FORM 2
The Patent Act 1970
(39 of 1970)
&
The Patent Rules, 2005

COMPLETE SPECIFICATION
(SEE SECTION 10 AND RULE 13)

TITLE OF THE INVENTION

“An automated method of sequential batch file processing for data loading”

APPLICANTS:

:
Name Nationality Address
HCL Technologies Limited Indian 50-53 Greams Road Chennai – 600006,
Tamil Nadu, India

The following specification particularly describes and ascertains the nature of this invention and the manner in which it is to be performed:-

FIELD OF INVENTION
[001] This invention relates to data processing systems, and more particularly to sequential file processing of files to be loaded/inserted into a database.

BACKGROUND OF INVENTION
[002] Document management systems use databases to store, track and trace electronics documents, scanned documents, digital documents and so on. Document management involves adding data records into a storage system. A data record may be in the form of a file, an email, and notification and so on.
[003] In case of file processing, files related to a specific process are stored in a sequential manner in one or more locations. The storage of files is based on metadata present in each file and can be implemented as a number of fields. These fields may be present in the same manner in all the files or the data in each file might change based on the content in the data record. All the files in a batch process may relate to each other through some common metadata. Each file will have a unique metadata or key associated with it for easy identification. Also all the files belonging to a batch process are loaded into storage in a serial manner. This means file 1 is loaded after which file 2 is loaded and so on.
[004] In small organizations where the file process is simpler and databases smaller, the process of creation, storage, and retrieval of files is easy. However big organizations face many challenges in batch processing of files. The organizations may receive files from internal as well as external clients in various formats. Identification of relevant files across the organization might be difficult. In addition, different batch process files might require storage on a daily basis for easy tracking by various members in the organization.
[005] There are many form based automatic document management solutions available. These allow users to name files using a form, where the user fills in data related to file as required in the file name, making batch processing easier. Implementing these form based document management solutions might work with internal clients, but when a large number of files come from external sources, the form based solution fails. These form based file names make identification of files easy, but they may not identify or store files in a sequential manner. Also, the files still require manual verification and checking before loading into a database.

OBJECT OF INVENTION
[006] The principal object of this invention is to automate the method of validating/verifying/checking files to be loaded into a database.
[007] Another object of the invention is to ensure that files are loaded in a predefined sequence into a database using metadata/events/keys in the file name.
[008] A further object of the invention is to define a custom file naming sequence specific to the batch file processing.

STATEMENT OF INVENTION
[009] Accordingly the invention provides a method and system for processing files to be loaded into a database, the method comprising of parsing name of a file to be loaded to the database; checking if the file name matches with a predefined event sequence; and loading the file to the database, if the file name fits in with the predefined event sequence. A notification may be sent to a stakeholder, if the file name fits in with the predefined event sequence. The notification is at least one of an email; a Short Messaging Service; an Instant Message; or an automated recording.
[0010] These and other aspects of the embodiments herein will be better appreciated and understood when considered in conjunction with the following description and the accompanying drawings. It should be understood, however, that the following descriptions, while indicating preferred embodiments and numerous specific details thereof, are given by way of illustration and not of limitation. Many changes and modifications may be made within the scope of the embodiments herein without departing from the spirit thereof, and the embodiments herein include all such modifications.

BRIEF DESCRIPTION OF FIGURES
[0011] This invention is illustrated in the accompanying drawings, through out which like reference letters indicate corresponding parts in the various figures. The embodiments herein will be better understood from the following description with reference to the drawings, in which:
[0012] FIG. 1 illustrates the environment of a document management system where file processing can be implemented, according to the embodiments as disclosed herein;
[0013] FIG. 2 illustrates the major blocks responsible for file processing and loading of batch files in a database server, according to the embodiments as disclosed herein;
[0014] FIG. 3 is an exemplary flowchart describing a method of processing and loading of files into a database server 103 in a sequential manner, according to the embodiments disclosed herein; and
[0015] FIG. 4 is an example of sequence of events defined for a particular batch process, according to embodiments disclosed herein.

DETAILED DESCRIPTION OF INVENTION
[0016] The embodiments herein, the various features, and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known components and processing techniques are omitted to not unnecessarily obscure the embodiments herein. The examples used herein are intended merely to facilitate an understanding of ways in which the embodiments herein may be practiced and to further enable those of skill in the art to practice the embodiments herein. Accordingly, the examples should not be construed as limiting the scope of the embodiments herein.
[0017] Referring now to the drawings, and more particularly to FIGS. 1 through 4, where similar reference characters denote corresponding features consistently throughout the figures, there are shown preferred embodiments.
[0018] FIG. 1 illustrates the environment of a document management system where batch file processing can be implemented, according to the embodiments as disclosed herein. An organization generally utilizes a document management system to create, store and retrieve documents in an organized manner. The organization may receive documents from both internal clients and external clients by way of a communication network. The clients (105-110) (internal and external) shown in figure 1 are computer systems. However, the client systems may be any electronic device with computing abilities. The host system 101 is generally a computing device capable of communicating with the database server 103 using communication network 104. The communication network 104 may be either a wired network 104 or a wireless network. The database server 103 is an electronic device, which has computing capabilities. The database server 103 can have a single or multiple databases stored. The database server 103 has necessary software for data communication and storage. The database server 103 may be present inside the internal network 104 of the organization. It may be accessible to internal clients who may be employees with privileges. These privileges may be assigned based on the batch process. Employees with privileges can access the database server 103 for information.
[0019] In case where files related to batch processing require storing into a database server 103, a batch file processing application 102 is present within the host system 101 and handles the batch processing in the host system 101. The host system 101 collects files from all the internal (105,106,107) and external clients (108,109,110,111) processes them using metadata and stores them in a sequential order in the database server 103. The metadata may comprise of the name of file.
[0020] FIG. 2 illustrates the major blocks responsible for file processing and loading of files in a database server, according to the embodiments as disclosed herein. The host system 101 may run several applications to support processing of files. Applications may be used for communication with the database server. The batch file processing application 102 refers to all the applications, which help in creating, storage, transmission and retrieval of files. The batch file processing application may simply be referred to as application 102 hereafter. The batch file processing application 102 processes files in three steps. The first step includes activities related to the processing 202 of the file and the second step is post processing 203.
[0021] Before starting the processing of a file, the batch process 206 needs to be set. Herein, the expected format of a file name is stored. For example, the file name for a file containing a pay order isclientid_firmID_POnumber_yearmonthdatetimefile format. Similarly, file names for other types of files are also defined. Once the batch process 206 is set, the event sequences 207 related to the batch process 206 need to be defined. When the first file in a batch process is received, the file name is parsed in 205 and the event sequence 207 is checked after which the file is validated by checking all the other information present in the file name. Consider an example of a file name 4567_6654_POA2345Z_20090417082403.csv for purchase order event or 6654_4567_ASN_1700AP_20090425092415 for Advance shipment notification event. The presence of PO in the file name denote that the file is a purchase order file with a purchase order number A2345Z from a client with client id 4567 to the firm with firm ID 6654, which was generated on 17th April, 2009 at 08.24.03. The file name describes “from” which client the file has been received, “to” which firm the purchase order has been sent and the date and time of receipt of the purchase order. The purchase order is the first event in this batch process. Hence, it is a valid event in an event sequence 207. Next, the “from” and “to” are checked against existing list of client and firm ID’s, after which the date and time are also checked. Once the file is completely validated, the processing 202 moves into the next stage. Like wise for advance shipment notification event sample, by using name of the file the existence of the entire pre-requisite PO, SO, BOL data in the database verified prior to staging the file for loading into database.
[0022] Successfully processed files 209 are loaded on to the database server 202. Documents, which cannot be validated because of incorrect ID or invalid date and time, are sent to failed documents 211 and appropriate notifications are sent to the stakeholder(s) of the file. The notification is sent to the stakeholder indicating that the name of the file is invalid and cannot be loaded into the database server 103. The notification may be an email, a Short Messaging Service (SMS), an Instant Message, an automated recorded call made to a phone number or any other suitable notification means.
[0023] The file origin can be traced using the input file 204 information received in the pre-processing stage 201. Batch processes 207 where files relating to certain event sequences 208 are present but not in the event, sequence 208 order defined are stored in temporary storage 211 for a certain period. The files stored in the temporary storage 211 are processed daily to check if any events in files become valid. If any of the events becomes valid, they are processed to check for complete validity of the file name and if they are completely validated, they become successful batch files 210 and are loaded on to the database server 103. If however, the file events do not get valid even after a period of time or the date/time or from/to location is invalid, the file is sent to failed documents 212, which sends a notification to the stakeholder of the file. The notification may notify the stakeholder that the file is invalid and cannot be loaded into the database server 103.
[001] FIG. 3 is a flowchart describing a method of processing and loading of files into a database server 103 in a sequential manner, according to the embodiments disclosed herein. The processing of files begins with setting (301) an event sequence 208 for a batch process 207. This flowchart follows a simple batch process 207 involving sales. The event sequences 208 in the batch process may include purchase order, sale order, Bill of lading receipt, Advance shipment notification, Shipment receipt, delivery receipt order, in the order given. These above mentioned order are sample and can be custom defined based on the need. Once the event sequence has been set and a file that may be uploaded has been detected, the file is formatted with a new formatted file 206 name, and is sent (302) to the batch file processing application 202 for further processing. The batch file processing application 102 extracts (303) the event related to the file and other information present in the formatted file name 206 by parsing the file name. Once the information in extracted, it is validated. The event of the file is checked against the event sequence of the batch process. For example, the purchase order file may be formatted as 4567_6654_POA2345Z_2009041708240523.csv. This is a purchase order file with purchase order A2345Z from a client with client id 4567 to the firm with firm ID 6654. The file name describes “from” as the client from where the file has been received, “to” as the firm to which the purchase order has been sent and the date and time of receipt of the purchase order. If the event is valid (304), the date and time information extracted from file name 206 is checked for validity (305). If the event is not valid (304), it is stored (306) in temporary storage for a certain period. The files stored in the temporary storage are processed daily to check (307) if any events in stored files become valid. If the batch file event becomes valid (308), they are processed to check for complete validity of the file name and if they are completely validated, they become successful batch files and are loaded on to the database server 103. If however, the file events do not get valid, or the date/time or from/to location is invalid and the time period expires (309) of time, the file is sent (311) to the failed document 211, which ensures that a notification is sent (312) to the stakeholder of the file. The notification may notify the stakeholder that the file is invalid and cannot be loaded into the database server 103. If the date and time are valid (305), the application 102 checks if the from location in the formatted file name is valid (310). If the date and time are not valid (305), a notification is sent to the stakeholder of the file. The notification may notify the stakeholder that the file is invalid and cannot be loaded into the database server 103. If the date and time and valid (305), the application 102 checks if the “from” information in the file name is valid (310). If the “from” field is valid (310), the application checks if the “to” field is valid (313). If the “from” or “to” location is not valid, the formatted file 206 is sent (311) to failed documents 212 which then sends (312) a notification with formatted file name to the stakeholder of the file. If the “to” information is valid, the application 102 does a final check (314) to determine if all the information in the formatted file name is valid. If all the information is valid (315), the formatted batch fie 206 are sent (316) to a successful batch file folder 210 and loaded (317) into the database server. The various actions in method 300 may be performed in the order presented or simultaneously. Further, in some embodiments, some actions listed in FIG. 3 may be omitted.
[0024] FIG. 4 is an example of sequence of events defined for a particular batch process, according to embodiments disclosed herein. The batch process revolves around a simple sales operation. The first column 401 shows event sequences 208 in the batch process 207, which may include includes purchase order, sale order, delivery receipt order, invoice order, payment order receipt in the order given. The second column 402 shows the formatted file names 206 for all the files involved in the process. All the file names in a batch process follow a generic format - From_To_PO_YYYYMMDDHH24MISS.csv OR FROM_TO_ASN_LOTID_YYYYMMDDHH24MISS . All the files shown in the second column 402 require to be validated, in the order present after which they can be loaded into the database server.
[0025] Embodiments disclosed herein use a file related to pay orders merely as an example. It may be obvious to a person of ordinary skill in the art that the embodiments herein may be used for any type of file.
[0026] The embodiments disclosed herein can be implemented through at least one software program running on at least one hardware device and performing functions to process files in a sequential manner for loading into a database server.
[0027] The foregoing description of the specific embodiments will so fully reveal the general nature of the embodiments herein that others can, by applying current knowledge, readily modify and/or adapt for various applications such specific embodiments without departing from the generic concept, and, therefore, such adaptations and modifications should and are intended to be comprehended within the meaning and range of equivalents of the disclosed embodiments. It is to be understood that the phraseology or terminology employed herein is for the purpose of description and not of limitation. Therefore, while the embodiments herein have been described in terms of preferred embodiments, those skilled in the art will recognize that the embodiments herein can be practiced with modification within the spirit and scope of the embodiments as described herein.

WE CLAIM:-
1. A method of processing files to be loaded into a database, the method comprising of:
Parsing name of a file to be loaded to said database;
checking if said file name matches with an event sequence; and
loading said file to said database, if said file name fits in with said predefined event sequence.
2. The method of claim 1, wherein said event sequence is pre-defined.
3. The method of claim1, wherein a notification is sent to a stakeholder, if said file name fits in with said predefined event sequence.
4. The method of claim 3, wherein said notification is at least one of
an email;
a Short Messaging Service;
an Instant Message; or
an automated recording.
5. A system performing a method as in at least one of preceding method claims 1 to 4.
Dated- 29th June 2011
Dr. Kalyan Chakravarthy
(Patent Agent)
ABSTRACT
An automated method of sequential batch files processing for data loading. This invention relates to data processing systems, and more particularly to sequential file processing, for files to be loaded into a database. Files related to a batch process are formatted and given a formatted file name defined for the process. A sequence of events is also defined for the files. The formatted files are individually validated by a file application. On successful validation, the files are sequentially loaded into a database server. If the files cannot be validated a notification is sent to the stakeholder of the event described in the file.FIG. 2