FIELD OF THE INVENTION
The present invention relates to image processing techniques, and more particularly to methods and systems for locating and retrieving required information from a document. The instant invention relates to extracting desired data in a template-based as welt as template-independent manner. Furthermore, the Instant invention can also be integrated with hardware as a device driver.
BACKGROUND OF THE INVENTION
In the present scenario, there Is an Increasing awareness towards authenticity of documents. Also, due to the automatic and computerized nature of processes, there is a need for digitization of all input. Data may directly be entered digitally or may be converted to a digital form acceptable by these processes.
In various areas, authenticity of a document is usually checked through handwritten signatures. The signature also serves as an identification means. User signatures are widely accepted as proof of originality of an instrument and the user's consent for the stated transaction effected by the Instrument. These serve an Important purpose in bank documents such as drafts, orders and cheques. Besides signatures, automatic extraction of user-entered fields such as date, courtesy amount etc from these documents may also be required.
Various methods have been suggested for the orientation detection of scanned document images. ITEE paper "Automatic Segmentation and Recognition of Bank Check Fields" describes removing background and noise during binarization using some features and a neural network. Identification is done based on each region and the features such as entropy energy, aspect ratio are calculated and fuzzy logic applied. This method is highly dependent on the fact that the aspect ratio of handwritten text and signature will be different, that is, the signature will be a purely graphical signature and not textual.
"Automatic Extraction of Signatures from Bank Check and Other Documents" uses sliding window and entropy as features on a template-based signature extraction

procedure. However, it does not handle the case of the signature overlaps with another component of the document. Also, taking the ideal scenario into account if the temf^ate is available then the entropy step becomes redundant and a simple sliding window based histogram analysis would be sufficient.
However, the above described systems and a majority of other systems in this field provide template-based extraction. As a result, they cannot directly be applied to or prove to be inefficient in real-life scenarios. This is due to the wide variety of available formats of each of the documents. Also, documents may also change from a Ome-to-time or situation-to-situation. Therefore, in template-based extraction systems the database of templates would need to be updated each time a new template is introduced or an existing one is changed. Also monitoring the use of these templates would be required since some templates that exist may become obsolete and would be therefore have to be removed.
A typical system for handwritten data extraction makes use of a document template to match it with the given document and extract the required information. This is useful in cases when the documents whose templates are already fed into the system are presented for data extraction. If however the given document has a different format and has no corresponding template then data extraction becomes tedious and time consuming. The user then may have to send the document for manual processing, as automatic extraction is not possible. This wastes a lot of time and involves extra labour. In addition, this incurs extra cost and a delay in services. In certain scenarios, this may also lead to higher fraud risks.
As an example, cheques from different banks have different formats. Even the same bank may have two or more different formats to suit the requirements of different type of users such as corporate and individual customers. Cheque fomiats also may keep evolving with time. Therefore, a format/templatebased approach is not feasible in this context and processing of bank cheques becomes complex. Also, in certain situations handwritten information on bank documents are covered by or interfere with the bank

stamp. This acts as noised attached with the data, which must be removed before extracting this information.
In the above described art, the disadvantage in template-independent systems Is the high dependency on a single feature, which in turn, compromises on robustness and efficiency.
Another main area of concern in the existing methods is that they are not able to extract user-entered handwritten information from documents due to the problem posed by varied backgrounds and the overlapping and intermixing of machine printed and handwritten infonnation.
Hence, there is a need for a robust and automated system that is able to function even in the non-availability of the relevant format. It must also be able to produce the desired results even when the required data is positioned In variable positions and orientations with respect to the expected location. Furt:her, it must also be able to identify the image even if It is overlaid with other elements in the document. It must also be flexible to handle grayscale, black-and-white as well as colored documents. There is also a requirement of extracting information filled in by more than one users, such as mandate scenario where more than one user sign on the same cheque.
OBJECTIVES AND SUMMARY OF THE INVENTION
It is an object of the invention to provide template-based extraction of user-handwritten information
It is another object of the invention to provide template-independent extraction of user-handwritten infonnation
It is also an object of the invention to capture required infonnation in documents filled by a single user as well as multiple users

It is another object of the invention to identify and locate regions where required data may be present
It is also an object of the invention to identify whether the handwritten data required to be extracted is present in the docunient or not
It Is yet another objective of the Invention to separate required information from another component overlaying it on the document
It Is also an objective of the present invention to identify the required infomnation even if it Is not exactly placed in ttie expected zone of the document
It is yet another objective of the system to aid in the faster processing of documents as a result of efficient identification and extraction of infomiatlon
To achieve the aforementioned objects, the present invention provides a method for extraction of user-handwritten Information from a document comprising the steps of:
- detecting the location of the required field on given document
- reading said field into memory, and
- extracting required handwritten information using template based extraction if the corresponding template is available, or
- extracting required handwritten information using non-template based extraction if the corresponding template is not available
Here, template-based extraction of handwritten information from a document comprises Hie steps of:
- finding the most probable region for the required Infomiation in said document
- extracting said region from both the identified template as well as said document
- performing relative component labeling of the image of said document with respect to said template

- deleting the matched components from said document image and image of said template
- identifying image labels of components left behind after deletion in both images
- determining a confidence value by matching said template image with said document image
- altering the document image based on the number of contact edges of the required information with said image labels
- segregating said required information by repeating above step for all labels
- obtaining finalized extracted image as required handwritten information
Also, template-independent extraction of handwritten information from a document comprises the steps of:
- identifying basic format applicable to the current document
- detecting known standard regions within said document
- detecting the most probable region for the required information in said document
- removing all recognizable text strings from said image
- comparing corresponding information of extracted image against existing image of concerned user taken from database
- labeling components in said retrieved image
- calculating the aspect ratio of components of said images
- applying path traversal to all components of said images to separate the attached components
- obtaining finalized extracted image as required handwritten information
The present invention further provides for a system for extraction of user-handwritten infonnation from a document comprising:
- input means for receiving user inputs to customize data extraction from the given document
- acquisition means for reading - in data from the document

- detection unit for detecting the presence and location of the required field on given document
- memory unit for storing data as it is being processed
- storage unit for storing the template database and the dictionary
- extraction unit for extracting required information from the given document
- output means for providing the extracted infomiation
BRIEF DESCRIPTION OF DRAWINGS
"Rie proposed method and system is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number Identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and components.
FIG. 1 illustrates an exemplary system for implementing a preferred embodiment of the present invention.
FIG 2 is a flow diagram depicting an exemplary method of classifying the given document to decide the type of extraction to be applied
FIG. 3 is a flow diagram illustrating an exemplary method of implementation of template based extraction of handwritten data
FIG. 4 is a flow diagram illustrating an exemplary method of implementation of template independent extraction of handwritten data
DETAILED DESCRIPTION OF DRAWINGS
Systems and methods for efficient means of automatic extraction of handwritten information are described. The description of the system or method shown herein below is intended only for illustration and disclosure of an operative embodiment and not to show all of the various forms or modifications in which this invention might be embodied or operated, since the same may be modified in various particulars or relations without departing from the spirit or scope of the claimed invention.

The present invention describes a method and system that offers a solution for automatic and efficient extraction of user's handwritten information from various instruments and paper-based documents such as bank cheques, drafts, orders etc. This may further be utilized for offline verification, analysis and similar other purposes. The instant invention enables both template-based as well as template-independent extraction of handwritten data. It also handles single as well as multiple user handwriting within the same document.
As an industrial application, one of the most important tasks in automatic bank cheque processing is the extraction of handwritten signatures from bank cheques for feeding them to an offline signature verification system which tests the signature for authenticity. The extraction and recognition of handwritten infomiation from a back cheque pose a formidable task which involves several subtasks such as extraction and recognition of signatures, courtesy amount, legal amount, payee, date etc.
The present invention can handle both template based as well as template independent extraction of data. Template-based extraction requires prior information about the layout of the document to extract the data from. It compares the document with a pre-specified template and tries to extract needed information from the fields earmariked for exb-action in the template. In template independent extraction there is no prior information about the layout of the document and fields cannot be marked in advance for extraction of data. In this scenario, a most probable region for the existence of the required data is found and the data present is then extracted. The extraction is done with prior knowledge of the approximate region in which the data is expected to be present.
The techniques described herein may be used in many different operating environments and systems. Multiple and varied implementations are described below. An exemplary implementation of the system and method of the present invention is discussed in the following section with respect to the accompanying figures.

EXEMPLARY SYSTEM
Figure 1 illustrates an exemplary system for implementing a preferred embodiment of the present invention. The system consists of an input device to accept the document and the data field which needs to be extracted. It may also be configured to accept document templates to be stored in the database of existing templates. It may further be configured to operatively update the dictionary and database of existing templates. It may also be configured to accept inputs to alter or update the database containing user-handwritten data samples. In another embodiment it may also be configured to accept basic formats used during template independent extraction. It may also be configured later or categorize the existing standard fomiats. These updates are then applied to the existing database and dictionary stored in the storage unit 105.
Further, acquisition means 102 can be coupled to a detection unit 103. The acquisition means read in data from the presented document into the memory unit 104 for further processing. These may comprise one or more digital input devices and one or more imaging devices such as an optical scanner. The acquisition means 102 may be connected to the detection unit 103 by an interface or in another embodiment, through a network.
The detection unit 103 receives an image of the document from the acquisition means 102 and converts it to digital data so that it is in a form that can be processed by the extraction unit 106. It then studies the read in data to determine whether the data that is required to be captured exists on the presented document by searching for Its intended location. It achieves this by working along with the extraction unit 106.
The system further includes a memory unit 106. At various stages during the extraction, the data is processed into many forms and is stored temporarily in the memory unit 104. After reading in the input data it is initially stored in the memory unit 104 and is thereafter accessed by the extraction unit 106 for further processing.
The storage unit 105 contains the templates database. In an embodiment it also contains the basic formats database, dictionary and the user handwriting data samples.

The extraction unit 106 is configured to accept the document image as input for processing and performs the actual task of extracting the required handwritten data from its identified location on the given document. In an embodiment, it accesses the existing templates database stored in the storage unit 105 to determine the mode of extraction to be applied. It includes means to detect each component of the document image. It further includes means to detect, analyze and compare characteristics of one or more components of the document image and apply one or more techniques such as aspect ratio, path tracing and so on using a plurality of techniques. It also makes use of the dictionary, standard formats and the templates database to facilitate the analysis and comparison while performing template-based extraction.
Once the extraction procedure is completed, the extraction unit 106 sends the final extracted image to the output means 107. However, if successful extraction has not been possible it signals 'recommendation for manual verification' to be given as output through the output means 107.
In one embodiment, the extraction unit 106 can reside on a standard expansion card for a personal computer. In another embodiment, the extraction unit 106 can also be incorporated into the hardware of an imaging device as a device driver. In yet another embodiment, the extraction unit 105 resides in the memory of a computing device. Thus, the system may operate both in offline or online mode.
EXEMPLARY METHOD (SExemplary methods for extraction of user-handwritten data from a document are described with reference to Figs. 2 to 4. The methods are illustrated as a collection of blocks in a logical flow graph, which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. The order in which the process is described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement the process, or an alternate process. Additionally, individual blocks may be deleted from the process without departing from the spirit and scope of the subject matter described herein.

Figure 2 illustrates a flow diagram representation of an exemplary implementation of classifying the given document to decide tiie type of extraction to be applied. At block 201, the document image acquired from the acquisition means is subject to processing for detecting known regions of the document. In an embodiment, in each document there is embedded an identification string. As an example, these may include the MICR region in the case of a bank cheque. The acquired image can be filtered to remove the noise and errors introduced during the process of scanning. At block 202, the known regions of the document are read into memory.
At block 202, it is detennined if the document template for the identified known regions of the given document is available. This is performed by accessing the existing templates already stored.
If there is a matching document template available for the given document, then at block 204 the selected mode for extraction becomes template-based extraction as disclosed below with respect to the description of Figure 3. Else, if no such template is found, then at block 205 the mode selected for extraction becomes template-Independent extraction as disclosed below with respect to the description of Rgure 4. In an embodiment, the database is subsequently updated with the new document template for future purposes.
Further, a plurality of novel features as described below are used to extract user-handwritten data from documents:
- Embedded data: Data that is embedded onto the document such as MICR in the case of a bank cheque are recognized through OCRing or similar techniques. This distinguishes handwritten data from printed data and thereafter a most probable region or quadrant where tiie required user data may exist can be found

- Aspect Ratio fWidth/Lenqth): Aspect ratio is calculated for all the elements on a presented document. For printed text, aspect ratio is quite high while it is relatively less for handwritten text
- Path of Handwritten Data: The path of the handwritten data is traced and the information of the following properties is saved in ttie dictionary:
o Change of angle
o Hde. The number of holes in the handwritten data are counted and the
biggest hole is compared with that of the original signature o Strokes. The number of strokes in the handwritten data are counted
Figure 3 illustrates an exemplary method of implementation of template-based extraction of handwritten data. At block 301 the region in the document where the required handwritten data is most likely to exist is detected. The coordinates of this region are then retrieved, This region is then extracted from both the template as well as the presented document into separate images. These images are then stored in the memory unit temporarily for processing.
Next, at block 302, relative component labeling of the extracted image of the presented document (S) is performed with respect to the extracted template image (T). Once the template is matched, matching of the individual components in the presented document image (S) with those in the template is performed and the matched components are subsequently deleted from both the images in block 303. The components that do not match and are left behind in the presented document image comprise the desired handwritten data. An error flag is set to default value "No', indicating that the presented image contains only the desired handwritten information.
At block 304, it is checked to determine if the extracted template image T is empty. If the template image T Is empty, then at block 305 the value of the error flag is determined. If the error flag is set to "Yes', it indicates at block 306 that all the components are not removed from the presented document image S. Therefore S contains the required information and possibly some noise or other non-required

elements. Further, if the error flag Is set to 'No', then it indicates at block 307 that the presented Image S contains only the required handwritten information.
However, If the extracted template Image T Is not empty, It Indicates that the system recognizes that the image contains some known elements but their separation from the required image is difficult These known elements are image labels. Therefore, in block 308, the template image T is now matched with the presented image S one element at a time. Based on the results obtained, a confidence value Is added to the extracted Image to further help during the verification process. The number of contact edges the compared element had with the extracted Image and the remaining labels 15 now counted at block 309. Based on the number of contact edges of the extracted data It is decided whether a label should be deleted.
If there is only a single contact point, then at block 310, the label is deleted from the template Image T as well as the extracted image S. If there are 2, 3 or 4 contact points, we first check for the case of 2 or 4 contact edges I.e. even number of contact edges. In tills scenario, the closest two edges are joined with a straight line of thickness of two pixels with horizontal biasing at block 311. However, if the number of contact edges Is odd, i.e. 3, they are ignored and retained as It is in the extracted Image S. Further, again in block 310, the label is deleted from the template image T as well as the extracted image S. Once again, in block 304, it is checked to determine if the extracted template image T is empty.
However, if the number of contact edges as determined in block 309 is more than 4, then at block 312, the error flag Is set to 'Yes' and the label is deleted from the template image T. The above-mentioned steps are then repeated until the extracted Image is. segregated from all the labels. If labels exist that cannot be segregated from the handwritten Information, tiie document is sent for manual extraction.
Rgure 4 illustrates an exemplary method of implementation of template Independent extraction of handwritten data. The extraction is performed based on a defined basic format standard of the document. In an embodiment this format may be defined by the

user. Also, this may be changed to suit the requirements of the user. Further, there may be more than one general fonnat. The general formats may also be classified into various categories. These categories can be defined and personalized by the user. The general fonnat is defined by the user is such a manner that the documents that are to be processed by the system are results of minor changes and personalization of the basic format.
Further, a sample of the handwritten data to be extracted is saved In the database of existing data samples. For example, in a scenario where user signatures have to be extracted from bank cheques, a database of user signatures may be maintained.
Based on the document presented, a particular document format, from the existing formats is identified. In block 401, the region in the document where the required handwritten data is most likely to exist is detected as per predefined rules. The same is then saved as the image of the presented document (S). Next, optical character recognition is applied on the extracted image and all recognizable text strings are removed.
Next, the corresponding image of the data to be extracted is taken from the existing database and compared against the data to be extracted from the presented document.
Furt:her, at block 402, the components in the image extracted from the presented document are labeled and the aspect ratio of the components in the two images is calculated. If the aspect ratio of a component in the extracted image of presented document is approximately the same as that of the existing image retiieved from the database, then at block 403, it is detected as the required information and extracted.
However, in a scenario where the user-handwritten data appears in the fomi of two or more components on the presented image, path traversal on all components is perfomied at block 404. The handwriting paths are partly traveled in both data images and the paths are calculated. If they are the same, then the combined aspect ratio of these components is determined for comparison against the aspect ratio of the existing

handwritten data image. A similar path traversal is performed if the presented handwritten data occurs as a single component but it is present as two or more components In the existing data image. After successful aspect ratio match is done with or without traversal, path traversal is applied to ail the components and the attached components are removed.
However, if successful extraction is not possible, at block 405, the document is sent for manual extraction.
The embodiments described above and illustrated in the figures are presented by way of example only and are not Intended as a limitation upon the concepts and principles of the present invention. Elements and components described herein may he further divided into additional components or joined together to form fewer components for performing the same functions. As such, it will be appreciated by one having ordinary skill in the art that various changes in the elements and their configuration and arrangement are possible without departing from the spirit and scope of the present invention as set forth in the appended claims.

We claim
1. A method for extraction of user-handwritten information from a document, said
method comprising the steps of:
- detecting the location of the required field on given document
- reading said field into memory, and
- extracting required handwritten infomiation using template based extraction if the corresponding template is available, or
- extracting required handwritten information using non-template based extraction if the corresponding template is not available

2. A method as claimed in claim 1, comprising the step of scanning said document to read the template while extracting the required information using template based extraction
3. A method as claimed in claim 2, comprising the step of searching the template database for the corresponding template of said document
4. A method as claimed in claim 2, comprising the step of updating the database with the new template after extracting the required information using template-independent extraction
5. A method as claimed in claim 2, wherein the required field on the document is read into memory through OCRing
6. A method as claimed in claim 2, wherein the required information is extracted by tracing the path of the said information
7. A method as claimed in claim 2, wherein the required information is extracted by calculating the aspect ratio of the components of the image

8. A method as claimed in claim 1, wherein performing template based extraction of
required information comprises:
- finding the most probable region for the required infbmfiation in said document
- extracting said region from both the identified template as well as said document
- perfonning relative component labeling of the image of said document with respect to said template
- deleting the matched components from said document Image and image of said template
- Identifying Image labels of components left behind after deletion in both images
- determining a confidence value by matching said template image with said document image
- altering the document image based on the number of contact edges of the required information with said Image labels
- segregating said required information by repeating above step for all labels
- obtaining finalized extracted Image as required handwritten information
9. A method as claimed in claim 1, wherein performing template independent
extraction of required information comprises:
- identifying basic format applicable to the current document
- detecting known standard regions within said document
- detecting tiie most probable region for the required information in said document
- removing all recognizable text strings from said image
- comparing corresponding Information of extracted image against existing Image of concerned user taken from database
- labeling components in said retrieved image
- calculating the aspect ratio of components of said images

- applying path traversal to all components of said Images to separate the attached components
- obtaining finalized extracted Image as required handwritten information

10. A method as claimed In claim 9, wherein said basic format Is defined and personalized by the user
11. A n:\ethod as claimed in claim 9, wherein said basic format can be changed to suit the requirements of the user
12. A method as claimed in claim 9, wherein there exists more than one basic format
13. A method as claimed In claim 12, wherein said basic formats can be classified into various categories
14. A method as claimed in claim 9, wherein said existing image of user-handwritten data is taken as Input and saved at the database of existing data samples for a particular user
15. A system for extraction of user-handwritten information from a document, comprising:

- input means for receiving user inputs to customize data extraction from the given document
- acquisition means for reading - in data from the document
- detection unit for detecting the presence and location of the required field on given document
- memory unit for storing data as it is being processed
- storage unit for storing the template database and the dictionary
- extraction unit for extracting required information from the given document
- output means for providing the extracted information

wherein the extraction unit can perform both template based and template independent extraction
16. The system as daimed in claim 15, wherein the input means accepts the data field to be extracted and the document from which it is to be extracted as input
17. The system as claimed in claim 15, wherein the input means accepts inputs to alter or update the document templates database
18. The system as claimed in claim 15, wherein the input means accepts inputs to alter or update the standard formats database
19. The system as claimed in claim 18, wherein the input means accepts inputs to categorize the data within the standard formats database
20. The system as daimed in daim 15, wherein the input means accepts Inputs to alter or update the database containing user handwriting data samples
21. The system as daimed In claim 15, wherein the acquisition means indudes one or more digital input devices and one or more imaging devices.
22. A system as daimed in claim 15, wherein predefined characteristics of the required information are stored in a dictionary at the storage means
23. A system as daimed in daim 22, wherein the said characteristics include attributes such as change of angle, number of holes and number of strokes
24. A system as daimed in daim 15, wherein the document templates database is stored at said storage means
25. A system as daimed in claim 15, wherein the standard formats database is stored at said storage means

26. A system as claimed in claim 15, wherein the user handwriting data samples are stored at said storage means
27. A system as claimed in claim 15, wherein said document contains an embedded identification string
28. A system as claimed in claim 15, wherein data read In at said acquisition means is filtered to remove noise and errors
29. The system as claimed in claim 15, wherein the extraction unit further comprises:

- template based extraction means
- template independent extraction means
30. The system as claimed in claim 15, wherein said system further includes:
- searching means for searching within the database and dictionary
31. The system as claimed in claim 15, wherein said system further includes:
- updating means for updating the template database and dictionary
32. The system as claimed In dalm 15, wherein said system operates both in offline as well as online mode
33. A computer program product for extraction of user-handwritten information from a document, comprising one or more computer readable media configured to perform the method as claimed In any of the claims 1-9

34. A method for extraction of user-handwritten information from a document
substantially as herein described with reference to and as illustrated by the
accompanying drawings.
35. A system for extraction of user-handwritten information from a document
substantially as herein described with reference to and as illustrated by the
accompanying drawings.