Field of Invention
The instant invention relates to a method and system for digitization of information present on business cards using the image-capture facility of hand held electronic devices.
Background of the invention
Driven by the dynamic business environment and consistent demands of enterprises and organizations to expand their operations, one is routinely on the move, engaging in business meetings, participating in events, attending conferences and exploring plethora of other networking opportunities for business expansions. In all these gatherings, exchange of business cards is a standard practice, which is one of the most fundamental and effective means of people networking. In the process, people collect a huge number of visiting cards of individuals, which are very crucial to their overall business prospects, networking opportunities and future growth.
Managing an ever-increasing number of business cards and retaining information about every individual becomes an onerous task. Accordingly, a method and system is needed to facilitate easy and immediate processing of information present on these business cards. Also, such information may be easily accessible. In addition, provision should be given to store context information along with the information present on the business card.
Prior work has been done in the area of capturing business card information. There are special purpose business card scanners that achieve the desired purpose, but the disadvantage with the conventional systems is that these solutions are not portable. The user will have to carry the card to his place, purchase the necessary hardware to get his work done. A user has to buy some extra hardware. He should know how to take correct and precise photographs.
Certain related inventions provide the facility to process the captured image locally within a portable terminal unit. However, correct extraction of information from the card image requires resource- and time-intensive operations. These operations include preprocessing, OCRing, Application logic that involves placement of various parts of information such as name, designation, company name, e-mail address, phone number, fax number, etc. Consequently, huge processing power and database are required to be used within the portable terminal. The portable devices with extensive processing power directly results in huge cost of the hardware, and is, therefore, not feasible for the communication devices such as mobiles to be used for

such functions. Moreover, incorporating a database within a portable terminal is not possible. Therefore, such portable terminal units produce inferior results to the ones obtained by a system, which handles the image processing at a remote server.
US23112476 talks specifically about a compact card scanner, which captures, process, and stores digital image and data in the scanner unit. It is cumbersome to carry everywhere a special scanner unit as against a mobile phone, which a business user commonly carries with him everywhere. Also, additional investments are needed for purchasing the scanner unit. The scanner unit also takes away the advantage of the image-capturing device doubling up as a communication device. Further, digital images occupy a large space.
The patent, GB02405003, mentions processing all the data captured locally using a portable terminal equipped with camera having an in-built software to process and manage captured images of business cards. This invention focuses on processing the captured data locally within the portable terminal unit. However, correct extraction of information requires resource- and time-intensive operations, extensive database search and comparison. This mandates huge processing power and database to be used within the portable terminal. Incorporating extensive processing power substantially increases the cost of the hardware, and is not feasible. Moreover, including a database in the portable terminal is not possible. Also, locally processing the image renders the portable terminal unusable during image processing. If the image-processing and text-extraction operations are performed at a remote server, the effective costs are much lesser, processing and text extraction is faster, and the portable terminal is not rendered unusable while the image is being processed and information extracted from it.
This patent describes managing an address book in a portable terminal, and various sections within that portable terminal for various steps of capturing, processing, managing and storing the information present on business cards. The information is stored in the permanent memory of the portable terminal. However, there is no way to recover the information if the information stored in the permanent memory of the input terminal is lost or rendered unusable. Also, there is no provision of metadata being stored with the information obtained from business cards. Absence of metadata may result in users not being able to efficiently obtain the desired information. Additionally, the information obtained from business cards is limited to be used only in the portable terminal.
JP23296353 explains a database of multiple business card images, where users access the pre-stored images using their unique ids. It explains a database of

multiple business card images,
where users access the pre-stored images using their unique ids. This system is shared by a number of companies and the system ensures authorized access to individual users. It does not solve the existing disadvantages.
Accordingly, a system and method is needed where the process for information extraction is easy and automated. The extracted information may be stored locally as well as on the remote server, and user may be able to query requisite data anytime, and hardware requirement is minimum.
Objectives and Summary
It is an objective of the instant invention to obviate the above drawbacks and provide a method and system for digitization of business cards.
It is an objective of the instant invention to provide easy extraction of information from business cards and use it when required.
It is another objective of the instant invention to provide digitization of business cards through handheld devices.
It is yet another objective of the instant invention to store the extracted data on a remote server.
It is further an objective of the instant invention to store the extracted data locally on the user's handheld device.
It is still another objective of the instant invention to provide a means to store metadata information along with the extracted information from the business card.
A method for digitization of information on a business cards in text form using a handheld electronic device comprising capturing the image of the business card using the handheld device, transmitting said captured image to an application server at the same instant, cleaning and processing said image and rendering it free of defects at said server, detecting blocks and components from the processed image, extracting, segregating and categorizing texts from said blocks and components and, updating said text in the user's folder at the server along with
metadata and transmitting said text to the handheld device for local storage. A system for instant digitization of information on a business card in text form using a handheld electronic device comprising image capturing means coupled to the handheld device for capturing image of the business card, transmission means in said handheld device for transmitting said captured image, a remote server for receiving said transmitted image, processing means in said server for removing defects of said image, layout analysis means for
defining the zones and extracting text, data classification means for categorizing extracted text and data storage means in said server for storing the extracted data in the user's folder.
Detailed description of the Invention
The instant invention provides a facility to users to automatically obtain the necessary information from an individual's business card in digital form and store them locally in the handheld electronic device and/or at a remote server, which is easily accessible through any handheld electronic device, which has image-capturing means, transmission means and display means.
The instant invention processes data at a remote server. This remote server also houses appropriate databases for quick and accurate search. The instant invention also enables users to store metadata (additional ancillary data) along with the user information extracted from the card and the image of the card. Metadata could be the brief description of conference/meeting in which the business card was presented to the user, date of the meeting, names of other associated persons, etc. The facility to store metadata is extremely useful when user wants to search for some information based on this metadata such as date or title of the conference/meeting and not on the information present on the card.
The information is also stored locally in the address book of the mobile phone so as to have instant access to the already processed information. Also, the same information is stored remotely at the server. At server, the image of the card is also stored along with the extracted information so that if the user wants to verify the information extracted from a business card, he can do so by querying for the image of the business card.
Figure 1 describes the system of the instant invention. The system makes use of a handheld device. Users usually always carry such handheld devices. In one embodiment, the handheld device includes mobiles. The handheld devices of the instant invention are equipped with camera. In one embodiment, the camera has a resolution of at least 2.0 mega pixels. A user, once he receives a business card (10), takes image of the business card with his handheld device camera (11). The transmission means within the handheld device sends this image to an application server (12). In one embodiment, the image is transmitted through the MMS facility in mobile phones. The image is further processed at the server. The server has memory with
modules stored therein. The server has layout analysis means for defining the zones and extracting text from the image zones. Imperfections, such as skew, noise, perspective in the image of the business card sent to the server hamper correct extraction of details from the business card.
An Image Processing and extraction module (13), specially designed to work on business cards and deployed at the server, performs the operations to render the image for correct extraction of printed details. The server comprises means to extract and segregate the text from the rendered image. The server also stores keyword database used to search and extract information from business cards.
The memory (14) at the server stores folders of the users. In one embodiment, the folder contains user's address book. The server stores the above extracted details in the respective folders. The user can access his folder at any stage through his handheld device and the required information is displayed to the user on the device. The extracted information is also stored in the handheld device memory.
The handheld device carried by user can be any electronic device such as PDA, mobile, pager, or communicator.
Figure 2 describes the steps of the method of the instant invention.
On receiving a business/ visiting card (20), a user, instead of keeping the card with him and referring to it every time he needs some information, captures an image of the card (21). For this purpose the user uses his handheld device equipped with image-capturing means.
As soon as the image is captured, the handheld device has transmission means, which transmit the captured image to a remote application server (22). The images are not stored locally. At the application server, Layout Analysis is done to prepare image for extracting the information (23). Next, the text is extracted for each block and component (24). Using intelligent business logic, the extracted textual information is segregated and categorized/classified under various heads (25) such as Name, Designation, Organization name, Email address, Phone number, Fax number, etc.
Different logics are used to categorize different types of information on the business card. The logic for categorizing a particular type of information on the business card is provided below.
A business card contains a wide variety of data as listed earlier. The instant invention extracts and determines various types of data. For example, to determine phone number, certain keywords are searched such as Telephone, Tel, Phone, Contact No, Mobile, Mob, Cell, etc. There is a separate keyword database, which is used to search for keywords for various types of data. If a keyword corresponding to phone number is matched, adjacent blocks are searched for long trails/strings of numeric and special characters. If such a component is found, individual characters are analysed one by one in a direction. If any character other than numeric or special character is encountered, such as an alphabet, then it is derived that the string does not represent a phone number. Similarly, once the entire string is traversed with only numeric and special characters encountered, it is derived that the string is a valid phone number if the string consists of more than 5 characters. Similarly, information for other fields is extracted.
After this, data validation and correction is done by matching the information against a huge database of valid words (26). This is done to correct the user data if extracted data is not accurate. Finally, the corrected data is stored at the server and also sent to the user who sent the card image (27).
Along with the information present on the card, the user can also send metadata information along with the image of the business card. This metadata can be used to query the data at a later date based on the context of the conference/meeting in which the user received the business card.
The user need not refer to the card whenever he needs information about any individual. Neither he needs to store and manage the cards. He may access his folder at the server through any of his handheld device from anywhere. The desired information is displayed to the user on his device through which the information has been requested. The information is also stored in the user's handheld communication device.
Figure 3 describes the steps of the Image Cleaning & processing. The captured image is first processed for removing noise (30) such as despeckling, which involves removing unwanted speckles that are present in the image. Then, preprocessing (31) is done to increase the accuracy as well as interpretation of the information present on the card. This is done by handling illumination, brightness, contrast, etc., present in the image.
Next, global thresholding is applied to the image (32). Thereafter, edges of the card image are detected (33) by using appropriate filters. Based on edges detected, corners of the images are determined (34). Next, if the card image has been captured at a certain angle, perspective correction (35) is done to make the image perpendicular to line of view. Then, the image is binarized (36). This is followed by detection of blocks and components (37). Components are grouped together based on factors such as component position, component height, size of component, and distance between components, etc. Finally, local thresholding is applied (38) on these blocks and components to render the image for correct text extraction.
Figure 4 describes user sending information query based on metadata such as conference name, conference date, etc. The information is received at the remote server, and based on the metadata-based query, requisite information is fetched and sent back to the user. The query can also be based on regular data such as Name, designation, etc. This would be needed if the user has lost the business card information stored on his handheld communication device.

We Claim:
1. A method for digitization of information on business cards using a handheld electronic device comprising:
capturing the image of the business card using the handheld device,
transmitting and storing said captured image to an application server at the
same instant,
cleaning and preprocessing said image and rendering it free of defects at
said server,
detecting blocks and components from the processed image, and storing this
image at the server
extracting, segregating and categorizing texts from said blocks and
components,
validation and correction of extracted text and,
updating said text in the user's folder at the server along with metadata and
transmitting said text to the handheld device for local storage.
2. The method as claimed in claim 1, wherein the card image is stored at the server for
verification of the extracted information.
3. The method as claimed in claim 1, wherein said cleaning and preprocessing
comprises:
removing noise and cleaning the image, said noise comprising unwanted
speckles,
increasing accuracy and interpretation from the cleaned image,
applying global thresholding to the image,
detecting the edges, applying perspective correction, binarizing said image.
4. The method as claimed in claim 1, wherein local thresholding is applied to render the
image for correct text extraction.
5. The method as claimed in claim 1, wherein categorizing information comprises:
searching for keywords in the extracted text using a keyword database, searching for strings of characters following the keyword found above and,
analysing each character of the above string for detecting numeric, special
character and alphanumeric character.
Inferring if character strings are valid values for a keyword
6. The method as claimed in claim 1, wherein user sends query to the server and receives
the required information.
7. The method as claimed in claim 6, wherein said query includes metadata based query
and regular query.
8. The method as claimed in claim 1, wherein said method comprises retrieving
information based on the metadata from said server and displaying to the user on said
handheld device when the user sends a request.
9. A system for instant digitization of information on a business cards in text form using a
handheld electronic device comprising:

• image capturing means coupled to the handheld device for capturing image
of the business card,
• transmission means in said handheld device for transmitting said captured
image,
• a remote server for receiving and storing said transmitted image,
• processing means in said server for removing defects of said image,
• layout analysis means for defining the zones and extracting text,
• means for categorizing extracted text,
• validation means for validating and correcting the extracted text and
• data storage means in said server for storing the extracted data and the
captured and preprocessed image in the user's folder.

10. The system as claimed in claim 9, wherein said means for categorizing extracted text
comprises database of keywords to be searched in the extracted text.
11. The system as claimed in claim 9, wherein said server comprises means to transmit the
extracted and categorized text to the handheld device.
12. The system as claimed in claim 6, wherein said handheld device comprises mobile
phones, pagers or PDA.
13. The system as claimed in claim 6, wherein said handheld device comprise display
means for displaying the retrieved information from the user's folder.
14. A method for digitization of information on business cards using a handheld electronic
device substantially as herein described with reference to the accompanying drawings.
15. A system for instant digitization of information on business cards in text form using a handheld electronic device substantially as herein described with reference to the accompanying drawings.