AUTOMATIC QUALITY ASSESSMENT OF DIGITAL DOCUMENTS

Field of the disclosure: -
The present disclosure relates to automatic quality assessment of digital documents and more specifically, but not limited to automatic quality assessment of digital documents against the original documents.

Background: -
Historically, every industry such as Banking, telecom, Insurance, Government services, Manufacturing and Education have relied heavily on paper based documents. Even with the advent of computers and technological advancements, the demand for paper for usage in day-to-day operations remains unaltered. However, many institutions such as banks, libraries and museums are turning to digitize their valuable documents such as manuscripts, maps, old magazines etc. for future use.

Also, publication houses are now opting for digital books instead of books printed on paper as they are more portable, easy to sell and distribute and can be accessed and shared easily. Businesses like banks, telecom operators, insurance companies etc. need to digitize their customers’ documents for further processing. Most of the businesses have started to opt for a paperless organization, as it facilitates easier retrieving, handling, processing and archiving, even for huge numbers of documents.

To cater to such requirements, establishments outsource the document scanning activity to specialist vendors. More and more companies, cutting across the industries, are choosing to outsource scanning activities, and thereby delineating their core focus areas from non-prime/support areas. These vendors dedicatedly scan documents, and have established massive scanning infrastructure for scanning documents in bulk. However, this outsourcing has its share of problems wherein the digitization task costs a huge amount of money and at the same time, the task has to be completed within a stipulated period. In a normal book/magazine digitization scenario wherein exact look and feel of the book is to be preserved, the system is normally manual. An operator manually types the text and formats the document to make it look similar to the original document in size, color etc. However, manual operation is always prone to human errors.

Another method for digitizing the document is Optical Character Recognition (OCR). However, OCR cannot be used to digitize ancient books and manuscripts as the fonts used by them are not standard fonts. Also, digitizing a book of the current generation is easier and cost-effective as once a book is digitized, multiple copies can be produced from them. Manuscript/ ancient book digitization is comparatively a costly affair as each manuscript is a unique creation.

Therefore, currently the standard method being followed is scanning of the book pages and then transforming it into text using OCR. The transformation into text is useful because scanning a book produces images, which are difficult to store on small devices, expensive to download, and cannot be searched. However, OCR is not always perfect and is prone to a lot of errors. This would ideally require an automated quality assessment of digitization which is presently performed manually.

Summary: -
Embodiments of the present disclosure relate to automatic quality assessment of digitized documents against the original document itself.

As opposed to the conventional art described above, the present disclosure focuses only on ascertaining the quality of digital documents and whether the original document needs to be digitized again based on the quality of the digital documents.

The present disclosure ensures that exact look and feel of the original document is maintained.

An embodiment of the present disclosure teaches a system for automatic quality assessment of digital documents comprising a document database, at least one acquisition unit coupled to the document database, at least one processor coupled to the acquisition unit, mismatch calculator coupled to the at least one comparator and an evaluation unit coupled to the mismatch calculator. According to another embodiment of the present disclosure, the document database is configured to store the digital documents as well as scanned images of corresponding original documents. The at least one acquisition unit is configured to acquire a particular digital document and scanned image of corresponding original document together for automatic quality assessment.

According to an embodiment of the present disclosure the at least one processor comprises a feature determination unit and a comparator coupled to the feature determination unit. In accordance with an embodiment, the feature determination unit is configured to determine values of pre-defined features of the digital document and scanned image of corresponding original document while the comparator is configured to compare the determined values of pre-defined features.

In accordance with an embodiment of the present disclosure, the output of the comparator is fed to the mismatch calculator, which is configured to calculate mismatch index. The mismatch index is then input to the evaluation unit, which is configured to evaluate output of the mismatch calculator and accordingly provide an output decision to the user.

According to an embodiment of the disclosure, the predefined features are such as physical layout of document, layout and structure of documents, textually, location of images, full text check and skew.

An embodiment of the present disclosure refers to a method for automatic quality assessment of digital documents wherein the method comprises acquiring a digital document and scanned image of corresponding original document, determining values of pre-defined features for the digital document and the scanned image of corresponding original document, comparing determined values of pre-defined features for the digital document and the scanned image of corresponding original document, calculating mismatch index based on comparison of determined values of pre-defined features, evaluation of calculated mismatch index and providing output to a user.

Yet another embodiment of the present disclosure describes a scanner comprising a document database, at least one acquisition unit coupled to the document database, at least one processor coupled to the acquisition unit, mismatch calculator coupled to the at least one comparator and an evaluation unit coupled to the mismatch calculator.

Brief Description of Drawings: -
The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and components.

Figure 1 illustrates a block diagram representation of a system for automatic quality assessment of digital documents according to an embodiment of the present disclosure.

Figures 2a-2e illustrate examples of mismatch between digital documents and original documents in accordance with embodiments of the present disclosure.

Figure 3 illustrates a block diagram representation of an example of a system for automatic quality assessment of digital documents according to an embodiment of the present disclosure.

Figure 4 illustrates a flow diagram representation of a method for automatic quality assessment of digital documents according to an embodiment of the present disclosure.

Figure 5 illustrates a flow diagram representation of an example of a method for automatic quality assessment of digital documents according to an embodiment of the present disclosure.

Figures 6a-6b illustrate a flow diagram representation of textual and non-textual pattern separation in accordance with an embodiment of the present disclosure.

Detailed Description: -
The following discussion provides a brief, general description of a suitable computing environment in which various embodiments of the present disclosure can be implemented. The aspects and embodiments are described in the general context of computer executable mechanisms such as routines executed by a general purpose computer e.g. a server or personal computer. The embodiments described herein can be practiced with other system configurations, including Internet appliances, hand held devices, multi-processor systems, microprocessor based or programmable consumer electronics, network PCs, mini computers, mainframe computers and the like. The embodiments can be embodied in a special purpose computer or data processor that is specifically programmed configured or constructed to perform one or more of the computer executable mechanisms explained in detail below.

Exemplary embodiments now will be described with reference to the accompanying drawings. The disclosure may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey its scope to those skilled in the art. The terminology used in the detailed description of the particular exemplary embodiments illustrated in the accompanying drawings is not intended to be limiting. In the drawings, like numbers refer to like elements.

The specification may refer to “an”, “one” or “some” embodiment(s) in several locations. This does not necessarily imply that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms “includes”, “comprises”, “including” and/or “comprising” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and/or” includes any and all combinations and arrangements of one or more of the associated listed items.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The figures depict a simplified structure only showing some elements and functional entities, all being logical units whose implementation may differ from what is shown. The connections shown are logical connections; the actual physical connections may be different. It is apparent to a person skilled in the art that the structure may also comprise other functions and structures. It should be appreciated that the functions, structures, elements and the protocols used in communication are irrelevant to the present disclosure. Therefore, they need not be discussed in more detail here.

Also, all logical units described and depicted in the figures include the software and/or hardware components required for the unit to function. Further, each unit may comprise within itself one or more components which are implicitly understood. These components may be operatively coupled to each other and be configured to communicate with each other to perform the function of the said unit.

Figure 1 describes a block diagram representation of a system for automatic quality assessment of digital documents in accordance with an embodiment of the present disclosure. The system 100 comprises a document database 101, which according to an embodiment of the present disclosure is configured to store digital documents and scanned images of corresponding original documents.

According to another embodiment of the present disclosure, the document database is configured to store only digital documents and when quality of a particular digital document is to be assessed, the corresponding original document is scanned at that instance itself.

According to an embodiment, the system 100 comprises at least one acquisition unit 102 coupled to the document database, which is configured to acquire a particular digital document and scanned image of corresponding original document together from the document database. The acquisition unit 102 rescales the acquired digital document and scanned image of corresponding original document to a pre-decided DPI’X’.

According to an embodiment of the present disclosure, the system 100 comprises two acquisition units configured to acquire the digital document and scanned image of corresponding original document separately from the document database.

The system 100 further comprises at least one processor 103 coupled to the acquisition unit 102 wherein the processor further comprises a feature determination unit 103a and a comparator 103b coupled to the feature determination unit 103a. According to an embodiment of the present disclosure, the system comprises two processors for the digital document and scanned image of the original document separately.

In accordance with an embodiment of the present disclosure, the feature determination unit 103a is configured to determine values of pre-defined features of the rescaled digital document and scanned image of corresponding original document. According to an embodiment of the disclosure, the pre-defined features are textual as well as non-textual such as physical layout of document i.e. torn corners, edges, streaks and black edges; layout and structure of documents; location of images; full text check and skew.

In accordance with an embodiment of the present disclosure, the feature determination unit 103a, is configured to first separate the textual and non-textual components of the documents to determine the values of the pre-defined features. The mechanism of separation of textual and non-textual components in accordance with an embodiment of the disclosure is described in Figure 6a-6b.

The comparator 103b, according to an embodiment of the present disclosure, compares the determined values of the pre-defined features and provides an output. This output is then, input to a mismatch calculator 104, which is configured to calculate a mismatch index. According to an embodiment of the disclosure, the mismatch index is then evaluated by an evaluation unit 105, which then provides an output to the user indicating whether there is any mismatch between the digital document and scanned image of corresponding original document. The evaluation of the mismatch index is carried out with a threshold mismatch index, wherein if the calculated mismatch index is above the threshold mismatch index, the quality of the digital document is not good and the original document requires rescanning. If the mismatch index is lower, then the quality of the digital document is satisfactory.

According to embodiments of the present disclosure, the pre-defined features and threshold mismatch index are set in accordance with the nature of the institution/establishment; the documents are being digitized for. Therefore, as an example, the pre-defined features and threshold mismatch index are different for a bank and a library.

Incase the quality of the digital document is not good, the processor 103 enables the system 100 to automatically detect the mismatch between the digital document and the original document such as identify parts of the original document which are not digitized, change in layout of text and images, change in captions, missing captions and title, change in color of text, differentiate whether an image is a pure image or a text image etc.

Figures 2a to 2e illustrate scenarios wherein according to embodiments of the disclosure, the system 100 shall output that the quality of digitized documents is not good and therefore, the document requires scanning. Figure 2a refers to an instance wherein the location of the image is not proper against the scanned image of the original document and therefore, evaluation of the mismatch index provides output as bad digitization of the document.

Figure 2b shows an instance wherein the layout and structure of the document has been changed in the digital document against the scanned image of the original document. Therefore, evaluation of the mismatch index provides output as bad digitization of the document. Figure 2c shows an instance wherein heading is missing in the digital document against the scanned image of the original document. Therefore, evaluation of the mismatch index provides output as bad digitization of the document.

Figure 2d refers to an instance wherein text is missing in the digital document against the scanned image of the original document. Therefore, evaluation of the mismatch index provides output as bad digitization of the document. Figure 2e refers to an instance wherein the image caption is missing in the digital document against the scanned image of the original document. Therefore, evaluation of the mismatch index provides output as bad digitization of the document.

Figure 3 illustrates a block diagram representation of an example of a system for automatic quality assessment of digital documents. The figure comprises a library collection of documents 301, a scanning unit 302 coupled to the library collection. According to an embodiment of the disclosure, the scanning unit comprises two options of manual entry 302a and an automated tool for digitization 302b. The scanning unit is then coupled to the system 100 as described in Figure 1. Accordingly, the output of the system 100 determines the quality of the digital document.

Embodiments of the method for automatic quality assessment of digital documents according to various embodiments of the present disclosure are described in Figures 4, 5, 6a and 6b. The methods are illustrated as a collection of blocks in a logical flow graph, which represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. The order in which the process is described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order to implement the process, or an alternate process.

Figure 4 refers to a flow diagram representation of a method for automatic quality assessment of digital documents wherein said method, according to an embodiment of the disclosure, comprises acquiring a digital document and scanned image of corresponding original document 401. According to an embodiment, the acquisition comprises rescaling the acquired documents to pre-decided DPI’X’. Values of pre-defined features for the digital document and the scanned image of corresponding original document are then determined 402 and the determined values are compared 403. Accordingly, the mismatch index is calculated 404 and evaluated 405. The evaluation provides an output to the user 406 as to whether the quality of the digital document is good or not and needs to be sent for rescanning.

According to an embodiment of the disclosure, the evaluation of the mismatch index is carried out with a threshold mismatch index, wherein if the calculated mismatch index is above the threshold mismatch index, the quality of the digital document is not good and the original document requires rescanning. If the mismatch index is lower, then the quality of the digital document is satisfactory.

In accordance with an embodiment of the present disclosure, Incase the quality of the digital document is not good, the mismatch between the digital document and the original document automatically detected. The mismatch identified are such as identify parts of the original document which are not digitized, change in layout of text and images, change in captions, missing captions and title, change in color of text, differentiate whether an image is a pure image or a text image etc.

According to an embodiment of the disclosure, the digital document and scanned image of corresponding original document are acquired separately and separately processed to determine values of the pre-defined features. This is illustrated by Figure 5, which describes a flow diagram representation of an example of a method for automatic quality assessment of digital documents according to an embodiment of the present disclosure. Scanned image and a file stored in the database are acquired separately 501, 502. The acquired documents are then rescaled separately to DPI’X’ 503, 504 and accordingly processed. The textual and non-textual components of the acquired documents are matched 505, 506 and their mismatch index is separately calculated 507, 508. It is then identified, whether there is a mismatch 509. If yes, then the quality of the digital document is bad 511 and the document needs to be rescanned else the digitization of the document is good 510.

Figures 6a-6b illustrate a flow diagram representation of textual and non-textual pattern separation in accordance with an embodiment of the present disclosure. The document image is acquired 6a1 and then, textual and Non-textual components are separated 6a2. This is achieved by way of mechanism illustrated in Figure 6b. After the textual and non-textual component separation, their respective patterns are separated 6a3 and then processed in accordance with the embodiment described in Figure 4.

According to Figure 6b, the document images are processed to remove noise 6b1 and filter the components of the document 6b2. Horizontal smearing is performed on the filtered components 6b3 with 0.05 LXDPI pixels. According to an embodiment of the disclosure, horizontal smearing is performed with 0.05 LXDPI pixels. Textual components are selected by width, height ratio of black component in the horizontal smeared image 6b4 and corresponding location of textual component is mapped on the original image 6b5, and textual and non-textual image is thus obtained 6b6.

The present disclosure describes a cost effective and efficient mechanism to gauge the quality of the digital document. Once the quality is known, the digital document can either be accepted for further processing or a request can be immediately sent to rescan the poor-quality digital document. This ensures that poor-quality digital documents are caught at the earliest in the process and immediate action can be taken. This results in huge gain for the organization, as it is not faced with scenarios where poor quality of digital document is determined only when it is to be actually used.

As will be appreciated by one of skill in the art, the present invention may be embodied as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, a software embodiment or an embodiment combining software and hardware aspects all generally referred to herein as a "circuit" or "module." Furthermore, the present invention may take the form of a computer program product on a computer-usable storage medium having computer-usable program code embodied in the medium.

Furthermore, the present invention was described in part above with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention.

It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

The flowchart and schematic diagrams of Figures 1-16 illustrate the architecture, functionality, and operations of some embodiments of methods, systems, and computer program products for media tuning. In this regard, each block may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that in other implementations, the function(s) noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending on the functionality involved.

In the drawings and specification, there have been disclosed exemplary embodiments of the invention. Although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation, the scope of the invention being defined by the following claims

We claim: -

1. A system for automatic quality assessment of digital documents comprising: -
a. a document database;
b. at least one acquisition unit coupled to the document database;
c. at least one processor coupled to the acquisition unit;
d. mismatch calculator coupled to the at least one comparator; and
e. an evaluation unit coupled to the mismatch calculator.

2. The system as claimed in claim 1 wherein the document database is configured to store the digital documents as well as scanned images of corresponding original documents.

3. The system as claimed in claim 1 wherein the at least one acquisition unit is configured to acquire a particular digital document and scanned image of corresponding original document together for automatic quality assessment.

4. The system as claimed in claim 1 wherein the at least one processor comprises: -
a. a feature determination unit; and
b. a comparator coupled to the feature determination unit.

5. The system as claimed in claim 4 wherein the feature determination unit is configured to determine values of pre-defined features of the digital document and scanned image of corresponding original document.

6. The system as claimed in claim 4 and 5 wherein the comparator is configured to compare determined values of pre-defined features of the digital document and scanned image of corresponding original document.

7. The system as claimed in claim 1 wherein the mismatch calculator is configured to calculate mismatch index based on output of the comparator.

8. The system as claimed in claim 1 wherein the evaluation unit is configured to evaluate output of the mismatch calculator and accordingly provide an output to the user.

9. The system as claimed in claim 5 wherein predefined features are such as physical layout of document, layout and structure of documents, textually, location of images, full text check and skew.

10. A method for automatic quality assessment of digital documents comprising: -
a. acquiring a digital document and scanned image of corresponding original document;
b. determining values of pre-defined features for the digital document and the scanned image of corresponding original document;
c. comparing determined values of pre-defined features for the digital document and the scanned image of corresponding original document;
d. calculating mismatch index based on comparison of determined values of pre-defined features;
e. evaluating calculated mismatch index; and
f. providing output to a user.

11. The method as claimed in claim 10 wherein predefined features are such as physical layout of document, layout and structure of documents, textually, location of images, full text check and skew.

12. The method as claimed in claim 10 wherein digital document and scanned image of corresponding original document is stored in a document database.

13. A scanner comprising: -
a. a document database;
b. at least one acquisition unit coupled to the document database;
c. at least one processor coupled to the acquisition unit;
d. mismatch calculator coupled to the at least one comparator; and
e. an evaluation unit coupled to the mismatch calculator.

14. The scanner as claimed in claim 13 wherein the document database is configured to store the digital documents as well as scanned images of corresponding original documents.

15. The scanner as claimed in claim 13 wherein the at least one acquisition unit is configured to acquire a particular digital document and scanned image of corresponding original document together for automatic quality assessment.

16. The scanner as claimed in claim 13 wherein the at least one processor comprises: -
a. a feature determination unit; and
b. a comparator coupled to the feature determination unit.

17. The scanner as claimed in claim 16 wherein the feature determination unit is configured to determine values of pre-defined features of the digital document and scanned image of corresponding original document.

18. The scanner as claimed in claim 16 and 17 wherein the comparator is configured to compare determined values of pre-defined features of the digital document and scanned image of corresponding original document.

19. The scanner as claimed in claim 13 wherein the mismatch calculator is configured to calculate mismatch index based on output of the comparator.

20. The scanner as claimed in claim 13 wherein the evaluation unit is configured to evaluate output of the mismatch calculator and accordingly provide an output to the user.

21. The scanner as claimed in claim 17 wherein predefined features are such as physical layout of document, layout and structure of documents, textually, location of images, full text check and skew.