DESC:TECHNICAL FIELD
[001] The present invention relates generally to a character recognition system for vehicle license plates and method thereof, and more particularly relates to a vehicle number recognition system.
BACKGROUND
[0001] Typically, vehicle license number plate recognition is similar to optical character recognition (OCR) for document processing. Optical character recognition (OCR) engines are used for document processing. However, existing OCR engines cannot be successfully used as license plate readers, because the OCR engines cannot tolerate an extreme range of illumination variations, such as non-homogeneous illumination and shadows, blurring due to dirt, screws, particles etc. In addition, the OCR engines are limited due of their memory and processing speed requirements.

[0002] The prior art discloses different methods of automatic number plate recognition from a front view of a vehicle, and optical character recognition. Different methods of character recognition from documents and number plates are described.

[0003] Patent Number US4949392 titled “Document Recognition and Automatic Indexing for Optical Character Recognition” discloses a character recognition scheme in which a library of templates defining the spacing between pre-printed lines and the corresponding line lengths for a plurality of different business forms is compared with the image data of an unknown document to determine the known business form (template) to which the document corresponds. Once the form of the document is determined, the optical character recognition system may intelligently associate the text characters in certain locations on the document with information fields defined by the pre-printed lines. The pre-printed lines in the image data are determined from the corresponding template and removed from the image data prior to optical character recognition processing.

[0004] Patent Number US7295694 titled “MICR-Based Optical Character Recognition System and Method” discloses a character recognition system and method. The system comprises an optical character reader (OCR) system for collecting character data by electro-optically scanning printed characters, a conversion system for converting the character data to a Magnetic Ink Character Recognition (MICR) format and a recognition engine for interpreting the converted character data using a MICR algorithm.

[0005] Patent Number US6553131 titled “License Plate Recognition with an Intelligent Camera” discloses an intelligent camera system and method for recognizing license plates. The system includes a camera to independently capture the license plate image and a processor for managing image data and executing a license plate recognition program device. The license plate recognition program device includes a program for detecting orientation, position, illumination conditions and blurring of the image and accounting for the orientations, positions, illumination conditions and blurring of the image to obtain a baseline image of the license plate. A segmenting program segments characters depicted in the baseline image by employing a projection along a horizontal axis of the base image to identify positions of the characters. A statistical classifier is adapted for classifying the characters.

[0006] Patent Number US5315664 titled "Number Plate Recognition System" discloses a number plate recognition system that has a structure for inputting an image, structure for calculating outlines of each configuration included in an input image, as well as for recognizing a configuration with the maximal outline size as the plate area. The areas in an image except the above plate area are masked, and the numbers are determined by using Euler numbers after calculating Euler numbers for each configuration within the above plate area.
[0007] There is still a need of an invention which solves the above defined problems and provides a character recognition system that recognizes characters from vehicle license plates.
SUMMARY
[002] This summary is provided to introduce concepts related to recognizing characters from a vehicle number plate. This summary is neither intended to identify essential features of the present invention nor is it intended for use in determining or limiting the scope of the present invention.
[003] For example, various embodiments herein may include one or more character recognition systems and methods for vehicle license plates are provided. In one of the embodiments, the method includes a step of capturing at least one image of a vehicle front view. The method includes a step of storing, in a database, images of a plurality of vehicles, pre-defined templates, structural features characteristic of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates. The method includes a step of detecting a vehicle plate region from the captured image. The method includes a step of localizing the vehicle plate region. The method includes a step of identifying a category of the vehicle plate region from the pre-defined categories stored in the database. The method includes a step of recognizing one or more characters from the vehicle plate region, wherein each of the characters have a signature. The method includes a step of identifying number of rows on the vehicle plate region. The method includes a step of selecting one or more characters from the identified rows of the vehicle plate region. The method includes a step of comparing the one or more characters with the pre-defined templates, and identifying best match characters. The method includes a step of analyzing the best match characters for identifying characters based on the structural features characteristic of each letter. The method includes a step of tuning the identified characters based on the syntax of the vehicle number plates.
[004] In another embodiment, a character recognition system is configured to recognize characters from a vehicle plate. The system includes a capturing module and a processing engine. The capturing module is configured to capture at least one image of a vehicle front view. The processing engine includes a memory, a processor, a database, a localization module, a category identifier, a character recognizer, a template matching module, a structural analyser, and a syntactical analyser. The memory is configured to store pre-determined rules. The processor is configured to generate system processing commands. The database is configured to store images of a plurality of vehicles, pre-defined templates, structural features characteristic of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates. The localization module is configured to detect and localize a vehicle plate region from the captured image. The category identifier is configured to identify a category of the vehicle plate region from the pre-defined categories stored in the database. The character recognizer is configured to recognize one or more characters from the vehicle plate region, wherein each of the characters have a signature. The character recognizer includes a row separator, and a character selector. The row separator is configured to identify the number of rows on the vehicle plate region. The character selector is configured to select one or more characters from the vehicle plate region. The template matching module is configured to compare the one or more characters with the pre-defined templates, and identify best match characters. The structural analyser is configured to analyse the best match characters to identify characters based on the structural features characteristic of each letter. The syntactical analyser is configured to fine tune the identified characters for accuracy based on the syntax of vehicle number plates.
BRIEF DESCRIPTION OF ACCOMPANYING DRAWINGS
[005] The detailed description is described with reference to the accompanying figures. In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The same numbers are used throughout the drawings to reference like features and modules.
[006] Figure 1 illustrates a block diagram depicting a character recognition system for vehicle license plates, according to an exemplary implementation of the present invention.
[007] Figure 2 illustrates a flow chart depicting a method of categorizing a vehicle plate region, according to an exemplary implementation of the present invention.
[008] Figure 3 illustrates a pattern depicting a horizontal level transition pattern for letter ‘A’, according to an exemplary implementation of the present invention.
[009] Figure 4 illustrates a flowchart depicting a method for recognizing characters, according to an exemplary implementation of the present invention.
[0010] It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative systems embodying the principles of the present invention. Similarly, it will be appreciated that any flowcharts, flow diagrams, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.
DETAILED DESCRIPTION
[0011] In the following description, for the purpose of explanation, specific details are set forth in order to provide an understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without these details. One skilled in the art will recognize that embodiments of the present invention, some of which are described below, may be incorporated into a number of systems.
[0012] The various embodiments of the present invention provide a character recognition system for vehicle license plates and method thereof.
[0013] Furthermore, connections between components and/or modules within the figures are not intended to be limited to direct connections. Rather, these components and modules may be modified, re-formatted or otherwise changed by intermediary components and modules.
[0014] References in the present invention to “one embodiment” or “an embodiment” mean that a particular feature, structure, characteristic, or function described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.
[0015] The present invention relates a character recognition system and method thereof for character recognition from vehicle license plate images. The character recognition system for vehicle license plates is configured to identify characters from different varieties of standard number plates which may be either single-row or two-row, based on a highly efficient character extraction and recognition approach. More specifically, the character recognition system is configured to categorize a private/commercial and defense/diplomatic vehicle license plates, simultaneous emphasizing of relevant characters and elimination of irrelevant patterns, row separation in case of two-row number plates, character segmentation and multi-feature based character recognition. The character recognition system robustly detects the relevant characters across different variations of the vehicle plate and then intelligently recognizes them by a combination of structural and syntactical features.
[0016] In an exemplary embodiment, the character recognition system is configured to recognize standard vehicle plates, having characters not connected to each other and in an upright font, covering a broad spectrum of sizes, aspect ratios and colors. Standard private and commercial license plates have black characters embossed on a white or yellow background, either in a single row or two rows. There are also defense and diplomatic license plates having white characters against black, blue or other dark background. The character recognition system is configured to cater to recognize all these categories of license plates in real time.
[0017] In one of the embodiments, the method includes a step of capturing at least one image of a vehicle front view. The method includes a step of storing, in a database, images of a plurality of vehicles, pre-defined templates, structural features characteristic of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates. The method includes a step of detecting a vehicle plate region from the captured image. The method includes a step of localizing the vehicle plate region. The method includes a step of identifying a category of the vehicle plate region from the pre-defined categories stored in the database. The method includes a step of recognizing one or more characters from the vehicle plate region, wherein each of the characters have a signature. The method includes a step of identifying a number of rows on the vehicle plate region. The method includes a step of selecting one or more characters from the identified rows of the vehicle plate region. The method includes a step of comparing the one or more characters with the pre-defined templates, and identifying best match characters. The method includes a step of analyzing the best match characters for identifying characters based on the structural features characteristic of each letter. The method includes a step of tuning the identified characters based on the syntax of the vehicle number plates.
[0018] In another implementation, the method includes a step of enhancing the one or more characters while diminishing an effect of a background of the vehicle plate region.
[0019] In another implementation, the method includes a step of disconnecting the one or more characters having wrongly connected with each other during a binarization technique.
[0020] In another implementation, the method includes a step of performing segmentation of the one or more characters using a vertical profile processing technique. The step of performing segmentation of the one or more characters further includes steps of thresholding and binarizing a filled image of the vehicle plate region, summing of pixels in vertical direction, and replacing the non-zero pixels by zeros in columns with sum less than a preferred threshold for eliminating erroneous connection between two neighboring characters.
[0021] In another implementation, the step of localizing the vehicle plate region includes cropping and selecting the vehicle plate region containing relevant characters from the vehicle plate region.
[0022] In another implementation, the step of cropping the vehicle plate region includes obtaining horizontal and vertical profiles of the vehicle plate region containing the relevant characters by performing summation of each row and column respectively, of the vehicle plate region; selecting at least one edge from a starting point of presence of a character from a horizontal direction, or a vertical direction; and cropping and selecting the vehicle plate region at peak positions with a sufficient tolerance to get a region containing relevant characters.
[0023] In another implementation, the step of recognizing the one or more characters includes resizing the one or more characters into a pre-defined size by eliminating irregular and isolated protrusions.
[0024] In another implementation, the step of identifying the category of the vehicle plate from the pre-defined categories by determining one or more dark characters on a light background, or one or more light characters on a dark background.
[0025] In another implementation, the step of identifying the category of the vehicle plate from the pre-defined categories includes a step of cropping a central region of the vehicle plate region. Further, the step identifying the category includes a step of comparing at least one property of the cropped region and an inverted region of the cropped region. The step identifying the category further includes a step of eliminating the cropped region from a morphologically filled region for finding holes in the cropped region. The step identifying the category further includes a step of eliminating the inverted cropped region from the morphologically filled region for finding holes in the inverted cropped region. The step identifying the category further includes a step of performing binarization and summation techniques on the cropped region and the inverted cropped region. The step identifying the category further includes a step of determining one or more dark characters on the light background by checking whether the sum of the cropped region is higher than the background. The step identifying the category further includes a step of determining one or more light characters on the dark background, by checking whether the sum of the inverted region is higher than the background. The step identifying the category further includes a step of inverting the vehicle plate based on the determined light characters on the dark background, or the dark characters on the light background.
[0026] In another implementation, the morphologically filled region is identified by eliminating the vehicle plate region from the cropped region to bring out the one or more characters.
[0027] In another implementation, identifying the number of rows on the vehicle plate region based on width and aspect ratio parameters of the cropped region, and the row separation for a two row vehicle plate region is identified from a valley near a central region of a horizontal profile.
[0028] In another implementation, the step of recognizing the one or more characters include the steps of identifying the one or more characters by using a combination of Euler number and horizontal and vertical level transition feature vectors, comparing the identified characters with the pre-defined templates and the best match characters, extracting structural features of letters including an Euler number, horizontal and vertical projection patterns, presence of corners, edges and lines at angles, left or right heaviness, and the sum of selected rows or columns; and imposing position based restrictions upon a permissible character set at different locations of a character sequence in the vehicle plate region.
[0029] In another implementation, the step of recognizing the one or more characters includes generating level transition patterns, having a total number of black to white or white to black transitions in a particular direction in a binary image.
[0030] In another embodiment, a character recognition system is configured to recognize characters from a vehicle plate. The system includes a capturing module and a processing engine. The capturing module is configured to capture at least one image of a vehicle front view. The processing engine includes a memory, a processor, a database, a localization module, a category identifier, a character recognizer, a template matching module, a structural analyser, and a syntactical analyser. The memory is configured to store pre-determined rules. The processor is configured to generate system processing commands. The database is configured to store images of a plurality of vehicles, pre-defined templates, structural features characteristic of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates. The localization module is configured to detect and localize a vehicle plate region from the captured image. The category identifier is configured to identify a category of the vehicle plate region from the pre-defined categories stored in the database. The character recognizer is configured to recognize one or more characters from the vehicle plate region, wherein each of the characters have a signature. The character recognizer includes a row separator, and a character selector. The row separator is configured to identify a number of rows on the vehicle plate region. The character selector is configured to select one or more characters from the vehicle plate region. The template matching module is configured to compare the one or more characters with the pre-defined templates, and identify best match characters. The structural analyser is configured to analyse the best match characters to identify characters based on the structural features characteristic of each letter. The syntactical analyser is configured to fine tune the identified characters for accuracy based on the syntax of vehicle number plates.
[0031] In another implementation, the processing engine includes a character emphasizer configured to enhance the one or more characters while diminishing an effect of background of the vehicle plate region.
[0032] In another implementation, the processing engine includes a character separator configured to disconnect the one or more characters having wrongly connected with each other during a binarization technique.
[0033] In another implementation, the character separator is configured to perform segmentation of the one or more characters using a vertical profile processing technique. The character separator includes threshold and binarization of a filled image of the vehicle plate region; summation of pixels in vertical direction; and replacing the non-zero pixels by zeros in columns with sum less than a preferred threshold to eliminate erroneous connection between two neighboring characters.
[0034] In another implementation, the localization module includes a cropping module configured to crop and select the vehicle plate region containing relevant characters from the vehicle plate region.
[0035] In another implementation, the cropping module is configured to obtain horizontal and vertical profiles of the vehicle plate region containing the relevant characters by performing summation of each row and column respectively of the vehicle plate region. Further, the cropping module is configured to select at least one edge from a starting point of presence of the one or more characters from a horizontal direction, or a vertical direction. The cropping module is configured to crop and select the vehicle plate region at peak positions with a sufficient tolerance to get a region containing relevant characters.
[0036] In another implementation, the character recognizer includes a resizing module configured to resize the one or more characters into a pre-defined size, after eliminating irregular and isolated protrusions.
[0037] In another implementation, the category identifier is configured to identify a category of the vehicle plate from the pre-defined categories by determining one or more dark characters on a light background, or one or more light characters on a dark background.
[0038] In another implementation, the category identifier is configured to crop a central region of the vehicle plate region. The category identifier is configured to compare at least one property of the cropped region and an inverted region of the cropped region. The category identifier is configured to eliminate the cropped region from a morphologically filled region to find holes in the cropped region. The category identifier is configured to eliminate the inverted cropped region from the morphologically filled region to find holes in the inverted cropped region. The category identifier is configured to perform binarization and summation techniques on the cropped region and the inverted cropped region. The category identifier is configured to determine one or more dark characters on a light background by checking whether the sum of the cropped image is higher than the background, and also determine one or more light characters on a dark background, by checking whether the sum of the inverted region is higher than the background. The category identifier is configured to invert the vehicle plate based on determined light characters on the dark background, or dark characters on the light background.
[0039] In another implementation, the character recognizer is configured to generate level transition patterns, having a total number of black to white or white to black transitions in a particular direction in a binary image.
[0040] Figure 1 illustrates a block diagram depicting a character recognition system (100) for vehicle license plates, according to an exemplary implementation of the present invention.
[0041] The character recognition system (100) for vehicle license plates (hereinafter referred as “system”) includes a capturing module (102), a network (104), and a processing engine (106). The capturing module (102) is configured to capture at least one image of a vehicle front view. In an embodiment, the capturing module (102) can be a camera. In one embodiment, the capturing module (102) can be a digital camera, which is configured to capture high resolution images of a vehicle front view. In another embodiment, the capturing module (102) is configured to capture high resolution images of a moving vehicle, and transmits the images to the processing engine (106) for further processing.
[0042] The capturing module (102) is configured to communicatively coupled with the processing engine (104) via a network (104). In one embodiment, the network (104) includes wired and wireless networks. Examples of the wired networks include a Wide Area Network (WAN) or a Local Area Network (LAN), a client-server network, a peer-to-peer network, and so forth. Examples of the wireless networks include Wi-Fi, a Global System for Mobile communications (GSM) network, and a General Packet Radio Service (GPRS) network, an enhanced data GSM environment (EDGE) network, 802.5 communication networks, Code Division Multiple Access (CDMA) networks, or Bluetooth networks.
[0043] The processing engine (106) includes a memory (108), a processor (110), a database (112), a localization module (114), a category identifier (118), a character recognizer (120), a template matching module (128), a structural analyser (130), and a syntactical analyser (132).
[0044] The memory (108) is configured to store pre-determined rules related to recognizing characters. In an embodiment, the memory (108) can include any computer-readable medium known in the art including, for example, volatile memory, such as static random-access memory (SRAM) and dynamic random-access memory (DRAM), and/or non-volatile memory, such as read only memory (ROM), erasable programmable ROM, flash memories, hard disks, optical disks, and magnetic tapes. The memory (108) also includes a cache memory to work with the system (100) more effectively.
[0045] The processor (110) is configured to cooperate with the memory (108) to receive the pre-determined rules. The processor (110) is further configured to generate system processing commands. In an embodiment, the processor (110) can be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and/or any devices that manipulate signals based on operational instructions. Among other capabilities, at least one processor (110) is configured to fetch the pre-determined rules from the memory (110) and execute different modules of the system (100).
[0046] The database (112) is configured to store images of a plurality of vehicles, which are captured by the capturing module (102), pre-defined templates, structural features characteristic of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates. In an embodiment, the database (112) includes a look up table configured to store data. In one embodiment, the database (112) can be implemented as enterprise database, remote database, local database, and the like. The database (112) can be located within the vicinity of the processing engine (106) or can be located at different geographic locations as compared to that of the processing engine (106). In another embodiment, the database (112) includes multiple databases, such as enterprise database, remote database, local database, and the like. These multiple databases may themselves be located either within the vicinity of each other or may be located at different geographic locations. Furthermore, the database (112) may be implemented inside the processing engine (106) and the database (112) may be implemented as a single database.
[0047] The localization module (114) is configured to detect and localize a vehicle plate region from the captured image of the vehicle front view. In an embodiment, the localization module (114) is configured to extract image features characteristic of the vehicle plate region from the captured image of the vehicle front view to locate a vehicle plate accurately.
[0048] In an embodiment, the localization module (114) includes a cropping module (116). The cropping module (116) is configured to crop and select the vehicle plate region containing relevant characters. The cropping module (116) is configured to crop and select the vehicle plate region by obtaining horizontal and vertical profiles of the vehicle plate region containing the relevant characters by performing summation on each row and column respectively. The cropped vehicle plate region plate region becomes the input for the character recognition system (100). In an embodiment, the cropped vehicle plate region plate region is a monochrome image. Further, the cropping module (116) is configured to select at least one edge from a starting point of the presence of the one or more characters from a horizontal direction, or a vertical direction, and crop and select the vehicle plate region at peak positions with a sufficient tolerance to get a region containing relevant characters.
[0049] The category identifier (118) is configured to cooperate with the database (112) and the localization module (114). The category identifier (118) is further configured to identify a category of the vehicle plate region from the pre-defined categories stored in the database (112). In an embodiment, the category identifier (118) is configured to identify a category of the vehicle plate region from the pre-defined categories by determining one or more dark characters on a light background, or one or more light characters on a dark background. In one embodiment, the category identifier (118) is configured to identify the category of the vehicle plate region by cropping a central region of the vehicle plate region, and inverting the cropped central region of the vehicle plate region. Further, the category identifier (118) is configured to compare at least one property of the cropped region with an inverted version of the cropped central region. The category identifier (118) is configured to eliminate the cropped central region from a morphologically filled region to find holes in the cropped central region, and also eliminate the inverted cropped image from the morphologically filled region to find holes in the inverted cropped region. In an embodiment, the morphologically filled region is identified by eliminating the vehicle plate region from the cropped region to bring out the one or more characters. The category identifier (118) is configured to perform binarization and summation techniques on the cropped central region and the inverted cropped region, and determine one or more dark characters on a light background by checking whether the sum of the cropped central region is higher than the background. Furthermore, the category identifier (118) is configured to determine one or more light characters on a dark background, by checking whether the sum of the inverted region is higher than the background. After determination, the category identifier (118) is configured to invert the vehicle plate region based on the determined light characters on the dark background, or dark characters on the light background.
[0050] More specifically, the category identifier (118) is configured to determine whether the vehicle plate region has dark letters on a light background (corresponding to private/commercial vehicles) or light letters on a dark background (corresponding to defense/diplomatic vehicles).
[0051] In another embodiment, a color based segmentation process is used to identify whether the vehicle plate region has a distinctive yellow background. If a single large connected patch with hue and saturation values corresponding to the yellow color is obtained during segmentation, then it is established that the vehicle plate region belongs to a commercial vehicle. However, such color based segmentation is not robust enough to distinguish all possible background colors under different environmental conditions. Hence, further analysis is required to identify plate category in cases where it cannot be categorically established that the vehicle plate region is having a yellow background. In one embodiment, the vehicle plate region is cropped from left and right ends and central columns alone are used for categorization. A decision is reached based on the properties of the cropped region, and the inverted form of the cropped region. The inverted cropped region is obtained by subtracting the pixel values from 255 for an 8-bit gray scale region/image. An image filling operation is performed on the cropped region. A morphologically filling refers to filling holes in the gray scale image where a hole is defined as an area of dark pixels surrounded by lighter pixels. This is followed by a differencing operation in which the cropped region is eliminated from the morphologically filled cropped region. This brings out the holes alone to prominence. In the current scenario, this means that for a vehicle plate region with dark characters on a lighter background, the characters alone will be highlighted. But, for the other case with light characters on a dark background, very few pixels will get emphasized, while majority will be eliminated out. The objective of the sequence of operations described here is to quantify the observation and derive a classification rule based on this information.
[0052] In another embodiment, binarization and summation techniques are applied on the differenced output. Similar operations are carried out on the inverted cropped region. A comparison is made between the sum values of the processed cropped region and the inverted cropped region. If cropped region sum exceeds the inverted cropped region sum, the category identifier (118) is concluded that the vehicle plate region has dark characters on a lighter background and is categorized as a private/commercial vehicle. Otherwise, the vehicle plate region is classified as a defense/diplomatic whose number plate has light characters against a dark background. Such a vehicle plate region is inverted to get dark characters on the light background and fed to the subsequent steps so that the remaining operations can be same regardless of the category.
[0053] In an embodiment, the processing engine (106) further includes a character emphasizer (134) and a character separator (136).
[0054] The character emphasizer (134) is configured to enhance the one or more characters while diminishing the effect of the background of the vehicle plate region.
[0055] In an embodiment, once the category is determined, morphologically cropped region and differencing operations are applied on either a cropped image or its inverted form based on the category of the vehicle plate region. An input gray scale image is filled followed by eliminating the cropped region from the morphologically filled region. The differenced region is then binarized using an adaptive global threshold. Some irregular protrusions may be present in the binarized region, which can cause two subsequent characters to get connected to each other. Separation of connected characters is achieved through vertical profile processing. This involves summation of pixels in each column and elimination of non-zero pixels in columns with sum less than a low threshold value as they are assumed to be spurious effects of binarization. The threshold values for character separation are pre-set by the user. These values are set very close to zero in order to separate out two characters getting connected accidentally during the binarization technique.
[0056] In another embodiment, an exact localization of the vehicle plate region containing characters can be achieved by analyzing horizontal and vertical profiles. The row and column sum along edges of the binary region gets low due to absence of characters. Therefore, starting from extreme ends on all four sides, the first surge in row/column sum is considered as start of the character presence from that side. The vehicle plate region is cropped till these points to exactly obtain the character area. This step is significant because size and aspect ratio of the character area provides an insight into number of rows in the vehicle plate region. If the vehicle plate geometry indicates that there are two lines in the vehicle plate region, then a row separation procedure is invoked to identify the two lines. This is based on the vehicle plate horizontal profile and finding the position corresponding to a minimum in projection profile around the central portion. The vehicle plate is divided into two rows along this horizontal line in the region.
[0057] The character separator (136) is configured to disconnect the one or more characters having wrongly connected with each other during a binarization technique. In an embodiment, the binarization technique converts a pixel image to a binary image. The character separator (136) is configured to perform segmentation of the one or more characters using a vertical profile processing technique, which includes threshold and binarization of the morphologically filled image of the vehicle plate region, summation of pixels, in vertical direction, and replacing the non-zero pixels by zeros in columns less than a preferred threshold to eliminate erroneous connection between two neighboring characters. In the vehicle plate region, where most of the elements are plate characters, elements located far away from average of row positions of all elements or having heights greatly different from a mean height can be discarded as spurious ones. The binarization technique is done based on a standard adaptive algorithm. An optimum threshold for minimizing the intra-class variance is obtained using Otsu’s method. Once the image is binarized, the foreground pixels have the value ‘1’ and background pixels are ‘0’. The sum of all the pixels is obtained and used for further decision making steps.
[0058] The character recognizer (120) is configured to cooperate with the character separator (136). The character recognizer (120) is configured to recognize one or more characters from the vehicle plate region, wherein each of the characters have a signature. In an embodiment, the character recognizer (120) is configured to generate level transition patterns, having a total number of black to white or white to black transitions in a particular direction. The character recognizer (120) includes a row separator (122), and a character selector (124). The row separator (122) is configured to identify a number of rows on the vehicle plate region. The character selector (124) is configured to cooperate with the row separator (122). The character selector (124) is further configured to select one or more characters from the vehicle plate region.
[0059] In an embodiment, the row separator (122) is configured to perform operations from left to right to get the characters of number plate in a proper order. In case of a multi-row plate, the row separator (122) analyses sequentially for the two rows from top to bottom. Elimination of the connected components which do not satisfy a proper size, aspect ratio and position criteria suited to characters of the vehicle plate region is also done at this stage.
[0060] In an embodiment, the character recognizer (120) includes a resizing module (126). The character resizing module (126) is configured to resize the one or more characters into a pre-defined size, by eliminating irregular and isolated protrusions. The character resizing module (126) is also configured to remove isolated pixels with an irregular appearance from the characters before bringing it to a pre-defined size. This is to ensure maximum correct recognitions. The characters can then be fed into the character recognizer (120) one after the other to get results.
[0061] In an embodiment, a transition count is performed on a binary image of the character, containing only two levels (white for the foreground character and black for the background). Transitions are counted both in horizontal and vertical directions. To count the transitions in horizontal direction, the number of changes from black to white and white to black in left to right direction of the character at every row of the image is checked. If the transition count is same for subsequent rows, then that particular value is considered only once while forming the horizontal level transition pattern. Similarly, to count the transitions in vertical direction, the number of changes from black to white and white to black in top to bottom direction of the character at every column is checked. If the transition count is same for subsequent columns, then that particular value is considered only once while forming the vertical level transition pattern.
[0062] In an embodiment, the character recognizer (120) is configured to recognize one or more characters by identifying characters using a combination of Euler number and horizontal and vertical level transition feature vectors. The character recognizer (120) is further configured to compare the identified characters with the pre-defined templates and the best match characters. Furthermore, the character recognizer (120) is configured to extract structural features of letters including Euler number, horizontal and vertical projection patterns, horizontal and vertical level transition feature vectors, presence of corners, edges and lines at angles, left or right heaviness, and the sum of selected rows or columns. Subsequently, the character recognizer (120) is configured to impose position based restrictions upon a permissible character set at different locations of a character sequence in the vehicle plate region. In one embodiment, recognizing the one or more characters includes generating level transition patterns, having a total number of black to white or white to black transitions in a particular direction in a binary image.
[0063] The template matching module (128) is configured to cooperate with the character recognizer (120) and the database (112). The template matching module (128) is configured to compare the one or more characters with the pre-defined templates, and identify best match characters. In an embodiment, remaining characters that are not yet identified are processed by the template matching module (128). Each of the test character is compared against entire template library, which contains instances of all relevant characters in a pre-defined font size. Thus, a best match is obtained corresponding to each character. This decision can be modified once the character is passed through subsequent modules.
[0064] The structural analyser (130) is configured to cooperate with the template matching module (128). The structural analyser (130) is configured to analyse the best match characters to identify characters based on the structural features characteristic of each letter. The preliminary structural analysis is aimed at classification of a few characters using a relatively less number of features. Characters possessing a unique signature corresponding to these structural features can be identified at this stage without passing them through the whole process. The features are Euler number and level transition patterns. The Euler number of a binary image is a scalar whose value is the total number of objects (connected components) in the vehicle plate region minus the total number of holes in those objects. For example, digit ‘8’ (as written here) contains a single object with two holes and hence has a Euler number -1.
[0065] In another embodiment, the structural analyser (130) is configured to analyse the best match characters based on combination of a number of features which aids in distinguishing between confusing characters. This implies that the set of structural features used for identification of different characters are different, depending on the specific character to be identified. The features of the binary image of a character in the pre-defined size used for classification are given below:
• Euler Number of the vehicle plate region or a part of the vehicle plate region as such or with modifications imposed by the character recognition system (100).
• Number of peaks encountered in horizontal and vertical projections of the vehicle plate region.
• Sum of specific regions within the binary image of the pre-defined size.
• Distribution of percentage of pixels in left and right halves or top and bottom halves of the vehicle plate region.
• Presence of straight lines, edges and corners in specific regions.
• First presence of a non-zero pixel from left or right at specific row positions.
[0066] In an exemplary embodiment, in order to illustrate the usefulness of these features, suppose template matching has found the best match of a character as letter ‘Z’. The other contestants for this character are ‘2’ and ‘7’ due to their similarity in appearance with ‘Z’. If sum of a few rows at the bottom is greater than a particular threshold, then the possibility of ‘7’ can be ruled out. Out of the remaining two choices, ‘Z’ has a sharp corner at the top right whereas ‘2’ has a rounded corner. If top right corner sum exceeds an empirical threshold, then the character can be classified as ‘Z’. A non-perfectly captured ‘Z’ need not satisfy this condition at all times. In such a situation, the decision is based on thresholding the sum of a few rows at the top since ‘Z’ has a flat top (indicating higher sum) whereas ‘2’ has a curved top (indicating lower sum). In cases where the possibility of ‘7’ remains intact, a further validation is done by counting the number of peaks in row projection. For ‘7’, there exists a single peak at the top whereas two peaks can be obtained for ‘2’ and ‘Z’ by appropriate selection of the threshold.
[0067] The syntactical analyser (132) is configured to cooperate with the structural analyser (130). The syntactical analyser (132) is configured to fine tune the identified characters for accuracy based on the syntax of the vehicle number plates.
[0068] In an embodiment, the positions of alphabets and digits, number of characters in the vehicle plate region, state codes etc. have some permissible values according to motor vehicle rules. These factors are utilized efficiently to improve plate recognition accuracy. For example, suppose the first character has been identified as ‘0’ for a commercial plate. However, the first character has to be compulsorily an alphabet for commercial vehicles. Therefore, this character can be unambiguously replaced with ‘O’. As another instance, consider that a license number obtained at output of the syntactical analyser (132) is ‘MH01AD34Z1’. Here, it is apparent that the last four characters must be actually digits. Hence ‘Z’ in the last but one position can be replaced by ‘2’ (features used are discriminative enough to rule out all other possibilities). Such rules are carefully applied to get the final character recognition output of the system (100).
[0069] Figure 2 illustrates a flow chart depicting a method of categorizing a vehicle plate region, according to an exemplary implementation of the present invention.
[0070] The flowchart (200) starts at step (202), providing a vehicle plate region as an input. In an embodiment, a localization module (114) is configured to provide a vehicle plate region as an input. At step (204), determining whether the vehicle plate region having a yellow background is done. If the vehicle plate region includes the yellow background, the category of a vehicle plate region is a commercial vehicle, as shown in step (206). In an embodiment, a category identifier (118) is configured to identify the category of the vehicle plate region from the pre-defined categories stored in a database (112). If the vehicle plate region does not include the yellow background, then cropping the vehicle plate region from left and right ends is performed, as shown in step (208). In an embodiment, the category identifier (118) is configured to crop the vehicle plate region from left and right ends. At step (212), inverting the cropped region. In an embodiment, the category identifier (118) is configured to invert the cropped region. At step (210), morphologically filling of the cropped region. In an embodiment, the category identifier (118) is configured to morphologically fill the cropped region. At step (214), eliminating the cropped region from the morphologically filled region. In an embodiment, the category identifier (118) is configured to eliminate the cropped region from the morphologically filled region. At step (216), binarizing the cropped region. In an embodiment, the category identifier (118) is configured to binarize the cropped region. At step (218), summing the cropped region. In an embodiment, the category identifier (118) is configured to perform a summation technique on the cropped region. At step (220), morphologically filling of the inverted cropped region. In an embodiment, the category identifier (118) is configured to morphologically fill the inverted cropped region. At step (222), eliminating the inverted cropped region from the morphologically filled region. In an embodiment, the category identifier (118) is configured to eliminate the inverted cropped region from the morphologically filled region. At step (224), binarizing the inverted cropped region. In an embodiment, the category identifier (118) is configured to binarize the inverted cropped region. At step (226), summing the inverted cropped region. In an embodiment, the category identifier (118) is configured to perform a summation technique on the inverted cropped region. At step (228), checking whether the cropped region sum is greater than the sum of the inverted cropped region. In an embodiment, the category identifier (118) is configured to check whether the cropped region sum is greater than the sum of the inverted cropped region. If the cropped region sum is greater than the sum of the inverted cropped region, the category of a vehicle plate region is a private/commercial vehicle, as shown in step (230). In an embodiment, a category identifier (118) is configured to identify the category of the vehicle plate region from the pre-defined categories stored in a database (112). If the cropped region sum is lesser than the sum of the inverted cropped region, the category of a vehicle plate region is a defense/ diplomatic vehicle, as shown in step (232). In an embodiment, a category identifier (118) is configured to identify a category of the vehicle plate region from the pre-defined categories stored in a database (112).
[0071] Figure 3 illustrates a pattern depicting a horizontal level transition pattern for letter ‘A’, according to an exemplary implementation of the present invention.
[0072] In an embodiment, a level transition pattern is the pattern of a total number of black to white or white to black transitions in either horizontal or vertical direction. In order to get a unique vector irrespective of the size of the object, a sequence of a same number of transitions is merged into a single number. Letter ‘A’, as given in Figure 3, can be considered as an example. As the letter ‘A’ is traversed from left to right, from top row to bottom row, there is a variation in the pattern of level transitions. As far as top rows are concerned, there is one white to black transition followed by a black to white transition (total number of transitions=2) (302). Coming down, the pattern becomes white to black, black to white, white to black and again black to white (total number of transitions=4) (304). At the region of horizontal stroke in the letter, the pattern is white to black and then black to white (total number of transitions=2) (306). Below the horizontal line, the previous pattern is again repeated (total number of transitions=4) (308). Thus, horizontal level transition feature vector for letter ‘A’ is obtained as [2 4 2 4]. Similarly, the vertical level transition vector is [2 4 2].
[0073] Figure 4 illustrates a flowchart depicting a method for recognizing characters, according to an exemplary implementation of the present invention.
[0074] The flowchart (400) starts at step (402), capturing at least one image of a vehicle front view. In an embodiment, a capturing module (102) is configured to capture at least one image of a vehicle front view.
[0075] At step (404), storing, in a database (114), images of a plurality of vehicles, pre-defined templates, structural features characteristic of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates. In an embodiment, a database (114) is configured to store images of a plurality of vehicles, pre-defined templates, structural features characteristics of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates.
[0076] At step (406), detecting a vehicle plate region from the captured image. In an embodiment, a localization module (114) is configured to detect a vehicle plate region from the captured image.
[0077] At step (408), localizing the vehicle plate region. In an embodiment, the localization module (114) is configured to localize the vehicle plate region.
[0078] At step (410), identifying a category of the vehicle plate region from the pre-defined categories stored in the database (114). In an embodiment, a category identifier (118) is configured to identify a category of the vehicle plate region from the pre-defined categories stored in the database (114).
[0079] At step (412), recognizing one or more characters from the vehicle plate region, wherein each of the characters have a signature. In an embodiment, a character recognizer (120) is configured to recognize one or more characters from the vehicle plate region, wherein each of the characters have a signature.
[0080] At step (414), identifying a number of rows on the vehicle plate region. In an embodiment, a row separator (122) is configured to identify a number of rows on the vehicle plate region.
[0081] At step (416), selecting one or more characters from the identified rows of the vehicle plate region. In an embodiment, a character selector (124) is configured to select one or more characters from the identified rows of the vehicle plate region.
[0082] At step (418), comparing the one or more characters with the pre-defined templates, and identifying best match characters. In an embodiment, a template matching module (128) is configured to compare the one or more characters with the pre-defined templates, and identify best match characters.
[0083] At step (420), analysing the best match characters for identifying characters based on the structural features characteristic of each letter. In an embodiment, a structural analyser (130) is configured to analyse the best match characters for identifying characters based on the structural features characteristic of each letter.
[0084] At step (422), tuning the identified characters based on the syntax of the vehicle number plates. In an embodiment, a syntactical analyser (132) is configured to fine tune the identified characters based on the syntax of the vehicle number plates.
,CLAIMS:
1. A method for recognizing characters, comprising:
capturing at least one image of a vehicle front view;
storing, in a database (112), images of a plurality of vehicles, pre-defined templates, structural features characteristics of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates;
detecting a vehicle plate region from said captured image;
localizing said vehicle plate region;
identifying a category of said vehicle plate region from said pre-defined categories stored in said database (112);
recognizing one or more characters from said vehicle plate region, wherein each of said characters have a signature;
identifying a number of rows on said vehicle plate region;
selecting one or more characters from said identified rows of said vehicle plate region;
comparing said one or more characters with said pre-defined templates, and identifying best match characters;
analysing said best match characters for identifying characters based on said structural features characteristic of each letter; and
tuning said identified characters based on said syntax of said vehicle number plates.

2. The method as claimed in claim 1, wherein said method includes enhancing said one or more characters while diminishing the effect of a background of said vehicle plate region.

3. The method as claimed in claim 1, wherein said method includes disconnecting said one or more characters having wrongly connected with each other during a binarization technique.

4. The method as claimed in claim 1, wherein said method includes performing segmentation of said one or more characters using a vertical profile processing technique, which includes:
thresholding and binarizing a morphologically filled image of said vehicle plate region;
summing of pixels in vertical direction; and
replacing the non-zero pixels by zeros in columns with sum less than a preferred threshold for eliminating erroneous connection between two neighboring characters.

5. The method as claimed in claim 1, wherein localizing said vehicle plate region includes cropping and selecting said vehicle plate region containing relevant characters from the vehicle plate region.

6. The method as claimed in claim 5, wherein cropping said vehicle plate region includes:
obtaining horizontal and vertical profiles of said vehicle plate region containing said relevant characters by performing summation on each row and column respectively;
selecting at least one edge from a starting point of presence of a character from a horizontal direction, or a vertical direction.
cropping and selecting said vehicle plate region at peak positions with a sufficient tolerance to get a region containing relevant characters.

7. The method as claimed in claim 1, wherein recognizing said one or more characters includes resizing said one or more characters into a pre-defined size by eliminating irregular and isolated protrusions.

8. The method as claimed in claim 1, wherein identifying said category of said vehicle plate region from said pre-defined categories by determining one or more dark characters on a light background, or one or more light characters on a dark background.

9. The method as claimed in claim 8, wherein identifying said category of said vehicle plate from said pre-defined categories includes:
cropping a central region of said vehicle plate region;
inverting said cropped central region of said vehicle plate region;
comparing at least one property of said cropped central region with an inverted region of said cropped central region;
eliminating said cropped region from said morphologically filled region for finding holes in said cropped region;
eliminating said inverted cropped region from said morphologically filled region for finding holes in said inverted cropped region;
performing binarization and summation techniques on said cropped central region and said inverted cropped region;
determining one or more dark characters on said light background by checking whether the sum of the cropped central region is higher than the background;
determining one or more light characters on said dark background, by checking whether the sum of the inverted region is higher than the background; and
inverting said vehicle plate region based on said determined light characters on said dark background, or said dark characters on said light background.

10. The method as claimed in claim 9, wherein said morphologically filled region is identified by eliminating said vehicle plate region from said cropped region to bring out said one or more characters.

11. The method as claimed in claim 1, wherein identifying said number of rows on said vehicle plate region based on width and aspect ratio parameters of the cropped region, and the row separation for a two row vehicle plate region is identified from a valley near a central region of a horizontal profile.

12. The method as claimed in claim 1, wherein recognizing said one or more characters include:
identifying said one or more characters by using a combination of a Euler number and horizontal and vertical level transition feature vectors;
comparing said identified characters with said pre-defined templates and said best match characters;
extracting structural features of letters including a Euler number, horizontal and vertical projection patterns, presence of corners, edges and lines at angles, left or right heaviness, and the sum of selected rows or columns; and
imposing position based restrictions upon a permissible character set at different locations of a character sequence in said vehicle plate region.

13. The method as claimed in claims 1 and 12, wherein recognizing said one or more characters includes generating level transition patterns, having a total number of black to white or white to black transitions in a particular direction.

14. A character recognition system (100), comprising:
a capturing module (102) configured to capture at least one image of a vehicle front view; and
a processing engine (106) configured to cooperate with said capturing module (102), said processing engine (106) comprising:
a memory (108) configured to store pre-determined rules;
a processor (110) configured to cooperate with said memory (108), said processor (110) configured to generate system processing commands;
a database (112) configured to store images of a plurality of vehicles, pre-defined templates, structural features characteristics of letters, at least one syntax of vehicle number plates, and pre-defined categories of vehicle plates;
a localization module (114) configured to detect and localize a vehicle plate region from said captured image;
a category identifier (118) configured to cooperate with said database (112) and said localization module (114), said category identifier (118) configured to identify a category of said vehicle plate region from said pre-defined categories stored in said database;
a character recognizer (120) configured to recognize one or more characters from said vehicle plate region, wherein each of said characters having a signature, said character recognizer includes:
a row separator (122) configured to identify a number of rows on said vehicle plate region; and
a character selector (124) configured cooperate with said row separator (122), said character selector (124) configured to select one or more characters from said vehicle plate region;
a template matching module (128) configured to cooperate with said character recognizer (120) and said database (112), said template matching module (128) configured to compare said one or more characters with said pre-defined templates, and identify best match characters;
a structural analyser (130) configured to cooperate with said template matching module (128), said structural analyser (130) configured to analyse said best match characters to identify characters based on said structural features characteristic of each letter; and
a syntactical analyser (132) configured to cooperate with said structural analyser, said syntactical analyser (132) configured to fine tune said identified characters based on said syntax of vehicle number plates.

15. The system (100) as claimed in claim 14, wherein said processing engine (106) includes a character emphasizer (134) configured to enhance said one or more characters while diminishing an effect of a background of said vehicle plate region.

16. The system (100) as claimed in claim 14, wherein said processing engine (106) includes a character separator (136) configured to disconnect said one or more characters having wrongly connected with each other during a binarization technique.

17. The system (100) as claimed in claim 14, wherein said character separator (136) is configured to perform segmentation of said one or more characters using a vertical profile processing technique, which includes:
threshold and binarization of a morphologically filled image of said vehicle plate region;
summation of pixels in vertical direction; and
replacing the non-zero pixels by zeros in columns with sum less than a preferred threshold to eliminate erroneous connection between two neighboring characters.

18. The system (100) as claimed in claim 14, wherein said localization module (114) includes a cropping module (116) configured to crop and select said vehicle plate region containing relevant characters from the vehicle plate region.

19. The system (100) as claimed in claim 18, wherein said cropping module (116) is configured to perform:
obtain horizontal and vertical profiles of said vehicle plate region containing said relevant characters by performing summation on each row and column respectively;
select at least one edge from a starting point of presence of said one or more characters from a horizontal direction, or a vertical direction.
crop and select said vehicle plate region at peak positions with a sufficient tolerance to get a region containing relevant characters.

20. The system (100) as claimed in claim 14, wherein said character recognizer (120) includes a resizing module (126) configured to resize said one or more characters into a pre-defined size, by eliminating irregular and isolated protrusions.

21. The system (100) as claimed in claim 1, wherein said category identifier (118) is configured to identify a category of said vehicle plate region from said pre-defined categories by determining one or more dark characters on a light background, or one or more light characters on a dark background.

22. The system (100) as claimed in claim 21, wherein said category identifier (118) is configured to:
crop a central region of said vehicle plate region;
invert said cropped central region of said vehicle plate region;
compare at least one property of said cropped region with said inverted region of said cropped region;
eliminate said cropped region from a morphologically filled region to find holes in said cropped region;
eliminate said inverted cropped region from said morphologically filled region to find holes in said inverted cropped region;
perform binarization and summation techniques on said cropped region and said inverted cropped region;
determine one or more dark characters on a light background by checking whether the sum of the cropped central region is higher than the background;
determine one or more light characters on a dark background, by checking whether the sum of the inverted region is higher than the background; and
invert said vehicle plate region based on determined light characters on said dark background, or dark characters on said light background.

23. The system (100) as claimed in claim 1, said character recognizer (120) configured to generate level transition patterns, having a total number of black to white or white to black transitions in a particular direction.