A Method For Readout And Residual Non Uniformity Correction Of An Image
Abstract:
A method and a system for readout and residual non-uniformity correction of an image has been described. In one of the embodiments, a fixed pattern noise correction unit is used for the correction of the readout and residual non-uniformity in the captured image and a bilateral filter is used for the removal of granular noise and artifacts and to preserve edges of the captured image.
Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence
Corporate Office, Outer Ring Road, Nagavara, Bangalore – 560045
Inventors
1. Neelabh Keshav
Central Research Laboratory Bharat Electronics Limited, Jalahalli Post, Bangalore-560013
2. Mastan Rao Kongara
Central Research Laboratory Bharat Electronics Limited, Jalahalli Post, Bangalore-560013
3. Uday Kumar Urimi
Central Research Laboratory Bharat Electronics Limited, Jalahalli Post, Bangalore-560013
4. Chinnappa Rajappa Patil
Central Research Laboratory Bharat Electronics Limited, Jalahalli Post, Bangalore-560013
Specification
Claims:
1. A method of correction of readout and residual non-uniformity in input data from electromagnetic radiation sensitive sensors, said method comprising:
dividing said input data of said captured image from said sensors to a plurality of blocks comprising a predetermined number of rows and a predetermined number of columns and designating exclusive numerals for each of said plurality of blocks;
determining reference mean of said each of said plurality of blocks column wise;
determining intermediary mean by performing a running average procedure on said determined reference mean of said each of said plurality of blocks;
determining absolute gradient of said each of said plurality of blocks column wise;
determining absolute mean of said determined absolute gradient of said each of said plurality of blocks column wise;
determining averaging mean by performing a running average procedure on said determined absolute mean of said determined absolute gradient of said each of said plurality of blocks; and
determining gain by using said absolute mean and said averaging mean for each of said plurality of blocks.
2. The method of claim 1, further comprising:
obtaining a first result by subtracting said input data with said reference mean for each of said plurality of blocks;
obtaining a second result by dividing said first result with said gain for each of said plurality of blocks;
obtaining corrected data corresponding to said input data by subtracting said intermediary mean from said second result for each of said plurality of blocks; and
obtaining an image with corrected readout and residual non-uniformity by said obtained corrected data corresponding to said input data.
3. The method of claim 1, wherein said sensors are sensitive to electromagnetic radiations comprising visible band, infrared band, and electromagnetic radiations capable of generating high dynamic range data.
4. The method of claim 1, wherein said input data comprises a captured image with said residual non-uniformity from said sensors.
5. The method of claim 1, wherein granular noise and artifacts in said obtained image with corrected read out and residual non-uniformity is eliminated by a bilateral filter.
, Description:FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
A METHOD FOR READOUT AND RESIDUAL NON-UNIFORMITY CORRECTION OF AN IMAGE
By
BHARAT ELECTRONICS LIMITED
Nationality: Indian
ADDRESS:
OUTER RING ROAD, NAGAVARA, BANGALORE 560045, INDIA
KARNATAKA, INDIA
The following specification particularly describes the invention and the manner in which it is to be performed.
TECHNICAL FIELD
The present disclosure relates generally to the field of signal processing of sensor acquired signals. More specifically, it relates to processing of signals from sensors for the purpose of readout and residual non uniformity correction.
BACKGROUND
A vision camera system comprises sensors sensitive to visible light and/or thermal and/or depending on the desired application and requirement. The vision camera systems are important in industrial, commercial, scientific and military applications such as surveillance, monitoring, inspection and vehicle sight particularly at night. The sensors in one form comprise of a planar array of detectors that are sensitive to selective regions of electromagnetic spectrum. Visible radiations are the electromagnetic radiationsthat are sensed by human eye and range from 400 nanometer to 900 nanometer. Infrared (IR) Radiation is electromagnetic radiation having wavelengths from 700 nanometers to about 1 millimeter. IR Radiations are invisible radiations that may be caused or produced by heat and emitted in proportion to the temperature and emissivity of the object. Regions of wavelength between 3-5 micrometer and 8-14 micrometer correspond to minima in atmospheric absorption enabling their easy detection from a distance. The sensors convert the incident radiations to electrical signals or electrical data that is processed to enable display for one or more viewers on a display device of the vision camera systems. The sensors convert the incident electromagnetic radiation into electrical voltages.
Digital image is an array of two dimensional elements called picture elements or pixels, value of each of which represents the apparent brightness level measured by the image sensor including thermal sensor and vary from image source to image source having a fixed range depending on the sensor. A set of digital image is generated by the vision camera system at a rate called as frame per second where each frame is one digital image. The digital image is a thermal image or infrared image, a charge coupled device (CCD) image, etc.
Thermal imaging is a process of converting infrared (IR) radiation or thermal radiation into visible images that depict the spatial distribution of temperature differences in a scene viewed by the vision camera system. The vision camera system comprises thermal sensors that are semiconductor devices sensitive to thermal signature of a scene and convert the incident thermal radiation to electrical voltages. The thermal signature depends on multiple factors such as shape and size of the object, temperature of the object, atmospheric conditions, etc. The thermal sensors are bonded to complementary metal oxide semiconductor (CMOS) based read-out integrated circuit (ROIC) to form a focal plane array (FPA) and give digital data via an analog-to-digital converter included as part of the infrared sensor which are multiplexed to the ROIC. The FPA is on a ceramic carrier as un-cooled detectors or may be encapsulated in a dewar and cooled by closed cycle sterling cooler as a cooled detectors. Most of the thermal sensor work in these two infrared windows and give digital data having high dynamic range- in order of 8192 to 65536 different discrete levels. The wavelength specified may be overlapping and not limiting.
The focal plane arrays (FPAs) comprise multiple detector elements, wherein each detector has a different gain and offset that change with time, due to FPA fabrication process, sensor operating temperature, temperature of the observed scene, electronic readout noise, etc. The difference in gain and offset amongst detectors produce fixed pattern noise (FPN) in the acquired imagery. The FPN affects the quality of the thermal image leading to generation of digital images with low dynamic range. The thermal images may be of low contrast and noisy and must undergo image processing to produce an enhanced image.
The prior art US7880777B2 mentions that the fixed pattern noise (FPN) observed in thermal images have peculiar characteristics in their pattern, they being predictable and repeatable. But even then, the problem of FPN intensity and polarity still exists. The stated work applied an iterative approach in correcting the fixed pattern noise. The advantage of using the iterative approach is that it uses the live data from the sensor. The methods and systems include correcting IR sensor data different strength levels of FPN data and selecting the particular strength level that produces corrected data with the least amount of FPN. The correction may occur over several frames of IR sensor data in order to find an optimal strength level for correction.
The prior art US20180063454A1 describes a yet another method to eliminate the fixed pattern noise in thermal video image sequence. The algorithm in the stated method is carried out as follows. Initially, a thermal image sequence is captured using a thermal imaging camera system. The average of pixels of the captured thermal image sequence is calculated. In the next step, the pixel values of the current received thermal image are subtracted from the pixel average calculated in the previous step. These pixel differences are then evaluated for updating the required condition. Based on the fulfillment of this updation condition, the required pixels on the image are located and the correction terms for fixed pattern noise are calculated. These fixed pattern noise correction terms are used to correct the next input thermal image frame. A sequence of fixed pattern noise corrected frames is generated in this manner.
The prior art US6473124B1 suggests a line storage system for pattern noise correction in the active pixel sensor. The active pixel sensors include associated circuitry within each pixel for amplifying and processing the signal. This associated circuitry can cause some amount of gain and losses in the signal. The gains and losses introduce a specific pattern. This pattern, which is representative of the associated circuitry, is called fixed pattern noise. As proposed in the given method, the information indicative of the fixed pattern noise is obtained and stored. This information indicative of the fixed pattern noise is then subtracted from the subsequent frames to reduce FPN.
The prior art In US20140037225A1, broadly two claims are made. One for fixed pattern noise removal and the other for non uniformity correction. Methods and systems are provided to reduce the fixed pattern noise in thermal images. In one example, a method includes receiving an image frame comprising a plurality of pixels arranged in a plurality of rows and columns. The pixels comprise thermal image data associated with a scene and noise introduced by an infrared imaging device. The image frame may be processed to determine a plurality of column correction terms, each associated with a corresponding one of the columns and determined based on relative relationships between the pixels of the corresponding column and the pixels of a neighbourhood of columns.
The prior art US6128039 claims the fixed pattern noise removal method from the hardware perspective. It invents a column amplifier for high fixed pattern noise reduction. The column amplifier includes a switching capacitor amplifier, a sample and hold stage, and an output buffer. The switching capacitor amplifier receives signals from a bit line that is coupled to a column of active pixel sensors. The switching capacitor amplifier is capacitively coupled to the bit line from a column of active pixel sensors and is able to cancel the common mode offset in the bit line. The common mode can also be adjusted in the switching capacitor amplifier such that the buffer stage of the column amplifier is not limited by the common mode level of the active pixel sensors. The switching capacitor amplifier includes an input capacitor and a feedback capacitor. The gain of the switching capacitor amplifier amplifies the pixel signals so that the fixed pattern noise introduced by stages after the switching capacitor amplifier will comprise a lower proportion of the total signal.
Therefore there exists a need for a method of correction of readout and residual non-uniformity in an image with low computational and memory requirement for real time applications.
SUMMARY
An aspect of the present invention is to address at least the above-mentioned problems and/or disadvantages and to provide at least the advantages described below.
A method of correction of readout and residual non-uniformity in input data from electromagnetic radiation sensitive sensors. The sensors are sensitive to electromagnetic radiations comprising visible band, infrared band, and electromagnetic radiations capable of generating high dynamic range data. The input data comprises a captured image with the residual non-uniformity from the sensors. The method comprises dividing the input data of a captured image from the sensors to multiple blocks. The blocks comprise a predetermined number of rows and a predetermined number of columns. Each block of the multiple blocks is designated exclusive numerals. Determining reference mean of each block column wise and determining intermediary mean by performing a running average procedure on the determined reference mean of each block. Determining absolute gradient of each block column wise and determining absolute mean of the determined absolute gradient of each block column wise. Determining averaging mean by performing a running average procedure on the determined absolute mean of the determined absolute gradient of each block and determining gain by using the absolute mean and the averaging mean for each block.
The method further comprises obtaining a first result by subtracting the input data with the reference mean for each block. Obtaining a second result by dividing the first result with the gain for each block. Obtaining corrected data corresponding to the input data by subtracting the intermediary mean from the second result for each block. Obtaining an image with corrected readout and residual non-uniformity by the obtained corrected data corresponding to input data. The granular noise and artifacts in the obtained image with corrected read out and residual non-uniformity is eliminated by a bilateral filter.
Other aspects, advantages, and salient features of the invention will become apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses exemplary embodiments of the invention.
BRIEF DESCRIPTION OF DRAWINGS
The above and other aspects, features, and advantages of certain exemplary embodiments of the present invention will be more apparent from the following description taken in conjunction with the accompanying drawings in which:
FIG. 1 exemplarily illustrates a method for readout and residual non-uniformity correction of an image.
FIG. 2 exemplarily illustrates a non-uniformity correction system employed to implement a method for readout and residual non-uniformity correction of an image according to one embodiment of the present invention.
FIG. 3 exemplarily illustrates the steps for readout and residual non-uniformity correction of the image.
FIGS. 4A-4B exemplarily illustrate an image with the fixed pattern noise and an image without the fixed pattern noise that was eliminated by the proposed method respectively.
Persons skilled in the art will appreciate that elements in the figures are illustrated for simplicity and clarity and may have not been drawn to scale. For example, the dimensions of some of the elements in the figure may be exaggerated relative to other elements to help to improve understanding of various exemplary embodiments of the present disclosure. Throughout the drawings, it should be noted that like reference numbers are used to depict the same or similar elements, features, and structures.
DETAILED DESCRIPTION OF DRAWINGS
The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of exemplary embodiments of the invention as defined by the claims and their equivalents. It includes various specific details to assist in that understanding but these are to be regarded as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. In addition, descriptions of well-known functions and constructions are omitted for clarity and conciseness.
The terms and words used in the following description and claims are not limited to the bibliographical meanings, but, are merely used by the inventor to enable a clear and consistent understanding of the invention. Accordingly, it should be apparent to those skilled in the art that the following description of exemplary embodiments of the present invention are provided for illustration purpose only and not for the purpose of limiting the invention as defined by the appended claims and their equivalents.
It is to be understood that the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Thus, for example, reference to “a component surface” includes reference to one or more of such surfaces.
By the term “substantially” it is meant that the recited characteristic, parameter, or value need not be achieved exactly, but that deviations or variations, including for example, tolerances, measurement error, measurement accuracy limitations and other factors known to those of skill in the art, may occur in amounts that do not preclude the effect the characteristic is intended to provide.
FIGS. 1 through 4B, discussed below, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way that would limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged processing system. The terms used to describe various embodiments are exemplary. It should be understood that these are provided to merely aid the understanding of the description, and that their use and definitions, in no way limit the scope of the invention. Terms first, second, and the like are used to differentiate between objects having the same terminology and are in no way intended to represent a chronological order, unless where explicitly stated otherwise. A set is defined as a non-empty set including at least one element.
Those skilled in this technology can make various alterations and modifications without departing from the scope and spirit of the invention. Therefore, the scope of the invention shall be defined and protected by the following claims and their equivalents.
FIGS. 1-4B are merely representational and are not drawn to scale. Certain portions thereof may be exaggerated, while others may be minimized. FIGS. 1-4B illustrate various embodiments of the invention that can be understood and appropriately carried out by those of ordinary skill in the art
In the foregoing detailed description of embodiments of the invention, various features are grouped together in a single embodiment for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments of the invention require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the detailed description of embodiments of the invention, with each claim standing on its own as a separate embodiment.
FIG. 1 exemplarily illustrates the method for readout and residual non-uniformity correction of an image. As used herein, “readout” refers to readout noise due to imperfect conversion of charge into a change in analogue voltage by an on-chip amplifier. As used herein, “residual non-uniformity” refers to noise or a problem that arises from sensor non-uniformities and these non-uniformities arise as each detector of a sensor has a different photo response from a neighbouring detector even though the neighbouring detectors are illuminated by same radiance. The non-uniformity described above results in a fixed pattern noise (FPN).The method comprises receiving input data of a captured image from the sensors and dividing 101 the input data of the captured image from the sensors to multiple blocks comprising a predetermined number of rows and a predetermined number of columns and designating exclusive numerals for each. For example, the received input data is divided to a multiple blocks to obtain f_n (x,y) where n is the block number and the n^th block f_n (x,y). The blocks from f_1 (x,y)to f_n (x,y) is obtained by various methods including dividing the image into blocks of a rows and ß columns where 0
Documents
Application Documents
#
Name
Date
1
201841049469-STATEMENT OF UNDERTAKING (FORM 3) [27-12-2018(online)].pdf
2018-12-27
2
201841049469-FORM 1 [27-12-2018(online)].pdf
2018-12-27
3
201841049469-DRAWINGS [27-12-2018(online)].pdf
2018-12-27
4
201841049469-DECLARATION OF INVENTORSHIP (FORM 5) [27-12-2018(online)].pdf