DESC:TECHNICAL FIELD
[1] The present invention relates to the general area of image processing devices, and methods thereof, and more specifically to noise robust detail enhancement techniques.

BACKGROUND
[2] Generally, modern day imaging sensors produce high dynamic range (HDR) raw images, which are generally of 14 or 16 bit resolution. As human visual system can distinguish approximately 128 gray levels, the 14-bit (> 16K levels of intensity values) information generated by the sensor has to be mapped to 7-bits. Further, most of the display devices require 8-bit values which effectively limit the output dynamic range of the imaging system to 256 levels.
[3] One of the existing arts, US Patent No. 6847738-B1, discusses a method where input image is filtered both spatially and temporally to obtained filtered signal. The input image (a frame of video signal) is passed through a spatial high pass filter, and a set of consecutive frames in a neighborhood around input image (frame) is passed through a temporal high pass filter. The output of spatial high pass filter is multiplied by a controllable fraction to obtain multiplied signal. Then the output of temporal high pass filtered is subtracted from this multiplied signal which is added back to input signal to generate enhanced output image. The controllable fraction is generated by a non-linear function.
[4] One of the existing arts, US Patent No. 0279730-A1, discusses a method where image is sharpened through an appropriate high-frequency compensation. An HPF extracts high frequency components from input image signal which is passed through a square operator followed by first differentiator to generate first differentiation signal. A second differentiator generates second differentiation signal from the input image signal. The first and second differentiation signals are multiplied in a multiplier to generate a multiplied signal. This multiplied signal is added with input image signal to generate output image signal.
[5] One of the existing arts, US Patent No. 7515765-B1, discusses a method where image is sharpened by processing in frequency domain. The input image is first transformed to frequency domain with predefined frequency bands. The amplitude of each of these frequency bands is adjusted by some weights. These weights are calculated either based on expected viewing distance of observer or based on the ability of display device to reproduce spatial frequencies in image. Weighted frequency bands are normalized to preserve overall power and transformed back to spatial domain from frequency domain to generate output sharpened image.
[6] One of the existing arts, US Patent No. 0189373-A1, discusses a method for adaptive sharpness enhancement. The method first generates two versions of sharpened image viz. linear sharpened image and non-linear sharpened image via linear sharpening and non-linear sharpening respectively. Then the two versions of sharpened images along with input image are blended together to generate output sharpened image. The blending parameters are generated using input image, linear sharpened image and non-linear sharpened image.
[7] One another existing art, US Patent No. 0150332-A1, discusses a method where image is sharpened by first converting the RGB image into a single channel of intensity data. This intensity data is processed to generate integral image data. In order to sharpen the image, a gain factor is calculated for each pixel. This gain factor is computed by applying a variable size filter to the integral image data to generate box filter data. The size of box filter for each pixel is estimated depending upon the amount of local blur about that pixel, thus as the blur changes, filter size also changes appropriately. The gain factor depends upon intensity data, box filter data and the filter size. Once gain factor is computed for each pixel, it is multiplied with original RGB data at corresponding pixel to generate output sharpened image.
[8] One of the existing arts, US Patent No. 6252995-B1, discusses a method for adaptive edge enhancement by carrying out sharpening only at non-uniform regions and not at uniform regions of image. The differentiation between uniform and non-uniform region is carried out by comparing a difference signal for each small region of image with a predefined threshold. This difference signal is computed by taking difference between the maximum and minimum value in a region. When the difference signal is more than threshold, corresponding region is declared as non-uniform and edge enhancement is carried out, while when the difference signal is less than threshold, corresponding region is declared as uniform and edge enhancement is not carried out.
[9] One of the existing arts, US Patent No. 0094890-A1, discusses a method for detecting slant edges and avoiding zigzagged slant edge artifacts in an image detail enhancement process. First the slant edges are detected, and a gain suppression factor is estimated for each pixel. Then the details in image are enhanced selectively using gain suppression factor to ensure that pixels detected on slant edge are not enhanced much relative to other pixels.
[10] One another existing art, PCT Patent Application No. WO 2009/008778 A1, discusses a method for infrared image processing wherein at-least one embodiment includes filtering of infrared image to provide a background portion of infrared image and a detail portion of infrared image. The background portion and/or the detail portion is scaled to provide a level of the detail portion relative to a level of the background portion. Then histogram equalization is performed on the background portion and the background portion is merged with detail portion after scaling to provide processed infrared image.
[11] Hence, there is a need of methods for compressing high dynamic range of these images to suit 8-10 bit displays. In addition to compressing dynamic range, these methods should also ensure that the perceptual image quality is not compromised, i.e. the methods should be capable of enhancing all minute details present in raw image while suppressing maximum noise content.
[12] For the reasons stated above, which will become apparent to those skilled in the art upon reading and understanding the specification, there is a need in the art for a system and method thereof for noise robust digital detail enhancement of high dynamic range infrared images that rules out all the void and tilting issues of the visual quality of high dynamic range image thereby solves all the issues and enhance image by suppressing unwanted noise with minimal computational efforts.

SUMMARY
[13] This summary is provided to introduce concepts related to enhancement of high dynamic range of images. This summary is neither intended to identify essential features of the present invention nor is it intended for use in determining or limiting the scope of the present invention.
[14] In an embodiment of the present invention, a method for digital detail enhancement of high dynamic range images is provided. The method includes receiving a raw image from a thermal sensor. A non-uniformity correction block performs non-uniformity correction and bad pixel replacement on the raw image to generate a corrected image. A bilateral filter performs edge preserving noise removal on the corrected image to generate a noiseless image with significant edge information preserved therein. A histogram projection block adjusts a dynamic range of the noiseless image to generate a dynamic range adjusted image. A dynamic range of the dynamic range adjusted image matches with a dynamic range of a display. A low pass filter extracts one or more low frequency components from the dynamic range adjusted image. A subtractor subtracts the extracted low frequency components from the dynamic range adjusted image to generate one or more noise free details. A multiplier multiplies the details in real-time by a user-define detail gain factor to obtain weighted details. An adder adds the weighted details to the dynamic range adjusted image to generate a final image. The display displays the final image.
[15] In another embodiment of the present invention, a system for digital detail enhancement of high dynamic range images is provided. The system includes a thermal sensor, a non-uniformity correction block, a bilateral filter, a histogram projection block, a lowpass filer, a subtractor, a multiplier, an adder, and a display. The thermal sensor generates a raw image. The non-uniformity correction block performs non-uniformity correction and bad pixel replacement on the raw image to generate a corrected image. The bilateral filter performs edge preserving noise removal on the corrected image to generate a noiseless image with significant edge information preserved therein. The histogram projection block adjusts a dynamic range of the noiseless image to generate a dynamic range adjusted image. The dynamic range of the dynamic range adjusted image matches with a dynamic range of the display. The low pass filter extracts one or more low frequency components from the dynamic range adjusted image. The subtractor subtracts the extracted low frequency components from the dynamic range adjusted image to generate one or more noise free details. The multiplier multiplies the details in real-time by a user-defined detail gain factor to obtain weighted details. The adder adds the weighted details to the dynamic range adjusted image to generate a final image. The display displays the final image.
[16] In an exemplary embodiment, the system includes a control unit which receives the detail gain factor from a user in real-time and provides the user-defined detail gain factor to the multiplier.
[17] In another exemplary embodiment, the low pass filter is a gaussian filter.

BRIEF DESCRIPTION OF ACCOMPANYING DRAWINGS
[18] Reference will be made to embodiments of the invention, examples of which may be illustrated in the accompanying figures. These figures are intended to be illustrative, not limiting. Although the invention is generally described in the context of these embodiments, it should be understood that it is not intended to limit the scope of the invention to these particular embodiments.
[19] The above and other objects, features, and advantages of the present invention will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings, in which:
[20] Figure 1 depicts a block diagram illustrating general process flow in accordance with an embodiment of the present invention.
[21] Figure 2 depicts a block diagram illustrating bilateral filtering of an image for detail preserving noise removal in accordance with an embodiment of the present invention.
[22] Figure 3 depicts a block diagram illustrating histogram projection of an image for mapping high dynamic range of input image to low dynamic range in accordance with an embodiment of the present invention.
[23] Figure 4 depicts a block diagram illustrating the detail enhancement method in accordance with an embodiment of the present invention.
[24] Figures 5(a) – 5(d) shows a sample image and output produced by the method in accordance with an embodiment of the present invention.
[25] It should be appreciated by those skilled in the art that any block diagram herein represents conceptual views of illustrative systems embodying the principles of the present invention. Similarly, it will be appreciated that any flow chart, flow diagram, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer or processor, whether or not such computer or processor is explicitly shown.

DETAILED DESCRIPTION
[26] The embodiments herein provide a system and method thereof for enhancing sharpness of HDR images. This invention also results in the enhancement of contrast in rendered image. Intensity of details present in output image can be adjusted using the run-time configurable detail gain control presented in this invention. An input image acquired by an imaging device, such as IR (InfraRed), CCD Sensor, or any other imaging device, is processed by the present invention to produce detail enhanced image.
[27] Further, the embodiments may be easily implemented in various image enhancement systems. Embodiments may also be implemented as one or more applications performed by stand alone or embedded systems. The systems and methods described herein are explained using examples with specific details for better understanding. However, the disclosed embodiments can be worked on by a person skilled in the art without the use of these specific details.
[28] Throughout this application, with respect to all reasonable derivatives of such terms, and unless otherwise specified (and/or unless the particular context clearly dictates otherwise), each usage of:
“a” or “an” is meant to read as “at least one.”
“the” is meant to be read as “the at least one.”
[29] Hereinafter, embodiments will be described in detail. For clarity of the description, known constructions and functions will be omitted. Parts of the description may be presented in terms of operations performed by an Electrical/Electronic system, using terms such as state, link, ground, fault, packet and the like, consistent with the manner commonly employed by those skilled in the art to convey the substance of their work to others skilled in the art. As is well understood by those skilled in the art, these quantities take the form of data stored/transferred in the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, and otherwise manipulated through mechanical and electrical components of the electronic/electrical systems; and the term electronic/electrical/computer system includes general purpose as well as special purpose data processing machines, switches, and the like, that are standalone, adjunct or embedded.
[30] In an embodiment, the present invention relates to noise robust detail enhancement techniques, in which the details present in an image are enhanced while suppressing unwanted noise with minimal computational efforts and provides a system and method thereof for enhancement of sharpness of HDR images and enhancement of contrast in rendered image.
[31] The present invention relates to the general area of image processing devices, and methods thereof for improving visual quality of high dynamic range image by enhancing the sharpness of image. More specifically, the present invention relates to noise robust detail enhancement techniques, in which the details present in an image are enhanced while suppressing unwanted noise with minimal computational efforts.
[32] According to the present invention a system and method thereof for enhancing sharpness of HDR images results in the enhancement of contrast in rendered image. Intensity of details present in output image can be adjusted using the run-time configurable detail gain control presented in this invention. An input image acquired by an imaging device, such as IR (InfraRed), CCD Sensor, or any other imaging device, is processed by the present invention to produce detail enhanced image. The details extracted from bilateral filtered image are noise free which when added back to input image enhances the sharpness of image without unwanted noise amplification, basis the fact that the output of bilateral filter has minimal amount of noise but significant amount of details. The embodiment in this invention first removes noise from input image while preserving significant details, then the dynamic range of noise-free image is adjusted to match the dynamic range of display device. Finally, details are extracted from dynamic range adjusted image and simply added back to generate detail enhanced image. Since the details are extracted solely from noise suppressed image, detail layer also has minimal noise which omits the requirement of content adaptive gain.
[33] According to one of the embodiments of present invention a system and method thereof for enhancing sharpness of HDR images may result in the enhancement of contrast in rendered image. Intensity of details present in output image can be adjusted using the run-time configurable detail gain control presented in this invention. An input image acquired by an imaging device, such as IR (InfraRed), CCD Sensor, or any other imaging device, is processed by the present invention to produce detail enhanced image. The details extracted from bilateral filtered image may be noise free, which when added back to input image enhances the sharpness of image without unwanted noise amplification, basis the fact that the output of bilateral filter has minimal amount of noise but significant amount of details. The embodiment in this invention first removes noise from input image while preserving significant details, then the dynamic range of noise-free image is adjusted to match the dynamic range of display device. Finally, details are extracted from dynamic range adjusted image and simply added back to generate detail enhanced image. Since the details are extracted solely from noise suppressed image, detail layer also has minimal noise which omits the requirement of content adaptive gain.
[34] According to an embodiment the present invention discloses a method for noise robust digital detail enhancement of raw images obtained from thermal imaging sensors. The method of present invention enhances details in noisy raw image captured by thermal imaging sensors especially under extreme weather conditions where the differential (difference between temperature levels present in the scene) is very low and corrupted with significant amount of noise. The system according to present invention comprises of image sensor, processing unit and display unit. Raw image data of sensor is acquired from the image sensor and is fed to processing unit. Processing unit performs edge preserving noise removal on raw data to generate noiseless raw image. Noiseless raw image is mapped to fit the dynamic range of display. Details without noise are extracted from this mapped raw image by subtracting low frequency components of this image from itself. A user defined detail gain parameter is multiplied with the extracted details and added back to the mapped image to generate noise free detail enhanced output image.
[35] In an embodiment of the present invention the method for enhancing details present in a raw image captured by thermal camera while suppressing the noise content and providing a run-time configurable detail level control comprise steps of receiving a plurality of signals from a detector representative of a scene. Further to the receiving a plurality of signals from a detector, Non-uniformity correction (NUC) and Bad Pixel replacement is performed prior to storing it in memory. An edge preserving noise removal using a bilateral filter to remove unwanted noise present in raw image while preserving the significant edge information is performed on the resultant signals of the previous processed step. The Compressing/Expanding the dynamic range of 13 -16 bit bilateral filtered image to fit in the dynamic range of display device (8 - 10 bits) at the user end is performed on the resultant signals of the previous processed step and low frequency components of dynamic range adjusted image are extracted by applying a low pass Gaussian filter onto the dynamic range adjusted image. The noise free details are extracted by subtracting low frequency components from the dynamic range adjusted image. The details are added back to the dynamic range adjusted image with a user defined detail gain multiplier.
[36] According to an embodiment of the present invention, by applying a low pass Gaussian filter (unsharp kernel) onto the dynamic range adjusted image, low frequency components of dynamic range adjusted image are extracted.
[37] According to an embodiment of the present invention, by subtracting low frequency components from the dynamic range adjusted image, the noise free details are extracted.
[38] According to an embodiment of the present invention, the step of adding the details back to the dynamic range adjusted image with an user defined detail gain multiplier comprise step of multiplying the details with detail gain and adding it back to the dynamic range adjusted image.
[39] Accordingly, the present invention aims to solve one or more of the aforesaid issues of the prior arts and to provide a noise robust digital detail enhancement system and method thereof which helps to compress dynamic range of HDR images while preserving and enhancing all significant details present in image. Further, the present invention provides a system and method thereof to suppress noise present in the image which might get amplified when performing detail enhancement.
[40] Referring to Figure 1, a general block diagram of image processing apparatus with associated electronics in accordance with the teachings of the present invention is provided. The Imaging system 100 includes a sensor 101, set of processing blocks 102 to 105, a control block and a display unit 107. Input Imagery from the scene is received by the sensor 101 and provided sequentially to the processing blocks, which will process the imagery and feed it to the display unit 107. The first processing block in the sequence is NUC (Non-Uniformity Correction) block 102, which is dedicated for removing inherent sensor noise also known as fixed pattern noise or sensor pattern noise from the image. The next three processing blocks viz. bilateral filter 103, histogram projection 104 and detail enhancement 105 together form major part 108 of this invention. Bilateral filter 103 removes noise from the image, histogram projection 104 does mapping from high dynamic range to low dynamic range and detail enhancement 105 enhances the sharpness of image. Control Unit 106 is interface between user and the processor. It receives the configuration parameter i.e. the detail gain control from the user and sends them to the detail enhancement unit 105.
[41] Figure 2 provides a block diagram of bilateral filtering for removing noise from image.
[42] Figure 3 provides a block diagram for mapping dynamic range of input image to the dynamic range of display.
[43] Figure 4 depicts internal block diagram of the system 400 for digital detail enhancement of high dynamic range images in accordance with an embodiment of the present invention. The system 400 includes a bilateral filter 401, a histogram projection block 402, a low pass filter 403, a subtractor 405, a multiplier 406, a control unit 407, and an adder 408.
[44] The image data obtained from NUC block 102 generally have significant amount of noise. This image data from NUC block is fed to the bilateral filter block 401 which suppresses the noise while ensuring that the details present in image remains preserved. The output Ib of this bilateral filter is computed form input Iin according to Equation 1.

(1)
where, is a Gaussian function of with variance , represents Euclidean distance between and , represents a neighborhood around pixel and is a normalizing term.
[45] The output of bilateral filter block is fed to histogram projection block 402 which maps the dynamic range of noise free image to suit the dynamic range of display device. The dynamic range adjusted image is obtained from bilateral filter output using a mapping function obtained from histogram projection as follows.

(2)

is the histogram of image , is intensity value in image , is corresponding intensity value in image , is a predefined threshold and is the bit depth of output image.
[46] After histogram projection, the image data is passed through detail enhancement block 404. inside this block, the histogram projection output Ih is first passed through a low pass Gaussian filter 403 to extract low frequency component ILPF, which is then subtracted from Ih in a subtractor 405 to get high frequency component i.e. detail layer Id.

(3)
where, is unit impulse function. Details extracted using Equation 3 have significant strength and reduced amount of noise. Strength of details is significant because the workflow in this invention first stretches the data onto full dynamic range of display device using histogram projection and then extracts the detail layer to get significant details. And, amount of noise is reduced because detail extraction is solely done on bilateral filtered image data which has minimal amount of noise unlike the NUC image data.
[47] The control unit 407 receives the detail gain factor from a user and provides the user-defined detail gain factor to the multiplier 406.
[48] The details obtained from subtractor 405 are multiplied in the multiplier 406 with a constant multiplication factor called detail gain control which is selected by the user through the control unit 407. The final detail enhanced image is obtained by adding the output of multiplier to histogram projected image in an addition block 408 using Equation 4.

(4)
[49] For faster real-time implementation, block 404 can be implemented in one filtering operation using Equation 5.

(5)

[50] In operation, the thermal sensor 101 generates a raw image. The non-uniformity correction block 102 performs non-uniformity correction and bad pixel replacement on the raw image to generate a corrected image. The bilateral filter 103 performs edge preserving noise removal on the corrected image to generate a noiseless image with significant edge information preserved therein. The histogram projection block 104 adjusts a dynamic range of the noiseless image to generate a dynamic range adjusted image. The dynamic range of the dynamic range adjusted image matches with a dynamic range of the display 107. The low pass filter 403 extracts one or more low frequency components from the dynamic range adjusted image. The subtractor 405 subtracts the extracted low frequency components from the dynamic range adjusted image to generate one or more noise free details. The multiplier 406 multiplies the details in real-time by a user-defined detail gain factor to obtain weighted details. The adder 408 adds the weighted details to the dynamic range adjusted image to generate a final image. The display 107 displays the final image.
[51] Figures 5(a) – 5(d) depict a sample image and the output images obtained by using method of the present invention.
[52] The present invention relates to the general area of image processing devices, and methods thereof for improving visual quality of high dynamic range image by enhancing the sharpness of image. More specifically, the present invention relates to noise robust detail enhancement techniques, in which the details present in an image are enhanced while suppressing unwanted noise with minimal computational efforts.
[53] According to the present invention a system and method thereof for enhancing sharpness of HDR images results in the enhancement of contrast in rendered image. Intensity of details present in output image can be adjusted using the run-time configurable detail gain control presented in this invention. An input image acquired by an imaging device, such as IR (InfraRed), CCD Sensor, or any other imaging device, is processed by the present invention to produce detail enhanced image. The noise from input image is first removed while preserving significant details, then the dynamic range of noise-free image is adjusted to match the dynamic range of display device. Finally, details are extracted from dynamic range adjusted image and simply added back to generate detail enhanced image. Since the details are extracted solely from noise suppressed image, detail layer also has minimal noise which omits the requirement of content adaptive gain.
[54] The present invention may overcome the challenges of the current scenario through a noise robust digital detail enhancement system and method thereof which helps to compress dynamic range of HDR images while preserving and enhancing all significant details present in image. Further, the present invention provides a system and method thereof to suppress noise present in the image which might get amplified when performing detail enhancement.
[55] The foregoing description of the invention has been set merely to illustrate the invention and is not intended to be limiting. Since modifications of the disclosed embodiments incorporating the spirit and substance of the invention may occur to person skilled in the art, the invention should be construed to include everything within the scope of the invention.
,CLAIMS:

1. A method for digital detail enhancement of high dynamic range images, comprising:
receiving a raw image from a thermal sensor (101);
performing non-uniformity correction and bad pixel replacement on the raw image by a non-uniformity correction block (102) to generate a corrected image;
performing edge preserving noise removal on the corrected image by a bilateral filter (103) to generate a noiseless image with significant edge information preserved therein;
adjusting a dynamic range of the noiseless image by a histogram projection block (104) to generate a dynamic range adjusted image, wherein a dynamic range of the dynamic range adjusted image matches with a dynamic range of a display (107);
extracting one or more low frequency components from the dynamic range adjusted image by applying a low pass filter (403);
subtracting the extracted low frequency components from the dynamic range adjusted image by a subtractor (405) to generate one or more noise free details;
multiplying the details in real-time by a user-defined detail gain factor by a multiplier (406) to obtain weighted details;
adding the weighted details to the dynamic range adjusted image by an adder (408) to generate a final image; and
displaying the final image on the display (407).

2. The method as claimed in claim 1, wherein a control unit (407) receives the detail gain factor from a user in real-time and provides the user-defined detail gain factor to the multiplier (406).

3. The method as claimed in claim 1, wherein the low pass filter (403) is a gaussian filter.

4. A system for digital detail enhancement of high dynamic range images, comprising:
a thermal sensor (101) configured to generate a raw image;
a non-uniformity correction block (102) configured to perform non-uniformity correction and bad pixel replacement on the raw image to generate a corrected image;
a bilateral filter (103) configured to perform edge preserving noise removal on the corrected image to generate a noiseless image with significant edge information preserved therein;
a histogram projection block (104) configured to adjust a dynamic range of the noiseless image to generate a dynamic range adjusted image, wherein a dynamic range of the dynamic range adjusted image matches with a dynamic range of a display (107);
a low pass filter (403) configured to extract one or more low frequency components of the dynamic range adjusted image;
a subtractor (405) configured to subtract the extracted low frequency components from the dynamic range adjusted image to generate one or more noise free details;
a multiplier (406) configured to multiply the details in real-time by a user-defined detail gain factor to obtain weighted details;
an adder (408) configured to add the weighted details to the dynamic range adjusted image to generate a final image; and
a display (407) configured to receive and display the final image.

5. The system as claimed in claim 4, comprising a control unit (407) configured to receive the detail gain factor from a user in real-time and provide the user-defined detail gain factor to the multiplier (406).

6. The system as claimed in claim 4, wherein the low pass filter (403) is a gaussian filer.