DESC:RELATED APPLICATION

The present invention claims benefit of the Indian Provisional Application No. 5205/CHE/2014 titled " METHOD AND APPARATUS FOR STORING, PROCESSING AND RECONSTRUCTING FULL RESOLUTION IMAGE OUT OF SUB BAND ENCODED IMAGES” by Samsung R&D Institute India – Bangalore Private Limited, filed on 17th October 2014, which is herein incorporated in its entirety by reference for all purposes.

FIELD OF THE INVENTION

The present invention generally relates to image processing and particularly relates to image reconstruction methods and systems. The present invention more particularly relates to a method and apparatus for storing, processing and reconstructing high resolution image out of sub-band information associated with the captured images.

BACKGROUND OF THE INVENTION

A wide range of electronic devices, including mobile wireless communication devices, personal digital assistants (PDAs), laptop computers, desktop computers, digital cameras, digital recording devices, and the like, have an assortment of image and video display capabilities. Some devices are capable of displaying two-dimensional (2D) images and video, three-dimensional (3D) images and video, or both.

Most of the time before capture of an image, previewing process happens. Currently, even during preview, processing by image signal processor (ISP) happens on the full resolution image. In most cases, display resolution is much lower than capture resolution which means that one has to down-scale the image to view it. The down-scaling of the previewed full scale image usually happens just before display of the image to fit display resolution, while the processing and editing of the images happen with resolution higher than display This would result in more computational resources. In order to display the image on the display of a user equipment with lesser display resolution (such as, but not limited to, mobile phone, tablet, PDA, and the like), the image has to be downscaled. But when the same image is to be viewed on the user equipment (UE) with larger display (such as, but not limited to, television, larger screen, monitor, laptop, and the like), the downscaled image needs to be upscaled to fit to the larger display resolution which would result in inferior image quality compared to the original image.. The image is captured at one instant of time and consumed (viewed/transferred) at some other instant of time (full resolution).

Figure 1 is a schematic diagram 100 illustrating displaying of the captured high resolution image on high resolution image and low resolution image, according to the existing art. According to the diagram 100, a high resolution image 102 of 16MP resolution can be previewed on a display of the UE (not shown in the Figure) for preview. As the 16MP image cannot be displayed on the display of the UE, using any of the known ISP, the high resolution image 102 can be processed to obtain downscaled image 104 of 2MP resolution.

As the downscaled image 104 can be edited or added effects by user to obtain an edited image 108, downscaled image can be used for editing. But, if the editing is done on the high resolution image to obtain the edited image 108, then the high resolution image needs to be edited, and then downscaled to fit the display of the UE, thereby resulting in slow image processing.

Further, when the downscaled image 104 is to be displayed on a larger screen 110 such as a television, laptop, and the like, then the downscaled image 104 needs to be upscaled. During upscaling of the downscaled image 104 the downscaled image 104 can be extrapolated such that the upscaled image can be obtained that can fit the larger display. But, the image quality will not be same after extrapolation of the downscaled image when compared with the high resolution image 102 that was previewed.

Existing computing applications can perform image signal processing, compression and decompression of images, and upscaling and downscaling of images. But there are no systems or methods available that can store down-scaled image using compression (Eg., JPEG) and encoded residual/edge image as part of JPEG header (Eg., JFIF).

Thus there is need for a method and system that allows the downscaled image to be processed multiple times and displayed on smaller resolution display devices. Further there is a need for a method and system which allows reconstruction of processed full resolution image with less power consumption.

More details about the proposed architecture which addresses the above mentioned shortcomings, disadvantages and problems are mentioned herein and which will be understood by reading and studying the following specification.

SUMMARY OF THE INVENTION

The various embodiments of the present invention disclose a method and apparatus for storing, processing and reconstructing full resolution image out of sub band encoded images.

According to an embodiment of the present invention, a method of extracting sub-band information associated with a selected image comprising creating a downscaled image by performing a downscaling of a selected image, extracting sub-band information associated with the downscaled image by processing the downscaled image using one or more parameters at one instance in time, compressing and encoding the sub-band information associated with the downscaled image and storing the compressed and encoded sub-band information as a single container file format.

In an embodiment of the present invention, the downscaled image is processed independent of the sub-band information, using one or more processing parameters comprising of a filter size, filter coefficients, intensity gains, color gains or any other linear or non-linear digital signal processing parameters. In another embodiment of the present invention, one or more parameters used for processing the downscaled information are stored in the same order as of the processing along with the compressed downscaled image. In another embodiment of the present invention, the downscaled image is decompressed for further processing, independent of the sub-band information.

In an embodiment of the present invention, the method further comprises steps of allowing independent modification to the downscaled image, storing the one or more processing parameters for post processing of the sub-band information, updating the downscaled image by modifying one or more parameters of the image, encoding the updated image and the modified one or more parameters and storing the updated image and the modified one or more parameters as a single file.

In an embodiment of the present invention, creating a high-resolution image from the compressed down-scaled image and the sub-band information comprises of de-compressing the downscaled image and the sub- band information, performing upscaling of the decompressed downscaled image processing the decompressed sub- band information using the one or more processing parameters, and reconstructing the high resolution image of the selected image by adding an extrapolated downscaled image and the processed sub-band information associated with the selected image.
In an embodiment of the present invention, the method further comprises of segregating impact of photo editor effects on the down-scaled image and the sub-band information, conditioning the down-scaled image by an image signal processor (ISP) to get a conditioned downscaled image, processing the conditioned down-scaled image using one or more photo editing parameters based on the photo editor effects during photo editing to get a processed downscaled image, and storing one or more photo editing parameters related to the processing of the downscaled image by appending the one or more photo editing parameters with the one or more processing parameters as a single data file format. Here the one or more processing parameters are applied to the sub-band information in a same order as the one or more processing parameters were stored before the reconstruction of the high resolution image.

In an embodiment of the present invention, the method further comprise of conditioning of the compressed downscaled data independent of the sub-band information, the method comprising decompressing of downscaled data from an existing file container, conditioning of the downscaled data using one or more photo-editor operations, compressing the processed downscaled data, replacing the stored compressed downscaled data with the newly conditioned compressed downscaled data in the same file container and appending the relevant photo-editing parameters in the same file container.

Embodiments herein further disclose a method for rendering a high resolution images based on a time-domain system. The method comprises of applying a blur filter on to a selected image to obtain a filtered image, extracting a sub-band information associated with the selected image at one instance in time by taking a difference between the selected image and the filtered image, performing downscaling of the filtered image, processing the down-scaled image by an image signal processor (ISP) and storing the related parameters, conditioning of the processed downscaled image using one or more photo editing operations and storing the related parameters, compressing and storing the conditioned downscaled image and the sub-band information associated with the selected image, de-compressing the downscaled image and the sub-band information, conditioning the decompressed sub-band image, using the one or more stored processing parameters used by the downscaled image and adding the conditioned sub-band information with an extrapolated conditioned downscaled image to reconstruct the high resolution image.

Embodiments herein further disclose a method for rendering a high resolution images based on a frequency-domain system. The method comprises of performing frequency transformation on a selected image, applying a low pass filter to obtain a low frequency data of the selected image, applying a high pass filter to obtain high frequency data of the selected image, processing the low frequency data of the selected image using one or more processing parameters, conditioning the low frequency data using photo editing operations and storing the related parameters, compressing and storing the low frequency data and the high frequency data associated with the selected image, and decompressing the low frequency data and the high frequency data, conditioning the decompressed high frequency data using the one or more stored processing parameters, adding conditioned low frequency data and conditioned high frequency data to get a full frequency spectrum conditioned data and applying inverse frequency transformation to reconstruct the high resolution image.

Embodiments herein further disclose a method for rendering a high resolution images based on a closed loop system. The method further comprising creating a downscaled image by performing a downscaling of a selected image, performing an upscaling of the downscaled image, extracting a sub-band information associated with the selected image by taking a difference between the upscaled image and the selected image, conditioning the down-scaled image by an image signal processor (ISP) and storing the processing parameters, processing the conditioned downscaled image using one or more photo-editing operations and storing the related parameters, compressing and storing, the downscaled image and the sub-band information, associated with the selected image, de-compressing, the downscaled image and the sub- band information, conditioning, the decompressed sub-band image, using the stored processing parameters used by the downscaled image and adding the conditioned sub-band information with an extrapolated conditioned downscaled image to reconstruct the high resolution image.

In an embodiment of the present invention, a method of rendering a low resolution image comprises of decompressing a copy of a downscaled image while retaining the downscaled image in a file container, conditioning of the downscaled data using at least one of a color conversion, resizing, rotation and blending and rendering the downscaled image to the display. Here the low resolution image is independent of sub-band information and related processing parameters.

Embodiments herein further disclose an apparatus for extracting sub-band information associated with a selected image, the apparatus comprising a downscaler adapted for downscaling of the selected image, an image processing unit adapted for creating a time-stamped downscaled image, and extracting sub-band information associated with the downscaled image by processing the downscaled image using one or more parameters at one instance in time; an encoding unit for compressing and encoding the sub-band information associated with the downscaled image and a storage unit for storing the compressed and encoded sub-band information as a single container file format.

According to an embodiment of the present invention, the image processing unit is further adapted for allowing independent modification to the downscaled image and storing the one or more processing parameters for post processing of the sub-band information, updating the downscaled image by modifying one or more parameters of the image, encoding the updated image and the modified one or more parameters, and storing the updated image and the modified one or more parameters as a single file.

The image processing unit is further adapted for creating a high-resolution image from the compressed down-scaled image and the sub-band information, comprising steps of de-compressing the downscaled image and the sub- band information, performing upscaling of the decompressed downscaled image, processing the decompressed sub- band information using the one or more processing parameters, reconstructing the high resolution image of the selected image by adding an extrapolated downscaled image and the processed sub-band information associated with the selected image, segregating impact of photo editor effects on the down-scaled image and the sub-band information, conditioning the down-scaled image to get a conditioned downscaled image, and processing the conditioned down-scaled image using one or more photo editing parameters based on the photo editor effects during photo editing to get a processed downscaled image.

According to an embodiment of the present invention, the storage unit is further adapted for storing one or more photo editing parameters related to the processing of the downscaled image by appending the one or more photo editing parameters with the one or more processing parameters as a single data file format.

The foregoing has outlined, in general, the various aspects of the invention and is to serve as an aid to better understanding the more complete detailed description which is to follow. In reference to such, there is to be a clear understanding that the present invention is not limited to the method or application of use described and illustrated herein. It is intended that any other advantages and objects of the present invention that become apparent or obvious from the detailed description or illustrations contained herein are within the scope of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS
The other objects, features and advantages will occur to those skilled in the art from the following description of the preferred embodiment and the accompanying drawings in which:

Figure 1 is a schematic diagram illustrating displaying of the captured high resolution image on high resolution image a low resolution display unit, according to the existing art.

Figure 2 is a schematic flow diagram illustrating a method for storing, processing and reconstructing full resolution image out of sub band encoded images, according to an embodiment of the present invention.

Figure 3a is a schematic diagram illustrating a method for storing, processing and reconstructing full resolution image out of sub band encoded images using time domain, according to an embodiment of the present invention.

Figure 3b is a schematic diagram illustrating method for storing, processing and reconstructing full resolution image out of sub band encoded images using frequency domain, according to an embodiment of the present invention.

Figure 3c is a schematic diagram illustrating a method for storing, processing and reconstructing full resolution image out of sub band encoded images using a closed loop system, according to an embodiment of the present invention.

Figure 4 is a schematic diagram illustrating a camera live preview operation, according to an embodiment of the present invention.

Figure 5 is a schematic diagram illustrating a method of capturing and storing of images, according to an embodiment of the present invention.

Figure 6 is a schematic diagram illustrating a fast preview operation after capturing the image, according to an embodiment of the present invention.

Figure 7 is a schematic diagram illustrating an editing operation of the image after capturing the image, according to an embodiment of the present invention.

Figure 8a is a schematic diagram illustrating use case application for storing, processing and reconstructing full resolution image out of sub band encoded images, according to an embodiment of the present invention.

Figure 8b is a schematic diagram illustrating another use case application for storing, processing and reconstructing full resolution image out of sub band encoded images, according to an embodiment of the present invention.

Figure 9 is a schematic diagram illustrating transmission of the captured image, according to an embodiment of the present invention.

Although specific features of the present invention are shown in some drawings and not in others, this is done for convenience only as each feature may be combined with any or all of the other features in accordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The various embodiments of the present invention disclose a method and apparatus for storing, processing and reconstructing full resolution image out of sub band encoded images. In the following detailed description of the embodiments of the invention, reference is made to the accompanying drawings that form a part hereof, and in which are shown by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that changes may be made without departing from the scope of the present invention. The following detailed description is, therefore, not to be taken in a limiting sense, and the scope of the present invention is defined only by the appended claims.

The specification may refer to “an”, “one” or “some” embodiment(s) in several locations. This does not necessarily imply that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment. Single features of different embodiments may also be combined to provide other embodiments.

As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless expressly stated otherwise. It will be further understood that the terms “includes”, “comprises”, “including” and/or “comprising” when used in this specification, specify the presence of stated features, integers, steps, operations, elements and/or components, but do not preclude the presence or addition of one or more other features integers, steps, operations, elements, components, and/or groups thereof. As used herein, the term “and/or” includes any and all combinations and arrangements of one or more of the associated listed items.

Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

The present invention discloses a method and system for rendering high resolution images based on sub-band encoded data of an image. The present invention is described with help of various embodiments. The person having ordinarily skilled in the art can understand that implement the present invention can be modified to achieve the objective of the present invention without departing from the scope of the invention.

According to an embodiment of the present invention, a method for rendering high resolution images based on sub-band encoded data of an image is described herein. The method comprises steps of performing downscaling of a selected image. According to the present method, an image can be received by user equipment (UE). In an embodiment of the present invention, the selected image can be a preview image that is of high resolution. In another embodiment of the present invention, the selected image can be any of high resolution images that can include, but not limited to, a captured image from the UE, an image received from the other UE, or any other image stored in any of the storage device. According to the present embodiment, the selected image can be image under review at the UE before capture, but the person having ordinarily skilled in the art can understand that the selected image can be any of the abovementioned image, without departing from the scope of the invention.

The selected image can be downscaled so that a downscaled image is obtained for the selected image. The person having ordinarily skilled in the art can understand that any of the downscaler can be used for downscaling the selected image, without departing from the scope of the invention.

The method further comprises step of creating a time-stamped downscaled image. As user of the UE is attempting to capture image, and there can be selected images at different time instances, the downscaler can create downscaled images to fit on the screen of the UE at that particular time instance and the image can be called as time-stamped downscaled image. In an embodiment of the present invention, the UE can generate only one time-stamped downscaled image. In another embodiment of the present invention, the UE can generate plurality of down-stamped downscaled images so that the user can select the best one among them. In another embodiment of the present invention, the UE can generate, continuous burst of time-stamped downscaled images. The person having ordinarily skilled in the art can understand that different image capturing methods can be used to capture and create time-stamped downscaled image, without departing from the scope of the invention.

Further, the method comprises step of extracting sub-band information associated with the selected image at one instance in time. For the selected image, the associated sub-band information can be extracted, wherein the sub-band information is residue of the selected image after creating time-stamped downscaled image. The sub-band information comprises of pixel data and other data related to the time-stamped downscaled image after extracting from the selected image.

In an embodiment of the present invention, the sub-band information corresponding to the selected image can be derived using at least one of, but not limited to, time-domain method, frequency-domain method, closed loop system, and the like. The method for deriving the sub-band information can be described later in the description with respect to corresponding figures.

Further, the method comprises of compressing the downscaled image and the sub-band information. Upon extracting the sub-band information, both the downscaled image and sub-band information can be compressed. In an embodiment of the present invention, the sub-band information can be compressed based on pre-computed weighted quantization values. In an embodiment of the present invention, any of the known image compression method can be used for compressing the image, without departing from the scope of the invention. In an embodiment of the present invention, the compressed image can be an extrapolated compressed image.

In another embodiment of the present invention, the sub-band information and the time-stamped downscaled image can be stored in the UE for future reference or use. In an embodiment of the present invention, the sub-band information and the time-stamped downscaled image can be stored in the same order as they are received during preview, at different time intervals, without departing from the scope of the invention. In an embodiment of the present invention, the storage module in which the time-stamped downscaled image and the corresponding sub-band information stored can include, but not limited to, internal storage of the UE, random access memory (RAM), external storage devices such as, but not limited to, memory card, pen drive, hard disk, storage module of another device or UE, server, cloud database, and the like, and the person having ordinarily skilled in the art can understand that the time-stamped downscaled image and the corresponding sub-band information can be stored in any of the known storage modules, without departing from the scope of the invention.

Further, the method comprises steps of decompressing and adding the sub-band information with extrapolated downscaled image to reconstruct and render high resolution image. During reconstruction of the compressed image, the extrapolated downscaled image and the sub-band information can be accessed, decompressed and reconstructed to render back the high resolution image.

In another embodiment of the present invention, compressed downscaled image can be displayed on a user device. Upon compressing the image, the time-stamped downscaled image of the selected image can be displayed on the user device. The time-stamped downscaled image can be decompressed along with the sub-band information, reconstructed and displayed only as a preview image to the user so that the user can understand that the image that user is trying to capture will be displayed as such.

In an embodiment of the present invention, the downscaled image can be processed independent of the sub-band information, using one or more processing parameters comprising of, but not limited to, a filter size, filter coefficients, intensity gains, color gains or any other linear or non-linear digital signal processing parameters, and the like. In another embodiment of the present invention, one or more parameters used for processing the downscaled information can be stored in the same order as of the processing along with the compressed downscaled image. In another embodiment of the present invention, the downscaled image can be decompressed for further processing, independent of the sub-band information.

Further, in an embodiment of the present invention, the method further comprises of de-compressing the downscaled image and the side band information. The downscaled image can be previewed on the display of the UE. But to maintain the consistency of the image and resolution quality, the downscaled image can be de-compressed. Also, the side band information corresponding to the downscaled image can also be de-compressed.

Further, the method comprises of processing the decompressed side band information using one or more processing parameters. In an embodiment of the present invention, the one or more processing parameters associated with the decompression of the side band information can include at least one of, but not limited to, a filter size, filter coefficients, intensity gains, color gains, any other linear or non-linear digital signal processing parameters, and the like. In an embodiment of the present invention, one or more parameters used for processing the sub-band information can be stored in a same order as of the processing.

Further, the method comprises of reconstructing a high resolution image of the selected image by adding the extrapolated downscaled image and the processed sub-band information associated with the selected image. The downscaled image and the corresponding sub-band information can be added to an adder, wherein the adder identifies the image information related to each pixel, combines both of the downscaled image and the sub-band information to reconstruct the high resolution image. In an embodiment of the present invention, the one or more parameters can also be applied to the sub-band information in the same order as they are stored before the reconstruction of the high resolution image.

In another embodiment of the present invention, the method further comprises of encoding the sub-band information associated with the selected image. According to the present invention, any of the known encoding method can be used for encoding the sub band information corresponding to the downscaled image, without departing from the scope of the invention.

Further, the downscaled image and the corresponding encoded sub-band information can be stored in a compressed single data container format. In an embodiment of the present invention, the sub-band image is compressed, encoded and stored as part of a header of an image file format. In an embodiment of the present invention, image file format can be one of a known image formats that includes, but not limited to, JPEG, GIF, PNG, TIF, and the like.

In another embodiment of the present invention, the downscaled image and the corresponding compressed encoded sub-band information can be stored in compressed separate containers with same file format. The downscaled image and the corresponding compressed encoded sub-band information can also store the location information so that when the downscaled image is accessed for reconstruction, the corresponding compressed encoded sub-band information can also be obtained easily. In another embodiment of the present invention, the downscaled image and the corresponding compressed encoded sub-band information can be stored in compressed separate containers with different file formats. The person having ordinarily skilled in the art can understand that various data storing methods can be used to store the downscaled image and the corresponding compressed encoded sub-band information, without departing from the scope of the invention.

In another embodiment of the present invention, the method further comprises of allowing independent modification to the downscaled image. The downscaled image of the selected image can be accessed from the storage module, and only the downscaled image can be accessed without accessing the sub-band information. The accessed downscaled image can be modified independently, without modifying the sub-band information.

Further, the method comprises of storing the one or more processing parameters for post processing of the sub-band image. The processing parameters associated with each of the downscaled image and the corresponding sub-band information can be retained as it is, so that the one or more processing parameters can be used during post processing of the sub-band image to reconstruct the high resolution image.

Figure 2 is a schematic flow diagram 200 illustrating a method for rendering high resolution images based on sub-band encoded data of an image, according to an embodiment of the present invention. According to the Figure 2, at step 202 the method comprises step of performing downscaling of a selected image. The selected image can be a preview image on user equipment (UE), which is actually a high resolution image. The selected image can be downscaled to match the display capability of the display of the UE.

Further, at step 204, the method comprises creating a time-stamped downscaled image. Based on the time duration of the preview, the time-stamped downscaled image can be created. At step 206, the method comprises of extracting sub-band information associated with the selected image, wherein the sub-band information can be residue data related corresponding to the selected image, after obtaining the time-stamped downscaled image after downscaling of the selected image. Further, at step 208, the method comprises of compressing the downscaled image and the sub-band information. Both the downscaled image and the corresponding sub-band information can be compressed. The compressed downscaled image and the corresponding sub-band information can be stored for future use. At step 210, the method further comprises of displaying the compressed downscaled image on a user device. The compressed image can be displayed on the UE as a preview image, which user can retain it or move to another image for capturing and saving.

Figure 3a is a schematic diagram 300 illustrating a method for storing, processing and reconstructing full resolution image out of sub band encoded images using time domain, according to an embodiment of the present invention. According to the Figure 3a, the high resolution image obtained during preview can be downscaled, compressed, stored, decompressed, added to obtain the high resolution image back during storing based on time factor of the image.

According to the Figure 3a, an input image x can be received during preview. The input image x can be transmitted to a blur function module 302, wherein the noise present in the image can be removed to obtain filtered image x’. Any of the known noise removal method and any type of noise present in the image can be removed using the blur function 302 to obtain the filtered image X’, without departing from the scope of the invention. The input image x can be substituted at a subtractor 304 with the filtered image x’ to obtain sub-band information d.

Further, the filtered image X’ can be transmitted to the downscaler 306 to obtain downscaled image x’, wherein the image can be downscaled such that the image can match the display resolution of the UE. Further, the downscaled image x’ can be transmitted to an image signal processor (ISP) 308, wherein the ISP 308 can process the downscaled image to obtain a processed image y. The ISP 308 can also obtain the one or more parameters P while processing the downscaled image x’. The parameters P obtained during processing of the image are described herein above, and hence not described herein again to avoid repetition.

Further, the processed image y can be displayed on display 310 of the UE. Further, the processed image y can be upscaled at an upscaler 312 to obtain an upscaled image Y. The sub-band information d can be encoded at an encoder 314, and the encoded sub-band information can be decompressed at a decompressor 316, wherein parameters P can be considered while encoding and decompressing the sub-band information d. The decompressed image can be conditioned at ISP conditioning module to obtain conditioned sub-band information d’. The ISP conditioning of the sub-band information d is equivalent to preconditioning of the ISP 308. The conditioned sub-band information d’ and the upscaled image Y can be added together and stored in a storage module 318.

The abovementioned process receives the downscaled image which is time stamped and performs processing on the time stamped downscaled image. The time domain based sub-band information processing method helps in reducing the aliasing effects in the downscaled image, and also helps in improving quality of the downscaled image.

Figure 3b is a schematic diagram 320 illustrating method for storing, processing and reconstructing full resolution image out of sub band encoded images using frequency domain, according to an embodiment of the present invention. According to the Figure 3b, the high resolution image obtained during preview can be downscaled, compressed, stored, decompressed, added to obtain the high resolution image back during storing based on time factor of the image.

According to the Figure 3b, an input image x can be received during preview. The input image x can be transmitted to a frequency transformer 322, wherein the frequency transformer 322 transforms the image data into frequency data of the image x. The frequency data of the image x can be transmitted to high frequency modulator 324, wherein the frequency of the image x can be modulated to high frequency data d.

Further, the frequency data of the image x can also be transmitted to a low frequency modulator 326, wherein the frequency of the image x can be modulated to low frequency data x’. Further, the low frequency image data x’ can be transmitted to an image signal processor (ISP) 328, wherein the ISP 328 can process the low frequency data of the image x’ to obtain a processed image y. The ISP 328 can also obtain the one or more parameters P while processing the low frequency image data x’. The parameters P obtained during processing of the image are described herein above, and hence not described herein again to avoid repetition.

Further, the processed image y can be displayed on display 330 of the UE. The high frequency image data d can be encoded at an encoder 332, and the encoded high frequency image data d can be decompressed at a decompressor 334, wherein parameters P can be considered while encoding and decompressing the high frequency image data d. The decompressed high frequency image data can be conditioned at ISP conditioning module to obtain conditioned high frequency image data d’. The ISP conditioning of the high frequency image data d is equivalent to preconditioning of the ISP 328. Further, the processed image y can be transformed at an inverse frequency transformer 332 by adding the high frequency image data d’ to obtain a high resolution image data and high resolution image data can be stored in a storage module 336.

Figure 3c is a schematic diagram 350 illustrating a method for storing, processing and reconstructing full resolution image out of sub band encoded images using a closed loop system, according to an embodiment of the present invention. According to the Figure 3c, the high resolution image obtained during preview can be downscaled, compressed, stored, decompressed, added to obtain the high resolution image back during storing based on closed loop system.

According to the Figure 3c, an input image x can be received during preview. The input image X can be transmitted to a downscaler 352 to obtain downscaled image x’, wherein the image can be downscaled such that the image can match the display resolution of the UE. The downscaled image x’ can be upscaled using an upscaler 354 to obtain an upscaled image X’. The input image X can be substituted at a subtractor 356 with the upscaled image X’ to obtain sub-band information d.

Further, the downscaled image x’ can be transmitted to an image signal processor (ISP) 358, wherein the ISP 358 can process the downscaled image x’ to obtain a processed image y. The ISP 358 can also obtain the one or more parameters P while processing the downscaled image x’. The parameters P obtained during processing of the image are described herein above, and hence not described herein again to avoid repetition.

Further, the processed image y can be upscaled at an upscaler 360 to obtain an upscaled image Y. The sub-band information d can be encoded at an encoder 362, and the encoded sub-band information can be decompressed at a decompressor 364, wherein parameters P can be considered while encoding and decompressing the sub-band information d. The decompressed image can be conditioned at ISP conditioning module 366 to obtain conditioned sub-band information d’. The ISP conditioning of the sub-band information d is equivalent to preconditioning of the ISP 358. The conditioned sub-band information d’ and the upscaled image Y can be added together and stored in a storage module 368.

The abovementioned process for closed loop system is one of the suitable methods as the abovementioned process replicates the exact process of upscaling during the reconstruction of full resolution image. Further, the losses are controlled in the closed loop system and hence better tuning capabilities can be provided in the system for making the right trade-offs.

Figure 4 is a schematic diagram 400 illustrating a camera live preview operation, according to an embodiment of the present invention. According to the Figure 4, an input image X can be received during preview. The input image X can be transmitted to a downscaler 402 to obtain downscaled image x’, wherein the image can be downscaled such that the image can match the display resolution of the UE. The downscaled image x’ can be upscaled using an upscaler 404 to obtain an upscaled image X’. The input image X can be substituted at a subtractor 406 with the upscaled image X’ to obtain sub-band information d.

Further, the downscaled image x’ can be transmitted to an image signal processor (ISP) 408, wherein the ISP 408 can process the downscaled image x’ to obtain a processed image y. The ISP 358 can also obtain the one or more parameters P while processing the downscaled image x’. The parameters P obtained during processing of the image are described herein above, and hence not described herein again to avoid repetition.

Further, the processed image y can be display on a display 410 of the UE. The sub-band information d can be encoded at an encoder 412 to obtain encoded sub-band information, wherein parameters P can be considered while encoding the sub-band information d. The encoded sub-band information and processed image y can be stored in a storage module 414, wherein the storage module 414 can be a zero shutter lag (ZSL) storage, and the person having ordinarily skilled in the art can understand the ZSL storage, and hence not described in detail.

According to the present invention, at different time-stamps, plurality of high resolution image previews can be captured and selected. For the selected images, time stamp information, indicating timing information at which the image is selected, can also be attached. The plurality of time-stamped images can be downscaled and subtracted with the selected image to obtain plurality of time stamped downscaled images along with sub-band information. The plurality of time stamped downscaled images can be processed with an image signal processor (ISP) to obtain corresponding parameters for the plurality of time stamped downscaled images. Now, the plurality of time stamped downscaled images along with corresponding sub-band information and parameters can be stored in a zero lag shutter (ZSL) buffer memory. The ZSL buffer can be a temporary memory and can delete the unwanted images once they are rejected. If user selects any particular downscaled image at particular time-stamp, then only the particular time stamped downscaled image along with corresponding sub-band information and the parameters can be stored in a storage as a single file format.

Figure 5 is a schematic diagram 500 illustrating a method of capturing and storing of images, according to an embodiment of the present invention. The Figure 5 illustrates obtaining/ capturing plurality of downscaled images at zero shutter lag (ZSL) buffer 502 along with encoded sub-band information and parameters at different time stamps and storing the selected downscaled image along with the encoded sub-band information and parameters. As described herein above, the ZSL buffer 502 comprises of plurality of downscaled images, along with corresponding sub-band information and parameters obtained at different time stamps. When the user selects downscaled image with corresponding sub-band information and parameters at time instance tk, then only the particular downscaled image with corresponding sub-band information and parameters at time instance tk can be stored in a storage module 504 within a single file format. In the current embodiment, the file format is mentioned as JPEG image. In another embodiment of the present invention, the file format can be any of the known file format, without departing from the scope of the invention.

According to an embodiment of the present invention, the downscaled image along with encoded sub-band information and parameters can be decoded and displayed on a display of user equipment during fast preview. During fast preview, the downscaled image can be obtained from a storage module and at a decoder, the downscaled image can be decoded by adding the sub-band information and the parameters to the image to obtain a fast preview image. The fast preview image can be displayed on the display of the UE.

Figure 6 is a schematic diagram 600 illustrating a fast preview operation after capturing the image, according to an embodiment of the present invention. According to the diagram 600, a downscaled image can be obtained from a storage module 602, along with the corresponding sub-band information and the parameters stored as a single JPEG image file format contents. The downscaled image and the corresponding sub-band information and the parameters can be decoded by a JPEG decoder 604 to obtain decoded image. The decoded image can be displayed on display 606 of the UE as a fast preview image.

According to another embodiment of the present invention, the stored downscaled image can be obtained by the user during preview, and the image can be edited by the user. Before editing, the downscaled image along with the corresponding sub-band information and the parameters can be obtained from a storage module and decoded by a decoder. During editing, the user can make changes in the image by adding/ modifying the parameters of the image. Upon editing, the changed parameters can also be considered and stored in the storage. Upon completion of editing, the edited image can be encoded by an encoder and the encoded image can be stored in the storage module. In an embodiment of the present invention, the edited image can be updated and stored in the same file. In another embodiment of the present invention, the edited image along with the sub-band information and the parameters can be stored as new file format content, without departing from the scope of the invention.

Figure 7 is a schematic diagram 700 illustrating an editing operation of the image after capturing the image, according to an embodiment of the present invention. According to the Figure 7, a downscaled image can be obtained from a storage module 702, along with the corresponding sub-band information and the parameters stored as a single JPEG image file format contents. The downscaled image and the corresponding sub-band information and the parameters can be decoded by a JPEG decoder 704 to obtain decoded image. The decoded downscaled image can be sent for photo editing 706, wherein the parameters 708 of the downscaled image can be added/ modified to obtain edited image. The updated parameters 708 can be stored in the single JPEG image file format contents present in the storage module 702. The edited image is encoded by a JPEG encoder 710, wherein the JPEG encoder encodes the edited image to obtain encoded downscaled image. The encoded downscaled image can be stored in the single JPEG image file format contents present in the storage module 702 along with sub-band information and the edited parameters.

Figure 8a is a schematic diagram 800 illustrating use case application for storing, processing and reconstructing full resolution image out of sub band encoded images, according to an embodiment of the present invention. According Figure 8a, the encoded sub-band information along with parameters can be accessed from a single JPEG file present in a storage module and the accessed encoded sub-band information and the parameters cane be decoded by a decoder to obtain decoded sub-band information. The decoded sub-band information can be again processed along with the parameters by an image signal processor (ISP) conditioning to obtain processed image d’.

Further, JPEG compressed downscaled image present in the JPEG file can be accessed and decoded using JPEG decoder to obtain decoded downscaled image. The decoded downscaled image can be upscaled by an upscaler to obtain upscaled image. The upscaled image and the processed image d’ can be added together to obtain a high resolution image, which can be output on display of user equipment (UE).

Figure 8b is a schematic diagram 850 illustrating another use case application for storing, processing and reconstructing full resolution image out of sub band encoded images, according to an embodiment of the present invention. According Figure 8b, only JPEG compressed downscaled image present in JPEG file can be accessed and decoded using JPEG decoder to obtain decoded downscaled image. The decoded downscaled image can be output on display of user equipment (UE).

Figure 9 is a schematic diagram 900 illustrating transmission of the captured image, according to an embodiment of the present invention. According to the Figure 9, it can be seen that the JPEG file stored in storage module of user equipment (UE) can comprise of JPEG compressed downscaled image along with corresponding encoded sub-band information, and parameters. From the stored JPEG file, the user can view both low resolution image as well as high resolution image. If the user accesses only the compressed downscaled image, then the compressed downscaled image can be decoded and displayed for the low resolution display of the image. If the user wishes to access high resolution display of the image, then the compressed downscaled image along with the encoded sub-band information and the parameters can be accessed, and processed to obtain the high resolution image.

The present embodiments have been described with reference to specific example embodiments; it will be evident that various modifications and changes may be made to these embodiments without departing from the broader spirit and scope of the various embodiments. Furthermore, the various devices, modules, and the like described herein may be enabled and operated using hardware circuitry, for example, complementary metal oxide semiconductor based logic circuitry, firmware, software and/or any combination of hardware, firmware, and/or software embodied in a machine readable medium. For example, the various electrical structure and methods may be embodied using transistors, logic gates, and electrical circuits, such as application specific integrated circuit.

Although the embodiments herein are described with various specific embodiments, it will be obvious for a person skilled in the art to practice the invention with modifications. However, all such modifications are deemed to be within the scope of the claims. It is also to be understood that the following claims are intended to cover all of the generic and specific features of the embodiments described herein and all the statements of the scope of the embodiments which as a matter of language might be said to fall there between.
,CLAIMS:
We Claim:

1. A method of extracting sub-band information associated with a selected image, comprising:
creating a downscaled image by performing a downscaling of a selected image; and
extracting sub-band information associated with the downscaled image by processing the downscaled image using one or more parameters at one instance in time.

2. The method of claim 1, further comprising:
compressing and encoding the sub-band information associated with the downscaled image; and
storing the compressed and encoded sub-band information as a single container file format.

3. The method of claim 1, wherein a processing parameter is at least one of a filter size, filter coefficients, intensity gains, color gains or any other linear or non-linear digital signal processing parameters.

4. . The method of claim 1, wherein one or more parameters used for processing the downscaled image are stored along with the compressed downscaled image in an order of processing.

5. The method of claim 1, wherein the sub-band information is compressed based on pre-computed weighted quantization values.

6. The method of claim 1, wherein the downscaled image is decompressed for further processing, independent of the sub-band information.

7. The method of claim 1, further comprising:
allowing independent modification to the downscaled image; and
storing the one or more processing parameters for post processing of the sub-band information.

8. The method of claim 1, further comprising:
updating the downscaled image by modifying one or more parameters of the image;
encoding the updated image and the modified one or more parameters; and
storing the updated image and the modified one or more parameters as a single file.

9. The method of claim 1, further comprising creating a high-resolution image from the compressed down-scaled image and the sub-band information, the method comprising:
de-compressing the downscaled image and the sub- band information;
performing upscaling of the decompressed downscaled image;
processing the decompressed sub- band information using the one or more processing parameters; and
reconstructing the high resolution image of the selected image by adding an extrapolated downscaled image and the processed sub-band information associated with the selected image.

10. The method of claim 9, further comprising:
segregating impact of photo editor effects on the down-scaled image and the sub-band information;
conditioning the down-scaled image by an image signal processor (ISP) to get a conditioned downscaled image;
processing the conditioned down-scaled image using one or more photo editing parameters based on the photo editor effects during photo editing to get a processed downscaled image; and
storing one or more photo editing parameters related to the processing of the downscaled image by appending the one or more photo editing parameters with the one or more processing parameters as a single data file format.

11. The method of claim 9, wherein the one or more processing parameters are applied to the sub-band information in a same order as the one or more processing parameters were stored before the reconstruction of the high resolution image.

12. The method of claim 9, further comprising conditioning of compressed downscaled data independent of the sub-band information, the method comprises of:
decompressing of downscaled data from an existing file container;
conditioning of the downscaled data using one or more photo-editor operations;
compressing the processed downscaled data;
replacing the stored compressed downscaled data with the newly conditioned compressed downscaled data in the same file container; and
appending the relevant photo-editing parameters in the same file container.

13. A method of rendering high resolution images based on a time-domain system, comprising:
applying a blur filter on to a selected image to obtain a filtered image;
extracting a sub-band information associated with the selected image at one instance in time by taking a difference between the selected image and the filtered image;
performing downscaling of the filtered image;
processing the down-scaled image by an image signal processor (ISP) and storing the related parameters;
conditioning of the processed downscaled image using one or more photo editing operations and storing the related parameters;
compressing and storing the conditioned downscaled image and the sub-band information associated with the selected image;
de-compressing the downscaled image and the sub-band information;
conditioning the decompressed sub-band image, using the one or more stored processing parameters used by the downscaled image; and
adding the conditioned sub-band information with an extrapolated conditioned downscaled image to reconstruct the high resolution image.

14. A method of rendering high resolution images based on a frequency-domain system, comprising:
performing frequency transformation on a selected image;
applying a low pass filter to obtain a low frequency data of the selected image;
applying a high pass filter to obtain high frequency data of the selected image;
processing the low frequency data of the selected image using one or more processing parameters;
conditioning the low frequency data using photo editing operations and storing the related parameters;
compressing and storing the low frequency data and the high frequency data associated with the selected image; and
decompressing the low frequency data and the high frequency data;
conditioning the decompressed high frequency data using the one or more stored processing parameters,;
adding conditioned low frequency data and conditioned high frequency data to get a full frequency spectrum conditioned data; and
applying inverse frequency transformation to reconstruct the high resolution image.

15. A method of rendering high resolution images based on a closed loop system, comprising:
creating a downscaled image by performing a downscaling of a selected image;
performing an upscaling of the downscaled image;
extracting a sub-band information associated with the selected image by taking a difference between the upscaled image and the selected image;
conditioning the down-scaled image by an image signal processor (ISP) and storing the processing parameters;
processing the conditioned downscaled image using one or more photo-editing operations and storing the related parameters;
compressing and storing, the downscaled image and the sub-band information, associated with the selected image;
de-compressing, the downscaled image and the sub- band information;
conditioning, the decompressed sub-band image, using the stored processing parameters used by the downscaled image; and
adding the conditioned sub-band information with an extrapolated conditioned downscaled image to reconstruct the high resolution image.

16. A method of rendering a low resolution image, comprising:
decompressing a copy of a downscaled image while retaining the downscaled image in a file container;
conditioning of the downscaled data using at least one of a color conversion, resizing, rotation and blending; and
rendering the downscaled image to the display;
where the low resolution image is independent of sub-band information and related processing parameters.

17. An apparatus for extracting sub-band information associated with a selected image, comprising:
a downscaler adapted for downscaling of the selected image;
an image processing unit adapted for:
creating a time-stamped downscaled image; and
extracting sub-band information associated with the downscaled image by processing the downscaled image using one or more parameters at one instance in time.
an encoding unit for compressing and encoding the sub-band information associated with the downscaled image; and
a storage unit for storing the compressed and encoded sub-band information as a single container file format.

18. The apparatus of claim 17, wherein the one or more processing parameters comprises of a filter size, filter coefficients, intensity gains, color gains or any other linear or non-linear digital signal processing parameters.

19. The apparatus of claim 17, wherein one or more parameters used for processing the downscaled image are stored in the same order as of the processing along with the compressed downscaled image.

20. The apparatus of claim 17, wherein the sub-band information is compressed based on pre-computed weighted quantization values.

21. The apparatus of claim 17, wherein the downscaled image is decompressed for further processing, independent of the sub-band information.

22. The apparatus of claim 17, wherein the image processing unit is further adapted for:
allowing independent modification to the downscaled image; and
storing the one or more processing parameters for post processing of the sub-band information.
updating the downscaled image by modifying one or more parameters of the image;
encoding the updated image and the modified one or more parameters; and
storing the updated image and the modified one or more parameters as a single file.

23. The apparatus of claim 17, wherein the image processing unit is further adapted for creating a high-resolution image from the compressed down-scaled image and the sub-band information, comprising steps of:
de-compressing the downscaled image and the sub- band information;
performing upscaling of the decompressed downscaled image;
processing the decompressed sub- band information using the one or more processing parameters; and
reconstructing the high resolution image of the selected image by adding an extrapolated downscaled image and the processed sub-band information associated with the selected image.
segregating impact of photo editor effects on the down-scaled image and the sub-band information;
conditioning the down-scaled image to get a conditioned downscaled image;
processing the conditioned down-scaled image using one or more photo editing parameters based on the photo editor effects during photo editing to get a processed downscaled image.

24. The apparatus of claim 17, wherein the storage unit is further adapted for storing one or more photo editing parameters related to the processing of the downscaled image by appending the one or more photo editing parameters with the one or more processing parameters as a single data file format.

25. The apparatus of claim 17, wherein the image processing unit is further adapted for conditioning of the compressed downscaled data independent of the sub-band information, comprising of:
decompressing of the downscaled image from an existing file container;
conditioning of the downscaled image using one or more photo-editor operations;
compressing the processed downscaled image;
replacing the stored compressed downscaled image with the newly conditioned compressed downscaled image in the same file container; and
appending the relevant photo-editing parameters in the same file container.

26. A apparatus for rendering high resolution images based on a time-domain method, comprises of:
a blur filter to obtain a filtered image from a selected image;
an image extractor for extracting a sub-band information associated with the selected image at one instance in time by taking a difference between the selected image and the filtered image;
a down scaler for performing downscaling of the filtered image;
an image signal processor (ISP) for processing the down-scaled image and storing the related parameters;
an image conditioner adapted for:
conditioning of the processed downscaled image using one or more photo editing operations and storing the related parameters;
compressing and storing the conditioned downscaled image and the sub-band information associated with the selected image;
de-compressing the downscaled image and the sub-band information;
conditioning the decompressed sub-band image, using the one or more stored processing parameters used by the downscaled image; and
an adder module adapted for adding the conditioned sub-band information with an extrapolated conditioned downscaled image to reconstruct the high resolution image.

27. An apparatus for rendering high resolution images based on a frequency-domain method, comprising at least one processing module adapted for:
performing frequency transformation on a selected image;
applying a low pass filter to obtain a low frequency data of the selected image;
applying a high pass filter to obtain high frequency data of the selected image;
processing the low frequency data of the selected image using one or more processing parameters;
conditioning the low frequency data using photo editing operations and storing the related parameters;
compressing and storing the low frequency data and the high frequency data associated with the selected image; and
decompressing the low frequency data and the high frequency data;
conditioning the decompressed high frequency data using the one or more stored processing parameters,;
adding conditioned low frequency data and conditioned high frequency data to get a full frequency spectrum conditioned data; and
applying inverse frequency transformation to reconstruct the high resolution image.

28. An apparatus for rendering high resolution images based on a closed loop method, comprising at least one processing module adapted for:
creating a downscaled image by performing a downscaling of a selected image;
performing an upscaling of the downscaled image;
extracting a sub-band information associated with the selected image by taking a difference between the upscaled image and the selected image;
conditioning the down-scaled image by an image signal processor (ISP) and storing the processing parameters;
processing the conditioned downscaled image using one or more photo-editing operations and storing the related parameters;
compressing and storing, the downscaled image and the sub-band information, associated with the selected image;
de-compressing, the downscaled image and the sub- band information;
conditioning, the decompressed sub-band image, using the stored processing parameters used by the downscaled image,; and
adding the conditioned sub-band information with an extrapolated conditioned downscaled image to reconstruct the high resolution image.

29. An apparatus for rendering a low resolution image, comprising at least one module adapted for:
decompressing a copy of a downscaled image without necessarily removing the downscaled image from a file container;
conditioning of the downscaled data using at least one of a color conversion, resizing, rotation and blending; and
rendering the downscaled image to the display;
where the low resolution image is independent of sub-band information and related processing parameters.