Sign In to Follow Application
View All Documents & Correspondence

"Image Processing Method And Image Processing Apparatus"

Abstract: To efficiently execute correction of pixel values of a motion-presence area of captured image data. In correction processing for pixel values of a particular area such as a motion-presence area, initial values of the differences or ratios between white balance adjusted image data and flash image data are set, and conversion of the initial values is performed in accordance with a smoothing filter or a conversion expression whose weight is set on the basis of flash image data, and estimated values of image differences or image ratios in the motion-presence area are calculated to apply the estimated values to execute pixel value correction of the motion-presence area. In addition, after white balance adjustment using a low resolution image and pixel value correction of the motion-presence section, a high resolution corrected image is generated on the basis of the correspondence of a corrected image to low resolution image data.

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
10 January 2006
Publication Number
34/2007
Publication Type
INA
Invention Field
COMPUTER SCIENCE
Status
Email
remfry-sagar@remfry.com
Parent Application

Applicants

SONY CORPORATION
7-35, KITASHINAGAWA 6-CHOME, SHINAGAWA-KU, TOKYO 141-0001, JAPAN.

Inventors

1. RUI YAMADA
1-14-10, HIGASHI GOTANDA, SHINAGAWA-KU, TOKYO, JAPAN.

Claims

1. An image processing method comprising the steps of: determining (S501) an image difference or an image ratio based on corresponding pixel values of first image data and second image data having different pixel values; determining (S506)an estimated value based on the image difference or the image ratio of a particular area of image data; and generating (S507) a corrected image of the particular area based on the estimated value of the image difference or the image ratio of the particular area which have been determined in the estimated-value determination step and the second image data; wherein: the first image data comprises a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash-assisted captured image, the second image data comprises a flash-assisted image I2, and the particular area comprises a motion-presence area in which a motion of a subject is detected; the image difference d(x, y) comprises a vector determined as: d(x, y) = R(x, y) — I2(x, y) where R(x, y) denotes a pixel value vector of each pixel (x, y) of the white balance adjusted image R, and I2(x, y) denotes a pixel value vector of the corresponding pixel (x, y) of the flash-assisted image I2; the image ratio d(x, y) is: d(x, y) - R(x, y)/(I2(x, y) + e) where e is a fixed value; and the corrected image generation step comprises a step of generating a corrected image of the motion-presence area on the basis of the estimated values of the image difference or the image ratio in the motion-presence area which are determined in the estimated value determination step and the flash-assisted image I2.

2. The image processing method as claimed in claim 1, wherein the image difference d(x, y) comprises a vector determined as: d(x, y) = A(x, y) - B(x, y) where A(x, y) denotes a pixel value vector of each pixel (x, y) of the first image data and B(x, y) denotes a pixel value vector of the corresponding pixel (x, y) of the second image data; and the image ratio d(x, y) is: d (x, y) = A(x, y)/(B(x, y) + e) where e is a fixed value.

3. The image processing method as claimed in claim 1, wherein the step of determining the estimated value further comprises: an initial-value setting step of setting an initial value of image difference or image ratio of the particular area of the image data on the basis of an image difference or an image ratio determined in an area in the vicinity of the particular area; and a smoothing process execution step of executing a smoothing process based on a smoothing filter applied against the initial value set in the initial-value setting step.

4. The image processing method as claimed in claim 3, wherein the step of setting the initial value comprises a step of setting an initial value of an initial-value-setting target pixel on the basis of an image difference or an image ratio of a pixel in the vicinity of the initial-value-setting target pixel for which the image-difference or the image-ratio is already set.

5. The image processing method as claimed in claim 4, wherein the step of setting the initial value comprises a step of applying a mask image in order to discriminate between the initial-value-setting target pixel and the pixel for which the image difference or the image ratio is already set.

6. The image processing method as claimed in claim 1, wherein the step of generating the corrected image further comprises a step of generating the corrected image of the particular area on the basis of the estimated values of the image differences or the image ratios in the particular area which are determined in the estimated-value determination step and the second image data; and if using the image difference, the step of generating the corrected-image is executed as a step of adding the estimated value of the image difference of the particular area determined in the estimated-value determination step to the second image data in the particular area; and if using the image ratio, the corrected-image generation step is executed as a step of multiplying the estimated value of the image difference of the particular area determined in the estimated-value determination step by the second image data in the particular area.

7. The image processing method as claimed in claim 1, wherein: the first image data comprises a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash-assisted captured image; the second image data comprises a flash-assisted image I2; the particular area comprises a motion-presence area in which a motion of a subject is detected; and if using the image difference, the step of generating the corrected-image is executed as a step of adding the estimated value of the image difference in the particular area determined in the estimated-value determination step to the flash-assisied image I2 in the motion-presence area; and if using the image ratio, the corrected-image generation step is executed as a step of multiplying the estimated value of the image ratio in the particular area determined in the estimated-value determination step by the flash-assisted c image I2 in the motion-presence area.

8. The image processing method as claimed in claim 1, wherein the step of determining the estimated-value further comprises: initial value setting step of setting an initial value of image difference or image ratio of the particular area of the image data on the basis of image difference or image ratio determined in an area in the vicinity of the particular area; and filtering process step executing pixel value conversion processing according to a pixel value conversion expression corresponding to filtering processing using a filter whose weight is set on the basis of the second image data against an initial value set in the initial-value setting step, and correcting the image difference or the image ratio of the particular area.

9. The image processing method as claimed in claim 8, wherein the initial-value setting step further comprises a step of setting an initial value of an initial-value-setting target pixel on the basis of an image difference or an image ratio of a pixel in the vicinity of the initial-value-setting target pixel for which the image-difference or the image-ratio is already set.

10. The image processing method as claimed in claim 8, wherein the initial value setting step further comprises a step of applying a mask image in order to discriminate between the initial-value-setting target pixel and the pixel for which the image difference or the image ratio is already set.

11. The image processing method as claimed in claim 8, wherein: the first image data comprises a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash-assisted captured image; the second image data comprises a flash-assisted image I2;. the particular area comprises a motion-presence area in which a motion of a subject is detected; and the filtering process step comprises a step of executing pixel value correction processing using an expression containing a function whose weight is set according to the pixel values of pixels constituting image data of the flash-assisted image I2.

12. The image processing method as claimed in claim 8, wherein: the filtering processing step comprises a step of executing pixel value correction process using the following conversion expression: (Formula Removed) where d(x, y, ch) and I2( y, y, ch) are a value corresponding to a difference image or image ratio d of each channel [ch] at a pixel position (x, y), and a pixel value of the flash-assisted image I2, respectively; d'(x, y, ch) is an updated pixel value of the image difference d of the channel [ch] at the pixel position (x, y); i and j are reference pixel positions which are used for determining the updated value d' of the value d at the pixel position (x, y); if k denotes an arbitrary natural number, x — k ≤ i ≤ x + k and y — k ≤ j < y + k, and w(x) is a weighting function which sets a weight, according to the pixel values of pixels constituting image data of the flash-assisted image I2.

13. The image processing method as claimed in claim 12, wherein the weighting function w(x) in the conversion expression is a function expressed by the following expression (Formula Removed)

14. The image processing method as claimed in claim 12, wherein the reference pixel positions i, j which are used for determining the updated value d' of the value of the pixel position (x, y) in the above conversion expression are xx — k ≤ i ≤ x + k and y — k ≤ j ≤ y + k, where k is an arbitrary natural number and k is any of 1, 2 and 3.

15. The image processing method as claimed in claim 1, further comprising the steps of: storing in memory no flash-assisted low resolution image data IIL based on a no flash- assisted image; storing in memory flash-assisted high resolution image data I2H based on a flash-assisted image and flash resolution image data I2L; storing in memory no flash-assisted low resolution image data I3L based on a no flash-assisted image; and detecting a motion-presence area on the basis of the no flash-assisted low resolution image data IIL and the no flash-assisted low resolution image data I3L; wherein the no flash-assisted low resolution image data IlL is set as the first image data and the flash-assisted low resolution image data I2L is set as the second image data to determine the estimated values and generate a white balance adjusted image R including white balance adjustment processing and pixel value correction processing for a motion area; and in the corrected-image generation step, a high resolution final corrected image RH is generated on the basis of corresponding pixel values of the pixel value adjusted image R, the flash-assisted high resolution image data I2H and the flash-assisted low resolution image data I2L.

16. The image processing method as claimed in claim 15, wherein the corrected-image generation step further comprises the steps of acquiring pixel value conversion information on the corresponding pixels of the pixel value adjusted image R relative to the flash-assisted low resolution image data I2L; and executing pixel value conversion of the flash-assisted high resolution image data I2H.

17. An image processing apparatus comprising: a determining means for determining an image difference or an image ratio based on corresponding pixel values of first image data and second image data having different pixel values; an estimation means for determining an estimated value based on the image difference or the image ratio of a particular area of image data; and a generating means for generating a corrected image of the particular area based on the estimated value of the image difference or the image ratio of the particular area which have been determined by the estimation means and the second image data.

18. The image processing apparatus as claimed in claim 17, wherein the estimation means further comprises: an initial-value setting unit for setting an initial value of image difference or image ratio of the particular area of the image data on the basis of an image difference or an image ratio determined in an area in the vicinity of the particular area; and a smoothing process execution unit for executing a smoothing process based on a smoothing filter applied against the initial value set by the initial-value setting unit.

19. The image processing apparatus as claimed in claim 17, wherein the estimation means further comprises: initial value setting unit of setting an initial value of image difference or image ratio of the particular area of the image data on the basis of image difference or image ratio determined in an area in the vicinity of the particular area; and filtering unit for executing pixel value conversion processing according to a pixel value conversion expression corresponding to filtering processing using a filter whose weight is set on the basis of the second image data against an initial value set in the initial-value setting step, and correcting the image difference or the image ratio of the particular area.

Specification

The present invention relates to an image processing method and image processing apparatus [0001] The present invention relates to an image processing method and an image processing device as well as a computer program, and more specifically, to an image processing method and an image processing device as well as a computer program each of which for enabling image adjustment to alleviate the problem of difference in color temperature between ambient light and flash light by using a plurality of images and which enables efficient high-speed correction processing even in the case where an image shake occurs between a plurality of images used for image adjustment. Background Art [0002] Flashes (electrical flashes or strobes) are used as auxiliary light sources during camera photography. In recent years, DSCs (Digital Still Cameras) are rapidly spreading, and flash photography is often performed with DSCs as well. Flashes can be used to perform various kinds of photography such as fill-in light (a technique for weakening a shadow which appears extremely strongly on the face of a person or the like), backlight compensation (a technique for preventing the face of a person from losing shadow detail when the person standing with the sun behind is photographed), catch light (a technique for beautifully photographing the pupil of an eye with a twinkhng "light spot" reflected in the pupil), and daylight synchronized flash (daylight synchro, a technique which uses flash as auxiliary light rays in the daytime or evening). On the other hand, when flash photography is performed, there is a case where color balance is impaired or loss of highlight detail occurs. One object of the present invention is to provide appropriate means capable of correcting undesirable phenomena which occur during flash photography. [0003] In general, in digital cameras, white balance (WB) is performed so that an image of a white subject can be taken in white light. For example, white balance adjustment is performed in such a manner that when an image is to be taken in a light component environment, such as natural light, illumination light or flash (strobe, in which the color temperature of light irradiated toward a subject is high and blue (B) light is strong, sensitivity to blue light is suppressed, whereas when an image is to be taken in a light component environment in which the color temperature of light irradiated toward a subject is low and red (B) light is strong, sensitivity to red light is suppressed and sensitivity to blue (B) light is made relatively high. [0004] White balance adjustment generally executes adjustment processing in which adjustment parameters are set according to light sources to be used during image capture. For example when image capture with flash is performed, white balance adjustment is performed in accordance with parameters corresponding to light components of flash light used. [0005] However, if image capture with flash is performed when there is ambient light other than flash light, two kinds of lights, i.e., flash light and ambient light, are irradiated to a subject and reflected light therefrom reaches the image pickup element of the camera, whereby image capture is performed. In the case of this image capture, if white balance adjustment is carried out according to flash light, the section of the subject which is irradiated with a large amount of flash light is adjusted to natural color. However, if white balance adjustment according to parameter settings matched to the light components of flash light is performed on an area which is out of the reach of flash light and is captured in reflected light of only ambient light, for example, a background image area, appropriate white balance adjustment is not executed and the area is outputted as an area having unnatural color. [0006] Conversely, if white balance adjustment matched to a background section, i.e., white balance adjustment based on the assumption that the image is taken with only ambient light, is executed on the entire captured image, the section irradiated with a large amount of flash light is adjusted to unnatural color. [0007] To cope with this problem, several constructions have been proposed. For example, Patent Document 1 (Japanese Patent Application Publication Hei8-51632) discloses a construction which acquires an image taken without flash and an image taken with flash, divides each of these two captured images into blocks, making a comparison between their luminance values in units of one block, and performs different white balance adjustment on each block of the image taken with flash on the basis of the result of the comparison between the luminance values. [0008] During white balance adjustment, any one of white balance adjustment matched to flash light, white balance adjustment matched to light intermediate between flash light and ambient light and white balance adjustment matched to ambient light is selected and executed. However, this construction needs to perform processing in units of blocks, resulting in problems such as the occurrence of block distortion and the problem that no correct processing can be effected when a subject moves. [0009] Patent Document 2 (Japanese Patent Application Publication JP-A-2000-308068) discloses the following processing construction. Namely, an image is taken with flash with a fully open aperture and a short exposure time, and after that, an image is taken without flash under the originally intended exposure conditions. The former and the latter are respectively called a first image and a second image. In addition, in the first image, pixels having not less than a predetermined level are registered as a main subject area, and the other pixels are registered as a background area. After that, the first image is subjected to white balance adjustment matched to flash light and the second image is subjected to white balance adjustment matched to ambient light, and the main subject area of the first image and the background area of the second area are combined to generate a final recorded image. [0010] However, in this construction, it is impossible to correctly effect white balance adjustment of a subject irradiated with both ambient light and flash light. [0011] Patent Document 3 (Japanese Patent Application Publication JP-A-2000-307940) discloses a construction in which image shake detection means is added to the above-mentioned construction of Patent Document 2. In this construction, if it is determined that a shake has occurred, the above-mentioned first image is used as a recorded image without modification and the processing of combining the first image and the second image is not executed. Accordingly, if a shake is detected, unnaturalness due to the difference in color temperature between flash light and ambient light cannot be solved. [0012] Patent Document 4 (Japanese Patent Application Publication Hei8-340542) discloses a construction which performs division on the luminance of each pixel of an image taken with flash and the luminance of the corresponding pixel of an image taken without flash to find the contribution of flash light, and performs white balance adjustment on the image taken with flash, on the basis of this contribution. [0013] In this construction, an image taken with mixed reflected lights of flash light and ambient light is simply interpolated with white balance parameters for flash light and ambient light on the basis of the contribution of flash light, thereby generating a final image. However, if a physical reflection model of light is taken into account, components originating from flash light and components originating from ambient light should be independently processed, and it is impossible to generate an optimum result image merely by processing the image taken with mixed reflected lights of flash light and ambient light. [Patent Document 1] Japanese Patent Application Publication Hei8'51632 [Patent Document 2] Japanese Patent Application Publication JP-A-2000-308068 [Patent Document 3] Japanese Patent Application Publication JP-A-2000-307940 [Patent Document 4] Japanese Patent Application Publication Hei8-340542 Disclosure of the Invention [Problem to be solved by the Invention] [0014] The present invention has been conceived in view of the problems of the above-mentioned related art, and provides an image processing method and an image processing device as well as a computer program each of which is capable of performing optimum image adjustment on an image taken under an environment where ambient light and flash light are mixed, and which is capable of performing pixel value correction such as optimum white balance adjustment efficiently at high speed without failure even when an image shake is detected. [Means for Solving the Problems] [0015] In a first aspect, a preferred embodiment of the present invention provides an image processing method including the steps of calculating an image difference or image ratio on the basis of corresponding pixel value of first image data and second image data having different pixel values! calculating estimated values based on the image difference or the image ratio in a particular area of image data! and generating a corrected image of the particular area on the basis of the second image data and the estimated values of the image difference or the image ratio in the particular area which have been calculated in the estimated-value calculation step. [0016] Further, another preferred embodiment of the present invention is characterized in that when A(x, y) denotes a pixel value vector of each pixel (x, y) of the first image data and B(x, y) denotes a pixel value vector of the corresponding pixel (x, y) of the second image data, the image difference d(x, y) is a vector calculated as: d(x, y) = A(x, y) - B(x, y) and the image ratio d(x, y) is' d(x, y) = A(x, y)/(B(x, y) + e) where e is a vector calculated as a fixed value. [0017] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the first image data is a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash captured image, the second image data is a flash image 1%, and the particular area is a motion-presence area in which a motion of a subject is detected; where R(x, y) denotes a pixel value vector of each pixel (x, y) of the white balance adjusted image R and ^(x, y) denotes a pixel value vector of the corresponding pixel (x, y) of the flash image lz, the image difference d(x, y) is a vector calculated as: d(x, y) = R(x, y) - I2(x, y) and the image ratio d(x, y) is: d(x, y) = R(x, y)/(I2(x, y) + e) where e is a vector calculated as a fixed value," and the corrected image generation step is a step of generating a corrected image of the motion-presence area on the basis of the flash image \z and the estimated values of the image difference or the image ratio in the motion-presence area which are calculated in the estimated value calculation step. [0018] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the estimated-value calculation step includes an initial-value setting step of setting initial values of image differences or image ratios of the particular area of the image data on the basis of image differences or image ratios determined in an area in the vicinity of the particular area, and a smoothing processing execution step of executing smoothing processing based on a smoothing filter as to the initial values set in the initial-value setting step. [0019] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the initial-value setting step is a step of setting an initial value of an initial-value-setting target pixel on the basis of an image difference or an image ratio of a pixel in the vicinity of the initial-value setting target pixel for which the image difference or image ratio is already set. [0020] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the initial-value setting step includes a step of applying a mask image in order to discriminate between the initial-value-setting target pixel and pixel to which the image difference or image ratio is already set. [0021] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the corrected-image generation step is a step of generating the corrected image of the particular area on the basis of the second image data and the estimated values of the image differences or the image ratios in the particular area which are calculated in the estimated-value calculation step. In processing using image differences, the corrected-image generation step is executed as a step of adding the estimated values of the image differences of the particular area calculated in the estimated-value calculation step to the second image data in the particular area, whereas in processing using image ratios, the corrected-image generation step is executed as a step of multiplying the estimated values of the image differences of the particular area calculated in the estimated-value calculation step by the second image data in the particular area. [0022] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the first image data are a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash captured image, the second image data are an flash image 12, and the particular area is a motion-presence area in which a motion of a subject is detected. In processing using image differences, the corrected-image generation step is executed as a step of adding the estimated values of the image differences in the particular area calculated in the estimated-value calculation step to the flash image la in the motion-presence area, whereas in processing using image ratios, the corrected-image generation step is executed as a step of multiplying the estimated values of the image ratio in the particular area calculated in the estimated-value calculation step by the flash image \2 in the motion-presence area. [0023] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the estimated-value calculation step includes an initial-value setting step of setting initial values of image differences or image ratios of the particular area of the image data on the basis of image differences or image ratio determined in an area in the vicinity of the particular area, and a filtering processing step of executing pixel value conversion processing according to a pixel value conversion expression corresponding to filtering processing using a filter whose weight is set on the basis of the second image data, as to the initial values set in the initial-value setting step, and correcting the image differences or the image ratio of the particular area. [0024] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the initial-value setting step is a step of setting an initial value of an initial-value-setting target pixel on the basis of an image difference or an image ratio of a pixel in the vicinity of the initial-value setting target pixel to which the image difference or the image ratio is already set. [0025] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the initial-value setting step includes a step of applying a mask image in order to discriminate between the initial-value-setting target pixel and the pixel to which the image difference or the image ratio is already set. [0026] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the first image data is a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash captured image, the second image data are a flash image 12, and the particular area is a motion-presence area in which a motion of a subject is detected! and in that the filtering processing step is a step of executing pixel value correction processing using an expression containing a function whose weight is set according to the pixel values of pixels constituting image data of the flash image \2. [0027] Further, another preferred embodiment of the image processing method of the present invention is characterized in that the filtering processing step is a step of executing pixel value correction processing using the following conversion expression^ where d(x, y, ch) and lz(y, y, ch) are a value corresponding to a difference image or image ratio d of each channel [ch] at a pixel position (x, y), and a pixel value of the flash image Lz, respectively; d'(x, y, ch) is an updated pixel value of the image difference d of the channel [ch] at the pixel position (x, y); i and j are reference pixel positions which are used for calculating the updated value d' of the value d at the pixel position (x, y). When k denotes an arbitrary natural number, x — k) on an image \z picked up by the two kinds of lights, ambient light and flash light, being reflected from the object will be considered here. It is assumed that in the above expression (Expression 3), the light source 1 is ambient light and the light source 2 is flash light. The color (lir, lig, lib) of the ambient light can be measured by a method used in automatic white balance adjustment which has heretofore been performed. Since the color (\2Z, bg, bb) of the flash light is peculiar to flash devices, this color is known and can be set in advance. Furthermore, if k' is known, the pixel (ir, ig, ib) can be decomposed into ambient light components (llr * or', llg * og', llb * ob') and flash light components (k' * I2r * or', k' * I2g * og', k' * I2b * ob'). The ambient light components and the flash light components are separated and are independently WB processed, and if the resultant images are added together and reconstructed, it is possible to solve the unnaturalness of the image due to the difference in color temperature between the ambient light and the flash light. [0081] The pixel value correction of Step Sill is executed in accordance with the above-mentioned processing as to the motion section detected in Step S107 in the flow described with reference to Fig. 2. A specific processing example will be described below. [0082] As described above, in Step S107, it is detected whether an image shake due to a motion of the subject itself has occurred, during the capture of the three images in Steps S101, S103 and S105. This processing of detecting whether an image shake due to a motion of the subject itself has occurred is performed by comparing two of the three images. [0083] For example, as shown in Fig. 5, in the case where a ball 200 is rolling while A the no flash-assisted image Ii, B the flash image 12, and C the no flash-assisted image Is are being continuously captured, a difference image D Is — Ii between the image Ii of A and the image Is of C is acquired to detect an area 210 in which an image shake due to the motion of the subject itself is occurring. [0084] A specific processing procedure for the pixel value correction processing of the motion section will be described below with reference to Figs. 6 and 7. Fig. 6 is a flowchart showing the specific processing procedure for the pixel value correction processing of the motion section, and Fig. 7 shows a pixel area containing the motion section which is a correction target, i.e., the area 210 shown in Fig. 5. [0085] As shown in Fig. 7, in an image area which is determined to be moving, the in the vicinity of pixels of an image area determined to be not moving (pixels each arranged at a position surrounded by 8 pixels) are defined as inner boundary pixels 250 of a motion-presence section. In addition, in the image area which is determined to be moving, the pixels other than the inner boundary pixels 250 are defined as motion-presence non-inner-boundary pixels 252. [0086] In addition, in the image area which is determined to be not moving, pixels arranged at the in the vicinity of positions of the image area determined to be moving (pixels each arranged at a position surrounded by 8 pixels) are defined as outer boundary pixels 251 of the motion-presence section. In addition, in the image area which is determined to be not moving, the pixels other than the outer boundary pixels 251 of the motion-presence section are defined as motion absence non-outer-boundary pixels 253. [0087] As to any of the pixels of the motion-presence section, the ratio of the intensity (scalar value) ki of the light source 1 (ambient light) to the intensity (scalar value) k2 of the light source 2 (flash light), i.e., the light intensity scalar ratio k' = k2/ki, is unknown. It is assumed here that an objective image which is correctly adjusted in white balance (WB) and is corrected for its motion section has a pixel construction in which the pixel values of the motion-presence section and its motion absence section are smoothly varied. [0088] Under this assumption, the value of the light intensity scalar ratio k' is found with regard to each of the motion-presence section outer boundary pixels 251. As to each of these motion-presence section outer boundary pixels 251, the ambient light components (llr * or', llg * og', llb * ob') in the above-mentioned expression (Expression 3) are equal to the values of the corresponding pixel in the no flash-assisted captured image data Ii, whereby the value of the light intensity scalar ratio k' = k2/ki can be found on the basis of the pixel values (ir, ig, ib) of the flash image 12 and the expression (Expression 3). The processing of calculating the ratio: k' = k2/ki of the intensity (scalar value) ki of the light source 1 (ambient light) to the intensity (scalar value) k2 of the light source 2 (flash light) is the processing of Step S301 of Fig. 6. [0089] From the processing of this step S301, the light intensity scalar ratio k' is found with regard to each of the motion-presence section outer boundary pixels 251. However, the values of the light intensity scalar ratios k' corresponding to the respective pixels contained in the motion-presence section are unknown. However, the values of the light intensity scalar ratios k' corresponding to these respective pixels contained in the motion-presence section are interpolated from the calculated values of k' corresponding to the motion-presence section outer boundary pixels 251. As one example of the interpolation method, processing using radial hasis functions (RBFs: Radial Basis Functions) can be enumerated. [0090] As a reference document regarding interpolation of data using radial basis functions (RBFs), for example, J. C. Carr, et al, "Reconstruction and Representation of 3D Objects with Radial Basis Function," ACM SIGGRAPH 2001, Los Angeles, CA, pp. 67-76, 12-17 August 2001 can be enumerated. [0091] The Radial Basis Functions (radial basis functions) are functions whose value monotonously decreases (or increases) with the increasing distance from the central point so that its contour lines form hyper spheres (in the case of three dimensions, circles or ellipses). It has been known that if the problem of estimating the values of heights of unknown points by constructing a function which passes sample points having known heights and becomes as smooth as possible is to be solved, an RBF centered at the known sample points may be superimposed. [0092] Specifically, if sample points are present in a two-dimensional space, the sample points are defined as follows' , where cix and ci^ respectively represent an x coordinate value and a y coordinate value at the sample point i. Letting {hi} = (l < i < n) denote heights at the respective points, the desired function' is expressed as the following expression (Expression 4) by using an RBF: =1 (Expression 4) Here, p(x) is: p(x) = p2y [0093] Incidentally, an example of the base function: is-. . 'or [0094] However, {d±} (1 < i < n) , {pj (0 < i < 2) cannot be specified with only the above expression (Expression 4). For this reason, {dj (1 < i < n) , {pj^} (0 < i < 2) which satisfies the following expression (Expression 5) is found: [0095] Therefore, from the following expression:and the expression (Expression 5), it is possible to find (the number of unknown numbers n + 3, the number of expressions n + 3) {d,} (1 < i < n),{Pi} (0 < i < 2). [0096] If the light intensity scalar ratios k' in the motion-presence section non-outer-boundary pixels 251 are used as samples and the following expression' is constructed, the light intensity scalar ratio k' at an arbitrary position can be found. [0097] This processing is an RBF construction processing based on the light intensity scalar ratios k' at the sample points in Step S302 shown in Fig. 6 (k' in the motion-presence section non-outer-boundary pixels 251). [0098] Through this RBF construction processing, estimation is performed on the ratios of the intensity (scalar value) ki of the light source 1 (ambient light) to the intensity (scalar value) k2 of the light source 2 (flash light), which ratios correspond to the respective pixels on the flash image \z in the section determined to be moving, i.e., the light intensity scalar ratios k': k' = k2/ki corresponding to the respective pixels contained in the motion-presence section. The estimated light intensity scalar ratios k' corresponding to the respective pixels are used to decompose the motion-presence area section of the image lz into ambient light components and flash components, and white balance (WB) adjustment processing is individually executed according to the color temperatures of the respective lights. [0099] Namely, since the light intensity scalar ratio k' is found at each pixel position in the motion-presence section, this k' and the pixel value (ir, ig and ib) at each pixel position in the flash image Iz as well as the known light component (lir, lig, lib) of the light source 1 (ambient light) and the known light component (\2i, he, bb) of the light source 2 (flash light) are substituted into the above-mentioned expression (Expression 3), whereby the reflectance (or', og', Ob') of the subject based on only the light source 1 (ambient light) is found. [0100] Furthermore, the pixel value of the subject in the case of being irradiated with only the ambient light components: llb * ob'), and the pixel value of the subject in the case of being irradiated with only the flash light components' (k' * I2r * or', k' * I2g * og', k' * I2b * ob'), are found, and the two white balance adjustments which are based on the independent settings of the parameters corresponding to the respective light components as mentioned above with reference to Figs. 3 and 4 are executed on the respective pixel values, and the final white balance adjusted pixel value is calculated by the processing of re-synthesizing these adjusted data. [0101] The procedure of correction of pixel values of a motion-presence section can be summarized as the following processing of a to f. a. First, as data corresponding to pixels of a motion absence section in the vicinity of the motion-presence section, the light intensity scalar ratio of two kinds of light sources, i.e., light irradiated from only the first light source and light irradiated with only ambient light without the first light source, is found as to each portion of a subject whose image is picked up by any one of the pixels. b. A radial basis function (RBF: Radial Basis Function) is applied to calculate a light intensity scalar ratio corresponding to each of the pixels of the motion-presence section. c. On the basis of the light intensity scalar ratio corresponding to each of the pixels of the motion-presence section, the pixel value of each of the pixels of the motion-presence section in an image corresponding to an image captured under an environment irradiated by only the first light source is calculated as a first pixel value. [0102] d. On the basis of the light intensity scalar ratio corresponding to each of the pixels of the motion-presence section, the pixel value of each of the pixels of the motion-presence section in an image corresponding to an image captured under an environment irradiated by ambient light which does not include the first light source is calculated as a second pixel value. e. Pixel value adjustment processing (white balance adjustment) is executed on the basis the first pixel value, and pixel value adjustment processing (white balance adjustment) is executed on the basis the second pixel value. f. The generated two adjusted pixel values are added together. [0103] In this manner, the white balance (WB) adjusted pixel values relative to the pixels contained in the motion section are overwritten onto the image data R generated in Step S110 of Fig. 2, to find a first corrected image data R. Namely, only the pixel values of the motion section due to the motion of the subject in the captured image are reset in Step Sill, and are overwritten onto the white balance adjusted image R generated in Step S110, to find the first corrected image data R'. [0104] Incidentally, as mentioned previously in the processing flow of Fig. 3, in the case where the level correction of the flash light (S202) has been performed, equivalent level correction is performed during the calculation of the pixel values of the motion section, and after that, addition is performed on the white balance adjusted pixel values based on the ambient light and the flash light component. [0105] In this manner, the pixel values of the motion section are reset, and the pixels of the image R which correspond to the motion section are rewritten. This processing is the processing of Step S303 of Fig. 6. [0106] However, in the first corrected image R' found by overwriting the white balance adjusted image R generated in Step S110 with the pixel values which have been reset in the motion-presence section, there is a case where the pixels of the motion absence section of the original image R and the boundary of the reset pixels of the motion-presence section are not smoothly connected. A cause of this case can be considered to be that the color of the ambient light failed to be correctly measured or a loss of highlight detail has occurred in the flash captured image 12. For this reason, to cope with that case, further color conversion processing for smoothing a boundary portion is executed. [0107] Specifically, the processing of Steps S304 to S306 in Fig. 6 is performed. In Step S304, the ratios of the pixel value of a certain pixel a 254 among the motion-presence section inner boundary pixels 250 of the first corrected image R' (refer to Fig. 7) to the average value of the values in the first corrected image R' of motion-presence section outer boundary pixels (pixels 255 in Fig. 7) among the in the vicinity of pixels of the pixel a 254 (pixels arranged at positions contained in the surrounding 8 pixels) are found with respect to individual color components (ar, ag, ab). These ratios are stored as color component data corresponding to the pixel a 254. [0108] Similarly, ratios relative to the individual color components (ar, ag, ab) which correspond to the respective pixels arranged at all positions among the motion-presence section inner boundary pixels 250 are calculated as pixel color component ratio data, and are stored as color component ratio data corresponding to the respective pixels. [0109] Then, in Step S305, an RBF based on the color component ratio data is constructed by using all of the motion-presence section inner boundary pixels 250 as sample points. Finally, in Step S306, as to each of the pixels of the motion-presence section, color component ratio data (ccr, cxg, ab) which correspond to the respective pixels of the motion-presence section are found on the basis of the RBF based on the constructed color component ratio data, and the pixel values of the respective pixels which are set in the first corrected image R' are multiplied by the color component ratio data (ocr, ag, ab) of the corresponding images, whereby new pixel values are calculated to execute second color conversion in which these pixel values are substituted for the pixels of the motion-presence section. A second corrected image R" obtained by executing this color conversion processing is provided as an output image. This processing is the processing of Step S306 of Fig. 6. [0110] Incidentally, if the motion-presence section image and the motion absence section image in the first corrected image R' obtained by performing the processing of Steps S301 to S303 in Fig. 6 are smoothly connected at the boundary therebetween, the processing of Step S304 to Step S306 may be omitted. [0111] The above description has referred to a method using an RBF, with regard to a method of interpolating points with values given at sample points other than the points, but this method is not limitative and other methods may be used to perform interpolation. The above-mentioned processing is a specific processing example of Step Sill of Fig. 2. In this manner, if the image shake based on the motion of the subject itself is detected from the continuously captured images, processing according to the flow shown in Fig. 6 is executed, and the second corrected image R" or the first corrected image R' generated by the above-mentioned processing is provided as a final output image. [0112] Incidentally, if it is known in advance that the subject itself does not move, the processing of Steps S107 to Sill of Fig. 2 are not necessary. In addition, since the decision in Step S108 is necessarily "No", the decision processing of Step S108 need not be performed, so that Steps S105 and S106 for creating and storing data to be used for the decision of Step S108 are also unnecessary. [0113] Accordingly, if it is known in advance that the subject itself does not move, the no flash-assisted captured image la which is captured in Step S105 is an unnecessary image, and in this case, two images, i.e., the no flash-assisted captured image Ii and the flash captured image Iz, need only to be captured. [0114] In this specification, the term "flash" has been used to describe an illuminating device which emits light in the case of a dark subject, but this device is also called a strobe. Accordingly, the present invention is not limited to flashes and can be generally applied to illuminating devices which emit light in the case of dark subjects. [0115] As described hereinabove, in the present embodiment, as to images which are respectively captured under a plurality of different light irradiations such as flash light and ambient light, images each captured under an environment illuminated with a single kind of light are acquired or generated, and white balance adjustments according to parameters which are set on the basis of the color components (color temperatures) of the respective irradiation lights are respectively executed on the images each captured under an environment illuminated with a single kind of light, and these adjusted images are synthesized. Accordingly, it is possible to realize appropriate white balance adjustment processing in which the difference in color temperature between flash light and ambient light is reduced. [0116] Fig. 8 is a block diagram showing the functional construction of a digital signal processing section (DSP) (which corresponds to the DSP 106 in Fig. l) which executes processing according to the present embodiment. [0117] The processing in the digital signal processing section (DSP) shown in Fig. 8 will be described, while referring to the flowchart shown in Fig. 2. [0118] In Steps S101 to S106 of Fig. 2, the no flash-assisted captured image Ii, the flash captured image lz, and the no flash-assisted captured image Is are respectively stored in frame memories 301, 302 and 303. Incidentally, as the image storing frame memories, a memory constructed in the digital signal processing section (DSP) or a bus-connected memory (the memory 112 of Fig. l) may also be used. [0119] The motion detection processing of Step S107 is executed by a motion detection section 309. This is executed as detection processing using difference data based on the no flash-assisted captured image Ii and the no flash-assisted captured image la as described previously with reference to Fig. 5. [0120] The white balance adjustment processing based on the above-mentioned plurality of images in Step S112 is the processing described previously with reference to Figs. 3 and 4. [0121] First, the difference image F = lz - Ii is found in a difference image calculation section 304 on the basis of the no flash-assisted captured image Ii and the flash captured image lz (Fig. 3, S201). Then, white balance adjustment processing according to parameters which are set on the basis of the components of flash light is executed by a white balance adjustment section 307 as to the difference image data F = \z - Ii, i.e., the image F corresponding to an image captured under the condition that only flash light is irradiated (Fig. 3, Step S202). Furthermore, white balance adjustment processing according to parameters which are set on the basis of an estimated value of ambient light components which is estimated by an ambient light component estimation section 306 is executed on the no flash-assisted captured image Ii by a white balance adjustment section 305 (Fig. 3, S203). [0122] Furthermore, the pixel values of the two images acquired by these two white balance adjustment processings are added together in a pixel value addition section 308 (Fig. 3, S204). [0123] If the captured image does not contain a motion section, no processing is executed in a motion-section corrected pixel value calculation section 310 and image data having the pixel values added together in the pixel value addition section 308 is outputted via an output switching section 312 as a white balance adjusted image. The image data is outputted to the D/A converter 108 which executes digital/analog conversion (refer to Fig. l), the codec 111 which executes coding processing, or the like. [0124] On the other hand, if a motion area of the subject itself is detected in the motion detection section 309 as the result of the motion detection using the difference data based on the no flash-assisted captured image Ii and the no flash-assisted captured image Is, the correction (conversion) of the pixel values of the motion-presence section which has previously been described with reference to Figs. 6 and 7 is, furthermore, performed in the motion-section corrected pixel value calculation section 310, and an image having pixel value data in which corrected pixel values are substituted for the motion-presence section is outputted via the output switching section 312. [0125] A white balance adjustment section 311 executes the processing of Step S113 in the processing flow of Fig. 2. Namely, if it is determined that correction is impossible, for example, in the case where a motion area is detected in the motion detection section 309 but the proportion of the motion area in the entire image is high, the white balance adjustment section 311 receives the input of the flash captured image \2, executes white balance adjustment according to preset parameters, and outputs this adjusted image via the output switching section 312. [0126] The construction shown in Fig. 8 is shown to be divided into individual processing sections for the convenience of description of its function, but actual processing can be executed by a processor in the DSP in accordance with a program which executes processing according to each of the above-mentioned processing flows. [0127] In the above-mentioned white balance adjustment processing, reference has been made to a construction example in which, as described with reference to Figs. 3 and 4, image data based on a single kind of irradiated light is found as to an ambient light component and a flash light component and as to each of those image data, white balance adjustment processing according to parameters which are set on the basis of the corresponding one of the ambient light component and the flash light is executed. [0128] The following description refers to a construction example in which white balance adjustment processing according to parameters which are set on the basis of ambient light is executed on the difference image data F = 12 • Ii corresponding to the condition that only flash light is irradiated. [0129] Fig. 9 shows a white balance adjustment processing flow based on a plurality of image data in the present embodiment, which flow corresponds to the flow of Fig. 3 in the previously-mentioned embodiment. [0130] In Step S401, the differences between the components of the flash image lz and the components of the respective colors of the pixels of the no flash-assisted captured image Ii are found, and the difference image F = 12 - Ii is generated and stored in the memory. The difference image F = Lj" Ii becomes equivalent to an image which is picked up by the solid state image pickup element of the camera when the subject is irradiated with only flash light with no ambient light existing at all and only the flash light is reflected from the subject and enters the solid state image pickup element. Then, in Step S402, white balance (WB) adjustment matched to the color temperature of ambient light is executed on the image F, thereby generating the corrected image F'. Namely, white balance (WB) adjustment is executed on the difference image data F on the basis of parameters which are set according to the color temperature of ambient light, whereby the corrected image F' is generated. [0131] At this time, by directly comparing each of the pixels of the difference image F and the corresponding one of the pixels of the no flash-assisted captured image Ii, white balance adjustment processing is executed so that flash light matches the color of ambient light. As a specific example of this WB processing, a pixel (n, gi, bi) of the no flash-assisted captured image Ii which is at the same position as a pixel (rf, gf, bf) of the difference image F is employed to execute pixel value conversion using the following Expressions 6 and 7 as to the R and B components of the pixel of the difference image F according to the level of the G signal of the pixel of the no flash-assisted captured image l\- rf' = rf * (g± I gf) ••• (Expression 6) br'= bf * (gA / gf) •••(Expression?) [0132] Then, r/ and rf as well as bf and bf are compared to obtain the following values^ ar = r± / rf' = (r. * gf) / (r£ * gj -"(Expressions) ab = bt /b£'= (bt * gf)/(bf * gj ••• (Expression 9) [0133] A WB parameter is found by averaging ar and ab which are found from Expressions 8 and 9, with respect to all pixels. The R component and the B component of each pixel of the image F are multiplied by the found parameter, whereby white balance adjustment is performed. Through this processing, the image F is converted to an image which looks as if it were taken under flash light having the same color as ambient light, and this image is stored as the image F'. [0134] Further, in Step S403, the difference image F' and the no flash-assisted captured image Ii are synthesized to generate a first white balance adjusted image Ri. The first white balance adjusted image Ri is an image in which the color temperatures of ambient light and flash light are coincident with each other. [0135] Finally, in Step S404, white balance adjustment is further executed on the first white balance adjusted image Ri, whereby a second white balance adjusted image R2 is generated. [0136] As a parameter for WB in Step S404, a value which is set by a user may be used, or known automatic white balance techniques may also be used to convert the first white balance adjusted image Ri so that the final second white balance adjusted image R2 has natural hue. [0137] According to the processing of this embodiment, it is possible to achieve white balance adjustment which takes the components of ambient light into greater account. Specifically, it is possible to achieve adjustment according to ambient light! for example, if ambient light is reddish owing to a sunset or the like, i.e., contains a large amount of R component, the entire image is adjusted with a reddish hue. [0138] Fig. 10 is a block diagram showing the functional construction of the digital signal processing section (DSP) (corresponding to the DSP 106 of Fig. l) which executes processing according to the present embodiment. [0139] The processing in the digital signal processing section (DSP) shown in Fig. 10 will be described, while referring to the flowchart shown in Fig. 9. [0140] In Steps S101 to S106 of Fig. 2, the no flash-assisted captured image Ii, the flash captured image \z, and the no flash-assisted captured image Is are respectively stored in frame memories 401, 402 and 403. [0141] The difference image data F = 12 - Ii is found in a difference image calculation section 404 on the basis of the no flash-assisted captured image Ii and the flash captured image 12 (Fig. 9, S401). Then, white balance adjustment processing according to parameters which are set on the basis of ambient light components is executed by a white balance adjustment section 405 as to the difference image data F = \z ' Ii, i.e., the image F corresponding to an image captured under the condition that only flash light is irradiated (Fig. 9, Step S402). Furthermore, the pixel values of the difference image F' acquired by this white balance adjustment processing and those of the no flash-assisted captured image Ii are added together in a pixel value addition section 406, whereby the first white balance adjusted image Ri is generated (Fig. 9, S403). Furthermore, in a white balance adjustment section 407, white balance adjustment is performed on the first white balance adjusted image Hi, whereby the second white balance adjusted image R2 is generated. [0142] If the captured image does not contain a motion section, no processing is executed in a motion-part corrected pixel value calculation section 409 and the second white balance adjusted image R2 is outputted via an output switching section 411 as a white balance adjusted image. The image R2 is outputted to the D/A converter 108 which executes digital/analog conversion (refer to Fig. l), the codec 111 which executes coding processing, or the like. [0143] On the other hand, if a motion area of the subject itself is detected in the motion detection section 408 as the result of the motion detection using the difference data based on the no flash-assisted captured image Ii and the no flash-assisted captured image Is, the correction (conversion) of the pixel values of the motion-presence section which has previously been described with reference to Figs. 6 and 7 is, furthermore, performed in the motion-part corrected pixel value calculation section 409, and an image having pixel value data in which corrected pixel values are substituted for the motion-presence section is outputted via the output switching section 411. [0144] A white balance adjustment section 410 executes the processing of Step S113 in the processing flow of Fig. 2. Namely, if it is determined that correction is impossible, for example, in the case where a motion area is detected in the motion detection section 408 but the proportion of the motion area in the entire image is high, the white balance adjustment section 410 receives the input of the flash captured image \z, executes white balance adjustment according to preset parameters, and outputs this adjusted image via the output switching section 411. [0145] The construction shown in Fig. 10 is shown to be divided into individual processing sections for the convenience of description of its function, but actual processing can be executed by the processor in the DSP in accordance with a program which executes processing according to each of the above-mentioned processing flows. [0146] According to the present embodiment, it is possible to achieve white balance adjustment which takes the components of ambient light into greater account. [0147] [Embodiment 2] [0147] As Embodiment 2 of the present invention, similarly to the above description, the following description refers to a construction which adopts a technique different from that of Embodiment 1 as the processing of Step Sill of Fig. 2, i.e., a pixel value correction method for a motion section, in an image processing method and an image processing device both of which execute optimum white balance adjustment processing for image capture under environments in which ambient light and flash light are mixed. The present method enables, for example, high speed processing of high resolution images. [0148] In the above-mentioned embodiment 1, reference has been made to a construction capable of reducing the difference in color temperature between flash light and ambient light. The construction is summarized as follows' first, only flash light components are extracted from a difference image between an image taken with flash and an image taken without flash, and after that, this difference image and the image taken without flash are independently subjected to color conversion and are re-synthesized to reduce the difference in color temperature between ambient light and flash light. [0149] According to this construction, although it is necessary to continuously capture a plurality of images, it is possible to cope with even the case where a subject or the like moves during that time. Namely, three images are continuously captured in the order of a flash image, a no flash-assisted image and a flash image, and a motion section of the subject is detected from the difference between the two no flash-assisted images, and as to each pixel contained in the motion section, interpolation processing is performed on the ratio of a reflected flash light component to a reflected ambient light component, from data in pixels other than the motion section, by using a radial basis function. [0150] Namely, in Embodiment 1, the processing of Step Sill shown in Fig. 1, i.e., the pixel value correction processing of the motion section in the white balance adjusted image R, is executed in accordance with the processing flow shown in Fig. 6. [0151] However, the above-mentioned motion section compensation method in some cases needs large memory sizes and high calculation costs. In particular, the method has the problem that higher resolutions of images to be corrected invoke greater increases in the necessary memory sizes and calculation costs. [0152] Embodiment 2 solves the above-mentioned problem, and provides a construction example in which during image capture which is performed with flash, it is possible to solve unnaturalness due to the difference in color temperature between ambient light and flash light, and further, even if an image shake is detected, it is possible to perform processing optimally and efficiently without failure, and even if the resolution of an objective image is high, it is possible to perform processing efficiently at high speed. [0153] An image processing device according to Embodiment 2 has a construction similar to the construction described previously in Embodiment 1 with reference to Fig. 1. Embodiment 2 is distinctive in the processing of the digital signal processing section (DSP) 106. Details of image processing according to Embodiment 2 will be described below. [0154] The image processing is realized in such a manner that, in the DSP 106, the arithmetic unit sequentially executes operations written in predetermined program codes, on a stream of input image signals. In the following, the order in which individual processing steps of the program are executed will be described with reference to a flowchart. However, the present invention may be constructed not in the form of a program which will be described in the present embodiment, but by incorporating a hardware circuit which realizes processing equivalent to functions which will be described below. [0155] The basic processing of the procedure of white balance (WB) correction processing in the present embodiment is executed in accordance with the flowchart shown in Fig. 2 mentioned above in Embodiment 1. Namely, three images are continuously captured in the order of a flash image, a no flash-assisted image and a flash image, and white balance adjustment is performed on the basis of these images. [0156] In the present embodiment, the pixel value correction processing of the motion section in Step Sill shown in Fig. 2 can be executed far more efficiently at far higher speeds. During the processing of high resolution image data in particular, appropriate pixel value correction processing can be performed at high speed even in the case of an apparatus having a small memory capacity. [0157] The case where the white balance adjustment processing based on the above-mentioned plurality of images is performed in Step S110 shown in Fig. 2 means the case where it is determined that an image shake due to a motion of the subject itself has occurred and the image shake is correctable. As to the image area of the image shake due to the motion of the subject itself, i.e., the motion section area, in the white balance adjusted image R generated in Step Slid, pixel value correction processing is executed in Step Sill. Namely, exceptional processing is performed on the pixel values of the motion section detected in Step S107, thereby modifying the white balance adjusted image R. As modification processing, there is, for example, a method of inputting the pixel values of the flash image lz corresponding to the section in which the motion is detected, referring to the pixel values of a section in which a motion is absent, in the white balance adjusted image R, determining the pixel values of the section in which the motion is detected, and synthesizing a final image. [0158] Details of pixel value correction processing according to the present embodiment will be described below. [0159] Fig. 11 is a schematic view showing one example of data which is acquired as the result of motion detection processing executed in Step S107 of Fig. 2. [0160] In Fig. 11, the section determined to be moving in Step S107 of the flow shown in Fig. 2 is called a motion-presence section pixel area 550, while the section determined to be not moving is called a motion absence section pixel area 551. [0161] The image obtained in Step S204 mentioned previously with reference to Fig. 3, i.e., the constituent pixels of the motion absence section pixel area 551 of the white balance adjusted image R, can be said to be pixels obtained by correctly subjecting the image 12 taken with flash and stored in the memory in Steps S103 and S104 shown in Fig. 2 to color conversion to reduce the difference in color between flash light and ambient light. On the other hand, as to the constituent pixels of the motion-presence section pixel area 550 of the white balance adjusted image R, since the motion of the subject exerts an influence on the pixel values, it can be said that there is a high possibility that color conversion is not correctly effected. [0162] Accordingly, as to the pixels of the motion-presence section pixel area 550, color conversion is directly applied to the flash image 12 to perform the processing of finding the final white balance adjusted image R. [0163] In Embodiment 1, the processing of Step Sill shown in Fig. 1, i.e., the pixel value correction processing of the motion section of the white balance adjusted image R, is executed as the interpolation processing using the radial basis function in accordance with the processing flow shown in Fig. 6. [0164] In the present embodiment, the pixel value correction processing of the motion section of the white balance adjusted image R is executed in a sequence different from Embodiment 1. The sequence of the pixel value correction processing of the motion section according to the present embodiment 2 will be described below with reference to the flowchart shown in Fig. 12. [0165] Fig. 12 is a flowchart showing the processing of Step Sill shown in Fig. 2, i.e., the details of the pixel value correction processing of the motion section of the white balance adjusted image R. The processing of each step will be described below. [0166] First, in Step S501, an image difference d is found on the basis of the following expression (Expression 21) using the white balance adjusted image R generated in accordance with Step S110 shown in Fig. 2, i.e., the processing flow of Fig. 3, and the image \z taken with flash and stored in the memory in Steps S103 and S104 shown in Fig. 2'- d(x, y) = R(x, y) - I2(x, y) ••• (Expression 21) [0167] In this expression, d(x, y), R(x, y) and I2(x, y) are vectors corresponding to the colors of the respective images at the pixel position (x, y). The above expression is executed as a vector calculation. [0168] In the expression (Expression 21), the image difference d between each pixel value in the white balance adjusted image R and in the flash image la is found, but instead of the difference between each pixel value in the white balance adjusted image R and in the flash image 12, the ratio of each pixel value of the white balance adjusted image R to the corresponding one of the flash image la, i.e., the image ratio d, may be calculated and adopted, as given by the following expression (Expression 22): d(x, y) = R(x, y) - (I2(x, y) + e) ••• (Expression 22) [0169] In the above expression (Expression 22), "e" is a vector having elements to which fully small fixed values are respectively set, for example, a floor value. The sign "/" in (Expression 22) represents the calculation of dividing the elements of a vector by the respective elements of a vector and setting the results as the respective elements of a vector, "e" is used for the purpose of preventing d(x, y) from becoming unable to be calculated when the elements of I2(x, y) become Os. [0170] If the image difference or the image ratio d(x, y) between the white balance adjusted image R and the flash image \2 is correctly found with respect to the entire image, it is possible to calculate the final white balance adjusted image R on the basis of the flash image la. Namely, it is possible to calculate R(x, y) = d(x, y) + I2(x, y) or R(x, y) = (I2(x, y) + e) * d(x, y). Incidentally, the sign "*" represents the calculation of multiplying the elements of a vector by the respective elements of a vector and setting the results as the respective elements of a vector. [0171] However, there is a possibility that the motion-presence section pixel area 550 appears as a motion image, so that if the image difference or the image ratio d(x, y) calculated from the above expression (Expression 21) or (Expression 22) is applied to the pixel area 550 without modification, an optimum color cannot be realized. [0172] For this reason, in the present embodiment, final pixel values relative to the motion-presence section pixel area 550, i.e., the final white balance adjusted image R, is calculated by performing smooth interpolation by using the image difference or the image ratio d(x, y) found from the motion absence section pixel area 551. Otherwise, the final white balance adjusted image R is calculated by performing smooth interpolation calculation using interpolation filtering processing with the image ratio d(x, y). The interpolation filtering processing can be realized by, for example, a method which, as will be described below, interpolates pixel values gradually from the periphery of the area 550 in accordance with the transition state of pixel values actually observed in the pixel area 550 or pixel values at the periphery of the motion-presence section pixel area 550, and further, performs low-pass filtering processing. Incidentally, the interpolation filtering processing of the present invention is not limited to this method. [0173] According to the present method, even as to the motion-presence section pixel area 550, it is possible to generate an image having a color similar to the final white balance adjusted image R while retaining the texture of the flash image lz of the motion-presence section pixel area 550, [0174] Fig. 13 is a view for describing the interpolation processing for finding final corrected pixel values relative to the motion-presence section pixel area 550 by using the above-mentioned (Expression 21), i.e., the expression for calculating the image difference d between each pixel value in the white balance adjusted image R and in the flash image \2. [0175] In Fig. 13, for ease of explanation, data are shown to be one-dimensionally arranged and the processing will be described on the basis of this illustration, but actually, the processing is executed on a two-dimensional plane. In each of Figs. 13A to 13C, the horizontal axis represents pixel position, while the vertical axis represents pixel value (luminance or R, G and B values). [0176] Fig. ISA shows the transition of pixel values of a white balance adjusted image R 560 and a flash image lz 561 in the motion absence section pixel area 551 and the motion-presence section pixel area 550. Fig. 13B shows a value 562 of the image difference d of a section in the motion absence section pixel area 551 which is close to the motion-presence section pixel area 550. Namely, Fig. 13B shows the value 562 of the above-mentioned (Expression 21), i.e., the image difference d between each pixel value in the white balance adjusted image R and in the flash image 12. [0178] The dashed line shown in the motion absence section pixel area 551 shown in Fig. 13B represents an estimated value 563 of the image difference d in the motion-presence section pixel area 550 which is obtained by smoothly interpolating the value 562 of the image difference d. A calculation processing method for the estimated value 563 of the image difference d will be described later. [0179] Fig. 13C is a view in which the result obtained by adding the estimated image difference d563 in the motion-presence section pixel area 550 shown in Fig. 13B to the value of the flash image \z 561 in the motion-presence section pixel area 550 shown in Fig. ISA is shown by a dotted line as a final white balance adjusted image R 564 in the motion-presence section pixel area 550. [0180] This white balance adjusted image R 564 shown by the dotted line in the motion-presence section pixel area 550 in Fig. 13C and the white balance adjusted image R 560 in the motion absence section pixel area 551 are outputted as the final white balance adjusted image R. The final output image becomes an image in which the white balance adjusted image R 564 shown by the dotted line in the motion-presence section pixel area 550 shown in Fig. 13C and the white balance adjusted image R 560 in the motion absence section pixel area 551 are smoothly connected. This output image is an image in which the texture of the flash image Iz 561 in the motion-presence section pixel area 550 is retained. [0181] It is to be noted that even if not the image difference d given by the above-mentioned expression (Expression 21) but the image ratio d using the above-mentioned expression (Expression 22) is applied as d(x, y), a correction method similar to the above-mentioned method can be executed. Namely, the image ratio d is used instead of the image difference d shown in Fig. 13B, and similarly to the dotted line shown in the motion absence section pixel area 551 shown in Fig. 13B, the values of the image ratios d are smoothly interpolated in the motion-presence section pixel area 550, thereby calculating estimated values of the image ratios d in the motion-presence section pixel area 550. After that, the image ratios d are multiplied by the pixels of the flash image lz 561 in the motion-presence section pixel area 550, whereby the white balance adjusted image R of the motion-presence section pixel area 550 can be calculated. [0182] Details of the interpolation method using the image difference or image ratio d between each pixel value in the white balance adjusted image R and in the flash image Iz will be described below. [0183] After the execution of Step S501 in the flow shown in Fig. 12, an initial value of the image difference or image ratio d(x, y) relative to each pixel of the motion-presence section pixel area 550 is found in Step S502 as preprocessing for interpolation. In the following description, reference is made to a method of finding the image difference d(x, y), but similar techniques can be applied to the case where the image ratio d(x, y) is to be found. [0184] First, a mask image M having pixels each expressed as either one of two values is prepared, and M(x, y) = 1 is set as to each pixel of the motion absence section pixel area 551, while M(x, y) = 0 is set as to each pixel of the motion-presence section pixel area 550. [0185] Each pixel of the image is checked to select a pixel (surrounded by 8 or 4 in the vicinity of pixels) which neighbors the pixels of M(x, y) = 1, i.e., the motion absence section pixel area 551, from among the pixels of M(x, y) = 0, i.e., the pixels of the motion-presence section pixel area 550. [0186] Then, the average of the image differences d(x, y) of the pixels of M(x, y) = 1 which neighbor the pixel of interest is calculated. This average value is set as an initial value of the image difference d(x, y) at the position of the pixel of interest. [0187] Specific processing will be described below with reference to Fig. 14. For example, in the case of an image having the motion-presence section pixel area 550 and the motion absence section pixel area 551 as shown in Fig. 14, the initial value of the image difference d(x, y) of a pixel 581 in the motion-presence section pixel area 550 which neighbors the motion absence section pixel area 551 is set as the average value of pixels 582, 583 and 584 of the motion absence section pixel area 551 which neighbor the pixel 581. [0188] Initial values relative to the pixels of the motion-presence section pixel area 550 which neighbor the motion absence section pixel area 551 are determined by a similar technique. Then, as to the pixel whose initial value has been newly set, the value of M(x, y) of the mask image is changed to a 1. Subsequently, after initial values have been set as to all pixels (each surrounded by 8 or 4 in the vicinity of pixels) that neighbor the pixels of M(x, y) = 1, from among the pixels of M(x, y) = 0, i.e., the pixels contained in the motion-presence section pixel area 550, M(x, y) = 1 is set as to the pixels whose initial values have been newly set. Namely, the initial values of the image differences d(x, y) of the motion-presence section pixel area 550 are sequentially determined from the peripheral section toward the central section of the motion-presence section pixel area 550 shown in Fig. 14. This processing is repeatedly performed until all the pixels are set to M(x, y) = 1. [0189] Through this processing, the initial values of the image differences d(x, y) of all the pixels contained in the motion-presence section pixel area 550 are determined. This processing is the processing of Step S502 of the flow shown in Fig. 12 [0190] After the initial values of the image differences d(x, y) in the motion-presence section pixel area 550 have heen found, the processing of Step S503 in the flow shown in Fig. 12 is executed. [0191] In Step S503, smoothing filtering is performed on the image differences d of only the pixels of the motion-presence section pixel area 550. The smoothing filtering may make use of, for example, moving average filtering which averages pixel values contained in the vicinity of a square of n x n. From this smoothing processing, it is possible to obtain the effect of smoothly connecting only the values of the image differences d of the motion-presence section pixel area 550 while fixing the values of the image differences d of the motion absence section pixel area 551. [0192] In this smoothing processing executed in Step S503, how each color component of the image difference d(x, y) has varied in each pixel through this smoothing filtering is checked, and the maximum value of the variation of each color component of the image difference d(x, y) (the pixel value variation maximum value) is stored in the memory. [0193] Then, in Step S504, it is determined whether the maximum value of the variation of each color component of the image difference d(x, y) (the pixel value variation maximum value) which has been stored in the smoothing processing of Step S502 is greater than a preset threshold. If the pixel value variation maximum value is smaller, the process proceeds to Step S506, and the smoothing processing is completed. [0194] If the maximum value of the variation of each color component of the image difference d(x, y) (the pixel value variation maximum value) which has been stored in the smoothing processing of Step S502 is greater than the preset threshold, it is determined that the image differences d of the motion-presence section pixel area 550 have not yet been smoothly interpolated, and the process proceeds to Step S505. [0195] In Step S505, it is determined whether the number of times by which the smoothing filtering has so far been performed on the pixels of the motion-presence section pixel area 550 of the image difference d is greater than a preset threshold, and if the number of times is greater, the process proceeds to Step S506. If the number of times is smaller, the process returns to Step S503, and the smoothing filtering is again performed to again execute the smoothing processing of the motion-presence section pixel area 550. [0196] In the case where the number of times by which the smoothing filtering has been performed on the pixels of the motion-presence section pixel area 550 of the image difference d is greater than the preset threshold, even if the smoothing filtering is repeatedly executed, no variations of the image differences d can be obtained. Accordingly, the predetermined maximum number of times of execution of the smoothing processing is determined as a threshold in advance, and if this threshold is reached, the smoothing processing is brought to an end at this point of time, and the process proceeds to the next step. [0197] In Step S506, the image differences d in the motion-presence section pixel area 550 which have been obtained in the above-mentioned smoothing processing are determined as the estimated image differences d 563. Namely, the image differences d are determined as the estimated image differences d 563 in the motion-presence section pixel area 550 which are shown in Fig. 13B. [0198] Then, in Step S507, the image differences d calculated from the above-mentioned processing and the flash image \z are used to generate the final white balance adjusted image R of the motion-presence section pixel area 550. [0199] Namely, the final white balance adjusted image R of the motion-presence section pixel area 550 shown in Fig. 13C is generated. The result obtained by adding the estimated image differences d 563 in the motion-presence section pixel area 550 shown in Fig. 13B to the values of the flash image \2 561 in the motion-presence section pixel area 550 shown in Fig. 13Ais set as the final white balance adjusted image R 564 in the motion-presence section pixel area 550. [0200] The above-mentioned processing example is an example which uses the white balance adjusted image R found in accordance with the above-mentioned expression (Expression 21) and the image difference d of each pixel value of the flash image \z. However, in the case where the white balance adjusted image R found in accordance with the above-mentioned expression (Expression 22) and the image difference d of each pixel value of the flash image 12 are used, in Step S507, the image ratio d in the motion-presence section pixel area 550 and the elements of each pixel of the flash image \% are multiplied to generate the final white balance adjusted image R. [0201] Through the above-mentioned processing, the final output image R can be generated. [0202] The construction of a motion-section corrected pixel value calculation section in the present embodiment will be described below with reference to Fig. 15. [0203] The motion-section corrected pixel value calculation section in the present embodiment is set to correspond to the motion-section corrected pixel value calculation section 310 shown in Fig. 8 mentioned previously in Embodiment 1 or the motion-section corrected pixel value calculation section 409 shown in Fig. 10. [0204] The construction shown in Fig. 15 will be described. A motion-section corrected pixel value calculation section 710 has an image difference (image ratio) calculation section 711, a motion-presence-pixel-area image difference (image ratio) d initial value calculation section 712, a smoothing processing section 713, and a motion-section final corrected pixel value calculation section 714. [0205] The image difference (image ratio) calculation section 711 receives the input of the motion-absence-section-pixel-area white balance adjusted image R 701 and the input of each image data of the flash image 12, and calculates the image differences d or the image ratios d in a motion absence section pixel area in accordance with the above-mentioned expression (Expression 21) or (Expression 22). This processing corresponds to "d" shown in Fig. 13A. [0206] The motion-presence-pixel-area image difference (image ratio) d initial value calculation section 712 sets the initial values of the image differences (image ratios) d in a motion-presence section pixel area. As described previously with reference to Fig. 14, this processing first sets as the initial values the average value of the image differences (image ratios) d of in the vicinity of pixels of the motion absence pixel area, which pixels are positioned in the section of the motion-presence pixel area which neighbors the motion absence pixel area, and then executes the processing of sequentially determining the initial values of the image differences (image ratios) d toward the inside of the motion-presence pixel area. [0207] The smoothing processing section 713 executes smoothing processing using, for example, smoothing filtering, on the basis of the initial values of the motion-presence-pixel-area image difference (image ratio) d which have been set in the motion-presence-pixel-area image difference (image ratio) d initial value calculation section 712, and determines the estimated values d of the image differences (image ratios) in the motion-presence pixel area. Namely, the estimated image difference d563 in the motion-presence section pixel area 550 shown in Fig. 13B is determined. [0208] The motion-section final corrected pixel value calculation section 714 receives the inputs of the estimated values d of the image differences (image ratios) smoothed in the smoothing processing section 713 and the flash image 12, and generates the final white balance adjusted image R of the motion-presence section pixel area 550. Namely, the final white balance adjusted image R of the motion-presence section pixel area 550 shown in Fig. 13C is generated. [0209] The result obtained by adding the estimated image differences d 563 in the motion-presence section pixel area 550 shown in Fig. 13B to the values of the flash image 12 561 in the motion-presence section pixel area 550 shown in Fig. 13A is set and outputted as the final white balance adjusted image R 564 of the motion-presence section pixel area 550. [0210] In the case where the white balance adjusted image R found in accordance with the expression (Expression 22) and the image difference d of each pixel value of the flash image \z are used, in Step S507, the image ratio d in the motion-presence section pixel area 550 and the elements of each pixel of the flash image la are multiplied to generate and output the final white balance adjusted image R. [0211] As described above, according to the pixel value correction processing of the present embodiment, it is possible to perform appropriate color conversion on the motion-presence section pixel area by means of simple processing using smoothing filtering. [0212] In the present embodiment, reference has bee made to an example in which white balance adjustment processing is performed. However, the above-de scribed technique can be used to solve not only white balance adjustment processing but also general data processing problems. [0213] Namely, the processing of the present invention is useful in the case where, as shown in Fig. 13, certain data are defined in a certain area (the area 551 of Fig. 13) but data are to be defined in the other area (the area 550 of Fig 13). [0214] It is assumed that an area indicated by the area 550 shown in Fig. 13 are given and reference data 561 are given in a data area 551 adjacent to the area. At this time, interpolation data (the interpolation data 564 of Fig. 13) having the same characteristics (the texture of an image) as the reference data 561 can be generated. [0215] This data processing can be applied to not only the white balance adjustment processing of image data in the above-mentioned embodiment but also general data processing. Namely, the present embodiment is not limited to the problem of white balance adjustment. [Embodiment 3] [0216] Embodiment 3 of the present invention will be described below with reference to a construction which adopts a technique different from those of Embodiments 1 and 2 as the processing of Step Sill in the flow of Fig. 2, i.e., a pixel value correction processing method for a motion section, in an image processing method and an image processing device both of which, similarly to the above-mentioned method and apparatus, execute optimum white balance adjustment processing for image capture under environments in which ambient light and flash light are mixed. [0217] In Embodiment 1, reference has been made to a construction which detects a motion section of a subject from the difference between two no flash-assisted images, and as to pixels contained in the motion section, performs interpolation processing on the ratio of a reflected flash light component to a reflected ambient light component, from data in pixels other than the motion section, by using a radial basis function. In Embodiment 2, reference has been made to a construction which performs pixel value correction on a motion section by processing simplified by a smoothing filter. [0218] Embodiment 3 provides a construction example in which sets a filter whose weight is dynamically determined according to image data of the flash image 12 and performs pixel value correction on motion-presence pixels by using the set filter. [0219] An image processing device of Embodiment 3 has a construction similar to the construction described previously with reference to Fig. 1, and filter setting processing according to image data of the flash image la and pixel value correction processing using the filter in Embodiment 3 are executed in the digital signal processing section (DSP) 106. Details of image processing according to Embodiment 3 will be described below. [0220] The image processing is realized in such a manner that the arithmetic unit sequentially executes operations written in predetermined program codes, on a stream of input image signals in the inside of the DSP 106. In the following, the order in which individual processing steps of the program are executed will be described with reference to a flowchart. However, the present invention may be constructed not in the form of a program which will be described in the present embodiment, but by incorporating a hardware circuit which realizes processing equivalent to functions which will be described below. [0221] The basic sequence of the procedure of white balance (WB) correction processing in the present embodiment is the processing according to the flowchart shown in Fig. 2, similarly to Embodiments 1 and 2 mentioned previously. Namely, three images are continuously captured in the order of a flash image, a no flash-assisted image and a flash image, and white balance adjustment is performed on the basis of these images. [0222] In the present embodiment, the pixel value correction processing of the motion section in Step Sill shown in Fig. 2 can be executed far more efficiently with far higher accuracy. The case where the white balance adjustment processing based on the above-mentioned plurality of images is performed in Step S110 shown in Fig. 2 means the case where it is determined that an image shake due to a motion of a subject itself has occurred and the image shake is correctable. As to the image area of the image shake due to the motion of the subject itself, i.e., the motion section area, in the white balance adjusted image R generated in Step S110, pixel value correction processing is executed in Step Sill. Namely, exceptional processing is performed on the pixel values of the motion section detected in Step S107, thereby modifying the white balance adjusted image R. As modification processing, there is, for example, a method of inputting the pixel values of the flash image 12 corresponding to the section in which the motion is detected, referring to the pixel values of a section in which a motion is absent, in the white balance adjusted image R, determining the pixel values of the section in which the motion is detected, and synthesizing a final image. [0223] Details of pixel value correction processing according to the present embodiment will be described below. [0224] As described in the previous embodiments, data which is acquired as the result of motion detection processing executed in Step S107 of Fig. 2 is data corresponding to the schematic view shown in Fig. 16. [0225] In Fig. 16, the section determined to be moving in Step S107 of the flow shown in Fig. 2 is the motion-presence section pixel area 550, while the section determined to be not moving is the motion absence section pixel area 551. [0226] The image obtained in Step S204 mentioned previously with reference to Fig. 3, i.e., the constituent pixels of the motion absence section pixel area 551 of the white balance adjusted image R, can be said to be pixels obtained by correctly subjecting the image 12 taken with flash and stored in the memory in Steps S103 and S104 shown in Fig. 2 to color conversion to reduce the difference in color between flash light and ambient light. On the other hand, as to the constituent pixels of the motion-presence section pixel area 550 of the white balance adjusted image R, since the motion of the subject exerts an influence on the pixel values, it can be said that there is a high possibility that color conversion is not correctly effected. [0227] Accordingly, as to the pixels of the motion-presence section pixel area 550, color conversion is directly applied to the flash image \z to perform the processing of finding the final white balance adjusted image R. [0228] In Embodiment 1, the processing of Step Sill shown in Fig. 1, i.e., the pixel value correction processing of the motion section of the white balance adjusted image R, is executed as the interpolation processing using the radial basis function in accordance with the processing flow shown in Fig. 6. In Embodiment 2, correction processing using smoothing filtering is executed in accordance with the processing flow shown in Fig. 12. [0229] In the present embodiment, the pixel value correction processing of the motion section of the white balance adjusted image R is executed in a sequence different from Embodiments 1 and 2. The sequence of the pixel value correction processing of the motion section according to the present embodiment 3 will be described below with reference to the flowchart shown in Fig. 16. [0230] Fig. 16 is a flowchart showing the processing of Step Sill shown in Fig. 2, i.e., a detailed sequence in which the pixel value correction processing of the motion section of the white balance adjusted image R is executed in accordance with the present embodiment 3. The processing of each step will be described below. [0231] First, in Step S601, the image difference d is found on the basis of the following expression (Expression 31) using the white balance adjusted image R generated in accordance with Step S110 shown in Fig. 2, i.e., the processing flow of Fig. 3, and the image 1% taken with flash and stored in the memory in Steps S103 and S104 shown in Fig. 2' d(x, y) = R(x, y) - I2(x, y) ••• (Expression 31) [0232] In this expression, d(x, y), R(x, y) and I2(x, y) are vectors corresponding to the colors of the respective images at the pixel position (x, y). The above expression is executed as a vector calculation. [0233] In the expression (Expression 31), the image difference d between each pixel value in the white balance adjusted image R and in the flash image lz is found, but instead of the difference between each pixel value in the white balance adjusted image R and in the flash image lz, the ratio of each pixel value of the white balance adjusted image R to the corresponding one of the flash image lz, i.e., the image ratio d, may be calculated and adopted, as given by the following expression (Expression 32): d(x, y) = R(x, y) - (I2(x, y) + e) ••• (Expression 32) [0234] In the above expression (Expression 32), "e" is a vector having elements to which fully small fixed values are respectively set, for example, a floor value. The sign "/" in (Expression 32) represents the calculation of dividing the elements of a vector by the respective elements of a vector and setting the results as the respective elements of a vector. "e" is used for the purpose of preventing d(x, y) from becoming unable to be calculated when the elements of ^(x, y) become Os. [0235] If the image difference or the image ratio d(x, y) between the white balance adjusted image R and the flash image lz is correctly found with respect to the entire image, it is possible to calculate the final white balance adjusted image R on the basis of the flash image \z. Namely, it is possible to calculate R(x, y) = d(x, y) + I2(x, y) or R(x, y) = (I2(x, y) + e) * d(x, y). Incidentally, the sign "*" represents the calculation of multiplying the elements of a vector by the respective elements of a vector and setting the results as the respective elements of a vector. [0236] However, there is a possibility that the motion-presence section pixel area 550 appears as a motion image, so that if the image difference or the image ratio d(x, y) calculated from the above expression (Expression 31) or (Expression 32) is applied to the pixel area 550 without modification, an optimum color cannot be realized. [0237] For this reason, in the present embodiment, final pixel values relative to the motion-presence section pixel area 550, i.e., the final white balance adjusted image R, is calculated by performing interpolation using the image difference or the image ratio d(x, y) found from the motion absence section pixel area 551. The image difference or the image ratio d(x, y) is used to perform pixel value correction of the motion-presence section 550. [0238] According to the present method, even as to the motion-presence section pixel area 550, it is possible to generate an image having a color similar to the final white balance adjusted image R while retaining the texture of the flash image 12 of the motion-presence section pixel area 550. [0239] Fig. 18 is a view for describing the interpolation processing for finding final corrected pixel values relative to the motion-presence section pixel area 550 by using the above-mentioned (Expression 31), i.e., the expression for calculating the image difference d between each pixel value in the white balance adjusted image R and in the flash image \z- [0240] In Fig. 18, for ease of explanation, data are shown to be one-dimensionally arranged and the processing will be described on the basis of this illustration, but actually, the processing is executed on a two-dimensional plane. In each of Figs. ISA to 18C, the horizontal axis represents pixel position, while the vertical axis represents pixel value (luminance or R, G and B values). [0241] Fig. ISA shows the transition of pixel values of a white balance adjusted image R 860 and a flash image la 861 in the motion absence section pixel area 551 and the motion-presence section pixel area 550. [0242] Fig. 18B shows a value 862 of the image difference d of a section in the motion absence section pixel area 551 which is close to the motion-presence section pixel area 550. Namely, Fig. 18B shows the value 862 of the image difference d between each pixel value in the white balance adjusted image R and in the flash image lz, which value 862 is calculated by using the above-mentioned (Expression 31). [0243] The dashed line shown in the motion absence section pixel area 550 shown in Fig. 18B represents a value estimated on the basis of an image difference d value 862a and an image difference d value 862b of the motion absence section pixel area 551 close to the motion-presence section pixel area 550, i.e., an estimated value 863 of the image difference d in the motion-presence section pixel area 550 which is obtained by smoothly interpolating the values 862a and 862b of the image difference d of the motion absence section pixel area 551. [0244] The estimated value 863 of the image difference d in the motion-presence section pixel area 550 is an estimated value determined by executing the following steps (l) and (2): (1) a step of setting an initial image difference d,' and (2) a step of correcting the initial image difference d by means of a filter whose weight is dynamically determined according to image data of the flash image 12. A calculation processing method for the estimated value 863 of the image difference d in the motion-presence section pixel area 550 will be described later. [0245] Fig. 18C is a view in which the result obtained by adding the estimated image difference d 863 in the motion-presence section pixel area 550 shown in Fig. 18B to the value of the flash image la 861 in the motion-presence section pixel area 550 shown in Fig. ISA is shown by a dotted line as a final white balance adjusted image R 864 in the motion-presence section pixel area 550. [0246] This white balance adjusted image R 864 shown by the dotted line in the motion-presence section pixel area 550 in Fig. 18C and the white balance adjusted image R 860 in the motion absence section pixel area 551 are outputted as the final white balance adjusted image R. The final output image becomes an image in which the white balance adjusted image R 864 shown by the dotted line in the motion-presence section pixel area 550 shown in Fig. 18C and the white balance adjusted image R 860 in the motion absence section pixel area 551 are smoothly connected. This output image is an image in which the texture of the flash image \2 861 in the motion-presence section pixel area 550 is retained. [0247] It is to be noted that even if not the image difference d given by the above-mentioned expression (Expression 31) but the image ratio d using the above-mentioned expression (Expression 32) is applied as d(x, y), a correction method similar to the above-mentioned method can be executed. Namely, the image ratio d is used instead of the image difference d shown in Fig. 18B, and similarly to the dotted line shown in the motion absence section pixel area 551 shown in Fig. 18B, the values of the image ratios d are smoothly interpolated in the motion-presence section pixel area 550, thereby calculating estimated values of the image ratios d in the motion-presence section pixel area 550. After that, the image ratios d are multiplied by the pixels of the flash image \2 861 in the motion-presence section pixel area 550, i.e., not addition processing but multiplication processing is performed, whereby the white balance adjusted image R of the motion-presence section pixel area 550 can be calculated. [0248] Details of the interpolation method using the image difference or image ratio d between each pixel value in the white balance adjusted image R and in the flash image 12 will be described below. [0249] As described above, the estimated value 863 of the image difference d in the motion-presence section pixel area 550 is determined by executing the following steps (l) and (2): (1) the step of setting the initial image difference d; and (2) the step of correcting the initial image difference d by means of the filter whose weight is dynamically determined according to image data of the flash image 12. First, the processing of setting the initial value of the image difference d in the motion-presence section pixel area 550 will be described below. [0250] After the execution of Step S601 in the flow shown in Fig. 17, an initial value of the image difference or image ratio d(x, y) relative to each pixel of the motion-presence section pixel area 550 is found in Step S602. In the following description, reference is made to a method of finding the image difference d(x, y), but similar techniques can be applied to the case where the image ratio d(x, y) is to be found. [0251] First, a mask image M having pixels each expressed as either one of two values is prepared, and M(x, y) = 1 is set as to each pixel (x, y) of the motion absence section pixel area 551, while M(x, y) = 0 is set as to each pixel (x, y) of the motion-presence section pixel area 550. [0252] Each pixel of the image is checked to select a pixel (surrounded by 8 or 4 in the vicinity of pixels) which neighbors the pixels of M(x, y) = 1, i.e., the motion absence section pixel area 551, from among the pixels of M(x, y) = 0, i.e., the pixels of the motion-presence section pixel area 550. [0253] Then, the average of the image differences d(x, y) of the pixels of M(x, y) = 1 which neighbor the pixel of interest is calculated. This average value is set as an initial value of the image difference d(x, y) at the position (x, y) of the pixel of interest. [0254] Specific processing will be described below with reference to Fig. 16. For example, in the case of an image having the motion-presence section pixel area 550 and the motion absence section pixel area 551 as shown in Fig. 16, the initial value of the image difference d(x, y) of a pixel 821 in the motion-presence section pixel area 550 which neighbors the motion absence section pixel area 551 is set as the average value of pixels 822, 823 and 824 of the motion absence section pixel area 551 which neighbor the pixel 821. [0255] Initial values relative to the pixels of the motion-presence section pixel area 550 which neighbor the motion absence section pixel area 551 are determined by a similar technique. Then, as to the pixel whose initial value has been newly set, the value of M(x, y) of the mask image is changed to a 1. Subsequently, after initial values have been set as to all pixels (each surrounded by 8 or 4 in the vicinity of pixels) that neighbor the pixels of M(x, y) = 1, from among the pixels of M(x, y) = 0, i.e., the pixels contained in the motion-presence section pixel area 550, M(x, y) = 1 is set as to the pixels whose initial values have been newly set. Namely, the initial values of the image differences d(x, y) of the motion-presence section pixel area 550 are sequentially determined from the peripheral section toward the central section of the motion-presence section pixel area 550 shown in Fig. 16. This processing is repeatedly performed until all the pixels are set to M(x, y) = 1. [0256] Through this processing, the initial values of the image differences d(x, y) of all the pixels contained in the motion-presence section pixel area 550 are determined. This processing is the processing of Step S602 of the flow shown in Fig. 17 [0257] After the initial values of the image differences d(x, y) in the motion-presence section pixel area 550 have been found through the above-mentioned processing, the processing of Steps S603 to S606 in the flow shown in Fig. 16 is executed. This processing is the above-mentioned processing: (2) the step of correcting the initial image difference d by means of the filter whose weight is dynamically determined according to image data of the flash image \2. [0258] In Step S603, corrected values d' of the initial image differences d relative to only the pixels of the motion-presence section pixel area 550 from among image differences d are calculated by filtering processing using a filter. [0259] In the present embodiment, the filter applied to the filtering processing for calculating the corrected values d' of the initial image differences d is not the smoothing filter applied to Embodiment 2 but a filter whose weight is dynamically determined according to the image data of the flash image \z. [0260] For example, the following expression is used as a calculation expression for finding an updated pixel value of the image difference d(x, y) at each pixel position (x, y) in the motion-presence section pixel area 550 by means of filtering processing corresponding to one pixel value correction process • • • (Expression 33) [0261] In the above expression (Expression 33), d(x, y, ch) and I2(y, y, ch) respectively represent a difference image d of each channel [ch] at the pixel position (x, y) and the pixel value of the flash image 12. The term "channel" means each channel for a color image, specifically, each of red, green and blue (RGB) channels. In addition, d'(x, y, ch) represent a new pixel value, i.e., an updated pixel value, of the difference image d of each channel [ch] at the pixel position (x, y). [0262] In addition, i and j of the above expression (Expression 33) represent the positions of reference pixels which are used for calculating the updated value d' of the pixel difference value d at the pixel position (x, y). The ranges of the values of i and j are x — k threshold" is not satisfied, the process proceeds to Step S606, and executes the processing of determining an image difference (image ratio) d estimated value relative to the motion-presence section pixel area 550. [0272] In Step S604, if it is determined that "maximum value of pixel value variation amount > threshold" is satisfied, it is determined that the image difference d(x, y) at the pixel position (x, y) in the motion-presence section pixel area 550 has not yet been smoothly corrected, and the process proceeds to Step S605. [0273] In Step S605, it is determined whether the number of times by which the pixel value updating processing (filtering processing) has been executed in Step S603 by using the above-mentioned expression (Expression 33) is greater than a preset threshold number. If "number of times of execution of pixel value updating processing (filtering processing) > threshold number" is satisfied, the process proceeds to Step S606. [0274] If "number of times of execution of pixel value updating processing (filtering processing) > threshold number" is not satisfied, the process returns to Step S603 and repeatedly executes the processing (filtering processing) of correcting the image difference d(x, y) at the pixel position (x, y) in the motion-presence section pixel area 550 in accordance with the above-mentioned expression (Expression 33). [0275] If it is determined in Step S604 that "maximum value of pixel value variation amount > threshold" is not satisfied, or if it is determined in Step S605 that "number of times of execution of pixel value updating processing (filtering processing) > threshold number" is satisfied, the processing (filtering processing) of correcting the image difference d(x, y) at the pixel position (x, y) in the motion-presence section pixel area 550 in accordance with the above-mentioned expression (Expression 33) is completed, and the process proceeds to Step S606 and executes the processing of determining an estimated value of the image difference (image ratio) d relative to the motion-presence section pixel area 550. [0276] In Step S606, the image difference d in the motion-presence section pixel area 550 which has been found in the above-mentioned filtering processing is determined as an estimated image difference d. The image difference d(x, y) at the pixel position (x, y) in the motion-presence section pixel area 550 which has been calculated by this processing corresponds to the estimated image difference d 863 in the motion-presence section pixel area 550 shown in Fig. 18B. [0277] Then, in Step S607, the processing of adding together the values (the estimated image differences d 863 shown in Fig. 18B) of the image differences d(x, y) at the pixel positions (x, y) in the motion-presence section pixel area 550 which value have been calculated by the filtering processing using the above-mentioned expression (Expression 33) and the flash image \% (the flash captured image \2 861 shown in Fig. 18A) is executed to generate the white balance adjusted image R in the motion-presence section pixel area 550. Namely, from R = Iz + d, the pixel values of the white balance adjusted image R in the motion-presence section pixel area 550 are calculated. [0278] This result is the white balance adjusted image R 864 shown by a dashed line in the motion-presence section pixel area 550 of Fig. 18C. A final corrected image, i.e., the white balance adjusted image R, is an image in which the white balance adjusted image R 860 in the motion absence section pixel area 551 and the white balance adjusted image R 864 shown by the dashed line in the motion-presence section pixel area 550 are connected. [0279] As shown in Fig. 18C, the final white balance adjusted image R is an image in which the white balance adjusted image R 864 shown by the dashed line in the motion-presence section pixel area 550 and the white balance adjusted image R 860 in the motion absence section pixel area 551 are smoothly connected. Further, this output image is an image in which the texture of the flash image 12 861 in the motion-presence section pixel area 550 is retained. [0280] It is to be noted that even if not the image difference d given by the above-mentioned expression (Expression 31) but the image ratio d using the above-mentioned expression (Expression 32) is used, a correction method similar to the above-mentioned method can be executed. In this case, in Step S607, the processing of multiplying the values of the image ratios d(x, y) at the pixel positions (x, y) in the motion-presence section pixel area 550 which value are calculated by filtering processing using the image ratios d by the flash image \2 is executed to generate the white balance adjusted image R in the motion-presence section pixel area 550. Namely, from R = I2 x d, the pixel values of the white balance adjusted image R in the motion-presence section pixel area 550 are calculated. [0281] Namely, the image ratio d is used instead of the image difference d shown in Fig. 18B, and similarly to the dashed line shown in the motion absence section pixel area 551 shown in Fig. 18B, the values of the image ratios d are smoothly interpolated in the motion-presence section pixel area 550, thereby calculating estimated values of the image ratios d in the motion-presence section pixel area 550. After that, the image ratios d are multiplied by the pixels of the flash image \z 561 in the motion-presence section pixel area 550, whereby the white balance adjusted image R of the motion-presence section pixel area 550 can be calculated. [0282] The construction of a motion-section corrected pixel value calculation section in the present embodiment will be described below with reference to Fig. 19. [0283] The motion-section corrected pixel value calculation section in the present embodiment is set to correspond to the motion-section corrected pixel value calculation section 310 shown in Fig. 8 mentioned previously in Embodiment 1 or the motion-section corrected pixel value calculation section 409 shown in Fig. 10. [0284] The construction shown in Fig. 19 will be described. A motion-section corrected pixel value calculation section 910 has an image difference (image ratio) calculation section 911, a motion-presence-pixel-area image difference (image ratio) d initial value calculation section 912, a filtering processing section 913, and a motion-section final corrected pixel value calculation section 914. [0285] The image difference (image ratio) calculation section 911 receives the input of a motion-absence-section-pixel-area white balance adjusted image R 901 and the input of each image data of the flash image lz which is stored in a flash image 12 storing frame memory 902, and calculates the image differences d or the image ratios d in a motion absence section pixel area in accordance with the above-mentioned expression (Expression 31) or (Expression 32). This processing corresponds to the image difference d 862 in the motion absence section pixel area 551 which is shown in Fig. 18B. [0286] The motion-presence-pixel-area image difference (image ratio) d initial value calculation section 912 sets the initial values of the image differences (image ratios) d in a motion-presence section pixel area. As described previously with reference to Fig. 16, this processing first sets as the initial values the average value of the image differences (image ratios) d of in the vicinity of pixels of the motion absence pixel area, which pixels are positioned in the section of the motion-presence pixel area which neighbors the motion absence pixel area, and then executes the processing of sequentially determining the initial values of the image differences (image ratios) d toward the inside of the motion-presence pixel area. [0287] The filtering processing section 913 executes the filtering processing of updating the initial values of the motion-presence-pixel-area image differences (image ratios) d which have been set in the motion-presence-pixel-area image difference (image ratio) d initial value calculation section 912, in accordance with the above-mentioned expression (Expression 33). This filtering processing is filtering processing using a filter generated on the basis of the flash image Iz, i.e., filtering processing which performs updating in accordance with the above-mentioned expression (Expression 33). [0288] The filtering processing section 913, as described with reference to the flow shown in Fig. 17, brings the filtering processing using the expression (Expression 33) to an end on the condition that the maximum variation of the pixel values by one filtering processing is smaller than a preset threshold or the number of times of filtering processing is greater than a preset threshold number, and determines the estimated values d of the image differences (the image ratios) in the motion-presence pixel area. Namely, the estimated image difference d 863 in the motion-presence section pixel area 550 shown in Fig. 18B is determined. [0289] The motion-section final corrected pixel value calculation section 914 receives the inputs of the estimated values d of the image differences (image ratios) subjected to the filtering processing in the filtering processing section 913 and the flash image \z from the image \z storing frame memory 902, and generates the final white balance adjusted image R of the motion-presence section pixel area 550. Namely, the final white balance adjusted image R of the motion-presence section pixel area 550 shown in Fig. 18C is generated. [0290] The result obtained by adding the estimated image differences d 863 in the motion-presence section pixel area 550 shown in Fig. 18B to the values of the flash captured image lz 861 in the motion-presence section pixel area 550 shown in Fig. 18A is set and outputted as the final white balance adjusted image R 864 of the motion-presence section pixel area 550 shown in Fig. 18C. [0291] In the case where the white balance adjusted image R found in accordance with the expression (Expression 32) and the image difference d of each pixel value of the flash captured image lz are used, the image ratio d in the motion-presence section pixel area 550 and the elements of each pixel of the flash captured image 12 are multiplied to generate and output the final white balance adjusted image R. [0292] As described above, according to the pixel value correction processing of the present embodiment, it is possible to perform appropriate color conversion on the motion-presence section pixel area by means of processing using a filter, whereby the corrected pixel values in the motion-presence section pixel area become pixel values which retain the texture of the flash captured image 12 and a far more natural image can be generated. [0293] When processing using the smoothing filter according to the second embodiment is executed, it is difficult to perform pixel correction corresponding to the image differences or the image ratios of an original image, and the fuzziness of edge sections and the blur of colors may occur according to the kind of image. However, the present embodiment adopts a construction which sets, in accordance with Expression 33, a pixel value conversion expression using a coefficient determined to take into account the pixel values of a second image, i.e., the flash image 1%, and performs the filtering processing. Accordingly, pixel value correction which reflects the texture of the flash image lz is performed and the fuzziness of edge sections, the blur of colors and the like are solved even in the motion-presence area, whereby it is possible to generate an image which reflects the texture of the flash image 12. [0294] In the present embodiment, reference has been made to an example in which white balance adjustment processing is executed, but the above-mentioned technique according to the present invention can be applied to not only white balance adjustment processing but also the case where a section of an image is to be modified to become similar to the feature of another image, in, for example, image processing which modifies the pixel values of a section of a certain image. Namely, a pixel value conversion expression using a coefficient determined by using the feature of a second image is set in accordance with Expression 33, and filtering processing is performed to modify the pixel values of a section of a first image, whereby it is possible to generate an image which has the feature of the second image while retaining the color of the first image. [Embodiment 4] [0295] Each of the above-mentioned Embodiments 1 to 3 has a construction which continuously captures three images, i.e., a no flash-assisted image, a flash image and a no flash-assisted image, and generates one output image. In each of the above-mentioned embodiments, it is assumed that these three images have the same resolution. However, since it is expected that processing is executed on an image having a resolution of, for example, several million pixels, image data for three images need to be stored in order to execute this processing, and further, a memory having a large capacity capable of storing various processing image data becomes necessary. Accordingly, with the increase of the number of pixels, the amount of calculations to be executed becomes huge. [0296] In the apparatus construction shown in Fig. 1, if, for example, a CCD (Charge Coupled Device) is used as the solid state image pickup element 103, it is difficult to continuously capture an image of several million pixels at high speed. General digital cameras have the function of, reading an image of low resolution from the solid state image pickup element 103 at high speed and successively displaying the image on a camera-attached display, at intervals of, for example, 1/30 second during image capture, [0297] The present embodiment provides a construction example capable of solving the difference in color temperature between ambient light and flash light efficiently at high speed by using an image of low resolution captured in this manner. [0298] A block diagram showing an image processing device and an image pickup apparatus according to the present embodiment is similar to Fig. 1, and the description of the block diagram is similar to the previous description and is omitted herein. [0299] Fig. 20 is a flowchart for describing the processing procedure of the present embodiment. Each step of the flowchart shown in Fig. 20 will be described below. [0300] First, in Step S701, image capture is performed without flash by using an aperture and a shutter speed which are set in advance, and in Step S702, a low resolution image IIL based on the image taken in Step S701 is stored in the memory. The low resolution image IIL is an image having a resolution lower than the resolution of the original captured linage of the solid state image pickup element 103 (refer to Fig. l). Various methods for generating the low resolution image IIL from a picked-up high resolution image are available. For example, there are a method of simply sub sampling pixels and a method of finding the average of a plurality of in the vicinity of pixels and forming one new pixel. As mentioned above, general digital cameras have the function of displaying an image picked up by the solid state image pickup element 103 during image capture on a low resolution display at a speed of, for example, 30 frames per second, and the method used at this time can be applied without modification. [0301] When the no flash-assisted low resolution image IIL based on the image taken in Step S701 is to be stored in the memory, the storing processing can be executed as low resolution image data storing processing, so that the amount of data to be stored in the memory is small and the processing time required to store the data in the memory is reduced. Accordingly, the process can proceed to the next step at high speed. [0302] Then, in Step S703, an image is taken with flash, and in Step S704, two images, i.e., a flash-assisted high resolution image I2H based on the image taken in Step S703 and a flash-assisted low resolution image IZL, are stored in the memory. The flash-assisted low resolution image I2L is a low resolution image having the same resolution as the no flash-assisted low resolution image IIL, and is found by the same method as the above-mentioned one. The flash-assisted high resolution image I2H is an image which has the desired resolution to be finally outputted and is higher in resolution than the no flash-assisted low resolution image IIL and the flash-assisted low resolution image IZL. [0303] Then, in Step S705, an image is again taken without flash, and in Step S706, a no flash-assisted low resolution image ISL based on the image taken in Step S705 is stored in the memory. The no flash-assisted low resolution image ISL is an image having the same resolution as the no flash-assisted low resolution image IIL and the flash-assisted low resolution image IZL, and is found by the same method as the above-mentioned one. [0304] The next processing of Steps S707 to S713 is completely the same as the processing of Steps S107 to S113 of the flowchart of Fig. 2 mentioned in the previous embodiment, and the description thereof is omitted. [0305] However, target image data to be processed in the processing of Steps S707 to S713 are low resolution images, i.e., the no flash-assisted low resolution image IIL, the flash-assisted low resolution image IZL and the no flash-assisted low resolution image ISL. The white balance adjusted image R is generated on the basis of these low resolution images. As to a motion section, the white balance adjusted image R is generated which has corrected pixel values calculated by the processing using the Radial Basis Function (radial basis function) mentioned previously in Embodiment 1, the processing based on the smoothing processing using the smoothing filter mentioned previously in Embodiment 2, the processing using the filter for which a weight based on a flash captured image is set as mentioned in Embodiment 3, or the like. [0306] Finally, in Step S714, a high resolution final image RH is generated on the basis of the white balance adjusted image R generated on the basis of the low resolution images, the flash-assisted low resolution image I£L and the flash-assisted high resolution image I2H. [0307] First, pixel values at pixel positions l2L(x', y') and R(x', y') relative to the respective positions (x', y') of the flash-assisted low resolution image I2L and the white balance adjusted image R are found with respect to each pixel l2H(x, y) of the flash-assisted high resolution image I2H. [0308] Incidentally, x' and y' are not necessarily integers. As the method of finding pixel values at (x', y'), generally widely used image interpolation techniques such as nearest in the vicinity of methods, bilinear methods and bicubic methods are employed. [0309] Then, the ratio of the pixel value of the pixel R(x', y') of the white balance adjusted image R generated on the basis of the low resolution images to the pixel value of the corresponding pixel l2L(x', y') of the flash-assisted low resolution image I2L is found. [0310] With respect to this pixel value ratio based on the low resolution image data' R(x', y'):I2L(x', y'), a pixel value ratio based on the high resolution image data is set: RH(X, y):l2H(x, y). Since each pixel value of the flash-assisted high resolution image I2H is known, the pixel l2H(x, y) and the pixel value ratio based on the low resolution image data: R(x', y'):l2L(x', y') are multiplied to calculate the pixel RH(X, y) of a high resolution final image RH. [0311] This calculation is performed on all the pixels of the flash-assisted high resolution image I2H to generate the final high resolution white balance adjusted image RH. Namely, conversion information for converting the flash-assisted low resolution image I2L to the white balance adjusted image R is acquired from the low resolution images, and on the basis of this conversion information, the processing of converting each pixel value of the high resolution image, i.e., the flash-assisted high resolution image I2H, is executed to generate the high resolution white balance adjusted image RH. [0312] According to this processing method, the target image data processed in the processing of Steps S707 to S713 are the low resolution images, i.e., the no flash-assisted low resolution image IIL, the flash-assisted low resolution image IZL and the no flash-assisted low resolution image ISL, and Steps S707 to S713 are executed as the processing of generating the white balance adjusted image R based on the low resolution images. Accordingly, the number of pixels is reduced and the amounts of calculations are also reduced, whereby high speed processing becomes possible. As to correction processing for a motion section as well, it is possible to achieve high speed processing in any processing such as the processing using the Radial Basis Function (radial basis function) mentioned previously in Embodiment 1, or the processing based on the smoothing processing using the smoothing filter mentioned previously in Embodiment 2, or the processing using the filter in which a weight based on a flash captured image is set as mentioned in Embodiment 3. [0313] In addition, since only the low resolution image data of each of the two no flash-assisted captured image data needs to be stored in the memory, the present embodiment can also be applied to apparatuses and models having small memory capacities. [0314] Accordingly, the processing of the present embodiment can also be executed in models and apparatuses which use processors having comparatively low processing capabilities and have small memory capacities, and enable these models and apparatuses to output finally obtained image data as high resolution white balance adjusted image data. [0315] The present invention has been described above in detail with reference to particular embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions without departing from the spirit of the present invention. Namely, the foregoing is illustrative of the present invention and is not to be construed as limiting the present invention. The spirit of the present invention is to be understood from the appended claims. Incidentally, in each of the above-mentioned embodiments, the term "flash" has been used to describe an illuminating device which emits light in the case of a dark subject, but this device is also called a strobe. Accordingly, the present invention is not limited to flashes and can be generally applied to illuminating devices which emit light in the case of dark subjects. [0316] The sequence of processing mentioned hereinabove can be executed by hardware, software or a complex construction of both hardware and software. In the case where processing using software is to be executed, a program which records a processing sequence may be installed in an executable state in a memory of a computer incorporated in dedicated hardware, or a program may be installed in an executable state in a general-purpose computer capable of executing various kinds of processing. [0317] For example, the program can be recorded in advance on hard disks or ROMs (Read Only Memories) which serve as recording media. Otherwise, the program can be temporarily or permanently stored (recorded) in removable recording media such as flexible discs, CD-ROMs (Compact Disc Read Only Memories), MO (Magneto-optical) discs, DVDs (Digital Versatile Discs), magnetic discs and semiconductor memories. These removable recording media can be provided as so-called package software. [0318] The program is not only installed into the computer from the above-mentioned type of recording medium, but also may be wirelessly transmitted from a download site to the computer or may be wire-transmitted to the computer via a network such as a LAN (Local Area Network) or the Internet. The computer receives the program transmitted in this manner, and can install the program into a recording medium such as an internal hard disk. [0319] It is to be noted that the various kinds of processing described in the specification are not only executed on a time-series basis according to the description, but may also be executed in parallel or individually according to the processing capability or requirements of an apparatus which executes processing. In addition, the term "system" used herein means a logical collection construction of a plurality of devices, and is not limited to a construction in which individual constituent devices are incorporated in the same housing. Industrial Applicability [0320] As described hereinabove, according to the preferred embodiment of the present invention, it is possible to efficiently execute correction processing of the pixel values of a particular area such as a motion-presence area on the basis of pixel values of a motion absence area, for example, white balance adjusted image data, and flash image data of the particular area such as the motion-presence area. Accordingly, it is possible to generate an image smoothly connected to the white balance adjusted image data, and it is also possible to generate an image which reflects texture information on the flash image data of the motion-presence area. Accordingly, the present invention can be applied to digital cameras which capture images having motion, and the like. [0321] According to the preferred embodiment of the present invention, in pixel value correction processing for a motion-presence area, after initial values of the difference or the ratio between white balance adjusted image data and flash image data are set in the motion-presence area, smoothing is performed by a smoothing filter and estimated values of the image difference or the image ratio in the motion-presence area are calculated to execute pixel value correction of the motion-presence area on the basis of the estimated values, whereby high speed processing using reduced amount of calculation is realized. Accordingly, the present invention can be applied to digital cameras which capture images having motion, and the like. [0322] Furthermore, according to the pixel value correction processing described as the third embodiment of the present invention, it is possible to perform appropriate color conversion of a motion-presence section pixel area by means of processing using a filter, and corrected pixel values relative to the motion-presence section pixel area become pixel values which retain the texture of a flash captured image \z, whereby it is possible to generate a far more natural image. When processing using a smoothing filter is executed, it is difficult to perform pixel correction corresponding to the image difference or the image ratio of an original image, and the fuzziness of edge sections and the blur of colors may occur according to the kind of image. However, according to the pixel value correction processing described as the third embodiment, there is provided a construction which performs filtering processing according to a pixel value conversion expression using a coefficient determined to take into account the pixel values of the flash captured image 12. Accordingly, pixel value correction which reflects the texture of the flash captured image \z is performed and the fuzziness of edge sections, the blur of colors and the like are solved even in the motion-presence area, whereby it is possible to generate an image which reflects the texture of the flash captured image I2. [0323] Furthermore, according to the present invention, after white balance adjustment using low resolution images and pixel value correction of a motion-presence section have been executed, it is possible to generate a high resolution corrected image on the basis of the correspondence of corrected image data to low resolution image data, whereby high speed processing can be achieved with a small memory amount and a high resolution corrected image can be finally acquired. Accordingly, the present invention is suitable for digital cameras having limited memory amounts. Brief Description of the Drawings [0324] Fig. 1 is a view showing the construction of an image processing device of the present invention,' Fig. 2 is a flowchart for describing the procedure of an image processing method of the present invention,' Fig. 3 is a flowchart for describing the procedure of white balance adjustment processing based on a plurality of image data in the image processing method of the present invention," Fig. 4 is a view for describing the principle of the white balance adjustment processing based on a plurality of image data in the image processing method of the present invention! Fig. 5 is a view for describing motion section detection processing based on a plurality of image data in image processing of the present invention! Fig. 6 is a flowchart for describing pixel value adjustment processing for a motion section in the image processing of the present invention. Fig. 7 is a view for describing pixel value adjustment processing for a motion section in the image processing of the present invention! Fig. 8 is a view for describing a mechanism which executes pixel value adjustment processing based on a plurality of image data in the image processing of the present invention! Fig. 9 is a flowchart for describing the procedure of white balance adjustment processing based on a plurality of image data in the image processing method of the present invention! Fig. 10 is a view for describing a mechanism which executes pixel value adjustment processing based on a plurality of image data in the image processing of the present invention! Fig. 11 is a view for describing pixel value adjustment processing for a motion section in the image processing of the present invention! Fig. 12 is a flowchart for describing pixel value adjustment processing for a motion section in a second embodiment of the present invention! Fig. 13 is a view for describing the pixel value adjustment processing for the motion section in the second embodiment of the present invention! Fig. 14 is a view for describing an initial-value setting method for an image difference d, which method is executed in the pixel value adjustment processing for the motion section in the second embodiment of the present invention! Fig. 15 is a view for describing the construction and the processing of a motion-section corrected pixel value calculation section in the second embodiment of the present invention! Fig. 16 is a view for describing pixel value adjustment processing for a motion section as well as a method of setting an initial value of an image difference d in a third embodiment of the present invention! Fig. 17 is a flowchart for describing the pixel value adjustment processing for the motion section in the third embodiment of the present invention; Fig. 18 is a view for describing the pixel value adjustment processing for the motion section in the third embodiment of the present invention! Fig. 19 is a view for describing the construction and the processing of a motion-section corrected pixel value calculation section in the third embodiment of the present invention! and Fig. 20 is a flowchart for describing processing in a fourth embodiment of the present invention. [Explanations of Numerals] [0325] 101 LENS 102 DIAPHRAGM 103 IMAGE PICKUP ELEMENT 104 CORRELATED DOUBLE SAMPLING CIRCUIT (CDS) 105 A/D CONVERTER 106 DIGITAL SIGNAL PROCESSING SECTION (DSP) 107 TIMING GENERATOR 108 D/A CONVERTER 109 VIDEO ENCODER 110 VIDEO MONITOR 111 CODEC (CODEC) 112 MEMORY 113 CPU 114 INPUT DEVICE 115 FLASH CONTROL SECTION 116 FLASH DEVICE 200 BALL 210 AREA 250 INNER BOUNDARY PIXEL OF MOTION PRESENCE SECTION 251 OUTER BOUNDARY PIXEL OF MOTION PRESENCE SECTION 252 NON-INNER-BOUNDARY PIXEL OF MOTION PRESENCE SECTION 253 NON-OUTER-BOUNDARY PIXEL OF MOTION ABSENCE SECTION 254 PIXEL a 255 NEIGHBORING PIXEL OF a 301,302,303 MEMORIES 304 DIFFERENCE IMAGE CALCULATION SECTION 305 WHITE BALANCE ADJUSTMENT SECTION 306 AMBIENT LIGHT COMPONENT ESTIMATION SECTION 307 WHITE BALANCE ADJUSTMENT SECTION 308 PIXEL VALUE ADDITION SE CTION 309 MOTION DETECTION SECTION 310 MOTION-SECTION CORRECTED PIXEL VALUE CALCULATION SECTION 311 WHITE BALANCE ADJUSTMENT SECTION 312 OUTPUT SWITCHING SECTION 401, 402, 403 FRAME MEMORIES 404 DIFFERENCE IMAGE CALCULATION SECTION 405 WHITE BALANCE ADJUSTMENT SECTION 406 PIXEL VALUE ADDITION SECTION 407 WHITE BALANCE ADJUSTMENT SECTION 408 MOTION DETECTION SECTION 409 MOTION-SECTION CORRECTED PIXEL VALUE CALCULATION SECTION 410 WHITE BALANCE ADJUSTMENT SECTION 411 OUTPUT SWITCHING SECTION 550 MOTION PRESENCE SECTION PIXEL AREA 551 MOTION ABSENCE SECTION PIXEL AREA 560 WHITE BALANCE ADJUSTED IMAGE R 561 FLASH IMAGE I2 564 WHITE BALANCE ADJUSTED IMAGE R 581 PIXEL IN MOTION PRESENCE SECTION PIXEL AREA 582 to 584 PIXELS OF MOTION ABSENCE SECTION PIXEL AREA 701 MOTION-ABSENCE-SECTION-PIXEL-AREA WHITE BALANCE ADJUSTED IMAGE R 702 IMAGE I2 STORING FRAME MEMORY 710 MOTION SECTION CORRECTED PIXEL VALUE CALCULATION SECTION 711 IMAGE DIFFERENCE (IMAGE RATIO) d CALCULATION SECTION 712 MOTION-PRESENCE-PIXEL-AREA IMAGE DIFFERENCE (IMAGE RATIO) d INITIAL VALUE SETTING SECTION 713 SMOOTHING PROCESSING SECTION 714 MOTION-SECTION FINAL CORRECTED PIXEL VALUE CALCULATION SECTION 821 PIXEL IN MOTION PRESENCE SECTION PIXEL AREA 822 to 824 PIXELS OF MOTION ABSENCE SECTION PIXEL AREA 860 WHITE BALANCE ADJUSTED IMAGE R 861 FLASH IMAGE I2 862 IMAGE DIFFERENCE d OF SECTION IN MOTION-PRESENCE SECTION PIXEL AREA 863 ESTIMATED VALUE OF IMAGE DIFFERENCE d IN MOTION-PRESENCE SECTION PIXEL ARE A 864 WHITE BALANCE ADJUSTED IMAGE R 901 MOTION-ABSENCE-SECTION-PIXEL-AREA WHITE BALANCE ADJUSTED IMAGE R 902 IMAGE I2 STORING FRAME MEMORY 910 MOTION-SECTION CORRECTED PIXEL VALUE CALCULATION SECTION 911 IMAGE DIFFERENCE (IMAGE RATIO) d CALCULATION SECTION 912 MOTION-PRESENCE-PIXEL-AREA IMAGE DIFFERENCE (IMAGE RATIO) d INITIAL VALUE SETTING SECTION 913 FILTERING PROCESSING SECTION 914 MOTION-SECTION FINAL CORRECTED PIXEL VALUE CALCULATION SECTION WE CLAIM: 1. An image processing method comprising the steps of: determining (S501) an image difference or an image ratio based on corresponding pixel values of first image data and second image data having different pixel values; determining (S506)an estimated value based on the image difference or the image ratio of a particular area of image data; and generating (S507) a corrected image of the particular area based on the estimated value of the image difference or the image ratio of the particular area which have been determined in the estimated-value determination step and the second image data; wherein: the first image data comprises a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash-assisted captured image, the second image data comprises a flash-assisted image I2, and the particular area comprises a motion-presence area in which a motion of a subject is detected; the image difference d(x, y) comprises a vector determined as: d(x, y) = R(x, y) — I2(x, y) where R(x, y) denotes a pixel value vector of each pixel (x, y) of the white balance adjusted image R, and I2(x, y) denotes a pixel value vector of the corresponding pixel (x, y) of the flash-assisted image I2; the image ratio d(x, y) is: d(x, y) - R(x, y)/(I2(x, y) + e) where e is a fixed value; and the corrected image generation step comprises a step of generating a corrected image of the motion-presence area on the basis of the estimated values of the image difference or the image ratio in the motion-presence area which are determined in the estimated value determination step and the flash-assisted image I2. 2. The image processing method as claimed in claim 1, wherein the image difference d(x, y) comprises a vector determined as: d(x, y) = A(x, y) - B(x, y) where A(x, y) denotes a pixel value vector of each pixel (x, y) of the first image data and B(x, y) denotes a pixel value vector of the corresponding pixel (x, y) of the second image data; and the image ratio d(x, y) is: d (x, y) = A(x, y)/(B(x, y) + e) where e is a fixed value. 3. The image processing method as claimed in claim 1, wherein the step of determining the estimated value further comprises: an initial-value setting step of setting an initial value of image difference or image ratio of the particular area of the image data on the basis of an image difference or an image ratio determined in an area in the vicinity of the particular area; and a smoothing process execution step of executing a smoothing process based on a smoothing filter applied against the initial value set in the initial-value setting step. 4. The image processing method as claimed in claim 3, wherein the step of setting the initial value comprises a step of setting an initial value of an initial-value-setting target pixel on the basis of an image difference or an image ratio of a pixel in the vicinity of the initial-value-setting target pixel for which the image-difference or the image-ratio is already set. 5. The image processing method as claimed in claim 4, wherein the step of setting the initial value comprises a step of applying a mask image in order to discriminate between the initial-value-setting target pixel and the pixel for which the image difference or the image ratio is already set. 6. The image processing method as claimed in claim 1, wherein the step of generating the corrected image further comprises a step of generating the corrected image of the particular area on the basis of the estimated values of the image differences or the image ratios in the particular area which are determined in the estimated-value determination step and the second image data; and if using the image difference, the step of generating the corrected-image is executed as a step of adding the estimated value of the image difference of the particular area determined in the estimated-value determination step to the second image data in the particular area; and if using the image ratio, the corrected-image generation step is executed as a step of multiplying the estimated value of the image difference of the particular area determined in the estimated-value determination step by the second image data in the particular area. 7. The image processing method as claimed in claim 1, wherein: the first image data comprises a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash-assisted captured image; the second image data comprises a flash-assisted image I2; the particular area comprises a motion-presence area in which a motion of a subject is detected; and if using the image difference, the step of generating the corrected-image is executed as a step of adding the estimated value of the image difference in the particular area determined in the estimated-value determination step to the flash-assisied image I2 in the motion-presence area; and if using the image ratio, the corrected-image generation step is executed as a step of multiplying the estimated value of the image ratio in the particular area determined in the estimated-value determination step by the flash-assisted c image I2 in the motion-presence area. 8. The image processing method as claimed in claim 1, wherein the step of determining the estimated-value further comprises: initial value setting step of setting an initial value of image difference or image ratio of the particular area of the image data on the basis of image difference or image ratio determined in an area in the vicinity of the particular area; and filtering process step executing pixel value conversion processing according to a pixel value conversion expression corresponding to filtering processing using a filter whose weight is set on the basis of the second image data against an initial value set in the initial-value setting step, and correcting the image difference or the image ratio of the particular area. 9. The image processing method as claimed in claim 8, wherein the initial-value setting step further comprises a step of setting an initial value of an initial-value-setting target pixel on the basis of an image difference or an image ratio of a pixel in the vicinity of the initial-value-setting target pixel for which the image-difference or the image-ratio is already set. 10. The image processing method as claimed in claim 8, wherein the initial value setting step further comprises a step of applying a mask image in order to discriminate between the initial-value-setting target pixel and the pixel for which the image difference or the image ratio is already set. 11. The image processing method as claimed in claim 8, wherein: the first image data comprises a white balance adjusted image R subjected to pixel value adjustment on the basis of a no flash-assisted captured image and a flash-assisted captured image; the second image data comprises a flash-assisted image I2;. the particular area comprises a motion-presence area in which a motion of a subject is detected; and the filtering process step comprises a step of executing pixel value correction processing using an expression containing a function whose weight is set according to the pixel values of pixels constituting image data of the flash-assisted image I2. 12. The image processing method as claimed in claim 8, wherein: the filtering processing step comprises a step of executing pixel value correction process using the following conversion expression: (Formula Removed) where d(x, y, ch) and I2( y, y, ch) are a value corresponding to a difference image or image ratio d of each channel [ch] at a pixel position (x, y), and a pixel value of the flash-assisted image I2, respectively; d'(x, y, ch) is an updated pixel value of the image difference d of the channel [ch] at the pixel position (x, y); i and j are reference pixel positions which are used for determining the updated value d' of the value d at the pixel position (x, y); if k denotes an arbitrary natural number, x — k ≤ i ≤ x + k and y — k ≤ j < y + k, and w(x) is a weighting function which sets a weight, according to the pixel values of pixels constituting image data of the flash-assisted image I2. 13. The image processing method as claimed in claim 12, wherein the weighting function w(x) in the conversion expression is a function expressed by the following expression (Formula Removed) 14. The image processing method as claimed in claim 12, wherein the reference pixel positions i, j which are used for determining the updated value d' of the value of the pixel position (x, y) in the above conversion expression are xx — k ≤ i ≤ x + k and y — k ≤ j ≤ y + k, where k is an arbitrary natural number and k is any of 1, 2 and 3. 15. The image processing method as claimed in claim 1, further comprising the steps of: storing in memory no flash-assisted low resolution image data IIL based on a no flash- assisted image; storing in memory flash-assisted high resolution image data I2H based on a flash-assisted image and flash resolution image data I2L; storing in memory no flash-assisted low resolution image data I3L based on a no flash-assisted image; and detecting a motion-presence area on the basis of the no flash-assisted low resolution image data IIL and the no flash-assisted low resolution image data I3L; wherein the no flash-assisted low resolution image data IlL is set as the first image data and the flash-assisted low resolution image data I2L is set as the second image data to determine the estimated values and generate a white balance adjusted image R including white balance adjustment processing and pixel value correction processing for a motion area; and in the corrected-image generation step, a high resolution final corrected image RH is generated on the basis of corresponding pixel values of the pixel value adjusted image R, the flash-assisted high resolution image data I2H and the flash-assisted low resolution image data I2L. 16. The image processing method as claimed in claim 15, wherein the corrected-image generation step further comprises the steps of acquiring pixel value conversion information on the corresponding pixels of the pixel value adjusted image R relative to the flash-assisted low resolution image data I2L; and executing pixel value conversion of the flash-assisted high resolution image data I2H. 17. An image processing apparatus comprising: a determining means for determining an image difference or an image ratio based on corresponding pixel values of first image data and second image data having different pixel values; an estimation means for determining an estimated value based on the image difference or the image ratio of a particular area of image data; and a generating means for generating a corrected image of the particular area based on the estimated value of the image difference or the image ratio of the particular area which have been determined by the estimation means and the second image data. 18. The image processing apparatus as claimed in claim 17, wherein the estimation means further comprises: an initial-value setting unit for setting an initial value of image difference or image ratio of the particular area of the image data on the basis of an image difference or an image ratio determined in an area in the vicinity of the particular area; and a smoothing process execution unit for executing a smoothing process based on a smoothing filter applied against the initial value set by the initial-value setting unit. 19. The image processing apparatus as claimed in claim 17, wherein the estimation means further comprises: initial value setting unit of setting an initial value of image difference or image ratio of the particular area of the image data on the basis of image difference or image ratio determined in an area in the vicinity of the particular area; and filtering unit for executing pixel value conversion processing according to a pixel value conversion expression corresponding to filtering processing using a filter whose weight is set on the basis of the second image data against an initial value set in the initial-value setting step, and correcting the image difference or the image ratio of the particular area.

Documents

Application Documents

# Name Date
1 190-delnp-2006-pct-373.pdf 2011-08-21
2 190-delnp-2006-pct-346.pdf 2011-08-21
3 190-delnp-2006-pct-338.pdf 2011-08-21
4 190-delnp-2006-pct-308.pdf 2011-08-21
5 190-delnp-2006-pct-304.pdf 2011-08-21
6 190-delnp-2006-pct-301.pdf 2011-08-21
7 190-delnp-2006-pct-237.pdf 2011-08-21
8 190-delnp-2006-gpa.pdf 2011-08-21
9 190-delnp-2006-form-5.pdf 2011-08-21
10 190-delnp-2006-form-3.pdf 2011-08-21
11 190-delnp-2006-form-2.pdf 2011-08-21
12 190-delnp-2006-form-18.pdf 2011-08-21
13 190-delnp-2006-form-1.pdf 2011-08-21
14 190-delnp-2006-drawings.pdf 2011-08-21
15 190-delnp-2006-description (complete).pdf 2011-08-21
16 190-delnp-2006-correspondence-others.pdf 2011-08-21
17 190-delnp-2006-correspondence-others-1.pdf 2011-08-21
18 190-delnp-2006-claims.pdf 2011-08-21
19 190-delnp-2006-abstract.pdf 2011-08-21
20 190-DELNP-2006-GPA-(03-07-2012).pdf 2012-07-03
21 190-DELNP-2006-Form-2-(03-07-2012).pdf 2012-07-03
22 190-DELNP-2006-Form-1-(03-07-2012).pdf 2012-07-03
23 190-DELNP-2006-Drawings-(03-07-2012).pdf 2012-07-03
24 190-DELNP-2006-Description (Complete)-(03-07-2012).pdf 2012-07-03
25 190-DELNP-2006-Correspondence Others-(03-07-2012).pdf 2012-07-03
26 190-DELNP-2006-Claims-(03-07-2012).pdf 2012-07-03
27 190-DELNP-2006-Abstract-(03-07-2012).pdf 2012-07-03
28 190-delnp-2006-Petition-137-(07-12-2012).pdf 2012-12-07
29 190-delnp-2006-Form-3-(07-12-2012).pdf 2012-12-07
30 190-delnp-2006-Correspondence Others-(07-12-2012).pdf 2012-12-07
31 190-delnp-2006-GPA-(06-08-2014).pdf 2014-08-06
32 190-delnp-2006-Correspondence Others-(06-08-2014).pdf 2014-08-06
33 190 DELNP 2006.pdf 2014-09-26
34 190-DELNP-2006_EXAMREPORT.pdf 2016-06-30