Specification
FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION (See section 10, rule 13)
“IMAGE PICKUP APPARATUS AND SIGNAL VALUE CORRECTION METHOD”
FUJIFILM Corporation., a company incorporated in Japan, of 26-30, Nishiazabu 2-chome, Minato-ku, Tokyo 1068620, Japan
The following specification particularly describes the invention and the manner in which it is to be performed.
{DESCRIPTION}
{Title of Invention}
IMAGE PICKUP APPARATUS AND SIGNAL VALUE CORRECTION METHOD
{Technical Field}
The present invention relates to an image pickup apparatus and a signal value correction method that by a simple configuration make it possible to quickly and accurately correct the variation in signal values caused by a color array for color filters of an image pickup element and the variation in signal values caused by a structure in which multiple pixels share a specific circuit element.
{B ackground Art}
CMOS (Complementary Metal Oxide Semiconductor) image pickup elements exhibit low power consumption, and are used in a variety of portable image pickup apparatuses such as digital cameras and mobile phones.
In CMOS image pickup elements, a technique in which multiple pixels share a single amplifier is widely used as a technique for reducing the number of transistors required on the substrate.
In particular, an image pickup element that has color filters in a Bayer array (GR/BG) and a sharing structure for sharing a single amplifier on a 2 x 2 pixels basis, exhibits a good matching because the repetition period of the Bayer array (2 x 2) is equal to the repetition period of the amplifier sharing structure (2 x 2), and therefore, the image pickup element is very commonly used.
PTL 1 discloses a configuration in which correction is performed using a vertical-line correction parameter for multiple lines in view of an amplifier sharing pattern.
PTL 2 describes a CMOS image pickup element that achieves space saving by sharing an amplifier on a vertical 2 x horizontal 2 pixel basis.
PTL 3 discloses a configuration in which four pixels around a pixel of interest separately have color-mixing correction parameters, in order to respond to the pixel-by-
pixel difference of the color-mixing rate, resulting from the asymmetry of the aperture of each pixel.
{Citation List} {Patent Literature} {PTL 1}
Japanese Patent Application Laid-Open No. 2008-288649 {PTL 2}
Japanese Patent Application Laid-Open No. 2000-78474 {PTL 3}
Japanese Patent Application Laid-Open No. 2007-142697
{Summary of Invention} {Technical Problem}
In image pickup elements, typically, there is a problem in that the image quality of a picked-up image deteriorates due to the presence of a sensitivity ratio caused by the difference of the colors (for example, RGB) of color filters. Furthermore, there is also a problem in that a coloring on the peripheral part of the picked-up image or a fixed pattern noise appears because different sensitivity ratios are distributed in an image pickup surface even among the same color pixels. Hence, by dividing the image pickup surface into multiple regions and multiplying the signal value (output value) of each pixel by the reciprocal of the sensitivity ratio among the pixels for each region, it is possible to correct the sensitivity ratio among the pixels caused by the color of the color filter.
In a CMOS image pickup element with an amplifier sharing structure, a difference of signal values occurs depending on the relative positions of the pixels to the shared amplifier, even among the same color pixels in the same region. That is, there is a problem in that among the pixels at different relative positions to the shared amplifier, the sensitivities vary depending on differences of the substrate layouts and the like, resulting in an adverse effect on image reproducibility. This problem has become more conspicuous with the refinement of pixel size.
Hence, there can be conceived an idea of previously storing correction coefficients whose number (MN x MN) corresponds to the least common multiple of the
repetition period (M x M) of a basic array pattern of the color filters and the repetition period (N x N) of an amplifier sharing structure, selecting an appropriate correction coefficient from them for each pixel, and multiplying the signal value of each pixel by the selected correction coefficient, in order to correct both the variation in signal values caused by a color array for the color filters of the image pickup element and the variation in signal values caused by a structure in which multiple pixels share a specific circuit element.
However, although only four correction coefficients are required if the color filters are arranged in the Bayer array of 2 x 2 and the amplifier sharing structure is a four pixel square array of 2 x 2, the number of correction coefficients (MN x MN) becomes enormous, for example, if the color filter array is an array of 3 x 3 or 6 x 6. Therefore, there is a problem of an increase in correction processing time and circuit size.
The configuration described in PTL 1 cannot be applied to the correction of the sensitivity ratio among the pixels caused by the variation in optical characteristics of the amplifier sharing structure, because of the use of light shielding pixels.
The configurations described in PTL 2 and PTL 3 result in a problem in that when the repetition arrangement period of the color filters is different from the repetition arrangement period of the amplifier sharing structure, the correction coefficients enormously increase.
In addition, since in the Bayer array, green (G) pixels are arranged in a plaid pattern (checkerboard pattern) and red (R) and blue (B) pixels are arranged line-sequentially, there is a problem in that a low-frequency coloring (color moire) occurs by a folding of high-frequency signals beyond the reproduction band for the colors and a deviation of the phase for the colors. Therefore, a configuration that can accurately and easily correct the sensitivity ratio even if the color filter array is other than the Bayer array is desired.
The present invention has been made in view of such circumstances, and has an object to provide an image pickup apparatus and a signal value correction method that by a simple configuration make it possible to quickly and accurately correct the variation in signal values caused by a color array for color filters of an image pickup element and the variation in signal values caused by a structure in which multiple pixels share a specific circuit element.
{Solution to Problem}
To achieve the object, the present invention provides an image pickup apparatus including an image pickup element in which a plurality of color filters are respectively arranged on a plurality of pixels arrayed two-dimensionally in a horizontal direction and a vertical direction, each of the pixels including a photoelectric conversion element; storage means that stores information for correcting a signal value of each of the pixels of the image pickup element; and correction means that corrects the signal value of each of the pixels of the image pickup element using the information stored in the storage means, in which the plurality of the pixels of the image pickup element share a specific circuit element on a multiple-pixel basis; the plurality of the color filters of the image pickup element are arranged such that a basic array pattern is repeated in the horizontal direction and the vertical direction, the basic array pattern mixedly including three or more color types of the color filters and having an arrangement period different from an arrangement period of a sharing structure pattern including the specific circuit element and the multiple pixels; the storage means stores a plurality of first correction coefficients and a plurality of second correction coefficients, the plurality of the first correction coefficients respectively corresponding to colors of the plurality of the color filters of the image pickup element, the plurality of the second correction coefficients respectively corresponding to a plurality of relative positions of the pixels to a position of the specific circuit element of the image pickup element; and when the correction means targets each of the plurality of the pixels of the image pickup element and corrects the signal value of each pixel of interest, the correction means selects a first correction coefficient corresponding to the color of the color filter on the pixel of interest from the plurality of the first correction coefficients stored in the storage means, selects a second correction coefficient corresponding to the relative position of the pixel of interest from the plurality of the second correction coefficients stored in the storage means, and performs a calculation with the selected first correction coefficient and the selected second correction coefficient, with respect to the signal value of the pixel of interest.
That is, only by storing the first correction coefficients corresponding to the colors of the color filters of the image pickup element and the second correction coefficients corresponding to the relative positions of the pixels to the position of the
specific circuit element of the image pickup element, in the storage means, an adequate combination of the first correction coefficients and the second correction coefficients is selected for each pixel, and a calculation with it is performed with respect to the signal value of each of the pixels. Therefore, with an essential minimum number of correction coefficients, it is possible to quickly and accurately correct the variation in signal values caused by the color array for the color filters of the image pickup element and the variation in signal values caused by the structure in which multiple pixels share the specific circuit element.
In an embodiment, the storage means stores a sensitivity-ratio correction coefficient for correcting a sensitivity ratio among the pixels, and a color-mixing correction coefficient for correcting a color mixing of the color filter on an adjacent pixel that is adjacent to each of the pixels, the sensitivity-ratio correction coefficient and the color-mixing correction coefficient including the first correction coefficient and the second correction coefficient. The correction means performs a calculation with the first correction coefficient and the second correction coefficient for one of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient, with respect to the signal value of the pixel of interest, and then performs a calculation with the first correction coefficient and the second correction coefficient for the other of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient, with respect to the calculation result. That is, as a first step correction, a calculation with the first correction coefficient and the second correction coefficient for one of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient (for the sensitivity-ratio correction coefficient, for example) is performed with respect to the signal value of each pixel of interest, and then as a second step correction, a calculation with the first correction coefficient and the second correction coefficient for the other of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient (for the color-mixing correction coefficient, for example) is performed with respect to the calculation result. Therefore, it is possible to correct the signal value of the pixel more accurately than conventional image pickup apparatuses.
In an embodiment, the image pickup apparatus further includes correction-coefficient calculation means that calculates the second correction coefficient, the correction-coefficient calculation means calculating the second correction coefficient by
comparing the signal values among the pixels that have different relative positions to the position of the specific circuit element, in which the storage means stores the second correction coefficient calculated by the correction-coefficient calculation means. That is, it is possible to determine the correction coefficient for the sensitivity ratio by a factor of the amplifier sharing structure, separately from the correction coefficient for the sensitivity ratio by a factor of the color filters.
In an embodiment, in a whole or a part of a picked-up image generated by the image pickup element, the correction-coefficient calculation means calculates the second correction coefficient by calculating an average value of the signal values of a plurality of the same color pixels over a plurality of the sharing structure patterns, for each of the relative positions to the position of the specific circuit element, and comparing the average values among the relative positions that are different from each other. That is, it is possible to easily adapt the sensitivity-ratio correction coefficient according to a change in photographing environment, such as a change in incidence angle by a lens exchange.
In an embodiment, the image pickup element includes a white color filter in the basic array pattern, and the correction-coefficient calculation means calculates the second correction coefficient by averaging the signal values of the pixels corresponding to the white color filter over a plurality of the basic array patterns. That is, it is possible to accurately determine the correction coefficient by utilizing the good sensitivity characteristic of white pixels.
Furthermore, the present invention provides a signal value correction method to correct a signal value of each pixel of an image pickup element in which a plurality of color filters are respectively arranged on a plurality of pixels arrayed two-dimensionally in a horizontal direction and a vertical direction, each of the pixels including a photoelectric conversion element, in which the plurality of the pixels of the image pickup element share a specific circuit element on a multiple-pixel basis; the plurality of the color filters of the image pickup element are arranged such that a basic array pattern is repeated in the horizontal direction and the vertical direction, the basic array pattern mixedly including three or more color types of the color filters and having an arrangement period different from an arrangement period of a sharing structure pattern including the specific circuit element and the multiple pixels; and the method includes,
previously storing a plurality of first correction coefficients and a plurality of second correction coefficients in a storage device, the plurality of the first correction coefficients respectively corresponding to colors of the plurality of the color filters of the image pickup element, the plurality of the second correction coefficients respectively corresponding to a plurality of relative positions of the pixels to a position of the specific circuit element of the image pickup element; and, when targeting each of the plurality of the pixels of the image pickup element and correcting the signal value of each pixel of interest, selecting a first correction coefficient corresponding to the color of the color filter on the pixel of interest from the plurality of the first correction coefficients, selecting a second correction coefficient corresponding to the relative position of the pixel of interest from the plurality of the second correction coefficients, and performing a calculation with the selected first correction coefficient and the selected second correction coefficient, with respect to the signal value of the pixel of interest.
{Advantageous Effects of Invention}
In accordance with the present invention, by a simple configuration, it is possible to quickly and accurately correct the variation in signal values caused by a color array for color filters of an image pickup element and the variation in signal values caused by a structure in which multiple pixels share a specific circuit element.
{Brief Description of Drawings}
{Figure 1} Figure 1 is a block diagram showing an overall configuration of an exemplary
image pickup apparatus according to a first embodiment.
{Figure 2} Figure 2 is a schematic diagram showing a part of an exemplary image pickup
element according to the first embodiment.
{Figure 3} Figure 3 is a diagram showing an exemplary color filter array of the image
pickup element.
{Figure 4} Figure 4 is a flowchart showing a flow of an exemplary sensitivity correction
process according to the first embodiment.
{Figure 5} Figure 5 is a flowchart showing a flow of an exemplary color-mixing
correction process according to the first embodiment.
{Figure 6} Figure 6 is a schematic diagram showing a part of an exemplary image pickup
element according to a second embodiment.
{Figure 7} Figure 7 is a block diagram showing an overall configuration of an exemplary
image pickup apparatus according to the second embodiment.
{Figure 8} Figure 8 is a schematic diagram showing a part of an exemplary image pickup
element according to a third embodiment.
{Figure 9} Figure 9 is a flowchart showing a flow of an exemplary color-mixing
correction process according to the third embodiment.
{Figure 10} Figure 10 is a schematic diagram showing a part of an exemplary image
pickup element according to a fourth embodiment.
{Figure 11} Figure 11 is a block diagram showing an overall configuration of an
exemplary image pickup apparatus according to the fourth embodiment.
{Figure 12} Figure 12 is a flowchart showing a flow of an exemplary color-mixing
correction process according to the fourth embodiment.
{Figure 13} Figure 13 is a diagram showing another exemplary sharing structure pattern.
{Figure 14} Figure 14 is a diagram showing a color filter array according to a first
example.
{Figure 15} Figure 15 is an explanatory diagram for explaining a basic array pattern in
the color filter array according to the first example.
{Figure 16} Figure 16 is a diagram showing a color filter array according to a second
example.
{Description of Embodiments}
Hereinafter, embodiments of the present invention are described in detail with reference to the accompanying drawings.
In the above first to fourth embodiments, as shown in Figure 3, Figure 6 and the like, the image pickup elements 12 in which the same type of sharing structure patterns CP are repeatedly arranged has been described as an example. However, the present invention is not limited to such a case. It is allowable to be an image pickup element 12 in which multiple types of sharing structure patterns differing in pixel arrangement patterns are repeatedly arranged.
Figure 13 shows a part of an exemplary image pickup element 12 in which multiple types (multiple sets) of sharing structure patterns CP1 and CP2 differing in pixel arrangement patterns are repeatedly arranged. In Figure 13, "R", "B" and "G" indicate that the color of the color filter is red, blue and green, respectively. The first sharing structure pattern CP1 includes an upper-left pixel Rll, a middle-left pixel G12, a lower-left pixel R13 and an upper-right pixel G21. The second sharing structure pattern CP2
includes an upper-left pixel B22, a middle-left pixel G23, a lower-left pixel B24 and a lower-right pixel G34. Then, the two types of sharing structure patterns CP1 and CP2 differing in pixel arrangement patterns are repeatedly arranged in the horizontal direction x and the vertical direction y.
The sensitivity correction unit 42 and the color-mixing correction unit 44 performs a correction of the signal value depending on the relative position of the pixel 62 to the shared amplifier 66, for each of the multiple types of sharing structure patterns CP1 and CP2.
In the present invention, the number of the types of different sharing structure patterns is not limited to two sets. [Various examples of color filter arrays]
In the following, various examples of color filter arrays are described in detail. (First example of color filter array)
A color filter array according to a first example is shown in Figure 14.
Figure 14 shows an example of an array of the color filters 64 provided in the image pickup element 12 (hereinafter, referred to as a "color filter array"). On each pixel, any one of three types of primary color filters of red (R), green (G) and blue (B) is arranged.
Here, the pixels 62 including the photoelectric conversion elements are hidden by the color filters 64 and the amplifiers 66 also are hidden by the color filters 64 so that the amplifiers 66 cannot be seen actually, but in Figure 14, the amplifiers 66 are drawn in perspective in order to discriminate the sharing structure for the amplifiers 66.
The color filter array shown in Figure 14 has the following first to third features.
(First feature) In the color filter array shown in Figure 14, a basic array pattern BP (a pattern shown by the thick-bordered frame), in which three color types (R, G and B) of color filters 64 are mixedly arrayed in a square array (in the example, horizontally six color filters and vertically six color filters), is repeatedly arranged in the horizontal direction x and the vertical direction y. That is, in this color filter array, each color filter of R, G and B (R filter, G filter and B filter) is arrayed with a predetermined period.
Thus, the R filter, G filter and B filter are arrayed with a predetermined period, and therefore, when performing a synchronization (interpolation) process (also called a
demosaic process) of R, G and B signals read from the image pickup element, it is possible to perform the process according to the repeated pattern.
(Second feature) The arrangement period (6 x 6) of the basic array pattern BP is three fold in both the horizontal direction x and the vertical direction y, compared to the arrangement period (2 x 2) of a sharing structure CP constituted by a shared amplifier 66 and 2x2 pixels. The basic array pattern BP includes therein at least one same-color square-array pattern GP constituted by 2 x 2 color filters 64 that respectively correspond to the 2 x 2 pixels in the amplifier sharing structure (sharing square-array pattern) and that have the same color. That is, the basic array pattern BP includes therein the 2 x 2 color filters (the color filters in the same-color square-array pattern CP) whose positions coincide with the positions of the 2 x 2 pixels in the amplifier sharing structure in both the horizontal direction x and the vertical direction y.
Thus, the basic array pattern BP includes therein the color filters in the same-color square-array pattern GP whose positions coincide with the positions of the 2 x 2 pixels in the sharing structure in both the horizontal direction x and the vertical direction y, and therefore, it is possible to easily detect the sensitivity difference caused by the amplifier sharing structure, based on the signal values of the 2 x 2 pixels corresponding to the same-color square-array pattern GP.
A third feature will be described. In Figure 14, if focusing on each of the plurality of colors (R, G and B) in the basic array pattern BP, then for each color, one or more color filters are arranged on each line in the horizontal direction x and vertical direction y in the basic array pattern BP. For example, as for the "G" color filter (hereinafter, merely referred to as "G"), one or more color filters are arranged on each line of x = 0 to 5 to the horizontal direction and on each line of y = 0 to 5 to the vertical direction y, in the basic array pattern BP. Similarly, as for the "R" color filter (hereinafter, merely referred to as "R"), one or more color filters are arranged on each line of x = 0 to 5 to the horizontal direction x and on each line of y = 0 to 5 to the vertical direction y, in the basic array pattern BP. Similarly, as for the "B" color filter (hereinafter, merely referred to as "B"), one or more color filters are arranged on each line of x = 0 to 5 to the horizontal direction x and on each line of y = 0 to 5 to the vertical direction y, in the basic array pattern BP. By this feature, it is possible to suppress an occurrence of color moire (false color).
A fourth feature will be described. In Figure 14, in the plurality of color filters, if focusing on each of the plurality of colors (R, G and B), there is such a line that color filters with the same color are arranged on the same line at two or more types of arrangement intervals, in both the horizontal direction x and the vertical direction y. For example, if focusing on "G" on the horizontal line of y = 0 (the topmost horizontal line in the figure), the interval between the "G" at the coordinates (0, 0) and the "G" at the coordinates (2, 0) is two pixels, and the interval between the "G" at the coordinates (2, 0) and the "G" at the coordinates (3, 0) is one pixel. The same goes for the horizontal lines of y = 2, 3 and 5. If focusing on "B" on the horizontal line of y = 1 (the second top horizontal line in the figure), the interval between the "B" at the coordinates (0, 1) and the "B" at the coordinates (2, 1) is two pixels, and the interval between the "B" at the coordinates (2, 1) and the "B" at the coordinates (6, 1) is four pixels. If focusing on "R" on the same horizontal line of y = 1, the interval between the "R" at the coordinates (3,1) and the "R" at the coordinates (5, 1) is two pixels, and the interval between the "R" at the coordinates (5, 1) and the "R" at the coordinates (9, 1) is four pixels. The same goes for the horizontal line of y = 4. Also, the same goes if focusing on each color on the vertical lines. Such lines exist in both the horizontal direction x and the vertical direction y, at least, at intervals not greater than the repetition period (six pixels in the horizontal direction x, six pixels in the vertical direction) of the basic array pattern BP. By this feature, it is possible to suppress an occurrence of geometric noise in a periodic pattern.
A fifth feature will be described. In the basic array pattern BP, the G filters corresponding to luminance pixels are arranged such that a portion including two or more successive G filters is included in each direction of the horizontal direction, the vertical direction, and the diagonal directions (NE and NW).
Since the G filters corresponding to luminance pixels are arranged on lines of the color filter array in the horizontal, vertical and diagonal (NE and NW) directions, it is possible to enhance reproduction accuracy of a synchronization process (demosaic process) in a high-frequency region, without depending on a direction of a high-frequency occurrence.
A sixth feature will be described. In Figure 14, when defining each of the color filters 64 constituting the same-color square-array pattern GP as the k-th same-
color filter (k represents the integers from 1 to 4 that designate the positions relative to the amplifier 66), among the plurality of color filters (clockwise from the direction of 12 o'clock, BRGGGBRG) adjacent to the first same-color filter (the "G" at the upper left relative to the amplifier 66), the plurality of color filters (clockwise from the direction of 3 o'clock, BRGGGBRG) adjacent to the second same-color filter (the "G" at the upper right relative to the amplifier 66), the plurality of color filters (clockwise from the direction of 6 o'clock, BRGGGBRG) adjacent to the third same-color filter (the "G" at the lower left relative to the amplifier 66), and the plurality of color filters (clockwise from the direction of 9 o'clock, BRGGGBRG) adjacent to the fourth same-color filter (the "G" at the lower right relative to the amplifier 66), the color combination (RGB) and the number for each color (two Rs, four Gs and two Bs) are common. In the example, whichever same-color filter is focused on, the adjacent color filters are arrayed in clockwise order: BRGGGBRG, in an identical same-color square-array pattern. That is, any same-color filter is surrounded by the color filters sequenced in the common color array.
Figure 15 shows a diagram relevant to a state in which the basic array pattern BP shown in Figure 14 is partitioned into four sets of 3 x 3 pixels. As shown in Figure 15, the basic array pattern BP can be regarded as an array in which the A array of 3 x 3 pixels surrounded by the full-line frame, and the B array of 3 x 3 pixels surrounded by the broken-line frame are alternately arranged in the horizontal direction and the vertical direction. In each of the A array and the B array, G filters, which are luminance pixels, are arranged at the four corners, at the center and on both diagonal lines. In the A array, B filters are arrayed in the horizontal direction and R filters are arrayed in the vertical direction, across the G filter at the center. On the other hand, in the B array, R filters are arrayed in the horizontal direction and B filters are arrayed in the vertical direction, across the G filter at the center. That is, in the A array and the B array, the positional relationship between R filters and B filters is reversed, but the other arrangements are common.
The basic array pattern BP shown in Figure 14 is point-symmetrical with respect to the center of the basic array pattern (the center in the four G filters). Also, as shown in Figure 15, each of the A array and the B array in the basic array pattern is point-symmetrical with respect to the G filter at the center, and is top-bottom and left-right
symmetrical (line-symmetrical). Furthermore, as shown in Figure 14, G filters are arranged on each of the diagonal lines (NE and NW) in the color filter array, and therefore, the color filter array of the image pickup element 12 has a feature that makes it possible to further enhance reproduction accuracy of a synchronization process in a high-frequency region. (Second example of color filter array)
Figure 16 shows a second example of a color filter array of the image pickup element.
As shown in Figure 16, this color filter array includes a basic array pattern BP that is a square array pattern of 7 x 7 pixels (a pattern shown by the thick-bordered frame), and this basic array pattern BP is repeatedly arranged in the horizontal direction and the vertical direction. That is, in this color filter array, similarly to the color filter array according to the first example shown in Figure 14, each color filter of R, G and B (R filter, G filter and B filter) is arrayed with a predetermined period (the first feature).
The basic array pattern BP includes therein 2x2 color filters (color filters in a same-color square-array pattern GP) whose positions coincide with the positions of the 2 x 2 pixels in the amplifier sharing structure in both the horizontal direction x and the vertical direction y (the second feature).
However, in the basic array pattern BP according to the example, an odd number of color filters are arranged in both the horizontal direction x and the vertical direction y. Thereby, the same-color patterns of 2 x 2 Gs are arranged such that the positions deviate from each other by an odd number of pixels (in the example, one pixel) in both the horizontal direction x and the vertical direction y.
The four pixels in the same-color pattern of 2 x 2 may be arranged so as to deviate in both the horizontal direction x and the vertical direction y. That is, the four pixels in the same-color pattern of 2 x 2 are only necessary to be arranged such that they are at four types of positions of (2n, 2m), (2n+l, 2m), (2n, 2m+l) and (2n+l, lm+1). In other words, the four same-color filters of 2 x 2 are only necessary to be at four different positions relative to the amplifier 66.
Thus, multiple same-color square-array patterns GP are arranged so as to deviate from each other by an odd number of pixels, and therefore, it is possible to provide basic array patterns BP each of which necessarily includes one or more same-color square-
array patterns GP matching with the sharing structure pattern (CP in Figure 2) in which the amplifier 66 is shared. By measuring the signal value of each pixel in the same-color square-array pattern GP, it is possible to adequately measure and correct the variation in characteristics caused by the positions of the pixels relative to the amplifier 66.
In the above examples, the image pickup element with three color types of color filters of primary colors, RGB, or four color types of color filters of RGBW has been described. However, the present invention is not limited to this, and can be applied to an image pickup element with four color types of color filters that have three primary colors, RGB, and another color (for example, emerald (E)).
In addition, the present invention can be applied to an image pickup element with four color types of complementary color filters that have cyan (C), magenta (M) and yellow (Y), which are complementary colors for primary colors RGB, along with G.
So far, the case in which multiple pixels share only an amplifier (amplifying element) has been described as an example. However, it goes without saying that the present invention can be applied to a case of another circuit element if it involves a sharing structure causing sensitivity difference. The sensitivity correction in such a case is also comprehended in the present invention.
Only the case in which the basic array pattern BP is constituted by 6 x 6 filters or 7 x 7 filters is shown in the figures. However, it goes without saying that other filter numbers (8 x 8, 9 x 9, ...) are also allowable. Furthermore, the color filter array in a basic array pattern is not limited to an N x N square array, and the present invention can be applied to a basic array pattern with an N x M array. Here, in view of ease of image processes such as a synchronization process (demosaic process) and a thinning process in taking a moving image, it is preferable that N and M be 10 or less.
The present invention is not limited to the examples described in the specification and the examples shown in the drawings, and naturally, various design modifications and improvements may be made without departing from the spirit of the present invention.
{Reference Signs List}
10 ... photographing lens, 12 ... image pickup element, 40 ... correction-coefficient storage unit, 42 ... sensitivity correction unit, 44 ... color-mixing correction unit, 46 ... sensitivity-correction-coefficient calculation unit, 50 ... CPU, 62 ... pixel, 64 ... color filter, 66 ... amplifier, 100 ... image pickup apparatus
{CLAIMS}
1. An image pickup apparatus comprising an image pickup element in which a
plurality of color filters are respectively arranged on a plurality of pixels arrayed two-dimensionally in a horizontal direction and a vertical direction, each of the pixels including a photoelectric conversion element; storage means that stores information for correcting a signal value of each of the pixels of the image pickup element; and correction means that corrects the signal value of each of the pixels of the image pickup element using the information stored in the storage means,
wherein the plurality of the pixels of the image pickup element share a specific circuit element on a multiple-pixel basis,
the plurality of the color filters of the image pickup element are arranged such that a basic array pattern is repeated in the horizontal direction and the vertical direction, the basic array pattern mixedly including three or more color types of the color filters and having an arrangement period different from an arrangement period of a sharing structure pattern including the specific circuit element and the multiple pixels,
the storage means stores a plurality of first correction coefficients and a plurality of second correction coefficients, the plurality of the first correction coefficients respectively corresponding to colors of the plurality of the color filters of the image pickup element, the plurality of the second correction coefficients respectively corresponding to a plurality of relative positions of the pixels to a position of the specific circuit element of the image pickup element, and
when the correction means targets each of the plurality of the pixels of the image pickup element and corrects the signal value of each pixel of interest, the correction means selects a first correction coefficient corresponding to the color of the color filter on the pixel of interest from the plurality of the first correction coefficients stored in the storage means, selects a second correction coefficient corresponding to the relative position of the pixel of interest from the plurality of the second correction coefficients stored in the storage means, and performs a calculation with the selected first correction coefficient and the selected second correction coefficient, with respect to the signal value of the pixel of interest.
2. The image pickup apparatus according to claim 1, wherein the storage means
stores a sensitivity-ratio correction coefficient for correcting a sensitivity ratio among the
pixels, and a color-mixing correction coefficient for correcting a color mixing of the
color filter on an adjacent pixel that is adjacent to each of the pixels, the sensitivity-ratio
correction coefficient and the color-mixing correction coefficient including the first
correction coefficient and the second correction coefficient, and
the correction means performs a calculation with the first correction coefficient and the second correction coefficient for one of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient, with respect to the signal value of the pixel of interest, and then performs a calculation with the first correction coefficient and the second correction coefficient for the other of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient, with respect to the calculation result.
3. The image pickup apparatus according to claim 1 or 2, wherein, as the plurality of the first correction coefficients, the storage means stores as many first correction coefficients as types of the color filters of the image pickup element, the types of the color filters being different from each other in spectral characteristic.
4. The image pickup apparatus according to any one of claims 1 to 3, wherein, as the plurality of the second correction coefficients, the storage means stores as many second correction coefficients as the pixels existing in one unit of the sharing structure pattern of the image pickup element.
5. The image pickup apparatus according to any one of claims 1 to 4, further comprising correction-coefficient calculation means that calculates the second correction coefficient, the correction-coefficient calculation means calculating the second correction coefficient by comparing the signal values among the pixels that have different relative positions to the position of the specific circuit element,
wherein the storage means stores the second correction coefficient calculated by the correction-coefficient calculation means.
6. The image pickup apparatus according to claim 5, wherein, in a whole or a part
of a picked-up image generated by the image pickup element, the correction-coefficient
calculation means calculates the second correction coefficient by calculating an average
value of the signal values of a plurality of the same color pixels over a plurality of the
sharing structure patterns, for each of the relative positions to the position of the specific
circuit element, and comparing the average values among the relative positions that are different from each other.
7. The image pickup apparatus according to claim 6, wherein the image pickup
element includes a white color filter in the basic array pattern, and
the correction-coefficient calculation means calculates the second correction coefficient by averaging the signal values of the pixels corresponding to the white color filter over a plurality of the basic array patterns.
8. The image pickup apparatus according to any one of claims 1 to 7, wherein
multiple types of the sharing structure patterns are repeatedly arranged in the image
pickup element, the multiple types of the sharing structure patterns being different from
each other in arrangement pattern of the pixels, and
the correction means corrects the signal value depending on the relative position of the pixel to the position of the specific circuit element, for each of the multiple types of the sharing structure patterns.
9. A signal value correction method to correct a signal value of each pixel of an
image pickup element in which a plurality of color filters are respectively arranged on a
plurality of pixels arrayed two-dimensionally in a horizontal direction and a vertical
direction, each of the pixels including a photoelectric conversion element,
wherein the plurality of the pixels of the image pickup element share a specific circuit element on a multiple-pixel basis,
the plurality of the color filters of the image pickup element are arranged such that a basic array pattern is repeated in the horizontal direction and the vertical direction, the basic array pattern mixedly including three or more color types of the color filters and having an arrangement period different from an arrangement period of a sharing structure pattern including the specific circuit element and the multiple pixels, and
the method comprises, previously storing a plurality of first correction coefficients and a plurality of second correction coefficients in a storage device, the plurality of the first correction coefficients respectively corresponding to colors of the plurality of the color filters of the image pickup element, the plurality of the second correction coefficients respectively corresponding to a plurality of relative positions of the pixels to a position of the specific circuit element of the image pickup element; and, when targeting each of the plurality of the pixels of the image pickup element and
correcting the signal value of each pixel of interest, selecting a first correction coefficient corresponding to the color of the color filter on the pixel of interest from the plurality of the first correction coefficients, selecting a second correction coefficient corresponding to the relative position of the pixel of interest from the plurality of the second correction coefficients, and performing a calculation with the selected first correction coefficient and the selected second correction coefficient, with respect to the signal value of the pixel of interest.
10. The signal value correction method according to claim 9, wherein the method
comprises storing a sensitivity-ratio correction coefficient for correcting a sensitivity
ratio among the pixels, and a color-mixing correction coefficient for correcting a color
mixing of the color filter on an adjacent pixel that is adjacent to each of the pixels, in the
storage device, the sensitivity-ratio correction coefficient and the color-mixing correction
coefficient including the first correction coefficient and the second correction coefficient;
and
performing a calculation with the first correction coefficient and the second correction coefficient for one of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient, with respect to the signal value of the pixel of interest, and then performing a calculation with the first correction coefficient and the second correction coefficient for the other of the sensitivity-ratio correction coefficient and the color-mixing correction coefficient, with respect to the calculation result.
11. The signal value correction method according to claim 9 or 10, wherein the method comprises calculating the second correction coefficient by comparing the signal values among the pixels that have different relative positions to the position of the specific circuit element, and storing the calculated second correction coefficient in the storage device.
12. The signal value correction method according to claim 11, wherein, in a whole or a part of a picked-up image generated by the image pickup element, the method comprises calculating the second correction coefficient by calculating an average value of the signal value of a plurality of the same color pixels over a plurality of the sharing structure patterns, for each of the relative positions to the position of the specific circuit element, and comparing the average values among the relative positions that are different from each other.
13. The signal value correction method according to claim 12, wherein the image
pickup element includes a white color filter in the basic array pattern, and
the method comprises calculating the second correction coefficient by averaging the signal values of the pixels corresponding to the white color filter over a plurality of the basic array patterns.