Specification
DESCRIPTION [
TECHNICAL FIELD I
The present invention relates to image processing methods and devices, as well as image processing programs and media for |
recording such programs. Particularly, the present invention is drawn to image processing methods and devices which are able to ;
•
suppress adverse effects of the change in saturation during the t
grayscale conversion of a color image, and exert control so that
the saturation may favorably be reproduced even with a converted
grayscale, as well as image processing programs for making a
computer implement such image processing methods and computer- readable recording media with such programs stored therein. I
BACKGROUND ART
When an image taken with a camera or produced with a
personal computer as an input image is to be outputted from a
printer as a hard copy image or displayed on a monitor as a soft
^ copy image, for instance, the color space of the input image
(input color space) is converted into the color space of the !
output device such as a printer or a monitor (output color
space).
If the input color space and the output color space are
different in dynamic range from each other, the grayscale is
compressed or expanded to thereby map the image so that the
dynamic range of the output device may effectively be used.
In the case where the output color space is smaller than
the input color space, the grayscale needs to be subjected to
conversion for compressing it in order to put the whole scene
luminance information on the input image within a limited
2~
t
dynamic range for luminance of the output device.
Compression of the grayscale is normally performed using a
one-dimensional lookup table (LUT) for converting a threechannel
input image signal, with the three channels being red
(R), green (G) and blue (B) channels, into a three-channel RGB
output image signal.
Fig. 11 shows an exemplary grayscale curve used for an LUT
for performing such compression of the grayscale.
The grayscale curve as shown is adapted for the compression I
of an RGB input image signal with a dynamic range of 5.3 into an '
-^ RGB output image signal with a dynamic range of 3.3, and has a
W
linear middle part with a slope of 45 degrees continuing at its
both ends into parts with reduced slopes, respectively, that is
to say, constitutes a so-called S curve.
The grayscale curve of S shape is used to maintain the
grayscale at its middle levels while compressing it at lower and higher levels more intensively and, consequently, allows J
luminance information on an input image to be put within the 1
dynamic range for luminance of an output image with no clamping
thereof. i
In another method for grayscale compression, an RGB signal
^p is converted into a luminance signal and a chromaticity signal
(in Yxy, YCC or L*a*b*, for instance), and the luminance signal !
Y or lightness signal L* is solely compressed using a grayscale
curve (LUT). !
Patent Literature 1 proposes yet another method for
grayscale compression. In the gradation correcting method as
disclosed in Patent Literature 1, a luminance signal is obtained ;
from R, G, and B input signals; the luminance signal is gammaconverted
according to desired gradation characteristics; the
ratio of the gamma-converted luminance signal to the luminance
signal before gamma conversion is multiplied by each of the R, G,
and B input signals for obtaining primary gradation-corrected R,
3 G, and B signals; the difference between each of the R, G, and B
input signals and the luminance signal before gamma conversion
is added to the gamma converted luminance signal for obtaining
secondary gradation-corrected R, G, and B signals; and final,
gradation-corrected R, G, and B output signals are obtained by
interpolation from the primary and secondary gradation-corrected R, G, and B signals, with the ratio between the two types of
gradation-corrected R, G, and B signals to be added being
controlled depending on the value of the luminance signal before !
gamma conversion.
It is stated in Patent Literature 1 that the disclosed
•
method is the gradation adjusting method which allows an
effective brightness adjustment in the dynamic range of a CRT or
a printer with no change in hue or saturation.
CITATION LIST
PATENT LITERATURE Patent Literature 1: JP 2699711 B
SUMMARY OF INVENTION
TECHNICAL PROBLEMS
^p If the grayscale is compressed by a method using the above
LUT for three-channel RGB image signals, a problem is raised in
that the saturation is simultaneously changed. During the
compression using such an S-shaped grayscale curve as shown in
Fig. 11, lighter as well as darker colors are reduced in
saturation with disadvantage.
In other words, during the compression or expansion of the
grayscale, change in gray level brings about not only change in
luminosity (or brightness) but in color reproduction, that is to
say, in hue and saturation, so that the color as represented by
the input image data appears differently upon output.
On the other hand, in the method which converts an RGB
t
signal into a luminance signal and a chromaticity signal, and
only compresses the luminance signal Y or lightness signal L*,
the reproduced color feels uncomfortable because the saturation
is kept intact. f
In the gradation correcting method as disclosed in Patent
Literature 1, the hue and the saturation are maintained by
maintaining the. difference between each of the R, G and B values, which are in a linear relationship to the luminance, and Y.
Since human color perception is not necessarily linear to the
luminance, the hue or the saturation may be changed with the
greatly changed R, G, B or Y value, or depending on the position {
^, of the chromaticity point of any primary color, R, G or B (as
defined by X, Y, and Z).
An object of the present invention is to solve the above
problems with the prior art, aiming at providing an image
processing method and an image processing device, each capable
of maintaining the saturation and the hue of a color, or
controlling the saturation at will, when the gray level is changed during the exchange and reproduction of images between |
color spaces or devices with different dynamic ranges and, accordingly, capable of performing the grayscale conversion
which realizes a favorable saturation reproduction, as well as j
^ an image processing program for making a computer implement such
an image processing method, and a computer-readable recording
medium having such a program stored therein.
Another object of the present invention is to provide image processing method and device as well as an image processing [.
program each allowing a proper adjustment of the hue and the
saturation, and a medium for recording such a program.
SOLUTION TO PROBLEMS In order to achieve the above objects, the present
invention provides in its first aspect an image processing
I
method adapted to convert grayscale characteristics of image
data on an image, comprising: first grayscale conversion in
which first grayscale characteristics conversion is performed
using a first lookup table of one-dimensional type corresponding
to RGB pixel data on the image; and second grayscale conversion
in which the RGB pixel data on the image is converted into
luminance data or lightness data and chromaticity data, and I
second grayscale characteristics conversion is performed only on J
the luminance data or lightness data using a second lookup table
of one-dimensional type corresponding to the luminance data or
lightness data, wherein weighting is performed with respect to
•
the first grayscale characteristics conversion and the second
grayscale characteristics conversion.
Also in order to achieve the above objects, the present
invention provides in its second aspect an image processing
device adapted to convert grayscale characteristics of image
data on an image, comprising: a first grayscale conversion means
for performing first grayscale characteristics conversion using a first lookup table of one-dimensional type corresponding to RGB pixel data on the image; and a second grayscale conversion ;
means for converting the RGB pixel data on the image into I
luminance data or lightness data and chromaticity data, and ;
^ performing second grayscale characteristics conversion only on
the luminance data or lightness data using a second lookup table
of one-dimensional type corresponding to the luminance data or lightness data, wherein the device performs weighting with j
respect to the first grayscale characteristics conversion and 1
the second grayscale characteristics conversion. ;
The present invention provides in its third aspect an image processing program for making a computer implement individual
steps of the image processing method of the first aspect. 5
The present invention provides in its fourth aspect a computer-readable recording medium in which an image processing I
c
I
program for making a computer implement individual steps of the
image processing method of the first aspect is stored.
In each of the above aspects, the luminance data or •
lightness data is preferably Y on CIE XYZ colorimetric system or
a value determined from the Y.
The chromaticity data is preferably a value determined from
Cr and Cb on YCC color system or from X, Y and Z on XYZ colorimetric system. It is preferable that the first grayscale conversion is
•
carried out such that: it precedes the second grayscale
conversion; and the first grayscale characteristics conversion
is performed on first RGB pixel data on an input image so as to
obtain second RGB pixel data, and the second grayscale
conversion is carried out such that: the obtained second RGB
pixel data is converted into first luminance data or lightness
data and the chromaticity data; the second grayscale characteristics conversion is performed only on the first [
luminance data or lightness data so as to obtain second :
luminance data or lightness data; and the obtained second
luminance data or lightness data and the chromaticity data are
converted together into third RGB pixel data on an output image. I
^ Preferably, during the second grayscale conversion, the
second luminance data or lightness data and the chromaticity
data are converted together into fourth RGB pixel data, and the
third RGB pixel data on the output image is obtained by performing balance adjustment on the fourth RGB pixel data. !
It is also preferable that the second grayscale conversion
is carried out prior to the first grayscale conversion such j
that: first RGB pixel data on an input image is converted into third luminance data or lightness data and the chromaticity data; the second grayscale characteristics conversion is ;
performed only on the third luminance data or lightness data so i
7
as to obtain fourth luminance data or lightness data; and the
obtained fourth luminance data or lightness data and the
chromaticity data are converted together into fifth RGB pixel
data, and the first grayscale conversion is carried out such
that the first grayscale characteristics conversion is performed
on the obtained fifth RGB pixel data so as to obtain third RGB
pixel data on an output image.
It is preferable that the weighting with respect to the
first grayscale characteristics conversion (grayscale
conversion) and the second grayscale conversion is performed by
j» using images of a color chart that have different luminosities
to determine a ratio between amounts of the first grayscale
characteristics conversion and of the second grayscale
characteristics conversion.
The ratio between the amounts of the first grayscale
characteristics conversion and of the second grayscale
characteristics conversion is preferably determined by
processing the images that have different luminosities while
varying the ratio between the amounts of the first grayscale
characteristics conversion and of the second grayscale ;
characteristics conversion, and calculating a*b* chromaticity on
CIE L*a*b* color space to confirm saturation.
^P The images that have different luminosities are preferably
obtained by photographing the color chart with a camera at
different exposures or at different luminosities of light with
which the color chart is illuminated or by calculating, from an
image of the color chart, another image to be obtained at a
different luminosity.
In addition, the weighting with respect to the first
grayscale conversion and the second grayscale conversion is
preferably performed in order to attain the aimed or specified
grayscale characteristics conversion and the aimed or specified
saturation reproduction, or for the fine adjustment of grayscale
I
characteristics conversion and/or saturation reproduction.
It is preferable that the weighting is performed in order
to synthesize a conversion characteristic curve used for
converting grayscale characteristics of the image data on an
image, and the image data on an image is converted using the
synthesized conversion characteristic curve.
It is also preferable that the weighting is performed in
order to change a conversion curve used for converting a
specified range of grayscale characteristics of the image data
on an image.
^ ADVANTAGEOUS EFFECTS OF INVENTION
According to the present invention, grayscale conversion
realizing a favorable saturation reproduction is performed when
images are exchanged to reproduction between color spaces or
devices with different dynamic ranges.
The present invention makes it possible to suppress adverse
effects of the change in saturation during the grayscale •
conversion of a color image, and exert control so that the
saturation may favorably be reproduced even with a converted
grayscale. In other words, according to the present invention,
the saturation and the hue of a color are maintained, or the
4b saturation is controlled at will, even though the gray level is
changed.
According to the present invention, it is further possible i
to maintain the hue and the saturation in the color space which
is non-linear to the luminance, that is to say, more adequate to j
human perception. Consequently, grayscale characteristics are
attainable with accurately maintained hue and saturation even if
dynamic range compression is performed between color spaces
having quite different dynamic ranges. Finally, according to the present invention, it is possible I
to design the degree of hue and saturation adjustment as
appropriate.
I
BRIEF DESCRIPTION OF DRAWINGS
Fig. 1 is a block diagram illustrating a general
configuration of an embodiment of the image processing device of
the present invention.
Figs. 2A and 2B are diagrams each showing the flow of an
embodiment of the image processing method of the present
invention. I
Fig. 3 is a graph showing an example of the grayscale '
conversion curve to be used in the present invention.
^M Fig. 4 is a graph provided in order to explain an example
of the first grayscale characteristics conversion of the image
processing method of the present invention.
Fig. 5 is a graph for explaining an example of the second
grayscale characteristics conversion of the image processing
method of the present invention.
Figs. 6A, 6B, and 6C are graphs for explaining an example
of RGB balance adjustment by the image processing method of the present invention, made for the colors R, G and B, respectively. j
Fig. 7A is a diagram showing a color chart used in the image processing method of the present invention to determine
the ratio between the amounts of the first grayscale I
f ^ characteristics conversion and of the second grayscale
characteristic conversion, Fig. 7B is a graph of the
chromaticity of color chart images with different luminosities,
and Fig. 7C is a graph of the chromaticity of the color chart !.
image having undergone grayscale conversion. r
Fig. 8 is a diagram showing the flow of another embodiment
of the image processing method of the present invention.
Fig. 9 is a diagram illustrating an image processing device
for implementing yet another embodiment of the image processing
method of the present invention.
Fig. 10 is a graph showing exemplary curves of compressive
conversion by individual conversion process units of the image
(0
i
f
processing device as shown in Fig. 9.
Fig. 11 is a graph showing a grayscale conversion curve.
DESCRIPTION OF EMBODIMENTS j
In the following, the image processing method and device,
as well as the image processing program and the medium for
recording the program, all according to the present invention,
are described in detail based on the preferred embodiments as shown in the accompanying drawings. J
Fig. 1 is a schematic block diagram illustrating a general
^n configuration of an embodiment of the image processing device of
the present invention for implementing the image processing
method of the present invention. Figs. 2A and 2B are flow charts
each showing the flow of an embodiment of the image processing
method of the invention implemented by the image processing
device as shown in Fig. 1. <
An image processing device 10 of the embodiment as shown in i
Fig. 1 includes an input image data acquiring unit 12 for acquiring RGB image data on an input image; a first grayscale j
conversion section 14 for subjecting the RGB image data to a f
specified grayscale conversion amount (compression/expansion
{0} amount) of first grayscale characteristics conversion process so
as to obtain converted RGB image data; a second grayscale
conversion section 22 provided with a colorimetric conversion >
unit 16 for colorimetrically converting RGB image data into I
luminance data (Y) or lightness data (L*) and chromaticity data (xy, CC or a*b*), a luminance grayscale conversion unit 18 for ^
subjecting the luminance data or lightness data (Y or L*) to a specified grayscale conversion amount (compression/expansion [
amount) of second grayscale characteristics conversion process,
and with an inverse conversion unit 20 for inversely converting
the grayscale-converted luminance data (Y or L*) and the nonconverted
chromaticity data (xy, CC or a*b*) as subjected to no
I
I
conversion together into RGB image data; an RGB balance
adjustment unit 24 for performing RGB balance adjustment on the
RGB image data so as to obtain RGB image data on an output
image; an output image data outputting unit 26 for outputting
the RGB image data on an output image to an external, image
output device not shown; and a memory 28 in which the ratio
between the grayscale conversion amounts for the first grayscale
conversion section 14 and for the luminance grayscale conversion
unit 18 of the second grayscale conversion section 22, as well
as LUTs used in the first grayscale conversion section 14, the j
luminance grayscale conversion unit 18, and the RGB balance
^» adjustment unit 24 are stored.
It is assumed in the present invention that image data (RGB,
XYZ, YCC or L*a*b*), luminance data (Y), lightness data (L*),
and chromaticity data (xy, CC or a*b*) refer to data on each
pixel of an image, namely, the image data, luminance data, [
lightness data and chromaticity data on each pixel, respectively,
as long as the data are involved with the image.
In the image processing device 10 of this embodiment, the
first grayscale characteristics conversion process (hereafter referred to as "grayscale conversion process") in the first
grayscale conversion section 14 may be performed before the
^P second grayscale conversion process in the second grayscale
conversion section 22 as shown in Fig. 1 with solid lines and in
Fig. 2A as well, or after the second grayscale conversion
process in the second grayscale conversion section 22 as shown
in Fig. 1 with dotted lines and in Fig. 2B. 1
The image processing method of the first embodiment of the
present invention as shown in Figs. 1 and 2A, in which the first
grayscale conversion process is followed by the second grayscale
conversion process, is initially described.
In the image processing method of the first embodiment that
is implemented by the image processing device 10 as shown, a \7-
I
I
j
target grayscale or a specified grayscale, or the total
grayscale conversion amount considered as the target grayscale
or the specified grayscale, the ratio between a first grayscale
conversion amount (compression/expansion amount) and a second
grayscale conversion amount, a first LUT (RGB-LUT) 14a used for
the first grayscale conversion process (RGB grayscale conversion
process) and a second LUT (luminance LUT) 18a used for the
second grayscale conversion process (luminance grayscale
conversion process), both prepared in accordance with the ratio i
as above, a third LUT (RGB balance adjustment LUT) 24a used for i
the RGB balance adjustment, and the like are stored in advance
^H in the memory 28 as initial values or set points, as seen from
Figs. 1 and 2A. The first, second and third LUTs 14a, 18a and
24a are set in the first grayscale conversion section 14, the
luminance grayscale conversion unit 18 and the RGB balance
adjustment unit 24, respectively, during preprocessing (step
S10) .
The first and second LUTs 14a and 18a are obtained by distributing the grayscale conversion amount (grayscale
compression/expansion amount) as the ultimate target or as
specified between the grayscale conversion amounts of the RGB
grayscale conversion and of the luminance grayscale conversion
^P (Y or L*) at a certain ratio. The ratio between the grayscale
conversion amounts may be made variable. The method of
determining the grayscale conversion amount distribution ratio
will be described later.
The following description is made on the typical case where
the target grayscale is set in the memory 28 as an initial value
or set point, although the present invention is not particularly
limited to such setting, and a specified grayscale or even an 1
arbitrary grayscale may be set as above. In addition, it is needless to say that the present invention is also applicable to
such a case where fine adjustment in grayscale is to be i
conducted with no special settings, or conversion of grayscale {
\2> !
I
characteristics and/or fine adjustment of the saturation
reproduction is aimed.
When the grayscale conversion as the image processing of
the present invention is started, the RGB image data which is
RGB pixel data on an input image (hereafter referred to as
"first RGB image data") is firstly acquired by the input image
data acquiring unit 12 in step S12 of Fig. 2A.
Secondly in step S14, the acquired first RGB image data is
subjected to a preset grayscale conversion amount of first
grayscale conversion process by the first grayscale conversion
_ section 14 using the first LUT (RGB-LUT) 14a as a one-
^^ dimensional LUT corresponding to the RGB pixel data, so as to
generate the second RGB image data as grayscale-converted with
the RGB pixel data. In other words, the first grayscale
conversion section 14 is adapted to perform grayscale conversion
in the RGB space as the first grayscale conversion process.
In the next step S16, the generated second RGB image data
is colorimetrically converted (subjected to luminance and
chromaticity conversion) by the colorimetric conversion unit 16
of the second grayscale conversion section 22 so as to generate
luminance data (Y) and chromaticity data (xy, CC (Cr, Cb)) on
A CIE XYZ colorimetric system or YCC color system in NSTC mode, or
lightness data (L*) and chromaticity data (a*b*) on CIE L*a*b*
colorimetric system.
In the next step S18, the luminance data or lightness data
(hereafter collectively referred to as "luminance data") (Y or
L*) thus generated is subjected to a preset grayscale conversion
amount of second grayscale conversion process by the luminance
grayscale conversion unit 18 of the second grayscale conversion
section 22 using the second LUT (luminance LUT) 18a as a onedimensional
LUT, so as to generate the second luminance data (Y ;
or L*) as grayscale-converted. In other words, the luminance
grayscale conversion unit 18 is adapted to perform grayscale I
f
conversion in the luminance-color difference area as the second
grayscale conversion process.
It should be noted that the chromaticity data (xy, CC or :
a*b*) is not converted by the luminance grayscale conversion
unit 18 and, accordingly, passes by the luminance grayscale
conversion unit 18 in step S20 as the non-converted chromaticity ;
data (CC or a*b*) as subjected to no conversion. In the next step S22, the grayscale-converted second luminance data (Y or L*) and the non-converted chromaticity data I
(CC or a*b*) as subjected to no conversion are inversely
^^ converted by the inverse conversion unit 20 of the second
grayscale conversion section 22 so as to generate RGB image data
(hereafter referred to as "fourth RGB image data").
In the next step S24, the fourth RGB image data as
generated by inverse conversion is subjected to RGB balance
adjustment by the RGB balance adjustment unit 24 using the third LUT (RGB balance adjustment LUT) 24a as a one-dimensional LUT
corresponding to the RGB pixel data, so as to generate RGB image
data on an output image (hereafter referred to as "third RGB
image data").
Finally in step S26, the generated third RGB image data is
Ai outputted as an output image from the output image data
outputting unit 26 to an external, image output device not shown, •
such as printing devices including printers and displaying
devices including monitors and display units. ;
The output image thus obtained is the image with a
desirable color reproduction in which the grayscale of the input
image has been converted (compressed/expanded) into the target
grayscale while the hue and the saturation are favorably :
maintained. I
The image processing method of another embodiment, namely,
the second embodiment of the present invention as shown in Figs. f
1
t
i
1 and 2B, in which the second grayscale conversion process is
followed by the first grayscale conversion process, is as
described below.
Initially in step S30 of Fig. 2B, a preprocessing step
similar to step S10 of Fig. 2A, the first, second and third LUTs
14a, 18a and 24a are set in the first grayscale conversion
section 14, the luminance grayscale conversion unit 18 and the
RGB balance adjustment unit 24, respectively. '
When the grayscale conversion as the image processing of the present invention is started, the first RGB image data on an
input image is firstly acquired by the input image data
^ acquiring unit 12 in step S32 of Fig. 2B, as in step S12 of Fig.
9 2A
Secondly in step S34, the first RGB image data is
colorimetrically converted (subjected to luminance and
chromaticity conversion) by the colorimetric conversion unit 16
of the second grayscale conversion section 22 so as to generate third luminance data (Y or L*) and chromaticity data (xy, CC or a*b*). •
In the next step S36, the third luminance data (Y or L*) is
subjected to the second grayscale conversion process by the
luminance grayscale conversion unit 18 of the second grayscale
^fe conversion section 22 using the second LUT (luminance LUT) 18a
so as to generate the fourth luminance data (Y or L*) as
grayscale-converted.
The chromaticity data (xy, CC or a*b*) passes by the
luminance grayscale conversion unit 18 in step S38 as the nonconverted
chromaticity data (CC or a*b*) as subjected to no conversion.
In the next step S40, the fourth luminance data (Y or L*) I
and the non-converted chromaticity data (CC or a*b*) are !
inversely converted by the inverse conversion unit 20 of the •
second grayscale conversion section 22 so as to generate RGB
image data (hereafter referred to as "fifth RGB image data").
14
I
I
In the next step S42, the fifth RGB image data is subjected ;
to the first grayscale conversion process by the first grayscale
conversion section 14 using the first LUT (RGB-LUT) 14a as a
one-dimensional LUT corresponding to the RGB pixel data, so as
to generate the third RGB image data on an output image that is
grayscale-converted with the RGB pixel data.
Finally in step S44, the generated third RGB image data is
outputted as an output image from the output image data
outputting unit 26 to an external, image output device not shown. The output image thus obtained is the image with a
^_ desirable color reproduction in which the grayscale of the input
^ image has been converted (compressed/expanded) into the target
grayscale while the hue and the saturation are favorably
maintained, as is the case with the first embodiment.
In the embodiments as described above, amounts obtained by
spectral dispersion of light into three channels of red, yellow, I
and blue are used as RGB data values of RGB image data (RGB pixel data), to which the present invention is not limited. Any j
amounts are available as long as they are obtained by spectral
dispersion of light into linearly independent three channels, or
values (such as y exponents or logarithms) obtained by performing '
^p, grayscale conversion on such amounts, XYZ data values on CIE XYZ
colorimetric system, for instance, may be used. In other words, I
in the present invention, RGB values may be defined as amounts
obtained by spectral dispersion of light into linearly
independent three channels or values obtained by performing i
grayscale conversion on such amounts. If XYZ image data is to be used as RGB image data, the »
colorimetric conversion in step S16 of the first embodiment as [
shown in Fig. 2A and step S34 of the second embodiment as shown in Fig. 2B is the conversion process in which Y as luminance }
data and xy as chromaticity data are calculated from the XYZ image data, and the inverse conversion in step S22 of the first f
n i
I
f
embodiment as shown in Fig. 2A and step S40 of the second
embodiment as shown in Fig. 2B is the conversion process in
which the XYZ image data is calculated from the luminance data Y
and the chromaticity data xy.
As described before, the first grayscale conversion section
14 is adapted to perform grayscale conversion in the RGB space
as the first grayscale conversion process. Also described before
is that the first grayscale conversion allows the change in gray
level without pixel data loss. If performed alone, however, the
first grayscale conversion will simultaneously cause the change
^- in hue and saturation, so that lighter colors as well as darker
^ colors are reduced in saturation during the compression of the
grayscale, for instance, which will make the color of an input
image appear differently upon output.
On the other hand, the luminance grayscale conversion unit
18 is adapted to perform grayscale conversion in the luminancecolor
difference area as the second grayscale conversion process,
as described before. The second grayscale conversion also allows
the change in gray level without pixel data loss. With the
second grayscale conversion alone, however, the reproduced color
will feel uncomfortable because the saturation is kept intact.
For this reason, in the present invention, the grayscale
£^ conversion amount as a target, the target compression amount for
instance, is distributed between the first grayscale conversion
amount of grayscale conversion in the RGB space and the second
grayscale conversion amount of grayscale conversion in the
luminance-color difference area so that the hue and the
saturation may be kept unchanged, and the saturation in
particular may not be so reduced as to cause uncomfortable
feeling.
In other words, it is possible in both the above first and i
second embodiments of the image processing method of the
invention to combine the first grayscale conversion (RGB
13
f
}
t
compression/expansion) based on the RGB-LUT which is the first
LUT 14a with the second grayscale conversion (luminance
compression/expansion) based on the luminance LUT which is the I
second LUT 18a, and perform weighting with respect to the first j
and second grayscale conversions, so as to accomplish the
grayscale conversion (compression/expansion) into the target grayscale and a favorable saturation reproduction in a
compatible manner with each other. As a result, the color jj
reproduction is desirably realized. In particular, a more desirable color reproduction can be realized by making the J
weighting for the first and second grayscale conversions
_ variable.
^^ The combination of the first and second grayscale
conversions with each other and the weighting for them may be
carried out so that the RGB compression/expansion and the
luminance compression/expansion as synthesized together may
correspond to the target grayscale. In the present invention,
the RGB-LUT and the luminance LUT may freely be designed
provided that the target grayscale is attained by the synthesis
of grayscale conversions. I
It is preferable in order to obtain an output image from an
input image to select in advance one of the flow sequences of
^k the image processing methods of the first and second embodiments
as shown in Figs. 2A and 2B, to which the present invention is
not limited. The two flow sequences of the first and second
embodiments may each be employed if the situation to which the
relevant flow sequence is adequate is automatically detected.
The combination of the first and second grayscale
conversions with each other and the weighting for them are
preferably performed in accordance with a particular situation i
or aim, such as photographing, printing, film shooting, and
displaying, or may be performed as required for such a situation
or aim.
I
I
I
The image processing method of the first embodiment as
shown in Fig. 2A is specifically described below in reference to
the practical case where grayscale compression is performed on
an input image (RGB image data).
It is assumed that the dynamic range (D.R.) of input has a
luminance ratio of logio (luminance range) = 6, namely 1,000,000:1,
and the output device has a luminance ratio of logio (luminance
range) = 4, namely 10,000:1. [
The D.R. should eventually be compressed from 6 to 4.
RGB values of image data are normally in the state where
the luminance undergoes y correction. It is preferable in that
£* case to undo y correction so as to convert the data into that in
the region of luminance.
r = R«1/T» :
g = G(1/Y>
b = B(1/y)
Fig. 3 shows a grayscale conversion curve for the dynamic
range compression from the input to the output. During grayscale
conversion, it is normal to use so-called S curve
characteristics having a slope of approximately one in the
region of medium luminosities (or brightness), and reduced
^^ slopes toward the white and black sides, respectively. A feature
of the characteristics is that they do not change the color
reproduction in the region of medium luminosities.
The curve of grayscale conversion from the input to the
output is preferably tuned while making evaluation using image
data for gray (R = G = B).
In this regard, the curve as designed is referred to as
"curve A."
In Fig. 3, the state of r = g = b is obtained by the output
gray with respect to the input gray (r = g = b), so that three
curves are depicted in one and the same manner. The output gray,
however, may be in the state of r * g ?s b. In that case, three
different curves may be produced for colors R, G and B,
respectively.
The total compression amount of the grayscale compression
as designed in the foregoing is distributed between the
compression amounts in the RGB space and in the luminance-color
difference area (space). The method of distributing the total
compression amount and the shape of the grayscale curve may be
selected at will as long as grayscale curve A as designed in the
foregoing is attained when the two grayscale conversions are
synthesized together. As a simple example, the method is
C described in which, as shown in Fig. 4, the ratio is linearly
changed between a straight line with no compression having a
slope of 45 degrees and represented with a dotted line and curve
A of compressed grayscale that is eventually to be obtained. In
the example as shown in Fig. 4, the grayscale compression whose !
amount is set at a half of the total grayscale compression
amount is represented with a thin solid line. -
This grayscale curve as adjusted to half compression is referred to as "curve B." As shown in Fig. 4, the dynamic range
(DR) of input of curve B is DR = 6, and its dynamic range (DR)
of output is DR = 5, an intermediate between the dynamic range
(DR) of output of the straight line with a slope of 45 degrees
^b as represented with a dotted line, 6, and the dynamic range (DR)
of output of curve A, 4.
While the compression amount of curve B may be achieved in the RGB space or the luminance-color difference space, the
description is made on the case where it is achieved in the RGB space. I
With grayscale conversion in the RGB space being performed j
by the first conversion processing section 14 as shown in Fig. 1 f
in step S14 of Fig. 2A, an LUT (lookup table) in the shape of [
curve B is conveniently produced as the first LUT 14a, and the I
first LUT 14a thus obtained is applied to image data on an input
image.
It is preferable to produce first LUTs in curve B shape for
colors R, G and B, respectively, so as to perform processing for
each color.
Next in step S16, the RGB signal (image data) as grayscaleconverted
in step S14 is converted by the colorimetric
conversion unit 16 of the second conversion processing section
22 into a signal in the luminance-color difference space in
order to perform the remaining compression in the luminancecolor
difference space.
-- Since the rgb values have already been made proportional to
the luminance by undoing y correction, they can be converted into 5
CIE tristimulus values XYZ by matrix operation.
If the RGB signal is in the sRGB color space, the RGB
signal is initially converted into an sRGB signal with the
following equations.
[Mathematical Formula 1]
If R, G, B>0. 03928
RsRGB = R/12. 92
GsRGB = G/12. 92
BsRGB = B/12. 92
m If R, G, B>0. 03928
RsRGB = {(R+0. 055)/1.055}2-4
GsRGB = {(G+0. 055)/1.055}2-4
BsRGB = {(B+0. 055)/1.055}2-4
From the obtained sRGB s i g n a l , r = RsRGB, g = GsRGB, and b
= BsRGB are found using the formula: :
[Mathematical Formula 2]
'Xl ["0,412 4 0357 6 0480 5 T ^ R G B ]
r = 0,212 6 0,715 2 0,072 2 G^QQ
Z\ [0,019 3 0,119 2 0^50 5j_5sRGBJ
so as to calculate XYZ.
I
In the XYZ color space, the luminance is denoted by Y, and
the chromaticity, namely x and y are calculated from:
x = X/(X + Y + Z); and
y = Y/(X + Y + Z) .
When grayscale conversion in the region of luminance is
performed in step S18, the luminance Y is converted by the
luminance grayscale conversion unit 18 of the second conversion
processing section 22, then X and Z are calculated anew so that
neither x nor y may be changed.
In step S18, L*a*b* may be used as a more uniform color
^ ^ space in order to suppress the change in chromaticity more
accurately. In that case, conversion from the XYZ colorimetric
system into the L*a*b* colorimetric system is carried out as
follows.
[Mathematical Formula 3]
L* =116Xf(Y/Yn)-16
a* =500X[f(X/Xn)-f(Y/Yn)]
b* =200X[f(Y/Yn)-f(Z/Zn)]
f(t)=ft1/3 :t>(6/29)3
L1/3X (29/6)2+4/29 : t being other than the above
^ In the equations, Xn, Yn, and Zn are normalizing values,
for which XYZ values of white are used.
In the case of the L*a*b* space, control may be exerted
such that L* is compressed and, at the same time, a* and b* are
free of change. In other words, with the value of Y being I
changed in accordance with L*, X and Z may be calculated anew so j
as to prevent both a* and b* from being changed. {
The curve of grayscale conversion based on the luminance I
may be obtained as follows. j
The output RGB of each of grayscale curves A and B is converted into that in the L*a*b* space. I
22>
i
i
Fig. 5 is a graph showing the relationship between L*s for
the two curves.
As shown in Fig. 5, the relationship is represented by one
curve, and the curve in itself serves as an LUT used to compress
L*. The curve is referred to as "curve C."
As shown in Fig. 5, the dynamic range (DR) of input of
curve C (output L* for curve B) is the dynamic range (DR) of
output of curve B, namely DR = 5, while the dynamic range (DR)
of output is the dynamic range (DR) of output of curve A, namely
DR = 4 .
By synthesizing Fig. 4 as above and Fig. 5 together, the
—~ curve is obtained of which the dynamic range (DR) of input is 6
^ ^ and the dynamic range (DR) of output is 4, the same dynamic
ranges (DRs) of input and output as those of curve A which are 6
and 4, respectively.
In the next step S22, the luminance signal (luminance data)
as grayscale-converted in step S18 and the non-converted
chromaticity signal are inversely converted by the inverse •
conversion unit 20 of the second conversion processing section
22 into a signal in the RGB space.
Finally in step S24, the RGB signal (image data) as
obtained in step S22 by inverse conversion is adjusted in RGB
^fe balance by the RGB balance adjustment unit 24.
In other words, the output is finely adjusted in RGB value
balance to thereby make it consistent with the input gray.
To be more specific: Gray data showing r = g = b is
inputted to curve B which is produced as above, then the output '
value is converted in the L*a*b* and L* is processed with curve
C. Y is obtained from L* as processed, and X and Z are obtained I
so that neither a* nor b* may be changed. X, Y and Z as obtained are converted into R, G and B. Figs. 6A, 6B, and 6C are graphs showing R, G and B as
obtained by following the above procedure on the x-axes, and R,
G and B outputted when the gray of r = g = b is inputted to 2 j
I
I
In the case as shown, it is preferable that the first-stage
compression based on RGB is followed by the luminance compression based on YCC because the control in saturation needs [
to be performed in the region of higher saturations and is, j
accordingly, hard to perform if the luminance compression based [
on YCC is initially performed. Thus in the case as shown, the
luminance compression based on YCC is performed between the j
first-stage and second-stage compressions based on RGB, to which t
the present invention is not limited. The compression based on j
RGB and the luminance compression based on YCC may be determined l
in number of times and in order as appropriate to the color
^^ space of a scene in an input image and the color space as a
reproduction target. !
The ratio of the compression amount of each of the firststage
compression based on RGB by the first compression unit 80, I
the luminance compression based on YCC by the second compression f
unit 82, and the second-stage compression based on RGB by the I
third compression unit 84 to the total grayscale compression amount is set in advance by the ratio setting unit 7 6 in the f
relevant compression unit. In the case as shown, it is preferable that the compression amount of the first-stage compression based on RGB is the I
^b largest and the compression amount of the following luminance
compression based on YCC is the smallest, to which the present !
invention is not limited. The ratio between compression amounts )
of different compressions may be determined as appropriate to a scene in an input image and a reproduction target according f
thereto. =
The image processing device 10 of the embodiment as shown in Fig. 1 as well as the image processing methods of the first [
and second embodiments as shown in Figs. 2A and 2B, respectively, are each adapted to obtain an output image by subjecting an input image to the first grayscale conversion (RGB grayscale
I
t
I
I
conversion) and the second grayscale conversion (luminance
grayscale conversion) as weighted, to which the present
invention is not limited. Although not shown, the weighting with respect to the first and second grayscale conversions may also {
be performed in order to synthesize a conversion characteristic I
curve used for subjecting image data on an image to grayscale
conversion, and convert an input image into an output image
using the synthesized conversion characteristic curve, or in
order to change a conversion curve used for converting a f
specified range of grayscale characteristics of image data on an (
image.
^* In the embodiments as described above, during the second i
grayscale conversion, the luminance data or lightness data is {
subjected to luminance grayscale conversion while the [
chromaticity data is made to pass by, to which the present i
invention is not limited. It is also possible to change the I
chromaticity data. If the chromaticity is to be changed, CC or a*b* may be 1
t
changed in step 38 of Fig. 2B or step S60 of Fig. 8. Change is I
possible by using equations: Cr' = Cr * kl and Cb* = Cb * k2 (kl [
and k2 being coefficient parameters) for CC (Cr, Cb) for instance, or equations: a*' = a* + k3 and b*' = b* + k4 (k3 and ••
A ) k4 being coefficient parameters).
The methods of changing Cr and Cb, as well as a* and b* are not limited to the above, that is to say, other operation I
s;
expressions or LUTs are also available. J
I
The image processing methods as described above are able to •
be implemented on a computer by running an image processing :
program. '.
For instance, the image processing program of the present invention includes procedures for making a computer, ;
specifically a CPU thereof, implement the individual steps of
the image processing methods as above. The program consisting of
I
I
such procedures may be constructed as one or more program f
modules.
The image processing program which consists of the I
procedures to be implemented by a computer may be stored in a I
memory (storage unit) of a computer or a server, or in a recording medium. Upon implementation, the program is read out }
of the memory or recording medium and implemented by the J
computer (CPU) of interest or other computer. Consequently, the I
i
present invention can provide the computer-readable memory or |
recording medium in which the image processing program for |
making a computer implement the image processing method of the (
above first embodiment is stored. !
The present invention is fundamentally as described above. While the image processing method and device, as well as the i
image processing program and the medium for recording the I
program, all according to the present invention, have been I
detailed above, the present invention is in no way limited to }
the above embodiments. It is needless to say that various I
«
improvements or modifications are possible within the gist of |
the present invention. !
LEGENDS |
10 Image processing device
12 Input image data acquiring unit i
14 First grayscale conversion section |
16 Colorimetric conversion unit 18 Luminance grayscale conversion unit [
20 Inverse conversion unit i
22 Second grayscale conversion section [
24 RGB balance adjustment unit i
2 6 Output image data outputting unit I
28 Memory 2>£
[Claim 1]
An image processing method adapted to convert grayscale
characteristics of image data of an image, comprising:
first grayscale conversion which performs first grayscale
characteristics conversion using a first lookup table of onedimensional
type corresponding to RGB pixel data of the image;
and
second grayscale conversion which converts the RGB pixel
data of the image into luminance data or lightness data and
# chromaticity data, and performs second grayscale characteristics
conversion on the thus converted luminance data or lightness
data using a second lookup table of one-dimensional type
corresponding to the luminance data or lightness data,
wherein weighting to the first grayscale conversion and the
second grayscale conversion is performed.
[Claim 2]
The image processing method according to claim 1, wherein
said luminance data or lightness data is Y on CIE XYZ
colorimetric system or a value determined from the Y.
[Claim 3]
The image processing method according to claim 1 or 2,
^^ wherein said chromaticity data is a value determined from Cr and
Cb on YCC color system or from X, Y and Z on XYZ colorimetric
system.
[Claim 4]
The image processing method according to any one of claims
1 through 3,
wherein said first grayscale conversion is carried out
prior to said second grayscale conversion,
wherein said first grayscale conversion performs said first
grayscale characteristics conversion on first RGB pixel data on
an input image to obtain second RGB pixel data, and
wherein the second grayscale conversion
S7
converts the thus obtained second RGB pixel data into first
luminance data or lightness data and said chromaticity data,
performs said second grayscale characteristics conversion
on the thus converted first luminance data or lightness data to
obtain second luminance data or lightness data, and
converts the thus obtained second luminance data or
lightness data and the chromaticity data into third RGB pixel
data of an output image.
[Claim 5]
The image processing method according to claim 4, wherein
said second grayscale conversion
converts said second luminance data or lightness data and
^^ said chromaticity data into fourth RGB pixel data, and
performs balance adjustment on the thus converted fourth
RGB pixel data to obtain said third RGB pixel data of said
output image.
[Claim 6]
The image processing method according to any one of claims
1 through 3,
wherein said second grayscale conversion is carried out
prior to said second grayscale conversion,
wherein said second grayscale conversion
converts first RGB pixel data of an input image into third
^b luminance data or lightness data and said chromaticity data,
performs said second grayscale characteristics conversion
on the thus converted third luminance data or lightness data to
obtain fourth luminance data or lightness data, and
converts the thus obtained fourth luminance data or
lightness data and the chromaticity data into fifth RGB pixel
data, and
wherein the first grayscale conversion performs said first
grayscale characteristics conversion on the thus obtained fifth
RGB pixel data to obtain third RGB pixel data of an output image.
[Claim 7]
The image processing method according to any one of claims
1 through 3,
wherein said first grayscale conversion is carried out in
parallel with said second grayscale conversion,
wherein said first grayscale conversion performs said first
grayscale characteristics conversion on first RGB pixel data of
an input image to obtain second RGB pixel data,
wherein the second grayscale conversion
converts the first RGB pixel data of the input image into
third luminance data or lightness data and said chromaticity
data,
performs said second grayscale characteristics conversion
on the thus converted third luminance data or lightness data to
^^ obtain fourth luminance data or lightness data, and
converts the thus obtained fourth luminance data or
lightness data and the chromaticity data into fifth RGB pixel
data, and
the weighting to the first grayscale conversion and the
second grayscale conversion is carried out such that weighted
addition is performed between the second RGB pixel data obtained
by the first grayscale conversion and the fifth RGB pixel data
obtained by the first grayscale conversion, so as to obtain
third RGB pixel data of an output image.
[Claim 8]
^k The image processing method according to any one of claims
1 through 7, wherein said weighting to said first grayscale
conversion and said second grayscale conversion is performed by
using images of a color chart that have different luminosities
to determine a ratio between amounts of said first grayscale
characteristics conversion and of said second grayscale
characteristics conversion.
[Claim 9]
The image processing method according to claim 8, wherein
the ratio between the amounts of said first grayscale
characteristics conversion and of said second grayscale
characteristics conversion is determined by processing said
21
images that have different luminosities while varying the ratio
between the amounts of the first grayscale characteristics
conversion and of the second grayscale characteristics
conversion, and calculating chromaticity on a color space to
confirm saturation.
[Claim 10]
The image processing method according to claim 9, wherein
said chromaticity on the color space is a*b* chromaticity on CIE
L*a*b* color space.
[Claim 11]
The image processing method according to any one of claims
8 through 10, wherein said images that have different
^•^ luminosities are obtained by photographing said color chart with
a camera at different exposures or at different luminosities of
light with which the color chart is illuminated or by
calculating, from an image of the color chart, another image to
be obtained at a different luminosity.
[Claim 12]
An image processing device adapted to convert grayscale
characteristics of image data on an image, comprising:
a first grayscale conversion means for performing first
grayscale characteristics conversion using a first loo]<;up table
of one-dimensional type corresponding to RGB pixel data of the
J^ image; and
a second grayscale conversion means for converting the RGB
pixel data of the image into luminance data or lightness data
and chromaticity data, and performing second grayscale
characteristics conversion on the thus converted luminance data
or lightness data using a second lookup table of one-dimensional
type corresponding to the luminance data or lightness data,
wherein weighting to the first grayscale characteristics
conversion and the second grayscale characteristics conversion
is performed.
[Claim 13]
An image processing program for making a computer implement
individual steps of the image processing method according to anyone
of claims 1 through 11.
[Claim 14]
A computer-readable recording medium in which an image
processing program for making a computer implement individual
steps of the image processing method according to any one of
claims 1 through 11 is stored.