DESCRIPTION
STEREOSCOPIC VIDEO PROCESSING APPARATUS, METHOD, AND PROGRAM
TECHNICAL FIELD
5 [0001]
The present invention relates to a stereoscopic video
processing apparatus, method, and program and, in particular,
to a stereoscopic video processing apparatus, method, and
program that can reduce visual fatigue on watching a
10 stereoscopic video.
BACKGROUND ART
[0002]
Humans stereoscopically recognize an object by using
15 a difference (parallax) between an image caught by the right
eye and an image caught by the left eye. The rotational
movements of the eyeballs change the angle of vergence, and
humans recognize this as a distance to an object. The angle
of vergence is an angle formed by two lines of vision which
20 cross each other.
[0003]
When two two-dimensional images for the left eye and
the right eye, which have a parallax therebetween, are prepared
by using characteristics of the eyes of humans, and then
25 separately projected on the left eye and the right eye, the
distance to an object is deluded due to the angle of vergence.
This gives humans stereoscopic perception. Here, the
parallax is a difference between an image for the left eye
and an image for the right eye.
30 [0004]
An image obtained by displaying an image for the left
1
SP262577WO00
eye and an image for the right eye is called a stereoscopic
image. An image obtained by preparing a plurality of images
for the left eye and a plurality of images for the right eye
and continuously changing the prepared images is called a
5 stereoscopic video. Moreover, an apparatus that can display
the stereoscopic video is called a stereoscopic video display
apparatus.
[0005]
Further, a stereoscopic video display apparatus has been
10 proposed which alternately displays images for the right eye
and images for the left eye on a display in such a manner that
a parallax exists therebetween by using shutter glasses in
which a pair of left and right lens units is alternately switched
between a perspective state and a light shielding state. This
15 apparatus does not require a switching operation of the shutter
glasses because the left and right lens units are controlled
to be alternately switched between the perspective state and
the light shielding state in synchronization with timing at
which the images for the right eye and the images for the left
20 eye are alternately switched (for example, refer to Patent
Document 1).
CITATION LIST
PATENT DOCUMENT
25 [0006]
Patent Document 1: Japanese Patent Application Laid-Open No.
2001-320734
SUMMARY OF THE INVENTION
30 PROBLEMS TO BE SOLVED BY THE INVENTION
[0007]
2
SP262577WO00
However, for example, when a scene change occurs, a
temporal discontinuity in parallax change tends to occur in
a stereoscopic video because quite different images are
continuously displayed.
5 [0008]
For example, it is assumed that a two-dimensional object
without a parallax is displayed before the scene change and
a stereoscopic object with a parallax is displayed after the
scene change. In this case, since the angle of vergence a
10 to see the two-dimensional object and the angle of vergence
P to see the stereoscopic object are greatly different, a viewer
is required to rotate the eyeballs and to rapidly change the
angle of vergence.
[0009]
15 The adjustment of the angle of vergence that follows
the rapid change in parallax was considered the cause of visual
fatigue, and the conventional art could not solve the problem,
for example, the visual fatigue attributable to the scene
change.
20 [0010]
The present invention is made in view of such as itmiLion,
and is intended to reduce visual fatigue when watching a
stereoscopic video.
25 SOLUTIONS TO PROBLEMS
[0011]
An aspect of the present invention is a stereoscopic
video processing apparatus including; a frame specifying unit
that specifies a frame that causes a temporal discontinuity
30 in change of maximum parallax value among frames of a
stereoscopic video signal that includes frame rate for a unit
3
SP262577WO00
time; a maximum parallax specifying unit that specifies a
maximum parallax value of a first frame that precedes a
reference frame by a predetermined time when the specified
frame is set as the reference frame, and a maximum parallax
5 value of a second frame that lags behind the reference frame
by the predetermined time; a parallax coefficient calculating
unit that sets frames between the first frame and the second
frame as a processing target frame and calculates a parallax
coefficient that is coefficient used to adjust the maximum
10 parallax value of the processing target frame such that changes
in the maximum parallax value are continuous over time based
on the maximum parallax value of the first frame, the maximum
parallax value of the second frame, and the maximum parallax
value of the processing target frame; and a parallax adjusting
15 unit that adjusts the maximum parallax value of the processing
target frame by multiplying the maximum parallax value by the
calculated parallax coefficient.
[0012]
A parallax of each of the frames of the stereoscopic
20 video signal may be information contained in each of the frames
of the stereoscopic video signal, and may be specified based
on a parallax plane that indicates a parallax for each pixel
[0013]
-
The parallax of each of the frames of the stereoscopic
25 video signal may be specified by calculating a difference in
pixel between right eye image data and left eye image data
that, are contained in each frame of the stereoscopic video
signal.
[0014]
30 The frame specifying unit may specify the frame which
causes a temporal discontinuity in change of the maximum
4
SP262577WO00
parallax value by detecting a scene change frame in a video
of the stereoscopic video signal.
[0015]
The frame specifying unit may specify the frame which
5 causes a temporal discontinuity in change of the maximum
parallax value based on a difference in the maximum parallax
value between two frames that are temporally consecutive.
[0016]
An aspect of the present invention is a method of
10 processing a stereoscopic video, the method including: at a
frame specifying unit, specifying a frame which causes a
temporal discontinuity in change of maximum parallax value
among frames of a stereoscopic video signal that has frame
rate for each unit time; at a maximum parallax specifying unit,
15 setting the specified frame as a reference frame, and
specifying a maximum parallax value of a first frame that
precedes the reference frame by a predetermined time and a
maximum parallax value of a second frame that lags behind the
reference frame by the predetermined time; at a parallax
20 coefficient calculating unit, setting a frame between the first
frame and the second frame as a processing target frame, and
calculating a parallax coefficient that is coefficient used
to adjust a maximum parallax value of the processing target
frame so that changes in the maximum parallax value are
25 temporally continuous, based on the maximum parallax value
of the first frame, the maximum parallax value of the second
frame, and the maximum parallax value of the processing target
frame; and at a parallax adjusting unit, adjusting the maximum
parallax value of the processing target frame by multiplying
30 the maximum parallax value of the processing target frame by
the calculated parallax coefficient.
5
SP262577WO00
[0017]
An aspect of the present invention is a program causing
a computer to function as a stereoscopic video processing
apparatus, the stereoscopic vide processing apparatus
5 including: a frame specifying unit that specifies a frame that
causes a temporal discontinuity in change of maximum parallax
value among frames of a stereoscopic video signal that includes
frame rate for a unit time; a maximum parallax specifying unit
that specifies a maximum parallax value of a first frame that
10 precedes a reference frame by a predetermined time and a maximum
parallax value of a second frame that lags behind the reference
frame by the predetermined time when the specified frame is
set as the reference frame; a parallax coefficient calculating
unit that sets frames between the first frame and the second
15 frame as a processing target frame and calculates a parallax
coefficient that is coefficient used to adjust the maximum
parallax value of the processing target frame such that changes
in the maximum parallax value are continuous over time based
on the maximum parallax value of the first frame, the maximum
20 parallax value of the second frame, and the maximum parallax
value of the processing target frame; and a parallax ad j lasting
unit that adjusts the maximum parallax value of the processing
target frame by multiplying the maximum parallax value by the
calculated parallax coefficient.
25 [0018]
According to one aspect of the present invention, a frame
which causes a temporal discontinuity in change of maximum
parallax value is specified among frames of a stereoscopic
video signal that includes frame rate for a unit time, the
30 specified frame is set as a reference 'frame, a maximum parallax
value of a first frame that precedes the reference frame by
6
SP262577WO00
a predetermined time and a maximum parallax value of a second
frame that lags behind the reference frame by the predetermined
time are specified, frames between the first frame and the
second frame are set as processing target frames, parallax
5 coefficients are calculatedbased on the maximumparallax value
of the first frame, the maximum parallax of the second frame,
and maximum parallax values of the processing target frames
in which the parallax coefficients are coefficient used to
adjust the maximum parallax values of the processing target
10 frames such that changes in the maximum parallax value are
continuous over time, and the maximum parallax values of the
processing target frames are adjusted by being multiplied by
the calculated corresponding coefficients.
15 EFFECTS OF THE INVENTION
[0019]
According to the present invention, visual fatigue on
watching a stereoscopic video can be reduced.
20 BRIEF DESCRIPTION OF DRAWINGS
[0020]
Fig. 1 is a diagram that describes the mechanism that
humans' eyes recognize a stereoscopic video.
- Fig. 2 is a diagram that illustrates an example of an
25 image for the left eye and an image for the right eye that
have a parallax therebetween.
Fig. 3 is a diagram that illustrates an example in which
a stereoscopic video illustrated in Fig. 2 is observed by
humans.
30 Fig. 4 is a diagram that illustrates a change in parallax
over time as a graph.
7
SP262577W000
Fig. 5 is a diagram that describes a change in angle
of vergence of the lines of vision when a stereoscopic video
is observed.
Fig. 6 is a block diagram that illustrates an example
5 of a configuration of a stereoscopic video display system
according to an embodiment of the present invention.
Fig. 7 is a block diagram that illustrates an example
of a detailed configuration of a parallax processing unit
illustrated in Fig. 6.
10 Fig. 8 is a diagram that illustrates an example of a
configuration of a frame of a stereoscopic' video signal.
Fig. 9 is a diagram that illustrates an example of a
scene change flag sequence.
Fig. 10 is a diagram that illustrates right eye planes,
15 left eye planes, and parallax planes of a frame with frame
number P®180 and a frame with frame number P+180.
Fig. 11 is a graph that illustrates changes in the maximum
parallaxvalue of a stereoscopic video signal before processing
by the parallax processing unit is performed.
20 Fig. 12 is a graph that illustrates changes in the maximum
parallax value of the stereoscopic video signal after the
processing by the parallax processing unit is performed.
Fig. 13 is a diagram that illustrates an example of a
righ-t eye plane, a left eye plane, and a parallax plane before
25 the processing by the parallax processing unit is performed.
Fig. 14 is diagram that illustrates an example of a right
eye plane, a left eye plane, and a parallax plane after the
processing by the parallax processing unit is performed.
Fig. 15 is a flowchart that describes an example of
30 processing of correcting a stereoscopic video signal.
Fig. 16 is a block diagram that illustrates an example
8
SP262577WO00
of a configuration of a personal computer.
MODE FOR CARRYING OUT THE INVENTION
[0021]
5 Hereafter, embodiments of the present invention will
be described with reference to the accompanying drawings.
[0022]
First, the display mechanism of a stereoscopic video
is described.
10 [0023]
Fig. 1 is a diagram that describes the mechanism that
human being's eyes recognize a stereoscopic video.
[0024]
As illustrated in the same drawing, humans
15 stereoscopically recognize an object using a difference
(parallax) between an image captured by the left eye and an
image captured by the right eye. In the example of Fig. 1,
the same object of a circular shape appears in both of the
image captured by the left eye and the image captured by the
20 right eye, but is displayed in different positions in the
images.
[0025]
To focus the lines of vision on an object to which a
human being pays attention, the eyeballs of the right eye and
25 the left eye will be rotated. The rotational movements of
the eyeballs change the angle of vergence, and the human being
recognizes the distance to the object by the angle of vergence.
As a result, the human being recognizes a stereoscopic video.
Here, the angle of vergence is an angle formed by the lines
30 of vision that cross each other.
[0026]
9
SP26257'7W000
Therefore, for example, when a stereoscopic video is
displayed on a two-dimensional display unit, the display may
be achieved by preparing an image for the right eye and an
image for the left eye that have a parallax therebetween. As
5 illustrated in Fig. 2, for example, when an image for the right
eye and an image for the left eye that have a parallax
therebetween are displayed on a two-dimensional display unit
at the same time, a triangular object in the diagram can be
stereoscopically displayed (for example, the object appears
10 to project out from the display unit) due to the parallax.
[0027]
Further, since an image for the right eye and an image
for the left eye are displayed, for example, in red light and
blue light, respectively in a superimposed manner and are
15 separated by glasses to which a red filter and a blue filter
are respectively attached, they are observed by the left eye
and the right eye of humans.
[0028]
Fig. 3 is a diagram that illustrates an example in which
20 the stereoscopic video illustrated in Fig. 2 is observed by
humans. As illustrated in the same drawing, the line of vision
of the left eye is focused on the almost center of an object
in an image for the left eye and the line of vision of the
right eye is focused on the almost center of an object in an
25 image for the right eye. This gives a sensation that the object
appears to project out at the position where the line of vision
of the left eye and the line of vision of the right eye, which
are illustrated with dotted lines, cross each other.
[0029]
30 However, for example, when a scene change occurs in the
stereoscopic video, a temporal discontinuity in parallax
10
SP262577W000
change is likely to occur because quite different images are
continuously displayed.
[0030]
Fig. 4 is a diagram that illustrates changes in parallax
5 over time as a graph in which the horizontal axis indicates
time, and the vertical axis represents a maximum parallax value
at each point in time in a stereoscopic video. in the example
of the same drawing, the scene change occurs at time point
t1.
10 [0031]
In the graph of Fig. 4, the discontinuous change occurs
at time point t1. That is, the temporal discontinuity in
parallax change occurs in the stereoscopic video.
[0032]
15 The adjustment of the angle of vergence attributable
to such a rapid parallax change is thought to be accompanied
by visual fatigue. To simplify the description, the scene
change is assumed such that, for example, in a video, a scene
in which an object being paid attention to is displayed
20 two-dimensionally is changed to a scene in which the object
is stereoscopically displayed.
[0033]
For example , in the scene in which the object is
two-dimensionally displayed as illustrated in Fig. 5, the
25 eyeballs of the left eye and the right eye of a human being
are turned in the direction in which the angle of vergence
between the lines of vision becomes a. When the scene change
occurs immediately after that and thus the scene is switched
to the scene in which the object is stereoscopically displayed,
30 the eyeballs of the left eye and the right eye are rotated
so that the angle of vergence between the lines of vision lines
11
SP262577WO00
becomes P. In this way, the rapid rotation of the eyeballs
results in the visual fatigue.
[0034]
The study on the change in the angle of vergence and
5 the visual fatigue is reported in articles , such as "An
Ergonomic Evaluation System for Stereoscopic 3-D images by
Shinsuke Kishi et al ., The Journal of The Institute of Image
Information and Television Engineers Vol. 60 ( 2006 ) No. 6 pp.
934-942 ", and "3-D images and Human Science by Hiroshi
10 Harashima et al., Ohmsha , Ltd., 2000."
[0035]
Accordingly, the present invention prevents the
temporal discontinuity in parallax change from occurring in
a stereoscopic video.
15 [0036]
Fig. 6 is a block diagram that illustrates an example
of a configuration of a stereoscopic video display system
according to an embodiment of the present invention. As
illustrated in the same drawing., a stereoscopic video display
20 systemlO is configured to include a stereoscopic video display
device 21 and a display 22.
[0037]
The stereoscopic video display device 21 is configured
to include a stereoscopic video signal acquiring unit 41 and
25 a parallax processing unit 42 inside thereof.
[0038]
The stereoscopic video signal acquiring unit 41 is
configured to include a unit, such as a tuner, which receives
a stereoscopic video signal transmitted as a broadcast wave.
30 Moreover, the stereoscopic video signal acquiring unit 41 may
be configured by using a unit, such as a drive, which reads
12
SP262577WO00
a stereoscopic video signal recorded in a recording medium,
such as a DVD or an HDD.
[0039]
The parallax processing unit 42 is assumed to be a
5 functional block that performs processing of correcting a
signal, e.g., the stereoscopic video signal acquired by the
stereoscopic video signal acquiring unit 41 so that there is
no temporal discontinuity in parallax change in astereoscopic
video.. The configuration of the parallax processing unit 42
10 is described later in detail with reference to Fig. '7.
[0040]
The display 22 is configured by using an LCD (Liquid
Crystal display) or the like and is assumed to have a
two-dimensional image display area. The user is assumed to
15 watch the stereoscopic video displayed on the display 22 while
wearing, for example, glasses provided with color filters,
or the like.
[0041]
Moreover, the display 22 may be configured to include
20 a screen or the like and the stereoscopic video projected from
the stereoscopic video display device 21 may be displayed on
the screen. In addition, the stereoscopic video display
system l0 maybe configured in the form in which the stereoscopic
video display device 21 and the display 22 are integrally
25 formed.
[0042]
Fig. 7 is a block diagram that illustrates an example
of a detailed configuration of the parallax processing unit
42 of Fig. 6. As illustrated in the same drawing, the parallax
30 processing unit 42 is configured to include a scene change
detecting unit 61 and a parallax adjusting unit 62.
13
SF262577WO00
[0043]
The scene change detecting unit 61 specifies a scene
change frame, at which the scene changes, from the stereoscopic
video signal. The scene change detecting unit 61 accumulates,
5 in an internal buffer or the like, frames of the stereoscopic
video signal corresponding to, for example, a stereoscopic
video for three seconds and specifies the scene change frame.
[0044]
Fig. 8 is a diagram that describes a configuration of
10 a frame of the stereoscopic video signal. The stereoscopic
video signal includes frames. Fig. 8 illustrates information
contained in one frame of the stereoscopic video signal.
Moreover, the stereoscopic video signal that is a signal to
display a moving image is assumed to include 60 frames in a
15 second. That is, the stereoscopic video for one second can
be displayed by the stereoscopic video signal corresponding
to 60 frames.
[0045]
In the example of Fig. 8, the frame of the stereoscopic
20 video signal includes a right eye plane, a left eye plane,
a parallax plane, and a scene change flag. Moreover, besides
those, information such as a frame header, an error detection
bit, and the like is further contained as appropriate.
[0046]
25 The right eye plane and the left eye plane are considered
as image data for one screen to be displayed on the display
22. In addition, in the example of Fig. 8, to simplify the
description, the image for one screen. to be displayed on the
display 22 is assumed to include 64 (=,8 x 8) pixels, and each
30 rectangle in the right eye plane and the left eye plane
represents one pixel.
14
SP262577W000
[0047]
Each of the right eye plane and the left eye plane includes
64 rectangles, and "0" or "1" is recorded in each rectangle.
Here, to simplify the description, the rectangles with "0"
5 in the diagram are assumed to represent black pixels and the
rectangles with "1" in the diagram are assumed to represent
white pixels.
[0048]
In the example of Fig. 8, illustrated is an image in
10 which an object of 2 x 4 white pixels is displayed on the black
background, and the object of 2 x 4 white pixels is displayed
in different positions in the right eye plane and the left
eye plane.
That is, as previously described with reference to Fig. 3,
15 images having a parallax are displayed and thus a sensation
that the object appears to project out from the screen is
obtained.
[0049]
The difference in the display position between the right
20 eye plane and the left eye plane can be specified by the parallax
plane. The parallax plane has 64 rectangles corresponding
to the respective pixels of each of the right eye plane and
the left eye plane, and is assumed to be information that
represents by how many pixels the right eye plane is shifted
25 from the left eye plane. In the example of Fig. 8, "2" is
recorded in each of the rectangles at the positions where 2
x 4 white pixels forming the object are displayed in the right
eye plane. This implies that the object of 2 x 4 white pixels
in the right eye plane is displaced by two pixels with respect
30 to the left eye plane.
[0050]
15
SP262577W000
The scene change flag is a flag for confirmation of the
occurrence of the scene change. For example, the scene change
flag is made up of a bit representing "0" or "1". When the
scene change flag represents "1", it implies that the scene
5 change occurs in the following frame. In the example of Fig.
8, since "0" is recorded as the scene change flag, the example
implies that the scene change does not occur in the following
frame.
[0051]
10 Although not illustrated in Fig. 8, each of the frames
of the stereoscopic video signal is given a frame number that
shows the order of the frame when counted from the leading
frame.
[0052]
15 As described above, the scene change detecting unit 61
accumulates, in an internal buffer or the like, the frames
of the stereoscopic video signal corresponding to, forexample,
a stereoscopic video for three seconds and specify a scene
change frame. That is, the frames shown in Fig. 8 are
20 accumulated in 180 (= 3 x 60) buffers. The scene change.
detecting unit 61 extracts the scene change detection flags
from the 180 frames that are accumulated, and generates a scene
change flag sequence. Fig. 9 is a diagram that illustrates
an example of the scene change flag sequence.
25 [0053]
In the example of Fig. 9, the scene change flag sequence
is generated by associating the scene change detection flags
with the frame numbers, respectively. In addition, the
leading frame (that is, the frame at the beginning of the image)
30 is assumed to be named frame number'0. In this example, the
scene change flag sequence is generated with scene change
16
SP262577WO00
detection flags of 180 frames (corresponding to three seconds)
named frame number 0 to 179.
[0054]
In the example of Fig. 9, since the scene change detection
5 flag corresponding to frame number 8 is set to "1", it can
be understood that the scene changes at the frame with frame
number 9. That is, the scene shown in the images of the frames
with frame number 0 to 8 is different from the scene shown
in the images of the frames with frame number 9 to 179.
10 [0055]
The parallax adjusting unit 62 acquires the frames of
the stereoscopic video signal based on the frame number given
to the scene change frame specified by the scene change
detecting unit 61, and accumulates them in the internal buffer
15 or the like. Here, the scene change frame specified by the
scene change detecting unit 61 is the frame of which the scene
change detection flag is "1", and is the frame with frame number
8 in the example of Fig. 9.
[0056]
20 For example, when the frame number given to the frame
specified by the scene change detecting unit 61 is assumed
to be P, the frame with frame number P-180 and the frame with
frame number P+180 are acquired and accumulated (stored).
That is, a frame that precedes the scene change frame by three
25 seconds and a frame that lags behind the scene change frame
by three seconds are acquired.
[0057]
In addition, for example, as in the example illustrated
in Fig. 9, when the frame with frame,number P-180 does not
30 exist, for example, the frame with frame number 0 may be acquired
instead of that frame.
17
SP262577W000
[0058]
The parallax adjusting unit 62 specifies the maximum
parallax value in the two frames mentioned above. That is,
the maximum value among numerical values in the parallax planes
5 of the respective frames is specified. Here, the maximum
parallax value of the frame with frame number P-180 is assumed
to be Db, and the maximum parallax value of the frame with
frame number P+180 is assumed to be Da.
[0059]
10 Next, the parallax adjusting unit 62 calculates a
parallax coefficient a that is coefficient used to adjust the
maximum parallax value of a processing target frame of signal
correction processing such that there is no temporal
discontinuity in parallax change, by using Expression. (1)
15 [0060]
[Expression 1]
a- S¢in (1)
[0061]
Wherein, in Expression (1), Pc is the frame number for
20 the processing target frame, and Dc is the maximum parallax
value of the frame with the frame number Pc. Moreover, F in
Expression (1) is a value which represents the number of frames
for one second, and the value of F in the previously described
example becomes 60.
25 [0062]
The parallax coefficient a I's calculated for each
processing target frame.
That is, for each of the frames, from the frame with frame
number P'180 to the frame with frame number P+180, the parallax
18
SP262577W000
coefficient a is calculated individually.
[0063]
For example, it is assumed that right eye planes, left
eye planes, and parallax planes of the frames with frame number
5 P-180 and frame number P+180 are illustrated in Fig. 10, The
parallax processing unit 42 specifies the maximum value among
numerical values of the parallax planes of the two frames as
the maximum parallax value. in this case, the value of Da
becomes 4 and the value of Db becomes 2.
10 [0064]
When the values are substituted for F, 'Da, and Db into
Expression (1), Expression (2) is obtained.
[0065]
[Expression 2]
15 a®a(P-(P 1B)+a
a6ODc (2)
[0066]
The parallax adjusting unit 62 adjusts the maximum
parallax of each of the frames, from the frame with frame number
P-180 to the frame with frame number P+180, by using the pa i allax
20 coefficients a that have been calculated in the previously
described manner. For example, the frame number of the frame
at which the scene change actually occurs is substituted into
P in Expression (2) described above, the frame number of the
processing target frame is substituted into Pc, and the maximum
25 parallax value is substituted into Dc, so that a is calculated
for each of the frame numbers respectively provided for the
processing target frames.
[0067]
Next, the parallax adjusting unit 62 adjusts (corrects)
19
SP262577WO00
the maximum parallax of the frame with frame number Pc by
multiplying a by the value of Dc. In addition, the frames,
from the frame with frame number P-180 to the frame with frame
number P+180, are assumed to be processing target frames, and
5 each of the parallax coefficients a that are individually
calculated is multiplied by the maximum parallax value of each
frame.
[0068]
Referring to Figs. 11 and 12, the description is made
10 further in detail. To simplify the description, the maximum
parallax value of each of the frames, from the frame with frame
number 0 to the frame with frame number P, is assumed to be
always 2, and the maximum parallax value of each of the frames,
from the frame with frame number P+1 to the last frame, is
15 assumed to be always 4.
[0069]
Fig. 11 is a graph that illustrates changes in the maximum
parallax value of the stereoscopic video signal before the
processing performed by the parallax processing unit 42 is
20 performed. In the same drawing, the horizontal axis indicates
frame numbers, the vertical axis indicates values of paraillax,
and the scene change frame is the frame with frame number P.
As illustrated in the same drawing, changes in the maximum
parallax value are discontinuous between the frame with frame
25 number P and the frame with frame number P+1.
[0070]
Fig. 12 is a graph that illustrates changes in the maximum
parallax value of the stereoscopic video signal after the
processing by the parallax processing, unit 42 is performed.
30 In the same drawing, similar to the case of Fig. 11, the
horizontal axis indicates frame numbers, the vertical axis
20
52262577W000
indicates values of parallax, and the scene change frame is
assumed to be the frame with frame number P. In the graph
of the same drawing, the changes in the maximum parallax value
of the stereoscopic video signal which has undergone the
5 processing by the parallax processing unit 42 are indicated
by a solid line, and the changes in the maximum parallax value
of the stereoscopic video signal which has not undergone the
processing by the parallax processing unit 42 are indicated
by a dotted line.
10 [0071]
As illustrated in Fig. 12, through the processing
performed by the parallax processing unit 42, the maximum
parallax of each of the frames, from the frame with frame number
P+180 to the frame with frame number P-180, is individually
15 adjusted.
That is, the maximum parallaxes of the frames, from the frame
with frame number P-180 to the frame with frame number P, are
adjusted to be increased compared to the original values, and
the maximum parallaxes of the frames, fromthe frame with frame
20 number P+1 to the frame with frame number P+180, are adjusted
to be decreased compared to the original values. As a rosult,
the maximum parallaxes have been adjusted so as to be gradually
increased in a direction from the frame with frame number P-180
to the frame with frame number P+180.
25 [0072]
That is, the processing of correcting a signal such that
there is no temporal discontinuity in parallax change has been
performed.
[0073]
30 For example, when the processing target frame is the
scene change frame, the case becomes such that Pc = P. Next,
21
SP262577WO00
it is assumed that the right eye plane, the left eye plane,
and,the parallax plane of the frame with frame number P are
as illustrated in Fig. 13. As illustrated in Fig. 13, the
maximum parallax value of the frame with frame number P is
5 2.
[0074]
If the frame number and the maximum parallax value shown
in Fig. 13 are substituted into variables in Expression (2),
the value of a becomes 1.5.
10 [0075]
The parallax adjusting unit 62 multiplies the maximum
parallax value in the parallax plane of Fig. 13 by a. In this
case, the maximum parallax value is adjusted so as to become
2 x 1.5 = 3. Therefore, the right eye plane, the left eye
15 plane, and the parallax plane of the frame with frame number
P that has undergone the processing by the parallax processing
unit 42 are illustrated in Fig. 14,
[0076]
After that, the stereoscopic video signal which has
20 undergone the correction of the parallax adjustment is output
to the display 22.
[0077]
In this way, it is possible to prevent the temporal
discontinuity in parallax change from occurring in the
25 stereoscopic video, and thus user's visual fatigue can be
reduced. Since the maximum parallax value of the corrected
stereoscopic video signal is adjusted so as to gradually change
from the time point at which the scene change occurs, an
unnatural impression may not be given to the user.
30 [0078]
Next, an example of the processing of correcting the
22
SP262577WO00
stereoscopic videosignalperformedbytheparallaxprocessing
unit 42 will be described with reference to Fig. 15.
[0079]
In Step 521, the scene change detecting unit 61 specifies
5 the frame number of the scene change frame based on the
stereoscopic video signal supplied from the stereoscopic video
signal acquiring unit 41. In this case, the scene change
detecting unit 61 accumulates, in an internal buffer or the
like, the frames of the stereoscopic video signal corresponding
10 to the stereoscopic video, for example, for three seconds,
and generates the scene change flag sequence as described above
with reference to Fig. 9. Then, the frame number P of the
scene change frame is specified based on the scene change flag
sequence.
15 [0080]
In Step S22, the parallax adjusting unit 62 acquires
the frames of the stereoscopic video signal based on the frame
number of the scene change frame specified by the scene change
detecting unit 61 during the process of Step 521. In this
20 case, the frame with frame number P-180 and the frame with
frame number P1180 are acquired.
[0081]
In Step 523, the parallax adjusting unit 62 specifies
the waximumparallax value for the two frames mentioned above.
25 That is, the maximum value among the numerical values of the
parallax planes of both the frames is specified.
[0082]
In Step 524, the parallax adjusting unit 62 calculates
the parallax coefficient a by Expression (1).
30 [0083]
In Step 525, the parallax adjusting unit 62 adjusts the
23
SP262577WO00
value of the maximum parallax of the parallax plane.
[0084]
Further, the processing of Step S24 and Step S25 is
performed on each of the frames, from the frame with frame
5 number P-180 to the frame with frame number P+180, that are
set as processing target frames. That is, the variable Pc
in Expression (1) is sequentially counted up, for example,
from P-180 to P+180, and thus the processing of Step S24 and
Step S25 is executed.
10 [0085]
In Step S26, the parallax adjusting unit 62 outputs the
stereoscopic video signal which has been corrected through
the processing of Step S25 for adjusting the parallax.
[0086]
15 In this way, the processing of correcting the
stereoscopic video signal is executed.
[0087]
The above description has been made based on the premise
that the parallax plane is contained in the frame of the
20 stereoscopic video signal. However, the present invention
can be applied even to a case where the parallax plane Is not
contained in the frame of the stereoscopic video signal. When
the parallax plane is not contained, the shift of the object
in the right eye plane and the left eye plane may be specified,
25 for example, using the block matching process or the like by
the parallax processing unit 42, and, as a result, the
information similar to the parallax plane may be obtained.
[0088]
Moreover, the above description, has been made based on
30 the premise that the scene change detection flag is contained
in the frame of the stereoscopic video signal. However, the
24
SP262577W000
present invention can be applied even to the case where the
scene change detection flag is not contained in the frame of
the stereoscopic video signal. When the scene change
detection flag is not contained, the scene change is detected
5 based on, for example, the histogram of the luminance of the
pixels of the image data obtained from the stereoscopic video
signal beforehand, and, as a result, information similar to
the scene change detection flag may be obtained.
[0089]
10 In addition, the above description has been made in
connection with an example in which the adjustment of the
parallax is performed using the scene change frame as a
reference frame. That is, the description has been made on
the assumption that frame number Pin Expression (1) represents
15 the scene change frame. However, the frame used as the
reference frame may be specified, for example, based on the
change in the maximum value of parallax of each frame. For
example, the frame with frame number P and the frame with frame
number P+1 are detected which satisfy a condition in which
20 an absolute value of a difference in the maximum parallax value
between the frames is equal to or greater than a threshold
value. After the detection, the parallax adjustment may be
performed based on the frame with frame number P which serves
as a reference frame.
25 [0090]
A series of processes described above can be executed
by hardware, and it also can be executed by software. When
the series of processes described above is executed by software,
a program that forms the software will be installed into a
30 computer built in special hardware from a network or a recording
medium. Moreover, it may be installed, from a network or a
25
SP262577W000
recording medium, into a general purpose personal computer
700 or the like, illustrated in Fig. 16, into which various
kinds of programs are installed and thus the computer can
execute various functions.
5 [0091]
In Fig. 16, a CPU (Central Processing Unit) 701 executes
various kinds of processing according to a program stored in
a ROM (Read Only Memory) 702 or a program loaded into a RAM
(Random Access Memory) 703 from a storage unit 708. The RAM
10 703 also appropriately stores data or the like which is
necessary for the CPU 701 to execute various kinds of
processing.
[0092]
The CPU 701, the RO M702, and the RAM 703 are connected
15 to one another via a bus 704. An I/O interface 705 is also
connected to the bus 704.
[0093]
The I/O interface 705 is also connected to an input unit
706 configured to include a keyboard, a mouse or the like,
20 a display configured to include an LCD (Liquid Crystal Display),
and an output unit 707 configured to include a speaker or the
like. Moreover, the I/O interface 705 is even connected to
the storage unit 708 configured to include a hard disk or the
like, a communication unit 709 configured to include a modem,
25 or a network interface card, such as a LAN card. The
communication unit 709 performs a communication process
through a network including the Internet.
[0094]
The I/O interface 705 is also connected to a drive 710
30 if necessary, and a removable medium 711 such as a magnetic
disk, an optical disk, a magnet-optical disk, a semiconductor
26
SP262577W000
memory is appropriately mounted. And, the computer program
read from the removable medium is installed into the storage
unit 708, if necessary.
[0095]
5 When the series of processes described above is executed
by using software, a program that composes the software is
installed from a network such as the Internet, or a recording
medium such as the removable medium 711.
[0096]
10 In addition, the recording medium is configured by using
amagneticdisk (including afloppydisk (registeredtrademark) ,
an optical disk (including a CD-ROM (Compact Disk-Read Only
Memory) and a DVD (Digital Versatile Disk)), amagneto-optical
disk (including an MD (Mini-Disk) (registered trademark)),
15 or the removable medium 711 configured by using a semiconductor
memory or the like, in each of which a program is stored and
each of which is distributed separately from the main body
of the apparatus illustrated in Fig. 16 to deliver the program
to the user. Besides, the recording medium may be configured
20 by using the ROM 702, a hard disk contained in the storage
unit 708, or the like in which the program is recorded and
which is delivered to the user in the state of being embedded
in the main body of the apparatus.
[0097]
25 Moreover, the series of processes which has been
described above in the specification of the present application
includes not only processes which are performed in time series
along the description order but also processes which are
executed in parallel with each other, or individually.
30 [0098]
Moreover, embodiments of the present invention are not
27
SP262577WO00
limited to the above-described embodiments but maybe modified
in various forms within the scope without departing from the
spirit of the invention.
5 REFERENCE SIGNS LIST
10
[0099]
10
21
22
41
42
61
62
Stereoscopic video display system
Stereoscopic video display apparatus
Display
Stereoscopic video signal acquiring unit
Parallax processing unit
Scene change detecting unit
Parallax adjusting unit
28
SP262577WO00
CLAIMS
1e A stereoscopic video processing apparatus comprising:
a frame specifying unit that specifies a frame that
5 causes a temporal discontinuity in change of maximum parallax
value among frames of a stereoscopic video signal that includes
frame rate for a unit time;
a maximum parallax specifying unit that specifies a
maximum parallax value of a first frame that precedes a
10 reference frame by a predetermined time when the specified
frame is set as the reference frame, and a maximum parallax
value of a second frame that lags behind the reference frame
by the predetermined time;
a parallax coefficient calculating unit that sets frames
15 between the first frame and the second frame as a processing
target frame and calculates a parallax coefficient that is
coefficient used to adjust the maximum parallax value of the
processing target frame such that changes in the maximum
parallax value are continuous over time based on the maximum
20 parallax value of the first frame, the maximum parallax value
of the second frame, and the maximum parallax value of the
processing target frame; and
a parallax adjusting unit that adjusts the maximum
parallax value of the processing target frame by multiplying
25 the maximum parallax value by the calculated parallax
coefficient.
2e The stereoscopic video processing apparatus according
to claim 1, wherein a parallax of each of the frames of the
30 stereoscopic video signal is information contained in each
of the frames of the stereoscopic video signal, and is specified
29
SP262577WO00
based on a parallax plane that indicates a parallax for each
pixel..
3. The stereoscopic video processing apparatus according
5 to claim 1, wherein the parallax of each of the frames of the
stereoscopic video signal is specified by calculating a
difference in pixel between right eye image data and left eye
image data that are contained in each frame of the stereoscopic
video signal.
10
4. The stereoscopic video processing apparatus according
to claim 1, wherein the frame specifying unit specifies the
frame which causes a temporal discontinuity in change of the
maximum parallax value by detecting a scene change frame in
15 a video of the stereoscopic video signal.
5. The stereoscopic video processing apparatus according
to claim 1, wherein the frame specifying unit specifies the
frame which causes a temporal discontinuity in change of the
20 maximum parallax value based on a difference in the maximum
parallax value between two frames that are temporally
consecutive.
6. - A method of processing a stereoscopic video, the method
25 comprising:
at a frame specifying unit, specifying a frame which
causes a temporal discontinuity in change of maximum parallax
value among frames of a stereoscopic video signal that has
frame rate for each unit time;
30 at a maximum parallax specifying unit, setting the
specified frame as a reference frame, and specifying a maximum
30
SP262577W000
parallax value of a first frame that precedes the reference
frame by a predetermined time and a maximum parallax value
of a second frame that lags behind the reference frame by the
predetermined time;
5 at a parallax coefficient calculating unit, setting a
frame between the first frame and the second frame as a
processing target frame, and calculating a parallax
coefficient that is coefficient used to adjust a maximum
parallax value of the processing target frame so that changes
10 in the maximum parallax value are temporally continuous, based
on the maximum parallax value of the first frame, the maximum
parallax value of the second frame, and the maximum parallax
value of the processing target frame; and
at a parallax adjusting unit, adjusting the maximum
15 parallax value of the processing target frame by multiplying
the maximum parallax value of the processing target frame by
the calculated parallax coefficient.
7. A program causing a computer to function as a
20 stereoscopic video processing apparatus, the stereoscopic
vide processing apparatus including:
a frame specifying unit that specifies a frame that
causes a temporal discontinuity in change of maximum parallax
value among frames of a stereoscopic video signal that includes
25 frame rate for a unit time;
a maximum parallax specifying unit that specifies a
maximum parallax value of a first frame that precedes a
reference frame by a predetermined time and a maximum parallax
value of a second frame that lags behind the reference frame
30 by the predetermined time when the specified frame is set as
the reference frame;
31
SP262577
a parallax coeffiu^ient calculatingunit that sets frames
between the first frame and the second frame as a processing
target frame and calculates a parallax coefficient that is
coefficient used to adjust the maximum parallax value of the
5 processing target frame such that changes in the maximum
parallax value are continuous over time based on the maximum
parallax value of the first frame, the maximum parallax value
of the second frame, and the maximum parallax value of the
processing target frame; and
10 a parallax adjusting unit that adjusts the maximum
parallax value of the processing target frame by multiplying
the maximum parallax value by the calculated parallax
coefficient.