SP336805WO00
DESCRIPTION
TRANSMITTING APPARATUS, TRANSMITTING METHOD, AND
RECEIVING APPARATUS
5 TECHNICAL FIELD
[0001]
The present technology relates to a transmitting
apparatus, a transmitting method, and a receiving
apparatus. In particular, the present technology relates
10 to a transmitting apparatus or the like, which transmits
superimposition information data and disparity
information together with stereoscopic image data
including left-eye image data and right-eye image data.
15 BACKGROUND ART
[0002]
For example. Patent Document 1 has proposed a
transmission scheme using television airwaves of
stereoscopic image data. In this transmission scheme,
20 stereoscopic image data having left-eye image data and
right-eye image data are transmitted to display a
stereoscopic image using a binocular disparity.
[0003]
Fig. 64 illustrates the relation between the
25 display positions of left and right images of an object
(thing) on a screen and the reproduction position of a
stereoscopic image thereof, in a stereoscopic image
display using a binocular disparity. For example, as for
an object A of which the left image La and the right
30 image Ra are displayed respectively at the right side and
the left side on the screen as illustrated, since the
1
SP336805WO00
left and right lines of sight intersect with each other
in front of a screen surface, the reproduction position
of a stereoscopic image thereof is in front of the screen
surface. DPa denotes a disparity vector of the object A
5 in the horizontal direction.
[0004]
Also, for example, as for an object B of which the
left image Lb and the right image Rb are respectively
displayed at the same position on the screen as
10 illustrated, since the left and right lines of sight
intersect with each other at the screen surface, the
reproduction position of a stereoscopic image thereof is
on the screen surface. In addition, for example, as for
an object C of which the left image Lc and the right
15 image Re are displayed respectively at the left side and
the right side on the screen as illustrated, since the
left and right lines of sight intersect with each other
inside the screen surface, the reproduction position of a
stereoscopic image thereof is inside the screen surface.
20 DPc denotes a disparity vector of the object C in the
horizontal direction.
CITATION LIST
PATENT DOCUMENT
25 [0005]
Patent Document 1: Japanese Patent Application Laid-Open
No. 2005-6114
SUMMARY OF THE INVENTION
30 PROBLEMS TO BE SOLVED BY THE INVENTION
[0006]
2
SP336805WOQ0
As described above, in a stereoscopic image display,
a viewer usually perceives the perspective of a
stereoscopic image by using a binocular disparity. Also,
it is expected that superimposition information
5 superimposed on an image, such as a caption, will be
rendered in conjunction with a stereoscopic image display
not only as a two-dimensional (2D) spatial depth feel but
also as a three-dimensional (3D) depth feel. For example,
in the case where an image and a caption are displayed in
10 a superimposed (overlaid) manner but not displayed in
front of a thing (object) in an image closest in terms of
the perspective, a viewer may feel a perspective
discrepancy.
[0007]
15 Thus, it is considered that disparity information
between a left-eye image and a right-eye image is
transmitted together with data of superimposition
information and a receiving side provides a disparity
between left-eye superimposition information and right-
20 eye superimposition information. In this manner,
disparity information is meaningful information in a
receiving apparatus capable of displaying a stereoscopic
image. On the other hand, the disparity information is
unnecessary a legacy 2D-compatible receiving apparatus.
25 In the 2D-compatible receiving apparatus, there is a need
to take some kinds of measures to prevent normal
reception processing from being interrupted by
transmission of the disparity information.
[0008]
30 An object of the present technology is to prevent
reception processing of a legacy 2D-compatible receiving
3
SP336805WO00
apparatus from being interrupted by transmission of
disparity information.
SOLUTIONS TO PROBLEMS
5 [0009]
A concept of the present technology is a
transmitting apparatus including:
an image data output unit configured to output
left-eye image data and right-eye image data for
10 displaying a stereoscopic image;
a superimposition information data output unit
configured to output superimposition information data to
be superimposed on an image by the left-eye image data
and the right-eye image data;
15 a disparity information output unit configured to
output disparity information for providing a disparity by
shifting the superimposition information to be
superimposed on the image by the left-eye image data and
the right-eye image data; and
20 a data transmitting unit configured to transmit a
multiplexed data stream including a video data stream
including the image data output from the image data
output unit, a first private data stream including the
superimposition information data output from the
25 superimposition information data output unit, and a
second private data stream including the disparity
information output from the disparity information output
unit,
wherein a first descriptor and a second descriptor
30 including respective pieces of language information
corresponding to the first private data stream and the
4
SP336805WO00
second private data stream are inserted into the
multiplexed data stream, and the language information
included in the second descriptor is set to represent a
non-language.
5 [0010]
In the present technology, the image data output
unit outputs left-eye image data and right-eye image data
for displaying a stereoscopic image. The superimposition
information data output unit outputs superimposition
10 information data to be superimposed on an image by the
left-eye image data and the right-eye image data. Herein,
the superimposition information includes a caption,
graphics, a text, and the like that are superimposed on
the image. The disparity information output unit outputs
15 disparity information for providing a disparity by
shifting the superimposition information to be
superimposed on the image by the left-eye image data and
the right-eye image data.
[0011]
20 The data transmitting unit transmits the
multiplexed data stream. The transport stream includes a
video data stream including image data, a first private
data stream including superimposition information data,
and a second private data stream including disparity
25 information.
[0012]
A first descriptor and a second descriptor
including respective pieces of language information
corresponding to the first private data stream and the
30 second private data stream are inserted into the
multiplexed data stream. The language information
5
SP336805WO00
included in the second descriptor is set to represent a
non-language.
[0013]
For example, the superimposition information data
5 is DVB (Digital Video Broadcasting) subtitle data, and
the descriptors are a component descriptor and a subtitle
descriptor. Also, for example, the language information
representing a non-language is "zxx" representing a nonlanguage
of an ISO language code, or any one of language
10 codes included in a space from "qaa" to "qrz" of an ISO
language code.
[0014]
In this manner, in the present technology, the
multiplexed data stream includes the first private data
15 stream including the superimposition information data,
and the second private data stream including the
disparity information. The first descriptor and the
second descriptor including the respective pieces of
language information are inserted into the multiplexed
20 data stream. Therefore, based on the language
information included in the descriptor, a legacy 2Dcompatible
receiving apparatus of a receiving side can
extract only the first private data stream from the
multiplexed data stream and decode the first private data
25 stream extracted. Thus, the second private data stream
can be prevented from being decoded. Accordingly,
reception processing of the legacy 2D-compatible
receiving apparatus can be prevented from being
interrupted by transmission of the disparity information.
30 [0015]
Also, in the present technology, for example, the
6
SP336805WO00
superimposition information data may be DVB subtitle data,
a first subtitle descriptor and a second subtitle
descriptor corresponding respectively to the first
private data stream and the second private data stream
5 may be inserted into the multiplexed data stream, and a
subtitle type represented by subtitle type information of
the first subtitle descriptor may be different from a
subtitle type represented by subtitle type information of
the second subtitle descriptor.
10 [0016]
In this case, based on the subtitle type
information included in the subtitle descriptor, the
legacy 2D-compatible receiving apparatus of the receiving
side can extract only the first private data stream from
15 the multiplexed data stream and decode the first private
data stream extracted. Thus, the second private data
stream can be prevented from being decoded. Accordingly,
reception processing of the legacy 2D-compatible
receiving apparatus can be prevented from being
20 interrupted by transmission of the disparity information.
[0017]
Also, in the present technology, for example, the
superimposition information data may be DVB subtitle data,
and a segment including the superimposition information
25 data of the first private data stream may be equal to a
page ID of a segment including the disparity information
of the second private data stream. In this case, based
on the page ID, a 3D-compatible receiving apparatus of
the receiving side can easily combine the segment
30 including the superimposition information data and the
segment including the disparity information.
7
SP336805WO00
[0018]
Also, in the present technology, the
superimposition information data may be DVB subtitle data,
and the disparity information included in the second
5 private data stream may be operated as an ancillary page.
In this case, when coping with a multi-language server,
since the second private data stream including the
disparity information can be commonly transmitted to each
service, efficient use of a transmission bandwidth can be
10 achieved.
[0019]
Another concept of the present technology is a
receiving apparatus including:
a data receiving unit configured to receive a
15 multiplexed data stream including a video data stream
including left-eye image data and right-eye image data
for displaying a stereoscopic image, a first subtitle
data stream including superimposition information data to
be superimposed on an image by the left-eye image data
20 and the right-eye image data, and a second subtitle data
stream including disparity information for providing a
disparity by shifting the superimposition information to
be superimposed on the image by the left-eye image data
and the right-eye image data;
25 a video decoding unit configured to extract the
video data stream from the multiplexed data stream
received by the data receiving unit and decode the video
data stream extracted; and
a subtitle decoding unit configured to extract the
30 first subtitle data stream from the multiplexed data
stream received by the data receiving unit and decode the
8
SP336805WO00
first subtitle data stream extracted,
wherein descriptors including respective pieces of
subtitle type information corresponding to the first
subtitle data stream and the second subtitle data stream
5 are inserted into the multiplexed data stream,
the respective pieces of subtitle type information
included in the descriptors corresponding to the first
subtitle data stream and the second subtitle data stream
are set to represent different subtitle types, and
10 the subtitle decoding unit determines the subtitle
data stream to be decoded after being extracted from the
multiplexed data stream, based on the subtitle type
information inserted into the descriptor.
[0020]
15 In the present technology, the data receiving unit
receives the multiplexed data stream including the video
data stream, the first subtitle data stream, and the
second subtitle data stream. The video decoding unit
extracts the video data stream from the multiplexed data
20 stream and decodes the video data stream extracted. Also,
the video decoding unit extracts the first subtitle data
stream from the multiplexed data stream and decodes the
first subtitle data stream extracted.
[0021]
25 Herein, the descriptors including the respective
pieces of subtitle type information corresponding to the
first subtitle data stream and the second subtitle data
stream are inserted into the multiplexed data stream.
The respective pieces of subtitle type information
30 included in the descriptors corresponding to the first
subtitle data stream and the second subtitle data stream
9
SP336805WO00
are set to represent different subtitle types.
[0022]
The subtitle decoding unit determines the subtitle
data stream to be decoded after being extracted from the
5 multiplexed data stream, based on the subtitle type
information inserted into the descriptor. Accordingly,
for example, in a 2D-compatible receiving apparatus of a
receiving side, since the second private data stream
including the disparity information can be prevented from
10 being decoded, reception processing of the legacy 2Dcompatible
receiving apparatus can be prevented from
being interrupted by transmission of the disparity
information.
[0023]
15 Another concept of the present technology is a
receiving apparatus including:
a data receiving unit configured to receive a
multiplexed data stream including a video data stream
including left-eye image data and right-eye image data
20 for displaying a stereoscopic image, a first subtitle
data stream including superimposition information data to
be superimposed on an image by the left-eye image data
and the right-eye image data, and a second subtitle data
stream including disparity information for providing a
25 disparity by shifting the superimposition information to
be superimposed on the image by the left-eye image data
and the right-eye image data;
a video decoding unit configured to extract the
video data stream from the multiplexed data stream
30 received by,the data receiving unit and decode the video
data stream extracted; and
10
SP336805WO00
a subtitle decoding unit configured to extract the
first subtitle data stream from the multiplexed data
stream received by the data receiving unit and decode the
first subtitle data stream extracted,
5 wherein descriptors including respective pieces of
language information corresponding to the first subtitle
data stream and the second subtitle data stream are
inserted into the multiplexed data stream,
the language information included in the descriptor
10 corresponding to the second subtitle data stream is set
to represent a non-language, and
the subtitle decoding unit determines the subtitle
data stream to be decoded after being extracted from the
multiplexed data stream, based on the language
15 information inserted into the descriptor.
[0024]
In the present technology, the data receiving unit
receives the multiplexed data stream including the video
data stream, the first subtitle data stream, and the
20 second subtitle data stream. The video decoding unit
extracts the video data stream from the multiplexed data
stream and decodes the video data stream extracted. Also,
the video decoding unit extracts the first subtitle data
stream from the multiplexed data stream and decodes the
25 first subtitle data stream extracted.
[0025]
Herein, the descriptors including the respective
pieces of language information corresponding to the first
subtitle data stream and the second subtitle data stream
30 are inserted into the multiplexed data stream. The
language information included in the descriptor
11
SP336805WO00
corresponding to the second subtitle data stream is set
to represent a non-language.
[0026]
The subtitle decoding unit determines the subtitle
5 data stream to be decoded after being extracted from the
multiplexed data stream, based on the language
information inserted into the descriptor. Accordingly,
for example, in a 2D-compatible receiving apparatus of a
receiving side, since the second private data stream
10 including the disparity information can be prevented from
being decoded, reception processing of the legacy 2Dcompatible
receiving apparatus can be prevented from
being interrupted by transmission of the disparity
information.
15
EFFECTS OF THE INVENTION
[0027]
According to the present technology, reception
processing of the legacy 2D-compatible receiving
2 0 apparatus can be prevented from being interrupted by
transmission of the disparity information.
BRIEF DESCRIPTION OF DRAWINGS
[0028]
25 Fig. 1 is a block diagram illustrating an example
of a configuration of an image transmitting/receiving
system according to an embodiment of the present
invention.
Fig. 2 is a block diagram illustrating an example
30 of a configuration of a transmission data generating unit
in a broadcasting station.
12
SP336805WO00
Fig. 3 is a diagram illustrating image data of a
1920x1080 pixel format.
Fig. 4 is a diagram for describing a Top & Bottom
scheme, a Side By Side scheme, and a Frame Sequential
5 scheme that are stereoscopic image data (3D image data)
transmitting schemes.
Fig. 5 is a diagram for describing an example of
detecting a disparity vector of a right-eye image with
respect to a left-eye image.
10 Fig. 6 is a diagram for describing the obtainment
of a disparity vector by a block matching scheme.
Fig. 7 is a diagram illustrating an example of an
image in the case where a value of a disparity vector of
each pixel is used as a luminance value of each pixel.
15 Fig. 8 is a diagram illustrating an example of a .
disparity vector of each block.
Fig. 9 is a diagram for describing downsizing
processing performed by a disparity information creating
unit of the transmission data generating unit.
20 Fig. 10 is a diagram illustrating an example of a
region defined on an image in subtitle data and a
subregion defined in the region.
Fig. 11 is a diagram illustrating a configuration
of a 2D stream and a 3D extension stream that are
25 included in a transport stream TS.
Fig. 12 is a diagram for describing the association
of a value of a time stamp PTS inserted into a PES header
of a 2D stream PESl(l): PES#1 with a value of a time
stamp PTS inserted into a PES header of a 3D extension
30 stream PES2(2): PES#2.
Fig. 13 is a diagram illustrating an example in
13
SP336805WO00
which the values of the time stamps PTS of the 2D stream
and the 3D extension stream are set to different values.
Fig. 14 is a diagram illustrating another example
in which the values of the time stamps PTS of the 2D
5 stream and the 3D extension stream are set to different
values.
Fig. 15 is a diagram illustrating a configuration
of a transport stream TS including a 2D stream and a 3D
extension stream.
10 Fig. 16 is a diagram illustrating a structure of a
PCS (page_composition_segment) constituting subtitle data.
Fig. 17 is a diagram illustrating the
correspondence relation between each value of
segment_type and a segment type.
15 Fig. 18 is a diagram for describing information
(Component_type = 0x15, 0x25) representing a 3D subtitle
format that is newly defined.
Fig. 19 is a diagram illustrating the extraction of
a subtitle descriptor (Subtitling_descriptor) and a
20 component descriptor (Component_descriptor) inserted into
a transport stream.
Fig. 20 is a diagram illustrating the extraction of
PES streams (2D stream and 3D extension stream) inserted
into a transport stream.
25 Fig. 21 is a diagram illustrating the use of any
one of language codes included in a space from "qaa" to
"qrz" of an ISO language code, as an ISO language code
representing a non-language.
Fig. 22 is a diagram illustrating the extraction of
30 an ISO language code (ISO 639-2 Code) list.
Fig. 23 is a diagram illustrating an example of a
14
SP336805WO00
stream configuration of subtitle data streams (2D stream
and 3D extension stream).
Fig. 24 is a diagram illustrating an example of a
syntax of the component descriptor (Component_descriptor).
5 Fig. 25 is a diagram illustrating an example of a
syntax of the subtitle descriptor (Subtitling_descriptor).
Fig. 2 6 is a diagram illustrating an example of
updating disparity information by using an interval
period and the case where an interval period is fixed and
10 is equal to an update period.
Fig. 27 is a diagram illustrating an example of
updating disparity information by using an interval
period and an example of updating disparity information
in the case where an interval period is set to, short.
15 Fig. 28 is a diagram illustrating an example of a
configuration of the 3D extension stream.
Fig. 29 is a diagram illustrating an example of
updating disparity information in the case of
sequentially transmitting DSS segments.
20 Fig. 30 is a diagram illustrating an example of
updating disparity information, in which an update frame
interval is expressed in a multiple of an interval
duration (ID) as a unit period.
Fig. 31 is a diagram illustrating an example of
25 displaying subtitles, in which two regions as caption
display regions are included in a page area (Area for
Page_default).
Fig. 32 is a diagram illustrating an example of the
disparity information curve of each region and page in
30 the case where disparity information in units of a region
and disparity information in units of a page including
15
SP336805WO00
all regions are included in a DSS segment, as disparity
information that is sequentially updated in a caption
display period.
Fig. 33 is a diagram illustrating a transmission
5 structure of disparity information of each page and
region.
Fig. 34 is a diagram (1/3) illustrating an example
of a syntax of the DSS.
Fig. 35 is a diagram (2/3) illustrating an example
10 of a syntax of the DSS.
Fig. 36 is a diagram (3/3) illustrating an example
of a syntax of the DSS.
Fig. 37 is a diagram (1/4) illustrating the main
data definition contents (semantics) of the DSS.
15 Fig. 38 is a diagram (2/4) illustrating the main
data definition contents (semantics) of the DSS.
Fig. 39 is a diagram (3/4) illustrating the main
data definition contents (semantics) of the DSS.
Fig. 40 is a diagram (4/4) illustrating the main
20 data definition contents (semantics) of the DSS.
Fig. 41 is a diagram illustrating the concept" of
broadcast reception in the case where a set-top box and a
television receiver are 3D-compatible devices.
Fig. 42 is a diagram schematically illustrating
25 extraction processing of a 2D stream and a 3D extension
stream in the set-top box (3D-compatible device).
Fig. 43 is a diagram illustrating the concept of
broadcast reception in the case where a set-top box and a
television receiver are legacy 2D-compatible devices.
30 Fig. 44 is a diagram schematically illustrating
extraction processing of only a 2D stream in the set-top
16
SP336805WO00
box (2D-compatible device).
Fig. 45 is a diagram illustrating the summarization
of the concept of broadcast reception in the case where a
receiver is a legacy 2D-compatible device (2D receiver)
5 and in the case where a receiver is a 3D-compatible
device (3D receiver).
Fig. 46 is a diagram illustrating an example of
displaying a caption (graphics information) on an image,
and the perspective of a background, a near-view object,
10 and the caption.
Fig. 47 is a diagram illustrating an example of
displaying a caption on an image, and a left-eye caption
LGI and a right-eye caption RGI for displaying the
caption.
15 Fig. 48 is a block diagram illustrating an example
of a configuration of a set-top box included in the image
transmitting/receiving system.
Fig. 49 is a block diagram illustrating an example
(3D-compatible) of a configuration of a bit stream
20 processing unit included in the set-top box.
Fig. 50 is a diagram illustrating an example of a
syntax of a multi-decoding descriptor that can be used to
associate a 2D stream with a 3D extension stream.
Fig. 51 is a diagram illustrating the contents
25 (semantics) of main information in the example of the
syntax of the multi-decoding descriptor.
Fig. 52 is a diagram illustrating an example of a
configuration of a transport stream TS in the case where
the multi-decoding descriptor is disposed.
30 Fig. 53 is a block diagram illustrating another
example (2D-compatible) of a configuration of a bit
17
SP336805WO00
stream processing unit included in the set-top box.
Fig. 54 is a block diagram illustrating an example
of a configuration of a television receiver included in
the image transmitting/receiving system.
5 Fig. 55 is a diagram illustrating an example of a
configuration of a transport stream TS in the case where
the disparity information included in the 3D extension
stream is operated as an ancillary page (ancillary_page).
Fig. 56 is a diagram illustrating the extraction of
10 a subtitle descriptor (Subtitling_descriptor) and a
component descriptor (Component_descriptor) inserted into
a transport stream.
Fig. 57 is a diagram illustrating the extraction of
PES streams (2D stream and 3D extension stream) inserted
15 into a transport stream.
Fig. 58 is a diagram illustrating an example of a
stream configuration of a subtitle data stream (2D stream
and 3D extension stream) in the case where the disparity
information included in the 3D extension stream is
20 operated as an ancillary page (ancillary_page).
Fig. 59 is a diagram schematically illustrating
extraction processing of a 2D stream and a 3D extension
stream in the 3D-compatible device.
Fig. 60 is a diagram schematically illustrating
25 extraction processing of only a 2D stream in the 2Dcompatible
device.
Fig. 61 is a block diagram illustrating another
example of a configuration of a set-top box included in
the image transmitting/receiving system.
30 Fig. 62 is a block diagram illustrating another
example of a configuration of a television receiver
18
SP336805WO00
included in the image transmitting/receiving system.
Fig. 63 is a block diagram illustrating another
example of a configuration of the image
transmitting/receiving system.
5 Fig. 64 is a diagram for describing the relation
between the display positions of left and right images of
an object on a screen and the reproduction position of a
stereoscopic image thereof, in a stereoscopic image
display using a binocular disparity.
10
MODE FOR CARRYING OUT THE INVENTION
[0029]
Hereinafter, modes for implementing the invention
(hereinafter, referred to as "embodiments") will be
15 described. In addition, the description will be made in
the following order.
1. Embodiments
2. Modifications
[0030]
20 <1. Embodiments>
[Example of Configuration of Image
Transmitting/Receiving System]
Fig. 1 illustrates an example of a configuration of
an image transmitting/receiving system 10 according to an
25 embodiment. The image transmitting/receiving system 10
includes a broadcasting station 100, a set-top box (STB)
200, and a television receiver (TV) 300.
[0031]
The set-top box 200 and the television receiver 300
30 are connected by a digital interface of HDMI (High
Definition Multimedia Interface). The set-top box 200
19
SP336805WO00
and the television receiver 300 are connected by using an
HDMI cable 400. The set-top box 200 is provided with an
HDMI terminal 202, The television receiver 300 is
provided with an HDMI terminal 302. One end of the HDMI
5 cable 400 is connected to the HDMI terminal 202 of the
set-top box 200, and the other end of the HDMI cable 400,
is connected to the HDMI terminal 302 of the television
receiver 300.
[0032]
10 [Description of Broadcasting Station]
The broadcasting station 100 transmits a transport
stream TS on a broadcast wave. The broadcasting station
100 includes a transmission data generating unit 110 that
generates a transport stream TS. The transport stream TS
15 includes stereoscopic image data, audio data,
superimposition information data, disparity information,
or the like. The stereoscopic image data has a
predetermined transmission format, and includes left-eye
image data and right-eye image data for displaying a
20 stereoscopic image. In general, the superimposition
information is a caption, graphics information, text
information, or the like. However, in the embodiment,
the superimposition information is a subtitle (caption).
[0033]
25 [Example of Configuration of Transmission Data
Generating Unit]
Fig. 2 illustrates an example of a configuration of
the transmission data generating unit 110 in the
broadcasting station 100. The transmission data
30 generating unit 110 transmits disparity information
(disparity vector) in a data structure that can easily
20
SP336805WO00
cooperate with a DVB (Digital Video Broadcasting) scheme
that is one of the existing broadcast standards. The
transmission data generating unit 110 includes a data
extracting unit 111, a video encoder 112, and an audio
5 encoder 113. Also, the transmission data generating unit
110 includes a subtitle generating unit 114, a disparity
information creating unit 115, a subtitle processing unit
116, a subtitle encoder 118, and a multiplexer 119.
[0034]
10 The data extracting unit 111 is, for example,
detachably mounted with a data recording medium Ilia.
The data recording medium Ilia stores the audio data and
the disparity information in association with the
stereoscopic image data including the left-eye image data
15 and the right-eye image data. The data extracting unit
111 extracts the stereoscopic image data, the audio data,
the disparity information, or the like from the data
recording medium Ilia and outputs the same. Examples of
the data recording medium Ilia include a disk-type
20 recording medium and a semiconductor memory.
[0035]
The stereoscopic image data recorded in the data
recording medium Ilia is stereoscopic image data of a
predetermined transmission scheme. An example of the
25 transmission scheme for transmitting the stereoscopic
image data (3D image data) will be described. Herein,
although the following first to third transmission
schemes will be described as an example, any other
transmission schemes may be used to transmit the
30 stereoscopic image data (3D image data), Also, herein,
the case where the left-eye (L) image data and the right-
21
SP336805WO00
eye (R) image data are image data with a predetermined
resolution, for example, a 1920x1080 pixel format as
illustrated in Fig. 3 will be described as an example.
[0036]
5 The first transmission scheme is a Top & Bottom
scheme, and is a scheme that transmits each line data of
the left-eye image data in the first half of the vertical
direction and transmits each line data of the right-eye
image data in the second half of the vertical direction
10 as illustrated in Fig. 4(a). In this case, since the
lines of the left-eye image data and the right-eye image
data are reduced by 1/2, the vertical resolution is
reduced by 1/2 with respect to the original signal.
[0037]
15 The second transmission scheme is a Side By Side
scheme, and is a scheme that transmits pixel data of the
left-eye image data in the first half of the horizontal
direction and transmits pixel data of the right-eye image
data in the second half of the horizontal direction as
20 illustrated in Fig. 4(b). In this case, the horizontaldirection
pixel data of each of the left-eye image data
and the right-eye image data is reduced by 1/2. The
horizontal resolution is reduced by 1/2 with respect to
the original signal.
25 [0038]
The third transmission scheme is a Frame Sequential
scheme or an L/R No Interleaving scheme, and is a scheme
that transmits the left-eye image data and the right-eye
image data by being sequentially switched for the
30 respective frames as illustrated in Fig. 4(c). In
addition, this scheme also includes a Full Frame scheme
22
SP336805WO00
or a Service Compatible scheme for the conventional 2D
format.
[0039]
Also, the disparity information recorded in the
5 data recording medium Ilia is, for example, a disparityvector
of each pixel constituting an image. An example
of the detection of the disparity vector will be
described. Herein, an example of detecting the disparity
vector of the right-eye image with respect to the left-
10 eye image will be described. As illustrated in Fig. 5,
the left-eye image is used as a detection image, and the
right-eye image is used as a reference image. In this
example, disparity vectors at the positions (xi, yi) and
(xj, yj) are detected.
15 [0040]
The case of detecting the disparity vector at the
position (xi, yi) will be described as an example. In
this case, in the left-eye image, for example, a 4x4, 8x8,
or 16x16 pixel block (disparity detection block) Bi is
20 set with the upper left pixel at the position (xi, yi).
In the right-eye image, a pixel block matched with the
pixel block Bi is searched for.
[0041]
In this case, in the right-eye image, a search
25 range around the position (xi, yi) is set. Using each
pixel within the search range as a sequential attention
pixel, a comparison block like the above-described pixel
block Bi, for example, a 4x4, 8x8, or 16x16 comparison
block is sequentially set.
30 [0042]
Between the pixel block Bi and the sequentially-set
23
SP336805WO00
comparison blocks, the sum of absolute difference values
for the respective corresponding pixels is obtained.
Herein, as illustrated in Fig. 6, when the pixel value of
the pixel block Bi is L(x, y) and the pixel value of the
5 comparison block is R(x, y), the sum of absolute
difference values between the pixel block Bi and the
comparison blocks is expressed as i;|L(x, y) - R(x, y)'I .
[0043]
When an n number of pixels are included in the
10 search range set in the right-eye image, an n number of
sums Si to Sn are finally obtained and the minimum sum
Smin among them is selected. The position (xi', yi') of
the upper left pixel is obtained from the comparison
block for which the minimum sum Smin is obtained.
15 Accordingly, the disparity vector at the position (xi,
yi) is detected as (xi' - xi, yi' - yi) . Although not
described in detail, for example, a 4x4, 8x8, or 16x16
pixel block Bj with the upper left pixel at the position
i^ji r yJ) is set in the left-eye image, and the disparity
20 vector at the position (xj, yj) is detected through the
same process.
[0044]
The video encoder 112 performs encoding, such as
MPEG4-AVC, MPEG2, or VC-1, on the stereoscopic image data
25 extracted by the data extracting unit 111, to generate a
video data stream (video elementary stream). The audio
encoder 113 performs encoding, such as AC3 or AAC, on the
audio data extracted by the data extracting unit 111, to
generate an audio data stream (audio elementary stream).
30 [0045]
The subtitle generating unit 114 generates subtitle
24
SP336805WO00
data as caption data of a DVB (Digital Video
Broadcasting) scheme. The subtitle data is subtitle data
for a two-dimensional image. The subtitle generating
unit 114 constitutes a superimposition information data
5 output unit.
[0046]
The disparity information creating unit 115
performs downsizing processing on the disparity vector
(horizontal-direction disparity vector) of a plurality of
10 pixels or each pixel extracted by the data extracting
unit 111, to generate disparity information of each layer
as described below. In addition, the disparity
information need not be necessarily generated by the
disparity information creating unit 115, and may also be
15 supplied separately from the outside.
[0047]
Fig. 7 illustrates an example of depth-direction
relative data that is provided as a luminance value of
each pixel. Herein, the depth-direction relative data
20 can be treated as a disparity vector of each pixel
through a predetermined conversion. In this example, a
luminance value of a person portion is set to be high.
This means that a disparity vector value of the person
portion is large, and thus means that the person portion
25 is perceived as being protrusive in the stereoscopic
image display. Also, in this example, a luminance value
of a background portion is set ,to be low. This means
that a disparity vector value of the backgrotind portion
is small, and thus means that the background portion is
30 perceived as being sunken in the stereoscopic image
display.
25
SP336805WO00
[0048]
Fig. 8 illustrates an example of a disparity vector
of each block. A block corresponds to the upper layer of
a pixel located at the lowermost layer. The block is
5 constructed by dividing an image (picture) region into a
predetermined size in the horizontal direction and the
vertical direction. The disparity vector of each block
is obtained, for example, by selecting a disparity vector
with the largest value from the disparity vectors of all
10 pixels present in the block. In this example, the
disparity vector of each block is represented by an arrow,
and the length of the arrow corresponds to the magnitude
of the disparity vector.
[0049]
15 Fig. 9 illustrates an example of the downsizing
processing performed by the disparity information
creating unit 115. First, as illustrated in Fig. 9(a),
the disparity information creating unit 115 obtains a
signed disparity vector of each block by using the
20 disparity vector of each pixel. As described above, the
block corresponds to the upper layer of a pixel located
at the lowermost layer, and is constructed by dividing an
image (picture) region into a predetermined size in the
horizontal direction and the vertical direction. The
25 disparity vector of each block is obtained, for example,
by selecting a disparity vector with the smallest value
or a negative disparity vector with the largest absolute
value from the disparity vectors of all pixels present in
the block.
30 [0050]
Next, as illustrated in Fig. 9(b), the disparity
26
SP336805WO0,0
information creating unit 115 obtains a disparity vector
of each group (Group Of Block) by using the disparity
vector of each block. The group corresponds to the upper
layer of the block, and is obtained by grouping a
5 plurality of adjacent blocks together. In the. example of
Fig. 9(b), each group includes four blocks bound by a
broken-line box. The disparity vector of each group is
obtained, for example, by selecting a disparity vector
with the smallest value or a negative disparity vector
10 with the largest absolute value from the disparity
vectors of all blocks in the group.
[0051]
Next, as illustrated in Fig. 9(c), the disparity
information creating unit 115 obtains a disparity vector
15 of each partition by using the disparity vector of each
group. The partition corresponds to the upper layer of
the group, and is obtained by grouping a plurality of
adjacent groups together. In the example of Fig. 9(c),
each partition includes two groups bound by a broken-line
20 box. The disparity vector of each partition is obtained,
for example, by selecting a disparity vector with the
smallest value or a negative disparity vector with the
largest absolute value from the disparity vectors of all
groups in the partition.
25 [0052]
Next, as illustrated in Fig. 9(d), the disparity
information creating unit 115 obtains a disparity vector
of the entire picture (entire image) located at the
uppermost layer by using the disparity vector of each
30 partition. In the example of Fig. 9(d), the entire
picture includes four partitions bound by a broken-line
27
SP336805WO00
box. The disparity vector of the entire picture is
obtained, for example, by selecting a disparity vector
with the smallest value or a negative disparity vector
with the largest absolute value from the disparity
5 vectors of all partitions included in the entire picture.
[0053]
In this way, the disparity information creating
unit 115 can obtain the disparity vector of each region
of each layer such as the block, the group, the partition,
10 and the entire picture by performing the downsizing
processing on the disparity vector of each pixel located
at the lowermost layer. Also, in the example of the
downsizing processing illustrated in Fig. 9, in addition
to the layer of the pixel, the disparity vectors of four
15 layers of the block, the group, the partition, and the
entire picture are finally obtained. However, the number
of layers, the method of dividing the region of each
layer, and the number of regions are not limited thereto.
[0054]
20 Returning to Fig. 2, the subtitle processing unit
116 can define a subregion in a region based on the
subtitle data generated by the subtitle generating unit
114. Also, the subtitle processing unit 116 sets
disparity information for shifting the display position
25 of the superimposition information in the left-eye image
and the right-eye image based on the disparity
information created by the disparity information creating
unit 115. The disparity information can be set for each
subregion, region or page.
30 [0055]
Fig. 10(a) illustrates an example of a region
28
SP336805WO00
defined on an image in the subtitle data and a subregion
defined in the region. In this example, two subregions
of SubRegion 1 and SubRegion 2 are defined in Region 0
with Region_Starting Position of RO. The horizontal
5 position x of the SubRegion 1 is SRI, and the horizontal
position X of the SubRegion 2 is SR2. In this example,
disparity information Disparity 1 is set for subregion
SubRegion 1, and disparity information Disparity 2 is set
for subregion SubRegion 2.
10 [0056]
Fig. 10(b) illustrates an example of the shift
adjustment in the subregion in the left-eye image by the
disparity information. Disparity information Disparity 1
is set for a subregion SubRegion 1. Therefore, as for
15 the subregion SubRegion 1, a shift adjustment is
performed such that the horizontal position x is SRI -
disparity 1. Also, disparity information Disparity 2 is
set for a subregion SubRegion 2. Therefore, as for the
subregion SubRegion 2, a shift adjustment is performed
20 such that the horizontal position x is SR2 - disparity 2.
[0057]
Fig. 10(c) illustrates an example of the shift
adjustment in the subregion in the right-eye image by the
disparity information. Disparity information Disparity 1
25 is set for a subregion SubRegion 1. Therefore, as for
the subregion SubRegion 1, a shift adjustment is
performed such that the horizontal position x is SRI +
disparity 1 as opposed to the above-described left-eye
image. Also, disparity information Disparity 2 is set
30 for a subregion SubRegion 2. Therefore, as for the
subregion SubRegion 2, a shift adjustment is performed
29
SP336805WO00
such that the horizontal position x is SR2 + disparity 2
as opposed to the above-described left-eye image.
[0058]
The subtitle processing unit 116 outputs display
5 control information such as the disparity information and
the region information of the above-described subregion,
together with the subtitle data generated by the subtitle
generating unit 114. Also, the disparity information may
also be set in units of a region or a page, in addition
10 to being set in units of a subregion as described above.
[0059]
The subtitle data includes segments such as DDS,
PCS, RCS, CDS, ODS, and EDS. The DDS (display definition
segment) specifies a display size for an HDTV. The PCS
15 (page composition segment) specifies a region position in
a page. The RCS (region composition segment) specifies a
size of a region or an encoding mode of an object, and
also specifies a starting position of the object.
[0060]
20 The CDS (CLUT definition segment) specifies a CLUT
content. The ODS (object data segment) includes encoded
pixel data Pixeldata. The EDS (end of display set
segment) indicates the end of the subtitle data starting
from the segment of DDS. In this embodiment, the segment
25 of DSS (Display Signaling Segment) is further defined.
The above-described display control information is
inserted into the segment of DSS.
[0061]
Returning to Fig. 2, the subtitle encoder 118
30 generates first and second private data streams (first
and second subtitle data streams). That is, the subtitle
30
SP336805WO00
encoder 118 generates the first private data stream (2D
stream) including the segments of DDS, PCS, RCS, CDS, ODS,
and EDS. Also, the subtitle encoder 118 generates the
second private data stream (3D extension stream)
5 including the segments of DDS, DSS, and EDS.
[0062]
The multiplexer 119 multiplexes the respective data
streams from the video encoder 112, the audio encoder 113,
and the subtitle encoder 118 to obtain a transport stream
10 TS as a multiplexed data stream. The transport stream TS
includes a video data stream, an audio data stream, and
first and second private data streams as PES (Packetized
Elementary Stream) streams.
[0063]
15 Fig. 11 illustrates a configuration of the first,
private data stream (2D stream) and the second private
data stream (3D extension stream) that are included in
the transport stream TS. Fig. 11(a) illustrates the 2D
stream, in which a PES header is disposed at the
20 beginning, followed by a PES payload including the
respective segments of DDS, PCS, RCS, CDS, ODS, and EDS.
[0064]
Also, Fig. 11(b) illustrates the 3D extension
stream, in which a PES header is disposed at the
25 beginning, followed by a PES payload including the
respective segments of DDS, DSS, and EDS. Also, as
illustrated in Fig. 11(c), the 3D extension stream may be
configured to include the respective segments of DDS, PCS,
DSS, and EDS in the PES payload. In this case, PAGE
30 STATE of PCS is NORMAL CASE, which indicates that there
is no change in superimposition data (bitmap).
31
SP336805WO00
[0065]
Herein, a page ID (page_id) of each segment
included in the 2D stream and a page ID (page_id) of each
segment included in the 3D extension stream are equal.
5 Accordingly, based on the page ID, a 3D-compatible
receiving apparatus of the receiving side can easily
combine the segment of the 2D stream and the segment of
the 3D extension stream.
[0066]
10 The multiplexer 119 includes synchronization
information for synchronization between the display by
the superimposition information data in the receiving
side and the shift control by the disparity information
in the 2D stream and the 3D extension stream.
15 Specifically, as illustrated in Fig. 12, the multiplexer
119 is set such that a value of a time stamp PTS
(Presentation Time Stamp) inserted into the PES header of
the 2D stream PESl(l): PES#1 is associated with a value
of a time stamp PTS inserted into the PES header of the
20 3D extension stream PES2(2): PES#2.
[0067]
Fig. 12 illustrates an example in which the values
of the time stamps PTS of the 2D stream and the 3D
extension stream are set to the same value, that is, PTSl.
25 In this case, in the receiving side (decoding side), the
display of a caption pattern by the subtitle data
(superimposition information data) is started from PTSl,
and the shift control by the disparity information for
displaying a caption pattern in 3D is also started from
30 PTSl.
[0068]
32
SP336805WO00
Also, in the example of Fig. 12, it is illustrated
that the 3D extension stream includes two pieces of
disparity information that are disparity information of a
PTSl frame and disparity information of a predetermined
5 subsequent frame. In the receiving side (decoding side),
it is indicated that the disparity information of an
arbitrary frame between the two frames can be obtained by
interpolation processing, and the shift control can be
dynamically performed.
10 [0059]
Also, in Fig. 12, "Conventional Segments" included
in the 2D stream represent the respective segments of DDS,
PCS, RCS, CDS, CDS, and EDS. Also, "Extended Segments"
included in the 3D extension stream represent the
15 respective segments of DDS, DSS, and EDS or the
respective streams of DDS, PCS, DSS, and EDS. Also, in
Fig. 12, "Elementary_PID" of the 2D stream is IDl, and
"Elementary_PID" of the 3D extension stream is ID2. The
same applies in Figs. 13 and 14 described below.
20 [0070]
Fig. 13 illustrates an example in which the values
of the time stamps PTS of the 2D stream and the 3D
extension stream are set to different values. That is,
in the example of Fig. 12, the value of the time stamp
25 ^TS of the 2D stream is set to PTSl, and the value of the
time stamp PTS of the 3D extension stream is set to PTS2
following PTSl. In this case, in the receiving side
(decoding side), the display of a caption pattern by the
subtitle data (superimposition information data) is
30 started from PTSl, and the shift control by the disparity
information for displaying a caption pattern in 3D is
33
SP336805WO00
started from PTS2.
[0071]
Also, in the example of Fig. 13, it is illustrated
that the 3D extension stream includes disparity
5 information of a PTS2 frame and disparity information of
a plurality of subsequent frames. In the receiving side
(decoding side), it is indicated that the disparity
information of an arbitrary frame between the plurality
of frames can be obtained by interpolation processing,
10 and the shift control can be dynamically performed.
[0072]
Like Fig. 13, Fig. 14 illustrates an example in
which the values of the time stamps PTS of the 2D stream
and the 3D extension stream are set to different values.
15 In addition. Fig. 14 illustrates an example in "which
there is a plurality of 3D extension streams with
different time stamp (PTS) values. That is, in the
example of Fig. 14, the value of the time stamp PTS of
the 2D stream is set to PTSl. Also, the values of the
20 time stamps PTS of a plurality of 3D extension frames are
set to PTS2, PTS3, PTS4, ... following PTSl.
[0073]
In this case, in the receiving side (decoding side),
the display of a caption pattern by the subtitle data
25 (superimposition information data) is started from PTSl.
Also, the shift control by the disparity information for
displaying a caption pattern in 3D is started from PTS2,
and then sequential update is performed. Also, in the
example of Fig. 14, the plurality of 3D extension streams
30 includes only the disparity information represented by
the respective type stamps. In the receiving side
34
SP336805WO00
(decoding side), it is indicated that the disparity
information of an arbitrary frame between the plurality
of frames can be obtained by interpolation processing,
and the shift control can be dynamically performed.
5 [0074]
An operation of the transmission data generating
unit 110 illustrated in Fig. 2 will be described briefly.
The stereoscopic image data extracted from the data
extracting unit 111 is supplied to the video encoder 112.
10 In the video encoder 112, encoding such as MPEG4-AVC,
MPEG2, or VC-1 is performed on the stereoscopic image
data, and a video data stream (video elementary stream)
including the encoded video data is generated. The video
data stream is supplied to the multiplexer 119.
15 [0075]
The audio data extracted from the data extracting
unit 111 is supplied to the audio encoder 113. In the
audio encoder 113, encoding such as MPEG-2 Audio AAC or
MPEG-4 AAC is performed on the audio data, and an audio
20 data stream including the encoded audio data is generated.
The audio data stream is supplied to the multiplexer 119.
[0076]
In the subtitle generating unit 114, subtitle data
being DVB caption data (for a 2D image) is generated.
25 The subtitle data is supplied to the disparity
information creating unit 115 and the subtitle processing
unit 116.
[0077]
The disparity vector for each pixel extracted from
30 the data extracting unit 111 is supplied to the disparity
information creating unit. 115. In the disparity
35
SP336805WO00
information creating unit 115, downsizing processing is
performed on the disparity vector for each pixel or the
disparity vector for a plurality of pixels, and disparity
information of each layer is generated. The disparity
5 information is supplied to the subtitle processing unit
116.
[0078]
In the subtitle processing unit 116, for example, a
subregion in a region is defined based on the subtitle
10 data generated by the subtitle generating unit 114. Also,
in the subtitle processing unit 116, disparity
information for shifting the display position of the
superimposition information in the left-eye image and the
right-eye image is set based on the disparity information
15 created by the disparity information creating unit 115.
In this case, the disparity information is set for each
subregion, region or page.
[0079]
The display control information and the subtitle
20 data output from the subtitle processing unit 116 are
supplied to the subtitle encoder 118. The display
control information includes the region information of a
subregion, the disparity information, and the lijce. In
the subtitle encoder 118, first and second two private
25 data streams (elementary streams) are generated.
[0080]
That is, the first private data stream (2D stream)
including the respective segments of DDS, PCS, RCS, CDS,
ODS, and EDS is generated. In addition, the second
30 private data stream (3D extension stream) including the
respective segments of DDS, DSS, and EDS is generated.
36
SP336805WO00
As described above, the segment of DSS is the segment
including the display control information.
[0081]
As described above, the respective data streams
5 from the video encoder 112, the audio encoder 113, and
the subtitle encoder 118 are supplied to the multiplexer
119. In the multiplexer 119, the respective data streams
are packetized and multiplexed into a PES packet, and a
transport stream TS is obtained as a multiplexed data
10 stream. The transport stream TS includes a video data
stream, an audio data stream, and first and second
private data streams (first and second subtitle data
streams) as PES streams.
[0082]
15 Fig. 15 illustrates an example of a configuration
of the transport stream TS. In addition, in Fig. 15, the
illustration of video and audio-related portions is
omitted for simplicity of illustration. The transport
stream TS includes a PES packet that is obtained by
20 packetizing each elementary stream.
[0083]
In this configuration example, the PES packet
Subtitle PESl of the 2D stream (first private data
stream) and the PES packet Subtitle PES2 of the 3D
25 extension stream (second private data stream) are
included. The 2D stream (PES stream) includes the
respective segments of DDS, PCS, RCS, CDS, ODS, and EDS.
The 3D extension stream (PES stream) includes the
respective segments of DDS, DSS, and EDS. In this case,
30 "Elementary_PID" of the 2D stream and "Elementary_PID" of
the 3D extension stream are set differently to PIDl and
37
SP336805WO00
PID2, thus indicating that these streams are different
PES streams.
[0084]
Fig. 16 illustrates a structure of PCS
5 (page_composition_segment). The segment type of the PCS
is 0x10 as illustrated in Fig. 17.
Region_Horizontal_Address and Rregion_Vertical_Address
indicate the starting position of a region. Also, with
respect to the other segments such as DDS, RCS, and CDS,
10 the illustration of structures thereof will be omitted.
As illustrated in Fig. 17, the segment type of DDS is
0x14, the segment type of RCS is 0x11, the segment type
of CDS is 0x12, the segment type of CDS is 0x13, and the
segment type of EDS is 0x80. In addition, as illustrated
15 in Fig. 17, the segment type of the DSS is 0x15. A
detailed structure of the segment of DSS will be
described below.
[0085]
Returning to Fig. 15, the transport stream TS also
2 0 includes a PMT (Program Map Table) as PSI (Program
Specific Information). The PSI is information describing
to which program, each elementary stream included in the
transport stream belongs. Also, the transport stream
includes an EIT (Event Information Table) as SI (Serviced
25 Information) for performing management on each event.
The EIT describes metadata for each program.
[0086]
A subtitle descriptor (Subtitling_descriptor)
representing the content of a subtitle is inserted into
30 the PMT. Also, a component descriptor
(Component_descriptor) representing the content of a
38
SP336805WO00
delivery for each stream is inserted into the EIT. As
illustrated in Fig. 18, in the case where the
"stream_content" of the component descriptor represents a
subtitle, when the "component_type" is 0x15 of 0x25, it
5 represents a 3D subtitle; and when the "component_type"
is any other value, it represents a 2D subtitle. As
illustrated in Fig. 15, the "subtitling_type" of the
subtitle descriptor is set to the value of
"component_type".
10 [0087]
Also, the PMT includes a subtitle elementary loop
having information related to a subtitle elementary
stream. In the subtitle elementary loop, information
such as a packet identifier (PID) is disposed for each
15 stream, and a descriptor describing information related
to the elementary stream is also disposed, although not
illustrated.
[0088]
Fig. 19 illustrates the extraction of the subtitle
20 descriptor (Subtitling_descriptor) and the component
descriptor (Component_descriptor) illustrated in Fig. 15.
Fig. 20 illustrates the extraction of the PES streams (2D
stream and 3D extension stream) illustrated in Fig. 15.
[0089]
25 A composition page ID (composition_page_id) of the
subtitle descriptor is set to indicate that each segment
included in the 3D extension stream is associated with
each segment of the 2D stream. That is, the
"composition_page_id" in the 3D extension stream and the
30 "composition_page_id" in the 2D stream are set to share
the same value (OxXXXX in the figure). Also, in order to
39
SP336805WO00
indicate that each segment included in the 3D extension
stream is associated with each segment of the 2D stream,
the "page_ids" of the respective associated segments in
both of the PES streams are encoded such as to have the
5 same value.
[0090]
In the subtitle descriptor and the component
descriptor, for example, an ISO language code
(ISO_639_language_code) is described as language
10 information. The ISO language code of the descriptor
corresponding to the 2D stream is set to represent the
language of a subtitle (caption). In the illustrated
example, the ISO language code is set to "eng"
representing English. The 3D extension stream has a
15 segment of the DSS with disparity information, but does
not have a segment of the ODS. Therefore, -the 3D
extension stream does not depend on languages. The ISO
language code described in the descriptor corresponding
to the 3D extension stream'is set to, for example, "zxx"
20 representing a non-language.'
[0091]
Also, it can be considered that any one of language
codes included in a space from "qaa" to "qrz" of an ISO
language code is used as an ISO language code
25 representing a non-language. Fig. 21 illustrates the
extraction of the subtitle descriptor
(Subtitling_descriptor) and the component descriptor
(Component_descriptor) in that case. Also, for reference.
Fig. 22 illustrates the extraction of an ISO language
30 code (ISO 639-2 Code) list.
[0092]
40
SP336805WO00
Fig. 23 illustrates an example of a stream
configuration of subtitle data streams (2D stream and 3D
extension stream). This example is an example of service
of two languages that are English "eng" and German "ger".
5 The 3D extension stream is included as
"composition_page_id" in the respective language services,
and is designated as "subtitling_type=3D",
"ISO_639_language_code=zxx".
[0093]
10 Fig. 24 illustrates an example of a syntax of the
component descriptor (Component_descriptor). An 8-bit
field of "descriptor_tag" indicates that the descriptor
is a component descriptor. An 8-bit field of
"descriptor_length" represents the entire byte size
15 following the field.
[0094]
A 4-bit field of "stream_content" represents the
stream type of a main stream such as a video, an audio,
and a subtitle. A 4-bit field of "component_type"
20 represents the component type of a main stream such as a
video, an audio, and a subtitle.
[0095]
In the case where the main stream is the 2D stream,
that is, in the component descriptor corresponding to the
25 2D stream, the "stream_content" is a subtitle "subtitle",
and the "component_type" is a two-dimensional "2D". Also,
in the case where the main stream is the 3D extension
stream, that is, in the component descriptor
corresponding to the 3D extension stream, the
30 "stream_content" is a subtitle "subtitle", and the
"component_type" is a three-dimensional "3D".
41
SP336805WO00
[0096]
An 8-bit field of the "component_tag" has the same
value as the "component_tag" in the stream identifier
descriptor (stream_identifier descriptor) corresponding
5 to the main stream. Accordingly, the stream identifier
descriptor and the component descriptor are associated
with the "component_tag". A 24-bit field of i
"ISO_639_language_code" represents an ISO language code.
[0097]
10 Fig. 25 illustrates an example of a syntax of the
subtitle descriptor (Subtitling_descriptor). An 8-bit
field of "descriptor_tag" indicates that the descriptor
is a subtitle descriptor. An 8-bit field of
"descriptor_length" represents the entire byte size
15 following the field.
[0098]
A 24-bit field of "ISO_639_language_code"
represents an ISO language code. An 8-bit field of
"subtitling_type" represents subtitle type information.
20 In the case where the main stream is the 2D stream, that
is, in the subtitle descriptor corresponding to the 2D
stream, the "subtitling_type" is "2D". On the other hand,
in the case where the main stream is the 3D extension
stream, that is, in the subtitle descriptor corresponding
25 to the 3D extension stream, the "subtitling_type" is "3D".
A 16-bit field of "composition_page_id" represents a
composition page ID, and has the same value as a page ID
(page_id) of a segment included in the main stream.
[0099]
30 [Update of Disparity Information]
As described above, the disparity information is
42
SP336805WO00
transmitted by the 3D extension stream. The update of
the disparity information will be described.
[0100]
Figs. 2 6 and 27 illustrate examples of the
5 disparity information update using an interval period.
Fig. 26 illustrates the case where an interval period is
fixed and is equal to an update period. That is, each of
the update periods of A-B, B-C, C-D, ... includes one
interval period.
10 [0101]
Fig. 27 corresponds to a general case, and
illustrates an example of the disparity information
update in the case where an interval period is set to be
a short period (may be, for example, a frame period). In
15 this case, the numbers of interval periods in the
respective update periods are M, N, P, Q, and R. Also,
in Figs. 2 6 and 27, "A" represents a starting frame
(starting point) of a caption display period, and "B" to
"F" represent subsequent update frames (update points).
20 [0102]
When the disparity information sequentially updated
in the caption display period is transmitted to the
receiving side (set-top box 200 or the like), the
receiving side can generate and use disparity information
25 of an arbitrary frame interval, for example, a 1-frame
interval, by performing interpolation processing on the
disparity information for each update period.
[0103]
Fig. 28 illustrates an example of a configuration
30 of the 3D extension stream. Also, in this configuration
example, although the case where the respective segments
43
SP336805WO00
of DDS, DSS, and EDS are included in a PES data payload
(PES data Payload) are illustrated, the same applies to
the case where the respective segments of DDS, PCS, DSS,
and EDS are included in the PES data payload.
5 [0104]
Fig. 28(a) illustrates an example in which only one
DSS segment is inserted. A PES header includes time
information (PTS). Also, the respective segments of DDS,
DSS, and EDS are included as PES payload data. These are
10 transmitted together before the start of a subtitle
display period. The DSS segment may include a plurality
of pieces of disparity information sequentially updated
in the caption display period.
[0105]
15 Also, the DSS segment may not include a plurality
of pieces of disparity information sequentially updated
in the caption display period, and the plurality of
pieces of disparity information may be transmitted to the
receiving side (set-top box 200 or the like). In this
2 0 case, a DSS segment is inserted into the 3D extension
stream at each update timing. Fig. 28(b) illustrates an
example of the configuration of the 3D extension stream
in this case.
[0106]
25 Fig. 29 illustrates an example of the disparity
information update in the case where the DSS segments are
sequentially transmitted as illustrated in Fig. 28(b) (
described above. Also, in Fig. 29, "A" represents a
starting frame (starting point) of a caption display
30 period, and "B" to "F" represent subsequent update frames
(update points).
44
SP336805WO00
[0107]
Also in the case where the disparity information
sequentially updated in the caption display period is
transmitted to the receiving side (set-top box 200 or the
5 like) by sequentially transmitting the DSS segments, the
receiving side can also perform the above-described
processing. That is, also in this case, the receiving
side can generate and use disparity information of an
arbitrary frame interval, for example, a 1-frame interval,
10 by performing interpolation processing on the disparity
information for each update period.
[0108]
Fig. 30 illustrates an example of the disparity
information update as illustrated in Fig. 27 described
15 above. An update frame interval is expressed in a
multiple of an interval duration (ID) ks a unit period.
For example, an update frame interval Division Period 1
is expressed as "ID*M", an update frame interval Division
Period 2 is expressed as "ID*N", and the subsequent
20 update frame intervals are expressed likewise. In the
example of the disparity information update illustrated
in Fig. 30, the update frame interval is not fixed, and
the update frame interval is set according to a disparity
information curve.
25 [0109]
Also, in the example of the disparity information
update, in the receiving side, a starting frame (starting
time) Tl_0 of the caption display period is provided as a
PTS (Presentation Time Stamp), that is inserted into the
30 header of a PES stream including the disparity
information. In the receiving side, each update time of
45
SP336805WO00
the disparity information is obtained based on
information about an interval duration (information about
a unit period), which is information about each update
frame interval, and information about the number of
5 interval durations.
[0110]
In this case, from the starting frame (starting
time) T1_0 of the caption display period, the respective
update times are sequentially obtained based on the
10 following equation (1). In Equation (1),
"interval_count" denotes the number of interval periods,
and are values corresponding to M, N, P, Q, R, and S in
Fig. 30. Also, in Equation (1), "interval_time" is a
value corresponding to the interval duration ID in Fig.
15 30.
[0111]
Tm_n = Tm_(n - 1) + (interval_time*interval_count)
...(1)
[0112]
20 For example, in the update example illustrated in
Fig. 30, each update time is obtained based on Equation
(1) in the following manner. That is, the update time
Tl_l is obtained as Tl_l = T1_0 + (ID*M) by using the
starting time T1_0, the interval duration ID, and the
25 number M. Also, the update time Tl_2 is obtained as Tl_2
= Tl_l + (ID*N) by using the update time Tl_l, the
interval duration ID, and the number N. Each of the
subsequent update times is obtained in the same manner.
[0113]
30 In the update example illustrated in Fig. 30, in
the receiving side, interpolation processing is performed
46
SP336805WO00
on the disparity information sequentially updated in the
caption display period, and the disparity information of
an arbitrary frame interval in the caption display period,
for example, a 1-frame interval is generated and used.
5 For example, as the above interpolation processing, by
performing not linear interpolation processing but
interpolation processing accompanied with low-pass filter
(LPF) processing in the time direction (frame direction),
a change in the disparity information of a predetermined
10 frame interval in the time direction (frame direction)
after the interpolation processing becomes smooth. A
broken line a of Fig. 30 represents an example of an LPF
output.
[0114]
15 Fig. 31 illustrates an example of the display of a
subtitle as a caption. In the display example, a page
region (Area for Page_default) includes two regions
(Regionl and Region2) as a caption display region. The
region includes one or more subregions. Herein, it is
2 0 assumed that the region includes one subregion, and the
region and the subregion are the same.
[0115]
Fig. 32 illustrates an example of the disparity
information curve of each region and page in the case
25 where disparity information in units of a region and
disparity information in units of a page are included in
a DSS segment, as disparity information that is
sequentially updated in the caption display period.
Herein, the disparity information curve of the page takes
30 the minimum value of the disparity information curve of
two regions.
47
SP336805WO00
[0116]
About the Regionl, there are seven pieces of
disparity information that are a starting time T1_0. and
subsequent update times Tl_l, Tl_2, Tl_3, ..., Tl_6. Also,
5 about the Region2, there are eight pieces of disparity
information that are a starting time T2_0 and subsequent
update times T2_l, T2_2, T2_3, ..., T2_7. In addition,
about the page (Page_default), there are seven pieces of
disparity information that are a starting time T0_0 and
10 subsequent update times T0_1, T0_2, T0_3, ..., T0_6.
[0117]
Fig. 33 illustrates a transmission structure of the
disparity information of each page and region illustrated
in Fig. 32. First, a page layer will be described. A
15 fixed value "page_default_disparity" of the disparity
information is disposed in the page layer. As for the
disparity information sequentially updated in the caption
display period, "interval_count" representing the number
of interval periods corresponding to a starting time and
20 subsequent update times, and "disparity_page_updete"
representing the disparity information are sequentially
disposed. Also, the "interval_count" at the starting
time is set to "0".
[0118]
25 Next, a region layer will be described. As for the
Regionl (Subregionl) , "subregion_disparity_integer_part"
and "subregion_disparity_fractional_part" being the fixed
values of the disparity information are disposed. Herein,
"subregion_disparity_integer_part" represents an integer
30 part of the disparity information, and
"subregion_disparity_fractional_part" represents a
48
SP336805WO00
fractional part of the disparity information.
[0119]
As for the disparity information sequentially
updated in the caption display period, "interval_count"
5 representing the number of interval periods corresponding
to a starting time and subsequent update times, and
"disparity_region_updete_integer_part" and
"disparity_region_updete_fractional_part" representing
the disparity information are sequentially disposed.
10 Herein, "disparity_region_updete_integer_part" represents
an integer part of the disparity information, and
"disparity_region_updete_fractional_part" represents a
fractional part of the disparity information. Also, the
"interval_count" at the starting time is set to "0".
15 [0120]
Like the above-described Regionl, as for the
Region2 (Subregion2), "subregion_disparity_integer_part"
and "subregion_disparity_fractional_part" being the fixed
values of the disparity information are disposed. As for
20 the disparity information sequentially updated in the
caption display period, "interval_count" representing the
number of interval periods corresponding to a starting
time and subsequent update times, and
"disparity_region_updete_integer_part" and
25 "disparity_region_updete_fractional_part" representing
the disparity information are sequentially disposed.
[0121]
Figs. 34 to 36 illustrate examples of the syntax of
a DSS (Disparity_Signaling_Segment). Figs. 37 to 40
30 illustrate the main data definition contents (semantics)
of a DSS. This syntax includes respective pieces of
49
SP336805WO00
information of "sync_byte", "segment_type", "page_id",
"segment_length", and "dss_version_number". The
"segment_type" is 8-bit data representing a segment type,
and herein is a value representing the DSS. The
5 "segment_length" is 8-bit data representing the number of
subsequent bytes.
[0122]
A 1-bit flag of
"disparity_shift_update_sequence_page_flag" indicates
10 whether disparity information sequentially updated in the
caption display period is present as disparity
information in units of a page. "1" represents presence,
and "0" represents absence. An 8-bit field of
"page_default_disparity_shift" represents fixed disparity
15 information in units of a page, that is, disparity
information that is commonly used in the caption display
period. When the flag of
"disparity_page_update_sequence_flag" described above is
"1", the reading of "disparity_shift_update_sequence()"
20 is performed.
[0123]
Fig. 36 illustrates an example of the syntax of
"disparity_shift_update_sequence()". The
"disparity_page_update_sequence_length" is 8-bit data
25 representing the number of subsequent bytes. A 24-bit
field of "interval_duration[23..0]" specifies an interval
duration (see Fig. 30) as a unit period in units of 90
KHz. That is, the "interval_ duration[23..0]" represents
a 24-bit value of the interval duration measured with a
30 90 KHz clock.
[0124]
50
SP336805WO00
The reason for being the 24-bit length with respect
to the 33-bit length of the PTS inserted into a header
portion of the PES is as follows. That is, a time
exceeding 24 hours can be represented by the 33-bit
5 length, but it is an unnecessary length as the interval
duration in the caption display period. Also, by the 24-
bit representation, the data size can be reduced and
compact transmission can be performed. Also, 24 bits is
8x3 bits, and byte alignment is facilitated.
10 [0125]
An 8-bit field of "division_period_count"
represents the number of division periods that are
influenced by the disparity information. For example, in
the case of the update example illustrated in Fig. 30,
15 the number of division periods is "7" corresponding to
the starting time T1_0 and the subsequent update times
Tl_l to Tl_6. A "for" loop below is repeated the number
of times represented by the 8-bit field of
"division_period_count".
20 [0126]
An 8-bit field of "interval_count" represents the
number of interval periods. For example, in the case of
the update example illustrated in Fig. 30, it correspond
to M, N, P, Q, R, and S. An 8-bit field of
25 "disparity_shift_update_integer_part" represents the
disparity information. The "interval_count" is "0"
corresponding to the disparity information at the
starting time (the initial value of the disparity
information). That is, when the "interval_count" is "0",
30 the "disparity_page_update" represents the disparity
information at the starting time (the initial value of
51
SP336805WO00
the disparity information).
[0127]
A "while" loop of Fig. 34 is repeated when the data
length processed up to that time (processed_length) does
5 not reach the segment data length (segment_length). In
the "while" loop, the disparity information in units of a
region or a subregion in the region is disposed. Herein,
the region includes one or more subregions, and the
region and the subregion may be the same.
10 [0128]
Information of "region_id" is included in the
"while" loop. A 1-bit flag of
"disparity_shift_update_sequence_region_flag" is flag
information indicating whether there is
15 "disparity_shift_update_sequence()" for all the
subregions in the region. A 2-bit field of •
"number_of_subregions_minus_l" represents the value equal
to the number of subregions in the region minus 1. When
number_of_subregions_minus_l = 0, the region includes one
2 0 subregion that has•the same dimension as the region.
[0129]
When number_of_subregions_minus_l > 0, the region
includes a plurality of subregions divided in the
horizontal direction. In the "for" loop of Fig. 35,
25 information of "subregion_horizontal_position" and
"subregion_width" corresponding to the number of
subregions is included. A 16-bit field of
"subregion_horizontal_position" represents the pixel
position of the left of the subregion. The
30 "subregion_width" represents the horizontal width of the
subregion with the number of pixels.
52
SP336805WO00
[0130]
An 8-bit field of
"subregion_ciisparity_shift_integer_part" represents fixed
disparity information in units of a region (in units of a
5 subregion), that is, an integer part of the disparity
information that is commonly used in the caption display
period. A 4-bit field of
"subregion_disparity_shift_fractional_part" represents
fixed disparity information in units of a region (in
10 units of a subregion), that is, a fractional part of the
disparity information that is commonly used in the
caption display period. When the flag of
"disparity_shift_update_sequence_region_flag" described
above is "1", the reading of
15 "disparity_shift_update_sequence0" (see Fig. 36) is
performed.
[0131]
[Concept of Broadcast Reception]
Fig. 41 illustrates the concept of broadcast
20 reception in the case where a set-top box 200 and a
television receiver 300 are 3D-compatible devices. In
this case, in the broadcasting station 100, a subregion
SR 00 is defined in a region Region 0, and the disparity
information Disparity 1 is set. Herein, it is assumed
25 that the region Region 0 and the subregion SR 00 are the
same region. Together with the stereoscopic image data,
the subtitle data and the display control information
(disparity information Disparity 1 and region information
Position of the subregion) are transmitted from the
30 broadcasting station 100.
[0132]
53
SP336805WO00
First, a description will be given of the case of
reception by the set-top box 200 that is a 3D-compatible
device. In this case, the set-top box 200 reads the
respective segment data constituting the subtitle data
5 from the 2D stream, reads the DSS segment data including
the display control information such as the disparity
information from the 3D extension stream, and uses the
read data. In this case, the set-top box 200 extracts a
2D stream and a 3D extension stream of a language
10 selected by a user from the transport stream TS based on
the subtitle type information _and the language
information, and decodes the 2D stream and the 3D
extension stream extracted.
[0133]
15 As described above, in the transport stream TS, the
subtitle descriptors are inserted in association with the
2D stream and the 3D extension stream (see Fig. 15). The
subtitle type, information "subtitling_type" of the
subtitle descriptor corresponding to the 2D stream is set
20 to "2D". Also, the subtitle type information
"subtitling_type" of the subtitle descriptor
corresponding to the 3D stream is set to "3D".
[0134]
Also, as described above, in the transport stream
25 TS, the component descriptor and the subtitle descriptor
are respectively inserted in association with the 2D
stream and the 3D extension stream (see Fig. 15). The
language information (ISO language code) of the
descriptor corresponding to the 2D stream is set to
30 represent a language, and the language information (ISO
language code) of the descriptor corresponding to the 3D
54
SP336805WO00
extension stream is set to represent a non-language.
[0135]
The set-top box 200 is a 3D-compatible device.
Therefore, based on the subtitle type information, the
5 set-top box 2 00 determines the 2D stream corresponding to
a subtitle type "2D(HD,SD)" and the 3D extension stream
corresponding to a subtitle type "3D" as a stream to be
extracted (see a "O" mark of Fig. 42 that will be
described below). Also, the set-top box 200 determines
10 the 2D stream of the language selected by the user and
the 3D extension stream with the language information
(ISO language code) representing a non-language as a
stream to be extracted (see a "O" mark of Fig. 42 that
will be described below).
15 [0136]
Fig. 42 schematically illustrates the abovedescribed
2D stream and 3D extension stream extraction
processing of the set-top box 200, for example, in the
case where the language service selected by the user is
20 English "eng" in the stream configuration example
illustrated in Fig. 23 described above. In this case, as
a result, based on the subtitle type information and the
language information, the 2D stream corresponding to
English "eng" and the 3D extension stream included in
25 "composition_page_id=A" among the language services are
determined as a stream to be extracted.
[0137]
The set-top box 200 generates region display data
for displaying a subtitle, based on the subtitle data.
30 The set-top box 200 obtains output stereoscopic image
data by superimposing the region display data on a left-
55
SP336805WO00
eye image frame (frameO) portion and a right-eye image
frame (framel) portion constituting the stereoscopic
image data.
[0138]
5 Herein, the set-top box 200 shifts the positions of
the respective superimposed display data based on the
disparity information. Also, the set-top box 200 changes
the superimposition position, the size, and the like
appropriately according to a transmission format of the
10 stereoscopic image data (Side By Side scheme. Top &
Bottom scheme, Frame Sequential scheme, or a format
scheme in which each view has a full-screen size).
[0139]
The set-top box 2 00 transmits the output
15 stereoscopic image data obtained as described above, to
the 3D-compatible television receiver 300 through, for
example, an HDMI digital interface. The television
receiver 300 performs 3D signal processing on the
stereoscopic image data received from the set-top box 200,
2 0 to generate left-eye image data and right-eye image data
on which the subtitle is superimposed. The television
receiver 300 displays a binocular disparity image (lefteye
image and right-eye image) on a display panel such as
an LCD to allow a user to recognize a stereoscopic image.
25 [0140]
Next, a description will be given of the case of
reception by the television receiver 300 that is a 3Dcompatible
device. In this case, the television receiver
300 reads the respective segment data constituting the
30 subtitle data from the 2D stream, reads the DSS segment
data including the display control information such as
56
SP336805WO00
the disparity information from the 3D extension stream,
and uses the read data. In this case, like the abovedescribed
set-top box 200, based on the subtitle type
information, the language information, or the like, the
5 television receiver 300 extracts the 2D stream and the 3D
extension stream of the language selected by the user
from the transport stream TS, and decodes the 2D stream
and the 3D extension stream extracted.
[0141]
10 The television receiver 300 generates region
display data for displaying a subtitle, based on the
subtitle data. The television receiver 300 superimposes
the region display data on the left-eye image data and
the right-eye image data obtained by performing
15 processing according to a transmission format on the
stereoscopic image data, to generate left-eye image data
and right-eye image data on which the subtitle is
superimposed. The television receiver 300 displays a
binocular disparity image {left-eye image and right-eye
20 image) on a display panel such as an LCD to allow a user
to recognize a stereoscopic image.
[0142]
Fig. 43 illustrates the concept of broadcast
reception in the case where the set-top box 200 and the
25 television receiver 300 are legacy 2D-compatible devices.
In this case, as in the case of Fig. 41, together with
the stereoscopic image data, the subtitle data and the
display control information (disparity information
"Disparity 1" and region information "Position" of the
30 subregion) are transmitted from the broadcasting station
100.
57
SP336805WO00
[0143]
First, a description will be given of the case of
reception by the set-top box 200 that is a legacy 2Dcompatible
device. In this case, the set-top box 200
5 reads the respective segment data constituting the
subtitle data from the 2D stream, and uses the read data.
In this case, the set-top box 200 extracts only the 2D
stream of the language selected by the user from the
transport stream TS based on the subtitle type
10 information, the language information, or the like, and
decodes the 2D stream extracted. That is, since the settop
box 200 does not read the DSS segment including the
display control information such as the disparity
information, the reception processing can be prevented
15 from being interrupted by the reading.
[0144]
As described above, in the transport stream TS, the
subtitle descriptors are inserted in association with the
2D stream and the 3D extension stream (see Fig. 15). The
20 subtitle type information "subtitling_type" of the
subtitle descriptor corresponding to the 2D stream is set
to "2D". Also, the subtitle type information
"subtitling_type" of the subtitle descriptor
corresponding to the 3D stream is set to "3D".
25 [0145]
Also, as described above, in the transport stream
TS, the component descriptor and the subtitle descriptor
are respectively inserted in association with th^ 2D
stream and the 3D extension stream {see Fig. 15). The
30 language information (ISO language code) of the
descriptor corresponding to the 2D stream is set to
58
SP336805WO00
"»
represent a language, and the language information (ISO
language code) of the descriptor corresponding to the 3D
extension stream is set to represent a non-language.
[0146]
5 The set-top box 200 is a 2D-compatible device.
•Therefore, based on the subtitle type information, the
set-top box 200 determines the 2D stream corresponding to
a subtitle type "2D(HD,SD)" as a stream to be extracted.
Also, the set-top box 200 determines the 2D stream of the
10 language selected by the user as a stream to be extracted.
[0147]
Fig. 44 schematically illustrates the 2D stream
extraction processing of the set-top box 200, for example,
in the case where the language service selected by the
15 user is English "eng" in the stream configuration example
illustrated in Fig. 23 described above. In this case,
based on the subtitle type information, the set-top box
200 can determine the 2D stream corresponding to a
subtitle type "2D(HD,SD)" as a stream to be extracted.
20 Herein, since the set-top box 200 is a legacy 2Dcompatible
device and cannot interpret the subtitle type
"3D", the set-top box 200 may also determine the 3D
extension stream as a stream to be extracted (illustrated
as a mark "A").
25 [0148]
However, in this case, based on the language
information, the set-top box 200 determines the 2D stream
corresponding to English "eng" as a stream to be
extracted, and does not determine the 3D extension stream
30 with the language information representing a non-language
as a stream to be extracted (illustrated as a mark "x").
59
SP336805WO00
As a result, the set-top box 200 determines only the 2D
stream corresponding to English "eng" as a, stream to be
extracted. Accordingly, since the set-top box 200 can
more securely prevent decoding processing from being
5 performed on the 3D extension stream including the DSS
segment having the disparity information, the reception
processing thereof can be prevented from being
interrupted by the decoding processing.
[0149]
10 The set-top box 200 generates region display data
for displaying a subtitle, based on the subtitle data.
The set-top box 200 obtains output 20 image data by
superimposing the region display data on the 2D image
data that has been obtained by performing the processing
15 according to the transmission format on the stereoscopic
image data.
[0150]
The set-top box 2 00 transmits the output 2D image
data obtained as described above, to the television
20 receiver 300 through, for example, an HDMI digital
interface. The television receiver 300 displays a 2D
image according to the 2D image data received from the
set-top box 200.
[0151]
25 Next, a description will be given of the case of
reception by the television receiver 300 that is a 2Dcompatible
device. In this case, the television receiver
300 reads the respective segment data constituting the
subtitle data from the 2D stream, and uses the read data.
30 In this case, like the set-top box 200 described above,
the television receiver 300 extracts only the 2D stream
60
SP336805WO00
of the language selected by the user from the transport
stream TS based on the subtitle type information and the
language information, and decodes the 2D stream extracted.
That is, since the television receiver 300 does not read
5 the DSS segment including the display control information
such as the disparity information, the reception
processing can be prevented from being interrupted by the
reading.
[0152]
10 The television receiver 300 generates region
display data for displaying a subtitle, based on the
subtitle data. The television receiver 300 obtains 2D
image data by superimposing the region display data on
the 2D image data that has been obtained by performing
15 the processing according to the transmission format on
the stereoscopic image data. The television receiver 300
displays a 2D image according to the 2D image data.
[0153]
Fig. 45 illustrates the summarization of the
20 concept of broadcast reception in the case where the
above-described receiver (set-top box 200, television
receiver 300) is a legacy 2D-compatible device {2D
receiver) and in the case where the receiver is a 3Dcompatible
device (3D receiver). Also, in Fig. 45, a
25 stereoscopic image data (3D image data) transmission
scheme is a Side By Side scheme.
[0154]
Also, in the case of the 3D-compatible device (3D
receiver), a 3D mode or a 2D mode can be selected. When
30 the 3D mode is selected by the user, the case is the same
as described with reference to Fig. 41. When the 2D mode
61
SP336805WO00
is selected by the user, the case is the same as the case
of the 2D-compatible device (2D receiver) described with
reference to Fig. 43.
[0155]
5 In the transmission data generating unit 110
illustrated in Fig. 2, the transport stream TS output as
a multiplexed data stream from the multiplexer 119
includes two private data streams (subtitle data streams).
That is, a 2D stream and a 3D extension stream are
10 included in the transport stream TS (see Fig. 11).
[0156]
Therefore, in an legacy 2D-compatible receiving
apparatus of the receiving side, the reception processing
can be performed by reading only the respective segments
15 constituting the subtitle data from the 2D stream. That
is, in the 2D-compatible receiving apparatus, since the
DSS segment need not be read from the 3D extension stream,
the reception processing can be prevented from being
interrupted by the reading.
20 [0157]
Also, in the transmission data generating unit 110
illustrated in Fig. 2, in the component descriptor and
the subtitle descriptor inserted into the transport
stream TS in association with the 2D stream and the 3D
25 extension stream, the subtitle type information and the
language information are set to identify the respective
streams. Therefore, based on the corresponding subtitle
type information and the language information, the 2Dcompatible
receiving apparatus can extract only the 2D
30 stream of the language selected by the user from the
transport stream TS and decode the 2D stream easily with
62
SP336805WO00
a high accuracy. Accordingly, since the 2D-compatible
receiving apparatus can more securely prevent decoding
processing from being performed on the 3D extension
stream including the DSS segment having the disparity
5 information, the reception processing thereof can be
prevented from being interrupted by the decoding
processing.
[0158]
Also, in the transmission data generating unit 110
10 illustrated in Fig. 2, page IDs (page_id) of the
respective segments included in the 2D stream and the 3D
extension stream inserted into the transport stream TS
are equal. Therefore, based on the page ID, a 3Dcompatible
receiving apparatus of the receiving side can
15 easily combine the segment of the 2D stream and the
segment of the 3D extension stream.
[0159]
Also, in the transmission data generating unit 110
illustrated in Fig. 2, since the DSS segment including
20 the disparity information sequentially updated in the
subtitle display period can be transmitted, the display
positions of the left-eye subtitle and the right-eye
subtitle can be dynamically controlled. Accordingly, in
the receiving side, the disparity provided between the
25 left-eye subtitle and the right-eye subtitle can be
dynamically changed in conjunction with a change in the
image content.
[0160]
Also, in the transmission data generating unit 110
30 illustrated in Fig. 2, the disparity information of the
frame for each update frame interval included in the DSS
63
SP336805WO00
segment obtained by the subtitle encoder 118 is not an
offset value from the previous disparity information, but
is the disparity information itself. Therefore, in the
receiving side, even when an error occurs in the
5 interpolation process, the recovery from the error can be
performed within a predetermined delay time.
[0161]
[Description of Set-top Box]
Returning to Fig. 1, the set-top box 200 receives a
10 transport stream TS that is transmitted on a broadcast
wave from the broadcasting station 100. The transport
stream TS includes audio data and stereoscopic image data
including left-eye image data and right-eye image data.
Also, the transport stream TS further includes subtitle
15 data (including display control information) for a
stereoscopic image for displaying a subtitle (caption).
[0162]
That is, the transport stream TS includes a video
data stream, an audio data stream, and first and second
20 private data streams (subtitle data streams) as PES
streams. As described above, the first and second
private data streams are respectively a 2D stream and a
3D extension stream (see Fig. 11).
[0163]
25 The set-top box 200 includes a bit stream
processing unit 201. When the set-top box 200 is a 3Dcompatible
device (3D STB), the bit stream processing
unit 201 acquires stereoscopic image data, audio data,
and subtitle data (including display control information)
30 from the transport stream TS. In this case, the bit
stream processing unit 201 acquires the respective
64
SP336805WO00
segment data constituting the subtitle data from the 2D
stream, and acquires the DSS segment data including the
display control information such as the disparity
information from the 3D extension stream.
5 [0164]
The bit stream processing unit 201 uses the
stereoscopic image data and the subtitle data (including
the display control information) to generate output
stereoscopic image data in which the subtitle is
10 superimposed on a left-eye image frame (frameO) portion
and a right-eye image frame (framel) portion (see Fig.
41) . In this case, a disparity can be provided between a
subtitle superimposed on a left-eye image (a left-eye
subtitle) and a subtitle superimposed on a right-eye
15 image (a right-eye subtitle).
[0165]
For example, as described above, the display
control information added to the subtitle data for a
stereoscopic image received from the broadcasting station
20 100 includes disparity information, and a disparity can •
be provided between a left-eye subtitle and a right-eye
subtitle based on the disparity information. In this
manner, since the disparity is provided between the lefteye
subtitle and the right-eye subtitle, the user can
25 recognize a subtitle (caption) in front of an image.
[0166]
Fig. 46(a) illustrates an example of the display of
a subtitle (caption) on an image. In the display example,
a caption is superimposed on an image including a
30 background and a near-view object. Fig. 46(b)
illustrates that the perspective of a background, a near-
65
SP336805WP00
view object and a caption is expressed and the caption is
recognized at the frontmost position.
[0167]
Fig. 47(a) illustrates an example of the display of
5 a subtitle (caption) on an image as in Fig. 46(a). Fig.
47(b) illustrates a left-eye caption LGI superimposed on
a left-eye image and a right-eye caption RGI superimposed
on a right-eye image. Fig. 47(c) illustrates that a
disparity is provided between the left-eye caption LGI
10 and the right-eye caption RGI so that the caption is
recognized at the frontmost position.
[0168]
Also, when the set-top box 200 is a legacy 2Dcompatible
device (2D STB), the bit stream processing
15 unit 201 acquires stereoscopic image data, audio data,
and subtitle data (bit map pattern data that does not
include display control information) from the transport
stream TS. The bit stream processing unit 201 uses the
stereoscopic image data and the subtitle data to generate
20 2D image data on which the subtitle is superimposed (see
Fig. 43) .
[0169]
In this case, the bit stream processing unit 201
acquires the respective segment data constituting the
25 subtitle data from the 2D stream. That is, in this case,
since the DSS segment is not read from the 3D extension
stream, the reception processing can be prevented from
being interrupted by the reading. In this case, based on
the subtitle type information and the language
30 information, the bit stream processing unit 201 extracts
only the 2D stream from the transport stream TS and
66
SP336805WO00
decodes the extracted 2D stream easily with a high
accuracy.
[0170]
[Example of Configuration of Set-top Box]
5 An example of the configuration of the set-top box
200 will be described. Fig. 48 illustrates an example of
the configuration of the set-top box 200. The set-top
box 200 includes a bit stream processing unit 201, an
HDMI terminal 202, an antenna terminal 203, a digital
10 tuner 204, a video signal processing circuit 205, an HDMI
transmitting unit 206, and an audio signal processing
circuit 207. Also, the set-top box 200 includes a CPU
211, a flash ROM 212, a DRAM 213, an internal bus 214, a
remote control receiving unit (RC receiving unit) 215,
15 and a remote control transmitter {RC transmitter) 216.
[0171]
The antenna terminal 2 03 is a terminal that is
configured to input a television broadcast signal
received through a reception antenna (not illustrated).
20 The digital tuner 204 processes the television broadcast
signal input to the antenna terminal 2 03, and outputs a
transport stream TS (bit stream data) corresponding to a
channel selected by a user.
[0172]
25 Based on the transport stream TS, the bit stream
processing unit 201 outputs audio data and output
stereoscopic image data on which a subtitle is
superimposed. When the set-top box 200 is a 3Dcompatible
device (3D STB), the bit stream processing
30 unit 201 acquires stereoscopic image data, audio data,
and subtitle data (including display control information)
67
SP336805WO00
from the transport stream TS.
[0173]
The bit stream processing unit 201 generates output
stereoscopic image data by superimposing the subtitle on
5 a left-eye image frame (frameO) portion and a right-eye
image frame (framel) portion constituting the
stereoscopic image data (see Fig. 41). Herein, based on
the disparity information, a disparity is provided
between a subtitle superimposed on the left-eye image
10 (left-eye subtitle) and a subtitle superimposed on the
right-eye image (right-eye subtitle).
[0174]
That is, the bit stream processing unit 201
generates region display data for displaying a subtitle,
15 based on the subtitle data. The bit stream processing
unit 201 obtains output stereoscopic image data by
superimposing the region display data on a left-eye image
frame (frameO) portion and a right-eye image frame
(framel) portion constituting the stereoscopic image data.
20 Herein, the bit stream processing unit 201 shifts the
positions of the respective superimposed display data
based on the disparity information.
[0175]
Also, when the set-top box 200 is a 2D-compatible
25 device (2D STB), the bit stream processing unit 201
acquires stereoscopic image data, audio data, and
subtitle data (not including display control information).
The bit stream processing unit 201 uses the stereoscopic
image data and the subtitle data to generate 2D image
30 data on which the subtitle is superimposed (see Fig. 43).
[0176]
68
SP336805WO00
That is, the bit stream processing unit 201
generates region display data for displaying a subtitle,
based on the subtitle data. The bit stream processing
unit 201 obtains output 2D image data by superimposing
5 the region display data on the 2D image data that has
been obtained by performing the processing according to
the transmission format on the stereoscopic image data.
[0177]
The video signal processing circuit 205 performs
10 image quality adjustment processing on the image data,
which has been obtained by the bit stream processing unit
201, as necessary, and supplies the processed image data
to the HDMI transmitting unit 206. The audio signal
processing circuit 207 performs sound quality adjustment
15 processing on the audio data, which has been output from
the bit stream processing unit 201, as necessary, and
supplies the processed audio data to the HDMI
transmitting unit 206.
[0178]
20 The HDMI transmitting unit 206 transmits, for
example, uncompressed image data and audio data to the
HDMI terminal 202 by HDMI-based communication. In this
case, being transmitted on an HDMI TMDS channel, the
image data and audio data are packed and output from the
25 HDMI transmitting unit 206 to the HDMI terminal 202.
[0179]
The CPU 211 controls an operation of each unit of
the set-top box 200, The flash ROM 212 stores control
software and data. The DRAM 213 constitutes a work area
30 of the CPU 211. The CPU 211 deploys the software or data
read from the flash ROM 212 on the DRAM 213 and activates
69
SP336805WO00
the software to control each unit of the set-top box 200.
[0180]
The RC receiving unit 215 receives a remote control
signal (remote control code) transmitted from the RC
5 transmitter 216, and supplies the received remote control
signal to the CPU 211. The CPU 211 controls each unit of
the set-top box 200 based on the remote control code.
The CPU 211, the flash ROM 212, and the DRAM 213 are
connected to the internal bus 214.
10 [0181]
An operation of the set-top box 200 will be
described briefly. The television broadcast signal input
to the antenna terminal 203 is supplied to the digital
tuner 204. The digital tuner 204 processes the
15 television broadcast signal and outputs a transport
stream TS (bit stream data) corresponding to a channel
selected by the user.
[0182]
The transport stream TS (bit stream data) output
20 from the digital tuner 204 is supplied to the bit stream
processing unit 201. The bit stream processing unit 201
generates output image data to be output to the
television receiver 300 as follows.
[0183]
25 When the set-top box 200 is a 3D-compatible device
(3D STB), stereoscopic image data, audio data, and
subtitle data (including.display control information) are
acquired from the transport stream TS. The bit stream
processing unit 201 generates output stereoscopic image
30 data by superimposing the subtitle on a left-eye image
frame (frameO) portion and a right-eye image frame
70
SP336805WO00
(framel) portion constituting the stereoscopic image data.
Herein, based on the disparity information, a disparity
is provided between a left-eye subtitle superimposed on a
left-eye image and a right-eye subtitle superimposed on a
5 right-eye image.
[0184]
Also, when the set-top box 200 is a 2D-compatible
device (2D STB), stereoscopic image data, audio data, and
subtitle data (not including display control information)
10 are acquired. The bit stream processing unit 201 uses
the stereoscopic image data and the subtitle data to
generate 2D image data on which the subtitle is
superimposed.
[0185]
15 The output image data obtained by the bit stream
processing unit 201 is supplied to the video signal
processing circuit 205. The video signal processing
circuit 205 performs image quality adjustment processing
on the output image data as necessary. The processed
20 image data output from the video signal processing
circuit 205 is supplied to the HDMI transmitting unit 206.
[0186]
Also, the audio data obtained by the bit stream
processing unit 201 is supplied to the audio signal
25 processing circuit 207. The audio signal processing
circuit 207 performs sound quality adjustment processing
on the audio data as necessary. The processed audio data
output from the audio signal processing circuit 207 is
supplied to the HDMI transmitting unit 206. The image
30' data and the audio data supplied to the HDMI transmitting
unit 2 06 are transmitted through an HDMI TMDS channel
71
SP336805WO00
from the HDMI terminal 202 to the HDMI cable 400.
[0187]
[Example of Configuration of Bit Stream Processing
Unit]
5 Fig. 49 illustrates an example of the configuration
of the bit stream processing unit 201 in the case where
the set-top box 200 is a 3D-compatible device (3D STB).
The t^it stream processing unit 201 has a configuration
corresponding to the transmission data generating unit
10 110 illustrated in Fig. 2 described above. The bit
stream processing unit 201 includes a demultiplexer 221,
a video decoder 222, and an audio decoder 229.
[0188]
Also, the bit stream processing unit 201 includes
15 an encoded data buffer 223, a subtitle decoder 224, a
pixel buffer 225, a disparity information interpolating
unit 226, a position control unit 227, and a video
superimposing unit 228. Herein, the encoded data buffer
223 constitutes a decoding buffer.
20 [0189]
The demultiplexer 221 extracts a video data stream
packet and an audio data stream packet from the transport
stream TS, and provides the extracted packets to the
respective decoders for decoding. In addition, the
25 demultiplexer 221 extracts the following streams and
temporarily stores the extracted streams in the encoded
data buffer 223. In this case, as described with
reference to Fig. 41 described above, the demultiplexer
221 extracts the 2D stream and the 3D extension stream of
30 the language selected by the user based on the subtitle
type information and the language information.
72
SP336805WO00
[0190]
The CPU 211 recognizes the necessity to decode both
of the PES streams based on the value of
"composition_page_id" in the subtitle descriptor disposed
5 in the ES loop inside the PMT in association with the 2D
stream and the 3D extension stream. That is, when the
value of "composition_page_id" is equal, it means that
both PES streams are decoded. Alternatively, when the
value of "composition_page_id" is equal and special value
10 (predefined), it means that both PES streams are decoded.
[0191]
Also, it can also be considered that a descriptor
associating two streams, which indicates the necessity to
decode both of the two PES streams (2D stream and 3D
15 extension stream), is newly defined, and the descriptor
is disposed at a predetermined position. By the
descriptor, the CPU 211 recognizes the necessity to
decode both of the PES streams, and controls the bit
stream processing unit 201.
20 [0192]
Fig. 50 illustrates an example of the syntax of a multidecoding
descriptor that can be used to associate the 2D
stream with the 3D extension stream. Fig. 51 illustrates
the main information contents (semantics) in the syntax
25 example.
[0193]
An 8-bit field of "descriptor_tag" indicates that
the descriptor is a multi-decoding descriptor. An 8-bit
field of "descriptor_length" represents the entire byte ^
30 size following the, field.
[0194]
73
SP336805WO00
A 4-bit field of "stream_content" represents the
stream type of a main stream such as a video, an audio,
and a subtitle. A 4-bit field of "component_type"
represents the component type of a main stream such as a
5 video, an audio, and a subtitle. The "stream_content"
and "component_type" are the same information as the
"stream_content" and "component_type" in the component
descriptor corresponding to the main stream.
[0195]
10 In this embodiment, the main stream is a 2D stream,
the "stream_content" is a subtitle "subtitle", and the
"component_type" is a two-dimensional "2D".
[0196]
The "component_tag" has the same value as the
15 "component_tag" in the stream_identifier descriptor
corresponding to the main stream. Accordingly, the
stream_identifier descriptor and the multi-decoding
descriptor are associated with the "component_tag".
[0197]
2 0 A 4-bit field of "multi_decoding_count" represents
the number of target streams associated with the main
stream. In this embodiment, the target stream associated
with the 2D stream being the main stream is the 3D
extension stream, and "multi_decoding_count" is "1".
25 [0198]
An 8-bit field of "target_stream_component_type"
represents the stream type of a stream added to the main
stream, such as a video, an audio, and a subtitle. A 4-
bit field of "component_type" represents the component
30 type of a target stream. Also, an 8-bit field of
"target_stream_component_tag" has the same value as the
74
SP336805WO00
"component_tag" in the stream_identifier descriptor
corresponding to the target stream.
[0199]
In this embodiment, the target stream is the 3D
5 extension stream, the "target_stream_component_type" is a
three-dimensional "3D", and the
"target_stream_component_tag" has the same value as the
"component_tag" of the 3D extension stream.
[0200]
10 The multi-decoding descriptor is disposed, for
example, under the PMT or in the EIT. Fig. 52
illustrates an example of a configuration of the
transport stream TS in the case where the multi-decoding
descriptor is disposed.
15 [0201]
Returning to Fig. 49, the video decoder 222
performs opposite processihg to the video encoder 112 of
the transmission data generating unit 110 described above.
That is, the video decoder 222 reconstructs a video data
20 stream from the video packet extracted by the
demultiplexer 221, performs encoding processing, and
obtains stereoscopic image data including left-eye image
data and right-eye image data. Examples of the
transmission format of the stereoscopic image data
25 include a Side By Side scheme, a Top & Bottom scheme, a
Frame Sequential scheme, and a video transmission format
scheme in which each view occupies a full-screen size.
[0202]
The subtitle decoder 224 performs opposite
30 processing to the subtitle encoder 125 of the
transmission data generating unit 110 described above.
75
SP336805WO00
That is, the subtitle decoder 224 reconstructs each
stream from each stream packet stored in the encoded data
buffer 223, and performs encoding processing to acquire
the following segment data.
5 [0203]
That is, the subtitle decoder 224 decodes the 2D
stream to acquire the respective segment data
constituting the subtitle data. Also, the subtitle
decoder 224 decodes the 3D extension stream to acquire
10 the DSS segment data. As described above, a page ID
(page_id) of each segment included in the 2D stream and a
page ID (page_id) of each segment included in the 3D
extension stream are equal. Therefore, based on the page
ID, the subtitle decoder 224 can easily combine the
15 segment of the 2D stream and the segment of the 3D
extension stream.
[0204]
Based on the respective segment data and the
subregion region information constituting the subtitle
20 data, the subtitle decoder 224 generates region display
data (bit map data) for displaying the subtitle. Herein,
a transparent color is allocated to a region that is
located in the region and is not surrounded by subregions.
The pixel buffer 225 temporarily stores the display data.
25 [0205]
The video superimposing unit 228 obtains output
stereoscopic image data Vout. In this case, the video
superimposing unit 228 superimposes the display data
stored in the pixel buffer 225, on a left-eye image frame
30 (frame0) portion and a right-eye image frame (frame1)
portion of the stereoscopic image data obtained by the
76
SP336805WO00
video decoder 222. In this case, the video superimposing
unit 228 changes the superimposition position, the size,
and the like appropriately according to a transmission
scheme of the stereoscopic image data (such as a Side By
5 Side scheme, a Top & Bottom scheme, a Frame Sequential
scheme, or an MVC scheme). The video superimposing unit
228 outputs the output stereoscopic image data Vout to
the outside of the bit stream processing unit 201.
[0206]
10 The disparity information interpolating unit 22 6
provides the disparity information obtained by the
subtitle decoder 224 to the position control unit 227.
As necessary, the disparity information interpolating
unit 226 performs interpolation processing on the
15 disparity information to be provided to the position
control unit 227. The position control unit 227 shifts
the position of the display data superimposed on each
frame, based on the disparity information (see Fig.. 41) .
In this case, based on the disparity information, the
20 position control unit 227 provides a disparity by
shifting the display data (caption pattern data)
superimposed on the left-eye image frame (frameO) portion
and the right-eye image frame (framel) portion to be in
opposite directions.
25 [0207]
Also, the display control information includes
disparity information that is commonly used in the
caption display period. Also, the display control
information may include disparity information that is
30 sequentially updated in the caption display period. As
described above, the disparity information sequentially
77
SP336805WO00
updated in the caption display period includes disparity
information of the initial frame of the caption display
period and disparity information of a frame for each of
the subsequent update frame intervals.
5 [0208]
As for the disparity information commonly used in
the caption display period, the position control unit 227
uses the disparity information without change. On the
other hand, as for the disparity information sequentially
10 updated in the caption display period, the position
control unit 227 uses the disparity information
interpolated by the disparity information interpolating
unit 226 as necessary. For example, the disparity
information interpolating unit 22 6 generates disparity
15 information of an arbitrary frame interval in the caption
display period, for example, disparity information of a
1-frame interval.
[0209]
As the interpolation processing, the disparity
20 information interpolating unit 22 6 performs not linear
interpolation processing but interpolation processing
accompanied with low-pass filter (LPF) processing in the
time direction (frame direction), for example.
Accordingly, a change in the disparity information of a
25 predetermined frame interval in the time direction (frame
direction) after the interpolation processing becomes
smooth.
[0210]
Also, the audio decoder 229 performs opposite
30 processing to the audio encoder 113 of the transmis.sion
data generating unit 110 described above. That is, the
78
SP336805WO00
audio decoder 22 9 reconstructs an audio elementary stream
from the audio packet extracted by the demultiplexer 221,
performs encoding processing, and obtains output audio
data Aout. The audio decoder 229 outputs the output
5 audio data Aout to the outside of the bit stream
processing unit 201.
[0211]
An operation of the bit stream processing unit 201
illustrated in Fig. 49 will be described briefly. The
10 transport stream TS output from the digital tuner 204
(see Fig. 48) is supplied to the demultiplexer 221. The
demultiplexer 221 extracts a video data stream packet and
an audio data stream packet from the transport stream TS,
and supplies the extracted packets to the respective
15 decoders. Also, the 2D stream packet and the 3D
extension stream packet of the language selected by the
user are extracted by the demultiplexer 221, and the
extracted packet is temporarily stored in the encoded
data buffer 223.
20 [0212]
The video decoder 222 reconstructs a video data
stream from the video data packet extracted by the
demultiplexer 221, performs decoding processing, and
obtains stereoscopic image data including left-eye image
25 data and right-eye image data. The stereoscopic image
data is supplied to the video superimposing unit 228.
[0213]
The subtitle decoder 224 reads the 2D stream packet
and the 3D extension stream packet from the encoded data
30 buffer 223, and decodes the read packet. Based on the
respective segment data and the subregion region
79
SP336805WO00
information constituting the subtitle data, the subtitle
decoder 224 generates region display data (bit map data)
for displaying the subtitle. The display data is
temporarily stored in the pixel buffer 225.
5 [0214]
The video superimposing unit 228 superimposes the
display data stored in the pixel buffer 225, on the lefteye
image frame (frameO) portion and the right-eye image
frame (framel) portion of the stereoscopic image data
10 obtained by the video decoder 222. In this case, the
superimposition position, the size, and the like are
changed appropriately according to a transmission scheme
of the stereoscopic image data (such as a Side By Side
scheme, a Top & Bottom scheme, a Frame Sequential scheme,
15 or an MVC scheme) . The output stereoscopic image data
Vout obtained by the video superimposing unit 22 8 is
output to the outside of the bit stream processing unit
201.
[0215]
20 Also, the disparity information obtained by the
subtitle decoder 224 is provided through the disparity
information interpolating unit 22 6 to the position
control unit 227. The disparity information
interpolating unit 22 6 performs interpolation processing
25 as necessary. For example, as for the disparity
information at several-frame intervals sequentially
updated in the caption display period, interpolation
processing is performed by the disparity information
interpolating unit 22 6 as necessary, to generate
30 disparity information of an arbitrary frame interval, for
example, a 1-frame interval.
80
SP336805WO00
[0216]
Based on the disparity information, the position
control unit 227 shifts the display data (caption pattern
data) superimposed on the left-eye image frame (frameO)
5 portion and the right-eye image frame (framel) portion by
the video superimposing unit 228, such that they are in
opposite directions. Accordingly, a disparity is
provided between a left-eye subtitle displayed on the
left-eye image and a right-eye subtitle displayed on the
10 right-eye image. Accordingly, the 3D display of a
subtitle (caption) is implemented according to the
contents of a stereoscopic image.
[0217]
Also, the audio decoder 229 reconstructs an audio
15 elementary stream from the audio packet extracted by the
demultiplexer 221, performs decoding processing, and
obtains audio data Aout corresponding to the above
stereoscopic image data Vout for display. The audio data
Aout is output to the outside,of the bit stream
20 processing unit 201.
[0218]
Fig. 53 illustrates an example of the configuration
of the bit stream processing unit 201 in the case where
the set-top box 200 is a 2D-compatible device (2D STB).
25 In Fig. 53, the units corresponding to those of Fig. 49
are denoted by like reference numerals, and a detailed
description thereof will be omitted. Hereinafter, for
the convenience of description, the bit stream processing
unit 201 illustrated in Fig. 49 will be referred to as
30 the 3D-compatible bit stream processing unit 201, and the
bit stream processing unit 2 01 illustrated in Fig. 53
81
SP336805WO00
will be referred to as the 2D-compatible bit stream
processing unit 201.
[0219]
In the 3D-coinpatible bit stream processing unit 201
5 illustrated in Fig. 49, the video decoder 222
reconstructs a video data stream from the video packet
extracted by the demultiplexer 221, performs decoding
processing, and obtains stereoscopic image data including
left-eye image data and right-eye image data. On the
10 other hand, in the 2D-compatible bit stream processing
unit 201 illustrated in Fig. 53, the video decoder 222
acquires stereoscopic image data, cuts out left-eye image
data or right-eye image data, and performs scaling
processing as necessary, to obtain 2D image data.
15 [0220]
Also, in the 3D-compatible bit stream processing
unit 201 illustrated in Fig. 49, the demultiplexer 221
extracts the 2D stream packet and the 3D extension stream
packet of the language selected by the user as described
20 above, and provides the extracted stream packets to the
subtitle decoder 224. On the other hand, in the 2Dcompatible
bit stream processing unit 201 illustrated in
Fig. 53, the demultiplexer 221 extracts only the 2D
stream packet of the language selected by the user as
25 described with reference to Fig. 43, and provides the
extracted stream packet to the subtitle decoder 224.
[0221]
In this case, based on the subtitle type
information and the language information, the
30 demultiplexer 221 extracts only the 2D stream -from the
transport stream TS and decodes the extracted 2D stream
82
SP336805WO00
easily with a high accuracy. That is, the component
descriptor and the subtitle descriptor inserted in
association with the 2D stream and the 3D extension
stream are inserted into the transport stream TS (see Fig.
5 15) .
[0222]
In these descriptors, the subtitle type information
"subtitling_type" and the language information
"ISO_639_language_code" are set to identify the 2D stream
10 and the 3D extension stream (see Figs. 15 and 19).
Therefore, based on the corresponding subtitle type
information and the language information, the
demultiplexer 221 can extract only the 2D stream from the
transport stream TS and decode the extracted 2D stream
15 easily with a high accuracy.
[0223]
Also, in the 3D-compatible bit stream processing
unit 201 illustrated in Fig. 49, the subtitle decoder 224
acquires the respective segment data constituting the
20 subtitle data, for example, from the 2D stream as
described above, and acquires the DSS segment data from
the 3D extension stream.
[0224]
On the other hand, in the 2D-compatible bit stream
25 processing unit 201 illustrated in Fig. 53, the subtitle
decoder 224 acquires only the respective segment data
constituting the subtitle data from the 2D stream. Based
on the respective segment data and the subregion region
information, the subtitle decoder 224 generates region
30 display data (bit map data) for displaying the subtitle,
and temporarily stores the generated data in the pixel
83
SP336805WO00
buffer 225. In this case, the subtitle decoder 224 does
not read the DSS segment data. Therefore, the reception
processing can be prevented from being interrupted by the
reading.
5 [0225]
Also, in the 3D-compatible bit stream processing
unit 201 illustrated in Fig. 49, the video superimposing
unit 228 obtains output stereoscopic image data Vout and
outputs the output stereoscopic image data Vout to the
10 outside of the bit stream processing unit 201. In this
case, the video superimposing unit 22 8 obtains the output
stereoscopic image data Vout by superimposing the display
data stored in the pixel buffer 225, on the left-eye
image frame (frameO) portion and the right-eye image
15 frame (framel) portion of the stereoscopic image data
obtained by the video decoder 222. Based on the
disparity information, the position control unit 227
shifts the display data to be in opposite directions, and
provides a disparity between the left-eye subtitle
20 displayed on the left-eye image and the right-eye
subtitle displayed on the right-eye image.
[0226]
On the other hand, in the 2D-compatible bit stream
processing unit 201 illustrated in Fig. 53, the video
25 superimposing unit 228 obtains output 2D image data Vout
by superimposing the display data stored in the pixel
buffer 225 on the 2D image data obtained by the video
decoder 222. The video superimposing unit 228 outputs
the output 2D image data Vout to the outside of the bit
30 stream processing unit 201.
[0227]
84
SP336805WO00
An operation of the 2D bit stream processing unit
201 illustrated in Fig. 53 will be described briefly.
Also, since an operation of the audio system is the same
as that of the 3D bit stream processing unit 201
5 illustrated in Fig. 49, a description thereof will be
omitted.
[0228]
The transport stream TS output from the digital .
tuner 204 (see Fig. 48) is supplied to the demultiplexer
10 221. The demultiplexer 221 extracts a video data stream
packet and an audio data stream packet from the transport
stream TS, and supplies the extracted packets to the
respective decoders. In addition, the demultiplexer 221
extracts the 2D stream packet and temporarily stores the
15 extracted 2D stream packet in the encoded data buffer 223.
[0229]
The video decoder 222 reconstructs a video data
stream from the video data packet extracted by the
demultiplexer 221, performs decoding processing, and
20 obtains stereoscopic image data including left-eye image
data and right-eye image data. The video decoder 222
cuts out the left-eye image data or the right-eye image
data from the stereoscopic image data, and performs
scaling processing as necessary, to obtain 2D image data.
25 The 2D image data is supplied to the video superimposing
unit 228.
[0230]
Also, the subtitle decoder 224 reads the 2D stream
from the encoded data buffer 223 and decodes the same.
30 Based on the respective segment data constituting the
subtitle data, the subtitle decoder 224 generates region
85
SP336805WO00
display data (bit map data) for displaying the subtitle.
The display data is temporarily stored in the pixel
buffer 225.
[0231]
5 The video superimposing unit 228 obtains output 2D image
data Vout by superimposing the display data (bit map
data) of the subtitle stored in the pixel buffer 225 on
the 2D image data obtained by the video decoder 222. The
output 2D image data Vout is output to the outside of the
10 bit stream processing unit 201.
[0232]
In the set-top box 200 illustrated in Fig. 48, the
transport stream TS output from the digital tuner 204
includes display control information in addition to
15 stereoscopic image data and subtitle data. The display
control information includes display control information
such as disparity information and region information of a
subregion. Therefore, a disparity can be provided to the
display positions of the left-eye subtitle and the right-
20 eye subtitle. Accordingly, in the display of a subtitle
(caption), the consistency of a perspective between
respective objects in an image can be maintained in an
optimal state.
[0233]
25 Also, in the set-top box 200 illustrated in Fig. 48,
when the display control information acquired by the
subtitle decoder 224 of the 3D-compatible bit stream
processing unit 201 (see Fig. 46) includes the disparity
information sequentially updated in the caption display
30 period, the display positions of the left-eye subtitle
and the right-eye subtitle can be dynamically controlled.
86
SP336805WO00
Accordingly, the disparity provided between the left-eye
subtitle and the right-eye subtitle can be dynamically
changed in conjunction with a change in the image content.
[0234]
5 Also, in the set-top box 200 illustrated in Fig. 48,
the disparity information interpolating unit 22 6 of the
3D bit stream processing unit 201 (see Fig. 49) performs
interpolation processing on disparity information of a
plurality of frames constituting the disparity
10 information that is sequentially updated in the caption
display period (the period of a predetermined number of
fames) . In this case, even when disparity information is
transmitted from the transmitting side at intervals of an
update frame, the disparity provided between the left-eye
15 subtitle and the right-eye subtitle can be controlled at
fine intervals, for example, every frame.
[0235]
Also, in the set-top box 200 illustrated in Fig. 48,
the interpolation processing in the disparity information
20 interpolating unit 22 5 of the 3D bit stream processing
unit 201 (see Fig. 49) may be accompanied with, for
example, low-pass filter processing in the time direction
(frame direction). Therefore, even when disparity
information is transmitted from the transmitting side at
25 intervals of an update frame, a change in the disparity
information in the time direction after the interpolation
processing can be made smooth. Accordingly, it is
possible to suppress a sense of discomfort that may be
caused when a shift of the disparity provided between the
30 left-eye subtitle and the right-eye subtitle becomes
discontinuous every frame interval.
'87
SP336805WO00
[0236]
Also, in the set-top box 200 illustrated in Fig. 48,
based on the subtitle type information and the language
information, the demultiplexer 221 of the bit stream
5 processing unit 201 (see Fig. 53) extracts only the 2D
stream from the transport stream TS and decodes the
extracted 2D stream easily with a high accuracy.
Accordingly, since the subtitle decoder 224 can more
securely prevent decoding processing from being performed
10 on the 3D extension stream including the DSS segment
having the disparity information, the reception
processing thereof can be prevented from being
interrupted by the decoding processing.
[0237]
15 In addition, although not described above, when the
set-top box 200 is a 3D-compatible device, the user may
select a 2D display mode or a 3D display mode. In this
case, when the 3D display mode is selected, the bit
stream processing unit 201 may have the same
20 configuration and operation as the 3D-compatible bit
stream processing unit 201 described above (see Fig. 49).
On the other hand, when the 2D display mode is selected,
the bit stream processing unit 201 may have substantially
the same configuration and operation as the 2D-compatible
25 bit stream processing unit 201 described above (see Fig.
53) .
[0238]
[Description of Television Receiver]
Returning to Fig. 1, when being a 3D-compatible
30 device, the television receiver 300 receives stereoscopic
image data that is transmitted from the set-top box 2 00
88
SP336805WO00
through the HDMI cable 400. The television receiver 300
includes a 3D signal processing unit 301. The 3D signal
processing unit 301 performs processing corresponding to
the transmission format (decoding processing) on the
5 stereoscopic image data to generate left-eye image data
and right-eye image data.
[0239]
[Example of Configuration of Television Receiver]
An example of the configuration of a 3D-compatible
10 television receiver 300 will be described. Fig. 54
illustrates an example of the configuration of the
television receiver 300. The television receiver 300
includes a 3D signal processing unit 301, an HDMI
terminal 302, an HDMI receiving unit 303, an antenna
15 terminal 304, a digital tuner 305, and a bit stream
processing unit 306.
[0240]
Also, the television receiver 300 includes a
video/graphic processing circuit 307, a panel driving
20 circuit 308, a display panel 309, an audio signal
processing circuit 310, an audio amplifying circuit 311,
and a speaker 312. Also, the television receiver 300
•includes a CPU 321, a flash ROM 322, a DRAM 323, an
internal bus 324, a remote control receiving unit (RC
25 receiving unit)' 325, and a remote control transmitter (RC
transmitter) 326.
[0241]
The antenna terminal 304 is a terminal that is
configured to input a television broadcast signal
30 received through a reception antenna (not illustrated).
The digital tuner 305 processes the television broadcast
89
SP336805WO00
signal input to the antenna terminal 304, and outputs a
transport stream TS (bit stream data) corresponding to a
channel selected by a user.
[0242]
5 Based on the transport stream TS, the bit stream
processing unit 306 outputs audio data and output
stereoscopic image data on which a subtitle is
superimposed. Although not described in detail, for
example, the bit stream processing unit 201 has the same
10 configuration as the 3D-compatible bit stream processing
unit 201 (see Fig. 49) of the set-top box 200 described
above. With respect to stereoscopic image data, the bit
stream processing unit 306 synthesizes display data of a
left-eye subtitle and a right-eye subtitle, and generates
15 and outputs output stereoscopic image data superimposed
with a subtitle.
[0243]
Also, for example, when a transmission format of
the stereoscopic image data is a Side By Side scheme, a
20 Top & Bottom scheme, or the like, the bit stream
processing unit 306 performs scaling processing to output
full-resolution left-eye image data and right-eye image
data. Also, the bit stream processing unit 306 outputs
audio data corresponding to the image data.
25 [0244]
The HDMI receiving unit 303 receives uncompressed
image data and audio data supplied through the HDMI cable
400 to the HDMI terminal 302, by HDMI-based communication.
The HDMI receiving unit 303 has a version of, for example,
30 HDMI 1.4a, and thus can process stereoscopic image data.
[0245]
90
SP336805WO00
The 3D signal processing unit 301 performs decoding
processing on the stereoscopic image data received by the
HDMI receiving unit 303, to generate full-resolution
left-eye image data and right-eye image data. The 3D
5 signal processing unit 301 performs the decoding
processing corresponding to a TMDS transmission data
format. Also, the 3D signal processing unit 301 does not
perform any processing on the full-resolution left-eye
image data and right-eye image data obtained by the bit
10 stream processing unit 306,
[0246]
The video/graphic processing circuit 307 generates
image data for displaying a stereoscopic image, based on
the left-eye image data and right-eye image data
15 generated by the 3D signal processing unit 301. Also,
the video/graphic processing circuit 307 performs image
quality adjustment processing on the image data as
necessary.
[0247]
20 Also, with respect to the image data, the
video/graphic processing circuit 307 synthesizes
superimposition information data such as a menu or a
program as necessary. The panel driving circuit 308
drives the display panel 309 based on the image data
25 output from the video/graphic processing circuit 307.
The display panel 309 includes, for example, an LCD
(Liquid Crystal Display), a PDP (Plasma Display Panel),
or the like.
[0248]
30 The audio signal processing circuit 310 performs
necessary processing such as D/A conversion on the audio
91
SP336805WO00
data that is received by the HDMI receiving unit 303 or
is obtained by the bit stream processing unit 306. The
audio amplifying circuit 311 amplifies an audio signal
output from the audio signal processing circuit 310, and
5 supplies the amplified audio signal to the speaker 312.
[0249]
The CPU 321 controls an operation of each unit of
the television receiver 300. The flash ROM 322 stores
control software and data. The DRAM 323 constitutes a
10 work area of the CPU 321. The CPU 321 deploys the
software or data read from the flash ROM 322 on the DRAM
323 and activates the software to control each unit of
the television receiver 300.
[0250]
15 The RC receiving unit 325 receives a remote control
signal (remote control code) transmitted from the RC
transmitter 326, and supplies the received remote control
signal to the CPU 321. The CPU 321 controls each unit of
the television receiver 300 based on the remote control
20 code. The CPU 321, the flash ROM 322, and the DRAM 323
are connected to the internal bus 324.
[0251]
An operation of the television receiver 300
illustrated in Fig. 54 will be described briefly. The
25 HDMI receiving unit 303 receives stereoscopic image data
and audio data transmitted from the set-top box 200
connected through the HDMI cable 400 to the HDMI terminal
302. The stereoscopic image data received by the HDMI
receiving unit 303 is supplied to the 3D signal
30 processing unit 301. Also, the audio data received by
the HDMI receiving unit 303 is supplied to the audio
92
SP336805WO00
signal processing circuit 310.
[0252]
The television broadcast signal input to the
antenna terminal 304 is supplied to the digital tuner 305,
5 The digital tuner 305 processes the television broadcast
signal and outputs a transport stream TS (bit stream
data) corresponding to a channel selected by the user.
The transport stream TS is supplied to the bit stream
processing unit 306.
10 [0253]
Based on the video data stream, the audio data
stream, the 2D stream, the 3D extension stream, and the
3D stream, the bit stream processing unit 306 obtains
audio data and output stereoscopic image data
15 superimposed with the subtitle. In this case, with
respect to the stereoscopic image data, display data of
the left-eye subtitle and the right-eye subtitle are
synthesized to generate output stereoscopic image data
superimposed with the subtitle (full-resolution left-eye
2 0 image data and right-eye image data). The output
stereoscopic image data is supplied through the 3D signal
processing unit 301 to the video/graphic processing
circuit 307.
[0254]
25 The 3D signal processing unit 301 performs decoding
processing on the stereoscopic image data received by the
HDMI receiving unit 303, to generate full-resolution
left-eye image data and right-eye image data. The lefteye
image data and the right-eye image data are supplied
30 to the video/graphic processing circuit 307. The
video/graphic processing circuit 307 generates image data
93
SP336805WO00
for displaying a stereoscopic image based on the left-eye
image data and the right-eye image data, and also
performs superimposition information data synthesizing
processing such as image quality adjustment processing
5 and OSD (On Screen Display) processing as necessary.
[0255]
The image data obtained by the video/graphic
processing circuit 307 is supplied to the panel driving
circuit 308. Therefore, a stereoscopic image is
10 displayed by the display panel 309. For example, the
display panel 309 alternately displays a left-eye image
corresponding to the left-eye image data and a right-eye
image corresponding to the right-eye image data in a
time-division manner. By wearing, for example, shutter
15 glasses having a left-eye shutter and a right-eye shutter
that are opened alternately in synchronization with the
display of the display pa;nel 309, a viewer can view only
a left-eye image with a left eye and can view only a
right-eye image with a right eye, thus recognizing a
20 stereoscopic image.
[0256]
Also, the audio data obtained by the bit stream
processing unit 305 is supplied to the audio signal
processing circuit 310. The audio signal processing
25 circuit 310 performs necessary processing such as D/A
conversion on the audio data that is received by the HDMI
receiving unit 303 or is obtained by the bit stream
processing unit 306. The audio data is amplified by the
audio amplifying circuit 311, and the amplified audio
30 data is supplied to the speaker 312. Therefore, a sound
corresponding to the display image of the display panel
94
' SP336805WO00
309 is output from the speaker 312.
[0257]
In addition, Fig. 54 illustrates the 3D-coinpatible
television receiver 300 as described above. Although not
5 described in detail, the legacy 2D-compatible television
receiver has substantially the same configuration.
However, in the case of the legacy 2D-compatible
television receiver, the bit stream processing unit 306
has the same configuration and operation as the 2D-
10 compatible bit stream processing unit 201 illustrated in
Fig. 53 described above. Also, in the case of the legacy
2D-compatible television receiver, the 3D signal
processing unit 301 is unnecessary.
[0258]
15 Also, in the case of the 3D-compatible television
receiver 300, the user may select a 2D display mode or a
3D display mode. In this case, when the 3D display mode
is selected, the bit stream processing unit 306 has the
same configuration and operation as described above. On
20 the other hand, when the 2D display mode is selected, the
bit stream processing unit 306 has the same configuration
and operation as the 2D-compatible bit stream processing
unit 201 illustrated in Fig. 53 described above.
[0259]
25 <2. Modification>
Also, in the above-described embodiment, it is
illustrated that the disparity information included in
the 3D extension stream is operated as a composition page
(composition_page) (see Figs. 15, 19 and 23). In this
30 case, as described above, the 3D extension stream is
included as "composition_page_id" in the respective
95
SP336805WO00
language services, and is designated as
"subtitling_type=3D" and "ISO_639_language_code=zxx".
[0260]
On the other hand, it can be considered that the
5 disparity information included in the 3D extension stream
is operated as an ancillary page (ancillary_page). In
this case, when the 3D extension stream is designated as
a common ancillary page ID (ancillary_page_id) of the
respective language services, the 3D extension stream is
10 commonly referred to from the respective language
services, so that the subtitle service can be efficiently
encoded. Even in this case, the 3D extension stream is
also designated as "subtitling_type=3D" and
"ISO_639_language_code=zxx". However, it applies to only
15 the case where the configuration of the 3D extension
stream is DDS, DSS, and EDS. This is because the PCS is
not allowed to be shared between services and thus
"page_id" is required to be always equal to
"composition_page_id".
2-0 [02 61]
Fig. 55 illustrates an example of a configuration
of the transport stream TS in that case. As in the
configuration example of the transport stream TS
illustrated in Fig. 15 described above, also in Fig. 55,
25 the illustration of video and audio-related portions is
omitted'for simplicity of illustration. In Fig. 55, the
units corresponding to those of Fig. 15 are denoted by
like reference numerals, and a detailed description
thereof will be appropriately omitted.
30 [02 62]
A subtitle descriptor (Subtitling_Descriptor)
96
SP336805WO00
corresponding respectively to the 2D stream and the 3D
extension stream is inserted into the PMT. In the 2D
stream, an ancillary page ID (ancillary_page_id) is not
operated. Therefore, the ancillary page ID
5 "ancillary_page_id" of the subtitle descriptor
corresponding to .the 2D stream is set to have the same
value as the composition page ID "composition_page_id"
(OxXXXX in the drawing). That is, this is because the
values of both of the page IDs are required to be equal
10 when the ancillary page ID is not operated. The
"page_id" of each segment of the 2D stream is encoded to
have the same value as the "composition_page_id".
[0263]
On the other hand, in the 3D extension stream, an
15 ancillary page ID (ancillary_page_id) is operated.
Therefore, the ancillary page ID "ancillary_page_id" of
the subtitle descriptor corresponding to the 3D extension
stream is set to have a different value from the
composition page ID "composition_page_id" (OxPPPP in the
2 0 drawing) . The "page_id" of each segment of the 3D
extension stream is encoded to have the same value as the
"ancillary_page_id". Accordingly, the 3D extension
stream is the value of the page ID "page_id" equal to the
ancillary page ID "ancillary_page_id", and is commonly
25 referred to in the respective language services.
[0264]
Fig. 56 illustrates the extraction of the subtitle
descriptor (Subtitling_descriptor) and the component
descriptor (Component_descriptor) illustrated in Fig. 55.
30 Also, Fig. 57 illustrates the extraction of the PES
streams (2D stream and 3D extension stream) illustrated
97
SP336805WO00
in Fig. 55.
[0265]
A composition page ID (composition_page_id) of the
subtitle descriptor is set to indicate that each segment
5 included in the 3D extension stream is associated with
each segment of the 2D stream. That is, the
"composition_page_id" in the 3D extension stream and the
"composition_page_id" in the 2D stream are set to share
the same value (OxXXXX in the figure).
10 [0256]
Also, in the subtitle descriptor and the component
descriptor, for example, an ISO language code
(ISO_639_language_code) is described as language
information. The ISO language code of the descriptor
15 corresponding to the 2D stream is set to represent the
language of a subtitle (caption). In the illustrated
example, the ISO language code is set to "eng"
representing English. The 3D extension stream has a
segment of the DBS with disparity information, but does
2 0 not have a segment of the ODS. Therefore, the 3D
extension stream does not depend on languages. The ISO
language code described in the descriptor corresponding
to the 3D extension stream is set to, for example, "zxx"
representing a non-language.
25 [0267]
Also, the ancillary page ID "ancillary_page_id" of
the subtitle descriptor corresponding to the 2D stream is
set to have the same value as the composition page ID
"composition_page_id" (OxXXXX in the drawing). On the
30 other hand, the ancillary page ID "ancillary_page_id" of
the subtitle descriptor corresponding to the 3D extension
98
SP336805WO00
stream is set to have a different value from the
composition page ID "composition_page_id" (OxPPPP in the
drawing).
[0268]
5 Fig. 58 illustrates an example of a stream
configuration of subtitle data streams (2D stream and 3D
extension stream). Like the configuration example of Fig.
23, this example is an example of service of two
languages that are English "eng" and German "ger".
10 Unlike the configuration example of Fig. 23, the 3D
extension stream is not included as "composition_page_id"
in the respective language services. In this
configuration example, the 3D extension stream is a
common ancillary page ID (ancillary_page_id) of the
15 respective language services, and is commonly referred to
from the respective language services.
[0269]
Fig. 59 schematically illustrates the 2D stream and
3D extension stream extraction processing in the 3D-
20 compatible device (set-top box, television receiver, or
the like) in the case where the language service selected
by the user is English "eng".
[0270]
In this case, based on the subtitle type
25 information, the 3D-compatible device determines the 2D
stream corresponding to a subtitle type "2D(HD,SD)" and
the 3D extension stream corresponding to a subtitle type
"3D" as a stream to be extracted (see a "O" mark) . Also,
the receiving apparatus determines the 2D stream of the
30 language selected by the user and the 3D extension stream
with the language information (ISO language code)
99
SP336805WO00
representing a non-language as a stream to be extracted
(see a "O" mark) .
[0271]
In this case, as a result, based on the subtitle
5 type information and the language information, the 2D
stream corresponding to English "eng" and the 3D
extension stream commonly referred to as
"ancillary_page_id" from the respective language services
are determined as a stream to be extracted.
10 [0272]
Fig. 60 schematically illustrates the extraction
processing of only the 2D stream in the legacy 2Dcompatible
device (set-top box, television receiver, or
the like) in the case where the language service selected
15 by the user is English "eng".
[0273]
In this case, based on the subtitle type
information, the 2D-compatible device can determine the
2D stream corresponding to a subtitle type "2D(HD,SD)" as
20 a stream to be extracted. Herein, since the 2Dcompatible
device cannot interpret a subtitle type "3D",
the 3D extension stream may also be determined as a
stream to be extracted (see a "A" mark).
[0274]
25 However, in this case, based on the language
information, the 2D-compatible device determines the 2D
stream corresponding to English "eng" as a stream to be
extracted, and does not determine the 3D extension stream
with the language information representing a non-language
30 as a stream to be extracted (illustrated as a mark "x").
As a result, the 2D-compatible device determines only the
100
SP3,36805WO00
2D stream corresponding to English "eng" as a stream to
be extracted. Accordingly, since the 2D-compatible
device can more securely prevent decoding processing from
being performed on the 3D extension stream including the
5 DSS segment having the disparity information, the
reception processing thereof can be prevented from being
interrupted by the decoding processing.
[0275]
In addition. Fig. 48 illustrates that the set-top
10 box 200 is provided with the antenna input terminal 203
connected to the digital tuner 204. However, a set-top
box receiving an RF signal transmitted through a cable
may also be configured in the same manner. In this case,
a cable terminal is provided instead of the antenna
15 terminal 203.
[0276]
Also, a set-top box, to which the internet and a
home network are connected directly or through a router,
may also be configured in the same manner. In this case,
20 the above-described transport stream TS is transmitted
from the internet and the home network to the set-top box
directly or through the router.
[0277]
Fig. 61 illustrates an example of the configuration
25 of a set-top box 200A in that case. In Fig. 61, the
units corresponding to those of Fig. 48 are denoted by
like reference numerals. The set-top box 200A includes a
network terminal 208 connected to a network interface 209.
A transport stream TS is output from the network
30 interface 209 and then supplied to the bit stream
processing unit 201. Although not described in detail,
101
SP336805WO00
the other units of the set-top box 200A have the same
configurations and operations as the corresponding units
of the set-top box 200 illustrated in Fig. 48.
[0278]
5 Also, Fig. 54 illustrates that the television
receiver 300 is provided with the antenna input terminal
304 connected to the digital tuner 204. However, a
television receiver receiving an RF signal transmitted
through a cable may also be configured in the same manner.
10 In this case, a cable terminal is provided instead of the
antenna terminal 304.
[0279]
Also, a television receiver, to which the internet
and a home network are connected directly or through a
15 router, may also be configured in the same manner. In
this case, the above-described transport stream TS is
transmitted from the Internet and the home network to the
television receiver directly or through the router.
[0280]
20 Fig. 62 illustrates an example of the configuration
of a television receiver 300A in that case. In Fig. 62,
the units corresponding to those of Fig. 54 are denoted
by like reference numerals. The television receiver 300A
includes a network terminal 313 connected to a network
25 interface 314. A transport stream TS is output from the
network interface 314 and then supplied to the bit stream
processing unit 306. Although not described in detail,
the other units of the television receiver 300A have the
same configurations and operations as the corresponding
30 units of the television receiver 300 illustrated in Fig.
54.
102
SP336805WO00
[0281]
Also, in the above-described embodiment, the image
transmitting/receiving system 10 is illustrated as
including the broadcasting station 100, the set-top box
5 200, and the television receiver 300. However, as
illustrated in Fig. 54, the television receiver 300
includes the bit stream processing unit 306 that
functions in the same way as the bit stream processing
unit 201 in the set-top box 200. Therefore, as
10 illustrated in Fig. 63, an image transmitting/receiving
system lOA may be designed to include the broadcasting
station 100 and the television receiver 300.
[0282]
Also, in the above-described embodiment, the set-
15 top box 200 and the television receiver 300 are
illustrated as being connected through an HDMI digital
interface. However, the present invention can be
similarly applied even when the set-top box 200 and the
television receiver 300 are connected through any other
20 digital interface (including a wireless interface as well
as a wired interface) that is equivalent to the HDMI
digital interface.
[0283]
Also, in the above-described embodiment, the
25 subtitle (caption) is treated as the superimposition
information. However, the present invention can be
similarly applied even when other types of information
such as graphics information and text information are
treated as the superimposition information, and even when
30 those divided into an elementary stream and an additional
stream and encoded so as to be output in an associated
103
SP336805WO00
manner are treated in relation to an audio stream.
[0284]
Also, the present technology may have the following
configurations.
5 (1) A transmitting apparatus including:
an image data output unit configured to output
left-eye image data and right-eye image data for
displaying a stereoscopic image;
a superimposition information data output unit
10 configured to output superimposition information data to
be superimposed on an image by the left-eye image data
and the right-eye image data;
a disparity information output unit configured to
output disparity information for providing a disparity by
15 shifting the superimposition information to be
superimposed on the image by the left-eye image data and
the right-eye image data; and
a data transmitting unit configured to transmit a
multiplexed data stream including a video data stream
2 0 including the image data output from the image data
output unit, a first private data stream including the
superimposition information data output from the
superimposition information data output unit, and a
second private data stream including the disparity
25 information output from the disparity information output
unit,
wherein a first descriptor and a second descriptor
including respective pieces of language information
corresponding to the first private data stream and the
30 second private data stream are inserted into the
multiplexed data stream, and the language information
104
SP336805WO00
included in the second descriptor is set to represent a
non-language.
(2) The transmitting apparatus according to the (1),
wherein
5 the superimposition information data is DVB
subtitle data, and
the descriptors are a component descriptor and a
subtitle descriptor.
(3) The transmitting apparatus according to the (1)
10 or (2), wherein the language information representing a
non-language is "zxx" representing a non-language of an
ISO language code.
(4) The transmitting apparatus according to the (1)
or (2), wherein the language information representing a
15 non-language is any one of language codes included in a
space from "qaa" to "qrz" of an ISO language code.
(5) The transmitting apparatus according to any one
of the (1) to (4), wherein
the superimposition information data is DVB
20 subtitle data,
a first subtitle descriptor and a second subtitle
descriptor corresponding respectively to the first
private data stream and the second private data stream
are inserted into the multiplexed data stream, and
25 a subtitle type represented by subtitle type
information of the first subtitle descriptor is different
from a subtitle type represented by subtitle type
information of the second subtitle descriptor.
(6) The transmitting apparatus according to any one
30 of the (1) to (5), wherein
the superimposition information data is DVB
105
SP336805WO00
subtitle data, and
a segment including the superimposition information
data of the first private data stream is equal to a page
ID of a segment including the disparity information of
5 the second private data stream.
(7) The transmitting apparatus according to any one
of the (1) to (5), wherein
the superimposition information data is DVB
subtitle data, and
10 the disparity information included in the second
private data stream is operated as an ancillary page.
(8) A transmitting method including the steps of:
outputting left-eye image data and right-eye image
data for displaying a stereoscopic image;
15 outputting superimposition information data to be
superimposed on an image by the left-eye image data and
the right-eye image data;
outputting disparity information for providing a
disparity by shifting the superimposition information to
2 0 be superimposed on the image by the left-eye image data
and the right-eye image data; and
transmitting a multiplexed data stream including a
video data stream including the image data output in the
image data output step, a first private data stream
25 including the superimposition information data output in
the superimposition information data output step, and a
second private data stream including the disparity
information output in the disparity information output
step,
30 wherein a first descriptor and a second descriptor
including respective pieces of language information
106
SP336805WO00
corresponding to the first private data stream and the
second private data stream are inserted into the
multiplexed data stream, and the language information
included in the second descriptor is set to represent a
5 non-language.
(9) A receiving apparatus including:
a data receiving unit configured to receive a
multiplexed data stream including a video data stream
including left-eye image data and right-eye image data
10 for displaying a stereoscopic image, a first subtitle
data stream including superimposition information data to
be superimposed on an image by the left-eye image data
and the right-eye image data, and a second subtitle data
stream including disparity information for providing a
15 disparity by shifting the superimposition information to
be superimposed on the image by the left-eye image data
and the right-eye image data;
a video decoding unit configured to extract the
video data stream from the multiplexed data stream
2 0 received by the data receiving unit and decode the video
data stream extracted; and
a subtitle decoding unit configured to extract the
first subtitle data stream from the multiplexed data
stream received by the data receiving unit and decode the
25 first subtitle data stream extracted,
wherein
descriptors including respective pieces of subtitle
type information corresponding to the first subtitle data
stream and the second subtitle data stream are inserted
30 into the multiplexed data stream,
the respective pieces of subtitle type information
107
SP336805WO00
included in the descriptors corresponding to the first
subtitle data stream and the second subtitle data stream
are set to represent different subtitle types, and
the subtitle decoding unit determines the subtitle
5 data stream to be decoded after being extracted from the
multiplexed data stream, based on the subtitle type
information inserted into the descriptor.
(10) A receiving apparatus including:
a data receiving unit configured to receive a
10 multiplexed data stream including a video data stream
including left-eye image data and right-eye image data
for displaying a stereoscopic image, a first subtitle
data stream including superimposition information data to
be superimposed on an image by the left-eye image data
15 and the right-eye image data, and a second subtitle data
stream including disparity information for providing a
disparity by shifting the superimposition information to
be superimposed on the image by the left-eye image data
and the right-eye image data;
20 a video decoding unit configured to extract the
video data stream from the multiplexed data stream
received by the data receiving unit and decode the video
data stream extracted; and
a subtitle decoding unit configured to extract the
25 first subtitle data stream from ,the multiplexed data
stream received by the data receiving unit, and decode the
first subtitle data stream extracted,
wherein
descriptors including respective pieces of language
30 information corresponding to the first subtitle data
stream and the second subtitle data stream are inserted
108
SP336805WO00
into the multiplexed data stream,
the language information included in the descriptor
corresponding to the second subtitle data stream is set
to represent a non-language, and
5 the subtitle decoding unit determines the subtitle
data stream to be decoded after being extracted from the
multiplexed data stream, based on the language
information inserted into the descriptor.
10 REFERENCE SIGNS LIST
[0285]
10, lOA Image transmitting/receiving system
100 Broadcasting station
lllL, lllR Camera
15 112 Video framing unit
113 Disparity vector detecting unit
114 Microphone
115 Data extracting unit
115a Data recording medium
20 116 to 118 Change-over switch
119 Video encoder
120 Audio encoder
121 Subtitle generating unit
122 Disparity information creating unit
25 123 Subtitle processing unit
125 Subtitle encoder
126 Multiplexer
200, 200A Set-top box (STB)
201 Video stream processing unit
30 202 HDMI terminal
2 03 Antenna terminal
109
SP336805WO00
204 Digital tuner
205 Video signal processing circuit
2 06 HDMI transmitting unit
207 Audio signal processing circuit
5 208 Network terminal
209 Network interface
211 CPU
215 Remote control receiving unit
216 Remote control transmitting unit
10 221 Demultiplexer
222 Video decoder
223 Encoded data buffer
224 Subtitle decoder
225 Pixel buffer
15 22 6 Disparity information interpolating unit
227 Position control unit
228 Video superimposing unit
229 Audio decoder
300, 300A Television receiver (TV)
20 301 3D signal processing unit
302 HDMI terminal
303 HDMI receiving unit
3 04 Antenna terminal
305 Digital tuner
25 306 Video stream processing unit
307 Video/graphic processing circuit
308 Panel driving circuit
309 Display panel
310 Audio signal processing circuit
30 311 Audio amplifying circuit
312 Speaker

SP336805WO00
CLAIMS
1. A transmitting apparatus comprising:
an image data output unit configured to output
5 left-eye image data and right-eye image data for
displaying a stereoscopic image;
a superimposition information data output unit
configured to output superimposition information data to
be superimposed on an image by the left-eye image data
10 and the right-eye image data;
a disparity information output unit configured to
output disparity information for providing a disparity by
shifting the superimposition information to be
superimposed on the image by the left-eye image data and
15 the right-eye image data; and
a data transmitting unit configured to transmit a
multiplexed data stream including a video data stream
including the image data output from the image data
output unit, a first private data stream including the
2 0 superimposition information data output from the
superimposition information data output unit, and a
second private data stream including the disparity
information output from the disparity information output
unit,
25 wherein a first descriptor and a second descriptor
including respective pieces of language information
corresponding to the first private data stream and the
second private data stream are inserted into the
multiplexed data stream, and the language information
30 included in the second descriptor is set to represent a
non-language.
Ill
SP336805WO00
2. The transmitting apparatus according to claim 1,
wherein
the superimposition information data is DVB
5 subtitle data, and
the descriptors are a component descriptor and a
subtitle descriptor.
3. The transmitting apparatus according to claim 1,
10 wherein the language information representing a nonlanguage
is "zxx" representing a non-language of an ISO
language code.
4. The transmitting apparatus according to claim 1,
15 wherein the language information representing a nonlanguage'
is any one of language codes included in a space
from "qaa" to "qrz" of an ISO language code.
5. The transmitting apparatus according to claim 1,
20 wherein
the superimposition information data is DVB
subtitle data,
a first subtitle descriptor and a second subtitle
descriptor corresponding respectively to the first
25 private data stream and the second private data stream
are inserted into the multiplexed data stream, and
a subtitle type represented by subtitle type
information of the first subtitle descriptor is different
from a subtitle type represented by subtitle type
30 information of the second subtitle descriptor.
112
SP336805WO00
6. The transmitting apparatus according to claim 1,
wherein
the superimposition information data is DVB
subtitle data, and
5 a segment including the superimposition information
data of the first private data stream is equal to a page
ID of a segment including the disparity information of
the second private data stream.
10 7. The transmitting apparatus according to claim 1,
wherein
the superimposition information data is DVB
subtitle data, and
the disparity information included in the second
15 private data stream is operated as an ancillary page.
8. A transmitting method comprising the steps of:
outputting left-eye image data and right-eye image
data for displaying a stereoscopic image;
2 0 outputting superimposition information data to be
superimposed on an image by the left-eye image data and
the right-eye image data;
outputting disparity information for providing a
disparity by shifting the superimposition information to
25 be superimposed on the image by the left-eye image data
and the right-eye image data; and
transmitting a multiplexed data stream including a
video data stream including the image data output in the
image data output step, a first private data stream
30 including the superimposition information data output in
the superimposition information data output step, and a
113
SP335805WO00
second private data stream including the disparity
information output in the disparity information output
step,
wherein a first descriptor and a second descriptor
5 including respective pieces of language information
corresponding to the first private data stream and the
second private data stream are inserted into the
multiplexed data stream, and the language information
included in the second descriptor is set to represent a
10 non-language.
9. A receiving apparatus comprising:
a data receiving unit configured to receive a
multiplexed data stream including a video data stream
15 including left-eye image data and right-eye image data
for displaying a stereoscopic image, a first subtitle
data stream including superimposition information data to
be superimposed on an image by the left-eye image data
and the right-eye image data, and a second subtitle data
20 stream including disparity information for providing a
disparity by shifting the superimposition information to
be superimposed on the image by the left-eye image data
and the right-eye image data;
a video decoding unit configured to extract the
25 video data stream from the multiplexed data stream
received by the data receiving unit and decode the video
data stream extracted; and
a subtitle decoding unit configured to extract the
first subtitle data stream from the multiplexed data
30 stream received by the data receiving unit and decode the
first subtitle data stream extracted,
114
SP336805WO00
wherein
descriptors including respective pieces of subtitle
type information corresponding to the first subtitle data
stream and the second subtitle data stream are inserted
5 into the multiplexed data stream,
the respective pieces of subtitle type information
included in the descriptors corresponding to the first
subtitle data stream and the second subtitle data stream
are set to represent different subtitle types, and
10 the subtitle decoding unit determines the subtitle
data stream to be decoded after being extracted from the
multiplexed data stream, based on the subtitle type
information inserted into the descriptor.
15 10. A receiving apparatus comprising:
a data receiving unit configured to receive a
multiplexed data stream including a video data stream
including left-eye image data and right-eye image data
for displaying a stereoscopic image, a first subtitle
20 data stream including superimposition information data to
be superimposed on an image by the left-eye image data
and the right-eye image data, and a second subtitle data
stream including disparity information for providing a
disparity by shifting the superimposition information to
25 be superimposed on the image by the left-eye image data
and the right-eye image data;
a video decoding unit configured to extract the
video data stream from the multiplexed data stream
received by the data receiving unit and decode the video
30 data stream extracted; and
a subtitle decoding unit configured to extract the
115
SP336805WO00
fi:i:st subtitle data stream from the multiplexed data
stream received by the data receiving unit and decode the
first subtitle data stream extracted,
wherein
5 descriptors including respective pieces of language
information corresponding to the first subtitle data
stream and the second subtitle data stream are inserted
into the multiplexed data stream,
the language information included in the descriptor
10 corresponding to the second subtitle data stream is set
to represent a non-language, and
the subtitle decoding unit determines the subtitle
data stream to be decoded after being extracted from the
multiplexed data stream, based on the language
15 information inserted into the descriptor.