SP338522WO00
DESCRIPTION
TRANSMISSION DEVICE, TRANSMISSION METHOD, AND RECEPTION
DEVICE
5 TECHNICAL FIELD
[0001]
The present technology relates to a transmission
device, a transmission method, and a reception device.
Particularly, the present technology relates to a •
10 transmission device that transmits data of overlapping
information and disparity information together with
stereoscopic image data including left-eye image data and
right-eye image data.
15 BACKGROUND ART
[0002]
For example, in Patent Document 1, a transmission
system of stereoscopic image data using a television
broadcast wave has been proposed. In this transmission
2 0 system, stereoscopic image data including left-eye image
data and right-eye image data is transmitted, and a
stereoscopic image display using binocular parallax is
performed.
[0003]
25 Fig. 79 illustrates a relation between a horizontal
display position ,of an object (body) on a screen and a
reproduction position of a stereoscopic image thereof in
a stereoscopic image display using binocular parallax.
For example, for object A of,which a left image La is
30 displayed to be shifted to the right side and a right
image Ra is displayed to be shifted to the left side as
1
SP338522WO00
illustrated on the screen in the figure, the left and
right lines of sights intersect with each other on a
further front side than the screen face, and so the
reproduction position of the stereoscopic image is
5 located on a further front side than the screen face.
DPa represents a disparity vector in a horizontal
direction related to the object A.
[0004]
In addition, for example, for object B of which a
10 left image Lb and a right image Rb are displayed at the
same position as illustrated on the screen in the figure,
the left and right lines of sights intersect with each
other on the screen face, and so the reproduction
position of the stereoscopic image is on the screen face.
15 Furthermore, for example, for object C of which the left
image Lc is displayed to be shifted to the left side and
the right image Re is displayed to be shifted to the
right side as illustrated on the screen in the figure,
the left and right lines of sights intersect with each
20 other on a further inner side than the screen face, and
so the reproduction position of the stereoscopic image is
located on a further inner side than the screen face.
DPc represents a disparity vector in a horizontal
direction related to the object C.
25
CITATION LIST
PATENT DOCUMENT
[0005]
Patent Document 1: Japanese Patent Application Laid-Open
30 No. 2005-6114
2
SP338522WO00
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0006]
As described above, when a stereoscopic image is
5 displayed, a viewer usually perceives a sense of
perspective of a stereoscopic image using binocular
parallax. Overlapping information overlapping an image
such as a subtitle is expected to be rendered in
conjunction with a stereoscopic image display with a
10 sense of depth in a three dimension (3D) space as well as
a two dimension (2D) space. For example, when a subtitle
is displayed to overlap (be overlaid with) an image, if
the subtitle is not displayed ahead of an object within a
closest image in terms of a sense of perspective, the
15 viewer is likely to feel an inconsistency in a sense of
perspective.
[0007]
In this regard, there may be considered a technique
in which disparity information between a left-eye image
20 and a right-eye image data are transmitted together with
data of overlapping information, and a reception side
causes parallax to occur between left-eye overlapping
information and right-eye overlapping information. In a
reception device capable of displaying a stereoscopic
25 image, disparity information is meaningful information.
When a broadcasting station transmits data of overlapping
information and disparity information through individual
private data streams (PES streams), a reception device
with a 3D function needs to easily recognize the presence
30 of two PES streams and acquire corresponding disparity
information together with data of overlapping information.
3
SP338522WO00
[0008]
The present technology is directed to causing a
reception device with a 3D function to be able to
appropriately acquire corresponding disparity information
5 tougher with data of overlapping information.
SOLUTIONS TO PROBLEMS
[0009]
A concept of the present technology is a
10 transmission device, including:
an image data output unit that outputs left-eye
image data and right-eye image data configuring a
stereoscopic image;
an overlapping information data output unit that.
15 outputs data of overlapping information overlapping an
image based on the left-eye image data and the right-eye
image data;
a disparity information output unit that outputs
disparity information for shifting the overlapping
20 information overlapping the image based on the left-eye
image data and the right-eye image data and causing
disparity to occur; and
a data transmitting unit that transmits multiplexed
data stream including a video data stream including the
25 image data, a first private data stream including the
data of the overlapping information, and a second private
data Stream including the disparity information,
wherein association information associating the
first private data stream with the second private data ,
30 stream is included in the multiplexed data stream.
[0010]
4
SP338522WO00
In the present technology, an image data output
unit outputs left-eye image data and right-eye image data
configuring a stereoscopic image. An overlapping
information data output unit outputs data of overlapping
5 information overlapping an image based on the left-eye
image data and the right-eye image data. Here,
overlapping information is information such as a subtitle,
a graphics, or a text which overlaps an image. A
disparity information output unit that outputs disparity
10 information for shifting the overlapping information
overlapping the image based on the left-eye image data
and the right-eye image data and causing disparity to
occur.
[0011]
15 A data transmitting unit transmits multiplexed data
stream. The multiplexed data stream includes a video
data stream including image data, a first private data
stream including data of overlapping information, and a
second private data stream including disparity
20 information. Association information associating the
first private data stream with the second private data
stream is included in the multiplexed data stream.
[0012]
In the present technology, for example,
25 identification information which is common to a first
descriptor describing information related to the first
private data stream and a second descriptor describing
information related to the second private data stream may
be described as the association information. In this
30 case, the common identification information may be
defined by a special value representing that the first
5
SP338522WO00
private data stream and the second private data stream
are present. .
[0013-]
Further, in the present technology, the multiplexed
5 data stream may include the first private data stream and
the second private data stream corresponding to each of a
plurality of language services, and the pieces of common
identification information corresponding to the
respective language services may be set to be different
10 from each other. Thus, even in the multilingual service,
a reception device with a 3D function at a reception side
can properly extract and decode the first.private' data
stream and the second private data stream which are
associated with each other for each language service.
15 [0014]
Further, in the present technology, for example,
the data of the overlapping information may be subtitle
data of a DVB format, and a common composition page ID
may be described in a first subtitle descriptor
2 0 corresponding to the first private data stream and a
subtitle descriptor corresponding to the second private
data stream.
[0015]
Further, in the present technology, for example,
25 linguistic information may be described in the first
descriptor and the second descriptor, and the linguistic
information described in the second descriptor may be set
to represent a non-language. In this case, for example,
the linguistic information representing the non-language
30 may be any one of language codes included in a space of
"zxx" or "qaa" to "qrz" representing an ISO language code.
6
SP338522WO00
Thus, a reception device with a legacy 2D function can
prevent the second private data stream including the
disparity information from being extracted and decoded,
and prevent the disparity information from interfering
5 with the reception process.
[0016]
As described above, in the present technology, the
association information associating the first private
data stream with the second private data stream is
10 included in the multiplexed data stream. Thus, the
reception device with the 3D function at the reception
side can efficiently and appropriately extract and decode
the first private data stream and the second private data
stream which are associated with each other based on the
15 association information.
[0017]
Further, in the present technology, for example,
identification information which is common to a first
descriptor describing information related to the first
20 private data stream and a second descriptor describing
information related to the second private data stream may
be described as the association information, and type
information representing information for a stereoscopic
image display may be described in the first descriptor
25 and the second descriptor. For example, the data of the
overlapping information may be subtitle data of a DVB
format, and the type information may be subtitle type
information.
[0018]
30 In this case, although the type information
representing the information for the stereoscopic image
7
SP338522WO00
display is described, since the disparity information is
not included in the first private data stream, the
reception device with the 3D function at the reception
side can recognize the second private data stream
5 including the disparity information, search for the
second private data stream including the disparity
information based on the common identification
information, and decode the second private data stream
including the disparity information.
10 [0019]
Further, in the present technology, for example, a
descriptor describing dedicated linking information for
linking the first private data stream with the second
private data stream may be included in the multiplexed
15 data stream. For example, the descriptor may be a
dedicated descriptor describing the dedicated linking
information. Alternatively, the existing descriptor may
be extended to describe the association information. In
this case, the descriptor includes the first descriptor
20 and the second descriptor respectively corresponding to
the first private data stream and the second private data
stream, and the identification information which is
common to the first descriptor and the second descriptor
is described.
25 [0020]
Another concept of the present technology is a
reception device, including:
a data receiving unit that receives multiplexed , '
data stream including a video data stream including left-
30 eye image data and right-eye image data configuring a
stereoscopic image, a first private data stream including
8
SP338522WO00
data of overlapping information overlapping an image
based on the left-eye image data and the right-eye image
data, and a second private data stream including,
disparity information for shifting the overlapping
5 information overlapping the image based on the left-eye
image data and the right-eye image data and causing
disparity to occur;
a first decoding unit that extracts the video data
stream from the multiplexed data stream and decodes the
10 video data stream; and
a second decoding unit that extracts the first
private data stream and the second private data stream
from the multiplexed data stream and decodes the first
private data stream and the second private data stream,
15 wherein association information associating the
first private data stream with the second private data
stream is included in the multiplexed data stream, and
the second decoding unit extracts the first private
data stream and the second private data stream from the
20 multiplexed, data stream based on the association
information.
[0021]
In 'the present technology, a data receiving unit
receives a multiplexed data stream including a video data
25 stream, a first private data stream, and a second private
data stream. Further, a first decoding unit extracts the
video data stream from the multiplexed data stream, and
decodes the video data stream. Further, a second
decoding unit extracts the first private data stream and
30 the second private data stream from the multiplexed data
stream, and decodes the first private data stream and the
9
SP338522WO00
second private data stream.
[0022]
Here, association information associating the first
private data stream with the second private data stream
5 is included in the multiplexed data stream. Thus, the
second decoding unit can properly extract and decode the
first private data stream and the second private data
stream which are associated with each other based on the
association information, and acquire data of overlapping
10 information and disparity information corresponding
thereto.
EFFECTS OF THE INVENTION
[0023]
15 According to the present technology, a reception
device with a 3D function can easily acquire
corresponding disparity information together with data of
overlapping information.
2 0 BRIEF DESCRIPTION OF DRAWINGS
[0024]
Fig. 1 is a block diagram illustrating a
configuration ..example of an image transceiving system
according to an embodiment of the invention.
25 Fig. 2 is a block diagram illustrating a
configuration example of a transmission data generating
unit in a broadcasting station.
Fig. 3 is a diagram illustrating image data of a
pixel format of 1920x1080.
30 Fig. 4 is a diagram to describe a "top-and-bottom"
format, a "side by side" format, and a "frame sequential"
10
SP338522WO00
format, which are a transmission format of stereoscopic
image data (3D image data).
Fig. 5 is a diagram for describing an example of
detecting a disparity vector of a right-eye•image to a
5 left-eye image.
Fig. 6 is a diagram for describing that a disparity
vector is obtained by a block matching method.
Fig. 7 is a diagram illustrating an example of an
image when a value of a disparity vector of each pixel is
10 used as a brightness value of each pixel.
Fig. 8 is a diagram illustrating an example of a
disparity vector of each block.
Fig. 9 is a diagram for describing a downsizing
process executed by a disparity information creating unit
15 of a transmission data generating unit.
Fig. 10 is a diagram illustrating an example of a
region defined on a screen and a sub region defined in
the region in subtitle data.
Fig. 11 is a diagram illustrating configurations of
20 a 2 D stream and a 3D extension stream included in a
transport stream TS.
Fig. 12 is a diagram for describing association of
time stamp (PTS) values included in PES headers of a 2D
stream (PESl (1): PES#1) and a 3D extension stream (PES2
25 (2) : PES#2) .
Fig. 13 is a diagram illustrating an example in
which a time stamp (PTS) values of a 2D stream and a 3D
extension stream are set to different values from each
other.
30 Fig. 14 is a diagram illustrating another example
in which a time stamp (PTS) values of 'a 2D stream and a
11
SP338522WO00
3D extension stream are set to different values from each
other.
Fig. 15 is a diagram illustrating a configuration
example of a transport stream TS including a 2D stream
5 and a 3D extension stream.
Fig. 16 is a diagram illustrating a structure of a
PCS (page_composition_segment) configuring subtitle data.
Fig. 17 is a diagram illustrating a correspondence
relation between each value of "segment_type" and a
10 segment type.
Fig. 18 is a diagram for describing information
(component_type = 0x15, 0x25) representing a format of a
newly defined 3D subtitle.
Fig. 19 is a diagram illustrating that a subtitle
15 descriptor (subtitling_descriptor) and a component
descriptor {component_descriptor) included in a transport
steam are extracted.
Fig. 20 is a diagram illustrating that a PES stream
(a 2D stream and a 30 extension stream) included in a
20 transport steam is extracted.
Fig. 21 is a diagram illustrating that any one of
language codes included in a space of '^qaa" to "qrz" '
which are ISO language codes are used as an ISO language
code representing a non-language.
25 Fig. 22 is a diagram illustrating an excerpt of an
ISO language code (ISO 639-2 Code) list.
Fig. 23 is a diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream).
30 Fig. 24 is a diagram illustrating an example of
syntax of a component descriptor (component_descriptor).
12
. SP338522WO00
Fig. 25 is a diagram illustrating an example of
syntax of a subtitle descriptor (subtitling_descriptor).
Fig. 2 6 is a diagram illustrating an example in
which disparity information is updated using an interval
5 period, the interval period is fixed, and the period is
equal to an update period.
Fig. 27 is a diagram illustrating an example in
which disparity information is updated using an interval
period, and a short period is used as the interval period.
10 Fig. 28 is a diagram illustrating a configuration
example of a 3D extension stream.
Fig. 29 is a diagram illustrating an update example
of disparity information when a DSS segment is
sequentially transmitted.
15 Fig. 30 is a diagram illustrating an update example
of the disparity information (disparity) in which an
update frame interval is represented by a multiple of an
interval period (ID: interval duration) serving as a unit
time.
20 Fig. 31 is a diagram illustrating a display example
of a subtitle in which two regions serving as a subtitle
display region are included in a page region (Area for
Page_default).
Fig. 32 is a diagram illustrating an example of a
25 disparity information curve of each region and a page
wh$n both disparity information of a region unit and
disparity information of a page unit including all
regions are included in a DSS segment as disparity
information (Disparity) to be sequentially updated in a
30 subtitle display period.
Fig. 33 is a diagram illustrating a structure in
13
SP338522WO00
which disparity information of a page and each region is
transmitted.
Fig. 34 is a diagram (1/3) illustrating an example
of syntax of a DSS.
5 Fig. 35 is a diagram (2/3) illustrating an example
of syntax of a DSS.
Fig. 36 is a diagram (3/3) illustrating an example
of syntax of a DSS.
Fig. 37 is a diagram (1/4) illustrating main data
10 specifying content (semantics) of a DSS.
Fig. 38 is a diagram (2/4) illustrating main data
specifying content (semantics).of a DSS.
Fig. 39 is a diagram (3/4) illustrating main data
specifying content (semantics) of a DSS.
15 Fig. 40 is a diagram (4/4) illustrating main data
specifying content (semantics) of a DSS.
Fig. 41 is a diagram illustrating a broadcast
reception concept when a set-top box and a television
receiver are devices with a 3D function.
20 Fig. 42 is a diagram illustrating a broadcast
reception concept when a set-top box and a television
receiver are devices with a legacy 2D function.
Fig. 43 is a diagram illustrating' a broadcast
reception concept when receivers are a device with a
25 legacy 2D function (2D receiver) and a device with a 3D
function (3D receiver).
Fig. 44 is a diagram illustrating a stream -
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream) when a bilingual
30 service is provided.
Fig. 45 is a diagram illustrating a configuration
14
SP338522WO00
example of a transport stream TS when a bilingual service
is provided.
Fig. 46 is a diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
5 stream and a 3D extension stream) when a 3D extension
stream is used as an ancillary page.
Fig. 47 is a diagram illustrating a configuration
example of a transport stream TS when a 3D extension
stream is used as an ancillary page.
10 Fig. 48 is a diagram for describing information
when 3D service determination is performed.
Fig. 49 is a flowchart illustrating an example of a
3D service determining process in a receiver (a set-top
box) .
15 Fig. 50 is a flowchart illustrating another example
of a 3D service determining process in a receiver (a settop
box) .
Fig. 51 is a flowchart schematically illustrating
the flow of a process of a receiver (a set-top box) when
20 it is determined that a service is a 3D service.
Fig. 52 is a diagram illustrating a configuration
example of a 3D PES stream {a 3D stream).
Fig. 53 is a diagram illustrating a display example
of a subtitle (graphics information) on an image and a
25 sense of perspective of a background, a near-view object,
and a subtitle.
Fig. 54 is a diagram illustrating a display example
of a subtitle on an image, and a left-eye subtitle LGI
and a right-eye subtitle RGI for displaying a subtitle.
30 Fig. 55 is a block diagram illustrating a
configuration example of a set-top box configuring an
15
SP338522WO00
image transceiving system.
Fig. 56 is a block diagram illustrating a
configuration example (3D support) of a bit stream
processing unit configuring a set-top box.
5 Fig. 57 is a block diagram illustrating an example
of syntax of a multi-decoding descriptor used to
associate with a 2D stream with a 3D extension stream.
Fig. 58 is a block diagram illustrating a
configuration example of a television receiver
10 configuring an image transceiving system.
Fig. 59 is a block diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream) when "subtitling_type"
of a 2D stream is set to 3D.
15 Fig. 60 is a block diagram illustrating a
configuration example of a transport stream TS when
"subtitling_type" of a 2D stream is set to 3D.
Fig. 61 is a block diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
20 stream and a 3D extension stream) when a- bilingual
service is provided.
Fig. 62 is a flowchart schematically illustrating
the flow of a process of a receiver (set-top-box) when a
service is determined as a 3D service.
25 Fig. 63 is a block diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream) when a composition page
ID is defined by a special value (special_valueA).
Fig. 64 is a diagram illustrating a configuration
30 example of a transport stream TS when a composition page
ID is defined by a special value (special_valueA).
16
SP338522WO00
Fig. 65 is a diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream) when a bilingual
service is provided.
5 Fig. 66 is a flowchart schematically illustrating
the flow of a process of a receiver (set-top-box) when a
service is determined as a 3D service.
Fig. 67 is a diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
10 stream and a 3D extension stream) when dedicated
information (linking information) for linking of a 3D
extension stream is described in a descriptor.
Fig. 68 is a diagram illustrating an example of
syntax of a stream association ID descriptor.
15 Fig. 69 is a diagram illustrating content
(semantics) of main information in a syntax example of a
stream association ID descriptor.
Fig. 70 is a diagram illustrating an example of
syntax of an extended component descriptor.
20 Fig. 71 is a diagram illustrating content
(semantics) of main information in a syntax example of an
extended component descriptor.
Fig. 72 is a diagram illustrating a configuration
example of the transport stream TS when dedicated
25 information (linking information) for linking of a 3D
extension stream is described in a descriptor.
Fig. 73 is a diagram illustrating a stream
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream) when a bilingual
30 service is provided.
Fig. 74 is a diagram illustrating a configuration
17
SP338522WO00
example of a transport stream TS when a bilingual service
is provided.
Fig. 75 is a flowchart schematically illustrating
the flow of a process of a receiver (set-top box) when a .
5 service is determined as a 3D service.
Fig. 76 is a block diagram illustrating another
configuration example of a set-top box configuring an
image transceiving system.
Fig. 77 is a block diagram illustrating another
10 configuration example of a television receiver
configuring an image transceiving system.
Fig. 78 is a block diagram illustrating another
configuration example of an image transceiving system.
Fig. 7 9 is a diagram for describing a relation
15 between display positions of left and right images of an
object on a screen and a reproduction position of a
stereoscopic image when a stereoscopic image is displayed
using binocular parallax.
2 0 MODE FOR CARRYING OUT THE INVENTION
[0025]
Hereinafter, a mode for carrying out the present
invention (hereinafter, referred to as an "embodiment")
will be described. The description will be presented in
25 the following order:
1. Embodiment
2. Modified Example
[0026]
<1. Embodiment>
30 [Configuration Example of Image Transceiving
System]
18
SP338522WO00
Fig. 1 illustrates a configuration example of an
image transceiving system 10 according to an embodiment.
The image transceiving system 10 includes a broadcasting
station 100, a set-top box (STB) 200, and a television
5 receiver (TV) 300.
[0027]
The set-top box 200 is connected-with the
television receiver 300 through a digital interface of
HDMI (High Definition Multimedia Interface). The set-top
10 box 200 is connected with the television receiver 300
using an HDMI cable 400. An HDMI terminal 202 is
disposed in the set-top box 200. An HDMI terminal 302 is
disposed in the television receiver 300. One end of the
HDMI cable 400 is connected to the HDMI terminal 202 of
15 the set-top box 200, and the other end of the HDMI cable
400 is connected to the HDMI terminal 302 of the
television receiver 300.
[0028]
[Description of Broadcasting Station]
20 The broadcasting station 100 transmits a transport
stream TS through a broadcast wave. The broadcasting
station 100 includes a transmission data generating unit
110 that generates the transport stream TS. The
transport stream TS includes image data, audio data, data
25 of overlapping information, disparity information, and
the like. Here, the image data (hereinafter, referred to
appropriately as a "stereoscopic.image data") includes
left-eye image data and right-eye image data that
configures a stereoscopic image. The stereoscopic image
30 data has a predetermined transmission format. The
overlapping information generally refers to a subtitle,
19
SP338522WO00
graphics information, text information, or the like, but
refers a subtitle in this embodiment.
[0029]
"Configuration Example of Transmission Data
5 Generating Unit"
Fig. 2 illustrates a configuration example of the
transmission data generating unit 110 in the broadcasting
station 100.
The transmission data generating unit 110 transmits
10' disparity information (a disparity vector) through a data
structure which is compatible with a digital video
broadcasting (DVB) scheme which is one of the existing
broadcast standards.
The transmission data generating unit 110 includes a data
15 extracting unit 111, a video encoder 112, and an audio
encoder 113. The transmission data generating unit 110
further includes a subtitle generating unit 114, a
disparity information creating unit 115, a subtitle
processing unit 116, a subtitle encoder 118, and a
20 multiplexer 119.
[0030]
For example, a data recording medium Ilia is
detachably mounted to the data extracting unit 111. In
the data recording medium Ilia, audio data and disparity
25 information are recorded in association with stereoscopic
image data including left-eye image data and right-eye
image data. The data extracting unit 111 extracts
stereoscopic image data, audio data, disparity
information, and the like from the data recording medium ,
30 Ilia. Examples of the data recording medium Ilia include
disk-shaped recording medium and a semiconductor memory.
20
SP338522WO00
[0031]
The stereoscopic image data recorded in the data
recording medium Ilia is stereoscopic image data of a
predetermined transmission format. An example of a
5 transmission format of stereoscopic image data (3D image
data) will be described. Here, first to third
transmission formats will be described, but any other
transmission format may be used. Here, a case in which
each of left eye (L) image data and right eye (R) image
10 data is image data with a pixel format of a predetermined
resolution, for example, 1920x1080 as illustrated in Fig.
3 will be described as an example.
[0032]
The first transmission format is a top-and-bottom
15 format, and is a format in which data of each line of the
left-eye image data is transmitted in the first half in
the vertical direction, and data of each line of the
left-eye image data is transmitted in the second half in
the vertical direction as illustrated in Fig. 4(a). In
20 such a case, since the lines of the left-eye image data '
and the lines of the right-eye image data are thinned out
to 1/2, the vertical resolution becomes half of that of
the original signal.
[0033]
2,5 The second transmission format is a side-by-side
format, and is a format in which pixel data of the lefteye
image data is transmitted in the first half in a
horizontal direction, and pixel data of the right-eye
image data is transmitted in the second half in the
30 horizontal direction as illustrated in Fig. 4(b). In
such a case, pixel data of each one of the left-eye image
21
SP338522WO00
data and the right-eye image data in the horizontal
direction is thinned out to 1/2. The horizontal
resolution becomes half of that of the original signal.
[0034]
5 The third transmission format is a frame sequential
format or a L/R no-interleaving format, and is a format
in which left-eye image data and right-eye image data are
switched and transmitted sequentially in units of frames
as illustrated in Fig. 4(c).
10 This format includes a full frame format or a service
compatible format with a 2D format.
[0035]
For example, the disparity information recorded in
the data recording medium Ilia refers to a disparity
15 vector of each pixel configuring an image. An example of
detecting a disparity vector will be described. Here, an
example will be described in which a disparity vector of
a right-eye image with respect to a left-eye image is
detected. As illustrated in Fig. 5, the left-eye image
20 is set as a detection image, and the right-eye image is
set as a reference image. In this example, disparity
vectors at the positions of (xi, yi) and (xj, yj) are
detected.
[0036]
25 A case will be described as an example in which a
disparity vector at the position of (xi, yi) is detected.
In this case, in the left-eye image, a pixel located at
the position of (xi, yi) is set as the upper left side,
and, for example, a pixel block (disparity detection
30 block) Bi of 4x4, 8x8, or 16x16 is set. Then, in the
right-eye image, a pixel block that matches the pixel
22
SP338522WO00
block Bi is searched for.
[0037]
In such a case, in the right-eye image, a search
range having the position of (xi, yi) as its center is
5 set, and respective pixels within the search range are
sequentially set as a pixel of interest, and comparison
blocks, for example, of 4x4, 8x8, or 16x16,.which are the
same as the above-described pixel block Bi, are
sequentially set.
10 [0038]
A sum of absolute values of differences between
corresponding respective pixels of the pixel block Bi and
the comparison blocks that are sequentially set is
calculated. Here, as illustrated in Fig. 6, when a pixel
15 value of the pixel block Bi is L(x, y) and a pixel value
of the comparison block is R(x, y), a sum of the absolute
values of differences between the pixel block Bi and a
specific comparison block is represented as S|L(x, y) -
R{x, y) I .
20 [0039]
When n pixels are included in the search range set
in the right-eye image, n sums SI to Sn are finally
acquired, and a minimum sum Smin is selected from among
them. Then, the position (xi' , yi') of the pixel located
25 on the upper left side can be acquired from the
comparison block from which the sum Smin is acquired.
Accordingly, a disparity vector at the position of (xi,
yi) is detected as (xi'-xi, yi'-yi). Although detailed
description will not be presented, also for a disparity
30 vector at the position of (xj, yj), a pixel located at
the position of (xj, yj) is set as the upper left side in
23
SP338522WO00
the left-eye image, and a pixel block Bj, for example, of
4x4, 8x8, or 16x16 is set, so that the disparity vector
. can be detected in a similar process.
[0040]
5 Returning to Fig. 2, the video encoder 112 encodes
the stereoscopic image data extracted from the data
extracting unit 111 using MPEG4-AVC, MPEG2, VC-1, or the
like, and generates a video data stream (video elementary
stream). The audio encoder 113 encodes the audio data
10 extracted from the data extracting unit 111 using AC3,
XAC, or the like, and generates an audio data stream
(audio elementary stream).
[0041]
The subtitle generating unit 114 generates subtitle
15 data which is subtitle data of DVB (digital video
broadcasting). The subtitle data is 2D image subtitle
data. The subtitle generating unit 114 configures an
overlapping information data output unit.
[0042]
20 The disparity information creating unit 115
executes a downsizing process on a disparity vector (a
disparity vector in a horizontal direction) of each pixel
or a plurality of pixels extracted from the data
extracting unit 111, and generates disparity information
25 of each player as follows. The disparity information
needs not be necessarily generated by the disparity
information creating unit 115 and may be separately
supplied from the outside.
[0043]
30 Fig. 7 illustrates an example of data in a relative
depth direction given like a brightness value of each
24
SP338522WO00
pixel. Here, data in the relative depth direction can be
dealt with as a disparity vector of each pixel bypredetermined
conversion. In this example, a person
portion has a high brightness value. This means that the
5 person portion has a large disparity vector value, and
thus means that in a stereoscopic image display, the
person portion is recognized in a standing-out state.
Further, in this example, a background portion has a
small brightness value. This means that the background
10 portion has a small disparity vector value, and thus
means that in a sto^reoscopic image display, the
background portion is recognized in a sunken state.
[0044]
Fig. 8 illustrates an example of a disparity vector
15 of each block. A block corresponds to a layer above a
pixel positioned at the lowest layer. The blocks are
configured such that an image (picture) region is divided
into a predetermined size in a horizontal direction and a
vertical direction. For example, a disparity vector
20 having a largest value among disparity vectors of all
pixels present in a block is selected as a disparity
vector of each block. In this example, a disparity
vector of each block is represented by an arrow, and the
size of an arrow corresponds to the size of a disparity
25 vector.
[0045]
Fig. 9 illustrates an example of the downsizing
process executed by the disparity information creating
unit 115. First, disparity information creating unit 115
30 obtains a signed disparity vector of each block using a
disparity vector of each pixel as illustrated in Fig.
25
SP338522WO00
9(a) . As described above, a block corresponds to a layer
above a pixel located on the lowermost layer, and is
configured by dividing an image (picture) region into a
predetermined size in a horizontal direction and a
5 vertical direction. Then, for example, a disparity
vector having a smallest value or a disparity vector
having a negative value whose absolute value is largest
among disparity vectors of all pixels pr-esent in a block
is selected as the disparity vector of each block.
10 [0046]'
Next, the disparity information creating unit 115
obtains a disparity vector of each group (group of block)
using a disparity vector of each block as illustrated in
Fig. 9(b) . A group corresponds to an upper layer of a
15 block, and is obtained by grouping a plurality of
neighboring blocks together. In the example of Fig. 9(b),
each group is configured with 4 blocks bound by a dotted
frame. Then, for example, a disparity vector having a
smallest value or a disparity vector having a negative
20 value whose absolute value is largest among disparity
vectors of all blocks in a corresponding group is
selected as of a disparity vector of each group.
[0047]
Next, the disparity information creating unit 115
25 obtains a disparity vector of each partition using a
disparity vector of each group as illustrated in Fig.
9(c) . A partition corresponds to an upper layer of a
group, and is obtained by grouping a plurality of
neighboring groups together... In the example of Fig. 9(c),
30 each partition is configured with 2 groups bound by a
dotted frame. Then, for example, a disparity vector
26
SP338522WO00
having a smallest value or a disparity vector having a
negative value whose absolute value is largest among
disparity vectors of all groups in a corresponding
partition is selected as of a disparity vector of each
5 partition.
[0048]
Next the disparity information creating unit 115
obtains a disparity vector of the entire picture (entire
image) located in a highest layer using a disparity
10 vector of each partition as illustrated in Fig. 9(d). In
the example of Fig. 9(d), four partitions bound by a
dotted frame are included in the entire picture. Then,
for example, a disparity vector having a smallest value
or a disparity vector having a negative value whose
15 absolute value is largest among disparity vectors of all
partitions included in the entire picture is selected as
of a disparity vector of the entire picture.
[0049]
In the above-described way, the disparity
20 information creating unit 115 can obtained the disparity
vector of each region of each of layers including the
bloclc, the group, the partition, and the entire picture
, by executing the downsizing process on the disparity
vector of each pixel located in the lowermost layer. In
25 the example of the downsizing process illustrated in Fig.
9, disparity vectors of four layers of the block, the
group, the partition, and the entire picture are finally
obtained in addition to the layer of the pixel. However,
the number of layers, a region dividing method of each
30 layer, and the number of regions are not limited to the
above example.
27 '
SP338522WO00
[0050]
Referring back to Fig. 2, the subtitle processing
unit 116 can define a sub region within a region based on
the subtitle data generated by the subtitle generating
5 unit 114. Further, the subtitle processing unit 116 sets
disparity information for shift-adjusting the display
position of the overlapping information in the left-eye
image and the right-eye image data based on the disparity
information generated by the disparity information
10 creating unit 115. The disparity information can be set
in units of sub regions, in units of regions, or in units
of pages.
[0051]
Fig. 10(a) illustrates an example of a region
15 defined on a screen and a sub region defined in the
region in subtitle data. In this example, two sub
regions of "SubRegion 1" and "SubRegion 2" are defined in
a region 0 in which "Region_Starting Position" is RO.
The position x (a horizontal position) of "SubRegion 1"
20 in the horizontal direction is SRI, and the position x (a
horizontal position) of "SubRegion 2" in the horizontal
direction is SR2. In this example, disparity information
"disparity 1" is set on the sub region "SubRegion 1", and
disparity information "disparity 2" is set on the sub
25 region "SubRegion 2."
[0052]
Fig. 10(b) illustrates a shift adjustment example
in a sub region in an left-eye image based on disparity
information. The disparity information "disparity 1" is
30 set on the sub region "SubRegion 1." For this reason,
shift adjustment is performed on the sub region
28
SP338522WO00
"SubRegion 1" such that the position x (horizontal
position) in the horizontal direction is SRl-disparity 1.
Further, the disparity information "disparity 2" is set
on the sub region "SubRegion 2." For this reason, shift
5 adjustment is performed on the sub region "SubRegion 2"
such that the position x (horizontal position) in the
horizontal direction is SR2-disparity 2.
[0053]
Fig. 10(c) illustrates a shift adjustment example
10 in a sub region in an right-eye image based on disparity
information. The disparity information "disparity 1" is
set'on the sub region "SubRegion 1." For this reason,
shift adjustment is performed on the sub region
"SubRegion 1" in a direction opposite to the left-eye
15 image such that the position x (horizontal position) in
the horizontal direction is SRl+disparity 1. Further,
the disparity information "disparity 2" is set on the sub
region "SubRegion 2." For this reason, shift adjustment
is performed on the sub region "SubRegion 2" in a
20 direction opposite to the left-eye image such that the
position x (horizontal position) in the horizontal
direction is SR2+disparity 2.
[0054]
The subtitle processing unit 116 outputs display
25 control information such as region information of a sub
region or disparity information together with the
. subtitle data generated by the subtitle generating unit
114. The disparity information can be set in units of
regions or in units of pages as well as in units of sub
30 regions as described above.
[0055]
29
SP338522WO00
The subtitle data includes segments such as a DDS,
PCS, an RCS, a CDS, an ODS, an EDS, and the like. The
DDS (display definition segment) designates a display
size for a HDTV. The PCS (page composition segment)
5 designates a region position in a page. The RCS (region
composition segment) designates the size of a region or
an encoding mode of an object or designates a starting
position of an object.
[0056]
10 The CDS (CLUT definition segment) designates CLUT
content. The ODS (object data segment) includes encoded
pixel data. The EDS (end of display set segment)
represents the end of subtitle data starting from the DDS
segment. In this embodiment, a DSS (Disparity Signaling
15 Segment) segment is further defined. The DSS segment
includes the display control information.
[0057]
Referring back to Fig. 2, the subtitle encoder 118
generates first and second private data streams (first
20 and second subtitle data streams). In other words, the
subtitle encoder 118 generates the first private data
stream (a 2D stream) including segments of the DDS, the
PCS, the RCS, the CDS, the ODS, and the EDS. Further,
the subtitle encoder 118 generates the second private
25 data stream (a 3D extension stream) including segments of
the DDS, the DSS, and the EDS.
[0058]
The multiplexer 119 obtains a transport stream TS
as a multiplexed data stream by multiplexing streams from
30 the video encoder 112, the audio encoder 113, and the
subtitle encoder 118. The transport stream TS includes a
30
SP338522WO00
video data stream, an audio data stream, and the first
and second private data streams as the PES (Packetized
Elementary Stream) stream.
[0059]
5 Fig. 11 illustrates configurations of a 2D stream
and a 3D extension stream included in the transport
stream TS. Fig. 11(a) illustrates a 2D stream in which a
PES header is arranged at the head, and a PES payload
including segments of the CDS, the PCS, the RCS, the CDS,
10 the ODS, and the EDS is subsequently arranged.
[0060],
Fig. 11(b) illustrates a 3D extension stream in
which a PES header is arranged at the head, and a PES
payload including segments of the DDS, the DSS, and the
15 EDS is subsequently arranged. The 3D extension stream
may be configured such that the segments of the DDS, the
. PCS, the DSS, and the EDS are included in the PES payload
as illustrated in Fig. 11(c). In this case, a page state
of the PCS is a normal case, and overlapping data
2 0 (bitmap) does not change.
[0061]
Here, the segments included in the 2D stream and
the 3D extension stream have the same page ID (page_id).
Thus, in the reception device with the 3D function at the
25 reception side, the segment of the 2D stream and the
segment of the 3D extension stream are easily connected
with each other based on the page ID.
[0062]
The multiplexer 119 includes synchronous
30 information, which is used to synchronize a display based
on data of overlapping information at the reception side
31
SP338522WO00
with shift control based on disparity information
(Disparity), in the 2D stream and the 3D extension stream.
Specifically, the multiplexer 119 is configured to
associate a value of a time stamp (PTS: presentation time
5 stamp) included in the PES header of the 2D stream (PESl
(1):PES#1) with a value of a presentation time stamp
(PTS) included in the PES header of the 3D extension
stream (PES2 (2):PES#2) as illustrated in Fig. 12.
[0063]
10 Fig. 12 illustrates an example in which the values
of the time stamps (PTS) of the 2D stream and the 3D
extension stream are set to the same value, that is, a
PTSl. In this case, at the reception side (the decoding
side), a display of a subtitle pattern based on subtitle
15 data (data of overlapping information) starts from the
PTSl, and shift control based on disparity information
for displaying a subtitle pattern in a 3D manner also
starts from the PTSl.
[0064]
20 The example of Fig. 12 illustrates that two pieces
of disparity information, that is, disparity information
of a PTSl frame and disparity information of a subsequent
predetermined, frame are included in the 3D extension
stream. The reception side (the decoding side) can
25 obtain disparity information of an arbitrary frame
between the two frames by an interpolating process,and
dynamically perform shift control.
[0065]
In Fig. 12, "conventional segments" included in the
30 2D stream mean segments of the DDS, the PCS, the RCS, the
CDS, the ODS, and the EDS. Further, "extended segments"
32
SP338522WO00
included in the 3D extension stream mean segments of the
DDS, the DSS, and the EDS or segments of the DDS, the PCS,
the DSS, and the EDS. In Fig. 12, "Elementary_PID" of
the 2D stream is IDl, and "Elementary_PID" of the 3D
5 extension stream is ID2. This applies similarly in Figs.
13 and 14.
[0066]
Fig. 13 illustrates an example in which the time
stamp (PTS) values of the 2D stream and the 3D extension
10 stream are set to different values from each other. In
other words. Fig. 12 illustrates an example in which the
time stamp (PTS) value of the 2D stream is set to a PTSl,
the time stamp (PTS) value of the 3D extension stream is
set to a PTS2 subsequent to the PTSl. In this case, at
15 the reception side (the decoding side), a display of a
subtitle pattern based on subtitle data (data of
overlapping information) starts from the PTSl, and shift
control based on disparity information for displaying a
subtitle pattern in a 3D manner also starts from the PTS2.
20 [0067]
The example of Fig. 13 illustrates that disparity
information of a PTS2 frame and disparity information of
a plurality of subsequent frames are included in the 3D
extension stream. The reception side (the decoding side)
25 can obtain disparity information of an arbitrary frame
between two of the plurality of frames by an
interpolating process and dynamically perform shift
control.
[00 68]
30 Fig. 14 Illustrates an example in which the time
stamp (PTS) values of the 2D stream and the 3D extension
•• 33
SP338522WO00
stream are set to different values from each other,
similarly to Fig. 13, and a plurality of 3D, extension
streams having different time stamp (PTS) values are
present. In other words, in the example of Fig. 14, the
5 time stamp (PTS) value of the 2D stream is set to a PTSl.
The time stamp (PTS) values of a plurality of 3D
extension frames are set to a PTS2, a PTS3, a PTS4, and
the like which are subsequent to the PTSl.
[0069]
10 In this case, at the reception side (the decoding
side), a display of a subtitle pattern based on subtitle
data (data of overlapping information) starts from the
PTSl. Further, shift control based on disparity
information for displaying a subtitle pattern in a 3D
15 manner also starts from the PTS2, and then an update is
sequentially performed. The example of Fig. 14
illustrates that disparity information of a frame
represented by each time stamp is included in each of a
plurality of 3D extension streams, and the reception side
20 (the decoding side) can obtain disparity information of
an arbitrary frame between two of the plurality of frames
by an interpolating process and dynamically perform shift ^
control.
[0070]
25 An operation of the transmission data generating
unit 110 illustrated in Fig. 2 will be briefly described.
The stereoscopic image data extracted from the data
extracting unit 111 is supplied to the video encoder 112.
The video,encoder 112 encodes the stereoscopic image data
30 using MPEG4-AVC, MPEG2, VC-1, or the li]<;e, and generates
a video data stream including encoded video data. The
34
SP338522WO00
video data stream is supplied to the multiplexer 119.
[0071]
The audio data extracted from the data extracting
unit 111 is supplied to the audio encoder 113. The audio
5 encoder 113 encodes the audio data using MPEG-2 Audio AAC,
MPEG-4'AAC, or the like, and generates an audio data
stream including encoded audio data. The audio data
stream is supplied to the multiplexer 119.
[0072]
10 The subtitle generating unit 114 generates subtitle
data (for a 2D image) which is subtitle data of DVB. The
subtitle data is supplied to the disparity information
creating unit 115 and the subtitle processing unit 116.
[0073]
15 The disparity vector of each pixel extracted from
the data extracting unit 111 is supplied to the disparity
information creating unit 115. The disparity informationcreating
unit 115 executes the downsizing process on the
disparity vector of each pixel or a plurality of pixels,
20 and generates disparity information (disparity) of each
layer. The disparity information is supplied to the
subtitle processing unit 116.
[0074]
For example, the subtitle processing unit 116
25 defines a sub region in a region based on the subtitle
data generated by the subtitle generating unit 114.
Further, the subtitle processing unit 116 sets disparity
information for shift-adjusting the display position of
overlapping information in the left-eye image and the
30 right-eye image data based on the disparity information
•generated by the disparity information creating unit 115.
35
SP338522WO00
In this case, the disparity information is set in units
of sub regions, in units of regions, or in units of pages.
[0075]
The subtitle data and the display control
5 information output from the subtitle processing unit 116
are supplied to the subtitle encoder 118. The display
control information includes region information of a sub
region, disparity information, and the like. The
subtitle encoder 118 generates the first and second
10 private data streams (elementary stream).
[0076]
In other words, the first private data stream (the
2D stream) including the segments of the DDS, the PCS,
the RCS, the CDS, the CDS, and the EDS is generated. In
15 addition, the second private data stream (the 3D
extension stream) including the segments of the DDS, the
DSS, and the EDS is generated. As described above, the
segment of the DSS is a segment including the display
control information.
20 [0077]
The data streams from the video encoder 112, the
audio encoder 113, and the subtitle encoder 118 are
applied to the multiplexer 119 as described above. The
multiplexer 119 obtains a transport stream TS as a
25 multiplexed data stream by converting each data stream
into a PES packet and multiplexing the PES packet. The
transport stream TS includes the first private data
stream (the 2D stream) and the second private data stream
(the 3D extension stream) as the PES stream as well as
30 the video data stream and the audio data stream.
[0078] '
36
SP338522WO00
Fig. 15 illustrates a configuration example of the
transport stream TS. In Fig. 15, for the sake of
simplification of the drawing, video- and audio-related
portions are not illustrated. The transport stream TS
5 includes the PES packet obtained by packetizing each
elementary stream.
[0079]
In this configuration example, a PES packet
"Subtitle PESl" of the 2D stream (the first private data
10 stream) and a PES packet "Subtitle PES2" of the 3D
extension stream (the second private data stream) are
included. The 2D stream (PES stream) includes the
segments of the DDS, the PCS, the RCS, the CDS, the ODS,
and the EDS (see Fig. 11(a)). The 3D extension stream
15 (PES stream) includes the segments of the DDS, the DSS,
and the EDS or the segments of the DDS, the PCS, the DSS,
and the EDS (Figs. 11(b) and 11(c)). In this case,
"Elementary_PID" of the 2D stream and "Elementary_PID" of
the 3D extension stream are set to different IDs such as
20 PIDl and PID2, and the streams are different PES streams.
[0080]
Fig. 16 illustrates the structure of the PCS
(page_composition_segment) . The segment type of the PCS
is "0x10" as illustrated in Fig. 17.
25 "region_horizontal_address" and "region_vertical_addre&s"
represent the start position of a region. The structures
of the other segments such as the DDS, the RCS, and the
ODS are not illustrated in the drawing. As illustrated
in Fig. 17, the segment type of the DDS is "0x14," the
30 segment type of the RCS is "0x11," the segment type of
the CDS is "0x12," the segment type of the ODS is "0x13,"
37 . •
SP338522WO00
and the segment type of the EDS is "0x8 0." Further, the
segment type of the DSS is "0x15" as illustrated in Fig.
17. The detailed structure of the DSS segment will be
described later.
5 [0081]
Referring back to Fig. 15,'the transport stream TS
includes a PMT (program map table) as PSI (program
specific information). The PSI is information
representing a program to which each elementary stream
10 included in the transport steam belongs. The transport
stream further includes an EIT (event information table)
as SI (serviced information) to perform management of an
event unit. Metadata of a program unit is described in
the EIT.
15 [0082]
A subtitle descriptor (subtitling_descriptor)
representing content of a subtitle is included in the PMT.
Further, a component descriptor (component_desc!;riptor)
representing delivery content is included in the EIT for
20 each stream. As illustrated in Fig. 18, when
"stream_content" of the component descriptor represents a
subtitle, "component_type" of "0x15" or "0x25" represents
a 3D subtitle, and the other values represent a 2D
subtitle. As illustrated in Fig. 15, "subtitling_type"
25 of the subtitle descriptor is set to the same value as
"component_type."
[0083]
A subtitle elementary loop (subtitle ES loop)
having information associated with a subtitle elementary
30 stream is present in the PMT. In the subtitle elementary
loop, not only information such as a paclcet identifier
38
SP338522WO00
(PID) but also a descriptor describing information
associated with a corresponding elementary stream are .
arranged for each stream.
[0084]
5 In Fig. 19, a subtitle descriptor
(subtitling_descriptor) and a component descriptor
(component_descriptor) illustrated in Fig. 15 are
extracted ^nd illustrated. In Fig. 20, the PES stream
(the 2D stream and the 3D extension stream) illustrated
10 in Fig. 15 is extracted and illustrated.
[0085]
In order to specify association between the 2D
stream and the 3D extension stream, a composition page
ID"composition_page_id" of a subtitle descriptor
15 corresponding to each stream is set as follows. In other
words, "composition_page_id" of the 2D stream and
"composition_page_id" of the 3D extension stream are set
to share the same value ("OxXXXX" in Fig. 19). Here,
"composition_page_id" configures association information.
2 0 Further, both PES streams are encoded such that "page_id"
of each associated segment has the same value (OxXXXX) so
that each segment included in the 3D extension stream is
associated with each segment of the 2D stream.
[0086]
25 Further, for example, an ISO language code
(ISO_639_language_code) is described in the subtitle
descriptor and the component descriptor as linguistic
information. The ISO language code of a descriptor
corresponding to the 2D stream is set to represent a
30 language of a subtitle. In the illustrated example,
"eng" representing English is set. The 3D extension
39
SP338522WO00
stream includes the DSS segment with disparityinformation,
but does not include the ODS segment and
thus does not depend on a language. For example, the ISO
language code described in the descriptor corresponding
5 to the 3D extension stream is set "zxx" representing a
non-language.
[0087]
Further, any one of language codes included in a
space of "qaa" to "qrz" which are the ISO language codes
10 may be used as an ISO language code representing a nonlanguage.
Fig. 21 illustrates a subtitle descriptor
(subtitling_descriptor) and a component descriptor
(component_descriptor) in this case. Fig. 22 illustrates
an excerpt of an ISO language code (ISO 639-2 Code) list
15 for reference.
. [0088]
Fig. 23 illustrates a stream configuration example
of a subtitle data stream (a 2D stream and a 3D extension
stream) . This example is a single language service
20 example of English "eng." The 3D extension stream is
extracted in with "composition_page_id = OxXXXX" which is
in common with the 2D stream, and designated by
"subtitling_type = ,3D" and "ISO_639_language_code = zxx."
The 2D stream is designated by "subtitling_type = 2D" and
25 "ISO_639_language_code = eng." Here, "subtitling_type =
3D" means a case in which a value of "subtitling_type" is
"0x15" or "0x25." Further, "subtitling_type = 2D" means
a case in which a value of "subtitling_type" is neither
"0x15" nor "0x25." This applies similarly in the
30 following.
[0089]
40
SP338522WO00
Fig. 24 illustrates an example of syntax of a
component descriptor (component_descriptor). An 8-bit
field of "descriptor_tag" represents that a descriptor is
a component descriptor. An 8-bit field of
5 "descriptor_length" represents an entire byte size
subsequent to this field.
[0090]
A 4-bit field of "stream_content" represents the
stream type of a main stream such as a video, an audio,
10 or a subtitle. A 4-bit field of "component_type"
represents the component type of a main stream such as a
video, an audio, or a subtitle.
[0091]
When a main stream is a 2D stream, that is, in a
15 component descriptor corresponding to a 2D stream,
"stream_content" is "subtitle," and "component_type" is
"2D" for two dimension. Further, when a main stream is,a
3D extension stream, that is, in a component descriptor
corresponding to a 3D extension stream, "stream_content"
20 is "subtitle," and "component_type" is "3D" for three
dimension.
[0092]
An 8-bit field of "component_tag" has the same
value as "component_tag" in a stream identifier
25 descriptor (stream_identifier descriptor) corresponding
to a main stream. Thus, the stream identifier descriptor
is associated with the component descriptor by
"component_tag." A 2 4-bit field of
"ISO_639_language_code" represents the ISO language code.
30 ,[0093]
Fig. 25 illustrates an example of syntax of the
41
SP338522WO00
subtitle descriptor (subtitling_descriptor). An 8-bit
field of "descriptor_tag" represents that this descriptor
is the subtitle descriptor. An 8-bit field of
"descriptor_length" represents the entire byte size
5 subsequent to this field.
[0094]
A 24-bit field of "ISO_639_language_code"
represents the ISO language code. An 8-bit field of
"subtitling_type" represents subtitle type information.
10 When a main stream is a 2D stream, that is, in a^subtitle
descriptor corresponding to a 2D stream,
"subtitling_type" is "2D." However, when a main stream
is a 3D extension stream, that is, in a subtitle
descriptor corresponding to a 3D extension stream,
15 "subtitling_type" is "3D." A 16 bit field of
"composition_page_id" represents the composition page ID,
and has the same value as a page ID "page_id" of a
segment included in a main stream.
[0095]
20 [Update of Disparity Information]
As described above, disparity information is
transmitted through a 3D extension stream. An update of
the disparity information will be described.
[0096]
25 Figs. 26 and 27 illustrate an example of updating
the disparity information using an interval period. Fig.
2 6 illustrates an example in which the interval period is
fixed, and the period is equal to an update period. In
other words, each of update periods A-B, B-C, and C-D is
30 composed of a one interval period.
[0097]
42
SP338522WO00
Fig. 27 illustrates an example of updating the
disparity information when a short period (which may be a
frame period, for example) is used as the interval period,
which is general). In this case, M, N, P, Q, and R in
& the update periods are the number of interval periods.
Further, in Figs. 26 and 27, "A" represents a starting
frame (a starting point of time) of a subtitle display
period, and "B" to "F" represent subsequent update frames
(update points of time).
10 [0098]
When the disparity information sequentially updated
within the subtitle display period is transmitted to the
reception side (the set-top box 200 or the like), the
reception side can generate and use disparity information
15 of an arbitrary frame interval, for example, a one frame
interval by executing the interpolating process on the
disparity information of an update period interval.
[0099]
Fig. 28 illustrates a configuration example of the
20 3D extension stream. This configuration example
illustrates a case in which the segments of the DDS, the
DSS, and the EDS are included in the PES data payload,
but it applies similarly even when the segments of the
DDS, the PCS, the DSS, and the EDS are included in the
25 PES data payload.
[0100]
Fig. 28(a) illustrates an example in which only one
DSS segment is included. The PES header includes the
time information (PTS). Further, the segments of the DDS,
30 the DSS, and the EDS are included as PES payload data. . ,
The segments are collectively transmitted before the
43
SP338522WO00 •
subtitle display period starts. A plurality of pieces of
disparity information to be sequentially updated in the
subtitle display period are included in one DSS segment.
[0101]
5 Alternatively, the plurality of pieces of disparity
information can be transmitted to the reception side (the
set-top box 200 or the like) without including, the
plurality of pieces of disparity information to be
sequentially updated in the subtitle display period in
10 one DSS. In this case, the DSS segment is included in
the 3D extension stream at each timing at which update is
performed. Fig. 28(b) illustrates a configuration
example of the 3D extension stream in this c-ase.
[0102]
15 Fig. 2 9 illustrates an update example of the
disparity information when the DSS segment is
sequentially transmitted as illustrated-in Fig. 28(b).
In Fig. 29, "A" represents the starting frame (the
starting point of time) of the subtitle display period,
20 and "B" to "F" represent sequent update frames (update
points of time).
[0103]
Even when the DSS segment is sequentially
transmitted and the disparity information subsequently
25 updated in the subtitle display period is transmitted to
the reception side (the set-top box 200 or the like), the
same process as described above can be performed at the
reception side. In other words, even in this case, the
reception side can generate and use disparity information
30 of an arbitrary frame interval, for example, a one frame
intervals by executing the interpolating process on the
44
SP338522WO00
disparity information of an update period interval.
[0104]
Fig. 30 illustrates an update example of the
disparity information (disparity), similarly to Fig. 27. ;
5 The update frame interval is represented by a multiple of
the interval period (ID: interval duration) serving as
the unit time. For example, an update frame interval
"Division Period 1" is represented by "ID*M," an update
frame interval "Division Period 2" is represented by
10 "ID*N," and the following update frame intervals apply
similarly. In the update example of the disparity
information illustrated in Fig. 30, the update frame
interval is not fixed, and the update frame interval is
set to corresponding to a disparity information curve.
15 [0105]
Further, in the update example of the disparity
information (disparity), at the reception side, the
starting frame (the starting time) Tl_0 of the subtitle
display period is given by the PTS (presentation time
20 stamp) included in the header of the PES stream including
the disparity information. Further, at the reception
side, each update time of the disparity information is
obtained based on information of the interval period
(information of the unit time) which is information each
25 update frame interval and information the number of
interval periods.
[0106]
In this case, each update time is sequentially
obtained from the starting frame (the starting time) T1_0
30 of the subtitle, display period by the following Formula
(1). In this Formula (1), "interval_count" represents
45
SP338522WO00
the number of interval periods, and is a value
corresponding to M, N, P, Q, R, and S in Fig. 30. In
this Formula (1), "interval_time" is a value
corresponding to the interval-period (ID) in Fig. 30.
5 [0107]
Tm_n = Tm_ (n-1) + (interval_time * interval_count)
— (1)
[0108]
For example, in the update example illustrated in
10 Fig. 30, each update time is obtained by.Formula (1) as
follows. In other words, an update time Tl_l is obtained
using the starting time (T1_0), the interval period (ID),
and a number (M) : "Tl_l = T1_0 + (ID * M) ." Further, an
update time Tl_2 is obtained using the update time (Tl_l),
15 the interval period (ID), and a number (N): "Tl_2 = Tl_l
+ (ID * N)."
The subsequent update times are obtained in the similar
manner.
[0109]
20 In the update example illustrated in Fig. 30, the
reception side executes the interpolating process on the
disparity information to be sequentially updated within
the subtitle display period, and generates and usesdisparity
information of an arbitrary frame interval, for
25 example, a one frame interval within the subtitle display
period. For example, the interpolating process including
a low pass filter (LPF) filter other than a linear
interpolating process is performed in a time direction',,(a
frame direction) as the interpolating process, and thus a
30 change of the disparity information of a predetermined
frame interval which has been subjected to the
46
SP338522WO00
interpolating process in the time direction (the frame
direction) becomes gentle. In Fig. 30, a dashed line a
represents an LPF output example.
[0110]
5 Fig. 31 illustrates a display example of a subtitle
serving as a caption. In this display example, two
regions (a region 1 and a region 2) serving as a subtitle
display region are included, in a page region (Area for
Page_default). The region includes one or more sub
10 regions. Here, one sub region is included in the region,
and thus the region is assumed to be equal to the sub
region.
[0111]
Fig. 32 illustrates an example of a disparity
15 information curve of each region and a page when both
disparity information of a region unit and disparity
information of a page unit are included in the DSS
segment as the disparity information (Disparity) to be
sequentially updated in the subtitle display period.
20 Here, disparity information curve of a page is formed
such that a minimum value of disparity information curves
of two regions is employed.
[0112]
A starting time T1_0 and seven pieces of disparity
25 information, that is, subsequent update times Tl_l to
Tl_6 ar& present in connection with a region 1 (regionl).
A starting time T2_0 and eight pieces of disparity
information, that is, subsequent update times T2_l to
T2_7 are present in connection with a region 2 (region2).
30 Further, a starting time T0_0 and seven pieces of
disparity information, that is, subsequent update times
47
SP338522WO00
T0_1 to T0_6 are present in connection with a page
(page_default).
[0113]
Fig. 33 illustrates a structure in which disparity
5 information of a page and each region illustrated in Fig.
32 is transmitted. First, a page layer will be described.
"page_default_disparity" which is a fixed value of
disparity information is arranged in the page layer.
Further, "interval_count" representing the number of
10 interval periods and "disparity_page_updete" representing
disparity information which correspond to the starting
time and the subsequent update time are sequentially
arranged in connection with the disparity information to
be sequentially updated in the subtitle display period.
15 Further, "interval_count" of the starting time is assumed
• to be zero (0).
[0114]
Next, a region layer will be described.
"subregion_disparity_integer_part" and
20 "subregion_disparity_fractional_part" which are fixed
values of disparity information are arranged on a region
1 (sub regionl). Here,
"subregion_disparity_integer_part" represents an integer
part of disparity information, and
25 "subregion_disparity_fractional_part" represents a
fractional part of disparity information.
[0115]
Further, "interval_count" representing the number
of interval periods,
30 "disparity_region_updete_integer_part" representing
disparity information, and
48
SP338522WO00
"disparity_region_updete_fractional_part", which
correspond to the starting time and the subsequent update
times, are sequentially arranged in connection with the
disparity information to be sequentially updated in the
5 subtitle display period. Here,
"disparity_region_updete_integer_part" represents an
integer part of disparity information, and
"disparity_region_updete_fractional_part" represents a
fractional part of disparity information. Further,
10 "interval_count" of the starting time is assumed to be
zero (0) .
[0116]
"subregion_disparity_integer_part" and
"subregion_disparity_fractional_part" which are fixed
15 values of display information are arranged on a region (a
sub region 2), similarly to the region 1. Further,
"interval_count" representing the number of interval
periods, "disparity_region_updete_integer_part"
representing disparity information, and
2 0 "disparity_region_updete_fractional_part", which
correspond to the starting time and the subsequent update
times, are sequentially arranged in connection with the
disparity information to be sequentially updated in the
subtitle display period.
25 [0117]
Figs. 34 to 36 illustrate an example of syntax of
the DSS (disparity_signaling_segment). Figs. 37 to 40
illustrate main data specifying content (semantics) of
the DSS. This syntax includes information of
30 "sync_byte," "sfegment_type," "page_id,"
"segment^length," and "dss_version_number."
49
SP338522WO00
"segment_type" is 8-bit data representing, a segment type,
and is used as a value representing the DSS.
"segment_length" is 8-bit data representing a subsequent -' (
byte number.
5 [0118]
A 1-bit flag of
"disparity_shift_update_sequence_page_flag" represents
whether or not there is disparity information to be
sequentially updated in the subtitle display period as
10 disparity information of a page unit. "1" represents
that there is disparity information to be sequentially
updated in the subtitle display period as disparity
information of a page unit, whereas "0" represents that
there is no disparity information to be sequentially
15 updated in the subtitle display period as disparity
information of a page unit. An 8-bit field of
"page_default_disparity_shift" represents fixed disparity
information of a page unit, that is, disparity
information commonly used within the subtitle display
20 period. When the flag of
"disparity_page_update_sequence_flag" is "1,"
"disparity_shift_update_sequence ()" is read out.
[0119]
Fig. 36 illustrates an example of syntax of
•25 "disparity_shift_update_sequence ()."
"disparity_page_update_sequence_length" is 8-bit data
representing a subsequent byte number. A 24-bit field of
"interval_duration [23.. 0]--" designates an interval period
(interval duration) (see Fig. 30) serving as a unit time
30 in units of 90 KHz. In other words, in "interval_
duration [23..0]," a value, which is obtained by
50
SP338522,WO00
measuring the interval period (interval duration) at a
clock of 90KHz, is represented by a 24-bit length.
[0120] , •
The reason why the PTS included in the header
5 portion of the PES has the 33-bit length but the interval
period has the 24-bit length is as follows. In other
words, the 33-bit length can represent a time exceeding
2 4 hours but is a length unnecessary as the interval
period (interval duration) within the subtitle display
10 period. Further, using 24 bits, the data size can be
reduced, and compact transmission can be performed.
Further, 24 bits are 8x3 bits, and byte alignment is
easily performed.
[0121]
15 An 8-bit field of "division_period_count" represent
the number of periods (division periods) affected by
disparity information. For example, in the case of the
update example illustrated in Fig. 30, the number is "7,"
corresponding to the starting time T1_0 and the
20 subsequent update times Tl_l to Tl_6. The following for
loop is repeated as many times as the number represented
by an 8-bit field of "division_period_count."
[0122]
An 8-bit field of "interval_count" represents the
25 number of interval periods. For example, in the case of
the update example illustrated in Fig. 30,
"interval_count" corresponds to M, N, P, Q, R, and S. An
8-bit field of "disparity_shift_update_integer_part"
represents disparity information. "interval_count" is
30 assumed to be zero. (0), correspond to disparity
information of the starting time (an initial value of
51
SP338522WO0g
disparity information). In other words, when
"interval_count" is "0," "disparity_page_update"
represents the disparity information of the starting time
(the initial value of the disparity information).
5 [0123]
The while loop of Fig. 34 is repeated when a data
length (processed_length) processed until then does not
reach a segment data length (segment_length). In'the
while loop, disparity information of a region unit or a
10 sub region unit in a region is arranged. Here, one or
more sub regions are included in the region, and the sub
region may be the same as the region.
[0124]
In the while loop, information of "region_id" is
15 included. A 1-bit flag of
"disparity_shift_update_sequence_region_flag" is flag
information representing whether or not there is
"disparity_shift_update_sequence ()" for all sub regions
in the region. A 2-bit field of
2 0 "number_of_subregions_minus_l" represents a value
obtained by subtracting one (1) from the number of sub
regions in the region. In the case of
"number_of_subregions_minus_l = 0," a region includes one
sub region having the same size as a region.
25 [0125]
In the case of "number_of_subregions_minus_l>0," a
region includes a plurality of sub regions divided in the
horizontal direction. In the for loop of Fig. 35, pieces
of information of "subregion_horizontal_position" and
30 "subregion_width" which correspond to the number of sub
regions are included. A 16 bit field of
52
SP338522WO00
"subregion_horizontal_position" represents a pixel
position of the left end of the sub region.
"subregion_width" represents the width of the sub region
in the, horizontal direction,through the number of pixels.
5 [0126]
An 8-bit field of • '
"subregion_disparity_shift_integer_part" represents an
integer part of fixed disparity information of a region
unit (a sub region unit), that is, an integer part of
10 disparity information commonly used in the subtitle
display period. A 4-bit field of
"subregion_disparity_shift_fractional_part" represents a
fractional part of fixed disparity information of a
region unit (a sub region unit), that is, a fractional
15 part of disparity information commonly used in the
subtitle display period. When the flag of
"disparity_shift_update_sequence_region_flag" is "1,"
"disparity_shift_update_sequence ()" (see Fig. 36) is
read out.
20 [0127]
[Broadcast Reception Concept]
Fig. 41 illustrates a broadcast reception concept
when the set-top box 200 and the television receiver 300
are devices with the 3D function. In this case, the
25 broadcasting station 100 defines a sub region "SR 00" in
a region "Region 0," and sets disparity information
"Disparity 1." .Here, the region "Region 0" and the 'sub
region '^SR GO" are assumed to be the same region.
Subtitle data and display control information (region
30 information "Position" of the sub region and the
disparity information "Disparity 1") are transmitted from
53 ;
SP338522WO00
the broadcasting station 100 together with stereoscopic
image data.
[0128]
First, an example in which reception is performed
5 by the set-top box 200 which is a device with the 3D
function will be described. In this case, the set-top
box 20.0 reads data of each segment configuring the
subtitle data from the 2D stream, and reads and uses data
of the DSS segment including the display control
10 information such as the disparity information from the 3D
extension stream.
[0129]
In this case, the set-top box 200 extracts the 2D
stream and the 3D extension stream, which are associated
15 with each other, from the transport stream TS, and
decodes the 2D stream and the 3D extension stream. At
this time, the set-top box 200 efficiently and
appropriately extract two associated streams based on a
common composition page ID described in the subtitle
20 ^ descriptor (see Fig. 15) corresponding to each stream.
[0130]
The set-top box 200 generates display data of a
region for displaying a subtitle based on the subtitle
data. Further, the set-top box 200 causes the display
25 data of the region to, overlap a left-eye image frame
(frameO) portion and a right-eye image data frame
(framel) portion which configure the stereoscopic image
data, and obtains output stereoscopic image data.
[0131]
30 At this time, the set-top box 200 shift-adjusts the
position of display, data overlapping each portion based
54
SP338522WO00
on the disparity information. Further, the set-top box
2 00 appropriately changes the overlapping position, the
size, and the like according to a transmission format (a
side by side format, a top-and-bottom format, a frame
5 sequential format, and a format in which each view has a
full screen size) of the stereoscopic image data.
[0132]
The set-top box 200 transmits the output
stereoscopic image data obtained as described above to
10 the television receiver 300 with the 3D function, for
example, through the HDMI digital interface. The
television receiver 300 executes 3D signal processing on
the stereoscopic image data transmitted from the set-top
box 200, and generates data of a left-eye image and a
15 right-eye image which the subtitle overlaps. Further,
the television receiver 300 causes a binocular disparity
image (the left-eye image and the right-eye image)
through which the user recognizes the stereoscopic image
to be displayed on a display panel such as a liquid
20 crystal display (LCD).
[0133]
Next, an example in which reception is performed by
the television receiver 300 which is a device with a 3D
function will be described. In this case, the television
25 receiver 300 reads data of each segment configuring the
subtitle data from the 2D stream, and reads and uses data
of the DSS segment including the display control
information such as the disparity information from the 3D
extension stream. The television receiver 300
30 efficiently apd appropriately extract the 2D stream and
the 3D extension stream, which are associated with each
55
SP338522WO00
other,^ from the transport stream TS based on a common
composition page ID, similarly to the set-top box 200.
[0134]
The television receiver 300 generates display data
5 of a region for displaying a subtitle based on the
subtitle data. Further, the television receiver 300
causes the display data of the region to overlap the
left-eye image data and the right-eye image data which
are obtained by executing processing corresponding to the
10 transmission format on the stereoscopic image data, and
generates data of the left-eye image and the right-eye
image which the subtitle overlap. Further, the
television receiver 300 causes a binocular disparity
image (the left-eye image and the right-eye image)
15 through which the user recognizes the stereoscopic image
to be displayed on a display panel such as an LCD.
[0135]
Fig. 42 illustrates a broadcast reception concept
when the set-top box 200 and the television receiver 300
20 are when the set-top box 200 and the television receiver
300 are devices with a legacy 2D function. In this case,
similarly to the example of Fig. 41, subtitle data and
display control information (region information
"Position" of the sub region and the disparity
25 information "Disparity 1") are transmitted from the
broadcasting station 100 together with stereoscopic image
data.
[0136]
First, an example in which reception is performed
30 by the set-top box 200 which is a device with the legacy
2D function will be described. In this case, the set-top
5 6
SP338522WO00
box 200 reads and uses data of each segment configuring
the subtitle data from the 2D stream. In this case, the
set-top box 200 extracts the 2D stream from the transport
stream TS based on the subtitle type information, the
5 linguistic information, and the like which are described
in subtitle descriptor (see Fig. 15) corresponding to
each segment, and decodes the 2D stream.
Thus, the set-top box 200 does not read the DSS segment
including the display control information such as the
10 disparity information, and thus can avoid reading
interfering with the reception process.
[0137]
As described above, the subtitle descriptors which
correspond to the 2D stream and the 3D extension stream,
15 respectively, are included in the transport stream TS
(see Fig. 15) . Further, the subtitle type information
"subtitling_type" of the subtitle descriptor
corresponding to the 2D stream is set to "2D." Further,
the subtitle type information "subtitling_type" of the
20 subtitle descriptor corresponding to the 3D stream is set
to "3D."
[0138]
In addition, as described above, the component
descriptor and the subtitle descriptor corresponding to
25 each of the 2D stream and the 3D extension stream are
included in the transport stream TS (see Fig. 15).
Further, the linguistic information (the ISO language
code) of the descriptor corresponding to the 2D stream is '
set to represent a language, and the linguistic
30 information (the ISO language code) of the descriptor
corresponding to the 3D extension stream is set to
57
SP338522WO00
represent a non-language.
[0139]
Since the set-top box 200 is the device with the 2D
function, the 2D stream corresponding to,the subtitle
5 type"2D (HD,SD)" is decided as a stream to be extracted
based on the subtitle type information. Further, the
set-top box 200 decides a 2D stream of a language which
is" selected by the user or automatically selected by the
device as a stream to be extracted.
10 [0140]
The set-top box 200 generates display data of a
region for displaying a subtitle based on the subtitle
data. Further, the set-top box 200 causes the display
data of the region to overlap the 2D image data obtained
15 by executing processing corresponding to the transmission
format on the stereoscopic image data, and obtains output
2D image data. .
[0141]
The set-top box 200 outputs the output 2D image
20 data obtained as described above to the television
receiver 300, for example, through the HDMI digital
interface. The television receiver 300 displays 2D image
based on the 2D image data transmitted from the set-top
box 200.
25 [0142]
Next, an example in which reception is performed by
the television receiver 300 which is the device with the
2D function will be described. In this case, the
television receiver 300 reads and uses data o'f, each
30 segment configuring the subtitle data from the 2D stream.
In this case, the television receiver 300 extracts a 2D
58
SP338522WO00
stream of a language selected by the user from the
transport stream TS based on the subtitle type
information and the linguistic information, and decodes
the 2D stream, similarly to the set-top box 200. In
5 other words, the television receiver 300 does not read
the DSS segment including the display control information
such as the disparity information, and thus can avoid
reading interfering with the reception process.
[0143] •
10 The television receiver 300 generates display data
of a region for displaying a subtitle based on the
subtitle data. The television receiver 300 causes the
display data of the region to overlap the 2D image data
obtained by executing processing corresponding to the
15 transmission format on the stereoscopic image data, and
obtains output 2D image data. Then, the television
receiver 300 displays a 2D image based on the 2D image
data.
[0144]
20 Fig. 43 illustrates a broadcast reception concept
when the above-described receivers' (the set-top box 200
and the television receiver 300) are the device with the
legacy 2D function (2D receiver) and the device with the
3D function (3D receiver). In Fig. 43, the transmission
25 format of the stereoscopic image data (3D image data) is
assumed as the side by side format.
[0145]
Further, in the device with the 3D function (3D
Receiver), either a 3D mode or a 2D mode can be selected.
30 When the user selects the 3D mode, the concept described
with reference to Fig. 41 applies. When the user selects
59 '
SP338522WO00
the 2D mode, the device with the 2D function (2D
receiver) described with reference to Fig. 42 similarly
applies.
[0146]
5 In the transmission data generating unit 110
illustrated in Fig. 2, the common composition page ID is
described in the subtitle descriptor included
corresponding to each of the 2D stream and the 3D
extension stream, and association with the two streams is
10 clarified. Thus> the reception device with the 3D
function at the reception side can properly extract and
decode the 2D stream and the 3D extension stream, which
are associated with each other, based on the association
information and obtain the disparity information together
15 with the subtitle data. '
[0147]
Further, in the transmission data generating unit
110 illustrated in Fig. 2, the subtitle type information
and the linguistic information are set to the component
2 0 descriptor and the subtitle descriptor which are included
corresponding to each of the 2D stream and the 3D
extension stream such that the respective streams are
identified. Thus, the reception device with the 2D
function can easily extract and decode the 2D image based
25 on the subtitle type information and the linguistic
information with a high degree of accuracy. Thus, the
reception device with the 2D function can strongly
prevent the reading of the DSS segment with the disparity
information and avoid the processing interfering with the
30 reception process.
[0148]
60
SP338522WO00
Further, in the transmission data generating unit
110 illustrated in Fig. 2, the page IDs (page_id) of the
segments included in the 2D stream and.the 3D extension
stream included in the transport stream TS are the same.
5 Thus, the reception device with the 3D function at the
reception side can easily connect the segment of the 2D
stream with the segment of the 3D extension stream based
on the page ID.
[0149]
10 Further, in the transmission data generating unit
110 illustrated in Fig. 2, since the DSS Segment
including the disparity information sequentially updated
in the subtitle display period can be transmitted, the
display positions of the left-eye subtitle and the right-
15 eye subtitle can be dynamically controlled. As a result,
the reception side can dynamically change disparity
occurring between the left-eye subtitle and the right-eye
subtitle in conjunction with a change in image content.
[0150]
20 Further, in the transmission data generating unit
110 illustrated in Fig. 2, the disparity information of
the frame of each update frame interval included in the
DSS segment obtained by the subtitle encoder 118 is not
an offset value from previous disparity information but
25 disparity information itself. Thus, the reception side
can be recovered from an error within a predetermined
delay time even when an error occurs in the interpolation
process.
[0151]
30 The example in which a one language service of
English "eng" is provided has been described above (see
61
SP338522W,O00
Fig. 23) . Of course, however, the present technology can
be similarly applied to a multilingual service. Fig. 44
illustrates a stream configuration example of a subtitle
data- stream (a 2D stream and a 3D extension stream) when
5 a bilingual service is provided. For example. Fig. 4 4
illustrates a bilingual service example of English "eng"
and German "ger."
[0152]
Regarding the English service, the 3D extension
10 stream is extracted in by "composition_page_id = OxXXXX"
which is in common with the 2D stream, designated by
"subtitling_type = 3D" and "ISO_639_language_code = zxx,"
and the 2D stream is designated by "subtitling_type = 2D"
and "ISO_539_language_code = eng." Meanwhile, regarding
15 the German service, the 3D extension stream is extracted
in by "composition_page_id = OxYYYY" which is in common
with the 2D stream and designated by "subtitling_type =
3D" and "ISO_639_language_code = zxx," and the 2D stream
is designated by "subtitling_type = 2D" and
20 "ISO_639_language_code = ger."
[0153]
As described above, in the case of the multilingual
service, the composition page IDs (composition_page_id)
corresponding to the respective language services are set
25 to be different from each other. Thus, even in the
multilingual service, the 3D reception device at the
reception side can efficiently and appropriately extract
and decode the 2D stream and the 3D extension stream
which are associated with each other for each language
30 service.
[0154]
62 ,
SP338522WO00
Fig. 45 illustrates a configuration example of the
transport stream TS. In Fig. 45, for the sake of
simplification of the drawing, video- and audio-related
portions not illustrated. The transport stream TS
5 includes a PES packet obtained by packetizing each
elementary stream.
[0155]
In this configuration example, a PES packet
"Subtitle PESl" of the 2D stream and a PES packet
10 "Subtitle PES2" of the 3D extension stream are included
in connection with the English service. Further, a PES
packet "Subtitle PES3" of the 2D stream and a PES packet
"Subtitle PES4" of the 3D extension stream are included
in connection with the German service. The 2D stream
15 (PES stream) includes the segments of the DDS, the PCS,
the RCS, the CDS, the ODS, and the EDS (see Fig. 11(a)).
The 3D extension stream (PES stream) includes the
segments of the DDS, the DSS, and the EDS or the segments
of the DDS, the PCS, the DSS, and the EDS (see Figs.
20 11(b) and 11(c)). In this case, "Elementary_PID" of the
respective streams are set to be different from each
other, and the streams are PES streams different from
each other.
[0156]
25 A subtitle elementary loop (a subtitle ES loop)
having information associated with a subtitle elementary
stream is present in the PMT. In the subtitle elementary
loop/ not only information such as a packet identifier
(PID) but also a descriptor describing information
30 associated with a corresponding elementary stream are
arranged for each stream.
63
SP338522WO00
[0157]
In order to specify association between the 2D
stream and the 3D extension stream in connection with th^
English service, a composition page ID of a subtitle
5 descriptor corresponding to each stream is set as follows.
In other words, "composition_page_id" of the 2D stream
and "composition_page_id" of the 3D extension stream, are
set to share the same value ("OxXXXX" in Fig. 44).
Further, both PES streams are encoded such that "page_id"
10 of each associated segments has the same value (OxXXXX)
so that each segment included in the 3D extension stream
is associated with each segment of the 2D stream.
[0158]
Similarly, in order to specify association between
15 the 2D stream and the 3D extension stream in connection
with the German service, a composition page ID of a
subtitle descriptor corresponding to each stream is set.
In other' words, "composition_page_id" of the 2D stream
and "composition_page_id" of the 3D extension stream are
2 0 set to share the same value ("OxYYYY" in Fig. 44).
Further, both PES streams are encoded such that "page_id"
of each associated segments has the same value (OxYYYY)
so that each segment included in the 3D extension stream
is associated with each segment of the 2D stream.
25 [0159]
When the transport stream TS has the subtitle data
stream related to the multiple language as described
above, the reception device extracts and decodes the
subtitle data stream related to the language service,
30 which is selected by the user or automatically selected.
At this time, the reception device with the 3D function
64
SP338522WO00
efficiently and appropriately extract the 2D stream and
the 3D extension stream associated with the selected
language service based on the common composition page ID.
Meanwhile, the reception device with the legacy 2D
5 function extracts and decodes the 2D stream of the
selected language service based on the subtitling type
information and the linguistic information.
[0160]
Further, the stream configuration example
10 illustrated in Fig. 44 illustrates that the 3D extension
stream is used as a composition page (composition_page).
In this case, as described above, the 2D stream and 3D
stream are present on each language service. The 3D
extension stream may be used as an ancillary page. In
15 this case, a steam can be configured such that a one 3D
extension stream which is common to the respective
language services is presented as the 3D extension stream.
Thus, a band of the PES stream can be effectively used.
[0161]
20 Fig. 46 illustrates a stream configuration example
of, a subtitle data stream (a .2D stream and a 3D extension
stream) when the 3D extension stream is used as the
ancillary page. This example is a bilingual service
example of English "eng" and German "ger." In this
25 configuration example, the 3D extension stream is not
extracted in by "composition_page_id" during each
language service. In this configuration example, the 3D
extension stream is referred to commonly from the
respective language services by an ancillary page ID
30 "ancillary_page_id" which is common to the respective
language services.
65
SP338522W;O00
[0162]
Regarding the English service, the 2D stream is
extracted in by "composition_page_id = OxXXXX," and
designated by "subtitling_type = 2D" and
5 "ISO_63 9_language_code = eng." Further, the 3D extension
stream is.extracted in by "ancillary_page_id = OxZZZZ,"
and designated by "subtitling_type = 3D" and
"ISO_639_language_code = zxx." Similarly, regarding the
German service, the 2D stream is extracted in by
10 . "composition_page_id = OxYYYY," and designated by
"subtitling_type = 2D" and "ISO_639_language_cpde = ger."
Further, the 3D extension stream is extracted in by
"ancillary_page_id = OxZZZZ," and designated by
"subtitling_type = 3D" and "ISO_639_language_code = zxx."
15 [0163]
Fig. 47 illustrates a configuration example of the
transport stream TS. In Fig. 47, for the sake of
simplification of the drawing, video- and audio-related
portions are not illustrated. The transport stream TS
20 includes a PES packet obtained by packetizing each
elementary stream.
[0164]
In this configuration example, a PES packet
"Subtitle PESl" of the 2D stream is included in
25 connection with the English service. '
Further, a PES packet "Subtitle PES2" of the 2D stream is
included in connection with the German service.
In addition, a PES packet "Subtitle PES3" of the 3D
extension stream is included in connection with the
30 English service and the German service. The 2D stream
(PES stream) includes the segments of the DDS, the PCS,
66
SP338522WO00
the RCS, the CDS, the ODS, and the EDS (see Fig^ 11(a)).
The 3D extension stream (PES stream) includes the
segments of the DDS, the DSS, and the EDS (see Fig.
11(b)) . In this case, "Elementary_PID" of the respective
5 streams are set to be different from each other, and the
streams are PES streams different, from each other.
[0165]
A subtitle elementary loop (a subtitle ES loop)
having information associated with a- subtitle elementary
10 stream is present in the PMT. In the subtitle elementary
loop, not only information such as a packet identifier
(PID) but also a descriptor describing information
associated with a corresponding elementary stream are
arranged for each stream.
15 [0166]
In order to specify association between the 2D
stream related to the English service and the 3D
extension stream which is common to the respective
language services, a composition page ID of a subtitle
20 descriptor corresponding to each stream is set as follows.
In other words,. ''?composition_page_id" of the 2D stream
and "composition_page_id" of the 3D extension stream are
set to share the same value ("OxXXXX" in Fig. 46). At
this time, in the subtitle descriptor corresponding to
25 the 3D extension stream, "ancillary_page_id" is set to a
value ("OxZZZZ" in Fig. 46) different from
"composition_page_id," and the 3D extension stream is
used as the ancillary page.
[0167]
30 Similarly, in order to specify association between
the 2D stream related to the German service and the 3D
67
SP338522WO00
extension stream which is common to the respective
language services, a composition page ID of a subtitle
descriptor corresponding to each stream is set. In other
words, "composition_page_id" of the 2D stream and
5 "composition_page_id" of the 3D extension stream are set
to share the same value ("OxYYYY" in Fig. 4 6) . At this
time, in the subtitle descriptor corresponding to the '3D
extension stream, "ancillary_page_id" is set to a value
("OxZZZZ" in Fig. 46) different from
10 "composition_page_id," and the 3D extension stream is
used as the ancillary page.
[0168]
[Description of Set-Top Box]
Referring back to Fig. 1, the set-top box 200
15 receives the transport stream TS transmitted from the
broadcasting station 100 through the broadcast wave. The
transport stream TS includes the stereoscopic image data
including the left-eye image data and the right-eye image
data, and the audio data. The transport stream TS
•20 further includes the subtitle data (including the display
control information) for the stereoscopic image for
displaying the subtitle.
[0169]
In other words, the transport stream TS includes
25 the video data stream, the audio data stream, the first
and second private data streams (subtitle data stream) as
the PES stream. As described above, the first and second
private data streams are the 2D stream and the 3D
extension stream, respectively, (see Fig. 11) .
30 [0170]
The set-top box 200 includes a bit stream
68
SP338522WO00
processing unit 201. When the set-top box 200 is the
device with the 3D function (3D STB), the bit stream
processing unit 201 acquires stereoscopic image data,
audio data, and subtitle data (including the display
5 control information) from the transport stream TS.
Further, the bit stream processing unit 201 acquires data
of each segment configuring the subtitle data from the 2D
stream, and acquires data of the DSS segment including
the display control information such as the disparity
10 information from the 3D extension stream.
[0171]
Then, the bit stream processing unit 201 generates
output stereoscopic image data in which a subtitle
overlaps each of a left-eye image frame (frameO) portion
15 and a right-eye image data frame (framel) portion using
the stereoscopic image data and the subtitle data
(including the display control information) (see Fig. 41).
In this case, disparity can be brought to occur between
the subtitle (the left-eye subtitle) overlapping the
20 left-eye image and the subtitle (the right-eye subtitle)
overlapping the right-eye image.
[0172]
For example, as described above, the disparity
information is included in the display control
2 5 information added to the subtitle data for the
stereoscopic image transmitted from the broadcasting
station 100, and disparity can be brought to occur
between the left-eye subtitle and the right-eye subtitle
based on the disparity information. As described above,
30 by brining disparity to occur between the left-eye
subtitle and the right-eye subtitle, the user can
69
SP338522WO00
recognize the subtitle (subtitle) short of an image.
[0173]
When it is determined that the service is the 3D
service, the set-top box 200 extracts and decodes the 2D
5 stream and the 3D extension stream which are associated
with each other from the transport stream TS based on the
common composition page ID. Further, the set-top box 200
performs a process (overlapping process) of pasting a
subtitle in a background image as described above using
10 the subtitle data and the disparity information. Further, ,
when it is difficult to extract the 3D extension stream,
the bit stream processing unit 201 performs a process
(overlapping process) of pasting a subtitle in a
background image according to a logic of the receiver.
15 [0174]
For example, the set-top box 200 determines that
the service is the 3D service in case of the following
(1) to (3) .
[0175]
20 (1) A case where, in an SDT, "service_type" of a
service descriptor (service descriptor) is a 3D (OxlC,
OxlD, OxlE = frame compatible) (see Fig. 48(a)).
[0176]
{2) A case where, in an SDT or an EXT,
25 "stream_content" of a component descriptor is an MPEG4-
AVC video (0x05), and "component_type" is a 3D format of
(0x80 to 0x83) (see Fig. 48(b)).
[0177]
(3) A case where both (1) and (2) are satisfied.
30 [0178]
A flowchart of Fig. 49 illustrates an example of a
70
SP338522WO00
3D service determining process in the set-top box 200.
The set-top box 200 starts a determining process in step
STl, and then proceeds to step ST2. In step ST2, the
set-top box 200 determines whether or not "service_type"
5 of the service descriptor is 3D. When it is determined
that "service_type" of the service descriptor is 3D, in
step ST3, the set-top box '200 determines that the service
is the 3D service.
[0179]
10 When it is determined in step ST2 that
"service_type" is not a 3D, the set-top box 200 causes
the proceed to step ST4. In step ST4, the set-top box
2 00 determines whether or not "stream_content" of the
component descriptor is an MPEG4-AVC video (0x05) and
15 "component_type" represents a 3D format. When the 3D
format is represented, in step ST3, the set-top box 200 '.
determines that the service is the 3D service. However,
when the 3D format is not represented, in step ST5, the
set-top box 200 determines that the 2D service is
20 provided.
[0180]
A flowchart of Fig. 50 illustrates another example
of a 3D service determining process in the set-top box
200. In step STll, the set-top box 200 starts a >
25 determining process, and then proceeds to step ST12. In
step ST12, the set-top box 200 determines whether or not
"serviGe_type" of the service descriptor is a 3D. When
"service_type" of the service descriptor is a 3D, the
set-top box 200 causes the process to proceed to step
30 ST13.
[0181]
71
SP338522WO00
In step ST13, the set-top box 200 determines
whether or not "strearti_content" of the component
descriptor is an MPEG4-AVC video (0x05) and
"compohent_type" represents a 3D format. When the 3D
5 format is represented, in step ST14, the set-top box 200
determines that the service is the 3D service. However,
when it is determined in step ST12 that the 3D format is
not represented or when it is determined in step ST13
that the 3D format is not represented, in step ST15, the
10 set-top box 200 determines that the 2D service is
provided.
[0182]
A flowchart of Fig. 51 schematically illustrates
the flow of a process of the set-top box 200 when it is
15 determined that the service is the 3D service. In step
ST21, the set-top box 200 determines that the service is
the 3D service, and then causes the process to proceed to
step ST22. In step ST22, the set-top box 200 determines
whether or not "component_type" of a component descriptor
20 of a first stream whose "stream_type" represents "0x03
(subtitle)" is 3D.
[0183]
When "component_type" is "3D," in step ST23, the
set-top box 200 determines that a target PES stream is a
25 3D PES stream. Here, the 3D PES stream (3D stream) is a
PES stream including the DSS segment having the disparity
information and the like as well as the segments
configuring data of the overlapping information (subtitle
data) such as the DDS, the PCS, the RCS, the CDS, the ODS,
30 and the EDS as illustrated in Fig. 52.
[0184]
72
SP338522WO00
Next, in step ST24, the set-top box 200 determines
that the 3D extension segment (the DSS segment) is
present in the target PES stream. Thereafter, in step
ST25, the set-top box 200 decodes the 2D segment, then
5 decodes the 3D extension segment, and causes the subtitle
to,, overlap the background image (3D video) in terms of
the disparity information.
[0185]
When it is determined in step ST22 that
10 "component_type" is not "3D," in step ST26, the set-top
box 200 determines that the target PES stream is the 2D
stream. Further, in step ST27, the set-top box 200
determines that the 3D extension stream is separately
present. Thereafter, in step ST28, the set-top box 200
15 searches for another PES stream sharing the composition
page ID, that is, the 3D extension stream. Thereafter,
the set-top box 200 causes the process to proceed to step
ST29.
10186]
20 In step ST29, the set-top box 200 determines
whether or not the 3D extension stream is separately
present. When it is determined that the 3D extension
stream is present, in step ST25, the set-top box 200
decodes the 2D segment, then decodes the 3D extension
25 segment, and causes the subtitle to overlap the
background image (3D video) in terms of the disparity
information.
[0187]
Howevei;, when it is determined that the 3D
30 extension stream is not present, in step ST30, the settop
box 200 decodes the 2D segment, causes disparity to
73
SP338522WO00
occur in the subtitle according to the specification of
the set-top box 200 (receiver), and causes the subtitle
to overlap the background image (3D video). For example,
the subtitle is positioned at a^ monitor position without
5 causing disparity to occur in the subtitle. Further, for
example, fixed disparity is brought to occur in the
subtitle, and the subtitle is positioned at the position
ahead of the monitor position.
[0188]
10 Fig. 53(a) illustrates a display example of a
subtitle on an image. In this display example, a
subtitle overlaps an image including a background and a
near-view object on an image. Fig. 53(b) illustrates a
, sense of perspective of a background, a near-view object,
15 and a subtitle, and the subtitle is recognized at the
vejry front.
[0189]
Fig. 54(a) illustrates a display example of a
subtitle (caption) on an image which is the same to Fig.
20 53(a). Fig. 54(b) illustrates a left-eye subtitle LGI,to
overlap a left-eye image and a right-eye subtitle RGI to .
overlap a right-eye image. Fig. 54(c) illustrates that
disparity is brought to occur between the left-eye
subtitle LGI and the right-eye subtitle RGI so that the
25 subtitle can be recognized at the very front.
[0190]
Further, when the set-top box 200 is the device
with the legacy 2D function (the 2D STB), the bit stream
processing unit 201 acquires stereoscopic image data,
30 audio data, and subtitle data (bitmap pattern data
including no display control information) from the
74
SP338522WO00
transport stream TS. "Then, the bit stream processing
unit 201 generates the 2D image data which the subtitle
overlaps using the stereoscopic image data and the
subtitle data (see Fig. 42).
5 [0191]
In this case, the bit stream processing unit 201
acquires data of each segment configuring the subtitle
data from the 2D stream. In other words, in this case,
since the DSS segment is not read from the 3D extension
10 stream, it is possible to avoid reading interfering with
the reception process. In this case, the bit stream
processing unit 201 easily extracts and decodes the 2D
stream frofn the transport stream TS based on the subtitle
type information and the linguistic information with a
15 high degree of accuracy.
[0192]
[Configuration Example of Set-Top Box]
A configuration example of the set-top box 2 00 will
be described. Fig. 55 illustrates a configuration
20 example of the set-top box 200. The set-top box 200
includes the bit stream processing unit 201, the HDMI
terminal 202, an antenna terminal 203, a digital tuner
204, a video signal processing circuit 205, an HDMI
transmission unit 206, and an audio signal processing
25 circuit 207. The set-top box 200 further includes a CPU
211, a flash ROM 212, a DRAM 213, an internal bus 214, a •''
remote control receiving unit (RC receiving unit) 215,
and a remote control transmitter (RC transmitter) 216.
[0193]
30 ' The antenna terminal 203 is a terminal to which a
digital broadcast signal received by a receiving antenna
75
SP338522WO00
(not illustrated) is input. The digital tuner 204
processes the television broadcast signal input to the
antenna terminal 203, and then outputs a transport stream
TS (bit stream data) corresponding to a channel selected
5 by the user.
[0194]
The bit stream processing unit 201 outputs the
output stereoscopic image data which the subtitle
overlaps and the audio data based on the transport stream
10 TS. When the set-top box 200 is the device with the 3D
function (the 3D STB), the bit stream processing unit 201
acquires stereoscopic image data, audio data, subtitle
data (including display control information) from the
transport stream TS.
15 [0195]
The bit stream processing unit 201 generates output
stereoscopic image data in which a subtitle overlaps each
of a left-eye image frame (frameO) portion and a righteye
image data frame (framel) portion which configure the
20 stereoscopic image data (see Fig. 41). At this time,
disparity can be brought to occur between the subtitle
(the left-eye subtitle) overlapping the left-eye image
and the subtitle (the right-eye subtitle) overlapping the
right-eye image based on the disparity information.
25 [0196]
In other words, the bit stream processing unit 201
generates display data of a region for displaying a
subtitle based on the subtitle data. Further, the bit
stream processing unit 201 causes the display data of the
30 region to overlap the left-eye image frame (frameO)
portion and the right-eye image data frame (framel)
76
SP338522WO00
portion which configure the stereoscopic image data, and
obtains output stereoscopic image data. At this time,
the bit stream processing unit 201 shift-adjusts the
position of, display data overlapping each portion based
5 on the disparity information.
[0197]
Further, when the set-top box 2 00 is the device
with the 2D function (the 2D STB), the bit stream
processing unit 201 acquires stereoscopic image data,
10 audio data, and subtitle data (including no display
control information). The bit stream processing unit 201
generates the 2D image data which the subtitle overlaps
using the stereoscopic image data and the subtitle data
(see Fig. 42) . .
15 [0198]
In other words, the bit stream processing unit 201
generates display data of a region for displaying a
subtitle based on the subtitle data. Further, the bit
stream processing unit 201 causes the display data of the
20 region to overlap the.2D image data obtained by executing
processing corresponding to the transmission format on
the stereoscopic image data, and obtains output 2D image
data.
[0199]
25 The video signal processing circuit 205 performs,
for example, an image quality adjustment process on image
data acquired by the bit stream processing unit 201 as
necessary, and then supplies the processed image data to
the HDMI transmission unit 2 06. The audio signal
30 processing circuit 207 performs, for example,, an acoustic
quality adjustment process on the audio data output from
77
{
SP338522WO00
the bit stream processing unit 201 as necessary, and then
supplies the processed audio data to the HDMI
transmission unit 206.
[0200]
5 The HDMI transmission unit 205 transmits, for
example, image data and audio data which are not
compressed through the HDMI terminal 202 by communication
that conforms to the HDMI. In this case, since the image
data and the audio data are transmitted through a TMDS
10 channel of the HDMI, the image data and the audio data
are packed and then output from the HDMI transmission
unit 206 to the HDMI terminal 202.
[0201]
The CPU 211 controls an operation of each component
15 of the set-top box 200. The flash ROM 212 stores control
software and data. The DRAM 213 provides a work area of
the CPU 211. The CPU 211 develops software or data read
from the flash ROM 212 to the DRAM 213, activates the
software, and controls each component of the set-top box
20 200.
[0202]
The RC receiving unit 215 receives a remote control
signal (remote control code) transmitted from the RC
transmitter 215, and supplies the remote control signal
25 to the CPU 2LI1. The CPU 211 controls each component of
the set-top box 200 based on the remote control code.
The CPU 211, the flash ROM 212, and the DRAM 213 are
connected to the internal bus 214.
[0203]
30 An operation of the set-top box 200 will be briefly
described. The digital broadcast signal input to the
78
SP338522WO00
antenna terminal 203 is supplied to the digital tuner 204.
The digital tuner 204 processes the digital broadcast
signal, and outputs a transport stream (bit stream data) •
TS corresponding to a channel'selected by the user.
5 [0204]
The transport steam (bit stream data) TS output
from the digital tuner 204 is supplied to the bit stream
processing unit 201. The bit stream processing unit 201
generates output image data to be output to the
10 television receiver 300 as follows.
[0205]
When the set-top box 200 is the device with the 3D
function (the 3D STB), stereoscopic image data, audio
data, and subtitle data (including display control
15 information) are acquired from the transport stream TS.
Then, the bit stream processing unit 201 generates output
stereoscopic image data in which a subtitle overlaps each
of a left-eye image frame (frameO) portion and a righteye
image data frame . (framel) portion which configure the
20 stereoscopic image data. At this time, disparity is
brought to occur between the left-eye subtitle
overlapping the left-eye image and the right-eye subtitle
overlapping the right-eye image ba&ed on the disparity
information.
25 [0206]
Further, when the set-top box 200 is the device
with the 2D function (the 2D STB), stereoscopic image
data, audio data, and subtitle data (including no display
control information) are acquired. Then, the bit stream
30 processing unit 201 generates 2D image data which the
subtitle overlaps using the stereoscopic image data and
79
SP338522WO00
the subtitle data.
[0207]
The output image data acquired by the bit stream
processing unit 201 is supplied to the video signal
5 processing circuit 205. The video signal processing
circuit 205 performs the image quality adjustment process
or the like on the output image data as necessary. The
processed image data output from the video signal
processing circuit 205 is supplied to the HDMI
10 transmission unit 206.
[0208]
The audio data acquired by the bit stream
processing unit 201 is supplied to the audio signal
processing circuit 207. The audio signal processing
15 circuit 207 performs the acoustic quality adjustment
process on the audio data. The processed audio data
output from the audio signal processing circuit 207 is
supplied to the HDMI transmission unit 206. Then, the
image data and the audio data which are supplied to the
20 HDMI transmission unit 206 are transmitted from the HDMI
terminal 202 to the HDMI cable 400 through the TMDS
channel of the HDMI.
[0209]
[Configuration Example of Bit Stream Processing
25 Unit]
Fig. 56 illustrates a configuration example of the
bit stream processing unit 201 when the set-top box 200
is the device with the 3D function (the 3D STB). The bit
stream processing unit 201 has a configuration
30 corresponding to the transmission data generating unit
110 illustrated in Fig. 2. The bit stream processing
80
SP338522WO00
unit 201 includes a demultiplexer 221, a video decoder
222, and an audio decoder 229.
[0210]
The bit stream processing unit 201 further includes
5 an encoded data buffer 223, a subtitle decoder 224, a
pixel buffer 225, a disparity information interpolating
unit 226, a position control unit 227, and a video
overlapping unit 228. Here, the encoded data buffer 223
configures a decoding buffer.
• 10 [0211]
The demultiplexer 221 extracts packets of the video
data stream and the audio data stream from the transport
stream TS, and transfers the packets to respective
decoders for decoding. Further, the demultiplexer 221
15 extracts the following stream, and causes the extracted
stream to be temporarily accumulated in the encoded data
buffer 223. In this case, when it is determined.that the
service is the 3D service as described above, the
demultiplexer 221 extracts a 2D stream and a 3D extension
20 stream of a language selected by the user or
automatically selected based on the common composition
page ID. In this case, there is a 3D extension stream
which is hardly extracted.
[0212]
25 The video decoder 222 performs processing reverse
to the video encoder 112 of the transmission data
generating unit 110. In other words, the video decoder
222 reconstructs a video data stream from the video
packet extracted by the demultiplexer 221, performs a
30 decoding process, and acquires stereoscopic image data
including left-eye image data and right-eye image data.
81
SP338522WO00
Examples of the transmission format of stereoscopic image
data includes a side-by-side format, a top-and-bottom
format, a frame sequential format, and a video
transmission format in which each view occupies a full
5 screen size.
[0213]
The subtitle decoder 224 performs processing
reverse to the subtitle encoder 125 of the transmission
data generating unit 110. In other words, the subtitle
10 decoder 224 reconstructs each stream from a packet of
each stream accumulated in the encoded data buffer 223,
performs a decoding process, and acquires the following
segment data.
[0214]
15 In other words, the subtitle decoder 224 decodes
the 2D stream, and acquires data of each segment
. configuring the subtitle data. Further, the subtitle
decoder 224 decodes the 3D extension stream, and acquires
data of the DSS segment. As described above, the page
20 IDs (page_id) of the segments of the 2D stream and the 3D
extension stream are the same as each other. Thus, the
subtitle decoder 224 can easily connect the segment of
the 2D stream with the segment of the 3D extension stream
based on the page ID.
25 [0215]
The subtitle decoder 224 generates display data •
(bitmap data) of a region for displaying a subtitle based
on data of each segment configuring subtitle data and
region information of a sub region. Here, transparent
30 color is allocated to a region which is within a region
but not surrounded by a sub region. The pixel buffer 225
82
SP338522WO00
temporarily accumulates the display data.
[0216]
The video overlapping unit 228 obtains output
stereoscopic image data Vout. In this case, the video
•5 overlapping unit 228 causes the display data accumulated
in the pixel buffer 225 to overlap each of the left-eye
image frame (frame0) portion and the right-eye image data
frame (framel) portion of the stereoscopic image' data
obtained by the video decoder 222. In this case, the
10 video overlapping unit 228 appropriately changes the
overlapping position, the size, and the like according to
a transmission format (a side by side format, a top-andbottom
format, a frame sequential format, an MVC format,
or the like) of the stereoscopic image data. The video
15 overlapping unit 228 outputs the output stereoscopic
image data Vout to the outside of the bit stream
processing unit 201.
[0217]
The disparity information interpolating unit 22 6
20 transfers the disparity information obtained by the
subtitle decoder 224 to the position control unit 227 .
The disparity information interpolating unit 226 executes
the interpolating process on the disparity information as
necessary, and transfers the processing result to the
2,5 position control unit 227. The position control unit 227
shift-adjusts the position of the display data
overlapping each frame based on the disparity information
(see Fig. 41). In this case, the position control unit
227 causes disparity to occur by shift-adjusting display
30 data (subtitle pattern data) overlapping the left-eye
image frame (frameO) portion and display data (subtitle
83
SP338522WO00
pattern data) overlapping the right-eye image data frame
(framel) portion in opposite directions based on the
disparity information. As described above, when the 3D
extension stream is not extracted, for example,
5 appropriate disparity is brought to occur based on the
fixed disparity information.
[0218]
In addition, the display control information
includes disparity information commonly used within the
10 subtitle display period. Further, the display control
information may further include disparity information
sequentially updated within the subtitle display period.
The disparity information sequentially updated within the
subtitle display period includes disparity information of
15 a first frame and disparity information of a frame of
each subsequent update frame interval as described above.
[0219]
The position control unit 227 uses the disparity
information commonly used within the subtitle display
20 period as is. Meanwhile, with regard to the disparity
information sequentially updated within the subtitle
display period, the position control unit 227 uses
disparity information which has been subjected to the
interpolating process as necessary by the disparity
25 information interpolating unit 22 6. For example, the
disparity information interpolating unit 22 6 generates
disparity information of an arbitrary frame interval, for
example, a one frame interval within the subtitle display
period.
,30 [0220]
For example, as the interpolating process, the
84
SP338522WO00
disparity information interpolating unit 22 5 performs the
interpolating process including the low pass filter (LPF)
process in the time direction rather than the linear
interpolating process. Thus, a change in disparity
5 information of a predetermined frame' interval in tlie time
direction (the frame direction) after the interpolating
process becomes gentle.
[0221]
The audio decoder 229 performs processing reverse
10 to the audio encoder 113 of the transmission data
generating unit 110. In other words, the audio decoder
229 reconstructs an audio elementary stream from the
audio packet extracted by the demultiplexer 221, performs
a decoding process, and obtains output audio data Aout.
15 Then, the audio decoder 229 outputs the output audio data
Aout to the outside of the bit stream processing unit 201.
[0222]
An operation of the bit stream processing unit 201
illustrated in Fig. 56 will be briefly described. The
20 transport stream TS output from the digital tuner 204
(see Fig. 55) is supplied to the demultiplexer 221. The
demultiplexer 221 extracts packets of the video data
stream and the audio data stream from the transport
stream TS, and transfers the packets to the corresponding
25 . decoders. Further, the demultiplexer 221 further
extracts packets of a 2D stream and a 3D extension stream
of a language selected by the user, and temporarily
accumulates the packets in the encoded data buffer 223.
[0223]
30 The video decoder 222 reconstructs a video data
stream from the video data packet extracted by the
85
SP338522WO00
demultiplexer 221, performs a decoding process, and
acquires stereoscopic image data including left-eye image
data and right-eye image data. The stereoscopic image
, data is supplied to the video overlapping unit 228.
5 [0224]
The subtitle decoder 224 reads the packets of the
2D stream and the 3D extension stream from the encoded
data buffer 223, and decodes the packets. Then, the
subtitle decoder 224 generates display data (bitmap data)
10 of a region for displaying a subtitle based on data of
each segment configuring subtitle data and region
information of a sub region. The display data is
temporarily accumulated in the pixel buffer 225.
[0225] •
15 The video overlapping unit 228 causes the display
data accumulated in the pixel buffer 225 to overlap each
of the left-eye image frame (frameO) portion and the
right-eye image data frame (framel) portion of the
stereoscopic image data obtained by the video decoder 222.
20 In this case, the overlapping position, the size, and the
like are appropriately changed according to a
transmission format (a side by side format, a top-andbottom
format, a frame sequential format, an MVC format,
or the like) of the stereoscopic image data. The output
25 stereoscopic image data Vout obtained by the video
overlapping unit 228 is output to the outside of the bitstream
processing unit 201.
[0226]
Further, the disparity information obtained by the
30 subtitle decoder 224 is transferred to the position
control unit 227 through the disparity information
86
SP338522WO00
interpolating unit 22 6. The disparity information
interpolating unit 226 performs the interpolating process
as necessary. For example, the disparity information
interpolating unit 22 6 executes the interpolating process
5 on the disparity information of several frame intervals
sequentially updated within the subtitle display period
as necessary, and generates disparity information of an
arbitrary frame interval, for example, a one frame
interval.
10 [0227]
The position control unit 227 shift-adjusts display
data (subtitle pattern data) overlapping the left-eye
image frame (frameOj portion and display data (subtitle
pattern data) overlapping the right-eye image data frame
15 (framel) portion in opposite directions based on the
disparity information through the video overlapping unit
228. As a result, disparity is brought to occur between
the left-eye subtitle on the displayed on the left-eye
image and the right-eye subtitle displayed on the right-
20 eye image data. Thus, a 3D display of a subtitle
according to content of the stereoscopic image is
implemented.
[0228]
The audio decoder 229 reconstructs an audio
2 5 elementary stream from the audio packet extracted by the
demultiplexer 221, performs a decoding process, and
obtains audio data Aout corresponding to the display
stereoscopic image data Vout. The audio data Aout is
output to the outside of the bit stream processing unit
30 201.
[0229]
87
SP338522WO00
Fig. 57 illustrates a configuration example of the
bit stream processing unit 201 when the set-top box 200
is the device with the 2D function (the 2D STB). In Fig.
57, components corresponding to those of Fig. 56 are
5 denoted by the same reference numerals, and a detailed
description thereof will not be made. In the following,
for the sake of convenience of description, the bit
stream processing unit 201 illustrated in Fig. 56 is
referred to as a 3D bit stream processing unit 201, and
10 the bit stream processing unit 201 illustrated in Fig. 57
is referred to as a 2D bit stream processing unit 201.
[0230]
In the 3D bit stream processing unit 201
illustrated in Fig. 56, the video decoder 222
15 reconstructs a video data stream from the video packet
extracted by the demultiplexer 221, performs a decoding
process, and acquires stereoscopic image data including
left-eye image data and right-eye image data. On the
other hand, in the 2D bit stream processing unit 201
20 illustrated in Fig. 57, after acquiring the stereoscopic
image data, the video decoder 222 clips left-eye image
data or right-eye image data, performs a scaling process
or the like as necessary, and obtains 2D image data.
[0231]
25, Further, in the 3D bit stream processing unit 201
• illustrated in Fig. 56, the demultiplexer 221 extracts
' packets of a 2D stream and a 3D extension stream of a
language selected by the user or automatically selected,
and transfers the packets to the subtit,le decoder 224 as
30 described above. On the other hand, in the 2D bit stream
processing unit 201 illustrated in Fig. 57, the
88
SP338522WO00
demultiplexer 221 extracts a packet of a 2D stream of a
language selected by the user or automatically selected,
and transfers the packet to the subtitle decoder 224 as
described with reference to Fig.^ 42.
5 [0232]
In this case, the demultiplexer 221 easily extracts
and decodes the 2D stream from the transport stream TS
based on the subtitle type information and the linguistic
information with a high degree of accuracy. In other
10 words, the component descriptor and the subtitle
descriptor included corresponding to the 2D stream and
the 3D extension stream are included in the transport
stream TS (see Fig. 15).
[0233]
15 The subtitle type information "subtitling_type" and
the linguistic information "ISO_639_language_code" are
set to the descriptors so that the 2D stream and the 3D
extension stream are identified (see Figs. 15 and 19).
Thus, the demultiplexer 221 can easily extract the 2D
20 stream from the transport stream TS based on the subtitle
type information and the linguistic information with a
high degree of accuracy.
[0234]
Further, in the 3D bit stream processing unit 201
25 illustrated in Fig. 56, the subtitle decoder 224 acquires
data of each segment configuring subtitle data, for
example, the 2D stream, and further acquires data of the
DSS segment from the 3D extension stream as described
above.
30 [0235]
On the other hand, in the 2D bit stream processing
89
SP338522WO00
unit 201 illustrated in Fig. 57, the subtitle decoder 224
acquires only data of each segment configuring subtitle
data from the 2D stream. Further, the subtitle decoder
224 generates display data (bitmap data) of a region for
5 displaying a subtitle based on data of each segment and
region information of a sub region, and temporarily
accumulate the display data in the pixel buffer 225. In
this case, the subtitle decoder 224 does not read data of
the DSS segment. Thus, it is possible to avoid reading
10 interfering with the reception process.
[0236]
Further, in the 3D bit stream processing unit 201
illustrated in Fig. 56, the video overlapping unit 228
acquires the output stereoscopic image data Vout, and
15 outputs the output stereoscopic image data Vout to the
outside of the bit stream processing unit 201. In this
case, the output stereoscopic image data Vout is obtained
by causing the display data accumulated in the pixel
buffer 225 to overlap each of the left-eye image frame
2 0 (frame0) portion and the right-eye image data frame
(framel) portion of the stereoscopic image data obtained
by the video decoder 222. Then, the position control
unit 227 shift-adjusts the display data in opposite
directions based on the disparity information, and thus
25 causes' disparity to occur between the left-eye subtitle
displayed on the left-eye image and the right-eye
subtitle displayed on the right-eye image data.
[0237]
On the other hand, in the 2D bit stream processing
30 unit 2 01 illustrated in Fig. 57, the video overlapping
unit 228 obtains the output 2D image data Vout by causing
90
SP338522WO00
the display data accumulated in the pixel buffer 225 to
overlap the 2D image data obtained by the video decoder
222. Then, the video overlapping unit 228 outputs the
output 2D image data Vout to the outside of the bit
5 stream processing unit 201.
[0238]
An operation the 2D bit stream processing unit 201
illustrated in Fig. 57 will be briefly described. An
operation of an audio system is similarly to that of the
10 3D bit stream processing unit 201 illustrated in Fig. .56,
and thus a description thereof will not be made.
[0239]
The transport stream TS output from the digital
tuner 204 (see Fig. 55) is supplied to the demultiplexer
15 221. The demultiplexer 221 extracts packets of the video
data stream and the audio data stream from the transport
stream TS, and supplies the extracted padkets to the
corresponding decoders. Further, the demultiplexer 221
further extracts a packet of the 2D stream, and
2 0 temporarily accumulates the packet in the encoded data
buffer 223.
[0240]
The video decoder 222 reconstructs a video data
stream from the video data packet extracted by the
25 demultiplexer 221, performs a decoding process, and
acguires stereoscopic image data including left-eye image
data and right-eye image data.
Further, the video decoder 222 further clips lefteye
image data or right-eye image data from the
30 stereoscopic image data, performs a scaling process or
the like as nece^ssary, and obtains 2D image data. The 2D
91
SP338522WO00
image data is supplied to the video overlapping unit 228.
[0241]
Further, the subtitle decoder 224 reads the 2D
stream from the encoded data buffer 223, and decodes the
5 2D stream. Then, the subtitle decoder 224 generates
display data (bitmap data) of a region for displaying a
subtitle based on data of each segment configuring
subtitle data. The display data is temporarily
accumulated in the pixel buffer 225.
10 [0242]
The video overlapping unit 228 obtains output 2D
image data Vout by causing the display data (bitmap data)
of the subtitle accumulated in the pixel buffer 225 to
overlap the 2D image data obtained by the video decoder
15 222. The output 2D image data Vout is output to the
outside of the bit stream processing unit 201.
[0243]
In the set-top box 200 illustrated in Fig. 55, the
subtitle descriptor included corresponding to the 2D
20 stream and the 3D extension stream describes a common
composition page ID is described, and specifies
association between the two streams. Thus, the set-top
box 200 can efficiently and appropriately extract and
decodes the 2D stream and the 3D extension stream, which
25 are associated with each other, based on the association
information, and obtain disparity information together
with subtitle data.
[0244]
Further, in the set-top box 200 illustrated in Fig.
30 55, the demultiplexer 221 of the 2D bit stream processing
unit 201 (see Fig. 57) easily extracts only the 2D stream
92
SP338522WO00
from the transport stream TS based on the subtitle type
information and the linguistic information with a high
degree of accuracy. As a result, the subtitle decoder
224 can strongly prevent the decoding process from being
5 performed on the 3D extension stream including the DSS
segment with the disparity information and thus avoid the
process interfering with the reception process.
[0245]
Further, in the set-top box 200 illustrated in Fig.
10 55, the transport stream TS output from the digital tuner
2 04 includes the display control information as well as
the stereoscopic image data and the subtitle data. The
display control information includes the display control
information (region information of a sub region,
15 disparity information, and the like). Thus, disparity
can be brought to occur at the display positions of the
left-eye subtitle and the right-eye subtitle, and the
consistency of a sense of perspective with each object in
an image when^ a subtitle is displayed can be maintained
20 to an optimal state.
[0246]
Further, in the set-top box 200 illustrated in Fig.
55, when the disparity information sequentially updated
within the subtitle display period is included in the
25 display control information acquired by the subtitle
decoder 224 of the 3D bit stream processing unit 201 (see
Fig. 46), the display positions of the left-eye subtitle
and the right-eye subtitle can be dynamically controlled.
Thus, disparity brought to occur between the left-eye
30 subtitle and the right-eye subtitle can be dynamically
changed in conjunction with a change in image content.
93
SP338522WO00
[0247]
Further, in the set-top box 200 illustrated in Fig.
55, the disparity information interpolating unit 22 6 of
the 3D bit stream processing unit 201 (see Fig. 49)
5 executes the interpolating process on disparityinformation
of a plurality of frames configuring
disparity information sequentially updated within the
subtitle display period (a period of a predetermined
number of frames). In this case, even when disparity
10 information is transmitted from a transmission, side at
update frame intervals, disparity brought to occur
between the left-eye subtitle and the right-eye subtitle
can be controlled at minute intervals, for example, in
units of frames.
15 [0248]
Further, in the set-top box 200 illustrated in Fig.
55, the interpolating process in the disparity
information interpolating unit 22 6 of the 3D bit stream
processing unit 201 (see Fig. 56) may be performed to be
20 accompanied by, for example, the low pass filter process
in the time direction (the frame direction). Thus, even
when disparity information is transmitted from the
transmission side at update frame intervals, a change of
disparity information in the time direction after the
25 interpolating process can become gentle, and thus it is
possible to suppress an uncomfortable feeling caused when
transition of disparity brought to occur between the
left-eye subtitle and the right-eye subtitle becomes
discontinuous at update frame intervals.
30 [0249]
Further^ although not described above, a
94
SP338522WO00
configuration in which the set-top box 200 is the device
with the 3D function, and either of the 2D display mode
and the 3D display mode can be selected by the user may
be provided. In this case, when the 3D display mode is
5 selected, the bit stream processing unit 201 has the same
configuration as the 3D bit stream processing unit 201
(see Fig. 56) and the performs the same operation as the
3D bit stream processing unit 201. Meanwhile, when the
2D display mode is selected, the bit stream processing
10 unit 2 01 has substantially the same configuration as the
2D bit stream processing unit 201 (see Fig. 57) and
performs the same operation as the 2D bit stream
processing unit 201.
[0250]
15 [Description of Television Receiver]
Referring back to Fig. 1, when the television
receiver 300 is the device with the 3D function, the
television receiver 300 receives the stereoscopic image
• data transmitted from the set-top box 200 through the
20 HDMI cable 400. The television receiver 300 includes a
3D signal processing unit 301. The 3D signal processing
unit 301 performs a process (a decoding process)
corresponding to a transmission format on the
stereoscopic image data, and so generates the left-eye
25 image data and the right-eye image data.
[0251]
[Configuration Example of Television Receiver]
A configuration example of the television receiver
300 with the 3D function will be described. Fig. 58
30 illustrates a configuration example of the television
receiver 300. The television receiver 300 includes the
95
SP338522WO00
3D signal processing unit 301, the HDMI terminal 302, an
HDMI reception unit 303, an antenna terminal 304, a
digital tuner 305, and a bit stream processing unit 306.
[0252]
5 The television receiver 300 further includes a
video/graphics processing circuit 307, a panel driving
circuit 308, a display panel 309, an audio signal
processing circuit 310, an audio 4i^plifying circuit 311,
and a speaker 312. The television receiver 300 further
10 includes a CPU 321, a flash ROM 322, a DRAM 323, an
internal bus 324, a remote control receiving unit (RC
receiving unit) 325, and a remote control transmitter (RC
transmitter) 326.
[0253]
15 The antenna terminal 304 is a terminal to which a
television broadcast signal received by a receiving
antenna (not illustrated) is input. The digital tuner
305 processes the television broadcast signal input to
the antenna terminal 304, and then outputs a transport
20 stream (bit stream data) TS corresponding to a channel
selected by the user.
[0254]
The bit stream processing unit 306 outputs the
output stereoscopic image data which the subtitle
25 overlaps and the audio data based on the transport stream
TS. Although a detailed description will not be made,
for example, the bit stream processing unit 201 has the
same configuration as the 3D bit stream processing unit
201 (see Fig. 56) of the set-top box 200. The bit stream
30 processing unit 306 synthesizes display data of the lefteye
subtitle and the right-eye subtitle with the
96
SP338522WO00
stereoscopic image data, and generates and outputs output
stereoscopic image data which the subtitle overlaps.
[0255]
Further, for example, when the transmission format
5 of the stereoscopic image data is the side by side format,
the top-and-bottom format, or the like, the bit stream
processing unit 306 executes the scaling process and
outputs left-eye image data and right-eye image data of
the full resolution. Further, the bit stream processing
10 unit 306 outputs audio data corresponding to image data.
[0256]:
The HDMI reception unit 303 receives image data and
audio data, which are not compressed, supplied to the
HDMI terminal 302 through the HDMI cable 400 by
15 communication that conforms to the HDMI. The HDMI
reception unit 303 supports, for example, an HDMI1.4a
version and can deal with the stereoscopic image data.
[02 57]
The 3D signal processing unit 301 performs a
20 decoding process on the stereoscopic image data which is
received by the HDMI reception unit 303, and generates
left-eye image data and right-eye image data of the full
resolution. The 3D signal processing unit 301 performs
the decoding process corresponding to a TMDS transmission
25 data format. Further, the 3D signal processing unit 301
does not perform any process on the left-eye image data
and the right-eye image data of the full resolution
obtained by the bit stream processing unit 306.
[0258]
30 The video/graphics processing circuit 307 generates
image data for displaying a stereoscopic image based on
97
SP338522WO00
the left-eye image data and the right-eye image data
generated by the 3D signal processing unit 301. Further,
the video/graphics processing circuit 307 performs an
image quality adjustment process on the image data as
5 necessary.
[0259]
Further, the video/graphics processing circuit 30t
synthesizes the image data with data of overlapping
information such as a menu or a program table as
10 necessary. The panel driving circuit 308 drives the
display panel 309 based on the image data output from the
video/graphics processing circuit 307. For example, the
display panel 309 is configured with an LCD (Liquid
Crystal Display), a PDP (Plasma Display Panel), or the
15 like.
[0260]
The audio signal processing circuit 310 performs a
necessary process such as digital to analog (D/A)
conversion on the audio data which is received by the
20 HDMI reception unit 303 or obtained by the bit stream
processing unit 306. The audio amplifying circuit 311
amplifies an audio signal output from the audio signal
processing circuit 310 and supplies the amplified audio
signal to the speaker 312.
25 [0261]
The CPU 321 controls an operation of each component
of television receiver 300. The flash ROM 322 stores
control software and data. The DRAM 323 provides a work
area of the CPU 321. The CPU 321 develops software and
30 data read from the flash ROM 322 to the DRAM 323,
activates the software, and controls each component of
98
SP338522WO00
the television receiver 300.
[0262]
The RC receiving unit 325 receives a remote control
signal (remote control code) transmitted from the RC
5 transmitter 326, and supplies the remote control signal
to the CPU 321. The CPU 321 controls each component of
the television receiver 300 based on the remote control
code. The CPU 321, the flash ROM 322, and the DRAM 323
are connected to the internal bus 324.
10 [0263]
An operation of the television receiver 300
illustrated in Fig. 58 will be briefly described. The
HDMI reception unit 303 receives the stereoscopic image
data and the audio data which are transmitted from the
15 set-top box 200 connected to the HDMI terminal 302
through the HDMI cable 400. The stereoscopic image data
received by the HDMI reception unit 303 is supplied to
the 3D signal processing unit 301. The audio data
received by the HDMI reception unit 303 is supplied to
20 the audio signal processing circuit 310.
[0264]
The television broadcast signal input to the
antenna terminal 304 is supplied to the digital tuner 305.
The digital tuner 305 processes the television broadcast
25 signal, arid outputs a transport stream (bit stream data)
TS corresponding to a channel selected by the user. The
transport stream TS is supplied to the bit stream
processing unit 306.
[0265]
30 The bit stream processing unit 306 obtains the
output stereoscopic image data which the subtitle
99
SP338522WO00
overlaps and the audio data based on the video data
stream, the audio data stream, the 2D stream, and the 3D
extension stream. In this case, the display data of the
left-eye subtitle and the right-eye subtitle is
5 synthesized with the stereoscopic image data, and thus
the output stereoscopic image data (the left-eye image
data and the right-eye image data of the full resolution)
which the subtitle overlaps is generated. The output
stereoscopic image data is supplied to the video/graphics
10 processing circuit 307 through the 3D signal processing
unit 301.
[0266]
The 3D signal processing unit 301 performs a
decoding process on the stereoscopic image data which is
15 received by the HDMI reception unit 303, and generates
left-eye image data and right-eye image data of the full
resolution. The left-eye image data and the right-eye
image data are supplied to the video/graphics processing
circuit 307. The video/graphics processing circuit 307
20 generates image data for displaying a stereoscopic image
based on the left-eye image data and the right-eye image
data, and performs an image quality adjustment process
and a synthesis process of the overlapping information
data such as OSD (object screen display) as necessary.
25 [0267]
The image data obtained by the video/graphics
processing circuit 307 is supplied to the panel driving
circuit 308. Thus, the stereoscopic image is displayed
through the display panel 309. For example, the left-eye
30 image based on left-eye image data and the right-eye
image based on the right-eye image data are alternately
100
SP338522WO00
displayed on the display panel 309 in a time division
manner. For example, a viewer can perceive a
stereoscopic image by wearing shutter glasses in which a
left-eye shutter and a right-eye shutter are alternately
5 opened in synchronization with a display of the display
panel 309 and then viewing only the left-eye image with
the left eye and only the right-eye image with the right
eye.
[0268]
10 Further, the audio data obtained by the bit stream
processing unit 306 is supplied to the audio signal
processing circuit 310. The audio signal processing
circuit 310 performs a necessary process such as D/A
conversion on the audio data which is received by the
15 HDMI reception unit 303 or obtained by the bit stream
processing unit 306. The audio data is amplified by the
audio amplifying circuit 311 and then supplied to the
speaker 312. Thus, a sound corresponding to a display
image of the display panel 309 is output from the speaker
20 312.
[0269]
Fig. 58 illustrates the television receiver 300
with the 3D function as described above. Although a
detailed description will not be made, the television
25 receiver 300 with the 3D function has almost the same
configuration as the television receiver with the legacy
2D function. However, in case of the television receiver
with the legacy 2D function, the bit stream processing
unit 306 has the same configuration as the 2D bit stream
30 processing unit 201 illustrated in Fig. 57 and performs
the same operation as the 2D bit stream processing unit
101
. SP338522WO00
201. Further, the television receiver with the legacy 2D
function does not need the 3D signal processing unit 301.
[0270]
Further, a configuration in which the television
5 receiver 300 has the 3D function and either of the 2D
display mode and the 3D display mode can be selected by
the user may be provided. In this case, when the 3D
display mode is selected, the bit stream processing unit
306 has the same configuration and performs the same
10 operation as described above. Meanwhile, when the 2D
display mode is selected, the bit stream processing unit
306 has the same configuration as the 2D bit stream
processing unit 201 illustrated in Fig. 57 and performs
the same operation as the 2D bit stream processing unit
15 201.
[0271]
<2. Modified Example>
"First Modified Example"
In the above embodiment, "subtitling_type"
20 described in the subtitle descriptor corresponding to the
2D stream is set to 2D. However,, "subtitling_type" may
be set to 3D. In this case, the reception device with
the 3D function at the reception side can recognize the
presence of another PES stream including the DSS segment
25 when the PES stream has 3D as its type but does not
indlude the DSS segment.
[0272]
Fig. 59 illustrates a stream configuration example
of a subtitle data stream (a 2D stream and a 3D extehsion
30 stream) in this case. This example is a single language
service example of English "eng." The 3D extension
102
SP338522WO00
stream is extracted in by "composition_page_id = OxXXXX"
which is in common with the 2D stream, and designated by
"subtitling_type = 3D" and "ISO_639_language_code = zxx."
Here, the 2D stream is designated by "subtitling_type =
5 3D" and "ISO_639_language_code = eng."
[0273]
Fig. 60 illustrates a configuration example of the
transport stream TS. In Fig. 60, for the sake of
simplification of the drawing, video- and audio-related
10 portions are not illustrated. The transport stream TS
includes the PES packet obtained by packetizing each
elementary stream.
[0274]
In this configuration example, the PES packet
15 "Subtitle PESl" of the 2D stream (the first private data
stream) and the PES packet "Subtitle PES2" of the 3D
extension stream (the second private data stream) are
included. The 2D stream (PES stream) includes the
segments of the DDS, the PCS, the RCS, the CDS, the ODS,
20 and the EDS (see Fig. 11(a)). The 3D extension stream
(PES stream) includes the segments of the DDS, the DSS,
and the EDS or the segments of the DDS, the PCS, the DSS,
and the EDS (Figs. 11(b) and 11(c)). In this case,
"Elementary_PID" of the 2D stream and "Elementary_PID" of
25 the 3D extension stream are set to different IDs such as
PIDl and PID2, and the streams are different PES streams.
[0275]
A subtitle descriptor (subtitling_descriptor)
representing content corresponding to the 2D stream and
30 the 3D extension stream is included in the PMT. Further,
a component descriptor (component descriptor)
103
SP338522WO00
representing delivery content is included in the EIT for
. each stream. When "stream_content" of the component
descriptor represents a subtitle, "component_type" of
"0x15" or "0x25" represents a 3D subtitle, and the other
5 values represent a 2D subtitle. "subtitling_type" of the
subtitle descriptor is set to the same value as
"component_type."
[0276]
Here, "component_type" of the component descriptor
10 and "subtitling_type" of the subtitle descriptor
corresponding to the 3D extension stream are set to 3D.
Further, "component_type" of the component descriptor and
'''subtitling_type" of the subtitle descriptor
corresponding to the 2D stream are also set to 3D.
15 [0277]
In order to specify association between the 2D
stream and the 3D extension stream, the composition page
ID "composition_page_id" of the subtitle descriptor
corresponding to each stream is set. In other words>
20 "composition_page_id" of the 2D stream and
"composition_page_id" of the 3D extension stream are set
to share the same value ("OxXXXX" in Fig. 60). Here,
"composition_page_id" configures association information.
Further, both PES streams are encoded such that "page_id"
25 of each associated segments has the same value (OxXXXX)
so that each segment included in the 3D extension stream
is associated with each segment of the 2D stream.
[0278]
Further, for example, the ISO language code
30" (ISO_639_language_code) is described in the subtitle
descriptor and the component descriptor as linguistic
104
SP338522WO00
information. The ISO language code of the descriptor
corresponding to the 2D stream is set to represent a
language of a subtitle. In this example, the ISO
language code is set to "eng" representing English. The
5 3D extension stream includes the DSS segment with
disparity information, but does not include the ODS
segment and thus does not rely on a language. The ISO
language code described in the descriptor corresponding
to the 3D extension stream is set to, for example, "zxx"
10 representing a non-language.
[0279]
The stream configuration example illustrated in Fig. •
59 illustrates an example in which a single language
service of English "eng" is present (see Fig. 23) . Of
15 course, however, the present technology can be similarly
applied to a multilingual service. Fig. 61 illustrates a
stream configuration example of a subtitle data stream (a
2D stream and a 3D extension stream) when a bilingual
service is provided. This example is a bilingual service
2 0 of English "eng" and German "ger."
[0280]
Regarding the English service, the 3D extension
stream is extracted in by "composition_page_id = OxXXXX"
which is in common with the 2D stream, designated by
25 "subtitling_type = 3D" and "ISO_639_language_code = zxx,"
and the 2D stream is designated by "subtitling_type = 3D"
and "ISO_639_language_code = eng." Meanwhile, regarding
the German service, the 3D extension stream is extracted
in by "composition__page_id = OxYYYY" which is in common
30 with the 2D stream and designated by "subtitling_type =
3D" and "ISO_639_language_code = zxx," and the 2D streairt
105
•SP338522WO0,0
i s d e s i g n a t e d by " s u b t i t l i n g _ t y p e = 3D" and
"ISO_63 9_language_code = g e r ."
[0281]
Next, the flow of a process when "subtitling_type"
5 described in the' subtitle descriptor corresponding to the
2D stream is set to 3D and the receiver with the 3D
function determines that the service is the 3D service
will be described with reference to a flowchart of Fig.
62. Here, the description will proceed with an example
10 in which the receiver is the set-top box 200.
[0282]
In step ST41, the set-top box 200 determines that
the service is the 3D service, and then causes the
process to proceed to step ST42. In step ST42, the set-
15 top box 200 determines whether or not "component_type" of
the component descriptor of the first stream whose
"stream_type" represents "0x03 (subtitle)" is "3D."
[0283]
When "component_type" is "3D," in step ST43, the
20 set-top box 200 determines that the target PES stream is
a 3D PES stream. Then, the set-top box 200 causes the
process to proceed to step ST44. In step ST44, the settop
box 200 determines whether or not a 3D expended
segment, that is, the DSS segment is present in the
25 target PES stream.
[0284]
When the DSS segment is present in the target PES
stream, in step ST45, the set-top box 200 determines that
the 3D extension segment (the DSS segment) is present in
30 the target PES stream (see Fig. 52). Thereafter, in step
ST46, the set-top box 200 decodes the 2D stream, then
106'
SP338522WO00
decodes the 3D extension segment, and causes the subtitle
to overlap the background image (3D video) in terms of
the disparity information.
[0285]
5 When it is determined in step ST44 that the DSS
segment is not present in the target PES stream, in step
ST47, the set-top box 200 determines that the target PES
stream is the 2'D stream. Then, in step ST48, the set-top
box 200 determines that the 3D extension stream is
10 separately present.-
[0286]
Thereafter, in step ST49, the set-top box 200
searches for another PES stream sharing the composition
page ID, that is, the 3D extension stream. Then, in step
15 ST46, the set-top box 200 decodes the 2D stream, then
decodes the 3D extension segment, and causes the subtitle
to overlap the background image (3D video) in terms of
the disparity information.
[0287]
20 Further, when it is determined in step ST42 that
"component_type" is not "3D,"in step ST50, the set-top
box 200 determines that the target PES stream is the 2D
stream. Then, in step ST51, the set-top box 200 decodes
the 2D stream, then decodes the 3D extension segment, and
25 causes the subtitle, to overlap the background image (3D
video) in terms of the disparity information. •
[0288] ';
"Second Modified Example"
The above embodiment has been described in
30 connection with the example in which the composition page
ID (composition_page_id) described in the subtitle
107 ,
SP338522WO00
descriptor is,shared by the 2D stream and the 3D
extension stream. Further, the composition page ID may
be defined by, a special value (special_valueA)
representing that the 2D stream and the 3D extension
5 stream which are associated with each other are present.
[0289]
In this case, the reception device with the 3D
function at the reception side can recognize that the 3D
extension stream is present in addition to the 2D stream
10 when the composition page ID is the special value
(special_valueA). In other words, the reception device
can recognize that the DSS segment is divided into
another PES stream, and avoid, for example, a useless
process of searching for the 3D extension stream when the
15 3D extension stream is not present.
[0290]
Fig. 63 illustrates a stream configuration example
of a subtitle data stream (a 2D stream and a 3D extension
stream) in this case. This example is a single language
20 service example of English "eng." The 3D extension
stream is extracted in "composition_page_id =
special_valueA" which is in common with the 2D stream,
and designated by "subtitling_type = 3D" and
"ISO_639_language_code = zxx." Further, the 2D stream is
25 designated by "subtitling_type = 2D" and
"ISO_63 9_language_code = eng."
[02 91]
Fig. 64 is a configuration example of the transport
stream TS. In Fig.. 64, for the sake of simplification df
30 the drawing, video- and audio-related portions are not
illustrated. The transport stream TS includes the PES
108
SP338522WO00
packet obtained by packetizing each elementary stream.
[0292]
In this configuration example, the PES packet
"Subtitle PESl" of the 2D stream and the PES packet
5 "Subtitle PES2" of the 3D extension stream are included.
The 2D stream (PES stream) includes the segments of the
DDS, the PCS, the RCS, the CDS, the ODS, and the EDS (see
Fig. 11(a)). The 3D extension stream (PES stream)
includes the segments of the DDS, the DSS, and the EDS or
10 the segments of the DDS, the PCS, the DSS, and the EDS
(Figs. 11(b) and 11(c)). In this case, "Elementary_PID"
of the respective streams are set to be different from
each other, and the streams are PES streams different
from each other.
15 [0293]
A subtitle elementary.loop (subtitle ES loop)
having information associated with a subtitle elementary
stream is present in the PMT. In the subtitle elementary
loop, not only information such as a packet identifier
20 (PID) but' also a descriptor describing information
associated with a corresponding elementary stream are
arranged for each stream.
[0294]
In order to specify association between the 2D
25 stream and the 3D extension stream, the composition page
ID of the subtitle descriptor corresponding to each
stream is set. In other words, "compositron_page_id" of
the 2D stream and "composition_page_id" of the 3D
extension stream are set to share the same value. This
30 value is the special value (special_valueA) representing
that the 2D stream and the 3D extension stream which are
109
SP338522WO00
associated with each other are present. Further, both
PES streams are encoded such that "page_id" of each
associated segment has the same value (special_valueA) so
that each segment included in the 3D extension stream is
5 associated with each segment of the 2D stream.
[0295]
The stream configuration example illustrated in Fig.
63 illustrates an example in which a single language
service of English "eng" is present. Of course, however,
10 the present technology can be similarly applied to a
multilingual service. Fig. 65 illustrates a stream
configuration example of a subtitle data stream (a 2D
stream and a 3D extension stream) when a bilingual
service is provided. This example is a bilingual service
15 of English "eng" and German "ger."
[0296]
Regarding the English service, the 3D extension
stream is extracted in by "composition_page_id =
special_valueA" which is in common with the 2D stream,
20 and designated by "subtitling_type = 3D" and
"ISO_639_language_code = zxx," and the 2D stream is
designated by "subtitling_type = 2D" and
"ISO_639_language_code = eng." Meanwhile, regarding the
German service, the 3D extension stream is extracted in
25 by "composition_page_id = special_valueB" which is in
common with the 2D stream and designated by
"subtitling_type = 3D" and "ISO_639_language_code = zxx,"
and the 2D stream is designated by "subtitling_type = 2D"
and "ISO_639_language_code = ger."
30 [0297]
Next, the flow of a process when the common
110
SP338522WO00
composition page ID . (composition_page_id) is set to the
special value, and the receiver with the 3D function
determines that the service is the 3D service will be
described with reference to a flowchart of Fig. 66. Here,
5 the description will proceed with an example in which the
receiver is the set-top box 200.
[0298]
In step ST51, the set-top box 200 determines that
the service is the 3D service, and then causes the
10 process to proceed to step ST62.
In step ST62, the set-top box 200 determines whether or
not "component_type" of the component descriptor of the
first stream whose "stream_type" represents "0x03
(subtitle)" is "3D."
15 [0299]
When "component_type" is "3D," in step ST63, the
set-top box 200 determines that the target PES stream is
a 3D PES stream. Here, the 3D PES stream (3D stream) is
a PES stream including the OSS segment with the disparity
2 0 information in addition to the segments configuring data
of the overlapping information (subtitle data) such as
the DDS, the PCS, the RCS, the CDS, the ODS, and the EDS
as illustrated in Fig. 52.
[0300]
25 Next, in step ST64, the set-top box 200' determines
that the 3D extension segment (the DSS segment) is
present in the target PES stream. Thereafter, in step
ST65, the set-top box 200 decodes the 2D stream, then
decodes the 3D extension segment, and causes the subtitle
30 to overlap the background image (3D video) in terms of
the disparity information.
Ill
SP338522WO00
[0301]
Further, when it is determined in step ST62 that
"coniponent_type" is "3D," in step ST66, the set-top box
2 00 determines that the target PES stream is the 2D
5 stream. Then, in step ST67, the set-top box 200
determines whether or not "composition_page_id" of the
subtitle descriptor representing the target PES stream is
the special value representing the dual stream (Dual
stream).
10 [0302]
When "composition_page_id" is the special value, in
step ST68, the set-top box 200 determines that the 3D
extension stream is separately present. Thereafter, in
step ST69, the set-top box 200 searches for another PES
15 stream sharing the composition page ID, that is, the 3D
extension stream. Then, step ST65, the set-top box 200
decodes the 2D stream, then decodes the 3D extension
segment, and causes the subtitle to overlap the
background image (3D video) in terms of the disparity
2 0 information.
[0303]
When it is determined in step ST67 that
"composition_page_id" is not the special value, in step
ST70, the set-top box 200 determines that there is no PES
25 stream related to the 2D stream. Then, in step ST71, the
set-top box 200 decodes the 2D segment, causes disparity
to occur in the subtitle according to the specification
of the set-top box 200 (receiver), and causes the
subtitle to overlap the background image (3D video). For
30 example, the subtitle is positioned at a monitor position
without causing disparity to occur in the subtitle.
112
SP338522WO00
Further, for example, fixed disparity is brought to occur
in the subtitle, and the subtitle is positioned at the
position ahead of the monitor position.
[0304]
5 "Third Modified Example"
The above embodiment has been described in
connection with the example in which the 2D stream and
the 3D extension stream are associated with each other
using the composition page ID (composition_page_id)
10 described in the subtitle descriptor. Dedicated
information (linking information) for linking the 2D
stream with the 3D extension stream may be described in
the descriptor. Thus, the 2D stream, the 3D extension
stream which are associated with each other can be more
15 strongly linked.
[0305]
Fig. 57 illustrates a stream configuration example
of a subtitle data stream (a 2D stream and a 3D extension
stream) in this case. This example is a single language
20 service example of English "eng." The 3D extension
stream is extracted in by "composition_page_id = OxXXXX"
which is in common with the 2D stream, and designated by
"subtitling_type = 3D" and "ISO_639_language_code = zxx."
Further, the 2D stream is designated by "subtitling_type
25 = 2D" and "ISO_639_language_code = eng."
[0306]
Further, by sharing the composition page ID
(composition_page_id) of the subtitle descriptor between
the 2D stream and the 3D extension stream, association
30 between the two streams is explicitly represented.
Further, by describing dedicated linking information in
113
SP338522WO00
the descriptor, linking of the two streams is specified.
(1) A newly defined descriptor or (2) an existing
descriptor may be used as the descriptor in this case.
[0307]
5 First, an example in which a new descriptor is
defined will be described. Here, a stream association ID
descriptor (stream_association_ID descriptor) in which
dedicated linking information (association_ID) is
described is newly defined.
10 [0308]
Fig. 68 illustrates an example of syntax of the
stream association ID descriptor. Fig. 69 illustrates
content (semantics) of main information in this syntax
example. An 8-bit field-of "descriptor_tag" represents
15 that the descriptor is the stream association ID
descriptor. An 8-bit field of "descriptor_length"
represents the entire byte size subsequent to the field.
[0309]
A 4-bit field of "stream_content" represents the
20 stream type of the main stream such as a video, an audio,
or a subtitle. A 4-bit field of "component_type"
represents the component type of the main stream such as
a video, an audio, or a subtitle. "stream_content" and
"component_type" are regarded as the same information as
25 "stream_content" and "component_type" within the
component descriptor corresponding to the main stream.
[0310]
A 4-bit field of "association_ID" represents
linking information. "association_ID" of the stream
30 association ID descriptor corresponding to each component
(PES stream) to be linked has the same value.
114
SP338522WO00
[0311]
Next, an example in which the existing descriptor
is used will be described. Here, an example in which an
extended component descriptor is defined and used will be
5 described. Fig. 70 illustrates an example of syntax of
an extended component descriptor. Fig. 71 illustrates
content (semantics) of main information in this syntax
example.
. [0312]
10 A 1-bit field of "extended_flag" represents whether
or not the descriptor is an extended descriptor. In this
case, "1" represents the extended descriptor, and "0"
represents a non-extended descriptor. When the component
descriptor is the extended descriptor, the descriptor has
15 "association_ID" serving as linking information.
[0313]
An 8-bit field of "extension_type" represents an
extension type. Here, for example, when "extension_type"
is set "0x01," it means that the component (PES stream)
20 corresponding to the component descriptor is linked by
"association_ID." "extension_length" represents the
extended byte size subsequent to the field. Further, a
4-bit field of "association_ID" represents linking
information. As described above, "association_ID" of the
25 stream association ID descriptor corresponding to each
component (PES stream) to be linked has the same value.
[0314]
Fig. 72 illustrates a configuration example of the
transport stream TS. In Fig. 72, for the sake of
30 simplification of the drawing, video- and audio-related
portions are not illustrated. The transport stream TS
115
SP338522WO00
includes the PES packet obtained by packetizing each
elementary stream.
[0315]
In this configuration example, the PES packet
5 "Subtitle PESl" of the 2D stream and the PES packet
"Subtitle PES2" of the 3D extension stream are included.
The 2D stream (PES stream) includes the segments of the
CDS, the PCS, the RCS, the CDS, the ODS, and the EDS (see
Fig. 11(a)). The 3D extension stream (PES stream)
10 includes the segments of the DDS, the DSS, and the EDS or
the segments of the DDS, the PCS, the DSS, and the EDS
(Figs. 11(b) and 11(c)). In this case, "Elementary_PID"
of the respective streams are set to be different from
each other, and the streams are PES streams different
15 from each other.
[0316]
In order to specify association between the 2D
stream and the 3D extension stream, the composition page
ID of the subtitle descriptor corresponding to each
2 0 stream is set. In other words, "composition_page_id" of
the 2D stream and "composition_page_id" of the 3D
extension stream are set to share the same value
("OxXXXX" in Fig. 72). Further, both PES streams are
encoded such that "page_id" of each associated segment
25 has the same value (OxXXXX) so that each segment included
in the 3D extension stream is associate^d with each
segment of the 2D stream.
[0317]
Further, the component descriptor corresponding to
30 each of the 2D stream and the 3D extension stream is
extended, and "association_ID" serving as the linking
116
SP338522WO00
information described in each component descriptor is set
to the same value. Thus, the 2D stream and the 3D
extension stream which are associated with each other are
linked by "association_ID."
5 [0318]
Further, the stream association ID descriptor
(stream_association_ID descriptor) corresponding to each
of the 2D stream and the 3D extension stream is included
in the transport stream TS. Further, "association_ID"
10 serving as linking information described in each
descriptor has the same value. Thus, the 2D stream and
the 3D extension stream which are associated with each
other are linked by "association_ID."
[0319]
15 In addition, in the configuration example of the
transport stream TS illustrated in Fig. 72, both
extension of the component descriptor and insertion of
the stream association ID descriptor are performed.
However, any one of extension of the component descriptor
20 and insertion of the stream association ID descriptor may
be performed. Further, of course, although not
illustrated, association of "component descriptor" and
"PID" is performed as necessary through "component_tag"
of "stream_identifier descriptor."
25 [0320]
Further, the stream configuration example
illustrated in Fig. 67 illustrates an example in which a
single language service of English "eng" is present . Of
course, however, the present technology can be similarly
30 applied to a multilingual service. Fig. 73 illustrates a
stream configuration example of a subtitle data stream (a
117
SP338522WO00
2D stream and a 3D extension stream) when a bilingual
service is provided. This example is a bilingual service
of English "eng" and German "ger."
[0321]
5 In this case, in each language service, the 2D
stream, is linked with the 3D extension stream such that
the dedicated linking information "association_ID" is
described in the corresponding descriptor. At this time,
the different language service is different in the value
10 of "association_ID." Fig. 74 illustrates a configuration
example of the transport stream TS in this case.
"association_ID" related to the language service of ,
English is "association_ID_l," and "association_ID"
related to the language service of German is
15 "association_ID_2."
[0322]
Next, the flow of a process when the dedicated
linking information is described in the descriptor and
the receiver with the 3D function determines that the
20 service is the 3D^ service will be described with
reference to a flowchart of Fig. 75. Here, the
description will proceed with an example in which the
receiver is the set-top box 200.
[0323]
25 In step ST81, the set-top box 200 determines that
the service is the 3D service, and then causes the
process to proceed to step ST82. In step ST82, the settop
box 200 determines whether or not "association_ID"
serving as the linking information is present in the
30 descriptor of the PMT or the KIT.
[0324]
118
SP338522WO00
When it is determined that "association_ID" is
present, in step ST83, the set-top box 200 compares
"association_ID" of the respective PES streams with each
other. Then, in step ST84, the set-top box 200
5 recognizes an association state of the 2D stream and the
3D extension stream, that is, linking of the 2D stream
and the 3D extension stream. Then, in step ST85, after
decoding the 2D stream in step ST65, the set-top box 200
decodes the 3D extension segment, and causes the subtitle
10 to overlap the background image (3D video) in terms of
the disparity information.
[0325]
When it is determined in step ST82 that
"association_ID" is not present, the set-top box 200
15 causes the process to proceed to step ST86. In step ST86,
the set-top box 200 determines whether or not
"component_type" of the component descriptor of the first
stream whose "stream_type" represents "0x03 (subtitle)",
is "30."
20 [0326]
When "component_type" is "3D", in step ST87, the
set-top box 200 determines that the target PES stream is
a 3D PES. stream. Here, the 3D PES stream (3D stream) is
a PES stream including the DSS segment with the disparity
25 information in addition to the segments configuring data
of the overlapping information (subtitle data) such as
the DDS, the PCS, the RCS, the CDS, the ODS, and .the EDS
as illustrated in Fig. 52.
[0327]
30 Next, in step ST88, the set-top box 200 determines
that the 3D extension segment (the DSS segment) is
119
SP338522WO00
present in the target PES stream. Thereafter/ in step
ST85, the set-top box 200 decodes the 2D stream, then
decodes the 3D extension segment, and causes the subtitle
to overlap the background image (3D video) in terms of
5 the disparity information.
[0328]
Further, when it is determined in step ST8 6 that
"component_type" is not "3D," in step ST89, the set-top
box 200 determines that the target PES stream is the 2D
10 stream. Then, in step ST90, the set-top box 200 decodes
the 2D segment, causes disparity to occur in the subtitle
according to the specification of the set-top box 200
(receiver), and causes the subtitle to overlap the
background image (3D video). For example, the subtitle
15 is positioned at a monitor .position without causing
disparity to occur in the subtitle. Further, for example,
fixed disparity is brought to occur in the subtitle, and
the subtitle is positioned at the position ahead of the
monitor position.
,20 [0329]
"Others"
In the set-top box 200 illustrated in Fig. 55, the
antenna input terminal 203 connected to the digital tuner
204 is,disposed. However, a set-top box that receives an
25 RF signal transmitted through a cable can be also
similarly configured. In this case, a cable terminal is
disposed instead of the antenna terminal 203.
[0330]
Further, a set-top box connected to the Internet or
30 the home network directly or via a router can be also
similarly configured. In other words, in this case, the
120
SP338522WO00
transport stream TS is transmitted to the set-top box
from the Internet or the home network directly or via the
router.
[0331]
5 Fig. 7 6 illustrates a configuration example of the
set-top box 200A in this case. In Fig. 76, components
corresponding to those of Fig. 55 are denoted by the same
reference numerals. The set-top box 200A includes a •
network terminal 208 connected to a network interface 209.
10 Further, the transport stream TS from the network
interface 2 09 is output and supplied to the bit stream
processing unit 201. Although a detailed description
will not be made, the remaining components of the set-top
box 200A are similar in configuration and operation to
15 the set-top box 200 illustrated in Fig. 55.
[0332]
Further, in the television receiver 300 illustrated
in Fig. 58, the antenna input terminal 304 connected to
the digital tuner 204 is disposed. However, a television
2 0 receiver that receives an RF signal transmitted through a
cable can be also similarly configured. In this case, a
cable terminal is disposed instead of the antenna
terminal 304.
[0333]
25 Further, a television receiver connected to the
Internet or the home network directly or via a router can
be also similarly configured. In other words, in this
case, the transport stream TS is transmitted to the
television receiver from the Internet or the home network
30 directly or via the router.
[0334]
121
SP338522WO00
Fig. 77 illustrates a configuration example of the
television receiver 300A in this case. In Fig. 77,
components corresponding to those of Fig. 58 are denoted
by the same reference numerals. The television receiver
5 300A includes a network terminal 313 connected to a
-'network interface 314. Further, the transport stream TS
from the network interface 314 is output and supplied to
the bit stream processing unit 306. Although a detailed
description will not be made, the remaining components of
10 the television receiver 300A are similar in configuration
and operation to the television receiver 300 illustrated
in Fig. 58.
[0335]
In the above embodiment, the image transceiving
15 system 10 is configured to include the broadcasting
station 100, the set-top box 200, and the television
receiver 300. However, the television receiver 300
includes the bit stream processing unit 306 that performs
the same function as the bit stream processing unit 201
20 of the set-top box 200 as illustrated in Fig. 58. Thus,
an image transceiving system lOA may be configured with
the broadcasting station 100 and the television receiver
300 as illustrated in Fig. 78. Although a detailed
description will not be made, the television receiver 300
25 performs the 3D service determining process (Figs. 49 and
50) and the reception process (Figs. 51, 62, 66, and 75)
when it is determined•the service is the 3D service,
similarly to the set-top box 200.
[0336] '
30 Further, the above embodiment has been described in
connection with the example in which the component
122
SP338522WO00
descriptor is present (see Fig. 15) . However, when the
EPG is not used, the EIT is not present, and the
component descriptor is not present. The present
technology is implemented even when the component
5 descriptor is not present. In other words, present
technology can be implemented by the subtitle descriptor
(Subtitle_descriptor), the stream association ID
descriptor (stream_association_ID descriptor), and the
like which are arranged in the PMT.
10 [0337]
Further, the above embodiment has been described in
connection with the example in which the segments of the
DDS, the DSS, and the EDS or the segments of the DDS, the
PCS, the DSS, and the EDS are included in the 3D
15 extension stream (see Figs. 11(b) and 11(c)). However,
the segment configuration of the 3D extension stream is
not limited to this example and may include any other
segment. In this case, the segments of the DDS, the PCS,
the RCS, the CDS, the CDS, the DSS, and the EDS at a
20 maximum are included in the 3D extension stream.
[0338]
Further, the above embodiment has been described in
connection with the example in which the set-top box 200
is connected with the television receiver 300 through the
25 digital interface of the HDMI. However, the invention
can be similarly applied even when the set-top box 200 is
connected with the television receiver 300 via a digital
interface (including a wireless interface as well as a
wired interface) that performs the same function as the
30 digital interface of the HDMI.
[0339]
123
SP338522WO00
Furthermore, the above embodiment has been
described in connection with the example in which the
subtitle is dealt as the overlapping information.
However, the invention can be similarly applied even when
5 starting from overlapping information such as graphics
information or text information, information encoded such
that streams divided into a basic stream and an
additional stream are output in association with each
other is dealt with in connection with an audio stream.
10 [0340]
Further, the present technology may have the
following configurations.
(1) A transmission device, including:
an image data output unit that outputs left-eye
15 image data and right-eye image data configuring a
stereoscopic image;
' an overlapping information data output unit that
outputs data of overlapping information overlapping an
image based on the left-eye image data and the right-eye
2 0 image data;
a disparity information output unit that outputs
disparity information for shifting the overlapping
information overlapping the image based on the left-eye
image data and the right-eye image data and causing
25 disparity to occur; and
a data transmitting unit that transmits multiplexed
data stream including a video data stream including the
image data, a first private data stream including the
data of the overlapping information, and a second private
30 data stream including the disparity information,
wherein association information associating the
124
SP338522WO00
first private data stream with the second private data
stream is included in the multiplexed data stream.
(2) The transmission device according to (1),
wherein identification information, which is common
5 to a first descriptor describing information related to
the first private data stream and a second descriptor
describing information related to the second private data
stream, is described as the association information.
(3) The transmission device according to (2),
10 wherein the common identification information is
defined by a special value representing that the first
private data stream and the second private data stream
are present.
(4) The transmission device according to (1) or (2),
15 wherein the multiplexed data stream includes the
first private data stream and the second private data
stream corresponding to each of a plurality of language
services, and
the pieces of common identification information
20 corresponding to the respective language services, are set
to be different from each other.
(5) The transmission device according to any one of
(2) to (4),
wherein the data of the overlapping information is
2 5 subtitle data of a DVB format, and
a common composition page ID is described in a
first subtitle descriptor corresponding to the first
private data stream and a subtitle descriptor
corresponding to the second private data stream.
, 30 , (6) The transmis'sion device according to any one of
(2) to (5),
125
SP338522WO00
wherein linguistic information is described in the
first descriptor and the second descriptor, and
the linguistic information described in the second
descriptor is set to represent a non-language.
5 (7) The transmission device according to (6),
wherein the linguistic information representing the
non-language is any one of language codes included in a
space of "zxx" or "qaa" to "qrz" representing an ISO
language code.
10 (8) The transmission device according to (1),
wherein identification information, which is common
to a first descriptor describing information related to
the first private data stream and a second descriptor
describing information related to the second private data .
15 stream, is described as the association information, and
type information representing information for a
stereoscopic image display is described in the first
descriptor and the second descriptor.
(9) The transmission device according to (8),
20 wherein the data of the overlapping information is
subtitle data of a DVB format, and
the type information is subtitle type information.
(10) The transmission device according to (8) or
(9),
25 wherein linguistic information is described in the
first descriptor and the second descriptor, and
the linguistic information described in the second
descriptor is set to represent a non-language.
(11) The transmission device according to (10),
30 wherein the linguistic information representing the,'
non-language is any one of language codes included in a
126
SP338522WO00
space of "zxx" or "qaa" to "qrz" representing an ISO
language code.
(12) The transmission device according to (1),
wherein a descriptor describing dedicated linking
5 information for.linking the first private data stream
with the second private data stream is included in the
multiplexed data stream.
(13) The transmission' device according to (12),
wherein the descriptor is a dedicated descriptor
10 describing the dedicated linking information.
(14) A transmission method, including the steps of:
outputting left-eye image data and right-eye image
data configuring a stereoscopic image;
outputting data of overlapping information
15 overlapping an image based on the left-eye image data and
the right-eye image data;
outputting disparity information for shifting the
overlapping information overlapping the image based on
the left-eye image data and the right-eye image data and
20 causing disparity to occur; and
transmitting multiplexed data stream including a
video data stream including the image data, a first
private data stream including the data of the overlapping
information, and a second private data stream including
25 the disparity information,
wherein association information associating the
first private data stream with the second private data
stream is included in the multiplexed data stream.
(15) A reception device, including:
30 a data receiving unit that receives multiplexed
data stream including a video data stream including left-
127
SP338522WO00
eye image data and right-eye image data configuring a
stereoscopic image, a first private data stream including
data of overlapping information overlapping an image
based on the left-eye image data and the right-eye image
5 data, and a second private data stream including
disparity information for shifting the overlapping
information overlapping the image based on the left-eye
image data and the right-eye image data and causing
disparity to occur;
10 a first decoding unit that extracts the video data
stream from the multiplexed data stream and decodes the
video data stream; and
a second decoding unit that extracts the first
private data stream and the second private data stream
15 from the multiplexed data stream and decodes the first
private data stream and the second private data stream,
wherein association information associating the
first private data stream with the second private data
stream is included in the multiplexed data stream, and
20 the second decoding unit extracts the first private
data stream and the second private data stream from the
multiplexed data stream based on the association
information.
[0341]
25 The main feature of the present technology is that
when the 2D stream including the 2D segments (the
segments of the DDS, the PCS, the RCS, the CDS, the ODS,
and the EDS) and the 3D extension stream including the 3D
extension segment (the DSS segment) separately from the
30 2D stream are transmitted (see Fig. 11), the association
information (common composition_page_id or the like) for
128
'••\ "~'"^.
SP338522WO00
associating the two streams with each other is included
in the transport stream TS (see Fig. 15), and thus the
reception device with the 3D function can efficiently and
appropriately extract and decode the two streams.
: 5
REFERENCE SIGNS LIST
[0342]
10, lOA Image transceiving system
100 Broadcasting station
10 111 Data extracting unit
Ilia Data recording medium
112 Video encdder
113 Audio encoder
114 Subtitle generating unit
15- 115 Disparity information creating unit
116 Subtitle processing unit
118 Subtitle encoder
119 Multiplexer
200, 200A Set-top box (STB)
20 . 201 Bit stream processing unit
202 HDMI terminal
2 03 Antenna terminal
204 Digital tuner
205 Video signal processing circuit
25 205 HDMI transmission unit
207 Audio signal processing circuit
2 08 Network terminal
209 Network interface
211 CPU
30 215 Remote control receiving unit
216 Remote control transmitter
129
SP338522WO00
221 Demultiplexer
222 Video decoder ^
223 Encoded data buffer
224 Subtitle decoder
5 225 Pixel buffer
22 6 Disparity information interpolating unit
227 Position control unit
22 8 Video overlapping unit
229 Audio decoder
10 300, 300A Television receiver (TV)
301 3D Signal processing unit
3 02 HDMI terminal
303 HDMI reception unit
304 Antenna terminal
15 305 Digital tuner
306 Bit stream processing unit
307 Video/graphics processing circuit
308 Panel driving circuit
309 'Display panel
20 310 Audio signal processing circuit
311 Audio amplifying circuit
312 Speaker
130

SP338522WO00
CLAIMS
1. A transmission device, comprising:
an image data output unit that outputs left-eye
5 image data and right-eye image data configuring a
stereoscopic image;'
an overlapping information data output unit that
outputs data of overlapping information overlapping an
image based on the left-eye image data and the right-eye
10 image data;
a disparity information output unit that outputs
disparity information for shifting the overlapping
information overlapping the image based on the left-eye
image data and the right-eye image data and causing
15 disparity to occur; and
a data transmitting unit that transmits multiplexed
data stream including a video data stream including the
image data, a first private data stream including the
data of the overlapping information, and a second private ^
20 data stream including the disparity information,
wherein association information associating the
first private data stream with the second private data
stream is included in the multiplexed data stream.
25 2. The transmission device according to claim 1,
wherein identification information, which is common
to a first descriptor describing information related to
the first private data stream and a second descriptor
describing information related to the second private data
3'0 stream, is described as the association information.
131
SP338522WO00
3. The transmission device according to claim 2,
wherein the common identification information is
defined by a special value representing that the first
private data stream and the second private data stream
5 are present.
4. The transmission device according to claim 2,
wherein the multiplexed data stream includes the
first private data stream and the second private data
10 stream corresponding to each of a plurality of language
services, and
the pieces of common identification information
corresponding to the respective language services are set
to be different from each other.
15 '
5. The transmission device according to claim 2,
wherein the data of the overlapping information is
subtitle data of a DVB format, and
a common composition page ID is described in a
20 first subtitle descriptor corresponding to the first
private data stream and a subtitle descriptor
corresponding to the second private data stream.
6. The transmission device according to claim 2,
25 wherein linguistic information is described in the
•first descriptor and the second descriptor, and
the linguistic information described in the second
descriptor is set to represent a non-language.
30 7. The transmission device according to claim 6,
wherein the linguistic information i^ep'resenting the
132
SP338522WO00
non-language is any one of language codes included in a
space of "'zxx" or "qaa" to "qrz" representing an ISO
language code.
5 8. The transmission device according to claim 1,
wherein identification information, which is common
to a first descriptor describing information related to
the first private data stream and a second descriptor
describing information related to the second private data
10 stream, is described as the association information, and
type information representing information for a
stereoscopic image display is described in the first
descriptor and the second descriptor.
15 9. The transmission device according to claim 8,
wherein the data of the overlapping information is
subtitle data of a DVB format, and
the type information is subtitle type information.
2 0 10. The transmission device according to claim 8,
wherein linguistic information is described in the
first descriptor and the second descriptor, and
the linguistic information described in the second
descriptor is set to represent a non-language.
25
11. The transmission device according to claim 10,
wherein the linguistic information representing the
non-language is any one of language codes included in a
space of "zxx" or "qaa" to "qrz" representing an ISO
30 language code.
133
SP338522WO00
12. The transmission device according to claim 1,
wherein a descriptor describing dedicated linking
information for linking the first private data stream
with the second private data stream is included in the
5 multiplexed data stream.
13. The transmission device according to claim 12,
wherein the descriptor is a dedicated descriptor
describing the dedicated linking information.
10
14. A transmission method, comprising the steps .of:
outputting left-eye image data and right-eye image
data configuring a stereoscopic image;
outputting data of overlapping information
15 overlapping an image based on the left-eye image data and
the right-eye image data;
outputting disparity information for shifting the
overlapping information overlapping the image based on
the left-eye image data and the right-eye image data and
2 0 causing disparity to occur; and
transmitting multiplexed data stream including a
video data stream including the image data, a first
private data stream including the data of the overlapping
information, and a second private data stream including
25 the disparity information,
wherein association information associating the
first private data stream with the second private data
stream is included in the multiplexed data stream.
3,0 ,15. A reception device, comprising:
a data receiving unit that receives multiplexed
134
SP338522WO00
data stream including a video data stream including lefteye
image .data and right-eye image data configuring a
stereoscopic image, a first private data stream including
data of overlapping information overlapping an image
5 based on the left-eye image data and the right-eye image
data, and a second private data stream including
disparity information for shifting the overlapping
information overlapping the image based on the left-eye
image data and the right-eye image data and causing
10 disparity to occur;
a first decoding unit that extracts the video data
stream from the multiplexed data stream and decodes the
- video data stream; and
a second decoding unit that extracts the first
15 private data stream and the second private data stream
from the multiplexed data stream and decodes the first
private data stream and the second private data stream,
wherein association information associating the
first private data stream with the second private data
20 stream is included in the multiplexed data stream, and
the second decoding unit extracts the first private
data stream and the secohd private data stream from the
multiplexed data stream based on the association
information.