DESCRIPTION
[Title of Invention]
RECORDING MEDIUM, PLAYBACK DEVICE, AND INTEGRATED CIRCUIT
[Technical Field]
[0001]
The present invention relates to a technology for stereoscopic, i.e.
three-dimensional (3D), video playback and especially to the structure of stream data on
a recording medium.
[Background Art]
[0002]
In recent years, general interest in 3D video has been increasing. For example,
amusement park attractions that incorporate 3D video images are popular. Furthermore,
throughout the country, the number of movie theaters showing 3D movies is increasing.
Along with this increased interest in 3D video, the development of technology that
enables playback of 3D video images in the home has also been progressing. There is
demand for this technology to store 3D video content on a portable recording medium,
such as an optical disc, while maintaining the 3D video content at high image quality.
Furthermore, there is demand for the recording medium to be compatible with a
two-dimensional (2D) playback device. That is, it is preferable for a 2D playback device
to be able to play back 2D video images and a 3D playback device to be able to play
back 3D video images from the same 3D video content recorded on the recording
medium. Here, a "2D playback device" refers to a conventional playback device that
can only play back monoscopic video images, i.e. 2D video images, whereas a "3D
playback device" refers to a playback device that can play back 3D video images. Note
that in the present description, a 3D playback device is assumed to be able to also play
back conventional 2D video images.
Fig. 109 is a schematic diagram illustrating the technology for ensuring the
compatibility of an optical disc storing 3D video content with 2D playback devices (see,
for example, Patent Document 1). An optical disc PDS stores two types of video
streams. One is a 2D/left-view video stream, and the other is a right-view video stream.
A "2D/left-view video stream" represents a 2D video image to be shown to the left eye
of a viewer during 3D playback, i.e. a "left view". During 2D playback, this stream
constitutes the 2D video image. A "right-view video stream" represents a 2D video
image to be shown to the right eye of the viewer during 3D playback, i.e. a "right view".
The left and right-view video streams have the same frame rate but different
presentation times shifted from each other by half a frame period. For example, when
the frame rate of each video stream is 24 frames per second, the frames of the
2D/left-view video stream and the right-view video stream are alternately displayed
every 1/48 seconds.
[0004]
As shown in Fig. 109, the left-view and right-view video streams are divided
into a plurality of extents EX1A-C and EX2A-C respectively on the optical disc PDS.
Each extent contains at least one group of pictures (GOP), GOPs being read together by
the optical disc drive. Hereinafter, the extents belonging to the 2D/left-view video
stream are referred to as "2D/left-view extents", and the extents belonging to the
right-view video stream are referred to as "right-view extents". The 2D/left-view extents
EX1A-C and the right-view extents EX2A-C are alternately arranged on a track TRC of
the optical disc PDS. Each two contiguous extents EX1A + EX2A, EX1B + EX2B, and
EX1C + EX2C have the same length of playback time. Such an arrangement of extents
is referred to as an "interleaved arrangement". A group of extents recorded in an
interleaved arrangement on a recording medium is used both in 3D video playback and
2D video image playback, as described below.
[0005]
From among the extents recorded on the optical disc PDS, a 2D playback
device PL2 causes an optical disc drive DD2 to read only the 2DAeft-view extents
EX1A-C sequentially from the start, skipping the reading of right-view extents EX2A-C.
Furthermore, an image decoder VDC sequentially decodes the extents read by the
optical disc drive DD2 into a video frame VFL. In this way, a display device DS2 only
displays left views, and viewers can watch normal 2D video images.
[0006]
A 3D playback device PL3 causes an optical disc drive DD3 to alternately read
2D/left-view extents and right-view extents from the optical disc PDS. When expressed
as codes, the extents are read in the order EX1A, EX2A, EX1B, EX2B, EX1C, and
EX2C. Furthermore, from among the read extents, those belonging to the 2D/left-view
video stream are supplied to a left-video decoder VDL, whereas those belonging to the
right-view video stream are supplied to a right-video decoder VDR. The video decoders
VDL and VDR alternately decode each video stream into video frames VFL and VFR,
respectively. As a result, left views and right views are alternately displayed on a
display device DS3. In synchronization with the switching of the views by the display
device DS3, shutter glasses SHG cause the left and right lenses to become opaque
alternately. Therefore, a viewer wearing the shutter glasses SHG sees the views
displayed by the display device DS3 as 3D video images.
[0007]
When 3D video content is stored on any recording medium, not only on an
optical disc, the above-described interleaved arrangement of extents is used. The
recording medium can thus be used both for playback of 2D video images and 3D video
images.
[Citation List]
[Patent Literature]
[0008]
[Patent Literature 1]
Japanese Patent Publication No. 3935507
[Summary of Invention]
[Technical Problem]
[0009]
General video content includes, in addition to a video stream, one or more
graphics streams representing graphics images such as subtitles and interactive screens.
When video images are played back from 3D video content, the graphics images are
also reproduced in three dimensions. Techniques of reproducing them in three
dimensions include 2 plane mode and 1 plane + offset mode. 3D video content in 2
plane mode includes a pair of graphics streams separately representing left-view
graphics images and right-view ones. A playback device in 2 plane mode generates a
separate left-view and right-view graphics plane from the graphics streams. 3D video
content in 1 plane + offset mode includes a graphics stream representing 2D graphics
images, and offset information provided for the graphics stream. A playback device in
1 plane + offset mode first generates a single graphics plane from the graphics stream,
and then provides a horizontal offset in the graphics plane in accordance with the offset
information. A pair of left-view and right-view graphics planes is thus generated from
the graphics stream. In either mode, left-view and right-view graphics images are
alternately displayed on the screen of the display device. As a result, viewers perceive
the graphics images as 3D images.
[0010]
If the graphics stream and the offset information are contained in separate files
of 3D video content, the playback device in 1 plane + offset mode processes these files
separately into corresponding data pieces, and use the data pieces to generate a pair of
left-view and right-view graphics images. Note that the graphics images and offset
information are generally changed in frame periods. However, reading and analyzing
the file storing the offset information each time a frame is displayed has a risk that "the
process is not completed in time and the images cannot be displayed correctly".
Accordingly, in order for the process to be synchronized with the frame period without
fail, it is necessary to expand the offset information in the memory in advance. In that
case, the capacity of the built-in memory in which the file storing the offset information
is to be expanded should necessarily be large because the total amount of the offset
information per graphics stream is large. Also, when a plurality of graphics images are
included in one scene, the built-in memory is required to have even larger capacity. In
this way, incorporating the graphics stream and the offset information as separate files
into 3D video content prevents a further reduction in capacity of the built-in memory.
[0011]
In order to solve the above-described problem, the offset information is
contained in the video stream at intervals of GOPs, for example. This allows a
decoder in a playback device to extract the offset information from the video stream
while decoding the video stream. As a result, the playback device can surely maintain
the correspondence between the graphics stream and the offset information. In addition,
the built-in memory only needs to have a capacity sufficient to expand the offset
information per GOP therein, for example. This can easily achieve both the support of
3D video contents with various graphics streams and the further reduction in capacity of
the built-in memory.
[0012]
Here, various means are conceivable as specific means used by the decoder in a
playback device to implement the function to extract offset information from video
streams, such as a means for incorporating the function into the hardware dedicated to
decoding of video streams, and a means for realizing the function by another hardware
or software. However, it is not preferable to vary the data structures of video streams
and offset information among those means.
[0013]
An object of the present invention is to solve the above problems, particularly
to provide a recording medium in which a video stream and offset information are
integrally recorded in a data structure usable in common in various modes of
implementing the function, which is to extract the offset information from the video
stream, into a playback device.
[Solution to Problem]
[0014]
On a recording medium according to the present invention, a main-view video
stream, a sub-view video stream, and a graphics stream are recorded. The main-view
video stream includes main-view pictures constituting main views of stereoscopic video
images. The sub-view video stream includes sub-view pictures and metadata, the
sub-view pictures constituting sub-views of stereoscopic video images. The graphics
stream includes graphics data constituting monoscopic graphics images. The
main-view pictures are each rendered on a main-view video plane, when being played
back. The sub-view pictures are each rendered on a sub-view video plane, when being
played back. The graphics data is rendered on a graphics plane, when being played
back. The metadata is provided in each group of pictures (GOP) constituting the
sub-view video stream and includes offset information. The offset information is
control information specifying offset control for a plurality of pictures constituting a
GOP. The offset control is a process to provide a left offset and a right offset for
horizontal coordinates in the graphics plane to generate a pair of graphics planes, and
then combine the pair of graphics planes separately with the main-view video plane and
the sub-view video plane. The sub-view video stream is multiplexed in a transport
stream (TS). TS packets constituting the TS each have a header including a TS
priority flag that indicates a priority of the TS packet. TS packets containing the
metadata have a different value of TS priority flag from TS packets containing the
sub-view pictures.
[Advantageous Effects of Invention]
[0015]
The recording medium according to the present invention enables the decoding
unit of a playback device to separate TS packets containing the metadata and TS packets
containing the sub-view pictures in accordance with the values of the TS priority flags.
Accordingly, the decoding unit can be equipped with separate function units; one for
extracting the offset information from TS packets containing the metadata, and the other
for decoding TS packets containing the sub-view pictures into uncompressed pictures.
In this case, specific configurations of these function units can be designed
independently of each other. On the other hand, the decoding unit in which the
function units are integrated enables the integrated function unit to process all the TS
packets containing the sub-view video stream, independently of the values of the TS
priority flags. In this way, the recording medium according to the present invention
enables a video stream and offset information to be integrally recorded therein in a data
structure usable in common in various modes of implementing the function, which is to
extract the offset information from the video stream, into a playback device.
[Brief Description of Drawings]
[0016]
Fig. 1 is a schematic diagram showing a home theater system that uses a
recording medium according to Embodiment 1 of the present invention.
Fig. 2 is a schematic diagram showing a data structure of a BD-ROM disc 101
shown in Fig. 1.
Figs. 3 A and 3B are lists of elementary streams multiplexed in a main TS and a
sub-TS on the BD-ROM disc, respectively.
Fig. 4 is a schematic diagram showing the arrangement of TS packets in the
multiplexed stream data 400.
Fig. 5A is a schematic diagram showing a data structure of a TS header 501H;
Fig. 5B is a schematic diagram showing the format of a sequence of TS packets 501
constituting multiplexed stream data; Fig. 5C is a schematic diagram showing the
formation of a sequence of source packets 502 composed of the TS packet sequence for
multiplexed stream data; and Fig. 5D is a schematic diagram of a sector group, in which
a sequence of source packets 502 are consecutively recorded, in the volume area of the
BD-ROM disc.
Fig. 6 is a schematic diagram showing the data structure of the PG stream 600.
Fig. 7 is a schematic diagram showing the pictures for a base-view video
stream 701 and a right-view video stream 702 in order of presentation time.
Fig. 8 is a schematic diagram showing details on a data structure of a video
stream 800.
Fig. 9 is a schematic diagram showing details on a method for storing a video
stream 901 into a PES packet sequence 902.
Fig. 10 is a schematic diagram showing correspondence between PTSs and
DTSs assigned to each picture in a base-view video stream 1001 and a dependent-view
video stream 1002.
Fig. 11 is a schematic diagram showing a data structure of offset metadata
1110 included in a dependent-view video stream 1100.
Fig. 12 is a table showing syntax of this offset metadata 1110 shown in Fig. 11.
Figs. 13A and 13B are schematic diagrams showing offset controls for a PG
plane 1310 and IG plane 1320 respectively; and Fig. 13C is a schematic diagram
showing 3D graphics images that a viewer 1330 is made to perceive from 2D graphics
images represented by graphics planes shown in Figs. 13 A and 13B.
Figs. 14A and 14B are graphs showing examples of offset sequences; and Fig.
14C is a schematic diagram showing 3D graphics images reproduced in accordance with
the offset sequences shown in Figs. 14A and 14B.
Fig. 15 is a schematic diagram showing a PES packet 1510 storing VAU #1
1500 in the dependent-view video stream, and a sequence of TS packets 1520 generated
from the PES packet 1510.
Fig. 16 is a schematic diagram showing a sequence of TS packets 1620 where
TS packets belonging to the first group 1521 and the second group 1522 shown in Fig.
15 indicate the same value of TS priority.
Fig. 17A is a schematic diagram showing a data structure of decoding switch
information 1750; and Figs. 17B and 17C are schematic diagrams showing sequences of
decoding counters 1710, 1720, 1730, and 1740 allocated to each picture in a base-view
video stream 1701 and a dependent-view video stream 1702.
Fig. 18 is a schematic diagram showing a data structure of a PMT 1810.
Fig. 19 is a schematic diagram showing a physical arrangement of multiplexed
stream data on the BD-ROM disc.
Fig. 20 A is a schematic diagram showing the arrangement of the main TS 2001
and sub-TS 2002 recorded separately and consecutively on a BD-ROM disc; Fig. 20B is
a schematic diagram showing an arrangement of dependent-view data blocks D[0], D[l],
D[2], ... and base-view data blocks B[0], B[l], B[2], ... recorded alternately on the
BD-ROM disc 101 according to Embodiment 1 of the present invention; and Figs. 20C
and 20D are schematic diagrams showing examples of the extent ATC times for a
dependent-view data block group D[n] and a base-view data block group B[n] recorded
in an interleaved arrangement (n = 0, 1, 2).
Fig. 21 is a schematic diagram showing a playback path 2101, 2102 in 2D
playback mode and L/R mode for an extent block group 1901-1903.
Fig. 22 is a schematic diagram showing a data structure of 2D clip information
file(01000.clpi)231.
Fig. 23A is a schematic diagram showing a data structure of an entry map
2230; Fig. 23B is a schematic diagram showing source packets in a source packet group
2310 belonging to a file 2D 241 that are associated with each EP_ID 2305 by the entry
map 2230; and Fig. 23C is a schematic diagram showing a data block group D[n], B[n]
(n = 0, 1, 2, 3, ...) on a BD-ROM disc 101 corresponding to the source packet group
2310.
Fig. 24A is a schematic diagram showing a data structure of extent start points
2242; Fig. 24B is a schematic diagram showing a data structure of extent start points
2420 included in dependent-view clip information file (02000.clpi) 232; Fig. 24C is a
schematic diagram representing the base-view data blocks B[0], B[l], B[2],... extracted
from the file SS 244A by the playback device 102 in 3D playback mode; Fig. 24D is a
schematic diagram representing correspondence between dependent-view extents
EXT2[0], EXT2[1], ... belonging to the file DEP (02000.m2ts) 242 and the SPNs 2422
shown by the extent start points 2420; and Fig. 24E is a schematic diagram showing
correspondence between an extent SS EXTSS[0] belonging to the file SS 244A and an
extent block on the BD-ROM disc.
Fig. 25 is a schematic diagram showing correspondence between a single
extent block 2500 recorded on the BD-ROM disc and each of the extent block groups in
a file 2D 2510, file base 2511, file DEP 2512, and file SS 2520.
Fig. 26 is a schematic diagram showing an example of entry points set in a
base-view video stream 2610 and a dependent-view video stream 2620.
Fig. 27 is a schematic diagram showing a data structure of a 2D playlist file.
Fig. 28 is a schematic diagram showing a data structure of PI #N shown in Fig.
27.
Figs. 29A and 29B are schematic diagrams showing a correspondence between
two playback sections 2901 and 2902 to be connected when CC is "5" or "6".
Fig. 30 is a schematic diagram showing a correspondence between PTSs
indicated by a 2D playlist file (OOOOl.mpls) 221 and sections played back from a file 2D
(O1000.m2ts)241.
Fig. 31 is a schematic diagram showing a data structure of a 3D playlist file.
Fig. 32 is a schematic diagram showing an STN table 3205 included in a main
path 3101 of the 3D playlist file shown in Fig. 31.
Fig. 33 is a schematic diagram showing a data structure of the STN table SS
3130 shown in Fig. 31.
Fig. 34 is a schematic diagram showing correspondence between PTSs
indicated by a 3D playlist file (00002.mpls) 222 and sections played back from a file SS
(01000.ssif)244A.
Fig. 35 is a schematic diagram showing a data structure of an index file
(index.bdmv) 211 shown in Fig. 2.
Fig. 36 is a flowchart of processing whereby the playback device 102 shown in
Fig. 1 selects a playlist file for playback by using six types of determination processes.
Fig. 37 is a functional block diagram of a 2D playback device 3700.
Fig. 38 is a list of system parameters (SPRMs) stored in the player variable
storage unit 3736 shown in Fig. 37.
Fig. 39 is a flowchart of 2D playlist playback processing by a playback control
unit 3735 shown in Fig. 37.
Fig. 40 is a functional block diagram of the system target decoder 3725 shown
in Fig. 37.
Fig. 41A is a flowchart of processing whereby the PG decoder 4072 shown in
Fig. 40 decodes a graphics object from one data entry in the PG stream; and Figs.
41B-41E are schematic diagrams showing the graphics object changing as the
processing proceeds.
Fig. 42 is a functional block diagram of a 3D playback device 4200.
Fig. 43 is a table showing a data structure of SPRM(27) and SPRM(28) stored
in the player variable storage unit 4236 shown in Fig. 42.
Fig. 44 is a flowchart of 3D playlist playback processing by a playback control
unit 4235 shown in Fig. 42.
Fig. 45 is a functional block diagram of the system target decoder 4225 shown
in Fig. 42 that implements the function to extract offset metadata by using a first means.
Fig. 46 is a functional block diagram of a system of processing video streams
in the system target decoder 4225 shown in Fig. 42 that implements the function to
extract offset metadata by using a second means.
Fig. 47 is a functional block diagram of a plane adder 4226 shown in Fig. 42.
Fig. 48 is a flowchart of offset control by the cropping units 4731-4734 shown
in Fig. 47.
Fig. 49B is a schematic diagram showing PG plane data GP to which the
second cropping unit 4732 is to provide offset control; and Figs. 49A and 49C are a
schematic diagrams showing PG plane data RPG to which a right offset has been
provided and PG plane data LPG to which a left offset has been provided.
Fig. 50 is a schematic diagram showing a PES packet 5010 storing VAU #1
5000 in the dependent-view video stream and a sequence of TS packets 5020 generated
from the PES packet 5010.
Fig. 51 is a functional block diagram showing a system of processing video
streams in the system target decoder 5125 that extracts offset metadata from the TS
packet sequence 5020 shown in Fig. 50.
Fig. 52A is a schematic diagram showing a data structure of offset metadata
5200 that uses a completion function; Fig. 52B is a graph showing the types of elements
in the completion function; and Fig. 52C is a graph showing offset values calculated by
a 3D playback device from offset sequence IDs = 0, 1,2 shown in Fig. 52A.
Fig. 53 is a schematic diagram showing (i) a data structure of a 3D playlist file
5300 that includes a plurality of sub-paths and (ii) a data structure of a file 2D 5310 and
two files DEP 5321 and 5322 that are referred to by the 3D playlist file 5300.
Fig. 54 is a schematic diagram showing an STN table 5400 in which two or
more offset adjustment values are set for one piece of stream data.
Figs. 55A-55C are schematic diagrams showing parallaxes PRA, PRB, and
PRC between left and right views displayed on a 32-inch screen SCA, 50-inch screen
SCB, and 100-inch screen SCC, respectively.
Fig. 56A is a schematic diagram showing a correspondence table between
screen sizes and output offset adjustment values; and Fig. 56B is a graph representing a
function between screen sizes and output offset adjustment values.
Fig. 57 is a block diagram showing the components of a 3D playback device
required for output offset adjustment.
Fig. 58A is a schematic diagram showing a data structure of a dependent-view
video stream 5800 representing only still images; and Fig. 58B is a schematic diagram
showing a left-view video plane sequence 5821, a right-view video plane sequence 5822,
and a graphics plane sequence 5830 that are played back in accordance with such a 3D
playlist.
Fig. 59 is a block diagram of the display device 103 that performs the
processing for compensating a misalignment between left and right views.
Fig. 60A is a plan view schematically showing horizontal angles of view FIAL
and HAR for a pair of video cameras CML and CMR filming 3D video images; Figs.
60B and 60C are schematic diagrams showing a left view LV filmed by the left-video
camera CML and a right view RV captured by the right-video camera CMR,
respectively; and Figs. 60D and 60E are schematic diagrams respectively showing a left
view LV represented by the processed left-video plane and a right view RV represented
by the processed right-video plane.
Fig. 61A is a plan view schematically showing vertical angles of view VAL
and VAR for a pair of video cameras CML and CMR filming 3D video images; Fig.
61B is a schematic diagram showing a left view LV filmed by the left-video camera
CML and a right view RV captured by the right-video camera CMR; and Fig. 61C is a
schematic diagram showing a left view LV represented by the processed left-video
plane and a right view RV represented by the processed right-video plane.
Fig. 62A is a schematic diagram showing an example of graphics images
represented by a graphics plane GPL; Figs. 62B and 62C are schematic diagrams
respectively showing processes of providing a right and left offset to the graphics plane
GPL; and Figs. 62D and 62E are schematic diagrams showing graphics images
represented by the graphics planes GP1 and GP2 with the right and left offsets,
respectively.
Fig. 63 is a schematic diagram showing a condition regarding the arrangement
of graphic elements for graphics planes played back from a PG stream or an IG stream
on a BD-ROM disc and for a graphics plane generated by a playback device 102.
Figs. 64A1 and 64A2 are schematic diagrams showing the same screen in the
letterbox display; Figs. 64B and 64C are schematic diagrams showing the screens in
which the primary video plane has been provided with upward and downward offsets of
131 pixels, respectively; and Fig. 64D is a schematic diagram showing the screen in
which the primary video plane has been provided with an upward offset of 51 pixels.
Fig. 65 is a functional block diagram showing the structure of the playback
device required for the video shift.
Fig. 66A is a table showing the data structures of the SPRM(32) and
SPRM(33); and Fig. 66B is a schematic diagram showing the STN table in the playlist
file for the video content of the letterbox display.
Figs. 67A-67C are schematic diagrams showing primary video planes VPA,
VPB, and VPC processed by the video shift unit 6501 in the Up mode, Keep mode, and
Down mode, respectively; Figs. 67D-67F are schematic diagrams showing PG planes
PGD, PGE, and PGF processed by the second cropping unit 4632 in the Up mode, Keep
mode, and Down mode, respectively; and Figs. 67G-67I are schematic diagrams
showing plane data PLG, PLH, and PLI combined by the second adder 4642 in the Up
mode, Keep mode, and Down mode, respectively.
Fig. 68A is a schematic diagram showing another example of the STN table in
the playlist file for the video content of the letterbox display; and Fig. 68B is a
schematic diagram showing the order of registration of a plurality of pieces of stream
attribute information 6803, each including the video shift mode 6812, in the STN table
shown in Fig. 68A.
Fig. 69 is a functional block diagram showing another example of the structure
of the playback device required for the video shift.
Fig. 70 A is a schematic diagram showing the data structure of the SPRM(37) in
the player variable storage unit 4236; Fig. 70B is a schematic diagram showing the
video image IMG and subtitle SUB displayed on the screen SCR in the case where the
background color of the subtitle represented by the PG stream is set to the colorless
transparent; and Fig. 70C is a schematic diagram showing the video image IMG and
subtitle SUB displayed on the screen SCR in the case where a color coordinate value of
the background color of the subtitle is stored in the SPRM(37).
Fig. 71A is a schematic diagram showing a further another example of the STN
table in the playlist file for the video content of the letterbox display; and Fig. 71B is a
functional block diagram showing a further another example of the structure of the
playback device required for the video shift.
Fig. 72A is a schematic diagram showing the subtitles SB1 and SB2 that
correspond to the Keep mode; Fig. 72B is a schematic diagram showing the subtitles
SB1 and SB2 that correspond to the Down mode; Fig. 72C is a schematic diagram
showing the subtitle SB1 displayed in the Keep mode; and Fig. 72D is a schematic
diagram showing the subtitle SB 3 displayed in the Up mode when the video upward
subtitle 7110 is not registered in the STN table.
Figs. 73 A and 73B are lists of elementary streams multiplexed in a first sub-TS
and a second sub-TS on a BD-ROM disc, respectively.
Fig. 74 is a schematic diagram showing a data structure of the STN table SS
3130 according to Embodiment 2 of the present invention.
Fig. 75 is a functional block diagram of a system target decoder 7525
according to Embodiment 2 of the present invention.
Fig. 76 is a partial functional block diagram of the plane adder 7526 in 2 plane
mode.
Figs. 77A, 77B, and 77C are schematic diagrams showing a left-view graphics
image GOB0 represented by the 2D PG stream and a right-view graphics images
GOB1-GOB3 represented by the right-view PG stream; and Figs. 77D, 77E, and 77F
are schematic diagrams showing the offset control performed onto the left-view
graphics image shown in Figs. 77A, 77B, and 77C.
Fig. 78 is a functional block diagram of a recording device 7800 according to
Embodiment 3 of the present invention.
Figs. 79A and 79B are schematic diagrams respectively showing a picture in a
left view and a right view used to display one scene of 3D video images; and Fig. 79C is
a schematic diagram showing depth information calculated from these pictures by the
video encoder 7802.
Fig. 80 is a flowchart of a method for recording movie content on a BD-ROM
disc using the recording device 7800 shown in Fig. 78.
Fig. 81 is a schematic diagram showing a method to align extent ATC times
between consecutive data blocks.
Figs. 82A-82C are schematic diagrams illustrating the principle behind
playback of 3D video images (stereoscopic video images) in a method using parallax
video images.
Fig. 83 is a schematic diagram showing an example of constructing a left-view
LVW and a right-view RVW from the combination of a 2D video image MVW and a
depth map DPH.
Fig. 84 is a block diagram showing playback processing in the playback device
in 2D playback mode.
Fig. 85A is a graph showing the change in the data amount DA stored in the
read buffer 3721 during operation of the playback processing shown in Fig. 84 in 2D
playback mode; and Fig. 85B is a schematic diagram showing the correspondence
between an extent block 8510 for playback and a playback path 8520 in 2D playback
mode.
Fig. 86 is an example of a correspondence table between jump distances Sjump
and maximum jump times Tjumpmax for a BD-ROM disc.
Fig. 87 is a block diagram showing playback processing in the playback device
in 3D playback mode.
Figs. 88A and 88B are graphs showing changes in data amounts DAI and DA2
stored in RBI 4221 and RB2 4222 shown in Fig. 87 when 3D video images are played
back seamlessly from a single extent block; and Fig. 88C is a schematic diagram
showing a correspondence between the extent block 8810 and a playback path 8820 in
3D playback mode.
Fig. 89B is a schematic diagram showing an (M + I)* (the letter M represents
an integer greater than or equal to 1) extent block 8901 and (M + if1 extent block 8902
and the correspondence between these extent blocks 8901 and 8902 and a playback path
8920 in 3D playback mode; and Fig. 89A is a graph group showing changes in data
amounts DAI and DA2 stored in RBI 4221 and RB2 4222 , as well as the changes in
the sum DAI + DA2, when 3D video images are continually played back seamlessly
from two extent blocks 8901 and 8902.
Figs. 90A and 90B are graphs showing changes in data amounts DAI and DA2
stored in RBI 4221 and RB2 4222 when 3D video images are played back seamlessly
from the two consecutive extent blocks 8901 and 8902 shown in Fig. 89B.
Fig. 91 is a block diagram showing the video stream processing system
provided in the system target decoder 4225 in the 3D playback mode.
Figs. 92A and 92B are graphs showing temporal changes in the base-view
transfer rate Rexti and the dependent-view transfer rate Rexh in that case, respectively;
and Fig. 92C is a graph showing the temporal change in the sum of the base-view
transfer rate Rexti and the dependent-view transfer rate Rext2 shown in Figs. 92A and
92B.
Fig. 93 is a schematic diagram showing the relationships between the TS
packets that are transferred in the system target decoder from the source depacketizer to
the PID filter and the ATC times.
Fig. 94A is a table showing the maximum extent sizes maxSEXTiM and
maxSEXT2[n] for an extent pair at various combinations of the base-view transfer rate
Rexti [n] and the dependent-view transfer rate Rext2[ii]; Fig. 94B is a schematic
diagram showing that an extent pair EXTl[n], EXT2[n] is located at the top of an extent
block 9401 arranged after a layer boundary LB, and the base-view data block B[n] of
the extent pair is arranged before the dependent-view data block D[n] thereof.
Figs. 95A and 95B are graphs showing changes in amounts DAI, DA2 of data
stored in RBI and RB2, respectively, when 3D video images are played back
seamlessly from the two extent blocks 9401 and 9402 shown in Fig. 94B.
Fig. 96A is a schematic diagram showing the syntax of the extent start point in
the case where the order of data blocks is reversed in the extent pair located in the
middle of the extent block; Fig. 96B is a schematic diagram showing the relationships
between the base-view extent EXTl[k] (k = 0, 1, 2, ...) belonging to the file base and
the extent start flag indicated by the extent start point; Fig. 96C is a schematic diagram
showing the relationships between the dependent-view extent EXT2[k] belonging to the
file DEP and the extent start flag; and Fig. 96D is a schematic diagram showing the
relationships between the extent SS EXTSS[0] belonging to the file SS and the extent
blocks on the BD-ROM disc.
Fig. 97C is a schematic diagram showing an arrangement of a data block which
requires the largest capacity of RBI 4221; Figs. 97A and 97B are graphs showing
changes in amounts DAI, DA2 of data stored in RBI 4221 and RB2 4222, respectively,
when 3D video images are played back seamlessly from the two extent blocks 9701 and
9702 shown in Fig. 97C; Fig. 97F is a schematic diagram showing an arrangement of a
data block which requires the largest capacity of RB2 4222; and Figs. 97D and 97E are
graphs showing changes in amounts DAI, DA2 of data stored in RBI 4221 and RB2
4222, respectively, when 3D video images are played back seamlessly from the two
extent blocks 9703 and 9704 shown in Fig. 97F.
Fig. 98C is a schematic diagram showing an extent block 9810 which includes
in the middle thereof an extent pair in which the order of data blocks is reversed; and
Figs. 98A and 98B are graphs showing changes in amounts DAI, DA2 of data stored in
RBI 4221 and RB2 4222, respectively, when 3D video images are played back
seamlessly from the extent block 9801 shown in Fig. 98C.
Fig. 99 is a schematic diagram showing a relationship between an extent block
9900 and AV stream files 9910-9920, the extent block 9900 including an extent pair in
the middle thereof, the extent pair having data blocks in reverse order.
Fig. 100 is a schematic diagram showing arrangement 1 of a data block group
recorded before and after a layer boundary LB on a BD-ROM disc.
Fig. 101 is a schematic diagram showing a playback path Al 10 in 2D playback
mode and a playback path A120 in 3D playback mode for the data block group in
arrangement 1 shown in Fig. 100.
Fig. 102 is a schematic diagram showing arrangement 2 of a data block group
recorded before and after a layer boundary LB on a BD-ROM disc.
Fig. 103 is a schematic diagram showing a playback path A310 in 2D playback
mode and a playback path A320 in 3D playback mode for the data block group in
arrangement 2 shown in Fig. 102.
Fig. 104 is a schematic diagram showing the relationships between the read
time Sexti[3]/Rud72 of the block exclusively for 3D playback B[3]ss located at the end
of the second extent block A202 and the lower limit of the capacity of RB2 4222.
Fig. 105 is a schematic diagram showing entry points A510 and A520 set for
extents EXT1 [k] and EXT2[k] (the letter k represents an integer greater than or equal to
0) in a file base A501 and a file DEP A502.
Fig. 106A is a schematic diagram showing a playback path when extent ATC
times and playback times of the video stream differ between contiguous base-view data
blocks and dependent-view data blocks; and Fig. 106B is a schematic diagram showing
a playback path when the playback times of the video stream are equal for contiguous
base-view and dependent-view data blocks.
Fig. 107A is a schematic diagram showing a playback path for multiplexed
stream data supporting multi-angle; Fig. 107B is a schematic diagram showing a data
block group A701 recorded on a BD-ROM disc and a corresponding playback path
A702 in L/R mode; and Fig. 107C is a schematic diagram showing an extent block
formed by stream data Ak, Bk, and Ck for different angles.
Fig. 108 is a schematic diagram showing (i) a data block group A801
constituting a multi-angle period and (ii) a playback path A810 in 2D playback mode
and playback path A820 in L/R mode that correspond to the data block group A801.
Fig. 109 is a schematic diagram illustrating the technology for ensuring the
compatibility of an optical disc storing 3D video content with 2D playback devices.
Fig. 110 is a functional block diagram of a playback device realized by using
the integrated circuit 3 according to Embodiment 4 of the present invention.
Fig. Ill is a functional block diagram showing a representative structure of the
stream processing unit 5 shown in Fig. 110.
Fig. 112 is a functional block diagram of the switching unit 53 and the
surrounding units shown in Fig. 110 when the switching unit 53 is DMAC.
Fig. 113 is a functional block diagram showing a representative structure of the
AV output unit 8 shown in Fig. 110.
Fig. 114 is a schematic diagram showing one example of the method of using
the memory 2 during the process of superimposing images.
Fig. 115 is a schematic diagram showing a method of superimposing the
graphics plane on the left-view plane by using the memory 2 shown in Fig. 114.
Fig. 116 is a schematic diagram showing a method of superimposing the
graphics plane on the right-view plane by using the memory 2 shown in Fig. 114.
Fig. 117 is a schematic diagram showing another example of the method of
using the memory 2 during the process of superimposing images.
Fig. 118 is a detailed functional block diagram of the AV output unit 8 and the
data output unit in the playback device shown in Fig. 113.
Figs. 119A and 119B are schematic diagrams showing examples of the
topology of a control bus and a data bus arranged in the integrated circuit 3 shown in
Fig. 110.
Fig. 120 is a functional block diagram showing the structure of the integrated
circuit according to Embodiment 4 and the surrounding units, which are incorporated in
a display device.
Fig. 121 is a detailed functional block diagram of the AV output unit 8 shown
in Fig. 120.
Fig. 122 is a flowchart of playback processing by a playback device using the
integrated circuit 3 shown in Fig. 110.
Fig. 123 is a flowchart showing details of the steps S1-S6 shown in Fig. 122.
[Description of Embodiments]
[0017]
The following describes a recording medium and a playback device pertaining
to preferred Embodiments of the present invention with reference to the drawings.
[0018]
Embodiment 1
[0019]
Fig. 1 is a schematic diagram showing a home theater system that uses a
recording medium according to Embodiment 1 of the present invention. This home
theater system adopts a 3D video image (stereoscopic video image) playback method
that uses parallax video images, and in particular adopts an alternate-frame sequencing
method as a display method (see «Supplementary Explanation» for details). As
shown in Fig. 1, this home theater system plays back a recording medium 101 and
includes a playback device 102, a display device 103, shutter glasses 104, and a remote
control 105. The playback device 102 and the display device 103 are provided
independently of each other as shown in Fig. 1. Alternatively, the playback device 102
and the display device 103 may be provided as one unit.
[0020]
The recording medium 101 is a read-only Blu-ray disc (BD)™, i.e. a BD-ROM
disc. The recording medium 101 can be a different portable recording medium, such as
an optical disc with a different format such as DVD or the like, a removable hard disk
drive (HDD), or a semiconductor memory device such as an SD memory card. This
recording medium, i.e. the BD-ROM disc 101, stores movie content as 3D video images.
This content includes video streams representing a left view and a right view for the 3D
video images. The content may further include a video stream representing a depth map
for the 3D video images. These video streams are arranged on the BD-ROM disc 101 in
units of data blocks and are accessed using a file structure described below. The video
streams representing the left view or the right view are used by both a 2D playback
device and a 3D playback device to play the content back as 2D video images.
Conversely, a pair of video streams representing a left view and a right view, or a pair
of video streams representing either a left view or a right view and a depth map, are
used by a 3D playback device to play the content back as 3D video images.
[0021]
A BD-ROM drive 121 is mounted on the playback device 102. The BD-ROM
drive 121 is an optical disc drive conforming to the BD-ROM format. The playback
device 102 uses the BD-ROM drive 121 to read content from the BD-ROM disc 101.
The playback device 102 further decodes the content into video data / audio data. The
playback device 102 is a 3D playback device and can play the content back as both 2D
video images and as 3D video images. Hereinafter, the operational modes of the
playback device 102 when playing back 2D video images and 3D video images are
respectively referred to as "2D playback mode" and "3D playback mode". In 2D
playback mode, video data only includes either a left-view or a right-view video frame.
In 3D playback mode, video data includes both left-view and right-view video frames.
[0022]
3D playback mode is further divided into left/right (L/R) mode and depth mode.
In "L/R mode", a pair of left-view and right-view video frames is generated from a
combination of video streams representing the left view and right view. In "depth
mode", a pair of left-view and right-view video frames is generated from a combination
of video streams representing either a left view or a right view and a depth map. The
playback device 102 is provided with an L/R mode. The playback device 102 may be
further provided with a depth mode.
[0023]
The playback device 102 is connected to the display device 103 via a
High-Definition Multimedia Interface (HDMI) cable 122. The playback device 102
converts video data / audio data into a video signal / audio signal in the HDMI format,
and transmits the signals to the display device 103 via the HDMI cable 122. In 2D
playback mode, only one of either the left-view or the right-view video frame is
multiplexed in the video signal. In 3D playback mode, both the left-view and the
right-view video frames are time-multiplexed in the video signal. Additionally, the
playback device 102 exchanges CEC messages with the display device 103 via the
HDMI cable 122. The playback device 102 can thus ask the display device 103 whether
it supports playback of 3D video images.
[0024]
The display device 103 is a liquid crystal display. Alternatively, the display
device 103 can be another type of flat panel display, such as a plasma display, an
organic EL display, etc., or a projector. The display device 103 displays video on the
screen 131 in response to a video signal, and causes the speakers to produce audio in
response to an audio signal. The display device 103 supports playback of 3D video
images. During playback of 2D video images, either the left view or the right view is
displayed on the screen 131. During playback of 3D video images, the left view and
right view are alternately displayed on the screen 131.
[0025]
The display device 103 includes a left/right signal transmitting unit 132. The
left/right signal transmitting unit 132 transmits a left/right signal LR to the shutter
glasses 104 via infrared rays or by radio transmission. The left/right signal LR indicates
whether the image currently displayed on the screen 131 is a left-view or a right-view
image. During playback of 3D video images, the display device 103 detects switching
of frames by distinguishing between a left-view frame and a right-view frame based on
a control signal that accompanies a video signal. Furthermore, the display device 103
causes the left/right signal transmitting unit 132 to switch the left/right signal LR
synchronously with the detected switching of frames.
[0026]
The shutter glasses 104 include two liquid crystal display panels 141L and
141R and a left/right signal receiving unit 142. The liquid crystal display panels 141L
and 141R respectively constitute the left and right lens parts. The left/right signal
receiving unit 142 receives a left/right signal LR, and in accordance with changes
therein, transmits the signal to the left and right liquid crystal display panels 141L and
141R. In response to the signal, each of the liquid crystal display panels 141L and 141R
either lets light pass through the entire panel or shuts light out. For example, when the
left/right signal LR indicates a left-view display, the liquid crystal display panel 141L
for the left eye lets light pass through, while the liquid crystal display panel 141R for
the right eye shuts light out. When the left/right signal LR indicates a right-view display,
the display panels act oppositely. The two liquid crystal display panels 141L and 141R
thus alternately let light pass through in sync with the switching of frames. As a result,
when the viewer looks at the screen 131 while wearing the shutter glasses 104, the left
view is shown only to the viewer's left eye, and the right view is shown only to the right
eye. The viewer is made to perceive the difference between the images seen by each eye
as the binocular parallax for the same stereoscopic image, and thus the video image
appears to be stereoscopic.
[0027]
The remote control 105 includes an operation unit and a transmitting unit. The
operation unit includes a plurality of buttons. The buttons correspond to each of the
functions of the playback device 102 and the display device 103, such as turning the
power on or off, starting or stopping playback of the BD-ROM disc 101, etc. The
operation unit detects when the user presses a button and conveys identification
information for the button to the transmitting unit as a signal. The transmitting unit
converts this signal into a signal IR and outputs it via infrared rays or radio transmission
to the playback device 102 or the display device 103. On the other hand, the playback
device 102 and display device 103 each receive this signal IR, determine the button
indicated by this signal IR, and execute the function associated with the button. In this
way, the user can remotely control the playback device 102 or the display device 103.
[0028]
[0230]
When playing back 2D video content from the BD-ROM disc 101 in 2D
playback mode, the playback device 102 operates as a 2D playback device. Fig. 37 is a
functional block diagram of a 2D playback device 3700. As shown in Fig. 37, the 2D
playback device 3700 includes a BD-ROM drive 3701, playback unit 3702, and control
unit 3703. The playback unit 3702 includes a read buffer 3721, system target decoder
3725, and plane adder 3726. The control unit 3703 includes a dynamic scenario memory
3731, static scenario memory 3732, user event processing unit 3733, program execution
unit 3734, playback control unit 3735, and player variable storage unit 3736. The
playback unit 3702 and the control unit 3703 are each implemented on a different
integrated circuit, but may alternatively be implemented on a single integrated circuit.
[0231]
When the BD-ROM disc 101 is loaded into the BD-ROM drive 3701, the
BD-ROM drive 3701 radiates laser light to the disc 101 and detects change in the
reflected light. Furthermore, using the change in the amount of reflected light, the
BD-ROM drive 3701 reads data recorded on the disc 101. Specifically, the BD-ROM
drive 3701 has an optical pickup, i.e. an optical head. The optical head has a
semiconductor laser, collimate lens, beam splitter, objective lens, collecting lens, and
optical detector. A beam of light radiated from the semiconductor laser sequentially
passes through the collimate lens, beam splitter, and objective lens to be collected on a
recording layer of the disc 101. The collected beam is reflected and diffracted by the
recording layer. The reflected and diffracted light passes through the objective lens, the
beam splitter, and the collecting lens, and is collected onto the optical detector. The
optical detector generates a playback signal at a level in accordance with the amount of
collected light. Furthermore, data is decoded from the playback signal.
[0232]
The BD-ROM drive 3701 reads data from the BD-ROM disc 101 based on a
request from the playback control unit 3735. Out of the read data, the extents in the file
2D, i.e. the 2D extents, are transferred to the read buffer 3721; dynamic scenario
information is transferred to the dynamic scenario memory 3731; and static scenario
information is transferred to the static scenario memory 3732. "Dynamic scenario
information" includes an index file, movie object file, and BD-J object file. "Static
scenario information" includes a 2D playlist file and a 2D clip information file.
[0233]
The read buffer 3721, dynamic scenario memory 3731, and static scenario
memory 3732 are each a buffer memory. Memory elements in the playback unit 3702
are used as the read buffer 3721. Memory elements in the control unit 3703 are used as
the dynamic scenario memory 3731 and the static scenario memory 3732. Alternatively,
different areas in a single memory element may be used as part or all of these buffer
memories 3721, 3731, and 3732.
[0234]
The system target decoder 3725 reads 2D extents from the read buffer 3721 in
units of source packets and demultiplexes the 2D extents. The system target decoder
3725 then decodes each of the elementary streams obtained by the demultiplexing. At
this point, information necessary for decoding each elementary stream, such as the type
of codec and attributes of the stream, is transferred from the playback control unit 3735
to the system target decoder 3725. The system target decoder 3725 outputs a primary
video stream, secondary video stream, IG stream, and PG stream after decoding
respectively as primary video plane data, secondary video plane data, IG plane data, and
PG plane data, in units of VAUs. On the other hand, the system target decoder 3725
mixes the decoded primary audio stream and secondary audio stream and transmits the
resultant data to an audio output device, such as an internal speaker 103 A of the display
device 103. In addition, the system target decoder 3725 receives graphics data from the
program execution unit 3734. The graphics data is used for rendering graphic elements
for a GUI, such as a menu, on the screen and is in a raster data format such as JPEG and
PNG. The system target decoder 3725 processes the graphics data and outputs the
processed data as image plane data. Details on the system target decoder 3725 are
provided below.
[0235]
The plane adder 3726 receives primary video plane data, secondary video plane
data, IG plane data, PG plane data, and image plane data from the system target decoder
3725 and superimposes these pieces of plane data to generate one combined video
frame or field. The combined video data is transferred to the display device 103 for
display on the screen.
[0236]
The user event processing unit 3733 detects a user operation via the remote
control 105 or the front panel of the playback device 102. Based on the user operation,
the user event processing unit 3733 requests the program execution unit 3734 or the
playback control unit 3735 to perform processing. For example, when a user instructs to
display a pop-up menu by pushing a button on the remote control 105, the user event
processing unit 3733 detects the push and identifies the button. The user event
processing unit 3733 further requests the program execution unit 3734 to execute a
command corresponding to the button, i.e. a command to display the pop-up menu. On
the other hand, when a user pushes a fast-forward or a rewind button on the remote
control 105, for example, the user event processing unit 3733 detects the push and
identifies the button. The user event processing unit 3733 then requests the playback
control unit 3735 to fast-forward or rewind the playlist currently being played back.
[0237]
The program execution unit 3734 is a processor that reads programs from
movie object files and BD-J object files stored in the dynamic scenario memory 3731
and executes these programs. Furthermore, the program execution unit 3734 performs
the following operations in accordance with the programs: (1) The program execution
unit 3734 orders the playback control unit 3735 to perform playlist playback processing;
(2) The program execution unit 3734 generates graphics data for a menu or game as
PNG or JPEG raster data and transfers the generated data to the system target decoder
3725 to be combined with other video data. Via program design, specific details on
these processes can be designed relatively flexibly. In other words, during the authoring
process of the BD-ROM disc 101, the nature of these processes is determined while
programming the movie object files and BD-J object files.
[0238]
The playback control unit 3735 controls transfer of different types of data, such
as 2D extents, an index file, etc. from the BD-ROM disc 101 to the read buffer 3721,
dynamic scenario memory 3731, and static scenario memory 3732. A file system
managing the directory file structure shown in Fig. 2 is used for this control. That is, the
playback control unit 3735 causes the BD-ROM drive 3701 to transfer the files to each
of the buffer memories 3721, 3731, and 3732 using a system call for opening files. The
"file opening" is composed of a sequence of the following processes. First, a file name
to be detected is provided to the file system by a system call, and an attempt is made to
detect the file name from the directory/file structure. When the detection is successful,
the file entry for the target file to be transferred is first transferred to memory in the
playback control unit 3735, and a File Control Block (FCB) is generated in the memory.
Subsequently, a file handle for the target file is returned from the file system to the
playback control unit 3735. Afterwards, the playback control unit 3735 can cause the
BD-ROM drive 3701 to transfer the target file from the BD-ROM disc 101 to each of
the buffer memories 3721, 3731, and 3732 by showing the file handle to the BD-ROM
drive 3701.
[0239]
The playback control unit 3735 decodes the file 2D to output video data and
audio data by controlling the BD-ROM drive 3701 and the system target decoder 3725.
Specifically, the playback control unit 3735 first reads a 2D playlist file from the static
scenario memory 3732, in response to an instruction from the program execution unit
3734 or a request from the user event processing unit 3733, and interprets the content of
the file. In accordance with the interpreted content, particularly with the playback path,
the playback control unit 3735 then specifies a file 2D to be played back and instructs
the BD-ROM drive 3701 and the system target decoder 3725 to read and decode this
file. Such playback processing based on a playlist file is referred to as "playlist playback
processing".
[0240]
In addition, the playback control unit 3735 sets various types of player
variables in the player variable storage unit 3736 using the static scenario information.
With reference to the player variables, the playback control unit 3735 further specifies
to the system target decoder 3725 elementary streams to be decoded and provides the
information necessary for decoding the elementary streams.
[0241]
The player variable storage unit 3736 is composed of a group of registers for
storing player variables. Types of player variables include system parameters (SPRM)
and general parameters (GPRM). An SPRM indicates the status of the playback device
102. Fig. 38 is a list of SPRMs. As shown in Fig. 38, each SPRM is assigned a serial
number 3801, and each serial number 3801 is associated with a unique variable value
3802. There are provided, for example, 64 SPRMs. The contents of SPRMs are shown
below. Here, the numbers in parentheses indicate the serial numbers 3801.
[0242]
SPRM(0): Language code
SPRM(l): Primary audio stream number
SPRM(2): Subtitle stream number
SPRM(3): Angle number
SPRM(4): Title number
SPRM(5): Chapter number
SPRM(6): Program number
SPRM(7): Cell number
SPRM(8): Key name
SPRM(9): Navigation timer
SPRM(10): Current playback time
SPRM(11): Player audio mixing mode for karaoke
SPRM(12): Country code for parental management
SPRM(13): Parental level
SPRM(14): Player configuration for video
SPRM(15): Player configuration for audio
SPRM(16): Language code for audio stream
SPRM(17): Language code extension for audio stream
SPRM(18): Language code for subtitle stream
SPRM(19): Language code extension for subtitle stream
SPRM(20): Player region code
SPRM(21): Secondary video stream number
SPRM(22): Secondary audio stream number
SPRM(23): Player status
OT>T>'K4Y'>/l\ CTim*^'l\. 11»________1
[0243]
The SPRM(IO) indicates the PTS of the picture currently being decoded and is
updated every time a picture is decoded and written into the primary video plane
memory. Accordingly, the current playback point can be known by referring to the
SPRM(IO).
[0244]
The parental level in SPRM(13) indicates a predetermined restricted age and is
used for parental control of viewing of titles recorded on the BD-ROM disc 101. A
user of the playback device 102 sets the value of the SPRM(13) via, for example, an
OSD of the playback device 102. "Parental control" refers to restricting viewing of a
title in accordance with the viewer's age. The following is an example of how the
playback device 102 performs parental control. The playback device 102 first reads
the age for which viewing of a title is permitted from the BD-ROM disc 101, and then
compares this age with the value of the SPRM(13). If this age is equal to or less than
the value of the SPRM(13), the playback device 102 continues with playback of the title.
If this age is greater than the value of the SPRM(13), the playback device 102 stops
playback of the title.
[0245]
The language code for audio stream in SPRM(16) and the language code for
subtitle stream in SPRM(18) show default language codes of the playback device 102.
These codes may be changed by a user with use of the OSD or the like of the playback
device 102, or the codes may be changed by an application program via the program
execution unit 3734. For example, if the SPRM(16) shows "English", then during -
playback processing of a playlist, the playback control unit 3735 first searches the STN
table in the PI showing the current playback section, i.e. the current PI, for a stream
entry having the language code for "English". The playback control unit 3735 then
extracts the PID from the stream identification information of the stream entry and
transmits the extracted PED to the system target decoder 3725. As a result, an audio
stream having the PID is selected and decoded by the system target decoder 3725.
These processes can be executed by the playback control unit 3735 with use of the
movie object file or the BD-J object file.
[0246]
During playback processing, the playback control unit 3735 updates the player
variables in accordance with the status of playback. The playback control unit 3735
updates the SPRM(l), SPRM(2), SPRM(21), and SPRM(22) in particular. These SPRM
respectively show, in the stated order, the STN of the audio stream, subtitle stream,
secondary video stream, and secondary audio stream that are currently being processed.
For example, suppose that the SPRM(l) has been changed by the program execution
unit 3734. In this case, the playback control unit 3735 first refers to the STN shown by
the new SPRM(l) and retrieves the stream entry that includes this STN from the STN
table in the current PI. The playback control unit 3735 then extracts the PE) from the
stream identification information of the stream entry and transmits the extracted PID to
the system target decoder 3725. As a result, an audio stream having the PID is selected
and decoded by the system target decoder 3725. This is how the audio stream to be
played back is switched. The subtitle stream and the secondary video stream to be
played back can be similarly switched.
[0247]
«2D Playlist Playback Processing»
[0248]
Fig. 39 is a flowchart of 2D playlist playback processing by a playback control
unit 3735. 2D playlist playback processing is performed according to a 2D playlist file
and is started by the playback control unit 3735 reading a 2D playlist file from the static
scenario memory 3732.
[0249]
In step S3 901, the playback control unit 3735 first reads a single PI from a
main path in the 2D playlist file and then sets the PI as the current PI. Next, from the
STN table of the current PI, the playback control unit 3735 selects PEDs of elementary
streams to be played back and specifies attribute information necessary for decoding the
elementary streams. The selected PIDs and attribute information are indicated to the
system target decoder 3725. The playback control unit 3735 further specifies a SUB_PI
associated with the current PI from the sub-paths in the 2D playlist file. Thereafter,
processing proceeds to step S3902.
[0250]
In step S3 902, the playback control unit 3735 reads reference clip information,
a PTS #1 indicating a playback start time INI, and a PTS #2 indicating a playback end
time OUT1 from the current PI. From this reference clip information, a 2D clip
information file corresponding to the file 2D to be played back is specified. Furthermore,
when a SUBPI exists that is associated with the current PI, similar information is also
read from the SUB_PI. Thereafter, processing proceeds to step S3 903.
[0251]
In step S3 903, with reference to the entry map of the 2D clip information file,
the playback control unit 3735 retrieves the SPN #1 and the SPN #2 in the file 2D
corresponding to the PTS #1 and the PTS #2. The pair of PTSs indicated by the SUB_PI
are also converted to a pair of SPNs. Thereafter, processing proceeds to step S3904.
[0252]
In step S3904, from the SPN #1 and the SPN #2, the playback control unit
3735 calculates a number of sectors corresponding to each of the SPN #1 and the SPN
#2. Specifically, the playback control unit 3735 first obtains the product of each of the
SPN #1 and the SPN #2 multiplied by the data amount per source packet, i.e. 192 bytes.
Next, the playback control unit 3735 obtains a quotient by dividing each product by the
data amount per sector, i.e. 2048 bytes: Nl = SPN #1 * 192 / 2048, N2 = SPN #2 x 192
/ 2048. The quotients Nl and N2 are the same as the total number of sectors, in the main
TS, recorded in portions previous to the source packets to which SPN #1 and SPN #2
are allocated, respectively. The pair of SPNs converted from the pair of PTSs indicated
by the SUBPI is similarly converted to a pair of numbers of sectors. Thereafter,
processing proceeds to step S3905.
[0253]
In step S3905, the playback control unit 3735 specifies, from the numbers of
sectors Nl and N2 obtained in step S3904, LBNs of the top and end of the 2D extent
group to be played back. Specifically, with reference to the file entry of the file 2D to be
played back, the playback control unit 3735 counts from the top of the sector group in
which the 2D extent group is recorded so that the LBN of the (Nl + 1)* sector = LBN
#1, and the LBN of the (N2 + 1)* sector = LBN #2. The playback control unit 3735
further specifies a range from the LBN#1 to the LBN#2 to the BD-ROM drive 121. The
pair of numbers of sectors converted from the pair of PTSs indicated by the SUB_PI is
similarly converted to a pair of LBNs and specified to the BD-ROM drive 121. As a
result, from the sector group in the specified range, a source packet group belonging to a
2D extent group is read in aligned units. Thereafter, processing proceeds to step S3906.
[02541
In step S3906, the playback control unit 3735 checks whether an unprocessed
PI remains in the main path. When an unprocessed PI remains, processing is repeated
from step S3901. When no unprocessed PI remains, processing ends.
[0255]
«System Target Decoder»
[0256]
Fig. 40 is a functional block diagram of the system target decoder 3725. As
shown in Fig. 40, the system target decoder 3725 includes a source depacketizer 4010,
ATC counter 4020, first 27 MHz clock 4030, PID filter 4040, STC counter (STC1)
4050, second 27 MHz clock 4060, primary video decoder 4070, secondary video
decoder 4071, PG decoder 4072, IG decoder 4073, primary audio decoder 4074,
secondary audio decoder 4075, text subtitle decoder 4076, image processor 4080,
primary video plane memory 4090, secondary video plane memory 4091, PG plane
memory 4092, IG plane memory 4093, image plane memory 4094, and audio mixer
4095.
[0257]
The source depacketizer 4010 reads source packets from the read buffer 3721,
extracts the TS packets from the read source packets, and transfers the TS packets to the
PID filter 4040. Furthermore, the source depacketizer 4010 synchronizes the time of the
transfer with the time shown by the ATS of each source packet. Specifically, the source
depacketizer 4010 first monitors the value of the ATC generated by the ATC counter
4020. In this case, the value of the ATC depends on the ATC counter 4020 and is
incremented in accordance with a pulse of a clock signal from the first 27 MHz clock
4030. Subsequently, at the instant the value of the ATC matches the ATS of a source
packet, the source depacketizer 4010 transfers the TS packets extracted from the source
packet to the PID filter 4040. By adjusting the time of transfer in this way, the mean
transfer rate of TS packets from the source depacketizer 4010 to the PID filter 4040
does not surpass the value Rxs specified by the system rate 2211 in the 2D clip
[0258]
The PID filter 4040 first monitors a PED that includes each TS packet outputted
by the source depacketizer 4010. When the PID matches a PID pre-specified by the
playback control unit 3735, the PID filter 4040 selects the TS packet and transfers it to
the decoder 4070-4075 appropriate for decoding of the elementary stream indicated by
the PID (the text subtitle decoder 4076, however, is excluded). For example, if a PID is
0x1011, the TS packets are transferred to the primary video decoder 4070. TS packets
with PIDs ranging from OxlBOO-OxlBlF, 0x1100-0x11 IF, OxlAOO-OxlAlF,
0x1200-0x121F, and 0xl400-0xl41F are transferred to the secondary video decoder
4071, primary audio decoder 4074, secondary audio decoder 4075, PG decoder 4072,
and IG decoder 4073, respectively.
[0259]
The PID filter 4040 further detects a PCR from TS packets using the PIDs of
the TS packets. At each detection, the PID filter 4040 sets the value of the STC counter
4050 to a predetermined value. Then, the value of the STC counter 4050 is incremented
in accordance with a pulse of the clock signal of the second 27 MHz clock 4060. In
addition, the value to which the STC counter 4050 is set is indicated to the PED filter
4040 from the playback control unit 3735 in advance. The decoders 4070-4076 each use
the value of the STC counter 4050 as the STC. Specifically, the decoders 4070-4076
first reconstruct the TS packets received from the PID filter 4040 into PES packets.
Next, the decoders 4070-4076 adjust the timing of the decoding of data included in the
PES payloads in accordance with the times indicated by the PTSs or the DTSs included
in the PES headers.
[0260]
The primary video decoder 4070, as shown in Fig. 40, includes a transport
stream buffer (TB) 4001, multiplexing buffer (MB) 4002, elementary stream buffer
(EB) 4003, compressed video decoder (DEC) 4004, and decoded picture buffer (DPB)
[0261]
The TB 4001, MB 4002, and EB 4003 are each a buffer memory and use an
area of a memory element internally provided in the primary video decoder 4070.
Alternatively, some or all of the buffer memories may be separated in discrete memory
elements. The TB 4001 stores the TS packets received from the PID filter 4040 as they
are. The MB 4002 stores PES packets reconstructed from the TS packets stored in the
TB 4001. Note that when the TS packets are transferred from the TB 4001 to the MB
4002, the TS header is removed from each TS packet. The EB 4003 extracts encoded
VAUs from the PES packets and stores the VAUs therein. A VAU includes a
compressed picture, i.e., an I picture, B picture, or P picture. Note that when data is
transferred from the MB 4002 to the EB 4003, the PES header is removed from each
PES packet.
[0262]
The DEC 4004 is a hardware decoder specifically for decoding of compressed
pictures and is composed of an LSI that includes, in particular, a function to accelerate
the decoding. The DEC 4004 decodes a picture from each VAU in the EB 4003 at the
time shown by the DTS included in the original PES packet. The DEC 4004 may also
refer to the decoding switch information 1250 shown in Fig. 12 to decode pictures from
VAUs sequentially, regardless of the DTSs. During decoding, the DEC 4004 first
analyzes the VAU header to specify the compressed picture, compression encoding
method, and stream attribute stored in the VAU, selecting a decoding method in
accordance with this information. Compression encoding methods include, for example,
MPEG-2, MPEG-4 AVC, and VC1. Furthermore, the DEC 4004 transmits the decoded,
uncompressed picture to the DPB 4005.
[0263]
Like the TB 4001, MB 4002, and EB 4003, the DPB 4005 is a buffer memory
that uses an area of a built-in memory element in the primary video decoder 4070.
Alternatively, the DPB 4005 may be located in a memory element separate from the
other buffer memories 4001, 4002, and 4003. The DPB 4005 temporarily stores the
decoded pictures. When a P picture or B picture is to be decoded by the DEC 4004, the
DPB 4005 retrieves reference pictures, in response to an instruction from the DEC 4004,
from among stored, decoded pictures. The DPB 4005 then provides the reference
pictures to the DEC 4004. Furthermore, the DPB 4005 writes the stored pictures into the
primary video plane memory 4090 at the time shown by the PTSs included in the
original PES packets.
[0264]
The secondary video decoder 4071 includes the same structure as the primary
video decoder 4070. The secondary video decoder 4071 first decodes the TS packets of
the secondary video stream received from the PID filter 4040 into uncompressed
pictures. Subsequently, the secondary video decoder 4071 writes the uncompressed
pictures into the secondary video plane memory 4091 at the time shown by the PTSs
included in the PES packets.
[0265]
The PG decoder 4072 decodes the TS packets received from the PID filter
4040 into uncompressed graphics data and writes the uncompressed graphics data to the
PG plane memory 4092 at the time shown by the PTSs included in the PES packets
[0266].
Fig. 41A is a flowchart of processing whereby the PG decoder 4072 decodes a graphics
object from one data entry in the PG stream. The processing is started when the PG
decoder 4072 receives a group of TS packets constituting one data entry shown in Fig. 6,
from the PID filter 4040. Figs. 41B-41E are schematic diagrams showing the graphics
object changing as the processing proceeds.
[0267]
In step S4101, the PG decoder 4072 first identifies an ODS having the same
object ID as the reference object ID 605 in the PCS. Next, the PG decoder 4072 decodes
a graphics object from the identified ODS, and writes the decoded graphics object into
the object buffer. Here, the "object buffer" is a buffer memory embedded in the PG
decoder 4072. The "smile mark" FOB shown in Fig. 41B is an example of the graphics
object written into the object buffer.
[0268]
In step S4102, the PG decoder 4072 performs the cropping process in
accordance with the cropping information 602 in the PCS, extracts a part of the graphics
object from the graphics object, and writes the extracted part into the object buffer. Fig.
41C shows that strips LST and RST are removed from the left-hand and right-hand ends
of the smile mark FOB, and the remaining part OBJ is written into the object buffer.
[0269]
In step S4103, the PG decoder 4072 first identifies a WDS having the same
window ID as the reference window ID 603 in the PCS. Next, the PG decoder 4072
determines a display position of the graphics object in the graphics plane from a
window position 612 indicated by the identified WDS and an object display position
601 in the PCS. In Fig. 41D, the upper-left position of the window WIN in the graphics
plane GPL and an upper-left position DSP of the graphics object OBJ are determined.
[0270]
In step S4104, the PG decoder 4072 writes the graphics object in the object
buffer into the display position determined in step S4103. When doing so, the PG
decoder 4072 determines a range in which the graphics object is rendered by using a
window size 613 indicated by the WDS. In Fig. 41D, the graphics object OBJ is written
into the graphics plane GPL in the range of window WIN starting from the upper-left
position DSP.
[0271]
In step S4105, the PG decoder 4072 first identifies a PDS having the same
pallet ID as the reference pallet ID 604 in the PCS. Next, the PG decoder 4072, by using
CLUT 622 in the PDS, determines color coordinate values to be indicated by each pixel
data in the graphics object OBJ. In Fig. 41E, the color of each pixel in the graphics
object OBJ have been determined. In this way, processing of rendering a graphics object
included in one data entry is completed. Steps S4101-S4105 are executed by the time
indicated by the PTS included in the same PES packet as the graphics object.
[0272]
The IG decoder 4073 decodes the TS packets received from the PJD filter 4040
into uncompressed graphics object. The IG decoder 4073 further writes the
uncompressed graphics object to the IG plane memory 4093 at the time shown by the
PTSs included in the PES packets restored from the TS packets. Details on these
processes are the same as in the PG decoder 4072.
[0273]
The primary audio decoder 4074 first stores the TS packets received from the
PID filter 4040 in a buffer provided therein. Subsequently, the primary audio decoder
4074 removes the TS header and the PES header from each TS packet in the buffer, and
decodes the remaining data into uncompressed LPCM audio data. Furthermore, the
primary audio decoder 4074 transmits the resultant audio data to the audio mixer 4095
at the time shown by the PTS included in the original PES packet. The primary audio
decoder 4074 selects the decoding method for compressed audio data in accordance
with the compression encoding method and stream attributes for the primary audio
stream included in the TS packets. Compression encoding methods include, for example,
AC-3 and DTS.
[0274]
The secondary audio decoder 4075 has the same structure as the primary audio
decoder 4074. The secondary audio decoder 4075 first reconstructs PES packets from
the TS packets of the secondary audio stream received from the PID filter 4040 and then
decodes the data included in the PES payloads into uncompressed LPCM audio data.
Subsequently, the secondary audio decoder 4075 transmits the uncompressed LPCM
audio data to the audio mixer 4095 at the times shown by the PTSs included in the PES
headers. The secondary audio decoder 4075 selects the decoding method for
compressed audio data in accordance with the compression encoding method and
stream attributes for the secondary audio stream included in the TS packets.
Compression encoding methods include, for example, Dolby Digital Plus and DTS-HD
LBR.
[0275]
The audio mixer 4095 receives uncompressed audio data from both the primary
audio decoder 4074 and the secondary audio decoder 4075 and then mixes the received
data. The audio mixer 4095 also transmits the synthesized sound yielded by mixing
audio data to, for example, an internal speaker 103 A of the display device 103.
[0276]
The image processor 4080 receives graphics data, i.e., PNG or JPEG raster data,
from the program execution unit 3734. Upon receiving the graphics data, the image
processor 4080 renders the graphics data and writes the graphics data to the image plane
memory 4094.
[0277]
[0278]
When playing back 3D video content from the BD-ROM disc 101 in 3D
playback mode, the playback device 102 operates as a 3D playback device. The
fundamental part of the device's structure is identical to the 2D playback device shown
in Figs. 37 and 40. Therefore, the following is a description on sections of the structure
of the 2D playback device that are enlarged or modified. Details on the fundamental
parts of the 3D playback device can be found in the above description on the 2D
playback device. The 3D playback device also uses the same structure as the 2D
playback device for 2D playlist playback processing. Accordingly, the details on this
structure can be found in the description on the 2D playback device. The following
description assumes playback processing of 3D video images in accordance with 3D
[0279]
Fig. 42 is a functional block diagram of a 3D playback device 4200. The 3D
playback device 4200 includes a BD-ROM drive 4201, playback unit 4202, and control
unit 4203. The playback unit 4202 includes a switch 4220, first read buffer (hereinafter,
abbreviated as RBI) 4221, second read buffer (hereinafter, abbreviated as RB2) 4222,
system target decoder 4225, plane adder 4226, and HDMI communication unit 4227.
The control unit 4203 includes a dynamic scenario memory 4231, static scenario
memory 4232, user event processing unit 4233, program execution unit 4234, playback
control unit 4235, and player variable storage unit 4236. The playback unit 4202 and the
control unit 4203 are each implemented on a different integrated circuit, but may
alternatively be implemented on a single integrated circuit. In particular, the dynamic
scenario memory 4231, static scenario memory 4232, user event processing unit 4233,
and program execution unit 4234 have an identical structure with the 2D playback
device shown in Fig. 37. Accordingly, details thereof can be found in the above
description on the 2D playback device.
[0280]
When instructed by the program execution unit 4234 or other unit to perform
3D playlist playback processing, the playback control unit 4235 reads a PI from the 3D
playlist file stored in the static scenario memory 4232 in order, setting the read PI as the
current PL Each time the playback control unit 4235 sets a current PI, it sets operation
conditions on the system target decoder 4225 and the plane adder 4226 in accordance
with the STN table of the PI and the STN table SS in the 3D playlist file. Specifically,
the playback control unit 4235 selects the PID of the elementary stream for decoding
and transmits the PED, together with the attribute information necessary for decoding
the elementary stream, to the system target decoder 4225. If a PG stream or IG stream is
included in the elementary stream indicated by the selected PID, the playback control
unit 4235 specifies the reference offset ID 3201 and offset adjustment value 3202
allocated to the stream data, setting the reference offset ID 3201 and offset adjustment
value 3202 to the SPRM(27) and SPRM(28) in the player variable storage unit 4236.
The playback control unit 4235 also selects the presentation mode of each piece of plane
data in accordance with the offset during pop-up 3311 indicated by the STN table SS,
indicating the selected presentation mode to the system target decoder 4225 and plane
adder 4226.
[0281]
Next, in accordance with the current PI, the playback control unit 4235
indicates the range of the LBNs in the sector group recorded in the extent SS to be read
to the BD-ROM drive 4201 via the procedures in the description on Fig. 24E.
Meanwhile, the playback control unit 4235 refers to the extent start points in the clip
information file stored in the static scenario memory 4232 to generate information
indicating the boundary of the data blocks in each extent SS. This information indicates,
for example, the number of source packets from the top of the extent SS to each
boundary. The playback control unit 4235 then transmits this information to the switch
4220.
[0282]
The player variable storage unit 4236 includes the SPRMs shown in Fig. 38,
like the player variable storage unit 3736 in the 2D playback device. However, unlike
Fig. 38, SPRM(24) includes the first flag, and SPRM(25) includes the second flag, as
shown in Fig. 36. In this case, when the SPRM(24) is "0", the playback device 102 only
supports playback of 2D video images, and when the SPRM(24) is "1", the playback
device 102 also supports playback of 3D video images. The playback device 102 is in
L/R mode when the SPRM(25) is "0" and is in depth mode when the SPRM(25) is "1".
Furthermore, the playback device 102 is in 2D playback mode when the SPRM(25) is
"2".
[0283]
Furthermore, in the player variable storage unit 4236, unlike Fig. 38, the
SPRM(27) includes a storage area for a reference offset ID for each graphics plane, and
the SPRM(28) includes a storage area for an offset adjustment value for each graphics
plane. Fig. 43 is a table showing a data structure of SPRM(27) and SPRM(28). As
shown in Fig. 43, SPRM(27) includes an area for storing four types of reference offset
IDs 4310-4313. These reference offset IDs 4310, 4311, 4312, and 4313 are respectively
for a PG plane (PG_ref_offset_id), IG plane (IG_ref_offset_id), secondary video plane
(SV_ref_offset_id), and image plane (IM_ref_offset_id). The SPRM(28) includes an
area for storing four types of offset adjustment values 4320-4323. These offset
adjustment values 4320, 4321, 4322, and 4323 are respectively for a PG plane
(PGoffsetadjustment), IG plane (IG_offset_adjusrment), secondary video plane
(SV_offset_adjustment), and image plane (IM_offset_adjustment).
[0284]
The BD-ROM drive 4201 includes the same components as the BD-ROM drive
3701 in the 2D playback device shown in Fig. 37. Upon receiving an indication from
the playback control unit 4235 of a range of LBNs, the BD-ROM drive 4201 reads data
from the sectors on the BD-ROM disc 101 as indicated by the range. In particular, a
source packet group belonging to an extent in the file SS, i.e. belonging to an extent SS,
are transmitted from the BD-ROM drive 4201 to the switch 4220. Each extent SS
includes one or more pairs of a base-view and dependent-view data block, as shown in
Fig. 19. These data blocks have to be transferred in parallel to different RBI 4221 and
RB2 4222 . Accordingly, the BD-ROM drive 4201 is required to have at least the same
access speed as the BD-ROM drive 3701 in the 2D playback device.
[0285]
The switch 4220 receives an extent SS from the BD-ROM drive 4201. On the
other hand, the switch 4220 receives, from the playback control unit 4235, information
indicating the boundary in each data block included in the extent SS, i.e. the number of
source packets from the top of the extent SS to each boundary. The switch 4220 then
refers to this information (i) to extract base-view extents from each extent SS and
transmit the extents to the RBI 4221, and (ii) to extract dependent-view extents and
transmit the extents to the RB2 4222.
[0286]
The RBI 4221 and RB2 4222 are buffer memories that use a memory element
in the playback unit 4202. In particular, different areas in a single memory element are
used as the RBI 4221 and RB2 4222 . Alternatively, different memory elements may be
used as the RBI 4221 and RB2 4222 . The RBI 4221 receives base-view extents from
the switch 4220 and stores these extents. The RB2 4222 receives dependent-view
extents from the switch 4220 and stores these extents.
[0287]
In 3D playlist playback processing, the system target decoder 4225 first
receives PIDs for stream data to be decoded, as well as attribute information necessary
for decoding the stream data, from the playback control unit 4235. The system target
decoder 4225 then reads source packets alternately from base-view extents stored in the
RBI 4221 and dependent-view extents stored in the RB2 4222. Next, the system target
decoder 4225 separates, from each source packet, elementary streams indicated by the
PIDs received from the playback control unit 4235 and decodes the elementary streams.
The system target decoder 4225 then writes the decoded elementary streams in internal
plane memory according to the type thereof. The base-view video stream is written in
the left-video plane memory, and the dependent-view video stream is written in the
right-video plane memory. On the other hand, the secondary video stream is written in
the secondary video plane memory, the IG stream in the IG plane memory, and the PG
stream in the PG plane memory. When the secondary video stream is composed of a
pair of a base-view and a dependent-view video stream, separate secondary video plane
memories are prepared for both the left-view and right-view pieces of plane data. The
system target decoder 4225 additionally renders graphics data from the program
execution unit 4234, such as JPEG, PNG, etc. raster data, and writes this data in the
image plane memory.
[0288]
The system target decoder 4225 associates the output mode of plane data from
the left-video and right-video plane memories with B-D presentation mode and B-B
presentation mode as follows. When the playback control unit 4235 indicates B-D
presentation mode, the system target decoder 4225 alternately outputs plane data from
the left-video and right-video plane memories. On the other hand, when the playback
control unit 4235 indicates B-B presentation mode, the system target decoder 4225
outputs plane data from only the left-video or right-video plane memory twice per frame
while maintaining the operation mode in 3D playback mode.
[0289]
When the playback control unit 4235 indicates 1 plane + offset mode, then
each time the system target decoder 4225 reads the VAU at the top of each video
sequence from the dependent-view video stream, the system target decoder 4225 reads
the offset metadata 1110 from the VAU. In the playback section of the video sequence,
the system target decoder 4225 first specifies the PTS stored in the same PES packet
along with each VAU and specifies the number of the frame represented by the
compressed picture data of the VAU. The system target decoder 4225 then reads the
offset information associated with the frame number from the offset metadata and
transmits the offset information to the plane adder 4226 at the time indicated by the
specified PTS.
[0290]
The plane adder 4226 receives each type of plane data from the system target
decoder 4225 and superimposes these pieces of plane data on one another to create one
combined frame or field. In particular, in L/R mode, left-video plane data represents a
left-view video plane, and right-view plane data represents a right-view video plane.
Accordingly, the plane adder 4226 superimposes other plane data representing the left
view on the left-video plane data and superimposes other plane data representing the
right view on the right-video plane data. On the other hand, in depth mode, the
right-video plane data represents a depth map for the video plane representing the
left-video plane data. Accordingly, the plane adder 4226 first generates a pair of
left-view and right-view pieces of video plane data from the corresponding pieces of
video plane data. Subsequently, the plane adder 4226 performs the same combination
processing as in L/R mode.
[0291]
When receiving an indication of 1 plane + offset mode or 1 plane + zero offset
mode from the playback control unit 4235 as the presentation mode for the secondary
video plane, PG plane, IG plane, or image plane, the plane adder 4226 performs offset
control on the plane data received from the system target decoder 4225. A pair of
left-view plane data and right-view plane data is thus generated.
[0292]
In particular, when 1 plane + offset mode is indicated, the plane adder 4226
first reads one of the reference offset IDs 4310-4313 that corresponds to each graphics
plane from the SPRM(27) in the player variable storage unit 4236. Next, the plane adder
4226 refers to the offset information received from the system target decoder 4225 to
retrieve offset information, namely an offset direction 1122 and offset value 1123,
belonging to the offset sequence 1113 indicated by each reference offset ID 4310-4313.
Subsequently, the plane adder 4226 reads one of the offset adjustment values 4320-4323
that corresponds to each graphics plane from the SPRM(28) in the player variable
storage unit 4236 and adds each offset adjustment value to the corresponding offset
value. The plane adder 4226 then uses each offset value to perform offset control on the
corresponding graphics plane.
[0293]
On the other hand, when 1 plane + zero offset mode is indicated, the plane
adder 4226 does not refer to either SPRM(27) or SPRM(28), but rather performs offset
control on each graphics plane with an offset value of "0". Accordingly, the same plane
data is used for both the left-view and right-view graphics planes and combined with
[0294]
The HDMI communication unit 4227, connected with the display device 103
via an HDMI cable 122, exchanges CEC messages with the display device 103 via the
HDMI cable 122. This causes the HDMI communication unit 4227 to perform an
HDMI authentication of the display device 103 and ask the display device 103 whether
or not playback of 3D video images is supported.
[0295]
«3D Playlist Playback Processing»
[0296]
Fig. 44 is a flowchart of 3D playlist playback processing by a playback control
unit 4235. 3D playlist playback processing is started by the playback control unit 4235
reading a 3D playlist file from the static scenario memory 4232.
[0297]
In step S4401, the playback control unit 4235 first reads a single PI from a
main path in the 3D playlist file and then sets the PI as the current PI. Next, from the
STN table of the current PI, the playback control unit 4235 selects PIDs of elementary
streams to be played back and specifies attribute information necessary for decoding the
elementary streams. The playback control unit 4235 further selects, from among the
elementary streams corresponding to the current PI in the STN table SS in the 3D
playlist file, a PID of elementary streams that are to be added to the elementary streams
to be played back, and playback control unit 4235 specifies attribute information
necessary for decoding these elementary streams. The selected PIDs and attribute
information are indicated to the system target decoder 4225. The playback control unit
4235 additionally specifies, from among sub-paths in the 3D playlist file, a SUB_PI to
be referenced at the same time as the current PI, specifying this SUB_PI as the current
SUB_PI. Thereafter, processing proceeds to step S4402.
In step S4402, the playback control unit 4235 selects the display mode for each
piece of plane data based on the offset during pop-up indicated by the STN table SS and
indicates the display mode to the system target decoder 4225 and the plane adder 4226.
In particular, when the value of the offset during pop-up is "0", B-D presentation mode
is selected as the video plane presentation mode, and 1 plane + offset mode is selected
as the presentation mode for the graphics plane. On the other hand, when the value of
the offset during pop-up is "1", B-B presentation mode is selected as the video plane
presentation mode, and 1 plane + zero offset mode is selected as the presentation mode
for the graphics plane. Thereafter, processing proceeds to step S4403.
[0299]
In step S4403, the playback control unit 4235 checks whether 1 plane + offset
mode or 1 plane + zero offset mode has been selected as the presentation mode of the
graphics plane. If 1 plane + offset mode has been selected, processing proceeds to step
S4404. If 1 plane + zero offset mode has been selected, processing proceeds to step
S4405.
[0300]
In step S4404, the playback control unit 4235 refers to the STN table of the
current PI and retrieves the PG stream, IG stream, or text subtitle stream from among
the elementary streams indicated by the selected PIDs. Furthermore, the playback
control unit 4235 specifies the reference offset ID and offset adjustment value allocated
to the pieces of stream data, setting the reference offset ID and offset adjustment value
to the SPRM(27) and SPRM(28) in the player variable storage unit 4236. Thereafter,
processing proceeds to step S4405.
[0301]
In step S4405, the playback control unit 4235 reads reference clip information,
a PTS #1 indicating a playback start time INI, and a PTS #2 indicating a playback end
time OUT1 from the current PI and the SUB_PI. From this reference clip information, a
clip information file corresponding to each of the file 2D and the file DEP to be played
In step S4406, with reference to the entry map in each of the clip information
files specified in step S4405, the playback control unit 4235 retrieves the SPN #1 and
SPN #2 in the file 2D, and the SPN #11 and SPN #12 in the file DEP, corresponding to
the PTS #1 and the PTS #2. As described with reference to Fig. 24, referring to extent
start points of each clip information file, the playback control unit 4235 further
calculates, from the SPN #1 and the SPN #11, the number of source packets SPN #21
from the top of the file SS to the playback start position. The playback control unit 4235
also calculates, from the SPN #2 and the SPN #12, the number of source packets SPN
#22 from the top of the file SS to the playback end position. Specifically, the playback
control unit 4235 first retrieves, from among SPNs shown by extent start points of the
2D clip information files, a value "Am" that is the largest value less than or equal to
SPN #1, and retrieves, from among the SPNs shown by extent start points of
dependent-view clip information files, a value "Bm" that is the largest value less than or
equal to the SPN #11. Next, the playback control unit 4235 obtains the sum of the
retrieved SPNs Am + Bm and sets the sum as SPN #21. Next, the playback control unit
4235 retrieves, from among SPNs shown by the extent start points of the 2D clip
information files, a value "An" that is the smallest value that is larger than the SPN #2.
The playback control unit 4235 also retrieves, from the SPNs of the extent start points
of the dependent-view clip information files, a value "Bn" that is the smallest value that
is larger than the SPN #12. Next, the playback control unit 4235 obtains the sum of the
retrieved SPNs An + Bn and sets the sum as SPN #22. Thereafter, processing proceeds
to step S4407.
[0303]
In step S4407, the playback control unit 4235 converts the SPN #21 and the
SPN #22, determined in step S4406, into a pair of numbers of sectors Nl and N2.
Specifically, the playback control unit 4235 first obtains the product of SPN #21 and the
data amount per source packet, i.e. 192 bytes. Next, the playback control unit 4235
divides this product the data amount per sector, i.e. 2048 bytes: SPN #21 x 192 / 2048.
The resulting quotient is the same as the number of sectors Nl from the top of the file
SS to immediately before the playback start position. Similarly, from the SPN #22, the
playback control unit 4235 calculates SPN #22 x 192 / 2048. The resulting quotient is
the same as the number of sectors N2 from the top of the file SS to immediately before
the playback end position. Thereafter, processing proceeds to step S4408.
[0304]
In step S4408, the playback control unit 4235 specifies, from the numbers of
sectors Nl and N2 obtained in step S4407, LBNs of the top and end of the extent SS
group to be played back. Specifically, with reference to the file entry of the file SS to be
played back, the playback control unit 4235 counts from the top of sector group in
which the extent SS group is recorded so that the LBN of the (Nl + 1) sector = LBN
#1, and the LBN of the (N2 + I)* sector = LBN #2. The playback control unit 4235
further specifies a range from the LBN#1 to the LBN#2 to the BD-ROM drive 4201. As
a result, from the sector group in the specified range, a source packet group belonging to
an extent SS group is read in aligned units. Thereafter, processing proceeds to step
S4409.
[0305]
In step S4409, referring to the extent start points of the clip information file
used in step S4406, the playback control unit 4235 generates information (hereinafter
referred to as "data block boundary information") indicating a boundary between
dependent-view blocks and base-view data blocks included in the extent SS group,
transmitting the data block boundary information to the switch 4220. As a specific
example, assume that the SPN #21 indicating the playback start position is the same as
the sum of SPNs indicating the extent start points, An + Bn, and that the SPN# 22
indicating the playback end position is the same as the sum of SPNs indicating the
extent start points, Am + Bm. In this case, the playback control unit 4235 obtains a
sequence of differences between SPNs from the respective extent start points, A(n + 1) -
An, B(n + 1) - Bn, A(n + 2) - A(n + 1), B(n + 2) - B(n + 1), ..., Am - A(m - 1), and Bm
- B(m - 1), and transmits the sequence to the switch 4220 as the data block boundary
information. As shown in Fig. 24E, this sequence indicates the number of source
packets of data blocks included in the extent SS. The switch 4220 counts, from zero, the
number of source packets of the extents SS received from the BD-ROM drive 4201.
Each time the count is the same as the difference between SPNs indicated by the data
block boundary information, the switch 4220 switches the destination of output of the
source packets between the RBI 4221 and the RB2 4222 and resets the count to zero.
As a result, {B(n + 1) - Bn} source packets from the top of the extent SS are output to
the RB2 4222 as the first dependent-view extent, and the following {A(n + 1) - An}
source packets are transmitted to the RBI 4221 as the first base-view extent. Thereafter,
dependent-view extents and base-view extents are extracted from the extent SS
alternately in the same way, alternating each time the number of source packets received
by the switch 4220 is the same as the difference between SPNs indicated by the data
block boundary information.
[0306]
In step S4410, the playback control unit 4235 checks whether an unprocessed
PI remains in the main path. When an unprocessed PI remains, processing is repeated
from step S4401. When no unprocessed PI remains, processing ends.
[0307]
«System Target Decoder»
[0308]
The following two means are conceivable as specific means used by the system
target decoder 4225 to implement the function to extract offset metadata from the
dependent-view video stream. The first means incorporates a TS priority filter and an
offset metadata processing unit into the system target decoder 4225 as modules separate
from the primary video decoder. The TS priority filter selects TS packets containing
offset metadata and TS packets containing dependent-view pictures, depending on the
values of TS priority flags. The offset metadata processing unit extracts offset
information from the TS packets containing offset metadata. The second means causes
TS packets containing a dependent-view video stream to be sent to the primary video
decoder in the system target decoder 4225, regardless of the values of TS priority flags.
The primary video decoder extracts offset information from the dependent-view video
stream in parallel with the process of decoding the dependent-view video stream.
[0309]
(First Means)
[0310]
Fig. 45 is a functional block diagram of the system target decoder 4225 by
using the first means. The components shown in Fig. 45 differ from the components
of the system target decoder 3724 in the 2D playback device shown in Fig. 40 as
follows: (A) the input system from the read buffers to the decoders has a duplex
configuration; and (B) the TS priority filter and the offset metadata processing unit are
provided. The primary audio decoder, secondary audio decoder, audio mixer, image
processor, and plane memories have similar structures to those in the 2D playback
device shown in Fig. 40. Accordingly, among the components shown in Fig. 45, ones
different from the components shown in Fig. 40 are described below. On the other
hand, details of similar components can be found in the description on Fig. 40.
Furthermore, since the video decoders each have a similar structure, the structure of the
primary video decoder 4515 is described below. This description is also valid for the
structures of other video decoders.
[0311]
The first source depacketizer 4511 reads source packets from the first read
buffer 4221. The first source depacketizer 4511 further retrieves TS packets included
in the source packets and transmits the TS packets to the first PID filter 4513. The
second source depacketizer 4512 reads source packets from the second read buffer 4222,
furthermore retrieving TS packets included in the source packets and transmitting the
TS packets to the second PID filter 4514. Each of the source depacketizers 4511 and
4512 further synchronizes the time of transferring the TS packets with the time shown
by the ATS of each source packet. This synchronization method is the same method
as the source depacketizer 4010 shown in Fig. 40. Accordingly, details thereof can be
found in the description provided for Fig. 40. With this sort of adjustment of transfer
time, the mean transfer rate of TS packets from the first source depacketizer 4511 to the
first PID filter 4513 does not exceed the system rate Rxsi indicated by the 2D clip
information file. Similarly, the mean transfer rate of TS packets from the second
source depacketizer 4512 to the second PID filter 4514 does not exceed the system rate
Rts2 indicated by the dependent-view clip information file.
[0312]
The first PID filter 4513 compares the PID of each TS packet received from the
first source depacketizer 4511 with the selected PID. The playback control unit 4235
designates the selected PID beforehand in accordance with the STN table in the 3D
playlist file. When the two PIDs match, the first PID filter 4513 transfers the TS packets
to the decoder assigned to the PID. For example, if a PID is 0x1011, the TS packets are
transferred to TBI 4501 in the primary video decoder 4515. On the other hand, TS
packets with PIDs ranging from OxlBOO-OxlBlF, 0x1100-0x11 IF, OxlAOO-OxlAlF,
0x1200-0x12IF, and 0x1400-0x14IF are transferred to the secondary video decoder,
primary audio decoder, secondary audio decoder, PG decoder, and IG decoder,
respectively.
[0313]
The second PID filter 4514 compares the PID of each TS packet received from
the second source depacketizer 4512 with the selected PID. The playback control unit
4235 designates the selected PID beforehand in accordance with the STN table SS in
the 3D playlist file. When the two PIDs match, the second PID filter 4514 transfers
the TS packets to the decoder assigned to the PID or the TS priority filter 4551. For
example, if a PID is 0x1012 or 0x1013, the TS packets are transferred to the TS priority
filter 4551. On the other hand, TS packets with PIDs ranging from 0xlB20-0xlB3F,
Oxl220-Oxl27F, and 0xl420-0xl47F are transferred to the secondary video decoder,
PG decoder, and IG decoder, respectively.
[0314]
The TS priority filter 4551 receives TS packets from the second PID filter 4514
and reads TS priority 511 from the TS header 501H in each of the TS packets. Here, TS
packets with PID = 0x1012 or 0x1013 are transferred from the second PID filter 4514 to
the TS priority filter 4551. These TS packets contain a dependent-view video stream.
[0315]
Among the TS packets in the sequence 1520 shown in Fig. 15, the first group
1521 and the third group 1523 have the TS priority of "0," and the second group 1522
has the TS priority of "1". The TS priority filter 4551 transfers TS packets with TS
priority = 0 from the sequence 1520 to the TB2 4508 in the primary video decoder 4515,
and TS packets with TS priority = 1 to the offset metadata processing unit 4552. As
shown in Fig. 15, the TS packets with TS priority = 1 belong to the second group 1522.
Accordingly, the TS payloads thereof include only the supplementary data 1504
consisting only of the offset metadata 1509. As a result, among the VAU #1 in the
dependent-view video stream, supplementary data consisting only of the offset metadata
1509 is transferred to the offset metadata processing unit 4552, and the remaining data,
which include other supplementary data, are transferred to the primary video decoder
4515.
[0316]
Among the TS packets in the sequence 1620 shown in Fig. 16, the first group
1621 and the second group 1622 have the TS priority of "1," and the third group 1623
has the TS priority of "0". The TS priority filter 4551 transfers TS packets with TS
priority = 0 from the sequence 1620 to the TB2 4508 in the primary video decoder 4515,
and TS packets with TS priority = 1 to both the TB2 4508 and the offset metadata
processing unit 4552. Accordingly, VAU #1 in the dependent-view video stream is
transferred to the primary video decoder 4515, while the elements from the sub-AU
identification code to the supplementary data are transferred to the offset metadata
processing unit 4552 as well.
[0317]
The primary video decoder 4515 includes a TBI 4501, MB1 4502, EB1 4503, TB2
4508, MB2 4509, EB2 4510, buffer switch 4506, DEC 4504, DPB 4505, and picture
switch 4507. The TBI 4501, MB1 4502, EB1 4503, TB2 4508, MB2 4509, EB2 4510
and DPB 4505 are all buffer memories. Each of these buffer memories uses an area of a
memory element included in the primary video decoder 4515. Alternatively, some or all
of these buffer memories may be separated on different memory elements.
[0318]
The TBI 4501 receives TS packets that include a base-view video stream from
the first PED filter 4513 and stores the TS packets as they are. The MB1 4502 stores
PES packets reconstructed from the TS packets stored in the TBI 4501. The TS headers
of the TS packets are removed at this point. The EB1 4503 extracts and stores encoded
VAUs from the PES packets stored in the MB 1 4502. The PES headers of the PES
packets are removed at this point.
[0319]
The TB2 4508 receives TS packets that include the dependent-view video
stream from the TS priority filter 4551 and stores the TS packets as they are. The MB2
4509 stores PES packets reconstructed from the TS packets stored in the TB2 4508. The
TS headers of the TS packets are removed at this point. The EB2 4510 extracts and
stores encoded VAUs from the PES packets stored in the MB2 4509. The PES headers
of the PES packets are removed at this point.
[0320]
The buffer switch 4506 transfers the headers of the VAUs stored in the EB1
4503 and the EB2 4510 in response to a request from the DEC 4504. Furthermore, the
buffer switch 4506 transfers the compressed picture data for the VAUs to the DEC 4504
at the times indicated by the DTSs included in the original PES packets. In this case, the
DTSs are equal between a pair of pictures belonging to the same 3D VAU between the
base-view video stream and dependent-view video stream. Accordingly, for a pair of
VAUs that have the same DTS, the buffer switch 4506 first transmits the VAU stored in
the EB1 4503 to the DEC 4504. Additionally, the buffer switch 4506 may cause the
DEC 4504 to return the decoding switch information 1750 in the VAU. In such a case,
the buffer switch 4506 can determine if it should transfer the next VAU from the EB1
4503 or the EB2 4510 by referring to the decoding switch information 1750.
[0321]
Like the DEC 4004 shown in Fig. 40, the DEC 4504 is a hardware decoder
specifically for decoding of compressed pictures and is composed of an LSI that
includes, in particular, a function to accelerate the decoding. The DEC 4504 decodes the
compressed picture data transferred from the buffer switch 4506 in order. During
decoding, the DEC 4504 first analyzes each VAU header to specify the compressed
picture, compression encoding method, and stream attribute stored in the VAU,
selecting a decoding method in accordance with this information. Compression
encoding methods include, for example, MPEG-2, MPEG-4 AVC, MVC, and VC1.
Furthermore, the DEC 4504 transmits the decoded, uncompressed picture to the DPB
4505.
[0322]
The DPB 4505 temporarily stores the uncompressed pictures decoded by the
DEC 4504. When the DEC 4504 decodes a P picture or a B picture, the DPB 4505
retrieves reference pictures from among the stored, uncompressed pictures in response
to a request from the DEC 4504 and supplies the retrieved reference pictures to the DEC
4504.
[0323]
The picture switch 4507 writes the uncompressed pictures from the DPB 4505
to either the left-video plane memory 4520 or the right-video plane memory 4521 at the
time indicated by the PTS included in the original PES packet. In this case, the PTSs are
equal between a base-view picture and a dependent-view picture belonging to the same
3D VAU. Accordingly, for a pair of pictures that have the same PTS and that are stored
by the DPB 4505, the picture switch 4507 first writes the base-view picture in the
left-video plane memory 4520 and then writes the dependent-view picture in the
right-video plane memory 4521.
[0324]
The offset metadata processing unit 4552 is implemented on the same chip as
the primary video decoder 4515, but configured as a module separate from the primary
video decoder 4515. Alternatively, the offset metadata processing unit 4552 may be
implemented on a chip separate from the chip on which the primary video decoder 4515
is implemented. Furthermore, the offset metadata processing unit 4552 may be
configured as dedicated hardware or realized as general-purpose hardware controlled by
software. The offset metadata processing unit 4552 analyzes TS packets transferred
from the TS priority filter 4551, and then reads offset metadata from supplementary data
stored in the TS payloads of the TS packets.
[0325]
The sequence of TS packets 1520 shown in Fig. 15 contains the PES header
belonging to the same PES packet as VAU #1 in the group of TS packets to be
transferred to the primary video decoder 4515. Accordingly, the offset metadata
processing unit 4552 reads the PTS of the frame represented by the VAU #1 from the
offset metadata. On the other hand, the sequence of TS packets 1620 shown in Fig. 16
contains the PES header in the group of TS packets to be transferred to the offset
metadata processing unit 4552, as well as the group of TS packets to be transferred to
the primary video decoder 4515. Accordingly, the offset metadata processing unit
4552 may read the PTS of the frame represented by the VAU #1 from either of the PES
header and the offset metadata.
[0326]
The offset metadata processing unit 4552 increments the frame number by 1 at
frame intervals, starting from 0 at the time indicated by the PTS. In synchronization
with the incrementing action, the offset metadata processing unit 4552 further retrieves
offset information associated with each frame number from the offset metadata, and
then transmits the offset information to the plane adder 4226. Here, the TS priority
filter 4551 prevents compressed picture data from being transferred from either of the
sequences of TS packets 1520 and 1620 shown in Figs. 15 and 16 to the offset metadata
processing unit 4552. Accordingly, the offset metadata processing unit 4552 can
reliably manage the offset information without interference from the compressed picture
data.
[0327]
(Second Means)
[0328]
Fig. 46 is a functional block diagram showing a system for processing video
streams, the system included in the system target decoder 4225 that uses the second
means. The system target decoder 4225 shown in Fig. 46 differs from that shown in
Fig. 45 in the function of the DEC 4604 in the primary video decoder 4614. Other
components are similar to corresponding ones. In Fig. 46, components similar to ones
shown in Fig. 45 are marked with the same reference numbers. Furthermore, details of
the similar components can be found in the description on Fig. 45.
[0329]
Like the DEC 4504 shown in Fig. 45, the DEC 4604 is a hardware decoder
specifically for decoding of compressed pictures, and is composed of an LSI that
includes, in particular, a function to accelerate the decoding. The DEC 4604 decodes
the compressed picture data transferred from the buffer switch 4506 in order, and
transfers the decoded uncompressed pictures to the DPB 4505. Furthermore, each
time it reads a VAU located at the top of each video sequence from the dependent-view
video stream, the DEC 4604 reads offset metadata from the VAU. In the playback
section of the video sequence, the DEC 4604 first identifies the PTS which is stored in
the same PES packet as the VAU, and the frame number represented by the compressed
picture data of the VAU. Next, the DEC 4604 reads offset information associated with
the frame number from the offset metadata, and sends the offset information to the
plane adder 4226 at the time indicated by the identified PTS.
[0330]
«Plane Adders»
[0331]
Fig. 47 is a functional block diagram of a plane adder 4226. As shown in Fig.
47, the plane adder 4226 includes a parallax video generation unit 4710, switch 4720,
four cropping units 4731-4734, and four adders 4741-4744.
[0332]
The parallax video generation unit 4710 receives left-video plane data 4701
and right-video plane data 4702 from the system target decoder 4225. In the playback
device 102 in L/R mode, the left-video plane data 4701 represents the left-view video
plane, and the right-video plane data 4702 represents the right-view video plane. At this
point, the parallax video generation unit 4710 transmits the left-video plane data 4701
and the right-video plane data 4702 as they are to the switch 4720. On the other hand, in
the playback device 102 in depth mode, the left-video plane data 4701 represents the
video plane for 2D video images, and the right-video plane data 4702 represents a depth
map for the 2D video images. In this case, the parallax video generation unit 4710 first
calculates the binocular parallax for each element in the 2D video images using the
depth map. Next, the parallax video generation unit 4710 processes the left-video plane
data 4701 to shift the presentation position of each element in the video plane for 2D
video images to the left or right according to the calculated binocular parallax. This
generates a pair of video planes representing the left view and right view. Furthermore,
the parallax video generation unit 4710 transmits the pair of video planes to the switch
4720 as a pair of pieces of left-video and right-video plane data.
[0333]
When the playback control unit 4235 indicates B-D presentation mode, the
switch 4720 transmits left-video plane data 4701 and right-video plane data 4702 with
the same PTS to the first adder 4741 in that order. When the playback control unit 4235
indicates B-B presentation mode, the switch 4720 transmits one of the left-video plane
data 4701 and right-video plane data 4702 with the same PTS twice per frame to the
first adder 4741, discarding the other piece of plane data.
[0334]
The first cropping unit 4731 includes the same structure as a pair of the
parallax video generation unit 4710 and switch 4720. These structures are used when
the secondary video plane data is a pair of a left view and a right view. In particular, in
the playback device 102 in depth mode, the parallax video generation unit in the first
cropping unit 4731 converts the secondary video plane data into a pair of left-view and
right-view pieces of plane data. When the playback control unit 4235 indicates B-D
presentation mode, the left-view and right-view pieces of plane data are alternately
transmitted to the first adder 4741. On the other hand, when the playback control unit
4235 indicates B-B presentation mode, one of the left-view and right-view pieces of
plane data is transmitted twice per frame to the first adder 4741, and the other piece of
plane data is discarded.
[0335]
When the playback control unit 4235 indicates 1 plane + offset mode, the first
cropping unit 4731 performs the following offset control on the secondary video plane
data 4703. The first cropping unit 4731 first receives offset information 4707 from the
system target decoder 4225. At this point, the first cropping unit 4731 reads the
reference offset ID (SV_ref_offset_id) 4212 corresponding to the secondary video plane
from the SPRM(27) 4751 in the player variable storage unit 4236. Next, the first
cropping unit 4731 retrieves the offset information belonging to the offset sequence
indicated by the reference offset ED from the offset information 4707 received from the
system target decoder 4225. Subsequently, the first cropping unit 4731 reads the offset
adjustment value (SV_offset_adjustment) 4222 corresponding to the secondary video
plane from the SPRM(28) 4752 in the player variable storage unit 4236 and adds the
offset adjustment value to the retrieved offset value. After that, the first cropping unit
4731 refers to the offset value to perform offset control on the secondary video plane
data 4703. As a result, the secondary video plane data 4703 is converted into a pair of
pieces of secondary video plane data representing a left view and a right view, and this
pair is alternately output.
[0336]
The playback control unit 4235 generally updates the values of the SPRM(27)
4751 and SPRM(28) 4752 each time the current PI changes. Additionally, the program
execution unit 4234 may set the values of the SPRM(27) 4751 and the SPRM(28) 4752
in accordance with a movie object or BD-J object.
[0337]
On the other hand, when the playback control unit 4235 indicates 1 plane +
zero offset mode, the first cropping unit 4731 does not perform offset control, instead
outputting the secondary video plane data 4703 twice as is.
[0338]
Similarly, the second cropping unit 4732 refers to the reference offset ID
(PG_ref_offset_id) 4310 for the PG plane and to the offset adjustment value
(PG_offset_adjustment) 4320 to perform offset control on the PG plane data 4704. The
third cropping unit 4733 refers to the reference offset ED (IG_ref_offset_id) 4311 for the
IG plane and to the offset adjustment value (IG_offset_adjustment) 4321 to perform
offset control on the IG plane data 4705. The first cropping unit 4734 refers to the
reference offset ID (EM_ref_offset_id) 4213 for the image plane and to the offset
adjustment value (EM_offset_adjustment) 4323 to perform offset control on the image
plane data 4706.
[Flowchart of Offset Control]
[0340]
Fig. 48 is a flowchart of offset control by the cropping units 4731-4734. Each
of the cropping units 4731-4734 begins offset control upon receiving offset information
4707 from the system target decoder 4225. In the following example, the second
cropping unit 4732 performs offset control on the PG plane data 4704. The other
cropping units 4731, 4733, and 4734 perform similar processing respectively on the
secondary video plane data 4703, IG plane data 4705, and image plane data 4706.
[0341]
In step S4801, the second cropping unit 4732 first receives PG plane data 4704
from the system target decoder 4225. At this point, the second cropping unit 4732 reads
the reference offset ID (PG_ref_offset_id) 4310 for the PG plane from the SPRM(27)
4751. Next, the second cropping unit 4731 retrieves the offset information belonging to
the offset sequence indicated by the reference offset ID 4310 from the offset
information 4707 received from the system target decoder 4225. Thereafter, processing
proceeds to step S4802.
[0342]
In step S4802, the second cropping unit 4732 reads the offset adjustment value
(PG_offset_adjustment) 4320 for the PG plane from the SPRM(28) 4752 and adds this
offset adjustment value to the offset value retrieved in step S4801. Thereafter,
processing proceeds to step S4803.
[0343]
In step S4803, the second cropping unit 4732 checks which of a left view and a
right view is represented by the video plane data selected by the switch 4720. If the
video plane data represents a left view, processing proceeds to step S4804. If the video
plane data represents a right view, processing proceeds to step S4807.
In step S4804, the second cropping unit 4732 checks the value of the retrieved
offset direction. Hereinafter, the following is assumed: if the offset direction value is
"0", the 3D graphics image is closer to the viewer than the screen, and if the offset
direction value is "1", the image is further back than the screen. In this context, when
the offset direction value is "0", processing proceeds to step S4805. If the offset
direction value is "1", processing proceeds to step S4806.
[0345]
In step S4805, the second cropping unit 4732 provides a right offset to the PG
plane data 4704. In other words, the position of each piece of pixel data included in the
PG plane data 4704 is shifted to the right by the offset value. Thereafter, processing
proceeds to step S4810.
[0346]
In step S4806, the second cropping unit 4732 provides a left offset to the PG
plane data 4704. In other words, the position of each piece of pixel data included in the
PG plane data 4704 is shifted to the left by the offset value. Thereafter, processing
proceeds to step S4810.
[0347]
In step S4807, the second cropping unit 4732 checks the value of the retrieved
offset direction. If the offset direction value is "0", processing proceeds to step S4808.
If the offset direction value is "1", processing proceeds to step S4809.
[0348]
In step S4808, the second cropping unit 4732 provides a left offset to the PG
plane data 4704, contrary to step S4805. In other words, the position of each piece of
pixel data included in the PG plane data 4704 is shifted to the left by the offset value.
Thereafter, processing proceeds to step S4810.
[0349]
In step S4809, the second cropping unit 4732 provides a right offset to the PG
plane data 4704, contrary to step S4806. In other words, the position of each piece of
pixel data included in the PG plane data 4704 is shifted to the right by the offset value.
Thereafter, processing proceeds to step S4810.
[0350]
In step S4810, the second cropping unit 4732 outputs the processed PG plane
data 4704 to the third cropping unit 4734. Processing then terminates.
[0351]
[Changes in Plane Data via Offset Control]
[0352]
Fig. 49B is a schematic diagram showing PG plane data GP to which the
second cropping unit 4732 is to provide offset control. As shown in Fig. 49B, the PG
plane data GP includes pixel data representing the subtitle "I love you", i.e. subtitle data
STL. This subtitle data STL is located at a distance DO from the left edge of the PG
plane data GP.
[0353]
Fig. 49A is a schematic diagram showing PG plane data RPG to which a right
offset has been provided. As shown in Fig. 49A, when providing a right offset to the PG
plane data GP, the second cropping unit 4732 changes the position of each piece of
pixel data in the PG plane data GP from its original position to the right by a number of
pixels OFS equal to the offset value. Specifically, the second cropping unit 4732
performs cropping to remove, from the right edge of the PG plane data GP, pixel data
included in a strip AR1 of a width OFS equal to the offset value. Next, the second
cropping unit 4732 forms a strip AL1 of width OFS by adding pixel data to the left edge
of the PG plane data GP. The pixel data included in this strip AL1 is set as transparent.
This process yields PG plane data RGP to which a right offset has been provided.
Subtitle data STL is actually located at a distance DR from the left edge of this PG
plane data RGP. This distance DR equals the original distance DO plus the offset value
OFS:DR = D0 + OFS.
Fig. 49C is a schematic diagram showing PG plane data LPG to which a left
offset has been provided. As shown in Fig. 49C, when providing a left offset to the PG
plane data GP, the second cropping unit 4732 changes the position of each piece of
pixel data in the PG plane data GP from its original position to the left by a number of
pixels OFS equal to the offset value. Specifically, the second cropping unit 4732
performs cropping to remove, from the left edge of the PG plane data GP, pixel data
included in a strip AL2 of a width OFS equal to the offset value. Next, the second
cropping unit 4732 forms a strip AR2 of width OFS by adding pixel data to the right
edge of the PG plane data GP. The pixel data included in this strip AR2 is set as
transparent. This process yields PG plane data LGP to which a left offset has been
provided. Subtitle data STL is actually located at a distance DL from the left edge of
this PG plane data RGP. This distance DL equals the original distance DO minus the
offset value OFS: DL = DO - OFS.
[0355]
Referring again to Fig. 47, the first adder 4741 receives video plane data from
the switch 4720 and receives secondary video plane data from the first cropping unit
4731. At this point, the first adder 4741 superimposes each pair of the plane data and
secondary video plane data and transmits the result to the second adder 4742. The
second adder 4742 receives PG plane data from the second cropping unit 4732,
superimposes this PG plane data on the plane data from the first adder 4741, and
transmits the result to the third adder 4743. The third adder 4743 receives IG plane data
from the third cropping unit 4733, superimposes this IG plane data on the plane data
from the second adder 4742, and transmits the result to the fourth adder 4744. The
fourth adder 4744 receives image plane data from the fourth cropping unit 4734,
superimposes this image plane data on the plane data from the third adder 4743, and
outputs the result to the display device 103. The adders 4741-4744 each make use of
alpha blending when superimposing plane data. In this way, the secondary video plane
data 4703, PG plane data 4704, IG plane data 4705, and image plane data 4706 are
superimposed in the order shown by the arrow 4700 in Fig. 47 on the left-video plane
data 4701 or right-video plane data 4702. As a result, the video images indicated by
each piece of plane data are displayed on the screen of the display device 103 so that the
left-video plane or right-video plane appears to overlap with the secondary video plane,
IG plane, PG plane, and image plane in that order.
[0356]
In addition to the above-stated processing, the plane adder 4724 converts the
output format of the plane data combined by the four adders 4741-4744 into a format
that complies with the display method of 3D video images adopted in a device such as
the display device 103 to which the data is output. If an alternate-frame sequencing
method is adopted in the device, for example, the plane adder 4724 outputs the
combined plane data pieces as one frame or one field. On the other hand, if a method
that uses a lenticular lens is adopted in the device, the plane adder 4724 combines a pair
of left-view and right-view pieces of plane data as one frame or one field of video data
with use of internal buffer memory. Specifically, the plane adder 4724 temporarily
stores and holds in the buffer memory the left-view plane data that has been combined
first. Subsequently, the plane adder 4724 combines the right-view plane data, and
further combines the resultant data with the left-view plane data held in the buffer
memory. During combination, the left-view and right-view pieces of plane data are each
divided, in a vertical direction, into small rectangular areas that are long and thin, and
the small rectangular areas are arranged alternately in the horizontal direction in one
frame or one field so as to re-constitute the frame or the field. In this way, the pair of
left-view and right-view pieces of plane data is combined into one video frame or field.
The plane adder 4724 then outputs the combined video frame or field to the
corresponding device.
[0357]
In the BD-ROM disc 101 according to Embodiment 1 of the present invention,
as shown in Figs. 15 and 16, TS priorities are assigned to a sequence of TS packets
which stores a VAU at the top of each video sequence constituting the dependent-view
video stream. In particular, different TS priorities are assigned to a TS packet group
storing offset metadata and a TS packet group storing compressed picture data. In that
case, the function to extract offset metadata may be realized in the system target
decoder 4225 by the first means shown in Fig. 45 or the second means shown in Fig. 46.
The system target decoder 4225 by the first means can easily sort out TS packets storing
offset metadata from the other by using the TS priorities. Accordingly, the primary
video decoder 4515 and the offset metadata processing unit 4552 can be implemented in
different forms. Especially even if the primary video decoder 4515 is composed of
hardware, irrespectively thereof the offset metadata processing unit 4552 may be
composed of dedicated hardware or realized by software with use of general-purpose
hardware. On the other hand, in the system target decoder 4225 by the second means,
the primary video decoder 4615 can execute in parallel both the function to decode the
dependent-view video stream and the function to extract offset metadata. Thus
regardless of the TS priorities all TS packets storing the dependent-view video stream
can be passed to the primary video decoder 4615. In this way, the data structure of the
dependent-view video stream and the offset metadata recorded on the BD-ROM disc
101 according to Embodiment 1 of the present invention can be used in common by the
system target decoder 4225 realized by the first means and the system target decoder
4225 realized by the second means.
[0359]
[0360]
(1-A) Video Stream
[0361]
In L/R mode according to Embodiment 1 of the present invention, the
base-view video stream represents the left view, and the dependent-view video stream
represents the right view. Conversely, however, the base-view video stream may
represent the right view and the dependent-view video stream the left view.
[0362]
On the BD-ROM disc 101 according to Embodiment 1 of the present invention,
the base-view video stream and the dependent-view video stream are multiplexed in
different TSs. Alternatively, the base-view video stream and the dependent-view video
stream may be multiplexed into a single TS.
[0363]
(1-B) Offset Metadata
[0364]
(1-B-l) The offset metadata may be stored in the base-view video stream
instead of in the dependent-view video stream. In this case as well, the offset metadata
is preferably stored in the supplementary data in the VAU located at the top of each
video sequence. Furthermore, the 3D playlist file may be provided with a flag indicating
whether the base-view video stream or the dependent-view video stream includes the
offset metadata. This allows for an increase in the degree of freedom when creating
each piece of stream data. Also, it may be prescribed that this flag is "prohibited from
being changed during between Pis in which video images are seamlessly connected via
CC = 5, 6".
[0365]
(l-B-2) Offset metadata may be stored in each VAU (i.e., each frame or field)
instead of only being stored in the top VAU in each video sequence (i.e., each GOP).
Alternatively, offset metadata may be set at arbitrary intervals, such as three frames or
greater, for each content. In this case, it is preferable that offset metadata always be
stored in the top VAU in each video sequence and that the interval between the offset
metadata and the immediately prior offset metadata be restricted to equal to or greater
than three frames. Accordingly, the playback device can reliably perform processing to
change offset information in parallel with interrupt playback.
[0366]
(l-B-3) Instead of being stored in the video stream, offset metadata may be
multiplexed in a main TS or a sub-TS as independent stream data. In this case, a unique
PED is allocated to the offset metadata. The system target decoder refers to this PID to
separate the offset metadata from other stream data. Alternatively, the offset metadata
may first be preloaded into a dedicated buffer and later undergo playback processing,
like the text subtitle stream. In this case, the offset metadata is stored at constant frame
intervals. Accordingly, a PTS is not necessary for the offset metadata, thus reducing the
data amount of the PES header. This reduces the capacity of the buffer for preloading.
[0367]
(l-B-4) Instead of being stored in the supplementary data of a VAU, offset
metadata may be embedded in the video stream with use of a video watermark.
Furthermore, the offset metadata may be embedded in the audio stream with use of an
audio watermark.
[0368]
(l-B-5) In the sub-TS according to Embodiment 1 of the present invention, as
shown in Fig. 15, the TS packets 1530 and 1550 located at the ends of the first group
1521 and the second group 1522, respectively, include AD fields 1532 and 1552 in
general. With this structure, the three groups 1521-1523 are separated from each other.
Alternatively, in the VAU #1 1500 in the dependent-view video stream, the size of the
padding data 1506 may be adjusted so that the three groups 1521-1523 are separated
from each other.
[0369]
(l-B-6) TS packets containing offset metadata may be selected in a system
target decoder depending on PIDs, instead of TS priority. Fig. 50 is a schematic
diagram showing a PES packet 5010 containing VAU #1 5000 in the dependent-view
video stream and a sequence of TS packets 5020 generated from the PES packet 5010.
The VAU #1 5000 is located at the top of the video sequence, and accordingly includes
supplementary data 5004 consisting only of offset metadata 5009. The PES payload
5012 of the PES packet 5010 contains the VAU #1 5000, and the PES header 5011
thereof includes DTS and PTS assigned to compressed picture data 5005 in the VAU #1
5000. The PES packet 5010 is stored in the TS packet sequence 5020 in order from
the top. With this arrangement, the TS packet sequence 5020 is divided into three
groups 5021, 5022, and 5023 in order from the top. The first group 5021 includes the
PES header 5011, sub-AU identification code 5001, sub-sequence header 5002, and
picture header 5003. The second group 5022 includes the supplementary data 5004
consisting only of the offset metadata 5009. The third group 5013 includes the
compressed picture data 5005, padding data 5006, sequence end code 5007, and stream
end code 5008. Hatched areas in Fig. 50 show the supplementary data 5004 consisting
only of the offset metadata 5009, and dotted areas show data 5011, 5001-5003 arranged
before the supplementary data in the PES packet 5010. Like the TS packets in the
sequence 1520 shown in Fig. 15, the TS packets 5030 and 5050 located at the ends of
the first group 5021 and the second group 5022, respectively, include AD fields 5032
and 5052 in general. With this structure, the three groups 5021-5023 are separated
from each other. The TS headers 5031 and 5061 of the TS packets 5030 and 5060
belonging to the first group 5021 and the third group 5023 each indicate PID = 0x1012.
Here, the TS headers 5031 of the TS packets 5030 belonging to the first group 5021
may indicate PID = 0x1022. On the other hand, the TS headers 5041 and 5051 of the
TS packets 5040 and 5050 belonging to the second group 5022 each indicate PID =
0x1022. The hexadecimal value "0x1022" may be replaced to any other value except
the hexadecimal values assigned to the other elementary streams. In this way, TS
packets belonging to the second group 5022 have a different PID from TS packets
belonging to the third group 5023. Accordingly, the system target decoder can easily
select TS packets belonging to the second group by using PIDs.
The system target decoder extracts offset metadata from the TS packet
sequence 5020 shown in Fig. 50 as follows. Fig. 51 is a functional block diagram
showing a system of processing video streams in the system target decoder 5125. The
system target decoder 5125 shown in Fig. 51, in contrast to that 4225 shown in Fig. 45,
does not include the TS priority filter 4551. Other components thereof are similar to
corresponding ones. In Fig. 51, components similar to ones shown in Fig. 45 are
marked with the same reference numbers. Furthermore, details of the similar
components can be found in the description on Fig. 45.
[0371]
The second PID filter 4514 transfers TS packets with PID = 0x1012 to TB2
4508 in the primary video decoder 4515, and transfers TS packets with PID = 0x1022 to
the offset metadata processing unit 4552. Here, TS packets with PID = 0x1022 may be
transferred to TB2 4508 in parallel. In this way, TS packets containing offset metadata
are transferred to the offset metadata processing unit 4552.
[0372]
Note that data different from either of TS priority and PED may be used to
select TS packets containing offset metadata from a sub-TS. If the data, like TS
priority and PID, allows a different value to be set for each TS packet, the data can be
used to select the above-described TS packets. This would be obvious for a person
skilled in the art from the above-described embodiment.
[0373]
(l-B-7) The offset metadata 1110 according to Embodiment 1 of the present
invention, as shown in Fig. 11, provides each frame with offset information.
Alternatively, when the video stream represents a frame in the interlace method (e.g.
601), the display unit is not a frame, but a field. In that case, the offset metadata may
provide each field with the offset information or provide the pair of fields constituting
each frame with the offset information.
(l-B-8) In the offset metadata according to Embodiment 1 of the present
invention, each offset sequence defines an offset value for each frame. Alternatively,
each offset sequence may define a function that represents a change over time in the
offset value for each presentation time, i.e. a completion function. In this case, the 3D
playback device uses the completion function at each presentation time to calculate the
offset value for each frame included in that presentation time.
[0375]
Fig. 52A is a schematic diagram showing a data structure of offset metadata
5200 that uses a completion function. As shown in Fig. 52A, the offset metadata 5200
includes a correspondence table between offset sequence IDs 5210 and offset sequences
5220. An offset sequence 5220 includes a starting offset value (offset_start) 5221, an
ending offset value (offset_end) 5222, offset function ID (offset_func_id) 5223, and
offset duration (offset_duration) 5224. When the offset metadata 5200 is stored in a
video sequence in the dependent-view video stream, the starting offset value 5221
indicates the offset value for the first frame represented by the video sequence. The
ending offset value 5222 indicates the offset value for the first frame represented by the
next video sequence. The offset function ID 5223 defines the type of completion
function. The type of completion function represents the shape of the changes in the
offset value during the presentation time of the video sequence. The offset duration
5224 indicates the length of the presentation time of the video sequence.
[0376]
Fig. 52B is a graph showing the types of elements in the completion function.
As shown in Fig. 52B, the x-axis represents the presentation time, and the y-axis
represents the offset value. In this context, the sign of the offset value is determined by
the depth of the graphics image, i.e. by whether the 3D graphics image is further back or
closer than the screen. Three types of elements in a completion function are provided: a
linear shape LNR, a convex shape CVX, and a concave shape CCV. The linear shape
LNR is defined by a linear function y = ax + b, whereas the convex shape CVX and
concave shape CCV are defined by a second degree curve y = ax + bx + c, a third
degree curve y = ax3 + bx2 + ex + d, or a gamma curve y = a(x + b)1/r + c. In this context,
the constants a, b, c, and d are parameters determined by the xy coordinates of each
edge A, B of each element, i.e. by a pair of presentation time and the offset value at that
point. On the other hand, the constant r is separately defined and is stored in each offset
sequence. The types of completion functions are defined by one of these elements LNR,
CVX, and CCV or by a combination thereof.
[0377]
Fig. 52C is a graph showing offset values calculated by a 3D playback device
from offset sequence IDs = 0, 1, 2 shown in Fig. 52A. As shown in Fig. 52C, the
horizontal axis of the graph represents the time elapsed since the first frame in each
video sequence was displayed; in the video sequence, an offset sequence is stored. The
black circles A0, BO, Al, Bl, A2, and B2 indicate coordinates defined by either the
starting offset value 5221 or ending offset value 5222 and the offset duration 5224. The
lines GRO, GR1, and GR2 that respectively connect the pairs of black circles A0 + BO,
Al + Bl, and A2 + B2 represent completion functions that are each determined by the
type of completion function specified in the offset function ID 5223 and by the
coordinate values of the black circles A0 + BO, Al + Bl, and A2 + B2 at the edges of
the lines. In the offset sequence with offset sequence ID = 0, the offset function ID 5223
i indicates "linear", and thus the black circles A0 and BO at either edge are connected by
a line #0 GRO with a linear shape LNR. In the offset sequence with offset sequence ID
= 1, the offset function ID 5223 indicates "curve #1", and thus the black circles Al and
Bl at either edge are connected by a line #1 GR1 with a convex shape CVX. In the
offset sequence with offset sequence ID = 2, the offset function ID 5223 indicates
"curve #2", and thus the black circles A2 and B2 at either edge are connected by a line
#2 GR2 that is formed by a combination of a convex shape CVX and a concave shape
CCV. The white circles represent pairs of a presentation time for a frame and an offset
value for the frame as calculated by the 3D playback device using the completion
function indicated by each of the lines GRO, GR1, and GR2. As is clear from these lines
GRO, GR1, and GR2, the mere combination of the starting offset value 5221, ending
offset value 5222, offset function ID 5223, and offset duration 5224 can represent a
variety of changes in offset value, i.e. in the depth of 3D graphics images. Accordingly,
the size of the overall offset metadata can be reduced without a loss in the ability to
express 3D graphics images.
[0378]
(1-C) In the AV stream file for 3D video images, data regarding the playback
format of 3D video images may be added to the PMT 1810 shown in Fig. 18. In this
case, the PMT 1810 includes 3D descriptors in addition to the PMT header 1801,
descriptors 1802, and pieces of stream information 1803. The 3D descriptors are
information on the playback format of 3D video images, are shared by the entire AV
stream file, and particularly include 3D format information. The 3D format information
indicates the playback format, such as L/R mode or depth mode, of the 3D video images
in the AV stream file. Each piece of stream information 1803 includes 3D stream
descriptors in addition to a stream type 1831, a PID 1832, and stream descriptors 1833.
The 3D stream descriptors indicate information on the playback format of 3D video
images for each elementary stream included in the AV stream file. In particular, the 3D
stream descriptors of the video stream include a 3D display type. The 3D display type
indicates whether the video images indicated by the video stream are a left view or a
right view when the video images are displayed in L/R mode. The 3D display type also
indicates whether the video images indicated by the video stream are 2D video images
or depth maps when the video images are displayed in depth mode. When a PMT thus
includes information regarding the playback format of 3D video images, the playback
system of these video images can acquire such information simply from the AV stream
file. This sort of data structure is therefore useful when distributing 3D video content
via a broadcast.
(1-D) Clip Information File
[0380]
The dependent-view clip information file may include, among stream attribute
information 2220 such as in Fig. 22, a predetermined flag in the video stream attribute
information allocated to PID=0xl012, 0x1013 of the dependent-view video stream.
When turned on, this flag indicates that the dependent-view video stream refers to the
base-view video stream. Furthermore, the video stream attribute information may
include information regarding the base-view video stream to which the dependent-view
video stream refers. This information can be used to confirm the correspondence
between video streams when verifying, via a predetermined tool, whether the 3D video
content has been created in accordance with a prescribed format.
[0381]
According to Embodiment 1 of the present invention, the size of base-view
extents and dependent-view extents can be calculated from the extent start points 2242
and 2420 included in the clip information file. Alternatively, a list of the size of each
extent may be stored in, for example, the clip information file as part of the meta data.
[0382]
(l-E)PlaylistFile
[0383]
(1-E-l) The 3D playlist file 222 shown in Fig. 31 includes one sub-path.
Alternatively, the 3D playlist file may include a plurality of sub-paths. For example, if
the sub-path type of one sub-path is "3D L/R", then the sub-path type of the other
sub-path may be "3D depth". By switching between these two types of sub-paths when
playing back 3D video images in accordance with the 3D playlist file, the playback
device 102 can easily switch between L/R mode and depth mode. In particular, such
switching can be performed more rapidly than switching the 3D playlist file itself.
[0384]
A plurality of dependent-view video streams may represent the same 3D video
images in combination with a shared base-view video stream. However, the parallax
between the left view and right view for the same scene differs between the
dependent-view video streams. These dependent-view video streams may be
multiplexed into one sub-TS, or separated into different sub-TSs. In this case, the 3D
playlist file includes a plurality of sub-paths. Each sub-path refers to a different
dependent-view video stream. By switching between sub-paths when playing back 3D
video images in accordance with the 3D playlist file, the playback device 102 can easily
change the sense of depth of the 3D video images. In particular, such processing can be
performed more rapidly than switching the 3D playlist file itself.
[0385]
Fig. 53 is a schematic diagram showing (i) a data structure of a 3D playlist file
5300 that includes a plurality of sub-paths and (ii) a data structure of a file 2D 5310 and
two files DEP 5321 and 5322 that are referred to by the 3D playlist file 5300. The file
2D 5310 includes a base-view video stream with a PID = 0x1011. The file DEP #1 5321
includes a dependent-view video stream #1 with a PID = 0x1012. The file DEP #2 5322
includes a dependent-view video stream #2 with a PID = 0x1013. In combination with
the base-view video stream in the file 2D 5310, the dependent-view video streams #1
and #2 separately represent the same 3D video images. However, the parallax between
the left view and right view for the same scene differs between the dependent-view
video streams #1 and #2. Furthermore, offset sequences with the same offset sequence
ID define different offset values for the same frame number.
[0386]
The 3D playlist file 5300 includes a main path 5330 and two sub-paths 5331
and 5332. The PI #1 of the main path 5330 refers to file 2D 5310, in particular to the
base-view video stream. The SUB_PI #1 of each of the sub-paths 5331 and 5332 shares
the same playback time as the PI #1 in the main path 5330. The SUB_PI #1 of the
sub-path #1 5331 refers to the file DEP #1 5321, in particular to the dependent-view
video stream #1. The SUB_PI #1 of the sub-path #2 5332 refers to the file DEP #2 5322,
in particular to the dependent-view video stream #2. This is also true in the PI #2 of the
main path 5330 and the SUB_PI #2 of each of the sub-paths 5331 and 5332.
[0387]
During 3D playlist playback processing of the 3D playlist file 5300, the
playback device 102 first has a user or an application program select the sub-path for
playback. Alternatively, the playback device 102 may select the sub-path for playback
according to the screen size of the display device 103, or may select the sub-path by
referring to the interpupillary distance of the viewer. By selecting the sub-path in this
way, the parallax between the left-view and right-view video planes can easily be
changed. Furthermore, since offset information changes caused by switching of the
dependent-view video stream, the offsets of the graphics planes played back from the
PG stream or IG stream included in the file 2D 5310 change. This makes it easy to
change the sense of depth of the 3D video images.
[0388]
(l-E-2) In the 3D playlist file shown in Fig. 31, the base-view video stream is
registered in the STN table 3205 in the main path 3101, and the dependent-view video
stream is registered in the STN table SS 3130 in the extension data 3103. Alternatively,
the dependent-view video stream may be registered in the STN table. In that case, the
STN table may include a flag indicating which of the base view and the dependent view
is represented by the registered video stream.
[0389]
(l-E-3) According to Embodiment 1 of the present invention, 2D playlist files
and 3D playlist files are stored separately in the BD-ROM disc 101. Alternatively, in a
similar manner to the extension data 3103, the sub-path 3102 shown in Fig. 31 may be
recorded in an area that is referenced only by the playback device 102 in the 3D
playback mode. In that case, the 3D playlist files as they are can be used as the 2D
playlist files since there is no risk that the sub-path 3102 causes the playback device 102
in the 2D playback mode to malfunction. As a result, the authoring of the BD-ROM disc
is simplified.
[0390]
(l-E-4) The reference offset IDs and offset adjustment values for the PG stream,
IG stream, and text subtitle stream may be stored in the STN table SS 3130 instead of in
the STN table 3205. Alternatively, this information maybe stored in the stream attribute
information 2220 in the clip information file. Furthermore, the reference offset ID may
be stored in the subtitle entry for each PG stream and text subtitle stream or may be
stored in each page of the IG stream.
[0391]
(l-E-5) When reference offset IDs are set in the 3D playlist file, the following
constraint conditions may be prescribed for seamless connection between Pis. For
example, when CC = 5 is set in the PI #2 of the main path 5330 shown in Fig. 53, video
images in the playback sections defined by PI #1 and PI #2 need to be connected
seamlessly. In this case, in PI #1 and PI #2, changes are prohibited to both the values of
the reference offset IDs and the number of offset sequences included in the
dependent-view video stream, i.e. the number of entries. Furthermore, changes to both
the offset adjustment values and the number of entries thereof may be prohibited. With
these constraint conditions, the playback device 102 can skip updating of the SPRM(27)
when changing the current PI from PI #1 to PI #2. Since the processing load for
seamless connection is thus reduced, the reliability of this processing can be further
improved. As a result, the quality of 3D video images can be improved.
[0392]
(l-E-6) In the STN table, two or more offset adjustment values may be set for
one piece of stream data. Fig. 54 is a schematic diagram showing such an STN table
5400. As shown in Fig. 54, the STN table 5400 associates an STN 5401 with a stream
entry 5410 of PG stream 1 and a piece of stream attribute information 5403. The
stream attribute information 5403 includes three types of offset adjustment values #l-#3
5411-5413 along with a reference offset ID 5410. These offset adjustment values are
used to change an offset depending on the screen size of a display device; the offset is to
be provided to a graphics plane generated from PG stream 1.
[0393]
Assume that the correspondence between the types of offset adjustment values
and screen sizes is specified in advance. Specifically, offset adjustment value #1 5411,
offset adjustment value #2 5412, and offset adjustment value #3 5413 are respectively
used when the screen size falls within the range of 0-33 inches, 34-66 inches, and 67
inches or greater. The offset adjustment values 5411-5413 are set to satisfy the
following condition: parallaxes between left-view and right-view graphics images
produced by providing an offset to a graphics plane have the maximum value equal to
or less than an interpupillary distance of a general viewer (in the case of a child, 5 cm or
less). As long as this condition is satisfied, the parallax will not exceed the viewer's
interpupillary distance. This can reduce the viewer's risk of suffering visually-induced
motion sickness and eye strain.
[0394]
Each time the change of current PI causes the change of the total number of
offset adjustment values allocated to a piece of stream data, the playback control unit
4235 of the playback device 102 selects offset adjustment values to be used actually
depending on the screen size of the display device 103. Specifically, the playback
control unit 4235 first acquires the screen size of the display device 103, if necessary,
by performing the HDMI authentication. The playback control unit 4235 next selects
one of offset adjustment value #l-#3, 4801-4803, depending on which range the screen
size of the display device 103 falls within; 0-33 inches, 34-66 inches, 67 inches or
greater. The playback control unit 4235 stores information representing the type of the
selected value as a player variable into the player variable storage unit 4236. Thus,
until the selection of offset adjustment values are performed again, the playback control
unit 4235 selects offset adjustment values of the type indicated by the player variable
from each STN table, and then updates the value of the SPRM(28) to the selected value.
(1-F) The index file 211 shown in Fig. 35 includes the 3D existence flag 3520
and 2D/3D preference flag 3530 shared by all the titles. Alternatively, the index file
may specify a different 3D existence flag or 2D/3D preference flag for each title.
[0396]
(1-G) SPRM(27), SPRM(28)
[0397]
(1-G-l) The program execution unit 4234 may set the values of the SPRM(27)
4751 and the SPRM(28) 4752 in accordance with a movie object or BD-J object. In
other words, the playback device 102 may cause an application program to set the
reference offset ID and offset adjustment value. Furthermore, such an application
program may be limited to an object associated with the item "first play" 3501 in the
index table 3510.
[0398]
(l-G-2) The playback control unit 4235 may have the viewer adjust the offset
to be provided to the graphics plane. Specifically, when the viewer operates the remote
control 105 or the front panel of the playback device 102 and requests to set the offset
adjustment value, first the user event processing unit 4233 receives the request and
notifies the playback control unit 4235 of the request. Next, in response to the request,
the playback control unit 4235 displays an operation screen for adjusting the offset on
the display device 103. Here, an OSD of the playback device 102 is used for displaying
this operation screen. The playback control unit 4235 further has the viewer select a
graphics plane for adjustment and increase/decrease of the offset value, through
operation of the remote control 105 or the like. The playback control unit 4235 then
updates SPRM(28) to add or subtract a predetermined value to/from the offset
adjustment value corresponding to the selected graphics plane. Preferably, during the
adjustment processing, the playback control unit 4235 causes the playback unit 4202 to
continue playback processing of the graphics plane. Here, the playback unit 4202 makes
the operation screen or the graphics image—whichever is displayed closer to the
viewer—semi-transparent, or displays the operation screen closer than the graphics
image. This makes the graphics image visible even when the operation screen is being
displayed, and thus the viewer can immediately confirm the effect of increasing or
decreasing the offset value in the same way as when adjusting the brightness or color of
the screen.
[0399]
(l-G-3) For offset control, each of the cropping units 4731-4734 shown in Fig.
47 uses the offset sequence specified by the reference offset IDs indicated by the
SPRM(27). Conversely, for offset control, each cropping unit 4731-4734 may be made
not to use the offset sequence specified by each offset sequence ID indicated by a
predetermined SPRM. In other words, the SPRM may indicate the offset sequence IDs
(PG_ref_offset_id_mask, IG_ref_offset_id_mask, SV_ref_offset_id_ mask,
IM_ref_offset_id_mask) that are to be masked during offset control. In this case, each
of the cropping units 4731-4734 may select the ID of the offset sequence that includes
the largest offset value from among the offset sequences that are received from the
system target decoder 4225 and are allocated to the offset sequence IDs not masked in
the offset information 4707. In this way, the depth of the graphics images represented
by the secondary video plane, PG plane, IG plane, and image plane can easily be
aligned. This allows for an increase in the degree of freedom when creating each piece
of stream data.
[0400]
(1-H) When displaying a menu unique to the playback device 102 as an OSD,
the playback device 102 may perform offset control on the graphics plane representing
the 2D video images in the menu, i.e. on the OSD plane. In this case, the playback
device 102 may select, within the offset information transmitted by the system target
decoder 4225 at the presentation time of the menu, the offset information that has an
offset direction that is closer to the viewer than the screen and that has the largest offset
value. The menu can thus be displayed closer than any 3D graphics image, such as
subtitles or the like, played back from the 3D video content.
[0401]
Alternatively, the playback device 102 may pre-store offset information for the
OSD plane. A specific offset sequence ID, such as offsetid = 0, is allocated to this
offset information. Furthermore, the following two conditions may be placed on the
offset information with an offset sequence ID = 0: (1) The offset direction is closer to
the viewer than the screen, and (2) The offset value is the same as the largest offset
value among those included in the pieces of offset information that (i) are allocated to
offset sequence IDs other than zero, (ii) correspond to the same frame number, and (iii)
have offset directions closer to the screen than the viewer. With this prescription, the
playback device 102 does not have to select offset information from among the offset
information transmitted by the system target decoder 4225, thus simplifying offset
control of the OSD plane. Also, each of the cropping units 4731-4734 may use offset
information for offset sequence ID = 0 as a substitute when unable to detect reference
offset IDs indicated by SPRM(27) among the offset information 4707 received from the
system target decoder 4225.
[0402]
(1-1) In the 3D playback device, in addition to the setting of parental level in
SPRM(13), 3D parental level may be set in SPRM(30). The 3D parental level indicates
a predetermined restricted age and is used for parental control of viewing of 3D video
titles recorded on the BD-ROM disc 101. Like the value in SPRM(13), a user of the 3D
playback device sets the value of the SPRM(30) via, for example, an OSD of the 3D
playback device. The following is an example of how the 3D playback device performs
parental control onto each 3D video title. The 3D playback device first reads, from the
BD-ROM disc 101, the age for which viewing of a title in the 2D playback mode is
permitted and compares this age with the value of the SPRM(13). If this age is equal to
or less than the value of the SPRMC13), the 3D playback device stops playback of the
title. If this age is greater than the value of the SPRM(13), the 3D playback device reads,
from the BD-ROM disc 101, the age for which viewing of a title in the 3D playback
mode is permitted and compares this age with the value of the SPRM(30). If this age is
equal to or greater than the value of the SPRM(30), the 3D playback device plays back
the title in the 3D playback mode. If this age is less than the value of the SPRM(30) and
equal to or greater than the value of the SPRM(13), the 3D playback device plays back
the title in the 2D playback mode. In this way, the difference in viewer's
interpupillary distance by the age taken into account, it is possible to realize a parental
control so that, for example, "children whose ages are less than a predetermined value
can view 3D video images only as 2D video images". Preferably the parental control is
performed when it is judged that "the display device supports playback of 3D video
images" in the processing of selecting a playlist file for playback shown in Fig. 36,
namely when it is judged YES in step S3605. Note that a value indicating
permission/prohibition of 3D playback mode may be set in SPRM(30) instead of
parental level, and the 3D playback device may judge whether the 3D playback mode is
valid or invalid in accordance with the value.
[0403]
(1-J) In the 3D playback device, a value indicating "which of 2D playback
mode and 3D playback mode is to be prioritized" may be set in SPRM(31). A user of
the 3D playback device sets the value of the SPRM(31) via, for example, an OSD of the
3D playback device. In step S3603 in the processing of selecting a playlist file for
playback shown in Fig. 36, the 3D playback device refers to the SPRM(31) as well as
the 2D/3D preference flag. When both the SPRM(31) and 2D/3D preference flag
indicate the 2D playback mode, the 3D playback device selects the 2D playback mode.
When both the SPRM(31) and 2D/3D preference flag indicate the 3D playback mode,
the 3D playback device proceeds to step S3 605 and performs the HDMI authentication,
without displaying the playback mode selection screen. As a result, when the display
device is supporting the 3D video images, the 3D playback device selects the 3D
playback mode. When the SPRM(31) and 2D/3D preference flag indicate different
playback modes, the 3D playback device executes step S3604, i.e. displays the playback
mode selection screen to have the user select a playback mode. Alternatively, the 3D
playback device may have the application program select a playback mode. In this way,
even if the 2D/3D preference flag is set in the 3D video content, it is possible to have
the user select a playback mode only when the playback mode indicated by the 2D/3D
preference flag does not match the playback mode indicated by the SPRM(31) which is
the playback mode having been set by the user in advance.
[0404]
An application program such as a BD-J object may select a playback mode by
referring to the SPRM(31). Furthermore, the application program may determine the
initial state of the menu to be displayed on the selection screen depending on the value
of the SPRM(31), when causing a user to select a playback mode at step S3 604 shown
in Fig. 36. For example, when the value of the SPRM(31) indicates that the 2D
playback mode has a high priority, the menu is displayed in the state in which a cursor
is positioned on a button for selecting the 2D playback mode; when the value of the
SPRM(31) indicates that the 3D playback mode has a high priority, the menu is
displayed in the state in which the cursor is positioned on a button for selecting the 3D
playback mode. Alternatively, when the 3D playback device has a function to manage
the accounts of a plurality of users such as a father, a mother, and a child, the 3D
playback device may set a value to the SPRM(31) depending on the account of a user
who is logged in at the current time.
[0405]
The value of the SPRM(31) may indicate "which of 2D playback mode and 3D
playback mode is to be always set", in addition to "which of 2D playback mode and 3D
playback mode is to be prioritized". When the value of the SPRM(31) indicates "2D
playback mode is to be always set", the 3D playback device always selects the 2D
playback mode irrespectively of the value of the 2D/3D preference flag. In that case, the
value of the SPRM(25) is set to indicate the 2D playback mode. When the value of the
SPRM(31) indicates "3D playback mode is to be always set", the 3D playback device
performs the HDMI authentication without displaying the playback mode selection
screen irrespectively of the value of the 2D/3D preference flag. In that case, the value of
the SPRM(25) is set to indicate the 3D playback mode (L/R mode or depth mode). In
this way, even if the 2D/3D preference flag is set in the 3D video content, it is possible
to allow the playback mode having been set by the user in advance to be always
prioritized.
[0406]
(1-K) The playback device 102 may have the user register an interpupillary
distance as a reserved SPRM, for example SPRM(32). In this case, the playback device
102 can adjust the offset adjustment value so that the maximum value of the parallax
between the left-view and right-view graphics images does not exceed the value
registered in the SPRM(32). Specifically, it suffices for the playback device 102 to
perform the following calculations for each offset value output by the system target
decoder. The playback device 102 first seeks the ratio of the value of the SPRM(32) to
the width (horizontal length) of the screen of the display device 103 and further seeks
the product of this ratio and the number of horizontal pixels of the display device 103.
This product represents two times the upper limit of the offset that can be provided to
the graphics plane via offset control. Next, the playback device 102 compares this
product with the double of each offset value. If the double of any offset value is equal to
or greater than this product, the playback device 102 identifies the ID of the offset
sequence that includes the offset value and reduces the offset adjustment value for the
graphics plane indicated by that ID. The amount of the reduction is set to at least half
the difference between the double of the offset value and the above product. The
maximum value of the parallax between a left-view and a right-view graphics image
thus does not exceed the viewer's interpupillary distance. This can reduce the viewer's
risk of suffering visually-induced motion sickness and eye strain.
(1-L) Output Offset Adjustment
[0408]
(1-L-l) Since the alternate-frame sequencing method represents a parallax
between left and the right views by the number of pixels in the horizontal direction, the
actual size of the parallax depends on the screen size of a display device, namely, the
size of a pixel. On the other hand, the sense of depth of 3D video images depends on
the actual size of the parallax. Accordingly, in order to avoid the sense of depth of the
3D video images on a screen of any size from impairing powerful impression of the 3D
video images and fatiguing viewer's eyes excessively, the parallax between the left and
right views needs to be adjusted to be appropriate to the screen size. As one method of
the adjustment, the 3D playback device further provides an offset to the final frame data
combined by the plane adders. The offset is provided in a manner similar to that in
which an offset is provided to a graphics plane in the 1 plane + offset mode. The
offset control that is further applied to the final frame data is referred to as "output
offset adjustment".
[0409]
Figs. 55A-55C are schematic diagrams showing parallaxes PRA, PRB, and
PRC between left and right views displayed on a 32-inch screen SCA, 50-inch screen
SCB, and 100-inch screen SCC, respectively. The images LAI, LA2, LB, LCI, and
LC2 drawn on the figures by solid lines represent left views, and the images RA1, RA2,
RB, RC1, and RC2 drawn by dashed lines represent right views. Here, assume that
video content specifies parallaxes between left and right views to produce an optimal
sense of depth when 3D video images are displayed on a 50-inch screen. As shown in
Fig. 55B, the parallax between the left view LB and the right view RB is equal to an
optimal value PRB when a 3D video image represented by the video content is
displayed on a 50-inch screen SCB.
rr\ a i m
The parallax DA between the left view LAI and the right view RA1 drawn by
thin lines in Fig. 55 A is equal in number of horizontal pixels to the parallax DB between
the left view LB and the right view RB drawn in Fig. 55B. On the other hand, a pixel
on the 32-inch screen SCA is smaller than a pixel on the 50-inch screen SCB..
Accordingly, 3D video images generated by the parallax DA between the left view LAI
and the right view RA1 drawn by the thin lines produce a sense of depth weaker than
the optimal one, in general. In that case, the output offset adjustment increases the
parallax between the left view and the right view by twice a predetermined adjustment
value CRA. The parallax PRA between the left view LA2 and the right view RA2
drawn by thick lines in Fig. 55A indicates a parallax after the output offset adjustment.
In this way, when a parallax between left and right views is increased, a sense of depth
of 3D video images is enhanced. This prevents the 3D video images from losing
powerful impression.
[0411]
The parallax DC between the left view LCI and the right view RC1 drawn by
thin lines in Fig. 55C is equal in number of horizontal pixels to the parallax PRB
between the left view LB and the right view RB drawn in Fig. 55B. On the other hand,
a pixel of the 100-inch screen SCC is larger than a pixel of the 50-inch screen SCB.
Accordingly, 3D video images generated by the parallax DC between the left view LCI
and the right view RC1 drawn by the thin lines produce a sense of depth stronger than
the optimal one, in general. In that case, the output offset adjustment decreases the
parallax between the left view and the right view by twice a predetermined adjustment
value CRC. The parallax PRC between the left view LC2 and the right view RC2
drawn by thick lines in Fig. 55C indicates the parallax after the output offset adjustment.
In this way, when a parallax between left and right views is decreased, a sense of depth
of 3D video images is suppressed. This prevents a viewer from suffering eye fatigue.
[0412]
The adjustment values CRA and CRC shown in Figs. 55A and 55C are referred
to as "output offset adjustment values". Video content includes a correspondence table
between screen sizes and output offset adjustment values stored in an index file, playlist
file, or clip information file. Fig. 56A is a schematic diagram showing the
correspondence table. As shown in Fig. 56A, an output offset adjustment value is set
for each range of screen size with a width of 10 inches. The magnitude of each output
offset adjustment value indicates the number of horizontal pixels, and the sign thereof
indicates increase/decrease of the parallax between left and right views. Note that the
range of screen size may-has a width other than 10 inches. The correspondence table
is set according to standards or by a user. Furthermore, two or more types of
correspondence tables may be recorded in a 3D playback device in advance, and video
content may specify the identifier of a type of the correspondence tables; the type is to
be used during playback of the video content.
[0413]
(l-L-2) A 3D playback device may use a predetermined function to select an
output offset adjustment value, instead of the above-described correspondence table.
Fig. 56B is a graph representing the function. The horizontal axis of this graph
indicates the screen size in inch, and the vertical axis thereof indicates the output offset
adjustment value represented by the number of pixels with a sign. The 3D playback
device uses the function represented by the graph to calculate an output offset
adjustment value from the screen size of a display device. As the graph shows, the
output offset adjustment value is a greater positive value when the screen size is smaller
than 50 inches, and a greater negative value when the screen size is greater than 50
inches. Note that the output offset adjustment value is maintained at a substantially
constant positive value when the screen size is 32 inches or less, and the output offset
adjustment value is maintained at a substantially constant negative value when the
screen size is 103 inches or more.
[0414]
(l-L-3) Video content may include an optimal value of screen size that is
assumed at the time of authoring (assumed_TV_size_when_authoring), and a 3D
playback device may determine an output offset adjustment value based on the optimal
value. For example, when the screen size of a display device exceeds the optimal
value, the 3D playback device first reduces the frame size of 3D video images to the
optimal value. The 3D playback device next superimposes a black border on the edges
of each frame, and causes the entirity of the frame and black border to be equal in size
the screen of the display device. The 3D playback device further adjusts the output
offset adjustment value so that the parallax between the left and right views displayed
inside the black border is equal in magnitude to a parallax between the left and right
views if they are displayed on the entirity of a screen with a size equal to the optimal
value. This enables a sense of depth of the 3D video images to be maintained equal to
that assumed at the time of authoring.
[0415]
(l-L-4) Fig. 57 is a block diagram showing the components of a 3D playback
device required for the output offset adjustment. The 3D playback device 5700 shown
in Fig. 57, in contrast to that 4200 shown in Fig. 42, includes an output offset
adjustment value application unit 5701. Other components thereof are similar to
corresponding ones. In Fig. 57, the components similar to ones shown in Fig. 42 are
marked with the same reference numbers. Furthermore, details of the similar
components can be found in the description on Fig. 42.
[0416]
The player variable storage unit 4236 stores output offset adjustment values in
the SPRM(36). The output offset adjustment values are based on a screen size
obtained by the HDMI communication unit 4237 from the display device 103 through
HDMI authentication, and determined by the playback control unit 4235 using the
correspondence table or the function shown in Fig. 56A or 56B. Alternatively, an
application program such as a BD-J object may automatically set the value of the
SPRM(35) or cause a user to set the value by using a GUI.
[0417]
The output offset adjustment value application unit 5701 uses an output offset
adjustment value indicated by the SPRM(35) to provide an offset to each of left-view
and right-view frame data combined by the plane adders 4226. The output offset
adjustment of the frame data by the output offset adjustment value application unit 5701
is similar to the offset control over the PG plane data GP by the second cropping unit
4732 shown in Fig. 49.
[0418]
(l-L-5) The screen size of the display device 103 may be stored in the
SPRM(35) instead of output offset adjustment values. In that case, the output offset
adjustment value application unit 5701 retrieves an output offset adjustment value
associated with the screen size indicated by the SPRM(35) from the correspondence
table shown in Fig. 56A.
[0419]
(l-L-6) The player variable storage unit 4236 may further store an output offset
adjustment value alpha into the SPRM(36). The output offset adjustment value alpha
represents a positive numeric value. The output offset adjustment value application
unit 5701 uses the product of an output offset adjustment value indicated by the
SPRM(35) and an output offset adjustment value alpha indicated by the SPRM(36) as
an actual output offset adjustment value. This enables depths of 3D video images
adjusted by the output offset adjustment to depend on not only screen sizes but also the
ages of viewers. For example, when the viewers include a child with a smaller
interpupillary distance than an adult, the output offset adjustment for small screen size
sets the output offset adjustment value alpha to a value smaller than "1," and the output
offset adjustment for large screen size sets it to a value greater than "1." This weakens
a sense of depth of 3D video images regardless of the screen size.
[0420]
The program execution unit 4234 or the playback control unit 4235 may use a
GUI or an OSD to cause a user to set an output offset adjustment value alpha. In that
case, acceptable levels of the output offset adjustment value alpha may be represented
by, for example, the following three levels: "the sense of depth of 3D video images is
strong", "normal", and "weak". Alternatively, an output offset adjustment value alpha
may be stored in supplementary data in a video stream, a descriptor in a PMT, or a PI in
a playlist file included in video content. With this structure, the output offset
adjustment value alpha can vary with scenes represented by the video stream. In
particular, the output offset adjustment value alpha can be set to be reduced in a scene
producing a strong sense of depth.
[0421]
(l-L-7) Output offset adjustment values may be changed depending on a
distance between a viewer and a screen, instead of the screen size. Furthermore,
output offset adjustment values alpha may depend on the distance. In that case, for
example, a distance sensor is mounted on the shutter glasses 104 shown in Fig. 1 and
used to measure a distance between the display device 103 and the screen 131. The
distance is transmitted from the shutter glasses 104 to the display device 103 at any time,
and further transmitted from the display device 103 to the playback device 102 via the
HDMI cable 122. The playback device 102 uses the distance to select an output offset
adjustment value or an output offset adjustment value alpha.
[0422]
(l-L-8) When the display device 103 is a projector, images are enlarged by
lenses and projected onto a screen. Accordingly, the size of a display area on the
screen changes depending on the distance between the projector and the screen. In that
case, the projector determines the size of the display area by, for example, either of the
following two methods. The first method first measures the distance between the
projector and the screen, and then calculates the size of the display area based on the
relationship between the distance and the characteristics of the optical system of the
projector, especially the spread angle of projection light. Here, a distance sensor
mounted on the projector is used to measure the distance. For example, the distance
sensor first emits infrared laser light and the like to the screen, and then detects
reflection light from the screen. At the same time, the distance sensor also measures
the length of the time elapsing from the emission of the laser light until the detection of
the reflection light. The distance sensor then calculates the distance between the
projector and the screen from the elapsed time. The second method causes the
projector to operate as follows: the projector first projects a reference graphic object
such as a line segment onto the screen, and next uses an OSD or the like to urge a
viewer to measure and enter the size of the reference graphic object on the screen. The
projector then calculates the size of the display area from the size of the reference
graphic object entered by the viewer.
[0423]
(1-M) In some video content, such as content for displaying song lyrics during
karaoke, graphics image of subtitles or the like are repeatedly displayed as still images,
and only the graphics images are frequently updated. When such content is formed into
3D video content, the VAU in which the offset metadata is placed further includes a
sequence end code. When the playback device 102 decodes this VAU, it stores the
offset information obtained from the offset metadata and does not change the offset
information until a VAU that includes new offset metadata is decoded.
[0424]
Fig. 58A is a schematic diagram showing a data structure of a dependent-view
video stream 5800 representing only still images. Each VAU in the dependent-view
video stream 5800 represents one still image. In this case, a sequence end code 5803,
5804 is placed at the end of each VAU. Meanwhile, offset metadata 5811, 5812 is
placed in the supplementary data 5801, 5802 of each VAU. The offset metadata 5811 in
VAU #1 includes an offset sequence [0] with an offset sequence ID = 0. This offset
sequence [0] includes only offset information on frame #1. Similarly, in the offset
metadata 5812 of VAU #2, the offset sequence [0] includes only offset information on
[0425]
It is assumed here that the 3D playlist file specifies the following two items: (1)
The still images represented by the VAUs in the dependent-view video stream 5800
switch at 10 second intervals, and (2) Graphics images represented by the graphics
stream are overlapped on each still image. Fig. 58B is a schematic diagram showing a
left-view video plane sequence 5821, a right-view video plane sequence 5822, and a
graphics plane sequence 5830 that are played back in accordance with such a 3D
playlist file. In Fig. 58B, the video planes at the time when the still image is switched
are shown with hatching. In the left-view video plane sequence 5821, the still image
indicated by the first video plane 5841 is repeatedly played back for the first 10 second
interval 5861, and the still image indicated by the next video plane 5851 is repeatedly
played back for the next 10 second interval 5871. In the right-view video plane
sequence 5822, the still image indicated by the first video plane 5842 is repeatedly
played back for the first 10 second interval 5862, and the still image indicated by the
next video plane 5852 is repeatedly played back for the next 10 second interval 5872.
[0426]
When the playback device 102 decodes VAU #1 in the dependent-view video
stream 5800, it reads offset information for frame #1 from the offset metadata 5811.
Furthermore, the playback device 102 detects the sequence end code 5803. At this point,
the playback device 102 stores the offset information for frame #1. In this way, during
the first 10 second interval 5861, the offset provided to the graphics plane sequence
5830 is maintained constant in accordance with the stored offset information. In other
words, the depth of the graphics images is maintained constant.
[0427]
Once 10 seconds have passed after decoding of VAU #1, the playback device
102 decodes VAU #2, and reads new offset information for frame #1 from the offset
metadata 5812. Furthermore, the playback device 102 detects the sequence end code
5804. At this point, the playback device 102 stores the offset information for frame #1.
In this way, during the next 10 second interval 5871, the offset provided to the graphics
plane sequence 5830 is changed and maintained constant in accordance with the newly
stored offset information. In other words, the graphics images are maintained constant
at a new depth.
[0428]
When a VAU includes a sequence end code, the playback device 102 is thus
caused to store existing offset information as is. Accordingly, even when a video stream
is composed only of still images, the playback device 102 can reliably maintain offset
control for the graphics plane.
[0429]
(1-N) Compensation of misalignment between left view and right view
[0430]
There are cases in which a "misalignment" occurs between a left view and a
right view. The playback device 102 or the display device 103 according to
Embodiment 1 of the present invention compensates the misalignment by using the
means described below. This prevents the risk that the misalignment may cause viewers
to feel uncomfortable.
[0431]
The playback device 102 uses the function units shown in Fig. 42 to
compensate the aforementioned misalignment. Alternatively, the display device 103
may perform the compensation processing. Fig. 59 is a block diagram of the display
device 103 that performs the compensation processing. As shown in Fig. 59, the display
device 103 includes a receiving unit 5901, a stream processing unit 5902, a signal
processing unit 5903, and an output unit 5904. The receiving unit 5901 receives
multiplexed stream data from various mediums such as a BD-ROM disc, semiconductor
memory device, external network, and broadcast wave, as well as from the playback
device 102, and passes the received multiplexed stream data to the stream processing
unit 5902. The stream processing unit 5902 separates various types of data such as
video, audio, and graphics from the multiplexed stream data, and passes the various
types of data to the signal processing unit 5903. The signal processing unit 5903
decodes each of the various types of data, and passes the results thereof to the output
unit 5904. The output unit 5904 converts each of the decoded data into a predetermined
format, and outputs the results thereof. The output of the output unit 5904 is video/audio
itself. Alternatively, the output may be a video/audio signal in the HDMI format. Except
for mechanical parts such as the disc drive, display panel, and speaker, the elements
5901, 5902, 5903, and 5904 shown in Fig. 59 are implemented on one or more
integrated circuits.
[0432]
(1-N-l) Horizontal Misalignment Between Left View and Right View
[0433]
Fig. 60A is a plan view schematically showing horizontal angles of view HAL
and HAR for a pair of video cameras CML and CMR filming 3D video images. As
shown in Fig. 60A, the pair of video cameras CML and CMR are placed side by side in
the horizontal direction. The left-video camera CML films the left view, and the
right-video camera CMR films the right view. The horizontal angles of view HAL and
HAR of the video cameras CML and CMR are the same size but differ in location. This
yields a strip AL that is only included in the horizontal angle of view HAL of the
left-video camera CML and a strip AR that is only included in the horizontal angle of
view HAR of the right-video camera CMR. The object OBC located in the section
common to both horizontal angles of view HAL and HAR is captured by both video
cameras CML and CMR. However, the object OBL located in strip AL, which is
included only in the horizontal angle of view HAL of the left-video camera CML, is
only captured by the left-video camera CML, and the object OBR located in strip AR,
which is included only in the horizontal angle of view HAR of the right-video camera
CMR, is only captured by the right-video camera CMR.
Fig. 60B is a schematic diagram showing a left view LV filmed by the
left-video camera CML, and Fig. 60C is a schematic diagram showing a right view RV
captured by the right-video camera CMR. As shown in Figs. 60B and 60C, the strip AL,
which is included only in the horizontal angle of view HAL of the left-video camera
CML, appears as a strip along the left edge of the left view LV. However, this strip AL
is not included in the right view RV. On the other hand, the strip AR, which is included
only in the horizontal angle of view HAR of the right-video camera CMR, appears as a
strip along the right edge of the right view RV. However, this strip AR is not included
in the left view LV. Accordingly, among the three objects OBL, OBC, and OBR shown
in Fig. 60A, the object on the right OBR is not included in the left view LV, and the
object on the left OBL is not included in the right view RV. As a result, the object on
the left OBL is only visible to the viewer's left eye, and the object on the right OBR is
only visible to the right eye. The left view LV and right view RV thus run the risk of
causing the viewer to feel uncomfortable.
[0435]
On the BD-ROM disc 101, information indicating the width WDH of the above
strips AL and AR included in each frame of the left view LV and right view RV is
stored in the dependent-view video stream. This information is stored in the
supplementary data of the VAU at the top of each video sequence. Note however that
this supplementary data is different from the supplementary data including the offset
metadata 1110 shown in Fig. 11. On the other hand, in the playback device 102, the
system target decoder 4225 reads information showing the width WDH of the above
strips AL and AR from the dependent-view video stream. Furthermore, the system
target decoder 4225 transmits this information to the parallax video generation unit
4710 in the plane adder 4226 or the output unit 5904 in the display device 103. When
the receiving unit 5901 in the display device 103 directly reads a 3D video content from
an information medium such as a BD-ROM disc, the above-mentioned information is
read from the dependent-view video stream and transmitted to the output unit 5904 by
the signal processing unit 5903 in the display device 103. The parallax video generation
unit 4710 or the output unit 5904 (hereinafter, referred to as "parallax video generation
unit 4710 etc.") refers to this information to process the left-video plane and the
right-video plane, uniformly painting the strips AL and AR a background color or black.
In other words, the pixel data included in the strips AL and AR is uniformly overwritten
with data that represents a background color or black.
[0436]
Figs. 60D and 60E are schematic diagrams respectively showing a left view LV
represented by the processed left-video plane and a right view RV represented by the
processed right-video plane. As shown in Fig. 60D, the strip AL, which is included only
in the horizontal angle of view HAL of the left-video camera CML, is hidden by a black
strip BL of width WDH. On the other hand, as shown in Fig. 60E, the strip AR, which is
included only in the horizontal angle of view HAR of the right-video camera CMR, is
hidden by a black strip BR of width WDH. As a result, both of the viewer's eyes see
only the area shared by the left view LV and the right view RV, which avoids the risk of
causing the viewer to feel uncomfortable.
[0437]
Furthermore, the parallax video generation unit 4710 etc. may perform
cropping similar to that shown in Fig. 47 to remove pixel data included in the outer half
of the strips AL and AR respectively located in the left-video plane and right-video
plane. In this case, the parallax video generation unit 4710 etc. uniformly paints the
remaining half of the strips AL and AR a background color or black and, in addition,
adds a background-color or black strip of half the width of the strips AL and AR to the
opposite side. In this way, both of the viewer's eyes see the area shared by the left view
LV and the right view RV in the center of the screen, with background color or black
strips at both edges of the screen. This avoids the risk of causing the viewer to feel
uncomfortable.
[0438]
Alternatively, the parallax video generation unit 4710 etc. may process the
left-video plane and right-video plane as follows. First, via cropping similar to that
shown in Fig. 49, the parallax video generation unit 4710 etc. removes the pixel data in
the strips AL and AR from each of the video planes. Next, the parallax video generation
unit 4710 etc. resizes each video plane from the pixel data in the remaining area via
scaling. The video image shown by the remaining area is thus expanded to fill the entire
frame. As a result, both of the viewer's eyes see only the area shared by the left view
LV and the right view RV, which avoids the risk of causing the viewer to feel
uncomfortable.
[0439]
(l-N-2) Vertical Misalignment Between Left View and Right View
[0440]
Fig. 61A is a plan view schematically showing vertical angles of view VAL
and VAR for a pair of video cameras CML and CMR filming 3D video images. As
shown in Fig. 61 A, the vertical angles of view VAL and VAR for the video cameras
CML and CMR are the same size but differ in location. This yields a strip AT that is
only included in the vertical angle of view VAL of the left-video camera CML and a
strip AB that is only included in the vertical angle of view VAR of the right-video
camera CMR. The object OBJ located in the section common to both vertical angles of
view VAL and VAR is captured by both video cameras CML and CMR. However,
objects located in strip AT, which is included only in the vertical angle of view VAL of
the left-video camera CML, are only captured by the left-video camera CML, and
objects located in strip AB, which is included only in the vertical angle of view VAR of
the right-video camera CMR, are only captured by the right-video camera CMR.
[0441]
Fig. 61B is a schematic diagram showing a left view LV filmed by the
left-video camera CML and a right view RV filmed by the right-video camera CMR. As
shown in Fig. 6IB, the strip AT, which is included only in the vertical angle of view
VAL of the left-video camera CML, appears as a strip along the top of the left view LV.
However, this strip AT is not included in the right view RV. On the other hand, the strip
AB, which is included only in the vertical angle of view VAR of the right-video camera
CMR, appears as a strip along the bottom edge of the right view RV. However, this strip
AB is not included in the left view LV. Note that the positions of the strips AT and AB
may be reversed between the left view LV and right view RV. In this way, when the left
view LV and right view RV differ with regards to inclusion of the strips AT and AB,
the vertical position of the object OBJ shown in Fig. 61A differs between the left view
LV and the right view RV by the height HGT of the strips AT and AB. As a result, the
vertical position of the object OBJ differs as seen by the viewer's left eye and right eye,
which has the risk of causing the viewer to feel uncomfortable.
[0442]
On the BD-ROM disc 101, information indicating the height HGT of the above
strips AT and AB included in each frame of the left view LV and right view RV is
stored in the dependent-view video stream. This information is stored in the
supplementary data of the VAU at the top of each video sequence. Note however that
this supplementary data is different from the supplementary data including the offset
metadata 1110 shown in Fig.-11. On the other hand, in the playback device 102, the
system target decoder 4225 reads the information indicating the height HGT of the
above strips AT and AB from the dependent-view video stream. Furthermore, the
system target decoder 4225 transmits this information to the parallax video generation
unit 4710 in the plane adder 4226 or the output unit 5904 in the display device 103.
When the receiving unit 5901 in the display device 103 directly reads a 3D video
content from an information medium such as a BD-ROM disc, the above-mentioned
information is read from the dependent-view video stream and transmitted to the output
unit 5904 by the signal processing unit 5903 in the display device 103.
The parallax video generation unit 4710 or the output unit 5904 (hereinafter,
referred to as "parallax video generation unit 4710 etc.") refers to the height HGT of the
strips AT and AB to process the left-video plane and the right-video plane as follows.
First, the parallax video generation unit 4710 etc. shifts the position of the pixel data in
the left-video plane up by half the height HGT, i.e. HGT/2, and shifts the position of the
pixel data in the right-video plane down by HGT/2. The vertical center of the video
image shown in the area of the video planes other than the strips AT and AB thus
matches the vertical center of the screen. In the left-video plane, half of the strip AT is
removed from the top, yielding an empty strip with a height of HDT/2 at the bottom. In
the right-video plane, half of the strip AB is removed from the bottom, yielding an
empty strip with a height of HDT/2 at the top. Next, the parallax video generation unit
4710 etc. uniformly paints the strips a background color or black. In other words, the
pixel data included in the strips is uniformly overwritten with data that represents a
background color or black.
[0444]
Fig. 61C is a schematic diagram showing a left view LV represented by the
processed left-video plane and a right view RV represented by the processed right-video
plane. As shown in Fig. 61C, the vertical centers of the left view LV and the right view
RV match. Accordingly, the vertical position of the object OBJ shown in Fig. 61A is the
same in the left view LV and the right view RV. At the top of the left view LV, the strip
AT, which is included only in the vertical angle of view VAL of the left-video camera
CML, is hidden by a black strip BT of height HGT/2, and at the bottom of the right
view RV, the strip AB, which is included only in the vertical angle of view VAR of the
right-video camera CMR, is hidden by a black strip BB of height HGT/2. Furthermore,
a black strip BB of height HGT/2 is added to the bottom of the left view LV, and a
black strip BT of height HGT/2 is added to the top of the right view RV. As a result,
both of the viewer's eyes see only the area shared by the left view LV and the right view
RV, and the vertical positions match between the object seen by each eye. This avoids
the risk of causing the viewer to feel uncomfortable.
[0445]
Alternatively, the parallax video generation unit 4710 etc. may process the
left-video plane and right-video plane as follows. First, via cropping similar to that
shown in Fig. 49, the plane adder 4126 removes the pixel data in the strips AT and AB
from each of the video planes. Next, the parallax video generation unit 4710 etc. resizes
each video plane from the pixel data in the remaining area via scaling. The video image
shown by the remaining area is thus expanded to fill the entire frame, and as a result,
both of the viewer's eyes see only the area shared by the left view LV and the right view
RV. Furthermore, the vertical positions match between the object seen by each eye. This
avoids the risk of causing the viewer to feel uncomfortable.
[0446]
(l-N-3) Misalignment of Graphics Images Between Left View and Right View
[0447]
When a playback device in 1 plane + offset mode provides a large offset to a
graphics plane to generate a pair of graphics planes, a region in the right or left edge of
one graphics plane may not be included in the right or left edge of the other graphics
plane.
[0448]
Fig. 62A is a schematic diagram showing an example of graphics images
represented by a graphics plane GPL. As shown in Fig. 62A, the graphics plane GPL
represents three types of graphic elements OBI, OB2, and OB3. In particular, the left
edge of the left graphic element OBI is located at a distance Dl from the left edge of
the graphics plane GPL, and the right edge of the right graphic element OB3 is located
at a distance D3 from the right edge of the graphics plane GPL. Figs. 62B and 62C are
schematic diagrams respectively showing processes of providing a right and left offset
to the graphics plane GPL. As shown in Fig. 62B, a strip AR1 of width OFS equal to the
offset value is removed from the right edge of the graphics plane GPL, and a transparent
strip AL1 of width OFS is added to the left edge, in a way similar to the cropping
process shown in Fig. 49. The horizontal positions of the graphic elements OB1-OB3
are thus shifted to the right from their original positions by a distance OFS equal to the
offset value. On the other hand, as shown in Fig. 62B, a strip AL2 of width OFS equal
to the offset value is removed from the left edge of the graphics plane GPL, and a
transparent strip AR2 of width OFS is added to the right edge. The horizontal positions
of the graphic elements OB1-OB3 are thus shifted to the left from their original
positions by the distance OFS.
[0449]
As shown in Figs. 62B and 62C, the distance OFS, which is equal to the offset
value, is larger than the distance Dl between the left edge of the left graphic element
OBI and the left edge of the graphics plane GPL. The distance OFS is also larger than
the distance D3 between the right edge of the right graphic element OB3 and the right
edge of the graphics plane GPL. Accordingly, a portion MP3 of the right edge of the
right graphic element OB3 is missing in the graphics plane GP1 to which a right offset
has been provided. Also, a portion MP1 of the left edge of the left graphic element OBI
is missing in the graphics plane GP2 to which a left offset has been provided. However,
the missing portion MP1 of the left graphic element OBI is included in the graphics
plane GP1 with the right offset, and the missing portion MP3 of the right graphic
element OB3 is included in the graphics plane GP2 with the left offset. As a result,
these missing portions MP1 and MP3 are only seen by one of the viewer's eyes, which
may make the viewer feel uncomfortable.
[0450]
In the playback device 102, each of the cropping units 4731-4734 in plane
adder 4226 refers to the offset information 4707 to perform offset control on the
graphics plane GPL. At this point, each of the cropping units 4731-4734 furthermore
removes a strip that has a width equal to the offset value and extends along the left or
right edge of the graphics plane GPL. In other words, the pixel data in the strip is
overwritten with data representing a transparent color. Alternatively, the output unit
5904 in the display device 103 may receive offset information from the system target
decoder 4225 or the signal processing unit 5903 in the display device 103 and refer to
the offset information to remove a strip from the left or right edge of the graphics plane
GPL Figs. 62B and 62C show the strips AS1 and AS2 to be removed. In the graphics
plane GP1 with the right offset, the strip AS1 to be removed includes the missing
portion MP1 of the left graphic element OBI. In the graphics plane GP2 with the left
offset, the strip AS2 to be removed includes the missing portion MP 3 of the right
graphic element OB3.
[0451]
Figs. 62D and 62E are schematic diagrams showing graphics images
represented by the graphics planes GP1 and GP2 with the right and left offsets,
respectively. As shown in Figs. 62D and 62E, in the graphics planes GP1 and GP2, the
shapes of the three types of graphic elements OB1-OB3 match. As a result, only the
shared part of the graphics images are visible to each of the viewer's eyes. This avoids
the risk of causing the viewer to feel uncomfortable.
[0452]
Alternatively, the following condition may be prescribed regarding the
arrangement of graphic elements for graphics planes played back from a PG stream or
an IG stream on a BD-ROM disc and for a graphics plane generated by a playback
device 102. Fig. 63 is a schematic diagram showing such a condition. As shown in Fig.
63, xy orthogonal coordinates are established on the graphics plane GPL, with an origin
(0, 0) at the upper-left corner. The x and y coordinates are respectively the horizontal
and vertical coordinates of the graphics plane GPL. The coordinates of the lower-right
corner of the graphics plane GPL are set to (TWD, THG). Using these xy coordinates,
the condition is set as follows: in each frame, the graphic elements OBI, OB2, and OB3
must be positioned within the rectangular area having four points (OFS, 0), (TWD-OFS,
0), (TWD-OFS, THG), and (OFS, THG) as vertices. In other words, graphic elements
are prohibited from being placed within the strips AL and AR of width OFS which
respectively extend along the left edge and right edge of the graphics plane GPL. As is
clear from Figs. 62B and 62C, these strips AL and AR are removed by offset control.
Accordingly, if graphic elements are prohibited from being placed within the strips AL
and AR, the shapes of the graphic elements do not change even when an offset is
provided to the graphics plane GPL. As a result, both of the viewer's eyes see the same
graphics images, which avoids the risk of causing the viewer to feel uncomfortable.
[0453]
(1-0) Letterbox Display
[0454]
The screen size assumed at the time of authoring of a video content depends on
the format of the content: Full-HD format adopted in digital TV broadcasting; or
cinemascope™ format adopted in movies. The aspect ratio of Full-HD is 16:9 (~1.78:1),
while the aspect ratio of cinemascope is 2.35:1. Accordingly, in home movie contents
recorded on BD-ROM discs, horizontal black strips are provided above and below each
video frame. The black strips are provided so that the whole aspect ratio of the video
frame and the black strips is adjusted to 16:9. This display method is referred to as
"letterbox display".
[0455]
Figs. 64A1 and 64A2 are schematic diagrams showing the same screen in the
letterbox display. As shown in Figs. 64A1 and 64A2, the resolution of the whole screen
is 1920x1080 pixels, and the aspect ratio is 16:9. On the other hand, the resolution of a
display area DRG for displaying the video images is 1920x818 pixels, and the aspect
ratio is 2.35:9. Black strips BT and BB, each 131-pixel high, extend horizontally above
and below the display area DRG.
[0456]
When such a letterbox display is adopted in the display of 3D video images, it
is preferable that the subtitle is displayed on either of the black strips BT and BB, not on
the display area DRG. This enables the 3D video images and the subtitle to be separated
from each other and both to be presented to the viewer in a reliable manner. However,
131 pixels of height of the black strips BT and BB is not necessarily sufficient to
display the subtitle. In that case, the plane adders 4226 of the playback device 102
provide the primary video plane with an offset in the vertical direction. This offset
control is referred to as "video shift". There are three types of video shifts: "Keep",
"Up", and "Down". In the Keep mode, the primary video plane is not provided with the
offset in the vertical direction. Thus as in the video frame shown in Figs. 64A1 and
64A2, the height of each of black strips BT and BB is kept to be 131 pixels. In the Up
mode, the primary video plane is provided with an upward offset. Fig. 64B is a
schematic diagram showing the screen in which the primary video plane has been
provided with an upward offset of 131 pixels. As shown in Fig. 64B, the black strip BT
has been removed from the upper portion and the height of the black strip BB in the
lower portion has been doubled. In the Down mode, the primary video plane is provided
with a downward offset. Fig. 64C is a schematic diagram showing the screen in which
the primary video plane has been provided with a downward offset of 131 pixels. As
shown in Fig. 64C, the black strip BB has been removed from the lower portion and the
height of the black strip BT in the upper portion has been doubled. In this way, the
plane adders 4226 increases either of the black strips BT and BB to a height sufficient
to display the subtitle by executing the video shift in the Up or Down mode.
[0457]
The size of the vertical offset may be other than 131 pixels. Fig. 64D is a
schematic diagram showing the screen in which the primary video plane has been
provided with an upward offset of 51 pixels. As shown in Fig. 64D, the height of the
black strip BT in the upper portion has been decreased to 131-51 =80 pixels, and the
height of the black strip BB in the lower portion has been increased to 131+ 51 = 182
pixels. In the following, it is assumed that the size of the offset is 131 pixels.
[0458]
Fig. 65 is a functional block diagram showing the structure of the playback
device 102 required for the video shift. The structure shown in Fig. 65, in contast to
that shown in Fig. 47, includes a video shift unit 6501, SPRM(32) 6502, and SPRM(33)
6503. The other components are similar. In Fig. 65, the components similar to ones
shown in Fig. 47, are marked with the same reference numbers. Furthermore, details
of the similar components can be found in the description on Fig. 47.
[0459]
The values indicated by the SPRM(32) and SPRM(33) are set by the program
execution unit 4234 in accordance with an application program such as a BD-J object or
a user's instruction via a GUI. The value indicated by the SPRM(33) is further updated
in accordance with the playlist file.
[0460]
Fig. 66A is a table showing the data structures of the SPRM(32) and
SPRM(33). As shown in Fig. 66A, the SPRM(32) stores a parameter that indicates the
video shift mode (video_shift_mode). The parameter can take any of three values "0",
"1", and "2" which correspond to three types of video shift modes. The SPRM(33)
stores four pairs of a video upward move shift value and a video downward move shift
value. The shift values are composed of: a pair for the PG plane
(PG_shift_value_for_Up, PG_shift_value_for_Down); a pair for the IG plane
(IG_shift_value_for_Up, IG_shift_value_for_Down); a pair for the secondary video
plane (SV_shift_value_for_Up, SV_shift_value_for_Down); and a pair for the image
plane (IM_shift_value_for_Up, IM_shift_value_for_Down). Each of the video upward
move shift value and the video downward move shift value represents the size of an
offset in the vertical direction that is provided to the PG plane or the like when an
upward or downward offset is provided to the primary video plane.
[0461]
Fig. 66B is a schematic diagram showing the STN table in the playlist file for
the video content of the letterbox display. As shown in Fig. 66B, in the STN table 6600,
an STN 6601 is associated with a stream entry 6602 of the PG stream 1 and stream
attribute information 6603. The stream attribute information 6603 includes a video
upward move shift value (PG_y_shift_value_for_Up) 6610 and a video downward
move shift value (PG__y_shift_value_for_Down) 6611. With this structure, these shift
values can be set for each PI. With regard to the other stream data such as the IG stream,
each shift value can be set. The playback control unit 4235 reads a shift value from the
STN table in each PI, and updates a value indicated by the SPRM(33) with the read shift
value.
[0462]
The video shift unit 6501 receives the left-view plane data 4701 and the
right-view plane data 4702 alternately from the switch 4720. Upon each reception
thereof, the video shift unit 6501 refers to the SPRM(32) in the player variable storage
unit 4236 and provides the primary video plane with a vertical offset in a video shift
mode indicated by the value in the SPRM(32). The video shift unit 6501 then transmits
the primary video plane to the second adder 4742.
[0463]
Figs. 67A-67C are schematic diagrams showing primary video planes VPA,
VPB, and VPC processed by the video shift unit 6501 in the Up mode, Keep mode, and
Down mode, respectively. When the SPRM(32) indicates the Keep mode, the video
shift unit 6501 does not provide the primary video plane with a vertical offset. Thus as
shown in Fig. 67B, the height of each of black strips BT and BB in the primary video
plane VPB is kept to be 131 pixels. When the SPRM(32) indicates the Up mode, the
video shift unit 6501, in a similar manner to the cropping process shown in Fig. 49, first
cuts the black strip BT that is 131 pixels high out of the upper portion of the original
primary video plane VPB. The video shift unit 6501 then, as shown in Fig. 67A, adds a
black strip AB that is 131 pixels high to the lower portion of the primary video plane
VPA. This moves the location of the pixel data other than the pixel data included in the
cut-out black strip BT upward by 131 pixels. On the other hand, the height of black
strips BB + AB is increased to 131 x 2 - 262 pixels. When the SPRM(32) indicates the
Down mode, the video shift unit 6501 cuts the black strip BB that is 131 pixels high out
of the lower portion of the original primary video plane VPB, and as shown in Fig. 67C,
adds a black strip AT that is 131 pixels high to the upper portion of the primary video
plane VPC. This moves the location of the pixel data other than the pixel data included
in the cut-out black strip downward by 131 pixels. On the other hand, the height of
black strips BT + AT is increased to 131 x 2 = 262 pixels.
[0464]
Referring again to Fig. 65, each time it receives the PG plane data 4704 from
the system target decoder 4225, the second cropping unit 4732 refers to the SPRM(32)
6502 and the SPRM(33) 6503 and provides the PG plane 4704 with a vertical offset in
accordance with values stored in the SPRMs. Furthermore, in the 1 plane + offset mode,
the second cropping unit 4732 provides the PG plane 4704 with a horizontal offset. The
second cropping unit 4732 then transmits the PG plane 4704 to the second adder 4742.
[0465]
Figs. 67D-67F are schematic diagrams showing PG planes PGD, PGE, and
PGF processed by the second cropping unit 4732 in the Up mode, Keep mode, and
Down mode, respectively. When the SPRM(32) indicates the Keep mode, the second
cropping unit 4732 does not provide the PG plane PGE with a vertical offset. Thus as
shown in Fig. 67E, the subtitle SUB in the PG plane PGE is kept to be at the original
position. When the SPRM(32) indicates the Up mode, the second cropping unit 4732
first reads a video upward move shift value (PG_shift_value_for_Up) "a" for the PG
plane from the SPRM(33) 6503. The second cropping unit 4732 then, in a similar
manner to the cropping process shown in Fig. 49, provides the PG plane PGE with a
downward offset which has the same size as the video upward move shift value "a".
More specifically, the second cropping unit 4732 first cuts the strip SBE that is "a"
pixels high out of the lower portion of the original PG plane PGE. The second cropping
unit 4732 then, as shown in Fig. 67D, adds a strip STD that is "a" pixels high to the
upper portion of the PG plane PGD. This moves the location of the subtitle SUB
downward by "a" pixels. When the SPRM(32) indicates the Down mode, the second
cropping unit 4732 first reads a video downward move shift value
(PG_shift_value_for_Down) "b" for the PG plane from the SPRM(33) 6503. The
second cropping unit 4732 then, in a similar manner to the cropping process shown in
Fig. 49, provides the PG plane PGE with an upward offset which has the same size as
the video downward move shift value "b". More specifically, the second cropping unit
4732 first cuts the strip STE that is "b" pixels high out of the upper portion of the
original PG plane PGE. The second cropping unit 4732 then, as shown in Fig. 67F, adds
a strip SBF that is "b" pixels high to the lower portion of the PG plane PGF. This moves
the location of the subtitle SUB upward by "b" pixels.
[0466]
The second adder 4742 receives the PG plane data from the second cropping
unit 4732, superimposes the PG plane data on the primary video plane data from the
video shift unit 6501 and transmits the result to the third adder 4743. Figs. 67G-67I are
schematic diagrams showing plane data PLG, PLH, and PLI combined by the second
adder 4742 in the Up mode, Keep mode, and Down mode, respectively. In the Keep
mode, as shown in Fig. 67H, the subtitle SUB is displayed on top of the primary video
images MVW. In the Up mode, as shown in Fig. 67G, the subtitle SUB is displayed in
the black strip BBG that is located below the primary video images MVW. This can be
realized by adjusting the video upward move shift value "a". In the Down mode, as
shown in Fig. 671, the subtitle SUB is displayed in the black strip BTI that is located
above the primary video images MVW. This can be realized by adjusting the video
downward move shift value "b".
[0467]
In the letterbox display, a dialog screen represented by the IG plane, video
images represented by the secondary video plane, or a pop-up menu represented by the
image plane may be displayed in the black strip, as well as the subtitle represented by
the PG plane. In those cases, the height of the black strip can be adjusted appropriately
by a method similar to the above-described method.
[0468]
(1-0-1) In the structure shown in Fig. 65, the second cropping unit 4732 reads
the video upward/downward move shift value from the SPRM(33) 6503. Alternatively,
the second cropping unit 4732 may read the video upward/downward move shift value
directly from the playlist file.
[0469]
(1-0-2) The height of the black strips BT and BB may be other than 131 pixels,
and may further be variable. The value thereof may be set in an SPRM in the player
variable storage unit 4236 in accordance with an application program or the user.
[0470]
(1-0-3) In Figs. 67D and 67F, the second cropping unit 4732 moves the
location of almost all pixel data included in the PG plane PGE upward and downward.
Alternatively, the PG decoder may change the object display position indicated by the
PCS by referring to the SPRM(33) 6503. For example, when the PCS indicates the
object display position = (x, y) and the SPRM(33) 6503 indicates the video upward
move shift value "a", the PG decoder changes the object display position to coordinates
(x, y+a). With this operation, like the subtitle SUB shown in Fig. 67D, the graphics
object represented by the PG stream is displayed below the object display position
indicated by the PCS. This also applies to the case where the display position of the
graphics object is moved upward. Note that the PCS may store the video
upward/downward move shift value.
[0471]
(1-0-4) In the Up mode and Down mode, as shown in Figs. 67D and 67F, the
upper and lower portions of the PG plane are cut out. At this point, in order to prevent
the upper and lower portions of the graphics object from being cut out, the area in which
the graphics object can be arranged may be limited to a predetermined range. As a
specific example, it is assumed that the height*width of the PG plane is HGTxWDH,
the video upward move shift value is "a", and the video downward move shift value is
"b". In that case, as shown in Fig. 67E, the arrangement of the graphics object is limited
to within the following horizontal strip: the x-y coordinates of the upper-left corner PUL
= (0, b); and the x-y coordinates of the lower-left corner PDR = (WDH, HGT - a). More
accurately, the PG stream satisfies the following conditions: (A) the object display
position indicated by the PCS is within the above-described strip; (B) even if the
graphics object is displayed at the object display position, the display area does not
exceed the range of the above-described strip; (C) the window position indicated by the
WDS is within the above-described strip; and (D) even if the window is set at the
window position, the range thereof does not exceed the range of the above-described
strip. In this way, it is possible to prevent the upper and lower portions of the graphics
object from being cut out.
[0472]
(1-0-5) Fig. 68 A is a schematic diagram showing another example of the STN
table in the playlist file for the video content of the letterbox display. As shown in Fig.
68A, in the STN table 6800, an STN 6801 is associated with a stream entry 6802 of the
PG stream 1 and stream attribute information 6803. The stream attribute information
6803 includes a video shift mode 6812 as well as a video upward/downward move shift
value 6810, 6811. In that case, the playback device 102 may use the following structure
for the video shift.
[0473]
Fig. 69 is a functional block diagram showing another example of the structure
of the playback device 102 required for the video shift. The structure shown in Fig. 69
differs from the structure shown in Fig. 65 in the video shift unit 6901 and SPRM(34)
6904. The other components are similar. In Fig. 69, the components similar to ones
shown in Fig. 65 are marked with the same reference numbers. Furthermore, details of
the similar components can be found in the description on Fig. 65.
[0474]
As shown in Fig. 66A, the SPRM(32) represents the video shift mode, and the
SPRM(33) represents the video upward move shift value and the video downward move
shift value. The parameters representing those are updated in accordance with the STN
table in the playlist file as shown in Fig. 68A. In the player variable storage unit 4236,
the SPRM(34) further stores a flag whose ON/OFF indicates whether the video shift is
to be performed. The value of the flag is set by the program execution unit 4234 in
accordance with an application program or the user. Each time it receives either the
left-video plane data 4701 or the right-video plane data 4702 from the switch 4720, the
video shift unit 6901 first refers to the flag in the SPRM(34) to determine whether to
perform the video shift. For example, when the value of the flag is "1", the video shift
unit 6901 refers to the SPRM(32) and provides the primary video plane with a vertical
offset in a video shift mode indicated by the value in the SPRM(32). On the other hand,
when the value of the flag is "0", the video shift unit 6901 transmits the primary video
plane to the second adder 4742 without performing the video shift. Similarly, each time
it receives the PG plane data 4704 from the system target decoder 4225, the second
cropping unit 4732 first refers to the SPRM(34) to determine whether to provide the PG
plane 4704 with a vertical offset. For example, when the value of the flag is "1", the
second cropping unit 4732 refers to the SPRM(32) and SPRM(33) and provides the PG
plane 4704 with a vertical offset in accordance with the values therein. On the other
hand, when the value of the flag is "0", the second cropping unit 4732 does not provide
the PG plane 4704 with a vertical offset.
[0475]
(1-0-6) When a plurality of pieces of stream attribute information 6803, each
including the video shift mode 6812 shown in Fig. 68A, are registered in the STN table,
the order of registration is set so that pieces of stream attribute information having the
same video shift mode become continuous. Fig. 68B is a schematic diagram showing
the order of registration. As shown in Fig. 68B, PIDs of nine PG streams 1-9 are
registered in the STN table, in correspondence with stream numbers (STNs) 5-13. The
video shift mode of PG streams 1-3 is set to the Keep mode, the video shift mode of PG
streams 4 and 5 is set to the Up mode, and the video shift mode of PG streams 6-9 is set
to the Down mode. In that case, continuous three STNs = 1-3 are assigned to the PG
streams 1-3, continuous two STNs = 4, 5 are assigned to the PG streams 4, 5, and
continuous four STNs = 6-9 are assigned to the PG streams 6-9. Each time it receives
notification of depression of a subtitle switch button from the remote control 105, the
playback device 102 selects a PG stream to be used in the display of subtitle from
among PG stream 1-9 in accordance with the registration order indicated in Fig. 68B.
Here, since the screen display of both the video images and the subtitle is continued
during the selection operation, generally the display positions of the video images and
the subtitle change when the subtitle switch button is pressed. However, as shown in Fig.
68B, PG streams of the same video shift mode are continuously registered in the STN
table. Thus generally the display positions of the video images and the subtitle change
only after the subtitle switch button is pressed a plurality of times. In this way, the
frequency of change is suppressed, thereby preventing the selection operation of the PG
stream from disrupting the display of the video images and the subtitle.
[0476]
(1-0-7) When switching between the video shift modes, the playback device
102 may change the display position of the video images and subtitle smoothly by using
the visual effects such as the fade-in/out. More preferably, the change of display
position of the subtitle is delayed than the change of display position of the video
images. This prevents the risk that the change of display position of the video images
and subtitle due to a switch between video shift modes may cause viewers to feel
uncomfortable.
[0477]
(1-0-8) In the PDS in the PG stream, "colorless transparent" is assigned to
color ID = 255, and in the WDS, color ID = 255 is assigned to the background color in
the window. Accordingly, when the PG stream represents the subtitle, the background
color of the subtitle is set to the colorless transparent. Fig. 70B is a schematic diagram
showing the video image IMG and subtitle SUB displayed on the screen SCR in that
case. As shown in Fig. 70B, in the window WIN1 indicating the display range of the
subtitle SUB, the background color is colorless transparent. Accordingly, in the window
WIN1, the video image IMG and subtitle SUB are displayed on top of each other.
[0478]
On the other hand, the playback device 102 may assign an opaque color such as
black to color ID = 255. Fig. 70A is a schematic diagram showing the data structure of
the SPRM(37) in the player variable storage unit 4236. As shown in Fig. 70A, a color
coordinate value of the background color of the subtitle is stored in the SPRM(37). The
value is preset by the program execution unit 4234 in accordance with an application
program or the user. When a color coordinate value is set in the SPRM(37), the PG
decoder in the system target decoder 4225 assigns the color coordinate value to color ID
= 255 irrespectively of the setting indicated by the PDS. Fig. 70C is a schematic
diagram showing the video image IMG and subtitle SUB displayed on the screen SCR
in that case. Here, the color coordinate value indicated by the SPRM(37) is an opaque
color such as black. As shown in Fig. 70C, in the window WIN2 indicating the display
range of the subtitle SUB, the background color is the opaque color. Accordingly, in the
window WTN2, the video image IMG is hidden by the background color, and only the
subtitle SUB is displayed. In this way, it is possible to present the video images and the
subtitle to the viewer in a reliable manner.
[0479]
(1-0-9) With a switch between video shift modes, not only the display position
of the graphics image represented by the PG stream but the PG stream itself may be
changed. Fig. 71A is a schematic diagram showing a further another example of the
STN table in the playlist file for the video content of the letterbox display. As shown in
Fig. 71 A, in the STN table 7100, an STN 7101 is associated with a stream entry 7102 of
the PG stream 1 and stream attribute information 7103. The stream attribute information
7103 includes a video upward/downward move shift value 7110, 7111. The video
upward move shift value 7110 and the video downward move shift value 7111 indicates
PEDs of the PG streams that are to be selected when the Up mode and the Down mode
are selected as the video shift mode, respectively. In the PG streams indicated by the
video upward move shift value 7110 and the video downward move shift value 7111,
initially the display position of the subtitle is set in each black strip of the lower and
upper portions of the primary video plane. In that case, the playback device 102 may
use the following structure for the video shift.
[0480]
Fig. 71B is a functional block diagram showing a further another example of
the structure of the playback device 102 required for the video shift. The structure
shown in Fig. 71B differs from the structure shown in Fig. 65 in the following points:
(A) each time the video shift mode is switched, the playback control unit 7135 specifies
the PID of the PG stream to be selected newly to the PG decoder 4072; (B) the second
cropping unit 4732 does not provide the PG plane 4704 with a vertical offset; (C) the
SPRM(33) and SPRM(34) may not be set in the player variable storage unit 4236. The
other components are the same. Thus in Fig. 7IB, the components that are also shown
in Fig. 65 are assigned the same reference numbers. Furthermore, details on these
components can be found in the description of Fig. 65.
[0481]
The video shift mode indicated by the SPRM(32) 6502 is changed by the
program execution unit 4234 in accordance with an application program or the user.
Each time it detects a change in the value stored in the SPRM(32) 6502, the playback
control unit 7135 refers to the STN table shown in Fig. 71 A. By doing so, the playback
control unit 7135 retrieves the PID of the PG stream corresponding to the video shift
mode after the change, and passes the PID to the PG decoder 4072. More specifically,
when the SPRM(32) 6502 indicates the Up mode, the playback control unit 7135
retrieves the PID indicated by the video upward move subtitle 7110; when the
SPRM(32) 6502 indicates the Down mode, the playback control unit 7135 retrieves the
PID indicated by the video downward move subtitle 7111; and when the SPRM(32)
6502 indicates the Keep mode, the playback control unit 7135 retrieves the PID
indicated by the stream entry 7102. As a result, the PG plane 4704 decoded by the PG
decoder 4072 represents a subtitle that corresponds to the video shift mode.
[0482]
Fig. 72A is a schematic diagram showing the subtitles SB1 and SB2 that
correspond to the Keep mode. Fig. 72B is a schematic diagram showing the subtitles
SB1 and SB2 that correspond to the Down mode. As shown in Fig. 72A, in the Keep
mode, a horizontal subtitle SB1 is displayed on top of the lower portion of the video
image display area VP1, and a vertical subtitle SB2 is displayed on top of the right-end
portion of the video image display area VP1. Furthermore, the black strips BT and BB
that are 131 pixels high are displayed in the portions above and below the video image
display area VP1, respectively. As shown in Fig. 72B, in the Down mode, the black
strip BT2 that is 262 pixels high is displayed in the portion above the video image
display area VP2. If the display position of the horizontal subtitle SB1 were moved into
the black strip BT2 of the upper portion by providing the PG plane with a vertical offset,
the display position of the vertical subtitle SB2 would be moved upward outside the
screen to the display position SB20. On the other hand, the PG stream indicated by the
video downward move subtitle 7111 represents the horizontal subtitle SBD, and the
display position thereof has been set in the black strip BT2 in upper portion in advance.
Accordingly, the playback control unit 7135 changes the PG stream that represents the
horizontal subtitle SB1 in the Keep mode to the PG stream indicated by the video
downward move subtitle 7111 in the Down mode. On the other hand, the playback
control unit 7135 uses the PG stream, which represents the vertical subtitle SB2 in the
Keep mode, as it is in the Down mode. With this structure, in the Down mode, as shown
in Fig. 72B, the horizontal subtitle SBD is displayed in the black strip BT2 in the upper
portion, and the vertical subtitle SB2 is displayed, as in the Keep mode, on top of the
right-end portion of the video image display area VP1. This also applies to the Up
mode.
[0483]
When the video upward move subtitle 7110, the video downward move subtitle
7111, or the stream entry 7102 is not registered in the STN table, no new PID is
specified by the playback control unit 7135, thus the PG decoder 4072 maintains the
PID held at this point as it is. In that case, the second cropping unit 4732 may provide
the PG plane 4704 with a vertical offset. This offset is the same as the offset provided
by the video shift unit 6501 to the primary video planes 4701, 4702. Fig. 72C is a
schematic diagram showing the subtitle SB1 displayed in the Keep mode. Fig. 72D is a
schematic diagram showing the subtitle SB3 displayed in the Up mode when the video
upward subtitle 7110 is not registered in the STN table. As shown in Fig. 72C, in the
Keep mode, the subtitle SB1 is displayed on top of the lower portion of the video image
display area VP1. Furthermore, the black strips BT and BB that are 131 pixels high are
displayed in the portions above and below the video image display area VP1,
respectively. As shown in Fig. 72D, in the Up mode, the black strip BB2 that is 262
pixels high is displayed in the portion below the video image display area VP2. If the
display position of the horizontal subtitle SB1 were maintained to the position in the
Keep mode, the lower portion of the subtitle SB1 would be displayed on top of the
black strip BB2. In contrast, when the second cropping unit 4732 provides the PG plane
4704 with a vertical offset, as shown in Fig. 72D, in the Up mode, the subtitle SB3 is
displayed in the lower portion of the video image display area VP2 at a position
separate from the black strip BB2 in a similar manner to the subtitle SB 1 in the Keep
mode.
[0484]
Embodiment 2
The BD-ROM disc according to Embodiment 2 of the present invention also
includes a pair of a base view and a dependent view for the PG stream and the IG
stream. On the other hand, the playback device according to Embodiment 2 of the
present invention is provided with 2 plane mode. "2 plane mode" is one of the display
modes for the graphics plane. When a sub-TS includes both a base-view and
dependent-view graphics stream, the playback device in 2 plane mode decodes and
alternately outputs left-view and right-view graphics plane data from the graphics
streams. 3D graphics images can thus be played back from the graphics streams. Apart
from these points, the BD-ROM disc and playback device according to Embodiment 2
have the same structure and functions as according to Embodiment 1. Accordingly, the
following is a description of the BD-ROM disc and playback device according to
Embodiment 2 insofar as these have been changed or expanded as compared to
Embodiment 1. Details on the parts of the BD-ROM disc and playback device that are
the same as according to Embodiment 1 can be found in the description of Embodiment
1.
[0486]
[0487]
Fig. 73A is a list of elementary streams multiplexed in a first sub-TS on a
BD-ROM disc 101. The first sub-TS is multiplexed stream data in MPEG-2 TS format
and is included in a file DEP for the L/R mode. As shown in Fig. 73A, the first sub-TS
includes a primary video stream 7311, left-view PG streams 7312A and 7312B,
right-view PG streams 7313A and 7313B, left-view IG stream 7314, right-view IG
stream 7315, and secondary video stream 7316. When the primary video stream 301 in
the main TS shown in Fig. 3 A represents the left view of 3D video images, the primary
video stream 7311, which is a right-view video stream, represents the right view of the
3D video images. The pairs of left-view and right-view PG streams 7312A + 7313A and
7312B + 7313B represent the left view and right view of graphics images, such as
subtitles, when these graphics images are displayed as 3D video images. The pair of
left-view and right-view IG streams 7314 and 7315 represent the left view and right
view of graphics images for an interactive screen when these graphics images are
displayed as 3D video images. When the secondary video stream 306 in the main TS
represents the left view of 3D video images, the secondary video stream 7316, which is
a right-view video stream, represents the right view of the 3D video images.
[0488]
PIDs are assigned to the elementary streams 7311-7316 as follows, for example.
A PID of 0x1012 is assigned to the primary video stream 7311. When up to 32 other
elementary streams can be multiplexed by type in one sub-TS, the left-view PG streams
7312A and 7312B are assigned any value from 0x1220 to 0xl23F, and the right-view
PG streams 7313A and 7313B are assigned any value from 0x1240 to 0xl25F. The
left-view IG stream 7314 is assigned any value from 0x1420 to 0xl43F, and the
right-view IG stream 7315 is assigned any value from 0x1440 to 0xl45F. The
secondary video stream 7316 is assigned any value from 0xlB20 to 0xlB3F.
[0489]
Fig. 73B is a list of elementary streams multiplexed in a second sub-TS on a
BD-ROM disc 101. The second sub-TS is multiplexed stream data in MPEG-2 TS
format and is included in a file DEP for the depth mode. Alternatively, the second
sub-TS may be multiplexed in the same file DEP as the first sub-TS. As shown in Fig.
73B, the second sub-TS includes a primary video stream 7321, depth map PG streams
7323A and 7323B, depth map IG stream 7324, and secondary video stream 7326. The
primary video stream 7321 is a depth map stream and represents 3D video images in
combination with the primary video stream 301 in the main TS. When the 2D video
images represented by the PG streams 323A and 323B in the main TS are used to
project 3D video images on a virtual 2D screen, the depth map PG streams 7323A and
7323B are used as the PG streams representing a depth map for the 3D video images.
When the 2D video images represented by the IG stream 304 in the main TS are used to
project 3D video images on a virtual 2D screen, the depth map IG stream 7324 is used
as the IG stream representing a depth map for the 3D video images. The secondary
video stream 7326 is a depth map stream and represents 3D video images in
combination with the secondary video stream 306 in the main TS.
[0490]
PIDs are assigned to the elementary streams 7321-7326 as follows, for example.
A PID of 0x1013 is assigned to the primary video stream 7321. When up to 32 other
elementary streams can be multiplexed by type in one sub-TS, the depth map PG
streams 7323A and 7323B are assigned any value from 0x1260 to 0xl27F. The depth
map IG stream 7324 is assigned any value from 0x1460 to 0xl47F. The secondary
video stream 7326 is assigned any value from 0xlB40 to 0xlB5F.
[0491]
[0492]
Fig. 74 is a schematic diagram showing a data structure of the STN table SS
3130 according to Embodiment 2 of the present invention. As shown in Fig. 74, the
stream registration information sequences 3301, 3302, 3303, ... in the STN table SS
3130 include a stream registration information sequence 7413 of a PG stream and a
stream registration information sequence 7414 of an IG stream in addition to an offset
during pop-up 3311 and a stream registration information sequence 3312 of a
dependent-view video stream that are shown in Fig. 33.
[0493]
The stream registration information sequence 7413 of a PG stream includes
stream registration information indicating the PG streams that can be selected for
playback from the sub-TS. The stream registration information sequence 7414 of an IG
stream includes stream registration information indicating the IG streams that can be
selected for playback from the sub-TS. These stream registration information sequences
7413 and 7414 are used in combination with the stream registration information
sequences, included in the STN table of the corresponding PI, that indicate PG streams
and IG streams. When reading a piece of stream registration information from an STN
table, the playback device 102 in 3D playback mode automatically also reads the stream
registration information sequence, located in the STN table SS, that has been combined
with the piece of stream registration information. When simply switching from 2D
playback mode to 3D playback mode, the playback device 102 can thus maintain
already recognized STNs and stream attributes such as language.
[0494]
As further shown in Fig. 74, the stream registration information sequence 7413
of the PG stream generally includes a plurality of pieces of stream registration
information 7431. These are the same in number as the pieces of stream registration
information in the corresponding PI that indicates the PG streams. The stream
registration information sequence 7414 of the IG stream includes the same sort of pieces
of stream registration information. These are the same in number as the pieces of stream
registration information in the corresponding PI that indicates the IG streams.
[0495]
Each piece of stream registration information 7431 includes an STN 7441,
stereoscopic flag (is_SS_PG) 7442, base-view stream entry (stream_entry_for_
base_view) 7443, dependent-view stream entry (stream_entry_for_dependent_view)
7444, and stream attribute information 7445. The STN 7441 is a serial number assigned
individually to pieces of stream registration information 7431 and is the same as the
STN assigned to the piece of stream registration information, located in the
corresponding PI, with which the piece of stream registration information 7431 is
combined. The stereoscopic flag 7442 indicates whether a BD-ROM disc 101 includes
both base-view and dependent-view PG streams. If the stereoscopic flag 7442 is on, the
sub-TS includes both PG streams. Accordingly, the playback device reads all of the
fields in the base-view stream entry 7443, the dependent-view stream entry 7444, and
the stream attribute information 7445. If the stereoscopic flag 7442 is off, the playback
device ignores all of these fields 7443-7445. Both the base-view stream entry 7443 and
the dependent-view stream entry 7444 include sub-path ID reference information 7421,
stream file reference information 7422, and PIDs 7423. The sub-path ID reference
information 7421 indicates a sub-path ID of a sub-path that specifies the playback paths
of the base-view and dependent-view PG streams. The stream file reference information
7422 is information to identify the file DEP storing the PG streams. The PIDs 7423 are
the PIDs for the PG streams. The stream attribute information 7445 includes attributes
for the PG streams, such as language type.
[0496]
Note that the stream registration information 7431 of the PG stream may be
stored in the STN table instead of the STN table SS. In that case, the stream registration
information 7431 is stored in PG streams in the main TS, in particular in the stream
attribute information therein.
[0497]
[0498]
Fig. 75 is a functional block diagram of a system target decoder 7525
according to Embodiment 2 of the present invention. As shown in Fig. 75, the PG
decoder 7501 supports 2 plane mode, unlike the PG decoder shown in Fig. 45.
Specifically, the PG decoder 7501 includes a base-view PG decoder 7511 and a
dependent-view PG decoder 7512. In addition to decoding the PG streams 303A and
303B in the main TS shown in Fig. 3A, the base-view PG decoder 7511 decodes the
left-view PG streams 7312A and 7312B in the first sub-TS shown in Fig. 73A into
plane data. The dependent-view PG decoder 7512 decodes the right-view PG streams
7313A and 7313B in the first sub-TS shown in Fig. 73A and the depth map PG streams
7323A and 7323B in the second sub-TS shown in Fig. 73B into plane data. The
secondary video decoder and the IG decoder both include a similar pair of decoders.
The system target decoder 7525 further includes a pair of PG plane memories 7521 and
7522. The base-view PG decoder 7511 writes the plane data into the left PG plane
memory 7521, and the dependent-view PG decoder 7512 writes the plane data into the
right PG plane memory 7522. The IG plane memory and the image plane memory both
have similar structures. The system target decoder 7525 further associates the output of
plane data from the graphics plane memory with 2 plane mode, 1 plane + offset mode,
and 1 plane + zero offset mode. In particular, when the playback control unit 4235
indicates 2 plane mode, the system target decoder 7525 alternately outputs plane data
from a pair of PG plane memories 7521 and 7522 to the plane adder 7526.
[0499]
[0500]
Fig. 76 is a partial functional block diagram of the plane adder 7526 in 2 plane
mode. As shown in Fig. 76, the plane adder 6226 includes a parallax video generation
unit 4710, switch 4720, first adder 4741, and second adder 4742, like the plane adder
4226 shown in Fig. 47. The plane adder 7526 further includes a second parallax video
generation unit 7610 and a second switch 7620 as units for input of PG plane data 7604
and 7605. A similar structure is included in the units for input of secondary video plane
data, IG plane data, and image plane data.
[0501]
The second parallax video generation unit 7610 receives left PG plane data
7604 and right PG plane data 7605 from the system target decoder 7525. In the
playback device 102 in L/R mode, the left PG plane data 7604 represents the left-view
PG plane, and the right PG plane data 7605 represents the right-view PG plane. At this
point, the second parallax video generation unit 7610 transmits the pieces of plane data
7604 and 7605 as they are to the switch 7620. On the other hand, in the playback device
102 in depth mode, the left PG plane data 7604 represents the PG plane of 2D graphics
images, and the right PG plane data 7605 represents a depth map corresponding to the
2D graphics images. In this case, the second parallax video generation unit 7610 first
calculates the binocular parallax for each element in the 2D graphics images using the
depth map. Next, the second parallax video generation unit 7610 processes the left PG
plane data 7604 to shift the presentation position of each element in the 2D graphics
image in the PG plane to the left or right in accordance with the calculated binocular
parallax. This generates a pair of PG planes representing a left view and right view.
Furthermore, the second parallax video generation unit 7610 outputs this pair of PG
planes to the second switch 7620.
[0502]
The second switch 7620 outputs the left PG plane data 7604 and the right PG
plane data 7605, which have the same PTS, to the second adder 4742 in this order. The
second cropping unit 4732 in the 2 plane mode outputs each of the PG plane data 7604
and 7605 to the second adder 4742 as it is. The second adder 4742 superimposes each of
the PG plane data 7604 and 7605 on the plane data from the first adder 4741 and
transmits the result to the third adder 4743. As a result, the left-view PG plane is
superimposed on the left-video plane data 7601, and the right-view PG plane is
superimposed on the right-video plane data 7602.
[0503]
The second cropping unit 4732 in the 2 plane mode, in a similar manner to the
second cropping unit 4732 in the 1 plane + offset mode shown in Fig. 47, may provide
each of PG plane data 7604 and 7605 with a horizontal offset by using the offset
adjustment value. This enables the depth of the 3D graphics video images to be adjusted
in conjunction with the screen size of the display device 103. Alternatively, the second
cropping unit 4732, in a similar manner to the second cropping unit 4732 in the 1 plane
+ offset mode shown in Figs. 65 and 69, may provide each of PG plane data 7604 and
7605 with a vertical offset by using the video upward/downward move shift value.
When a 3D video image is presented in a letterbox display, this enables a 3D graphics
image (e.g. subtitle) to be presented in a black strip which is located above or below the
3D video image.
[0504]