DESCRIPTION
[Title of Invention]
RECORDING MEDIUM, REPRODUCTION DEVICE, AND INTEGRATED CIRCUIT
[Technical Field]
[0001]
The present invention relates to a technology for
stereoscopic, i.e. three-dimensional (3D), video playback and
especially to the allocation of a video stream on a recording
medium.
[Background Art]
[0002]
In recent years, general interest in 3D video has been
increasing. For example, amusement park attractions that
incorporate 3D video images are popular. Furthermore,
throughout the country, the number of movie theaters showing
3D movies is increasing. Along with this increased interest in
3D video, the development of technology that enables playback
of 3D video images in the home has also been progressing. There
is demand for this technology to store 3D video content on a
portable recording medium, such as an optical disc, while
maintaining the 3D video content at high image quality.
Furthermore, there is demand for the recording medium to be
compatible with a two-dimensional (2D) playback device. That
is, it is preferable for a 2D playback device to be able to play
back 2D video images and a 3D playback device to be able to play
back 3D video images from the same 3D video content recorded
on the recording medium. Here, a "2D playback device" refers
to a conventional playback device that can only play back
monoscopic video images, i.e. 2D video images, whereas a "3D
playback device" refers to a playback device that can play back
3D video images. Note that in general a 3D playback device can
also play back 2D video images.
[0003]
Fig. 63 is a schematic diagram illustrating the
technology for ensuring the compatibility of an optical disc
storing 3D video content with 2D playback devices (see Patent
Literature 1) . An optical disc 63 01 stores two types of AV (Audio
Visual) stream files. One of these files is a 2D/left-view
stream file, and the other is a right-view stream file. The
2D/left-view stream file contains a 2D/left-view stream. A
"2D/left-view stream" represents a 2D video image to be shown
to the left eye of a viewer during 3D playback, i.e. a
"left-view." During 2D playback, this stream constitutes the
2D video image. The right-view stream file contains a right-view
stream. A "right-view stream" represents a 2D video image to
be shown to the right eye of a viewer during 3D playback, i.e.
a "right-view." The left and right video streams have the same
frame rate but different presentation times shifted from each
other by half a frame period. For example, when the frame rate
of each video stream is 24 frames per second, the frames of the
left-view and right-view streams are alternately displayed
every 1/48 seconds.
[0004]
As shown in Fig. 63, the left-view and right-view stream
files are divided into a plurality of extents 6302A-C and
6303A-C respectively on the optical disc 6301. Each extent
contains at least one GOP (group of pictures) which is read
together from the optical disc. Hereinafter, the extents
belonging to the 2D/left-view stream file are referred to as
"left-view extents," and the extents belonging to the
right-view stream file are referred to as "right-view extents. "
The left-view extents 6302A-C and the right-view extents
6303A-C are alternately arranged on a track 6301A of the optical
disc 6301. Each two adjacent extents 6302A-6303A, 6302B-6303B
and 6302C-6303C have the same length of playback time. Such an
arrangement of extents is referred to as an interleaved
arrangement. Groups of extents recorded in an interleaved
arrangement on a recording medium are used both in 3D video
playback and 2D video image playback, as described below.
[0005]
From among the extents recorded on the optical disc 6301,
a 2D playback device 6304 causes an optical disc drive 63 04A
to read only the left-view extents 63 02A-C sequentially from
the start, skipping the reading of right-view extents 63 03A-C.
Furthermore, an image decoder 63 04B sequentially decodes the
extents read by the optical disc drive 6304A into a left-view
frame 6306L. In this way, a display device 6307 only displays
left-views, and viewers can watch normal 2D video images.
[0006]
A 3D playback device 6305 causes a 3D optical disc drive
63 05A to alternately read left-view extents and right-view
extents from the optical disc 6301. Expressed as codes, the
extents are read in the order 6302A, 6303A, 6302B, 6303B, 6302C,
and 6303C. Furthermore, from among the read extents, those
belonging to the 2D/left-view stream are supplied to a left
video decoder 63 05L, whereas those belonging to the right-view
stream are supplied to a right-video decoder 6305R. The video
decoders 63 05L and 63 05R alternately decode each video stream
into video frames 6306L and 6306R, respectively. As a result,
left-views and right-views are alternately displayed on a
display device 6308. In synchronization with the switching of
the views by the display device 6308, shutter glasses 6309 cause
the left and right lenses to become opaque alternately.
Therefore, a viewer wearing the shutter glasses 6309 sees the
views displayed by the display device 6308 as 3D video images.
[0007]
When 3D video content is stored on any recording medium,
not only on an optical disc, the above-described interleaved
arrangement of extents is used. In this way, the recording
medium can be used both for playback of 2D video images and 3D
video images.
[Citation List]
[Patent Literature]
[0008]
[PTL 1]
Japanese Patent No. 3935507
[Summary of Invention]
[Technical Problem]
[0009]
As shown in Fig. 63, when 2D video images are played back
from extent groups in an interleaved arrangement, the optical
disc drive 6304A performs a "jump" at each recording area for
the right extents 63 03A-C. That is, the optical disc drive 63 04A
skips over the reading of data from these recording areas.
During the jump period, since no data from the optical disc drive
6304A is provided to the buffer in the 2D playback device 6304,
the amount of data stored in the buffer decreases as processing
progresses in the video decoder 6304B. Therefore, in order to
seamlessly playback 2D video images, the amount of data, or the
size, of each of the left-view extents 6302A-C needs to be equal
to or greater than the size at which buffer underflow can be
prevented during the jump period.
[0010]
On the other hand, when 3D video images are played back
from the same extent groups, none of the right-view extents
6303A-C can be read while one of the left-view extents 6302A-C
is being read. Therefore, at this time, the amount of data for
the right-view extents 63 03A-C stored in the buffer in the 3D
playback device 63 05 decreases as processing progresses in the
right video decoder 63 05R. Conversely, while one of the
right-view extents 63 03A-C is being read, the amount of data
for the left-view extents 6302A-C stored in the buffer decreases
as processing progresses in the left video decoder 63 05L.
Therefore, in order to seamlessly play back 3D video images,
the size of each of the extents 6302A-C and 6303A-C needs to
be equal to or greater than the size at which data for one type
of extents will not be depleted while the other type of extents
is being read.
[0011]
Two or more different right-view streams can be recorded
for a common 2D/left-view stream on a recording medium. In this
case, by combining different right-view streams with the
left-view stream, a 3D playback device can, for example, play
back 3D video images with different binocular disparities from
the same recording medium.
[0012]
A combination of a video stream that represents 2D video
images and a "depth map stream" can also be stored on the
recording medium. Such 2D video images represent 3D video images
projected on a hypothetical 2D picture plane. The depth map
stream includes a depth map for each frame in the 3D video images.
A depth map represents a depth of the 3D video images at each
pixel as compared to the 2D picture plane. A 3D playback device
can play back 3D video images by constructing a combination of
a left-view stream and a right-view stream from a combination
of the video stream and the depth map stream. Here, the
above-mentioned 2D/left-view stream can be used as the video
stream representing the 2D video images. In this case, in
addition to the 2D/lef t-view stream and the right-view stream,
a depth map stream is recorded on the recording medium.
[0013]
In the above-described ways, when in addition to a single
combination of left and right video streams, another video
stream is recorded on one recording medium, three or more types
of extents are arranged in an interleaved manner on the
recording medium. Therefore, when 3D video images are played
back from such extent groups, the optical disc drive performs
a jump at each recording area for the extents that are not
targeted for playback, as it does during playback of 2D video
images. As a result, in order to seamlessly play back 3D video
images, the size of each extent needs to be equal to or greater
than the size at which buffer underflow or depletion of data
for one type of the extents can be avoided during each jump
period.
[0014]
In all of the above-mentioned cases, if the size of each
extent is simply large, buffer underflow and the like can be
avoided. If the extents are too large, however, it is difficult
to further decrease the capacity of the buffer. This is not
preferable, as it makes it difficult to further reduce the
circuit size of the playback device and to further reduce
manufacturing cost.
[0015]
An object of the present invention is to provide a
recording medium having two or more stream files recorded
thereon in a more appropriate arrangement so as to allow further
reduction in the buffer capacity of a playback device without
causing buffer underflow to occur.
[Solution to Problem]
[0016]
On a recording medium according to the present invention,
a base-view stream file and a dependent-view stream file are
recorded. The base-view stream file is used for monoscopic video
playback. The dependent-view stream file is used for
stereoscopic video playback in combination with the base-view
stream file. The recording medium further includes a contiguous
area in which a plurality of base-view extents belonging to the
base-view stream file and a plurality of dependent-view extents
belonging to the dependent-view stream file are arranged in an
interleaved manner. A lower limit of a size of each base-view
extent is a larger one of a first lower limit value and a second
lower limit value. The first lower limit value is determined
so that, in monoscopic video playback, buffer underflow does
not occur in a playback device during a jump period from each
base-view extent to a next base-view extent. The second lower
limit value is determined so that, in stereoscopic video
playback, buffer underflow does not occur in a playback device
during a read period of a next dependent-view extent after each
base-view extent. A lower limit of a size of each dependent-view
extent is a third lower limit value. The third lower limit value
is determined so that, in stereoscopic video playback, buffer
underflow does not occur in a playback device during a read
period of a next base-view extent after each dependent-view
extent.
[Advantageous Effects of Invention]
[0017]
In the above-described recording medium according to the
present invention, the lower limits for the sizes of the
base-view extents and the dependent-view extents are clear. As
a result, it is easy to appropriately design the size of each
extent so that the buffer capacity of the playback device can
be further reduced without causing buffer underflow to occur
during playback from the recording medium of both monoscopic
video images and stereoscopic video images.
[Brief Description of Drawings]
[0018]
Fig. 1 is a schematic diagram showing a home theater system
using a recording medium according to Embodiment 1 of the
present invention.
Fig. 2 is a schematic diagram showing the data structure
of the BD-ROM disc shown in Fig. 1.
Fig. 3 is a schematic diagram showing an index table
included in the index file shown in Fig. 2.
Fig. 4 is a schematic diagram showing elementary streams
multiplexed in the first AV stream file shown in Fig. 2.
Fig. 5 is a schematic diagram showing an arrangement of
packets in each elementary stream multiplexed in the AV stream
file shown in Fig. 2.
Fig. 6 is a schematic diagram showing the pictures in the
video stream shown in Fig. 5 in the time order in which they
are displayed.
Fig. 7 is a schematic diagram showing details of the data
structure of the video stream shown in Fig. 5.
Fig. 8 is a schematic diagram showing details of a method
for storing a video stream into PES packets shown in Fig. 5.
Figs . 9A, 9B, and 9C are schematic views showing the format
of a TS packet and a source packet which constitute the 2D/AV
stream file shown in Fig. 5.
Fig. 10 is a schematic diagram showing the data structure
of a PMT.
Fig. 11 is a schematic diagram showing the data structure
of the part of the first clip information file shown in Fig.
2 that is used as a 2D clip information file.
Fig. 12 is a schematic diagram showing the data structure
of stream attribute information shown in Fig. 11.
Fig. 13A is a schematic diagram showing the data structure
of the entry map shown in Fig. 11, and Fig. 13B is a schematic
diagram showing the source packets, from among the source
packets included in the first AV stream file, that are
associated with each EP_ID by the entry map.
Fig. 14 is a schematic diagram showing the data structure
of a 2D playlist file.
Fig. 15 is a schematic diagram showing the data structure
of playitem information shown in Fig. 14.
Figs. 16A and 16B are schematic diagrams showing the
relationship between playback sections specified by each piece
of playitem information to be connected when the connection
condition shown in Fig. 15 indicates "5" and "6, " respectively.
Fig. 17 is a schematic diagram showing the data structure
of a 2D playlist file when the playback path includes sub-paths.
Fig. 18 is a functional block diagram of a 2D playback
device.
Fig. 19 is a list of system parameters stored in the player
variable storage unit shown in Fig. 18.
Fig. 20 is a functional block diagram of a system target
decoder shown in Fig. 18.
Fig. 21 is a schematic diagram showing a physical
arrangement of extents on the BD-ROM disc shown in Fig. 2.
Fig. 22 is a schematic diagram showing the playback
process system inside the 2D playback device shown in Fig. 18.
Fig. 23 is a graph showing the change of the accumulated
data amount in the read buffer shown in Fig. 22 during a playback
process period of an AV stream file.
Fig. 24 is a table showing an example of the relationship
between jump distances and jump times specified for BD-ROM
discs.
Fig. 25 is a schematic diagram showing an example of a
physical arrangement of extents when 2D video images are
continuously played back from three different sections of an
AV stream file in turn.
Figs. 26A, 26B, 26C are schematic diagrams illustrating
the principle of stereoscopic video playback according to a
method using parallax video.
Fig. 27 is a schematic diagram showing an example of
constructing a left-view and a right-view from a combination
of 2D video images and a depth map.
Fig. 28 is a schematic diagram showing the relationship
between a movie object and a BD-J object, as related to the
playback of 3D video images described in an index table, and
three types of playlist files.
Fig. 29 is a flowchart showing processes, performed when
a title of 3D video images is selected, to select a playlist
file to be played back.
Fig. 30 is a schematic diagram showing an example of the
structures of the first playlist file and the second playlist
file shown in Fig. 2.
Fig. 31 is a schematic diagram showing an example of the
structures of the first playlist file and the third playlist
file shown in Fig. 2.
Fig. 32 is a schematic diagram showing another example
of the structures of a first playlist file and a 3D playlist
file.
Fig. 33A is a schematic diagram showing the elementary
stream multiplexed in the second AV stream file shown in Fig.
2, and Fig. 33B is a schematic diagram showing the elementary
stream multiplexed in the third AV stream file shown in Fig.
2.
Fig. 34 is a schematic diagram showing the pictures
included in the left-view stream and the pictures included in
the right-view stream in the time order in which they are
displayed.
Figs. 35A and 35B are schematic diagrams showing the
relationship between PTSs and DTSs allocated to each picture
in the base-view stream and the dependent-view stream.
Fig. 36 is a schematic diagram showing the data
configuration of a video access unit in the left-view stream,
the right-view stream, and the depth map stream.
Figs. 37A and 37B are schematic diagrams showing the
values of decode counters allocated to each picture in the
base-view stream and the dependent-view stream.
Fig. 38A is a schematic diagram showing the data structure
of a base-view clip information file, and Fig. 3 8B is a schematic
diagram showing the data structure of a dependent-view clip
information file.
Figs. 39A and 39B are schematic diagrams showing the data
structure of the 3D meta data shown in Fig. 38A.
Fig. 4 0A is a schematic diagram showing the data structure
of the entry map for the dependent-view clip information file
shown in Fig. 38B, and Fig. 40B is a schematic diagram showing
the relationship between the SPN shown by an entry point and
an extent on the BD-ROM disc.
Fig. 41 is a schematic diagram showing an example of entry
points set in the base-view stream and the dependent-view
stream.
Fig. 42 is a functional block diagram of a 3D playback
device.
Fig. 43 is a functional block diagram of the plane adder
unit shown in Fig. 42.
Figs. 44A and 44B are schematic diagrams showing the
cropping processing by each of the first through third cropping
processing units shown in Fig. 43.
Figs. 45A and 45B are schematic diagrams respectively
showing a left-view and a right-view superimposed by cropping
processes shown in Figs. 44A and 44B, and Fig. 45C is a schematic
diagram showing the resulting 3D video images that the viewer
is made to perceive.
Fig. 46 is a functional block diagram of the system target
decoder shown in Fig. 42.
Figs. 47A and 47B are schematic diagrams showing the
arrangement on a disc of the extents belonging to a base-view
stream file and a dependent-view stream file.
Figs. 48A and 48B are schematic diagrams showing the
relationship between the playback time of the video stream per
extent and the playback path.
Fig. 4 9 is a schematic diagram showing the position on
a BD-ROM disc of the extent groups belonging to a 2D/left-view
stream file, a right-view stream file, and a depth map stream
file.
Fig. 50 is a schematic diagram showing (i) the entry points
set for a combination Dn, Rn, and Ln of the nth extent among
the interleaved arrangement shown in Fig. 49 and (ii) the
playback path during interrupt playback.
Fig. 51 is a schematic diagram showing the playback
process system in the 3D playback device shown in Fig. 42.
Fig. 52 is a graph showing the change of the accumulated
data amount in the first read buffer when the 2D playback device
plays back 2D video images from the extent groups shown in Fig.
49.
Figs. 53A, 53B, and 53C are graphs showing the change of
the accumulated data amount in each read buffer when the 3D
playback device plays back 3D video images in L/R mode from the
extent groups shown in Fig. 49.
Figs. 54A, 54B, and 54C are graphs showing the change of
the accumulated data amount in each read buffer when the 3D
playback device plays back 3D video images in depth mode from
the extent groups shown in Fig. 49.
Fig. 55 is a schematic diagram showing the arrangement
of extent groups belonging to each file when only two types of
files, a 2D/left-view stream file and a right-view stream file,
are recorded on a BD-ROM disc.
Fig. 56 is a schematic diagram showing (i) a jump JLY caused
by layer switching occurring during the playback path of 3D
video images in L/R mode and (ii) an object JBdj caused by the
read process of the BD-J object file.
Figs. 57A and 57B are graphs showing the change of the
accumulated data amount in each read buffer when a margin time
is added to the zero sector transition time.
Figs. 58A and 58B are schematic diagrams showing the
relationship between a section in which interrupt playback is
prohibited and the change of the accumulated data amount in the
first read buffer.
Fig. 5 9 is a schematic diagram showing the arrangement
of the extent groups in an interleaved arrangement shown in Fig.
49 when an extent belonging to a file other than the AV stream
file is inserted into the extent groups.
Figs. 60A and 6OB are graphs showing the change of the
accumulated data amount in each read buffer when a different
method is used to guarantee the available buffer capacity in
each read buffer.
Fig. 61 is a block diagram showing the inner structure
of a recording device according to Embodiment 2 of the present
invention.
Figs. 62A, 62B, and 62C are schematic diagrams showing
processes to calculate depth information from a pair of left
and right pictures by the video encoder shown in Fig. 61.
Fig. 63 is a schematic diagram showing technology to
ensure compatibility of an optical disc with 3D video content
recorded thereon with a 2D playback device.
[Description of Embodiments]
[0019]
The following describes a recording medium and a playback
device pertaining to preferred embodiments of the present
invention with reference to the drawings.
[0020]
[Embodiment 1]
[0021]
Fig. 1 is a schematic diagram showing a home theater system
using a recording medium according to Embodiment 1 of the
present invention. In Fig. 1, the recording medium 101 is a
read-only Blu-ray disc (BD) , i.e. a BD-ROM disc. The recording
medium 101 can be a different portable recording medium, such
as an optical disc with a different format such as DVD or the
like, a removable hard disk drive (HDD), or a semiconductor
memory device such as an SD memory card. This recording medium,
i.e. the BD-ROM disc 101, stores a movie content as 3D video
images . The home theater center includes a playback device 102,
a display device 103, shutter glasses 104, and a remote control
105.
[0022]
A BD-ROM drive 121 is mounted on the playback device 102.
The BD-ROM drive 121 is an optical disc drive conforming to the
BD-ROM format. The playback device 102 uses the BD-ROM drive
121 to read a movie content from the BD-ROM disc 101. The playback
device 102 further decodes the movie content into video
data/audio data. When the playback device 102 is a 2D playback
device, or when the display device 103 only supports playback
of 2D video images, then the video data only includes either
a left-view or a right-view video frame. On the other hand, when
the playback device 102 is a 3D playback device, and when the
display device 103 supports playback of 3D video images, then
the video data includes both left-view and right-view video
frames.
[0023]
The playback device 102 is connected to the display device
103 via an HDMI (High-Definition Multimedia Interface) cable
122. The playback device 102 converts the video data/audio data
into a video signal/audio signal in the' HDMI format and sends
the signals to the display device 103 via the HDMI cable 122.
Here, when the playback device 102 is a 2D display device, or
when the display device 103 only supports playback of 2D video
images, then only one of either the left-view or the right-view
video frame is multiplexed in the video signal. On the other
hand, when the playback device 102 is a 3D playback device, and
when the display device 103 supports playback of 3D video images,
then both the left-view and the right-view video frames are
time-multiplexed in the video signal. Additionally, the
playback device 102 exchanges CEC messages with the display
device 103 via the HDMI cable 122. In this way, the playback
device 102 asks the display device 103 whether it supports
playback of 3D video images.
[0024]
The display device 103 is a liquid crystal display.
Alternatively, the display device 103 can be another type of
flat panel display, such as a plasma display, an organic EL
display, etc. , or a projector. The display device 103 displays
video on the screen 131 in accordance with a video signal, and
causes the speakers to produce audio in accordance with an audio
signal. Here, when the playback device 102 is a 2D display device,
or when the display device 103 only supports playback of 2D video
images, then only one of either the left-view or the right-view
is displayed on the screen 131. On the other hand, when the
playback device 102 is a 3D playback device, and when the display
device 103 supports playback of 3D video images, then the
left-view and the right-view are displayed alternately on the
screen 131.
[0025]
When the display device 103 supports playback of 3D video
images, a left/right signal transmitting unit 132 is further
mounted on the display device 103. The display device 103 uses
this unit to transmit a left/right signal LR to the shutter
glasses 104 via infrared rays or by radio transmission. The
left/right signal LR indicates whether the image displayed on
the screen 131 at that point is a left-view or a right-view image.
The display device 103 distinguishes between a left-view frame
and a right-view frame using the control signal that accompanies
a video signal and synchronizes the switching of the waveform
for the left/right signal LR with the switching of the frame.
[0026]
The shutter glasses 104 include two liquid crystal
display panels 141L and 141R and a left/right signal receiving
unit 142. Each of the liquid crystal display panels 141L and
141R constitute each of the left and right lens parts. The
left/right signal receiving unit 142 receives a left/right
signal LR, and in accordance with the waveform, sends the signal
to the left and right liquid crystal display panels 141L and
141R. In accordance with the signal, each of the liquid crystal
display panels 141L and 141R either lets light pass through the
entire panel or shuts light out. For example, when the
left/right signal LR indicates a left-view display, the liquid
crystal display panel 141L for the left eye lets light pass
through, while the liquid crystal display panel 141R for the
right eye shuts light out. When the left/right signal LR
indicates a right-view display, the display panels act
oppositely. In this way, the two liquid crystal display panels
141L and 141R alternately let light pass through in sync with
the switching of frames. As a result, when a viewer looks at
the screen 131 while wearing the shutter glasses 104, the
left-view is shown only to the viewer's left eye, and the
right-view is shown only to the right eye. At that time, the
viewer is made to perceive the difference between the images
seen by each eye as the binocular parallax for a single
stereoscopic image, and thus the video appears to be
stereoscopic.
[0027]
The remote control 105 includes an operation unit and a
transmitting unit. The operation unit includes a plurality of
buttons. The buttons correspond to each of the functions of the
playback device 102 and the display device 103, such as turning
the power on or off, starting or stopping playback of the BD-ROM
disc 101, etc. The operation unit detects when the user presses
a button and passes a signal that specifies the button to the
transmitting unit. The transmitting unit sends this signal as
a signal IR via infrared rays or radio transmission to the
playback device 102 or the display device 103. In this way, the
user can remotely control the playback device 102 or the display
device 103.
[0028]
>
[0032]
UDF (Universal Disc Format) is used as the file system
for the volume area 202B. Alternatively, the file system may
be ISO9660. The data recorded on the volume area 202B is
represented in a directory/file format in accordance with the
file system. In other words, the data is accessible in units
of directories or files.
[0033]
More specifically, when a UDF is employed as the file
system, the volume area 2 02B includes a directory area, a
recording area for a file set descriptor, and a recording area
for a terminating descriptor. A "directory area" is ordinarily
a plurality of areas, each of which is a recording area for data
constituting a single directory. The "file set descriptor"
indicates a logical block number (LBN) of a sector that stores
the file entry of the root directory. The "terminating
descriptor" indicates the termination of the file set
descriptor.
[0034]
Each directory area shares a common data structure. In
particular, each directory area has a file entry, a directory
file, and recording areas for each subordinate file.
[0035]
The "file entry" includes a descriptor tag, an ICB tag,
and an allocation descriptor. The "descriptor tag" indicates
that the data that includes the descriptor tag is the file entry.
For example, when a descriptor tag has a value of "261," that
data is a file entry. The "ICB tag" indicates attribute
information of the file entry itself. The "allocation
descriptor" indicates the LBN of the sector on which the
directory file belonging to the same directory area is recorded.
[0036]
The "directory file" includes the file identifier
descriptor of a subordinate directory and the file identifier
descriptor of a subordinate file. The "file identifier
descriptor of a subordinate directory" is reference information
used for accessing the subordinate directory located
immediately below the directory recorded in the directory area.
In particular, this file identifier descriptor includes
identification information of the subordinate directory, the
length of the directory name, a file entry address, and the
actual directory name. Here, the file entry address indicates
the LBN of the sector on which the file entry of the subordinate
directory is recorded. The "file identifier descriptor of a
subordinate file" is reference information for accessing the
subordinate file located immediately below the directory
recorded in the directory area. This file identifier descriptor
includes identification information of the subordinate file,
the length of the file name, a file entry address, and the actual
file name. Here, the file entry address indicates the LBN of
the file entry of the subordinate file. By tracing the file
identifier descriptors of subordinate directories/files, the
file entries of the subordinate directories/files can be
sequentially found, starting from the file entry of the root
directory.
[0037]
The "subordinate file" includes the file entry and the
body of the subordinate file located immediately below the
directory recorded on the directory area. The "file entry"
includes a descriptor tag, an ICB tag, and allocation
descriptors. The "descriptor tag" indicates that the data that
includes the descriptor tag is a file entry. The "ICB tag"
indicates attribute information of the file entry itself. The
"allocation descriptors" indicate the arrangement of the
extents constituting the body of the subordinate file. Here,
"extent" refers to a data sequence in which logical addresses
are continuous. Each allocation descriptor is assigned to one
of the extents. Therefore, when the subordinate file is divided
into a plurality of extents, the file entry includes a plurality
of allocation descriptors. More specifically, each allocation
descriptor includes the size of an extent and an LBN.
Furthermore, the two most significant bits of each allocation
descriptor indicate whether an extent is actually recorded at
the sector for that LBN. More specifically, when the two most
significant bits indicate "0, " an extent has been allocated to
the sector and has been actually recorded thereat. When the two
most significant bits indicate "1, " an extent has been allocated
to the sector but has not been yet recorded thereat. The logical
addresses of the extents constituting each file can be found
by referencing the allocation descriptors of the file entry of
the file.
[0038]
Like the above-described file system employing a UDF,
when each file recorded on the volume area 2 02B is divided into
a plurality of extents, the file system for the volume area 202B
also generally stores the information showing the locations of
the extents, as with the above-mentioned allocation descriptors,
in the volume area 202B. By referencing the information, the
location of each extent, particularly the logical address
thereof, can be found.
[0039]
<>
[0040]
As further shown in Fig. 2, in the directory/file
structure 204 on the BD-ROM disc 101, a BD movie (BDMV) directory
242 is located immediately below a ROOT directory 241. Below
the BDMV directory 242 are an index file (index.bdmv) 242A and
a movie object file (MovieObject.bdmv) 242B.
[0041]
The index file 242A contains information for managing as
a whole the content recorded on the BD-ROM disc 101. In
particular, this information includes information to make the
playback device recognize the content, as well as an index table.
The index table specifies the correspondence between a title
constituting the content and a program to control the operation
of the playback device. In particular, this program is called
an "object. " Object types are a movie object and a BD-J (BD Java
(registered trademark)) object.
[0042]
Fig. 3 is a schematic diagram showing an index table 310
in the index file 242A. The index table 310 stores items such
as "first play" 3 01, "top menu" 3 02, and "title k" 3 03 (k = 1,
2, . . . , n; an integer n is equal to or greater than one) . Each
item is associated with either a movie object MVO-2D, MVO-3D,
etc., or with a BD-J object BDJO-2D, BDJO-3D, etc. Each time
a title or a menu is called in response to a user operation or
an application program, a control unit of the playback device
102 refers to a corresponding item in the index table 310, and
calls an object corresponding to the item from the disc 101.
The control unit then executes a variety of processes in
accordance with the called object. More specifically, the
"first play" 301 specifies an object to be called when the disc
101 is loaded into the BD-ROM drive 121. The "top menu" 302
specifies an object for displaying a menu on the display device
103 when a command "go back to menu" is input, for example, by
user operation. In the "title k" 303, the titles that constitute
the content on the disc 101 are individually allocated. For
example, when a title for playback is specified by user
operation, in the item "title k" in which the title is allocated,
the object for playing back a video from the AV stream file
corresponding to the title is specified.
[0043]
The movie object file 242B generally stores a plurality
of movie objects. Each movie object stores a sequence of
navigation commands. A navigation command is a control command
causing the playback device to execute playback processes
similarly to general DVD players. Types of navigation commands
are, for example, a read-out command to read out a playlist file
corresponding to a title, a playback command to play back stream
data from an AV stream file indicated by a playlist file, and
a transition command to make a transition to another title.
Navigation commands are written in an interpreted language and
are deciphered by an interpreter, i.e. a job control program,
included in the playback device to make the control unit execute
the desired job. A navigation command is composed of an opcode
and an operand. The opcode describes the operation that the
playback device is to execute, such as dividing, playing back,
or calculating a title, etc. The operand indicates
identification information targeted by the operation such as
the title's number, etc . The control unit of the playback device
102 calls a movie object in response, for example, to a user
operation and executes navigation commands included in the
called movie object in the order of the sequence. Thus, in a
manner similar to general DVD players, the playback device 102
first makes the display device 103 display a menu to allow the
user to select a command. The playback device 102 then executes
playback start/stop of a title, switches to another title, etc.
in accordance with the selected command, thereby dynamically
changing the progress of video playback.
[0044]
As shown in Fig. 2, the BDMV directory 242 further contains
a playlist (PLAYLIST) directory 243; a clip information
(CLIPINF) directory 244; a stream (STREAM) directory 245 ; a BD-J
object (BDJO: BD Java Object) directory 246; and a Java archive
(JAR: Java Archive) directory 247.
[0045]
Three types of AV stream files (00001.m2ts) 245A,
(00002.m2ts) 245B, and (00003.m2ts) 245C are located in the
STREAM directory 245. An "AV stream file" is the body of a video
content and represents video, audio, subtitles, etc. The types
of AV stream files are a 2D/AV stream file, a left-view stream
file, a right-view stream file, and a depth map AV stream file.
A "2D/AV stream file" refers to an AV stream file that can be
used alone for playback of 2D video images. A "left-view stream
file" refers to an AV stream file representing the left-view
of 3D video images. A "right-view stream file" refers to an AV
stream file representing the right-view of 3D video images. A
left-view stream file and a right-view stream file are used as
a pair to display 3D video images. A "depth map AV stream file"
refers to an AV stream file representing a depth map for 3D video
images. A depth map AV stream file is used in combination with
a 2D/AV stream file for playback of 3D video images. In the
example shown in Fig. 2, the first AV stream file (00001.m2ts)
245A is a 2D/AV stream file and is also a left-view stream file.
The second AV stream file (00002.m2ts) 245B is a right-view
stream file, and is used in combination with the first AV stream
file 245A for playback of 3D video images. The third AV stream
file (00003.m2ts) 245C is a depth map AV stream file, and is
used in combination with the first AV stream file 245A for
playback of 3D video images.
[0046]
Three types of clip information files, (00001.clpi) 244A,
(00002.clpi) 244B, and (00003.clpi) 244C are located in the
CLIPINF directory 244. A "clip information file" is associated
on a one-to-one basis with an AV stream file, and mainly
represents the correspondence between the presentation time of
the video shown by the AV stream file and the logical address
in the AV stream file. The types of clip information files are
a 2D clip information file, a left-view clip information file,
a right-view clip information file, and a depth map clip
information file, which are clip information files respectively
corresponding to a 2D/AV stream file, a left-view stream file,
a right-view stream file, and a depth map AV stream file. In
the example shown in Fig. 2, the first clip information file
(00001. clpi) 244A is a 2D clip information file and is also a
left-view clip information file corresponding to the first AV
stream file (00001.m2ts) 245A. The second clip information file
(00002. clpi) 244B is a right-view clip information file
corresponding to the second AV stream file (00002.m2ts) 245B.
The third clip information file (00003.clpi) 244C is a depth
map clip information file corresponding to the third AV stream
file (00003.m2ts) 245C.
[0047]
Three types of playlist files, (00001.mpls) 243A,
(00002.mpls) 243B, and (00003.mpls) 243C are located in the
PLAYLIST directory 243. A "playlist file" specifies the
playback path of an AV stream file, i.e. the part of an AV stream
file to decode, and the order of decoding. The types of playlist
files are a 2D playlist file and a 3D playlist file. A "2D
playlist file" refers to a playlist file specifying the playback
path of a 2D/AV stream file. A "3D playlist file" refers to a
playlist file that specifies (i) the playback path of a
combination of a left-view stream file and a right-view stream
file, or (ii) the playback path of a combination of a 2D/AV stream
file and a depth map AV stream file. The first playlist file
(00001.mpls) 243A is a 2D playlist file, and the second playlist
file (00002.mpls) 243B and the third playlist file (00003.mpls)
are both 3D playlist files. In particular, when 3D video images
are played back from the combination of a left-view stream file
and a right-view stream file, the second playlist file 243B is
used. On the other hand, when 3D video images are played back
from the combination of a 2D/AV stream file and a depth stream
file, the third playlist file 243C is used.
[0048]
A BD-J object file (XXXXX.bdjo) 246A is located in the
BDJO directory 24 6. The BD-J object file 246A includes a single
BD-J object. The BD-J object is a bytecode program to cause a
Java virtual machine mounted on the playback device 102 to
execute the processes of title playback and graphics rendering.
The BD-J object is written in a compiler language such as Java
or the like. The BD-J object includes an application management
table and identification information for the playlist file to
which is referred. The application management table is a list
of the Java application programs to be executed by the Java
virtual machine and their period of execution (lifecycle) . The
identification information of the playlist file to which is
referred identifies a playlist file that corresponds to a title
to be played back. The Java virtual machine calls a BD-J object
in accordance with a user operation or an application program,
and executes the Java application program according to the
application management table included in the BD-J object.
Consequently, the playback device 102 dynamically changes the
progress of the video for each title played back, or causes the
display device 103 to display graphics independently of the
title video.
[0049]
A JAR file (YYYYY.jar) 24 7A is located in the JAR directory
247. The JAR directory 247 generally includes a plurality of
bodies of Java application programs to be executed in accordance
with the application management table shown in the BD-J object.
A Java application program is a bytecode program written in a
compiler language such as Java or the like, as is the BD-J obj ect.
Types of Java application programs include programs causing the
Java virtual machine to execute playback of a title process and
programs causing the Java virtual machine to execute graphics
rendering. The JAR file 247A is a Java archive file, and when
it is read by the playback device 102, it is extracted in internal
memory. In this way, a Java application program is stored in
memory.
[0050]
More specifically, the directory file structure 204 is
implemented to have a ROOT directory area, a BDMV directory area,
a PLAYLIST directory area, a CLIPINF directory area, a STREAM
directory area, a BDJO directory area, and a JAR directory area
in the volume area 202B of the BD-ROM disc 101. By tracing the
above-mentioned file identifier descriptor, a series of file
entries in each directory can be found sequentially from the
ROOT directory. For example, the file entry of the ROOT
directory can lead to the file entry of the BDMV directory.
Furthermore, the file entry of the BDMV directory can lead to
the file entry of the PLAYLIST directory. Similarly, the file
entry of the BDMV directory can lead to the file entries of the
CLIPINF directory, the STREAM directory, the BDJO directory,
and the JAR directory areas.
[0051]

[0052]
The following describes the data structure of each file
associated with playback of 2D video content from among the
group of files in the BDMV directory 242.
[0053]
<<0bjects Recorded in the Index Table>>
[0054]
In the example in Fig. 3, the movie object associated with
the item "title 1, " MV0-2D, includes a group of commands related
to playback processes for 2D video images using the first
playlist file 243A. When the playback device 102 refers to the
item "title 1, " then in accordance with the movie object MV0-2D,
the first playlist file 243A is read from the disc 101, and
playback processes for 2D video images are executed in
accordance with the specifications therein.
[0055]
In the example in Fig. 3, the BD-J object associated with
the item "title 2, " BDJO-2D, includes an application management
table related to playback processes for 2D video images using
the first playlist file 243A. When the playback device 102
refers to the item "title 2," then in accordance with the
application management table in the BD-J object BDJO-2D, a Java
application program is called from the JAR file 247A and
executed. In this way, the first playlist file 243A is read from
the disc 101, and playback processes for 2D video images are
executed in accordance with the specifications therein.
[0056]
<<2D/AV Stream File>>
[0057]
The first AV stream file 245A is a digital stream in MPEG-2
transport stream (TS) format. A plurality of elementary streams
are multiplexed in the first AV stream file 245A. Fig. 4 is a
schematic diagram showing elementary streams multiplexed in the
first AV stream file 245A. The first AV stream file 245A shown
in Fig. 4 includes a primary video stream 4 01, primary audio
streams 4 02A and 402B, presentation graphics (PG) streams 4 03A
and 4 03B, an interactive graphics (IG) stream 4 04, secondary
video streams 405A and 405B, and a secondary audio stream 406.
[0058]
The primary video stream 4 01 represents the primary video
of a movie, and the secondary video streams 4 05A and 4 05B
represent secondary video of the movie. The primary video is
the major video of a content, such as the main feature of a movie,
and is displayed on the entire screen, for example. On the other
hand, the secondary video is displayed simultaneously with the
primary video with the use, for example, of a picture-in-picture
method, so that the secondary video images are displayed in a
smaller window presented on the full screen displaying the
primary video image. Each video stream is encoded by a video
compression encoding method, such as MPEG-2, MPEG-4 AVC, or
SMPTE VC-1.
[0059]
The primary audio streams 402A and 402B represent the
primary audio of the movie. Here, the two primary audio streams
402A and 402B are in different languages. The secondary audio
stream 4 06 represents secondary audio to be mixed with the
primary audio. Each audio stream is encoded by a method such
as AC-3, Dolby Digital Plus ("Dolby Digital" is a registered
trademark), MLP, DTS (Digital Theater System: registered
trademark), DTS-HD, or linear PCM (Pulse Code Modulation) .
[0060]
The PG streams 4 03A and 4 03B represent subtitles of the
movie. Here, the PG streams 403A and 403B represent subtitles
in a different language. The IG stream 4 04 represents graphical
user interface (GUI) graphics components, and the arrangement
thereof, for constructing an interactive screen on the screen
131 in the display device 103.
[0061]
The elementary streams 401-406 are identified by packet
IDs (PIDs). For example, the primary video stream 401 is
assigned a hexadecimal value of 0x1011 as a PID. The primary
audio streams 4 02A and 4 02B are each assigned any value from
0x1100 to OxlllF as PIDs. The PG streams 403A and 403B are each
assigned any value from 0x1200 to 0xl21F as PIDs. The IG stream
404 is assigned any value from 0x1400 to 0xl41F as a PID. The
secondary video streams 4 05A and 4 05B are each assigned any
value from 0x1B00 to 0x1B1F as PIDs. The secondary audio stream
406 is assigned any value from 0x1AOO to 0x1A1F as a PID.
[0062]
Fig. 5 is a schematic diagram showing the arrangement of
packets in each of the elementary streams 501, 504, 507, 510
multiplexed in the 2D/AV stream file 513. First, a video stream
501 having a plurality of video frames 501 is converted to a
series of PES packets 502 . Then, each PES packet 502 is converted
to a TS packet 503 . Similarly, an audio stream having a plurality
of audio frames 5 04 is converted into a series of PES packets
505, after which this series is converted into a series of TS
packets 506. Similarly, the PG stream 507 and the IG stream 510
are converted into series of PES packets 508 and 511 and
afterwards are converted into series of TS packets 5 09 and 512,
respectively. Lastly, these TS packets 503, 506, 509, and 512
are time-multiplexed into one stream 513 . In this way, the 2D/AV
AV stream file 513 is constructed.
[0063]
Using a video stream as an example, details regarding
packetization of the elementary streams shown in Fig. 5 are
provided below.
[0064]
Fig. 6 is a schematic diagram showing the pictures
611-614, ..., 621-624 included in the video stream 600 in the
time order in which they are displayed. Each picture
611-614, . . . , 621-624 represents one frame or one field of video
data. In the video stream 600, each picture 611-614, ...,
621-624 is compressed with one of the above-mentioned video
compression encoding methods such as MPEG-2.
[0065]
Compression of each picture by the above-mentioned
encoding method uses the picture's spatial or temporal
redundancy. Here, picture encoding that only uses the picture's
spatial redundancy is referred to as "intra-picture encoding."
On the other hand, picture encoding that uses the similarity
between data for multiple pictures displayed sequentially is
referred to as "inter-picture predictive encoding." In
inter-picture predictive encoding, first, a picture earlier or
later in presentation time is assigned to the picture to be
encoded as a reference picture. Next, a motion vector is
detected between the picture to be encoded and the reference
picture, and then motion compensation is performed using the
motion vector. Furthermore, the difference value between the
picture after motion compensation and the picture to be encoded
is sought, and temporal redundancy is removed using the
difference value. In this way, the amount of data for each
picture is compressed.
[0066]
As shown in Fig. 6, the pictures included in the video
stream 600 are generally divided into a plurality of GOPs 610
and 620. Here, a "GOP" refers to a sequence of pictures starting
with an I picture. An "I Picture" refers to a picture compressed
by intra-picture encoding. A GOP generally has a P picture and
a B picture as well as an I picture. A "P picture" refers to
a picture compressed by inter-picture predictive encoding,
having used as a reference picture either an I picture earlier
in presentation time or a different P picture. A "B picture"
refers to a picture compressed by inter-picture predictive
encoding, having used two reference pictures that are I or P
pictures earlier or later in presentation time.
[0067]
In the example in Fig. 6, the pictures in the GOPs 610
and 620 are compressed in the following order. First, the first
pictures in GOP 610 and 620 are compressed as I0 picture 611
and I7 picture 621 respectively. Here, the subscripted number
indicates the sequential number allotted to each picture in the
order of presentation time. Next, the fourth pictures in GOP
610 and 620 are compressed as P3 picture 614 and P10 picture 624
respectively, using I0 picture 611 and I7 picture 621 in the
same GOP respectively as a reference picture, as indicated by
the arrows in Fig. 6. Then the second and third pictures in the
first GOP 610 are compressed as B1 picture 612 and B2 picture
613 respectively, using the earlier I0 picture 611 and the later
P3 picture 614 as reference pictures. Similarly, the second and
third pictures in the second GOP 620 are compressed as B8 picture
622 and B9 picture 623 respectively, using the earlier I7 picture
621 and the later Pi0 picture 624 as reference pictures. When
subsequent pictures after the P3 picture 614 exist, as in the
first GOP 610, these pictures are similarly compressed.
[0068]
In this way, as each GOP always contains an I picture at
the top, the pictures in each GOP can be decoded. For example,
in the first GOP 610, the I0 picture 611 is first decoded alone.
Next, the P3 picture 614 is decoded using the decoded I0 picture
611. Then the B1 picture 612 and B2 picture 613 are decoded using
the decoded I0 picture 611 and P3 picture 614. In this way, the
video stream 600 can be randomly accessed in units of GOPs.
Therefore, editing or playback of the video stream 600 can be
performed in units of GOPs.
[0069]
Fig. 7 is a schematic diagram showing details of the data
structure of the video stream 70 0 . As shown in Fig. 7, inaddition
to the bodies of the GOPs 610 and 620 shown in Fig. 6, a header
is assigned to each picture 611, 612, etc. in the GOPs 710 and
720, which constitute the actual video stream 700. This
combination of a header and the body of each picture is called
a "video access unit." In other words, in the GOPs 710 and 720,
each picture is organized as a video access unit 711, 712, etc.
Each picture can be read from the video stream 7 00 in video access
units.
[0070]
As shown in Fig. 7, the top video access unit 711 in the
GOPs 710 and 720 includes the body of the I picture 611 and a
header. The body of the I picture 611 is stored as compressed
picture data 711E. The header includes an access unit (AU)
identification code 711A, a sequence header 711B, a picture
header 711C, and supplementary data 711D. The AU identification
code 711A is a predetermined code indicating the top of each
picture access unit. The sequence header 711B, also called a
GOP header, includes information shared by the whole GOP 710,
e.g. the resolution, frame rate, aspect ratio, and bit rate.
The picture header 711C includes information necessary for
decoding of the I picture 611, such as the encoding method, etc.
The supplementary data 711D includes additional information
regarding questions other than the decoding of the I picture
611, for example closed caption text information, as well as
time code information. The second or subsequent video access
unit 712 in the GOPs 710 and 72 0 has the same structure as the
first video access unit 711, with the exception that the header
does not include a sequence header 711B. The actual content of
each element in the video access units 711, 712 varies according
to the encoding method for the video stream 700. For example,
when the encoding method is MPEG-4 AVC, the video access unit
711 is comprised of a plurality of NAL units, and the AU
identification code 711A, sequence header 711B, picture header
711C, and supplementary data 711D respectively correspond to
an AU delimiter (Access Unit Delimiter), SPS (sequence
parameter set), PPS (picture parameter set), and SEI
(Supplemental Enhancement Information).
[0071]
Fig. 8 is a schematic diagram showing details of a method
for storing the video stream 801 into PES packets 802. Here,
in the actual video stream, pictures are multiplexed in the
order of encoding, not in the order of presentation time. In
other words, as shown in Fig. 8, in each video access unit
comprising the video stream 801, I picture yyl, P picture yy2,
B pictures yy3, yy4, etc. are stored in order from the top. The
I picture yyl is used as a reference picture for encoding the
P picture yy2, and the I picture yyl and the P picture yy2 are
used as reference pictures for encoding the B pictures yy3 and
yy4 . Each of these video access units is stored as a difference
PES packet 802. As shown in Fig. 8, each PES packet 802 includes
a PES payload 8 02P and a PES header 802H. Each video access unit
is stored in a PES payload 802P, whereas each PES header 802H
includes a presentation time for the picture stored in the PES
payload 802P in the same PES packet 802, i.e., a PTS
(presentation time-stamp) , and a decoding time for the picture,
i.e. a DTS (decoding time-stamp).
[0072]
As with the video stream 801 shown in Fig. 8, the other
elementary streams shown in Fig. 5 are stored in PES payloads
in a sequence of PES packets. Furthermore, the PES header in
each PES packet includes the PTS for the data stored in the PES
payload for the PES packet.
[0073]
Figs. 9A, 9B, and 9C schematically show the format of a
TS packet 901 and a source packet 902 constituting the 2D/AV
stream file 513. The TS packet 901 is 188 bytes long. As shown
in Fig. 9A, the TS packet 901 is composed of a 4-byte long TS
header 901H and a 184-byte long TS payload 901P. The PES packets
in which each of the elementary streams shown in Fig. 5 are stored
are typically divided into a plurality of sections, with each
section being stored in the TS payload 901P of a different TS
packet 901. Each TS header 9 01H includes a PID. The PID
identifies an elementary stream having data stored in the PES
payload 802P when the PES packet 802 is reconstructed from data
stored in the TS payload 901P of the same TS packet 901. When
the 2D/AV stream file 513 is written on the BD-ROM disc 101,
as shown in Fig. 9B, a 4-byte long header (TP_Extra_Header) 902H
is further added to each TS packet 901. The header 902H
particularly includes an ATS (Arrival_Time_Stamp). The ATS
shows the transfer start time at which the TS packet 901 is to
be transferred to a PID filter inside a system target decoder,
which is described later. The 192-byte long packet 902 formed
by combining this header 902H with a TS packet 901 is called
a "source packet." This source packet 902 is written in the 2D/AV
stream file 513. Consequently, as shown in Fig. 9C, the
plurality of source packets 902 is sequentially arranged in the
2D/AV stream file 513. The source packets 902 are assigned
serial numbers from the top of the 2D/AV stream file 513 in the
order 0, 1, 2, etc. The serial numbers are called SPNs (source
packet numbers).
[0074]
The types of the TS packets contained in the AV stream
file include not only those that are converted from the
elementary streams shown in Figs . 4 and 5, but also a PAT (Program
Association Table) , a PMT (Program Map Table) , and a PCR
(Program Clock Reference) . The PCR, PMT, and PAT are specified
by European Digital Broadcasting Standard and are intended to
regulate the partial transport stream constituting a single
program. By using PCR, PMT, and PAT, the AV stream file can be
regulated in the same way as the partial transport stream.
Concretely, the PAT shows the PID of a PMT included in the same
AV stream file. The PID of the PAT itself is 0. The PMT includes
the PIDs for the elementary streams representing video, audio,
subtitles, etc. included in the same AV stream file, as well
as the attribute information of the elementary streams. The PMT
also includes various descriptors relating to the AV stream file.
The descriptors particularly include copy control information
showing whether copying of the AV stream file is permitted or
not. The PCR stores information indicating the value of an STC
(System Time Clock) to be associated with an ATS of the packet.
Here, the STC is a clock used as a reference for the PTS and
the DTS in a decoder. With the use of PCR, the decoder
synchronizes the STC with the ATC that is the reference for the
ATS.
[0075]
Fig. 10 is a schematic diagram showing the data structure
of the PMT 1010. The PMT 1010 includes, from the top thereof,
a PMT header 1001, a plurality of descriptors 1002, and a
plurality of pieces of stream information 1003. The PMT header
1001 indicates the length of data, etc. stored in the PMT 1010.
Each descriptor 1002 relates to the entire AV stream file that
includes the PMT 1010. The aforementioned copy control
information is described in one of the descriptors 1002. Each
piece of stream information 1003 relates to a different one of
the elementary streams included in the AV stream file, and each
piece of stream information 1003 is assigned to a different
elementary stream. Each piece of stream information 1003
includes a stream type 1003A, a PID 1003B, and a stream
descriptor 1003C. The stream type 1003A includes identification
information of the codec used for compressing the elementary
stream. The PID 1003B indicates the PID of the elementary stream.
The stream descriptor 1003C includes attribute information of
the elementary stream, such as a frame rate and an aspect ratio.
[0076]
By using PCR, PMT, and PAT, the decoder in the playback
device can be made to process the AV stream file in the same
way as the partial transport stream in the European Digital
Broadcasting Standard. In this way, it is possible to ensure
compatibility between a playback device for the BD-ROM disc 101
and a terminal device conforming to the European Digital
Broadcasting Standard.
[0077]
<<2D Clip Information File>>
[0078]
Fig. 11 is a schematic diagram showing the data structure
of the section of the first clip information file (00001.clip)
244A that is used as a 2D clip information file. A 2D clip
information file generally has this data structure. As shown
in Fig. 11, the first clip information file 244A includes clip
information 1101, stream attribute information 1102, and an
entry map 1103.
[0079]
As shown in Fig. 11, the clip information 1101 includes
a system rate 1101A, a playback start time 1101B, and a playback
end time 1101C. The system rate 1101A indicates the maximum
value of the transfer speed at which "TS packets" in the first
AV stream file 245A are transferred from the read buffer in the
playback device 102 to the system target decoder. Note that the
read buffer and the system target decoder are described later.
The interval between the ATSs of the source packets in the first
AV stream file 245A is set so that the transfer speed of the
TS packet is limited to the system rate or lower. The playback
start time 1101B indicates the PTS of the video access unit
located at the top of the first AV stream file 245A. For instance,
the playback start time 110IB shows the PTS of the top video
frame. The playback end time 1101C shows the value of the STC
delayed a predetermined time from the PTS of the video access
unit located at the end of the first AV stream file 245A. For
instance, the playback end time 1101C shows the sum of the PTS
of the last video frame and the playback time of one frame.
[0080]
Fig. 12 is a schematic diagram showing the data structure
of the stream attribute information 1102. As shown in Fig. 12,
the stream attribute information 1102 is a table associating
the PIDs 1201 for each elementary stream included in the first
AV stream file 245A with pieces of attribute information 1202.
Here, each piece of attribute information 1202 is different
depending on whether it corresponds to a video stream, an audio
stream, a PG stream, or an IG stream. For example, the piece
of attribute information 1210 corresponding to the PID 0x1011
for the primary video stream includes a codec type 1211 used
for the compression of the video stream, as well as a resolution
1212, an aspect ratio 1213 , and a frame rate 1224 for the pictures
constituting the video stream. On the other hand, the piece of
attribute information 1220 corresponding to the PID 0x1101 for
the primary audio stream includes a codec type 1221 used for
compressing the audio stream, a number of channels 1222 included
in the audio stream, a language 1223, and a sampling frequency
1224. The playback device 102 uses these pieces of attribute
information 1202 to initialize the decoder.
[0081]
Fig. 13A is a schematic diagram showing the data structure
of the entry map 1103. As shown in Fig. 13A, the entry map 1103
includes tables 13 00 which are assigned to video streams in the
first AV stream file 245A. Each table 1300 corresponds to the
PID for the assigned video stream. Each table 1300 includes an
entry map header 1301 and a plurality of entry points 1302. The
entry map header 1301 includes the PID corresponding to the
table 1300 and the total number of entry points 13 02 included
in the table 1300. The entry point 1302 shows a pair of PTS 1303
and SPN 1304 in correspondence with a different entry map ID
(EP_ID) 1305. The PTS 1303 is equivalent to the PTS for one of
the I pictures included in the video stream associated with each
section 1300. The SPN 1304 is equivalent to the SPN for the top
of the source packet group stored in the corresponding I
picture.
[0082]
Fig. 13B schematically shows, out of the source packets
included in the first AV stream file 245A, source packets whose
correspondence with the EP_IDs 1305 are shown by the entry map
1103 . With reference to the entry map 1103, the playback device
102 can specify the SPN within the first AV stream file 245A
corresponding to a scene at an arbitrary point during the
playback of the video from the video stream. For example, to
execute trickplay such as fast-forward or rewind, the playback
device 102 specifies source packets of the SPNs 1304
corresponding to the EP_IDs 1305 by using the entry map 1103
and selectively extracts and decodes the source packets. As a
result, the I picture can be selectively played back. Thus, the
playback device 102 can efficiently perform trickplay without
analyzing the first AV stream file 245A.
[0083]
<<2D Playlist File>>
[0084]
Fig. 14 is a schematic diagram showing the data structure
of a 2D playlist file 1400. The first playlist file 243A
indicated in Fig. 2 has this data structure. The 2D play list
file 1400 specifies the playback path of the 2D/AV stream file
14 04, i.e. the sections P1, P2, and P3 of the 2D/AV stream file
14 04 which are to be decoded, as well as the order of decoding
thereof. The 2D playlist file 1400 particularly specifies with
PTSs a range for each section P1, P2, and P3 . The specified PTS
are converted to SPNs of the AV stream file 1404 using the clip
information file 14 03 corresponding to the 2D/AV stream file
14 04. As a result, the range of each section P1, P2, and P3 is
specified with SPNs. Note that, unlike Fig. 14, any of the
sections P1, P2, and P3 can be a section of a different AV stream
file than the 2D/AV stream file 1404.
[0085]
As shown in Fig. 14, the 2D playlist file 1400 includes
at least one piece of playitem (PI) information 1401. Each piece
of playitem information 14 01 is identified by a unique playitem
ID. Each piece of playitem information 1401 specifies a
different playback section in the playback path via a pair of
PTSs. For example, the first piece of playback item information
PI#1 specifies the first playback section via a pair of PTSs
T1, representing the start time, and T2, representing the end
time. The start time T1 and the end time T2 are converted into
SPNs A1 and A2 in the 2D/AV stream file 1404 by using the 2D
clip information file 1403 . As a result, the range of the section
P1 that is to be decoded in the playback section is specified
by the SPNs A1 and A2 at either end. Similar to the PI#l-3 shown
in Fig. 14, when a plurality of pieces of playitem information
1401 are recorded in the 2D playlist file 1400, the order of
the recording is indicated by the order of the corresponding
playback sections in the playback path. Here, the playback path
of a series that links the playback sections specified by the
pieces of playitem information 14 01 is referred to as a "main
path." Furthermore, a series of pieces of playitem information
PI#l-3 that specify a main path are also referred to as the "main
path" 14 05.
[0086]
The 2D playlist file 1400 further includes an entry mark
1402. The entry mark 1402 indicates a time point in the main
path 1405 to be actually played back. The entry mark 1402 can
be assigned to a playback section defined by the playitem
information 14 01. For example, as shown in Fig. 14, a plurality
of entry marks 1402 can be assigned to one piece of playitem
information PI #1. The entry mark 1402 is particularly used for
searching for a start position of playback when random access
is made. When the 2D playlist file 1400 defines a playback path
for a movie title, for instance, the entry marks 14 02 may be
assigned to the top of each chapter. Consequently, the playback
device 102 can play back a movie title starting from any of the
chapters.
[0087]
Fig. 15 is a schematic diagram showing the data structure
of playitem information 1500. As shown in Fig. 15, the playitem
information 1500 includes reference clip information 1501, a
playback start time 1502, a playback end time 1503, a connection
condition 1504 , and a stream selection table 1505 . The reference
clip information 1501 identifies a clip information file that
is necessary for converting PTSs to SPNs. The playback start
time 1502 and the playback end time 1503 respectively indicate
the PTSs of the top and the end of the section of the AV stream
file to be decoded. The playback device 102 refers to the entry
map from the clip information file indicated by the reference
clip information 1501, and obtains SPNs respectively
corresponding to the playback start time 1502 and the playback
end time 1503. Thus, the playback device 102 identifies the
section that is to be read from the AV stream file and starts
to play back the section.
[0088]
The connection condition 1504 specifies a condition for
connecting video in the playback section specified by a playback
start time 1502 and a playback end time 1503 to video in the
playback section specified by the previous piece of playitem
information in the main path. The connection condition 1504 has
three types, for example "1, " "5, " and "6." When the connection
condition 1504 indicates "1, " the video to be played back from
the section of the AV stream file specified by the piece of
playitem information does not need to be seamlessly connected
to the video played back from the section of the AV stream file
specified by the previous piece of playitem information. On the
other hand, when the connection condition 1504 indicates "5"
or "6," both videos need to be seamlessly connected.
[0089]
Figs. 16A and 16B each schematically show the
relationships between playback sections specified by each piece
of playitem information to be connected when the connection
condition 1504 indicates "5" or "6." When the connection
condition 1504 indicates "5," as shown in Fig. 16A, the STCs
of two pieces of playitem information PI#1 and PI#2 may be
nonconsecutive. That is, the PTS TE at the end of the first AV
stream file section 1601 defined by the first piece of playitem
information PI#1, in a previous position in the main path, and
the PTS TS at the top of the second AV stream file section 1601
defined by the second piece of playitem information PI#2, in
a later position, may be nonconsecutive. Note that, in this case,
several constraint conditions need to be satisfied. For example,
when the second section 1602 is supplied to a decoder
subsequently after the first section 1601, the sections 1601
and 1602 need to be created so that the decoder can smoothly
decode the file. Furthermore, the last frame of the audio stream
contained in the first section 1601 needs to overlap the top
frame of the audio stream contained in the second section 1602.
On the other hand, when the connection condition 1504 indicates
"6, " as shown in Fig. 16B, the first section 1601 and the second
section 1602 need to be able to be handled as a series of sections,
in order to allow the decoder to duly perform the decode
processing. That is, STCs and ATCs need to be consecutive
between the first section 1601 and the second section 1602.
[0090]
Referring to Fig. 15 again, the stream selection table
1505 shows a list of elementary streams that the decoder in the
playback device 102 can select from the AV stream file during
the time between the playback start time 1502 and the playback
end time 1503. The stream selection table 1505 particularly
includes a plurality of stream entries 1509. Each of the stream
entries 1509 includes a stream selection number 1506, stream
path information 1507, and stream identification information
1508 of a corresponding elementary stream. The stream selection
numbers 1506 are serial numbers assigned to the stream entries
1509, and used by the playback device 102 to identify the
elementary streams. Each piece of stream path information 1507
shows an AV stream file to which an elementary stream to be
selected belongs. For example, if the stream path information
1507 shows "main path, " the AV stream file corresponds to the
clip information file indicated by the reference clip
information 1501. If the stream path information 1507 shows
"sub-path ID = 1, " the AV stream file to which the elementary-
stream to be selected belongs is an AV stream specified by a
piece of sub-playitem information included in the sub-path
whose sub-path ID = 1. Here, the sub-playitem information piece
defines a playback section that falls between the playback start
time 1502 and the playback end time 1503 . Note that the sub-path
and the sub-playitem information are described in the next
section of this specification. Each piece of stream
identification information 1508 indicates the PID of a
corresponding one of the elementary streams multiplexed in an
AV stream file specified by the stream path information 1507.
The elementary streams indicated by the PIDs are selectable
during the time between the playback start time 1502 and the
playback end time 1503. Although not shown in Fig. 15, each
stream entry 1509 also contains attribute information of a
corresponding elementary stream. For example, the attribute
information of an audio stream, a PG stream, and an IG stream
indicates a language type of the stream.
[0091]
Fig. 17 is a schematic diagram showing the data structure
of a 2D playlist file 17 00 when the playback path to be defined
includes sub-paths. As shown in Fig. 17, the 2D playlist file
1700 may include one or more sub-paths in addition to the main
path 1701. Sub-paths 1702 and 1703 are each a playback path
parallel to the main path 1701. The serial numbers are assigned
to the sub-paths 17 02 and 1703 in the order they are registered
in the 2D playlist file 1700. The serial numbers are each used
as a sub-path ID for identifying the sub-path. Similarly to the
main path 1701 that is a playback path of a series of playback
sections specified by the pieces of playitem information #1-3,
each of the sub-paths 1702 and 17 03 is a playback path of a series
of playback sections specified by sub-playitem information #1-3 .
The data structure of the sub-playitem information 1702A is
identical with the data structure of the playitem information
1500 shown in Fig. 15. That is, each piece of sub-playitem
information 1702A includes reference clip information, a
playback start time, and a playback end time. The playback start
time and the playback end time of the sub-playitem information
are expressed on the same time axis as the playback time of the
main path 1701. For example, in the stream entry 1509 included
in the stream selection table 1505 of the playitem information
#2, assume that the stream path information 1507 indicates
"sub-path ID = 0," and the stream identification information
1508 indicates the PG stream #1. Then, in the sub-path 1702 with
sub-path ID = 0, for the playback section of the playitem
information #2, the PG stream #1 is selected as the decode target
from an AV stream file corresponding to the clip information
file shown by the reference clip information of the sub-playitem
information #2.
[0092]
Furthermore, the sub-playitem information includes a
field called an SP connection condition. The SP connection
condition carries the same meaning as a connection condition
of the playitem information. That is, when the SP connection
condition indicates "5" or "6," each AV stream file section
specified by two adjacent pieces of sub-playitem information
also needs to satisfy the above-described conditions.
[0093]

[0094]
Next, the configuration for the playback device 102 to
play back the above-described 2D video content from the BD-ROM
disc 101, i.e., the configuration of a 2D playback device, is
described below.
[0095]
Fig. 18 is a functional block diagram showing a 2D playback
device 1800. As shown in Fig. 18, the 2D playback device 1800
has a BD-ROM drive 18 01, a playback unit 1800A, and a control
unit 1800B. The playback unit 1800A has a read buffer 1802, a
system target decoder 1803, and a plane adder 1810. The control
unit 1800B has a dynamic scenario memory 1804, a static scenario
memory 1805, a program execution unit 1806, a playback control
unit 1807, a player variable storage unit 1808, and a user event
processing unit 1809. The playback unit 1800A and the control
unit 1800B are each implemented on a different integrated
circuit. Alternatively, the playback unit 1800A and the control
unit 1800B may also be implemented on a single integrated
circuit.
[0096]
When the BD-ROM disc 101 is loaded into the BD-ROM drive
1801, the BD-ROM drive 1801 radiates laser light to the disc
101, and detects change in the light reflected from the disc
101. Furthermore, using the change in the amount of reflected
light, the BD-ROM drive 1801 reads data recorded on the disc
101. Concretely, the BD-ROM drive 1801 has an optical pickup,
i.e. an optical head. The optical head has a semiconductor laser,
a collimate lens, a beam splitter, an objective lens, a
collecting lens, and an optical detector. A beam of light
radiated from the semiconductor laser sequentially passes
through the collimate lens, the beam splitter, and the objective
lens to be collected on a recording layer of the BD-ROM disc
101. The collected beam is reflected and diffracted by the
recording layer. The reflected and diffracted light passes the
objective lens, the beam splitter, and the collecting lens, and
is collected onto the optical detector. The optical detector
generates a playback signal at a level in accordance with the
amount of collected light. Furthermore, data is decoded from
the playback signal.
[0097]
The BD-ROM drive 1801 reads data from the BD-ROM disc 101
based on a request from the playback control unit 1807. Out of
the read data, an AV stream file is transferred to the read buffer
1802; dynamic scenario information such as an index file, movie
object file, and BD-J object file is transferred to the dynamic
scenario memory 1804 ; and a playlist file and a clip information
file, i.e. static scenario information, are transferred to the
static scenario memory 1805.
[0098]
The read buffer 1802, the dynamic scenario memory 1804,
and the static scenario memory 1805 are each a buffer memory.
A memory device in the playback unit 1800A is used as the read
buffer 1802. Memory devices in the control unit 1800B are used
as the dynamic scenario memory 1804 and the static scenario
memory 1805. In addition, different areas in a single memory
device maybe used as these buff er memories 1802, 1804 and 1805.
The read buffer 1802 stores an AV stream file, the dynamic
scenario memory 1804 stores dynamic scenario information, and
the static scenario memory 1805 stores static scenario
information.
[0099]
The system target decoder 1803 reads an AV stream file
from the read buffer 1802 in units of source packets and
demultiplexes the AV stream file. The system target decoder 1803
then decodes each of elementary streams obtained by the
demultiplexing. Here, information necessary for decoding each
elementary stream, such as the type of codec and attribute of
the stream, is transferred from the playback control unit 1807
to the system target decoder 1803. The system target decoder
1803 outputs a primary video stream, a secondary video stream,
an IG stream, and a PG stream that have been decoded in video
access units. The output data are used as primary video plane
data, secondary video plane data, IG plane data, and PG plane
data, respectively. On the other hand, the system target decoder
1803 mixes the decoded primary audio stream and secondary audio
stream and sends the resultant data to an audio output device,
such as an internal speaker 103A of a display device. In addition,
the system target decoder 1803 receives graphics data from the
program execution unit 1806. The graphics data is used for
rendering graphics such as a GUI menu on a screen, and is in
a raster data format such as JPEG and PNG. The system target
decoder 1803 processes the graphics data and outputs the data
as image plane data. Details of the system target decoder 1803
will be described below.
[0100]
The user event processing unit 1809 detects a user
operation via the remote control 105 or the front panel of the
playback device 102 . Based on the user operation, the user event
processing unit 1809 requests the program execution unit 1806
or the playback control unit 1807 to perform a relevant process.
For example, when a user instructs to display a pop-up menu by
pushing a button on the remote control 105, the user event
processing unit 1809 detects the push and identifies the button.
The user event processing unit 1809 further requests the program
execution unit 1806 to execute a command corresponding to the
button, which is a command to display the pop-up menu. When a
user pushes a fast-forward or a rewind button on the remote
control 105, for example, the user event processing unit 1809
detects the push and identifies the button. In addition, the
user event processing unit 1809 requests the playback control
unit 1807 to fast-forward or rewind the playlist currently being
played back.
[0101]
The playback control unit 1807 controls transfer of files,
such as an AV stream file and an index file, from the BD-ROM
disc 101 to the read buffer 1802, the dynamic scenario memory
1804, and the static scenario memory 1805. A file system
managing the directory file structure 204 shown in Fig. 2 is
used for this control. That is, the playback control unit 1807
causes the BD-ROM drive 1801 to transfer the files to each of
the buffer memories 1802, 1804 and 1805 using a system call for
opening files. The file opening is composed of a series of the
following processes. First, a file name to be detected is
provided to the file system by a system call, and an attempt
is made to detect the file name from the directory/file
structure 204. When the detection is successful, the file entry
for the target file is first transferred to memory in the
playback control unit 1807, and an FCB (File Control Block) is
generated in the memory. Subsequently, a file handle for the
target file is returned from the file system to the playback
control unit 1807. After this, the playback control unit 1807
can transfer the target file from the BD-ROM disc 101 to each
of the buffer memories 1802, 18 04 and 1805 by showing the file
handle to the BD-ROM drive 1801.
[0102]
The playback control unit 1807 decodes the AV stream file
to output video data and audio data by controlling the BD-ROM
drive 1801 and the system target decoder 1803 . More specifically,
the playback control unit 1807 first reads a playlist file from
the static scenario memory 18 05, in response to an instruction
from the program execution unit 1806 or a request from the user
event processing unit 1809, and interprets the content of the
file. In accordance with the interpreted content, particularly
with the playback path, the playback control unit 1807 then
specifies an AV stream file to be played back and instructs the
BD-ROM drive 1801 and the system target decoder 1803 to read
and decode this file. Such playback processing based on a
playlist file is called "playlist playback." In addition, the
playback control unit 1807 sets various types of player
variables in the player variable storage unit 18 08 using the
static scenario information. With reference to the player
variables, the playback control unit 1807 further specifies to
the system target decoder 1803 elementary streams to be decoded
and provides - the information necessary for decoding the
elementary streams.
[0103]
The player variable storage unit 18 08 is composed of a
group of registers for storing player variables. Types of player
variables include system parameters (SPRM) and general
parameters (GPRM). Fig. 19 is a list of SPRMs. Each SPRM is
assigned a serial number 1901, and each serial number 1901 is
associated with a unique variable value 1802. The contents of
major SPRM are shown below. Here, the numbers in parentheses
indicate the serial numbers 1901.
[0104]
SPRM(0) :Language code
SPRM(l) :Primary audio stream number
SPRM(2) :Subtitle stream number
SPRM(3) :Angle number
SPRM(4) :Title number
SPRM(5) :Chapter number
SPRM(6) :Program number
SPRM(7) :Cell number
SPRM(8) :Selected key name
SPRM(9) :Navigation timer
SPRM(10) :Current playback time
SPRM(11) :Player audio mixing mode for Karaoke
SPRM(12) :Country code for parental management
SPRM(13) :Parental level
SPRM(14) :Player configuration for video
SPRM(15) :Player configuration for audio
SPRM(16) :Language code for audio stream
SPRM(17) :Language code extension for audio stream
SPRM(18) :Language code for subtitle stream
SPRM (19) : Language code extension for subtitle stream
SPRM(20) :Player region code
SPRM(21) :Secondary video stream number
SPRM(22) :Secondary audio stream number
SPRM(23) :Player status
SPRM(24) :Reserved
SPRM(25) :Reserved
SPRM(26) :Reserved
SPRM(27) :Reserved
SPRM(28) :Reserved
SPRM(29) :Reserved
SPRM(30) :Reserved
SPRM(31) :Reserved
[0105]
The SPRM(IO) indicates the PTS of the picture currently
being decoded and is updated every time a picture is decoded
and written into the primary video plane memory. Accordingly,
the current playback point can be known by referring to the
SPRM(IO).
[0106]
The language code for the audio stream of the SPRM(16)
and the language code for the subtitle stream of the SPRM(18)
show default language codes of the playback device 102. These
codes may be changed by a user with use of the OSD (On Screen
Display) or the like of the playback device 102, or may be changed
by an application program via the program execution unit 1806.
For example, if the SPRM(16) shows "English," in playback
processing of a playlist, the playback control unit 1807 first
searches the stream selection table in the playitem information
for a stream entry having the language code for "English. " The
playback control unit 1807 then extracts the PID from the stream
identification information of the stream entry and transmits
the extracted PID to the system target decoder 1803 . As a result,
an audio stream having the same PID is selected and decoded by
the system target decoder 1803. These processes can be executed
by the playback control unit 18 07 with use of the movie object
file or the BD-J object file.
[0107]
During playback processing, the playback control unit
1807 updates the player variables in accordance with the status
of the playback. The playback control unit 1807 updates the
SPRM(l), the SPRM(2), the SPRM(21) and the SPRM(22) in
particular. These SPRM respectively show, in the stated order,
the stream selection numbers of the audio stream, the subtitle
stream, the secondary video stream, and the secondary audio
stream that are currently being processed. As an example, assume
that the audio stream number SPRM(l) has been changed by the
program execution unit 1806. In this case, the playback control
unit 18 07 first searches the stream selection table in the
playitem information currently being played back for a stream
entry including a stream selection number that matches the
stream selection number shown by the changed SPRM(l). The
playback control unit 18 07 then extracts the PID from the stream
identification information in the stream entry and transmits
the extracted PID to the system target decoder 1803 . As a result,
the audio stream having the same PID is selected and decoded
by the system target decoder 1803. This is how the audio stream
targeted for playback is switched. The subtitle stream and the
secondary video stream to be played back can be switched in a
similar manner.
[0108]
The playback execution unit 1806 is a processor and
executes programs stored in the movie object file or the BD-J
object file. The playback execution unit 1806 executes the
following controls in particular in accordance with the
programs. (1) The playback execution unit 1806 instructs the
playback control unit 18 07 to perform playlist playback
processing. (2) The playback execution unit 1806 generates
graphics data for a menu or a game as PNG or JPEG raster data,
and transfers the generated data to the system target decoder
1803 to be composited with other video data. Specific contents
of these controls can be designed relatively flexibly through
program designing. That is, the contents of the controls are
determined by the programming procedure of the movie object file
and the BD-J object file in the authoring procedure of the BD-ROM
disc 101.
[0109]
The plane adder 1810 receives primary video plane data,
secondary video plane data, IG plane data, PG plane data, and
image plane data from the system target decoder 1803, and
composites these data into a video frame or a field by
superimposition. The resultant composited video data is
outputted to the display device 103 and displayed on a screen
thereof.
[0110]
<>
[0111]
Fig. 20 is a functional block diagram of the system target
decoder 18 03. As shown in Fig. 20, the system target decoder
1803 includes a source depacketizer 2010, an ATC counter 2020,
a first 27 MHz clock 2030, a PID filter 2040, an STC counter
(STC1) 2050, a second 27 MHz clock 2060, a primary video decoder
2070, a secondary video decoder 2071, a PG decoder 2072, an IG
decoder 2073, a primary audio decoder 2074, a secondary audio
decoder 2075, an image processor 2080, a primary video plane
memory 2090, a secondary video plane memory 2091, a PG plane
memory 2092, an IG plane memory 2 093 , an image plane memory 2094,
and an audio mixer 2095.
[0112]
The source depacketizer 2010 reads source packets from
the read buffer 1802, extracts the TS packets from the read
source packets, and transfers the TS packets to the PID filter
2040. The source depacketizer 2010 further adjusts the time of
the transfer in accordance with the ATS of each source packet.
Specifically, the source depacketizer 2010 first monitors the
value of the ATC generated by the ATC counter 2020. Here, the
value of the ATC depends on the ATC counter 2020, and is
incremented in accordance with a pulse of the clock signal of
the first 27 MHz clock 2030. Subsequently, at the instant the
value of the ATC and the ATS of a source packet are identical,
the source depacketizer 2 010 transfers the TS packet extracted
from the source packet to the PID filter 204 0 at the recording
speed RTS1.
[0113]
The PID filter 2040 first selects, from among the TS
packets outputted from the source depacketizer 2010, TS packets
which have a PID that matches a PID pre-specified by the playback
control unit 1807. The PID filter 2040 then transfers the
selected TS packets to the decoders 2070-2075 depending on the
PID of the TS packets. For example, a TS packet with PID 0x1011
is transferred to the primary video decoder 2070, TS packets
with PIDs ranging from 0x1B00 to 0x1B1F, 0x1100 to 0x111F,
0x1A00 to 0x1A1F, 0x1200 to 0x121F, and 0x1400 to 0x141F are
transferred to the secondary video decoder 2071, the primary
audio decoder 2074, the secondary audio decoder 2075, the PG
decoder 2072, and the IG decoder 2073, respectively.
[0114]
The PID filter 204 0 further detects PCRs from each TS
packet using the PID of the TS packet. In this case, the PID
filter 2040 sets the value of the STC counter 2050 to a
predetermined value. Herein, the value of the STC counter 2050
is incremented in accordance with a pulse of the clock signal
of the second 27 MHz clock 2060. In addition, the value to which
the STC counter 2050 is set to is instructed to the PID filter
2040 from the playback control unit 1807 in advance. The
decoders 2070-2075 each use the value of the STC counter 2050
as the STC. That is, the decoders 2070-2075 perform decoding
processing on the TS packets outputted from the PID filter 2040
at the time indicated by the PTS or the DTS shown by the TS
packets.
[0115]
The primary video decoder 2070, as shown in Fig. 20,
includes a TB (Transport Stream Buffer) 2001, an MB
(Multiplexing Buffer) 2002, an EB (Elementary Stream Buffer)
2003, a compressed video decoder (Dec) 2004, and a DPB (Decoded
Picture Buffer) 2005. The TB 2001, the MB 2002, the EB 2003,
and the DPB 2005 each are a buffer memory and use an area of
a memory device internally provided in the primary video decoder
2070. Some or all of the TB 2001, the MB 2002, the EB 2003, and
the DPB 2005 may be separated in different memory devices. The
TB 2001 stores the TS packets received from the PID filter 2040
as they are. The MB 2002 stores PES packets reconstructed from
the TS packets stored in the TB 2001. Note that when the TS
packets are transferred from the TB 2001 to the MB 2002, the
TS header is removed from each TS packet. The EB 2003 extracts
an encoded video access unit from the PES packets and stores
the extracted encoded video access unit therein. The video
access unit includes compressed pictures, i.e., an I picture,
B picture, and P picture. Note that when data is transferred
from the MB 2002 to the EB 2003, the PES header is removed from
each PES packet. The compressed video decoder 2004 decodes each
video access unit in the MB 2002 at the time of the DTS shown
by the original TS packet. Herein, the compressed video decoder
2004 changes a decoding scheme in accordance with the
compression encoding formats, e.g., MPEG-2, MPEG-4 AVC, and VC1,
and the stream attribute of the compressed pictures stored in
the video access unit. The compressed video decoder 2 004 further
transfers the decoded pictures, i.e., a video frame or field,
to the DPB 2005. The DPB 2005 temporarily stores the decoded
pictures. When decoding a P picture or a B picture, the
compressed video decoder 2004 refers to the decoded pictures
stored in the DPB 2005. The DPB 2005 further writes each of the
stored pictures into the primary video plane memory 2090 at the
time of the PTS shown by the original TS packet.
[0116]
The secondary video decoder 2071 includes the same
structure as the primary video decoder 2070 . The secondary video
decoder 2071 first decodes the TS packets of the secondary video
stream received from the PID filter 2040 into uncompressed
pictures . Subsequently, the secondary video decoder 2071 writes
the resultant uncompressed pictures into the secondary video
plane memory 2091 at the time of the PTS shown by the TS packet.
[0117]
The PG decoder 2 072 decodes the TS packets received from
the PID filter 204 0 into uncompressed graphics data and writes
the resultant uncompressed graphics data to the PG plane memory
2092 at the time of the PTS shown by the TS packet.
[0118]
The IG decoder 2073 decodes the TS packets received from
the PID filter 2 04 0 into uncompressed graphics data and writes
the resultant uncompressed graphics data to the IG plane memory
2093 at the time of the PTS shown by the TS packet.
[0119]
The primary audio decoder 2074 first stores the TS packets
received from the PID filter 204 0 in a buffer provided therein.
Subsequently, the primary audio decoder 2074 removes the TS
header and the PES header from each TS packet in the buffer,
and decodes the remaining data into uncompressed LPCM audio data.
The primary audio decoder 2074 further sends the resultant audio
data to the audio mixer 2 095 at the time of the PTS shown by
the original TS packet. The primary audio decoder 2074 changes
a decoding scheme of the uncompressed audio data in accordance
with the compression encoding format, e.g. AC-3 or DTS, and the
stream attribute of the primary audio stream, included in the
TS packets.
[0120]
The secondary audio decoder 2075 has the same structure
as the primary audio decoder 2074. The secondary audio decoder
2075 first decodes the TS packets of the secondary audio stream
received from the PID filter 2040 into uncompressed LPCM audio
data. Subsequently, the secondary audio decoder 2075 sends the
uncompressed LPCM audio data to the audio mixer 2095 at the time
of the PTS shown by the original TS packet. The secondary audio
decoder 2075 changes a decoding scheme of the uncompressed audio
data in accordance with the compression encoding format, e.g.
Dolby Digital Plus or DTS-HD LBR, and the stream attribute of
the primary audio stream, included in the TS packets.
[0121]
The audio mixer 2095 mixes (superimposes) the
uncompressed audio data received from the primary audio decoder
2074 and from the secondary audio decoder 2075. The audio mixer
2095 further sends the resultant composited audio to an internal
speaker 103A of the display device 103 or the like.
[0122]
The image processor 2080 receives graphics data, i.e.,
PNG or JPEG raster data, along with the PTS thereof from the
program execution unit 1806. Upon the reception of the graphics
data, the image processor 2 08 0 appropriately processes the
graphics data and writes the graphics data to the image plane
memory 2094 at the time of the PTS thereof.
[0123]

[0124]
When AV stream files representing 2D video images are
stored on the BD-ROM disc 101, their physical arrangement needs
to be designed so that the 2D video images can be played back
seamlessly. Such a physical arrangement is described below.
Here, seamless playback means that video images and sounds are
played back smoothly and continuously from AV stream files.
[0125]
AV stream files are recorded on the BD-ROM disc 101 as
groups of extents. Here, logical addresses are consecutive
within a single extent. As described previously, logical
addresses in the BD-ROM disc 101 are substantially equivalent
to physical addresses. Accordingly, the physical addresses in
an extent are also substantially consecutive. Therefore, the
BD-ROM drive 1801 can continuously read one extent without
making the optical pickup perform a seek.
[0126]
In the volume area 202B shown in Fig. 2, each extent is
generally recorded in a plurality of physically consecutive
sectors. These sectors belong to the recording area for an AV
stream file in the STREAM directory area. Therefore, the logical
address of each extent can be known from each allocation
descriptor recorded in the file entry of the same recording
area.
[0127]
Fig. 21 is a schematic diagram showing a physical
arrangement of extents on the BD-ROM disc 101. As shown in Fig.
21, an AV stream file 2100 is divided into three extents 2101A,
2101B, and 2101C recorded on the track 202 on the disc 101. As
shown in Fig. 21, each of the extents 2101A-C is continuously
arranged, but different extents are not consecutive with each
other. To seamlessly play back video from the extents 2101A-C,
the physical arrangement of the extents 2101A-C needs to satisfy
the conditions described below.
[0128]
A group of arrows A1 shown in Fig. 21 indicates a playback
path. As the arrows A1 show, when video images are played back
from the AV stream file 2100, the extents 2101A, 2101B, and 2101C
are sequentially read by the playback device 102 . When the first
extent 2101A has been read to its end EA, the BD-ROM drive
temporarily stops the reading operation by the optical pickup,
increases the revolving speed of the BD-ROM disc 101, and
quickly moves the top TB of the next extent 2101B to the location
of the optical pickup. These operations of causing the optical
pickup to suspend the reading operation and then, while
suspended, position the optical pickup over the next area to
be read is referred to as a "jump." In Fig. 21, convex portions
J1 and J2 in the playback path Al indicate the period in which
a jump is performed on the playback path Al, i.e. the "jump
period."
[0129]
Types of jumps include a track jump and a focus jump, in
addition to the operation of increasing or decreasing the
revolving speed of the BD-ROM disc 101. Track jumps are
operations of moving the optical pickup in a radial direction
of the disc. Focus jumps are operations of moving the focus
position of the optical pickup from one recording layer to
another when the BD-ROM disc 101 is a multi-layer disc. These
types of jumps generally cause a longer seek time and a larger
number of sectors skipped in the reading processes, and thus
are referred to as "long jumps." During a jump period, the
optical pickup stops the reading operation. Accordingly, during
the jump periods J1 and J2 shown in Fig. 21, data is not read
from the corresponding portions G1 and G2 on the track 202,
respectively. The length of a portion skipped in a reading
process during a jump period, i.e. the length of the portions
Gl and G2, is called a "jump distance." Jump distances are
normally expressed by the number of sectors included in skipped
portions. A long jump is specifically defined as a jump whose
jump distance exceeds a predetermined threshold value. For
example, the BD-ROM standards specify that the threshold value
is to be 40,000 sectors based on the type of the BD-ROM disc
and the reading capability of the BD-ROM drive.
[0130]
During a jump period, the BD-ROM drive 1801 cannot read
data from the BD-ROM disc 101. For the playback device 102 to
seamlessly play back video from the AV stream file 2100, the
system target decoder 1803 should be able to continue decoding
processing even during a jump period and should be able to
maintain continual output of decoded video data. The physical
arrangement of the extents on the disc 101 thus needs to be
designed accordingly.
[0131]
Fig. 22 is a schematic diagram showing the playback
process system inside the 2D playback device shown in Fig. 18.
This playback process system converts an AV stream file read
from the BD-ROM disc 101 into video data VD and audio data AD.
As shown in Fig. 22, the BD-ROM drive 1801 reads an AV clip from
the BD-ROM disc 101 in units of extents and then stores the AV
stream file in the read buffer 1802 at a read rate of Rud. The
system target decoder 1803 reads source packets from each extent
in the read buffer 1802 at a mean transfer rate of Rext and then
decodes the source packets into video data VD and audio data
AD. Note that the mean transfer rate of Rext does not exceed
192/188 times the system rate 1101A shown in Fig. 11.
[0132]
Fig. 23 is a graph showing the change of the accumulated
data amount DA in the read buffer 18 02 during a playback process
period of an AV stream file. This is a graph of when video is
played back sequentially from the three extents 2101A-C shown
in Fig. 21.
[0133]
First, the top extent 2101A is read from the BD-ROM disc
101 into the read buffer 1802. During the read period PR1, the
accumulated data amount DA, as shown by the arrow 23 01 in Fig.
23, increases at a rate equivalent to the difference between
the read rate Rud and the mean transfer rate Rext, i.e. Rud - Rext.
As shown in Fig. 21, when the last EA in the top extent 2101A
is read, a jump Jl is performed to the top TB in the next extent
2101B. During the corresponding jump period PR J, reading of data
from the BD-ROM disc 101 is suspended. Therefore, as shown by
the arrow 2303 in Fig. 23, the accumulated data amount DA
decreases at the mean transfer rate Rext- Here, the accumulated
data amount in the read buffer 1802 during the read period PR1,
i.e. the size of the top extent 2101A, is assumed to be equivalent
to the data amount transferred from the read buffer 1802 to the
system target decoder 1803 from the read period PR1 through the
jump period PRJ. In this case, as shown in Fig. 23, the
accumulated data amount DA returns to the value at the start
of the read period PR1 by the end of the jump period PRJ. As
shown in Fig. 21, after the jump Jl, reading of the next extent
2101B begins . During the corresponding read period PR2, as shown
by arrow 2303 in Fig. 23, the accumulated data amount DA once
again increases at a rate equivalent to the difference between
data transfer rates, or Rud - Rext.
[0134]
The BD-ROM drive 1801 does not actually perform
read/transfer operations continuously, as shown by the arrows
23 01 and 23 03 in Fig. 23, but rather intermittently. This
ensures that the accumulated data amount DA during the read
periods PR1 and PR2 for the extents does not exceed the amount
of the read buffer 1802, or in other words, that overflow does
not occur in the read buffer 1802. Accordingly, the arrows 2301
and 23 03 in Fig. 23 represent what is actually a step-wise
increase as an approximated straight increase.
[0135]
To seamlessly play back video from the top extent 2101A
and the next extent 2101B regardless of the jump J1, during the
entire jump period PRJ, it is necessary to maintain provision
of data from the read buffer 1802 to the system target decoder
1803 and ensure continual output from the decoder 1803. Here,
as made clear by Fig. 23, even if the top extent 2101A is small,
if it is equivalent to the data amount transferred from the read
buffer 18 02 to the system target decoder 1803 from the read
period PR1 through the jump period PRJ, then the accumulated
data amount DA will not reach zero during the jump period PRJ.
That is, underflow will not occur in the read buffer 1802. If
this is the case, then the system target decoder 1803 can
maintain continual output of video data, and seamless playback
of video from this video data is possible. Accordingly, for
seamless playback of video, the size Sext of the top extent 2101A
should fulfill Equation 1.
[0136]
Equation 1

[0137]
In Equation 1, the jump period Tjump represents the length
of the jump period PRJ in units of seconds. The read rate RUd
represents the rate of reading data from the BD-ROM disc 101
to the read buffer 1802 in bits per second. The mean transfer
rate Rext represents, in bits per second, the mean transfer rate
at which the source packets contained in each extent are
transferred from the read buffer 1802 to the system target
decoder 1803. Note that dividing the mean transfer rate Rext in
the second line of Equation 1 by the number "8" is for converting
the unit of the extent size Sext from bits to bytes. That is,
the extent size Sext is represented in bytes. The function CEIL()
represents an operation to round up fractional numbers after
the decimal point of the value in parentheses. Hereinafter, the
minimum value of the extent size Sext expressed by the right-hand
side of Equation 1 is referred to as "minimum extent size."
[0138]
More specifically, the mean transfer rate Rext is
evaluated using the value obtained by dividing the size of an
extent expressed in bits by the extent ATC time. Here, the "size
of an extent expressed in bits" is equivalent to eight times
the product of the number of source packets in the extent and
the number of bytes per source packet (192 bytes) . The "extent
ATC time" is defined as the range of the ATS assigned to each
source packet in the extent, i.e. the value of the ATC
representing the time interval from the ATS for the top source
packet in the extent to the ATS for the top source packet in
the next extent. Accordingly, the extent ATC time is equal to
the time required to transfer all the data included in the extent
from the read buffer 1802 to the system target decoder 1803.
[0139]
On the other hand, the finite size of the read buffer 1802
restricts the maximum value of the jump time Tjump in which
seamless playback is possible. That is, even if the accumulated
data amount DA reaches the full capacity of the readbuf f er 1802,
an excessively long jump time TjUmp due to an excessively long
jump distance to the next extent would cause the accumulated
data amount DA to reach zero during the jump period PRJ, and
accordingly, would cause depletion of the data accumulated in
the read buffer 1802. In this case, the system target decoder
1803 would stop providing video data, and therefore seamless
playback could not be achieved. Hereinafter, the length of time
required for the accumulated data amount DA to decrease from
the full capacity of the read buffer 1802 to zero while data
supply from the BD-ROM disc 101 to the read buffer 18 02 is
suspended, that is, the maximum value of the jump time Tjump
allowable for seamless playback, is referred to as "maximum jump
time."
[0140]
Standards of optical discs normally predetermine the
relationship between jump distances and jump times by using the
access speed of an optical disc drive and the like. Fig. 24 is
a table showing an example of the relationship between jump
distances Sjump and jump times Tjump specified for BD-ROM discs.
In Fig. 24, jump distances Sjump are represented in units of
sectors. Here, 1 sector = 2048 bytes. As shown in Fig. 24, when
jump distances fall within the range of 0-10000, 10001-20000,
20001-40000 sectors, 40001 sectors-1/10 stroke, and 1/10 stroke
or greater, the corresponding jump times are 250 ms, 3 00 ms,
350 ms, 700 ms, and 14 00 ms, respectively. The minimum extent
sizes are calculated according to the regulation shown in Fig.
24 . Furthermore, the AV stream file is divided into a plurality
of extents and arranged on the BD-ROM disc 101 in accordance
with the minimum extent sizes. When the BD-ROM disc 101 is such
a disc, the BD-ROM drive 18 01 of the playback device 102 complies
with the regulation shown in Fig. 24, thereby being able to
seamlessly play back video images from the BD-ROM disc 101.
[0141]
When the BD-ROM disc 101 is a multi-layer disc and a
recording layer to be read is switched to another layer, 350
ms are needed for operations such as a focus jump to switch layers
in addition to the jump time Tjump specified in Fig. 24. This
length of time is hereinafter referred to as "layer switching
time." When there is a layer boundary between two extents to
be consecutively read, the minimum extent size is accordingly
determined based on the sum of the jump time Tjump corresponding
to the jump distance Sjump between the two extents and the layer
switching time.
[0142]
The maximum jump distance corresponding to the maximum
jump time is determined from the regulation in Fig. 24 and the
layer switching time. For example, assuming that the maximum
jump time is 700 ms, then the maximum jump distance is 1/10 stroke
(approximately 1.2 GB) and 40000 sectors (approximately 78.1
MB) without and with a layer boundary between two consecutive
extents, respectively.
[0143]
When video that extends from one section of an AV stream
file through a second, different section is played back, to
seamlessly connect the video played back from these sections,
the arrangement of the last extent of the previous section in
the playback path and the top extent of the next section should
satisfy the following conditions. First, the size of the last
extent should at least be equivalent to the minimum extent size
calculated based on the jump distance to the top extent. Next,
the jump distance should not exceed the maximum jump distance.
[0144]
Fig. 25 is a schematic diagram showing an example of a
physical arrangement of extents when 2D video images are
continuously played back from three different sections
2511-2513 of an AV stream file in turn. As shown in Fig. 25,
a playlist file 2500 includes three pieces of playitem
information (PI#l-3) 2501-2503. These pieces of playitem
information 2501-2503 specify the entireties of the three
different sections 2511-2513, respectively, as a playback
section. The Sections 2511-2513 are divided into a plurality
of extents 2521A, 2521B, 2522A, 2522B, and 2523 and recorded
on the track 202 of the BD-ROM disc 101.
[0145]
The sizes of the extents 2 521A and 2521B in the recording
area of the top section 2511 are at least equivalent to the
corresponding minimum extent size. Here, the minimum extent
size for the top extent 2521A is calculated from the jump
distance G1 from this extent 2521A to the next extent 2521B.
The minimum extent size for the last extent 2521B is calculated
from the jump distance G2 from this extent 2521B to the top extent
2522A in the second AV clip 2512. Furthermore, both jump
distances G1 and G2 do not exceed the maximum jump distance.
Similarly, in the recording area for the second section 2512,
the sizes of the extents 2521A and 2521B are at least equivalent
to the corresponding minimum extent size, and the minimum extent
sizes are calculated from the jump distances G3 and G4 between
extents. Furthermore, both jump distances Gl and G2 do not
exceed the maximum jump distance.
[0146]
By designing the arrangement of each extent in this way,
data can be provided to the system target decoder 1803 without
underflow occurring in the read buffer 1802 during any of the
jump periods. Therefore, during all of the jump periods, the
system target decoder 1803 can continue decoding processing and
maintain continual output of video data after decoding. In this
way, 2D video images can be played back seamlessly from the
sections 2511-2513.
[0147]

[0148]
Playback methods of 3D video images are roughly
classified into two categories: methods using a holographic
technique, and methods using parallax video.
[0149]
A method using a holographic technique is characterized
by allowing a viewer to perceive objects in video as
stereoscopic by giving the viewer's visual perception
substantially the same information as optical information
provided to visual perception by human beings of actual obj ects.
However, although a technical theory for utilizing these
methods for moving video display has been established, it is
extremely difficult to construct, according to the present
technology, a computer that is capable of real-time processing
of the enormous amount of calculation required for moving video
display and a display device having super-high resolution of
several thousand lines per 1 mm. Accordingly, at the present
time, the realization of these methods for commercial use is
hardly in sight.
[0150]
"Parallax video" refers to a pair of 2D video images shown
to each of a viewer's eyes for the same scene, i.e. the pair
of a left-view and a right-view. A method using a parallax video
is characterized by playing back the left-view and right-view
of a single scene so that the viewer sees each view in only one
eye, thereby allowing the user to perceive the scene as
stereoscopic.
[0151]
Figs. 26A, 26B, 26C are schematic diagrams illustrating
the principle of playing back 3D video images (stereoscopic
video) according to a method using parallax video. Fig. 26A
shows from above when a viewer 2601 is looking at a cube 2602
placed in front of the viewer' s face. Figs. 26B and 26C
respectively show the outer appearance of the cube 2602 as a
2D video image as perceived by the left eye 2601L and the right
eye 2601R of the viewer 2601. As is clear from comparing Fig.
26B and Fig. 26C, the outer appearances of the cube 2602 as
perceived by the eyes are slightly different. The difference
of the outer appearances, i.e., the binocular parallax allows
the viewer 2601 to recognize the cube 2602 as three-dimensional.
Thus, according to a method using parallax video, first, left
and right 2D video images with different viewpoints are prepared
for a single scene. For example, for the cube 2602 shown in Fig.
26A, the left view of the cube 2602 shown in Fig. 26B and the
right view shown in Fig. 26C are prepared. Here, the position
of each viewpoint is determined by the binocular parallax of
the viewer 2601. Next, each video image is played back so as
to allow the corresponding eye of the viewer 2601 to perceive
it. Consequently, the viewer 2601 recognizes the scene played
back on the screen, i.e., the video of the cube 2602 as
stereoscopic. In this way, unlike methods using the holography
technique, methods using parallax video have the advantage of
requiring preparation of 2D video images from merely two
viewpoints.
[0152]
Several concrete methods for how to use parallax video
have been proposed. From the standpoint of how these methods
show left and right 2D video images to the viewer's eyes, the
methods are divided into alternate frame sequencing methods,
methods that use a lenticular lens, and two-color separation
methods.
[0153]
In alternate frame sequencing, left and right 2D video
images are alternately displayed on a screen for a predetermined
time, while the viewer observes the screen using shutter glasses.
Here, each lens in the shutter glasses is, for example, formed
by a liquid crystal panel. The lenses pass or block light in
a uniform and alternate manner in synchronization with
switching of the 2D video images on the screen. That is, each
lens functions as a shutter that periodically blocks an eye of
the viewer. More specifically, while a left video image is
displayed on the screen, the shutter glasses make the left-side
lens transmit light and the right-hand side lens block light.
Conversely, while a right video image is displayed on the screen,
the shutter glasses make the right-side glass transmit light
and the left-side lens block light. As a result, the eyes of
the viewer see afterimages of the right and left video images,
which are overlaid with each other, and perceive a 3D video
image.
[0154]
According to the alternate-frame sequencing, as
described previously, right and left video images are
alternately displayed in a predetermined cycle. For example,
when 24 video frames are displayed per second for playing back
a normal 2D movie, 4 8 video frames in total for both right and
left eyes need to be displayed for a 3D movie. Accordingly, a
display device able to quickly execute rewriting of the screen
is preferred for this method.
[0155]
In a method using a lenticular lens, a right video frame
and a left video frame are respectively divided into reed-shaped
small and narrow areas whose longitudinal sides lie in the
vertical direction of the screen. In the screen, the small areas
of the right video frame and the small areas of the left video
frame are alternately arranged in the landscape direction of
the screen and displayed at the same time. Here, the surface
of the screen is covered by a lenticular lens. The lenticular
lens is a sheet-shaped lens constituted from parallel-arranged
multiple long and thin hog-backed lenses. Each hog-backed lens
lies in the longitudinal direction on the surface of the screen.
When a viewer sees the left and right video frames through the
lenticular lens, only the viewer' s left eye perceives light from
the display areas of the left video frame, and only the viewer' s
right eye perceives light from the display areas of the right
video frame. This is how the viewer sees a 3D video image from
the parallax between the video images respectively perceived
by the left and right eyes. Note that according to this method,
another optical component having similar functions, such as a
liquid crystal device, may be used instead of the lenticular
lens. Alternatively, for example, a longitudinal polarization
filter may be provided in the display areas of the left image
frame, and a lateral polarization filter may be provided in the
display areas of the right image frame. In this case, the viewer
sees the display through polarization glasses. Here, for the
polarization glasses, a longitudinal polarization filter is
provided for the left lens, and a lateral polarization filter
is provided for the right lens. Consequently, the right and left
video images are respectively perceived only by the
corresponding eyes, thereby allowing the viewer to recognize
a stereoscopic video image.
[0156]
In a method using parallax video, in addition to being
constructed from the start by a combination of left and right
video images, the 3D video content can also be constructed from
a combination of 2D video images with a depth map. The 2D video
images represent 3D video images projected on a hypothetical
2D picture plane, and the depth map represents the depth of each
portion of the 3D video image as compared to the 2D picture plane
in units of pixels. When the 3D content is constructed from a
combination of 2D video images with a depth map, the 3D playback
device or the display device first constructs left and right
videos from the combination of 2D video images with a depth map
and then creates 3D video images from these left and right videos
using one of the above-described methods.
[0157]
Fig. 27 is a schematic diagram showing an example of
constructing a left-view 2703-L and a right-view 2703-R from
a combination of a 2D video image 2701 and a depth map 2702.
As shown in Fig. 27, a circular plate 2711 is shown in the
background 2712 of the 2D video image 2701. The depth map 2702
indicates the depth for each pixel in each portion of the 2D
video image 2701. According to the depth map 2702, in the 2D
video image 2701, the display area 2721 of the circular plate
2711 is closer to the viewer than the screen, and the display
area 2722 of the background 2712 is deeper than the screen. The
parallax video generation unit 2700 in the playback device 102
first calculates the binocular parallax for each portion of the
2D video image 2701 using the depth of each portion indicated
by the depth map 2702 . Next, the parallax video generation unit
2700 shifts the display position of each portion in the 2D video
image 2701 in accordance with the calculated binocular parallax
to construct the left-view 2703L and the right-view 2703R. In
the example shown in Fig. 27, the parallax video generation unit
2700 shifts the display position of the circular plate 2711 in
the 2D video image 2701 as follows: the display position of the
circular plate 2731L in the left-view 2703L is shifted to the
right by half of its binocular parallax, S1, and the display
position of the circular plate 2731R in the right-view 2703R
is shifted to the left by half of its binocular parallax, S1.
In this way, the viewer perceives the circular plate 2711 as
being closer than the screen. Conversely, the parallax video
generation unit 2700 shifts the display position of the
background 2712 in the 2D video image 2701 as follows: the
display position of the background 2732L in the left-view 2703L
is shifted to the left by half of its binocular parallax, S2,
and the display position of the background 2732R in the
right-view 2703R is shifted to the right by half of its binocular
parallax, S2. In this way, the viewer perceives the background
2712 as being deeper than the screen.
[0158]
A playback system for 3D video images with use of parallax
video has already been established for use in movie theaters,
attractions in amusement parks, and the like. Accordingly, this
method is also useful for implementing home theater systems that
can play back 3D video images. In the embodiments of the present
invention, among methods using parallax video, an
alternate-frame sequencing method or a method using
polarization glasses is assumed to be used. However, apart from
these methods, the present invention can also be applied to
other methods, as long as they use parallax video. This will
be obvious to those skilled in the art from the following
explanation of the embodiments.
[0159]

[0160]
Next, among the file group in the BDMV directory 242 shown
in Fig. 2, an explanation is provided for the data structure
of the files related to playback of 3D video content. Here,
fundamental parts of the data structure are identical with those
of the data structure for 2D content, which is shown in Figs.
2-17. Accordingly, the following will mainly describe expanded
or changed portions with respect to the data structure for the
2D video content, incorporating by reference the aforementioned
explanation for the fundamental parts.
[0161]
<>
[0162]
Fig. 28 is a schematic diagram showing the relationship
between a movie object MVO-3D and a BD-J object BDJ0-3D, as
related to the playback of 3D video images described in the index
table 310, and three types of playlist files 243A-C. The first
playlist file (00001.mpls) 243A is a 2D playlist file. The
second playlist file (00002.mpls) 243B and the third playlist
file (00003.mpls) 243C are 3D playlist files . In the index table
310, the items "title 3" and "title 4" are allocated to 3D video
titles. The movie object MVO-3D associated with the item "title
3" contains, in addition to a group of commands related to 2D
video image playback processing using the first playlist file
243A, a group of commands related to 3D video image playback
processing using either of the other playlist files 243B and
243C. In the BD-J object BDJO-3D associated with the item "title
4," the application management table specifies, in addition to
a Java application program related to 2D video image playback
processing using the first playlist file 243A, a Java
application program related to 3D video image playback
processing using either of the other playlist files 243B and
243C.
[0163]
For example, when title 3 is selected by user operation,
the playback device 102 refers to the item "title 3" in the index
table 310, and calls and executes the movie object MVO-3D. In
this way, the playback device 102 first performs the following
four types of distinguishing processes: 1) Does the playback
device 102 itself support playback of 3D video images? 2) Has
the user selected 3D video image playback? 3) Does the display
device 103 support playback of 3D video images? and 4) Is the
3D video image playback mode of the playback device 102 set to
L/R mode or depth mode? Here, "L/R (left/right) mode" refers
to a mode in which 3D video images are played back from a
combination of a left-view stream file and a right-view stream
file, whereas "depth mode" refers to a mode in which 3D video
images are played back from a combination of a 2D/AV stream file
and a depth map AV stream file. Next, the playback device 102
selects one of the playlist files 243A-C to be played back in
accordance with the results of the above-described
distinguishing processes. When, on the other hand, the playback
device 102 refers to the item "title 4" in the index table 310
and calls the BD-J object BDJO-3D, the playback device 102
executes each Java application program in accordance with the
application management table. In this way, the playback device
102 performs the above-described distinguishing processes and
selects a playlist file in accordance with the results thereof.
[0164]
Fig. 29 is a flowchart showing processes, performed when
a title of 3D video images is selected, to select a playlist
file to be played back. In the example shown in Fig. 28, when
the item "title 3" in the index table 310 is referred to,
selection processes are executed in accordance with the movie
object MVO-3D. When the item "title 4" is referred to, selection
processes are executed in accordance with the Java application
programs specified by the BD-J object BDJO-3D.
[0165]
Here, the selection processes are assumed to be performed
when the SPRM shown in Fig. 19 further include the following
two types of flags: (A) a flag for identifying "whether the
playback device 102 only supports playback of 2D video images
or also supports 3D video images, " and (B) a flag for identifying
"whether the 3D video image playback mode of the playback device
102 is set to L/R mode or depth mode. " Specifically, this assumes
that the SPRM(24) includes flag (A) , and the SPRM(25) includes
flag (B). Furthermore, the value of each flag is assumed to
indicate the following: when the SPRM(24) is "0," the playback
device 102 only supports playback of 2D video images, and when
it is "1, " the playback device 102 also supports 3D video image
playback. When the SPRM(25) is "0, " the 3D video image playback
mode of the playback device 102 is L/R mode, and when it is "1, "
the 3D video image playback mode is depth mode.
[0166]
In step S2901, the playback device 102 checks the value
of the SPRM(24) . If the value is 0, the process advances to S2905 .
If the value is 1, the process advances to S2902.
[0167]
In step S2902, the playback device 102 causes the display
device 103 to display a menu and allows the user to select 2D
video image playback or 3D video image playback. If the user
selects 2D video image playback with an operation of a remote
control or the like, the process advances to step S2905. On the
other hand, if the user selects 3D video image playback, the
process advances to step S2903.
[0168]
In S2903, the playback device 102 checks whether the
display device 103 supports 3D video image playback.
Specifically, the playback device 102 exchanges CEC messages
with the display device 103 via an HDMI cable 122, and the
playback device 102 and asks the display device 103 whether it
supports 3D video image playback. If the display device 103 does
not support 3D video image playback, the process advances to
step S2905, and when it does support 3D video image playback,
the process advances to step S2904.
[0169]
In step S2904, the playback device 102 checks the value
of the SPRM(25) . When the value is 1, the process advances to
step S2907.
[0170]
In step S2905, the playback device 102 selects the first
playlist file 243A as the playback target. Note that in this
case, the playback device 102 may cause the display device 103
to display the reason the 3D video image playback was not
selected.
[0171]
In step S2906, the playback device 102 selects the second
playlist file 243B as the playback target.
[0172]
In step S2907, the playback device 102 selects the third
playlist file 243C as the playback target.
[0173]
<<3D Playlist File>>
[0174]
Fig. 3 0 is a schematic diagram showing an example of the
structures of the first playlist file 243A and the second
playlist file 243B. The first AV clip group 3001 is formed by
sections LCL1-3 in the first AV stream file 245A. The second
AV clip group 3002 is formed by sections RCL1-3 in the second
AV stream file 245B. Both the main path 3003 in the first playlist
file 243A and the main path 3004 in the second playlist file
243B include three pieces of playitem information #1-3. The
three pieces of playitem information #1-3 all specify playback
areas in the first AV clip group 3001. Unlike the first playlist
file 243A, the second playlist file 243B further includes a
sub-path 3005. The sub-path 3005 includes three pieces of
sub-playitem information #1-3, and each piece of sub-playitem
information #1-3 specifies a playback area in the second AV clip
group 3002. The pieces of sub-playitem information #1-3 are in
one-to-one correspondence with the pieces of playitem
information #1-3 . The length of the playback area of each piece
of sub-playitem information is equivalent to the length of the
playback area of the corresponding piece of playitem
information. The sub-path 3 005 further includes a sub-path type
3006. The "sub-path type" indicates whether the playback
process should by synchronous between the main path and the
sub-path. In Fig. 30, the value of the sub-path type 3006 is
"3D L/R." This value indicates that the sub-path 3005 is to be
used for playback of 3D video images in L/R mode, and in
particular that the sub-path 3005 specifies the playback path
for the right-view stream file.
[0175]
When the first playlist file 243A is selected as the
playback target by the selection process shown in Fig. 29, the
playback device 102 plays back 2D video images from the first
AV clip group 3 001 in accordance with the corresponding main
path 3003 . On the other hand, when the second playlist file 243B
is selected as the playback target, the playback device 102
executes 3D video image playback in L/R mode. In particular,
the playback device 102 generates a left-view from the first
AV clip group 3 001 in accordance with the main path 3 004 of the
second playlist file 24 3B and generates a right-view from the
second AV clip group 3002 in accordance with the sub-path 3005.
At that time, the playback device 102 detects that the value
of the sub-path type 3 006 is "3D L/R" and synchronizes playback
processing between the main path 3004 and the sub-path 3005.
In this way, playback of 3D video images in L/R mode is performed.
Here, the first AV clip group 3 0 01 is used in common in 2D video
image playback and in 3D video image playback in L/R mode.
[0176]
In each piece of playitem information in the second
playlist file 243B, a stream entry for the 2D/left-view stream
and a stream entry for the right-view stream have been added
to the stream selection table 1505 shown in Fig. 15. The stream
entries 1509 for the 2D/left-view stream and the right-view
stream have the same contents such as the frame rate, resolution,
and video format. Note that each stream entry 1509 may further
have a flag for identifying the 2D/left-view stream and the
right-view stream added therein.
[0177]
Fig. 31 is a schematic diagram showing an example of the
structures of the first playlist file 243A and the third
playlist file 243C. The first AV clip group 3101 is formed by
the sections LCL1-3 in the first AV stream file 245A. The third
AV clip group 3102 is formed by the sections DCL1-3 in the third
AV stream file 245C. Both the main path 3103 in the first playlist
file 243A and the main path 3104 in the third playlist file 243C
include three pieces of playitem information #1-3. The three
pieces of playitem information #1-3 all specify playback areas
in the first AV clip group 3101. Unlike the first playlist file
243A, the third playlist file 243C further includes a sub-path
3105. The sub-path 3105 includes three pieces of sub-playitem
information #1-3, and each piece of sub-playitem information
#1-3 specifies a playback area in the third AV clip group 3102.
The pieces of sub-playitem information #1-3 are in one-to-one
correspondence with the pieces of playitem information #1-3.
The length of the playback area of each piece of sub-playitem
information is equivalent to the length of the playback area
of the corresponding piece of playitem information. Furthermore,
the value of the sub-path type 3106 in the sub-path 3105 is "3D
depth." This value indicates that the sub-path 3105 is to be
used for playback of 3D video images in depth mode, and in
particular that the sub-path 3105 specifies the playback path
for the depth map AV stream file.
[0178]
When the first playlist file 243A is selected as the
playback target by the selection process shown in Fig. 29, the
playback device 102 plays back 2D video images from the first
AV clip group 3101 in accordance with the corresponding main
path 3103. On the other hand, when the third playlist file 243C
is selected as the playback target, the playback device 102
executes 3D video image playback in depth mode. In particular,
the playback device 102 generates 2D video images from the first
AV clip group 3101 in accordance with the main path 3104 of the
third playlist file 243C and generates a depth map from the third
AV clip group 3102 in accordance with the sub-path 3105. At that
time, the playback device 102 detects that the value of the
sub-path type 3 006 is "3D depth" and synchronizes playback
processing between the main path 3104 and the sub-path 3105.
Furthermore, the playback device 102 generates a pair of a
left-view and a right-view from the generated 2D video images
and depth map. In this way, playback of 3D video images in depth
mode is performed. Here, the third AV clip group 3102 is used
in combination with the first AV clip group 3101.
[0179]
In each piece of playitem information in the third
playlist file 243C, a stream entry for the depth map stream has
been added to the stream selection table 1505 shown in Fig. 15.
The primary video stream and the depth map stream share the
contents of the stream entry 1509 . A flag to distinguish between
the primary video stream and the depth map stream may further
be added to the stream entry 1509.
[0180]
Fig. 32 is a schematic diagram showing another example
of the structures of a first playlist file 243A and a 3D playlist
file 3200 . The first AV clip group 3201 is formed by the sections
LCL1-3 in the first AV stream file 245A. The second AV clip group
3202 is formed by the sections RCL1-3 in the second AV stream
file 245B. The third AV clip group 32 02D is formed by the sections
DCL1-3 in the third AV stream file 245C. Both the main path 3103
in the first playlist file 243A and the main path 3204 in the
3D playlist file 3200 include three pieces of playitem
information #1-3 . The three pieces of playitem information #1-3
all specify playback areas in the first AV clip group 3201.
Unlike the second playlist file 243B and the third playlist file
243C, the 3D playlist file 3200 includes two types of sub-paths.
In the first sub-path 3205R, the sub-path type 3206R is "3D L/R, "
and each piece of sub-playitem information #1-3 specifies a
playback area in the second AV clip group 3202R. On the other
hand, in the second sub-path 3205D, the sub-path type 3206D is
"3D depth," and each piece of sub-playitem information #1-3
specifies a playback area in the third AV clip group 32 02D. The
pieces of sub-playitem information #1-3 are in one-to-one
correspondence with the pieces of playitem information #1-3.
The length of the playback area of each piece of sub-playitem
information is equivalent to the length of the playback area
of the corresponding piece of playitem information.
[0181]
When 3D video images are played back in accordance with
the 3D playlist file 3200, the playback device 102 can switch
the playback mode between L/R mode and depth mode as the sub-path
targeted for playback is switched between two types of sub-paths,
3205R and 3205D. In particular, this switching process can be
performed more rapidly than by a switching process between the
second playlist file 243B and the third playlist file 243C.
[0182]
Unlike Fig. 32, the 3D playlist file can include a
plurality of sub-paths with equivalent sub-path types. For
example, when 3D video images with differing binocular
disparities for the same scene are represented by a common
left-view and differing right-views, a different right-view
stream file can be recorded on the BD-ROM disc 101 for each
different right-view. In other words, the STREAM directory in
the BD-ROM disc 101 may include two or more right-view stream
files for the same left-view stream file. In this case, a
plurality of sub-paths whose sub-path type is "3D L/R" is
provided in the 3D playlist file. These sub-paths individually
specify playback paths for each right-view stream file. In
addition, two or more types of depth map AV stream files may
be included for one 2D/AV stream file. In this case, a plurality
of sub-paths whose sub-path type is "3D depth" is provided in
the 3D playlist file. These sub-paths individually specify
playback paths for each depth map AV stream file. When 3D video
images are played back in accordance with this kind of 3D
playlist file, since the sub-path targeted for playback can be
switched rapidly, for example in accordance with user operation,
the binocular parallax can be changed without substantially
interrupting the 3D video images. In this way, the user can
easily select the 3D video images for the desired binocular
parallax.
[0183]
<>
[0184]
Figs. 33A and 33B are schematic diagrams showing the
elementary stream multiplexed in the second AV stream file 245B
and the third AV stream file 245C. Unlike the first AV stream
file 245A shown in Fig. 4, the second AV stream file 245B and
the third AV stream file 245C include only the primary video
streams 3321 and 3331, respectively. The primary video stream
3321 in the second AV stream file 245B represents the right-view
stream, and the primary video stream 3331 in the third AV stream
file 245C represents the depth map stream. Accordingly,
PID=0xl012 and 0x1013 are allocated for the primary video
streams 3321 and 3331; these differ from the PID=0xl011 for the
primary video stream in the first AV stream file 245A, which
represents the left-view stream.
[0185]
Fig. 34 is a schematic diagram showing the pictures
3411-3414, etc. included in the left-view stream 3410 and the
pictures 3421-3424, etc . included in the right-view stream 3420
in the time order in which they are displayed. Each picture
3411-3414 and 3421-3424 represents one frame or field of video
data.
[0186]
As shown in Fig. 34, each picture 3411-3414 in the
left-view stream 3410 is compressed with inter-picture
predictive encoding. This encoding method is exactly the same
as the encoding method for each picture in the video stream 600
shown in Fig. 6. Specifically, the top picture in a GOP is first
compressed into an I0 picture 3411 by intra-picture encoding.
Here, the subscripted number indicates the sequential number
allotted to each picture in the order of presentation time. Next,
the fourth picture is compressed as P3 picture 3414 using I0
picture 3411 as a reference picture, as indicated by the arrows
in Fig. 34. Then the second and third pictures are compressed
as Bri picture 3412 and Br2 picture 3413 respectively, using
the earlier I0 picture 3411 and the later P3 picture 3414 as
reference pictures. Here, "Br pictures" are the pictures among
the B pictures that are used as reference pictures in
inter-picture predictive encoding of other pictures.
[0187]
Each picture 3421-3424 in the right-view stream 3420 is
compressed with inter-picture predictive encoding. However,
this encoding method differs from the encoding method for each
picture 3411-3414 in the left-view stream 3410, as in addition
to the video's temporal redundancy, it uses the redundancy
between the left and right videos. That is, the reference
pictures for each picture 3421-3424, as shown by the arrows in
Fig. 34, are selected not only from the same right-view stream
3420 but also from the left-view stream 3410. The presentation
times for each picture 3421-3424 in the right-view stream 3420
and for each reference picture selected from the left-view
stream 3410 are substantially equivalent. In other words, the
pictures represent the right-view and the left-view of the same
scene in the 3D video images. Specifically, the top picture in
the right-view stream 342 0 is first compressed as P0 picture
3421 using I0 picture 3411 in the left-view stream 3410 as a
reference picture. Next, the fourth picture is compressed as
P3 picture 3424 using P0 picture 3421 and P3 picture 3414 in the
left-view stream 3410 as reference pictures. Then the second
picture is compressed as B1 picture using Br1 picture 3412 in
the left-view stream 3410 along with P0 picture 3421 and P3
picture 3424 as reference pictures. Similarly, the third
picture is compressed as B2 picture using Br2 picture 3413 in
the left-view stream 3410 along with P0 picture 3421 and P3
picture 3424 as reference pictures.
[0188]
The revised standards for MPEG-4 AVC/H.264, called MVC
(Multiview Video Coding), are known as a video compression
encoding method that makes use of this sort of correlation
between left and right videos. MVC was created in July of 2008
by the JVT (Joint Video Team) , a joint project between ISO/IEC
MPEG and ITU-T VCEG, and is a standard for collectively encoding
video that can be seen from a plurality of perspectives. With
MVC, not only temporal similarity in video is used for
inter-video predictive encoding, but also similarity between
videos from differing perspectives. This type of predictive
encoding has a higher video compression ratio than predictive
encoding that individually compresses video seen from each
perspective.
[0189]
As described previously, pictures in the left-view stream
3410 are used as reference pictures for compression of pictures
in the right-view stream 3420. Therefore, unlike the left-view
stream 3410, the right-view stream 3420 cannot be decoded
independently. On the other hand, however, since the
correlation between the left-view and the right-view is high,
the pictures in the right-view stream 3420 generally have a
significantly higher compression rate than the pictures in the
left-view stream 3410, and thus the amount of data is
significantly smaller.
[0190]
Each picture in the depth map stream is compressed with
exactly the same inter-picture predictive encoding as the
encoding method for the pictures in the video stream 600 shown
in Fig. 6. Here, this encoding method is equivalent to the
encoding method for the right-view stream. For example, when
the right-view stream is encoded with the MVC format, the depth
map stream is also encoded with the MVC format. In this case,
as with the right-view stream, the depth map stream cannot be
decoded independently. On the other hand, however, when the
playback device 102 switches the 3D video image playback mode
from L/R mode to depth mode, the playback device 102 can smoothly
switch from the right-view stream to the depth map stream.
[0191]
Hereinafter, a video stream that can be decoded
independently, like the left-view stream 3410, and that can be
used for playback of monoscopic video images is referred to as
a "base-view stream." Furthermore, an AV stream file that
includes a base-view stream is referred to as a "base-view
stream file." On the other hand, a video stream that requires
a base-view stream for decoding, like the right-view stream 3420
and the depth map stream, and that is used in combination with
that base-view stream for playback of stereoscopic video images
is referred to as a "dependent-view stream." Furthermore, an
AV stream file that includes a dependent-view stream is referred
to as a "dependent-view stream file."
[0192]
Figs. 35A and 35B are schematic diagrams showing the
relationship between PTSs and DTSs allocated to each picture
in the base-view stream 3501 and the dependent-view stream 3502 .
As shown in Figs. 35A and 35B, the same PTS and DTS are allocated
to a set of pictures used for displaying the same frame or field
in the 3D video images in the video streams 3501 and 3502. For
example, the top frame or field in the 3D video images is
represented with a combination of Ix picture 3511 in the
base-view stream 3501 and P1 picture 3521 in the dependent-view
stream 3502. Therefore, the PTS and DTS are equivalent between
the pair of pictures 3511 and 3521. Here, the subscripted number
indicates the serial number allocated to each picture in order
of the DTS. Similarly, the PTS and DTS are equivalent between
the second pictures in the video streams 3501 and 3502, i.e.
P2 picture 3512 and P2 picture 3522. The PTS and DTS are
equivalent between the third pictures in the video streams 3501
and 3502, i.e. Br3 picture 3513 and B3 picture 3523. They are
also equivalent between Br4 picture 3514 and B4 picture 3524.
[0193]
As in the data structure of the video stream 700 shown
in Fig. 7, one video access unit is formed for each picture in
the GOPs constituting the actual base-view stream and the
dependent-view stream. Furthermore, the sequence header in the
top video access unit in each pair of GOPs in the base-view stream
and the dependent-view stream includes the same resolution,
frame rate, and aspect ratio. In particular, the frame rate is
equivalent to the value for when the base-view stream is decoded
independently by a 2D playback device.
[0194]
A pair of video access units between the base-view stream
3501 and the dependent-view stream 3502 that contains pictures
with equivalent PTSs and DTSs is referred to as a "3D video image
access unit. " By allocating PTSs and DTSs as shown in Figs. 35A
and 35B, the decoder of a 3D playback device can easily process
the base-view stream 3501 and the dependent-view stream 3502
in parallel in units of 3D video image access units. In this
way, the pair of pictures that is to be used for display of the
same frame or field in a 3D video image is assured of being
processed in parallel by the decoder.
[0195]
Fig. 36 is a schematic diagram showing the data
configuration of a video access unit 3600 in the left-view
stream, the right-view stream, and the depth map stream. As
shown in Fig. 36, in addition to the data structure in Fig. 7,
the video access unit 3600 further includes decoding switch
information 3601. In particular, the decoding switch
information 3601 is stored in the supplementary data 711D, i.e.
in the SEI message as per MPEG-4 AVC. The decoding switch
information 3601 is information to allow the 3D playback device
to easily specify the video access unit that is the target of
decoding. Here, as described below, the decoder in the 3D
playback device alternately decodes the base-view stream and
the dependent-view stream by video access units. At that time,
in accordance with the time of the DTS assigned to each video
access unit, the decoder specifies the next video access unit
to be decoded. However, many types of decoders ignore the DTS,
decoding video access units sequentially. With this sort of
decoder, it is preferable for each video access unit to include
decoding switch information 36 01 in addition to the DTS.
[0196]
As shown in Fig. 36, the decoding switch information 3601
includes a subsequent access unit type 3602, a subsequent access
unit size 3603 and a decode counter 3604.
[0197]
The subsequent access unit type 3602 indicates to which
of the base-view stream and the dependent-view stream the
subsequent video access unit to be decoded belongs . For example,
when the value shown by the subsequent access unit type 3602
is "1," the subsequent video access unit belongs to the
base-view stream, and when the value shown by the subsequent
access unit type 3602 is "2," the subsequent video access unit
belongs to the dependent-view stream. When the value shown by
the subsequent access unit type 3602 is "0," the subsequent
video access unit is at the end of the stream targeted for
decoding, and no subsequent video access unit for decoding
exists.
[0198]
A subsequent access unit size 3603 indicates the size of
each subsequent video access unit that is to be decoded. By
referring to each subsequent access unit size 3603, the decoder
in the 3D playback device can specify the size of a video access
unit without analyzing its structure. Accordingly, the decoder
can easily extract video access units from the buffer.
[0199]
The decode counter 3204 shows a decoding order of the video
access unit to which it belongs. The order is counted from video
access unit that includes an I picture in the base-view stream.
Figs. 3 7A and 3 7B are schematic diagrams showing the values
3604A and 3604B of the decode counters allocated to each picture
in the base-view stream 3701 and the dependent-view stream 3702 .
As shown in Figs. 37A and 37B, there are two ways of allocating
values.
[0200]
In Fig. 37A, the decode counter values 3606A and 3604B
are incremented alternately between the base-view stream 3701
and the dependent-view stream 3702. For example, for the I
picture 3711 in the base-view stream 3701, a value of "1" is
allocated to the decode counter value 3604A. Next, a value of
"2" is allocated to the decode counter value 3604B for the next
picture to be decoded, the P picture 3721 in the dependent-view
stream 3702. Furthermore, a value of "3" is allocated to the
decode counter value 3604A for the following picture to be
decoded, the P picture 3712 in the base-view stream 3701. By
allocating values in this way, even when the decoder in the 3D
playback device fails to read one of the video access units due
to some error, the decoder can immediately specify the missing
picture using the decode counter values 3604A and 3604B.
Therefore, the decoder can perform error processing
appropriately and promptly.
[0201]
In Fig. 3 7A, for example, an error occurs during the
reading of the third video access unit in the base-view stream
3701, and the Br picture 3713 is missing. During decoding
processing of the P picture 3722 contained in the second video
access unit in the dependent-view stream 3702, however, the
decoder has read the decode counter value 3604B for this video
access unit and retained the value. Accordingly, the decoder
can predict the decode counter value 3604A for the next video
access unit to be processed. Specifically, the decode counter
value 3604B in the video access unit that includes the P picture
3722 is "4." Therefore, the decode counter value 3604A for the
next video access unit to be read can be predicted to be "5."
The next video access unit that is actually read, however, is
the fourth video access unit in the base-view stream 3701, whose
decode counter value 3604A is " 7 . " The decoder thus detects that
it failed to read a video access unit. Accordingly the decoder
can execute error processing to "skip decoding processing of
the B picture 3723 extracted from the third video access unit
in the dependent-view stream 3702, since the Br picture 3713
to be used as a reference is missing. " In this way, the decoder
checks the decode counter values 3604A and 3604B during each
decoding process. Consequently, the decoder can promptly detect
errors during reading of video access units and can promptly
execute appropriate error processing. As a result, the decoder
can prevent noise from contaminating the playback video.
[0202]
In Fig. 37B, decode counter values 3604C and 3604D are
incremented successively in the video streams 3701 and 3702.
Therefore, the decode counter values 3604C and 36 04D are
equivalent for a pair of pictures in the same 3D video image
access unit. In this case, when the decoder decodes a video
access unit in the base-view stream 3701, it can predict that
"the decode counter value 3604C is equivalent to the decode
counter value 3604D for the next video access unit to be decoded
in the dependent-view stream 3702." Conversely, when the
decoder decodes a video access unit in the dependent-view stream
3702, it can predict that "the decode counter value 3604C for
the next video access unit to be decoded in the base-view stream
3701 is equivalent to the decode counter value 3604D plus one. "
Accordingly, at any point in time, the decoder can promptly
detect an error in reading a video access unit using the decode
counter values 3604C and 3604D and can promptly execute
appropriate error processing. As a result, the decoder can
prevent noise from contaminating the playback video.
[0203]
<>
[0204]
Figs. 3 8A and 38B are schematic diagrams showing the data
structure of a base-view clip information file 3801 and of a
dependent-view clip information file 3802, respectively. Here,
a "base-view clip information file" is a clip information file
that should be associated with a base-view stream file, and in
the present embodiment corresponds to a 2D/left-view clip
information file. A "dependent-view clip information file" is
a clip information file that should be associated with a
dependent-view stream file, and in the present embodiment
corresponds to a right-view clip information file and a depth
map clip information file. Like the 2D clip information file
shown in Fig. 11, the clip information files 3801 and 3 802
include clip information 3811 and 3821, stream attribute
information 3812 and 3822, and an entry map 3813 and 3823 . These
data structures are equivalent to the data structure in the 2D
clip information file shown in Figs. 11-13. The base-view clip
information file 3801 further contains 3D meta data 3814.
Additionally, conditions are placed on the stream attribute
information 3822 in the dependent-view clip information file
3802, and information is added to the entry map 3823.
[0205]
<<3D Meta Data>>
[0206]
Fig. 3 9A and Fig. 3 9B are schematic diagrams showing the
data structure of the 3D meta data 3814. The 3D meta data 3814
is information used for the cropping processes on the PG stream,
IG stream and the secondary video stream that are multiplexed
into the base-view stream file. "Cropping processes" refer to
the processes that add depth to the 2D video images played back
from each stream. During these processes, a left-view and
right-view pair is generated by shifting each 2D video image
in a horizontal direction. The amount of shifting corresponds
to the binocular parallax that generates the depth that should
be given to that 2D video image. In particular, the 3D meta data
3814 includes a value that represents the number of pixels of
the amount of shifting, i.e. the "offset value."
[0207]
As shown in Fig. 3 9A, the 3D meta data 3814 includes a
table 3901 for each PID in the PG stream, the IG stream, and
the secondary video stream. Each table 3901 generally includes
a plurality of pairs of PTS 3902 and offset values 3903. Each
PTS 3 902 shows a display time of a frame or a field in the PG
stream, the IG stream or the secondary video stream. The offset
value 3 903 represents the offset value for the frame or field
to which the PTS 3902 is allocated. The offset values 3902 may-
be negative values. A pair 3904 of the PTS 3902 and the offset
value 3903 is referred to as an "offset entry. " The valid section
of each offset entry ranges from the PTS of the offset entry
to the PTS of the subsequent offset entry. In the example in
Fig. 39A, the PTS of offset entry #1 is 180000, the PTS of offset
entry #2 is 270000, and the PTS of offset entry #3 is 360000.
In this case, as shown in Fig. 3 9B, an offset value of +5 in
the offset entry #1 is valid in an STC range 3904A of 180000
to 270000, and an offset value of +3 in the offset entry #2 is
valid in an STC range 3904B from 270000 to 360000.
[0208]
<>
[0209]
As was explained with regards to Fig. 34, the
dependent-view stream is compressed using the base-view stream.
Accordingly, the video stream attributes of the dependent-view
stream become equivalent to the base-view stream. Here, the
video stream attribute information for the base-view stream is
associated with PID=0xl011 in the stream attribute information
3812 in the base-view clip information file 3801. The video
stream attribute information for the dependent-view stream is
associated with PID=0xl012 in the stream attribute information
3822 in the dependent-view clip information file 3802.
Accordingly, the items shown in Fig. 12, i.e. the codec 1211,
resolution 1212, aspect ratio 1213, and frame rate 1214, match
between these two pieces of video stream attribute information.
If the codec type matches, then a reference relationship between
pictures in the base-view stream and the dependent-view stream
is established during coding, and thus each picture can be
decoded. If the resolution, aspect ratio, and frame rate all
match, then on-screen presentation of the left and right videos
can be synchronized. Therefore, these videos can be shown as
3D video images without making the viewer feel uncomfortable.
[0210]
<>
[0211]
Fig. 4 0A is a schematic diagram showing the data structure
of the entry map 3823 for the dependent-view clip information
file 3802 shown in Fig. 38B. As shown in Fig. 40A, the entry
map 3823 includes a table 4001 allocated to the dependent-view
stream. The table 4 001 includes an entry map header 4 011 and
a plurality of entry points 4012. The entry map header 4011
indicates the PID for the dependent-view stream allocated in
the table 4001, i.e. either 0x1012 or 0x1013 . In each entry point
4012, a pair of a PTS 4013 and an SPN 4014 is associated with
an EP_ID 4016. The PTS 4013 is equivalent to the PTS for the
top picture in one of the GOPs included in the dependent-view
stream. The SPN 4014 is equivalent to the top SPN of the source
packet group recorded by the top picture. In particular, among
the PTSs in the entry map 3813 in the base-view clip information
file 3801, the PTS 4013 for the entry point 4012 is equivalent
to the PTS for the entry point in the table allocated to the
base-view stream. In other words, whenever an entry point is
set to the top of a source packet that includes one of a set
of pictures included in the same 3D video image access unit,
an entry point is also set to the top of the source packet group
that includes the other picture.
[0212]
Fig. 41 is a schematic diagram showing an example of entry
points set in the base-view stream 4110 and the dependent-view
stream 4120. In the two video streams 4110 and 4120, GOPs that
are the same number from the top represent video for the same
playback period. As shown in Fig. 41, in the base-view stream
4110, entry points 4101B, 4103B, and 4105B are set to the top
of the first GOP #1, the third GOP #3, and the fifth GOP #5,
respectively. Accordingly, in the dependent-view stream 4120
as well, entry points 4101D, 4103D, and 4105D are set to the
top of the first GOP #1, the third GOP #3, and the fifth GOP
#5, respectively. In this case, when the 3D playback device
begins interrupt playback of 3D video images from, for example,
GOP #3, it can immediately acquire the top SPN of the source
packet group to be played back from the corresponding entry
points 4103B and 4103D. In this way, even during playback of
3D video images, it is possible to improve response speed for
processing that requires random access to the video stream, such
as interrupt playback or the like.
[0213]
As further shown by Fig. 40A, each entry point 4 012 further
contains an extent start flag 4015. The extent start flag 4015
indicates whether, from among the source packet group stored
by the dependent-view stream, the source packet specified by
the SPN 4 014 for the same entry point 4 012 is at the top of any
of the extents on the BD-ROM disc 101. Fig. 40B is a schematic
diagram showing the relationship between the SPN shown by an
entry point and an extent on the BD-ROM disc. For example, the
value of the extent start flag 4015 in the entry point EP_ID=0
is "1. " In this case, the value of the SPN 4014 in the same entry
point, "3, " indicates the SPN for the source packet at the top
of the first extent 4022 in the track 202 on the disc 101. The
value of the extent start flag 4 015 in the entry point EP_ID=1
is "0. " In this case, the value of the SPN 4014, "1500, " indicates
the SPN for a source packet recorded in a section other than
the top of the first extent 4022. The value of the extent start
flag 4015 in the entry point EP_ID=2 is "1." In this case, the
value of the SPN 4014, "3200," indicates the SPN for a source
packet at the top of the second extent 4023. Accordingly, the
SPN 4014 for entry points where EP_ID=0 and EP_ID=2 indicates
the range of the SPN for the source packet included in the first
extent 4 022. The 3D playback device is able to seek the size
of the first extent 4022 and the extent ATC time from the size
of this range. Furthermore, the 3D playback device can indicate
to the BD-ROM drive the range to be read without seeking, from
the file system, the range of the logical addresses of the first
extent. Consequently, the 3D playback device can promptly
execute the read process. Also, the entry point 4 012 is always
set, from among the source packet group stored by the
dependent-view stream, to the top of each extent on the BD-ROM
disc 101. Accordingly, the 3D playback device can promptly
execute a read process for any of the extents.
[0214]
A similar extent start flag is also included in the entry-
point of the entry map for the base-view clip information file
3801. Accordingly, the 3D playback device can acquire the size
of each extent stored by the base-view stream from the extent
start flag 4015. Therefore, the 3D playback device can promptly
read these extents from the BD-ROM disc 101.
[0215]
Additionally, each entry map header 4011 includes an
extent start type. The extent start type indicates which is
positioned first on the track 202 on the BD-ROM disc 101: an
extent storing the base-view stream, or an extent storing the
dependent-view stream. Accordingly, by referring to the extent
start type, the 3D playback device can easily determine which
of the extents storing one of the video streams it should first
request that the BD-ROM read.
[0216]

[0217]
The following describes the structure necessary for the
playback device 102 to play back 3D video image content from
the above-described BD-ROM disc 101, i.e. the structure
necessary for a 3D playback device. The fundamental part of this
structure is identical to the 2D playback device shown in Figs.
18 to 20. Therefore, the following is a description of sections
of the structure of the 2D playback device that are enlarged
or modified, incorporating by reference the afore-mentioned
description of the 2D playback device for the fundamental parts.
Regarding the playback processing of 2D video images in
accordance with the 2D playlist files that specify the playback
path of the 2D video images, i.e. the playback processing of
the 2D playlist, the 3D playback device has the same structure
as the 2D playback device. The details thereof are incorporated
in the following by reference. The following describes the
playback processing of 3D video images in accordance with the
3D playlist files that define the playback path of the 3D video
images, i.e. 3D playlist playback processing.
[0218]
Fig. 42 is a functional block diagram of the 3D playback
device 4200 . The 3D playback device 4200 includes a BD-ROM drive
4201, a playback unit 4200A, and a control unit 4200B. The
playback unit 4200A includes a switch 4212, a first read buffer
4221, a second read buffer 4222, a system target decoder 4203,
and a plane adder 4210 . The control unit 4200B includes a dynamic
scenario memory 4204, a static scenario memory 4205, a program
execution unit 4206, a playback control unit 4207, a player
variable storage unit 4208, and a user event processing unit
4209. The playback unit 4200A and the control unit 4200B are
mounted on a different integrated circuit. Alternatively, these
units may be mounted on a single integrated circuit. In
particular, the dynamic scenario memory 4204, the static
scenario memory 4205, the program execution unit 4206, and the
user event processing unit 4209 have an identical structure with
the 2D playback device shown in Fig. 18. Accordingly, details
thereof are incorporated by reference to the above explanation
of the 2D playback device.
[0219]
The BD-ROM drive 4201 includes elements identical to the
BD-ROM drive 1801 in the 2D playback device shown in Fig. 18.
With use of these elements, the BD-ROM drive 4201 reads data
from the BD-ROM disc 101 in accordance with a request from the
playback control unit 4207. Among this data, the AV stream file
is selectively sent from the BD-ROM drive 4201 to either the
first read buffer 4221 or the second read buffer 4222.
Specifically, the BD-ROM drive 42 01 first alternately receives
read requests for the base-view stream file and the
dependent-view stream file from the playback control unit 4207
in units of extents . In accordance with each request, the BD-ROM
drive 42 01 alternately sends extents belonging to each AV stream
file to the switch 4212. From among these extents, the switch
4212 sends those belonging to the base-view stream file to the
first read buffer 4221 and those belonging to the dependent-view
stream file to the second read buffer 4222. During playback of
3D video images, both the base-view stream file and the
dependent-view stream file need to be sent in this way in
parallel to the read buffers 4221 and 4222. Accordingly, the
BD-ROM drive 4201 needs to have a faster access speed than the
BD-ROM drive 1801 in the 2D playback device.
[0220]
The first read buffer 4221 and the second read buffer 4222
are buffer memories that use a memory element in the playback
unit 4200A. In particular, different areas in a single memory
element are used as the read buffers 4221 and 4222.
Alternatively, different memory elements may be used as the read
buffers 4221 and 4222. The first read buffer 4221 stores the
base-view stream file transferred from the BD-ROM drive 4201.
The second read buffer 4222 stores the dependent-view stream
file transferred from the BD-ROM drive 42 01.
[0221]
Receiving a request from, for example, the program
execution unit 42 06 for performing 3D playlist playback
processing, the playback control unit 4207 first refers to the
3D playlist file stored in the static scenario memory 4205. For
example, as shown in Figs. 30 and 31, the 3D playlist files 243B
and 243C specify main paths 3004 and 3014 and sub-paths 3005
and 3105. Subsequently, the playback control unit 4207 reads
pieces of playitem information #1-3 in order from the main paths
3004 and 3005, and uses these pieces of information to specify
base-view stream file sections LCL1-3 in order. In parallel,
the playback control unit 4207 further reads pieces of
sub-playitem information #1-3 in order from the sub-paths 3005
and 3105, and uses these pieces of information to specify
dependent-view stream file sections RCL1-3 and DCL1-3 in order.
Then, the playback control unit 4207 accesses the static
scenario memory 4205 and, from among the clip information files
associated with each section, refers to the entry maps 1103 and
3823 shown in Figs. 13Aand40A. In this way, the playback control
unit 4207 determines whether the extent at the playback start
point belongs to the base-view stream file or the dependent-view
stream file, based on the extent start type written in the entry
map headers 13 01 and 4 011. From these results, the playback
control unit 4207 determines an initial position of the switch
4212. Subsequently, the playback control unit 4207 makes a
request to the BD-ROM drive 4201 to alternately read the extents
of the base-view stream file and the dependent-view stream file
from the playback start point. In particular, out of the two
stream files, the request specifies to start reading the AV clip
indicated by the above-mentioned determination results first.
After the BD-ROM drive 4201 transfers the whole first extent
from the BD-ROM drive 4201 to either of the read buffers 4221
or 4222, the extent is further transferred from the read buffer
4221 or 4222 to the system target decoder 4203. In addition to
such processing, the playback control unit 4207 reads the 3D
meta data 3814 shown in Figs. 3 9A and 3 9B from the base-view
clip information file stored in the static scenario memory 4205,
and transfers the 3D meta data 3814 to the plane adder 4210.
[0222]
First, the system target decoder 4203 reads source
packets alternately from the base-view stream file stored in
the first read buffer 4221 and the dependent-view stream file
stored in the second read buffer 4222. Then, the system target
decoder 4203 demultiplexes these read source packets to
separate the elementary streams therein. Subsequently, the
system target decoder 4203 decodes each of the elementary
streams separately. Furthermore, the system target decoder 4203
writes a decoded base-view stream, dependent-view stream,
secondary video stream, IG stream, and PG stream into built-in
dedicated memories, namely a 2D/left video plane memory, aright
video plane memory, a sub-video plane memory, an IG plane memory,
and a PG plane memory, respectively. The details of the system
target decoder 4203 are described below.
[0223]
As in the 2D playback device, the player variable storage
unit 4208 includes the SPRM shown in Fig. 19. However, any two
of the SPRM(24) - (32) that were reserved in Fig. 19 include the
above-described two types of flags (A) and (B). For example,
the SPRM(24) includes flag (A) , and the SPRM(25) includes flag
(B). When the SPRM(24) is "0," the playback device 102 only
supports playback of 2D video images, and when it is "1," the
playback device 102 also supports 3D video image playback. When
the SPRM(25) is "0," the 3D video image playback mode of the
playback device 102 is L/R mode, and when it is "1, " the 3D video
image playback mode is depth mode.
[0224]
The plane adder 4210 receives the 2D/lef t video plane data,
right video plane data, sub-video plane data, IG plane data,
PG plane data, and image plane data from the system target
decoder 4203 and superimposes all of these data to compose one
video frame or field. The composed video frame or field is sent
to the display device 103 and displayed on the screen.
[0225]
Fig. 43 is a functional block diagram of the plane adder
4210 . The plane adder 4210 includes a parallax video generation
unit 4320, a switch 4310, four adders 4311-4314, and four
cropping processing units 4321-4324.
[0226]
The parallax video generation unit 4320 receives 2D/left
video plane data 43 01 and right video plane data 43 02 from the
system target decoder 4203. When the 3D video image playback
mode of the playback device 102 is set to L/R mode, the 2D/left
video plane data 43 01 represents a video frame or field in the
left-view stream, and the right video plane data 43 02 represents
a video frame or field in the right-view stream. In this case,
the parallax video generation unit 4320 sends the 2D/lef t video
plane data 4301 and the right video plane data 43 02 as they are
to the switch 4310. On the other hand, when the 3D video image
playback mode is set to depth mode, the 2D/lef t video plane data
4301 represents a video frame or field in the 2D video stream,
and the right video plane data 43 02 represents a depth map for
that video frame or field. In this case, the parallax video
generation unit 4320 first calculates the binocular parallax
for each element in the video frame or field using the depth
map. Next, the parallax video generation unit 4320 shifts the
display position of each element in the video frame or field
to the left or right according to the calculated binocular
parallax to construct the video frames or fields representing
the left-view and the right-view. The parallax video generation
unit 4320 further sends left and right video frames or fields
to the switch 4310 as the 2D/lef t video plane data 4301 and the
right video plane data 4302.
[0227]
The switch 4310 sends 2D/left video plane data 4301 and
right video plane data 43 02 with equivalent PTSs to the first
adder 4311 in that order.
[02281
The first adder 4311 alternately receives the 2D/left
video plane data 4301 and the right video plane data 4302 from
the switch 4310. It also alternately receives a pair of
sub-video plane data pieces representing the left-view and the
right-view from the first cropping processing unit 4321. Next,
the first adder 4311 superimposes the sub-video plane data
representing the left-view on the 2D/left video plane data 43 01
and superimposes the sub-video plane data representing the
right-view on the right video plane data 4302. After
superimposing is complete, the left and right video plane data
are alternately sent to the second adder 4312.
[0229]
The second adder 4312 alternately receives left and right
video plane data from the first adder 4311. It also alternately
receives a pair of PG plane data pieces representing the
left-view and the right-view from the second cropping
processing unit 4322. Next, the second adder 4312 superimposes
the PG plane data representing the left-view on the plane data
representing the left-view and superimposes the PG plane data
representing the right-view on the plane data representing the
right-view. After superimposing is complete, the left and right
video plane data are alternately sent to the third adder 4313.
[0230]
The third adder 4313 alternately receives left and right
video plane data from the second adder 4312 . It also alternately
receives a pair of IG plane data pieces representing the
left-view and the right-view from the third cropping processing
unit 4323. Next, the third adder 4313 superimposes the IG plane
data representing the left-view on the plane data representing
the left-view and superimposes the IG plane data representing
the right-view on the plane data representing the right-view.
After superimposing is complete, the left and right video plane
data are alternately sent to the fourth adder 4314.
[0231]
The fourth adder 4314 alternately receives left and right
video plane data from the third adder 4313 . It also alternately
receives a pair of image plane data pieces representing the
left-view and the right-view from the fourth cropping
processing unit 4324. Next, the fourth adder 4314 superimposes
the image plane data representing the left-view on the plane
data representing the left-view and superimposes the image
plane data representing the right-view on the plane data
representing the right-view. After superimposing is complete,
the left and right video plane data are alternately sent to the
display device 103.
[0232]
In this way, as shown by the arrows 4300 in Fig. 43, the
2D/left video plane data 4301, right video plane data 4302,
sub-video plane data 4303, IG plane data 4304, PG plane data
4305 and image plane data 4306 are superimposed in this order.
By such combination processes, video images indicated by the
sets of plane data are displayed on the screen of the display
device 103 such that the 2D/left video plane or the right video
plane, the sub-video plane, the IG plane, the PG plane, and the
image plane are overlaid in this order.
[0233]
The first cropping processing unit 4321 uses 3D meta data
3814 to perform cropping processing on the sub-video plane data
4303. In this way, the sub-video plane data 4303 is transformed
into a pair of sub-video plane data pieces that represent a
left-view and a right view and are alternately transmitted.
[0234]
The second cropping processing unit 4322 uses 3D meta data
3814 to perform cropping processing on the PG plane data 4304.
In this way, the PG plane data 43 04 is transformed into a pair
of PG plane data pieces that represent a left-view and a right
view and are alternately transmitted.
[0235]
The third cropping processing unit 4323 uses 3D meta data
3814 to perform cropping processing on the IG plane data 4305.
In this way, the IG plane data 43 05 is transformed into a pair
of IG plane data pieces that represent a left-view and a right
view and are alternately transmitted.
[0236]
Figs. 44A and 44B are schematic diagrams showing the
cropping processing by each of the first through third cropping
processing units 4321-4323. Figs. 44A and 44B show an example
of cropping processing performed on the PG plane data 4304 by
the second cropping processing unit 4322. First, the second
cropping processing unit 4322 searches among the 3D meta data
3814 shown in Figs. 39A and 39B for the table 3901 that is
associated with the PID = 0x1200 of the PG stream. Then, the
second cropping processing unit 4322 searches within the table
3 901 for an offset entry 3 904 that is currently valid, and
acquires the offset value 3903. If video plane data to be
superimposed on the PG plane data 4304 is 2D/left video plane
data 43 01, the second cropping processing unit 4322 shifts the
PG plane data 43 04 in a horizontal direction with respect to
the 2D/left video plane data 4301 by a number of pixels 4401L
equivalent to the acquired offset value, as shown in Fig. 44A.
At that time, if the offset value is positive, the second
cropping processing unit 4322 shifts the PG plane data 4304 to
the right, and if the offset value is negative, it shifts the
data to the left. Subsequently, the second cropping processing
unit 4322 removes (crops) an area 4402L of the PG plane data
43 04 that extends out of the range of the 2D/left video plane
data 43 01 and superimposes the remaining area 44 03L on the
2D/left video plane data 4301. On the other hand, if the video
plane data on which data is superimposed is the right video plane
data 43 02, the second cropping processing unit 4322 shifts the
PG plane data 43 04 in a horizontal direction with respect to
the right video plane data 4302 by the number of pixels 4401R
equivalent to the acquired offset value, as shown in Fig. 44B.
At that time, if the offset value is positive, the second
cropping processing unit 4322 shifts the PG plane data 4304 to
the left, and if the offset value is negative, it shifts the
data to the right, opposite from the case shown in Fig. 44A.
Subsequently, the second cropping processing unit 4322 removes
(crops) an area 4402R of the PG plane data 4304 that extends
out of the range of the right video plane data 43 02 and
superimposes the remaining area 44 03R on the right video plane
data 43 02, as in the case shown in Fig. 44A. The first cropping
processing unit 4321 similarly performs cropping processing on
the sub-video plane data 4303, and the third cropping processing
unit 4323 similarly performs cropping processing on the IG plane
data 4305.
[0237]
Figs. 45A and 45B are schematic diagrams respectively
showing a left-view and a right-view superimposed by cropping
processes shown in Figs. 44Aand 44C, and Fig. 45C is a schematic
diagram showing the resulting 3D video image that the viewer
is made to perceive. In the left-view, as shown in Fig. 45A,
the PG plane 4502 is shifted to the right with respect to the
left video plane 4501L by an offset value 4401L, and thus the
area 4503L on the left side of the PG plane 4502 appears
superimposed on the left video plane 4501L. As a result, the
2D video image 4 504, i.e. subtitles for the PG plane 4502,
appears shifted to the right from its original position by the
offset value 4401L. Conversely, in the right view, as shown in
Fig. 45B, the PG plane 4502 is shifted to the left with respect
to the right video plane 4501R by an offset value 4401R, and
thus the area 4503R on the right side of the PG plane 4502 appears
superimposed on the right video plane 4501R. As a result, the
2D video image 4504, i.e. subtitles for the PG plane 4502,
appears shifted to the left from its original position by the
offset value 4401R. Consequently, as shown in Fig. 45C, the 3D
video image 4504 of the subtitles appears to be closer to a viewer
4505 than the video plane 4506 . Clipping processing is thus used
to create a pair of left-view and right-view plane data pieces
from a single plane data piece. In this way, it is possible to
play back a parallax video. That is, depth can be added to a
monoscopic image. In particular, it is possible to allow the
viewer to see the monoscopic image as popping out from the
screen.
[0238]
Once again referring to Fig. 43 , the image plane data 4306
is graphics data sent from the program execution unit 42 06 to
the system target decoder 4203 and decoded by the system target
decoder 4203 . The graphics data is raster data such as JPEG data
or PNG data, and shows a GUI graphics part such as a menu. The
fourth cropping processing unit 4324 performs the cropping
processing on the image plane data 4306 as do the other cropping
processing units 4321-4323. However, unlike the other cropping
processing units 4321-4323, the fourth cropping processing unit
4324 reads the offset value from offset information specified
by a program API 4330 instead of from the 3D meta data 3814.
Here, the program API 433 0 is executed by the program execution
unit 4206. In this way, the offset information corresponding
to the depth of the image representing the graphics data is
calculated and output to the fourth cropping processing unit
4324.
[0239]
In addition to the above-stated processing, the plane
adder 4210 performs processing to convert an output format of
the plane data combined by the four plane adders 4311-4314 into
a format that complies with the 3D display method adopted in
a device such as the display device 103 to which the data is
outputted. If an alternate-frame sequencing method is adopted
in the device, for example, the plane adder 4210 outputs the
superimposed plane data pieces as one frame or one field. On
the other hand, if a method that uses a lenticular lens is adopted
in the device, the plane adder 4210 superimposes the left and
right plane data pieces as one frame or one field of video data
with use of the built-in buffer memory. More specifically, the
plane adder 4210 temporarily stores and holds in the buffer
memory the left video plane data that has been superimposed
first. Subsequently, the plane adder 4210 superimposes the
right video plane data, and further superimposes the resultant
data with the left video plane data held in the buffer memory.
In the superimposing, the left and right plane data pieces are
each divided, in a vertical direction, into small rectangular
areas that are long and thin, and the small rectangular areas
are arranged alternately in the horizontal direction in one
frame or one field so as to re-constitute the frame or the field.
In this way, the left and right plane data pieces are combined
into one video frame or field, which the plane adder 4210 then
outputs to the appropriate device.
[0240]
<>
[0241]
Fig. 46 is a functional block diagram of the system target
decoder 4203. The structural elements shown in Fig. 46 differ
from the 2D playback device shown in Fig. 20 in the following
two points: 1) the input channel from the read buffer to the
main video decoder is doubled, and 2) the main video decoder
is a 3D video image decoder that can alternately decode a
base-view stream and a dependent-view stream. On the other hand,
the secondary video decoder, IG decoder, PG decoder, primary-
audio decoder, secondary audio decoder, audio mixer, image
processor, and plane memories are similar to those included in
the 2D playback device shown in Fig. 20. Accordingly, among the
structural elements shown in Fig. 46, those differing from the
structural elements shown in Fig. 20 are described below, and
details about similar structural elements are incorporated by
reference to the explanation for Fig. 20.
[0242]
The first source depacketizer 4611 reads source packets
from the first read buffer 4221, fetches TS packets included
in the source packets, and transmits the TS packets to the first
PID filter 4613. The second source depacketizer 4612 reads
source packets from the second read buffer 4222, fetches TS
packets included in the source packets, and transmits the TS
packets to the second PID filter 4614. Each of the source
depacketizers 4611 and 4612 further adjusts the time of
transferring the TS packets, in accordance with the ATS of the
source packets. This adjustment is made in the same manner as
made by the source depacketizer 2010 shown in Fig. 20. Thus,
the detailed explanation of the adjustment provided for Fig.
20 is incorporated in the following by reference.
[0243]
The first PID filter 4 613 compares the PIDs of the TS
packets received from the first source depacketizer 4611 with
the PIDs previously designated by the playback control unit 4207 .
When two PIDs match, the first PID filter 4 613 transfers the
corresponding TS packet to the TB(1) 4601, the secondary video
decoder, the IG decoder, the PG decoder, the audio decoder or
the secondary audio decoder of the 3D video image decoder 4315,
depending on the PID. Similarly, the second PID filter 4614
transfers the TS packets, received from the second source
depacketizer 4612, to the decoders, according to the PID of each
TPS packet. Here, as shown in Figs. 33A and 33B, the
dependent-view stream file generally includes only the
dependent-view stream. Thus, for the 3D playlist playback, the
second PID filter 4614 transfers the TS packets mainly to the
TB(2) 4608 of the 3D video decoder 4615.
[0244]
As shown in Fig. 46, the 3D video decoder 4615 includes
a TB(1) 4601, an MB(1) 4602, an EB(1) 4603, a TB(2) 4608, an
MB (2) 4609, an EB(2) 4610, a buffer switch 4606, a compressed
video decoder 4604, a DPB 4605, and a picture switch 4607. The
TB(1) 4601, the MB (1) 4602, theEB(l) 4603, theTB(2) 4608, the
MB (2) 4609, the EB(2) 4610 and the DPB 4605 are all buffer
memories, each of which uses an area of the memory elements
included in the 3D video decoder 4615. Note that some or all
of these buffer memories may be separated on different memory
elements.
[0245]
The TB(1) 4601 receives TS packets that include a
base-view stream from the first PID filter 4613, and stores the
TS packets as they are. The MB(1) 4602 stores PES packets
reconstructed from the TS packets stored in the TB(1) 4601. The
TS headers of the TS packets are removed at this point. The EB (1)
4603 extracts and stores encoded video access units from the
PES packets stored in the MB(1) 4602. The PES headers of the
PES packets are removed at this point.
[0246]
The TB(2) 4608 receives TS packets that include a
dependent-view stream from the second PID filter 4614, and
stores the TS packets as they are. The MB(2) 4609 stores PES
packets reconstructed from the TS packets stored in the TB(2)
4608 . The TS headers of the TS packets are removed at this point.
The EB(2) 4610 extracts and stores encoded video access units
from the PES packets stored in the MB (2) 4609. The PES headers
of the PES packets are removed at this point.
[0247]
The buffer switch 4606 transfers the video access units
stored in the EB(1) 4603 and the EB(2) 4610 to the compressed
video decoder 4604 at the times of the DTSs indicated by the
original TS packets. Here, as shown in Figs. 35A and 35B, the
DTSs for a pair of pictures belonging to the same 3D video image
access unit in the base-view stream 3501 and the dependent-view
stream 3502 are equivalent. Accordingly, from among the pairs
of video access units with equivalent DTSs stored by the EB(1)
4603 and the EB(2) 4610, the buffer switch 4606 first sends a
pair stored in the EB(1) 4603 to the compressed video decoder
4604. Additionally, the buffer switch 4606 may receive the
decode switch information 3601, included in the corresponding
video access unit 3600 shown in Fig. 36, back from the compressed
video decoder 4604. If this is the case, the buffer switch 4606
can determine if it should first transfer the next video access
unit to the EB(1) 4603 or to the EB(2) 4610 by using the decode
switch information 3601.
[0248]
The compressed video decoder 4604 decodes each video
access unit transferred from the buffer switch 4606, at the time
of the DTS of the corresponding TS packet. Here, the compressed
video decoder 4604 uses different decoding methods according
to the encoding format (e.g. MPEG-2, MPEG-4 AVC and VC1) of the
compressed pictures contained in the video access unit, and
according to the stream attribute. The compressed video decoder
4604 further transfers the decoded pictures, namely the video
frames or fields, to the DPB 4 605.
[0249]
The DPB 4605 temporarily holds the decoded pictures. The
compressed video decoder 4604 refers to the decoded pictures
held by the DPB 4 605 to decode the P pictures and the B pictures.
The DPB 4605 further transfers each of the pictures to the
picture switch 4607 at the time of the PTS of the original TS
packet.
[0250]
The picture switch 4607 writes the picture transferred
from the compressed video decoder 4604 to either the 2D/left
video plane memory 4620 or the right video plane memory 4621.
When the picture belongs to the 2D/left-view stream, the picture
switch 4607 writes the picture to the 2D/lef t video plane memory
4620, and when it belongs to the right-view stream, the picture
switch 4607 writes it to the right video plane memory 4621.
[0251]

[0252]
Next, an explanation is provided for the physical
arrangement of AV stream files when recording a pair of AV stream
files representing 3D video images, i.e. a base-view stream file
and a dependent-view stream file, on the BD-ROM disc 101, such
that the arrangement is useful for seamless playback of 3D video
images.
[0253]
During 3D video image playback, the 3D playback device
has to process in parallel a base-view stream file and a
dependent-view stream file whose playback periods match. Figs.
47A and 47B are schematic diagrams showing the arrangement on
the disc 101 of the extents belonging to both AV stream files.
[0254]
Assume that, as shown in Fig. 47A, the entirety of a
base-view stream file is sequentially recorded on the disc 101
as a single extent 4 701, after which the entirety of a
dependent-view stream file is sequentially recorded as a single
extent 4702 . When the 3D playback device processes the AV stream
files 4701 and 4702 in parallel, the capacity of the buffer
useable for the processing is generally limited. Therefore, as
shown by the arrows (1) - (4) in Fig. 47A, the playback path of
the 3D video images proceeds alternately for portions of the
extents 4701 and 4702. As a result, each time the extent to be
read switches, a jump occurs, as shown by the dotted lines in
Fig. 47A. In particular, the jump distance is approximately the
same as the length of each entire AV stream file. This makes
it difficult for the read process of each AV stream file by the
BD-ROM drive to keep up with the decoding process by the 3D video
image decoder, and thus it is difficult to ensure continual
seamless playback.
[0255]
In Embodiment 1, as shown in Fig. 47B, the extents for
both AV stream files are recorded in an interleaved arrangement.
Specifically, one base-view stream file is divided into a
plurality of extents 4701A, 4701B, etc., and one dependent-view
stream file is divided into a plurality of extents 4702A, 4702B,
etc. Furthermore, the extents for both AV stream files are
arranged alternately on the disc 101. In this case, as shown
by the arrows (l)-(4) in Fig. 47B, the playback path for the
3D video images proceeds in order starting with extents 4701A,
4701B, 4702A, 4702B, etc. In this way, the 3D playback device
does not have to jump the entirety of the AV stream file and
can read both AV stream files alternately in units of extents.
This is useful for seamless playback of 3D video images.
[0256]
<>
[0257]
The playback time for the video stream for each extent
fulfills the conditions described below. Figs. 48A and 48B are
schematic diagrams showing the relationship between this
playback time and the playback path.
[0258]
Assume that, as shown in Fig. 48A, an extent 4801 for the
base-view stream file and an extent 4 8 02 for the dependent-view
stream file are consecutive. The playback time of the base-view
stream included in the extent 4801 is four seconds, and the
playback time of the dependent-view stream included in the
extent 4802 is one second. Here, the playback time of the
base-view stream required for decoding of the dependent-view
stream is equivalent to the playback time of the dependent-view
stream. Accordingly, to economize the buffer capacity within
the 3D playback device, it is preferable that the 3D video image
playback path proceed alternately between the extents 4801 and
4802 for each AV stream file for the same playback time, for
example one second, as shown by the arrow 4810 in Fig. 4 8A.
However, in that case, as shown by the dotted line in Fig. 48A,
a jump occurs during the reading of the extent 4801, whose video
stream has a long playback time.
[0259]
In Embodiment 1, as shown in Fig. 48A, the sections of
each AV stream file with matching playback periods are included
in a pair of consecutive extents. In particular, the playback
times of the video streams are equivalent between these extents.
Specifically, the playback time of the video streams in the top
extent 4801A in the base-view stream file and the top extent
4802A in the dependent-view stream file are both equivalent to
one second, and the playback time of the video stream in the
second extent in each AV stream file, 4801B and 4802B, are both
equivalent to 0.7 seconds. The 3D video image playback path
arranged in this way, as shown by the arrow 4820 in Fig. 48B,
proceeds in order starting with the extents 4801A, 4802A, 4801B,
4802B, etc. Furthermore, no jump occurs during the reading of
any of the extents . Accordingly, the 3D playback device can read
extents continuously without causing any jumps. This is useful
for seamless playback of 3D video images.
[0260]
<>
[0261]
The top of each extent belonging to the base-view stream
file includes an I picture for the base-view stream, and the
top of each extent belonging to the dependent-view stream file
includes a P picture for the dependent-view stream. This P
picture, as shown in Fig. 34, is compressed using the I picture
in the base-view stream as a reference picture. In this case,
the 3D playback device can specify the size of each extent using
the entry point in the clip information file. Therefore, the
3D playback device can simplify the processing to alternately
read extents of both AV stream files from the BD-ROM disc 101.
[0262]
<>
[0263]
Fig. 49 is a schematic diagram showing the position on
a BD-ROM disc of the extent groups belonging to a 2D/left-view
stream file, a right-view stream file, and a depth map stream
file. The AV stream files correspond to the three types of AV
stream files 245A-245C shown in Fig. 2. As shown in Fig. 49,
these groups of extents are recorded in an interleaved
arrangement similar to the arrangement shown in Fig. 47B.
Specifically, the 2D/left-view stream file is first divided
into a plurality of extents L1, L2, L3, etc., the right-view
stream file is divided into a plurality of extents R1, R2, R3,
etc. , and the depth map stream file is divided into a plurality
of extents D1, D2, D3, etc. Hereinafter, the plurality of
extents L1, L2, L3, etc. belonging to the 2D/left-view stream
file are referred to as "left-view extents," the plurality of
extents R1, R2, R3, etc. belonging to the right-view stream file
are referred to as "right-view extents," and the plurality of
extents D1, D2, D3, etc. belonging to the depth map stream file
are referred to as "depth map extents." The left-view extents
L1-3, the right-view extents R1-3, and the depth map extents
D1-3 are alternately arranged one at a time. In this interleaved
arrangement, sections of each AV stream file with matching
playback periods are further included individually in three
consecutive extents, similar to the arrangement shown in Fig.
48B. In particular, between these extents, the playback times
of the video streams are equivalent. For example, the top
left-view extent L1, the top right-view extent Rl, and the top
depth map extent Dl are arranged consecutively. The sections
of the AV stream files included in the extents L1, R1, and D1
have matching playback periods, and the playback time of the
video streams is equivalent. The same is true for the next three
consecutive extents, L2, R2, and D2.
[0264]
As shown in Fig. 49, for each three consecutive extents,
i.e. for a combination of a left-view extent, a right-view
extent, and a depth map extent with matching playback periods,
the arrangement order is fixed. In particular, extents are
arranged starting with the smallest extent. For example, the
picture R1 included in the top right-view extent is compressed
using the picture included in the top left-view extent L1 as
a reference picture, as shown in Fig. 34. On the other hand,
the amount of data per pixel in the depth map, i.e. the number
of bits of the depth value, is in general smaller than the amount
of data per pixel of the picture included in the 2D/left-view
stream, i.e. the number of bits of the chromatic coordinate
value and the a value. Furthermore, as shown in Figs. 4 and 33,
unlike the dependent-view stream file, the base-view stream
file includes elementary streams, such as a primary audio stream,
in addition to the primary video stream. Therefore, the size
of the top right-view extent Rl, Sext2[1] / and the size of the
top depth map extent D1, Sext3[1] / are both generally smaller
than the size of the top left-view extent L1, Sext1[1] : Sext2 [1]
>
[0274]
Fig. 50 is a schematic diagram showing (i) the entry points
set for a combination Dn, Rn, and Ln of the nth extent among
the interleaved arrangement shown in Fig. 49 and (ii) the
playback path during interrupt playback. As shown in Fig. 50,
four entry points are set in each of the extents Dn, Rn, and
Ln. Similar to the entry points shown in Fig. 41, the entry points
in the extents Dn, Rn, and Ln located at the same position when
counted from their respective tops have the same PTS. For
example, the PTS for the top entry point EPR1 in the right-view
extent Rn is equivalent to the PTS for the top entry point EPL1
in the left-view extent Ln, and the PTS for the second entry-
point EPR2 in the right-view extent Rn is equivalent to the PTS
for the second entry point EPL2 in the left-view extent Ln.
[0275]
Fig. 50 further shows the types of playback paths 5001
and 5002 that can be chosen when interrupt playback of 3D video
images is performed from the third entry point EPR3 in the
right-view extent Rn. Both playback paths 5001 and 5002 first
traverse the right-view extent Rn from the third entry point
EPR3 to the end. In the first playback path 5001, a jump JEP
occurs from the end of the right-view extent Rn to the third
entry point EPL3 of the left-view extent Ln. Subsequently, the
first playback path 5001 traverses the left-view extent Ln
starting from the third entry point EPL3. On the other hand,
the second playback path 5002 proceeds continuously from the
end of the right-view extent Rn to the top of the left-view extent
Ln and further proceeds along the left-view extent Ln without
a jump. However, the source packets included from the top entry
point EPL1 through the third entry point EPL3 in the left-view
extent Ln are discarded without being decoded.
[0276]
One of the two playback paths 5001 and 5002 is chosen as
follows. First, the jump distance of the jump JEP, i.e. the
number of sectors from the top entry point EPL1 to the third
entry point EPL3 in the left-view extent Ln, is calculated from
the entry map 3823 in the base-view clip information file 3801
shown in Fig. 38A. Next, according to the regulation shown in
Fig. 24, the jump time Tjump_EP corresponding to the jump distance
is determined to be the maximum jump time for the jump JEP. On
the other hand, the time Tread-EP required for the BD-ROM drive
42 01 to read the left-view extent Ln from the top entry point
EPL1 through the third entry point EPL3 and transfer data to
the first read buffer 4221 is calculated by multiplying the data
transfer rate from the BD-ROM drive 4201 to the first read buffer
4221 by the above-mentioned number of sectors. Next, this time
Tread-EP is compared with the jump time Tjump-EP- When Tread-EP is
shorter than TjUmp-EP/ i.e. Tread-EP < Tjump-EP, then the first
playback path 5001 is selected. Conversely, when Tread-EP is
longer than Tjump-Ep, i.e. Tread-EP > Tjump-Ep, then the second
playback path 5002 is selected. In this way, decoding of the
range from the top entry point EPL1 through the third entry point
EPL3 in the left-view extent Ln is skipped as quickly as possible.
As a result, upon interrupt playback, the time until video
images are actually played back can be reduced.
[0277]
<>
[0278]
Here, as an assumption for the following explanation, the
data transfer rate of the playback process system in the 3D
playback device is defined. Fig. 51 is a schematic diagram
showing the playback process system in the 3D playback system
shown in Fig. 42. This playback process system converts the
base-view stream file and the dependent-view stream file read
by the BD-ROM disc 101 into 3D video image data VD and audio
data AD. As shown in Fig. 51, the BD-ROM drive 4201 alternately
reads both AV stream files in units of extents and stores them
alternately in the first read buffer 4221 and the second read
buffer 4222 at a rate RUd-3D (bits/second) . This rate RUd-3D is
referred to as the "3D video image read rate. " The 3D video image
read rate RUd-3D is, for example, 72 Mbps. The system target
decoder 4203 reads source packets from the first read buffer
4221 at a mean transfer rate of Rext1-3D (bits/second) . This mean
transfer rate of Rext1-3D is referred to as the "left-view transfer
rate. " The left-view transfer rate Rext1-3D is generally different
for each extent. The maximum value Rmax1-3D of the left-view
transfer rate Rext1-3D is equivalent to 192/188 times the system
rate for the 2D/left-view stream file. On the other hand, the
system target decoder 4203 reads source packets that include
a section of a right-view stream file from the second read buffer
4222 at a first mean transfer rate Rext2 (bits/second) and reads
source packets that include a section of the depth map stream
file at a second mean transfer rate ReXt3 (bits/second) . The first
mean transfer rate Rext2 is referred to as the "right-view
transfer rate," and the second mean transfer rate Rext3 is
referred to as the "depth map transfer rate." Both transfer
rates Rext2 and Rext3 are generally different for each extent.
The maximum value Rmax2 of the right-view transfer rate Rext2 is
equivalent to 192/188 times the system rate for the right-view
stream file, and the maximum value RmaX3 of the depth map transfer
rate Rext3 is equivalent to 192/188 times the system rate for
the depth map stream file. Note that the system rate, as shown
in Figs. 11 and 38, is recorded in the clip information in the
clip information file. While the BD-ROM drive 4201 is reading
an extent from the disc 101, in order to prevent underflow in
the read buffers 4221 and 4222 caused by the decoding process
in the system target decoder, the 3D video image read rate Rua-3D
should be equal to or greater than each of the maximum values
of the transfer rates, Rmax1-3D/ Rmax2, and Rmax3, as in Equation
2.
[0279]
Equation 2

[0280]
The playback process system in the 3D playback device
shown in Fig. 51 is further assumed to be useable as a playback
process system to play back 2D video images from the BD-ROM disc
101. In playback of 2D video images, the BD-ROM drive 42 01 only
stores the left-view extents in the first read buffer 4221 at
a rate RUd-2D (bits/second) . This rate RUd-2D is referred to as
the "2D video image read rate." The 2D video image read rate
Rud-2D is, for example, 54 Mbps. The system target decoder 4203
reads source packets from the first read buffer 4221 at a mean
transfer rate Rext1-2D (bits/second) . This mean transfer rate
Rext1-2D is referred to as the "2D video image transfer rate."
The 2D video image transfer rate Rext1-2D is generally different
for each extent. The maximum value Rmax1-2D of the 2D video image
transfer rate Rext1-2D is equivalent to 192/188 times the system
rate for the 2D/left-view stream file. While the BD-ROM drive
4201 is reading an extent from the disc 101, in order to prevent
underflow in the first read buffer 4221 caused by the decoding
process in the system target decoder, the 2D video image read
rate Rud-2d should be higher than the maximum value of the transfer
rate, Rmax1-2D/ as in Equation 3.
[0281]
Equation 3

[0282]
<>
[0283]
As shown in Fig. 4 9, in the playback path 4 901 for 2D video
images, jumps J2d1/ J2d2, etc., occur respectively in each
recording area of the dependent view extents D2, R2, D3, R3,
etc. On the other hand, data transfer from the first read buffer
4221 to the system target decoder 4203 is continuous.
Accordingly, for the 2D playback device to seamlessly play back
2D video images, a sufficient amount of data should be
accumulated in the first read buffer 4221 during reading of
extents, so that underflow will not occur in the first read
buffer 4221 due to data transfer to the system target decoder
4203 while reading of extents is suspended. In other words, as
explained with reference to Figs. 22-24, the size of each
base-view extent should be at least equivalent to the size of
the smallest extent, and the gap between base-view extents
should be smaller than the maximum jump distance.
[0284]
Fig. 52 is a graph showing the change of the accumulated
data amount Dl in the first read buffer 4221 when the 2D playback
device plays back 2D video images from the extent groups shown
in Fig. 49. First, the top left-view extent L1 is read from the
BD-ROM disc 101 and output to the first read buffer 4221. During
the read period PR2d1/ as shown by the arrow 5201 in Fig. 52,
the accumulated data DAI increases at a rate equivalent to the
difference between the 2D video image read rate RUd-2D and the
2D video image transfer rate Rext1-2D[1] / or RUd-2D-Rext1-2D [1] • When
the end of the top left-view extent L1 is read, a jump J2d1 occurs .
During the jump period PJ2d1/ reading of the two extents D2 and
R2 positioned immediately after the top left-view extent L1 is
skipped, and thus reading of data from the BD-ROM disc 101 stops .
Accordingly, during the jump period PJ2d1/ as shown by the arrow
5202 in Fig. 52, the accumulated data amount DAI decreases at
the 2D video image transfer rate Rext1-2D[1] •
[0285]
Here, during the read period PR2D1, the accumulated data
amount in the first read buffer 4221, i.e. the size Sext1[1] of
the top left-view extent L1, is assumed to be equivalent to the
data amount transferred from the first read buffer 4221 to the
system target decoder 4203 between this read period PR2d1 and
the jump period PJ2d1. In this case, as shown in Fig. 52, at
the end of the jump period PJ2d1/ the accumulated data amount
DA1 only falls to the value it was at the start of the read period
PR2D1. After the jump J2d1/ reading of the next left-view extent
L2 begins. During the corresponding read period PR2d2 , as shown
by the arrow 5203 in Fig. 52, the accumulated data amount DA1
once again rises at a rate equivalent to the difference in data
transfer rates, RUd-2D-Rext1-2D[2] .
[0286]
The above-described change of the accumulated data amount
DAI is not limited to the top two left-view extents L1 and L2,
but is also similar when playing back 2D video images from the
nth and (n+l)th left-view extents Ln and L(n+1) (an integer n
is equal to or greater than two) . Accordingly, in order to play
back 2D video images seamlessly from these two left-view extents
Ln and L(n+1) regardless of the jump between them, the following
two conditions should be fulfilled, as in the case shown in Fig.
23 . First, the size Sext1 [n] of the nth left-view extent Ln should
be at least equivalent to the size of the smallest extent
expressed by the right-hand side of Equation 4.
[0287]
Equation 4

[0288]
In Equation 4, the jump time TjUmp.2D [n] expresses the
length of the jump period from the end of the nth left-view extent
Ln to the top of the (n+1)th left-view extent L(n+1) in seconds.
The jump time Tjump-2D[n] is determined by the sum of two
parameters, TJ [n] and TL [n] : Tjump.2D [n] =TJ [n] +TL [n] . The first
parameter TJ [n] represents the value assigned to the number of
sectors, i.e. the jump distance, from the end of the nth left-view
extent Ln to the top of the (n+l)th left-view extent L(n+1) as
per the regulation shown in Fig. 24 . The second parameter TL [n]
represents either (i) the layer switching time, i.e. 350 ms,
when there is a layer boundary LB between the nth left-view extent
Ln and the (n+l)th left-view extent L(n+1) or (ii) zero, when
there is no layer boundary LB therebetween.
[0289]
Next, the gap between the nth and the (n+l)th left-view
extents Ln and L(n+1) should be smaller than the maximum jump
distance. The maximum jump distance is equivalent to the value
associated with the jump time TjUmp.2D [n] as per the regulation
shown in Fig. 24.
[0290]
<>
[0291]
As shown in Fig. 49, the playback path 4902 for 3D video
images in L/R mode alternately reads right-view extents R1, R2,
etc. and left-view extents L1, L2, etc. Accordingly, while an
extent is being read into one of the two read buffers 4221 and
4222, extents are not read into the other buffer. Furthermore,
in the playback path 4902, jumps Jlr1, Jlr2 , etc. occur in the
recording areas for the depth map extents D2, D3, etc. On the
other hand, data transfer from the read buffers 4221 and 4222
to the system target decoder 42 03 is continuous. Accordingly,
in order for the 3D playback device to play back 3D video images
seamlessly, a sufficient amount of data should be accumulated
in the read buffers 4221 and 4222 during reading of extents,
so that underflow will not occur in the read buffers 4221 and
4222 due to data transfer to the system target decoder 42 03 while
reading of extents is suspended.
[0292]
Figs. 53A, 53B, and 53C are graphs showing the change of
the accumulated data amounts DAI and DA2 in the read buffers
4221 and 4222 when the 3D playback device plays back 3D video
images in L/R mode from the extent groups shown in Fig. 49. For
convenience of explanation, it is assumed that the right-view
extents and left-view extents have already been read into the
read buffers 4221 and 4222 from the top pair of extents R1 and
L1 in order through the (n-1) thpair of extents R(n-l) andL(n-l) .
It is also assumed that the integer n is sufficiently larger
than one. In this case, the accumulated data amounts DAI and
DA2 in the read buffers 4221 and 4222 are already maintained
above the lower limits UL1 and UL2 respectively. These lower
limits UL1 and UL2 are referred to as the "buffer margin amount."
Furthermore, the zero sector transition time TjUmp-o is assumed
to be equivalent to a value that does not include a margin time,
i.e. equivalent to a defined value that is evaluated only on
the time actually required for a zero sector transition. The
method for guaranteeing the buffer margin amounts UL1 and UL2
is described later.
[0293]
In accordance with the playback path 53 01 shown in Fig.
53C, the nth right-view extent Rn is first read from the BD-ROM
disc 101 during the (2n-1)th read period PRRn and output to the
second read buffer 4222. During the (2n-1)th read period PRRn,
as shown in Fig. 53B, the accumulated data amount DA2 in the
second read buffer 4222 increases at a rate equivalent to the
difference between the 3D video image read rate RUd-3D and the
right-view transfer rate Rext2 [n] , or RUd-3D-Rext2 [n] . Conversely,
as shown in Fig. 53A, the accumulated data amount DA1 in the
first read buffer 4221 decreases at the left-view transfer rate
Rext1-3D [n] .
[0294]
When the end of the nth right-view extent Rn is read, the
nth zero sector transition J0n occurs. During the nth zero sector
transition period PJ0n, reading of data from the BD-ROM disc
101 is suspended. Therefore, the accumulated data amount DA1
in the first read buffer 4221 decreases at the left-view
transfer rate Rext1-3D [n-1] , and the accumulated data amount DA2
in the second read buffer 4222 decreases at the right-view
transfer rate Rext2 [n] .
[0295]
At the point the nth zero sector transition period PJ0n
ends, the 2nth read period PRLn begins . During the 2nth read period
PRLn, the nth left-view extent Ln is read from the BD-ROM disc
101 and output to the first read buffer 4221. Accordingly, as
shown in Fig. 53A, the accumulated data amount DAI in the first
read buffer 4221 increases at a rate equivalent to the
difference between the 3D video image read rate RUd-3D and the
left-view transfer rate Rext1-3D [n] , or RUd-3D-Rext1-3D [n] .
Conversely, as shown in Fig. 53B, the accumulated data amount
DA2 in the second read buffer 4 222 decreases at the right-view
transfer rate Rext2 [n] .
[0296]
When the end of the nth left-view extent Ln is read, the
jump JLRn occurs. During the jump period PJLRn, reading of the
depth map extent D(n+1) located immediately after the nth
left-view extent Ln is skipped, and thus reading of data from
the BD-ROM disc 101 is suspended. Therefore, during the jump
period PJLRn, as shown in Fig. 53A, the accumulated data amount
DAI in the first read buffer 4221 decreases at the left-view
transfer rate Rext1-3D [n] , whereas, as shown in Fig. 53B, the
accumulated data amount DA2 in the second read buffer 4222
decreases at the right-view transfer rate Rext2 [n] .
[0297]
The accumulated data amount in the second read buffer 4222
during the (2n-1)th read period PRRn, i.e. the size Sext2 tn] of
the nth right-view extent Rn, is at least equivalent to the data
amount transferred from the second read buffer 4222 to the
system target decoder 4203 from the (2n-1)th read period PRRn
through the jump period PJLRn. Accordingly, at the end of the
jump period PJLRn, as shown in Fig. 53B, the accumulated data
amount DA2 in the second read buffer 4222 does not fall below
the second buffer margin amount UL2.
[0298]
At the point the jump period PJLRn ends, the (2n+l)th read
period PRR(n+l) begins. During the (2n+l)th read period PRR(n+l) ,
the (n+l)th right-view extent R(n+1) is read from the BD-ROM
disc 101 and output to the second read buffer 4222 . Accordingly,
as shown in Fig. 53B, the accumulated data amount DA2 in the
second read buffer 4222 increases at a rate equivalent to the
difference between the 3D video image read rate RUd-3D and the
right-view transfer rate Rext2 [n+1] , or RUd-3D-Rext2 [n+1] .
Conversely, as shown in Fig. 53A, the accumulated data amount
DAI in the first read buffer 4221 continues to decrease at the
left-view transfer rate Rext1-3D [n] .
[0299]
When the end of the (n+1)th right-view extent R(n+1) is
read, the (n+1)th zero sector transition J0 (n+1) occurs. During
the (n+1)th zero sector transition period PJ0(n+1) , reading of
data from the BD-ROM disc 101 is suspended. Therefore, the
accumulated data amount DAI in the first read buffer 4221
decreases at the left-view transfer rate Rext1-3D [n] , and the
accumulated data amount DA2 in the second read buffer 4222
decreases at the right-view transfer rate Rext2 [n+1] .
[0300]
The accumulated data amount in the first read buffer 4221
during the 2nth read period PRLn, i.e. the size Sext1 [n] of the
nth left-view extent Ln, is at least equivalent to the data amount
transferred from the first read buffer 4221 to the system target
decoder 4203 from the 2nth read period PRLn through the (n+1)th
zero sector transition period PJ0(n+1) . Accordingly, at the end
of the (n+1)th zero sector transition period PJ0(n+1) , as shown
in Fig. 53A, the accumulated data amount DAI in the first read
buffer 4221 does not fall below the first buffer margin amount
UL1.
[0301]
At the point the (n+1)th zero sector transition period
PJ0(n+1) ends, the (2n+2)th read period PRL (n+1) begins. During
the (2n+2)th read period PRL(n+1) , the (n+1)th left-view extent
L(n+1) is read from the BD-ROM disc 101 and output to the first
read buffer 4221. Accordingly, as shown in Fig. 53A, the
accumulated data amount DAI in the first read buffer 4221
increases at a rate equivalent to the difference between the
3D video image read rate RUd-3D and the left-view transfer rate
Rext1-3D[n+1] , or RUd-3D-Rext1-3D[n+1] . Conversely, as shown in Fig.
53B, the accumulated data amount DA2 in the second read buffer
4222 continues to decrease at the right-view transfer rate
Rext2 [n+1] .
[0302]
The above-described change of the accumulated data
amounts DAI and DA2 is similar for each extent after the (n+2)th
extents. Accordingly, in order to play back 3D video images
seamlessly from the right-view extents Rn, R(n+1) , etc. and the
left-view extents Ln, L(n+1) , etc. , regardless of jumps between
them, the following two conditions need to be fulfilled.
[0303]
The first condition is that the size Sext1 [n] of the nth
left-view extent Ln should be equivalent to the data amount
transferred from the first read buffer 4221 to the system target
decoder 4203 from the 2nth read period PRLn through the (n+1)th
zero sector transition period PJ0 (n+1) . Here, the length of the
2nth read period PRLn is equivalent to the size Sext1 [n] of the
nth left-view extent Ln divided by the 3D video image read rate
Rud-3D, or Sext1 [n] /RUd-3D- The length of the (2n+l)th read period
PRR(n+1) is equivalent to the size Sext2 [n+1] of the (n+1)th
right-view extent R (n+1) divided by the 3D video image read rate
RUd-3D/ or Sext2 [n+1] /RUd-3D- Accordingly, the size Sext1 [n] of the
nth left-view extent Ln is at least equivalent to the smallest
extent size represented by the right-hand side of Equation 5.
[0304]
Equation 5

[0305]
The second condition is that the size Sext2 [n] of the nth
right-view extent Rn should be at least equivalent to the data
amount transferred from the second read buffer 4222 to the
system target decoder 4203 from the (2n-1)th read period PRRn
through the nth jump period PJLRn. Here, the length of the (2n-1)th
read period PRRn is equivalent to the size Sext2[n] of the nth
right-view extent Rn divided by the 3D video image read rate
Rud-3D, or Sext2 [n] /RUd-3D- Accordingly, the size Sext2 [n] of the nth
right-view extent Rn is at least equivalent to the smallest
extent size represented by the right-hand side of Equation 6.
[0306]
Equation 6

[0307]
In Equations 5 and 6, the jump time Tjump-3D [n] expresses
the length of the jump period PJLRn from the end of the nth
left-view extent Ln to the top of the (n+1)th right-view extent
R(n+1) in seconds. More specifically, the jump time Tjump-sc [n]
is determined by the sum of two parameters, TJ [n] and TL [n] :
TjUmp-3D [n] =TJ [n] +TL [n] . The first parameter TJ [n] represents
the value assigned to the number of sectors, i.e. the jump
distance, from the end of the nth left-view extent Ln to the
top of the (n+1)th right-view extent R(n+1) as per the regulation
shown in Fig. 24. The second parameter TL [n] represents either
(i) the layer switching time, i.e. 350 ms, when there is a layer
boundary LB between the nth left-view extent Ln and the (n+1)th
right-view extent R(n+1) or (ii) zero, when there is no layer
boundary LB therebetween.
[0308]
<>
[0309]
As shown in Fig. 49, the playback path 4903 for 3D video
images in depth mode alternately reads depth map extents D1,
D2, etc. and left-view extents L1, L2, etc. Accordingly, while
an extent is being read into one of the two read buffers 4221
and 4222, extents are not read into the other buffer.
Furthermore, in the playback path 4 903, jumps Jlr1, Jlr2 , etc.
occur in the recording areas for the depth map extents D2, D3,
etc. On the other hand, data transfer from the read buffers 4221
and 4222 to the system target decoder 4203 is continuous.
Accordingly, in order for the 3D playback device to play back
3D video images seamlessly, a sufficient amount of data should
be accumulated in the read buffers 4221 and 4222 during reading
of extents, so that underflow will not occur in the read buffers
4221 and 4222 due to data transfer to the system target decoder
4203 while reading of extents is suspended.
[0310]
Figs. 54A, 54B, and 54C are graphs showing the change of
the accumulated data amounts DA1 and DA2 in the read buffers
4221 and 4222 when the 3D playback device plays back 3D video
images in depth mode from the extent groups shown in Fig. 49.
For convenience of explanation, it is assumed that the depth
map extents and left-view extents have already been read into
the read buffers 4221 and 4222 from the top pair of extents D1
and L1 in order through the (n-1)th pair of extents D(n-1) and
L(n-l) . It is also assumed that the integer n is sufficiently
larger than one. In this case, the accumulated data amounts DA1
and DA2 in the read buffers 4221 and 4222 are already maintained
above the buffer margin amounts UL1 and UL2 respectively.
Furthermore, the zero sector transition time Tjump-0 is assumed
to be equivalent to a value that does not include a margin time,
i.e. equivalent to a defined value that is evaluated only on
the time required for an actual zero sector transition.
[0311]
In accordance with the playback path 54 01 shown in Fig.
54C, the nth depth map extent Dn is first read from the BD-ROM
disc 101 during the (2n-1)th read period PRDn and output to the
second read buffer 4222. During the (2n-1)th read period PRDn,
as shown in Fig. 54B, the accumulated data amount DA2 in the
second read buffer 4222 increases at a rate equivalent to the
difference between the 3D video image read rate RUd-3D and the
depth map transfer rate Rext3 [n] , or RUd-3D-Rext3 [n]. Conversely,
as shown in Fig. 54A, the accumulated data amount DAI in the
first read buffer 4221 decreases at the left-view transfer rate
Rext1-3D [n-1] .
[0312]
When the end of the nth depth map extent Dn is read, the
nth jump JLDn occurs. During the nth jump period PJldH, reading
of the right-view extent Rn located immediately after the nth
depth map extent Dn is skipped, and thus reading of data from
the BD-ROM disc 101 is suspended. Therefore, during the nth jump
period PJLRn, as shown in Fig. 54A, the accumulated data amount
DA1 in the first read buffer 4221 decreases at the left-view
transfer rate Rext1-3D [n-1] , whereas, as shown in Fig. 54B, the
accumulated data amount DA2 in the second read buffer 4222
decreases at the depth map transfer rate Rext3 [n] .
[0313]
At the point the nth jump period PJLRn ends, the 2nth read
period PRLn begins. During the 2nth read period PRLn, the nth
left-view extent Ln is read from the BD-ROM disc 101 and output
to the first read buffer 4221. Accordingly, as shown in Fig.
54A, the accumulated data amount DAI in the first read buffer
4221 increases at a rate equivalent to the difference between
the 3D video image read rate RUd-3D and the left-view transfer
rate Rext1-3D[n] , or RUd-3D_Rext1-3D [n] • Conversely, as shown in Fig.
54B, the accumulated data amount DA2 in the second read buffer
4222 continues to decrease at the depth map transfer rate
Rext3 [n] .
[0314]
When the end of the nth left-view extent Ln is read, the
nth zero sector transition J0n occurs. During the nth zero sector
transition period PJ0n, reading of data from the BD-ROM disc
101 is suspended. Therefore, the accumulated data amount DA1
in the first read buffer 4221 decreases at the left-view
transfer rate Rext1-3D[n] , and the accumulated data amount DA2
in the second read buffer 4222 decreases at the depth map
transfer rate Rext3 [n] .
[0315]
The accumulated data amount in the second read buffer 4222
during the (2n-1)th read period PRDn, i.e. the size Sext3 [n] of
the nth depth map extent Dn, is at least equivalent to the data
amount transferred from the second read buffer 4222 to the
system target decoder 4203 from the (2n-1)th read period PRpn
through the nth zero sector transition period PJ0n. Accordingly,
at the end of the nth zero sector transition period PJ0n, as shown
in Fig. 54B, the accumulated data amount DA2 in the second read
buffer 4222 does not fall below the second buffer margin amount
UL2.
[0316]
At the point the nth zero sector transition period PJ0n
ends, the (2n+l)th read period PRD(n+1) begins. During the
(2n+l)th read period PRD(n+1), the (n+1)th depth map extent
D(n+1) is read from the BD-ROM disc 101 and output to the second
read buffer 4222. Accordingly, as shown in Fig. 54A, the
accumulated data amount DA1 in the first read buffer 4221
continues to decrease at the left-view transfer rate Rext1-3D [n] .
Conversely, as shown in Fig. 54B, the accumulated data amount
DA2 in the second read buffer 4222 increases at a rate equivalent
to the difference between the 3D video image read rate Rud-3D
and the depth map transfer rate Rext3 [n+1] , or RUd-3D-Rext3 [n+1].
[0317]
When the end of the (n+1)th depth map extent D(n+1) is
read, the (n+1)th jump JLD(n+1) occurs. During the (n+1)th jump
period PJ(n+1) , reading of the right-view extent R(n+1)
located immediately after the (n+1)th depth map extent D(n+1)
is skipped, and thus reading of data from the BD-ROM disc 101
is suspended. Therefore, during the (n+1)th jump period PJld (n+1) ,
the accumulated data amount DA1 in the first read buffer 4221
decreases at the left-view transfer rate Rext1-3D [n] , and the
accumulated data amount DA2 in the second read buffer 4222
decreases at the depth map transfer rate Rext3 [n+1] .
[0318]
At the point the (n+1)th jump period PJld (n+1) ends, the
(2n+2)th read period PRL(n+1) begins. During the (2n+2)th read
period PRL (n+1) , the (n+1)th left-view extent L (n+1) is read from
the BD-ROM disc 101 and output to the first read buffer 4221.
Accordingly, as shown in Fig. 54A, the accumulated data amount DA1 in the first read buffer 4221 increases at a rate equivalent
to the difference between the 3D video image read rate RUd-3D
and the left-view transfer rate Rext1-3D [n+1] , or
Rud-3D-Rext1-3D [n+1] . Conversely, as shown in Fig. 54B, the
accumulated data amount DA2 in the second read buffer 4222
continues to decrease at the depth map transfer rate Rext3 [n+1] .
[0319]
The accumulated data amount in the first read buffer 4221
during the 2nth read period PRLn, i.e. the size Sext1 [n] of the
nth left-view extent Ln, is at least equivalent to the data amount
transferred from the first read buffer 4221 to the system target
decoder 4203 from the 2nth read period PRLn through the (n+1)th
jump period PJLD(n+1) . Accordingly, at the end of the (n+1)th
jump period PJLD(n+1), as shown in Fig. 54A, the accumulated
data amount DA1 in the first read buffer 4221 does not fall below
the first buffer margin amount UL1.
[0320]
The above-described change of the accumulated data
amounts DA1 and DA2 is similar for each extent after the (n+2)th
extents. Accordingly, in order to play back 3D video images
seamlessly from the depth map extents Dn, D(n+1) , etc. and the
left-view extents Ln, L(n+1) , etc., regardless of jumps between
them, the following two conditions need to be fulfilled.
[0321]
The first condition is that the size Sext1 [n] of the nth
left-view extent Ln should be equivalent to the data amount
transferred from the first read buffer 4221 to the system target
decoder 4203 from the 2nth read period PRLn through the (n+1)th
jump period PJLD(n+1) . Here, the length of the 2nth read period
PRLn is equivalent to the size Sext1 [n] of the nth left-view extent
Ln divided by the 3D video image read rate RUd-3D/ or Sext1 [n] /Rud-3D-
The length of the (2n+l)th read period PRD(n+1) is equivalent
to the size Sext3 [n+1] of the (n+1)th depth map extent D(n+1)
divided by the 3D video image read rate RUd-3D/ or Sext3 [n+1] /RUd-3D-
Accordingly, the size Sext1 [n] of the nth left-view extent Ln is
at least equivalent to the smallest extent size represented by
the right-hand side of Equation 7.
[0322]
Equation 7

[0323]
The second condition is that the size Sext3 [n] of the nth
depth map extent Dn should be at least equivalent to the data
amount transferred from the second read buffer 4222 to the
system target decoder 4203 from the (2n-1)th read period PRDn
through the nth zero sector transition period PJ0n. Here, the
length of the (2n-1)th read period PRDn is equivalent to the size
Sext3 [n] of the nth depth map extent Dn divided by the 3D video
image read rate RUd-3D/ or Sext3 [n]/RUd-3D- Accordingly, the size
Sext3 [n] of the nth depth map extent Dn is at least equivalent
to the smallest extent size represented by the right-hand side
of Equation 8.
[0324]
Equation 8

[0325]
In Equations 7 and 8, the jump time Tjump.3D [n] expresses
the length of the jump period PJLRn from the end of the nth depth
map extent Dn to the top of the nth left-view extent Ln in seconds.
More specifically, the jump time Tjump.3D[n] is equivalent to the
value assigned to the number of sectors, i.e. the jump distance,
from the end of the nth depth map extent Dn to the top of the
nth left-view extent Ln as per the regulation shown in Fig. 24.
[0326]
When there is no layer boundary LB between the nth
left-view extent Ln and the (n+1)th depth map extent D(n+1),
the zero sector transition time Tjump-0 [n] is equivalent to a
defined value that is evaluated only on the time actually
required for a zero sector transition. On the other hand, when
there is a layer boundary LB between the nth left-view extent
Ln and the (n+1)th depth map extent D(n+1) , then the zero sector
transition time TjUmp-0 [n] is replaced by the jump time necessary
to skip the layer boundary LB. This jump time is longer than
the value associated with the number of sectors between the nth
left-view extent Ln and the (n+1)th depth map extent D(n+1),
as per the regulation shown in Fig. 24, by the time for switching
layers, i.e. 350 ms.
[0327]
To enable seamless playback of 2D video images, 3D video
images in L/R mode, and 3D video images in depth mode from the
extent groups shown in Fig. 49, the size of each extent needs
to fulfill Equations 4-8 above. In particular, the size of the
smallest extent in the 2D/left-view stream file needs to be
equivalent to the largest value among the values corresponding
to the right-hand side of Equations 4, 5, and 7.
[0328]
<>
[0329]
When only L/R mode is used for 3D video playback, the depth
map extents may be removed from the arrangement in Fig. 4 9 . Fig.
55 is a schematic diagram showing the arrangement of extent
groups belonging to each file when only two types of files, a
2D/left-view stream file and a right-view stream file, are
recorded on a BD-ROM disc. As shown in Fig. 55, these two types
of extent groups are recorded in an interleaved arrangement.
In this interleaved arrangement, sections of each AV stream file
with matching playback periods are included individually in two
consecutive extents. In particular, between these extents, the
playback times of the video streams are equivalent. For example,
the top right-view extent R1 and the top left-view extent L1
are consecutive, their playback periods match, and the playback
time of the video streams is equivalent. The conditions for
extent sizes required to make seamless 2D video playback and
seamless 3D video playback compatible are determined for this
arrangement in the same way as in the arrangement shown in Fig.
49.
[0330]
First, in 2D video playback processing, only the
left-view extents L1, L2, etc. are read in accordance with the
playback path 5501 shown in Fig. 55, and reading of the
right-view extents Rl, R2, etc. is skipped by jumps J2d1/ J2d2,
etc. Here, the extent groups shown in Fig. 55 do not include
depth map extents, unlike the extent groups shown in Fig. 49.
Accordingly, the playback path 5501 for 2D video images differs
from the playback path 4901 for 2D video images shown in Fig.
49 only in the jump distance between the nth and (n+1)th left-view
extents Ln and L(n+1) (an integer n is equal to or greater than
two). Accordingly, for seamless 2D video playback, the size
Sext1 [n] of the nth left-view extent Ln should fulfill Equation
4.
[0331]
Next, in playback processing of 3D video images, the
right-view extents R1, R2, etc. and the left-view extents L1,
L2, etc. are read in order from the top in accordance with the
playback path 5502 shown in Fig. 55. The extent groups shown
in Fig. 55 do not include depth map extents, unlike the extent
groups shown in Fig. 49. Accordingly, the playback path 5502
for 3D video images differs from the playback path 4902 for 3D
video images in L/R mode shown in Fig. 4 9 in that a jump JLRn
does not occur immediately after each left-view extent. Instead,
only a zero sector transition J0n occurs (n=1, 2, etc.).
Therefore, for 3D video images to be played back seamlessly,
the size Sext1 [n] of the nth left-view extent Ln should fulfill
Equation 9 below instead of Equation 5, and the size Sext2 [n]
of the nth right-view extent Rn should fulfill Equation 10 below
instead of Equation 6. Both Equations 9 and 10 are equivalent
to Equations 5 and 6, replacing the jump time Tjump-3D with the
zero sector transition time Tjump-0.
[0332]
Equation 9

Equation 10

[0333]
Accordingly, the size of the smallest extent in the
2D/left-view stream file is equivalent to the larger of the
values corresponding to the right-hand side of Equations 4 and
9.
[0334]
Note that when the consecutive extents shown in Fig. 55
are read continuously by the BD-ROM drive 4201, the zero sector
transition time Tjump-0 [n] in Equations 9 and 10 can be considered
to be zero. In other words, the size Sext1 [n] of the nth left -view
extent Ln and the si ze Sext2 [n] of the nth right-view extent Rn
should fulfill the following equations.
[0335]

[0336]
<>
[0337]
The lower limits UL1 and UL2 for the accumulated data
amounts DA1 and DA2 in the read buffers 4221 and 4222 shown in
Figs. 53A-C and 54A-C represent the buffer margin amount for
each buffer. The "buffer margin amounts" refer to the lower
limit of the accumulated data amount that should be maintained
in each of the read buffers 4221 and 4222 in order to prevent
underflow, which is caused during the reading of an AV stream
file mainly when the recording layer to be read is switched,
or when reading is interrupted by the read process of another
file. Here, the term "another file" means a file other than the
AV stream file shown in Fig. 2, and for example includes the
movie object file 242B, the BD-J object file 246A, and the JAR
file 247A.
[0338]
Fig. 56 is a schematic diagram showing (i) a jump JLY caused
by layer switching occurring during the playback path of 3D
video images in L/R mode and (ii) an object JBdj caused by the
read process of the BD-J object file 246A. The buffer margin
amounts UL1 and UL2 necessary for the jumps JLy and JBdj are
calculated as follows.
[0339]
For the jump JLY caused by switching layers, the maximum
jump time Tjump.Ly is equivalent to the sum of the jump distance
as per the regulation shown in Fig. 24 and the layer switching
time. The jump distance is the value associated with the number
of sectors between the extents Ln and D(n+1) on either side of
the layer boundary LB. As per Equation 2, the left-view transfer
rate Rext1-3D does not exceed the maximum value Rmax1-3D-
Accordingly, the data amount that is consumed from the first
read buffer 4221 during the jump period caused by layer
switching does not exceed the product of the left-view transfer
rate Rext1-3D and the maximum jump time Tjump.LY. The value of this
product is determined to be the first buffer margin amount UL1.
In other words, the first buffer margin amount UL1 is calculated
by Equation 11.
[0340]
Equation 11

[0341]
For example, when the maximum jump distance is 4 0000
sectors, as per the regulation shown in Fig. 24, the maximum
jump time Tjump-Ly is 700 ms, which includes the layer switching
time of 350 ms. Accordingly, when the system rate corresponding
to the 2D/left-view stream file is 4 8 Mbps, the first buffer
margin amount UL1 is equivalent to (48 Mbpsxl92/188)x0.7
seconds = approximately 4.09 MB.
[0342]
Similarly, the maximum value of the data amount that is
consumed from the second read buffer 4222 during the jump period
caused by layer switching, i.e. the product of the maximum value
Rmax2 of the right-view transfer rate and the maximum jump time
Tjump-LY, is determined to be the second buffer margin amount UL2 .
In other words, the second buffer margin amount UL2 is
calculated in Equation 12.
[0343]
Equation 12
[0344]
For example, when the maximum jump distance is 40000
sectors, meaning that the maximum jump time Tjump-LY is 700 ms,
and when the system rate corresponding to the right-view stream
file is 16 Mbps, the second buffer margin amount UL2 is
equivalent to (16 Mbpsxl92/188)x0.7 seconds = approximately
1.36 MB.
[0345]
When a read process of the BD-J object file 246A interrupts
the read period of the AV stream file, as shown in Fig. 56, a
first jump JBdj1 occurs. In this way, the position of the read
target shifts from the recording area of the (n-1)th left-view
extent L (n-1) to the recording area of the BD-J object file 246A.
The corresponding jump time TBdj is set to a predetermined fixed
value, e.g. 900 ms. Next, the BD-J object file 246A is read.
The time required for reading is equivalent to eight times the
size SBdj of the extent belonging to the file 24 6A in bytes divided
by the 3D video image read rate RUd-3D, or 8xSBDJ [n]/Ruti-3D. Next,
a second jump JBdj2 occurs. In this way, the position of the read
target returns from the recording area of the BD-J object file
246A back to the recording area of the (n-1)th left-view extent
L(n-l) . The corresponding jump time TBDj is equivalent to the
first jump period, e.g. 900 ms . During the two jumps JBDj1 and
JBDJ2 and the reading of the BD-J object file 246A, data is not
read into the first read buffer 4221. Accordingly, the maximum
value of the amount of data that is consumed from the first read
buffer 4221 during this time is determined to be the first read
buffer margin amount UL1. In other words, the first read buffer
margin amount UL1 is calculated in Equation 13.
[0346]
Equation 13

[0347]
Similarly, the maximum value of the amount of data that
is consumed from the second read buffer 4222 during the two jumps
Jbdj1 and JBdj2 and the reading of the BD-J object file 246A is
determined to be the second read buffer margin amount UL2 . In
other words, the second read buffer margin amount UL2 is
calculated in Equation 14.
[0348]
Equation 14

[0349]
The first buffer margin amount UL1 is set to the larger
of the values corresponding to the right-hand side of Equations
11 and 13 . The second buffer margin amount UL2 is set to the
larger of the values corresponding to the right-hand side of
Equations 12 and 14.
[0350]
<>
[0351]
The buffer margin amounts UL1 and UL2 are guaranteed in
the following way. First, a margin time is added to the zero
sector transition time Tjump-0 [n] substituted into Equations 5-8 .
The margin time is, for example, 100 ms. In this way, since the
minimum extent size is enlarged, the accumulated data amounts DA1 and DA2 in the read buffers 4221 and 4222 increase by a
predetermined amount each time a pair of a base-view extent and
a dependent-view extent is decoded, unlike the change shown in
Figs. 53 and 54. Accordingly, by reading a certain number of
extents into the read buffers 4221 and 4222, the buffer margin
amounts UL1 and UL2 are guaranteed.
[0352]
Figs. 57A and 57B are graphs showing the change of the
accumulated data amounts DA1 and DA2 in the read buffers 4221
and 4222 when a margin time is added to the zero sector transition
time. In the following explanation, 3D video images are assumed
to be played back in L/R mode from the top of the extent groups
shown in Fig. 49. This explanation is also applicable when 3D
video images are played back in depth mode and when 3D video
images are played back from an arbitrary extent via interrupt
playback.
[0353]
First, during the first read period PRr1, the top
right-view extent R1 is read from the BD-ROM disc 101 and output
to the second read buffer 4222. Meanwhile, the system target
decoder 4203 is placed on standby. Accordingly, in the first
read period PRR1, as shown in Fig. 57B, the accumulated data
amount DA2 in the second read buffer 4222 increases from 0 at
the 3D video image read rate RUd-3D.
[0354]
When the end of the top right-view extent R1 is read, the
first zero sector transition J01 occurs. During the first zero
sector transition period PJ01, reading of data from the BD-ROM
disc 101 is suspended. Meanwhile, the system target decoder 4203
continues on stand-by. Therefore, during the first zero sector
transition period PJ01, the accumulated data amount DA1 in the
first read buffer 4221 and the accumulated data amount DA2 in
the second read buffer 4222 are both maintained constant.
[0355]
At the point the first zero sector transition period PJ01
ends, the second read period PRL1 begins . During the second read
period PRL1/ the top left-view extent L1 is read from the BD-ROM
disc 101 and output to the first read buffer 4221. Meanwhile,
the system target decoder 4203 beings the decoding process in
the second read period PRL1. Accordingly, as shown in Fig. 57A,
the accumulated data amount DA1 in the first read buffer 4221
increases from 0 at a rate equivalent to the difference between
the 3D video image read rate RUd-3D and the left-view transfer
rate Rext1-3D[1] , or RUd-3D-Rext1-3D [1] • Conversely, as shown in Fig.
57B, the accumulated data amount DA2 in the second read buffer
4222 decreases at the right-view transfer rate Rext2[1].
[0356]
When the end of the top left-view extent L1 is read, the
first jump Jlr1 occurs. During the first jump period PJlr1,
reading of the second depth map extent D2 located immediately
after the top left-view extent L1 is skipped, and thus reading
of data from the BD-ROM disc 101 is suspended. Therefore, during
the first jump period PJlrI, as shown in Fig. 57A, the accumulated
data amount DA1 in the first read buffer 4221 decreases at the
left-view transfer rate Rext1-3D[1], whereas, as shown in Fig.
57B, the accumulated data amount DA2 in the second read buffer
4222 decreases at the right-view transfer rate Rext2[1].
[0357]
The accumulated data amount in the second read buffer 4222
during the first read period PRR1, i.e. the size Sext2 [1] of the
top right-view extent R1, is at least equivalent to the smallest
extent size expressed by the right-hand side of Equation 6 . Here,
a margin time, e.g. 100 ms, is added to the zero sector transition
time Tjump-0 [1] . Furthermore, during the first read period PRR1,
data is not transferred from the second read buffer 4222 to the
system target decoder 4203. Accordingly, the size Sext2 [1] of
the top right-view extent Rl is larger than the data amount
transferred from the second read buffer 4222 to the system
target decoder 4203 from the second read period PRL1 through
the first jump period P Jlr1 . As a result, at the end of the first
jump period PJlr1, as shown in Fig. 57B, the accumulated data
amount DA2 in the second read buffer 4222 increases to a value
DM2 larger than zero. This value DM2 is evaluated as the product
of the maximum value Rmax2 of the right-view transfer rate
multiplied by a margin time, e.g. 100 ms: DM2=Rmax2xTjump-0 [1].
[0358]
At the point the first jump period PJlrI ends, the third
read period PRR2 begins. During the third read period PRR2, the
second right-view extent R2 is read from the BD-ROM disc 101
and output to the second read buffer 4222 . Accordingly, as shown
in Fig. 57B, the accumulated data amount DA2 in the second read
buffer 4222 increases at a rate equivalent to the difference
between the 3D video image read rate RUd-3D and the right-view
transfer rate Rext2 [2] , or RUd-3D_Rext2 [2] . Conversely, as shown
in Fig. 57A, the accumulated data amount DA1 in the first read
buffer 4221 continues to decrease at the left-view transfer rate
Rextl-3D [1] •
[0359]
When the end of the second right-view extent R2 is read,
the second zero sector transition J02 occurs. During the second
zero sector transition period PJ02, reading of data from the
BD-ROM disc 101 is suspended. Therefore, the accumulated data
amount DA1 in the first read buffer 4221 decreases at the
left-view transfer rate Rext1-3D [2] , and the accumulated data
amount DA2 in the second read buffer 4222 decreases at the
right-view transfer rate Rext2
[2] .
[0360]
The accumulated data amount in the first read buffer 4221
during the second read period PRl1/ i.e. the size Sext1[1] of
the top left-view extent L1, is at least equivalent to the
smallest extent size expressed by the right-hand side of
Equation 5. Here, a margin time, e.g. 100 ms, is added to the
zero sector transition time Tjump-0[2] . Accordingly, the size
Sext1[1] of the top left-view extent L1 is larger than the data
amount transferred from the first read buffer 4221 to the system
target decoder 4203 from the second read period PRL1 through
the second zero sector transition period PJ02. As a result, at
the end of the second zero sector transition period PJ02, as
shown in Fig. 57A, the accumulated data amount DA1 in the first
read buffer 4221 increases to a value DM1 larger than zero. This
value DM1 is evaluated as the product of the maximum value Rmax1-3D
of the left-view transfer rate multiplied by a margin time, e.g.
100 ms: DM1=Rmax1-3DxT
[0361]
By repeating the above-described process, each time a
pair of a left-view extent and a right-view extent are read into
the read buffers 4221 and 4222, the accumulated data amounts DA1 and DA2 in the read buffers 4221 and 4222 increase by the
values DM1 and DM2. Accordingly, once a certain number of
extents have been read into the read buffers 4221 and 4222, the
buffer margin amounts UL1 and UL2 are guaranteed. For example,
when the margin time is 100 ms, in order to guarantee the buffer
margin amounts UL1 and UL2 required to prevent underflow caused
by a 700 ms jump, the above-described process should be repeated
700 ms/100 ms = seven times.
[0362]
Conversely, as long as no large jump occurs, the
accumulated data amounts DA1 and DA2 in the read buffers 4221
and 4222 continue to increase by values DM1 and DM2. Therefore,
when the accumulated data amounts DA1 and DA2 exceed a
predetermined threshold, the 3D playback device interrupts the
read/transfer operation of the BD-ROM drive 4201. In this way,
the 3D video image read rate RUd-3D decreases, and thus the rate
of increase of the accumulated data amounts DA1 and DA2 is held
down. Overflow in the read buffers 4221 and 4222 can thus be
avoided.
[0363]
<>
[0364]
When the arrangement on the BD-ROM disc 101 of the extents
belonging to the AV stream file is as described above, the
minimum values of the capacities required for the read buffers
4221 and 4222 are calculated as follows.
[0365]
When playing back 3D video images in either L/R mode or
depth mode, in order to read the nth left-view extent Ln, the
capacity RBI[n] of the first read buffer 4221 should fulfill
Equation 15.
[0366]
Equation 15

[0367]
On the other hand, when playing back 3D video images in
L/R mode, in order to read the nth right-view extent Rn, the
capacity RB2LR[n] of the second read buffer 4222 should fulfill
Equation 16.
[0368]
Equation 16

[0369]
Here, any of the right-view extents may be read first by
interrupt playback. In that case, as shown in Fig. 57B, the
system target decoder 4203 does not read data from the second
read buffer 4222 until the entire right-view extent is stored
in the second read buffer 4222 . Accordingly, unlike the capacity
RBI [n] of the first read buffer 4221, the capacity RB2LR[n] of
the second read buffer 4222 needs to further fulfill the
condition of being "at least larger than the size Sext2 [n] of
the nth right-view extent Rn."
[0370]
When playing back 3D video images in depth mode, in order
to read the nth depth map extent Dn, the capacity RB2LD[n] of
the second read buffer 4222 should fulfill Equation 17.
[0371]
Equation 17

[0372]
Similar to the right-view extents, any of the depth map
extents may be read first by interrupt playback. Accordingly,
the capacity RB2LD [n] of the second read buffer 4222 needs to
further fulfill the condition of being "at least larger than
the size Sext3 [n] of the nth depth map extent Dn."
[0373]
The minimum values of the sizes of the base-view extents
and dependent-view extents on the BD-ROM disc 101 in Embodiment
1 of the present invention are clearly expressed by the
right-hand side of Equations 4, 5, and 6. As a result, underflow
does not occur in either of the read buffers 4221 and 4222 during
playback of either 2D video images or 3D video images from the
BD-ROM disc, and it is easy to appropriately set the size of
extents to further reduce the capacities of the buffers.
[0374]

[0375]
(1) Embodiment 1 of the present invention pertains to the
arrangement of extents when storing 3D video images on a
recording medium. However, the present invention can also be
used for storage of high frame rate video on a recording medium.
Specifically, for example the high frame rate video can be
divided into an odd-numbered frame group and an even-numbered
frame group, which can be considered as a base-view stream and
a dependent-view stream and recorded on a recording medium with
the arrangement of extents as described in Embodiment 1. A
playback device that only supports video playback at a normal
frame rate can play back video for the odd-numbered frame group
from the recording medium. Conversely, a playback device that
supports video playback at a high frame rate can choose to play
back video for only the odd-numbered frame group or video for
both frame groups. In this way, compatibility with a playback
device that only supports video playback at a normal frame rate
can be ensured on a recording medium on which a high frame rate
video is stored.
[0376]
(2) Embodiment 1 of the present invention assumes that
the left-view stream is the base-view stream. However, the
right-view stream may also be the base-view stream. In this case,
the left-view stream can be compressed as a dependent-view
stream using the right-view stream.
[0377]
(3) In order to accurately calculate the extent ATC time
when the mean transfer rate Rext of data from the read buffer
1802 to the system target decoder 1803, as shown in Fig. 22,
is evaluated, the size of each extent can be regulated as a fixed
multiple of the source packet length. Furthermore, when a
particular extent includes more source packets than this
multiple, the sum of (i) the product of the number of source
packets exceeding the multiple and the transfer time per source
packet and (ii) the extent ATC time corresponding to the
multiple can be considered to be the extent ATC time for the
particular extent. Additionally, the extent ATC time can be
defined as the sum of (iii) the value of the time interval from
the ATS of the top source packet in an extent until the ATS of
the last source packet in the same extent and (iv) the transfer
time per source packet. In this case, reference to the next
extent is unnecessary for calculation of the extent ATC time,
and thus the calculation can be simplified. Note that in the
above-described calculation of extent ATC time, the occurrence
of wraparound in the ATS needs to be taken into consideration.
[0378]
(4) As shown in Figs. 35A and 35B, a pair of pictures
belonging to the same 3D video image access unit have the same
PTS and the same DTS. Additionally, the PTS and DTS allocated
to one picture in one of the pairs of pictures may be later than
the PTS and DTS allocated to the other picture by a predetermined
amount, e.g. a half cycle. In this case, the switch 4310 shown
in Fig. 43 would output the 2D/left video plane data 43 01 and
the right video plane data 4302 to the first adder 4311 in order
of PTSs. Furthermore, the buffer switch 4606 shown in Fig. 46
would transfer video access units accumulated in the EB (1) 4603
and the EB (2) 4610 respectively to the compressed video decoder
4 604 in accordance with the times of DTSs shown by original TS
packets.
[0379]
(5) Among the stream attribute information 3821 in the
dependent-view clip information file 3802 shown in Fig. 38B,
the video stream attribute information regarding the
dependent-view stream may have an additional flag indicating
that reference to the base-view stream is necessary for decoding
of the dependent-view stream. Furthermore, the identification
information for the AV stream file that includes the base-view
stream may also be added to the video stream attribute
information. In this case, when verifying whether or not the
data that is to be recorded on the disc 101 has been created
according to a specified format during the authoring process
of the BD-ROM disc 101, the suitability of the correspondence
between the base-view stream and the dependent-view stream can
be assessed using the above-described additional information.
[0380]
(6) The 3D meta data 3814 shown in Fig. 3 9A can be set,
for example, for each plane, unlike the setting for each PID
as shown in the figure. In this way, the 3D playback device can
simplify analysis of the 3D meta data. Additionally, taking into
consideration the performance of the 3D playback device with
regards to the plane combining process, a condition of being,
for example, one second or longer may be placed on the length
of the valid section of the offset entry.
[0381]
(7) In the example shown in Fig. 40A, an extent start flag
4015 is newly added to each entry point 4012. Additionally, the
one bit area reserved in each entry point 4012 for an angle
switching flag may be used as the area for the extent start flag
4015. Here, the "angle switching flag" is a flag indicating
whether or not the entry point has been set at a position in
which the angle can be changed when the AV stream file is
expressed as multi-angle video images. By using this one bit
area as both an "angle switching flag" and an "extent start
flag, " the total bit volume in the entry map 3823 can be reduced.
In this case, another flag indicating whether the one bit area
is being used as an area for either flag can be newly added to
the entry map header 4011. By checking this flag, the 3D playback
device can promptly interpret the above-described one bit area.
Accordingly, the 3D playback device can promptly switch between
playback processing of multi-angle video images and playback
processing of 3D video images.
[0382]
(8) Unlike the example shown in Fig. 40A, information
other than the extent start flag 4015 can be used for determining
the size of extents belonging to an AV stream file. For example,
a list of the size of each extent can be stored in the clip
information file as meta data. Additionally, a plurality of bits
associated on a one-to-one basis with the entry points in the
entry map may be stored apart from the entry map. Each bit is
set to "1" only when the corresponding entry point is set to
the top of an extent.
[0383]
(9) In the interleaved arrangement shown in Fig. 49, the
three types of extents Dn, Rn, and Ln, which have matching
playback periods, preferably all have the same extent ATC time.
In other words, each of the extents has the same difference in
ATS from the top source packet in the extent to the top source
packet in the next extent. In this case, in the same time measured
by the ATC, the first source depacketizer 4611 fetches TS
packets from all of the source packets in the base-view extent
Ln and outputs them to the first PID filter 4613, and the second
source depacketizer 4612 fetches TS packets from all of the
source packets in the dependent-view extent Dn or Rn and outputs
them to the second PID filter 4614. Accordingly, in particular
during interrupt playback, the 3D video decoder 4615 can easily
synchronize decoding processing of TS packets between the
base-view stream and the dependent-view stream.
[0384]
(10) The extent groups shown in Fig. 49 may include,
instead of the depth map stream, two or more different
right-view streams. In this case, the extents belonging to each
stream would be arranged alternately on the track on the BD-ROM
disc 101 similarly to the interleaved arrangement shown in Fig.
49. Furthermore, the size of each extent would fulfill
conditions similar to Equations 4-8. Specific details are as
follows.
[0385]
First, for a shared base-view stream file, it is assumed
that N types of dependent-view stream files 1-N are recorded.
Here, an integer N is two or greater. In this case, a plurality
of base-view extents and a plurality of dependent-view extents
belonging to the (k-1)th dependent-view stream file (k=2, 3,
4 . . . N+1) are alternately arranged one by one on the BD-ROM
disc 101. In particular, similar to the interleaved arrangement
shown in Fig. 49, the nth extent in the (k-l)th dependent-view
stream file is placed between the nth base-view extent and the
(n+1)th base-view extent. To seamlessly play back video from
these extent groups, the size Sext1 [n] of the nth (n=2, 3, ...)
base-view extent and the size Sextk [n] of the nth extent in the
(k-l)th dependent-view stream file should fulfill the following
conditions.
[0386]
During playback of 2D video images, when the jump from
the nth (an integer n is sufficiently large) base-view extent
to the (n+1)th base-view extent requires a time length TjUmp-2D [n] ,
each base-view extent is read into the first read buffer 4221
at a rate RUd-2D/ and the base-view stream file is transferred
from the first read buffer 4221 to the system target decoder
4203 at a mean rate of Rext1-2D [n] , then the size Sext1 [n] of the
nth base-view extent should fulfill Equation 18.
[0387]
Equation 18

[0388]
During playback of 3D video images, when the jump from
the nth base-view extent to the nth extent in the (k-l)th
dependent-view stream file requires a time length TFjump-k-3D [n] ,
the jump from the nth extent in the (k-1)th dependent-view stream
file to the (n+1)th base-view extent requires a time length
TBjump-k-3D [n] , each base-view extent is read into the first read
buffer 4221 at a rate Rud-3D/ each extent in the (k-1)th
dependent-view stream file is read into the second read buffer
4222 at the rate RUd-3D/ and the base-view stream file is
transferred from the first read buffer 4221 to the system target
decoder 4203 at a mean rate of Rext1-3D[n] , the size Sext1 [n] of
the nth base-view extent should fulfill Equation 19.
[0389]
Equation 19

[0390]
Furthermore, in playback of 3D video images, when the jump
from the nth base-view extent to the (n+1)th extent in the (k-1)th
dependent-view stream file requires a time length TFjUmp-k-3D [n+1] ,
and the (k-1)th dependent-view stream file is transferred from
the second read buffer 4222 to the system target decoder 4203
at a mean rate of Rextk-3D [n] , the size Sextk [n] of the nth extent
in the (k-1)th dependent-view stream file should fulfill
Equation 20.
[0391]
Equation 20

[0392]
(11) When playing back 3D video images from the extent
groups shown in Fig. 49, the buffer margin times UL1 and UL2
are guaranteed by successively reading a certain number of
extents into the read buffers 4221 and 4222, as can be easily
understood from Figs. 57A and 57B. Accordingly, if there were
few extents placed between a starting position of interrupt
playback and a layer boundary LB, there would be a risk that
the accumulated data amounts DA1 and DA2 in the read buffers
4221 and 4222 might not reach the buffer margin amounts UL1 and
UL2 before the jump produced by layer switching. In order to
avoid that risk, interrupt playback should be prohibited in
extent groups located a predetermined number of extents
immediately before a layer boundary LB.
[0393]
Figs. 58A and 58B are schematic diagrams showing the
relationship between a section in which interrupt playback is
prohibited and the change of the accumulated data amount DA1
in the first read buffer 4221. As shown in Fig. 58A, the depth
map extent group D, right-view extent group R, and left-view
extent group L are recorded in an interleaved arrangement on
two recording layers separated by a layer boundary LB, similar
to the arrangement shown in Fig. 49. The interrupt playback
prohibited area 5801 is set for extent groups consecutively
located a predetermined number of extents immediately before
the layer boundary LB. Interrupt playback is prohibited in the
interrupt playback prohibited area 5801. In the example shown
in Fig. 58A, since the first entry point 5811 is located before
the interrupt playback prohibited area 5801, interrupt playback
at this point is allowed. On the other hand, since the second
entry point 5812 is located in the interrupt playback prohibited
area 5801, interrupt playback at this point is prohibited.
[0394]
Fig. 58B is a graph showing the change of the accumulated
data amount DA1 in the first read buffer 4221 during interrupt
playback from the first entry point 5811. This interrupt
playback is performed in L/R mode, and in accordance with the
playback path 5810 shown in Fig. 58A, the left-view extents L
included in the area from the first entry point 5811 to the layer
boundary LB are read into the first read buffer 4221. At that
time, as shown in Fig. 58B, each time a left-view extent L is
processed, the accumulated data amount DA1 increases. In
particular, when the left-view extent 5802 located immediately
before the layer boundary LB is read into the first read buffer
4221, the accumulated data amount DA1 exceeds the first buffer
margin amount UL1. Therefore, from the starting point of the
jump JLY that occurs immediately afterwards to the point when
the top of the next left-view extent 5803 is read, underflow
does not occur in the first read buffer 4221. The same is true
for the second read buffer 4222.
[0395]
Based on the above description, the length of the
interrupt playback prohibited area 58 01 should fulfill the
following two conditions: (A) the number of left-view extents
located from the top of the interrupt playback prohibited area
5801 to immediately before the layer boundary LB should exceed
the value yielded by dividing the first buffer margin amount
UL1 by the margin time, and (B) the number of right-view extents
located from the top of this area to immediately before the layer
boundary LB should exceed the value yielded by dividing the
second buffer margin amount UL2 by the margin time.
[0396]
The interrupt playback prohibited area 5801 is specified
to the 3D playback device in the following way. First, in the
clip information file corresponding to the AV stream file to
which each extent belongs, an interrupt playback prohibited
mark is added to each entry point 4 012 in the entry map 3 823
shown in Fig. 40A. The "interrupt playback prohibited mark" is
a flag and indicates whether or not the position in the extent
shown by the entry point is included in an interrupt playback
prohibited area 5801. When the 3D playback device is instructed
to perform interrupt playback of 3D video images by user
instruction or the like, the playback control unit 4207 shown
in Fig. 42 refers to the interrupt playback prohibited mark in
the clip information file. In this way, the 3D playback device
can determine whether or not the starting position of interrupt
playback is included in an interrupt playback prohibited area
5801. Note that the "interrupt playback prohibited mark" may
be included in the playitem information in the 3D playlist file
243B shown in Fig. 30. When instructed to perform interrupt
playback of 3D video images, the 3D playback device can
determine whether or not to permit interrupt playback by making
the playback control unit 4207 refer to the interrupt playback
prohibited mark in the 3D playlist file.
[0397]
(12) Extents belonging to a different file than the AV
stream files may be inserted in the extent groups in an
interleaved arrangement shown in Fig. 49. Such a different file
may, for example, be the movie object file 242B, the BD-J object
file 246A, or the JAR file 247A, which are shown in Fig. 2. Fig.
59 is a schematic diagram showing the arrangement of the extent
groups in such a case. As shown in Fig. 59, an extent Ex is
inserted between the nth left-view extent Ln and the (n+1)th depth
map extent D(n+1) . The file to which this extent Ex belongs is
different from either AV stream file. In this case, the
insertion of extents belonging to a file other than the AV stream
files is prohibited in m sets of extents arranged consecutively
immediately before the extent Ex, i.e. D(n-m+l), R(n-m+l) ,
L(n-m+l), D(n-m+2), R(n-m+2) , L(n-m+2), ..., Dn, Rn, Ln. Here,
three types of consecutive extents Dk, Rk, Lk (k=l, 2,3, etc.)
are defined as a set. Furthermore, the number m is at least
equivalent to the value yielded by dividing the buffer margin
amounts UL1 and UL2 by the margin time. When these m sets of
extents are read in order from the top into the read buffers
4221 and 4222, the accumulated data amounts DA1 and DA2 exceed
the buffer margin amounts UL1 and UL2. Accordingly, underflow
does not occur in the read buffers 4221 and 4222 while reading
of the extent Ex is being skipped by the immediately subsequent
jump.
[0398]
Here, in the regulation shown in Fig. 24, when the jump
distance exceeds 4 0,000 sectors, the corresponding jump time
becomes significantly longer. The buffer capacity necessary for
such a long jump time can easily become excessively large.
Accordingly, it is preferable that the size of the extent Ex
be set so that the interval between the left-view extents Ln
and L(n+1) placed immediately before and after the extent Ex
not exceed 4 0,000 sectors.
[0399]
(13) When playing back 3D video images from the extent
groups shown in Fig. 49, the buffer margin amounts UL1 and UL2
are guaranteed by successively reading a certain number of
extents into the read buffers 4221 and 4222, as can be easily
understood from Figs. 57A and 57B. Accordingly, interruption
by read processes of files other than AV stream files, such as
the BD-J object file 246A or the like, is prohibited from the
point at which reading of the AV stream files from the BD-ROM
disc 101 starts until a predetermined number of extents have
been read into the read buffers 4221 and 4222. Additionally,
the accumulated data amounts DA1 and DA2 in the read buffers
4221 and 4222 are monitored, and while either falls below the
corresponding buffer margin amount UL1 or UL2, interruption by
read processes of other files may be prohibited.
[0400]
In the example shown in Figs. 57A and 57B, when a certain
number of extents have been successively read into the read
buffers 4221 and 4222, the buffer margin amounts UL1 and UL2
are guaranteed. Additionally, the buffer margin amounts UL1 and
UL2 may be guaranteed immediately after reading of the AV stream
files into the read buffers 4221 and 4222 has begun using the
method described below. Immediately after the BD-ROM drive 4201
starts to read the AV stream files, this method makes the system
target decoder 4203 wait to perform decoding processing not only
during reading of the first dependent-view extent, but also
until a predetermined time has passed after reading of the first
base-view extent begins. This predetermined time is set so that,
during the standby period, a sufficient data amount will
accumulate in the read buffers 4221 and 4222. In this way, when
reading of the second base-view extent and dependent-view
extent begins, the accumulated data amounts DA1 and DA2 are
already maintained at the buffer margin amounts UL1 and UL2.
[0401]
Figs. 60A and 60B are graphs showing the change of the
accumulated data amounts DA1 and DA2 in the read buffers 4221
and 4222 when this method is used on the extent groups shown
in Fig. 49. In the following explanation, 3D video images are
assumed to be played back in L/R mode from the top of the extent
groups shown in Fig. 49. This explanation is also applicable
when 3D video images are played back in depth mode and when 3D
video images are played back from an arbitrary extent via
interrupt playback. Note that the size of each extent has
already been determined so as to fulfill Equations 5 and 6 . Here,
as described below, a margin time does not need to be included
in the zero sector transition time Tjump.0[.] to be substituted
into Equations 5 and 6.
[0402]
First, during the first read period PRr1, the top
right-view extent Rl is read from the BD-ROM disc 101 and output
to the second read buffer 4222. In the first read period PRR1,
as shown in Fig. 60B, the accumulated data amount DA2 in the
second read buffer 4222 increases from 0 at the 3D video image
read rate RUd-3D-
[0403]
At the end of the first read period PRR1, the first zero
sector transition period PJ01 starts. During the first zero
sector transition period PJol, reading of data from the BD-ROM
disc 101 is suspended. Therefore, the accumulated data amounts
DA1 and DA2 in the read buffers 4 221 and 4222 are both maintained
constant.
[0404]
At the point the first zero sector transition period PJ01
ends, the second read period PRL1 begins . During the second read
period PRlI, the top left-view extent L1 is read from the BD-ROM
disc 101 and output to the first read buffer 4221. However, the
system target decoder 4203 continues on standby until a
predetermined time dt has passed from the start of the second
read period PRL1. Accordingly, as shown in Fig. 60A, the
accumulated data amount DA1 in the first read buffer 4221
increases from 0 at the 3D video image read rate RUd-3D. Conversely,
as shown in Fig. 60B, the accumulated data amount DA2 in the
second read buffer 4222 is maintained constant.
[0405]
When the predetermined time dt has passed from the start
of the second read period PRL1, the system target decoder 4203
is activated and starts to read and decode source packets from
the read buffers 4221 and 4222. Accordingly, as shown in Fig.
60A, the rate of increase of the accumulated data DA1 in the
first read buffer 4221 decreases from the 3D video image read
rate RUd-3D by the left-view transfer rate Rext1-3D [1]. Conversely,
as shown in Fig. 6OB, the accumulated data amount DA2 in the
second read buffer 4222 decreases by the right-view transfer
rate Rext2 [1].
[0406]
When the end of the top left-view extent L1 is read, the
first jump Jlr1 is performed to skip over the immediately
subsequent depth map extent D2 to the next right-view extent
R2. During the first jump period PJlr1, reading of data from
the BD-ROM disc 101 is suspended. Therefore, as shown in Fig.
60A, the accumulated data amount DA1 in the first read buffer
4221 decreases at the left-view transfer rate Rext1-3D [1], whereas,
as shown in Fig. 60B, the accumulated data amount DA2 in the
second read buffer 4222 continues to decrease at the right-view
transfer rate Rext2 [1].
[0407]
Unlike Fig. 57B, in Fig. 60B, data is not transferred from
the second read buffer 4222 to the system target decoder 4203
until the predetermined time dt has passed from the start of
the second read period PRL1. In other words, the amount of time
that the accumulated data amount DA2 in the second read buffer
4222 decreases is shortened by the predetermined time dt. As
a result, the data amount transferred from the second read
buffer 4222 to the system target decoder 4203 from the second
read period PRL1 through the first jump period PJlrI is smaller
than the size Sext2 [1] of the top right-view extent R1 by the
second buffer margin amount UL2 . Accordingly, at the end of the
first jump period PJlr1, as shown in Fig. 60B, the accumulated
data amount DA2 in the second read buffer 4222 does not fall
below the second buffer margin amount UL2.
[0408]
At the point the first jump period PJlrI ends, the third
read period PRR2 begins. During the third read period PRR2, the
second right-view extent R2 is read from the BD-ROM disc 101
and output to the second read buffer 4222 . Accordingly, as shown
in Fig. 60B, the accumulated data amount DA2 in the second read
buffer 4222 increases at a rate equivalent to the difference
between the 3D video image read rate Rud-3D and the right-view
transfer rate Rext2[2] , or RUd-3D-Rext2 [2] . Conversely, as shown
in Fig. 60A, the accumulated data amount DA1 in the first read
buffer 4221 continues to decrease at the left-view transfer rate
Rextl-3D [1].
[0409]
When the end of the second right-view extent R2 is read,
the second zero sector transition period PJ02 starts. During
the second zero sector transition period PJ02, reading of data
from the BD-ROM disc 101 is suspended. Therefore, the
accumulated data amount DA1 in the first read buffer 4221
decreases at the left-view transfer rate Rext1-3D[2], and the
accumulated data amount DA2 in the second read buffer 4222
decreases at the right-view transfer rate Rext2 [2] .
[0410]
Unlike Fig. 57B, in Fig. 60B, data is not transferred from
the first read buffer 4221 to the system target decoder 4203
until the predetermined time dt has passed from the start of
the second read period PRL1. In other words, the transfer time
is shortened by the predetermined time dt. As a result, the data
amount transferred from the first read buffer 4221 to the system
target decoder 4203 from the second read period PRL1 through
the second zero sector transition period PJ02 is smaller than
the size SextitH of the top left-view extent L1 by the first
buffer margin amount UL1. Accordingly, at the end of the first
jump period PJ02, as shown in Fig. 60A, the accumulated data
amount DA1 in the first read buffer 4221 does not fall below
the first buffer margin amount UL1.
[0411]
After the second zero sector transition period PJ02, the
size of each extent should be set so that the accumulated data
amounts DA1 and DA2 in the read buffers 4221 and 4222 exhibit
the change shown in Figs. 53A-B. In other words, each time an
extent is processed, the accumulated data amounts DA1 and DA2
in the read buffers 4221 and 4222 should return to the respective
buffer margin amounts UL1 and UL2. Accordingly, a margin time
does not need to be included in the zero sector transition time
Tjump-0[.] substituted into Equations 5 and 6.
[0412]
During interrupt playback, the first extents to be read
are not limited to the top extents R1 and L1, but may be any
arbitrary extents Rn, Ln (excluding, however, extents in
interrupt playback prohibited areas) . Therefore, based on the
above-mentioned considerations, the predetermined time dt
should fulfill Equations 21 and 22 for an arbitrary integer n.
[0413]
Equation 21

Equation 22

[0414]
As shown by Fig. 60A, during the read process of the top
left-view extent L1, the accumulated data amount DA1 in the
first read buffer 4221 reaches its maximum value at the end of
the second read period PRL1. Accordingly, to read the nth
left-view extent Ln, the capacity RB1 [n] of the first read
buffer 4221 should fulfill Equation 23 instead of Equation 15.
[0415]
Equation 23

[0416]
On the other hand, as shown by Fig. 60B, during the read process
of the top right-view extent R1, the accumulated data amount
DA2 in the second read buffer 4222 should at least be equivalent
to the size of the top right-view extent R1. Accordingly, to
read the nth right-view extent Rn, the capacity RB2 [n] of the
second read buffer 4222 should fulfill the above-mentioned
Equation 16.
[0417]
[Embodiment 2]
[0418]
The following describes, as the second embodiment of the
present invention, a recording device and a recording method
for recording the recording medium of Embodiment 1 of the
present invention.
[0419]
The recording device described here is called an
authoring device. The authoring device, generally located at
a creation studio that creates movie contents to be distributed,
is used by authoring staff. First, in accordance with an
operation by the authoring staff, the recording apparatus
converts movie content into a digital stream compression
encoded in accordance with an MPEG specification, i.e. into an
AV stream file. Next, the recording device generates a scenario,
which is information defining how each title included in the
movie content is to be played back. Specifically, the scenario
includes the above-described dynamic scenario information and
static scenario information. Then, the recording device
generates a volume image or an update kit for a BD-ROM disc from
the aforementioned digital stream and scenario. Lastly, the
recording device records the volume image on the recording
medium in accordance with the arrangements of extents explained
in the afore-described Embodiment 1.
[0420]
Fig. 61 is a block diagram of an internal structure of
the above-described recording device. As shown in Fig. 61, the
recording device includes a video encoder 6101, a material
creation unit 6102, a scenario generation unit 6103, a BD
program creation unit 6104, a multiplex processing unit 6105,
a format processing unit 6106, and a database unit 6107.
[0421]
The database unit 6107 is a nonvolatile storage device
embedded in the recording device. Specifically, the database
unit 6107 is a hard disk drive (HDD) . Alternatively, the
database unit 6107 may be an external HDD connected to the
recording device, a nonvolatile semiconductor memory device
embedded in the recording device, or an external nonvolatile
semiconductor memory device connected to the recording device.
[0422]
The video encoder 6101 receives video data, such as
uncompressed bitmap data, from the authoring staff, and
compresses the received video data in accordance with a
compression/encoding scheme such as MPEG-4 AVC or MPEG-2. This
process converts primary video data into a primary video stream,
and secondary video data into a secondary video stream. In
particular, 3D video image data is converted into a 2D/left-view
stream or a right-view stream. As shown in Figs. 34A and 34B,
the video encoder 6101 forms the 2D/left-view stream as a
base-view stream by performing inter-picture predictive
encoding on the pictures included in the 2D/left-view stream.
On the other hand, the video encoder 6101 forms the right-view
stream as a dependent-view stream by performing inter-picture
predictive encoding on both the pictures included in the
2D/lef t-view stream and the pictures included in the right-view
stream. Alternatively, the right-view stream and the
2D/left-view stream may be formed as the base-view stream and
the dependent-view stream, respectively. The converted video
streams 6111 are stored into the database unit 6107.
[0423]
During the above-described process of inter-picture
predictive encoding, the video encoder 6101 further detects
motion vectors between left video images and right video images
and calculates depth information of each 3D video image based
on the detected motion vectors. The calculated depth
information of each 3D video image is organized into the frame
depth information 6110 that is stored in the database unit 6107 .
[0424]
Figs. 62A, 62B, and 62C are schematic diagrams showing
processes to calculate depth information from a pair of left
and right pictures by the video encoder 6101.
[0425]
The video encoder 6101 first compresses each picture
using the redundancy between the left and right pictures. At
that time, the video encoder 6101 compares an uncompressed left
picture and an uncompressed right picture on a per-macroblock
basis (each macroblock containing 8 x 8 or 16 x 16 pixels, with
all the macroblocks representing a matrix) so as to detect a
motion vector for each image in the two pictures. Specifically,
as shown in Figs. 62A and 62B, a left video picture 6201 and
a right video picture 6202 are each divided into tnacroblocks
6203, the entirety of which represents a matrix. Next, the areas
occupied by the image data in picture 6201 and picture 6202 are
compared for each macroblock 6203, and a motion vector between
these pieces of image data is detected based on the result of
the comparison. For example, the area occupied by image 6204
showing a "house" in picture 62 01 is substantially the same as
that in picture 6202. Accordingly, a motion vector is not
detected from such areas. On the other hand, the area occupied
by image 6205 showing a "circle" in picture 6201 is
substantially different from the area in picture 6202.
Accordingly, a motion vector indicating the displacement
between the images 6205 showing the "circles" in the pictures
6201 and 6202 is detected from these areas.
[0426]
The video encoder 6101 next makes use of the detected
motion vector not only when compressing the pictures 6201 and
6202, but also when calculating the binocular parallax
pertaining to a 3D video image constituted from the pieces of
image data 6204 and 6205. Furthermore, in accordance with the
binocular parallax thus obtained, the video encoder 6101
calculates the "depths" of each image, such as the images 6204
and 6205 of the "house" and "circle. " The information indicating
the depth of each image may be organized, for example, into a
matrix 6206 the same size as the matrix of the macroblocks in
pictures 6201 and 6202 as shown in Fig. 62C. The frame depth
information 6110 shown in Fig. 61 includes this matrix 6206.
In this matrix 6206, blocks 6207 are in one-to-one
correspondence with the macroblocks 6203 in pictures 6201 and
6202. Each block 6207 indicates the depth of the image shown
by the corresponding macroblocks 6203 by using, for example,
eight bits. In the example shown in Figs. 62A-C, the depth of
the image 6205 of the "circle" is stored in each of the blocks
in an area 6208 in the matrix 6206. This area 6208 corresponds
to the entire areas in the pictures 6201 and 6202 that show the
image 6205.
[0427]
Returning to Fig. 61, the material creation unit 6102
creates elementary streams other than video streams, such as
an audio stream 6112, a PG stream 6113, and an IG stream 6114,
and stores the created streams into the database unit 6107. For
example, the material creation unit 6102 receives uncompressed
LPCM audio data from the authoring staff, encodes the
uncompressed LPCM audio data in accordance with a
compression/encoding scheme such as AC-3, and converts the
encoded LPCM audio data into the audio stream 6112 . The material
creation unit 6102 also receives a subtitle information file
from the authoring staff and creates the PG stream 6113 in
accordance with the subtitle information file. The subtitle
information file defines image data for showing subtitles,
display timings of the subtitles, and visual effects to be added
to the subtitles (e.g., f ade-in and fade-out) . Furthermore, the
material creation unit 6102 receives bitmap data and a menu file
from the authoring staff and creates the IG stream 6114 in
accordance with the bitmap data and the menu file. The bitmap
data shows images that are to be presented on a menu. The menu
file defines how each button on the menu is to be transitioned
from one status to another, and visual effects to be added to
each button.
[0428]
The scenario generation unit 6103 creates BD-ROM scenario
data 6115 in accordance with an instruction that has been issued
by the authoring staff and received via GUI and then stores the
created BD-ROM scenario data 6115 in the database unit 6107.
The BD-ROM scenario data 6115 described here is a file group
that defines methods of playing back the elementary streams 6111
to 6114 stored in the database unit 6107 . Of the file group shown
in Fig. 2, the BD-ROM scenario data 6115 includes the index file
242A, the movie object file 242B, and the playlist files 243A-C.
The scenario generation unit 6103 further creates a parameter
file 6116 and transfers the created parameter file 6116 to the
multiplex processing unit 6105 . The parameter file 6116 defines,
from among the elementary streams 6111 to 6114 stored in the
database unit 6107, one or more streams to be multiplexed to
form each AV stream file.
[0429]
The BD program creation unit 6104 provides the authoring
staff with a programming environment for programming a BD-J
object and Java application programs. The BD program creation
unit 6104 receives a request from a user via GUI and creates
each program's source code according to the request. The BD
program creation unit 6104 further creates the BD-J object file
246A from the BD-J object and compresses the Java application
programs in the JAR file 247A. The files 246A and 247A are
transferred to the format processing unit 6106.
[0430]
Here, it is assumed that the BD-J object is programmed
in the following way: the BD-J object causes the program
execution unit 4206 shown in Fig. 42 to transfer graphics data
for GUI to the system target decoder 4203 . Furthermore, the BD-J
object causes the system target decoder 4203 to process the
graphics data as the image plane data 4306 shown in Fig. 43.
In this case, the BD program creation unit 6104 may set offset
information corresponding to the image plane data 4306 in the
BD-J object by using the frame depth information 6110 stored
in the database unit 6107.
[0431]
In accordance with the parameter file 6116, the multiplex
processing unit 6105 multiplexes each of the elementary streams
6111-6114 stored in the database unit 6107 to form a stream file
in an MPEG-2 TS format. More specifically, as shown in Fig. 5,
each of the elementary streams 6111 to 6114 is converted into
a source packet series, and the source packets included in each
series are assembled to construct a single stream file. In this
manner, the AV stream files 245A-C shown in Fig. 2 are created.
[0432]
In parallel with the aforementioned processing, the
multiplex processing unit 6105 creates the clip information
files which respectively correspond to the AV stream files
24 5A-C by the following procedure. First, the entry maps 1103
and 3823 shown in Figs. 13A and 40A are generated. Next, for
each AV stream file, the stream attribute information 1102 shown
in Fig. 12 is extracted from each elementary stream to be
multiplexed. Furthermore, as shown in Figs. 11, 38A, and 3 8B,
a pair of an entry map and a piece of stream attribute information
is associated with the clip information that indicates each AV
stream file.
[0433]
The format processing unit 6106 creates a BD-ROM disc
image 6120 of the directory structure 204 shown in Fig. 2 from
(i) the BD-ROM scenario data 6115 stored in the database unit
6107, (ii) a group of program files including, among others,
a BD-J object file created by the BD program creation unit 6104,
and (iii) AV stream files and clip information files generated
by the multiplex processing unit 6105. In this directory
structure 204, UDF is used as a file system.
[0434]
When creating a file entry of an AV stream file, the format
processing unit 6106 refers to the entry map of a corresponding
clip information file. In this manner, the SPN of each entry-
point is used for creation of allocation descriptors. In
particular, allocation descriptors in a file entry of an AV
stream file of 3D video images are created such that, with a
depth map extent at the start of the file, three types of extents
are arranged alternately, as shown in Fig. 49. Furthermore, the
size of each extent is set so as to fulfill Equations 4-8, and
the value of the logical address shown by each allocation
descriptor is determined accordingly.
[0435]
In addition, by using the frame depth information 6110
stored in the database unit 6107, the format processing unit
6106 creates the 3D meta data 3 814 shown in Figs. 3 9A and 3 9B
for each of the secondary video stream 6111, the PG stream 6113 ,
and the IG stream 6114 . Here, the positions of image data pieces
within left and right video frames are automatically adjusted
so that 3D video images represented by one stream avoid overlap
with 3D video images represented by other streams in the same
visual direction. Furthermore, an offset value for each video
frame is also automatically adjusted so that depths of 3D video
images represented by one stream avoid agreement with depths
of 3D video images represented by other streams.
[0436]
Thereafter, the BD-ROM disc image 6120 generated by the
format processing unit 6106 is converted into data suited for
pressing of a BD-ROM disc, then recorded on the master to be
utilized for creation of the BD-ROM disc. Mass production of
the BD-ROM disc 101 pertaining to Embodiment 1 of the present
invention is made possible by using the master in the press
process.
[0437]
Supplementary Explanation>
[0438]
«Data Distribution via Broadcasting or Communication
Circuit»
[0439]
The recording medium according to Embodiment 1 of the
present invention may be, in addition to an optical disc, a
general removable medium available as a package medium, such
as a portable semiconductor memory device including an SD memory
card. Also, Embodiment 1 describes an example of an optical disc
in which data has been recorded beforehand, namely, a
conventionally available read-only optical disc such as a
BD-ROM or a DVD-ROM. However, the embodiment of the present
invention is not limited to these. For example, when a terminal
device writes a 3D video content that has been distributed via
broadcasting or a network into a conventionally available
writable optical disc such as a BD-RE or a DVD-RAM, arrangement
of the extents according to the above-described embodiment may
be used. Here, the terminal device may be incorporated in a
playback device, or may be a device different from the playback
device.
[0440]
<>
[0441]
The following describes a data read unit of a playback
device in the case where a semiconductor memory card is used
as the recording medium according to Embodiment 1 of the present
invention instead of an optical disc.
[0442]
A part of the playback device that reads data from an
optical disc is composed of an optical disc drive, for example.
Compared with this, a part of the playback device that reads
data from a semiconductor memory card is composed of an
exclusive interface (I/F) . In more detail, a card slot is
provided with the playback device, and the I/F is mounted in
the card slot. When the semiconductor memory card is inserted
into the card slot, the semiconductor memory card is
electrically connected with the playback device via the I/F.
Furthermore, the data is read from the semiconductor memory card
to the playback device via the I/F.
[0443]
<>
[0444]
Here, the mechanism for protecting copyright of data
recorded on a BD-ROM disc is described, as an assumption for
the following supplementary explanation.
[0445]
From a standpoint, for example, of improving copyright
protection or confidentiality of data, there are cases in which
a part of the data recorded on the BD-ROM is encrypted. The
encrypted data is, for example, a video stream, an audio stream,
or other stream. In such a case, the encrypted data is decoded
in the following manner.
[0446]
The playback device has recorded thereon beforehand a
part of data necessary for generating a "key" to be used for
decoding the encrypted data recorded on the BD-ROM disc, namely,
a device key. On the other hand, the BD-ROM disc has recorded
thereon another part of the data necessary for generating the
"key, " namely, an MKB (Media Key Block) , and encrypted data of
the "key, " namely, an encrypted title key. The device key, the
MKB, and the encrypted title key are associated with one another,
and each are further associated with a particular identifier
written into a BCA 201 recorded on the BD-ROM disc 101 shown
in Fig. 2, namely, a volume ID. When the combination of the device
key, the MKB, the encrypted title key, and the volume ID is not
correct, the encrypted data cannot be decoded. In other words,
only when the combination is correct, the above-mentioned
"key," namely the title key, can be generated. Specifically,
the encrypted title key is first decrypted using the device key,
the MKB, and the volume ID. Only when the title key can be
obtained as a result of the decryption, the encrypted data can
be decoded using the title key as the above-mentioned "key."
[0447]
When a playback device tries to play back the encrypted
data recorded on the BD-ROM disc, the playback device cannot
play back the encrypted data unless the playback device has
stored thereon a device key that has been associated beforehand
with the encrypted title key, the MKB, the device, and the volume
ID recorded on the BD-ROM disc. This is because a key necessary
for decoding the encrypted data, namely a title key, can be
obtained only by decrypting the encrypted title key based on
the correct combination of the MKB, the device key, and the
volume ID.
[0448]
In order to protect the copyright of at least one of a
video stream and an audio stream that are to be recorded on a
BD-ROM disc, a stream to be protected is encrypted using the
title key, and the encrypted stream is recorded on the BD-ROM
disc. Next, a key is generated based on the combination of the
MKB, the device key, and the volume ID, and the title key is
encrypted using the key so as to be converted to an encrypted
title key. Furthermore, the MKB, the volume ID, and the
encrypted title key are recorded on the BD-ROM disc. Only a
playback device storing thereon the device key to be used for
generating the above-mentioned key can decode the encrypted
video stream and/or the encrypted audio stream recorded on the
BD-ROM disc using a decoder. In this manner, it is possible to
protect the copyright of the data recorded on the BD-ROM disc.
[0449]
The above-described mechanism for protecting the
copyright of the data recorded on the BD-ROM disc is applicable
to a recording medium other than the BD-ROM disc. For example,
the mechanism is applicable to a readable and writable
semiconductor memory device and in particular to a portable
semiconductor memory card such as an SD card.
[0450]
<>
[0468]
Embodiment 2 of the present invention is based on the
assumption that an AV stream file and a playlist file are
recorded on a BD-ROM disc using the prerecording technique of
the authoring system, and the recorded AV stream file and
playlist file are provided to users. Alternatively, it may be
possible to record, by performing real-time recording, the AV
stream file and the playlist file in a writable recording medium
such as a BD-RE disc, aBD-Rdisc, a hard disk, or a semiconductor
memory card (hereinafter, "BD-RE disc or the like") , and provide
the user with the recorded AV stream file and playlist file.
In such a case, the AV stream file may be a transport stream
that has been obtained as a result of real-time decoding of an
analog input signal performed by a recording device.
Alternatively, the AV stream file may be a transport stream
obtained as a result of partialization of a digitally input
transport stream performed by the recording device.
[0469]
The recording device performing real-time recording
includes a video encoder, an audio encoder, a multiplexer, and
a source packetizer. The video encoder encodes a video signal
to convert it into a video stream. The audio encoder encodes
an audio signal to convert it into an audio stream. The
multiplexer multiplexes the video stream and audio stream to
convert them into a digital stream in the MPEG-2 TS format. The
source packetizer converts TS packets in the digital stream in
MPEG-2 TS format into source packets. The recording device
stores each source packet in the AV stream file and writes the
AV stream file on the BD-RE disc or the like.
[0470]
In parallel with the processing of writing the AV stream
file, the control unit of the recording device generates a clip
information file and a playlist file in the memory and writes
the files on the BD-RE disc or the like. Specifically, when a
user requests performance of recording processing, the control
unit first generates a clip information file in accordance with
an AV stream file and writes the file on the BD-RE disc or the
like. In such a case, each time a head of a GOP of a video stream
is detected from a transport stream received from outside, or
each time a GOP of a video stream is generated by the video
encoder, the control unit acquires a PTS of an I picture
positioned at the head of the GOP and an SPN of the source packet
in which the head of the GOP is stored. The control unit further
stores a pair of the PTS and the SPN as one entry point in an
entry map of the clip information file. Here, an
"is_angle_change" flag is added to the entry point. The
is_angle_change flag is set to "on" when the head of the GOP
is an IDR picture, and "off" when the head of the GOP is not
an IDR picture. In the clip information file, stream attribute
information is further set in accordance with an attribute of
a stream to be recorded. In this manner, after writing the AV
stream file and the clip information file into the BD-RE disc
or the like, the control unit generates a playlist file using
the entry map in the clip information file, and writes the file
on the BD-RE disc or the like.
[0471]
<>
[0472]
The playback device according to Embodiment 1 of the
present invention may write a digital stream recorded on the
BD-ROM disc 101 on another recording medium via a managed copy.
Here, managed copy refers to a technique for permitting copy
of a digital stream, a playlist file, a clip information file,
and an application program from a read-only recording medium
such as a BD-ROM disc to a writable recording medium only in
the case where authentication with the server via communication
succeeds . Here, the writable recording medium may be a writable
optical disc such as a BD-R, a BD-RE, a DVD-R, a DVD-RW, and
a DVD-RAM, and a portable semiconductor memory device such as
a hard disk, an SD memory card, a Memory Stick (TM), a Compact
Flash (TM) , a Smart Media (TM) , and a Multimedia Card (TM) . A
managed copy allows for limitation of the number of backups of
data recorded on a read-only recording medium and for charging
for backups.
[0473]
When a managed copy is performed from a BD-ROM disc to
a BD-R disc or a BD-RE disc and the two discs have an equivalent
recording capacity, the bit streams recorded on the original
disc may be copied in order as they are.
[0474]
If a managed copy is performed between different types
of recording media, a trans code needs to be performed. Here,
a "trans code" refers to processing for adjusting a digital
stream recorded on the original disc to the application format
of a recording medium that is the copy destination. For example,
the trans code includes the process of converting an MPEG-2 TS
format into an MPEG-2 program stream format and the process of
reducing a bit rate of each of a video stream and an audio stream
and re-encoding the video stream and the audio stream. During
the trans code, an AV stream file, a clip information file, and
a playlist file need to be generated in the above-mentioned
real-time recording.
[0475]
<>
[0476]
Among the data structures in Embodiment 1 of the present
invention, a repeated structure "there is a plurality of pieces
of information having a predetermined type" is defined by
describing an initial value of a control variable and a cyclic
condition in a "for" sentence. Also, a data structure "if a
predetermined condition is satisfied, predetermined
information is defined" is defined by describing, in an "if"
sentence, the condition and a variable to be set at the time
when the condition is satisfied. In this manner, the data
structure described in Embodiment 1 is described using a high
level programming language. Accordingly, the data structure is
converted by a computer into a computer readable code via the
translation process performed by a compiler, which includes
"syntax analysis, " "optimization, " "resource allocation, " and
"code generation, " and the data structure is then recorded on
the recording medium. By being described in a high level
programming language, the data structure is treated as a part
other than the method of the class structure in an
object-oriented language, specifically, as an array type member
variable of the class structure, and constitutes a part of the
program. In other words, the data structure is substantially
equivalent to a program. Therefore, the data structure needs
to be protected as a computer related invention.
[0477]
<>
[0478]
When a playlist file and an AV stream file are recorded
on a recording medium, a playback program is recorded on the
recording medium in an executable format. The playback program
makes the computer play back the AV stream file in accordance
with the playlist file. The playback program is loaded from a
recording medium to a memory device of a computer and is then
executed by the computer. The loading process includes compile
processing or link processing. By these processes, the playback
program is divided into a plurality of sections in the memory
device. The sections include a text section, a data section,
a bss section, and a stack section. The text section includes
a code array of the playback program, an initial value, and
non-rewritable data. The data section includes variables with
initial values and rewritable data. In particular, the data
section includes a file, recorded on the recording device, that
can be accessed at any time. The bss section includes variables
having no initial value. The data included in the bss section
is referenced in accordance with commands indicated by the code
in the text section. During the compile processing or link
processing, an area for the bss section is set aside in the
computer's internal RAM. The stack section is a memory area
temporarily set aside as necessary. During each of the processes
by the playback program, local variables are temporarily used.
The stack section includes these local variables. When the
program is executed, the variables in the bss section are
initially set at zero, and the necessary memory area is set aside
in the stack section.
[0479]
As described above, the playlist file and the clip
information file are already converted on the recording device
into computer readable code. Accordingly, at the time of
execution of the playback program, these files are each managed
as "non-rewritable data" in the text section or as a "file
accessed at any time" in the data section. In other words, the
playlist file and the clip information file are each included
as a compositional element of the playback program at the time
of execution thereof. Therefore, the playlist file and the clip
information file fulfill a greater role in the playback program
than mere presentation of data.
[0480]
<>
[0481]
A playback device according to Embodiment 1 of the present
invention includes middleware, a system LSI, and hardware other
than the system LSI. The playback device further includes an
interface for the middleware, an interface between the
middleware and the system LSI, an interface between the
middleware and other hardware, and a user interface. When these
elements are incorporated in a playback device, they operate
in cooperation with one another. As a result, each element
provides the playback device with a particular function. By
appropriately defining the interface for the middleware and the
interface between the middleware and the system LSI, it is
possible to develop the user interface, middleware, and system
LSI in the playback device independently, in parallel, and
efficiently. Note that in each interface, there is generally
a variety of such appropriate definitions.
[Industrial Applicability]
[0482]
The present invention relates to a technique for playing
back stereoscopic video images . As described above, the present
invention provides clear lower limits for the base-view extents
and the dependent-view extents recorded on a recording medium.
It is thus clear that the present invention is industrially
applicable.
[Reference Signs List]
[0483]
DA1 Accumulated data amount in first read buffer
DA2 Accumulated data amount in second read buffer
UL1 Buffer margin amount for first read buffer
UL2 Buffer margin amount for second read buffer
Dn nth depth map extent
Rn nth right-view extent
Ln nth left-view extent
PRRn Read period for nth right-view extent
PJ0n Zero sector transition period immediately after nth
right-view extent
PRLn Read period for nth left-view extent
PJLRn Jump period immediately after nth left-view extent
J0n Zero sector transition immediately after nth
right-view extent
JLRn Jump immediately after nth left-view extent
We claim
1. A recording medium on which a base-view stream file used for
monoscopic video playback and a dependent-view stream file used
for stereoscopic video playback in combination with the
base-view stream file are recorded,
the recording medium comprising a contiguous area in
which a plurality of base-view extents belonging to the
base-view stream file and a plurality of dependent-view extents
belonging to the dependent-view stream file are arranged in an
interleaved manner, wherein
a lower limit of a size of each base-view extent is a larger
one of a first lower limit value and a second lower limit value,
the first lower limit value is determined so that, in
monoscopic video playback, buffer underflow does not occur in
a playback device during a jump period from each base-view
extent to a next base-view extent,
the second lower limit value is determined so that, in
stereoscopic video playback, buffer underflow does not occur
in a playback device during a read period of a next
dependent-view extent after each base-view extent,
a lower limit of a size of each dependent-view extent is
a third lower limit value, and
the third lower limit value is determined so that, in
stereoscopic video playback, buffer underflow does not occur
in a playback device during a read period of a next base-view
extent after each dependent-view extent.
2. The recording medium in Claim 1, wherein
a dependent-view extent is arranged at the head of the
contiguous area,
an nth base-view extent has a si ze Sext1 [n] , where an integer
n is larger than one,
during monoscopic video playback by a first playback
device that includes a first read buffer and a first decoder,
when (i) a jump time Tjump-2D(n) from the nth base-view extent
to an (n+1)th base-view extent is required, (ii) each base-view
extent is read into the first read buffer at a rate RUd-2D, and
(iii) the base-view stream file is transferred from the first
read buffer to the first decoder at a mean rate Rext1-2D, then
the first lower limit value is expressed by the right-hand side
of the following equation,

an (n+1)th dependent-view extent located next after the
nth base-view extent has a size Sext2 [n+1] ,
during stereoscopic video playback by a second playback
device that includes a second read buffer, a third read buffer,
and a second decoder, when (i) each base-view extent is read
into the second read buffer at a rate RUd-3D, (ii) each
dependent-view extent is read into the third read buffer at a
rate RUd-3D, and (iii) the base-view stream file is transferred
from the second read buffer to the second decoder at a mean rate
Rext1-3D, then the second lower limit value is expressed by the
right-hand side of the following equation,

during the stereoscopic video playback, when the
dependent-view stream file is transferred from the third read
buffer to the second decoder at a mean rate Rext2-3D, then the
third lower limit value is expressed by the right-hand side of
the following equation,

3. A playback device for playing back video images from a
recording medium, the playback device comprising:
a reading unit operable to read, from the recording medium,
a base-view stream file used for monoscopic video playback and
a dependent-view stream file used for stereoscopic video
playback in combination with the base-view stream file;
a separation unit operable to separate extents from the
read base-view stream file and the read dependent-view stream
file;
a first read buffer storing, from among separated extents,
base-view extents belonging to the base-view stream file;
a second read buffer storing, from among separated
extents, dependent-view extents belonging to the
dependent-view stream file; and
a decoder operable to receive, from each read buffer,
compressed pictures included in each extent and to decode the
compressed pictures,
the recording medium comprising a contiguous area in
which a plurality of base-view extents and a plurality of
dependent-view extents are arranged in an interleaved manner,
wherein
a lower limit of a size of each base-view extent is a larger
one of a first lower limit value and a second lower limit value,
the first lower limit value is determined so that, in
monoscopic video playback, underflow does not occur in the first
read buffer during a jump period from each base-view extent to
a next base-view extent,
the second lower limit value is determined so that, in
stereoscopic video playback, underflow does not occur in the
first read buffer during a read period of a next dependent-view
extent after each base-view extent,
a lower limit of a size of each dependent-view extent is
a third lower limit value, and
the third lower limit value is determined so that, in
stereoscopic video playback, underflow does not occur in the
second read buffer during a read period of a next base-view
extent after each dependent-view extent.
4 . An integrated circuit loaded on a playback device for playing
back video images from a recording medium, the playback device
comprising:
a reading unit operable to read, from the recording medium,
a base-view stream file used for monoscopic video playback and
a dependent-view stream file used for stereoscopic video
playback in combination with the base-view stream file;
a separation unit operable to separate extents from the
read base-view stream file and the read dependent-view stream
file;
a first read buffer storing, from among separated extents,
base-view extents belonging to the base-view stream file; and
a second read buffer storing, from among separated
extents, dependent-view extents belonging to the
dependent-view stream file,
the integrated circuit comprising:
a decoder operable to receive, from each read buffer,
compressed pictures included in each extent and to decode the
compressed pictures; and
a control unit operable to control provision of
compressed pictures from each read buffer to the decoder,
the recording medium comprising a contiguous area in
which a plurality of base-view extents and a plurality of
dependent-view extents are arranged in an interleaved manner,
wherein
a lower limit of a size of each base-view extent is a larger
one of a first lower limit value and a second lower limit value,
the first lower limit value is determined so that, in
monoscopic video playback, underflow does not occur in the first
read buffer during a jump period from each base-view extent to
a next base-view extent,
the second lower limit value is determined so that, in
stereoscopic video playback, underflow does not occur in the
first read buffer during a read period of a next dependent-view
extent after each base-view extent,
a lower limit of a size of each dependent-view extent is
a third lower limit value, and
the third lower limit value is determined so that, in
stereoscopic video playback, underflow does not occur in the
second read buffer during read period of a next base-view extent
after each dependent-view extent.

Provided is a recording medium in which extents for a
base-view and a dependent-view are arranged alternately. The
lower limit of the size of the base-view extents is the larger
of a first lower limit value and a second lower limit value.
The lower limit of the size of the dependent-view extent is a
third lower limit value. The first lower limit value is
determined so that, in monoscopic video playback, buffer
underflow does not occur during jump periods between base-view
extents. The second lower limit value is determined so that,
in stereoscopic video playback, buffer underflow does not occur
during read periods from each base-view extent to a next
dependent-view extent. The third lower limit value is
determined so that, in stereoscopic video playback, buffer
underflow does not occur during read periods from each
dependent-view extent to the next base-view extent.