"A Playback Apparatus And A Playback Method For Obtaining Playback Management Information"
Abstract:
The present invention relates to a playback apparatus, a playback method, a program, a recording medium, and a data structure that enable interactive operations when playing back AV content. A controller 34 obtains an order list of audio stream numbers in advance. In response to an instruction to switch audio by a user, the controller obtains the audio stream number subsequent to the audio stream number which is being played back. It is then checked whether a stream that is found to be playable by the playback apparatus is included in a main clip or a sub clip, and the main clip referred to by the Main Path is read together with the clip in which the corresponding audio stream is multiplexed. Then, the audio stream file of the corresponding clip and the file to be played back included in the main clip are selected by the switches 57 through 59, and 77. The selected files are combined by a video data processor 96 and an audio data processor 97, and the combined data is output. The present invention is applicable to playback apparatuses.
Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence
7-35, Kitashinagawa 6-chome, Shinagawa-ku, Tokyo, 141-0001, JAPAN.
Inventors
1. MOTOKI KATO
C/O SONY CORPORATION
7-35, KITASHINAGAWA 6-CHOME, SHINAGAWA-KU, TOKYO JAPAN.
2. TOSHIYA HAMADA,
C/O SONY CORPORATION
7-35, KITASHINAGAWA 6-CHOME, SHINAGAWA-KU, TOKYO JAPAN.
Specification
The present invention relates to a playback apparatus and a playback method for obtaining playback management information
Technical Field (0001}
The present invention relates to playback apparatuses, playback methods, programs, recording media, and data structures. More particularly, the invention relates to a playback apparatus, a playback method, a program, a recording medium, and a data structure that enable interactive operations when AV content is played back. Background Art [0002]
In DVD (Digital Versatile Disc) video standards, interactive operations can be performed, that is, users can switch sound or subtitles when playing back AV (Audio Visual) content, such as a movie, recorded on an information recording medium [for example, see Non-Patent Document 1). More specifically, the user operates a sound switch button 11 or a subtitle switch button 12 of a remote controller 2 to switch sound or subtitles of AV content displayed on a display device 1 shown in Fig. 1. For example, if the user operates the sound switch button 11 when sound 1 is set in the initial state in the display device 1, sound 1 is
switched to sound 2, as shown in Fig. 2.
[0003]
AV content on DVD video is recorded in the form of an MPEG (Moving Picture Experts Group)2 program stream. In the MPEG2 program stream, as shown in Fig. 3, a video stream
(Video in Fig. 3), a plurality of audio streams (indicated by Audio I, 2, and 3 in Fig. 3), and a plurality of sub-picture streams (Sub-pictures 1, 2, and 3) are multiplexed such that the audio streams and the sub-picture streams are AV-synchronized with the video stream. The sub-picture streams (sub-pictures 1, 2, and 3) are streams in which bitmap images are run-length coded, and are mainly used for subtitles.
[0004]
Generally, a plurality of audio streams are used for recording sound of different languages, and a plurality of sub-picture streams are used for recording subtitles of different languages. The user can interactively select sound or subtitles of a desired language by using the remote controller 2 while video is being played back.
[0005]
The DVD video defines a table structure, provided to users, indicating the relationship between sound numbers and subtitle numbers for a plurality of audio streams (audio 1, 2, and 3) and a plurality of sub-picture streams (sub-
pictures I, and 3) in a program stream. [0006]
Fig. 4 illustrates a stream number table indicating the relationship between audio signals and subtitle signals. In this table, the sound numbers are referred to as "A_SN (Audio Stream Number)", and the subtitle numbers are referred to as "S_SN (Sub-picture Stream Number)". In Fig. 4, each of the plurality of audio streams is provided with A_SN, and each of the plurality of sub-picture streams is provided with S_SN. More specifically, A_SN=1: audio 2, A_SN=2: audio 1, and A_SN-3: audio 3. Also, S_SN=1: sub-picture 3, S_SN=2: sub-picture 1, and S_SN=3: sub-picture 2. In this case, a smaller number of A_SN or S_SN indicates an audio signal or a subtitle signal to be provided to users with higher priority. That is, A_SN=1 is an audio stream played back as a default, and S_SN=1 is a sub-picture stream played back as a default. [0007]
More specifically, sound 1 played back in the initial state in Fig. 1 is audio 2, which is A_SN=1 (Fig. 4), and sound 2 played back after being switched from sound 1 in Fig. 2 is audio 1, which is A_SN=2.
Non-Patent Document 1: DVD Specifications for Read-Only Disc Part 3; Version 1.1 Disclosure of Invention
Problems to be Solved by the Invention [0008]
According to the DVD video, however, when the user switches sound or subtitles while playing back a video program stream, the user can select only from audio streams or sub-picture streams multiplexed into the program stream which is currently played back. That is, when playing back an MPEG2 program stream, such as that shown in Fig. 3, the user can select only from audio 1 through audio 3 when switching sound. [0009]
Accordingly, even if another stream having audio streams and subtitles different from a program stream which is currently played back is available, the user cannot switch sound or subtitles to the audio streams or subtitles in the different stream. Accordingly, the extensibility in selecting streams is low. [0010]
The present invention has been made in view of the above background. An object of the present invention is to select sound and subtitles from streams or data files different from a main AV stream when the user switches sound or subtitles.
Means for Solving the Problems [0011]
A playback apparatus of the present invention includes: obtaining means for obtaining playback management information including first information having a main playback path indicating the position of an AV stream file recorded on a recording medium and second information having a plurality of sub playback paths indicating the positions of sub files including accompanying data to be played back simultaneously with playback of main image data included in the AV stream file; selection means for selecting accompanying data to be played back, based on an instruction from a user, from among accompanying data to be played back simultaneously with the main image data included in the AV stream file referred to by the main playback path and the accompanying data included in the sub files referred to by the sub playback paths; reading means for reading, if the accompanying data selected by the selection means is included in a sub file referred to by a sub playback path, the sub file referred to by the sub playback path together with the AV stream file referred to by the main playback path; and playback means for playing back the main image data included in the AV stream file read by the reading means and the accompanying data included in the sub file selected by the selection means and read by the reading means. [0012]
The first information may include a table defining the accompanying data included in the AV stream file referred to by the main playback path and the accompanying data referred to by the sub playback paths, and the selection means may select the accompanying data to be played back, based on the instruction from the user, from among the accompanying data defined in the table. [0013]
The playback apparatus may further include determining means for determining whether the playback apparatus has a function of playing back the accompanying data selected by the selection means. If it is determined by the determining means that the playback apparatus has a function of playing back the accompanying data and if the accompanying data is included in a sub file referred to by a sub playback path, the reading means may read the sub file referred to by the sub playback path together with the AV stream file referred to by the main playback path, and the playback means may play back the main image data included in the AV stream file read by the reading means and the accompanying data included in the sub file selected by the selection means and read by the reading means. [0014]
The playback apparatus may further include determining means for determining whether the playback apparatus has a
function of playing back the accompanying data selected by the selection means. If it is determined by the determining means that the playback apparatus has a function of playing back the accompanying data and if the accompanying data is included in a sub file referred to by a sub playback path, the reading means may read the sub file referred to by the sub playback path together with the AV stream file referred to by the main playback path, and the playback means may play back the main image data included in the AV stream file read by the reading means and the accompanying data included in the sub file selected by the selection means and read by the reading means. [0015]
The table may further define accompanying information concerning accompanying data, and the determining means may determine whether the playback apparatus has a function of playing back the accompanying data based on attribute information concerning the accompanying data defined in the table. [0016]
The second information may include type information concerning the types of sub playback paths, file names -of the sub files referred to by the sub playback paths, and IN points and OUT points of the sub files referred to by the sub playback paths.
[0017]
The second information may further include specifying information for specifying the AV stream file referred to by the main playback path to play back the sub playback paths simultaneously with the main playback path, and a time on the main playback path for allowing the IN points to be started in synchronization with the main playback path on the time axis of the main playback path.
[0018]
A playback method of the present invention includes: an obtaining step of obtaining playback management information including first information having a main playback path indicating the position of an AV stream file recorded on a recording medium and second information having a plurality of sub playback paths indicating positions of sub files including accompanying data to be played back simultaneously with playback of main image data included in the AV stream file; a selection step of selecting accompanying data to be played back, based on an instruction from a user, from among accompanying data to be played back simultaneously with the main image data included in the AV stream file referred to by the main playback path and the accompanying data included in the sub files referred to by the sub playback paths; a reading step of reading, if the accompanying data selected by processing of the selection step is included in a sub
file referred to by a sub playback path, the sub file referred to by the sub playback path together with the AV stream file referred to by the main playback path; and a playback step of playing back the main image data included in the AV stream file read by processing of the reading step and the accompanying data included in the sub file selected by processing of the selection step and read by processing of the reading step. [0019]
A program of the present invention includes: an obtaining step of obtaining playback management information including first information having a main playback path indicating the position of an AV stream file recorded on a recording medium and second information having a plurality of sub playback paths indicating the positions of sub files including accompanying data to be played back simultaneously with playback of main image data included in the AV stream file; a selection step of selecting accompanying data to be played back, based on an instruction from a user, from among accompanying data to be played back simultaneously with the main image data included in the AV stream file referred to by the main playback path and the accompanying data included in the sub files referred to by the sub playback paths; a reading step of reading, if the accompanying data selectedby processing of the selection step is included in a sub
file referred to by a sub playback path, the sub file referred to by the sub playback path together with the AV stream file referred to by the main playback path; and a playback step of playing back the main image data included in the AV stream file read by processing of the reading step and the accompanying data included in the sub file selected by processing of the selection step and read by processing of the reading step. [0020]
According to a first aspect of the present invention, playback management information including first information having a main playback path indicating the position of an AV stream file recorded on a recording medium and second information having a plurality of sub playback paths indicating the positions of sub files including accompanying data to be played back simultaneously with playback of main image data included in the AV stream file is obtained. Then, accompanying data to be played back is selected, based on an instruction from a user, from among accompanying data to be played back simultaneously with the main image data included in the AV stream file referred to by the main playback path and the accompanying data included in the sub files referred to by the sub playback paths. If the selected accompanying data is included in a sub file referred to by a sub playback path, the sub file referred to by the sub playback path is
read together with the AV stream file referred to by the main playback path. Then, main image data included in the read AV stream file and the accompanying data included in the selected and read sub file are played back. [0021]
Association data recorded on a first recording medium of the present invention indicates whether accompanying data is included in a clip used by a main playback path indicating the position of the AV stream file or in clips used by a plurality of sub playback paths indicating the positions of sub files including the accompanying data played back simultaneously with the AV stream file. If the association data indicates that the accompanying data is included in the clips used by the plurality of sub playback paths indicating the positions of the sub files including the accompanying data, the association data includes at least an ID of the sub playback path to be played back selected from the ID for specifying the sub playback path to be played back, an ID for specifying a clip used by the sub playback path, and an ID for specifying an elementary stream to be played back by the clip. [0022]
According to a second aspect of the present invention, the association data indicates whether accompanying data is included in a clip used by a main playback path indicating
the position of the AV stream file or in clips used by a plurality of sub playback paths indicating the positions of sub files including the accompanying data played back simultaneously with the AV stream file. If the association data indicates that the accompanying data is included in the clips used by the plurality of sub playback paths indicating the positions of the sub files including the accompanying data, the association data includes at least an ID of the sub playback path to be played back selected from the ID for specifying the sub playback path to be played back, an ID for specifying a clip used by the sub playback path, and an ID for specifying an elementary stream to be played back by the clip. [0023]
A playback control file recorded on a second recording medium of the present invention includes a sub playback path indicating the position of a sub file including accompanying data to be played back simultaneously with main image data included in an AV stream file. The main playback path includes a table defining a list of elementary streams that are selectable while the main playback path is being played back. The table includes data indicating whether the selectable elementary streams are included in the AV stream file selected by the main playback path or the sub file selected by the sub playback path.
[0024]
According to a data structure of the present invention, a playback control file includes a sub playback path indicating the position of a sub file including accompanying data to be played back simultaneously with main image data included in an AV stream file. The main playback path includes a table defining a list of elementary streams that are selectable while the main playback path is being played back. The table includes data indicating whether the selectable elementary streams are included in the AV stream file selected by the main playback path or the sub file selected by the sub playback path.
[0025]
According to a third aspect of the present invention, a playback control file includes a sub playback path indicating the position of a sub file including accompanying data to be played back simultaneously with main image data included in an AV stream file. The main playback path includes a table defining a list of elementary streams that are selectable while the main playback path is being played back. The table includes data indicating whether the selectable elementary streams are included in the AV stream file selected by the main playback path or the sub file selected by the sub playback path. Advantages
[0026]
According to the present invention, interactive operations can be performed when playing back an AV stream file. In particular, according to the present invention, interactive operations can also be performed on sub files referred to by sub playback paths, which are different from the AV stream file referred to by the main playback path. Brief Description of the Drawings [0027]
[Fig. 1] Fig. 1 illustrates known sound switching.
[Fig. 2] Fig. 2 illustrates known sound switching.
[Fig. 3] Fig. 3 illustrates the structure of an MPEG2 program stream.
[Fig. 4] Fig. 4 illustrates a stream number table indicating the relationship between sound signals and subtitle signals provided to a user.
[Fig. 5] Fig. 5 illustrates an example of an application format on a recording medium installed in a playback apparatus to which the present invention is applied.
[Fig. 6] Fig. 6 illustrates the structure of Main Path and Sub Path.
[Fig. 7] Fig. 7 illustrates an example of Main Path and Sub Path.
[Fig. 8] Fig. 8 illustrates another example of Main Path and Sub Path.
[Fig. 9] Fig. 9 illustrates still another example of Main Path and Sub Path.
[Fig. 10] Fig. 10 illustrates another example of Main
Path and Sub Path.
[Fig. 11] Fig. 11 illustrates the syntax of
PlayList().
[Fig. 12] Fig. 12 illustrates the syntax of SubPathO.
[Fig. 13] Fig. 13 illustrates the syntax of
SubPlayltem(i).
[Fig. 14] Fig. 14 illustrates the syntax of
PlayltemO .
[Fig. 15] Fig. 15 illustrates the syntax of
STN_table().
[Fig. 16] Fig. 16 illustrates an example of the syntax
of stream_entry().
[Fig. 17] Fig. 17 illustrates the syntax of
stream_attribute().
[Fig. 18] Fig. 18 illustrates stream_cording_type.
[Fig. 19] Fig. 19 illustrates video_format.
[Fig. 20] Fig. 20 illustrates frame_rate.
[Fig. 21] Fig. 21 illustrates aspect_ratio.
[Fig. 22] Fig. 22 illustrates audio_presentation_type.
[Fig. 23] Fig. 23 illustrates sampling_frequency.
[Fig. 24] Fig. 24 illustrates Character code.
[Fig. 25] Fig. 25 is a block diagram illustrating an
example of the configuration of a playback apparatus to which the present invention is applied.
[Fig. 26] Fig. 26 is a flowchart illustrating playback processing performed by the playback apparatus shown in Fig. 25.
[Fig. 27] Fig. 27 is a flowchart illustrating the playback processing performed by the playback apparatus shown in Fig. 25.
[Fig. 28] Fig. 28 is a flowchart illustrating the playback processing performed by the playback apparatus shown in Fig. 25.
[Fig. 29] Fig. 29 is a flowchart illustrating processing when an instruction to switch audio is given by a user.
[Fig. 30] Fig. 30 is a flowchart illustrating processing when an instruction to switch subtitles is given by a user.
[Fig. 31] Fig. 31 illustrates the configuration of a personal computer.
[Fig. 32A] Fig. 32A illustrates another example of the syntax of PlayListO .
[Fig. 32B] Fig. 32B illustrates another example of the syntax of PlayListO •
[Fig. 33] Fig. 33 illustrates another example of the syntax of STN_table().
[Fig. 34] Fig. 34 illustrates the types in STN_table() shown in Fig. 33. Reference Numerals [0028]
20 playback apparatus, 31 storage drive, 32 switch, 33 AV decoder, 34 controller, 51 through 54 buffers, 55, 56 PID filters, 57 through 59 switches, 71 background decoder, 72 MPEG2 video decoder, 73 presentation graphics decoder, 74 interactive graphics decoder, 75 audio decoder, 76 Text-ST composition, 77 switch, 91 background plane generator, 92 video plane generator, 93 presentation graphics plane generator, 94 interactive graphics plane generator, 95 buffer, 96 video data processor, 97 audio data processor
Best Mode for Carrying Out the Invention [0029]
An embodiment of the present invention is described below with reference to the accompanying drawings. [0030]
Fig. 5 illustrates an example of an application format on a recording medium to be installed in a playback apparatus 20 (which is discussed below with reference to Fig. 25) to which the present invention is applied. The recording medium may be an optical disc, a magnetic disk, or a semiconductor memory, which is discussed below.
[0031]
The application format has two layers, i.e., PlayList and Clip, for managing AV (Audio Visual) streams. In this case, a pair of one AV stream and one item of clip information accompanying the AV stream are considered to be as one object, which is referred to as a "clip". An AV stream is also referred to as an "AV stream file". Clip information is also referred to as a "clip information file". [0032]
Generally, files used in computers are handled as byte strings. On the other hand, content of AV stream files is expanded onto a time axis, and access points in clips are mainly specified by PlayLists by using time stamps. That is, it can be said that PlayList and Clip form two layers for managing AV streams. [0033]
If access points in Clip are indicated by PlayList by using time stamps, a clip information file is used for finding, from the time stamps, information concerning an address at which decoding in an AV stream file is started. [0034]
PlayList is a set of playback zones of an AV stream. One playback zone in an AV stream is referred to as "Playltem", which is indicated by a pair of an IN point (playback start point) and an OUT point (playback end point)
on the time axis. Accordingly, PlayList has one or a plurality of Playltems, as shown in Fig. 5.
[0035]
In Fig. 5, the first PlayList from the left has two Playltems, which refer to the first half and the second half of the AV stream contained in the clip at the left side in Fig. 5. The second PlayList from the left has one Playltem, which refers to the entirety of the AV stream contained in the clip at the right side. The third PlayList from the left has two Playltems, which refer to a certain portion of the AV stream contained in the clip at the left side and a certain portion of the AV stream contained in the clip at the right side.
[0036]
If a disc navigation program shown in Fig. 5 designates the left Playltem contained in the first PlayList from the left as information concerning the current playback position, the first half of the AV stream contained in the left clip, which is referred to by the designated Playltem, is played back.
[0037]
The disc navigation program has a function of controlling the playback order specified in PlayList and interactive playback operations by using PlayList. The disc navigation program also has a function of displaying a menu
creen for allowing a user to give instructions to perform various types of playback operations. The disc navigation program is described in a programming language, for example, Java™, and is recorded on a recording medium. [0038]
In this embodiment, a playback path including at least one Playltem (sequential Playltems) in PlayList is referred to as a "Main Path", and a playback path including at least one Sub Path (formed by sequential or non-sequential SubPlayltems) disposed in parallel with the Main Path in a PlayList is referred to as a "Sub Path". That is, the application format on a recording medium installed in the playback apparatus 20 (which is discussed below with reference to Fig. 25) has at least one Sub Path, which is played back in association with the Main Path, in a PlayList [0039]
Fig. 6 illustrates the structure of the Main Path and Sub Path. The PlayList is allowed to have the single Main Path and at least one Sub Path. One Sub Path includes at least one SubPlayltem. [0040]
The PlayList shown in Fig. 6 has one Main Path including three Playltems and three Sub Paths. The Playltems forming the Main Path are provided with IDs (Identifications) from the head. More specifically, the
Main Path includes Playltems, such as Playltem_id=0, Playltem_id=ll and Playltem_id=2. The Sub Path are also provided with IDs from the head, such as Subpath_id=0, Subpath_id=l, and Sub Path_id=2. Subpath_id=0 has one SubPlayltem, Subpath_id=l has two SubPlaylterns, and Sub Path_id=2 has one SubPlayltem. [0041]
Sub Path_id=l is applied to, for example, Director's Cut, and can be inserted into a predetermined portion of the AV stream file as the director's commentaries. [0042]
A clip AV stream file referred to by one Playltem includes at least video stream data (main image data). The clip AV stream file may also include at least one audio stream, which is played back simultaneously with (in synchronization with) the video stream (main image data) which is also contained in the clip AV stream file. The clip AV stream file may also include at least one bitmap subtitle stream file which is played back in synchronization with the video stream which is also contained in the clip AV stream file. The clip AV stream file may also include at least one interactive graphics stream file which is played back in synchronization with the video stream which is also contained in the clip AV stream file. The video stream contained in the clip AV stream file and the audio stream,
bitmap subtitle stream, or interactive graphics stream, which is played back in synchronization with the video stream, are multiplexed. In other words, a clip AV stream file referred to by one Playltem includes video stream data and at least 0 audio stream data, at least 0 bitmap subtitle stream data, or at least 0 interactive graphics stream data, which is played back in synchronization with the video stream data, such that they are multiplexed into the clip AV stream file. [0043]
A SubPlayltem refers to, for example, audio stream data or subtitle data contained in a stream different from the clip AV stream file referred to by the Playltem. [0044]
When playing back a PlayList including only a Main Path, the user can select sound and subtitles only from audio streams and sub-picture streams multiplexed into a clip referred to by that Main Path. In contrast, when playing back a PlayList including a Main Path and a Sub Path, the user can refer to audio streams and sub-picture streams multiplexed into a clip referred to by the SubPlayltem in addition to audio streams and sub-picture streams multiplexed into a clip AV stream file referred to by the Main Path. [0045]
As discussed above, a plurality of Sub Paths are included in one PlayList, and each SubPath refers to the corresponding SubPlayltem. Accordingly, AV streams having high extensibility and high flexibility can be provided. That is, SubPlayltems can be added afterwards. [0046]
Fig. 7 illustrates an example of the Main Path and an example of the Sub Path. In Fig. 7, an audio playback path played back simultaneously with (in AV-synchronization with) the Main Path is indicated by using the Sub Path. [0047]
The PlayList shown in Fig. 7 includes one Playltem, i.e., Playltem_id=0, as the Main Path, and one SubPlayltem as the Sub Path. The SubPlayltem() includes .the following data. SubPlayltem() includes Clip_Information_file_name for specifying the clip referred to by the Sub Path in the PlayList. In the example in Fig. 7, the SubPlayltem refers to an Auxiliary audio stream of SubClip_entry_id=0. SubPlayltem() also includes SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback zone of the Sub Path contained in the clip (in this case, the Auxiliary audio stream). The SubPlayltem() also includes sync_Play!tem_id and sync_start_PTS_of_Play!tem for specifying the time at which the playback operation of the Sub Path is started on the time axis of the Main Path. In
Fig. 7, sync_Play!tem_id=0 and sync_start_PTS_of_Play!tem=tl With this information, the time tl at which the playback operation of the Sub Path is started on the time axis of Playltem_id=0 of the Main Path can be specified. That is, in the example in Fig. 7, the playback start time of the Main Path and the playback start time of the Sub Path is the same, i.e., tl. [0048]
The clip AV audio stream referred to by the Sub Path should not include STC non-sequential points (system time base non-sequential points). The clip audio sample clock used for the Sub Path is locked to the audio sample clock used for the Main Path. [0049]
In other words, SubPlayltemO includes information for specifying the clip referred to by the Sub Path, information for specifying the playback zone of the Sub Path, and information for specifying the time at which the playback operation of the Sub Path is started on the time axis of the Main Path. Since the clip AV stream used for the Sub Path does not include STC, the user can refer to a clip AV audio stream different from the clip AV stream (main AV stream) referred to by the Main Path on the basis of the information included in SubPlayltemO (information for specifying the clip referred to by the Sub Path, information for specifying
the playback zone of the Sub Path, and information for specifying the time at which the playback operation of the Sub Path is started on the time axis of the Main Path), and plays back the clip AV audio stream. [0050]
As stated above, the Playltem and the SubPlayltem individually manage clip AV stream files. The clip AV stream file managed by the Playltem is different from the clip AV stream file managed by the SubPlayltem. [0051]
In a manner similar to the example shown in Fig. 7, a subtitle stream playback path played back simultaneously with the Main Path may be indicated by using a Sub Path. [0052]
Fig. 8 illustrates another example of Main Path and another example of Sub Path. In Fig. 8, an audio playback path played back asynchronously with the Main Path is indicated by using a Sub Path. The main clip AV stream file referred to by the Playltem of the Main Path is similar to that in Fig. 7, and an explanation thereof is thus omitted. [0053]
The configuration shown in Fig. 8 is used when, for example, the Main Path is used as the slideshow of still images and the audio path of the Sub Path is used as the BGM (background music) of the Main Path. That is, the
configuration shown in Fig. 8 is used for allowing BGM to be continuously played back when the user instructs a playback apparatus (player) to update the images of the slideshow. [0054]
In Fig. 8, Playltem_id=0, 1, and 2 are disposed in the Main Path, and one SubPlayltem is disposed in the Sub Path. The Sub Path includes SubPlayItem_IN_time and SubPlayItem_OUT_time for specifying the playback zone of the Sub Path in the clip (Auxiliary audio stream). In the example in Fig. 8, the clip (Auxiliary audio stream) is referred to by the SubPlayltem. By comparing Fig. 8 with Fig. 7, it can be understood that the SubPlayltem in Fig. 8 does not include sync_Play!tem__id and
sync_start_PTS_of_Play!tem. The reason for this is that, since the playback timing of the AV stream (video data) referred to by the Main Path is not related to the audio playback timing, it is not necessary to specify the time at which the Sub Path starts the playback operation on the time axis of the Main Path. That is, information indicating that the AV stream referred to by the Main Path and the audio stream referred to by the Sub Path are played back together is sufficient. [0055]
It has been described that the playback timing of the video stream data included in the AV stream is different
from that of the audio stream referred to by the Sub Path.
To put it more specifically, this means that the playback
Q