Specification
Description
RECORDING MEDIUM, REPRODUCTION APPARATUS , RECORDING METHOD,
PROGRAM, AND REPRODUCTION METHOD
Technical Field
The present invention relates to a recording medium suchas aBD-ROM, and to a reproduction apparatus . Thepresent invention particularly relates to an interactive control technology for combining, with a motion picture made of a plurality of pictures, an interactive display made of a plurality of buttons, and for realizing reproduction control in accordance with a user operation directed to the buttons.
Background Art
The interactive control described above is an indispensable function for reproduction apparatuses in receiving a user operation on selection of title and chapter to be reproduced, a reply to a quiz question, and the like, and is widely known to be realized on DVD reproduction apparatuses. The interactive control by a DVD reproduction apparatus is an application of OSD (on screen display) technology by which a button being a character string attached to a graphics is displayed on a screen, and the color of a frame of the button is changed according to a user operation. Such color change enables a user to intuitionally understand which button on the interactive display is in a selected state .
A disadvantage of the interactive control on a DVD is that there is hardly any factor for amusement. Specifically,
users hardly find it amusing during operation of the DVD interactive control. In view of such criticism, animation display of buttons is attempted on the interactive display of a BD-ROM (Blu-ray disk prerecorded format) whose standardization is underway. To be more specific, in the interactive display presented by a BD-ROM, each button is presented in a form of original animation, and is overlayed with a scene of a movie, which is the main video image. In addition, the content of this animation display changes according to a user operation . When such an animation display is used to represent a character appearing on a movie work, a user can change the expression and action of the character by his operation directed to rhe interactive display. Such a kind of animation enables creation of interactive display by which even a child is amused.
However, the animation display for buttons incurs a large amount of decoding load, and so there is a problem that a waiting time, until an initial display of the interactive display is realized, becomes long. For example, assume that an interactive display such as shown in FIG.l is overlayed with a scene of a movie work. This interactive display has four buttons, each of which haH three states: normal state; selected state; and active state . Also assume a case in which each state of button is presented in 2-3 seconds of animation display. Even if one page of graphics data is displayed at intervals of 5 frames in an imaae signal, about 3G pages of graphics data are required to rea Lize 2- 3 seconds of animation Not only that, since there are threestates for a button (normal
state, selected state, and active state) , there will be total of 90 pages (3*30) of graphics data. So as to dispose four buttons on an interactive display, it requires decoding of 360 pages (4*90), which is an enormous amount of graphics data. Decoding load for one page of graphics data is light. However, it takes about dozens of seconds so as to decode several hundreds of pages of graphics data . Even for a purpose of making amusing interactive displays, it is too much to keep users waiting dozens of seconds to display each interactive display, which deserves users' criticism.
Disclosure of the invention
The object of the present invention is to provide a recording medium that realizes interactive display with animation, without delay.
So as to achieve the above -described object, the present invention provides a recording medium having: a graphics stream; where: said graphics stream represents an interactive display to be overlayed with a motion picture made of a plurality of pictures, the interactive display including at least one graphical button material; said graphics stream includes a plurality of pieces of graphics data that are grouped under state sets; and the state sets respectively correspond to different states of the at least one button material and are disposed in a sequential order.
Under the above construction, suppose there are 360 pages of graphics data for rendering animation, and there are three states for each bucton na ter iai . Then , the graphics
data will be grouped under three state sets (e.g. 120+120+120
pages). Then each of the state sets is placed so that the
set that appears earlier in the animation will be placed nearer
to the top, and the set that appears later will be placed
farther to the top. Accordingly, in reproduction, the
earlier appearing button-state set will be loaded to the
reproduction apparatus early, postponing loading of later
appearing button-state sets. By doing so, when once
reading/decoding of about 1/3-2/3 of the entire graphics data
are complete, the preparation for an initial display is ready,
even though not all of the .3 60 pages of graphics data have
not been finished being decoded.
This further means that in a case where there is an enormous amount of graphics data to be read/decoded, execution of initial display will not be delayed. According to this, the interactive display with animation can be performed without delay.
Brief Description of the Drawings
FIG. 1 illustrates an interactive display composed of animation.
FIG.2A illustrates an example of use of a recording medium according to the present invention.
FIG.2B illustrates a key disposition on a remote
controller 400, through which, a user operation directed to
the interactive display is received.
FIG.3 is a diagram shewing a structure or a BD-ROM. FIG. 4 is a diagram schematically illustrating a
structure of an AV Clip.
FIG . 5 is a diagram illustrating an internal structure of Clip information.
FIG.6 is a diagram illustrating an internal structure of playlist information (PL information) .
FIG . 7 is a diagram schematically showing an indirect reference by means of PL information.
FIG.8A is a diagram illustrating a structure of the interactive graphics stream.
FIG.8B is a diagram illustrating an internal structure of each of an ICS and an CDS.
FIG. 9 is a diagram illustrating a logical structure constituted by a various kind of functional segments.
FIG.10A is a diagram illustrating a data structure of an ODS, which defines a graphics object.
FIG.10B is a diagram illustrating a data structure of a PDS.
FIG.11 is a diagram illustrating a data structure of an Interactive Composition Segment.
FIG.12 is a diagram illustrating a relation between ODSs in a DSn and an ICS.
FIG.13 is a diagram illustrating a screen composition at a display timing of arbitrary picture data ptl.
FIG.14 is a diagram illustrating a setting example of button information in ICS.
FIG.15 is a diagram illustrating one example of the state transition of buttons A-D.
FIG.16 is a diagram illustrating one example of the
design images of the 0DS11, 21, 31, and 41.
FIG.17 is a diagram illustrating one example of the design images of the 0DS11-19 of the button A.
FIG.18 is a diagram illustrating one example of ICS and ODS included in a DS.
FIG.19 is a diagram illustrating the order of ODSs
belonging to a Display Set. as well as button-state sets.
FIG.20 is a diagram illustrating a state transition
of an interactive display at which the button-state sets of
FIG.19 are disposed.
FIG. 21 is a diagram illustrating the order of ODSs in a Display Set.
FIG . 22 is a diagram illustrating difference in the order of ODSs in a S-ODSs, between a case when default_selected__button_number is "0", and when it is "button B".
FIG.23A and FIG.23B are diagrams for showing the concrete value for ΣSIZE(DSn [ ICS . BUTTON[ i]] ) , in a case when N-ODSs include a plurality of ODSs constituting the buttons A-D, and S-ODSs include a plurality of ODSs constituting the buttons A-D.
FIG.24 is a diagram showing a timing of synchronized display by means of ICS.
FIG.25 is a diagram showing how to set DTS and PTS, in a case when a plurality of OCSs compose an initial display of an interactive display and the defaui_seiected_button_numbet is valid.
FIG.26 is a diagram shoving how to set DTS and PTS,
in a ca.se when a pluralit y of ODSs compose an initial display of an interactive display, and the default selected button is invalid.
FIG.27 is a diagram shcwing the internal structure of a reproduction apparatus according to the present invention. FIG.28 is a diagram showing a storage content of the object buffer 15, in contrast to the graphics plane 8.
FIG.29 is a diagram showing the processing performed by the graphics controller 17 at an initial display.
FIG. 30 is a diagram showing processing of the graphics controller 17 at an update of an interactive display according to IstUserAction(MoveRight).
FIG. 31 is a diagram shoving processing of the graphics controller 17 at an update of an interactive display according to IstUserAction (MOveDown).
FIG. 32 is a diagram showing processing of the graphics controller 17 at an update of an interactive display according to IstUserAction (Activated).
FIG. 33 is a timing chart showing pipelining performed by the reproduction apparatus.
FIG. 34 is a timing chart showing pipelining performed by the reproduction apparatus in a case where the default selected button dynamically cnanges.
FIG.35 is a flowchart showing the execution procedure for LinkPL function, performed by the control unit 20.
FIG. 36 is a flowchart showing the procedure of loading processing for Segment.
FIG. 37 isa diagram shewing ine example of multiplexing .
FIG. 38 is a diagram showing the manner in which DS10 is loaded to the coded data buffer 13 of the reproduction apparatus.
FIG. 39 is a diagram shov.-ing a case in which a normal reproduction is performed.
FIG.4 0 is a diagram showing how the loading of DSl, DS10, and DS20 is performed when a normal reproduction is performed as in FIG.39.
FIG . 41 is a flowchart in which processing corresponding to the main routine in the processing of the graphics controller 17 is described.
FIG . 42 is a flowchart showing the processing procedure for synchronized control performed by means of a time stamp. FIG . 43 is a flowchart showing the procedure for writing processing directed to the graphics plane 8.
FIG. 44 is a flowchart showing a processing procedure for auto-activating a default selected button.
FIG.4 5 is a flowchart showing a processing procedure for an animation display.
FIG.46 is a flowchart showing a processing procedure of a UO processing.
FIG.47 is a flowchart showing a procedure of changing processing for a current button.
FIG.4 8 is a flowchart showing a procedure of numerical inputting processing.
FIG . 4 9 is a diaqram showing a pipeline in a reproduction apparatus, based on DTS, and PCS in PDS.
FIG.50 is a diagram showing a meaning of an END during
pipelining of the reproduction apparatus.
FIG.51 is a flowchart showing a production method of a BD-ROM that relates to the second embodiment.
FIG.52 is a diagram showing an internal structure of PL information relating to the third embodiment.
FIG.53 is a diagram schematically showing synchronization of sub Playltem according to "sync__PlayItem_id,sync_start_PTS_of_PlayItem".
FIG. 54 is a diagram showing file identification byway of "Clip_In forma tion_file_ name" within the playlist information on an HD.
Best Mode for Carrying Out the Invention (First Embodiment)
The following describes an embodiment of a recording medium, relating to the present invention. Firstly, among the embodiments of a recording medium according to the present invention, an example of how the recording medium is used is explained as follows. FIG.2A is a diagram illustrating a use of the recording medium according to the present invention. In FIG.2A, the recording medium of the present invention is a BD-ROM 100 . This BD-ROM 100 is used to supply a movie work to a Home Theatre System structured by a reproduction apparatus 200, a television 300, and a remote controller 400. Among them, the remote controller 4 00 is used to receive an operation to change the state of the interactive display, from a user, and is closely related to the recording medium of the present invention. FIG.2B
illustrates keys of the remote controller 4 00, through which a user operation directed to the interactive display is received. As shown in this drawing, the remote controller 400 is provided with a MoveUp key, a MoveDown key, a MoveRight key, and a MoveLeft key . Here, the buttons on the interactive display have three states: normal state; selected state; and active state. The MoveUp key , the MoveDown key, the MoveRight key, and the MoveLeft key are used to receive a user operation to change the state of the buttons into normal state—selected state—active state. The normal state is a state in which mere display is provided, whereas the selected state is a state in which a focus is given according to a user operation, but confirmation has not been received. The active state is a state in which confirmation is received. The MoveUp key is for, when a button in the interactive display is in a selected state, setting a bu tton above this selected button, to be in a selected state. The MoveDown key is for setting a button below this selected button to be in a selected state. The MoveRight key is for setting a key on the right of the selected key to be in a selected state, and the MoveLeft key is for setting a key on the left of the selected key to be in a selected state.
The Activated key is for setting the selected button to be in an active state (i.e. to activate) . Numerical keys "0"-"9" are for setting a button to which a corresponding number is assigned, to be in a selected state. "+10" key is for receiving an operation t.o add 10 to the already inputted numerical values. It should be noted here that "0" key and
n+10" key are both for receiving input of numerical value of no smaller than 10 digits. Therefore any one of them is enough for the remote controller 4 00.
So far, the description is about howthe recording medium of the present invention is used.
As follows, among the embodiments of the recording medium of the present invention, a production example is explained. The recording medium of the present invention is produced by improving the application layer of a BD-ROM. FIG.3 is a diagram showing a structure of a BD-ROM.
In the drawing, the BD-ROM is shown at a fourth row of the drawing, and a track on the BD-ROM is shown in the third row. The track is actually in a spiral shape on the disk, but shown as a line extending in the lengthwise direction of the drawing. The track includes a lead-in area, a volume area, and a lead-out area. The volume area in this drawing has a physical layer, a file system layer, and an application layer. In the first row of the drawing, an application format of the BD-ROM is illustrated using a directory structure. As illustrated in the drawing, the BD-ROM has a directory BDMV under the root directory, and the BDMV directory contains such files as XXX.M2TS, XXX.CLPI, and YYY.MPLS. By forming the above application format, it is possible to produce the recording medium according to the present invention. In a case in which there is more than one file for each kind, it is preferable to provide three directories named STREAM, CLIPINF, and PLAYLIST under trie BDMV to store the files with the same extension in one directory. Specifically, it is desirable to store the files with the extension M2TS in the
STREAM, the files with the extension CLPI in the CLIPINF. and the files with the extension MPLS in the PLAYLIST.
As follows, each file in the application format is explained. Firstly, an explanation about the AV Clip (XXX.M2TS) is given.
The AV Clip (XXX.M2TS) is a digital stream in MPEG-TS format (TS is Transport Stream) obtained by multiplexing a video stream, at least one audio stream, a presentation graphics stream, and an interactive graphics stream. The video stream represents motion pictures of the movie, the audio stream represents sound of the movie, the presentation graphics stream represents subtitles of the movie, and the interactive graphics stream represents dynamic reproduction control procedure directed to menus. FIG.4 is a diagram schematically illustrating a structure of the AV Clip.
The AV Clip (middle row) is structured in a following
manner. A video stream made of plural vide frames (pictures
pjl, pj2, and pj3) , and an audio stream made of plural audio
frames (top row of the drawing) are respectively converted
into a PES packet sequence (second to top row of the drawing) ,
and then into a TS packet secuence (third to top row of the
drawing). Then a presentation graphics stream and an
interactive graphics stream .;bottom row of the drawing) are
converted into a PES packet sequence (second to bottom row
of the drawing), and then into a TS packet sequence (third
to bottom row of the drawing) . The TS packet sequences are
multiplexed, to obtain the 4-.V Clip.
The AV Clip constituted in the above manner is divided into more than one extent, like ordinary computer files, and stored in areas in the BD-ROX The AV Clio is made of one
or more ACCESS UNITs. Each ACCESS UNIT functions as a unit
of random access, as well as a decoding unit starting with
an Intra (I) picture.
Clip information (XXX.CLFI) is management information
for an AV Clip. FIG . 5 is a diagram illustrating an internal
structure of the Clip information. Since an AV Clip is
obtained by multiplexing a video stream and an audio stream,
and has ACCESS UNITS that each functions as a unit of random
access, specific items managed by the Clip information are:
attributes of each of the video stream and the audio stream;
and at which position in the AV Clip a random access point
exists .
In the drawing, the broken leader lines help clarify
a structure of the Clip information. As the leader line hnl
shows, the Clip information (XXX.CLPI) is comprised of:
"attribute information" about the video stream and the audio
stream; and "EP_map" being a reference table used in making
a random access to the ACCESS UNITs.
As the leader line hn2 shows, the attribute information
("At tribute " ) is comprised of : "Video attribute information"
that is attribute information on a video stream; "Number"
indicating the number of sets of attribute information; and
"Audio attribute information #]-i*m" that each representing
attribute information for a coriesponding one of a plurality
of audio streams to be multiplexed onto the AV Clip. As the
leader line hn3 shows, the Video attribute information
indicates what kind of compression method was employed to
compress the video stream (Coding), and a resolution
(Resolution), an aspect ratio (Aspect), and a frame rate
(Framerate) for each set of picture data constituting the video stream.
As the leader line hn4 shows, audio attribute information #1- #m (attribute information on the audio stream) indicates what kind of compression method was employed to compress the audio stream (Coding), a channel of the audio stream (Ch.), which language the audio stream corresponds to (Lang), and a sampling frequency of the audio stream. EP_map is a reference table used for performing indirect reference to the addresses of a plurality of random access points, using time information . As the leader line hn5 shows', the EP_map is comprised of: a plurality of sets of entry information (ACCESS UNIT#1 entry, ACCESS UNIT#2 entry, ACCESS UNIT#3 entry...), and entry number (Number). As the leader line hn6 shows, each entry indicates a reproduction start time of a corresponding ACCESS UNIT and the address of the ACCESS UNIT, in association (Note that it is possible to enter the size of the first I picture in the ACCESS UNIT (I-size)) . A reproduction start time of an ACCESS UNIT is expressed by a time stamp of picture data positioned at the beginning of the ACCESS UNIT (presentation time stamp). In addition, an address of an ACCESS UNIT is expressed by a serial number of a corresponding TS packet (SPN(Source Packet Number)). In coding, the variable length coding compression method is adopted. Therefore, even if "here are varieties in size and reproduction time between ea L ACCESS UNIT including GOP, it becomes possible to perform random access, from any reproduction time, to the picture data of the ACCESS UNIT
corresponding to a destination reproduction time, by referring to the entry of this ACCESS UNIT.
Note that the file name XXX in XXX.CLPI is identical to the name of an AV Clip whose Clip information matches. For example, the file name of the AV Clip in this drawing is XXX, and so it corresponds to the AV Clip (XXX.M2TS) . Next, playlist information (PL information) is explained.
YYY.MPLS (playlist information) is a table constituting a playlist being reproduction path information, and is comprised of a plurality of pieces of Playltem information (PlayIteminformation#l, #2, E3...#n), and Playltem information number (Number). FIG.6 is a diagram illustrating an internal structure of playlist information. One piece of Playltem inf ormation defines one or more logical reproduction sections that constitute a playlist. The leader line hsl in the drawing helps clarify a structure of one piece of Playltem information . As this leader line shows , the piece of Playltem information is comprised of: "Clip__Information_file_name" representing a file name of reproduction section information of an AV Clip to which the In-point and Out-point of the reproduction section belongs; "Clip__codee_identifier" representing a coding method used to encode the AV Clip; "IN_time" being time information indicating a start point of the reproduction section; and "OUT_point" being time information indicating an ending point of the reproduction section.
The Playltem information is characterized by its notation method . Specifically , in the PlayItem information .
reproduction sections are defined in an indirect way by means of time, using the EP_map as a reference table. FIG.7 is a diagram schematically illustrating this indirect reference In this drawing, an AV Clip is comprised of a plurality of ACCESS UNITs. The EP_map of the Clip information specifies the addresses of the ACCESS UNITs as shown by the arrows ayl, 2, 3, and 4. The arrows jyl, 2, 3, and 4 schematically show-how the pieces of Playltem information are used to make a reference to the ACCESS UNITs. In other words, reference made by means of the pieces of Playltem information (i.e. the arrows jy 1, 2, 3, and 4) is able to specify the addresses of ACCESS UNITS within the AV Clip, via the EP_map, and so can be regarded as an indirect reference using time concept by means of EP_map.
A reproduction section on a BD-ROM, comprised of a combination of Playltem information, Clip information, and AV Clip is called "Playltem". A logical reproduction unit on a BD-ROM, comprised of a combination of PL information, Clip information, and AV Clip is called "playlist"(abbreviated as PL) A movie work recorded in a BD-ROM is structured by such logical reproduction units (i.e. PL) . Therefore, it is easy to produce another movie work made of all the scenes with a character, by defining pLs specifying scenes with the cnaracter.
Because movie works recorded in BD-ROMs have the aforementioned logical structure, quoting of an AV Clip constituting a scene of a movie work in another movie work can be efficiently performed.
Next, the interactive graphics stream is described. FIG.8A is a diagram illustrating a structure of the interactive graphics stream. The first row represents a TS packet sequence constituting the AV Clip. The second row represents a PES packet sequence constituting the graphics stream. The PES packet sequence in the second row is structured by extracting payloads from TS packets of the first row that include a predetermined PID, and then linking the extracted payloads together. Note that the presentation graphics stream is not a main focus of the present invention, and so is not explained here.
The third row represents the structure of the graphics
stream. The graphics stream is made of functional segments
of: ICS(Interactive Composition Segment); PDS (Palette
Definition Segment); ODS (Object_Definition_Segment), and
END (END of Display Set Segment). Among these functional
segments, ICS is called a display composition segment, and
PDS, ODS, and END are respectively called a definition segment.
The PES packet and each of the functional segments correspond
one to one, or one to plurality. in other words, one
functional segment is either recorded in the BD-ROM after
converted into one PES packet, or after divided into fragments
and converted into more than one PES packet.
FIG.8B is a diagram illustrating PES packets obtained by converting the functional segments. As FIG.8B shows, a PES packet is made of a packet header and a payload, where the payload is the substance of a functional segment. In addition, the packet header includes a DTS and a PTS
corresponding to this functional segment. Hereinafter, DTS and PTS included in a packet header are referred to as DTS and PTS of a functional segment.
These various kinds of functional segments constitute
a logical structure as shown in FIG.9. In this drawing, the
third row represents functional segments, the second row
represents Display Sets , and the first row represents Epochs .
The second row is a collection of Display Sets
(abbreviated as "DS" ) , each of which corresponds to graphics
that make up one screen of graphics. The broken line in the
drawing indicates to which DS the functional segments of the
third row belong. As can be understand from the above, a
series of functional segments of "ICS-PDS-ODS-END"
constitutes one DS . A reproduction apparatus can constitute
one screen of graphics by reading a series of functional
segments constituting one DS, from the BD-ROM.
Each Epoch in the first row represents a period having memory management continuity on a reproduction time axis for an AV Clip, and also corresponds to a data set assigned to this period. Here the assumed memory is a graphics plane that is for storing one screen of graphics, or an object buffer for storing graphics data in decompressed state. By stating a graphics plane or an object, buffer has memory management continuity, it is meant that a flash does not occur in the graphics plane or in the object buffer in a period represented by the Epoch, and deletion/re-drawing for graphics is performed only within a predete rmined rectangular area within the graphics plane (Here, the flash means clearing all the
contents stored in the plane- or in the buffer). The size (length/width) and position of this rectangular area are fixed throughout an Epoch. As lcng as deletion/re-drawing of graphics is performed within this fixed area of the graphics plane, seamless reproduction is guaranteed. That is, an Epoch can also be considered as one unit on a reproduction time axis, at which seamless reproduction is guaranteed. If a user wants to change the area, he has to define a change time on the reproduction time axis, and create a new Epoch that corresponds to the time after this change time. In this case, seamless reproduction will not be guaranteed between these two Epochs.
Note that what is meant by "seamless reproduction" is that deletion/re-drawing of graphics will complete within a predetermined video frame number. For an interactive graphics stream, this video frame number is 4 to 5 frames. This number of video frames is determined by a ratio of a fixed area to an entire graphics plane, and a transfer rate between the object buffer and the graphics plane.
In the drawing, the broken lines hkl, 2 represent to which Epoch the functional segments in the third row belong. As can be understood from this drawing, a series of DS (namely, Epoch Start, Acquisition Point, and Normal Case) constitutes an Epoch of the first row. "Epoch Start", " AcquisitionPoint" , and "Normal Case" are respectively a type of DS . Note that the order of "Acquisition Point" and "Normal Case'' is merely one example, and can be reversed.
"Epoch Start" is a DS tnat produces a display effect
of "new display", and indicates a start of a new Epoch. Therefore an Epoch Start has to include all the functional segments required to compose a next screen, and is disposed at a position in an AV Clip, which is destined for a random access target (e.g. chapter of a movie work).
"Acquisition Point" is a Display Set that produces a display effect of "display refresh", and relates to a preceding "Epoch Start". The Acquisition Point is classified into two types: Duplicate and Inherit. A Duplicate is a Display Set totally identical to a preceding Epoch Start, whereas an Inherit is a Display Set inheriting the functional segments from a preceding Epoch Start but has different button commands from the preceding Epoch Start. Although not representing a start time of an Epoch, an Acquisition-Point DS includes all the functional segments required to compose a next screen. Therefore if a random access is performed to an Acquisition-Point DS, graphics display is guaranteed . In other words , an Acquisition-Point DS enables a screen composition during an Epoch in progress. An Acquisition-Point Display Set is incorporated into a position that is a random access target. Such a position can be specified by time search. Time search is an operation in which input of time (minute and second) is received from a user, and random access is made to a reproduction time corresponding to the inputted time. Such time input is performed in a rough unit such as 10 minutes or 10 seconds, and so time search can specify reproduction points in 10 minutes interval or 10 seconds interval. By providing
Acquisition Points at positions that can be specified by the time search, graphics stream reproduction will be facilitated at the time search.
"Normal Case" is a DS that produces a display effect of "display update", and only includes a difference with the prior display composition. For example, suppose that a button for a DSv has a same image design as a preceding DSu, but is under different state control from the DSu. In this case, the DSv is arranged either to only include ICS, or to only include ICS and PDS, and set the DSv as a Normal-Case DS . By doing so, it is not necessary to provide overlapping ODSs, and so helps reduce the occupied areas in a BD-ROM. Note that a Normal-Case DS cannot compose a screen by itself, because it represents only the difference.
An interactive display defined by these DS is created by disposing GUI parts on a screen. By stating that a DS has interactivity, it means that the state of the GUI parts is changeable in accordance with a user operation. In the present embodiment, such GUI parts, which are a user operation target, are called "button" . The states of a button include "normal state", "selected state", and "active state", each of which is composed by a plurality of graphics in decompressed state. Each decompressed graphics representing a state of a button is called "graphics object". The reason why one state of each button is represented by a plurality of decompressed graphics is for the purpose of an imation display . Next, the Definition Segments (ODS, PDS) are explained . "Object_Definitiot_Segment" is information defining
a graphics object. The foliov.-ing explains this graphics object. An AV Clip, recorded in a BD-ROM, features high-quality images that can be compared to those of a high-definition television. Accordingly, the resolution of the graphics object is high, which is 1920*1080 pixels, so as to produce a high definition . As for the color of a pixel, an index value for one pixel is set as 8 bit length. Here, the index value includes red color-difference component (Cr__value), blue color-difference component (Cb_value), brightness component (Y_vaiue), and transparency level (T_value). By this setting, arbitrary 256 colors can be selected from 16,777,216 colors (full color), thereby enabling setting of the 256 colors as colors for the pixel. An ODS has a data structure as shown in FIG.10A, for defining a graphics object. An ODS is comprised of: "segment_type" that indicates that it is an ODS; "segment_length" that indicates a data length of the ODS; "object_id" that uniquely identifies the graphics object that corresponds to this ODS in the Epoch; "object_version_number" that indicates a version of the ODS in the Epoch; "last in sequence flag"; and a continuous byte-length data "object_data_fragment" that is either a part or all of the graphics object.
"object_id" uniquely identifies the graphics object that corresponds to this ODS in the Epoch. However, when animation is constituted by a plurality of graphics objects defined by a plurality of ODSs, "objeot_id" assigned to each of the plurality of ODSs will be serial numbers.
Next, "last in sequence flag'' and "object__data_fragment'' are explained. There is a case where decompressed graphics constituting a button cannot be stored in one ODS, because of a constraint on the pay load of a PES packet. In such a case, a division (fragment) of a button command is set as "object_data_fragment" . When storing one graphics object in a plurality of ODSs, all the fragments have to have the same size, except the last fragment. That is, the last fragment will have smaller size than the other fragments. The plurality of ODSs storing these fragments will appear in the identical order in a DS. The last part of the graphics object is specified by an ODS having "last in sequence flag" . The above-described data structure of an ODS assumes adoption of a storing method by which storing for a new PES packet does not start until a current PES packet becomes full. However, it is also possible to adopt a storing method by which there will be empty space in each PES packet. "palette_definition__segment (PDS)" is information that defines a palette for color conversion . A data structure of a PDS is shown in FIG.1OB. As shown by FIG.1OB, a PDS is comprised of: "segment_type" that indicates that it is a PDS; "segment_length" that indicates a data length of the PDS; "palette_id" that uniquely identifies the palette included in thePDS; vpalette_version_number" that indicates a version of the PDS in the Epoch; and "palette_entry" that i s information about each entry . "palette_en try " represents a red color-difference component (Cr_vaiue), blue color-difference component (Cb..value), brightness component
(Rvalue) , and transparency level (T_value), for each entry. Next, END of Display Set Segment is explained. END of Display Set Segment is an index indicating an end of a Display Set transfer, and is disposed immediately after the last ODS . The internal structure of this "END of Display Set Segment" is comprised of: segment_type that indicates that this functional segment is an END of Display Set Segment; and segment_length that indicates a data length of the functional segment, which do not require particular explanation. Therefore the drawing does not show the internal structure.
Next, an Interactive Composition Segment (ICS) is
explained. An ICS is a functional segment constituting an
interactive display. An ICS has a data structure shown in
FIG. 11. As shown in this drawing, an ICS is comprised of:
segment_type; segment_len?th; composition_number;
composition_state; command_update_flag;
composition_timeout_PTS; selection_timeout_PTS;UO_mask_ta
ble; animation_frame_rate_code;
default_selected_button_number;
default_activated_button_number; and ''button information set (button info (1)(2)(3)).
"composition_number" indicates whether update is performed in the DS to which che ICS belongs, and takes a number from 0 to 15.
"composition_state' indicates whether the DS that starts with the particular ICS is a normal case, acquisition case, or epoch start.
"command_update_flag" indicates whether the button commands in the ICS have any change from the counterparts of the preceding ICS. For example, when a DS that a certain ICS belongs to is an Acquisition Point, the ICS will have the same content as the immediately preceding ICS, in principle. However, when this command_update_flag is set to be ON, it becomes possible to set, in the ICS, a button command different from the preceding DS. By this flag, a preceding graphics object can be appropriated, however the command gets valid when it is instructed to be changed.
xxcomposition__timeout_PTS" describes an ending time of an interactive display by means of buttons. At the ending time, display of the interactive display is no more valid, and therefore is not presented, compos it ion_timeout_PTS is preferably described in a time accuracy of the reproduction time axis for the motion picture data.
"selection_timeout_PTS" describes an ending time of
a valid button selection period. At the time shown by the
selection_timeout_PTS, a button specified by
default_activated_button_number gets activated.
selection_timeout_PTS is equal to the time specified by
composition_timeout_PTS or shorter.
selection_timeout_PTS is described in a time accuracy of a video frame.
"UO__mask_table" indicates permission/non-permission of a user operation in a Display Set corresponding to an ICS. When "non-permission" is set in this mask fieid. any user operation directed to a reproduction apparatus becomes
invalid.
"animation_frame_rate_code" describes a frame rate to be applied to a button displayed in animation . The animation frame rate is given by dividing the video frame rate by a value of this field. If this field indicates a value of 00, only ODS identified by start_object_id_xxx is displayed and in non-animation, among the ODS defining graphics objects for all the buttons.
"default_selected_button__number" indicates a button number whose default is to be set in a selected state at the start of an interactive display. When this field indicates 0, the button assigned a button number stored in the register of a reproduction apparatus is set to be an active state automatically. If this field indicates other than 0, the field is interpreted to identify a corresponding button. "default_activated_button_number" indicates a button to be set in active state automatically, in a case when a user has not set any button in active state within the time defined by the selection_timeout_PTS. When the default_activated_button_number indicates *FF", the button currently in selected state is automatically selected at the time defined by the selection_timeout_PTS. When the default_activated_button_number indicates "00", the automatic selection is not performed. When the defaut_activated__button__number indicates other than "00" and "FF", then this field is interpreted to identify a corresponding button.
"button information(button_info)" is information
defining each button to be overlived in an interactive display In the drav/ing, the leader i ine hpl helps clarify the internal structure of the button information i, which is information about the ith button to be controlled by the ICS (button i) . The following is an explanation on information elements constituting this button information i.
wbutton_number" is a numerical value uniquely identifying a button i in an ICS.
"numerically_selectable_flag" is a flag indicating whether a numerical-value selection is permitted for the button i.
"auto_action_f lag" indicates whether to automatically set the button i to be active state. When auto_action_flag is set to be ON (i.e. bit value of 1), the button i is set to be in active state, instead of selected state. When the auto_action_flag is set to be OFF (i.e. bit value of 0) , the button i will be in mere selected state even when the button i is actually selected.
"object__horizontal_position",vobject_vertical_posi tion" respectively indicate a horizontal position and a vertical position of the upper left pixel of the button i in an interactive display.
"upper_button_number" indicates a button number to be in selected state, instead of the button i, when the MOVEUP key is pressed while the button i is in selected state. If the number for the button i i; set in this field, the press of the MOVEUP key is ignorec.
" lo'/;er_button_number" , w ief t_button__number" , and
"right_button_number" respectively indicate the button numbers to be in selected state, instead of the button i, when the MOVEDOWN key, the MOVELEFT key, the MOVERIGHT key are pressed while the button i is in selected state. When the number for the button i is set in this field, the press for these keys is ignored.
"start_object__id_normal" indicates the first number of serial numbers assigned to a plurality of ODSs constituting animation, when the button i in normal state is to be drawn in animation.
wend_object_id_normal" indicates the last number of the object_id being the serial numbers assigned to the plurality of ODSs constituting the animation, when the button i in normal state is to be drawn in animation. If the ID indicated by this end_object_id_normal is identical to the ID indicated by the start_object_id_normal, then the still image corresponding to a graphics object identified by this ID will be the image design of this button i.
"repeated_normal_flag" indicates whether to repeatedly continue the animation display for the button i in normal state.
"start_object_id_seiected" indicates the first number of the serial numbers assigned to the plurality of ODSs constituting animation, when the button i in selected state is drawn in animation. If the ID indicated by end_object_id__selected is identical to the ID indicated by this start_object_id_selected, then the still image corresponding to a graphics object identified by this ID will
be the image design of this button i.
"end_object_id_selected" indicates the last number of the object_id being the serial numbers assigned to the plurality of ODSs constituting the animation.
"repeat_selected_flag" indicates whether to repeatedly continue the animation display for the button i in selected state. When start_object_id_selected has the same value as end_object_id_selected, 00 is set in this field . "start_object_id_activated" indicates the first number of the serial numbers assigned to the plurality of ODSs constituting animation, when the button i in active state is drawn in animation.
"end_object_id_activated" indicates the last number of object_id being the serial numbers assigned to the plurality of ODSs constituting animation, when the button in active state is drawn in animation.
Next, the button commands are explained, "button command (buttor:_command) " indicates a command to be executed when the button i is put into active state. The button command can instruct a reproduction apparatus to per form reproduct ion directed to PL and Playltem. A command that instructs a reproduction apparatus to perform reproduction directed to PL and Playltem is called "LinkPL command" . This command can start reproduction of a playlist specified by a first argument,, from a position specified by a second argument
Format: LinkPL(first argument, second argument)
The first argument specifies, by means of a number assigned to a playlist, a PL to be reproduced. The second argument specifies a reproduction start position, by means of a Playltem included in the PL, or a chapter and a mark that are included in the PL
A LinkPL function for specifying a reproduction start position by means of Playltem is expressed as ttLinkPLatPlayitem()".
A LinkPL function for specifying a reproduction start position by means of Chapter is expressed as "LinJcPLatChapterf) " .
A LinkPL function for specifying a reproduction start position by means of Mark is expressed as "LinkPLatMark() " . A button command is used to give instructions to a reproduction apparatus to perform a state obtaining and a state setting. A state of a reproduction apparatus is indicated by 64 Player Status Registers (whose set value is called PSR), and 4096 General Purpose Registers (whose set value is called GPR) . The button command is classified into the following (i)-(iv), and performs setting of a value to these registers, and obtaining of a value from these registers.
(i) "Get value of Player Status Register" command
Format: Get value of plaver Status Register (argument) This function is used when obtaining a set value of
a Player Status Register specified by an argument.
(ii) ttSet value of Player Status Register" command
Format: Set value of Player Status Register (first
argument, second argument)
The function is used when setting a value specified
by a second argument to a Player Status Register specified
by a first argument.
(iii) wGet value of General Purpose Register" command
Format: Get value of General Purpose Register
(argument)
This function is used when obtaining a set value of
a General Purpose Register specified by an argument.
(iv) "Set value of General Purpose Register" command
Format: Set value of General Purpose Register (first
argument, second argument)
This function is used when setting a value specified
by a second argument to a General Purpose Register specified
by a first argument.
Next, a concrete example of interactive control performed by such an ICS is explained . This concrete example assumes the ODSs and the ICS as shown in FIG. 12. FIG. 12 is a diagram illustrating a relation between ODSs in a DSn, and an ICS. This DSn is supposed to include 0DS11-19, 21-29, 31-39, and 41-49. Among these ODSs , 0DS11- 19 draw each state cf a button A. ODS21-29 draw each state of a button B.
ODS31-39 draw each state of a button C. and ODS41-49 draw each state of a button D (as shown by a sign w}" in the drawing) . In addition, button_info{1) , (2),(3) , ( 4 ) respectively indicate the state control of these buttons A-buttons D (Refer to the arrows bhl,2,3,4 in the drawing).
When the execution timing of the control by means of this ICS coincides with a display timing of any picture data ptl, which is among the pictures of FIG.13, an interactive display tml made of buttons A-D is displayed (gs2) by being overlayed (gsl) with this picture data ptl. An interactive display made of a plurality of buttons is displayed in' accordance with the content of a motion picture. Therefore an ICS can realize realistic effect with use of buttons. FIG . 14 illustrates a description example of an ICS when the state transition of the buttons A-D (shown in FIG.15) is performed.
In FIG.15, arrows hhl and hh2 symbolically represent a state transition by means of neighbor_info() of button info(l). In the neighbor_info() of the button info(l), the lower_button_number is set as "button C" , and so the button C will be in selected state (sjl of FIG. 15) if an UO of MOVEDOWN key press occurs while the button A is in selected state (upl of FIG. 15). In the neighbor_info( ) of the button info (1), the right_button_number is set as "button B'' , and so the button B will be in selected state ( s j 2 of FIG . 15 ) if an UO of MOVERIGHT key press occurs while the button A is in c^Iectsd state (up2 of FIG.15) .
The arrow hh3 in FIG.15 indicates a state transition
control by means of neighbcr_info() of the button info (3) . In the neighbor_info( } of the button info (3), upper__button_number is set as 'button A", and so the button A will return to the selected state if an UO of MOVEUP key press occurs while the button C is in selected state (up3). Next, the image designs of the buttons A- Dare described. Suppose here that 0DS11, 21, 31. and 41 have the image designs shown in FIG . 16, and that the 0DS11-19 assigned to the button A have the image designs shown in FIG.17. Because, in the normal_state_info() in the button_info (1) of the ICS, start__object_id_normal and end__object_id_normal specify 0DS11-13 , and so the normal state of the button A is presented in animation according to the 0DS11-13 . In addition, in the selected_state_info() of the button_info(1), start_object_id_selected and end_object_id_selected specify ODS14-16, and so the selected state of the button A is presented according to the ODS14-16. When a user puts the button A into selected state, the image design of the button A will change from the design according to theODSll-13 ,-to the design according to rhe ODS14-16. Here, in the normal_state_info() and the selected_state_info(), if the repeat_normal_f lag and repeat._select_f lag , are set as 1, the animation according to the GDSll-13 and the animation according to the ODS14-16 v^ill be repeatedly continued respectively, as shown by "-.(Ai", "(A)-", and "-(B)", ,V(B)-" in the drawing.
If a plurality of ODSs presentable in animation are assigned to the buttons A-D, ,-,:\4 that the control for these
CDS is described in the ICS, realistic-looking button state control will be realized (e.g. to change the expression of a character according to each user operation).
Next, application of numerically__selectable_flag is described.
FIG.18 is a diagram illustrating one example of ICS and ODS included in a DS. In this drawing, the ODS31-33 respectively indicate a portrait, a name, and a uniform number of a corresponding one of three baseball players shown in the top row of the drawing. The ICS to which this DS belongs includes three sets of button information, where the button information (1) ' s start_object_idis set to indicate the ODS 31, the button information (2)'s start__object__id is set to indicate the ODS32, and the button information (3)'s start_object__id is set to indicate the ODS33. In addition, the button information (1) 's button number is set as 99, the button information (2)'s button information is set as 42, and the button information (3)'s button information is set as 94. Furthermore, for all the button information (l)-{3), the numerically_selectable_f lag is set as 1. This means that any of the buttons corresponding to the button information (l)-{3) is selectable. Therefore if the numerical value of "99" is inputted by a user through the remote controller 400, the button corresponding to Mr. Beginner's Luck will be put to selected state. The input of the numerical value v99" ii'.ay be received through sequential press of "9" and "9". Alternatively, it may be realized by a combination of a press of v9" and 9 times of press directed to v'-10'' key. When the
numerical value of M2" is inputted, the button for Mr. Careless Mistake will be put to selected state, and when the numerical value of "94" is inputted, the button for Mr. Dead Stock will be put to selected state.
When auto__action_flag for these pieces of button information (l)-(3) are set. as 1, these pieces of button information (l)-(3) will be put to active state, instead of put to selected state, and the button commands included therein, namely (LinkPL(PL#21) , LinkPL(PL#22), LinkPL(PL#23 ) ) will be executed. When the link destinations PL#21, #22, and #23 of the button commands, included in the three pieces of button information, are batting/throwing scenes of respective players, then these batting/throwing scenes will be reproduced in response to an input of a numerical value corresponding to the uniform number of a corresponding player . Since it is possible to select buttons directly using a uniform number that is well-known, the user operability will remarkably improve.
Next, the order of ODSs in a Display Set is described. As described above, each ODS belonging to a Display Set is specified by an ICS to display one state of a button. The order of ODSs in a Display Set is decided according to such a specification, i.e. which state of a button an ODS represents.
To be more specific, 0E:3sina Display Set are class if led into sets of (1) those representing normal state, (2) those representing selected sta:e, and (3) those representing active state. A set of OD;fs that represents one state of
a particular button is called "button-state set". These button-state sets are disposed in the order of "normal state-selected state-active state". As shown above, the order of each ODS in the Display Set is decided according to which state of a button the ODS represents.
FIG.19 is a diagram illustrating the order of ODSs
belonging to a Display Set. In the second row of this drawing,
three button-state sets in the Display Set are illustrated.
This drawing depicts : a set of ODSs for rendering normal state
of the buttons (ODSs for Normal state); a set of ODSs for
rendering selected state of the buttons (ODSs for Selected
state); and a set of ODSs for rendering active state of the
buttons (ODSs for Active state). The order of these
button-state sets is as follows: normal state—selected
state—active state. The purpose of this order is for
facilitating reading of the ODSs constituting an initial
display of an interactive display early, putting off the
reading of the ODSs constituting a screen display after
update.
The first row of FIG.19 represents graphics objects An, Bn, Cn, Dn, As, Bs, Cs, Ds, Aa, Ba, Ca, Da, which will be rendered by these button-state sets. The subscript "n" assigned to An,Bn,Cn,Dn represents normal state of a corresponding button . In the same manner, the subscript *s" of As,Bs,Cs,Ds represents selected state of a corresponding button, and the subscript "a" represents active state of a corresponding button. The second row of FIG.19 shows button-state sets to which the Graphics Objects of the first
row belong. It should be no;ed that, in this drawing, the sets of ODSl-ODSn respectively belonging to N-ODSs, S-ODSs, and A-ODSs are assigned the same number to each other, (i.e. there is 0DS1 belonging to N-ODSs , and at the same time, there is 0DS1 belonging to S-ODSs . ) . However, each set is different from one another . This also applies to every similar drawing hereinafter.
FIG. 20 is a drawing illustrating a state transition of an interactive display at which the button-state sets of FIG.19 are disposed.
The interactive display in this drawing has a plurality of states, namely, "initial display", "update display by 1st user action", and "update display by 2nd update display". The arrows in the drawing represent user actions that trigger the state transition. By this drawing, the four buttons A, B, C, and D respectively have three states of "normal state", "selected state", and "active state". Among them, what is necessary for an initial display is three graphics objects for rendering a normal state and one graphics object for rendering a selected state.
Even when the default selected button is not decided yet, and it is uncertain which of the buttons A-D will be put to selected state, an initial display can be realized when decodingof graphics object representing the normal state and the selected state of each button has been complete. In view of this, in the present. r'-nbodiruent, the button-state sets are disposed in the ord-e: of ''normal state-selected state-active state" as the .--^cond row of FIG. 19 shows.
According to this disposition of button-state sets, initial display can be realized even v.hen the reading and decoding ■of ODSs constituting active state has not yet been complete, and it becomes also possible to shorten a time period between the reading start of a Display Set and completion of an initial display.
Next, the order in which the ODSs shown in FlGs.16 and 17 should be disposed is explained. FIG.21 is a diagram illustrating the order of ODSs in a Display Set. In this drawing, ODSs for Normal state is composed of 0DS11-13, ODS21-23, ODS31-33, ODS41-43. ODSs for Selected state is composed of ODS14-16, ODS24-26, ODS34-36, and ODS44-46. Finally, ODSs for Active state is composed of ODS17-19, ODS27-29, ODS37-39, and ODS47-49. ODS11-13 are for rendering expression change of a character as shown in FIG . 17 , and the same thing can be said to the ODS21-23, the ODS31-33 , and ODS41-43 . Therefore, by disposing these ODSs in the first button-state set, preparation for an initial display can be made even during reading of the Display Set. By the above arrangement, it becomes possible to perform an interactive display to which animation is incorporated, without delay. Next, the order of ODSs to be referenced by a plurality of button states (multi-reference) is explained. Multi-reference means that ob;.ect_id of one ODS is specified oy two or more sets of normal_state_inf o, selected_state_inf o, and act L;"a ted_state_inf o . By such mult iple-reference , it .oecomes possible to use a graphics object for rendering the normal state of a button, for renderinc another button's
selected state. This means that it is possible to share the design image of one graphics object. Such sharing helps reduce the number of ODSs. Here, it is problematic which button-state set an ODS to be multiple-referenced should be made to belong.
For example, suppose when a normal state of a button and a selected state of another button are drawn by one ODS . Then it is problematic whether this ODS belongs to a button-state set corresponding to Normal state, or a button-state set corresponding to Selected state.
In principle, this ODS will be disposed only in a button-state set that corresponds to an earliest-appearing state among the plurality of states.
For instance, when an ODS is multiple-referenced by a normal state and a selected state, the ODS will be disposed in a button-state set for Normal state (N-ODSs), not in a button-state set for Selected state (S-ODSs ) . In a different example, if an ODS is multipe-referenced by a selected state and an active state, then the ODS will be disposed in a button-state set for Selected state (S-ODSs), and not in a button-state set for Active state(A-ODSs). In this way, a multiple-referenced ODS is disposed only in a button-state set corresponding to the earliest appearing state.
Next, the order of ODSs in S-ODSs is explained. In S-ODSs, which ODS comes first depends on whether the default selected button is statically determined or dynamically determined. A statically-determined default selected button is a button specified by a valid value (other than
00) set in default_selected__button_number in the ICS. When a valid value other than 00 is set as the default_selected_button_number, and that no ODS representing a default selected button does not exist in the N-ODSs, the ODS representing the default selected button is disposed at the beginning of the S-ODSs.
When the default_selected_button_number indicates a value of 00, then the button to be set in selected state, as a default, will dynamically change depending on the state at a reproduction apparatus side.
The case where the default_selected_button_number is set to indicate the value 00 includes a case when the AV Clip onto which the Display Set has been- multiplexed exists at a joint point of a plurality of reproduction paths. For example, suppose a case when the preceding reproduction paths are respectively first, second, and third chapters, and the Display set at the joint point is for displaying buttons corresponding to the first, second, and third chapters . In such a case, it is not appropriate to predetermine the button to be in selected state as a default, by the default_selected_button_number.
In such a case, it is ideal to change the button to be in selected state according to which one of the preceding plurality of reproduction paths has been passed, until reaching this Display Set ;e.g. the second-chapter button when reaching from Lne first chapter , the third-chapter button when reaching from the secono. chapter, and the fourth-chapter-button when reaching from tr-e third button) . In a case where
the button to be in selected .state changes depending on the
preceding reproduction path, the
default_selected_button_numter will be set to indicate"invalid", i.e. the zero value is set thereto . Since the button to be in selected state will change; an arrangement of disposing a certain ODS at the beginning of the button-state set is not performed.
FIG. 22 is a diagram illustrating difference in the order of ODSs in a S-ODSs, between a case when default_selected_button_number indicates w0", and when it indicates "button B" . In this drawing, the broken line ssl indicates the order of ODSs in the S-ODSs when default_selected_button__number indicates the button B, and the broken line ss2 indicates the order of ODSs in the S-ODSs when default_selected_button_.number indicates 0 . As shown in the representation of this drawing, when default_selected_button_number indicates the button B, the ODSBs indicating the selected state of the button B is placed at the beginning of S-ODSs, with the other buttons ' ODSs being placed thereafter. On the other hand, when default_selected_button_numbsr indicates 0, the ODSAs indicating the selected state of the button A is placed at the beginning of the S-ODSs. In this way, whether the default_selected__button__number is valid or not affects the order of ODSs within the s-ODSs.
Next, how a Display Sev having such 1C3 and ODS is assigned to a reproduction time axis of an AVCI ip is described . An Epoch is a period during whicn memory management continues
on the reproduction time axis. Since an Epoch is composed of one or more Display Sets, it is an issue how to assign Display Sets to the reproduction time axis of the AV Clip. Here, the reproduction time axis of an AV Clip is used to define decode/reproduction timing of each piece of picture data constituting a video stream multiplexed into an AV Clip . In this reproduction time axis, a decode timing and a reproduction timing are expressed in a time accuracy of 90KHz . DTS and PTS assigned to these ICS and ODS in a Display Set, indicate a timing to realize synchronized control on this reproduction time axis. Synchronized control by means of DTS and PTS assigned to ICS and ODS is assigning of Display-Set to the reproduction time axis.
First, what kind of synchronized control is performed
by means of DTS and PTS assigned to ODS is described.
A DTS indicates a decode start time for an ODS, in a
time accuracy of 90KHz. A PTS indicates a decode end time
for an ODS.
Decoding of an ODS does not end in an instant, and takes
a certain time to complete.
So as to clarify the start point/end point, a DTS and
a PTS respectively indicate a decode start time and a decode
end time, for an ODS.
Since the value of a PTS indicates an ending time, it
is necessary that the decoding of ODSj has been complete before
the time shown hy the PTS. and that the decompressed graphics
object should be obtained in the object buffer of a
reproduction apparatus by "hen.
Since the decode stcrt time of an arbitrary ODSj belonging to a Display Set v;/' is expressed as DTS (DSn [ODSJ ) in a time accuracy of 90KHz, the time obtained by adding a maximum time required for the decoding to this decode start time will be the guaranteed decode end time of the ODSj of the Display Set.
Assume that the size of ODSj after decompression is SIZE(DSn[ODSj]) , and the decode rate of the ODS is Rd. Then the maximum time required for decoding will be SIZE(DSn[ODSj J)//Rd.
Note that the operator "//" used here is a division in which decimals are raised to the next whole number.
- This maximum time is converted into a time accuracy of 90KHz, and added to the DTS of ODSj. In this way, the decode end time ( 90KHz ) to be indicated by the PTS is obtained. The PTS of ODSj belonging to the DSn is expressed as follows.
PTS(DS[ODSj]=DTS(DSn[ODSjf)+90,000*(SIZE(DSn[ODSjj))// RD)
Then between the two ODS(ODSj, ODSj+1), the following relation should be satisfied.
PTS(DSn[ODSj j )PTS(DSn[S-ODSsfirst]) ^9 0,000*(8*video_width*video_height//128,000,000) +9 0,000*(£SIZE(DSn[ICS.BUTTON[i]] )12 8,000,000 )
On the other hand, when the def au.1 t_selected_button_number is invalid, it is only necessary to replace PTS(DSn[S-ODSsfirst]) with
PTS(DSn[S-ODSslastj ) . The concrete expression will be as follows:
PTS(DSn[ICS])>PTS(DSn[S-ODSslast]) +90,000*(8*video_width*video_height//128,000, 000) +90,000*(£SIZE(DSn[ICS.BUTTONli]])128,000,000)
FIG.24 shows one example of realizing synchronized display by setting PTS and DTS in the above way. In this diagram, assumption is made such that a button is displayed at a display timing of arbitrary picture data "pyl" which is in the motion picture in this drawing. In this case, the PTS value of the ICS should be set so as to coincide with the display timing of this picture data.
Decoding of the ODS whose decode time finishes the last, among the ODSs constituting the initial display of the DSn, should be complete by the time obtained by subtracting, from the PTS of the ICS, the screen-clearing period cdl and the graphics-object transfer period tdl. Therefore the PTS value of the ODS should be set at the timing (itl) shown in this drawing. Furthermore, the decoding of the ODS takes the period ddl, and so the DTS value of this ODS should be set at the ddl period earlier than this PTS.
In FIG. 24, there is only one ODS to be overlayed with the motion picture , which is a simplified case. If the initial display of the interactive display to be overlayed with the motion picture is realized by a plurality of ODSs. PTS and DTS of the ICS, and PTS and DTS of ODS should be set as in
FIG.25.
FIG. 25 is a diagram shov.-ing how to set DTS and PTS, in a case when a plurality of ODSs compose the initial display of the interactive display, and the default selected button is statically decided. If the decoding of the ODS whose decoding takes the longest in S-ODSsfirst (the ODSs for realizing the initial display) finishes when the time ddl has passed in the drawing, then the PTS of this S-ODSsfirst (DSn[S-ODSsfirst] ) is set to indicate the time at which the period ddl has passed.
Furthermore, it is necessary to perform screen clearing
and transferring of decoded graphics objects, for realizing
initial display. Therefore, the PTS of the ICS (DSn[ICS])
should be set after the time obtained by adding, to the value
of this PTS(DSn[S-ODSsfirst]}, a period required to perform
screen clearing
(90,000*(8*video_width*videe_height//128,000.000)), and transferring time of decoded graphics object (90,000*(£SIZE(DSn[ICS.BUTTON[i] ] )12 8,00 0,000)) .
FIG. 26 is a diagram showing how to set DTS and PTS, in a case when a plurality of ODSs compose the initial display of the interactive display, and the default selected button is not decided. If the decoding of the S-ODSslast, whose decoding takes the longest in S-ODSs for realizing the initial display, finishes when the time dd2 has passed in the drawing, then the PTS of this S-ODSslasr, (DSn [S-ODSslast ] ) is set to indicate the time at which t:e period dd2 has passed.
Furthermore, so as to realize an initial display, ic
is necessary to perform screen clearing, and transferring
of decoded graphics object. Therefore the PTS(DSn[ICS] ) of
the ICS should be set after trie time obtained by adding, to
the value of this PTS(DSn[S-ODSslast]), a period required
to perform the screen
clearing(90,000*(8*video_width*video_height//128,000 , 000 ) and transferring time of decoded graphics object (90,000*(£SIZE(DSn[ICS.BUTTON[i]])12 8,000,000)).
In a DVD, a period during which an interactive control is valid corresponds to a period of a VOBU, which functions as a GOP of the video stream. However, in a BD-ROM, such a valid period can be set arbitrarily by means of PTS and DTS in ICS included in an Epoch. Therefore the interactive control by means of a BD-ROM is not in dependence relation with a GOP.
Note that a synchronized control by means of a PTS of an ICS is not limited to perform control so as to display a button at a certain timing on a reproduction time axis, and also includes performing of control so as to enable display of a Popup menu during a certain period on a reproduction time axis. A Popup menu is a menu that will be popped up on a screen, by a press of a menu key provided for the remote controller 400, and the synchronized control by means of PTS of ICS also include enabLing of display of such a Popup menu at a display timing of certain picture data in an AV Clip. The ODSs constituting a Popup x-enu are decoded a;;d a graphics object obtained by the decoding are written to a graphics plane, just as the ODSs constituting a button. Unless the
writing to a graphics plane is not complete, it is not possible to respond to a menu call by a user. In view of this, a time at which the Popup display becomes possible is indicated by the PTS of ICS, in performing synchronized display of a Popup menu.
The data structure of a Display Set (ICS,PDS,ODS), described so far, is an instance in a class structural body written in a programming language. A producer who performs authoring is able to obtain the structures on a BD-ROM, by describing this class structural body.
Next, an embodiment of a reproduction apparatus according to the present invention is explained. FIG . 27 is a diagram showing the internal structure of a reproduction apparatus according to the present invention. The reproduction apparatus according to the present invention is produced industrially based on the interior shown in this drawing. The reproduction apparatus of the present invention is mainly composed of three parts: a system LSI, a drive apparatus, and a micro system. The industrial production is pursued by implementing these parts to the cabinet and to the substrate of the reproduction apparatus. The system LSI is an integrated circuit onto which varieties of processing units, having functions of the reproduction apparatus, are integrated. The reproduction apparatus produced in such a way is comprised of: a BD drive 1, a track buffer 2, a PID fiiLar 3 s transport buffers 4a, b, and c. a peripheral circuit 4d. a video decoder 5, a video plane 6, an audio decoder 7, a gra:--i.ics plane 8, a GLUT unit 9.
an addition device 10, a graphics decoder 12, a coded data buffer 13, a peripheral circuit 13a, a stream graphics processor 14, an object buffer 15, a composition buffer 16, a graphics controller 17 , a 00 controller 18, a player register group 19, and a control unit 20.
The BD-ROM drive 1 executes access to the BD-ROM by performing loading/reading/ejecting for a BD-ROM.
The track buffer 2 is a FIFO memory, and stores TS packets read from the BD-ROM, on a first-in first-out basis.
The PID filter 3 performs filtering to a plurality of TS packets outputted from the track buffer 2 . The filtering by the PID filter 3 is to write only TS packets having a desired PID, to the transport buffers 4 a , b, andc, and does not require buffering. Therefore the TS packets inputted to the PID filter 3 will be written to the transport buffers 4a, b, and c, without time-delay.
The transport buffers 4a, b, and c are respectively a memory for storing TS packets outputted from the PID filter 3, on a first-in, first-out basis.
The peripheral circuit 4d isa wire logic that performs processing to convert TS packets read from the transport buffer 4a, into functional segments. The functional segments obtained by the conversion are stored in the coded data buffer 13.
The video decoder 5 decodes a plurality of TS packets outputted from the PID filter 3 to obtain decompressed pictures, and write them to the video plane 6.
The video plane 6 is £ plane for motion pictures.
The audio decoder 7 decodes the TS packets outputted form the PID filter 3 to obtain decompressed audio data, and outputs the decompressed audio data.
The graphics plane 8 is a memory having an area corresponding to one screen, and is able to store decompressed graphics for one screen.
The CLUT un it 9 converts index colors of the decompressed graphics stored in the graphics plane 8, based on the Y, Cr, Cb values shown by the PDS.
The addition device 10 multiplies the decompressed graphics resulted from color conversion at the CLUT unit 9, with a T value (transmission rate) shown by the PDS, and adds the result and the decompressed picture data stored in the video plane 6, pixel by pixel, to obtain an overlayed image, and outputs the overlayed image.
The graphics decoder 12 decodes a graphics stream, to obtain decompressed graphics, and writes this decompressed graphics to the graphics plane 8, as a graphics object. By decoding of the graphics stiream, subtitles and menus will appear on a screen. This graphics decoder 12 is comprised of a coded data buffer 13, a peripheral circuit 13a, a stream graphics processor 14, an object buffer 15, a composition buffer 16, and a graphics controller 17.
The coded data buffer 13 is a buffer in which the functional segments are stored together with DTS and PTS. Such functional segments let-ult from removing a TS packet header, and a PES packet header from each TS packet in a transport stream stored in the transport buffer 4a, and then
disposing the payloads sequenLially. Among the removed TS packet header and the PES packet header, PTS/DTS are stored in association with a corresponding PES packet.
The peripheral circuit 13a is a wire logic that realizes : transferring between the coded data buffer 13 and the stream graphics processor 14; and transferring between the coded data buffer 13 and the composition buffer 16. In such transferring processing, when it reaches a time shown by the DTS of an ODS, the ODS is transferred from the coded data buffer 13 to the stream graphics processor 14 . In addition, when it reaches the time shown by the DTS of an ICS and a PDS, the ICS and the PDS are transferred to the composition buffer 16.
The stream graphics processor 14 decodes an ODS, and writes the decompressed graphics composed of index colors resulting fromdecoding, to the object buff er 15 , as a graphics object. The decoding by the stream graphics processor 14 starts at the time of a DTS associated with the ODS, and ends at the time of a PTS associated with the ODS. The decode rate "Rd" for the aforementioned graphics object corresponds to an output rate of this stream graphics processor 14.
The graphics object obtained from the decoding of the stream graphics processor 14 is stored in the object buffer 15. FIG.28 is a diagram showing a storage content of the object buffer 15, in contrast to the graphics plane 8. This storage content assumes a case /.'here the ODSs in z'r.e example shown by FIGs.16 and 17 are written to the object buffer 15. In the example of FIGs.16 and 1T, four-button animation is
realized by 36 ODSs (ODS11-49). In this example, the ODSs representing all the frames of this animation are stored in the object buffer 15. In contrast, a display position of each of the ODSs stored in the object buffer 15 is defined in the graphics plane 8. This display position is defined by button_horizontal position and button_vertical position of each piece of button information. The animation is realized by transferring the ODSs from the object buffer 15, frame by frame, so as to write the ODSs to the corresponding display positions of the graphics plane 8.
The composition buffer 16 is a memory in which ICS and PDS are disposed.
The graphics controller 17 decodes the ICS disposed in the composition buffer 16, and performs control according to the ICS. The execution timing of this control is based on the value of PTS assigned to the ICS. One of the most important functions of this graphics controller 17 is writing processing at initial display and at update of an interactive display. The following explains writing processing of the graphics controller 17 at initial display and at update, by referring to FIG.29. FIG.29 is a diagram showing the processing performed by the graphics controller 17 at an initial display. As shown in this drawing, the graphics controller 17 performs control to v/rite the ODSs belonging to the S-ODSs of the button ?., to a display position defined by the button_hor zontal_position and the button_vertical_position of the button A's button information, and to write the ODSs belonainq to the N-ODSs
of the buttons B, C, and D, respectively to display positions defined by the button_horizontal_position and the button_vertical_position of the button information of the buttons B, C, and D (The arrows wl, w2 , w3 , and w4 in the drawing symbolically show the above-described writing) . By the writing, the initial display of FIG.20 is realized. Please note here that not all t.he ODSs are not necessary for an initial display of an interactive display. The initial display is realized when the ODSs belonging to the S-ODSs of the default selected button and the ODSs belonging to the N-ODSs of the other buttons have been ready on the object buffer 15. That is, the graphics controller 17 can start writing for an initial display, if decoding has been complete for the ODSs belonging to the S-ODSs of the default selected button, and for the ODSs belonging to the N-ODSs of the other buttons.
FIG. 30 is a diagram showing processing of the graphics
controller 17 at an update of an interactive display according
to lstUserAction(MoveRight) . As shown in this drawing, the
graphics controller 17 performs control to write the ODSs
belonging to the S-ODSs of the button B, to a display position
defined by the button_:iorizontal_position and the
button_vertical_position of the button B's button
information, and to write the ODSs belonging to the N-ODSs
of the button A, to a display position defined by the
buttcn_ricrizontal_posit ion and the
button_vertical_position 3: the button As button
information (The arrows v. 5 , ,:>; , w7, and wS symbolically show
thewriting). By this writing, the state transition as shown in FIG.20 is realized. The buttons C and D stay in normal state, just as in the initial display of the interactive display, but the writing for these buttons to the graphics plane 8 is continued so as to continue the animation.
FIGs . 31 and 32 respectively show the processing of the graphics controller 17 , in each cases where the IstUserAction is MoveDown and Activated . At an interactive-display update, S-ODSs and A-ODSs for the buttons other than the default selected button are involved, and so all the ODSs are required to exist in the object buffer 15.
The UO controller 18 detects a user operation directed to the remote controller and to the front panel of the reproduction apparatus. and outputs information corresponding to the user operation (hereinafter "UOfuser operation)"), to the control unit 20.
The player register group 19 is a register embedded in the control unit 20, and is comprised of 32 player status registers and 32 general purpose registers. The meaning of the set value of the player status registers ( PSR) is described below. In the following, the notation as "PSR(x) " represents a set value of the xth player status register.
PSR(O) : Reserved
PSR(l) : stream no. of audio stream to be decoded
PSR(2) : stream no. of sub-picture stream to be
decoded
PSR(3) : no. representing angle setting by a user
PSR(4) : title no. currently reproduced PSR(5) : chapter no. currently reproduced PSR(6) : PL no. currently reproduced PSR(7) : PlayItem no. currently reproduced PSR(8) : time information indicating current
reproduction point PSR(9) : counted value at navigation timer PSR(IO) : button no. currently in selected state PSR(ll)-(12): Reserved
PSR(13) : parental level setting by a user PSR(14) : video reproduction setting of reproduction
apparatus PSR(15) : audio reproduction setting of
reproduction apparatus PSR(16) : language code representing audio setting
of reproduction apparatus PSR(17) .- language code representing sub-title
setting of reproduction apparatus PSR(18) : language setting for menu rendering PSR(19)-(63) : Reserved
The PSR(8) is updated every time each piece of picture data of a AV Clip is displayed. In other words, when the reproduction apparatus displays a new piece of picture data, the PSR( 8 ) is updated to indicate the value shown by the display start time (presentation time) of the new piece of picture data. By referring to this PSK(8), it is possible to know the current reproduction po.\nt.
The control unit 20 performs integration control through
bi-directional communication with the graphics decoder 12. The communication from the control unit 20 to the graphics decoder 12 is to output an UO received at the UO controller 18 to the graphics decoder 12. The communication from the graphics decoder 12 to the control unit 20 is to output a button command included in en ICS to the control unit 20. In the reproduction apparatus constituted as above, each constituting element performs decoding processing using pipelining.
FIG.33 is a timing chart showing the pipelining performed by the reproduction apparatus. The fourth row represents a Display Set in a BD-ROM, and the third row represents reading periods that are respectively for reading the ICS, PDS, andODS, to the coded data buff er 13 . The second row represents decoding periods for ODSs, performed by the stream graphics processor 14. The first row represents processing periods by the Graphics controller 17. The decoding start time for each ODS is shown by theDTSll, DTS12, and DTS13 in the drawing. The storing of ODS(N-ODSs[ODS1J) (being the first ODS among the N -ODSs) to the coded data buffer 13, finishes by DTS11, and the storing of ODS(N-ODSs[ODSn]) (being the last ODS among the N ODSs) to the coded data buf fer 13, finishes by DTS12 . In this way, each ODS will have been read to the coded data buffei 13 by the time shown by its own DTS.
On the other hand, the decode end time for each ODS
is shown by PTSll, PTS12, PTS13, in the drawing. Decoding of N-ODSs(ODSl) performed by the stream graphics processor 14 finishes by PTSll, and decoding of N-ODSs(ODSn) finishes by PTS12. In the above vay, each ODS is read to the coded data buffer 13 by the time shown by the DTS of the ODS, and the ODS read to the coded data buffer 13 is decoded and written to the object buffer 15 by the time shown by the PTS of the ODS . The stream graphics processor 14 performs the described series of processing using pipelining.
When the default selected button is statically decided, the time at which all the graphics objects for an initial display of an interactive display are ready is when the decoding of a first ODS of t he button-state set corresponding to the normal state and of a first ODS of the button-state set corresponding to the selected state is complete . In this diagram, at the time shown by the PTS13, all the graphics objects necessary for the initial display of the interactive display are ready.
In the drawing, the period cdl in the first row represents a period required to clear the graphics plane 8 by the graphics controller 17. In addition, the period tdl is a period required to write, to the graphics plane 8 , graphics objects that correspond to the first page of the interactive display, among those obtained in the object buffer 15. The writing destination in the graphics plane 8 is shown by button_hor i <.
Documents
Application Documents
| # |
Name |
Date |
| 1 |
2425-KOLNP-2009-FORM 13 [27-08-2022(online)].pdf |
2022-08-27 |
| 1 |
abstract-2425-kolnp-2009.jpg |
2011-10-07 |
| 2 |
2425-KOLNP-2009-POA [27-08-2022(online)].pdf |
2022-08-27 |
| 2 |
2425-kolnp-2009-specification.pdf |
2011-10-07 |
| 3 |
2425-KOLNP-2009-RELEVANT DOCUMENTS [27-08-2022(online)].pdf |
2022-08-27 |
| 3 |
2425-kolnp-2009-gpa.pdf |
2011-10-07 |
| 4 |
2425-KOLNP-2009_EXAMREPORT.pdf |
2016-06-30 |
| 4 |
2425-kolnp-2009-form 5.pdf |
2011-10-07 |
| 5 |
2425-kolnp-2009-form 3.pdf |
2011-10-07 |
| 5 |
2425-KOLNP-2009-(14-12-2015)-ANNEXURE TO FORM 3.pdf |
2015-12-14 |
| 6 |
2425-KOLNP-2009-FORM 3.1.1.pdf |
2011-10-07 |
| 6 |
2425-KOLNP-2009-Correspondence-160915.pdf |
2015-11-20 |
| 7 |
2425-kolnp-2009-form 2.pdf |
2011-10-07 |
| 7 |
2425-KOLNP-2009-(25-08-2015)-ABSTRACT.pdf |
2015-08-25 |
| 8 |
2425-KOLNP-2009-FORM 18.pdf |
2011-10-07 |
| 8 |
2425-KOLNP-2009-(25-08-2015)-ANNEXURE TO FORM 3.pdf |
2015-08-25 |
| 9 |
2425-KOLNP-2009-(25-08-2015)-CLAIMS.pdf |
2015-08-25 |
| 9 |
2425-kolnp-2009-form 1.pdf |
2011-10-07 |
| 10 |
2425-KOLNP-2009-(25-08-2015)-CORRESPONDENCE.pdf |
2015-08-25 |
| 10 |
2425-kolnp-2009-drawings.pdf |
2011-10-07 |
| 11 |
2425-KOLNP-2009-(25-08-2015)-FORM-1.pdf |
2015-08-25 |
| 11 |
2425-kolnp-2009-description (complete).pdf |
2011-10-07 |
| 12 |
2425-KOLNP-2009-(25-08-2015)-FORM-2.pdf |
2015-08-25 |
| 12 |
2425-kolnp-2009-correspondence.pdf |
2011-10-07 |
| 13 |
2425-KOLNP-2009-(25-08-2015)-FORM-3.pdf |
2015-08-25 |
| 13 |
2425-KOLNP-2009-CORRESPONDENCE 1.1.pdf |
2011-10-07 |
| 14 |
2425-KOLNP-2009-(25-08-2015)-FORM-5.pdf |
2015-08-25 |
| 14 |
2425-kolnp-2009-claims.pdf |
2011-10-07 |
| 15 |
2425-KOLNP-2009-(25-08-2015)-OTHERS.pdf |
2015-08-25 |
| 15 |
2425-kolnp-2009-abstract.pdf |
2011-10-07 |
| 16 |
2425-KOLNP-2009-(13-01-2014)-CORRESPONDENCE.pdf |
2014-01-13 |
| 16 |
2425-KOLNP-2009-(25-08-2015)-PETITION UNDER RULE 137.pdf |
2015-08-25 |
| 17 |
2425-KOLNP-2009-(13-01-2014)-ANNEXURE TO FORM 3.pdf |
2014-01-13 |
| 18 |
2425-KOLNP-2009-(25-08-2015)-PETITION UNDER RULE 137.pdf |
2015-08-25 |
| 18 |
2425-KOLNP-2009-(13-01-2014)-CORRESPONDENCE.pdf |
2014-01-13 |
| 19 |
2425-KOLNP-2009-(25-08-2015)-OTHERS.pdf |
2015-08-25 |
| 19 |
2425-kolnp-2009-abstract.pdf |
2011-10-07 |
| 20 |
2425-KOLNP-2009-(25-08-2015)-FORM-5.pdf |
2015-08-25 |
| 20 |
2425-kolnp-2009-claims.pdf |
2011-10-07 |
| 21 |
2425-KOLNP-2009-(25-08-2015)-FORM-3.pdf |
2015-08-25 |
| 21 |
2425-KOLNP-2009-CORRESPONDENCE 1.1.pdf |
2011-10-07 |
| 22 |
2425-KOLNP-2009-(25-08-2015)-FORM-2.pdf |
2015-08-25 |
| 22 |
2425-kolnp-2009-correspondence.pdf |
2011-10-07 |
| 23 |
2425-KOLNP-2009-(25-08-2015)-FORM-1.pdf |
2015-08-25 |
| 23 |
2425-kolnp-2009-description (complete).pdf |
2011-10-07 |
| 24 |
2425-kolnp-2009-drawings.pdf |
2011-10-07 |
| 24 |
2425-KOLNP-2009-(25-08-2015)-CORRESPONDENCE.pdf |
2015-08-25 |
| 25 |
2425-KOLNP-2009-(25-08-2015)-CLAIMS.pdf |
2015-08-25 |
| 25 |
2425-kolnp-2009-form 1.pdf |
2011-10-07 |
| 26 |
2425-KOLNP-2009-(25-08-2015)-ANNEXURE TO FORM 3.pdf |
2015-08-25 |
| 26 |
2425-KOLNP-2009-FORM 18.pdf |
2011-10-07 |
| 27 |
2425-KOLNP-2009-(25-08-2015)-ABSTRACT.pdf |
2015-08-25 |
| 27 |
2425-kolnp-2009-form 2.pdf |
2011-10-07 |
| 28 |
2425-KOLNP-2009-Correspondence-160915.pdf |
2015-11-20 |
| 28 |
2425-KOLNP-2009-FORM 3.1.1.pdf |
2011-10-07 |
| 29 |
2425-KOLNP-2009-(14-12-2015)-ANNEXURE TO FORM 3.pdf |
2015-12-14 |
| 29 |
2425-kolnp-2009-form 3.pdf |
2011-10-07 |
| 30 |
2425-kolnp-2009-form 5.pdf |
2011-10-07 |
| 30 |
2425-KOLNP-2009_EXAMREPORT.pdf |
2016-06-30 |
| 31 |
2425-KOLNP-2009-RELEVANT DOCUMENTS [27-08-2022(online)].pdf |
2022-08-27 |
| 31 |
2425-kolnp-2009-gpa.pdf |
2011-10-07 |
| 32 |
2425-kolnp-2009-specification.pdf |
2011-10-07 |
| 32 |
2425-KOLNP-2009-POA [27-08-2022(online)].pdf |
2022-08-27 |
| 33 |
2425-KOLNP-2009-FORM 13 [27-08-2022(online)].pdf |
2022-08-27 |