FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEPTION DEVICE, AND
RECEPTION METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-0075, Japan
The following specification particularly describes the invention and the manner in which it is to
be performed.
2
DESCRIPTION
TRANSMISSION APPARATUS, TRANSMISSION METHOD, RECEPTION
APPARATUS, AND RECEPTION METHOD
5 TECHNICAL FIELD
[0001]
The present technology relates to a transmission
apparatus, a transmission method, a reception apparatus,
and a reception method, and specifically relates to a
10 transmission apparatus or the like which transmits text
information together with video information.
BACKGROUND ART
[0002]
15 In, for example, digital video broadcasting (DVB),
information on subtitles has been transmitted as bitmap
data. Recently, it has been proposed that information on
subtitles is transmitted as a text character code, that
is, on a text basis. In this case, the font is expanded
20 according to the resolution on a reception side.
[0003]
Furthermore, it has been proposed that text
information is to have timing information when
information on subtitles is transmitted on a text basis.
25 As this text information, for example, a timed text
markup language (TTML) is proposed by the World Wide Web
Consortium (W3C) (see Patent Document 1).
CITATION LIST
30 PATENT DOCUMENT
[0004]
3
Patent Document 1: Japanese Patent Application Laid-Open
No. 2012-169885
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED 5 BY THE INVENTION
[0005]
A purpose of the present technology is to
excellently perform display timing control of subtitles
on a reception side.
10
SOLUTIONS TO PROBLEMS
[0006]
A concept of the present technology is a
transmission apparatus including:
15 a video encoder which generates a video stream
having coded image data;
a subtitle encoder which generates a subtitle
stream having text information and display timing
information on subtitles; and
20 a transmission unit which transmits a transport
stream containing the video stream and the subtitle
stream.
[0007]
In the present technology, the video stream having
25 the coded image data is generated by the video encoder.
The subtitle stream having the text information and the
display timing information on the subtitles is generated
by the subtitle encoder. Then, the transport stream
containing the video stream and the subtitle stream is
30 transmitted by the transmission unit.
[0008]
4
For example, the subtitle encoder may generate the
subtitle stream on the basis of the text information on
the subtitles in a predetermined format having the
display timing information. In this case, for example,
the text information on the 5 subtitles in the
predetermined format may be in a timed text markup
language (TTML) or in a derivative format of the TTML.
[0009]
Furthermore, for example, the display timing
10 information on the subtitles may have information on, at
least, display start timing and display duration. In
this case, for example, the subtitle stream may be
constituted by a PES packet constituted by a PES header
and a PES payload, the text information and the display
15 timing information on the subtitles may be arranged in
the PES payload, and the display start timing may be
indicated by a display offset from a presentation time
stamp (PTS) inserted in the PES header.
[0010]
20 In the present technology, the subtitle stream
having the text information and the display timing
information on the subtitles is transmitted together with
the video stream. Thus, it is possible to excellently
perform display timing control of subtitles on a
25 reception side.
[0011]
Furthermore, another concept of the present
technology is a reception apparatus including:
a reception unit which receives a transport stream
30 containing a video stream having coded image data and a
subtitle stream having text information and display
5
timing information on subtitles;
a video decoder which performs decoding processing
to the video stream and obtains image data;
a subtitle decoder which performs decoding
processing to the subtitle stream and obtains 5 bitmap data
of the subtitles and the display timing information on
the subtitles; and
a display control unit which controls timing to
superimpose the bitmap data of the subtitles on the image
10 data on the basis of the display timing information.
[0012]
In the present technology, the transport stream is
received by the reception unit. The transport stream
contains the video stream having the coded image data and
15 the subtitle stream having the text information and the
display timing information on the subtitles.
[0013]
The decoding processing is performed to the video
stream and the image data is obtained by the video
20 decoder. Furthermore, the decoding processing is
performed to the subtitle stream, and the bitmap data of
the subtitles and the display timing information on the
subtitles are obtained by the subtitle decoder. Then,
the timing to superimpose the bitmap data of the
25 subtitles on the image data is controlled by the display
control unit on the basis of display timing control
information.
[0014]
For example, the display timing information on the
30 subtitles may have information on, at least, display
start timing and display duration. In this case, for
6
example, the subtitle stream may be constituted by a PES
packet constituted by a PES header and a PES payload, the
text information and the display timing information on
the subtitles may be arranged in the PES payload, and the
display start timing may be indicated by 5 a display offset
from a PTS inserted in the PES header.
[0015]
In the present technology, the subtitle stream has
the text information and the display timing information
10 on the subtitles, and the timing to superimpose the
bitmap data of the subtitles on the image data is
controlled on the basis of the display timing information.
Thus, it is possible to excellently perform display
timing control of subtitles.
15
EFFECTS OF THE INVENTION
[0016]
According to the present technology, it is possible
to excellently perform display timing control of
20 subtitles. Note that, the effects described in the
present specification are merely exemplified and not
limited, and there may be additional effects.
BRIEF DESCRIPTION OF DRAWINGS
25 [0017]
Fig. 1 is a block diagram illustrating a
configuration example of a transceiver system of an
embodiment.
Fig. 2 is a block diagram illustrating a
30 configuration example of a stream generation unit of a
broadcast transmission system.
7
Fig. 3 is a diagram explaining a TTML structure.
Figs. 4(a) to 4(c) are diagrams illustrating
examples of structures of each element of metadata,
styling, and layout in the head of the TTML structure.
Fig. 5 is a diagram illustrating 5 a configuration
example of the body of the TTML structure.
Fig. 6 is a diagram illustrating a configuration
example of a PES packet.
Figs. 7(a) and 7(b) are diagrams illustrating
10 examples of structures of a text header metadata segment
(THMS).
Figs. 8(a) and 8(b) are diagrams illustrating
examples of structures of a text header styling segment
(THSS).
15 Figs. 9(a) and 9(b) are diagrams illustrating
examples of structures of a text header layout segment
(THLS).
Figs. 10(a) and 10(b) are diagrams illustrating
examples of structures of a text body segment (TBS).
20 Figs. 11(a) and 11(b) are diagrams illustrating
examples of structures of a text whole segment (TWS).
Fig. 12 is a diagram illustrating an example of a
structure of a timing control segment (TCS).
Fig. 13 is a diagram explaining the settings of
25 “PTS”, “display offset”, and “display duration” when the
TTML is converted into a segment.
Fig. 14 is a diagram illustrating a configuration
example of a decoder buffer model.
Fig. 15 is a diagram illustrating an example of
30 basic operation of the decoder buffer model.
Fig. 16 is a diagram illustrating an example of
8
more practical operation of the decoder buffer model.
Fig. 17 is a diagram illustrating another example
of more practical operation of the decoder buffer model.
Fig. 18 is a diagram illustrating another
configuration example of the decoder 5 buffer model.
Fig. 19 is a diagram illustrating an example of
basic operation of the decoder buffer model.
Fig. 20 is a diagram illustrating an example of
more practical operation of the decoder buffer model.
10 Fig. 21 is a diagram illustrating another example
of more practical operation of the decoder buffer model.
Fig. 22 is a diagram illustrating a configuration
example of a transport stream TS.
Fig. 23 is a diagram illustrating a configuration
15 example of a television receiver.
Fig. 24 is a diagram illustrating a configuration
example of a subtitle decoder.
Fig. 25 is a diagram illustrating another
configuration example of the subtitle decoder.
20
MODE FOR CARRYING OUT THE INVENTION
[0018]
Hereinafter, a mode for carrying out the invention
(hereinafter, referred to as an “embodiment”) is
25 described. Note that, the description is made in the
following order:
1. Embodiment
2. Modified example
[0019]
30 <1. Embodiment>
[Configuration example of Transceiver system]
9
Fig. 1 illustrates a configuration example of a
transceiver system 10 of an embodiment. The transceiver
system 10 includes a broadcast transmission system 100
and a television receiver 200. The broadcast
transmission system 100 transmits a transport 5 stream TS
on a broadcast wave. The transport stream TS contains a
video stream having coded image data, an audio stream
having coded audio data, and a subtitle stream having
text information and display timing information on
10 subtitles.
[0020]
The broadcast transmission system 100 generates a
subtitle stream on the basis of the text information on
the subtitles in a predetermined format having the
15 display timing information. In this embodiment, the text
information is, for example, a timed text markup language
(TTML) proposed by the World Wide Web Consortium (W3C).
[0021]
In this embodiment, the display timing information
20 on the subtitles contained in the subtitle stream has
information on, at least, display start timing and
display duration. Here, the subtitle stream is
constituted by a PES packet constituted by a PES header
and a PES payload, the text information and the display
25 timing information on the subtitles are arranged in the
PES payload, and, for example, the display start timing
is indicated by a display offset from a PTS inserted in
the PES header.
[0022]
30 The television receiver 200 receives the transport
stream TS transmitted on a broadcast wave from the
10
broadcast transmission system 100. The transport stream
TS contains, as described above, the video stream having
the coded image data and the subtitle stream having the
text information and the display timing information on
5 the subtitles.
[0023]
The television receiver 200 receives image data
from the video stream, and bitmap data of the subtitles
and the display timing information on the subtitles from
10 the subtitle stream. Then, the television receiver 200
controls the timing to superimpose the bitmap data of the
subtitle on the image data on the basis of the display
timing information, and displays the image on which the
subtitles are superimposed.
15 [0024]
[Configuration example of Stream generation unit of
Broadcast transmission system]
Fig. 2 illustrates a configuration example of a
stream generation unit 110 of the broadcast transmission
20 system 100. The stream generation unit 110 includes a
control unit 111, a video encoder 112, an audio encoder
113, a text format conversion unit 114, and a subtitle
encoder 115.
[0025]
25 The control unit 111 controls the operation of the
components in the stream generation unit 110. The video
encoder 112 receives image data DV, performs coding to
the image data DV, and generates a video stream having
the coded image data (PES stream). The audio encoder 113
30 receives audio data DA, performs coding to the audio data
DA, and generates an audio stream having the coded audio
11
data (PES stream).
[0026]
The text format conversion unit 114 receives text
data (character code) DT as information on subtitles, and
obtains text information on subtitles 5 in a predetermined
format having display timing information. In this
embodiment, the text information is converted into the
TTML.
[0027]
10 Fig. 3 illustrates a TTML structure. The TTML is
described on an XML basis. In the head, the elements of
metadata, styling, and layout exist. Fig. 4(a)
illustrates an example of a structure of the metadata
(TTM:TTML Metadata). The metadata contains metadata
15 title information and copyright information.
[0028]
Fig. 4(b) illustrates an example of a structure of
the styling (TTS:TTML Styling). The styling contains an
identifier (id) and information on a color, a font
20 (fontFamily), a size (fontSize), an alignment (textAlign),
and the like. Fig. 4(c) illustrates an example of a
structure of the layout (region:TTML layout). The layout
contains an identifier (id) of a region where subtitles
are arranged, and information on an area (extent), an
25 offset (padding), a background color (backgroundColor),
an alignment (displayAlign), and the like.
[0029]
Fig. 5 illustrates an example of a structure of the
body. In the illustrated example, the information on
30 three subtitles of a subtitle 1 (subtitle1), a subtitle 2
(subtitle2), and a subtitle 3 (subtitle3) is contained.
12
The display start timing, the display end timing, and the
text data are described for each subtitle. For example,
with regard to the subtitle 1, the display start timing
is “0.76s”, the display end timing is “3.45s”, and the
text data is “It seems a paradox, 5 dose it not,”.
[0030]
Returning back to Fig. 2, the subtitle encoder 115
converts the TTML obtained by the text format conversion
unit 114 into various segments, and generates a subtitle
10 stream constituted by the PES packet in which the
segments are arranged in the payload.
[0031]
Fig. 6 illustrates a configuration example of the
PES packet. The PES header contains a presentation time
15 stamp (PTS). Furthermore, the PES payload contains the
segments of a timing control segment (TCS), a text header
metadata segment (THMS), a text header styling segment
(THSS), a text header layout segment (THLS), and a text
body segment (TBS), or the segments of a timing control
20 segment (TCS) and a text whole segment (TWS).
[0032]
Fig. 7(a) illustrates an example of a structure
(syntax) of the text header metadata segment (THMS). The
syntax contains information on “sync_byte”,
25 “segment_type”, “page_id”, “segment_length”, and
“segment_payload()”. “segment_type” is 8-bit data
indicating a segment type, and is, for example, “0x21”
indicating the THMS in this example. “segment_length” is
8-bit data indicating the length (size) of the segment.
30 The metadata illustrated in Fig. 7(b) is arranged in
“segment_payload()” as XML information. The metadata is
13
the same as the elements of the metadata existing in the
head of the TTML (see Fig. 4(a)).
[0033]
Fig. 8(a) illustrates an example of a structure
(syntax) of the text header styling segment 5 (THSS). The
syntax contains information on “sync_byte”,
“segment_type”, “page_id”, “segment_length”, and
“segment_payload()”. “segment_type” is 8-bit data
indicating a segment type, and is, for example, “0x22”
10 indicating the THSS in this example. “segment_length” is
8-bit data indicating the length (size) of the segment.
The metadata illustrated in Fig. 8(b) is arranged in
“segment_payload()” as XML information. The metadata is
the same as the elements of the styling existing in the
15 head of the TTML (see Fig. 4(b)).
[0034]
Fig. 9(a) illustrates an example of a structure
(syntax) of the text header layout segment (THLS). The
syntax contains information on “sync_byte”,
20 “segment_type”, “page_id”, “segment_length”, and
“segment_payload()”. “segment_type” is 8-bit data
indicating a segment type, and is, for example, “0x23”
indicating the THLS in this example. “segment_length” is
8-bit data indicating the length (size) of the segment.
25 The metadata illustrated in Fig. 9(b) is arranged in
“segment_payload()” as XML information. The metadata is
the same as the elements of the layout existing in the
head of the TTML (see Fig. 4(c)).
[0035]
30 Fig. 10(a) illustrates an example of a structure
(syntax) of the text body segment (TBS). The syntax
14
contains information on “sync_byte”, “segment_type”,
“page_id”, “segment_length”, and “segment_payload()”.
“segment_type” is 8-bit data indicating a segment type,
and is, for example, “0x24” indicating the TBS in this
example. The metadata illustrated 5 in Fig. 10(b) is
arranged in “segment_payload()” as XML information. The
metadata is the same as the body of the TTML (see Fig. 5).
[0036]
Fig. 11(a) illustrates an example of a structure
10 (syntax) of the text whole segment (TWS). The syntax
contains information on “sync_byte”, “segment_type”,
“page_id”, “segment_length”, and “segment_payload()”.
“segment_type” is 8-bit data indicating a segment type,
and is, for example, “0x25” indicating the TWS in this
15 example. “segment_length” is 8-bit data indicating the
length (size) of the segment. The metadata illustrated
in Fig. 11(b) is arranged in “segment_payload()” as XML
information. The metadata is the whole TTML (see Fig. 3).
The syntax is to maintain the compatibility in the whole
20 TTML, and to put the whole TTML in one segment.
[0037]
Fig. 12 illustrates an example of a structure
(syntax) of the timing control segment (TCS). The syntax
contains information on “sync_byte”, “segment_type”,
25 “page_id”, “segment_length”, “format_type”,
“number_of_regions”, “region_id”, “number_of_display_set”,
“display offset”, and “display duration”. “segment_type”
is 8-bit data indicating a segment type, and is, for
example, “0x20” indicating the TCS in this example.
30 “segment_length” is 8-bit data indicating the length
(size) of the segment.
15
[0038]
The 8-bit field of “format_type” indicates a type
of a transmission format of subtitles. “1” indicates
“DVB subtitle”, “2” indicates “EBU-TT-D”, and “3”
indicates “TTML”. In this embodiment, 5 the 8-bit field of
“format_type” is to be “2” or “3”
[0039]
The 8-bit field of “number_of_regions” indicates
the number of regions. The 16-bit field of “region_id”
10 indicates an identifier of a region. The 8 bits of
“number_of_display_set” indicated the number of displays
of the region. The 16-bit field of “display offset”
indicates an offset value from the PTS as the display
start timing in the region. The 16-bit field of “display
15 duration” indicates the display duration of the region.
[0040]
As illustrated in Fig. 13, when converting the TTML
into the segments, the subtitle encoder 115 sets “PTS”,
“display offset”, and “display duration” of each subtitle
20 referring to the system time information (the PCR and the
synchronization time of the video and the audio) on the
basis of the description on the display start timing
(begin) and the display end timing (end) of each subtitle
contained in the body of the TTML. At this time, a
25 subtitle segment encoder 115 sets “PTS”, “display offset”,
and “display duration” while verifying that the operation
of the reception side is correctly performed using a
decoder buffer model.
[0041]
30 Fig. 14 illustrates a configuration example of a
decoder buffer model 300A. The decoder buffer model 300A
16
includes a coded buffer 301, a subtitle segment decoder
302, a font expansion unit 303, and a bitmap buffer 304.
The coded buffer 301 temporarily holds received segment
data.
5 [0042]
The subtitle segment decoder 302 performs decoding
processing to the segment data of each region held by the
coded buffer 301 at a predetermined timing, and obtains
the text data and the control code of each region. The
10 font expansion unit 303 expands the font on the basis of
the text data and the control code of each region
obtained by the subtitle segment decoder 302, and obtains
the bitmap data of each region.
[0043]
15 The bitmap buffer 304 temporarily holds the bitmap
data of each region obtained by the font expansion unit
303 and control information for displaying the bitmap
data. The bitmap data of each region held by the bitmap
buffer 304 is read from the display start timing, and
20 superimposed on image data, and it lasts for the display
duration.
[0044]
Fig. 15 illustrates an example of basic operation
of the decoder buffer model 300A. The segment data of a
25 region 1 is buffered in the coded buffer 301. Then,
after all the segment data of the region 1 is stored in
the coded buffer 301 and a certain window period (Decode
Window) elapses, the segment data is decoded by the
subtitle segment decoder 302, the font is expanded by the
30 font expansion unit 303, and the bitmap data of the
region 1 is obtained.
17
[0045]
With the decoding processing, the segment data of
the region 1 instantly disappears from the coded buffer
301. The bitmap data obtained by the font expansion unit
303 is held by the bitmap buffer 304. The 5 display of the
region 1 is started and lasts for the display duration.
In the illustrated example, the region 1 having the
characters “ABC” is displayed on the screen immediately
after the decoding for the display duration. When the
10 display duration elapses, the bitmap data of the region 1
instantly disappears from the bitmap buffer 304.
[0046]
After all the segment data of the region 1 is
stored in the coded buffer 301, the segment data of a
15 region 2 is buffered in the coded buffer 301. The
processing similar to the above region 1 is sequentially
performed for the region 2.
[0047]
In other words, after all the segment data of the
20 region 2 is stored in the coded buffer 301 and a certain
window period (Decode Window) elapses, the segment data
is decoded by the subtitle segment decoder 302, the font
is expanded by the font expansion unit 303, and the
display of the region 2 is started and lasts for the
25 display duration. In the illustrated example, the region
2 having the characters “DEFG” is displayed on the screen
immediately after the decoding for the display duration.
[0048]
Thereafter, similar processing is continued.
30 [0049]
Fig. 16 illustrates an example of more practical
18
operation of the decoder buffer model 300A. This example
shows that the region 1 is only displayed at a certain
timing, the region 2 is simultaneously displayed from the
next timing, the region 1 is not displayed at a certain
timing, and then, the display of a region 5 3 is added to
the display of the region 2.
[0050]
The segment data of a region 1 is buffered in the
coded buffer 301. Then, after all the segment data of
10 the region 1 is stored in the coded buffer 301 and a
certain window period (Decode Window) elapses, the
segment data is decoded by the subtitle segment decoder
302, the font is expanded by the font expansion unit 303,
and the bitmap data of the region 1 is obtained at the
15 timing of the PTS1. The shortest window period is 0.
This means that all the segment data necessary for the
decoding needs to be buffered in the coded buffer 301 by
the decode timing.
[0051]
20 With the decoding processing, the segment data of
the region 1 instantly disappears from the coded buffer
301. The bitmap data of the region 1 obtained by the
font expansion unit 303 is held by the bitmap buffer 304.
The data storage amount of the coded buffer 301 at the
25 timing of the PTS1 is CBc_1, and the data storage amount
of the bitmap buffer 304 at the timing of the PTS1 is
BBc_1.
[0052]
Here, “display offset” is defined. This indicates
30 the delay from the PTS timing to the display start timing.
In the illustrated example, the display of the region 1
19
is started at the timing of the PTS1, which means
“display offset1 = 0”.
[0053]
Furthermore, “display duration” is defined here.
This indicates the display duration. 5 The illustrated
example shows “display duration1 = T1”. In the
illustrated example, the region 1 having the characters
“ABC” is displayed on the screen for the duration T1.
When the display duration elapses, the bitmap data of the
10 region 1 instantly disappears from the bitmap buffer 304.
In other words, while the region 1 is being displayed,
the bitmap data of the region 1 is maintained in the
bitmap buffer 304.
[0054]
15 Here, the buffer model is restricted so that the
data storage amount CBc_k of the coded buffer 301 is
smaller than its buffer size at all times and the data
storage amount BBc_k of the bitmap buffer 304 is smaller
than its buffer size at all times.
20 [0055]
After all the segment data of the region 1 is
stored in the coded buffer 301, the segment data of the
region 2 is buffered in the coded buffer 301. Then,
after all the segment data of the region 2 is stored in
25 the coded buffer 301 and a certain window period (Decode
Window) elapses, the segment data is decoded by the
subtitle segment decoder 302, the font is expanded by the
font expansion unit 303, and the bitmap data of the
region 2 is obtained at the timing of the PTS2.
30 [0056]
With the decoding processing, the segment data of
20
the region 2 instantly disappears from the coded buffer
301. The bitmap data of the region 2 obtained by the
font expansion unit 303 is held by the bitmap buffer 304.
The data storage amount of the coded buffer 301 at the
timing of the PTS2 is CBc_2, and the data 5 storage amount
of the bitmap buffer 304 at the timing of the PTS2 is
BBc_2.
[0057]
Here, the display of the region 2 is started at the
10 timing of the PTS2, which means “display offset2_1 = 0”.
Then, the region 2 is displayed for the duration T2_1,
which means “display duration2_1 = T2_1”. In the
illustrated example, the region 2 having the characters
“DEF” is displayed on the screen for the duration T2_1.
15 In this case, the display of the region 2 is added to the
display of the region 1 from the timing of the PTS2.
Note that, the display position of the region 1 is
shifted upward at the timing of the PTS2.
[0058]
20 Furthermore, with regard to the region 2, the
bitmap data of the region 2 does not disappear from the
bitmap buffer 304 at the end timing of the duration T2_1,
but is held. The display of the region 2 is resumed at
the timing of the PTS3 after a predetermined duration
25 from the timing of the PTS2, which means “display
offset2_1 = !0”. Then, the region 2 is displayed for the
duration T2_2, which means “display duration2_2 = T2_2”.
[0059]
Furthermore, after all the segment data of the
30 region 2 is stored in the coded buffer 301, the segment
data of the region 3 is buffered in the coded buffer 301.
21
Then, after all the segment data of the region 3 is
stored in the coded buffer 301 and a certain window
period (Decode Window) elapses, the segment data is
decoded by the subtitle segment decoder 302, the font is
expanded by the font expansion unit 303, 5 and the bitmap
data of the region 3 is obtained at the timing of the
PTS3.
[0060]
With the decoding processing, the segment data of
10 the region 3 instantly disappears from the coded buffer
301. The bitmap data obtained by the font expansion unit
303 is held by the bitmap buffer 304. The data storage
amount of the coded buffer 301 at the timing of the PTS3
is CBc_3, and the data storage amount of the bitmap
15 buffer 304 at the timing of the PTS3 is BBc_3.
[0061]
Here, the display of the region 3 is started at the
timing of the PTS3, which means “display offset3 = 0”.
Then, the region 3 is displayed for the duration T3,
20 which means “display duration3 = T3”. In the illustrated
example, the region 3 having the characters “GHJK” is
displayed on the screen for the duration T3. In this
case, the regions 2 and 3 are displayed from the timing
of the PTS3. Note that, the display position of the
25 region 2 is shifted upward at the timing of the PTS3.
[0062]
Note that, the example illustrates that there is
the time difference between the end timing of the
duration T2_1 and the timing of the PTS3, but it may be
30 the same timing.
[0063]
22
Fig. 17 illustrates another example of more
practical operation of the decoder buffer model 300A.
This example shows, as similarly to the example
illustrated in Fig. 16, that the region 1 is only
displayed at a certain timing, 5 the region 2 is
simultaneously displayed from the next timing, the region
1 is not displayed at a certain timing, and then, the
display of the region 3 is added to the display of the
region 2. However, unlike the example illustrated in Fig.
10 16, the regions 1 and 2 are decoded simultaneously.
[0064]
The segment data of the region 1 and the segment
data of the region 2 are buffered in the coded buffer 301.
Then, after all the segment data of the regions 1 and 2
15 is stored in the coded buffer 301 and a certain window
period (Decode Window) elapses, the segment data is
decoded by the subtitle segment decoder 302, the font is
expanded by the font expansion unit 303, and the bitmap
data of the regions 1 and 2 is obtained at the timing of
20 the PTS1.
[0065]
With the decoding processing, the segment data of
the regions 1 and 2 instantly disappears from the coded
buffer 301. The bitmap data of the regions 1 and 2
25 obtained by the font expansion unit 303 is held by the
bitmap buffer 304. The data storage amount of the coded
buffer 301 at the timing of the PTS1 is CBc_1, and the
data storage amount of the bitmap buffer 304 at the
timing of the PTS1 is BBc_1.
30 [0066]
Here, “display offset” is defined. This indicates
23
the delay from the PTS timing to the display start timing.
In the illustrated example, the display of the region 1
is started at the timing of the PTS1, which means
“display offset1 = 0”.
5 [0067]
Furthermore, “display duration” is defined here.
This indicates the display duration. The illustrated
example shows “display duration1 = T1”. In the
illustrated example, the region 1 having the characters
10 “ABC” is displayed on the screen for the duration T1.
When the display duration elapses, the bitmap data of the
region 1 instantly disappears from the bitmap buffer 304.
In other words, while the region 1 is being displayed,
the bitmap data of the region 1 is maintained in the
15 bitmap buffer 304.
[0068]
Here, the buffer model is restricted so that the
data storage amount CBc_k of the coded buffer 301 is
smaller than its buffer size at all times and the data
20 storage amount BBc_k of the bitmap buffer 304 is smaller
than its buffer size at all times.
[0069]
Furthermore, the display of the region 2 is started
at the timing after the duration T2_0 from the timing of
25 the PTS1, which means “display offset2_1 = T2_0”. Then,
the region 2 is displayed for the duration T2_1, which
means “display duration2_1 = T2_1”. In the illustrated
example, the region 2 having the characters “DEF” is
displayed on the screen for the duration T2_1. In this
30 case, the display of the region 2 is added to the display
of the region 1. Note that, the display position of the
24
region 1 is shifted upward.
[0070]
Furthermore, with regard to the region 2, the
bitmap data of the region 2 does not disappear from the
bitmap buffer 304 at the end timing of 5 the duration T2_1,
but is held. The display of the region 2 is resumed at
the timing of the PTS3 after a predetermined duration
from the timing of the PTS1, which means “display
offset2_1 = !0”. Then, the region 2 is displayed for the
10 duration T2_2, which means “display duration2_2 = T2_2”.
[0071]
Furthermore, after all the segment data of the
regions 1 and 2 is stored in the coded buffer 301, the
segment data of the region 3 is buffered in the coded
15 buffer 301. Then, after all the segment data of the
region 3 is stored in the coded buffer 301 and a certain
window period (Decode Window) elapses, the segment data
is decoded by the subtitle segment decoder 302, the font
is expanded by the font expansion unit 303, and the
20 bitmap data of the region 3 is obtained at the timing of
the PTS3.
[0072]
With the decoding processing, the segment data of
the region 3 instantly disappears from the coded buffer
25 301. The bitmap data obtained by the font expansion unit
303 is held by the bitmap buffer 304. The data storage
amount of the coded buffer 301 at the timing of the PTS3
is CBc_3, and the data storage amount of the bitmap
buffer 304 at the timing of the PTS3 is BBc_3.
30 [0073]
Here, the display of the region 3 is started at the
25
timing of the PTS3, which means “display offset3 = 0”.
Then, the region 3 is displayed for the duration T3,
which means “display duration3 = T3”. In the illustrated
example, the region 3 having the characters “GHJK” is
displayed on the screen for the duration 5 T3. In this
case, the regions 2 and 2 are displayed from the timing
of the PTS3. Note that, the display position of the
region 2 is shifted upward at the timing of the PTS3.
[0074]
10 Note that, the example illustrates that there is
the time difference between the end timing of the
duration T2_1 and the timing of the PTS3, but it may be
the same timing.
[0075]
15 Fig. 18 illustrates a configuration example of a
decoder buffer model 300B. The decoder buffer model 300B
includes a coded buffer 311, a subtitle segment decoder
312, and a font expansion unit 313. The coded buffer 311
temporarily holds received segment data.
20 [0076]
The subtitle segment decoder 312 continuously
performs decoding processing for each frame to the
segment data of each region held by the coded buffer 311
and obtains the text data and the control code of each
25 region during the display duration of each region. The
font expansion unit 313 expands, for each frame, the font
on the basis of the text data and the control code
obtained by the subtitle segment decoder 312 and obtains
the bitmap data of each region during the display
30 duration of each region. The bitmap data of each region
obtained by the font expansion unit 313 is superimposed
26
on the image data.
[0077]
Fig. 19 illustrates an example of basic operation
of the decoder buffer model 300B. The segment data of
the region 1 is buffered in the coded 5 buffer 311. Then,
after all the segment data of the region 1 is stored in
the coded buffer 311 and a certain window period (Decode
Window) elapses, the segment data is decoded by the
subtitle segment decoder 312, the font is expanded by the
10 font expansion unit 313, and the bitmap data of the
region 1 is obtained. Thus, the display of the region 1
is started and lasts for the display duration. In the
illustrated example, the region 1 having the characters
“ABC” is displayed on the screen for the display duration.
15 [0078]
In the subtitle segment decoder 312 and the font
expansion unit 313, the processing is repeated for each
frame during the display duration. When the display
duration of the region 1 elapses, the processing of the
20 subtitle segment decoder 312 and the font expansion unit
313 corresponding to the segment data of the region 1 is
stopped, and the segment data of the region 1 instantly
disappears from the coded buffer 311 at the timing.
[0079]
25 After all the segment data of the region 1 is
stored in the coded buffer 311, the segment data of a
region 2 is buffered in the coded buffer 311. The
processing similar to the above region 1 is sequentially
performed for the region 2.
30 [0080]
In other words, after all the segment data of the
27
region 2 is stored in the coded buffer 311 and a certain
window period (Decode Window) elapses, the segment data
is decoded by the subtitle segment decoder 312, the font
is expanded by the font expansion unit 313, and the
bitmap data of the region 2 is obtained. 5 Thus, the
display of the region 2 is started and lasts for the
display duration. In the illustrated example, the region
2 having the characters “DEFG” is displayed on the screen
for the display duration.
10 [0081]
Thereafter, similar processing is continued.
[0082]
Fig. 20 illustrates an example of more practical
operation of the decoder buffer model 300B. This example
15 shows that the region 1 is only displayed at a certain
timing, the region 2 is simultaneously displayed from the
next timing, the region 1 is not displayed at a certain
timing, and then, the display of a region 3 is added to
the display of the region 2.
20 [0083]
The segment data of the region 1 is buffered in the
coded buffer 311. Then, after all the segment data of
the region 1 is stored in the coded buffer 311 and a
certain window period (Decode Window) elapses, the timing
25 of the PTS1 comes.
[0084]
Here, “display offset” is defined. This indicates
the delay from the PTS timing to the display start timing.
In the illustrated example, the display of the region 1
30 is started at the timing of the PTS1, which means
“display offset1 = 0”. In this case, the segment data of
28
the region 1 is decoded by the subtitle segment decoder
312, the font is expanded by the font expansion unit 313,
and the bitmap data of the region 1 is obtained at the
timing of the PTS1. Thus, the display of the region 1 is
started. In the illustrated example, 5 the display of the
region 1 having the characters “ABC” is started.
[0085]
Furthermore, “display duration” is defined here.
This indicates the display duration. The example shows
10 “display duration1 = T1”, and the display of the region 1
lasts for the duration T1. In the subtitle segment
decoder 312 and the font expansion unit 313, the
processing is repeated for each frame during the display
duration of the region 1. When the display duration of
15 the region 1 elapses, the processing of the subtitle
segment decoder 312 and the font expansion unit 313
corresponding to the segment data of the region 1 is
stopped, and the segment data of the region 1 instantly
disappears from the coded buffer 311 at the timing.
20 [0086]
The data storage amount of the coded buffer 311 at
the timing of the PTS1 is CBc_1. Here, the buffer model
is restricted so that the data storage amount CBc_k of
the coded buffer 311 is smaller than its buffer size at
25 all times.
[0087]
After all the segment data of the region 1 is
stored in the coded buffer 311, the segment data of the
region 2 is buffered in the coded buffer 311. Then,
30 after all the segment data of the region 2 is stored in
the coded buffer 311 and a certain window period (Decode
29
Window) elapses, the timing of the PTS2 comes. The data
storage amount of the coded buffer 311 at the timing of
the PTS2 is CBc_2.
[0088]
The display of the region 2 5 is started at the
timing of the PTS2, which means “display offset2-1 = 0”.
In this case, the segment data of the region 2 is decoded
by the subtitle segment decoder 312, the font is expanded
by the font expansion unit 313, and the bitmap data of
10 the region 2 is obtained at the timing of the PTS2. Thus,
the display of the region 2 is started. In the
illustrated example, the display of the region 2 having
the characters “DEF” is started. In this case, the
display of the region 2 is added to the display of the
15 region 1 from the timing of the PTS2. Note that, the
display position of the region 1 is shifted upward at the
timing of the PTS2.
[0089]
Then, the region 2 is displayed for the duration
20 T2_1, which means “display duration2_1 = T2_1”. In the
illustrated example, the region 2 having the characters
“DEF” is displayed on the screen for the duration T2_1.
In the subtitle segment decoder 312 and the font
expansion unit 313, the processing is repeated for each
25 frame during the display duration of the region 2.
[0090]
Furthermore, with regard to the region 2, the
segment data of the region 2 does not disappear from the
coded buffer 311 at the end timing of the duration T2_1,
30 but is held. The display of the region 2 is resumed at
the timing of the PTS3 after a predetermined duration
30
from the timing of the PTS2, which means “display
offset2_1 = !0”. Then, the region 2 is displayed for the
duration T2_2, which means “display duration2_2 = T2_2”.
When all the display duration of the region 2 elapses,
the segment data of the region 2 instantly 5 disappears
from the coded buffer 311.
[0091]
Furthermore, after all the segment data of the
region 2 is stored in the coded buffer 311, the segment
10 data of the region 3 is buffered in the coded buffer 311.
Then, after all the segment data of the region 3 is
stored in the coded buffer 311 and a certain window
period (Decode Window) elapses, the timing of the PTS3
comes. The data storage amount of the coded buffer 311
15 at the timing of the PTS3 is CBc_3.
[0092]
The display of the region 3 is started at the
timing of the PTS3, which means “display offset3 = 0”.
In this case, the segment data of the region 3 is decoded
20 by the subtitle segment decoder 312, the font is expanded
by the font expansion unit 313, and the bitmap data of
the region 3 is obtained at the timing of the PTS3. Thus,
the display of the region 3 is started. In the
illustrated example, the display of the region 3 having
25 the characters “GHJK” is started. In this case, the
display of the region 3 is added to the display of the
region 2 from the timing of the PTS3. Note that, the
display position of the region 2 is shifted upward at the
timing of the PTS3.
30 [0093]
Then, the region 3 is displayed for the duration T3,
31
which means “display duration3 = T3”. In the illustrated
example, the region 3 having the characters “GHJK” is
displayed on the screen for the duration T3. In the
subtitle segment decoder 312 and the font expansion unit
313, the processing is repeated for each 5 frame during the
display duration of the region 3.
[0094]
Note that, the example illustrates that there is
the time difference between the end timing of the
10 duration T2_1 and the timing of the PTS3, but it may be
the same timing.
[0095]
Fig. 21 illustrates another example of more
practical operation of the decoder buffer model 300B.
15 This example shows, as similarly to the example
illustrated in Fig. 20, that the region 1 is only
displayed at a certain timing, the region 2 is
simultaneously displayed from the next timing, the region
1 is not displayed at a certain timing, and then, the
20 display of the region 3 is added to the display of the
region 2. However, unlike the example illustrated in Fig.
20, after all the segment data of the regions 1 and 2 is
stored in the coded buffer 311, the display of the region
1 is started.
25 [0096]
The segment data of the region 1 and the segment
data of the region 2 are buffered in the coded buffer 301.
Then, after all the segment data of the regions 1 and 2
is stored in the coded buffer 301 and a certain window
30 period (Decode Window) elapses, the timing of the PTS1
comes.
32
[0097]
Here, “display offset” is defined. This indicates
the delay from the PTS timing to the display start timing.
In the illustrated example, the display of the region 1
is started at the timing of the 5 PTS1, which means
“display offset1 = 0”. In this case, the segment data of
the region 1 is decoded by the subtitle segment decoder
312, the font is expanded by the font expansion unit 313,
and the bitmap data of the region 1 is obtained at the
10 timing of the PTS1. Thus, the display of the region 1 is
started. In the illustrated example, the display of the
region 1 having the characters “ABC” is started.
[0098]
Furthermore, “display duration” is defined here.
15 This indicates the display duration. The example shows
“display duration1 = T1”, and the display of the region 1
lasts for the duration T1. In the subtitle segment
decoder 312 and the font expansion unit 313, the
processing is repeated for each frame during the display
20 duration of the region 1. When the display duration of
the region 1 elapses, the processing of the subtitle
segment decoder 312 and the font expansion unit 313
corresponding to the segment data of the region 1 is
stopped, and the segment data of the region 1 instantly
25 disappears from the coded buffer 311 at the timing.
[0099]
The data storage amount of the coded buffer 311 at
the timing of the PTS1 is CBc_1. Here, the buffer model
is restricted so that the data storage amount CBc_k of
30 the coded buffer 311 is smaller than its buffer size at
all times.
33
[0100]
Furthermore, the display of the region 2 is started
at the timing after the duration T2_0 from the timing of
the PTS1, which means “display offset2_1 = T2_0”. The
segment data of the region 2 is decoded 5 by the subtitle
segment decoder 312, the font is expanded by the font
expansion unit 313, and the bitmap data of the region 2
is obtained at the timing. Thus, the display of the
region 2 is started. In the illustrated example, the
10 display of the region 2 having the characters “DEF” is
started. In this case, the display of the region 2 is
added to the display of the region 1. Note that, the
display position of the region 1 is shifted upward.
[0101]
15 Then, the region 2 is displayed for the duration
T2_1, which means “display duration2_1 = T2_1”. In the
illustrated example, the region 2 having the characters
“DEF” is displayed on the screen for the duration T2_1.
In the subtitle segment decoder 312 and the font
20 expansion unit 313, the processing is repeated for each
frame during the display duration of the region 2.
[0102]
Furthermore, with regard to the region 2, the
segment data of the region 2 does not disappear from the
25 coded buffer 311 at the end timing of the duration T2_1,
but is held. The display of the region 2 is resumed at
the timing of the PTS3 after a predetermined duration
from the timing of the PTS2, which means “display
offset2_1 = !0”. Then, the region 2 is displayed for the
30 duration T2_2, which means “display duration2_2 = T2_2”.
When all the display duration of the region 2 elapses,
34
the segment data of the region 2 instantly disappears
from the coded buffer 311.
[0103]
Furthermore, after all the segment data of the
region 2 is stored in the coded buffer 5 311, the segment
data of the region 3 is buffered in the coded buffer 311.
Then, after all the segment data of the region 3 is
stored in the coded buffer 311 and a certain window
period (Decode Window) elapses, the timing of the PTS3
10 comes. The data storage amount of the coded buffer 311
at the timing of the PTS3 is CBc_3.
[0104]
The display of the region 3 is started at the
timing of the PTS3, which means “display offset3 = 0”.
15 In this case, the segment data of the region 3 is decoded
by the subtitle segment decoder 312, the font is expanded
by the font expansion unit 313, and the bitmap data of
the region 3 is obtained at the timing of the PTS3. Thus,
the display of the region 3 is started. In the
20 illustrated example, the display of the region 3 having
the characters “GHJK” is started. In this case, the
display of the region 3 is added to the display of the
region 2 from the timing of the PTS3. Note that, the
display position of the region 2 is shifted upward at the
25 timing of the PTS3.
[0105]
Then, the region 3 is displayed for the duration T3,
which means “display duration3 = T3”. In the illustrated
example, the region 3 having the characters “GHJK” is
30 displayed on the screen for the duration T3. In the
subtitle segment decoder 312 and the font expansion unit
35
313, the processing is repeated for each frame during the
display duration of the region 3.
[0106]
Note that, the example illustrates that there is
the time difference between the 5 end timing of the
duration T2_1 and the timing of the PTS3, but it may be
the same timing.
[0107]
Returning back to Fig. 2, a TS formatter 116
10 multiplexes the video stream generated by the video
encoder 112, the audio stream generated by the audio
encoder 113, and the subtitle stream generated by the
subtitle encoder 115 by transport-packetizing them, and
obtains the transport stream TS as a multiplexed stream.
15 [0108]
The operation of the stream generation unit 110
illustrated in Fig. 3 is briefly described. The image
data DV is supplied to the video encoder 112. In the
video encoder 112, coding is performed to the image data
20 DV, and a video stream having the coded image data (PES
stream) is generated. The video stream is supplied to
the TS formatter 116. Furthermore, the audio data DA is
supplied to the audio encoder 113. In the audio encoder
113, coding is performed to the audio data DA, and an
25 audio stream having the coded audio data (PES stream) is
generated. The audio stream is supplied to the TS
formatter 116.
[0109]
Furthermore, the text data (character code) DT as
30 the subtitle information is supplied to the text format
conversion unit 114. In the text format conversion unit
36
114, the text information on the subtitles in a
predetermined format having the display timing
information, here the TTML, is obtained. The TTML is
supplied to the subtitle encoder 115.
5 [0110]
In the subtitle encoder 115, the TTML is converted
into various segments, and a subtitle stream constituted
by the PES packet in which the segments are arranged in
the payload is generated. The subtitle stream is
10 supplied to the TS formatter 116.
[0111]
The payload part of the PES packet contains the
segments of the TCS, the THMS, the THSS, the THLS, and
the TBS, or the segments of the TCS and the TWS. The
15 display timing information on the subtitles (region)
(having information on display start timing and display
duration) is transmitted by the TCS segment, and the TTML
information is transmitted by the other segments.
[0112]
20 In the subtitle encoder 115, when the TTML is
converted into the segments, “PTS” to be inserted in the
PES header and “display offset” and “display duration”
inserted in the TCS are set referring to the system time
information (the PCR and the synchronization time of the
25 video and the audio) on the basis of the description on
the display start timing and the display end timing of
each subtitle contained in the body of the TTML.
[0113]
In the TS formatter 116, the video stream generated
30 by the video encoder 112, the audio stream generated by
the audio encoder 113, and the subtitle stream generated
37
by the subtitle encoder 115 are multiplexed by transportpacketizing
them, and the transport stream TS is obtained
as a multiplexed stream.
[0114]
Fig. 22 illustrates a configuration 5 example of the
transport stream TS. In this configuration example,
there exists a PES packet “Video PES” of a video stream
identified by PID1. Furthermore, in this configuration
example, there exists a PES packet “Audio PES” of an
10 audio stream identified by PID2. Furthermore, in this
configuration example, there exists a PES packet
“Subtitle PES” of a subtitle stream identified by PID3.
[0115]
A PES packet is constituted by a PES header and a
15 PES payload. In the PES packet of the video stream, a
DTS/PTS time stamp is inserted in the PES header, and a
coded video stream is inserted in the PES payload.
Furthermore, in the PES packet of the audio stream, a PTS
time stamp is inserted in the PES header, and a coded
20 audio stream is inserted in the PES payload.
[0116]
Furthermore, in the PES packet of the subtitle
stream, a PTS time stamp is inserted in the PES header,
and a plurality of subtitle segments is inserted in the
25 PES payload. In other words, the subtitle segments are
the segments of the TCS, the THMS, the THSS, the THLS,
and the TBS, or the segments of the TCS and the TWS.
[0117]
Furthermore, the transport stream TS contains a
30 program map table (PMT) as program specific information
(PSI). The PSI is the information in which it is
38
described that each elementary stream contained in the
transport stream belongs to which program. Furthermore,
the transport stream TS contains an event information
table (EIT) as serviced information (SI) for performing
management in event units. The metadata 5 in program units
is described in the EIT.
[0118]
There exists a program descriptor for describing
information on all the programs in the PMT. Furthermore,
10 there exists an elementary stream loop having information
related to each elementary stream in the PMT. In this
configuration example, there exist a video elementary
stream loop (Video ES loop), an audio elementary stream
loop (Audio ES loop), and a subtitle elementary stream
15 loop (Subtitle ES loop). In each loop, the information
on a packet identifier (PID) and the like are arranged,
and a descriptor for describing the information on the
elementary stream is also arranged although not
illustrated.
20 [0119]
[Configuration example of Television receiver]
Fig. 23 illustrates a configuration example of the
television receiver 200. The television receiver 200
includes a reception unit 201, a TS analysis unit 202, a
25 video decoder 203, a video superimposing unit 204, a
panel drive circuit 205, and a display panel 206.
Furthermore, the television receiver 200 includes an
audio decoder 207, an audio output circuit 208, a speaker
209, and a subtitle decoder 210. Furthermore, the
30 television receiver 200 includes a CPU 221, a flash ROM
222, a DRAM 223, an internal bus 224, a remote control
39
reception unit 225, and a remote control transmitter 226.
[0120]
The CPU 221 controls the operation of the
components in the television receiver 200. The flash ROM
222 stores control software and data. The 5 DRAM 223 forms
the work area of the CPU 221. The CPU 221 expands
software or data read from the flash ROM 222 in the DRAM
223, starts the software, and controls the components in
the television receiver 200.
10 [0121]
The remote control reception unit 225 receives a
remote control signal (remote control code) transmitted
from the remote control transmitter 226 and supplies the
signal to the CPU 221. The CPU 221 controls the
15 components in the television receiver 200 on the basis of
the remote control code. The CPU 221, the flash ROM 222,
and the DRAM 223 are connected with the internal bus 224.
[0122]
The reception unit 201 receives the transport
20 stream TS transmitted on a broadcast wave from the
broadcast transmission system 100. As described above,
the transport stream TS contains the video stream, the
audio stream, and the subtitle stream. The TS analysis
unit 202 extracts the packets of the video, audio, and
25 subtitle streams from the transport stream TS.
[0123]
The audio decoder 207 performs decoding processing
to the audio stream extracted by the TS analysis unit 202,
and obtains audio data. The audio output circuit 208
30 performs necessary processing, such as D/A conversion and
amplification, to the audio data, and supplies it to the
40
speaker 209. The video decoder 203 performs decoding
processing to the video stream extracted by the TS
analysis unit 202, and obtains image data.
[0124]
The subtitle decoder 210 processes 5 the segment data
of each region extracted by the TS analysis unit 202, and
outputs the bitmap data of each region to be superimposed
on the image data. Figs. 24 and 25 illustrate
configuration examples of the subtitle decoder 210.
10 [0125]
The subtitle decoder 210 illustrated in Fig. 24 has
a configuration corresponding to the decoder buffer model
300A in Fig. 14. In other words, the subtitle decoder
210 includes a coded buffer 231, a subtitle segment
15 decoder 232, a font expansion unit 233, and a bitmap
buffer 234.
[0126]
The coded buffer 231 temporarily holds received
segment data. The subtitle segment decoder 232 performs
20 decoding processing to the segment data of each region
held by the coded buffer 231 at predetermined timing, and
obtains the text data and the control code of each region.
The font expansion unit 233 expands the font on the basis
of the text data and the control code of each region
25 obtained by the subtitle segment decoder 232, and obtains
the bitmap data of each region.
[0127]
The bitmap buffer 234 temporarily holds the bitmap
data of each region obtained by the font expansion unit
30 233. The bitmap data of each region held by the bitmap
buffer 234 is read from the display start timing, and
41
superimposed on the image data, and which lasts for the
display duration.
[0128]
Here, the subtitle segment decoder 232 extracts a
PTS from the PES header of the PES packet. 5 Furthermore,
the subtitle segment decoder 232 extracts the information
on “display offset” and “display duration” of each region
from the TCS segment (see Fig. 12). The information is
transmitted to the CPU 221. The CPU 221 controls, on the
10 basis of the information, the processing timing of the
subtitle segment decoder 232 and the font expansion unit
233, the timing to read the bitmap data of each region
from the bitmap buffer 234, and the like so that the
operation similar to that of the decoder buffer model
15 300A explained with Fig. 14 is performed (see Figs. 15 to
17).
[0129]
Furthermore, the subtitle decoder 210 illustrated
in Fig. 25 has a configuration corresponding to the
20 decoder buffer model 300B in Fig. 18. In other words,
the subtitle decoder 210 includes a coded buffer 241, a
subtitle segment decoder 242, and a font expansion unit
243.
[0130]
25 The coded buffer 241 temporarily holds received
segment data. The subtitle segment decoder 242
continuously performs decoding processing for each frame
to the segment data of each region held by the coded
buffer 241 and obtains the text data and the control code
30 of each region during the display duration of each region.
The font expansion unit 243 expands, for each frame, the
42
font on the basis of the text data and the control code
obtained by the subtitle segment decoder 242 and obtains
the bitmap data of each region during the display
duration of each region. The bitmap data of each region
obtained by the font expansion unit 243 5 is superimposed
on the image data.
[0131]
Here, the subtitle segment decoder 242 extracts a
PTS from the PES header of the PES packet. Furthermore,
10 the subtitle segment decoder 242 extracts the information
on “display offset” and “display duration” of each region
from the TCS segment (see Fig. 12). The information is
transmitted to the CPU 221. The CPU 221 controls, on the
basis of the information, the processing timing of the
15 subtitle segment decoder 232 and the font expansion unit
233 and the like so that the operation similar to that of
the decoder buffer model 300B explained with Fig. 18 is
performed (see Figs. 19 to 21).
[0132]
20 Returning back to Fig. 23, the video superimposing
unit 204 superimposes the bitmap data of each region
obtained by the subtitle decoder 210 on the image data
obtained by the video decoder 203. The panel drive
circuit 205 drives the display panel 206 on the basis of
25 the image data for display obtained by the video
superimposing unit 204. The display panel 206 is
constituted by, for example, a liquid crystal display
(LCD), an organic electroluminescence display, or the
like.
30 [0133]
The operation of the television receiver 200
43
illustrated in Fig. 23 is briefly described. In the
reception unit 201, the transport stream TS transmitted
on a broadcast wave from the broadcast transmission
system 100 is received. The transport stream TS contains
the video stream, the audio stream, 5 and the subtitle
stream.
[0134]
The transport stream TS is supplied to the TS
analysis unit 202. In the TS analysis unit 202, the
10 packets of the video, audio, and subtitle streams are
extracted from the transport stream TS. The video stream
(PES stream) extracted by the TS analysis unit 202 is
supplied to the video decoder 203. In the video decoder
203, decoding processing is performed to the video stream
15 and image data is obtained. The image data is supplied
to the video superimposing unit 204.
[0135]
Furthermore, the subtitle stream (PES stream)
extracted by the TS analysis unit 202 is supplied to the
20 subtitle decoder 210. In the subtitle decoder 210, the
segment data of each region is processed, and the bitmap
data of each region to be superimposed on the image data
is output. In this case, in the subtitle decoder 210,
the output timing of the bitmap data of each region is
25 controlled on the basis of the information on “display
offset” and “display duration” extracted from the PTS and
the TCS segments extracted from the PES header.
[0136]
The bitmap data of each region output from the
30 subtitle decoder 210 is supplied to the video
superimposing unit 204. In the video superimposing unit
44
204, the bitmap data of each region output from the
subtitle decoder 210 is superimposed on the image data
obtained by the video decoder 203. The image data for
display obtained by the video superimposing unit 204 is
supplied to the panel drive circuit 5 205. In the panel
drive circuit 205, the display panel 206 is driven on the
basis of the video data of display. Thus, the image in
which the regions are superimposed is displayed on the
display panel 206.
10 [0137]
Furthermore, the audio stream (PES stream)
extracted by the TS analysis unit 202 is supplied to the
audio decoder 207. In the audio decoder 207, decoding
processing is performed to the audio stream, and audio
15 data is obtained. The audio data is supplied to the
audio output circuit 208. In the audio output circuit
208, necessary processing, such as D/A conversion and
amplification, is performed to the audio data. Then, the
processed audio data is supplied to the speaker 209.
20 Thus, the audio corresponding to the display image on the
display panel 206 is output from the speaker 209.
[0138]
As described above, in the transceiver system 10
illustrated in Fig. 1, the subtitle stream having the
25 text information and the display timing information on
the subtitles is transmitted together with the video
stream. Thus, it is possible to excellently perform the
display timing control of the subtitles on the reception
side.
30 [0139]
Furthermore, in the transceiver system 10
45
illustrated in Fig. 1, the subtitle stream contains the
segment having the TTML information having the text
information and the display timing information on the
subtitles, and the TCS segment having the display timing
information (information on display 5 start timing and
display duration) corresponding to the display timing
information of the TTML. Thus, it is possible to easily
obtain the display timing information from the TCS
segment, and to reduce the processing load on the
10 reception side.
[0140]
Furthermore, in the transceiver system 10
illustrated in Fig. 1, when the TTML is converted into
the segments on the transmission side, “PTS”, “display
15 offset”, and “display duration” of each subtitle are set
referring to the system time information (the PCR and the
synchronization time of the video and the audio) on the
basis of the description on the display start timing and
the display end timing of each subtitle contained in the
20 body of the TTML. At this time, “PTS”, “display offset”,
and “display duration” are set while it is verified that
the operation of the reception side is correctly
performed using the decoder buffer model (see Figs. 14
and 18), and the display timing control of the subtitles
25 is excellently performed on the reception side.
[0141]
<2. Modified example>
Note that, it has been exemplified that the TTML is
used as the text information on the subtitles in a
30 predetermined format having the display timing
information in the above embodiment. However, the
46
present technology is not limited to this, and other
timed text information having information equivalent to
the TTML may be used. For example, a derivative format
of the TTML may be used.
5 [0142]
Furthermore, the transceiver system 10 constituted
by the broadcast transmission system 100 and the
television receiver 200 has been described in the above
embodiment, but the transceiver system to which the
10 present technology can be applied is not limited to this.
For example, the part of the television receiver 200 may
be a set top box and a monitor connected by a digital
interface, such as High-Definition Multimedia Interface
(HDMI). Note that, “HDMI” is a registered trademark.
15 [0143]
Furthermore, the present technology can be
following configurations:
(1) A transmission apparatus including:
a video encoder which generates a video stream
20 having coded image data;
a subtitle encoder which generates a subtitle
stream having text information and display timing
information on subtitles; and
a transmission unit which transmits a transport
25 stream containing the video stream and the subtitle
stream.
(2) The transmission apparatus according (1), in
which the subtitle encoder generates the subtitle stream
on the basis of the text information on the subtitles in
30 a predetermined format having the display timing
information.
47
(3) The transmission apparatus according to (2), in
which the text information on the subtitles in the
predetermined format is in a TTML or in a derivative
format of the TTML.
(4) The transmission apparatus according 5 to any one
of (1) to (3), in which the display timing information on
the subtitles has information on, at least, display start
timing and display duration.
(5) The transmission apparatus according to (4), in
10 which
the subtitle stream is constituted by a PES packet
constituted by a PES header and a PES payload,
the text information and the display timing
information on the subtitles are arranged in the PES
15 payload, and
the display start timing is indicated by a display
offset from a PTS inserted in the PES header.
(6) A transmission method including:
a video encoding step of generating a video stream
20 having coded image data;
a subtitle encoding step of generating a subtitle
stream having text information and display timing
information on subtitles; and
a transmission step of transmitting, by a
25 transmission unit, a transport stream containing the
video stream and the subtitle stream.
(7) A reception apparatus including:
a reception unit which receives a transport stream
containing a video stream having coded image data and a
30 subtitle stream having text information and display
timing information on subtitles;
48
a video decoder which performs decoding processing
to the video stream and obtains image data;
a subtitle decoder which performs decoding
processing to the subtitle stream and obtains bitmap data
of the subtitles and the display timing 5 information on
the subtitles; and
a display control unit which controls timing to
superimpose the bitmap data of the subtitles on the image
data on the basis of the display timing information.
10 (8) The reception apparatus according to (7), in
which the display timing information on the subtitles has
information on, at least, display start timing and
display duration.
(9) The reception apparatus according to (8), in
15 which
the subtitle stream is constituted by a PES packet
constituted by a PES header and a PES payload,
the text information and the display timing
information on the subtitles are arranged in the PES
20 payload, and
the display start timing is indicated by a display
offset from a PTS inserted in the PES header.
(10) A reception method including:
a reception step of receiving, by a reception unit,
25 a transport stream containing a video stream having coded
image data and a subtitle stream having text information
and display timing information on subtitles;
a video decoding step of performing decoding
processing to the video stream and obtaining image data;
30 a subtitle decoding step of performing decoding
processing to the subtitle stream and obtaining bitmap
49
data of the subtitles and the display timing information
on the subtitles; and
a display control step of controlling timing to
superimpose the bitmap data of the subtitles on the image
data on the basis of the display timing 5 information.
[0144]
A main feature of the present technology is to
excellently perform display timing control of subtitles
on a reception side by transmitting a subtitle stream
10 having text information on the subtitles and display
timing information together with a video stream (see Fig.
22).
REFERENCE SIGNS LIST
15 [0145]
10 Transceiver system
100 Broadcast transmission system
110 Stream generation unit
111 Control unit
20 112 Video encoder
113 Audio encoder
114 Text format conversion unit
115 Subtitle encoder
116 TS formatter
25 200 Television receiver
201 Reception unit
202 TS analysis unit
203 Video decoder
204 Video superimposing unit
30 205 Panel drive circuit
206 Display panel
50
207 Audio decoder
208 Audio output circuit
209 Speaker
210 Subtitle decoder
5 221 CPU
231, 241 Coded buffer
232, 242 Subtitle segment decoder
233, 243 Font expansion unit
234 Bitmap buffer
10 300A, 300B Decoder buffer model
301, 311 Coded buffer
302, 312 Subtitle segment decoder
303, 313 Font expansion unit
304 Bitmap buffer
15
51
CLAIMS
1. A transmission apparatus comprising:
a video encoder configured to generate a video
stream having 5 coded image data;
a subtitle encoder configured to generate a
subtitle stream having text information and display
timing information on subtitles; and
a transmission unit configured to transmit a
10 transport stream containing the video stream and the
subtitle stream.
2. The transmission apparatus according to claim 1,
wherein the subtitle encoder generates the subtitle
15 stream on the basis of the text information on the
subtitles in a predetermined format having the display
timing information.
3. The transmission apparatus according to claim 2,
20 wherein the text information on the subtitles in the
predetermined format is in a TTML or in a derivative
format of the TTML.
4. The transmission apparatus according to claim 1,
25 wherein the display timing information on the subtitles
has information on, at least, display start timing and
display duration.
5. The transmission apparatus according to claim 4,
30 wherein
the subtitle stream is constituted by a PES packet
52
constituted by a PES header and a PES payload,
the text information and the display timing
information on the subtitles are arranged in the PES
payload, and
the display start timing is indicated 5 by a display
offset from a PTS inserted in the PES header.
6. A transmission method comprising:
a video encoding step of generating a video stream
10 having coded image data;
a subtitle encoding step of generating a subtitle
stream having text information and display timing
information on subtitles; and
a transmission step of transmitting, by a
15 transmission unit, a transport stream containing the
video stream and the subtitle stream.
7. A reception apparatus comprising:
a reception unit configured to receive a transport
20 stream containing a video stream having coded image data
and a subtitle stream having text information and display
timing information on subtitles;
a video decoder configured to perform decoding
processing to the video stream and to obtain image data;
25 a subtitle decoder configured to perform decoding
processing to the subtitle stream and to obtain bitmap
data of the subtitles and the display timing information
on the subtitles; and
a display control unit configured to control timing
30 to superimpose the bitmap data of the subtitles on the
image data on the basis of the display timing information.
53
8. The reception apparatus according to claim 7,
wherein the display timing information on the subtitles
has information on, at least, display start timing and
5 display duration.
9. The reception apparatus according to claim 8,
wherein
the subtitle stream is constituted by a PES packet
10 constituted by a PES header and a PES payload,
the text information and the display timing
information on the subtitles are arranged in the PES
payload, and
the display start timing is indicated by a display
15 offset from a PTS inserted in the PES header.
10. A reception method comprising:
a reception step of receiving, by a reception unit,
a transport stream containing a video stream having coded
20 image data and a subtitle stream having text information
and display timing information on subtitles;
a video decoding step of performing decoding
processing to the video stream and obtaining image data;
a subtitle decoding step of performing decoding
25 processing to the subtitle stream and obtaining bitmap
data of the subtitles and the display timing information
on the subtitles; and
a display control step of controlling timing to
superimpose the bitmap data of the subtitles on the image
30 data on the basis of the display timing information.