DESCRIPTION
TRANSMISSION APPARATUS, TRANSMISSION METHOD, RECEPTION
APPARATUS, AND RECEPTION METHOD
5 Technical Field
[0001] The present technology relates to a
transmission apparatus, a transmission method, a
reception apparatus, and a reception method. More
specifically, the present technology relates to a
10 transmission apparatus and the like that hierarchically
encode and transmit image data of each picture
constituting moving image data.
Background Art
[0002] When compressed moving images are provided
15 over broadcasting, network services, and the like, the
upper limit of a frame frequency that can be played
back is restricted by a decoding capability of a
receiver. Therefore, a service side is required to take
the playback performance of prevalent receivers into
20 account, and restrict the service to a low frame
frequency only or simultaneously provide multiple high
and low frame frequency services.
[0003] A support for high frame frequency services
increases the cost of the receiver, which becomes a
25 barrier to rapid diffusion of the service. If only lowcost
receivers dedicated to low frame frequency
3
services are widespread in early times, and the service
side starts a high frame frequency service in the
future, the new service is completely unwatchable
without a new receiver, which becomes a barrier to
diffusion 5 of the service.
[0004] For example, a time direction scalability is
proposed, in which image data of each picture
constituting moving image data is subjected to
hierarchical encoding in HEVC (High Efficiency Video
10 Coding) (see Non-patent Document 1). A reception side
can identify a hierarchy of each picture on the basis
of a temporal ID (temporal_id) inserted into the header
of a NAL (Network Abstraction Layer) unit and perform
selective decoding to a hierarchy corresponding to a
15 decoding capability.
[0005] [Non-patent Document 1] Gary J. Sullivan,
Jens-Rainer Ohm, Woo-Jin Han, Thomas Wiegand, "Overview
of the High Efficiency Video Coding (HEVC) Standard"
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO
20 TECNOROGY, VOL. 22, NO. 12, pp. 1649-1668, DECEMBER
2012
Summary of Invention
Problem to be solved by the Invention
[0006] It is an object of the present technology to
25 enable satisfactory decoding processing corresponding
to a decoding capability on a reception side.
4
Means for solving the Problem
[0007] A concept of the present technology resides
in a transmission apparatus including: an image
encoding unit that sorts image data of pictures
constituting moving image 5 data into multiple
hierarchies, encodes image data of pictures of each of
the sorted hierarchies, and generates video data
including the encoded image data of the pictures of
each of the hierarchies; a transmission unit that
10 transmits a container of a predetermined format
including the generated video data; and an
identification information insertion unit that divides
the multiple hierarchies into a predetermined number of
hierarchy groups, the predetermined number being two or
15 more, and inserts identification information for
identifying a hierarchy group to which encoded image
data of each picture included in the video data belongs,
into a packet as a container of the video data.
[0008] In the present technology, by the image
20 encoding unit, image data of pictures constituting
moving image data are encoded and video data is
generated. In this case, the image data of pictures
constituting moving image data are sorted into the
multiple hierarchies and encoded, and the video data
25 including the encoded image data of the pictures of
each of the hierarchies is generated.
5
[0009] By the transmission unit, a container of a
predetermined format including the video data described
above is transmitted. For example, the container may be
a transport stream (MPEG-2 TS) adopted in the standard
of digital broadcasting. Further, 5 for example, the
container may be an MP4 used in distribution over the
Internet and the like, or a container of any other
format.
[0010] By the identification information insertion
10 unit, the multiple hierarchies are divided into a
predetermined number of hierarchy groups, the
predetermined number being two or more, and
identification information for identifying a hierarchy
group to which encoded image data of each picture
15 included in the video data belongs is inserted into a
packet as a container of the video data. For example,
the identification information may be degree-ofpriority
information that is set to be higher for a
hierarchy group on a low hierarchy side.
20 [0011] For example, the identification information
may be inserted into a header of a PES packet including
the encoded image data for each picture in a payload.
In this case, for example, the identification
information may be inserted using a field of a PES
25 priority of the header. Further, for example, the
identification information may be inserted into an
6
adaptation field of a TS packet including the
adaptation field. In this case, for example, the
identification information may be inserted using a
field of an ES priority indicator of the adaptation
field. Further, for example, 5 the identification
information may be inserted into a box of a header
related to a track of an appropriate picture.
[0012] As described above, in the present technology,
the identification information for identifying a
10 hierarchy group to which encoded image data of each
picture included in the video data belongs is inserted
into a packet as a container of the video data.
Therefore, on the reception side, it is easily possible
to selectively decode encoded image data of a picture
15 of a hierarchy equal to or lower than a predetermined
hierarchy corresponding to a decoding capability by
using the identification information.
[0013] It should be noted that in the present
technology, for example, the image encoding unit may
20 generate a single video stream including the encoded
image data of the pictures of each of the hierarchies
or may divide the multiple hierarchies into a
predetermined number of hierarchy groups, the
predetermined number being two or more, and generate a
25 predetermined number of video streams each including
the encoded image data of the pictures of each
7
hierarchy group, and may further include a
configuration information insertion unit that inserts
configuration information of the video streams included
in the container into a layer of the container. In this
case, for example, on the reception 5 side, it is
possible to easily grasp the configuration of the video
streams on the basis of the configuration information
of the video streams included in the container.
[0014] Further, another concept of the present
10 technology resides in a reception apparatus including:
an reception unit that receives a container of a
predetermined format including video data including
encoded image data of pictures of each of hierarchies,
the encoded image data being obtained by sorting image
15 data of pictures constituting moving image data into
multiple hierarchies and encoding the image data; and
an image decoding unit that selectively takes in a
buffer encoded image data of a picture of a hierarchy
equal to or lower than a predetermined hierarchy
20 corresponding to a decoding capability from the video
data included in the received container, decodes the
encoded image data of each picture taken in the buffer,
and obtains image data of the picture of the hierarchy
equal to or lower than the predetermined hierarchy.
25 [0015] In the present technology, by the reception
unit, a container of a predetermined format is received.
8
In the container, video data including image data of
pictures of each hierarchy, which is obtained by
sorting image data of pictures constituting moving
image data into multiple hierarchies and encoding the
image 5 data, is included.
[0016] By the image decoding unit, the encoded image
data of a picture of a hierarchy equal to or lower than
a predetermined hierarchy corresponding to a decoding
capability is selectively taken in the buffer from the
10 video data included in the received container, the
encoded image data of each picture taken in the buffer
is decoded, and the image data of the picture of the
hierarchy equal to or lower than the predetermined
hierarchy is obtained.
15 [0017] For example, the multiple hierarchies may be
divided into a predetermined number of hierarchy groups,
the predetermined number being two or more, and
identification information for identifying a hierarchy
group to which the encoded image data of each picture
20 included in the video data belongs may be inserted into
a packet as a container of the video data, and the
image decoding unit may take in the buffer the encoded
image data of the picture of a predetermined hierarchy
group corresponding to the decoding capability and
25 decode the encoded image data on the basis of the
identification information.
9
[0018] In this case, for example, the identification
information may be inserted into a header of a PES
packet including the encoded image data for each
picture in a payload. Further, in this case, for
example, the identification information 5 may be inserted
into an adaptation field of a TS packet including the
adaptation field. Further, in this case, for example,
the identification information may be inserted into a
box of a header related to a track of an appropriate
10 picture.
[0019] Further, for example, the multiple
hierarchies may be divided into a predetermined number
of hierarchy groups, the predetermined number being two
or more, and the received container may include the
15 predetermined number of video streams respectively
including encoded image data of pictures of the
predetermined number of hierarchy groups, and the image
decoding unit may take in the buffer the encoded image
data of the picture of the predetermined hierarchy
20 group corresponding to the decoding capability and
decode the encoded image data on the basis of stream
identification information. In this case, for example,
the image decoding unit may convert, when the encoded
image data of the picture of the predetermined
25 hierarchy group is included in multiple video streams,
the encoded image data of each picture into one scream
10
on the basis of decoding timing information and takes
the stream in the buffer.
[0020] As described above, in the present technology,
the encoded image data of a picture of a hierarchy
equal to or lower than a predetermined 5 hierarchy
corresponding to a decoding capability is selectively
taken in the buffer from the received video data and
then decoded. Therefore, it is possible to perform
appropriate decoding processing corresponding to the
10 decoding capability.
[0021] It should be noted that in the present
technology, for example, the image decoding unit may
have a function of rewriting a decoding time stamp of
the encoded image data of each picture selectively
15 taken in the buffer and adjusting decoding intervals of
low hierarchy pictures. In this case, realistic
decoding processing can be performed even in a decoder
having a low decoding capability.
[0022] Further, in the present technology, for
20 example, the reception apparatus may further include a
post-processing unit that matches a frame rate of the
image data of each picture obtained in the image
decoding unit with the display capability. In this case,
the image data of the frame rate matched with a high
25 display capability can be obtained even when the
decoding capability is low.
11
Effect of the Invention
[0023] According to the present technology, it is
possible to perform satisfactory decoding processing
corresponding to a decoding capability on a reception
side. It should be noted that the 5 effects described
herein are not necessarily limited but may be any
effect described herein.
Brief Description of Drawings
[0024] [Fig. 1] Fig. 1 is a block diagram showing a
10 configuration example of a transmission and reception
system as an embodiment.
[Fig. 2] Fig. 2 is a block diagram showing a
configuration example of a transmission apparatus.
[Fig. 3] Fig. 3 is a diagram showing an example of
15 hierarchical encoding performed in an encoder.
[Fig. 4] Fig. 4 is a diagram showing a structural
example (Syntax) of a NAL unit header and details
(Semantics) of main parameters in the structural
example.
20 [Fig. 5] Fig. 5 is a diagram for describing a
configuration of encoded image data in each picture by
HEVC.
[Fig. 6] Fig. 6 is a diagram showing an example of
encoding, decoding, display order, and delay in
25 hierarchical encoding.
[Fig. 7] Fig. 7 is a diagram showing an encoded stream
12
of hierarchical encoding and a display expectation
(display order) in a specified hierarchy.
[Fig. 8] Fig. 8 is a diagram showing a structural
example (Syntax) of an HEVC descriptor
5 (HEVC_descriptor).
[Fig. 9] Fig. 9 is a diagram showing details
(Semantics) of main information in the structural
example of the HEVC descriptor.
[Fig. 10] Fig. 10 is a diagram showing a structural
10 example (Syntax) of a scalability extension descriptor
(scalability_extension_descriptor).
[Fig. 11] Fig. 11 is a diagram showing details
(Semantics) of main information in the structural
example of the scalability extension descriptor.
15 [Fig. 12] Fig. 12 is a block diagram showing a
configuration example of a multiplexer.
[Fig. 13] Fig. 13 is a diagram showing an example of a
processing flow of the multiplexer.
[Fig. 14] Fig. 14 is a diagram showing a configuration
20 example of a transport stream TS when distribution by a
single stream is performed.
[Fig. 15] Fig. 15 is a block diagram showing a
configuration example of a reception apparatus.
[Fig. 16] Fig. 16 is a block diagram showing a
25 configuration example of a demultiplexer.
[Fig. 17] Fig. 17 is a diagram showing a case where a
13
single video stream (encoded stream) is included in a
transport stream TS.
[Fig. 18] Fig. 18 is a diagram showing a case where two
video streams (encoded stream) of a base stream and an
extended stream are included in a transport 5 stream TS.
[Fig. 19] Fig. 19 is a diagram for describing a
function of rewriting a decoding time stamp of encoded
image data of each picture and adjusting decoding
intervals of low hierarchy pictures.
10 [Fig. 20] Fig. 20 is a diagram showing an example of a
processing flow (1 frame) of the demultiplexer.
[Fig. 21] Fig. 21 is a diagram showing an example of a
processing flow (2 frames) of the demultiplexer.
[Fig. 22] Fig. 22 is a block diagram showing a
15 configuration example of a decoder.
[Fig. 23] Fig. 23 is a diagram showing a configuration
example of a post-processing unit.
[Fig. 24] Fig. 24 is a diagram showing an example of a
processing flow of the decoder and the post-processing
20 unit.
[Fig. 25] Fig. 25 is a diagram showing an arrangement
example of an adaptation field.
[Fig. 26] Fig. 26 is a block diagram showing a
configuration example of a multiplexer when
25 identification information of a hierarchy group is
inserted into an adaptation field.
14
[Fig. 27] Fig. 27 is a diagram showing a configuration
example of a transport stream TS when identification
information of a hierarchy group is inserted into an
adaptation field.
[Fig. 28] Fig. 28 is a block 5 diagram showing a
configuration example of a demultiplexer when
identification information of a hierarchy group is
inserted into an adaptation field.
[Fig. 29] Fig. 29 is a diagram showing a configuration
10 example of an MP4 stream.
[Fig. 30] Fig. 30 is a diagram showing a structural
example of "SampleDependencyTypeBox".
[Fig. 31] Fig. 31 is a diagram showing details of main
information in the structural example of
15 "SampleDependencyTypeBox".
[Fig. 32] Fig. 32 is a diagram showing a structural
example of "SampleScalablePriorityBox".
[Fig. 33] Fig. 33 is a diagram showing detains of main
information in the structural example of
20 "SampleScalablePriorityBox".
Mode(s) for Carrying Out the Invention
[0025] Hereinafter, a mode for carrying out the
invention (hereinafter, called "embodiment") will be
described. It should be noted that the description is
25 provided in the following order.
1. Embodiment
15
2. Modified Example
[0026] <1. Embodiment>
[Transmission and Reception System]
Fig. 1 shows a configuration example of a
transmission and reception system 10 5 as an embodiment.
The transmission and reception system 10 has a
configuration including a transmission apparatus 100
and a reception apparatus 200.
[0027] The transmission apparatus 100 transmits a
10 transport stream TS as a container on a broadcast wave.
In the transport stream TS, image data of pictures
constituting moving image data are sorted into multiple
hierarchies, and a video stream including encoded data
of image data of pictures of each of the hierarchies is
15 included. In this case, for example, encoding such as
H.264/AVC and HEVC is performed, and a referenced
picture is encoded so as to belong to a hierarchy of
the referenced picture and/or a hierarchy lower than
the hierarchy of the referenced picture.
20 [0028] Hierarchy identification information for
identifying a hierarchy to which encoded image data
belongs is added to the encoded image data of each
picture of each hierarchy. In this embodiment, in a
header portion of a NAL unit (nal_unit) of each picture,
25 hierarchy identification information
("nuh_temporal_id_plus1" representing temporal_id) is
16
arranged. By addition of the hierarchy identification
information in such a manner, on the reception side, it
is possible to identify a hierarchy of each picture in
a layer of the NAL unit and to perform decoding
processing by selectively taking out 5 encoded image data
of a hierarchy equal to or lower than a predetermined
hierarchy.
[0029] In this embodiment, the multiple hierarchies
are divided into a predetermined number of, i.e., two
10 or more hierarchy groups, and identification
information for identifying a hierarchy group to which
encoded image data of each picture included in the
video stream belongs is inserted into a layer of the
video stream.
15 [0030] In this embodiment, the identification
information is considered as degree-of-priority
information that is set to be higher for a hierarchy
group on the low hierarchy side, and is inserted into a
header of a PES packet including encoded image data of
20 each picture in a payload. The identification
information enables the reception side to take in a
buffer and process only encoded image data of a picture
of a hierarchy group corresponding to a decoding
capability of the reception side.
25 [0031] The transport stream TS includes a single
video stream including encoded image data of a picture
17
of each hierarchy, or a predetermined number of video
streams each including encoded image data of a picture
of each hierarchy group described above. In the
transport stream TS, hierarchy information of
hierarchical encoding and configuration 5 information of
a video stream are inserted. Those pieces of
information enable the reception side to easily grasp a
hierarchy configuration or a stream configuration and
preform appropriate decoding processing.
10 [0032] The reception apparatus 200 receives the
above-mentioned transport stream TS transmitted from
the transmission apparatus 100 on a broadcast wave. The
reception apparatus 200 selectively takes in a buffer
encoded image data of a picture of a hierarchy lower
15 than a predetermined hierarchy selected according to a
decoding capability from a video stream included in the
transport stream TS, performs decoding, acquires image
data of each picture, and perform image reproduction.
[0033] For example, as described above, the
20 transport stream TS may include a single video stream
including encoded image data of pictures of the
multiple hierarchies. In such a case, on the basis of
the above-mentioned identification information, the
encoded image data of a picture of a predetermined
25 hierarchy group corresponding to a decoding capability
is taken in the buffer and processed.
18
[0034] Further, for example, as described above, the
transport stream TS may include a predetermined number
of video streams including encoded image data of
pictures of a predetermined number of, i.e., two or
more hierarchy groups, which are obtained 5 by dividing
the multiple hierarchies. In such a case, the encoded
image data of a picture of a predetermined hierarchy
group corresponding to a decoding capability is taken
in the buffer and processed on the basis of stream
10 identification information.
[0035] Further, the reception apparatus 200 performs
processing of rewriting a decoding time stamp of
encoded image data of each picture selectively taken in
the buffer and adjusting decoding intervals of low
15 hierarchy pictures. This adjusting processing enables
realistic decoding processing even in a decoder having
a low decoding capability.
[0036] Further, the reception apparatus 200 performs
post-processing of matching a frame rate of image data
20 of each picture, which is obtained by decoding as
described above, with a display capability. The postprocessing
enables obtaining image data of a frame rate
matched with a high display capability even when the
decoding capability is low, for example.
25 [0037] "Configuration of Transmission Apparatus"
Fig. 2 shows a configuration example of the
19
transmission apparatus 100. The transmission apparatus
100 includes a CPU (Central Processing Unit) 101, an
encoder 102, a coded picture buffer (cpb) 103, a
multiplexer 104, and a transmission unit 105. The CPU
101 is a control unit and controls 5 operations of
respective units of the transmission apparatus 100.
[0038] The encoder 102 inputs uncompressed moving
image data and performs hierarchical encoding. The
encoder 102 sorts image data of pictures constituting
10 the moving image data into multiple hierarchies. The
encoder 102 then encodes the image data of pictures of
each sorted hierarchy and generates a video stream
having encoded image data of pictures of each hierarchy.
For example, the encoder 102 performs encoding such as
15 H.264/AVC and HEVC. At that time, the encoder 102
performs encoding such that a picture to be referenced
(referenced picture) belongs to a hierarchy of the
referenced picture and/or a hierarchy lower than the
hierarchy of the referenced picture.
20 [0039] Fig. 3 shows an example of hierarchical
encoding performed in the encoder 102. In this example,
five hierarchies from 0 to 4 are provided by sorting,
and image data of pictures of each hierarchy is
subjected to encoding, for example, HEVC.
25 [0040] The vertical axis represents a hierarchy. 0
to 4 are set as temporal_id (hierarchy identification
20
information) arranged in a header portion of a NAL unit
(nal_unit) constituting the encoded image data of
pictures of the hierarchies of 0 to 4, respectively. On
the other hand, the horizontal axis represents a
display order (POC: picture order of 5 composition), in
which the left side represents an earlier display time
and the right side represents a later display time.
[0041] Part (a) of Fig. 4 shows a structural example
(Syntax) of a NAL unit header, and part (b) of Fig. 4
10 shows details (Semantics) of main parameters in the
structural example. 0 is indispensable for a 1-bit
field of "Forbidden_zero_bit". A 6-bit field of
"Nal_unit_type" represents a NAL unit type. A 6-bit
field of "Nuh_layer_id" assumes 0. A 3-bit field of
15 "Nuh_temporal_id_plus1" represents a temporal_id and
takes values (1 to 7) to which 1 is added.
[0042] Referring back to Fig. 3, each rectangular
frame represents a picture, and a number represents an
order of encoded picture, that is, encoded order
20 (decoded order on the reception side). 16 pictures from
"1" to "17" (excluding "2") constitute a sub group of
pictures, and "1" is a head picture of the sub group of
pictures. "2" is a head picture of the next sub group
of pictures. Alternatively, 16 pictures from "2" to
25 "17" except for "1" constitute a sub group of pictures,
and "2" is a head picture of the sub group of pictures.
21
[0043] The picture "1" may be a head picture of a
GOP (Group Of Pictures). The encoded image data of the
head picture of the GOP is constituted of NAL units of
AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS as shown
in Fig. 5. On the other hand, pictures 5 other than the
head picture of the GOP are each constituted of NAL
units of AUD, PPS, PSEI, SLICE, SSEI, and EOS. The VPS
is transmittable to a sequence (GOP) one time together
with the SPS, and the PPS is transmittable in My
10 Picture.
[0044] Referring back to Fig. 3, solid arrows
represent a reference relationship of pictures in
encoding. For example, a picture "1" is an I picture
that does not reference other pictures. A picture "2"
15 is a P picture that references the picture "1" and is
encoded. Further, a picture "3" is a B picture that
references the pictures "1" and "3" and is encoded.
Hereinafter, the other pictures similarly reference a
near picture in the display order and are encoded. It
20 should be noted that the pictures of the hierarchy 4
are not referenced by other pictures.
[0045] The encoder 102 generates a single video
stream (single stream) including encoded image data of
pictures of each hierarchy, or divides the multiple
25 hierarchies into a predetermined number of, i.e., two
or more hierarchy groups and generates a predetermined
22
number of video streams (multi-stream) each including
encoded image data of pictures of each hierarchy group.
For example, in the example of the hierarchical
encoding of Fig. 3, when the hierarchies are divided
into two hierarchy groups such that 5 the hierarchies 0
to 3 are set to a hierarchy group of a lower hierarchy
and the hierarchy 4 is set to a hierarchy group of a
higher hierarchy, the encoder 102 generates two video
streams (encoded streams) each including encoded image
10 data of pictures of each hierarchy group.
[0046] Irrespective of the number of generated video
streams, as described above, the encoder 102 divides
the multiple hierarchies into a predetermined number of,
i.e., two or more hierarchy groups and adds
15 identification information for identifying a hierarchy
group to which encoded image data belongs to the
encoded image data of pictures of each hierarchy group.
In this case, for example, for the identification
information, "general_level_idc" as a level
20 specification value of a bit stream included in the SPS
is used, and a hierarchy group on the high hierarchy
side takes a higher value. It should be noted that
"sub_layer_level_idc" can be transmitted in the SPS on
a sublayer-by-sublayer basis, and thus
25 "sub_layer_level_idc" may be used as the identification
information. Those above values are supplied not only
23
in the SPS but also in the VPS.
[0047] In this case, a value of the level
specification value of each hierarchy group is set to a
value corresponding to a frame rate constituted of
pictures of this hierarchy group and 5 pictures of all
the hierarchy groups on the hierarchy side lower than
this hierarchy group. For example, in the example of
the hierarchical encoding of Fig. 3, the level
specification value of the hierarchy group including
10 the hierarchies from 0 to 3 is a value corresponding to
a frame rate constituted of only the pictures of the
hierarchies from 0 to 3, and the level specification
value of the hierarchy group including the hierarchy 4
is a value corresponding to a frame rate constituted of
15 the pictures of all the hierarchies from 0 to 4.
[0048] Fig. 6 shows an example of encoding, decoding,
display order, and delay in the hierarchical encoding.
This example corresponds to the example of the abovementioned
hierarchical encoding in Fig. 3. This example
20 shows a case where all hierarchies (all layers) are
subjected to hierarchical encoding at a full temporal
resolution. Part (a) of Fig. 6 shows encoder inputs.
As shown in part (b) of Fig. 6, the pictures are
encoded in the encoding order with delay corresponding
25 to 16 pictures, and an encoded stream is obtained.
Further, part (b) of Fig. 6 shows decoder inputs, and
24
the pictures are decoded in the decoding order. As
shown in part (c) of Fig. 6, image data of the pictures
are obtained in the display order with delay
corresponding to 4 pictures.
[0049] Part (a) of Fig. 7 shows 5 an encoded stream,
which is similar to the above-mentioned encoded stream
shown in part (b) of Fig. 6, in three stages of the
hierarchies 0 to 2, the hierarchy 3, and the hierarchy
4. Here, "Tid" represents temporal_id. Part (b) of Fig.
10 7 shows a display expectation (display order) in the
case where pictures of the hierarchies 0 to 2, i.e.,
partial hierarchies when Tid = 0 to 2 are selectively
decoded. Further, part (c) of Fig. 7 shows a display
expectation (display order) in the case where pictures
15 of the hierarchies 0 to 3, i.e., partial hierarchies
when Tid = 0 to 3 are selectively decoded. Furthermore,
part (d) of Fig. 7 shows a display expectation (display
order) in the case where pictures of the hierarchies 0
to 4, i.e., all the hierarchies when Tid = 0 to 4 are
20 selectively decoded.
[0050] To perform decoding processing on the encoded
stream shown in part (a) of Fig. 7 according to a
decoding capability, a decoding capability having
temporal resolution at a full rate is required. However,
25 when decoding where Tid =0 to 2 is performed, a decoder
having a 1/4 decoding capability for the encoded full
25
temporal resolution should perform processing. Further,
when decoding where Tid = 0 to 3 is performed, a
decoder having a 1/2 decoding capability for the
encoded full temporal resolution should perform
5 processing.
[0051] However, when pictures belonging to a low
hierarchy and referenced in the hierarchical encoding
are successive and encoded at a timing of a full
temporal resolution, the capability of a decoder that
10 performs partial decoding does not catch up with the
encoding. This situation corresponds to a period A in
part (a) of Fig. 7. A decoder that decodes partial
hierarchies where Tid = 0 to 2 or Tid = 0 to 3 performs
decoding/display with a capability having 1/4 or 1/2 of
15 the time axis, as shown in the example of display. Thus,
pictures successive at the encoded full temporal
resolution in the period A cannot be decoded.
[0052] Ta represents a time required for decoding
processing on a picture-by-picture basis in the decoder
20 that decodes Tid = 0 to 2. Tb represents a time
required for decoding processing on a picture-bypicture
basis in the decoder that decodes Tid = 0 to 3.
Tc represents a time required for decoding processing
on a picture-by-picture basis in the decoder that
25 decodes Tid = 0 to 4 (all hierarchies). A relationship
of those times is Ta > Tb > Tc.
26
[0053] In this embodiment, as will be described
later, when the reception apparatus 200 includes a
decoder having a low decoding capability and
selectively decodes low hierarchy pictures, the
reception apparatus 200 is provided 5 with a function of
rewriting a decoding time stamp (DTS) and adjusting
decoding intervals of the low hierarchy pictures. This
enables realistic decoding processing even in a decoder
having a low decoding capability.
10 [0054] Referring back to Fig. 2, the coded picture
buffer (cpb) 103 temporarily stores the video stream
generated in the encoder 102, the video stream
including encoded data of pictures of each hierarchy.
The multiplexer 104 reads the video stream stored in
15 the coded picture buffer 103 and converts the video
stream into PES packets and further into transport
packets to be multiplexed, thus obtaining a transport
stream TS as a multiplexed stream.
[0055] In this embodiment, as described above, the
20 multiple hierarchies are divided into a predetermined
number of, i.e., two or more hierarchy groups. The
multiplexer 104 inserts the identification information
into the header of a PES packet (PES header). The
identification information is for identifying a
25 hierarchy group to which encoded image data of each
picture included in the video stream belongs. This
27
identification information enables the reception side
to take in a buffer and process only encoded image data
of pictures of a hierarchy group corresponding to a
decoding capability of the reception side.
[0056] For example, when the multiple 5 hierarchies
are divided into a low hierarchy group and a high
hierarchy group, the multiplexer 104 uses a 1-bit field
of a known PES priority (PES_priority), located in the
PES header. When the PES payload includes encoded image
10 data of a picture of a hierarchy group on the low
hierarchy side, the 1-bit field is set to "1", that is,
set to have a high degree of priority. On the other
hand, when the PES payload includes encoded image data
of a picture of a hierarchy group on the high hierarchy
15 side, the 1-bit field is set to "0", that is, set to
have a low degree of priority.
[0057] As described above, the transport stream TS
includes a single video stream including encoded image
data of a picture of each hierarchy, or a predetermined
20 number of video streams each including encoded image
data of a picture of each hierarchy group described
above. The multiplexer 104 inserts hierarchy
information and stream configuration information into
the transport stream TS.
25 [0058] The transport stream TS includes a PMT
(Program Map Table) as one kind of PSI (Program
28
Specific Information). In the PMT, a video elementary
loop (video ES1 loop) including information related to
each video stream exists. In the video elementary loop,
information such as a stream type and a packet
identifier (PID), and a descriptor 5 that describes
information related to that video stream are arranged
to correspond to each video stream.
[0059] The multiplexer 104 inserts an HEVC
descriptor (HEVC_descriptor) as one of the descriptor
10 and further inserts a scalability extension descriptor
(scalability_extension_descriptor) that is newly
defined.
[0060] Fig. 8 shows a structural example (Syntax) of
the HEVC descriptor (HEVC_descriptor). Further, Fig. 9
15 shows details (Semantics) of main information in the
structural example.
[0061] An 8-bit field of "descriptor_tag" shows a
descriptor type and shows an HEVC descriptor here. An
8-bit field of "descriptor_length" shows the length
20 (size) of the descriptor. As the length of the
descriptor, a subsequent number of bytes is shown.
[0062] An 8-bit field of "level_idc" shows a level
specification value of a bit rate. Further, when
"temporal_layer_subset_flag = 1", a 5-bit field of
25 "temporal_id_min" and a 5-bit field of
"temporal_id_max" are present. "temporal_id_min" shows
29
a value of temporal_id on the lowest hierarchy of
hierarchically encoded data included in a corresponding
video stream. "temporal_id_max" shows a value of
temporal_id on the highest hierarchy of hierarchically
encoded data included in a corresponding 5 video stream.
[0063] A 1-bit field of "level_constrained_flag" is
newly defined and represents that a level specification
value (general_level_idc) of a bit stream included in a
NAL unit of the VPS may change on a picture-by-picture
10 basis. "1" represents a possibility of change, and "0"
represents no change.
[0064] As described above, for example,
"general_level_idc" is used as identification
information of a hierarchy group to which encoded image
15 data belongs, when the multiple hierarchies are divided
into a predetermined number of, i.e., two or more
hierarchy groups. Therefore, in the case of a video
stream including encoded image data of pictures of
multiple hierarchy groups, "general_level_idc" may
20 change on a picture-by-picture basis. On the other hand,
in the case of a video stream including encoded image
data of pictures of a single hierarchy group,
"general_level_idc" does not change on a picture-bypicture
basis. Alternatively, "sublayer_level_idc" is
25 given to each sublayer, and the decoder reads packets
of temporal_id within a decodable range, and thus data
30
of a corresponding hierarchy is processed.
[0065] A 3-bit field of "scalability_id" is newly
defined and is an ID representing a scalability given
to each stream when multiple video streams supply a
scalable service. "0" represents a base 5 stream, and "1"
to "7" each represent an ID that increases according to
the degree of scalability from the base stream.
[0066] Fig. 10 shows a structural example (Syntax)
of a scalability extension descriptor
10 (scalability_extension_descriptor). Further, Fig. 11
shows details (Semantics) of main information in the
structural example.
[0067] An 8-bit field of
"scalability_extension_descriptor_tag" shows a
15 descriptor type and shows a scalability extension
descriptor here. An 8-bit field of
"scalability_extension_descriptor_length" shows the
length (size) of the descriptor. As the length of the
descriptor, a subsequent number of bytes is shown. A 1-
20 bit field of "extension_stream_existing_flag" is a flag
showing the presence of an extension service by a
different stream. "1" represents that an extended
stream is present, and "0" represents that an extended
stream is absent.
25 [0068] A 3-bit field of "extension_type" shows a
type of extension. "001" represents that extension is
31
time direction scalable. "010" represents that
extension is spatial direction scalable. "011"
represents that extension is bit rate scalable.
[0069] A 4-bit field of "number_of_streams" shows
the total number of streams involving 5 a distribution
service. A 3-bit field of "scalability_id" is an ID
representing a scalability given to each stream when
multiple video streams supply a scalable service. "0"
represents a base stream, and "1" to "7" each represent
10 an ID that increases according to the degree of
scalability from the base stream.
[0070] A 3-bit field of "number_of_layers" shows the
total number of hierarchies of the stream. An 8-bit
field of "sublayer_level_idc shows a value of level_idc
15 at which an appropriate sublayer represented by
temporal_id includes a layer lower than the sublayer to
be handled by the decoder. "Number of layers" contains
all values of "Nuh_temporal_id_plus1" of the NAL unit
header, and when the demultiplexer (demuxer) detects
20 this, it is possible to preliminarily recognize to
which hierarchy a decoder corresponding to a
predetermined level_idc can perform decoding by using
"sublayer_level_idc".
[0071] As described above, in this embodiment, the
25 level specification value (general_level_idc) of a bit
rate included in the SPS and the like are used as
32
identification information of a hierarchy group to
which encoded image data belongs, when the multiple
hierarchies are divided into a predetermined number of,
i.e., two or more hierarchy groups. A value of the
level specification value of each 5 hierarchy group is
set to a value corresponding to a frame rate
constituted of pictures of this hierarchy group and
pictures of all the hierarchy groups on the hierarchy
side lower than this hierarchy group.
10 [0072] Fig. 12 shows a configuration example of the
multiplexer 104. The multiplexer 104 includes a PES
priority generation unit 141, a section coding unit 142,
PES packetization units 143-1 to 143-N, a switch unit
144, and a transport packetization unit 145.
15 [0073] The PES packetization units 143-1 to 143-N
respectively read video streams 1 to N stored in the
coded picture buffer 103 and generate PES packets. At
that time, the PES packetization units 143-1 to 143-N
provide time stamps of a DTS (Decoding Time Stamp) and
20 a PTS (Presentation Time Stamp) to the PES headers on
the basis of HRD information of the video streams 1 to
N. In this case, "cpu_removal_delay" and
"dpb_output_delay" of each picture are referenced,
converted into a DTS and a PTS with accuracy
25 synchronized with a STC (System Time Clock) time, and
arranged at a predetermined position of the PES header.
33
[0074] Information on the number of hierarchies
(Number of layers) and the number of streams (Number of
streams) are supplied to the PES priority generation
unit 141 from the CPU 101. The PES priority generation
unit 141 generates degree-of-priority 5 information of
each hierarchy group, in the case where the multiple
hierarchies represented by the number of hierarchies
are divided into a predetermined number of, i.e., two
or more hierarchy groups. For example, when the
10 multiple hierarchies are divided into two, the PES
priority generation unit 141 generates a value ("1" for
the low hierarchy group, "0" for the high hierarchy
group) to be inserted into a 1-bit field of
"PES_priority" of the header of a PES packet.
15 [0075] The degree-of-priority information of each
hierarchy group, which is generated in the PES priority
generation unit 141, is supplied to the PES
packetization units 143-1 to 143-N. The PES
packetization units 143-1 to 143-N insert the degree of
20 priority of each hierarchy group, as identification
information, into the header of a PES packet including
encoded image data of a picture of that hierarchy group.
[0076] It should be noted that processing of
inserting the degree of priority of a hierarchy group
25 to which the picture belongs into the header of the PES
packet as header information on a picture-by-picture
34
basis in such a manner may be limited to a case where a
single video stream (single stream) is generated in the
encoder 102. In this case, processing is performed only
in the PES packetization unit 143-1.
[0077] The switch unit 144 selectively 5 takes out the
PES packet generated in the PES packetization units
143-1 to 143-N on the basis of a packet identifier
(PID) and transmits the PES packet to the transport
packetization unit 145. The transport packetization
10 unit 145 generates a TS packet including the PES packet
in the payload and obtains a transport stream TS.
[0078] The section coding unit 142 generates various
types of section data to be inserted into the transport
stream TS. Information on the number of hierarchies
15 (Number of layers) and the number of streams (Number of
streams) are supplied to the section coding unit 142
from the CPU 101. On the basis of the information, the
section coding unit 142 generates the above-mentioned
HEVC descriptor (HEVC_descriptor) and scalability
20 extension descriptor (scalability_extension_descriptor).
[0079] The section coding unit 142 transmits various
types of section data to the transport packetization
unit 145. The transport packetization unit 145
generates a TS packet including this section data and
25 inserts the TS packet into the transport stream TS.
[0080] Fig. 13 shows a processing flow of the
35
multiplexer 104. This is an example in which the
multiple hierarchies are divided into two of a low
hierarchy group and a high hierarchy group. The
multiplexer 104 starts processing in Step ST1 and then
moves to processing of Step ST2. 5 In Step ST2, the
multiplexer 104 sets temporal_id_ of each picture of
the video stream (video elementary stream) and the
number of constituting encoded streams.
[0081] Next, in Step ST3, the multiplexer 104
10 references HRD information
(cpu_removal_delay,dpb_output_delay), determines DTS
and PTS, and inserts the DTS and the PTS into a
predetermined position of the PES header.
[0082] Next, in Step ST4, the multiplexer 104
15 determines whether the stream is a single stream
(single video stream) or not. When the stream is a
single stream, in Step ST5, the multiplexer 104
proceeds with multiplexing processing using one PID
(packet identifier), and then moves to processing of
20 Step ST7.
[0083] In Step ST7, the multiplexer 104 determines
whether each picture is a picture (slice) of a low
hierarchy group or not. When each picture is a picture
of a low hierarchy group, in Step ST8, the multiplexer
25 104 sets "1" for "PES_priority" of the header of a PES
packet including encoded image data of that picture in
36
the payload. On the other hand, when each picture is a
picture of a high hierarchy group (non-low hierarchy
group), in Step ST9, the multiplexer 104 sets "0" for
"PES_priority" of the header of a PES packet including
encoded image data of that picture in 5 the payload. The
multiplexer 104 moves to processing of Step ST10 after
the processing of Steps ST8 and ST9.
[0084] Here, an association between a picture and a
slice will be described. The picture is a concept and
10 the same as the slice in a structural definition. One
picture can be divided into multiple slices, but the
multiple slices are the same to serve as an access unit,
as found from a parameter set.
[0085] When the stream is not a single stream in
15 Step ST4 described above, in Step ST6, the multiplexer
104 proceeds with the multiplexing processing using
multiple packet PIDs (packet identifier), and then
moves to processing of Step ST10. In Step ST10, the
multiplexer 104 inserts the encoded stream (video
20 elementary stream) into the PES payload and converts
the encoded stream into PES packets.
[0086] Next, in Step ST11, the multiplexer 104 codes
the HEVC descriptor, the scalability extension
descriptor, and the like. The multiplexer 104 then
25 converts those descriptors into transport packets in
Step ST12, and obtains a transport stream TS. The
37
multiplexer 104 then terminates the processing in Step
ST13.
[0087] Fig. 14 shows a configuration example of a
transport stream TS when distribution by a single
stream is performed. This transport 5 stream TS includes
one video stream. In other words, in this configuration
example, a PES packet "video PES1" of a video stream
including encoded image data by HEVC, for example, of
pictures of the multiple hierarchies and a PES packet
10 "audio PES1" of an audio stream exist.
[0088] In the encoded image data of each picture,
the NAL units of VPS, SPS, SEI, and the like exist. As
described above, in the header of the NAL unit of each
picture, temporal_id representing a hierarchy of that
15 picture is inserted. Further, for example, the VPS
includes the level specification value
(general_level_idc) of a bit rate. Further, for example,
picture timing SEI includes "cpb_removal_delay" and
"dpb_output_delay".
20 [0089] Further, in the header of the PES packet (PES
header), a field of one bit showing the degree of
priority of "PES_priority" exists. By this
"PES_priority", whether encoded image data of a picture
included in the PES payload is of a picture of the low
25 hierarchy group or of a picture of the high hierarchy
group is distinguishable.
38
[0090] Further, the transport stream TS contains a
PMT (Program Map Table) as one type of PSI (Program
Specific Information). This PSI is information
describing to which program each elementary stream
included in the transport 5 stream belongs.
[0091] In the PMT, a program loop that describes
information related to the entire program exists.
Further, in the PMT, an elementary loop including
information related to each elementary stream exists.
10 In this configuration example, a video elementary loop
(video ES1 loop) and an audio elementary loop (audio
ES1 loop) exist.
[0092] In the video elementary loop, information
such as a stream type and a packet identifier (PID),
15 and a descriptor that describes information related to
that video stream are arranged to correspond to the
video stream (video PES1). As one descriptor, the
above-mentioned HEVC descriptor (HEVC_descriptor) and
scalability extension descriptor
20 (scalability_extension_descriptor) are inserted.
[0093] Referring back to Fig. 2, the transmission
unit 105 modulates the transport stream TS in a
modulation method suitable for broadcasting such as
QPSK/OFDM, and transmits an RF-modulated signal from a
25 transmission antenna.
[0094] The operation of the transmission apparatus
39
100 shown in Fig. 2 will be described briefly.
Uncompressed moving image data is input to the encoder
102. In the encoder 102, hierarchical encoding is
performed on the moving image data. In other words, in
the encoder 102, image data of pictures 5 constituting
the moving image data are sorted into multiple
hierarchies and encoded, so that a video stream
including encoded image data of the pictures of the
hierarchies is generated. At that time, the encoding is
10 performed such that a referenced picture belongs to a
hierarchy of the referenced picture and/or a hierarchy
lower than the hierarchy of the referenced picture.
[0095] In the encoder 102, a single video stream
including encoded image data of pictures of each
15 hierarchy is generated. Alternatively, the multiple
hierarchies are divided into a predetermined number of,
i.e., two or more hierarchy groups, and a predetermined
number of video streams each including encoded image
data of pictures of each hierarchy group are generated.
20 [0096] Further, the video stream including encoded
data of pictures of each hierarchy, which is generated
in the encoder 102, is supplied to the coded picture
buffer (cpb) 103 and temporarily stored. In the
multiplexer 104, the video stream stored in the coded
25 picture buffer 103 is read, converted into PES packets,
further converted into transport packets to be
40
multiplexed, thus obtaining a transport stream TS as a
multiplexed stream.
[0097] In the multiplexer 104, for example, in the
case of a single video stream (single stream), the
identification information for identifying 5 a hierarchy
group to which encoded image data of each picture of
the video stream belongs is inserted into the header of
the PES packet (PES header). For example, when the
multiple hierarchies are divided into two of a low
10 hierarchy group and a high hierarchy group, a 1-bit
field of PES priority (PES_priority) of the PES header
is used.
[0098] Further, in the multiplexer 104, the
hierarchy information and the stream configuration
15 information are inserted into the transport stream TS.
In other words, in the multiplexer 104, the HEVC
descriptor (HEVC_descriptor) and the scalability
extension descriptor (scalability_extension_descriptor)
are inserted into a video elementary loop corresponding
20 to each video stream.
[0099] The transport stream TS generated in the
multiplexer 104 is transmitted to the transmission unit
105. In the transmission unit 105, the transport stream
TS is modulated in a modulation method suitable for
25 broadcasting such as QPSK/OFDM, and an RF-modulated
signal is transmitted from a transmission antenna.
41
[0100] "Configuration of Reception Apparatus"
Fig. 15 shows a configuration example of the
reception apparatus 200. The reception apparatus 200
includes a CPU (Central Processing Unit) 201, a
reception unit 202, a demultiplexer 5 203, and a coded
picture buffer (cpb) 204. Further, the reception
apparatus 200 includes a decoder 205, a decoded picture
buffer (dpb) 206, and a post-processing unit 207. The
CPU 201 constitutes a control unit and controls
10 operations of respective units of the reception
apparatus 200.
[0101] The reception unit 202 demodulates the RFmodulated
signal received by an reception antenna and
acquires the transport stream TS. The demultiplexer 203
15 selectively takes out the encoded image data of a
picture of a hierarchy group corresponding to a
decoding capability (Decoder temporal layer capability)
from the transport stream TS and transmits the encoded
image data to the coded picture buffer (cpb) 204.
20 [0102] Fig. 16 shows a configuration example of the
demultiplexer 203. The demultiplexer 203 includes a TS
adaptation field extraction unit 231, a clock
information extraction unit 232, a TS payload
extraction unit 233, a section extraction unit 234, a
25 PSI table/descriptor extraction unit 235, and a PES
packet extraction unit 236. Further, the demultiplexer
42
203 includes a PES header extraction unit 237, a time
stamp extraction unit 238, an identification
information extraction unit 239, a PES payload
extraction unit 240, and a stream configuration unit
(stream 5 composer) 241.
[0103] The TS adaptation field extraction unit 231
extracts the adaptation field from a TS packet
including the adaptation field of the transport stream
TS. The clock information extraction unit 232 extracts
10 the PCR (Program Clock Reference) from an adaptation
field including a PCR and transmits the PCR to the CPU
201.
[0104] The TS payload extraction unit 233 extracts
the TS payload from a TS packet including a TS payload
15 in the transport stream TS. The section extraction unit
234 extracts the section data from a TS payload
including section data. The PSI table/descriptor
extraction unit 235 analyzes the section data extracted
in the section extraction unit 234 and extracts a PSI
20 table or a descriptor. The PSI table/descriptor
extraction unit 235 then transmits a minimum value
(min) and a maximum value (max) of temporal_id to the
CPU 201 and also to the stream configuration unit 241.
[0105] The PES packet extraction unit 236 extracts
25 the PES packet from a TS payload including a PES packet.
The PES header extraction unit 237 extracts a PES
43
header from the PES packet extracted in the PES packet
extraction unit 236. The time stamp extraction unit 238
extracts time stamps (DTS, PTS) inserted into the PES
header on a picture-by-picture basis and transmits the
time stamps to the CPU 201 and 5 also to the stream
configuration unit 241.
[0106] The identification information extraction
unit 239 extracts identification information for
identifying a hierarchy group to which a picture
10 belongs, the identification information being inserted
into the PES header on a picture-by-picture basis, and
transmits the identification information to the stream
configuration unit 241. For example, when the multiple
hierarchies are divided into two of a low hierarchy
15 group and a high hierarchy group, the identification
information extraction unit 239 extracts degree-ofpriority
information of a 1-bit field of "PES_priority"
of the PES header and transmits the degree-of-priority
information to the stream configuration unit 241. It
20 should be noted that the identification information is
definitely inserted on the transmission side when a
single video stream is included in the transport stream
TS, but may not be inserted on the transmission side
when multiple video streams are included in the
25 transport stream TS.
[0107] The PES payload extraction unit 240 extracts
44
a PES payload, that is, encoded image data of a picture
of each hierarchy, from the PES packet extracted in the
PES packet extraction unit 236. The stream
configuration unit 241 selectively takes out encoded
image data of a picture of 5 a hierarchy group
corresponding to a decoding capability (Decoder
temporal layer capability) from the encoded image data
of a picture of each hierarchy taken out in the PES
payload extraction unit 240, and transmits the encoded
10 image data to the coded picture buffer (cpb) 204. In
this case, the stream configuration unit 241 references
the hierarchy information and the stream configuration
information obtained in the PSI table/descriptor
extraction unit 235, the identification information
15 (degree-of-priority information) extracted in the
identification information extraction unit 239, and the
like.
[0108] For example, it is assumed that a frame rate
of a video stream (encoded stream) included in the
20 transport stream TS is 120 fps. For example, it is
assumed that the multiple hierarchies are divided into
two of a hierarchy group on the low hierarchy side and
a hierarchy group on a high hierarchy side, and a frame
rate of pictures of each hierarchy group is 60 fps. For
25 example, in the example of the hierarchical encoding
described above in Fig. 3, the hierarchies 0 to 3 are
45
assumed as a hierarchy group on the low hierarchy side,
for which a decoder that supports level_idc of 60 fps
can perform decoding. Further, the hierarchy 4 is
assumed as a hierarchy group on the high hierarchy side,
for which a decoder that supports level_5 idc of 120 fps
can perform decoding.
[0109] In this case, in the transport stream TS, a
single video stream (encoded stream) including encoded
data of pictures of each hierarchy is included, or two
10 video streams (encoded streams) of a base stream
(B_str) including encoded image data of pictures of the
hierarchy group on the low hierarchy side and an
extended stream (E_str) including encoded image data of
pictures of the hierarchy group on the high hierarchy
15 side are included.
[0110] When the decoding capability supports 120 fps,
the stream configuration unit 241 takes out the encoded
image data of pictures of all the hierarchies and
transmits the encoded image data to the coded picture
20 buffer (cpb) 204. On the other hand, when the decoding
capability does not support 120 fps but supports 60 fps,
the stream configuration unit 241 takes out only
encoded image data of pictures of the hierarchy group
on the low hierarchy side and transmits the encoded
25 image data to the coded picture buffer (cpb) 204.
[0111] Fig. 17 shows an example of picture (slice)
46
selection of the stream configuration unit 241 when the
transport stream TS includes a single video stream
(encoded stream). Here, "High" represents a picture of
a hierarchy group on the high hierarchy side, and "Low"
represents a picture of a hierarchy 5 group on the low
hierarchy side. Further, "P" represents "PES_priority".
[0112] When the decoding capability supports 120 fps,
the stream configuration unit 241 takes out encoded
image data of pictures of all the hierarchies and
10 transmits the encoded image data to the coded picture
buffer (cpb) 204. On the other hand, when the decoding
capability supports not 120 fps but 60 fps, the stream
configuration unit 241 performs filtering based on the
"PES_priority", takes out only pictures of the
15 hierarchy group on the low hierarchy side where P=1,
and transmits the pictures to the coded picture buffer
(cpb) 204.
[0113] Fig. 18 shows an example of picture (slice)
selection of the stream configuration unit 241 when the
20 transport stream TS includes two video streams (encoded
streams) of a base stream and an extended stream. Here,
"High" represents a picture of a hierarchy group on the
high hierarchy side, and "Low" represents a picture of
a hierarchy group on the low hierarchy side. Further,
25 it is assumed that a packet identifier (PID) of the
base stream is PID A, and a packet identifier (PID) of
47
the extended stream is PID B.
[0114] When the decoding capability supports 120 fps,
the stream configuration unit 241 takes out encoded
image data of pictures of all the hierarchies and
transmits the encoded image data to 5 the coded picture
buffer (cpb) 204. In this case, the stream
configuration unit 241 converts the encoded image data
of each picture into one stream on the basis of
decoding timing information and transmits the stream to
10 the coded picture buffer (cpb) 204.
[0115] In such a case, the value of the DTS is
considered as a decoding timing, and the streams are
collected to one such that the values monotonically
increase between the pictures. Such processing of
15 collecting the pictures into one stream may be
performed on multiple streams that are read from
multiple coded picture buffers (cpb) 204 provided to
correspond to the number of streams, and decoding
processing may be performed on that one stream.
20 [0116] On the other hand, when the decoding
capability supports not 120 fps but 60 fps, the stream
configuration unit 241 performs filtering based on the
packet identifier (PID), takes out only pictures of the
hierarchy group on the low hierarchy side having PID A,
25 and transmits the pictures to the coded picture buffer
(cpb) 204.
48
[0117] It should be noted that the stream
configuration unit 241 has a function of selectively
rewriting a decoding time stamp of the encoded image
data of each picture to be transmitted to the coded
picture buffer (cpb) 204 and 5 adjusting decoding
intervals of the low hierarchy pictures. This enables
realistic decoding processing even in a decoder 205
having a low decoding capability.
[0118] Fig. 19 shows a case where the hierarchy
10 groups are divided into the hierarchy group on the low
hierarchy side and the hierarchy group on the high
hierarchy side in the example of hierarchical encoding
shown in Fig. 3 and where encoded image data of
pictures belonging to the low hierarchy group are
15 selectively taken out in the stream configuration unit
241 and transmitted to the coded picture buffer (cpb)
204.
[0119] Part (a) of Fig. 19 shows a decoding timing
before the decoding intervals are adjusted. In this
20 case, the decoding intervals of pictures vary, and the
shortest decoding intervals are equal to the decoding
intervals at full resolution of 120 fps. In contrast to
this, part (b) of Fig. 19 shows a decoding timing after
the decoding intervals are adjusted. In this case, the
25 decoding intervals of pictures are made equal, and the
decoding intervals are 1/2 of the decoding intervals at
49
full resolution. In such a manner, in each hierarchy,
the decoding intervals are adjusted according to the
capability of a target decoder.
[0120] Fig. 20 shows an example of a processing flow
of the demultiplexer 203. The processing 5 flow shows a
case where the transport stream TS includes a single
video stream (encoded stream).
[0121] The demultiplexer 203 starts processing in
Step ST31 and then moves to processing of Step ST32. In
10 Step ST32, a decoding capability (Decoder temporal
layer capability) is set from the CPU 201. Next, the
demultiplexer 203 determines in Step ST33 whether a
capability to decode all the hierarchies (layers) is
provided or not.
15 [0122] When a capability to decode all the
hierarchies is provided, the demultiplexer 203
demultiplexes all TS packets that pass through an
appropriate PID filter and performs section parsing in
Step ST34. The demultiplexer 203 then moves to
20 processing of Step ST35.
[0123] In Step ST33, when a capability to decode all
the hierarchies is not provided, the demultiplexer 203
demultiplexes TS packets having "PES_priority" of "1"
and performs section parsing in Step ST36. The
25 demultiplexer 203 then moves to processing of Step ST35.
[0124] In Step ST35, the demultiplexer 203 reads the
50
HEVC descriptor (HEVC_descriptor) and the scalability
extension descriptor (scalability_extension_descriptor)
in the target PID section and obtains the presence or
absence of an extended stream, a scalable type, the
number of streams and IDs, maximum 5 and minimum values
of temporal_id, and a decoder supporting level of each
layer.
[0125] Next, the demultiplexer 203 transfers the
encoded stream as a target of PID to the coded picture
10 buffer (cpb) 204 in Step ST37 and notifies the DTS and
the PTS to the CPU 201. After the processing of Step
ST37, the demultiplexer 203 terminates the processing
in Step ST38.
[0126] Fig. 21 shows an example of a processing flow
15 of the demultiplexer 203. The processing flow shows a
case where the transport stream TS includes two video
streams (encoded streams) of a base stream and an
extended stream.
[0127] The demultiplexer 203 starts processing in
20 Step ST41 and then moves to processing of Step ST42. In
Step ST42, a decoding capability (Decoder temporal
layer capability) is set from the CPU 201. Next, the
demultiplexer 203 determines in Step ST43 whether a
capability to decode all the hierarchies (layers) is
25 provided or not.
[0128] When a capability to decode all the
51
hierarchies is provided, the demultiplexer 203
demultiplexes multiple streams constituting all the
hierarchies by the PID filter and performs section
parsing in Step ST44. The demultiplexer 203 then moves
to processing 5 of Step ST45.
[0129] In Step ST43, when a capability to decode all
the hierarchies is not provided, the demultiplexer 203
demultiplexes a stream where PID = PID A and performs
section parsing in Step ST46. The demultiplexer 203
10 then moves to processing of Step ST45.
[0130] In Step ST45, the demultiplexer 203 reads the
HEVC descriptor (HEVC_descriptor) and the scalability
extension descriptor (scalability_extension_descriptor)
in the target PID section and obtains the presence or
15 absence of an extended stream, a scalable type, the
number of streams and IDs, maximum and minimum values
of temporal_id, and a decoder supporting level of each
layer.
[0131] Next, the demultiplexer 203 connects encoded
20 streams as target of PID into one stream on the basis
of DTS (PTS if there is no DTS) information, and
transfers the stream to the coded picture buffer (cpb)
204 and also notifies the DTS and the PTS to the CPU
201 in Step ST47. After the processing of Step ST47,
25 the demultiplexer 203 terminates the processing in Step
ST48.
52
[0132] Referring back to Fig. 15, the coded picture
buffer (cpb) 204 temporarily stores the video stream
(encoded stream) taken out in the demultiplexer 203.
The decoder 205 takes out encoded image data of a
picture of a hierarchy specified as 5 a hierarchy to be
decoded, from the video stream stored in the coded
picture buffer 204. The decoder 205 then decodes the
taken encoded image data of each picture at a decoding
timing of that picture and transmits the resultant data
10 to the decoded picture buffer (dpb) 206.
[0133] Here, for the decoder 205, a hierarchy to be
decoded is specified by temporal_id from the CPU 201.
The specified hierarchy refers to all hierarchies
included in the video stream (encoded stream) taken out
15 in the demultiplexer 203 or to some hierarchies on the
low hierarchy side, and is set by the CPU 201
automatically or set according to a user operation.
Further, a decoding timing is given to the decoder 205
from the CPU 201 on the basis of a DTS (Decoding Time
20 stamp). It should be noted that when the decoder 205
decodes encoded image data of each picture, the decoder
205 reads image data of the referenced picture from the
decoded picture buffer 206 for use as necessary.
[0134] Fig. 22 shows a configuration example of the
25 decoder 205. The decoder 205 includes a temporal ID
analysis unit 251, a target hierarchy selection unit
53
252, and a decoding unit 253. The temporal ID analysis
unit 251 reads the video stream (encoded stream) stored
in the coded picture buffer 204 and analyzes
temporal_id inserted into the NAL unit header of the
encoded image data 5 of each picture.
[0135] The target hierarchy selection unit 252 takes
out encoded image data of a picture of a hierarchy
specified as a hierarchy to be decoded, from the video
stream read from the coded picture buffer 204 on the
10 basis of an analysis result of the temporal ID analysis
unit 251. The decoding unit 253 decodes the encoded
image data of each picture, which is taken out in the
target hierarchy selection unit 252, at a decoding
timing sequentially and transmits the resultant data to
15 the decoded picture buffer (dpb) 206.
[0136] In this case, the decoding unit 253 analyzes
VPS and SPS, grasps a level specification value
"sublayer_level_idc" of a bit rate of each sublayer,
for example, and checks whether decoding is possible
20 within the decoding capability or not. Further, in this
case, the decoding unit 253 analyzes SEI, grasps
"initial_cpb_removal_time" and "cpb_removal_delay", for
example, and checks whether a decoding timing from the
CPU 201 is adequate or not.
25 [0137] When decoding a slice, the decoding unit 253
acquires from a slice header
54
"ref_idx_l0_active(ref_idx_l1_active) as information
representing a prediction destination of a time
direction, and predicts a time direction. It should be
noted that a decoded picture has an index of
"short_term_ref_pic_set_idx" or "it_5 idx_sps" obtained
from the slice header and is processed to be referenced
by another picture.
[0138] Referring back to Fig. 15, the decoded
picture buffer (dpb) 206 temporarily stores image data
10 of each picture, which is decoded in the decoder 205.
The post-processing unit 207 performs processing of
matching a frame rate of the image data of each picture
sequentially read at a display timing from the decoded
picture buffer (dpb) 206 with a display capability. In
15 this case, a display timing is given from the CPU 201
on the basis of the PTS (Presentation Time stamp).
[0139] For example, when the frame rate of the
decoded image data of each picture is 120 fps and when
the display capability is 120 fps, the post-processing
20 unit 207 transmits the decoded image data of each
picture to a display as it is. Further, for example,
when the frame rate of the decoded image data of each
picture is 120 fps and when the display capability is
60 fps, the post-processing unit 207 performs
25 subsampling processing on the decoded image data of
each picture such that the time-direction resolution
55
becomes 1/2 times, and transmits the data to the
display as image data of 60 fps.
[0140] Further, for example, when the frame rate of
the decoded image data of each picture is 60 fps and
the display capability is 120 fps, the 5 post-processing
unit 207 performs interpolation processing on the
decoded image data of each picture such that the timedirection
resolution becomes twice, and transmits the
data to the display as image data of 120 fps. Further,
10 for example, when the frame rate of the decoded image
data of each picture is 60 fps and when the display
capability is 60 fps, the post-processing unit 207
transmits the decoded image data of each picture to the
display as it is.
15 [0141] Fig. 23 shows a configuration example of the
post-processing unit 207. This is an example manageable
in the case where the frame rate of the decoded image
data of each picture is 120 fps or 60 fps and the
display capability is 120 fps or 60 fps as described
20 above.
[0142] The post-processing unit 207 includes an
interpolation unit 271, a subsampling unit 272, and a
switch unit 273. The decoded image data of each picture
from the decoded picture buffer 206 is directly input
25 to the switch unit 273, input to the switch unit 273
after the frame rate is made twice in the interpolation
56
unit 271, or input to the switch unit 273 after the
frame rate is made 1/2 times in the subsampling unit
272.
[0143] The switch unit 273 is provided with
selection information from the CPU 201. 5 The selection
information is generated automatically by the CPU 201
referencing the display capability or generated
according to a user operation. The switch unit 273
selectively outputs any of the inputs on the basis of
10 the selection information. Thus, the frame rate of the
image data of each picture sequentially read from the
decoded picture buffer (dpb) 206 at a display timing is
matched with the display capability.
[0144] Fig. 24 shows an example of a processing flow
15 of the decoder 205 and the post-processing unit 207.
The decoder 205 and the post-processing unit 207 start
processing in Step ST51 and then move to processing of
Step ST52. In Step ST52, the decoder 205 reads a video
stream as a decoding target stored in the coded picture
20 buffer (cpb) 204 and selects a picture of a hierarchy
specified as a decoding target from the CPU 201 on the
basis of temporal_id.
[0145] Next, the decoder 205 sequentially decodes
encoded image data of each selected picture at a
25 decoding timing, transfers the decoded image data of
each picture to the decoded picture buffer (dpb) 206,
57
and temporalily stores the decoded image data in Step
ST53. Next, the post-processing unit 207 reads the
image data of each picture from the decoded picture
buffer (dpb) 206 at the display timing in Step ST54.
[0146] Next, the post-processing unit 5 207 determines
whether the frame rate of the read image data of each
picture matches with the display capability or not.
When the frame rate does not match with the display
capability, the post-processing unit 207 matches the
10 frame rate with the display capability to be
transmitted to the display in Step ST56, and then
terminates the processing in Step ST57. On the other
hand, when the frame rate matches the display
capability, the post-processing unit 207 transmits the
15 image data to the display without changing the frame
rate in Step ST58 and then terminates the processing in
Step ST57.
[0147] The operation of the reception apparatus 200
shown in Fig. 15 will be described briefly. In the
20 reception unit 202, an RF-modulated signal received by
a reception antenna is demodulated and a transport
stream TS is acquired. The transport stream TS is
transmitted to the demultiplexer 203. In the
demultiplexer 203, encoded image data of a picture of a
25 hierarchy group corresponding to a decoding capability
(Decoder temporal layer capability) is selectively
58
taken out from the transport stream TS, transmitted to
the coded picture buffer (cpb) 204, and temporalily
stored.
[0148] In the decoder 205, encoded image data of a
picture of a hierarchy specified as 5 a hierarchy to be
decoded is taken out from the video stream stored in
the coded picture buffer 204. In the decoder 205, the
taken-out encoded image data of each picture is then
decoded at a decoding timing of that picture,
10 transmitted to the decoded picture buffer (dpb) 206,
and temporalily stored. In this case, when the encoded
image data of each picture is decoded, image data of a
referenced picture is read from the decoded picture
buffer 206 for use as necessary.
15 [0149] The image data of each picture sequentially
read from the decoded picture buffer (dpb) 206 at a
display timing is transmitted to the post-processing
unit 207. In the post-processing unit 207,
interpolation or subsampling for matching the frame
20 rate of the image data of each picture with the display
capability is performed. The image data of each picture
processed in the post-processing unit 207 is supplied
to the display, and displaying of a moving image by the
image data of that picture is performed.
25 [0150] As described above, in the transmission and
reception system 10 shown in Fig. 1, on the
59
transmission side, the identification information for
identifying a hierarchy group to which encoded image
data of each picture included in a video stream belongs
is inserted into a layer (header of a PES packet) of
this video stream. Therefore, for 5 example, on the
reception side, using this identification information
easily enables selective decoding of encoded image data
of a picture of a hierarchy equal to or lower than a
predetermined hierarchy corresponding to the decoding
10 capability.
[0151] Further, in the transmission and reception
system 10 shown in Fig. 1, on the transmission side,
the scalability extension descriptor
(scalability_extension_descriptor) and the like are
15 inserted into the layer of the transport stream TS.
Therefore, for example, on the reception side, it is
possible to easily grasp the hierarchy information in
hierarchical encoding, the configuration information of
the video stream included in the transport stream TS,
20 and the like and perform appropriate decoding
processing.
[0152] Further, in the transmission and reception
system 10 shown in Fig. 1, on the reception side,
encoded image data of a picture of a hierarchy equal to
25 or lower than a predetermined hierarchy corresponding
to the decoding capability (Decoder temporal layer
60
capability) is selectively taken in the coded picture
buffer 204 from the received video stream and then
decoded. Therefore, for example, it is possible to
perform appropriate decoding processing corresponding
to the decoding 5 capability.
[0153] Further, in the transmission and reception
system 10 shown in Fig. 1, on the reception side, the
function of selectively rewriting a decoding time stamp
of the encoded image data of each picture taken in the
10 coded picture buffer 204 and adjusting decoding
intervals of low hierarchy pictures is provided.
Therefore, for example, this enables realistic decoding
processing even when the decoder 205 has a low decoding
capability.
15 [0154] Further, in the transmission and reception
system 10 shown in Fig. 1, on the reception side, the
frame rate of decoded image data of each picture is
matched with the display capability in the postprocessing
unit 207. Therefore, for example, the image
20 data of the frame rate matched with a high display
capability can be obtained even when the decoding
capability is low.
[0155] <2. Modified Example>
It should be noted that the embodiment described
25 above has shown the example in which the identification
information for identifying a hierarchy group to which
61
encoded image data of each picture included in a video
stream belongs among a predetermined number of
hierarchy groups is inserted into the header of a PES
packet (PES header). However, the position at which
this identification information 5 is inserted is not
limited thereto.
[0156] For example, the multiplexer 104 (see Fig. 2)
may insert this identification information into the
adaptation field of a TS packet having an adaptation
10 field. For example, when the multiple hierarchies are
divided into two of a low hierarchy group and a high
hierarchy group, the multiplexer 104 uses a 1-bit field
of a known elementary stream priority indicator
(elementary_stream_priority_indicator), located in the
15 adaptation field.
[0157] When a PES packet including in a payload
encoded image data of a picture of a hierarchy group on
the low hierarchy side is included in a payload of a
subsequent TS packet, the 1-bit field is set to "1",
20 that is, set to have a high degree of priority. On the
other hand, when a PES packet including in a payload
encoded image data of a picture of a hierarchy group on
the low hierarchy side is included in a payload of a
subsequent TS packet, the 1-bit field is set to "0",
25 that is, set to have a low degree of priority.
[0158] Fig. 25 shows an arrangement example of an
62
adaptation field. This is an example in which the
multiple hierarchies are divided into two of a low
hierarchy group and a high hierarchy group, and a 1-bit
field of an elementary stream priority indicator
(elementary_stream_priority_5 indicator) is used.
[0159] In the example shown in the figure, a TS
packet including an adaptation field is arranged
immediately before each group including a predetermined
number of TS packets, the TS packets separately
10 including a PES packet including encoded image data of
one picture in a payload. In this case, when the one
picture is a picture of a hierarchy group on the low
hierarchy side, the 1-bit field of the elementary
stream priority indicator is set to "1". On the other
15 hand, when the one picture is a picture of a hierarchy
group on the high hierarchy side, the 1-bit field of
the elementary stream priority indicator is set to "0".
[0160] As shown in Fig. 25, arranging the TS packet
including the adaptation field enables the reception
20 side to easily identify a hierarchy group to which
encoded data of a picture belongs, for each encoded
image data of a picture included in a video stream. It
should be noted that the arrangement example of Fig. 25
shows that a TS packet including an adaptation field is
25 arranged on a picture-by-picture basis. However, each
time a hierarchy group to which a picture belongs is
63
changed, a TS packet including an adaptation field may
be arranged immediately therebefore.
[0161] Fig. 26 shows a configuration example of a
multiplexer 104A of the transmission apparatus 100 when
identification information of a 5 hierarchy group is
inserted into an adaptation field as described above.
In Fig. 26, portions corresponding to those in Fig. 12
are denoted by the same reference symbols and detailed
description thereof will be omitted. The multiplexer
10 104A includes an adaptation field priority instruction
unit 146 instead of the PES priority generation unit
141 of the multiplexer 104 of Fig. 12.
[0162] The priority instruction unit 146 is provided
with information on the number of hierarchies (Number
15 of layers) and the number of streams (Number of
streams) from the CPU 101. The priority instruction
unit 146 generates degree-of-priority information of
each hierarchy group when the multiple hierarchies
represented by the number of hierarchies are divided
20 into a predetermined number of, i.e., two or more
hierarchy groups. For example, when the multiple
hierarchies are divided into two, the priority
instruction unit 146 generates a value to be inserted
into the 1-bit field of the elementary stream priority
25 indicator ("1" for the low hierarchy group, "0" for the
high hierarchy group).
64
[0163] The degree-of-priority information of each
hierarchy group, which is generated in the priority
instruction unit 146, is supplied to the transport
packetization unit 145. The transport packetization
unit 145 arranges a TS packet including 5 an adaptation
field immediately before each group including a
predetermined number of TS packets, the TS packets
separately including a PES packet including encoded
image data of one picture in a payload. In such a case,
10 the transport packetization unit 145 then inserts, as
identification information, the degree-of-priority
information corresponding to a hierarchy group to which
the picture belongs into the adaptation field.
[0164] Fig. 27 shows a configuration example of a
15 transport stream TS when identification information of
a hierarchy group is inserted into an adaptation field
as described above. This configuration example is
substantially the same as the configuration example
described above shown in Fig. 14. In this configuration
20 example, a TS packet including an adaptation field
exists, and identification information for identifying
a hierarchy groups to which each picture belongs is
inserted into the adaptation field. For example, when
the multiple hierarchies are divided into two of a low
25 hierarchy group and a high hierarchy group, the 1-bit
field of the elementary stream priority indicator
65
(elementary_stream_priority_indicator) is used.
[0165] Fig. 28 shows a configuration example of a
demultiplexer 203A of the reception apparatus 200 when
identification information of a hierarchy group is
inserted into an adaptation field as 5 described above.
In Fig. 28, portions corresponding to those in Fig. 16
are denoted by the same reference symbols and detailed
description thereof will be omitted. The demultiplexer
203A includes an identification information extraction
10 unit 242 instead of the identification information
extraction unit 239 of the demultiplexer 203 of Fig. 16.
[0166] The identification information extraction
unit 242 extracts identification information from an
adaptation field and transmits the identification
15 information to the stream configuration unit 241. For
example, when the multiple hierarchies are divided into
two of a low hierarchy group and a high hierarchy group,
the identification information extraction unit 242
extracts degree-of-priority information of the 1-bit
20 field of "elementary_stream_priority_indicator" of the
adaptation field, and transmits the degree-of-priority
information to the stream configuration unit 241.
[0167] The stream configuration unit 241 selectively
takes out encoded image data of a picture of a
25 hierarchy group corresponding to a decoding capability
(Decoder temporal layer capability) from the encoded
66
image data of pictures of each hierarchy taken out in
the PES payload extraction unit 240, and transmits the
encoded image data to the coded picture buffer (cpb)
204. In this case, the stream configuration unit 241
references the hierarchy information 5 and stream
configuration information obtained in the PSI
table/descriptor extraction unit 235, the
identification information (degree-of-priority
information) extracted in the identification
10 information extraction unit 242, and the like.
[0168] Further, in the embodiment described above,
the transmission and reception system 10 constituted by
the transmission apparatus 100 and the reception
apparatus 200 has been described, but the configuration
15 of the transmission and reception system to which the
present technology is applicable is not limited thereto.
For example, the following configuration may be
provided: the portion of the reception apparatus 200
may be a set-top box and a monitor connected by a
20 digital interface such as (HDMI (High-Definition
Multimedia Interface). It should be noted that "HDMI"
is a registered trademark.
[0169] Further, in the embodiment described above,
the example in which the container is a transport
25 stream (MPEG-2 TS) has been described. However, the
present technology is similarly applicable to a system
67
having a configuration using a network such as the
Internet for distribution to a reception terminal. In
the distribution over the Internet, the distribution is
performed in an MP4 or a container of any other format
in many cases. Specifically, containers 5 of various
formats such as a transport stream (MPEG-2 TS) adopted
in the standard of digital broadcasting and an MP4 used
in the distribution over the Internet are applied as a
container.
10 [0170] For example, Fig. 29 shows a configuration
example of an MP4 stream. In the MP4 stream, boxes of
"moov", "moof", "mdat", and the like exist. In the box
of "mdat", as tracks, a video elementary stream
"track1:video ES1" as an encoded stream of a video
15 exists, and an audio elementary stream "track1:audio
ES1" as an encoded stream of audio exists.
[0171] Further, in the box of "moof", "mfhd (movie
fragment header" exists as a header portion, and "track
fragment" corresponding to each track exists as a data
20 portion thereof. In "track1 fragment(video)"
corresponding to the video elementary stream
"track1:video ES1", "Independent and disposal samples"
exists. In "Independent and disposal samples", a box of
"SampleDependencyTypeBox" corresponding to each picture
25 is inserted.
[0172] In this box, identification information for
68
identifying a hierarchy group to which encoded image
data of each picture belongs can be inserted. For
example, when the multiple hierarchies are divided into
two hierarchy groups of the uppermost hierarchy and the
other lower hierarchy, the identification 5 information
can be inserted using a 2-bit field of
"sample_depends_on" and a 2-bit field of
"sample_is_depended_on".
[0173] Fig. 30 shows a structural example (Syntax)
10 of "SampleDependencyTypeBox". Further, Fig. 31 shows
details (Semantics) of main information in the
structural example. In this case, "1" is set for
"sample_depends_on" so as to indicate that it is used
to reference another picture and is not an I picture,
15 and "2" is set for "sample_is_depended_on" so as to
indicate that it is not referenced by another picture.
Thus, it is possible to identify a picture belonging to
a group of the uppermost hierarchy. Further, in other
situations, it is possible to identify that picture as
20 a picture belonging to a hierarchy group of a hierarchy.
[0174] It should be noted that use of a box
"SampleScalablePriorityBox" newly defined can also be
conceived instead of using a box of
"SampleDependencyTypeBox". Fig. 32 shows a structural
25 example (Syntax) of "SampleScalablePriorityBox".
Further, Fig. 33 shows details (Semantics) of main
69
information in the structural example.
[0175] In this case, when the multiple hierarchies
are divided into two hierarchy groups of the lowermost
hierarchy group and the high hierarchy group, the
identification information is inserted 5 using a 2-bit
field of "base_and_priority". In other words, for
example, when "1" is set for "base_and_priority", it is
possible to identify a picture having a low degree of
priority and belonging to the high hierarchy group. On
10 the other hand, for example, when "2" is set for
"base_and_priority", it is possible to identify a
picture having a high degree of priority and belonging
to the low hierarchy group.
[0176] Further, the present technology can have the
15 following configurations.
(1) A transmission apparatus, including:
an image encoding unit that sorts image data of
pictures constituting moving image data into multiple
hierarchies, encodes image data of pictures of each of
20 the sorted hierarchies, and generates video data
including the encoded image data of the pictures of
each of the hierarchies;
a transmission unit that transmits a container of
a predetermined format including the generated video
25 data; and
an identification information insertion unit that
70
divides the multiple hierarchies into a predetermined
number of hierarchy groups, the predetermined number
being two or more, and inserts identification
information for identifying a hierarchy group to which
encoded image data of each picture 5 included in the
video data belongs, into a packet as a container of the
video data.
(2) The transmission apparatus according to (1), in
which
10 the identification information is degree-ofpriority
information that is set to be higher for a
hierarchy group on a low hierarchy side.
(3) The transmission apparatus according to (1), in
which
15 the identification information is inserted into a
header of a PES packet including the encoded image data
for each picture in a payload.
(4) The transmission apparatus according to (3), in
which
20 the identification information is inserted using a
field of a PES priority of the header.
(5) The transmission apparatus according to (1), in
which
the identification information is inserted into an
25 adaptation field of a TS packet including the
adaptation field.
71
(6) The transmission apparatus according to (5), in
which
the identification information is inserted using a
field of an ES priority indicator of the adaptation
5 field.
(7) The transmission apparatus according to (1), in
which
the identification information is inserted into a
box of a header related to a track of an appropriate
10 picture.
(8) The transmission apparatus according to any one of
(1) to (7), in which
the image encoding unit
generates a single video stream including the
15 encoded image data of the pictures of each of the
hierarchies or a predetermined number of video data
items each including the encoded image data of the
pictures of each of the hierarchies, and
further includes a configuration information
20 insertion unit that inserts configuration information
of the video stream included in the container into a
layer of the container.
(9) A transmission method, including:
an image encoding step of sorting image data of
25 pictures constituting moving image data into multiple
hierarchies, encoding image data of pictures of each of
72
the sorted hierarchies, and generating video data
including the encoded image data of the pictures of
each of the hierarchies;
a transmission step of transmitting, by a
transmission unit, a container 5 of a predetermined
format including the generated video data; and
an identification information insertion step of
dividing the multiple hierarchies into a predetermined
number of hierarchy groups, the predetermined number
10 being two or more, and inserting identification
information for identifying a hierarchy group to which
encoded image data of each picture included in the
video data belongs, into a packet as a container of the
video data.
15 (10) A reception apparatus, including:
an reception unit that receives a container of a
predetermined format including video data including
encoded image data of pictures of each of hierarchies,
the encoded image data being obtained by sorting image
20 data of pictures constituting moving image data into
multiple hierarchies and encoding the image data; and
an image decoding unit that selectively takes in a
buffer encoded image data of a picture of a hierarchy
equal to or lower than a predetermined hierarchy
25 corresponding to a decoding capability from the video
data included in the received container, decodes the
73
encoded image data of each picture taken in the buffer,
and obtains image data of the picture of the hierarchy
equal to or lower than the predetermined hierarchy.
(11) The reception apparatus according to (10), in
5 which
the multiple hierarchies are divided into a
predetermined number of hierarchy groups, the
predetermined number being two or more, and
identification information for identifying a hierarchy
10 group to which the encoded image data of each picture
included in the video data belongs is inserted into a
packet as a container of the video data, and
the image decoding unit takes in the buffer the
encoded image data of the picture of a predetermined
15 hierarchy group corresponding to the decoding
capability and decodes the encoded image data on the
basis of the identification information.
(12) The reception apparatus according to (11), in
which
20 the identification information is inserted into a
header of a PES packet including the encoded image data
for each picture in a payload.
(13) The reception apparatus according to (11), in
which
25 the identification information is inserted into an
adaptation field of a TS packet including the
74
adaptation field.
(14) The transmission apparatus according to (11), in
which
the identification information is inserted into a
box of a header related to a track 5 of an appropriate
picture.
(15) The reception apparatus according to (10), in
which
the multiple hierarchies are divided into a
10 predetermined number of hierarchy groups, the
predetermined number being two or more, and the
received container includes the predetermined number of
video streams respectively including encoded image data
of pictures of the predetermined number of hierarchy
15 groups, and
the image decoding unit takes in the buffer the
encoded image data of the picture of the predetermined
hierarchy group corresponding to the decoding
capability and decodes the encoded image data on the
20 basis of stream identification information.
(16) The reception apparatus according to (15), in
which
the image decoding unit converts, when the encoded
image data of the picture of the predetermined
25 hierarchy group is included in multiple video streams,
the encoded image data of each picture into one scream
75
on the basis of decoding timing information and takes
the stream in the buffer.
(17) The reception apparatus according to any one of
(10) to (16), in which
the image decoding unit 5 has a function of
rewriting a decoding time stamp of the encoded image
data of each picture selectively taken in the buffer
and adjusting decoding intervals of low hierarchy
pictures.
10 (18) The reception apparatus according to any one of
(10) to (17), , further including a post-processing
unit that matches a frame rate of the image data of
each picture obtained in the image decoding unit with
the display capability.
15 (19) A reception method, including:
a reception step of receiving, by a reception unit,
a container of a predetermined format including video
data including encoded image data of pictures of each
of hierarchies, the encoded image data being obtained
20 by sorting image data of pictures constituting moving
image data into multiple hierarchies and encoding the
image data; and
an image decoding step of selectively taking in a
buffer encoded image data of a picture of a hierarchy
25 equal to or lower than a predetermined hierarchy
corresponding to a decoding capability from the video
76
data included in the received container, decoding the
encoded image data of each picture taken in the buffer,
and obtaining image data of the picture of the
hierarchy equal to or lower than the predetermined
5 hierarchy.
[0177] A main feature of the present technology is
to insert identification information for identifying a
hierarchy group to which encoded image data of each
picture included in a video data belongs into a packet
10 as a container of video data, and thus easily enable
the reception side to selectively decode encoded image
data of a picture of a hierarchy equal to or lower than
a predetermined hierarchy corresponding to a decoding
capability by using the identification information (see
15 Fig. 12).
Description of Symbols
[0178]
10 transmission and reception system
100 transmission apparatus
20 101 CPU
102 encoder
103 coded picture buffer (cpb)
104,104A multiplexer
105 transmission unit
25 141 PES priority generation unit
142 section coding unit
77
143-1 to 143-N PES packetization unit
144 switch unit
145 transport packetization unit
146 adaptation field priority instruction unit
200 reception 5 apparatus
201 CPU
202 reception unit
203 demultiplexer
204 coded picture buffer (cpb)
10 205 decoder
206 decoded picture buffer (dpb)
207 post-processing unit
231 TS adaptation field extraction unit
232 clock information extraction unit
15 233 TS payload extraction unit
234 section extraction unit
235 PSI table/descriptor extraction unit
236 PES packet extraction unit
237 PES header extraction unit
20 238 time stamp extraction unit
239 identification information extraction unit
240 PES payload extraction unit
241 stream configuration unit
242 identification information extraction unit
25 251 temporal ID analysis unit
252 target hierarchy selection unit
78
253 decoding unit
271 interpolation unit
272 subsampling unit
273 switch unit
5
79
Claims
[1] A transmission apparatus, comprising:
an image encoding unit that sorts image data of
pictures constituting moving image data into multiple
hierarchies, encodes image data of pictures 5 of each of
the sorted hierarchies, and generates video data
including the encoded image data of the pictures of
each of the hierarchies;
a transmission unit that transmits a container of
10 a predetermined format including the generated video
data; and
an identification information insertion unit that
divides the multiple hierarchies into a predetermined
number of hierarchy groups, the predetermined number
15 being two or more, and inserts identification
information for identifying a hierarchy group to which
encoded image data of each picture included in the
video data belongs, into a packet as a container of the
video data.
20 [2] The transmission apparatus according to claim 1,
wherein
the identification information is degree-ofpriority
information that is set to be higher for a
hierarchy group on a low hierarchy side.
25 [3] The transmission apparatus according to claim 1,
wherein
80
the identification information is inserted into a
header of a PES packet including the encoded image data
for each picture in a payload.
[4] The transmission apparatus according to claim 3,
5 wherein
the identification information is inserted using a
field of a PES priority of the header.
[5] The transmission apparatus according to claim 1,
wherein
10 the identification information is inserted into an
adaptation field of a TS packet including the
adaptation field.
[6] The transmission apparatus according to claim 5,
wherein
15 the identification information is inserted using a
field of an ES priority indicator of the adaptation
field.
[7] The transmission apparatus according to claim 1,
wherein
20 the identification information is inserted into a
box of a header related to a track of an appropriate
picture.
[8] The transmission apparatus according to claim 1,
wherein
25 the image encoding unit
generates a single video stream including the
81
encoded image data of the pictures of each of the
hierarchies or a predetermined number of video data
items each including the encoded image data of the
pictures of each of the hierarchies, and
further includes a configuration 5 information
insertion unit that inserts configuration information
of the video stream included in the container into a
layer of the container.
[9] A transmission method, comprising:
10 an image encoding step of sorting image data of
pictures constituting moving image data into multiple
hierarchies, encoding image data of pictures of each of
the sorted hierarchies, and generating video data
including the encoded image data of the pictures of
15 each of the hierarchies;
a transmission step of transmitting, by a
transmission unit, a container of a predetermined
format including the generated video data; and
an identification information insertion step of
20 dividing the multiple hierarchies into a predetermined
number of hierarchy groups, the predetermined number
being two or more, and inserting identification
information for identifying a hierarchy group to which
encoded image data of each picture included in the
25 video data belongs, into a packet as a container of the
video data.
82
[10] A reception apparatus, comprising:
an reception unit that receives a container of a
predetermined format including video data including
encoded image data of pictures of each of hierarchies,
the encoded image data being obtained 5 by sorting image
data of pictures constituting moving image data into
multiple hierarchies and encoding the image data; and
an image decoding unit that selectively takes in a
buffer encoded image data of a picture of a hierarchy
10 equal to or lower than a predetermined hierarchy
corresponding to a decoding capability from the video
data included in the received container, decodes the
encoded image data of each picture taken in the buffer,
and obtains image data of the picture of the hierarchy
15 equal to or lower than the predetermined hierarchy.
[11] The reception apparatus according to claim 10,
wherein
the multiple hierarchies are divided into a
predetermined number of hierarchy groups, the
20 predetermined number being two or more, and
identification information for identifying a hierarchy
group to which the encoded image data of each picture
included in the video data belongs is inserted into a
packet as a container of the video data, and
25 the image decoding unit takes in the buffer the
encoded image data of the picture of a predetermined
83
hierarchy group corresponding to the decoding
capability and decodes the encoded image data on the
basis of the identification information.
[12] The reception apparatus according to claim 11,
5 wherein
the identification information is inserted into a
header of a PES packet including the encoded image data
for each picture in a payload.
[13] The reception apparatus according to claim 11,
10 wherein
the identification information is inserted into an
adaptation field of a TS packet including the
adaptation field.
[14] The transmission apparatus according to claim 11,
15 wherein
the identification information is inserted into a
box of a header related to a track of an appropriate
picture.
[15] The reception apparatus according to claim 10,
20 wherein
the multiple hierarchies are divided into a
predetermined number of hierarchy groups, the
predetermined number being two or more, and the
received container includes the predetermined number of
25 video streams respectively including encoded image data
of pictures of the predetermined number of hierarchy
84
groups, and
the image decoding unit takes in the buffer the
encoded image data of the picture of the predetermined
hierarchy group corresponding to the decoding
capability and decodes the encoded 5 image data on the
basis of stream identification information.
[16] The reception apparatus according to claim 15,
wherein
the image decoding unit converts, when the encoded
10 image data of the picture of the predetermined
hierarchy group is included in multiple video streams,
the encoded image data of each picture into one scream
on the basis of decoding timing information and takes
the stream in the buffer.
15 [17] The reception apparatus according to claim 10,
wherein
the image decoding unit has a function of
rewriting a decoding time stamp of the encoded image
data of each picture selectively taken in the buffer
20 and adjusting decoding intervals of low hierarchy
pictures.
[18] The reception apparatus according to claim 10,
further comprising a post-processing unit that matches
a frame rate of the image data of each picture obtained
25 in the image decoding unit with the display capability.
[19] A reception method, comprising:
85
a reception step of receiving, by a reception unit,
a container of a predetermined format including video
data including encoded image data of pictures of each
of hierarchies, the encoded image data being obtained
by sorting image data of pictures constituting 5 moving
image data into multiple hierarchies and encoding the
image data; and
an image decoding step of selectively taking in a
buffer encoded image data of a picture of a hierarchy
10 equal to or lower than a predetermined hierarchy
corresponding to a decoding capability from the video
data included in the received container, decoding the
encoded image data of each picture taken in the buffer,
and obtaining image data of the picture of the
15 hierarchy equal to or lower than the predetermined
hierarchy.