TECHNICAL FIELD
[0001]
The present invention relates to a transmission device,
a transmission method, a reception device, and a reception
method, more specifically, to a transmission device that
10 subjects image data of pictures constituting moving image data
to hierarchical encoding and transmits the same, and others.
BACKGROUND ART
[0002]
15 To service compressed moving images by way of
broadcasting, networks, or the like, there is an upper limit
on replayable frame frequency depending on decoding capability
of a receiver. Therefore, service providers need to limit
their services to low-frame frequency services or provide
20 concurrently high-frame frequency services and low-frame
frequency services, with consideration given to replaying
capabilities of the prevailing receivers.
[0003]
To correspond to high-frame frequency services, the
25 receivers become higher in cost, which is a disincentive to
popularization. When inexpensive receivers dedicated to
low-frame frequency services are initially in widespread use
and service providers start high-frame frequency services in
the future, customers cannot receive the high-frame frequency
30 services without new receivers, which is a disincentive to
proliferation of the new services.
3
[0004]
For example, there is proposed time-direction
scalability by subjecting image data of pictures constituting
moving image data to hierarchical encoding by high efficiency
5 video coding (HEVC) (refer to Non-patent Document 1). At the
reception side, the levels of the pictures can be identified
based on temporal ID (temporal_id) information inserted in
the header of a network abstraction layer (NAL) unit, which
allows selective decoding up to the level corresponding to
10 decoding capability.
CITATION LIST
NON-PATENT DOCUMENT
[0005]
15 Non-patent Document 1: Gary J. Sullivan, Jens-Rainer Ohm,
Woo-Jin Han, Thomas Wiegand, "Overview of the High Efficiency
Video Coding (HEVC) Standard" IEEE TRANSACTIONS ON CIRCUITS
AND SYSTEMS FOR VIDEO TECNOROGY, VOL. 22, NO. 12, pp. 1649-1668,
DECEMBER 2012
20
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0006]
An object of the subject technique is to allow favorable
25 decoding at the reception side.
SOLUTIONS TO PROBLEMS
[0007]
A concept of the subject technique lies in a transmission
30 device including:
an image encoding unit that classifies image data of
4
pictures constituting moving image data into a plurality of
levels and encodes the classified image data of the pictures
at the respective levels to generate a video stream having
the encoded image data of the pictures at the respective levels;
5 and
a transmission unit that transmits a container in a
predetermined format containing the generated video stream,
wherein
the image encoding unit equalizes hierarchical
10 composition between a low-level side and a high-level side,
and combines corresponding pictures on the low-level side and
the high-level side into one set and encodes the same
sequentially.
[0008]
15 According to the subject technique, the image encoding
unit encodes the image data of the pictures constituting the
moving image data to generate the video stream (encoded stream).
In this case, the image data of the pictures constituting the
moving image data is classified into a plurality of levels
20 and encoded to generate the video stream having the image data
of the pictures at the respective levels. The hierarchical
composition is equalized between the low-level side and the
high-level side. Corresponding pictures on the low-level
side and the high-level side are combined into one set and
25 are sequentially encoded.
[0009]
The transmission unit transmits the container in the
predetermined format including the foregoing video stream.
For example, the image encoding unit may generate a single
30 video stream having the encoded image data of the pictures
at the respective levels or divide the plurality of levels
5
into two sets of the upper-level side and the lower-level side
and generate two video streams having the encoded image data
of the pictures in the respective level sets.
[0010]
5 According to the subject technique, the hierarchical
composition is equalized between the low-level side and the
high-level side, and corresponding pictures on the low-level
side and the high-level side are combined into one set and
are sequentially encoded. This allows the reception side to
10 decode the encoded image data of the pictures on the low-level
side and the high-level side with a smaller buffer size and
a reduced decoding delay.
[0011]
In the subject technique, for example, a hierarchical
15 information insertion unit that inserts hierarchical
information into a layer of the container may further be
included. In this case, for example, the hierarchical
information may have information on level specified values
for the respective levels. In addition, in this case, for
20 example, the hierarchical information insertion unit may
insert the hierarchical information into the layer of the
container at positions in synchronization with the encoded
image data of the pictures in the video stream.
[0012]
25 For example, the hierarchical information insertion
unit may insert the hierarchical information into an extension
field of a PES packet. In this case, the hierarchical
information insertion unit may insert the hierarchical
information into the extension field of the PES packet at least
30 for each coded video sequence. In addition, in this case,
for example, an information insertion unit that inserts
6
information for describing explicitly whether the
hierarchical information is inserted into the extension field
of the PES packet may further be included under a program map
table.
5 [0013]
In addition, for example, the hierarchical information
insertion unit may insert the hierarchical information under
a program map table. In addition, for example, the
hierarchical information insertion unit may insert the
10 hierarchical information under an event information table.
[0014]
The hierarchy information is inserted in the layer of
the container, and the reception side can refer to the hierarchy
information to retrieve selectively from the video stream the
15 encoded image data of the pictures up to the level commensurate
with the capability of the decoder in an easy manner.
[0015]
In addition, another concept of the subject technique
lies in a reception device including a reception unit that
20 receives a container in a predetermined format that contains
a video stream having encoded image data of pictures obtained
by classifying image data of the pictures constituting moving
image data into a plurality of levels and encoding the same,
wherein
25 in the encoding, hierarchical composition is equalized
between a low-level side and a high-level side, and
corresponding pictures on the low-level side and the high-level
side are combined into one set and are sequentially encoded,
and
30 the reception device further includes a processing unit
that processes the received container.
7
[0016]
According to the subject technique, the reception unit
receives the container in the predetermined format. The
container contains the video stream having image data of the
5 pictures at the respective levels obtained by classifying the
image data of the pictures constituting the moving image data
into a plurality of levels and encoding the same. In this
case, in the process of encoding, the hierarchical composition
is equalized between the low-level side and the high-level
10 side, and corresponding pictures on the low-level side and
the high-level side are combined into one set and are
sequentially encoded.
[0017]
The processing unit processes the received container.
15 For example, the processing unit may be configured to retrieve
selectively the encoded image data of the pictures at a
predetermined level and lower ones from the video stream and
decode the same based on the hierarchy information, thereby
obtaining the image data of the pictures at the predetermined
20 level and lower ones.
[0018]
As described above, according to the subject technique,
in the video stream contained in the received container, the
hierarchical composition is equalized between the low-level
25 side and the high-level side, and corresponding pictures on
the low-level side and the high-level side are combined into
one set and are sequentially encoded. This makes it possible
to decode the encoded image data of the pictures on the low-level
side and the high-level side with a smaller buffer size and
30 a reduced decoding delay.
[0019]
8
According to the subject technique, hierarchical
information may be inserted into a layer of the container,
and
the processing unit may retrieve selectively from the
5 video stream the encoded image data of the pictures at a
predetermined level and lower ones and decode the same, based
on the hierarchical information, to obtain the image data of
the pictures at the predetermined level and lower ones. In
this case, it is easy to retrieve selectively from the video
10 stream the encoded image data of the pictures at the level
commensurate with the capability of the decoder in an easy
manner.
EFFECTS OF THE INVENTION
15 [0020]
According to the subject technique, the reception side
can perform favorable decoding. The advantages of the
technique are not limited to the ones described here but may
be any of advantages described in the subject disclosure.
20
BRIEF DESCRIPTION OF DRAWINGS
[0021]
Fig. 1 is a block diagram of a transmission/reception
system as an embodiment.
25 Fig. 2 is a block diagram of a configuration example
of a transmission device.
Fig. 3 is a diagram illustrating an example of
hierarchical encoding performed by an encoder.
Figs. 4(a) and 4(b) are diagrams illustrating a
30 structural example (Syntax) of an NAL unit header and the
contents (Semantics) of major parameters in the structural
9
example.
Fig. 5 is a diagram illustrating another example of
hierarchical encoding performed by the encoder.
Fig. 6 is a diagram illustrating a configuration example
5 of encoded image data of pictures.
Figs. 7(a) and 7(b) are diagrams illustrating a
structural example (Syntax) of a layer decoding descriptor
(Layer_decoding_descriptor).
Fig. 8 is a diagram illustrating the contents (Semantics)
10 of major information in the structural example of the layer
decoding descriptor.
Fig. 9 is a diagram illustrating a structural example
(Syntax) of a PES extension field data
"pes_extension_field_data()".
15 Figs. 10(a) and 10(b) are diagrams illustrating a
structural example (Syntax) of a PES extension descriptor
(PES_extension_descriptor) and the contents (Semantics) of
major information in the structural example.
Fig. 11 is a diagram of a configuration example of a
20 transport stream TS in the case where single-stream
distribution is performed and hierarchical information is
inserted under a program map table.
Fig. 12 is a diagram of a configuration example of the
transport stream TS in the case where two-stream distribution
25 is performed and hierarchical information is inserted under
the program map table.
Fig. 13 is a diagram of a configuration example of the
transport stream TS in the case where single-stream
distribution is performed and hierarchical information is
30 inserted under an event information table.
Fig. 14 is a diagram of a configuration example of the
10
transport stream TS in the case where two-stream distribution
is performed and hierarchical information is inserted under
the event information table.
Fig. 15 is a diagram of a configuration example of the
5 transport stream TS in the case where single-stream
distribution is performed and hierarchical information is
inserted into an extension field of a PES packet header.
Fig. 16 is a diagram of a configuration example of the
transport stream TS in the case where two-stream distribution
10 is performed and hierarchical information is inserted into
the extension field of the PES packet header.
Fig. 17 is a block diagram illustrating a configuration
example of a reception device.
Fig. 18 is a diagram illustrating an example of
15 correspondence between signaling of "level_idc" and
hierarchical information of coded image data with changes in
hierarchical composition.
MODE FOR CARRYING OUT THE INVENTION
20 [0022]
An embodiment for carrying out the invention
(hereinafter, referred to as "embodiment") will be described
below. The descriptions will be given in the following order:
1. Embodiment
25 2. Modification example
[0023]
<1. Embodiment>
[Transmission/reception system]
Fig. 1 illustrates a configuration example of a
30 transmission/reception system 10 as an embodiment. The
transmission/reception system 10 has a transmission device
11
100 and a reception device 200.
[0024]
The transmission device 100 transmits a transport stream
TS as a container carried on broadcast waves or in packets
5 over a network. The transport stream TS contains a video stream
in which image data of pictures constituting moving image data
is classified into a plurality of levels and encoded data of
the image data of the pictures at the respective levels is
included. In this case, the transport stream TS contains a
10 single video stream having the encoded image data of the
pictures at the respective levels or two video streams in which
the plurality of levels is divided into two sets of high-level
side and low-level side and the encoded image data of the
pictures at the respective level sets is included.
15 [0025]
For example, the referenced pictures are encoded
according to H.264/AVC or H.265/HEVC such that they belong
to their levels and/or lower ones. In this case, the
hierarchical composition is equalized between the low-level
20 side and the high-level side, and corresponding pictures on
the low-level side and the high-level side are combined into
one set and are sequentially encoded. Performing such
encoding allows the reception side to decode the encoded image
data of the pictures on the low-level side and the high-level
25 side with a smaller buffer size and a reduced decoding delay.
[0026]
Level identification information is added to the encoded
image data of the pictures at the respective levels to identify
the levels to which the pictures belong. In the embodiment,
30 the level identification information
("nuh_temporal_id_plus1" indicative of temporal_id) is added
12
to the headers of the NAL units (nal_unit) of the pictures.
Adding the level identification information allows the
reception side to retrieve selectively the encoded image data
at the predetermined level and lower ones for decode
5 processing.
[0027]
Hierarchical information including information on
level specified values at the respective levels and others
is inserted into the layer of the container. The reception
10 side can refer to the hierarchical information to retrieve
selectively from the video stream the encoded image data of
the pictures at the level commensurate with the capability
of the decoder in an easy manner. For example, the hierarchical
information is inserted under a program map table (PMT) or
15 under an event information table (EIT).
[0028]
In addition, the hierarchical information is inserted
into PES extension fields of the headers of PES packets at
positions in synchronization with the encoded image data of
20 the pictures in the video stream, for example. This allows
the reception side to, even with changes in the hierarchical
composition, retrieve selectively from the video stream the
encoded image data of the pictures at the level commensurate
with the capability of the decoder.
25 [0029]
When the hierarchical information is inserted into the
extension field of the PES packet as described above,
identification information indicating that the hierarchical
information is inserted into the extension field of the PES
30 packet is inserted under the program map table. In this case,
the reception side can identify a situation that the
13
hierarchical information is inserted into the extension field
of the PES packet based on the identification information.
[0030]
The reception device 200 receives the transport stream
5 TS sent from the transmission device 100 on broadcast waves
or in packets over a network. The reception device 200
processes the transport stream TS. In this case, the reception
device 200 retrieves selectively from the video stream the
encoded image data of the pictures at a predetermined level
10 and lower ones commensurate with the capability of the decoder
and decodes the same based on the hierarchy information
contained in the layer of the container, thereby obtaining
the image data of the pictures at the predetermined level and
lower ones.
15 [0031]
"Configuration of the transmission device"
Fig. 2 illustrates a configuration example of the
transmission device 100. The transmission device 100 has a
central processing unit (CPU) 101, an encoder 102, a compressed
20 data buffer (cpb: coded picture buffer) 103, a multiplexer
104, and a transmission unit 105. The CPU 101 is a control
unit that controls the operations of the components of the
transmission device 100.
[0032]
25 The encoder 102 inputs uncompressed moving image data
VD to perform hierarchical encoding. The encoder 102
classifies the image data of pictures constituting the moving
image data VD into a plurality of levels. Then, the encoder
102 encodes the classified image data of the pictures at the
30 respective levels to generate a video stream having the encoded
image data of the pictures at the respective levels.
14
[0033]
The encoder 102 performs encoding such as H.264/AVC or
H.265/HEVC. At that time, the encoder 102 performs encoding
such that the referenced pictures belong to their levels and/or
5 lower ones. The encoder 102 also divides the plurality of
levels into low-level side and high-level side, and equalizes
the hierarchical composition between the low-level side and
the high-level side, and combines corresponding pictures on
the low-level side and the high-level side into one set and
10 encodes the same sequentially.
[0034]
Fig. 3 illustrates an example of hierarchical encoding
performed by the encoder 102. In this example, the image data
of the pictures is classified into six levels of 0 to 5, and
15 is subjected to encoding.
[0035]
The vertical axis indicates the levels. The values 0
to 5 are set as the temporal_id (level identification
information) added to the headers of the NAL units (nal_unit)
20 constituting the encoded image data of the pictures at the
levels 0 to 5. Meanwhile, the horizontal axis indicates the
picture order of composition (POC), the display time is earlier
with increasing proximity to the left side and is later with
increasing proximity to the right side.
25 [0036]
Fig. 4(a) illustrates a structural example (Syntax) of
the NAL unit header, and Fig. 4(b) illustrates the contents
(Semantics) of major parameters in the structural example.
The 1-bit field "Forbidden_zero_bit" is essentially 0. The
30 6-bit field "Nal_unit_type" indicates the NAL unit type. The
6-bit field "Nuh_layer_id" is 0 as a precondition. The 3-bit
15
field "Nuh_temporal_id_plus1" indicates temporal_id and
takes the value increased by one (1 to 6).
[0037]
Returning to Fig. 3, rectangular frames indicate
5 pictures, and numbers in the rectangular frames indicate the
order of coded pictures, that is, the encoding order (the
decoding order at the reception side). For example, eight
pictures of "0" to "7" constitute a sub group of pictures,
and the picture "0" becomes the first picture in the sub group
10 of pictures. Several sub groups of pictures are collected
into a group of pictures (GOP).
[0038]
In this example, three levels of 0 to 2 are on the low-level
side and three levels of 3 to 5 are on the high-level side.
15 As illustrated in the drawing, the hierarchical composition
is equalized between the low-level side and the high-level
side, and corresponding pictures on the low-level side and
the high-level side are combined into one set and are
sequentially encoded. For example, the picture "0" on the
20 low-level side and the picture "1" on the high-level side are
first combined into one set and subjected to encoding, and
then the picture "2" on the low-level side and the picture
"3" on the high-level side are combined into one set and
subjected to encoding. The pictures at the following levels
25 are encoded in the same manner. In this case, the low levels
are limited to levels lower than a specific level. Accordingly,
to decode the pictures at the low levels, only the pictures
at the limited low levels can be decoded and displayed in a
stable manner. This matter also applies even when the pictures
30 are not divided into the low levels and the high levels.
[0039]
16
Returning to Fig. 3, solid-line and broken-line arrows
indicate the reference relationships between the pictures in
encoding. For example, the picture "0" is an intra picture
(I picture) that needs no reference to other pictures, and
the picture "5 1" is a P picture that is encoded with reference
to the "1" picture. The picture "2" is a B picture that is
encoded with reference to the "0" picture and a picture in
the previous sub group of pictures (not illustrated). The
picture "3" is a B picture that is encoded with reference to
10 the "0" and "2" pictures. Similarly, the other pictures are
encoded with reference to pictures close to them in the picture
order of composition. The code "D" indicates how much each
picture is distant from the referenced picture in the picture
order of composition. Without the indication of "D," D = 1.
15 [0040]
Fig. 5 illustrates another example of hierarchical
encoding performed by the encoder 102. Although no detailed
description will be provided, in the example of Fig. 3, the
picture order of composition on the high-level side is one
20 picture behind the picture order of composition on the
low-level side, whereas, in the example of Fig. 5, the picture
order of composition on the high-level side is one picture
ahead of the picture order of composition on the low-level
side. In this case, the hierarchical composition is equalized
25 between the low-level side and the high-level side, and
corresponding pictures on the low-level side and the high-level
side are combined into one set and are sequentially encoded.
[0041]
The encoder 102 generates a video stream having the
30 encoded image data of the pictures at the respective levels.
For example, the encoder 102 generates a single video stream
17
having the encoded image data of the pictures at the respective
levels or generates two video streams having the encoded image
data of the pictures on the upper-order level side and the
lower-order level side.
5 [0042]
Fig. 6 illustrates a configuration example of encoded
image data of the pictures. The encoded image data of the
first picture of the GOP is composed of NAL units of AUD, VPS,
SPS, PPS, PSEI, SLICE, SSEI, and EOS. Meanwhile, the pictures
10 other than the first picture of the GOP are composed of NAL
units of AUD, PPS, PSEI, SLICE, SSEI, and EOS. The unit VPS
can be transmitted together with the unit SPS once per sequence
(GOP), and the unit PPS can be transmitted for each picture.
The unit EOS may not exist.
15 [0043]
The bit stream level specified value
"general_level_idc" is inserted into the sequence parameter
set (SPS). In addition, when the pictures belonging to the
levels indicated by "temporal_id" are bound into sub layers
20 (sub_layer) and "Sublayer_level_presented_flag" is set to
"1," the bit rate level specified value for each sub layer
"sublayer_level_idc" can also be inserted into the SPS. This
matter is applied to not only the SPS but also the VPS.
[0044]
25 For example, the example of hierarchical encoding
illustrated in Fig. 3 will be discussed. The value of
"general_level_idc" inserted into the SPS is a level value
including all the pictures at the levels 0 to 5. For example,
when the frame rate is 120P, the value is "Level 5.2." The
30 value of "sublayer_level_idc[2]" inserted into the SPS becomes
a level value including only the pictures at the levels 0 to
18
2. For example, when the frame rate is 60P, the value is "Level
5.1."
[0045]
Returning to Fig. 2, the compressed data buffer (cpb)
5 103 accumulates temporarily the video stream containing the
encoded image data of the pictures at the respective levels
generated by the encoder 102. The multiplexer 104 reads the
video stream accumulated in the compressed data buffer 103,
turns the same into a PES packet, and further turns the same
10 into a transport packet to multiplex the same, thereby
obtaining a transport stream TS as a multiplexed stream. The
transport stream TS contains one or more video streams as
described above.
[0046]
15 The multiplexer 104 inserts the hierarchical
information into the layer of the container. The transmission
unit 105 transmits the transport stream TS obtained by the
multiplexer 104 on broadcast waves or in packets over a network
to the reception device 200.
20 [0047]
[Insertion of the hierarchical information]
The insertion of the hierarchical information by the
multiplexer 104 will be further explained. The multiplexer
104 inserts the hierarchical information to the layer of the
25 container by any of the following methods (A), (B), and (C),
for example:
(A) Insert the hierarchical information under the
program map table (PMT);
(B) Insert the hierarchical information under the event
30 information table (EIT); and
(C) Insert the hierarchical information into the
19
extension field of the header of the PES packet.
[0048]
"Description of (A)"
The transport stream TS contains a PMT as program
5 specific information (PSI). The PMT has a video elementary
loop (video ES1 loop) with information related to each video
stream. In the video elementary loop, information such as
stream type and packet identifier (PID) is arranged and
descriptors describing information related to each video
10 stream are also arranged in correspondence with the video
stream.
[0049]
The multiplexer 104 inserts a layer decoding descriptor
(Layer_decoding_descriptor) newly defined as one of the
15 descriptors. Figs. 7(a) and 7(b) illustrate a structural
example (Syntax) of the layer decoding descriptor. Fig. 8
illustrates the contents (Semantics) of major information in
the structural example.
[0050]
20 The 8-bit field "Layer_decoding_descriptor_tag"
indicates descriptor type, and in this example, layer decoding
descriptor. The 8-bit field
"Layer_decoding_descriptor_length" indicates the length
(size) of the descriptor as the number of bytes of the subsequent
25 "layer_information()."
[0051]
Fig. 7(b) illustrates a structural example (Syntax) of
"layer_information()." The 3-bit field "layer_minimum LMI"
indicates the level (layer) indicated by the minimum value
30 of "temporal_id." The 3-bit field "layer_maximum LMX"
indicates the level (layer) indicated by the maximum value
20
of "temporal_id." In this example, the number of layers to
which "temporal_id" is assigned is (LMX – LMI + 1). The 8-bit
field "layer_level_idc[i]" indicates "level_idc" as level
specified value of the bit rate at each level.
5 [0052]
"Description of (B)"
The transport stream TS also contains EIT as SI (serviced
information) for management of each event. The multiplexer
104 arranges the layer decoding descriptor described above
10 (see Figs. 7(a) and 7(b)) under the EIT. In this case, the
multiplexer 104 also arranges a conventionally known component
descriptor under the EIT to make a link with the PES stream.
[0053]
"Description of (C)"
15 PES extension field can be provided in the header of
the PES packet. The multiplexer 104 inserts PES extension
field data having hierarchical information into the extension
field. In this manner, the multiplexer 104 provides the PES
extension field in the header of the PES packet to insert the
20 PES extension field data having hierarchical information at
least for each coded video sequence (CVS), for example. Fig.
9 illustrates a structural example (Syntax) of the PES
extension field data "pes_extension_field_data()."
[0054]
25 The "PES_extension field length" is given outside the
syntax structure. The 8-bit field "start_sync_byte"
indicates the code value representing the start of the
extension field. The 8-bit field "extension_field_type"
indicates the type of the extension field, which means the
30 supply of hierarchical information in this example. The
"layer_information()" has fields "layer_minimum LMI,"
21
"layer_minimum LMX," and "layer_level_idc[i]," as described
above (see Fig. 7(b)).
[0055]
In this case, the multiplexer 104 arranges a PES
5 extension descriptor (PES_extension_descriptor) as one of the
descriptors in the video elementary loop to describe explicitly
that the hierarchical information is inserted into the PES
extension field.
[0056]
10 Fig. 10(a) illustrates a structural example (Syntax)
of the PES extension descriptor (PES_extension_descriptor).
Fig. 10(b) illustrates the contents (Semantics) of major
information in the structural example. The 8-bit field
"PES_extention_descriptor_tag" indicates the type of the
15 descriptor, which means the PES extension descriptor in this
example.
The 8-bit field "PES_extention_descriptor_length"
indicates the length (size) of the descriptor as the number
of subsequent bytes. The 1-bit field "PES_extension_existed"
20 indicates whether the PES extension field of the applicable
PES stream is encoded. The value "1" indicates that the PES
extension field is encoded, and the value "0" indicates that
the PES extension field is not encoded.
[0057]
25 [Configuration of the transport stream TS]
Fig. 11 illustrates a configuration example of the
transport stream TS in the case where single-stream
distribution is performed and the hierarchical information
is inserted under the program map table (PMT) (the foregoing
30 case (A)). In this configuration example, there is a PES packet
"video PES1" of a video stream having image data encoded by
22
HEVC of pictures at a plurality of levels, for example.
[0058]
The encoded image data of the pictures have NAL units
such as VPS, SPS, PPS, SLICE, and SEI. As described above,
5 the level identification information
("nuh_temporal_id_plus1" indicative of temporal_id) for the
picture is arranged in the headers of the NAL units. The level
specified value of the bit stream "general_level_idc" is
inserted into the SPS. In addition, the pictures belonging
10 to the levels indicated by "temporal_id" are bound into sub
layers (sub_layer) and "Sublayer_level_presented_flag" is
set to "1," whereby the bit rate level specified value for
each sub layer "sublayer_level_idc" is inserted into the SPS.
[0059]
15 The transport stream TS also contains the program map
table (PMT) as program specific information (PSI). The PSI
is information describing to which program each elementary
stream contained in the transport stream belongs.
[0060]
20 The PMT has a program loop describing information related
to the entire program. The PMT also has an elementary loop
with information related to each elementary stream. In this
configuration example, there exists a video elementary loop
(video ES loop).
25 [0061]
In the video elementary loop, information such as stream
type and packet identifier (PID) is arranged in correspondence
with the video stream (video PES1), and descriptors describing
information related to the video stream are also arranged.
30 As one of the descriptors, the layer decoding descriptor
(Layer_decoding_descriptor) described above is inserted.
23
[0062]
For example, in the examples of hierarchical encoding
illustrated in Figs. 3 and 5, the contents described by the
descriptor are as follows: "layer_minimum LMI" = 0,
5 "layer_maximum LMX" = 5, "layer_level_idc[0]" = Level 5,
"layer_level_idc[1]" = Level 5, "layer_level_idc[2]" = Level
5.1, "layer_level_idc[3]" = Level 5.2, "layer_level_idc[4]"
= Level 5.2, and "layer_level_idc[5]" = Level 5.2.
[0063]
10 Fig. 12 illustrates a configuration example of the
transport stream TS in the case where two-stream distribution
is performed and the hierarchical information is inserted under
the program map table (PMT) (the foregoing case (A)). In this
configuration example, a plurality of levels is divided into
15 two sets of low-level side and high-level side, and there exist
PES packets "video PES1" and "video PES2" of the video streams
having image data encoded by HEVC of the pictures of the two
sets, for example.
[0064]
20 The encoded image data of the pictures on the low-level
side have NAL units such as VPS, SPS, PPS, SLICE, and SEI.
The hierarchical identification information
("nuh_temporal_id_plus1" indicative of temporal_id) of the
picture is arranged in the header of the NAL units. The level
25 specified value of the bit stream "general_level_idc" is
inserted into the SPS. In addition, the pictures belonging
to the levels indicated by "temporal_id" are bound into sub
layers (sub_layer) and "sublayer_level_presented_flag" is
set to "1," whereby the bit rate level specified value for
30 each sub layer "sublayer_level_idc" is inserted into the SPS.
[0065]
24
Meanwhile, the encoded image data of the pictures on
the high-level side have NAL units such as PPS and SLICE. The
hierarchical identification information
("nuh_temporal_id_plus1" indicative of temporal_id) of the
5 picture is arranged in the headers of the NAL units.
[0066]
The transport stream TS also contains the program map
table (PMT) as program specific information (PSI). The PSI
is information describing to which program each elementary
10 stream contained in the transport stream belongs.
[0067]
The PMT has a program loop describing information related
to the entire program. The PMT also has an elementary loop
with information related to each elementary stream. In this
15 configuration example, there exist two video elementary loops
(video ES1 loop and video ES2 loop).
[0068]
In the video elementary loop, information such as stream
type and packet identifier (PID) is arranged in correspondence
20 with the video streams (video PES1 and video PES2), and
descriptors describing information related to the video
streams are also arranged. As one of the descriptors, the
layer decoding descriptor (Layer_decoding_descriptor)
described above is inserted.
25 [0069]
For example, in the examples of hierarchical encoding
illustrated in Figs. 3 and 5, the contents described by the
descriptors corresponding to the PES packets "video PES1" and
"video PES2" are as follows: the descriptor corresponding to
30 the PES packet "video PES1" describes "layer_minimum LMI" =
0, "layer_maximum LMX" = 2, "layer_level_idc[0]" = Level 5,
25
"layer_level_idc[1]" = Level 5, and "layer_level_idc[2]" =
Level 5.1; and the descriptor corresponding to the PES packet
"video PES1" describes "layer_minimum LMI" = 3, "layer_maximum
LMX" = 5, "layer_level_idc[3]" = Level 5.2,
5 "layer_level_idc[4]" = Level 5.2, and "layer_level_idc[5]"
= Level 5.2.
[0070]
Fig. 13 illustrates a configuration example of the
transport stream TS in the case where single-stream
10 distribution is performed and the hierarchical information
is inserted under the event information table (EIT) (the
foregoing case (B)). In this configuration example, as in
the configuration example of Fig. 11, there exists the PES
packet "video PES1" of the video stream having the image data
15 encoded by HEVC of the pictures at a plurality of levels, for
example.
[0071]
The transport stream TS contains the program map table
(PMT) as program specific information (PSI). The PSI is
20 information describing to which program each elementary stream
contained in the transport stream belongs.
[0072]
The PMT has a program loop describing information related
to the entire program. The PMT also has elementary loops with
25 information related to each elementary stream. The PMT has
a program loop describing information related to the entire
program. The PMT also has an elementary loop with information
related to each elementary stream. In this configuration
example, there exists a video elementary loop (video ES loop).
30 In the video elementary loop, information such as stream type
and packet identifier (PID) is arranged in correspondence with
26
the video stream (video PES1), and descriptors describing
information related to the video stream are also arranged.
[0073]
The transport stream TS also contains EIT as SI (serviced
5 information) for management of each event. The layer decoding
descriptor (Layer_decoding_descriptor) described above is
arranged under the EIT. Although not explained in detail,
the contents described by the descriptor are the same as those
in the configuration example of Fig. 11. A conventionally
10 known component descriptor is arranged under the EIT to make
a link with the PES packet "video PES1."
[0074]
Fig. 14 illustrates a configuration example of the
transport stream TS in the case where two-stream distribution
15 is performed and the hierarchical information is inserted under
the event information table (EIT) (the foregoing case (B)).
In this configuration example, a plurality of levels is divided
into two sets of low-level side and high-level side, and there
exist PES packets "video PES1" and "video PES2" of the video
20 streams having image data encoded by HEVC of the pictures of
the two sets, for example, as in the configuration example
of Fig. 12.
[0075]
The transport stream TS also contains the program map
25 table (PMT) as program specific information (PSI). The PSI
is information describing to which program each elementary
stream contained in the transport stream belongs.
[0076]
The PMT has a program loop describing information related
30 to the entire program. The PMT also has an elementary loop
with information related to each elementary stream. In this
27
configuration example, there exist two video elementary loops
(video ES1 loop and video ES2 loop). In the video elementary
loop, information such as stream type and packet identifier
(PID) is arranged in correspondence with the video streams
5 (video PES1 and video PES2), and descriptors describing
information related to the video streams are also arranged.
[0077]
The transport stream TS also contains EIT as serviced
information (SI) for management of each event. The layer
10 decoding descriptors (Layer_decoding_descriptor)
corresponding to the PES packets "video PES1" and "video PES2"
are arranged under the EIT. Although not explained in detail,
the contents described by the descriptors are the same as those
in the configuration example of Fig. 12. A conventionally
15 known component descriptor is arranged under the EIT to make
links with the PES packets "video PES1" and "video PES2."
[0078]
Fig. 15 illustrates a configuration example of the
transport stream TS in the case where single-stream
20 distribution is performed and the hierarchical information
is inserted into the extension field of the header of the PES
packet (the foregoing case (C)). In this configuration
example, there is a PES packet "video PES1" of a video stream
having image data encoded by HEVC of pictures at a plurality
25 of levels, for example, as in the configuration example of
Fig. 11.
[0079]
A PES extension field is provided in the header of the
PES packet, and PES extension field data
30 "pes_extension_field_data()" having "layer_information()"
is inserted into the PES extension field. Although not
28
described in detail, the contents described in
"layer_information()" are the same as those described by the
layer decoding descriptor in the configuration example of Fig.
11.
5 [0080]
The transport stream TS also contains the program map
table (PMT) as program specific information (PSI). The PSI
is information describing to which program each elementary
stream contained in the transport stream belongs.
10 [0081]
The PMT has a program loop describing information related
to the entire program. The PMT also has an elementary loop
with information related to each elementary stream. In this
configuration example, there exists a video elementary loop
15 (video ES loop).
[0082]
In the video elementary loop, information such as stream
type and packet identifier (PID) is arranged in correspondence
with the video streams (video PES1 and video PES2), and
20 descriptors describing information related to the video
streams are also arranged. As one of the descriptors, a PES
extention descriptor (PES_extention_descriptor) is inserted.
The PES extention descriptor is a descriptor to describe
explicitly that the hierarchical information is inserted into
25 the PES extension field.
[0083]
Fig. 16 illustrates a configuration example of the
transport stream TS in the case where two-stream distribution
is performed and the hierarchical information is inserted into
30 the extension field of the header of the PES packet (the
foregoing case (C)). In this configuration example, a
29
plurality of levels is divided into two sets of low-level side
and high-level side, and there exist PES packets "video PES1"
and "video PES2" of the video streams having image data encoded
by HEVC of the pictures of the two sets, for example, as in
5 the configuration example of Fig. 12.
[0084]
A PES extension field is provided in the header of the
PES packet "video PES1", and PES extension field data
"pes_extension_field_data()" having "layer_information()"
10 is inserted into the PES extension field. Although not
described in detail, the contents described in
"layer_information()" are the same as those described by the
layer decoding descriptor corresponding to the PES packet
"video PES1" in the configuration example of Fig. 12.
15 [0085]
A PES extension field is provided in the header of the
PES packet "video PES2", and PES extension field data
"pes_extension_field_data()" having "layer_information()"
is inserted into the PES extension field. Although not
20 described in detail, the contents described in
"layer_information()" are the same as those described by the
layer decoding descriptor corresponding to the PES packet
"video PES2" in the configuration example of Fig. 12.
[0086]
25 The transport stream TS also contains the PMT (program
map table) as PSI (program specific information). The PSI
is information describing to which program each elementary
stream contained in the transport stream belongs.
[0087]
30 The PMT has a program loop describing information related
to the entire program. The PMT also has an elementary loop
30
with information related to each elementary stream. In this
configuration example, there exist two video elementary loops
(video ES1 loop and video ES2 loop).
[0088]
5 In the video elementary loop, information such as stream
type and packet identifier (PID) is arranged in correspondence
with the video streams (video PES1 and video PES2), and
descriptors describing information related to the video
streams are also arranged. As one of the descriptors, a PES
10 extention descriptor (PES_extention_descriptor) is inserted.
The PES extention descriptor is a descriptor to describe
explicitly that the hierarchical information is inserted into
the PES extension field.
[0089]
15 Operation of the transmission device 100 illustrated
in Fig. 2 will be briefly described. Uncompressed moving image
data VD is input into the encoder 102. The encoder 102 subjects
the moving image data VD to hierarchical encoding.
Specifically, the encoder 102 classifies image data of pictures
20 constituting the moving image data VD into a plurality of levels
and encodes the same, thereby generating a video stream having
encoded image data of the pictures at the respective levels.
[0090]
In this case, the referenced pictures are encoded such
25 that they belong to their levels and/or lower ones. In this
case, a plurality of levels is divided into two of low-level
side and high-level side, and the hierarchical composition
is equalized between the low-level side and the high-level
side, and corresponding pictures on the low-level side and
30 the high-level side are combined into one set and are
sequentially encoded. Also in this case, a single video stream
31
having the encoded image data of the pictures at the respective
levels is generated, or two video streams having the encoded
image data of the pictures on the upper-order level side and
the lower-order level side are generated.
5 [0091]
The video stream generated by the encoder 102 and
containing the encoded data of pictures at the respective
levels is supplied to the compressed data buffer (cpb) 103
and is temporarily accumulated there. The multiplexer 104
10 reads the video stream from the compressed data buffer 103,
turns the same into PES packet, further turns the same into
transport packet for multiplexing, thereby obtaining the
transport stream TS as a multiplexed stream. The transport
stream TS contains one or more video streams as described above.
15 [0092]
When the multiplexer 104 generates the transport stream
TS, the hierarchical information is inserted in the layer of
the container under the program map table (PMT), under the
event information table (EIT), or in the extension field of
20 the header of the PES packet. The transport stream TS generated
by the multiplexer 104 is sent to the transmission unit 105.
The transmission unit 105 transmits the transport stream TS
on broadcast waves or in packets over a network to the reception
device 200.
25 [0093]
"Configuration of the reception device"
Fig. 17 illustrates a configuration example of the
reception device 200. The reception device 200 has a central
processing unit (CPU) 201, a reception unit 202, a
30 demultiplexer 203, and a compressed data buffer (cpb: coded
picture buffer) 204. The reception device 200 also has a
32
decoder 205, a decompressed data buffer (dpb: decoded picture
buffer) 206, a post-processing unit 207, and a display unit
208. The CPU 201 constitutes a control unit that controls
operations of the components of the reception device 200.
5 [0094]
The reception unit 202 receives the transport stream
TS on broadcast waves or in packets over a network transmitted
from the transmission device 100. The demultiplexer 203
retrieves selectively from the transport stream TS the encoded
10 image data of the pictures at the level commensurate with the
capability of the decoder 205, and sends the same to the
compressed data buffer (cpb: coded picture buffer) 204. In
this case, the demultiplexer 203 refers to the value of
"nuh_temporal_id_plus1" indicative of "temporal_id" arranged
15 in the headers of the NAL units (nal_unit) of the pictures.
[0095]
In this case, the demultiplexer 203 extracts the
hierarchical information inserted in the layer of the container,
recognizes "layer_level_idc" at the respective levels from
20 the hierarchical information, and detects up to which level
decoding is enabled according to the capability of the decoder
205. For example, in the examples of hierarchical encoding
of Figs. 3 and 5, it is assumed that "layer_level_idc[0]" =
Level 5, "layer_level_idc[1]" = Level 5, "layer_level_idc[2]"
25 = Level 5.1, "layer_level_idc[3]" = Level 5.2,
"layer_level_idc[4]" = Level 5.2, and "layer_level_idc[5]"
= Level 5.2. In this case, when the decoder 205 has a capability
of 60P, that is, "Level 5.1," the demultiplexer 203 detects
that decoding is enabled up to the level 2. In addition, in
30 this case, when the decoder 205 has a capability of 120P, that
is, "Level 5.2," the demultiplexer 203 detects that decoding
33
is enabled up to the level 5.
[0096]
The compressed data buffer (cpb) 204 accumulates
temporarily the encoded image data of pictures at the
5 respective levels sent from the demultiplexer 203. The
decoder 205 reads and decodes the encoded image data of the
pictures accumulated in the compressed data buffer 204 at
decode timings given by decoding time stamps (DTS) of the
pictures, and sends the same to the decompressed data buffer
10 (dpb) 206.
[0097]
The decompressed data buffer (dpb) 206 accumulates
temporarily the image data of the pictures decoded by the
decoder 205. The post-processing unit 207 matches the frame
15 rate for the image data of the pictures read sequentially at
display timings given by presentation time stamps (PTS) from
the decompressed data buffer (dpb) 206 with the display
capability.
[0098]
20 For example, when the frame rate of image data of the
pictures after decoding is 60 fps and the display capability
is 120 fps, the post-processing unit 207 performs interpolation
in the image data of the pictures after decoding such that
the time-direction resolution becomes doubled, and sends the
25 same as image data of 120 fps to the display unit 208.
[0099]
The display unit 208 is composed of a liquid crystal
display (LCD), an organic electro-luminescence (EL) panel,
or the like, for example. The display unit 208 may be an
30 external device connected to the reception device 200.
[0100]
34
Operations of the reception device 200 illustrated in
Fig. 17 will be described briefly. The reception unit 202
receives the transport stream TS on broadcast waves or in
packets over a network from the transmission device 100. The
5 transport stream TS is sent to the demultiplexer 203. The
demultiplexer 203 retrieves selectively from the transport
stream TS the encoded image data of pictures at the level
commensurate with the capability of the decoder 205 based on
the hierarchical information inserted in the layer of the
10 container, and sends the same to the compressed data buffer
(cpb) 204 for temporary accumulation.
[0101]
The decoder 205 retrieves the encoded image data of
pictures at the respective levels accumulated in the compressed
15 data buffer 204. The decoder 205 then decodes the retrieved
encoded image data of the pictures at the respective decode
timings for the pictures, sends the same to the decompressed
data buffer (dpb) 206 for temporary accumulation.
[0102]
20 Then, the image data of the pictures read sequentially
at the display timings from the decompressed data buffer (dpb)
206 is sent to the post-processing unit 207. The
post-processing unit 207 subjects the image data of the
pictures to interpolation or sub sampling to match the frame
25 rate with the display capability. The image data of the
pictures processed by the post-processing unit 207 is supplied
to the display unit 208 for display of moving images.
[0103]
As described above, in the transmission/reception
30 system 10 illustrated in Fig. 1, the transmission device 100
equalizes the hierarchical composition between the low-level
35
side and the high-level side, and combines corresponding
pictures on the low-level side and the high-level side into
one set and encodes the same sequentially. Accordingly, the
reception device 200 can decode the encoded image data of the
pictures on 5 the low-level side and the high-level side at one
collective timing, thereby reducing the buffer size and
decreasing decode delay.
[0104]
In addition, in the transmission/reception system 10
10 illustrated in Fig. 1, the transmission device 100 inserts
the hierarchical information into the layer of the container
to generate a transport stream containing a video stream having
the image data of the encoded pictures at the respective levels.
Accordingly, the reception device 200 can refer to the
15 hierarchical information to retrieve selectively from the
video stream the encoded image data of the pictures up to the
level commensurate with the capability of the decoder in an
easy manner, for example.
[0105]
20 In the transmission/reception system 10 illustrated in
Fig. 1, the transmission device 100 inserts the hierarchical
information into the PES extension field of the header of the
PES packet in the position synchronized with the encoded image
data of the pictures of the video stream at least for each
25 coded video sequence (CVS). This allows the reception side
to, even with changes in the hierarchical composition, retrieve
selectively from the video stream the encoded image data of
the pictures up to the level commensurate with the capability
of the decoder.
30 [0106]
Fig. 18 illustrates an example of correspondence between
36
signaling of "level_idc" and hierarchical information of
encoded image data with changes in hierarchical composition.
In this example, the hierarchical composition changes from
a first 50P CVS system in which encoding is performed at three
5 levels of 0 to 2 to a second 50P CVS system in which encoding
is performed at four levels of 0 to 3, and further changes
to a 100P CVS system in which encoding is performed at six
levels of 0 to 5. In the illustrated example, the hierarchical
information is inserted under the PMT. However, the foregoing
10 matter also applies to the case where the hierarchical
information is inserted under the EIT or into the PES extension
field as described above.
[0107]
In the period of the first 50P CVS system, data is
15 distributed in a single video stream. The value of
"general_level_idc" inserted into the SPS of the encoded image
data is set to "Level 5.1" as a level value containing all
the pictures at the levels of 0 to 2. The value of
"sublayer_level_idc[1]" as the level specified value of bit
20 rate at the level of 1 is set to "Level 5." In this case,
the hierarchical information is described as
"layer_level_idc[0]" = Level 4.1, "layer_level_idc[1]" =
Level 5, and "layer_level_idc[2]" = Level 5.1."
[0108]
25 In the period of the second 50P CVS system, data is
distributed in a single video stream. The value of
"general_level_idc" inserted into the SPS of the encoded image
data is set to "Level 5.1" as a level value containing all
the pictures at the levels of 0 to 3. The value of
30 "sublayer_level_idc[2]" as the level specified value of bit
rate at the level of 2 is set to "Level 5." In this case,
37
the hierarchical information is described as
"layer_level_idc[0]" = Level 4, "layer_level_idc[1]" = Level
4.1, "layer_level_idc[2]" = Level 5", and
"layer_level_idc[3]" = Level 5.1."
5 [0109]
In the period of the 100P CVS system, data is distributed
in two video streams. The value of "general_level_idc"
inserted into the SPS of the encoded image data is set to "Level
5.2" as a level value containing all the pictures at the levels
10 of 0 to 5. The value of "sublayer_level_idc[2]" as the level
specified value of bit rate at the level of 2 is set to "Level
5.1." In this case, the hierarchical information is described
as "layer_level_idc[0]" = Level 4.1, "layer_level_idc[1]"
=Level 5, "layer_level_idc[2]" = Level 5.1,
15 "layer_level_idc[3]" = Level 5.2, "layer_level_idc[4]" =
Level 5.2, and "layer_level_idc[5]" = Level 5.2."
[0110]
While the hierarchical composition changes as
illustrated in the drawing, if the decoder 205 of the reception
20 device 200 corresponds to 50P, for example, the demultiplexer
203 retrieves the levels of 0 to 2 in the period of the first
50P CVS system, retrieves the levels of 0 to 3 in the second
50P CVS system, and retrieves the levels of 0 to 2 in the period
of the 100P CVS system, based on the hierarchical information,
25 and sends the same to the compressed data buffer 204. The
decoder 205 decodes the encoded image data of the pictures
at their respective decode timings to obtain 50P image data.
[0111]
<2. Modification example>
30 In the foregoing embodiment, the
transmission/reception system 10 is composed of the
38
transmission device 100 and the reception device 200. However,
the configuration of the transmission/reception system to
which the subject technique is applicable is not limited to
this. For example, part of the reception device 200 may be
5 formed as a set-top box and a monitor connected via a digital
interface such as a high-definition multimedia interface
(HDMI). The "HDMI" is a registered trademark.
[0112]
In the foregoing embodiment, the container is a transport
10 stream (MPEG-2 TS). However, the subject technique is also
applicable to other systems in which data is distributed to
reception terminals via a network such as the Internet. In
the Internet delivery, data is frequently distributed by a
container in MP4 or other formats. That is, the container
15 may be a transport stream (MPEG-2 TS) employed under digital
broadcasting standards, or any other container in various
formats such as MP4 used in the Internet delivery.
[0113]
The subject technique may be configured as described
20 below.
(1) A transmission device including:
an image encoding unit that classifies image data of
pictures constituting moving image data into a plurality of
levels and encodes the classified image data of the pictures
25 at the respective levels to generate a video stream having
the encoded image data of the pictures at the respective levels;
and
a transmission unit that transmits a container in a
predetermined format containing the generated video stream,
30 wherein
the image encoding unit equalizes hierarchical
39
composition between a low-level side and a high-level side,
and combines corresponding pictures on the low-level side and
the high-level side into one set and encodes the same
sequentially.
5 (2) The transmission device according to (1), further including
a hierarchical information insertion unit that inserts
hierarchical information into a layer of the container.
(3) The transmission device according to (2), wherein
the hierarchical information has information on level
10 specified values for the respective levels.
(4) The transmission device according to (2) or (3), wherein
the hierarchical information insertion unit inserts the
hierarchical information into the layer of the container at
positions in synchronization with the encoded image data of
15 the pictures in the video stream.
(5) The transmission device according to (4), wherein
the hierarchical information insertion unit inserts the
hierarchical information into an extension field of a PES
packet.
20 (6) The transmission device according to (5), wherein
the hierarchical information insertion unit inserts the
hierarchical information into the extension field of the PES
packet at least for each coded video sequence.
(7) The transmission device according to (5) or (6), further
25 including
an information insertion unit that inserts information
for describing explicitly whether the hierarchical
information is inserted into the extension field of the PES
packet under a program map table.
30 (8) The transmission device according to (2) or (3), wherein
the hierarchical information insertion unit inserts the
40
hierarchical information under a program map table.
(9) The transmission device according to (2) or (3), wherein
the hierarchical information insertion unit inserts the
hierarchical information under an event information table.
5 (10) The transmission device according to any of (1) to (9),
wherein
the image encoding unit generates a single video stream
having the encoded image data of the pictures at the respective
levels or divides the plurality of levels into two sets of
10 the upper-level side and the lower-level side and generates
two video streams having the encoded image data of the pictures
in the respective level sets.
(11) A transmission method including:
an image encoding step of classifying image data of
15 pictures constituting moving image data into a plurality of
levels and encoding the classified image data of the pictures
at the respective levels to generate a video stream having
the encoded image data of the pictures at the respective levels;
and
20 a transmission step by a transmission unit of
transmitting a container in a predetermined format containing
the generated video stream, wherein
at the image encoding step, hierarchical composition
is equalized between a low-level side and a high-level side,
25 and corresponding pictures on the low-level side and the
high-level side are combined into one set and are sequentially
encoded.
(12) A reception device including a reception unit that
receives a container in a predetermined format that contains
30 a video stream having encoded image data of pictures obtained
by classifying image data of the pictures constituting moving
41
image data into a plurality of levels and encoding the same,
wherein
in the encoding, hierarchical composition is equalized
between a low-level side and a high-level side, and
5 corresponding pictures on the low-level side and the high-level
side are combined into one set and are sequentially encoded,
and
the reception device further includes a processing unit
that processes the received container.
10 (13) The reception device according to (12), wherein
hierarchical information is inserted into a layer of
the container, and
the processing unit retrieves selectively from the video
stream the encoded image data of the pictures at a predetermined
15 level and lower ones and decodes the same, based on the
hierarchical information, to obtain the image data of the
pictures at the predetermined level and lower ones.
(14) A reception method including a reception step by a
reception unit of receiving a container in a predetermined
20 format containing a video stream that has encoded image data
of pictures at a plurality of levels obtained by classifying
image data of the pictures constituting moving image data into
the respective levels and encoding the same,
in the encoding, hierarchical composition is equalized
25 between a low-level side and a high-level side, and
corresponding pictures on the low-level side and the high-level
side are combined into one set and are sequentially encoded,
and
the reception method further includes a processing step
30 of processing the received container.
42
INDUSTRIAL APPLICABILITY
[0114]
A main feature of the subject technique is in that the
hierarchical composition is equalized between the low-level
5 side and the high-level side, and corresponding pictures on
the low-level side and the high-level side are combined into
one set and are sequentially encoded, thereby allowing the
reception side to decode the encoded image data of the pictures
on the low-level side and the high-level side with a smaller
10 buffer size and a reduced decoding delay (see Figs. 3 and 5).
Another main feature of the subject technique is in that the
hierarchical information is inserted into the layer of the
container to allow the reception side to retrieve selectively
from the video stream the encoded image data of the pictures
15 up to the level commensurate with the capability of the decoder
in an easy manner (see Figs. 7(a) and 7(b), and Figs. 11 to
16).
REFERENCE SIGNS LIST
20 [0115]
10 Transmission/reception system
100 Transmission device
101 CPU
102 Encoder
25 103 Compressed data buffer (cpb)
104 Multiplexer
105 Transmission unit
200 Reception device
201 CPU
30 202 Reception unit
203 Demultiplexer
43
204 Compressed data buffer (cpb)
205 Decoder
206 Decompressed data buffer (dpb)
207 Post-processing unit
5 208 Display unit
44
CLAIMS
1. A transmission device comprising an image encoding unit
that subjects image data of pictures constituting moving image
5 data to hierarchical encoding to generate a first stream having
encoded image data of pictures on a low-level side and a second
stream having encoded image data of pictures on a high-level
side, wherein
the encoded image data has an NAL unit structure, and
10 the image encoding unit inserts a level specified value of
the first stream together with a level specified value of a
bit stream formed from a combination of the first stream and
the second stream into an NAL unit of SPS of the first stream,
and
15 the transmission unit further comprises:
a multiplexer that multiplexes the first stream and the
second stream generated by the image encoding unit to obtain
a container in a predetermined format, inserts a first
descriptor describing the level specified value of the first
20 stream into the container in correspondence with the first
stream, and inserts a second descriptor describing the level
specified value of the bit stream formed from a combination
of the first stream and the second stream into the container
in correspondence with the second stream; and
25 a transmission unit that transmits the container in the
predetermined format obtained by the multiplexer.
2. The transmission device according to claim 1, wherein
the first descriptor further describes maximum and minimum
30 information on the levels of the pictures contained in the
first stream, and the second descriptor further describes
45
maximum and minimum information on the levels of the pictures
contained in the second stream.
3. The transmission device according to claim 1, wherein
5 the container is a transport stream, and
the first descriptor and the second descriptor are
inserted under a program map table.
4. A transmission method comprising an image encoding step
10 by an image encoding unit of subjecting image data of pictures
constituting moving image data to hierarchical encoding to
generate a first stream having encoded image data of pictures
on a low-level side and a second stream having encoded image
data of pictures on a high-level side, wherein
15 the encoded image data has an NAL unit structure, and
at the image encoding step, the image encoding unit inserts
a level specified value of the first stream together with a
level specified value of a bit stream formed from a combination
of the first stream and the second stream into an NAL unit
20 of SPS of the first stream, and
the transmission method further comprises:
a multiplexing step by a multiplexer of multiplexing
the first stream and the second stream generated at the image
encoding step to obtain a container in a predetermined format,
25 inserting a first descriptor describing the level specified
value of the first stream into the container in correspondence
with the first stream, and inserting a second descriptor
describing the level specified value of the bit stream formed
from a combination of the first stream and the second stream
30 into the container in correspondence with the second stream;
and
46
a transmission step by a transmission unit of
transmitting the container in the predetermined format
obtained at the multiplexing step.
5 5. A reception device comprising a reception unit that
receives a container in a predetermined format containing a
first stream having encoded image data of pictures on a
low-level side and a second stream having encoded image data
of pictures on a high-level side generated by subjecting image
10 data of the picture constituting moving image data to
hierarchical encoding, and containing a first descriptor
describing a level specified value of the first stream in
correspondence with the first stream and a second descriptor
describing a level specified value of a bit stream formed from
15 a combination of the first stream and the second stream in
correspondence with the second stream,
the encoded image data has an NAL unit structure, and
the level specified value of the first stream is inserted
together with the level specified value of the bit stream formed
20 from a combination of the first stream and the second stream
into an NAL unit of SPS of the first stream, and
the reception device further includes a processing unit
that decodes the first stream or both the first stream and
the second stream contained in the received container depending
25 on decoding capability to obtain the image data of the pictures
constituting moving image data.
6. A reception method comprising a reception step by a
reception unit of receiving a container in a predetermined
30 format containing a first stream having encoded image data
of pictures on a low-level side and a second stream having
47
encoded image data of pictures on a high-level side generated
by subjecting image data of the pictures constituting moving
image data to hierarchical encoding, and containing a first
descriptor describing a level specified value of the first
5 stream in correspondence with the first stream and a second
descriptor describing a level specified value of a bit stream
formed from a combination of the first stream and the second
stream in correspondence with the second stream,
the encoded image data has an NAL unit structure, and
10 the level specified value of the first stream is inserted
together with the level specified value of the bit stream formed
from a combination of the first stream and the second stream
into an NAL unit of SPS of the first stream, and
the reception method further comprises a processing step
15 by a processing unit of decoding the first stream or both the
first stream and the second stream contained in the received
container depending on decoding capability to obtain the image
data of the pictures constituting moving image data.
20 7. The transmission device according to claim 1, wherein
flag information indicative of insertion of the level specified
value of the first stream is further inserted into the NAL
unit of the SPS.