FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEPTION DEVICE, AND
RECEPTION METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-0075, Japan
The following specification particularly describes the invention and the manner in which it is to
be performed.
2
DESCRIPTION
TRANSMISSION APPARATUS, TRANSMISSION METHOD, RECEPTION
APPARATUS, AND RECEPTION METHOD
5 TECHNICAL FIELD
[0001]
The present technology relates to a transmission
apparatus, a transmission method, a reception apparatus, and
a reception method, and relates to a transmission apparatus
10 or the like which hierarchically encodes and transmits image
data of each picture consisting moving picture data.
BACKGROUND ART
[0002]
15 When a service of a compressed moving picture is provided
by broadcasting, the internet, or the like, the upper limit
of a reproducible frame frequency is defined by the decoding
capability of receivers. Thus, service sides need to restrict
services having a low frame frequency and simultaneously
20 provide services having a plurality of high and low frame
frequencies taking the reproducible capability of widespread
receivers into consideration.
[0003]
The cost of a receiver is too high to deal with a service
25 having a high frame frequency, which is an obstruction factor
of spreading. When service sides start a service having a
high frame frequency although inexpensive receivers dedicated
for services having a low frame frequency have been widespread,
the service having a high frame frequency can never be viewed
30 without a new receiver, which is an obstruction factor of
spreading of the service.
3
[0004]
For example, in High Efficiency Video Coding (HEVC),
a time direction scalability by hierarchically encoding image
data of each picture consisting moving picture data has been
presented (Non Patent Document 1). A 5 receiving side can
identify, based on a temporal ID (temporal_id) inserted in
a header of a network abstraction layer (NAL) unit, the layer
of each picture, and can selectively perform decoding to a
layer corresponding to decoding capability.
10
CITATION LIST
NON PATENT DOCUMENT
[0005]
Non Patent Document 1: Gary J. Sullivan, Jens-Rainer Ohm,
15 Woo-Jin Han, Thomas Wiegand, "Overview of the High Efficiency
Video Coding (HEVC) Standard" IEEE TRANSACTIONS ON CIRCUITS
AND SYSTEMS FOR VIDEO TECNOROGY, VOL. 22, NO. 12, pp. 1649-1668,
DECEMBER 2012
20 SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0006]
A purpose of the present technology is to enable a
receiving side to perform excellent decode processing
25 according to decoding capability.
SOLUTIONS TO PROBLEMS
[0007]
A concept of the present technology is
30 a transmission apparatus including an image encoding
unit which classifies image data of each picture consisting
4
moving picture data into a plurality of layers, encodes the
classified image data of the picture in each of the plurality
of layers, and generates a video stream having the encoded
image data of the picture in each of the plurality of layers,
a data transmission unit which transmits 5 the video data,
and an information transmission unit which transmits
a level designation value of a bit stream and information on
a layer range in each of a plurality of layer ranges having
a different maximum layer.
10 [0008]
In the present technology, image data of each picture
consisting moving picture data is encoded and video data is
generated by the image encoding unit. In this case, the image
data of each picture consisting the moving picture data is
15 classified into a plurality of layers and encoded, and a data
stream having the image data of the picture in each of the
plurality of layers is generated. The video data is
transmitted by the transmission unit.
[0009]
20 A level designation value of a bit stream and information
on a layer range in each of a plurality of layer ranges having
a different maximum layer are transmitted by the information
transmission unit. For example, the information transmission
unit may insert the information in a layer of a container
25 containing the video data and transmit the information. In
this case, for example, the container is an MPEG2-TS, and the
information transmission unit may insert the information under
a program map table, and transmit the information.
Furthermore, for example, the information transmission unit
30 may insert the information in a metafile having
meta-information related to the video data, and transmit the
5
information. In this case, for example, the metafile may be
an MPD file.
[0010]
As described above, in the present technology, a level
designation value of a bit stream and information 5 on a layer
range in each of a plurality of layer ranges having a different
maximum layer are transmitted. Thus, it is possible for a
receiving side to easily decode encoded image data of a picture
in a layer range according to decoding performance.
10 [0011]
Note that, in the present technology, the information
transmission unit may transmit, together with information on
a profile, the level designation value of the bit stream and
the information on the layer range in each of the plurality
15 of layer ranges having a different maximum layer. In this
case, it is possible for the receiving side to confirm the
layer range according to the decoding performance on the basis
of the information on the profile in addition to the level
designation value of the bit stream.
20 [0012]
Furthermore, another concept of the present technology
is
a reception apparatus including a data reception unit
which receives video data having encoded image data of a picture
25 in each of a plurality of layers obtained by classifying image
data of each picture consisting moving picture data into the
plurality of layers and encoding the image data,
an information reception unit which receives a level
designation value of a bit stream and information on a layer
30 range in each of a plurality of layer ranges having a different
maximum layer,
6
and a processing unit which extracts, from the video
data, the image data of the picture in a layer lower than a
predetermined layer and decodes the image data on the basis
of the information.
5 [0013]
In the present technology, video data is received by
the reception unit. The video data has encoded image data
of a picture in each of a plurality of layers obtained by
classifying image data of each picture consisting moving
10 picture data into the plurality of layers and encoding the
image data.
[0014]
A level designation value of a bit stream and information
on a layer range in each of a plurality of layer ranges having
15 a different maximum layer are received by the information
reception unit. For example, the information reception unit
may acquire the information from a layer of a container
containing the video data. Furthermore, for example, the
information reception unit may acquire the information from
20 a metafile having meta-information related to the video data.
The image data of the picture in a layer lower than a
predetermined layer is extracted from the video data and
decoded on the basis of the information by the processing unit.
[0015]
25 As described above, in the present technology, a level
designation value of a bit stream and information on a layer
range in each of a plurality of layer ranges having a different
maximum layer are received, and image data of the picture in
a layer lower than a predetermined layer is extracted from
30 the video data and decoded on the basis of the information.
Thus, it is possible to easily decode encoded image data of
7
a picture in a layer range according to decoding performance.
EFFECTS OF THE INVENTION
[0016]
According to the present technology, 5 it is possible for
a receiving side to perform excellent decode processing
according to decoding capability. Note that, effects are not
necessarily limited to the effects described here and may be
any one of the effects described in the present disclosure.
10
BRIEF DESCRIPTION OF DRAWINGS
[0017]
Fig. 1 is a block diagram illustrating a configuration
example of a transceiver system of an embodiment.
15 Figs. 2(a) and 2(b) are structure examples of a NAL unit
header and a diagram illustrating semantics of a main parameter
in the structure example.
Figs. 3(a) to 3(c) are diagrams illustrating an example
of hierarchical encoding.
20 Fig. 4 is a diagram illustrating "level_idc" and an
example of the value.
Fig. 5 is a block diagram illustrating a configuration
example of a transmission apparatus.
Fig. 6 is a diagram illustrating a structure example
25 of an element-added HEVC descriptor.
Fig. 7 is a diagram illustrating a structure example
of a layer/signaling/descriptor.
Fig. 8 is a diagram illustrating semantics of main
information in the structure example of the
30 layer/signaling/descriptor.
Figs. 9(a) to 9(c) are diagrams explaining a
8
configuration example of a profile layer.
Fig. 10 is a diagram illustrating a configuration example
of a transport stream TS.
Fig. 11 is a block diagram illustrating a configuration
example of a reception 5 apparatus.
Fig. 12 is a diagram schematically illustrating a
configuration of a demultiplexer and a decoder.
Fig. 13 is a diagram schematically illustrating behavior
of a temporal ID analysis unit.
10 Fig. 14 is an example of a processing flow of a decoder
and a post-processing unit.
Figs. 15(a) and 15(b) are diagrams illustrating a
configuration example of an MPEG-DASH-based stream
distribution system.
15 Figs. 16(a) to 16(d) are diagrams illustrating an example
of a relation between structures hierarchically arranged in
an MPD file.
Fig. 17 is a diagram illustrating a configuration example
of a FragmentedMP4 stream in association with an MPD file.
20 Fig. 18 is a diagram illustrating an implementation of
an MPD file.
MODE FOR CARRYING OUT THE INVENTION
[0018]
25 Hereinafter, mode for carrying out the invention
(hereinafter, referred to as an "embodiment") is described.
Note that, the description is made in the following order:
1. Embodiment
2. Modified example
30 [0019]
<1. Embodiment>
9
[Transceiver system]
Fig. 1 illustrates a configuration example of a
transceiver system 10 of an embodiment. The transceiver
system 10 includes a transmission apparatus 100 and a reception
5 apparatus 200.
[0020]
The transmission apparatus 100 transmits a transport
stream TS as a container on a broadcast wave. The transport
stream TS contains video data having encoded image data of
10 a picture in each of a plurality of layers obtained by
classifying image data of the picture consisting moving picture
data into the plurality of layers and encoding the image data.
In this case, by, for example, performing encoding such as
H.264/AVC or H.265/HEVC, the image data is encoded so that
15 a referred picture belongs to the own layer and/or a layer
lower than the own layer.
[0021]
Layer identification information is added to the encoded
image data of the picture of each of the layers in order to
20 identify the layer to which each picture belongs. In this
embodiment, the layer identification information
("nuh_temporal_id_plus1" indicating temporal_id) is arranged
in a header part of a NAL unit (nal_unit) of each picture.
By adding the layer identification information in this manner,
25 it is possible for a receiving side to selectively extract
encoded image data in a layer lower than a predetermined layer
and perform decode processing.
[0022]
Fig. 2(a) illustrates a structure example (Syntax) of
30 a NAL unit header, and Fig. 2(b) illustrates semantics of a
main parameter in the structure example. A 1-bit field of
10
"Forbidden_zero_bit" is necessarily 0. A 6-bit field of
"Nal_unit_type" indicates a NAL unit type. A 6-bit field of
"Nuh_layer_id" assumes 0. A 3-bit field of
"Nuh_temporal_id_plus1" indicates temporal_id, and takes a
value obtained by 5 adding 1 (1 to 7).
[0023]
The transport stream TS contains a single video stream.
Furthermore, a level designation value of a bit stream and
information on a layer range in each of a plurality of layer
10 ranges having a different maximum layer are inserted in the
transport stream TS. The information is inserted, for example,
under a program map table (PMT).
[0024]
Figs. 3(a) to 3(c) illustrate an example of hierarchical
15 encoding, and each rectangular frame indicates a picture. Fig.
3(a) is an example in which the uppermost layer is 3. For
example, a level designation value of a bit stream of a layer
range whose maximum layer is layer 3 is "level5.2"
corresponding to a service of 2160/100P, and the information
20 on the layer range indicates 0 to 3. Furthermore, for example,
a level designation value of a bit stream of a layer range
whose maximum layer is layer 2 is "level5.1" corresponding
to a service of 2160/50P, and the information on the layer
range indicates 0 to 2.
25 [0025]
Fig. 3(b) is an example in which the uppermost layer
is 3. For example, a level designation value of a bit stream
of a layer range whose maximum layer is layer 3 is "level5.2"
corresponding to a service of 2160/100P, and the information
30 on the layer range indicates 0 to 3. Furthermore, for example,
a level designation value of a bit stream of a layer range
11
whose maximum layer is layer 2 is "level5.1" corresponding
to a service of 2160/50P, and the information on the layer
range indicates 0 to 2.
[0026]
Fig. 3(c) is an example in which the 5 uppermost layer
is 4. For example, a level designation value of a bit stream
of a layer range whose maximum layer is layer 4 is "level5.2"
corresponding to a service of 2160/100P, and the information
on the layer range indicates 0 to 4. Furthermore, for example,
10 a level designation value of a bit stream of a layer range
whose maximum layer is layer 3 is "level5.1" corresponding
to a service of 2160/50P, and the information on the layer
range indicates 0 to 3. Furthermore, for example, a level
designation value of a bit stream of a layer range whose maximum
15 layer is layer 2 is "level5" corresponding to a service of
2160/25P, and the information on the layer range indicates
0 to 2.
[0027]
Here, a value of "level_idc" is described. Fig. 4
20 illustrates "level_idc" and an example of the value. For
example, "level_idc" corresponding to a service of 1080/50P
is "level4.1", and the value is "123" in decimal and "0x7b"
in hexadecimal. To indicate this "level4.1", "b", which is
the lower 4 bits, is to be described as
25 "ls4b_sublayer_level_idc" in an HEVC descriptor, which will
be described later. Furthermore, for example, "level_idc"
corresponding to a service of 1080/100P is "level4.2", and
the value is"126" in decimal and "0x7e" in hexadecimal.
[0028]
30 Furthermore, for example, "level_idc" corresponding to
a service of 2160/50P is "level5.1", and the value is "153"
12
in decimal and "0x99" in hexadecimal. To indicate this
"level5.1", "9", which is the lower 4 bits, is to be described
as "ls4b_sublayer_level_idc" in an HEVC descriptor, which will
be described later. Furthermore, for example, "level_idc"
corresponding to a service of 2160/100P 5 is "level5.2", and
the value is "156" in decimal and "0x9c" in hexadecimal.
[0029]
Furthermore, for example, "level_idc" corresponding to
a service of 4320/50P s "level6.1", and the value is"183" in
10 decimal and "0xb7" in hexadecimal. To indicate this
"level6.1", "7", which is the lower 4 bits, is to be described
as "ls4b_sublayer_level_idc" in an HEVC descriptor, which will
be described later. Furthermore, for example, "level_idc"
corresponding to a service of 4320/100P is "level6.2", and
15 the value is "186" in decimal and "0xba" in hexadecimal.
[0030]
The reception apparatus 200 receives the above described
transport stream TS transmitted from the transmission
apparatus 100 on a broadcast wave or on an internet packet.
20 The reception apparatus 200 extracts the encoded image data
of the picture in the layer lower than the predetermined layer
from the video data contained in the transport stream TS and
decodes the encoded image data according to the own decoding
capability. At this time, the reception apparatus 200
25 performs decoding on the basis of the level designation value
of the bit stream and the information on the layer range in
each of the layer ranges having a different maximum layer which
are inserted in the transport stream TS as described above.
[0031]
30 "Configuration of the transmission apparatus"
Fig. 5 illustrates a configuration example of the
13
transmission apparatus 100. The transmission apparatus 100
includes a central processing unit (CPU) 101, an encoder 102,
a compressed data buffer (cpb: coded picture buffer) 103, a
multiplexer 104, and a transmission unit 105. The CPU 101
is a control unit, and controls the operations 5 of the units
of the transmission apparatus 100.
[0032]
The encoder 102 inputs decoded moving picture data VD
and hierarchically encodes the data. The encoder 102
10 classifies image data of each picture consisting the moving
picture data VD into a plurality of layers. Then, the encoder
102 encodes the classified image data of the picture in each
of the layers, and generates a video stream (video data) having
the encoded image data of the picture in each of the layers.
15 [0033]
The encoder 102 performs encoding, such as H.264/AVC
or H.265/HEVC. At this time, the encoder 102 performs encoding
so that a picture to be referred (a referred picture) belongs
to the own layer and/or a layer lower than the own layer. The
20 coded picture buffer (cpb) 103 temporarily stores the video
stream containing the encoded image data of the picture in
each of the layers and generated by the encoder 102.
[0034]
The multiplexer 104 reads and PES-packetizes the video
25 stream stored in the compressed data buffer 103, multiplexes
the video stream by transport-packetizing the video stream,
and obtains the transport stream TS as a multiplexed stream.
The transport stream TS contains a single video stream as
described above. The multiplexer 104 inserts, in a layer of
30 a container, the level designation value of the bit stream
and the information on the layer range in the layer ranges
14
(level layers) having a different maximum layer. The
transmission unit 105 transmits the transport stream TS
obtained by the multiplexer 104 to the reception apparatus
200 on a broadcast wave or an internet packet.
5 [0035]
[Insertion of information]
The insertion of information by the multiplexer 104 is
further described. To insert the information, an existing
HEVC descriptor (HEVC_descriptor) or a newly defined
10 layer/signaling/descriptor (Layer_signaling descriptor) is
used.
[0036]
Fig. 6 illustrates a structure example (Syntax) in the
case of an existing HEVC descriptor to which elements are added.
15 An 8-bit field of "descriptor_tag" indicates a descriptor type,
and indicates an HEVC descriptor here. An 8-bit field of
"descriptor_length" indicates the length (size) of the
descriptor, and indicates the following number of bytes as
the length of a descriptor.
20 [0037]
An 8-bit field of "profile_idc" indicates a profile of
a bit stream. An 8-bit field of "level_idc" indicates a level
designation value of a bit stream in the uppermost layer. A
4-bit field of "ls4b_sublayer_level_idc" indicates a level
25 designation value of a bit stream in a layer lower than the
uppermost layer (for example, the layer one level below the
uppermost layer). In this case, lower 4 bits are arranged
in hexadecimal.
[0038]
30 Furthermore, in the case of
"temporal_layer_subset_flag=1", there exists a 3-bit field
15
of each of "temporal_id_min", "temporal_id_max",
"temporal_id_sublayer_min", and "temporal_id_sublayer_max".
"temporal_id_max" indicates a value of temporal_id of the
uppermost layer of the layer range in which the maximum layer
is the highest layer, that is, the uppermost 5 layer, and
"temporal_id_min" indicates a value of temporal_id of the
lowest layer of the layer range. Furthermore
"temporal_id_sublayer_max" indicates a value of temporal_id
of the uppermost layer of the layer range in which the maximum
10 layer is lower than the uppermost layer (normally, the layer
one level below the uppermost layer), and
"temporal_id_sublayer_min" indicates a value of temporal_id
of the lowest layer of the layer range.
[0039]
15 For example, a specific example of each field description
in the hierarchical encoding example illustrated in Fig. 3(a)
is described. "0x9c", which is the value of "level5.2", is
described in the 8-bit field of "level_idc". "9" of 0x99",
which is the value of "level5.1" is described in the 4-bit
20 field of "ls4b_sublayer_level_idc".
[0040]
"011" indicating temporal_id=3 is described in the 3-bit
field of "temporal_id_max", and "000" indicating t
temporal_id=0 is described in the 3-bit field
25 "temporal_id_min". Furthermore, "010" indicating
temporal_id=2 is described in the 3-bit field of
"temporal_id_sublayer_max", and "000" indicating
temporal_id=0 is described in the 3-bit field of
"temporal_id_min".
30 [0041]
Fig. 7 illustrates a structure example (Syntax) of a
16
layer/signaling/descriptor (Layer_signaling descriptor).
Furthermore, Fig. 8 illustrates semantics of main information
in the structure example. By the above described HEVC
descriptor (Fig. 6), a level designation value of a bit stream
and information on a layer range in each of 5 two layer ranges
are transmitted. However, by the layer/signaling/descriptor,
a level designation value of a bit stream and information on
a layer range in each of three or more layer ranges can be
transmitted.
10 [0042]
The 8-bit field of "descriptor_tag" indicates a
descriptor type, and indicates a layer/signaling/descriptor
here. An 8-bit field of "descriptor_length" indicates the
length (size) of the descriptor, and indicates the following
15 number of bytes as the length of a descriptor.
[0043]
An 8-bit field of "overall_profile_idc" indicates a
profile of the maximum range related to scalable encode tools.
An 8-bit field of "highest_level_idc" indicates the maximum
20 level of a scalable range. An 8-bit field of
"number_of_profile_layers" indicates the number of profile
layers having a scalable function. "number_of_level_layers"
indicates the number of level layers.
[0044]
25 An 8-bit field of "layer_profile_idc[i]" indicates a
profile of each profile layer. An 8-bit field of
"layer_level_idc[i][j]" indicates a level of each level layer.
An 8-bit field of "temporal_id_layer_min[i][j]" indicates a
value of the minimum temporal_id in each level layer. An 8-bit
30 field of "temporal_id_layer_max[i][j]" indicates a value of
the maximum temporal_id in each level layer.
17
[0045]
For example, a specific example of each field description
related to a level in the hierarchical encoding example
illustrated in Fig. 3(c) is described. "0x9c", which is the
value of "level5.2", is described in 5 the 8-bit field of
"highest_level_idc". When three layer ranges in which each
of layers 4, 3, and 2 is the maximum layer are set, "3" indicating
that that the number of level layers is three is described
in an 8-bit field of "number_of_level_layers".
10 [0046]
Then, with regard to a first level layer, "0x9c", which
is the value of "level5.2", is described in the 8-bit field
of "layer_level_idc[i][j]", "100" indicating temporal_id=4
is described in the 8-bit field of
15 "temporal_id_layer_max[i][j]", and "000" indicating
temporal_id=0 is described in the 8-bit field of
"temporal_id_layer_min[i][j]".
[0047]
Furthermore, with regard to a second level layer, "0x99",
20 which is the value of "level5.1", is described in the 8-bit
field of "layer_level_idc[i][j]", "011" indicating
temporal_id=3 is described in the 8-bit field of
"temporal_id_layer_max[i][j]", and "000" indicating
temporal_id=0 is described in the 8-bit field of
25 "temporal_id_layer_min[i][j]".
[0048]
Furthermore, with regard to a third level layer, "0x96",
which is the value of "level5", is described in the 8-bit field
of "layer_level_idc[i][j]", "010" indicating temporal_id=2
30 is described in the 8-bit field of
"temporal_id_layer_max[i][j]", and "000" indicating
18
temporal_id=0 is described in the 8-bit field of
"temporal_id_layer_min[i][j]".
[0049]
Here, a configuration example of a profile layer is
described with reference to Figs. 9(a) to 5 9(c). Fig. 9(a)
is an example of encoding an 8-bit image whose image quality
is improved with two streams by encoding difference data as
an extended stream of a scalable main profile of
"Profile_layering_id=1" in addition to a base stream of a main
10 profile of "Profile_layering_id=0", compared with the base
stream alone.
[0050]
Fig. 9(b) is an example of encoding a 10-bit image whose
image quality is improved with two streams by encoding
15 difference data as an extended stream of a scalable main10
profile of "Profile_layering_id=1" in addition to a base stream
of a main10 profile of "Profile_layering_id=0", compared with
the base stream alone.
[0051]
20 Fig. 9(c) is an example of encoding an 8-bit image whose
image quality is improved with two streams by encoding
difference data as an extended stream 1 of a scalable main
profile of "Profile_layering_id=1" in addition to a base stream
of a main profile of "Profile_layering_id=0", compared with
25 the base stream alone. Furthermore, an image having an
accuracy of 10 bits obtained by encoding difference data
between 10 bits and 8 bits as an extended stream 2 of a scalable
main profile of "Profile_layering_id=2" is encoded.
[0052]
30 Note that, the added value of displaying by the scalable
extended stream not only improves the above described image
19
quality, but also applies to scalable extension related to
increase of a spatial resolution, expansion of a color gamut,
and an expansion of a luminance level. With regard to these
streams, by analyzing a packet from a decoder input buffer
of a receiver and appropriately discriminating 5 the packet,
it is possible to perform desired decoding.
[0053]
[Configuration of the transport stream TS]
Fig. 10 illustrates a configuration example of the
10 transport stream TS. The transport stream TS contains one
video stream. In other words, in this configuration example,
there exists a PES packet "video PES1" of a video stream having
encoded image data of pictures of a plurality of layers by,
for example, an HEVC.
15 [0054]
In the encoded image data of each picture, there exists
an NAL unit, such as a VPS, SPS, PPS, SLICE, or SEI. As described
above, the layer identification information on the picture
("nuh_temporal_id_plus1" indicating temporal_id) is arranged
20 in the header of the NAL unit. "general_level_idc", which is
a level designation value of a bit stream, is inserted in SPS
[0055]
Furthermore, the transport stream TS contains a program
map table (PMT) as program specific information (PSI). The
25 PSI is the information in which it is described that elementary
streams contained in the transport stream each belong to which
program.
[0056]
In the PMT, there exists a program/loop (Program loop)
30 describing information related to an entire program.
Furthermore, there exists an elementary/loop having
20
information related to each elementary stream in the PMT. In
the configuration example, there exists a video
elementary/loop (video ES1 loop).
[0057]
In the video elementary/loop, information, 5 such as a
stream type and a packet identifier (PID), corresponding to
a video stream (video PES1), and a descriptor describing
information related to the video stream is arranged. As one
of the descriptors, the above described HEVC descriptor
10 (HEVC_descriptor) or layer/signaling/descriptor
(Layer_signaling descriptor) is inserted. Note that, the
layer/signaling/descriptor is not inserted when the
element-added HEVC descriptor illustrated in Fig. 6 is used.
[0058]
15 The operations of the transmission apparatus 100
illustrated in Fig. 5 are briefly described. The decoded
moving picture data VD is input to the encoder 102. The encoder
102 performs hierarchical encoding to the moving picture data
VD. In other words, the encoder 102 classifies the image data
20 of each picture consisting the moving picture data VD into
a plurality of layers, encodes the image data, and generates
a video stream (video data) having the encoded image data of
the picture in each of the layers. In this case, the image
data is encoded so that a picture to be referred belongs to
25 the own layer and/or a layer lower than the own layer.
[0059]
The video stream containing the encoded data of the
picture in each of the layers and generated by the encoder
102 is supplied to the compressed data buffer (cpb) 103 and
30 temporarily stored. By the multiplexer 104, the video stream
stored in the compressed data buffer 103 is read,
21
PES-packetized, and multiplexed by being
transport-packetized, and the transport stream TS as a
multiplexed stream is obtained. The transport stream TS
contains a single video stream.
5 [0060]
When the transport stream TS is generated by the
multiplexer 104 in this manner, the level designation value
of the bit stream and the information on the layer range in
the layer ranges having a different maximum layer are inserted
10 in the layer of the container. For example, the element-added
HEVC descriptor (see Fig. 6) or the layer/signaling/descriptor
(see Fig. 7) is inserted under the program map table (PMT).
The transport stream TS generated by the multiplexer 104 is
transmitted to the transmission unit 105. The transport
15 stream TS is transmitted to the reception apparatus 200 on
a broadcast wave or an internet packet by the transmission
unit 105.
[0061]
"Configuration of the reception apparatus"
20 Fig. 11 illustrates a configuration example of the
reception apparatus 200. The reception apparatus 200
includes a central processing unit (CPU) 201, a reception unit
202, a demultiplexer 203, and a compressed data buffer (cpb:
coded picture buffer) 204. Furthermore, the reception
25 apparatus 200 includes a decoder 205, a decompressed data
buffer (dpb: decoded picture buffer) 206, a post-processing
unit 207, and a display unit 208. The CPU 201 consists a control
unit, and controls the operations of the units of the reception
apparatus 200.
30 [0062]
The reception unit 202 receives the transport stream
22
TS transmitted from the transmission apparatus 100 on a
broadcast wave or on an internet packet. The demultiplexer
203 extracts, from the transport stream TS, a TS packet
consisting the video stream contained in the transport stream
TS after filtering the TS packet with a PID filter, 5 and transmits
the transport stream TS to the compressed data buffer (cpb:
coded picture buffer) 204.
[0063]
Furthermore, the demultiplexer 203 extracts section
10 data data from the transport stream TS, and transmits the
section data data to the CPU 201. The section data contains
the above described HEVC descriptor (HEVC_descriptor) and
layer/signaling/descriptor (Layer_signaling descriptor).
The CPU 201 determines the layer range which the decoder 205
15 can decode from the layer ranges indicated by these descriptors
with the level designation value of the bit stream and the
information on the layer range, and transmits the information
on the temporal ID (temporal_id) of the layer range to the
decoder 205.
20 [0064]
Furthermore, the demultiplexer 203 extracts a program
clock reference (PCR) from the TS packet containing the PCR,
and transmits the PCR to the CPU 201. Furthermore, the
demultiplexer 203 extracts time stamps (DTS and PTS) inserted
25 in a PES header for each picture, and transmits the time stamps
to the CPU 201.
[0065]
The compressed data buffer (cpb) 204 temporarily stores
the encoded image data of each picture according to the TS
30 packet transferred from the demultiplexer 203. The decoder
205 reads and decodes the encoded image data of each picture
23
stored in the compressed data buffer 204 at a decode timing
supplied by a decoding time stamp (DTS) of the picture, and
transmits the decoded image data to the decompressed data
buffer (dpb: decoded picture buffer) 206. At this time, the
decoder 205 selectively decodes only the 5 encoded image data
of the picture contained in the decodable layer range on the
basis of the information on the temporal ID (temporal_id) of
the decodable layer range supplied by the CPU 201.
[0066]
10 Fig. 12 schematically illustrates a configuration
example of the demultiplexer 203 and the decoder 205. The
demultiplexer 203 includes a video multiplexing buffer 203a
and a section data buffer 203b. The TS packet corresponding
to a program number equivalent to a broadcasting service
15 channel and extracted from the transport stream TS with the
PID filter is temporarily stored in the video multiplexing
buffer 203a, and then, is transferred to the compressed data
buffer 204. Furthermore, the section data extracted from the
transport stream TS with the PID filter is temporarily stored
20 in the section data buffer 203b, and then, is transmitted to
the CPU 201.
[0067]
Furthermore, the decoder 205 includes a temporal ID
analysis unit 205a and a decode processing unit 205b. The
25 temporal ID analysis unit 205a sequentially reads the encoded
data of each picture stored in the compressed data buffer 204
at the decode timing, and analyzes the information on the
temporal ID (temporal_id) inserted in the NAL unit header.
Then, the temporal ID analysis unit 205a transfers the encoded
30 data to the decode processing unit 205b when determining the
encoded data is within the decodable layer range, and discards
24
the encoded data without transferring the encoded data to the
decode processing unit 205b when determining the encoded data
is not within the decodable layer range. Note that, the
information on the temporal ID (temporal_id) of the decodable
layer range is supplied to the temporal ID 5 analysis unit 205a
by the CPU 201.
[0068]
For example, the case of the hierarchical encoding
example of Fig. 3(a) is described. For example, when the
10 decoder 205 is a 100p decoder, "0 to 3" is supplied to the
temporal ID analysis unit 205a as the information on the
temporal ID of the decodable layer ranges by the CPU 201. Thus,
the temporal ID analysis unit 205a transmits the encoded image
data of the pictures in the layers 0 to 3 to the decode processing
15 unit 205b.
[0069]
On the other hand, when the decoder 205 is a 50p decoder,
"0 to 2" is supplied to the temporal ID analysis unit 205a
as the information on the temporal ID of the decodable layer
20 ranges by the CPU 201. Thus, the temporal ID analysis unit
205a transmits the encoded image data of the pictures in the
layers 0 to 2 to the decode processing unit 205b. On the other
hand, the temporal ID analysis unit 205a discards the encoded
image data of the picture in the layer 3 without transmitting
25 the encoded image to the decode processing unit 205b. Fig.
13 schematically illustrates such behavior of the temporal
ID analysis unit 205a. Note that, "tid=high" indicates the
layer 3, and "tid=low" indicates the layers 0 to 2.
[0070]
30 Returning back to Fig. 11, the decompressed data buffer
(dpb) 206 temporarily stores the image data of each picture
25
decoded by the decoder 205. The post-processing unit 207
performs, to the image data of each picture sequentially read
at the display timing supplied by the presentation time stamp
(PTS) from the decompressed data buffer (dpb) 206, processing
to adjust the frame rate to the display 5 capability.
[0071]
For example, when the frame rate of the image data of
each picture after decoding is 50 fps and the display capability
is 100 fps, the post-processing unit 207 performs interpolation
10 processing to the image data of each picture after decoding
so that the time direction resolution becomes twice, and
transmits the image data of 100 fps to the display unit 208.
[0072]
The display unit 208 is constituted by, for example,
15 a liquid crystal display (LCD), an organic
electro-luminescence (EL) panel, or the like. Note that, the
display unit 208 may be an external device connected to the
reception apparatus 200.
[0073]
20 Fig. 14 illustrates an example of a processing flow of
the decoder 205 and the post-processing unit 207. In step
ST1, the processing is started, and then moves to the processing
in step ST2. In step ST2, the decoder 205 reads, from the
compressed data buffer (cpb) 204, the video stream to be decoded
25 at the decode timing (the encoded image data of the picture
to be decoded), and detects the temporal ID (temporal_id).
[0074]
Next, in step ST3, the decoder 205 determines whether
the temporal ID (temporal_id) detected in step ST2 is within
30 the decodable range. When the temporal ID is not within the
decodable range, the decoder 205 does not perform the decode
26
processing, and returns back to the processing in step ST2.
On the other hand, when the temporal ID is within the decodable
range, the decoder 205 moves to the processing in step ST4.
In step ST4, the decoder 205 performs the decode processing,
and transfers the image data of the picture 5 after decoding
to the decompressed data buffer (dpb) 206.
[0075]
Next, in step ST5, the post-processing unit 207 reads,
from the decompressed data buffer (dpb) 206, the image data
10 of the image data of the picture to be displayed at the display
timing. Next, in step ST6, the post-processing unit 207
determines whether a display frequency and a read frequency
from the decompressed data buffer (dpb) 206 are different.
When the frequencies are different, in step ST7, the
15 post-processing unit 207 adjusts the read frequency to the
display frequency by performing frame interpolation or
thinning of the picture. After the processing in step ST7,
the processing is terminated in step ST8. Furthermore, when
the frequencies are not different in step ST6, the processing
20 is immediately terminated in step ST8.
[0076]
The operations of the reception apparatus 200
illustrated in Fig. 11 are briefly described. The transport
stream TS transmitted from the transmission apparatus 100 on
25 a broadcast wave or on an internet packet is received by the
reception unit 202. The transport stream TS is transmitted
to the demultiplexer 203. The TS packet corresponding to a
service is extracted, from the transport stream TS, on the
basis of the PID information contained in the PMT by the
30 demultiplexer 203. The TS packet is transmitted to the
compressed data buffer (cpb) 204, and temporarily stored.
27
[0077]
Furthermore, the section data is extracted from the
transport stream TS, and transmitted to the CPU 201 by the
demultiplexer 203. The layer range which the decoder 205 can
decode is determined from the layer ranges 5 described by the
HEVC descriptor or the layer/signaling/descriptor with the
level designation value of the bit stream and the information
on the layer range, and the information on the temporal ID
(temporal_id) of the layer range is transmitted to the decoder
10 205 by the CPU 201.
[0078]
The encoded image data of each picture stored in the
compressed data buffer 204 is decoded at the decode timing
of the picture, transmitted to the decompressed data buffer
15 (dpb) 206, and temporarily stored by the decoder 205. In this
case, by the decoder 205, the encoded image data of the picture
within the decodable layer range is only selectively decoded
on the basis of the information on the temporal ID (temporal_id)
of the decodable layer range supplied by the CPU 201.
20 [0079]
The image data of each picture stored in the decompressed
data buffer (dpb) 206 is sequentially read at the display timing,
and transmitted to the post-processing unit 207.
Interpolation, subsample, or thinning is performed to the image
25 data of each picture to adjust the frame rate to the display
capability by the post-processing unit 207. The image data
of each picture processed by the post-processing unit 207 is
supplied to the display unit 208, and the moving picture is
displayed with the image data of each picture.
30 [0080]
As described above, in the transceiver system 10
28
illustrated in Fig. 1, the element-added HEVC descriptor (see
Fig. 6) or the layer/signaling/descriptor (see Fig. 7) is
inserted in the layer of the container of the transport stream
TS by the transmission apparatus 100. These descriptors have
the level designation value of the 5 bit stream and the
information on the layer range in each of the layer ranges
having a different maximum layer. Thus, for example, it is
possible for the receiving side to decode the encoded image
data of the picture in the layer range according to the decoding
10 performance on the basis of the information.
[0081]
<2. Modified example>
[Application to the MPEG-DASH-based stream distribution
system]
15 Note that, in the above described embodiment, the example
in which the container is a transport stream (MPEG-2 TS) has
been described. However, the present technology can be
similarly applied to a system having a configuration in which
a stream is distributed to a reception terminal using a network,
20 such as the internet. In an internet distribution, a stream
is mainly distributed by a container of an MP4 or other formats.
[0082]
Fig. 15(a) illustrates a configuration example of an
MPEG-DASH-based stream distribution system 30. The stream
25 distribution system 30 has a configuration in which a DASH
stream file server 31 and a DASH MPD server 32 are connected
to N number of receivers 33-1, 33-2, ..., and 33-N through
a content delivery network (CDN) 34.
[0083]
30 The DASH stream file server 31 generates, on the basis
of media data (video data, audio data, subtitle data, or the
29
like) of predetermined content, a stream segment conforming
to DASH (hereinafter, appropriately referred to as a "DASH
segment"), and transmits the segment in response to an HTTP
request from the receiver. The DASH stream file server 31
may be a streaming-dedicated server, or used 5 as a web server.
[0084]
Furthermore, the DASH stream file server 31 transmits,
in response to a request of a segment of a predetermined stream
transmitted from the receiver 33 (33-1, 33-2, ..., and 33-N)
10 through the CDN 34, the segment of the stream to the receiver,
which is the request source, through the CDN 34. In this case,
the receiver 33 performs the request by referring to the value
of the rate described in a media presentation description (MPD)
file and selecting a stream of an optimal rate according to
15 a network environment where a client is placed.
[0085]
The DASH MPD server 32 is a server to generate an MPD
file to acquire the DASH segment generated by the DASH stream
file server 31. The MPD file is generated based on content
20 metadata from a content management server (not illustrated)
and an address (url) of the segment generated by the DASH stream
file server 31.
[0086]
In an MPD format, using an element of representation
25 for each stream of a video and audio, each attribute is described.
For example, by separating the representation for each of a
plurality of video data streams having a different rate, each
rate is described in the MPD file. The receiver 33 can select
an optimal stream according to the conditions of the network
30 environment where the receiver 33 is placed by referring to
the value of the rate as described above.
30
[0087]
Fig. 15(b) also illustrates a configuration example of
an MPEG-DASH-based stream distribution system 30A. The
stream distribution system 30A is constituted by a broadcast
transmission system 36 with which the DASH 5 stream file server
31 and the DASH MPD server 32 are connected, and M number of
receivers 35-1, 35-2, ..., and 35-M. In the case of the stream
distribution system 30A, the broadcast transmission system
36 transmits a stream segment conforming to DASH (DASH segment)
10 generated by the DASH stream file server 31 and the MPD file
generated by the DASH MPD server 32 on a broadcast wave.
[0088]
Figs. 16(a) to 16(d) illustrate an example of a relation
between structures hierarchically arranged in the above
15 described MPD file. As illustrated in Fig. 16(a), a plurality
of periods divided by time intervals exists in a media
presentation as a whole MPD file. For example, a first period
starts from 0 second, the next period starts from 100 seconds,
and so on.
20 [0089]
As illustrated in Fig. 16(b), a plurality of
representations exists in the periods. A stream attribute
grouped by AdaptationSet, such as a representation group
associated with a video data stream having different rate and
25 the same contents exists in the representations.
[0090]
As illustrated in Fig. 16(c), the representation
contains SegmentInfo. As illustrated in Fig. 16(d), there
exist an initialization segment and a plurality of media
30 segments, in which information on a segment into which the
period is further divided is described, in the SegmentInfo.
31
There exist information and the like on an address (url) to
actually acquire segment data, such as a video or audio, in
the media segment.
[0091]
Note that, switching stream can be 5 freely performed
between the representations grouped by AdaptationSet. Thus,
according to the conditions of a network environment where
an IPTV client is placed, it is possible to select a stream
of an optimal rate, and perform seamless moving picture
10 distribution.
[0092]
Fig. 17 illustrates a configuration example of a
FragmentedMP4 stream in association with an MPD file. The
FragmentedMP4 stream contains FragmentedMP4 obtained by
15 packetizing a video stream. Encoded video data of
predetermined pictures of the video stream, for example, 1
GOP is inserted in the part of "mdat" of FragmentedMP4.
[0093]
The present technology can be applied to the stream
20 distribution systems 30 and 30A illustrated in Figs. 15(a)
and (b). In this case, the video stream has been hierarchically
encoded similarly to the above described embodiment.
Furthermore, the contents equivalent to the description
contents of, for example, the above described
25 layer/signaling/descriptor (see Fig. 7) are inserted in the
MPD file, and transmitted to the receiving side. Fig. 18
illustrates an implementation of an MPD file.
[0094]
Furthermore, the transceiver system 10 constituted by
30 the transmission apparatus 100 and the reception apparatus
200 has been described in the above described embodiment,
32
however, the transceiver system to which the present technology
can be applied is not limited to this. For example, the part
of the reception apparatus 200 may be, for example, a set top
box and monitor connected by a digital interface, such as
High-Definition Multimedia Interface (5 HDMI). Note that,
"HDMI" is a registered trademark.
[0095]
Furthermore, the present technology can be following
configurations:
10 (1) A transmission apparatus includes an image encoding
unit which classifies image data of each picture consisting
moving picture data into a plurality of layers, encodes the
classified image data of the picture in each of the plurality
of layers, and generates video data having the encoded image
15 data of the picture in each of the plurality of layers,
a data transmission unit which transmits the video data,
and an information transmission unit which transmits
a level designation value of a bit stream and information on
a layer range in each of a plurality of layer ranges having
20 a different maximum layer.
(2) The transmission apparatus according to the (1),
in which the information transmission unit inserts the
information in a layer of a container containing the video
data and transmits the information.
25 (3) The transmission apparatus according to the (2),
in which the container is an MPEG2-TS,
and the information transmission unit inserts the
information under a program map table and transmits the
information.
30 (4) The transmission apparatus according to the (1),
in which the information transmission unit inserts the
33
information in a metafile having meta-information related to
the video data, and transmits the information.
(5) The transmission apparatus according to the (4),
in which the metafile is an MPD file.
(6) The transmission apparatus according 5 to any one of
the (1) to (5),
in which the information transmission unit transmits,
together with information on a profile, the level designation
value of the bit stream and the information on the layer range
10 in each of the plurality of layer ranges having a different
maximum layer.
(7) A transmission method includes an image encoding
step for classifying image data of each picture consisting
moving picture data into a plurality of layers, encoding the
15 classified image data of the picture in each of the plurality
of layers, and generating video data having the encoded image
data of the picture in each of the plurality of layers,
a data transmitting step for transmitting the video data,
and an information transmitting step for transmitting
20 a level designation value of a bit stream and information on
a layer range in each of a plurality of layer ranges having
a different maximum layer.
(8) A reception apparatus includes a data reception unit
which receives video data having encoded image data of a picture
25 in each of a plurality of layers obtained by classifying image
data of each picture consisting moving picture data into the
plurality of layers and encoding the image data,
an information reception unit which receives a level
designation value of a bit stream and information on a layer
30 range in each of a plurality of layer ranges having a different
maximum layer,
34
and a processing unit which extracts, from the video
data, the encoded image data of the picture in a layer lower
than a predetermined layer and decodes the encoded image data
on the basis of the information.
(9) The reception apparatus according 5 to the (8),
in which the information reception unit acquires the
information from a layer of a container containing the video
data.
(10) The reception apparatus according to the (8),
10 in which the information reception unit acquires the
information from a metafile having meta-information related
to the video data.
(11) A reception method includes a data receiving step
for receiving video data having encoded image data of a picture
15 in each of a plurality of layers obtained by classifying image
data of each picture consisting moving picture data into the
plurality of layers and encoding the image data,
an information receiving step for receiving a level
designation value of a bit stream and information on a layer
20 range in each of a plurality of layer ranges having a different
maximum layer,
and a processing step for extracting, from the video
data, the encoded image data of the picture in a layer lower
than a predetermined layer and decoding the encoded image data
25 on the basis of the information.
[0096]
The main feature of the present technology is by
transmitting a level designation value of a bit stream and
information on a layer range in each of a plurality of layer
30 ranges having a different maximum layer when video data
hierarchically encoded is transmitted, it is possible for a
35
receiving side to easily decode the encoded image data of the
picture of the layer range according to decoding performance
(see Fig. 10, Fig. 6, and Fig. 7).
REFERENCE 5 SIGNS LIST
[0097]
10 Transceiver system
30, 30A MPEG-DASH-based stream distribution system
31 DASH stream file server
10 32 DASH MPD server
33-1, 33-2, ..., 33-N, 35-1, 35-2, ..., 35-M receiver
34 Content delivery network (CDN)
36 Broadcast transmission system
100 Transmission apparatus
15 101 CPU
102 Encoder
103 Compressed data buffer (cpb)
104 Multiplexer
105 Transmission unit
20 200 Reception apparatus
201 CPU
202 Reception unit
203 Demultiplexer
203a Video multiplexing buffer
25 203b Section data buffer
204 Compressed data buffer (cpb)
205 Decoder
205a Temporal ID analysis unit
205b Decode processing unit
30 206 Decompressed data buffer (dpb)
207 Post-processing unit
36
208 Display unit
37
CLAIMS
1. A transmission apparatus comprising:
an image encoding unit configured to classify image data
of each picture consisting moving picture data 5 into a plurality
of layers, encode the classified image data of the picture
in each of the plurality of layers, and generate video data
having the encoded image data of the picture in each of the
plurality of layers;
10 a data transmission unit configured to transmit the video
data; and
an information transmission unit configured to transmit
a level designation value of a bit stream and information on
a layer range in each of a plurality of layer ranges having
15 a different maximum layer.
2. The transmission apparatus according to claim 1, wherein
the information transmission unit inserts the information in
a layer of a container containing the video data and transmits
20 the information.
3. The transmission apparatus according to claim 2, wherein
the container is an MPEG2-TS, and
the information transmission unit inserts the
25 information under a program map table and transmits the
information.
4. The transmission apparatus according to claim 1, wherein
the information transmission unit inserts the information in
30 a metafile having meta-information related to the video data,
and transmits the information.
38
5. The transmission apparatus according to claim 4, wherein
the metafile is an MPD file.
6. The transmission apparatus according to 5 claim 1, wherein
the information transmission unit transmits, together with
information on a profile, the level designation value of the
bit stream and the information on the layer range in each of
the plurality of layer ranges having a different maximum layer.
10
7. A transmission method comprising:
an image encoding step for classifying image data of
each picture consisting moving picture data into a plurality
of layers, encoding the classified image data of the picture
15 in each of the plurality of layers, and generating video data
having the encoded image data of the picture in each of the
plurality of layers;
a data transmitting step for transmitting the video data;
and
20 an information transmitting step for transmitting a
level designation value of a bit stream and information on
a layer range in each of a plurality of layer ranges having
a different maximum layer.
25 8. A reception apparatus comprising:
a data reception unit configured to receive video data
having encoded image data of a picture in each of a plurality
of layers obtained by classifying image data of each picture
consisting moving picture data into the plurality of layers
30 and encoding the image data;
an information reception unit configured to receive a
39
level designation value of a bit stream and information on
a layer range in each of a plurality of layer ranges having
a different maximum layer; and
a processing unit configured to extract, from the video
data, the encoded image data of the picture 5 in a layer lower
than a predetermined layer and decode the encoded image data
on the basis of the information.
9. The reception apparatus according to claim 8, wherein
10 the information reception unit acquires the information from
a layer of a container containing the video data.
10. The reception apparatus according to claim 8, wherein
the information reception unit acquires the information from
15 a metafile having meta-information related to the video data.
11. A reception method comprising:
a data receiving step for receiving video data having
encoded image data of a picture in each of a plurality of layers
20 obtained by classifying image data of each picture consisting
moving picture data into the plurality of layers and encoding
the image data;
an information receiving step for receiving a level
designation value of a bit stream and information on a layer
25 range in each of a plurality of layer ranges having a different
maximum layer; and
a processing step for extracting, from the video data,
the encoded image data of the picture in a layer lower than
30
40
a predetermined layer and decoding the encoded image data on
the basis of the information.
Dated this 12th day of October 2016
5
MAYANK SOOD
IN/PA 1850
OF K & S PARTNERS
10 ATTORNEY FOR THE APPLICANT(S)