FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“TRANSMISSION APPARATUS, TRANSMISSION METHOD,
RECEPTION APPARATUS AND RECEPTION METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-0075, Japan
The following specification particularly describes the invention and the manner in
which it is to be performed.
2
Description
Title of Invention
TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEPTION DEVICE,
5 AND RECEPTION METHOD
Technical Field
[0001]
The present technology relates to a transmission device, a transmission
10 method, a reception device, and a reception method, and more particularly to a
transmission device or the like which transmits basic format image data and a
predetermined number of pieces of high-quality format image data.
Background Art
15 [0002]
In the related art, it is known to transmit basic format image data and highquality
format image data, and to selectively use the basic format image data or the
high-quality format image data on a receiving side. For example, Patent Document
1 describes that a stream of a base layer for a low-resolution video service and a
20 stream of an extended layer for a high-resolution video service are generated by
performing scalable media encoding and broadcast signals including these streams
are transmitted. A high-quality format includes a high frame frequency, a high
dynamic range, a wide color gamut, a high bit length, and the like in addition to a
high resolution.
25
Citation List
Patent Literature
[0003]
Patent Literature 1: JP 2008-543142T
30
Summary of Invention
3
Technical Problem
[0004]
An object of the present technology is to successfully transmit basic format
image data and a predetermined number of pieces of high-quality format image data.
5
Solution to Problem
[0005]
A concept of the present technology is a transmission device including: an
image encoding unit which generates a basic video stream obtained by encoding
10 basic format image data and a predetermined number of extended video streams
obtained by encoding the predetermined number of pieces of high-quality format
image data; a transmission unit which transmits a container in a predetermined
format including the basic video stream and the predetermined number of extended
video streams generated by the image encoding unit; and an identification
15 information insertion unit which inserts identification information in a high-quality
format corresponding to each of the predetermined number of extended video
streams into a layer of the container and/or the video stream.
[0006]
In the present technology, a basic video stream and a predetermined number
20 of extended video streams are generated by an image encoding unit. Here, the basic
video stream is obtained by encoding basic format image data. In addition, the
predetermined number of extended video streams are obtained by encoding the
predetermined number of pieces of high-quality format image data, respectively.
[0007]
25 For example, the image encoding unit may generate, with respect to the
basic format image data, the basic video stream by performing a prediction encoding
process in the basic format image data, and generate, with respect to the high-quality
format image data, the extended video stream by selectively performing a prediction
encoding process in the high-quality format image data or a prediction encoding
30 process between the high-quality format image data and the basic format image data
or between the high-quality format image data and other high-quality format image
4
data.
[0008]
The transmission unit transmits a container in a predetermined format
including the basic video stream and the predetermined number of extended video
streams generated by the image encoding unit. The 5 container may be, for example,
a transport stream (MPEG-2 TS) that has been employed in digital broadcasting
standards. In addition, the container may be, for example, MP4 that has been used
in distribution on the Internet and the like, or a container in a format other than that.
[0009]
10 An identification information insertion unit inserts identification
information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the container or the video stream.
For example, when the container is MPEG2-TS and the identification information
insertion unit inserts the identification information into the layer of the container, the
15 identification information may be inserted into respective video elementary stream
loops (video ES loop) corresponding to the predetermined number of extended video
streams present under a program map table (PMT). In addition, for example, a
video stream has a network abstraction layer (NAL) unit structure, and the
identification information insertion unit may insert the identification information into
20 a header of the NAL unit.
[0010]
According to the present technology, identification information in a highquality
format corresponding to the predetermined number of extended video streams
is inserted into a layer of the container or the video stream and transmitted.
25 Therefore, it is easy to obtain image data in accordance with display capability on a
receiving side by selectively performing a decoding process on a predetermined
video stream based on the identification information.
[0011]
In the present technology, for example, the identification information
30 inserted into a layer of the container may include information indicating whether
each of the predetermined number of extended video streams is generated by
5
performing a prediction encoding process between the high-quality format image
data and the basic format image data or by performing a prediction encoding process
between the high-quality format image data and the high-quality format image data.
In this case, it is possible to easily check on the receiving side whether basic format
image data is referred to or other high-quality format image 5 data are referred to in a
prediction encoding process when each of the predetermined number of extended
video streams is generated.
[0012]
In the present technology, for example, the identification information
10 inserted into a layer of the container may include information indicating a video
stream corresponding to image data referred to in a prediction encoding process
between the high-quality format image data and the basic format image data or
between the high-quality format image data and other high-quality format image data
performed when each of the predetermined number of extended video streams is
15 generated. In this case, it is possible to easily recognize on the receiving side which
video stream corresponds to the image data referred to in the prediction encoding
process when each of the predetermined number of extended video streams is
generated.
[0013]
20 Another concept of the present technology is a reception device including: a
reception unit which receives a container in a predetermined format including a basic
video stream obtained by encoding basic format image data and a predetermined
number of extended video streams obtained by encoding the predetermined number
of pieces of high-quality format image data. Identification information in a high25
quality format corresponding to each of the predetermined number of extended video
streams is inserted into a layer of the container and/or a layer of the video stream,
and the reception device further includes a process unit which processes each of the
video streams included in the received container based on the identification
information.
30 [0014]
In the present technology, a reception unit receives a container including a
6
basic video stream and a predetermined number of extended video streams. Here,
the basic video stream is obtained by encoding basic format image data. In addition,
the predetermined number of extended video streams are obtained by encoding the
predetermined number of pieces of high-quality format image data. Identification
information in a high-quality format corresponding 5 to each of the predetermined
number of extended video streams is inserted into a layer of the container or the
video stream.
[0015]
For example, the basic video stream may be generated by performing, with
10 respect to the basic format image data, a prediction encoding process in the basic
format image data, and the extended video stream may be generated by selectively
performing, with respect to the high-quality format image data, a prediction encoding
process in the high-quality format image data or a prediction encoding process
between the high-quality format image data and the basic format image data or
15 between the high-quality format image data and other high-quality format image data.
[0016]
The process unit processes each of the video streams included in the
received container based on the identification information. For example, the
process unit may acquire image data in accordance with display capability by
20 performing a decoding process on the basic video stream and a predetermined one of
the extended video streams based on the identification information and display
capability information.
[0017]
In this manner, each of the video streams is processed based on
25 identification information in a high-quality format corresponding to each of the
predetermined number of extended video streams inserted into a layer of the
container or the video stream and transmitted in the present technology.
Accordingly, image data in accordance with reception capability is easily obtained by
selectively performing a decoding process on a predetermined video stream.
30
Advantageous Effects of Invention
7
[0018]
According to the present technology, it is possible to successfully transmit
basic format image data and the predetermined number of pieces of high-quality
format image data. It should be noted that the effects described herein are not
necessarily limitative, and any effect described 5 in the present disclosure may be
exhibited.
Brief Description of Drawings
[0019]
10 [FIG. 1] FIG. 1 is a block diagram showing a configuration example of a transmission
and reception system of an embodiment.
[FIG. 2] FIG. 2 is a block diagram showing a configuration example of a transmission
device.
[FIG. 3] FIG. 3 is a block diagram showing a configuration example of an image data
15 generation unit which generates a piece of basic format image data Vb and three
pieces of high-quality format image data Vh1, Vh2, and Vh3.
[FIG. 4] FIG. 4 is a block diagram showing a configuration example of a main
portion of an encoding unit.
[FIG. 5] FIG. 5 is a diagram showing a structure example of an NAL unit header and
20 a content of a main parameter in the structure example.
[FIG. 6] FIG. 6 is a diagram showing a configuration example of a basic video stream
STb and extended video streams STe1, STe2, and STe3.
[FIG. 7] FIG. 7 is a diagram showing a structure example of a scalable extension
descriptor.
25 [FIG. 8] FIG. 8 is a diagram showing a content of principle information in the
structure example of the scalable extension descriptor.
[FIG. 9] FIG. 9 is a diagram showing a correspondence relationship between a value
of “type of enhancement” field of the scalable extension descriptor and a value of
“nuh_layer_id” of the NAL unit header.
30 [FIG. 10] FIG. 10 is a diagram showing a configuration example of a transport stream
TS.
8
[FIG. 11] FIG. 11 is a diagram showing a configuration example of a reception device.
[FIG. 12] FIG. 12 is a block diagram showing a configuration example of a main
portion of a decoding unit.
5 Description of Embodiments
[0020]
Hereinafter, embodiments for implementing this technology (hereinafter
referred to as “embodiments”) will be described. Note that description will be
10 provided in the following order.
1. Embodiment
2. Modified example
[0021]
<1. Embodiment>
15 [Transmission and reception system]
FIG. 1 shows a configuration example of a transmission and reception
system 10 as an embodiment. This transmission and reception system 10 is
configured to have a transmission device 100 and a reception device 200.
[0022]
20 A transmission device 100 transmits a transport stream TS as a container by
placing it on a broadcast wave or a net packet. This transport stream TS includes a
basic video stream and a predetermined number of extended video streams.
[0023]
The basic video stream is generated by performing encoding such as
25 H.264/AVC and H.265/HEVC on basic format image data. Here, with respect to
the basic format image data, the basic video stream is generated by performing a
prediction encoding process within the basic format image data.
[0024]
The predetermined number of extended video streams is generated by
30 performing encoding such as H.264/AVC and H.265/HEVC on the predetermined
number of pieces of high-quality image data. Here, with respect to the high-quality
9
format image data, an extended video stream is generated by selectively performing a
prediction encoding process in the high-quality format image data or a prediction
encoding process between the high-quality format image data and the basic format
image data or between the high-quality format image data and other high-quality
5 format image data.
[0025]
Identification information in a high-quality format corresponding to each of
the predetermined number of extended video streams is inserted into a layer of the
container. The high-quality format corresponding to each of the predetermined
10 number of extended video streams can be easily grasped on a receiving side in the
layer of the container based on the identification information. In the embodiment,
the identification information is inserted into respective video elementary stream
loops corresponding to the predetermined number of extended video streams present
under a program map table.
15 [0026]
The identification information includes information indicating whether each
of the predetermined number of extended video streams is generated by performing a
prediction encoding process between the high-quality format image data and basic
format image data or by performing a prediction encoding process between the high20
quality format image data and high-quality format image data. Using the
information, it is possible to easily recognize on the receiving side in the layer of the
container whether the basic format image data is referred to or other high-quality
format image data is referred to in a prediction encoding process when each of the
predetermined number of extended video streams is generated.
25 [0027]
In addition, the identification information includes information indicating a
video stream corresponding to image data referred to in a prediction encoding
process performed between the high-quality format image data and the basic format
image data or between the high-quality format image data and other high-quality
30 format image data performed when each of the predetermined number of extended
video streams is generated. Using the information, it is possible to easily recognize
10
on the receiving side in the layer of the container which video stream corresponds to
the image data referred to in the prediction encoding process when each of the
predetermined number of extended video streams is generated.
[0028]
Identification information in a high-quality 5 format corresponding to each of
the predetermined number of extended video streams is inserted into a layer of the
video stream. Using the identification information, it is possible to easily grasp a
high-quality format corresponding to each of the predetermined number of extended
video streams on a receiving side. In the embodiment, the identification
10 information is inserted into a header of an NAL unit.
[0029]
A reception device 200 receives the transport stream TS placed on a
broadcast wave or a net packet and transmitted by the transmission device 100.
Identification information in a high-quality format corresponding to each of the
15 predetermined number of extended video streams included in the transport stream TS
is inserted into a layer of the container or the video stream as described above. The
reception device 200 processes each video stream included in the transport stream TS
and acquires image data in accordance with display capability based on the
identification information.
20 [0030]
[Configuration of transmission device]
FIG. 2 shows a configuration example of the transmission device 100. The
transmission device 100 deals with a piece of basic format image data Vb and three
pieces of high-quality format image data Vh1, Vh2, and Vh3 as transmission image
25 data. Here, the basic format image data Vb is low dynamic range (LDR) image data
whose frame frequency is 50 Hz. The high-quality format image data Vh1 is LDR
image data whose frame frequency is 100 Hz. The LDR image data has a
brightness range from 0 % to 100 % with respect to luminance of a white peak of an
LDR image of the related art.
30 [0031]
The high-quality format image data Vh2 is a high dynamic range (HDR)
11
image data whose frame frequency is 50 Hz. The high-quality format image data
Vh3 is HDR image data whose frame frequency is 100 Hz. The HDR image data
has a brightness range of 0 to 100 %*N, for example, 0 to 400 % or 0 to 800 %, if the
luminance of the white peak of the LDR image of the related art is set to 100 %.
5 [0032]
FIG. 3 shows a configuration example of an image data generation unit 150
which generates the basic format image data Vb and three pieces of high-quality
format image data Vh1, Vh2, and Vh3. The image data generation unit 150
includes a HDR camera 151, a frame rate conversion unit 152, a dynamic range
10 conversion unit 153, and a frame rate conversion unit 154.
[0033]
The HDR camera 151 images a subject and outputs a HDR image data
whose frame frequency is 100 Hz, that is, the high-quality format image data Vh3.
The frame rate conversion unit 152 performs a process of converting the frame
15 frequency from 100 Hz to 50 Hz on the high-quality format image data Vh3 output
by the HDR camera 151, and outputs a HDR image data whose frame frequency is
50 Hz, that is, the high-quality format image data Vh2.
[0034]
The dynamic range conversion unit 153 performs a process of converting a
20 HDR into a LDR on the high-quality format image data Vh3 output by the HDR
camera 151, and outputs a LDR image data whose frame frequency is 100 Hz, that is,
the high-quality format image data Vh1. The frame rate conversion unit 154
performs a process of converting the frame frequency from 100 Hz to 50 Hz on the
high-quality format image data Vh1 output by the dynamic range conversion unit
25 153, and outputs an LDR image data whose frame frequency is 50 Hz, that is, the
basic format image data Vb.
[0035]
Returning to FIG. 2, the transmission device 100 includes a control unit 101,
LDR photoelectric conversion units 102 and 103, HDR photoelectric conversion
30 units 104 and 105, a video encoder 106, a system encoder 107, and a transmission
unit 108. The control unit 101 is configured to include a central processing unit
12
(CPU), and controls operations of each unit of the transmission device 100 based on
a control program.
[0036]
The LDR photoelectric conversion unit 102 obtains basic format image data
Vb for transmission by applying 5 photoelectric conversion characteristics (LDR
OETF curve) for an LDR image to the basic format image data Vb. The LDR
photoelectric conversion unit 103 obtains high-quality format image data Vh1 for
transmission by applying the photoelectric conversion characteristics for an LDR
image to the high-quality format image data Vh1.
10 [0037]
The HDR photoelectric conversion unit 104 obtains a high-quality format
image data Vh2 for transmission by applying photoelectric conversion characteristics
(HDR OETF curve) for a HDR image to the high-quality format image data Vh2.
The HDR photoelectric conversion unit 105 obtains a high-quality format image data
15 Vh3 for transmission by applying the photoelectric conversion characteristics for a
HDR image to the high-quality format image data Vh3.
[0038]
The video encoder 106 includes four encoding units 106-0, 106-1, 106-2,
and 106-3. The encoding unit 106-0 performs a prediction encoding process such
20 as H.264/AVC or H.265/HEVC on the basic format image data Vb for transmission,
and generates a basic video stream STb. In this case, the encoding unit 106-0
performs prediction within the image data Vb.
[0039]
The encoding unit 106-1 performs the prediction encoding process such as
25 H.264/AVC or H.265/HEVC on the high-quality format image data Vh1 for
transmission, and generates an extended video stream STe1. In this case, the
encoding unit 106-1 selectively performs prediction in the image data Vh1 or
prediction between the image data Vh1 and the image data Vb for each encoding
block to reduce a prediction residue.
30 [0040]
The encoding unit 106-2 performs the prediction encoding process such as
13
H.264/AVC or H.265/HEVC on the high-quality format image data Vh2 for
transmission, and generates an extended video stream STe2. In this case, the
encoding unit 106-2 selectively performs prediction in the image data Vh2 or
prediction between the image data Vh2 and the image data Vb for each encoding
5 block to reduce a prediction residue.
[0041]
The encoding unit 106-3 performs the prediction encoding process such as
H.264/AVC or H.265/HEVC on the high-quality format image data Vh3 for
transmission, and generates an extended video stream STe3. In this case, the
10 encoding unit 106-3 selectively performs prediction in the image data Vh3 or
prediction between the image data Vh3 and the image data Vh2 for each encoding
block to reduce a prediction residue.
[0042]
FIG. 4 shows a configuration example of a main portion of the encoding unit
15 160. The encoding unit 160 can be applied to the encoding unit 106-1, 106-2, or
106-3. The encoding unit 160 includes an intra-layer prediction unit 161, an interlayer
prediction unit 162, a prediction adjusting unit 163, a selection unit 164, and an
encoding function unit 165.
[0043]
20 The intra-layer prediction unit 161 obtains, with respect to image data V1 to
be encoded, prediction residual data by performing prediction (intra-layer prediction)
in the image data V1 to be encoded. The inter-layer prediction unit 162 obtains,
with respect to the image data V1 to be encoded, prediction residual data by
performing prediction (inter-layer prediction) between the image data V1 and image
25 data V2 to be referred to.
[0044]
The prediction adjusting unit 163 performs the following processes
according to a type of scalable extension of the image data V1 to the image data V2
to efficiently perform inter-layer prediction by the inter-layer prediction unit 162.
30 In a case of dynamic range extension, level adjustment for converting an LDR into a
HDR is performed. In a case of spatial scalable extension, a block is enlarged to a
14
predetermined size. In a case of frame rate extension, a process is bypassed. In a
case of color gamut extension, mapping is performed on each of brightness and color
difference. In a case of bit length extension, a conversion to align MSBs of a pixel
is performed.
5 [0045]
For example, in a case of the encoding unit 106-1, the image data V1 is a
high-quality format image data Vh1’ (100 Hz, LDR), the image data V2 is basic
format image data Vb’ (50 Hz, LDR), and the type of scalable extension is frame rate
extension. For this reason, the image data Vb’ is bypassed as it is by the prediction
10 adjusting unit 163.
[0046]
In addition, for example, in a case of the encoding unit 106-2, the image
data V1 is a high-quality format image data Vh2’ (50 Hz, HDR), the image data V2
is the basic format image data Vb’ (50 Hz, LDR), and the type of scalable extension
15 is dynamic range extension. For this reason, the level adjustment for converting an
LDR into a HDR is performed on the image data Vb’ by the prediction adjusting unit
163.
[0047]
Moreover, for example, in a case of the encoding unit 106-3, the image data
20 V1 is a high-quality format image data Vh3’ (100 Hz, HDR), the image data V2 is
the high-quality format image data Vh2’ (50 Hz, HDR), and the type of scalable
extension is frame rate extension. For this reason, the image data Vb’ is bypassed
as it is by the prediction adjusting unit 163.
[0048]
25 The selection unit 164 selectively takes out either prediction residual data
obtained by the intra-layer prediction unit 161 or prediction residual data obtained by
the inter-layer prediction unit 162 for each encoding block, and transmits the result to
the encoding function unit 165. In this case, the selection unit 164 takes out, for
example, a smaller one of these pieces of prediction residual data. The encoding
30 function unit 165 obtains a video stream ST by performing an encoding process such
as conversion encoding, quantization, or entropy encoding on the prediction residual
15
data taken out by the selection unit 164.
[0049]
Returning to FIG. 2, the video encoder 106 inserts identification information
of each corresponding high-quality format into layers of the extended video streams
STe1, STe2, and STe3. 5 The video encoder 106 inserts identification information
into, for example, a header of the NAL unit.
[0050]
FIG. 5(a) shows a structure example (Syntax) of an NAL unit header, and
FIG. 5(b) shows a content (Semantics) of a main parameter in the structure example.
10 A 1-bit field of “Forbidden_zero_bit” is required to be zero. A 6-bit field of
“nal_unit_type” shows a type of an NAL unit. A 6-bit field of “Nuh_layer_id” is an
ID showing a layer extension type of a stream. A 3-bit field of
“nuh_temporal_id_plus1” shows a temporal_id (0 to 6), and takes a value (1 to 7)
obtained by adding 1 thereto.
15 [0051]
In the embodiment, the 6-bit field of “nuh_layer_id” shows identification
information (extended category information of a stream) in a high-quality format
corresponding to each extended video stream. For example, “0” shows a base
stream. “1 to 4” show spatially-extended streams. “5 to 8” show frame rate20
extended streams. “9 to 12” show dynamic range-extended streams. “13 to 16”
show color gamut-extended streams. “17 to 20” show bit length-extended streams.
“21 to 24” show spatial extension and frame rate extension. “25 to 28” show frame
rate extension and dynamic range extension.
[0052]
25 For example, the basic video stream STb corresponds to a base stream, and
thus “nuh_layer_id” in the header of the NAL unit which configures the basic video
stream STb is set to “0”. In addition, for example, the extended video stream STe1
corresponds to a frame rate-extended stream, and thus “nuh_layer_id” in the header
of the NAL unit which configures the extended video stream STe1 is set to any of a
30 range of “5 to 8”.
[0053]
16
Moreover, for example, the extended video stream STe2 corresponds to a
dynamic range-extended stream, and thus “nuh_layer_id” in the header of the NAL
unit which configures the extended video stream STe2 is set to any of a range of “9
to 12”. Furthermore, for example, the extended video stream STe3 corresponds to a
stream of frame rate extension and 5 dynamic range extension, and thus
“nuh_layer_id” in the header of the NAL unit which configures the extended video
stream STe3 is set to any of a range of “25 to 28”.
[0054]
FIG. 6 shows a configuration example of the basic video stream STb and the
10 extended video streams STe1, STe2, and STe3. A horizontal axis represents a
display order (picture order of composition (POC)), the left side is set to be prior to a
display time, and the right side is set to be after the display time. Each of
rectangular frames represents a picture, and a solid line arrow represents a reference
relationship of pictures in prediction encoding.
15 [0055]
The basic video stream STb is configured by encoding image data of
pictures of “00”, “01”, …. The extended video stream STe1 is configured by
encoded image data of pictures of “10”, “11”, … positioned between respective
pictures of the basic video stream STb. The extended video stream STe2 is
20 configured by encoded image data of pictures of “20”, “21”, … at the same positions
as respective pictures of the basic video stream STb. Then, the extended video
stream STe3 is configured by encoded image data of pictures of “30”, “31”, …
positioned between respective pictures of the extended video stream STe2.
[0056]
25 Returning to FIG. 2, the system encoder 107 generates a transport stream TS
including the basic video stream STb and the extended video streams STe1, STe2,
and STe3 generated by the video encoder 106. Then, the transmission unit 108
places the transport stream TS on a broadcast wave or a net packet and transmits the
result to the reception device 200.
30 [0057]
At this time, the system encoder 107 inserts a high-quality format
17
corresponding to each of the extended video streams STe1, STe2, and STe3 into a
layer of the container (transport stream). In the embodiment, for example, the
system encoder 107 inserts a scalable extension descriptor including identification
information into a video elementary stream loop corresponding to each of the
extended video stre 5 ams present under the program map table (PMT).
[0058]
FIG. 7 shows a structure example (Syntax) of the scalable extension
descriptor. FIG. 8 shows a content (Semantics) of principle information in the
structure example shown in FIG. 7. An 8-bit field of “descriptor_tag” shows a
10 descriptor type, and herein shows a scalable extension descriptor. An 8-bit field of
“descriptor_length” shows a length (size) of a descriptor, and shows the following
number of bytes as the length of the descriptor.
[0059]
A 4-bit field of “type of enhancement” shows identification information
15 (extended category information of a stream) in a high-quality format corresponding
to each of the extended video streams. For example, “1” represents spatial scalable
extension. “2” represents frame rate scalable extension. “3” represents dynamic
range scalable extension. “4” represents color gamut scalable extension. “5”
represents bit length scalable extension. “6” represents spatial and frame rate
20 scalable extension. “7” represents frame-rate and dynamic range scalable extension.
[0060]
For example, the extended video stream STe1 corresponds to frame rate
scalable extension, and thus “type of enhancement” of a scalable extension descriptor
corresponding to the extended video stream STe1 is set to “2”.
25 [0061]
In addition, for example, the extended video stream STe2 corresponds to
dynamic range scalable extension, and thus “type of enhancement” of a scalable
extension descriptor corresponding to the extended video stream STe2 is set to “3”.
[0062]
30 Moreover, for example, the extended video stream STe3 corresponds to
frame rate/dynamic range scalable extension, and thus “type of enhancement” of a
18
scalable extension descriptor corresponding to the extended video stream STe3 is set
to “7”.
[0063]
FIG. 9 shows a correspondence relationship between a value of this “type of
enhancement” field and a value of “nuh_layer_id” field 5 of the NAL unit header. In
this manner, it can be understood that identification information (extended category
information of a stream) in a high-quality format corresponding to each of the
extended video streams, even if from any field, can be grasped similarly.
[0064]
10 Returning to FIG. 7, a 4-bit field of “scalable_priority” shows a priority in
the same extended category of each of the extended video streams. That is, the field
shows whether each of the extended video streams is generated by performing the
prediction encoding process between the high-quality format image data and basic
format image data, respectively, or generated by performing the prediction encoding
15 process between the high-quality format image data and high-quality format image
data, respectively.
[0065]
For example, “0” represents a first priority stream referencing a basic stream,
that is, that each of the extended video streams is generated by performing the
20 prediction encoding process between the high-quality format image data and basic
format image data. In addition, for example, “1” represents a second priority
stream referencing the first priority stream, that is, that each of the extended video
streams is generated by performing the prediction encoding process between the
high-quality format image data and high-quality format image data.
25 [0066]
For example, the extended video stream STe1 is related to encoding of the
high-quality format image data Vh1’ and is generated by performing a prediction
encoding process between the high-quality format image data Vh1’ and the basic
format image data Vb’. For this reason, “scalable_priority” of a scalable extension
30 descriptor corresponding to the extended video stream STe1 is set to “0”.
[0067]
19
In addition, for example, the extended video stream STe2 is related to
encoding of the high-quality format image data Vh2’ and is generated by performing
a prediction encoding process between the high-quality format image data Vh2’ and
the basic format image data Vb’. For this reason, “scalable_priority” of a scalable
extension descriptor corresponding 5 to the extended video stream STe2 is set to “0”.
[0068]
In addition, for example, the extended video stream STe3 is related to
encoding of the high-quality format image data Vh3’ and is generated by performing
the prediction encoding process between the high-quality format image data Vh3’
10 and the high-quality format image data Vh2’. For this reason, “scalable_priority”
of a scalable extension descriptor corresponding to the extended video stream STe3
is set to “1”.
[0069]
A 32-bit field of “enhancement reference PID” shows a PID value of a
15 reference stream. In other words, this field shows a PID value of a video stream
corresponding to image data referred to in the prediction encoding process between
the high-quality format image data and basic format image data or between the highquality
format image data and other high-quality format image data, which is
performed when each of the extended video streams is generated, respectively.
20 [0070]
For example, the extended video stream STe1 is related to encoding of the
high-quality format image data Vh1’ and is generated by performing the prediction
encoding process between the high-quality format image data Vh1’ and the basic
format image data Vb’. For this reason, “enhancement reference PID” of a scalable
25 extension descriptor corresponding to the extended video stream STe1 shows a PID
value of the basic video stream STb.
[0071]
In addition, for example, the extended video stream STe2 is related to
encoding of the high-quality format image data Vh2’ and is generated by performing
30 the prediction encoding process between the high-quality format image data Vh2’
and the basic format image data Vb’. For this reason, “enhancement reference PID”
20
of a scalable extension descriptor corresponding to the extended video stream STe2
shows the PID value of the basic video stream STb.
[0072]
In addition, for example, the extended video stream STe3 is related to
encoding of the high-quality format image data Vh3’ 5 and is generated by performing
a prediction encoding process between the high-quality format image data Vh3’ and
the high-quality format image data Vh2’. For this reason, “enhancement reference
PID” of a scalable extension descriptor corresponding to the extended video stream
STe3 shows a PID value of the extended video stream STe2.
10 [0073]
[Configuration of transport stream TS]
FIG. 10 shows a configuration example of a transport stream TS. The
transport stream TS includes four video streams of the basic video stream STb and
the extended video streams STe1, STe2, and STe3. In the configuration example, a
15 PES packet “video PES” of each video stream is present.
[0074]
A packet identifier (PID) of the basic video stream STb is set to, for
example, PID1. In encoding image data of each picture of the video stream, NAL
units such as AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS are present.
20 “nuh_layer_id” in headers of these NAL units is set to “0”, which represents a basic
video stream (refer to FIG. 9).
[0075]
In addition, a packet identifier (PID) of the extended video stream STe1 is
set to, for example, PID2. In encoding image data of each picture of the video
25 stream, NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS are
present. “nuh_layer_id” in headers of these NAL units is set to “5”, for example,
which represents a frame rate-extended stream (refer to FIG. 9).
[0076]
Moreover, a packet identifier (PID) of the extended video stream STe2 is set
30 to, for example, PID3. In encoding image data of each picture of the video stream,
NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS are present.
21
“nuh_layer_id” in headers of these NAL units is set to “9”, which represents a
dynamic range-extended stream (refer to FIG. 9).
[0077]
Furthermore, a packet identifier (PID) of the extended video stream STe3 is
set to, for example, PID4. In encoding image 5 data of each picture of the video
stream, NAL units such as AUD, SPS, PPS, PSEI, SLICE, SSEI, and EOS are
present. “nuh_layer_id” in headers of these NAL units is set to “25”, for example,
which represents a frame rate-extended and dynamic range-extended stream (refer to
FIG. 9).
10 [0078]
In addition, the transport stream TS includes a program map table (PMT) as
program specific information (PSI). This PSI is information describing to which
program each elementary stream included in the transport stream is affiliated.
[0079]
15 A program loop which describes information related to an entire program is
present in the PMT. In addition, an elementary stream loop having information
related to each elementary stream is present in the PMT. In this configuration
example, four video elementary stream loops (video ES loops) corresponding to four
video streams of the basic video stream STb and the extended video streams STe1,
20 STe2, and STe3 are present. Information of a stream type (ST0), a packet identifier
(PID1), and the like is disposed in a video elementary stream loop corresponding to
the basic video stream STb.
[0080]
In addition, information of a stream type (ST1), a packet identifier (PID2),
25 and the like is disposed, and a descriptor which describes information related to the
extended video stream STe1 is disposed in a video elementary stream loop
corresponding to the extended video stream STe1. As one of the descriptors, the
scalable extension descriptor described above is inserted.
[0081]
30 “type of enhancement” in the descriptor is set to “2”, which represents a
frame rate-extended stream (frame rate scalable extension) (refer to FIG. 9). In
22
addition, “scalable_priority” in the descriptor is set to “0”, which represents the first
priority stream referencing the basic stream. Moreover, “enhancement reference
PID” in the descriptor is set to “PID1”, which represents reference to the basic video
stream STb.
5 [0082]
In addition, information of a stream type (ST2), a packet identifier (PID3),
and the like is disposed, and a descriptor which describes information related to the
extended video stream STe2 is disposed in a video elementary stream loop
corresponding to the extended video stream STe2. As one of the descriptors, the
10 scalable extension descriptor described above is inserted.
[0083]
“type of enhancement” in the descriptor is set to “3”, which represents a
dynamic range-extended stream (dynamic range scalable extension) (refer to FIG. 9).
In addition, “scalable_priority” in the descriptor is set to “0”, which represents the
15 first priority stream referencing a basic stream. Moreover, “enhancement reference
PID” in the descriptor is set to “PID1”, which represents reference to the basic video
stream STb.
[0084]
In addition, information of a stream type (ST3), a packet identifier (PID4),
20 and the like is disposed, and a descriptor which describes information related to the
extended video stream STe3 is disposed in a video elementary stream loop
corresponding to the extended video stream STe3. As one of the descriptors, the
scalable extension descriptor described above is inserted.
[0085]
25 “type of enhancement” in the descriptor is set to “7”, which represents a
frame rate-extended and dynamic range-extended stream (frame rate/dynamic range
scalable extension) (refer to FIG. 9). In addition, “scalable_priority” in the
descriptor is set to “1, which represents a second priority stream referencing the first
priority stream. Moreover, “enhancement reference PID” in the descriptor is set to
30 “PID3”, which represents reference to the extended video stream STe2.
[0086]
23
An operation of the transmission device 100 shown in FIG. 2 will be briefly
described. The basic format image data Vb which is an LDR image data with a
frame frequency of 50 Hz is supplied to the LDR photoelectric conversion unit 102.
The LDR photoelectric conversion unit 102 applies photoelectric conversion
characteristics (LDR OETF curve) for an LDR image 5 to the basic format image data
Vb, and thereby the basic format image data Vb’ for transmission is obtained. The
basic format image data Vb’ is supplied to the encoding units 106-0, 106-1, and 106-
2 of the video encoder 106.
[0087]
10 In addition, the high-quality format image data Vh1 which is an LDR image
data with a frame frequency of 100 Hz is supplied to the LDR photoelectric
conversion unit 103. The LDR photoelectric conversion unit 103 applies the
photoelectric conversion characteristics (LDR OETF curve) for an LDR image to the
high-quality format image data Vh1, and thereby the high-quality format image data
15 Vh1’ for transmission is obtained. The high-quality format image data Vh1’ is
supplied to the encoding unit 106-1 of the video encoder 106.
[0088]
Moreover, the high-quality format image data Vh2 which is a HDR image
data with a frame frequency of 50 Hz is supplied to the HDR photoelectric
20 conversion unit 104. The HDR photoelectric conversion unit 104 applies
photoelectric conversion characteristics (HDR OETF curve) for a HDR image to the
high-quality format image data Vh2, and thereby the high-quality format image data
Vh2’ for transmission is obtained. The high-quality format image data Vh2’ is
supplied to the encoding units 106-2 and 106-3 of the video encoder 106.
25 [0089]
Furthermore, the high-quality format image data Vh3 which is a HDR image
data with a frame frequency of 100 Hz is supplied to the HDR photoelectric
conversion unit 105. The HDR photoelectric conversion unit 105 applies the
photoelectric conversion characteristics (HDR OETF curve) for a HDR image to the
30 high-quality format image data Vh3, and thereby the high-quality format image data
Vh3’ for transmission is obtained. The high-quality format image data Vh3’ is
24
supplied to the encoding unit 106-3 of the video encoder 106.
[0090]
The video encoder 106 performs an encoding process on each of the basic
format image data Vb’ and the high-quality format image data Vh1’, Vh2’, and Vh3’,
and thereby a video stream is generated. 5 In other words, the encoding unit 106-0
performs a prediction encoding process such as H.264/AVC or H.265/HEVC on the
basic format image data Vb’ for transmission, and thereby the basic video stream
STb including encoding image data of each picture is generated. In this case,
prediction in the image data Vb’ is performed by the encoding unit 106-0.
10 [0091]
In addition, the encoding unit 106-1 performs the prediction encoding
process such as H.264/AVC or H.265/HEVC on the high-quality format image data
Vh1’ for transmission, and thereby the extended video stream STe1 including
encoding image data of each picture is generated. In this case, the encoding unit
15 106-1 selectively performs prediction in the image data Vh1’ or prediction between
the high-quality format image data Vh1’ and the image data Vb’ for each encoding
block to reduce a prediction residue.
[0092]
Moreover, the encoding unit 106-2 performs the prediction encoding
20 process such as H.264/AVC or H.265/HEVC on the high-quality format image data
Vh2’ for transmission, and thereby the extended video stream STe2 including
encoding image data of each picture is generated. In this case, the encoding unit
106-2 selectively performs prediction in the image data Vh2’ or prediction between
the image data Vh2’ and the image data Vb’ for each encoding block to reduce the
25 prediction residue.
[0093]
Furthermore, the encoding unit 106-3 performs the prediction encoding
process such as H.264/AVC or H.265/HEVC on the high-quality format image data
Vh3’ for transmission, and thereby the extended video stream STe3 including
30 encoding image data of each picture is generated. In this case, the encoding unit
106-3 selectively performs prediction in the image data Vh3’ or the prediction
25
between the image data Vh3’ and the image data Vh2’ for each encoding block to
reduce the prediction residue.
[0094]
In addition, the video encoder 106 inserts identification information in each
corresponding high-quality format into 5 layers of the extended video streams STe1,
STe2, and STe3, respectively. That is, the video encoder 106 sets identification
information (extended category information of a stream) in a high-quality format
corresponding to each of the extended video streams in a field of “nuh_layer_id” in
the header of the NAL unit (refer to FIGS. 5 and 9).
10 [0095]
The basic video stream STb and the extended video streams STe1, STe2,
and STe which are generated by the video encoder 106 are supplied to the system
encoder 107. The system encoder 107 generates a transport stream TS including
each video stream.
15 [0096]
The system encoder 107 inserts identification information in a high-quality
format corresponding to each of the extended video streams STe1, STe2, and STe3
into the layer of the container (transport stream). In other words, the system
encoder 107 inserts a scalable extension descriptor including identification
20 information (extended category information of a stream) into a video elementary
stream loop corresponding to each of the extended video streams present under the
PMT (refer to FIGS. 7 and 9).
[0097]
The transport stream TS generated by the system encoder 107 is transmitted
25 to the transmission u nit 108. The transmission unit 108 places the transport stream
TS on a broadcast wave or a net packet and transmits the result to the reception
device 200.
[0098]
[Configuration of reception device]
30 FIG. 11 shows a configuration example of the reception device 200. The
reception device 200 corresponds to a configuration example of the transmission
26
device 100 of FIG. 2. The reception device 200 includes a control unit 201, a
reception unit 202, a system decoder 203, a video decoder 204, LDR electric-photo
conversion units 205 and 206L, HDR electric-photo conversion units 207 and 208,
and a display unit (display device) 209. The control unit 201 is configured to
include a central processing unit (CPU), and controls 5 operations of each unit of the
reception device 200 based on a control program stored in a storage not shown.
[0099]
The reception unit 202 receives the transport stream TS placed on a
broadcast wave or a net packet and transmitted from the transmission device 100.
10 The system decoder 203 extracts the basic video stream STb and the extended video
streams STe1, STe2, and STe3 from the transport stream TS.
[0100]
In addition, the system decoder 203 extracts various types of information
inserted into the layer of the container (transport stream) and transmits the
15 information to the control unit 201. The information also includes the scalable
extension descriptor described above. The control unit 201 can grasp identification
information (extended category information of a stream) in a high-quality format
corresponding to each of the extended video streams STe1, STe2, and STe3 from a
field of “type of enhancement” of the descriptor.
20 [0101]
Moreover, the control unit 201 can grasp a priority in the same extended
category in each of the extended video streams STe1, STe3, and STe3, that is,
whether the first priority stream referencing the basic stream or the second priority
stream referencing the first priority stream, from a field of “scalable_priority” of the
25 descriptor. Furthermore, the control unit 201 can grasp a PID value of a video
stream referred to by each of the extended video streams STe1, STe2, and STe3 from
a field of “enhancement reference PID” of the descriptor.
[0102]
The video decoder 204 includes four decoding units 204-0, 204-1, 204-2,
30 and 204-3. The decoding unit 204-0 performs a decoding process on the basic
video stream STb and generates the basic format image data Vb’. In this case, the
27
decoding unit 204-0 performs prediction compensation within the image data Vb’.
[0103]
The decoding unit 204-1 performs a decoding process on the extended video
stream STe1 and generates the high-quality format image data Vh1’. In this case,
the decoding unit 204-1 performs prediction 5 compensation in the image data Vh1’ or
prediction compensation between the image data Vh1’ and the image data Vb’ for
each encoding block in accordance with prediction at a time of encoding.
[0104]
The decoding unit 204-2 performs a decoding process on the extended video
10 stream STe2 and generates the high-quality format image data Vh2’. In this case,
the decoding unit 204-2 performs prediction compensation in the image data Vh2’ or
prediction compensation between the image data Vh2’ and the image data Vb’ for
each encoding block in accordance with prediction at a time of encoding.
[0105]
15 The decoding unit 204-3 performs a decoding process on the extended video
stream STe3 and generates the high-quality format image data Vh3’. In this case,
the decoding unit 204-3 performs prediction compensation in the image data Vh3’ or
the prediction compensation between the image data Vh3’ and the image data Vh2’
for each encoding block in accordance with prediction at a time of encoding.
20 [0106]
FIG. 12 shows a configuration example of a main portion of the decoding
unit 240. The decoding unit 240 can be applied to the decoding units 204-1, 204-2,
and 204-3. The decoding unit 240 performs a process reverse to a process of the
encoding unit 165 of FIG. 4. The decoding unit 240 includes a decoding function
25 unit 241, an intra-layer prediction compensation unit 242, an inter-layer prediction
compensation unit 243, a prediction adjusting unit 244, and a selection unit 245.
[0107]
The decoding function unit 241 performs a decoding process other than the
prediction compensation on the video stream ST and obtains prediction residual data.
30 The intra-layer prediction compensation unit 242 performs prediction compensation
(intra-layer prediction compensation) in the image data V1 on the prediction residual
28
data and obtains the image data V1. The inter-layer prediction compensation unit
243 performs prediction compensation (inter-layer prediction compensation) between
the image data V1 and the image data V2 to be referred to on the prediction residual
data and obtains the image data V1.
5 [0108]
The prediction adjusting unit 244, even if detailed description is omitted,
performs a process according to a type of scalable extension of the image data V1 to
the image data V2 in the same manner as the prediction adjusting unit 163 of the
encoding unit 160 of FIG. 4. The selection unit 245 selectively takes out and
10 outputs either an image data V1 obtained by the intra-layer prediction compensation
unit 242 or an image data V1 obtained by the inter-layer prediction compensation
unit 243 for each encoding block, corresponding to prediction at a time of encoding.
[0109]
Returning to FIG. 11, the video decoder 204 transmits header information of
15 an NAL unit of each video stream to the control unit 201. The control unit 201 can
grasp identification information (extended category information of a stream) in a
high-quality format corresponding to each of the extended video streams STe1, STe2,
and STe3 from a field of “nuh_layer_id” of the header information.
[0110]
20 The LDR electric-photo conversion unit 205 performs electric-photo
conversion of characteristics, reverse to the conversion characteristics of the LDR
photoelectric conversion unit 102 in the transmission device 100 described above, on
the basic format image data Vb’ obtained by the decoding unit 204-0, and obtains the
basic format image data Vb. The basic format image data is an LDR image data
25 whose frame frequency is 50 Hz.
[0111]
In addition, the LDR electric-photo conversion unit 206 performs electricphoto
conversion of characteristics, reverse to the conversion characteristics of the
LDR photoelectric conversion unit 103 in the transmission device 100 described
30 above, on the high-quality format image data Vh1’ obtained by the decoding unit
204-1, and obtains the high-quality format image data Vh1. The high-quality
29
format image data Vh1 is an LDR image data whose frame frequency is 100 Hz.
[0112]
Moreover, the HDR electric-photo conversion unit 207 performs electricphoto
conversion of characteristics, reverse to the conversion characteristics of the
HDR photoelectric conversion 5 unit 104 in the transmission device 100 described
above, on the high-quality format image data Vh2’ obtained by the decoding unit
204-2, and obtains the high-quality format image data Vh2. The high-quality
format image data Vh2 is a HDR image data whose frame frequency is 50 Hz.
[0113]
10 Furthermore, the HDR electric-photo conversion unit 208 performs electricphoto
conversion of characteristics, reverse to the conversion characteristics of the
HDR photoelectric conversion unit 105 in the transmission device 100 described
above, on the high-quality format image data Vh3’ obtained by the decoding unit
204-3, and obtains the high-quality format image data Vh3. The high-quality
15 format image data Vh3 is a HDR image data whose frame frequency is 100 Hz.
[0114]
The display unit 209 is configured by, for example, a liquid crystal display
(LCD), an organic electro-luminescence (EL) panel, and the like. The display unit
209 displays an image caused by any of the basic format image data Vb and the high20
quality format image data Vh1, Vh2, and Vh3 according to display capability.
[0115]
In this case, the control unit 201 controls image data such that it is supplied
to the display unit 209. The control is performed based on identification
information (extended category information of a stream) in a high-quality format
25 corresponding to each of the extended video streams STe1, STe2, and STe3 grasped
by the control unit 201 as described above and display capability information of the
display unit 209.
[0116]
That is, when the display unit 209 cannot perform either a display of a high
30 frame frequency or a display of a high dynamic range, the control is performed so
that the basic format image data Vb according to decoding of the basic video stream
30
STb is supplied to the display unit 209. In this case, the control unit 201 controls so
that the decoding unit 204-0 decodes the basic video stream STb and the LDR
electric-photo conversion unit 205 outputs the basic format image data Vb.
[0117]
In addition, when the display unit 209 can 5 perform the display of a high
frame frequency but cannot perform the display of a high dynamic range, the control
is performed so that the high-quality format image data Vh1 according to decoding
of the extended video stream STe1 is supplied to the display unit 209. In this case,
the control unit 201 controls so that the decoding unit 204-0 decodes the basic video
10 stream STb, the decoding unit 204-1 decodes the extended video stream STe1, and
the LDR electric-photo conversion unit 206 outputs the high-quality format image
data Vh1.
[0118]
In addition, when the display unit 209 cannot perform the display of a high
15 frame frequency but can perform the display of a high dynamic range, the control is
performed so that the high-quality format image data Vh2 according to decoding of
the extended video stream STe2 is supplied to the display unit 209. In this case, the
control unit 201 controls so that the decoding unit 204-0 decodes the basic video
stream STb, the decoding unit 204-2 decodes the extended video stream STe2, and
20 the HDR electric-photo conversion unit 207 outputs the high-quality format image
data Vh2.
[0119]
In addition, when the display unit 209 can perform both the display of a
high frame frequency and the display of a high dynamic range, the control is
25 performed so that the high-quality format image data Vh3 according to decoding of
the extended video stream STe3 is supplied to the display unit 209. In this case, the
control unit 201 controls so that the decoding unit 204-0 decodes the basic video
stream STb, the decoding unit 204-2 decodes the extended video stream STe2, the
decoding unit 204-3 decodes the extended video stream STe3, and the HDR electric30
photo conversion unit 208 outputs the high-quality format image data Vh3.
[0120]
31
Operations of the reception device 200 shown in FIG. 11 will be briefly
described. The reception unit 202 receives a transport stream TS placed on a
broadcast wave or a net packet and transmitted from the transmission device 100.
The transport stream TS is supplied to the system decoder 203. The system decoder
203 extracts the basic video stream STb and 5 the extended video streams STe1, STe2,
and STe3 from the transport stream TS.
[0121]
In addition, the system decoder 203 extracts various types of information
inserted into the layer of the container (transport stream) and transmits the
10 information to the control unit 201. The information also includes a scalable
extension descriptor. The control unit 201 grasps identification information
(extended category information of a stream) in a high-quality format corresponding
to each of the extended video streams STe1, STe2, and STe3 from a field of “type of
enhancement” of the descriptor.
15 [0122]
When the display unit 209 cannot perform either the display of a high frame
frequency or the display of a high dynamic range, the basic format image data Vb is
supplied to the display unit 209 from the LDR electric-photo conversion unit 205.
An image according to the basic format image data Vb, that is, an LDR image data
20 whose frame frequency is 50 Hz, is displayed on the display unit 209.
[0123]
In this case, the basic video stream STb extracted by the system decoder 203
is supplied to the decoding unit 204-0. The decoding unit 204-0 performs a
decoding process on the basic video stream STb, and thereby the basic format image
25 data Vb’ is generated. Here, the decoding unit 204-0 can confirm that a supply
video stream is the basic video stream STb from a field of “nuh_layer_id” of the
header of the NAL unit.
[0124]
The basic format image data Vb’ generated by the decoding unit 204-0 is
30 supplied to the LDR electric-photo conversion unit 205. The LDR electric-photo
conversion unit 205 obtains the basic format image data Vb by performing electric32
photo conversion on the basic format image data Vb’ and the basic format image data
Vb is supplied to the display unit 209.
[0125]
In addition, when the display unit 209 can perform the display of a high
frame frequency but cannot perform the display of a 5 high dynamic range, the highquality
format image data Vh1 is supplied to the display unit 209 from the LDR
electric-photo conversion unit 206. An image according to the high-quality format
image data Vh1, that is, an LDR image data whose frame frequency is 100 Hz, is
displayed on the display unit 209.
10 [0126]
In this case, the basic video stream STb extracted by the system decoder 203
is supplied to the decoding unit 204-0. The decoding unit 204-0 performs a
decoding process on the basic video stream STb, and thereby the basic format image
data Vb’ is generated. In addition, the extended video stream STe1 extracted by the
15 system decoder 203 is supplied to the decoding unit 204-1. The decoding unit 204-
1 performs a decoding process on the extended video stream STe1 referencing the
basic format image data Vb’, and thereby the high-quality format image data Vh1’ is
generated.
[0127]
20 Here, the decoding unit 204-0 can confirm that a supply video stream is the
basic video stream STb from the field of “nuh_layer_id” of the header of the NAL
unit. In addition, the decoding unit 204-1 can confirm that a supply video stream is
the extended video stream STe1 from the field of “nuh_layer_id” of the header of the
NAL unit.
25 [0128]
The high-quality format image data Vh1’ generated by the decoding unit
204-1 is supplied to the LDR electric-photo conversion unit 206. The LDR electricphoto
conversion unit 206 obtains the high-quality format image data Vh1 by
performing electric-photo conversion on the high-quality format image data Vh1’ and
30 the high-quality format image data Vh1 is supplied to the display unit 209.
[0129]
33
In addition, when the display unit 209 cannot perform the display of a high
frame frequency but can perform the display of a high dynamic range, the highquality
format image data Vh2 is supplied to the display unit 209 from the HDR
electric-photo conversion unit 207. An image according to the high-quality format
image data Vh2, that is, a HDR image data whose 5 frame frequency is 50 Hz, is
displayed on the display unit 209.
[0130]
In this case, the basic video stream STb extracted by the system decoder 203
is supplied to the decoding unit 204-0. The decoding unit 204-0 performs a
10 decoding process on the basic video stream STb, and thereby the basic format image
data Vb’ is generated. In addition, the extended video stream STe2 extracted by the
system decoder 203 is supplied to the decoding unit 204-2. The decoding unit 204-
2 performs a decoding process on the extended video stream STe2 referring to the
basic format image data Vb’, and thereby the high-quality format image data Vh2’ is
15 generated.
[0131]
Here, the decoding unit 204-0 can confirm that a supply video stream is the
basic video stream STb from the field of “nuh_layer_id” of the header of the NAL
unit. In addition, the decoding unit 204-2 can confirm that a supply video stream is
20 the extended video stream STe1 from the field of “nuh_layer_id” of the header of the
NAL unit.
[0132]
The high-quality format image data Vh2’ generated by the decoding unit
204-2 is supplied to the HDR electric-photo conversion unit 207. The HDR
25 electric-photo conversion unit 207 obtains the high-quality format image data Vh2 by
performing electric-photo conversion on the high-quality format image data Vh2’ and
the high-quality format image data Vh2 is supplied to the display unit 209.
[0133]
In addition, when the display unit 209 can perform both the display of a
30 high frame frequency and the display of a high dynamic range, the high-quality
format image data Vh3 is supplied to the display unit 209 from the HDR electric34
photo conversion unit 208. An image according to the high-quality format image
data Vh3, that is, a HDR image data whose frame frequency is 100 Hz, is displayed
on the display unit 209.
[0134]
In this case, the basic video stream S 5 Tb extracted by the system decoder 203
is supplied to the decoding unit 204-0. The decoding unit 204-0 performs a
decoding process on the basic video stream STb, and thereby the basic format image
data Vb’ is generated. In addition, the extended video stream STe2 extracted by the
system decoder 203 is supplied to the decoding unit 204-2. The decoding unit 204-
10 2 performs a decoding process on the extended video stream STe2 referencing the
basic format image data Vb’, and thereby the high-quality format image data Vh2’ is
generated.
[0135]
Furthermore, the extended video stream STe3 extracted by the system
15 decoder 203 is supplied to the decoding unit 204-3. The decoding unit 204-3
performs a decoding process on the extended video stream STe3 referencing the
high-quality format image data Vh2’, and thereby the high-quality format image data
Vh3’ is generated.
[0136]
20 Here, the decoding unit 204-0 can confirm that a supply video stream is the
basic video stream STb from the field of “nuh_layer_id” of the header of the NAL
unit. In addition, the decoding unit 204-2 can confirm that a supply video stream is
the extended video stream STe1 from the field of “nuh_layer_id” of the header of the
NAL unit. Moreover, the decoding unit 204-3 can confirm that a supply video
25 stream is the extended video stream STe3 from the field of “nuh_layer_id” of the
header of the NAL unit.
[0137]
The high-quality format image data Vh3’ generated by the decoding unit
204-3 is supplied to the HDR electric-photo conversion unit 208. The HDR
30 electric-photo conversion unit 208 obtains the high-quality format image data Vh3 by
performing electric-photo conversion on the high-quality format image data Vh3’ and
35
the high-quality format image data Vh3 is supplied to the display unit 209.
[0138]
As described above, in the transmission and reception system 10 shown in
FIG. 1, identification information (extended category information of a stream) in a
high-quality format corresponding to each 5 of the predetermined number of extended
video streams included in the transport stream TS is inserted into the layer of the
container or the video stream and transmitted by the transmission device 100. For
this reason, it is easy to obtain image data in accordance with display capability on a
receiving side by selectively performing a decoding process on a predetermined
10 video stream based on the identification information.
[0139]
<2. Modification example>
In the embodiment described above, an example is shown in which
identification information (extended category information of a stream) in a high15
quality format corresponding to each of the predetermined number of extended video
streams included in the transport stream TS is inserted into the layer of both the
container and the video stream and transmitted. However, the identification
information being inserted into the layer of the container only or the layer of the
video stream only may also be considered.
20 [0140]
Moreover, instead of transmitting information which indicates an ID
representing a layer extension type of a stream, an extended category of a stream,
and a priority in the extended category, it is possible to indicate a combination state
of these using a value of “stream_type”. For example, as shown in FIG. 10, a basic
25 stream can be set as “Stream_type=ST0”, a first stream of frame rate scalable
extended stream can be set as “Stream_type=ST1”, a first stream of dynamic range
scalable extended stream can be set as “Stream_type=ST2”, and a frame
rate/dynamic range scalable extended stream (second extended stream) can be set as
“Stream_type=ST3”.
30 [0141]
It should be noted that, although the above-described embodiment shows the
36
transmission and reception system 10 constituted by the transmission device 100 and
the reception device 200, a configuration of a transmission and reception system to
which the present technology can be applied is not limited thereto. For example,
the reception device 200 part may be configured with, for example, a set-top box
connected with a digital interface such as 5 High-Definition Multimedia Interface
(HDMI) and a monitor, or the like. In this case, the set-top box can obtain display
capability information by acquiring extended display identification data (EDID) from
a monitor, and the like. “HDMI” is a registered trademark.
[0142]
10 In addition, the above-described embodiment shows the example in which
the container is a transport stream (MPEG-2 TS). The present technology, however,
can also be similarly applied to a system configured to distribute data to a reception
terminal using a network such as the Internet. In distribution on the Internet, there
are many cases of distribution using a container in MP4 or other formats. In other
15 words, as containers, various formats including transport streams (MPEG-2 TS)
employed in digital broadcasting standards, MP4 used in Internet distribution, and
the like are equivalent thereto.
[0143]
Additionally, the present technology may also be configured as below.
20 (1)
A transmission device including:
an image encoding unit which generates a basic video stream obtained by
encoding basic format image data and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
25 format image data;
a transmission unit which transmits a container in a predetermined format
including the basic video stream and the predetermined number of extended video
streams generated by the image encoding unit; and
an identification information insertion unit which inserts identification
30 information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the container.
37
(2)
The transmission device according to (1),
wherein the image encoding unit
generates, with respect to the basic format image data, the basic video
stream by performing a prediction encoding 5 process in the basic format image data,
and
generates, with respect to the high-quality format image data, the extended
video stream by selectively performing a prediction encoding process in the highquality
format image data or a prediction encoding process between the high-quality
10 format image data and the basic format image data or between the high-quality
format image data and other high-quality format image data.
(3)
The transmission device according to (2),
wherein the identification information inserted into a layer of the container
15 includes information indicating whether each of the predetermined number of
extended video streams is generated by performing a prediction encoding process
between the high-quality format image data and the basic format image data or by
performing a prediction encoding process between the high-quality format image
data and the high-quality format image data.
20 (4)
The transmission device according to (2) or (3),
wherein the identification information inserted into a layer of the container
includes information indicating a video stream corresponding to image data referred
to in a prediction encoding process between the high-quality format image data and
25 the basic format image data or between the high-quality format image data and other
high-quality format image data performed when each of the predetermined number
of extended video streams is generated.
(5)
The transmission device according to any of (1) to (4),
30 wherein the container is an MPEG2-TS, and
the identification information insertion unit inserts the identification
38
information into respective video elementary stream loops corresponding to the
predetermined number of extended video streams present under a program map table.
(6)
The transmission device according to any of (1) to (5),
wherein 5 the identification information insertion unit further inserts
identification information in a high-quality format corresponding to each of the
predetermined number of extended video streams into a layer of the video stream.
(7)
The transmission device according to (6),
10 wherein the video stream has a structure of a NAL unit, and
the identification information insertion unit inserts the identification
information into a header of the NAL unit.
(8)
A transmission method including:
15 an image encoding step of generating a basic video stream obtained by
encoding basic format image data and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
format image data;
a transmission step of transmitting, by a transmission unit, a container in a
20 predetermined format including the basic video stream and the predetermined
number of extended video streams generated in the image encoding step; and
an identification information insertion step of inserting identification
information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the container.
25 (9)
A transmission device including:
an image encoding unit which generates a basic video stream obtained by
encoding basic format image data and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
30 format image data;
a transmission unit which transmits a container in a predetermined format
39
including the basic video stream and the predetermined number of extended video
streams generated by the image encoding unit; and
an identification information insertion unit which inserts identification
information in a high-quality format corresponding to each of the predetermined
5 number of extended video streams into a layer of the video stream.
(10)
The transmission device according to (9),
wherein the image encoding unit
generates, with respect to the basic format image data, the basic video
10 stream by performing a prediction encoding process in the basic format image data,
and
generates, with respect to the high-quality format image data, the extended
video stream by selectively performing a prediction encoding process in the highquality
format image data or a prediction encoding process between the high-quality
15 format image data and the basic format image data or between the high-quality
format image data and other high-quality format image data.
(11)
The transmission device according to (9) or (10),
wherein the video stream has a structure of a NAL unit, and
20 the identification information insertion unit inserts the identification
information into a header of the NAL unit.
(12)
A transmission method including:
an image encoding step of generating a basic video stream obtained by
25 encoding basic format image data and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
format image data;
a transmission step of transmitting, by a transmission unit, a container in a
predetermined format including the basic video stream and the predetermined
30 number of extended video streams generated in the image encoding step; and
an identification information insertion step of inserting identification
40
information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the video stream.
(13)
A reception device including:
a reception unit which receives a 5 container in a predetermined format
including a basic video stream obtained by encoding basic format image data and a
predetermined number of extended video streams obtained by encoding the
predetermined number of pieces of high-quality format image data,
wherein identification information in a high-quality format corresponding to
10 each of the predetermined number of extended video streams is inserted into a layer
of the container, and
the reception device further includes a process unit which processes each of
the video streams included in the received container based on the identification
information.
15 (14)
The reception device according to (13),
wherein the process unit acquires image data in accordance with display
capability by performing a decoding process on the basic video stream and a
predetermined one of the extended video streams based on the identification
20 information and display capability information.
(15)
The reception device according to (13) or (14),
wherein the basic video stream is generated by performing, with respect to
the basic format image data, a prediction encoding process in the basic format image
25 data, and
the extended video stream is generated by selectively performing, with
respect to the high-quality format image data, a prediction encoding process in the
high-quality format image data or a prediction encoding process between the highquality
format image data and the basic format image data or between the high30
quality format image data and other high-quality format image data.
(16)
41
A reception method including:
a reception step of receiving, by a reception unit, a container in a
predetermined format which includes a basic video stream obtained by encoding
basic format image data and a predetermined number of extended video streams
obtained by encoding the predetermined 5 number of pieces of high-quality format
image data,
wherein identification information in a high-quality format corresponding to
each of the predetermined number of extended video streams is inserted into a layer
of the container, and
10 the reception method further includes a process step of processing each of
the video streams included in the received container based on the identification
information.
(17)
A reception device including:
15 a reception unit which receives a container in a predetermined format
including a basic video stream obtained by encoding basic format image data and a
predetermined number of extended video streams obtained by encoding the
predetermined number of pieces of high-quality format image data,
wherein identification information in a high-quality format corresponding to
20 each of the predetermined number of extended video streams is inserted into a layer
of the video stream, and
the reception device further includes a process unit which processes each of
the video streams included in the received container based on the identification
information.
25 (18)
The reception device according to (17),
wherein the process unit acquires image data in accordance with display
capability by performing a decoding process on the basic video stream and a
predetermined one of the extended video streams based on the identification
30 information and display capability information.
(19)
42
The reception device according to (17) or (18),
wherein the basic video stream is generated by performing, with respect to
the basic format image data, a prediction encoding process in the basic format image
data, and
the extended video stream is generated 5 by selectively performing, with
respect to the high-quality format image data, a prediction encoding process in the
high-quality format image data or a prediction encoding process between the highquality
format image data and the basic format image data or between the highquality
format image data and other high-quality format image data.
10 (20)
A reception method including:
a reception step of receiving, by a reception unit, a container in a
predetermined format which includes a basic video stream obtained by encoding
basic format image data and a predetermined number of extended video streams
15 obtained by encoding the predetermined number of pieces of high-quality format
image data,
wherein identification information in a high-quality format corresponding to
each of the predetermined number of extended video streams is inserted into a layer
of the video stream, and
20 the reception method further includes a process step of processing each of
the video streams included in the received container based on the identification
information.
[0144]
A main feature of the present technology is that it is easy to obtain image
25 data in accordance with display capability on a receiving side by inserting and
transmitting identification information (extended category information of a stream)
in a high-quality format corresponding to each of the predetermined number of
extended video streams included in the transport stream TS into a layer of the
container or the video stream (refer to FIG. 10).
30
Reference Signs List
43
[0145]
10 transmission and reception system
100 transmission device
101 control unit
5 102, 103 LDR photoelectric conversion unit
104, 105 HDR photoelectric conversion unit
106 video encoder
106-0, 106-1, 106-1, 106-1 encoding unit
107 system encoder
10 108 transmission unit
150 image data generation unit
151 HDR camera
152, 154 frame rate conversion unit
153 dynamic range conversion unit
15 160 encoding unit
161 intra-layer prediction unit
162 inter-layer prediction unit
163 prediction adjusting unit
164 selection unit
20 165 encoding function unit
200 reception device
201 control unit
202 reception unit
203 system decoder
25 204 video decoder
204-0, 204-1, 204-1, 204-1 decoding unit
205, 206 LDR electric-photo conversion unit
207, 208 HDR electric-photo conversion unit
209 display unit
30 240 decoding unit
241 decoding function unit
44
242 intra-layer prediction compensation unit
243 inter-layer prediction compensation unit
244 prediction adjusting unit
245 selection unit
5
45
CLAIMS
Claim 1
A transmission device comprising:
an image encoding unit which generates a basic video stream obtained by
encoding basic format image data 5 and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
format image data;
a transmission unit which transmits a container in a predetermined format
including the basic video stream and the predetermined number of extended video
10 streams generated by the image encoding unit; and
an identification information insertion unit which inserts identification
information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the container.
15 Claim 2
The transmission device according to claim 1,
wherein the image encoding unit
generates, with respect to the basic format image data, the basic video
stream by performing a prediction encoding process in the basic format image data,
20 and
generates, with respect to the high-quality format image data, the extended
video stream by selectively performing a prediction encoding process in the highquality
format image data or a prediction encoding process between the high-quality
format image data and the basic format image data or between the high-quality
25 format image data and other high-quality format image data.
Claim 3
The transmission device according to claim 2,
wherein the identification information inserted into a layer of the container
30 includes information indicating whether each of the predetermined number of
extended video streams is generated by performing a prediction encoding process
46
between the high-quality format image data and the basic format image data or by
performing a prediction encoding process between the high-quality format image
data and the high-quality format image data.
5 Claim 4
The transmission device according to claim 2,
wherein the identification information inserted into a layer of the container
includes information indicating a video stream corresponding to image data referred
to in a prediction encoding process between the high-quality format image data and
10 the basic format image data or between the high-quality format image data and other
high-quality format image data performed when each of the predetermined number
of extended video streams is generated.
Claim 5
15 The transmission device according to claim 1,
wherein the container is an MPEG2-TS, and
the identification information insertion unit inserts the identification
information into respective video elementary stream loops corresponding to the
predetermined number of extended video streams present under a program map table.
20
Claim 6
The transmission device according to claim 1,
wherein the identification information insertion unit further inserts
identification information in a high-quality format corresponding to each of the
25 predetermined number of extended video streams into a layer of the video stream.
Claim 7
The transmission device according to claim 6,
wherein the video stream has a structure of a NAL unit, and
30 the identification information insertion unit inserts the identification
information into a header of the NAL unit.
47
Claim 8
A transmission method comprising:
an image encoding step of generating a basic video stream obtained by
encoding basic format image data 5 and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
format image data;
a transmission step of transmitting, by a transmission unit, a container in a
predetermined format including the basic video stream and the predetermined
10 number of extended video streams generated in the image encoding step; and
an identification information insertion step of inserting identification
information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the container.
15 Claim 9
A transmission device comprising:
an image encoding unit which generates a basic video stream obtained by
encoding basic format image data and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
20 format image data;
a transmission unit which transmits a container in a predetermined format
including the basic video stream and the predetermined number of extended video
streams generated by the image encoding unit; and
an identification information insertion unit which inserts identification
25 information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the video stream.
Claim 10
The transmission device according to claim 9,
30 wherein the image encoding unit
generates, with respect to the basic format image data, the basic video
48
stream by performing a prediction encoding process in the basic format image data,
and
generates, with respect to the high-quality format image data, the extended
video stream by selectively performing a prediction encoding process in the highquality
format image data or a prediction encoding 5 process between the high-quality
format image data and the basic format image data or between the high-quality
format image data and other high-quality format image data.
Claim 11
10 The transmission device according to claim 9,
wherein the video stream has a structure of a NAL unit, and
the identification information insertion unit inserts the identification
information into a header of the NAL unit.
15 Claim 12
A transmission method comprising:
an image encoding step of generating a basic video stream obtained by
encoding basic format image data and a predetermined number of extended video
streams obtained by encoding the predetermined number of pieces of high-quality
20 format image data;
a transmission step of transmitting, by a transmission unit, a container in a
predetermined format including the basic video stream and the predetermined
number of extended video streams generated in the image encoding step; and
an identification information insertion step of inserting identification
25 information in a high-quality format corresponding to each of the predetermined
number of extended video streams into a layer of the video stream.
Claim 13
A reception device comprising:
30 a reception unit which receives a container in a predetermined format
including a basic video stream obtained by encoding basic format image data and a
49
predetermined number of extended video streams obtained by encoding the
predetermined number of pieces of high-quality format image data,
wherein identification information in a high-quality format corresponding to
each of the predetermined number of extended video streams is inserted into a layer
5 of the container, and
the reception device further includes a process unit which processes each of
the video streams included in the received container based on the identification
information.
10 Claim 14
The reception device according to claim 13,
wherein the process unit acquires image data in accordance with display
capability by performing a decoding process on the basic video stream and a
predetermined one of the extended video streams based on the identification
15 information and display capability information.
Claim 15
The reception device according to claim 13,
wherein the basic video stream is generated by performing, with respect to
20 the basic format image data, a prediction encoding process in the basic format image
data, and
the extended video stream is generated by selectively performing, with
respect to the high-quality format image data, a prediction encoding process in the
high-quality format image data or a prediction encoding process between the high25
quality format image data and the basic format image data or between the highquality
format image data and other high-quality format image data.
Claim 16
A reception method comprising:
30 a reception step of receiving, by a reception unit, a container in a
predetermined format which includes a basic video stream obtained by encoding
50
basic format image data and a predetermined number of extended video streams
obtained by encoding the predetermined number of pieces of high-quality format
image data,
wherein identification information in a high-quality format corresponding to
each of the predetermined number of extended 5 video streams is inserted into a layer
of the container, and
the reception method further includes a process step of processing each of
the video streams included in the received container based on the identification
information.
10
Claim 17
A reception device comprising:
a reception unit which receives a container in a predetermined format
including a basic video stream obtained by encoding basic format image data and a
15 predetermined number of extended video streams obtained by encoding the
predetermined number of pieces of high-quality format image data,
wherein identification information in a high-quality format corresponding to
each of the predetermined number of extended video streams is inserted into a layer
of the video stream, and
20 the reception device further includes a process unit which processes each of
the video streams included in the received container based on the identification
information.
Claim 18
25 The reception device according to claim 17,
wherein the process unit acquires image data in accordance with display
capability by performing a decoding process on the basic video stream and a
predetermined one of the extended video streams based on the identification
information and display capability information.
30
Claim 19
51
The reception device according to claim 17,
wherein the basic video stream is generated by performing, with respect to
the basic format image data, a prediction encoding process in the basic format image
data, and
the extended video stream is generated 5 by selectively performing, with
respect to the high-quality format image data, a prediction encoding process in the
high-quality format image data or a prediction encoding process between the highquality
format image data and the basic format image data or between the highquality
format image data and other high-quality format image data.
10
Claim 20
A reception method comprising:
a reception step of receiving, by a reception unit, a container in a
predetermined format which includes a basic video stream obtained by encoding
15 basic format image data and a predetermined number of extended video streams
obtained by encoding the predetermined number of pieces of high-quality format
image data,
wherein identification information in a high-quality format corresponding to
each of the predetermined number of extended video streams is inserted into a layer
20 of the video stream, and
the reception method further includes a process step of processing each of
the video streams included in the received container based on the identification
information.
25 Dated this 24th day of January 2017
SENTHIL KUMAR S.
IN/PA 1546
OF K&S PARTNERS
AGENT FOR THE APPLICANT[S]