Description
Title of Invention
ENCODING DEVICE, ENCODING METHOD, TRANSMISSION DEVICE,
DECODING DEVICE, 5 DECODING METHOD, AND RECEPTION DEVICE
Technical Field
[0001]
The present technology relates to an encoding device, an encoding method,
10 a transmission device, a decoding device, a decoding method, and a reception device.
More particularly, the present technology relates to an encoding device which
encodes image data in an interlacing scheme in an NAL unit structure.
Background Art
15 [0002]
Encoding schemes of encoding image data in an NAL unit structure are
known in the related art. For example, Non-Patent Literature 1 discloses encoding
in a High Efficiency Video Coding (HEVC) scheme.
20 Citation List
Non-Patent Literature
[0003]
Non-Patent Literature 1: “Overview of the High Efficiency Video Coding
(HEVC) Standard” written by Gary J. Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and
25 Thomas Wiegand, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR
VIDEO TECNOROGY, VOL. 22, NO. 12, pp. 1649-1668, December 2012.
Summary of Invention
Technical Problem
30 [0004]
When image data of the interlacing scheme is encoded in the HEVC scheme,
3
if a picture forming a field pair with an intra picture constituting a random access
point is set to a trailing picture (TP), there is no way to insert a leading picture (LP).
Thus, in that case, it is not possible to encode an open Group of Pictures (open GOP).
[0005]
An objective of the present technology 5 is to enable image data of the
interlacing scheme to undergo encoding of an open GOP.
Solution to Problem
[0006]
10 A concept of the present technology is an encoding device including: an
image encoding unit configured to encode image data of each picture constituting
image data in an interlacing scheme in an NAL unit structure and generate a video
stream having the encoded image data of each picture. The image encoding unit
sets an NAL unit type of a slice of encoded image data of a picture forming a field
15 pair with an intra picture constituting a random access point to a unique type which
indicates a picture that is dependent on the intra picture and forms the field pair with
the intra picture.
[0007]
In the present technology, the image encoding unit encodes image data of
20 each picture constituting image data in the interlacing scheme in the NAL unit
structure, and thereby generates a video stream having the image data of each picture.
For example, the image encoding unit may classify the image data of each picture
constituting the image data in the interlacing scheme into a plurality of layers,
encode the image data of pictures of each classified layer, and thereby generate a
25 video stream having the encoded image data of the pictures of each layer. In
addition, for example, the image encoding unit may encode the image data in the
interlacing scheme in the HEVC scheme.
[0008]
Here, the NAL unit type of a slice of encoded image data of a picture
30 forming a field pair with an intra picture constituting a random access point is set to a
unique type which indicates a picture that is dependent on the intra picture and forms
4
the field pair with the intra picture. For this reason, an LP can be inserted as a
succeeding picture and thus encoding of an open GOP becomes possible.
[0009]
Another concept of the present technology is a decoding device including:
an image decoding unit configured to decode a video 5 stream having encoded image
data of each picture obtained by encoding image data of each picture constituting
image data in an interlacing scheme in an NAL unit structure to obtain the image
data in the interlacing scheme. In the video stream, an NAL unit type of a slice of
encoded image data of a picture forming a field pair with an intra picture constituting
10 a random access point is set to a unique type which indicates a picture that is
dependent on the intra picture and forms the field pair with the intra picture.
[0010]
In the present technology, the image decoding unit decodes a video stream
and thereby obtains image data in the interlacing scheme. This video stream has
15 encoded image data of each picture obtained by encoding image data of each picture
constituting the image data in the interlacing scheme in the NAL unit structure.
[0011]
In the video stream, the NAL unit type of a slice of encoded image data of a
picture forming a field pair with an intra picture constituting a random access point is
20 set to a unique type which indicates a picture that is dependent on the intra picture
and forms the field pair with the intra picture. For this reason, the video stream
may be subject to encoding of an open GOP which includes an LP.
[0012]
In the present technology, for example, the video stream may have image
25 data of a picture of each layer obtained by encoding image data of a picture of each
layer, which is obtained by classifying the image data of each picture constituting the
image data in the interlacing scheme into a plurality of layers. The image decoding
unit may decode encoded image data of a picture of a layer designated in layer
designation information and obtain image data of a picture of a layer equal to or
30 lower than a predetermined layer. In this case, for example, decoding
commensurate with a decoding capability of the device is possible.
5
[0013]
In addition, in the present technology, for example, a scheme conversion
unit which converts image data in the interlacing scheme obtained by the image
decoding unit into image data of a progressive scheme and a post-processing unit
which causes a frame rate of the image data of 5 each picture obtained by the scheme
conversion unit to match a display capability may be further provided. In this case,
even when the decoding capability is low, image data of a frame rate commensurate
with a high display capability can be obtained.
10 Advantageous Effects of Invention
[0014]
According to the present technology, encoding of an open GOP can also be
performed for image data in an interlacing scheme. It should be noted that the
effects described herein are not necessarily limitative, and any effect described in the
15 present disclosure may be exhibited.
Brief Description of Drawings
[0015]
[FIG. 1] FIG. 1 is a block diagram showing a configuration example of a transmission
20 and reception system of an embodiment.
[FIG. 2] FIG. 2 is a diagram showing an example of hierarchical encoding of image
data of the interlacing scheme performed by an encoder.
[FIG. 3] FIG. 3 is a diagram showing a structure example (syntax) of an NAL unit
header and the content (semantics) of principal parameters in the structure example.
25 [FIG. 4] FIG. 4 is a diagram for describing a configuration of encoded image data of
each picture based on HEVC.
[FIG. 5] FIG. 5 is a diagram showing an example of hierarchical encoding of image
data of the interlacing scheme performed by an encoder.
[FIG. 6] FIG. 6 is a diagram showing an example of hierarchical encoding of image
30 data of the interlacing scheme performed by an encoder.
[FIG. 7] FIG. 7 is a diagram for describing a closed GOP and an open GOP.
6
[FIG. 8] FIG. 8 is a diagram showing an example of hierarchical encoding when an
NAL unit type of slice of encoded image data of a picture forming a field pair with a
picture of an IRAP is set to RAP_dependent.
[FIG. 9] FIG. 9 is a diagram showing an example of encoding of image data of the
5 interlacing scheme by an encoder.
[FIG. 10] FIG. 10 is a diagram showing another example of encoding of image data
of the interlacing scheme by an encoder.
[FIG. 11] FIG. 11 is a block diagram showing a configuration example of a
transmission device.
10 [FIG. 12] FIG. 12 is a block diagram showing a configuration example of an encoder.
[FIG. 13] FIG. 13 is a block diagram showing a configuration example of a reception
device.
[FIG. 14] FIG. 14 is a block diagram showing a configuration example of a decoder.
[FIG. 15] FIG. 15 is a block diagram showing a configuration example of a post15
processing unit.
Description of Embodiments
[0016]
20 Hereinafter, embodiments for implementing this technology (hereinafter
referred to as “embodiments”) will be described. Note that description will be
provided in the following order.
1. Embodiment
2. Modified example
25 [0017]
<1. Embodiment>
[Transmission and reception system]
FIG. 1 shows a configuration example of a transmission and reception
system 10 as an embodiment. This transmission and reception system 10 is
30 configured to have a transmission device 100 and a reception device 200.
[0018]
7
The transmission device 100 transmits a transport stream TS as a container
by causing the stream to be carried on broadcast waves. The transport stream TS
includes a video stream having encoded image data of each of pictures which is
obtained by encoding image data of the pictures which constitute image data of the
interlacing scheme in the NAL unit structure. 5 In this embodiment, image data of
the interlacing scheme is encoded in the High Efficiency Video Coding (HEVC)
scheme.
[0019]
In addition, this video stream is set to have encoded data of image data of
10 pictures of each layer which is obtained by classifying image data of each picture
constituting image data of the interlacing scheme into a plurality of layers. In this
case, encoding is performed such that a reference source picture is affiliated to its
own layer and/or a layer lower than its own layer.
[0020]
15 Layer identification information for identifying an affiliated layer of each
picture is added to encoded image data of pictures of each layer. Layer
identification information (“nuh_temporal_id_plus1” which means temporal_id) is
disposed in the header part of an NAL unit (nal_unit) of each picture. By adding
the layer identification information in this manner, a reception side can selectively
20 take out encoded image data of a layer equal to or lower than a predetermined layer
and perform a decoding process thereon.
[0021]
FIG. 2 shows an example of hierarchical encoding performed by an encoder.
In this example, layers are classified into four layers from 0 to 3, and encoding is
25 performed on image data of pictures of each of the layers. The vertical axis
represents layers. 0 to 3 are respectively set as temporal_id (layer identification
information) which is disposed in the header part of an NAL unit (nal_unit) which
constitutes encoded image data of pictures in Layers 0 to 3. The horizontal axis
represents a display order (picture order of composition (POC)) with the left side
30 thereof representing earlier positions and the right side thereof representing later
positions in that order.
8
[0022]
FIG. 3(a) shows a structure example (syntax) of the header of an NAL unit,
and FIG. 3(b) shows the content (semantics) of principal parameters in the structure
example. The one-bit field of “forbidden_zero_bit” should have 0. The six-bit
field of “nal_unit_type” represents the type 5 of NAL unit. The six-bit field of
“nuh_layer_id” is assumed to have 0. The three-bit field of
“nuh_temporal_id_plus1” represents temporal_id, and has the value obtained by
adding one (1 to 7).
[0023]
10 Returning to FIG. 2, each of the rectangular frames represents a picture, and
the number therein represents the position of a picture being encoded, i.e., in an
encoding order (decoding order on a reception side). The solid-line arrows
represent encoding reference relations between pictures. A structure of pictures
(SOP) is constituted with 16 pictures from “0” to “17” (excluding “2” and “3”), and
15 “0” is the leading picture. “2” is the leading picture of the next SOP. Alternatively,
excluding “0” and “1,” the structure of pictures (SOP) is constituted by 16 pictures
from “2” to “3” and up to “17,” and in this case, “2” is the leading picture of the SOP.
Two pictures surrounded by a dotted-line oval represent a field pair, “F” represents a
first field (top field), and “S” represents a second field (bottom field). It should be
20 noted that layers of two pictures forming a field pair are the same.
[0024]
The picture of “1” can be the leading picture of a group of pictures (GOP).
Encoded image data of the leading picture of a GOP is composed of NAL units that
are AUD, VPS, SPS, PPS, PSEI, SLICE, SSEI, and EOS as shown in FIG. 4. On
25 the other hand, a picture other than the leading picture of the GOP is composed of
NAL units that are AUD, PPS, PSEI, SLICE, SSEI, and EOS. The VPS can be
transmitted along with the SPS once in a sequence (GOP), and the PPS can be
transmitted for each picture.
[0025]
30 In the present embodiment, encoding of an open GOP is possible. In other
words, an NAL unit type of a slice of encoded image data of a picture forming a field
9
pair with a picture of an IRAP is set to a unique type which indicates a picture that is
dependent on the picture of the IRAP and forms the field pair with the picture of the
IRAP.
[0026]
In encoding in the HEVC scheme, an intra 5 random access point (IRAP), a
trailing picture (TP), and a leading picture (LP) are defined as the NAL unit types of
a slice. An “IRAP” indicates an intra picture constituting a random access point,
and includes each type of “BLA_W_LP,” “BLA_W_RADL,” “BLA_N_LP,”
“IDR_W_RADL,” “IDR_N_LP,” and “CRA_NUT.”
10 [0027]
A “TP” indicates a picture later in a decoding order than the IRAP and later
in a display order than the IRAP, and includes each type of “TRAIL_N” and
“TRAIL_R.” An “LP” indicates a picture later in a decoding order than the IRAP
and earlier in a display order than the IRAP, and includes each type of “RADL_N,”
15 “RADL_R,” “RASL_N,” and “RASL_R.”
[0028]
Originally, a picture forming a field pair with a picture of an IRAP becomes
a TP later in a decoding order than the IRAP and later in a display order than the
IRAP. In this case, only the TP can serve as a succeeding picture as shown in FIG. 5,
20 leaving no room for insertion of an LP, and thus encoding of an open GOP is not
possible.
[0029]
In this embodiment, a type called “RAP_dependent” is newly defined.
This type indicates a picture which is not a picture of an IRAP but is dependent on a
25 picture of an IRAP, and forms a field pair with the picture of the IRAP.
Accordingly, an NAL unit type of a slice of encoded image data of a picture forming
a field pair with a picture of an IRAP is set to RAP_dependent. In this case, an LP
can be inserted as a succeeding picture, and thus encoding of an open GOP is
possible.
30 [0030]
The concepts of a closed GOP and an open GOP will be described. FIG.
10
7(a) schematically shows a closed GOP. A closed GOP is a GOP in which the
leading I-picture in a decoding order is an instantaneous decoding refresh (IDR)
picture. FIG. 7(b) schematically shows an open GOP. An open GOP is a GOP in
which the leading I-picture in a decoding order is a non-IDR picture.
5 [0031]
As shown in the drawing, an N-th GOP is assumed to be a GOP Nth and an
(N−1)-th GOP is assumed to be a GOP (N−1)th. In a GOP Nth, an IDR picture
comes first in a decoding order, an LP comes next, and a TP comes next after that.
In addition, in the GOP Nth, an LP comes first in a display order, an IDR picture of a
10 non-IDR picture comes next, and a TP comes next after that.
[0032]
In the case of a closed GOP, when decoding starts from the beginning of a
GOP Nth, decoding of all pictures included in the GOP Nth is guaranteed. In other
words, the IDR picture is decoded first and then the LP is decoded. This LP is
15 prohibited from overtaking the IDR picture in the decoding order to refer to a GOP
(N−1)th. For this reason, when there is an LP, the LP is permitted only to refer to a
picture include in the GOP Nth, and thus it can be decoded without information of
the GOP (N−1)th. Likewise, since a TP is also prohibited to refer to the GOP
(N−1)th, it can be decoded without information of the GOP (N−1)th.
20 [0033]
On the other hand, in the case of an open GOP, when decoding starts from
the beginning of a GOP Nth, an LP which is earlier than a non-IDR picture in a
display order is permitted to refer to a GOP (N−1)th, and thus it may be undecodable.
It should be noted that, since a TP is prohibited from referring to the GOP (N−1)th, it
25 can be decoded without information of the GOP (N−1)th.
[0034]
In this manner, when decoding starts from the beginning of a GOP Nth, all
pictures included in the GOP Nth, can be decoded in the case of a closed GOP, but in
the case of an open GOP, it is not possible to decode an LP. On the other hand,
30 since referring to a GOP (N−1)th is not permitted in the case of a closed GOP, a bit
rate thereof becomes higher than an open GOP to obtain the same image quality.
11
For this reason, a closed GOP and an open GOP were separately used according to a
situation in the past.
[0035]
FIG. 8 shows an example of hierarchical encoding when an NAL unit type
of a slice of encoded image data of a picture forming 5 a field pair with a picture of an
IRAP is set to RAP_dependent. The vertical axis represents layers. The
horizontal axis represents a display order (picture order of composition or POC) with
the left side thereof representing earlier display times and the right side thereof
representing later display times. In this example, layers are classified into four
10 layers of 0 to 3 and encoding is performed on image data of pictures of each of the
layers as in the example of FIG. 2 described above.
[0036]
In this example, each of the rectangular frames represents a picture, and the
number therein represents the position of a picture being encoded, i.e., in the
15 encoding order (decoding order on the reception side). The solid-line arrows
represent encoding reference relations between pictures. Two pictures surrounded
by a dotted-line oval represent a field pair, “F” represents a first field (top field), and
“S” represents a second field (bottom field).
[0037]
20 In this example, a picture of “0” in Layer 0 is encoded as a picture of an
IRAP. Next, a picture of “1” forming a field pair with the picture of “0” is encoded
as a picture of RAP_dependent. Then, pictures of “2” to “15” in Layers 1 to 3 are
encoded as LPs, and then pictures of “16” and “17” in Layer 0 and pictures of “18”
to “31” in Layers 1 to 3 are encoded as TPs.
25 [0038]
In this example, by encoding the picture of “1” forming a field pair with the
picture of “0” which is the picture of the IRAP as a picture of RAP_dependent,
insertion of an LP thereafter is possible. For this reason, encoding of an open GOP
is possible.
30 [0039]
It should be noted that enabling an LP to be inserted as a succeeding picture
12
by setting an NAL unit type of a slice of encoded image data of a picture forming a
field pair with a picture of an IRAP to RAP_dependent is not limited in hierarchical
encoding.
[0040]
FIG. 9 shows an example of encoding in a 5 stream order and a display order.
In this example, an N-th GOP is assumed to be a GOP Nth, and an (N−1)-th GOP is
assumed to be a GOP (N−1)th. In addition, each of the solid-line rectangular
frames represent a picture, and the number therein represents the position of a picture
being encoded, i.e., in an encoding order (a decoding order on a reception side).
10 Two pictures surrounded by a dashed-line rectangular frame represent a field pair
with a first field (top field) and a second field (bottom field).
[0041]
In this example, each picture of the GOP Nth is encoded as below, and
thereby a video stream is generated. In other words, the picture of “0” is encoded
15 as a picture of the IRAP. Next, the picture of “1” forming a field pair with the
picture of “0” is encoded as a picture of RAP_dependent. Then, the pictures of “2”
to “5” positioned before the picture of “0” in the display order are encoded as LPs,
and then the pictures of “6” to “9” positioned after the picture of “1” in the display
order are encoded as TPs.
20 [0042]
In this example, by encoding the picture of “1” as a picture of
RAP_dependent as described above, insertion of an LP thereafter is possible. Thus,
encoding of an open GOP is possible.
[0043]
25 FIG. 10 also shows an example of encoding in a stream order and a display
order. This example shows a case in which an NAL unit type of a slice of encoded
image data of a picture forming a field pair with a picture of an IRAP is set to a TP,
rather than RAP_dependent.
[0044]
30 In this example, the respective pictures of the GOP Nth are encoded as
below, and thereby a video stream is generated. In other words, the picture of “0” is
13
encoded as a picture of an IRAP. Next, the picture of “1” forming the field pair
with the picture of “0” is encoded as a TP. Then, the pictures of “2” to “9”
positioned after the picture of “1” in the display order are encoded as TPs.
[0045]
In this example, by encoding the picture of “5 1” as a TP as described above,
succeeding pictures become only TPs. For this reason, encoding of an open GOP
that is based on the premise of insertion of an LP is not possible.
[0046]
Returning to FIG. 1, the reception device 200 receives the transport stream
10 TS carried on and sent by broadcast waves from the transmission device 100. The
reception device 200 selectively takes and decodes encoded image data of a layer
equal to or lower than a predetermined layer from a video stream included in this
transport stream TS according to its own decoding capability, acquires image data of
each of pictures, and thereby obtains image data in the interlacing scheme.
15 [0047]
In addition, the reception device 200 converts the image data in the
interlacing scheme obtained through the decoding described above into image data in
a progressive scheme, and further performs post-processing to cause the frame rate
thereof to match a display capability. Through this post-processing, it is possible to
20 obtain image data of a frame rate commensurate with a high display capability even
when, for example, a decoding capability is low.
[0048]
[Configuration of transmission device]
FIG. 11 shows a configuration example of the transmission device 100.
25 This transmission device 100 has an encoder 102, a compressed data buffer (coded
picture buffer or cpb) 103, a multiplexer 104, and a transmission unit 105.
[0049]
The encoder 102 receives an input of uncompressed image data in the
interlacing scheme and performs hierarchical encoding in the HEVC scheme. The
30 encoder 102 classifies image data of each of pictures constituting the image data in
the interlacing scheme into a plurality of layers, and generates a video stream having
14
encoded data of the image data of pictures of each layer. In this case, the encoder
102 encodes data such that a picture to be referred to (reference source picture) is
affiliated to its own layer and/or a layer lower than its own layer (see FIG. 2).
[0050]
At this time, the encoder 102 encodes 5 each picture such that encoding of the
open GOP becomes possible. In other words, the encoder 102 sets an NAL unit
type of a slice of encoded image data of a picture forming a field pair with a picture
of an IRAP to a unique type which indicates a picture that is dependent on the picture
of the IRAP and forms the field pair with the picture of the IRAP. In this
10 embodiment, this unique type is the newly defined “RAP_dependent” as described
above (see FIGS. 8 and 9).
[0051]
FIG. 12 shows a configuration example of the encoder 102. This encoder
102 has a parameter set/SEI encoding unit 121, a slice encoding unit 122, an NAL
15 packetizing unit 123, and a temporal ID generation unit 124.
[0052]
The parameter set/SEI encoding unit 121 generates parameter sets such as a
VPS, an SPS, a PPS, and the like and SEI of pictures of each layer according to the
number of streams to be encoded. The slice encoding unit 122 obtains slice data by
20 encoding image data of pictures of each layer.
[0053]
The NAL packetizing unit 123 generates encoded image data of the pictures
of each layer based on the parameter sets and SEI generated by the parameter set/SEI
encoding unit 121 and the slice data generated by the slice encoding unit 122, and
25 obtains video streams (encoded streams).
[0054]
The temporal ID generation unit 124 generates temporal_id for each picture
to be encoded based on layer information of the picture, and sends the data to the
NAL packetizing unit 123. The NAL packetizing unit 123 receives information of
30 the NAL unit type of each picture to be encoded. The NAL packetizing unit 123
inserts temporal_id corresponding to a picture of each layer into the header of the
15
NAL unit constituting encoded image data of the picture and further sets an NAL
unit type (see FIG. 3).
[0055]
Returning to FIG. 11, the compressed data buffer (cpb) 103 temporarily
accumulates a video stream including the encoded data 5 of the pictures of each layer
obtained by the encoder 102. The multiplexer 104 reads the video stream
accumulated in the compressed data buffer 103, makes them into PES packets,
further makes them into transport packets and multiplexes them, and thereby obtains
a transport stream TS as a multiplexed stream.
10 [0056]
This transport stream TS includes a single video stream having the encoded
image data of the pictures of each layer, or a predetermined number of video streams
having the encoded image data of the pictures of each layer set that is obtained by
dividing the plurality layers into a predetermined number of layer sets, the number
15 being equal to or greater than two. The multiplexer 104 inserts layer information
and stream configuration information into the transport stream TS.
[0057]
As one of program specific information (PSI), the transport stream TS
includes a program map table (PMT). This PMT has a video elementary loop
20 (video ES1 loop) with information relating to each video stream. In this video
elementary loop, information of a stream type, a packet identifier (PID), and the like
as well as a descriptor describing the information relating to the video stream are
disposed to correspond to each video stream.
[0058]
25 The transmission unit 105 modulates the transport stream TS in a
modulation scheme proper for broadcasting, for example, QPSK-OFDM, and
transmits an RF modulation signal from a transmission antenna.
[0059]
An operation of the transmission device 100 shown in FIG. 11 will be
30 briefly described. The encoder 102 receives an input of uncompressed image data
in an interlacing scheme. The encoder 102 performs hierarchical encoding on this
16
image data in an HEVC scheme. In other words, the encoder 102 classifies image
data of respective pictures constituting the image data into a plurality of layers and
encodes the data, and thereby generates a video stream having encoded image data of
the pictures of each layer. At this time, encoding is performed such that a picture to
be referred to is affiliated to its own layer 5 and/or a layer lower than its own layer.
[0060]
In addition, at this time, each picture is encoded such that encoding of an
open GOP becomes possible. In other words, an NAL unit type of a slice of
encoded image data of a picture forming a field pair with a picture of an IRAP is set
10 to “RAP_dependent” which is a unique type which indicates a picture that is
dependent on the picture of the IRAP and forms the field pair with the picture of the
IRAP.
[0061]
The video stream which is generated by the encoder 102, including the
15 encoded data of the pictures of each layer, is supplied to the compressed data buffer
(cpb) 103 and temporarily stored therein. In the multiplexer 104, the video stream
accumulated in the compressed data buffer 103 is read, PES packetized, and further
transport packetized and multiplexed, and thereby a transport stream TS is obtained
as a multiplexed stream.
20 [0062]
The transport stream TS obtained by the multiplexer 104 is sent to the
transmission unit 105. In the transmission unit 105, the transport stream TS is
modulated in a modulation scheme proper for broadcasting, for example, QPSKOFDM,
and an RF modulation signal is transmitted from a transmission antenna.
25 [0063]
[Configuration of a reception device]
FIG. 13 shows a configuration example of the reception device 200. This
reception device 200 has a reception unit 202, a demultiplexer 203, and a compressed
data buffer (coded picture buffer or cpb) 204. Further, the reception device 200 has
30 a decoder 205, an uncompressed data buffer (decoded picture buffer or dpb) 206, and
a post-processing unit 207.
17
[0064]
The reception unit 202 demodulates an RF modulation signal received by a
reception antenna to acquire the transport stream TS. The demultiplexer 203 takes
out a video stream (encoded stream) by performing a depacketizing process on the
transport stream TS and sends the stream 5 to the compressed data buffer (coded
picture buffer or cpb) 204.
[0065]
The compressed data buffer (cpb) 204 temporarily accumulates the video
stream taken out by the demultiplexer 203. The decoder 205 takes out encoded
10 image data of pictures of a layer that has been designated as a layer to be decoded
from the video stream accumulated in the compressed data buffer 204. Then, the
decoder 205 decodes the taken encoded image data of each picture at each decoding
timing of the picture, and sends the data to the uncompressed data buffer (dpb) 206.
When decoding the encoded image data of each picture, the decoder 205 reads and
15 uses image data of a reference source picture from the uncompressed data buffer 206
when necessary.
[0066]
FIG. 14 shows a configuration example of the decoder 205. This decoder
205 has a temporal ID analysis unit 251, a target layer selection unit 252, and a
20 decoding unit 253. The temporal ID analysis unit 251 reads the video stream
(encoded stream) accumulated in the compressed data buffer 204 and analyzes
temporal_id inserted into the NAL unit header of the encoded image data of each
picture.
[0067]
25 The target layer selection unit 252 takes out encoded image data of each
picture of a layer designated as a layer to be decoded from the video stream read
from the compressed data buffer 204 based on the analysis result of the temporal ID
analysis unit 251, and sends the data to the decoding unit 253. The decoding unit
253 sequentially decodes the encoded image data of each picture acquired by the
30 target layer selection unit 252 at decoding timings, and sends the data to the
uncompressed data buffer (dpb) 206.
18
[0068]
In this case, the decoding unit 253 analyzes the VPS and the SPS to
ascertain a level designation value of a bit rate of each sublayer
“sublayer_level_idc,” and checks whether decoding is possible within the decoding
capability. In addition, in this case, the decoding unit 5 253 analyzes the SEI to
ascertain, for example, “initial_cpb_removal_time” and “cpb_removal_delay,” and
checks whether a decoding timing is proper. Alternatively, decoding is started at a
decoding timing obtained from a DTS (decoding timestamp) obtained by the
demultiplexer 203. In addition, display is performed at a display timing obtained
10 from a PTS (presentation timestamp) obtained by the demultiplexer 203.
[0069]
The decoding unit 253 acquires information which indicates a prediction
destination in a time direction from a slice header when a slice is decoded to perform
time-direction prediction. In addition, when a slice is decoded, the decoding unit
15 253 manages a reading order after decoding for each type of the IRAP,
RAP_dependent, an LP and a TP described in the NAL unit type of the NAL unit
header.
[0070]
Returning to FIG. 13, the uncompressed data buffer (dpb) 206 temporarily
20 accumulates the image data of each picture decoded by the decoder 205. The postprocessing
unit 207 converts the image data in the interlacing scheme sequentially
read from the uncompressed data buffer (dpb) 206 at display timings into image data
in the progressive scheme. Furthermore, the post-processing unit 207 performs a
process of causing the frame rate of the image data of each picture converted into the
25 progressive scheme to match the display capability.
[0071]
For example, when the frame rate of the decoded image data of the pictures
is 120 fps and the display capability is 120 fps, the post-processing unit 207 sends
the decoded image data of the pictures to a display as it is. In addition, when, for
30 example, the frame rate of the decoded image data of the pictures is 120 fps and the
display capability is 60 fps, the post-processing unit 207 performs a subsampling
19
process so that time-direction resolution is half that of the decoded image data of the
pictures, and sends the data to the display as image data of 60 fps.
[0072]
In addition, when the decoded image data of each picture has an interlaced
field rate of 60 fps and the display capability thereof 5 is a progressive frame rate of
120 fps, the post-processing unit 207 performs I-P conversion on the decoded image
data of each picture to set the progressive frame rate to 60 fps, further performs an
interpolation process to set time direction resolution to twice that of pictures of the
60 fps, and sends the data to a display as image data of 120 fps. In addition, for
10 example, when the image data of each picture after decoding has a frame rate of 60
fps and the display capability thereof is 60 fps, the post-processing unit 207 sends the
decoded image data of each picture directly to the display.
[0073]
FIG. 15 shows a configuration example of the post-processing unit 270.
15 This is an example which can deal with the case described above in which the frame
rate of the decoded image data of the pictures is 120 fps or 60 fps and the display
capability is 120 fps or 60 fps,
[0074]
The post-processing unit 207 has an I-P conversion unit 271, an
20 interpolation unit 272, a subsampling unit 273, and a switching unit 274. The I-P
conversion unit 271 converts image data of each picture after decoding (image data
in the interlacing scheme) from the uncompressed data buffer 206 into image data in
the progressive scheme.
[0075]
25 The interpolation unit 272 performs an interpolation process in the time
direction on the image data obtained by the I-P conversion unit 271 to double the
frame rate. The subsampling unit 273 performs a subsampling process in the time
direction on the image data obtained by the I-P conversion unit 271 to halve the
frame rate. The switching unit 274 selectively outputs any output image data of the
30 I-P conversion unit 271, the interpolation unit 272, and the subsampling unit 273
based on selection information.
20
[0076]
The selection information is automatically generated by, for example, a
control unit that is not illustrated or according to a user operation. Accordingly,
image data of each picture sequentially read from the uncompressed data buffer (dpb)
206 at display timings is converted into the progressive 5 scheme, and the frame rate
matches the display capability.
[0077]
An operation of the reception device 200 shown in FIG. 15 will be briefly
described. The reception unit 202 demodulates the RF modulation signal received
10 by the reception antenna to acquire the transport stream TS. This transport stream
TS is sent to the demultiplexer 203. The demultiplexer 203 takes a video stream
(encoded stream) by performing a depacketizing process on the transport stream TS.
This video stream is sent to and temporarily accumulated in the compressed data
buffer (cpb) 204.
15 [0078]
The decoder 205 takes the encoded image data of the pictures of the layer
designated as a layer to be decoded from the video stream accumulated in the
compressed data buffer 204. Then, the decoder 205 decodes the encoded image
data of each of the taken pictures at decoding timings of the pictures, and sends the
20 data to the uncompressed data buffer (dpb) 206 to cause the data to be temporarily
stored. In this case, when the encoded image data of the respective pictures is to be
decoded, image data of a reference source picture is read from the uncompressed
data buffer 206 to be used.
[0079]
25 The image data of the pictures sequentially read from the uncompressed
data buffer (dpb) 206 at display timings is sent to the post-processing unit 207. The
post-processing unit 207 converts the image data in the interlacing scheme into
image data in the progressive scheme, and further performs an interpolation process
or a subsampling process on the image data of each picture converted into the
30 progressive scheme to cause the frame rate thereof to match the display capability.
The image data of the pictures processed by the post-processing unit 207 is supplied
21
to the display and a dynamic image of the image data of the pictures is displayed.
[0080]
As described above, in the transmission and reception system 10 shown in
FIG. 1, when image data in the interlacing scheme is encoded in the HEVC scheme,
the transmission side sets an NAL unit type of a slice 5 of encoded image data of a
picture forming a field pair with an intra picture constituting a random access point
to a unique type which indicates a picture that is dependent on the intra picture and
forms the field pair with the intra picture (to “RAP_dependent”). Thus, inserting an
LP as a succeeding picture becomes possible and thereby encoding of an open GOP
10 becomes possible.
[0081]
<2. Modified example>
It should be noted that, although the above-described embodiment shows the
transmission and reception system 10 constituted by the transmission device 100 and
15 the reception device 200, a configuration of a transmission and reception system to
which the present technology can be applied is not limited thereto. For example,
the reception device 200 part may be configured with a set-top box connected with a
digital interface such as High-Definition Multimedia Interface (HDMI) and a monitor,
or the like. Note that “HDMI” is a registered trademark.
20 [0082]
In addition, although a case of an interlaced-type image has been described
in the above embodiment, also in the case of a progressive-type image, a picture is
allowed to be disposed later than an IRAP in its decoding order and display order due
to the type of RAP_dependent, and thus a stream in which LPs and RPs are flexibly
25 arranged can be configured.
[0083]
In addition, the above-described embodiment shows the example in which
the container is a transport stream (MPEG-2 TS). The present technology, however,
can also be similarly applied to a system configured to distribute data to a reception
30 terminal using a network such as the Internet. In distribution on the Internet, there
are many cases of distribution using a container in MP4 or other formats. In other
22
words, as containers, various formats including transport streams (MPEG-2 TS)
employed in digital broadcasting standards, MP4 used in Internet distribution, and
the like are equivalent thereto.
[0084]
Additionally, the present 5 technology may also be configured as below.
(1)
An encoding device including:
an image encoding unit configured to encode image data of each picture
constituting image data in an interlacing scheme in an NAL unit structure and
10 generate a video stream having the encoded image data of each picture,
wherein the image encoding unit sets an NAL unit type of a slice of encoded
image data of a picture forming a field pair with an intra picture constituting a
random access point to a unique type which indicates a picture that is dependent on
the intra picture and forms the field pair with the intra picture.
15 (2)
The encoding device according to (1), wherein the image encoding unit
classifies the image data of each picture constituting the image data in the interlacing
scheme into a plurality of layers, encodes image data of a picture of each classified
layer, and generates a video stream having the encoded image data of the picture of
20 each layer.
(3)
The encoding device according to (1) or (2), wherein the image encoding
unit encodes the image data in the interlacing scheme in an HEVC scheme.
(4)
25 An encoding method including:
an image encoding step by an image encoding unit of encoding image data
of each picture constituting image data in an interlacing scheme in an NAL unit
structure and generating a video stream having the encoded image data of each
picture,
30 wherein, in the image encoding step, an NAL unit type of a slice of encoded
image data of a picture forming a field pair with an intra picture constituting a
23
random access point is set to a unique type which indicates a picture that is
dependent on the intra picture and forms the field pair with the intra picture.
(5)
A transmission device including:
an image encoding unit configured to encode 5 image data of each picture
constituting image data in an interlacing scheme in an NAL unit structure and
generate a video stream having the encoded image data of each picture; and
a transmission unit configured to transmit a container in a predetermined
format which includes the generated video stream,
10 wherein the image encoding unit sets an NAL unit type of a slice of encoded
image data of a picture forming a field pair with an intra picture constituting a
random access point to a unique type which indicates a picture that is dependent on
the intra picture and forms the field pair with the intra picture.
(6)
15 A decoding device including:
an image decoding unit configured to decode a video stream having encoded
image data of each picture obtained by encoding image data of each picture
constituting image data in an interlacing scheme in an NAL unit structure to obtain
the image data in the interlacing scheme,
20 wherein, in the video stream, an NAL unit type of a slice of encoded image
data of a picture forming a field pair with an intra picture constituting a random
access point is set to a unique type which indicates a picture that is dependent on the
intra picture and forms the field pair with the intra picture.
(7)
25 The decoding device according to (6),
wherein the video stream has image data of a picture of each layer obtained
by encoding image data of a picture of each layer, which is obtained by classifying
the image data of each picture constituting the image data in the interlacing scheme
into a plurality of layers, and
30 wherein the image decoding unit decodes encoded image data of a picture of
a layer designated in layer designation information and obtains image data of a
24
picture of a layer equal to or lower than a predetermined layer.
(8)
The decoding device according to (6) or (7), further including:
a scheme conversion unit configured to convert the image data in the
interlacing scheme obtained by the image 5 decoding unit into image data in a
progressive scheme; and
a post-processing unit configured to cause a frame rate of image data of
each picture obtained by the scheme conversion unit to match a display capability.
(9)
10 A decoding method including:
an image decoding step by an image decoding unit of decoding a video
stream having encoded image data of each picture obtained by encoding image data
of each picture constituting image data in an interlacing scheme in an NAL unit
structure to obtain the image data in the interlacing scheme,
15 wherein, in the video stream, an NAL unit type of a slice of encoded image
data of a picture forming a field pair with an intra picture constituting a random
access point is set to a unique type which indicates a picture that is dependent on the
intra picture and forms the field pair with the intra picture.
(10)
20 A reception device including:
a reception unit configured to receive a container in a predetermined format
which includes a video stream having encoded image data of each picture obtained
by encoding image data of each picture constituting image data in an interlacing
scheme in an NAL unit structure; and
25 an image decoding unit configured to obtain the image data in the
interlacing scheme by decoding the video stream included in the received container,
wherein, in the video stream, an NAL unit type of a slice of encoded image
data of a picture forming a field pair with an intra picture constituting a random
access point is set to a unique type which indicates a picture that is dependent on the
30 intra picture and forms the field pair with the intra picture.
[0085]
25
The main feature of the present technology is to enable insertion of an LP as
a succeeding picture possible and encoding of an open GOP possible by setting,
when image data in the interlacing scheme is encoded in the form of an NAL unit, an
NAL unit type of a slice of encoded image data of a picture forming a field pair with
an intra picture constituting a random access 5 point to a unique type which indicates a
picture that is dependent on the intra picture and forms the field pair with the intra
picture (see FIGS. 8 and 9).
Reference Signs List
10 [0086]
10 transmission and reception system
100 transmission device
102 encoder
103 compressed data buffer (cpb)
15 104 multiplexer
105 transmission unit
121 parameter set/SEI encoding unit
122 slice encoding unit
123 NAL packetizing unit
20 124 temporal ID generation unit
200 reception device
202 reception unit
203 demultiplexer
204 compressed data buffer (cpb)
25 205 decoder
206 uncompressed data buffer (dpb)
207 post-processing unit
251 temporal ID analysis unit
252 target layer selection unit
30 253 decoding unit
271 I-P conversion unit
26
272 interpolation unit
273 subsampling unit
274 switching unit
5
27
CLAIMS
Claim 1
An encoding device comprising:
an image encoding unit configured to encode image data of each picture
constituting image data in an interlacing scheme 5 in an NAL unit structure and
generate a video stream having the encoded image data of each picture,
wherein the image encoding unit sets an NAL unit type of a slice of encoded
image data of a picture forming a field pair with an intra picture constituting a
random access point to a unique type which indicates a picture that is dependent on
10 the intra picture and forms the field pair with the intra picture.
Claim 2
The encoding device according to claim 1, wherein the image encoding unit
classifies the image data of each picture constituting the image data in the interlacing
15 scheme into a plurality of layers, encodes image data of a picture of each classified
layer, and generates a video stream having the encoded image data of the picture of
each layer.
Claim 3
20 The encoding device according to claim 1, wherein the image encoding unit
encodes the image data in the interlacing scheme in an HEVC scheme.
Claim 4
An encoding method comprising:
25 an image encoding step by an image encoding unit of encoding image data
of each picture constituting image data in an interlacing scheme in an NAL unit
structure and generating a video stream having the encoded image data of each
picture,
wherein, in the image encoding step, an NAL unit type of a slice of encoded
30 image data of a picture forming a field pair with an intra picture constituting a
random access point is set to a unique type which indicates a picture that is
28
dependent on the intra picture and forms the field pair with the intra picture.
Claim 5
A transmission device comprising:
an image encoding unit configured to encode 5 image data of each picture
constituting image data in an interlacing scheme in an NAL unit structure and
generate a video stream having the encoded image data of each picture; and
a transmission unit configured to transmit a container in a predetermined
format which includes the generated video stream,
10 wherein the image encoding unit sets an NAL unit type of a slice of encoded
image data of a picture forming a field pair with an intra picture constituting a
random access point to a unique type which indicates a picture that is dependent on
the intra picture and forms the field pair with the intra picture.
15 Claim 6
A decoding device comprising:
an image decoding unit configured to decode a video stream having encoded
image data of each picture obtained by encoding image data of each picture
constituting image data in an interlacing scheme in an NAL unit structure to obtain
20 the image data in the interlacing scheme,
wherein, in the video stream, an NAL unit type of a slice of encoded image
data of a picture forming a field pair with an intra picture constituting a random
access point is set to a unique type which indicates a picture that is dependent on the
intra picture and forms the field pair with the intra picture.
25
Claim 7
The decoding device according to claim 6,
wherein the video stream has image data of a picture of each layer obtained
by encoding image data of a picture of each layer, which is obtained by classifying
30 the image data of each picture constituting the image data in the interlacing scheme
into a plurality of layers, and
29
wherein the image decoding unit decodes encoded image data of a picture of
a layer designated in layer designation information and obtains image data of a
picture of a layer equal to or lower than a predetermined layer.
5 Claim 8
The decoding device according to claim 6, further comprising:
a scheme conversion unit configured to convert the image data in the
interlacing scheme obtained by the image decoding unit into image data in a
progressive scheme; and
10 a post-processing unit configured to cause a frame rate of image data of
each picture obtained by the scheme conversion unit to match a display capability.
Claim 9
A decoding method comprising:
15 an image decoding step by an image decoding unit of decoding a video
stream having encoded image data of each picture obtained by encoding image data
of each picture constituting image data in an interlacing scheme in an NAL unit
structure to obtain the image data in the interlacing scheme,
wherein, in the video stream, an NAL unit type of a slice of encoded image
20 data of a picture forming a field pair with an intra picture constituting a random
access point is set to a unique type which indicates a picture that is dependent on the
intra picture and forms the field pair with the intra picture.
Claim 10
25 A reception device comprising:
a reception unit configured to receive a container in a predetermined format
which includes a video stream having encoded image data of each picture obtained
by encoding image data of each picture constituting image data in an interlacing
scheme in an NAL unit structure; and
30 an image decoding unit configured to obtain the image data in the
interlacing scheme by decoding the video stream included in the received container,
30
wherein, in the video stream, an NAL unit type of a slice of encoded image
data of a picture forming a field pair with an intra picture constituting a random
access point is set to a unique type which indicates a picture that is dependent on the
intra picture and forms the field pair with the intra picture.