IMAGE PROCESSING DEVICE AND METHOD
5
Technical Field
[0001]
The present disclosure relates to an image processing device and method,
and more particularly, to an image processing device and method which are capable
10 of performing a decoding process at a proper timing in scalable video coding.
Background Art
[0002]
Recently, devices for compressing and encoding an image by adopting a
15 encoding scheme of handling image information digitally and performing
compression by an orthogonal transform such as a discrete cosine transform and
motion compensation using image information-specific redundancy for the purpose
of information transmission and accumulation with high efficiency when the image
information is handled digitally have become widespread. Moving Picture Experts
20 Group (MPEG), H.264, MPEG-4 Part 10 (Advanced Video Coding) (hereinafter
referred to as H.264/AVC), and the like are examples of such encoding schemes.
[0003]
Therefore, for the purpose of improving encoding efficiency compared to
H.264/AVC, standardization of a encoding scheme referred to as high efficiency
25 video coding (HEVC) by Joint Collaboration Team-Video Coding (JCTVC), which is
a joint standardizing organization of International Telecommunication Union
Telecommunication Standardization Sector (ITU-T) and International Organization
for Standardization (ISO)/International Electrotechnical Commission (IEC), is
currently in progress, and Non-Patent Literature 1 has been issued as a draft of the
30 scheme.
[0004]
SP351652WO00
2/133
Meanwhile, the existing image encoding schemes such as MPEG-2 and
AVC have a scalability function of dividing an image into a plurality of layers and
encoding the plurality of layers.
[0005]
5 In other words, for example, for a terminal having a low processing
capability such as a mobile phone, image compression information of only a base
layer is transmitted, and a moving image of low spatial and temporal resolutions or a
low quality is reproduced, and for a terminal having a high processing capability
such as a television or a personal computer, image compression information of an
10 enhancement layer as well as a base layer is transmitted, and a moving image of high
spatial and temporal resolutions or a high quality is reproduced. That is, image
compression information according to a capability of a terminal or a network can be
transmitted from a server without performing the transcoding process.
[0006]
15 In the HEVC, it is possible to designate a hypothetical reference decoder
(HRD) parameter so that an overflow or an underflow of a buffer does not occur
when a decoding process for image compression information is performed.
Particularly, it is possible to designate an HRD parameter for each layer when
scalable video coding is performed (see Non-Patent Literature 2).
20
Citation List
Non-Patent Literature
[0007]
Non-Patent Literature 1: Benjamin Bross, Woo-Jin Han, Jens-Raincr Ohm,
25 Gary J. Sullivan, Thomas Wiegand," High efficiency video coding (HEVC) text
specification draft 9," JCTVC-K1003, 2012,10,21
Non-Patent Literature 2: Jill Boyce, Ye-Kui Wang, "NAL unit header and
parameter set designs for HEVC extensions," JCTVC-K1007, 2012,10,19
30 Summary of Invention
Technical Problem
3/133
SP351652WO00
[0008]
However, when the HRD parameter is designated for each layer or a time
layer serving as one of sublayers, it is difficult to detect whether the decoding
process is performed by a single decoding device or a plurality of decoding devices.
5 [0009]
The present disclosure was made in light of the foregoing, and it is desirable
to perform a decoding process at a proper timing.
Solution to Problem
10 [0010]
According to one aspect of the present disclosure, there is provided an
image processing device including a receiving section configured to receive a
bitstream obtained by encoding an image having at least one layer and buffer
management parameter information of each layer indicating at least one of that a
15 parameter for managing a decoder buffer is a parameter for performing a decoding
process of only a corresponding layer and that the parameter for managing the
decoder buffer is a parameter for performing a decoding process of a corresponding
layer and a lower layer, and a decoding section configured to decode the bitstream
received by the receiving section and generate an image.
20 [0011]
The layer can include a layer and a sublayer.
[0012]
The layer is a view of multi-view coding.
[0013]
25 The layer is a layer of scalable video coding.
[0014]
The buffer management parameter information is described in supplemental
enhancement information (SE1).
[0015]
30 The buffer management parameter information is described in
buffering_period_SEI.
-1.
SP351652WO00
4/133
[0016]
Parameter presence/absence information indicating a presence or absence of
the parameter for managing the decoder buffer serving as the parameter for
performing the decoding process of only the corresponding layer is described in a
5 vps (video parameter set) extension.
[0017]
The receiving section can receive an AVC flag indicating that a layer lower
than the corresponding layer is encoded by MPEG-4 Parti 0 Advanced Video Coding
(AVC) and the buffer management parameter information of each layer indicating
10 that the parameter for managing the decoder buffer is the parameter for performing
the decoding process of only the corresponding layer.
[0018]
According to one aspect of the present disclosure, there is provided an
image processing method including receiving, by an image processing device, a
15 bitstream obtained by encoding an image having at least one layer and buffer
management parameter information of each layer indicating at least one of that a
parameter for managing a decoder buffer is a parameter for performing a decoding
process of only a corresponding layer and that the parameter for managing the
decoder buffer is a parameter for performing a decoding process of a corresponding
20 layer and a lower layer, receiving, by the image processing device, a bitstream
obtained by encoding an image including at least one layer using a parameter
corresponding to the buffer management parameter information, and decoding, by
the image processing device, the received bitstream and generating an image.
[0019]
25 According to another aspect of the present disclosure, there is provided an
image processing device including a setting section configured to set buffer
management parameter information of each layer indicating at least one of that a
parameter for managing a decoder buffer is a parameter for performing a decoding
process of only a corresponding layer and that the parameter for managing the
30 decoder buffer is a parameter for performing a decoding process of a corresponding
layer and a lower layer, an encoding section configured to encode an image having at
SP351652WO00
5/133
least one layer and generate a bitstream, and a transmitting section configured to
transmit the buffer management parameter information set by the setting section and
the bitstream generated by the encoding section.
[0020]
5 The layer can include a layer and a sublayer.
[0021]
The layer is a view of multi-view coding.
[0022]
The layer is a layer of scalable video coding.
10 [0023]
The buffer management parameter information is described in supplemental
enhancement information (SE1).
[0024]
The buffer management parameter information is described in
15 buffering_period_SEL
[0025]
Parameter presence/absence information indicating a presence or absence of
the parameter for managing the decoder buffer serving as the parameter for
performing the decoding process of only the corresponding layer is described in a
20 vps (video parameter set)_extension.
[0026]
The setting section can set an AVC flag indicating that a layer lower than the
corresponding layer is encoded by MPEG-4 Parti0 Advanced Video Coding (AVC)
and the buffer management parameter information of each layer indicating that the
25 parameter for managing the decoder buffer is the parameter for peiforming the
decoding process of only the corresponding layer.
[0027]
According to another aspect of the present disclosure, there is provided an
image processing method including setting, by an image processing device, buffer
30 management parameter information of each layer indicating at least one of that a
parameter for managing a decoder buffer is a parameter for performing a decoding
SP351652WO00
6/133
process of only a corresponding layer and that the parameter for managing the
decoder buffer is a parameter for performing a decoding process of a corresponding
layer and a lower layer, encoding, by the image processing device, an image having
at least one layer and generating a bitstreatn, and transmitting, by the image
5 processing device, the set buffer management parameter information and the
generated bitstream.
[0028]
According to one aspect of the present disclosure, a bitstream obtained by
encoding an image having at least one layer and buffer management parameter
10 information of each layer indicating at least one of that a parameter for managing a
decoder buffer is a parameter for performing a decoding process of only a
corresponding layer and that the parameter for managing the decoder buffer is a
parameter for performing a decoding process of a corresponding layer and a lower
layer are received. Then, the received bitstream is decoded to generate an image.
15 [0029]
According to another aspect of the present disclosure, buffer management
parameter information of each layer indicating at least one of that a parameter for
managing a decoder buffer is a parameter for performing a decoding process of only
a corresponding layer and that the parameter for managing the decoder buffer is a
20 parameter for performing a decoding process of a corresponding layer and a lower
layer is set, and an image having at least one layer is encoded to generate a bitstream.
Then, the set buffer management parameter information and the generated bitstream
are transmitted.
[0030]
25 Also, the above-described image processing device may be an independent
device or an inner block constituting one image encoding device or image decoding
device.
Advantageous Effects of Invention
30 [0031]
According to one aspect of the present disclosure, it is possible to decode an
SP361662WO00
7/133
image. Particularly, it is possible to perform a decoding process at a proper timing.
[0032]
According to another aspect of the present disclosure, it is possible to
encode an image. Particularly, it is possible to perform a decoding process at a
5 proper timing.
Brief Description of Drawings
[0033]
[FIG. I] FIG. 1 is a diagram for describing an example of a configuration of a coding
10 unit.
[FIG. 2] FIG. 2 is a diagram for describing an example of spatial scalable video
coding.
[FIG. 3] FIG. 3 is a diagram for describing an example of temporal scalable video
coding.
15 [FIG. 4] FIG. 4 is a diagram for describing an example of scalable video coding of a
signal to noise ratio.
[FIG. 5] FIG. 5 is a diagram illustrating an example of syntax of an HRD parameter
ofaHEVC.
[FIG. 6] FIG. 6 is a diagram for describing a parallel process of scalability video
20 coding.
[FIG. 7] FIG. 7 is a diagram illustrating an example of syntax of an HRD parameter
according to the present technology.
[FIG. 8] FIG. 8 is a diagram illustrating another example of syntax of an HRD
parameter according to the present technology.
25 [FIG. 9] FIG. 9 is a diagram illustrating another example of syntax of an HRD
parameter according to the present technology.
[FIG. 10] FIG. 10 is a block diagram illustrating an example of a main configuration
of a scalable encoding device.
[FIG. 11] FIG. 11 is a block diagram illustrating an example of a main configuration
30 of an enhancement layer image encoding section.
[FIG. 12] FIG. 12 is a block diagram illustrating an example of a main configuration
8/133
SP351652WO00
of an accumulation buffer and an HRD type setting section.
[FIG. 13] FIG. 13 is a diagram for describing an example of a layer structure.
[FIG. 14] FIG. 14 is a flowchart for describing an example of a flow of an encoding
process.
5 [FIG. 15] FIG. 15 is a flowchart for describing an example of a layer encoding
process.
[FIG. 16] FIG 16 is a flowchart for describing an HRD parameter encoding process.
[FIG. 17] FIG. 17 is a flowchart for describing an HRD parameter calculation process.
[FIG. 18] FIG. 18 is a flowchart for describing an HRD parameter of the time layer
10 calculation process.
[FIG. 19] FIG. 19 is a flowchart for describing another example of an HRD parameter
encoding process.
[FIG. 20] FIG. 20 is a block diagram illustrating an example of a main configuration
of a scalable decoding device.
15 [FIG 21] FIG. 21 is a block diagram illustrating an example of a main configuration
of an enhancement layer image decoding section.
[FIG. 22] FIG. 22 is a block diagram illustrating an example of a main configuration
of an accumulation buffer and an HRD type decoding section.
[FIG 23] FIG. 23 is a flowchart for describing an example of a flow of a decoding
20 process.
[FIG 24] FIG. 24 is a flowchart for describing an example of a flow of a layer
decoding process.
[FIG. 25] FIG. 25 is a flowchart for describing another example of a flow of an HRD
parameter decoding process.
25 [FIG 26] FIG. 26 is a flowchart for describing another example of a flow of an
accumulation buffer monitoring process.
[FIG. 27] FIG. 27 is a diagram illustrating an example of syntax of vpsextension.
[FIG 28] FIG. 28 is a diagram illustrating an example of syntax of sps extension.
[FIG. 29] FIG. 29 is a diagram illustrating an example of syntax of vps.
30 [FIG 30] FIG. 30 is a diagram illustrating an example of semantics of
layer_id_jncluded_flag.
9/133
SP351652WO00
[FIG 31] FIG. 31 is a diagram for describing a setting example ofLayerSet.
[FIG 32] FIG 32 is a diagram illustrating an example of syntax of
b u ffer i n g_perio d_SEI.
[FIG. 33] FIG. 33 is a diagram illustrating an example of syntax of
5 buffering_period_SEI,
[FIG 34] FIG. 34 is a diagram illustrating an example of syntax of
buffering_period_SEI.
[FIG. 35] FIG. 35 is a diagram illustrating an example of syntax of
buffering_penod_SEI.
10 [FIG 36] FIG. 36 is a block diagram illustrating another example of a main
configuration of an enhancement layer image encoding section.
[FIG. 37] FIG 37 is a block diagram illustrating an example of a configuration of a
buffering period SEI setting section.
[FIG. 38] FIG. 38 is a flowchart for describing an example of a layer encoding
15 process.
[FIG. 39] FIG. 39 is a flowchart for describing an example of a buffering period SEI
encoding process.
[FIG 40] FIG 40 is a block diagram illustrating another example of a main
configuration of an enhancement layer image decoding section.
20 [FIG. 41] FIG 41 is a block diagram illustrating an example of a configuration of a
buffering period SET decoding section,
[FIG. 42] FIG 42 is a flowchart for describing an example of a layer decoding
process.
[FIG 43] FIG. 43 is a flowchart for describing an example of a buffering period SEI
25 decoding process.
[FIG. 44] FIG. 44 is a flowchart for describing an example of an HRD parameter
encoding process in the case of an AVC flag.
[FIG 45] FIG. 45 is a flowchart for describing an example of a buffering period SEI
encoding process in the case of an AVC flag.
30 [FIG. 46] FIG. 46 is a diagram illustrating an example of a multi-view image
encoding scheme.
SP351652WO00
10/133
[FIG, 47] FIG. 47 is a diagram illustrating an example of a main configuration of a
multi-view image encoding device to which the present disclosure is applied.
[FIG. 48] FIG. 48 is a diagram illustrating an example of a main configuration of a
multi-view image decoding device to which the present disclosure is applied.
5 [FIG 49] FIG 49 is a block diagram illustrating an example of a main configuration
of a computer.
[FIG 50] FIG. 50 is a block diagram illustrating an example of a schematic
configuration of a television device.
[FIG 51] FIG. 51 is a block diagram illustrating an example of a schematic
10 configuration of a mobile phone.
[FIG 52] FIG. 52 is a block diagram illustrating an example of a schematic
configuration of a recording/reproduction device.
[FIG. 53] FIG. 53 is a block diagram illustrating an example of a schematic
configuration of an image capturing device.
15 [FIG. 54] FIG. 54 is a block diagram illustrating an example of using scalable video
coding.
[FIG 55] FIG. 55 is a block diagram illustrating another example of using scalable
video coding.
[FIG. 56] FIG. 56 is a block diagram illustrating another example of using scalable
20 video coding.
Description of Embodiments
[0034]
25 Hereinafter, modes (hereinafter referred to as "embodiments") for carrying
out the present disclosure will be described. The description will proceed in the
following order:
0. Overview
1. First embodiment (image encoding device)
30 2. Second embodiment (image decoding device)
3. Third embodiment (example of syntax)
SP351652WO00
11/133
4. Fourth embodiment (example of buffering_period_SEI)
5. Fifth embodiment (example of AVC flag)
6. Sixth embodiment (multi-view image encoding device/multi-view image
decoding device)
5 7. Seventh embodiment (computer)
8. Applications
9. Applications of scalable video coding
[0035]
<0. Overview>
10
Hereinafter, the present technology will be described in connection with an
application to image encoding and decoding of a High Efficiency Video Coding
(HEVC) scheme.
[0036]
15
In an Advanced Video Coding (AVC) scheme, a hierarchical structure based
on a macrobiock and a sub macrobiock is defined. However, a macrobiock of
16x16 pixels is not optimal for a large image frame such as a Ultra High Definition
(UHD) (4000x2000 pixels) serving as a target of a next generation encoding scheme.
20 [0037]
On the other hand, in the HEVC scheme, a coding unit (CU) is defined as
illustrated in FIG. 1.
[0038]
A CU is also referred to as a coding tree block (CTB), and serves as a partial
25 area of an image of a picture unit undertaking the same role of a macrobiock in the
AVC scheme. The latter is fixed to a size of 16x16 pixels, but the former is not
fixed to a certain size but designated in image compression information in each
sequence.
[0039]
30 For example, a largest coding unit (LCU) and a smallest coding unit (SCU)
of a CU are specified in a sequence parameter set (SPS) included in encoded data to
SP351652WO00
12/133
be output.
[0040]
As split-flag"l is set in a range in which each LCU is not smaller than an
SCU> a coding unit can be divided into CUs having a smaller size. In the example
5 of FIG. I, a size of an LCU is 128, and a largest scalable depth is 5. A CU of a size
of 2N*2N is divided into CUs having a size of N*N serving as a layer that is onelevel
lower when a value of split flag is 1.
[0041]
Further, a CU is divided in prediction units (PUs) that are areas (partial
10 areas of an image of a picture unit) serving as processing units of intra or inter
prediction, and divided into transform units (TUs) that are areas (partial areas of an
image of a picture unit) serving as processing units of orthogonal transform.
Currently, in the HEVC scheme, in addition to 4x4 and 8><8, orthogonal transform of
16x16 and 32x32 can be used.
15 [0042]
As in the HEVC scheme, in the case of an encoding scheme in which a CU
is defined and various kinds of processes are performed in units of CUs, in the AVC
scheme, a macroblock can be considered to correspond to an LCU, and a block (sub
block) can be considered to correspond to a CU. Further, in the AVC scheme, a
20 motion compensation block can be considered to correspond to a PU. Here, since a
CU has a hierarchical structure, a size of an LCU of a topmost layer is commonly set
to be larger than a macroblock in the AVC scheme, for example, such as 128x128
pixels.
[0043]
25 Thus, hereinafter, an LCU is assumed to include a macroblock in the AVC
scheme, and a CU is assumed to include a block (sub block) in the AVC scheme. In
other words, a "block" used in the following description indicates an arbitrary partial
area in a picture, and, for example, a size, a shape, and characteristics thereof are not
limited. In other words, a "block" includes an arbitrary area (a processing unit)
30 such as a TU, a PU, an SCU, a CU, an LCU, a sub block, a macroblock, or a slice.
Of course, a "block" includes other partial areas (processing units) as well. When it
SP351652WO00
13/133
is necessary to limit a size, a processing unit, or the like, it will be appropriately
described.
[0044]

5 Meanwhile, in the AVC and tlEVC encoding schemes, in order to achieve
high encoding efficiency, it is important to select an appropriate prediction mode.
[0045]
As an example of such a selection method, there is a method implemented
in reference software (found at http://iphome.hhi.de/suehring/tmi/index.htm) of
10 H.264/MPEG-4 AVC called a joint model (JM).
[0046]
In the JM, as will be described later, it is possible to select two mode
determination methods, that is, a high complexity mode and a low complexity mode.
In both modes, cost function values related to respective prediction modes are
15 calculated, and a prediction mode having a smaller cost function value is selected as
an optimal mode for a corresponding block or macroblock.
[0047]
A cost function in the high complexity mode is represented as in the
following Formula (1):
20 [0048]
Cost(ModeeO)=im*R ... (1)
[0049]
Here, O indicates a universal set of candidate modes for encoding a
corresponding block or macroblock, and D indicates differential energy between a
25 decoded image and an input image when encoding is performed in a corresponding
prediction mode. X indicates Lagrange's undetermined multiplier given as a
function of a quantization parameter. R indicates a total coding amount including
an orthogonal transform coefficient when encoding is performed in a corresponding
mode.
30 [0050]
In other words, in order to perform encoding in the high complexity mode, it
14/133
SP351652WO00
is necessary to perform a temporary encoding process once by all candidate modes in
order to calculate the parameters D and R, and thus a large computation amount is
required.
[0051]
5 A cost function in the low complexity mode is represented by the following
Formula (2):
[0052]
Cost(Modeen)=D+QP2Quant(QP)*FIeaderBit... (2)
[0053]
10 Here, D is different from that of the high complexity mode and indicates
differential energy between a prediction image and an input image. QP2Quant (QP)
is given as a function of a quantization parameter QP, and HeaderBit indicates a
coding amount related to information belonging to a header such as a motion vector
or a mode including no orthogonal transform coefficient.
15 [0054]
In other words, in the low complexity mode, it is necessary to perforin a
prediction process for respective candidate modes, but since a decoded image is not
necessary, it is unnecessary to perform an encoding process. Thus, it is possible to
implement a computation amount smaller than that in the high complexity mode.
20 [0055]

Meanwhile, the existing image encoding schemes such as MPEG2 and AVC
have a scalability function as illustrated in FIGS. 2 to 4. Scalable video coding
refers to a scheme of dividing (hierarchizing) an image into a plurality of layers and
25 performing encoding for each layer.
[0056]
In hierarchization of an image, one image is divided into a plurality of
images (layers) based on a certain parameter. Basically, each layer is configured
with differential data so that redundancy is reduced. For example, when one image
30 is hierarchized into two layers, that is, a base layer and an enhancement layer, an
image of a lower quality than an original image is obtained using only data of the
SP351652WO00
15/133
base layer, and an original image (that is, a high-quality image) is obtained by
combining data of the base layer with data of the enhancement layer.
[0057]
As an image is hierarchized as described above, it is possible to obtain
5 images of various qualities according to the situation. For example, for a terminal
having a low processing capability such as a mobile phone, image compression
information of only a base layer is transmitted, and a moving image of low spatial
and temporal resolutions or a low quality is reproduced, and for a terminal having a
high processing capability such as a television or a personal computer, image
10 compression information of an enhancement layer as well as a base layer is
transmitted, and a moving image of high spatial and temporal resolutions or a high
quality is reproduced. In other words, image compression information according to
a capability of a terminal or a network can be transmitted from a server without
performing the transcoding process.
15 [0058]
As a parameter having scalability, for example, there is spatial resolution
(spatial scalability) as illustrated in FIG. 2. When the spatial scalability differs,
respective layers have different resolutions. In other words, each picture is
hierarchized into two layers, that is, a base layer of a resolution spatially lower than
20 that of an original image and an enhancement layer that is combined with an image
of the base layer to obtain an original image (an original spatial resolution) as
illustrated in FIG, 2. Of course, the number of layers is an example, and each
picture can be hierarchized into an arbitrary number of layers.
[0059]
25 As another parameter having such scalability, for example, there is a
temporal resolution (temporal scalability) as illustrated in FIG. 3. In the case of the
temporal scalability, respective layers have different frame rates. In other words, in
this case, each picture is hierarchized into layers having different frame rates, a
moving image of a high frame rate can be obtained by combining a layer of a high
30 frame rate with a layer of a low frame rate, and an original moving image (an
original frame rate) can be obtained by combining all the layers as illustrated in FIG.
SP351652WO00
16/133
3. The number of layers is an example, and each picture can be hierarchized into an
arbitrary number of layers.
[0060]
Further, as another parameter having such scalability, for example, there is a
5 signal-to-noise ratio (SNR) (SNR scalability). In the case of the SNR scalability,
respective layers having different SNRs. In other words, in this case, each picture is
hierarchized into two layers, that is, a base layer of an SNR lower than that of an
original image and an enhancement layer that is combined with an image of the base
layer to obtain an original SNR as illustrated in FIG. 4. In other words, for base
10 layer image compression information, information related to an image of a low
PSNR is transmitted, and a high PSNR image can be reconstructed by combining the
information with the enhancement layer image compression information. Of course,
the number of layers is an example, and each picture can be hierarchized into an
arbitrary number of layers.
15 [0061]
A parameter other than the above-described examples may be applied as a
parameter having scalability. For example, there is bit-depth scalability in which
the base layer includes an 8-bit image, and a 10-bit image can be obtained by adding
the enhancement layer to the base layer.
20 [0062]
Further, there is chroma scalability in which the base layer includes a
component image of a 4:2:0 format, and a component image of a 4:2:2 format can be
obtained by adding the enhancement layer to the base layer.
[0063]
25 Further, as a parameter having scalability, there is a multi-view. In this
case, an image is hierarchized into layers of different views.
[0064]
For example, layers described in the present embodiment include spatial,
temporal, SNR, bit depth, color, and view of scalability video coding described
30 above.
[0065]
SP351652WO00
17/133
Further, a term "layer" used in this specification includes a layer of scalable
video coding and each view when a multi-view of a multi-view is considered.
[0066]
Further, the term "layer" used in this specification is assumed to include a
5 main layer (corresponding to sub) and a sublayer. As a specific example, a main
layer may be a layer of spatial scalability, and a sublayer may be configured with a
layer of temporal scalability.
[0067]
In the present embodiment, a layer (Japanese) and a layer have the same
10 meaning, a layer (Japanese) will be appropriately described as a layer.
[0068]

Meanwhile, in the HEVC, when the decoding process of image compression
information is performed, it is possible to designate a Hypothetical Reference
15 Decoder (HDR) parameter illustrated in FIG. 5 so that no overflow or underflow of a
buffer occurs. In other words, the HRD parameter is a parameter used to manage a
decoder buffer. Particularly, when scalable video coding is performed, it is possible
to designate the HRD parameter for each layer in a video parameter set (VPS).
[0069]
20
In an example of FIG 6, two examples (exl and ex2) using a sequence
including an I picture, a b picture, a B picture, a b picture, and a B picture are
illustrated on the left side of FIG. 6 as an example of temporal scalability. In this
sequence, the I picture, the B picture, and the B picture are a lower time layer, and
25 the b picture and the b picture are an upper time layer.
[0070]
Here, the B picture indicates a picture that is referred to, and the b picture
indicates a picture that is not referred to.
[0071]
30 exl is an example in which all the pictures are decoded by a decoding
device #0. On the other hand, ex2 is an example in which the lower time layer of
18/133
SP351652WO00
the I picture, the B picture, and the B picture is decoded by the decoding device #0,
and the upper time layer of the b picture and the b picture is decoded by a decoding
device #1.
[0072]
5 On the right side of FIG. 6, as a scalable HEVC example, two examples
(exll and exl2) using a sequence including the I picture, the B picture, and the B
picture of the EL (enhancement layer) serving as the upper layer and the I picture, the
B picture, and the B picture of the BL (base layer) serving as the lower layer are
illustrated. The scalable HEVC means scalable video coding defined in the HEVC.
10 [0073]
exll is an example in which all the pictures are decoded by the decoding
device #0. On the other hand, exl2 is an example in which the lower layer of the I
picture, the B picture, and the B picture of the BL is decoded by the decoding device
#0, and the upper layer of the I picture, the B picture, and the B picture of the EL is
15 decoded by the decoding device #1.
[0074]
For each layer of the scalable HEVC, each time layer of temporal scalability
at the right side is configured as a sublayer.
[0075]
20 As described above, in the temporal scalability of the related art, in the
scalable HEVC, a process may be performed by a single decoding device, and a
parallel process may be performed by a plurality of decoding devices. Further,
through the syntax of FIG. 5, it is possible to designate the HRD parameter serving as
the parameter used to manage the decoder buffer for each layer or a time layer that is
25 one of sublayers.
[0076]
However, as illustrated in FIG. 6, it is difficult to detect whether the
decoding process is performed by a single decoding device or a plurality of decoding
devices.
30 [0077]
In this regard, in the present technology, the HRD parameter is transmitted
SP351652WO00
19/133
through syntax illustrated in FIG. 7. In other words, in the present technology,
information indicating whether the HRD parameter is a parameter for performing a
decoding process only in a corresponding layer or a parameter for performing a
decoding process of a corresponding layer and a lower layer is set. Thus, as
5 illustrated in FIG. 6, it is clearly defined whether the decoding process is performed
by a single decoding device or a plurality of decoding devices, and thus it is possible
to perform a decoding process at a proper timing.
[0078]

10 FIG. 7 is a diagram illustrating an example of syntax of the HRD parameter
according to the present technology. Numbers on the left of each row are row
numbers added for description.
[0079]
In an example of FIG. 7, hrd_parameters type flag is defined in a 10th row.
15 When a value of hrdparameterstypejflag is I, a value for performing a decoding
process of only a corresponding layer is set as an HRD parameter of a subsequent
paragraph. When a value of hrdparameters typejflag is 0, a value for performing
a decoding process of a corresponding layer and a lower layer is set as an HRD
parameter of a subsequent paragraph.
20 [0080]
Further, hrdparameterstypeflag may be included in an if statement
starting from an 11th row.
[0081]
sub hid_parameters_type[i] flag is defined in a 25th row. When a value
25 of sub_hrd_parameters_type[i]Jflag is 1, a value for performing a decoding process
of only a corresponding time layer is set as a sub HRD parameter of a subsequent
paragraph. When a value of hrd_parameters_typejflag is 0, a value for performing
a decoding process of a corresponding time layer and a lower time layer is set as a
sub HRD parameter of a subsequent paragraph.
30 [0082]
FIG. 7 illustrates the example in which the designating is performed by any
SP351652WO00
20/133
one method (either only a corresponding layer is included or a lower layer is also
included) for each layer and each time layer, but the present technology is not limited
to this example. For example, the HRD parameter in which the HRD parameter is
designated by both methods may be included.
5 [0083]

FIG. 8 is a diagram illustrating another example of syntax of the HRD
parameter according to the present technology. Numbers on the left of each row are
row numbers added for description.
10 [0084]
In an example of FIG. 8, hrd parameters_typel_present flag is defined in
an 11th row. When a value of hrd_parameters_typel_present_flag is 1, a value for
performing a decoding process of only a corresponding layer is set as an HRD
parameter of typel set in 13th to 24th rows. When a value of
15 hrd_parameters_typel_present_flag is 0, a value for performing a decoding process
of a corresponding layer and a lower layer is set as the HRD parameter of typel.
[0085]
hrd_parameters_type2_present_flag is defined in a 12th row. When a
value of hrdparameters„type2 present_flag is 1, a value for performing a decoding
20 process of only a corresponding layer is set as an HRD parameter of type2 defined in
25th to 36th rows. When a value of hrd_parameters_typel_present_flag is 0, a
value for performing a decoding process of a corresponding layer and a lower layer
is set as the HRD parameter of type2.
[0086]
25 Similarly to the example described above with reference to FIG. 7, the flags
of the 11th and 12th rows may be described before an if statement starting from a
10th row.
[0087]
sub_hrd_parameters typel_present_flag is defined in a 40th row. When a
30 value of sub hrd_parameters_typel_present_flag is 1, a value for performing a
decoding process of only a corresponding time layer is set as an HRD parameter of
21/133
SP351652WO00
typel set in 45th to 52nd rows. When a value of
sub_hrd_parameters_typel_present_flag is 0, a value for performing a decoding
process of a corresponding time layer and a lower time layer is set as the HRD
parameter of typel.
5 [0088]
subjird_parameters_type2_present flag is defined in a 41st row. When a
value of sub_hrd_parameters_type2_present_flag is 1, a value for performing a
decoding process of only a corresponding time layer is set as an HRD parameter of
type2 set in 53rd to 60th rows. When a value of
10 sub_hrd_parameters_typel_present_flag is 0, a value for performing a decoding
process of a corresponding time layer and a lower time layer is set as an HRD
parameter of type2.
[0089]
As described above, in the present technology, the parameters of typel and
15 type2 serving as the HRD parameter for the decoding process of only the
corresponding layer and the HRD parameter for the decoding process of the
corresponding layer and the lower layer are set at the encoding side. Thus, the
decoding side can select the parameter according to a device or a received bitstream.
[0090]
20 Further, when the image compression information includes only one layer,
that is, a scalability layer or a temporal scalability layer, hrdparameter type flag
and sub_hi*d_pai'ameter_type_flag may have any value, and the decoding process is
not affected.
[0091]
25 Next, the present technology will be described in connection with
applications to a specific device. For the sake of convenience of description, the
following description will proceed with a case of exl2 of the scalable HEVC and exl
of temporal scalability in FIG. 6. Here, the present technology is not limited to this
case. For example, there may be a case of exl2 of the scalable HEVC and ex2 of
30 temporal scalability in FIG 6, a case of ex 11 of the scalable F1EVC and ex2 of
temporal scalability in FIG 6, and a case of exl 1 of the scalable HEVC and exl of
SP351652WO00
22/133
temporal scalability in FIG. 6.
[0092]
<1. First embodiment

5 FIG. 10 is a block diagram illustrating an example of a main configuration
of a scalable encoding device.
[0093]
A scalable encoding device 100 illustrated in FIG 10 encodes each layer of
image data hierarchized into a base layer and an enhancement layer.
10 [0094]
The scalable encoding device 100 is configured to include a base layer
image encoding section 101-1, an enhancement layer image encoding section 101-2,
and an encoding control section 102.
[0095]
15 The base layer image encoding section 101-1 acquires image information
(base layer image information) of the base layer. The base layer image encoding
section 101-1 encodes the base layer image information without referring to other
layers, generates encoded data (base layer encoded data) of the base layer, and
outputs the generated encoded data.
20 [0096]
The enhancement layer image encoding section 101-2 acquires image
information (enhancement layer image information) of the enhancement layer. The
enhancement layer image encoding section 101-2 encodes the enhancement layer
image information. At this time, the enhancement layer image encoding section
25 101-2 performs inter-layer prediction with reference to information related to
encoding of the base layer as necessary.
[0097]
Further, the enhancement layer image encoding section 101-2 sets the FIRD
parameter type for each layer, and calculates the HRD parameter serving as the
30 parameter used to manage the decoder buffer based on state information of an
accumulation buffer according to the set HRD parameter type. The enhancement
i
SP351652WO00
23/133
layer image encoding section 101-2 encodes the calculated HRD parameter.
[0098]
Specifically, the HRD parameter type indicates whether the HRD parameter
is the parameter for decoding only a corresponding layer or the parameter for
5 performing the decoding process of a corresponding layer and a lower layer. This
type may set both of the parameters as well as any one of the parameters. The
encoding side calculates the HRD parameter according to the flag (information)
indicating the set type, and transmits the flag indicating the set type and the
calculated HRD parameter to the decoding side. Hereinafter, the flag indicating the
10 HRD parameter type is appropriately refened to as an "HRD parameter type flag."
[0099]
When the flag indicating the HRD parameter type is 1, the enhancement
layer image encoding section 101-2 calculates the HRD parameter based on the state
information of its own accumulation buffer. When the flag indicating the HRD
15 parameter type is 0, the enhancement layer image encoding section 101-2 acquires
state information of the whole accumulation buffer of the base layer image encoding
section 101-1, and calculates the HRD parameter based on the state information of
the base layer image encoding section 101-1 and its own accumulation buffer. This
process is performed on a layer and a sublayer (time layer). In the base layer image
20 encoding section 101-1, this process is performed only on the sublayer.
[0100]
The enhancement layer image encoding section 101-2 generates encoded
data (enhancement layer encoded data) of the enhancement layer through the above
encoding, and outputs the generated encoded data.
25 [0101]
The base layer image encoding section 101-1 and the enhancement layer
image encoding section 101-2 are appropriately referred to collectively as a "layer
image encoding section 101."
[0102]
30 The encoding control section 102 controls the encoding process of the layer
image encoding sections 101, for example, in view of the reference relation of the
SP351652WO00
24/133
layer image encoding sections 101.
[0103]
In the example of FIG. 10, one enhancement layer image encoding section
101-2 is illustrated, but when there is an upper layer, enhancement layer image
5 encoding sections 101-3, 101-4, ... that encode the upper layer are provided for each
of the upper layers.
[0104]

FIG. II is a block diagram illustrating an example of a main configuration
10 of the enhancement layer image encoding section 101-2. The base layer image
encoding section 101-1 of FIG. 10 has basically the same configuration as the
enhancement layer image encoding section 101-2 of FIG. 11 except that a type of an
image serving as a target is different. For the sake of convenience of description, in
the example of FIG. II, a configuration of the enhancement layer image encoding
15 section 101-2 will be described as an example.
[0105]
As illustrated in FIG. 11, the enhancement layer image encoding section
101-2 includes an A/D converting section 111, a screen reordering buffer 112, an
operation section 113, an orthogonal transform section 114, a quantization section
20 115, a lossless encoding section 116, an accumulation buffer 117, an inverse
quantization section 118, and an inverse orthogonal transform section 119. The
enhancement layer image encoding section 101-2 further includes an operation
section 120, a loop filter 121, a frame memory 122, a selecting section 123, an intra
prediction section 124, a motion prediction/compensation section 125, a predictive
25 image selecting section 126, and a rate control section 127. The enhancement layer
image encoding section 101-2 further includes an HRD type setting section 128.
[0106]
The A/D converting section 111 performs A/D conversion on input image
data (the enhancement layer image information), and supplies the converted image
30 data (digital data) to be stored in the screen reordering buffer 112. The screen
reordering buffer 112 reorders images of frames stored in a display order in a frame
25/133
SP351652WO00
order for encoding according to a Group Of Pictures (GOP), and supplies the images
in which the frame order is reordered to the operation section 113. The screen
reordering buffer 112 also supplies the images in which the frame order is reordered
to the intra prediction section 124 and the motion prediction/compensation section
5 125.
[0107]
The operation section 113 subtracts a predictive image supplied from the
intra prediction section 124 or the motion prediction/compensation section 125 via
the predictive image selecting section 126 from an image read from the screen
10 reordering buffer 112, and outputs differential information thereof to the orthogonal
transform section 114. For example, in the case of an image that has been subjected
to intra coding, the operation section 113 subtracts the predictive image supplied
from the intra prediction section 124 from the image read from the screen reordering
buffer 112. Further, for example, in the case of an image that has been subjected to
15 inter coding, the operation section 113 subtracts the predictive image supplied from
the motion prediction/compensation section 125 from the image read from the screen
reordering buffer 112.
[0108]
The orthogonal transform section 114 performs an orthogonal transform
20 such as a discrete cosine transform or a Karhunen-Loeve Transform on the
differential information supplied from the operation section 113. The orthogonal
transform section 114 supplies transform coefficients to the quantization section 115.
[0109]
The quantization section 115 quantizes the transform coefficients supplied
25 from the orthogonal transform section 114. The quantization section 115 sets a
quantization parameter based on information related to a target value of a coding
amount supplied from the rate control section 127, and performs the quantizing.
The quantization section 115 supplies the quantized transform coefficients to the
lossless encoding section 116.
30 [0110]
The lossless encoding section 116 encodes the transform coefficients
SP351652WO00
26/133
quantized in the quantization section 115 according to an arbitrary encoding scheme.
Since coefficient data is quantized under control of the rate control section 127, the
coding amount becomes a target value (or approaches a target value) set by the rate
control section 127.
5 [0111]
The lossless encoding section 116 acquires information indicating an intra
prediction mode or the like from the intra prediction section 124, and acquires
information indicating an inter prediction mode, differential motion vector
information, or the like from the motion prediction/compensation section 125.
10 Further, the lossless encoding section 116 appropriately generates an NAL unit of the
enhancement layer including a sequence parameter set (SPS), a picture parameter set
(PPS), and the like.
[0112]
The lossless encoding section 116 encodes various kinds of information
15 according to an arbitrary encoding scheme, and sets (multiplexes) the encoded
information as part of encoded data (also referred to as an "encoded stream"). The
lossless encoding section 116 supplies the encoded data obtained by the encoding to
be accumulated in the accumulation buffer 117.
[0113]
20 Examples of the encoding scheme of the lossless encoding section 116
include variable length coding and arithmetic coding. As the variable length coding,
for example, there is Context-Adaptive Variable Length Coding (CAVLC) defined in
the H.264/AVC scheme. As the arithmetic coding, for example, there is Context-
Adaptive Binary Arithmetic Coding (CABAC).
25 [0114]
The accumulation buffer 117 temporarily holds the encoded data
(enhancement layer encoded data) supplied from the lossless encoding section 116.
The accumulation buffer 117 outputs the held enhancement layer encoded data to a
recording device (recording medium), a transmission path, or the like (not illustrated)
30 at a subsequent stage at a certain timing. In other words, the accumulation buffer
117 serves as a transmitting section that transmits the encoded data as well. Further,
g
SP351652WO00
27/133
when there is a request from the HRD type setting section 128, the accumulation
buffer fl7 supplies information indicating a state of the accumulation buffer 117.
Further, for example, when there is an enhancement layer image encoding section
I0I-3 of an upper layer, and there is a request from its HRD type setting section 128
5 as indicated by a dotted line, the accumulation buffer 117 supplies the information
indicating the state of the accumulation buffer 117 to the enhancement layer image
encoding section 101-3 of the upper layer.
[0115]
The transform coefficients quantized by the quantization section 115 are
10 also supplied to the inverse quantization section 118. The inverse quantization
section 118 inversely quantizes the quantized transform coefficients according to a
method corresponding to the quantization performed by the quantization section 115.
The inverse quantization section 1 \ 8 supplies the obtained transform coefficients to
the inverse orthogonal transform section 119.
15 [0116]
The inverse orthogonal transform section 119 performs an inverse
orthogonal transform on the transform coefficients supplied from the inverse
quantization section 118 according to a method corresponding to the orthogonal
transform process performed by the orthogonal transform section 114. An output
20 (restored differential information) that has been subjected to the inverse orthogonal
transform is supplied to the operation section 120.
[0117]
The operation section 120 obtains a locally decoded image (a decoded
image) by adding the predictive image supplied from the intra prediction section 124
25 or the motion prediction/compensation section 125 via the predictive image selecting
section 126 to the restored differential information serving as an inverse orthogonal
transform result supplied from the inverse orthogonal transform section 119. The
decoded image is supplied to the loop filter 121 or the frame memory 122.
[0118]
30 The loop filter 121 includes a deblock filter, an adaptive offset filter, an
adaptive loop filter, or the like, and appropriately performs a filter process on the
SP351652WO00
28/133
reconstructed image supplied from the operation section 120. For example, the
loop filter 121 performs the deblock filter process on the reconstructed image, and
removes block distortion of the reconstructed image. Further, for example, the loop
filter 121 improves the image quality by performing the loop filter process on the
5 deblock filter process result (the reconstructed image from which the block distortion
has been removed) using a Wiener filter. The loop filter 121 supplies the filter
process result (hereinafter referred to as a "decoded image") to the frame memory
122.
[0119]
10 The loop filter 121 may further perform any other arbitrary filter process on
the reconstructed image. The loop filter 121 may supply information used in the
filter process such as a filter coefficient to the lossless encoding section 116 as
necessary so that the information can be encoded.
[0120]
15 The frame memory 122 stores the reconstructed image supplied from the
operation section 120 and the decoded image supplied from the loop filter 121. The
frame memory 122 supplies the stored reconstructed image to the intra prediction
section 124 via the selecting section 123 at a certain timing or based on an external
request, for example, from the intra prediction section 124. Further, the frame
20 memory 122 supplies the stored decoded image to the motion
prediction/compensation section 125 via the selecting section 123 at a certain timing
or based on an external request, for example, from the motion
prediction/compensation section 125.
[0121]
25 The frame memory 122 stores the supplied decoded image, and supplies the
stored decoded image to the selecting section 123 as a reference image at a certain
timing.
[0122]
The selecting section 123 selects a supply destination of the reference image
30 supplied from the frame memory 122. For example, in the case of the intra
prediction, the selecting section 123 supplies the reference image (a pixel value of a
29/133
SP351652WO00
current picture) supplied from the frame memory 122 to the motion
prediction/compensation section 125. Further, for example, in the case of the inter
prediction, the selecting section 123 supplies the reference image supplied from the
frame memory 122 to the motion prediction/compensation section 125.
5 [0123]
The intra prediction section 124 performs the intra prediction (intra-screen
prediction) for generating the predictive image using the pixel value of the current
picture serving as the reference image supplied from the frame memory 122 via the
selecting section 123. The intra prediction section 124 performs the intra prediction
10 in a plurality of intra prediction modes that are prepared in advance.
[0124]
The intra prediction section 124 generates predictive images in all the intra
prediction modes serving as the candidates, evaluates cost function values of the
predictive images using the input image supplied from the screen reordering buffer
15 112, and selects an optimal mode. When the optimal intra prediction mode is
selected, the intra prediction section 124 supplies the predictive image generated in
the optimal mode to the predictive image selecting section 126.
[0125]
As described above, the intra prediction section 124 appropriately supplies,
20 for example, the intra prediction mode information indicating the employed intra
prediction mode to the lossless encoding section 116 so that the information is
encoded.
[0126]
The motion prediction/compensation section 125 performs the motion
25 prediction (the inter prediction) using the input image supplied from the screen
reordering buffer 112 and the reference image supplied from the frame memory 122
via the selecting section 123. Although not illustrated, in the motion
prediction/compensation section 125, the reference image supplied from the frame
memory 122 of the base layer image encoding section 101-1 is also referred to as
30 necessary. The motion prediction/compensation section 125 performs a motion
compensation process according to a detected motion vector, and generates a
i
SP351652WO00
30/133
predictive image (inter-predictive image information). The motion
prediction/compensation section 125 performs the inter prediction in a plurality of
inter prediction modes that are prepared in advance.
[0127]
5 The motion prediction/compensation section 125 generates predictive
images in all the inter prediction modes serving as a candidate. The motion
prediction/compensation section 125 evaluates cost function values of the predictive
images using the input image supplied from the screen reordering buffer 112,
information of the generated differential motion vector, and the like, and selects an
10 optimal mode. When the optimal inter prediction mode is selected, the motion
prediction/compensation section 125 supplies the predictive image generated in the
optimal mode to the predictive image selecting section 126.
[0128]
The motion prediction/compensation section 125 supplies information
15 indicating the employed inter prediction mode, information necessary for performing
processing in the inter prediction mode when the encoded data is decoded, and the
like to the lossless encoding section 116 so that the information is encoded. For
example, as the necessary information, there is information of a generated
differential motion vector, and as prediction motion vector information, there is a
20 flag indicating an index of a prediction motion vector.
[0129]
The predictive image selecting section 126 selects a supply source of the
prediction image to be supplied to the operation section 113 and the operation section
120. For example, in the case of the intra coding, the predictive image selecting
25 section 126 selects the intra prediction section 124 as the supply source of the
predictive image, and supplies the predictive image supplied from the intra
prediction section 124 to the operation section 113 and the operation section 120.
For example, in the case of the inter coding, the predictive image selecting section
126 selects the motion prediction/compensation section 125 as the supply source of
30 the predictive image, and supplies the predictive image supplied from the motion
prediction/compensation section 125 to the operation section 113 and the operation
SP351652WO00
31/133
section 120.
[0130]
The rate control section 127 controls a rate of a quantization operation of the
quantization section 115 based on the coding amount of the encoded data
5 accumulated in the accumulation buffer 117 such that no overflow or underflow
occurs.
[0131]
The HRD type setting section 128 decides the HRD parameter type
according to the user's instruction, and acquires information indicating an
10 accumulation state from the accumulation buffer 117 or the accumulation buffer (the
lower layer) 117 of the base layer image encoding section 101-1 according to the
decided HRD parameter type. The HRD type setting section 128 calculates the
HRD parameter based on the acquired information, and causes the lossless encoding
section 116 to encode the flag indicating the HRD parameter type and the HRD
15 parameter.
[0132]
Further, when the image compression information (encoded data) to be
output includes one layer, the value of the flag indicating the HRD parameter type is
arbitrary, and does not affect the process at the decoding side.
20 [0133]

FIG. 12 is a block diagram illustrating an example of a configuration of the
accumulation buffer and the HRD type setting section of FIG. 11.
25 [0134]
In an example of FIG. 12, the accumulation buffer 117 is configured to
include a partial accumulation buffer 131 and a whole accumulation buffer 132.
[0135]
The HRD type setting section 128 is configured to include a layer HRD
30 parameter calculating section 141, a time layer HRD parameter calculating section
142, an HRD parameter type setting section 143, and a time HRD parameter type
SP351652WO00
setting section 144.
[0136]
The partial accumulation buffer 131 is configured with each accumulation
buffer that accumulates encoded data related to each upper time layer among the
5 encoded data (codes) accumulated in the whole accumulation buffer 132. The
information indicating the state of each accumulation buffer is supplied to the time
layer HRD parameter calculating section 142 on request.
[0137]
The whole accumulation buffer 132 accumulates the encoded data (codes)
10 encoded by the lossless encoding section 116. Further, information indicating a
state of the whole accumulation buffer of the whole accumulation buffer 132 is
supplied to the layer HRD parameter calculating section 141 and the time layer HRD
parameter calculating section 142 on request. Further, there are cases in which
there is the enhancement layer image encoding section 101-3 of the upper layer as
15 indicated by a dotted line. In this case, the information indicating the state of the
whole accumulation buffer of the whole accumulation buffer 132 is also supplied to
the HRD type setting section (upper layer) 128 according to the request of the HRD
type setting section (upper layer) 128 of the enhancement layer image encoding
section 101-3.
20 [0138]
The layer HRD parameter calculating section 141 acquires the information
indicating the state of the whole accumulation buffer 132 and the information
indicating the state of the accumulation buffer (the lower layer) 117 of the base layer
image encoding section 101-1 according to the HRD parameter type (flag) supplied
25 from the HRD parameter type setting section 143. Practically, information is
acquired from the whole accumulation buffer 132 of the accumulation buffer of the
base layer image encoding section 101-1.
[0139]
When the HRD parameter type flag indicates 1, the information indicating
30 the state of the whole accumulation buffer 132 is acquired. When the HRD
parameter type flag indicates 0, the information indicating the state of the whole
32/133
33/133
SP351652WO00
accumulation buffer 132 and the information indicating the state of the accumulation
buffer (the lower layer) 117 of the base layer image encoding section 101-1 are
acquired.
[0140]
5 The layer HRD parameter calculating section 141 calculates a layer HRD
parameter based on the HRD parameter type flag supplied from the HRD parameter
type setting section 143 and the acquired information, and supplies the calculated
layer HRD parameter to the lossless encoding section 116.
[0141]
10 The time layer HRD parameter calculating section 142 acquires the
information indicating the state of the whole accumulation buffer 132 and the
information indicating the state of the accumulation buffer of the corresponding time
layer of the partial accumulation buffer 131 according to the sub HRD parameter
type (flag) supplied from the time HRD parameter type setting section 144.
15 [0142]
When the sub HRD parameter type flag indicates 1, the information
indicating the state of the accumulation buffer of the corresponding time layer of the
partial accumulation buffer 131 is acquired. When the sub HRD parameter type
flag indicates 0, the information indicating the state of the whole accumulation buffer
20 132 and the information indicating the state of the accumulation buffer of the
corresponding time layer of the paitial accumulation buffer 131 are acquired.
[0143]
The time layer HRD parameter calculating section 142 calculates a time
layer HRD parameter based on the sub HRD parameter type supplied from the time
25 HRD parameter type setting section 144 and the acquired information, and supplies
the calculated time layer HRD parameter to the lossless encoding section 116.
[0144]
The HRD parameter type setting section 143 sets the HRD parameter type
according to the user's instruction, and supplies the flag indicating the set HRD
30 parameter type to the lossless encoding section 116 and the layer HRD parameter
calculating section 141.
SP351652WO00
34/133
[0145]
The time HRD parameter type setting section 144 sets the sub HRD
parameter type according to the user's instruction, and supplies a flag indicating the
set sub HRD parameter type to the lossless encoding section 116 and the time layer
5 HRD parameter calculating section 142.
[0146]
The lossless encoding section 116 encodes the flag indicating the HRD
parameter type supplied from the HRD parameter type setting section 143 and the
layer HRD parameter supplied from the layer HRD parameter calculating section 141,
10 and sets the encoded information as header information of the encoded data. The
lossless encoding section 116 encodes the flag indicating the sub HRD parameter
type supplied from the time HRD parameter type setting section 144, and the time
layer HRD parameter supplied from the time layer HRD parameter calculating
section 142, and sets the encoded information as the header information of the
15 encoded data. The encoded data is output to the whole accumulation buffer 132.
[0147]

In the scalable video coding, image data is hierarchized into a plurality of
layers as described above with reference to FIGS. 2 to 4. In the following, for the
20 sake of description, this layer is referred to as a main layer.
[0148]
A picture group of each main layer configures a sequence in the main layer.
The pictures in the sequence form a hierarchical structure (a GOP structure) as
illustrated in FIG. 13, similarly to moving image data of a single main layer. In the
25 following, for the sake of description, a layer in one main layer is referred to as a
sublayer.
[0149]
In the example of FIG. 13, a main layer is configured with two layers, that is,
a base layer (BaseLayer) and an enhancement layer (EnhLayer). The base layer is a
30 layer in which an image is formed by only its own main layer without depending on
other main layers. Data of the base layer is encoded and decoded without referring
35/133
SP351652WO00
to other main layers. The enhancement layer is a main layer that is combined with
data of the base layer to obtain an image. Data of the enhancement layer can be
used by a prediction process (an inter-main layer prediction process) (which is also
referred to as "inter-layer prediction") with a corresponding base layer.
5 [0150]
The number of main layers of encoded data hierarchized by the scalable
video coding is arbitrary. In the following, each main layer is assumed to be set as a
base layer or an enhancement layer, and in each enhancement layer, any one base
layer is assumed to be set as a reference destination.
10 [0151]
In the example of FIG. 13, each of the base layer and the enhancement layer
has a GOP structure configured with three sublayers, that is, a sublayer 0 (SublayerO),
a sublayer 1 (Sublayerl), and a sublayer 2 (Sublayer2). A square illustrated in FIG
13 indicates a picture, and a character in the square indicates a type of a picture.
15 For example, a square in which "I" is written indicates an I picture, a square in which
"B" is written indicates a B picture that is referable to, and a square in which "b" is
written indicates a B picture that is not referred to. Further, a dotted line between
squares indicates a dependence relation (a reference relation). As indicated by
individual dotted lines, a picture of an upper sublayer depends on a picture of a lower
20 sublayer. In other words, the picture of the sublayer 1 or the picture of the sublayer
0 is referred to by the picture of the sublayer 2 (Sublayer2). Further, the picture of
the sublayer 0 is referred to by the picture of the sublayer 1. The picture of the
sublayer 0 is appropriately referred to by the picture of the sublayer 0.
[0152]
25 The number of sublayers (a sublayer number) is arbitrary. The GOP
structure is also arbitrary, and not limited to the example of FIG. 13.
[0153]
Here, a correspondence relation with the present embodiment will be
described. Encoded data of all pictures of the enhancement layer is accumulated in
30 the whole accumulation buffer 132 of FIG. 12.
[0154]
1
SP351652WO00
36/133
The partial accumulation buffer 131 of FIG. 12 includes an accumulation
buffer of the sublayer 1 and an accumulation buffer of the sublayer 2. In other
words, for example, encoded data of the B pictures of the sublayer 1 indicated by B2,
B4, and B6 in the enhancement layer of FIG. 13 is accumulated in the accumulation
5 buffer of the sublayer 1. Encoded data of the B pictures of the sublayer 2 indicated
by bl, b3, b5, and b7 in the enhancement layer is accumulated in the accumulation
buffer of the sublayer 2.
[0155]
Further, encoded data of all pictures of the base layer of FIG 13 is
10 accumulated in (the whole accumulation buffer 132 of) the accumulation buffer 117
of the base layer image encoding section I0I-1 illustrated in FIG. 12, and
information indicating the buffer state is supplied to the layer HRD parameter
calculating section 141 as information indicating a state of the lower layer whole
accumulation buffer.
15 [0156]
Further, although not illustrated, the partial accumulation buffer 131 of the
accumulation buffer 117 of the base layer image encoding section 101-1 includes an
accumulation buffer of the sublayer 1 and an accumulation buffer of the sublayer 2.
In other words, for example, encoded data of the B pictures of the sublayer 1
20 indicated by B2, B4, and B6 in the base layer of FIG. 13 is accumulated in the
accumulation buffer of the sublayer 1. Encoded data of the B pictures of the
sublayer 2 indicated by bl, b3, b5, and b7 in the base layer is accumulated in the
accumulation buffer of the sublayer 2.
[0157]
25
Next, the flow of the process performed by the scalable encoding device 100
will be described. First, an example of the flow of an encoding process will be
described with reference to a flowchart of FIG. 14.
[0158]
30 When the encoding process starts, in step SI01, the encoding control section
102 of the scalable encoding device 100 decides a layer of a processing target in
37/133
SP351652WO00
view of the reference relation of an image or the like.
[0159]
In step S102, the base layer image encoding section 101-1 performs a layer
encoding process under control of the encoding control section 102. The layer
5 encoding process will be described later with reference to FIG. 15. When the
process of step SI02 ends, the process proceeds to step S103.
[0160]
In step S103, the encoding control section 102 determines whether or not all
the main layers have been processed. When it is determined that there is a non-
10 processed main layer, the process proceeds to step S104.
[0161]
In step SI04, the encoding control section 102 sets a next non-processed
main layer as a processing target (current main layer). When the process of step
S104 ends, the process returns to step S102. In step S102, the enhancement layer
15 image encoding section 101-2 performs the layer encoding process under control of
the encoding control section 102. The process of steps S102 to S104 is repeatedly
performed to encode the main layers as described above. The process of step SI 02
may be processed in parallel by a plurality of layer image encoding sections 101
having no reference relation.
20 [0162]
Then, when all the main layers are determined to have been processed in
step S103, the encoding process ends.
[0163]

25 Next, the layer encoding process in step S102 of FIG. 14 will be described
with reference to a flowchart of FIG. 15. An example of FIG. 15 will be described
in connection with an example of the enhancement layer image encoding section
101-2.
[0164]
30 In step S i l l , the A/D converting section 111 of the enhancement layer
image encoding section 101-2 performs A/D conversion on input image information
SP351652WO00
38/133
(image data) of the enhancement layer. In step SI 12, the screen reordering buffer
112 stores image information (digital data) of the enhancement layer that has been
subjected to the A/D conversion, and reorders the pictures arranged in the display
order in the encoding order.
5 [0165]
In step S113, the intra prediction section 124 performs the intra prediction
process in the intra prediction mode. In step SI 14, the motion
prediction/compensation section 125 performs an inter motion prediction process in
which motion prediction and motion compensation in the inter prediction mode are
10 performed. In step SI 15, the predictive image selecting section 126 decides an
optimal mode based on the cost function values output from the intra prediction
section 124 and the motion prediction/compensation section 125. In other words,
the predictive image selecting section 126 selects either of the predictive image
generated by the intra prediction section 124 and the predictive image generated by
15 the motion prediction/compensation section 125. In step SI16, the operation
section 113 calculates a difference between the image reordered in the process of step
SI 12 and the predictive image selected in the process of step SI 15. The differential
data is smaller in a data amount than the original image data. Thus, it is possible to
compress a data amount to be smaller than when an image is encoded without change.
20 [0166]
In step S117, the orthogonal transform section 114 performs the orthogonal
transform process on the differential information generated in the process of step
SI 16. Tn step SI 18, the quantization section 115 quantizes the orthogonal transform
coefficients obtained in the process of step SI 17 using the quantization parameter
25 calculated by the rate control section 127.
[0167]
The differential information quantized in the process of step SI 18 is locally
decoded as follows. In other words, in step SI 19, the inverse quantization section
118 performs inverse quantization on the quantized coefficients (which are also
30 referred to as "quantization coefficients") quantized in the process of step SI 18
according to characteristics corresponding to characteristics of the quantization
39/133
SP351652WO00
section 115. In step SI20, the inverse orthogonal transform section 119 performs
the inverse orthogonal transform on the orthogonal transform coefficients obtained in
the process of step SI 17. In step S121, the operation section 120 generates a locally
decoded image (an image corresponding to an input of the operation section 113) by
5 adding the predictive image to the locally decoded differential information.
[0168]
In step S122, the loop filter 121 performs filtering on the image generated in
the process of step S121. As a result, for example, block distortion is removed. In
step S123, the frame memory 122 stores the image in which, for example, the block
10 distortion has been deleted in the process of step S122. The image that is not
subjected to the filter process performed by the loop filter 121 is also supplied from
the operation section 120 and stored in the frame memory 122. The image stored in
the frame memory 122 is used in the process of step S113 or the process of step S114.
[0169]
15 In step SI24, the HRD type setting section 128 performs an HRD parameter
encoding process. The HRD parameter encoding process will be described later
with reference to FIG. 16, and through this process, the flag indicating the HRD
parameter type and the HRD parameter are supplied to the lossless encoding section
116.
20 [0170]
In step SI25, the lossless encoding section 116 encodes the coefficients
quantized in the process of step S118. In other words, lossless coding such as
variable length coding or arithmetic coding is performed on data corresponding to
the differential image.
25 [0171]
At this time, the lossless encoding section 116 encodes information related
to the prediction mode of the predictive image selected in the process of step SI 15,
and adds the encoded information to the encoded data obtained by encoding the
differential image. In other words, the lossless encoding section 116 also encodes,
30 for example, information according to the optimal intra prediction mode information
supplied from the intra prediction section 124 or the optimal inter prediction mode
SP351652WO00
40/133
supplied from the motion prediction/compensation section 125, and adds the encoded
information to the encoded data. Further, the lossless encoding section 116 also
encodes information such as a flag indicating the HRD parameter type and the HRD
parameter supplied in the process of step S124, and adds the encoded information to
5 the encoded data.
[0172]
In step S126, the accumulation buffer 117 accumulates the enhancement
layer encoded data obtained in the process of step S125. The enhancement layer
encoded data accumulated in the accumulation buffer 117 is appropriately read and
10 transmitted to the decoding side via a transmission path or a recording medium.
[0173]
In step S127, the rate control section 127 controls the quantization operation
of the quantization section 115 based on the coding amount (the generated coding
amount) of the encoded data accumulated in the accumulation buffer 117 in the
15 process of step SI26 so that no overflow or underflow occurs. Fuither, the rate
control section 127 supplies information related to the quantization parameter to the
quantization section 115.
[0174]
When the process of step S127 ends, the encoding process ends, and the
20 process returns to step S102 of FIG. 14.
[0175]

Next, an example of encoding the HRD parameter illustrated in FIG. 7 will
be described with reference to a flowchart of FIG. 16.
25 [0176]
In step S131, the HRD parameter type setting section 143 sets the HRD
parameter type according to the user's instruction. The HRD parameter type setting
section 143 supplies the flag indicating the set HRD parameter type to the lossless
encoding seetion 116 and the layer HRD parameter calculating section 141.
30 [0177]
In step SI32, the layer HRD parameter calculating section 141 performs a
SP351652WO00
41/133
process of calculating the HRD parameter of the corresponding layer according to
the flag indicating the HRD parameter type supplied from the HRD parameter type
setting section 143. The process of calculating the HRD parameter will be
described later with reference to FIG. 17.
5 [0378]
In step S133, the layer HRD parameter calculating section 141 supplies the
HRD parameter of the corresponding layer calculated in step S132 to the lossless
encoding section 116 so that the HRD parameter of the corresponding layer is
encoded.
10 [0179]
The flag indicating the HRD parameter type supplied in step SI31 and the
HRD parameter of the layer supplied in step S133 are encoded in step S125 of FIG.
15.
[0180]
15 In step S134, the time HRD parameter type setting section 144 sets the sub
HRD parameter type according to the user's instruction. The time HRD parameter
type setting section 144 supplies the flag indicating the set sub HRD parameter type
to the lossless encoding section 116 and the time layer HRD parameter calculating
section 142.
20 [0181]
In step S135, the time layer HRD parameter calculating section 142
performs a process of calculating the HRD parameter of the corresponding time layer
according to the flag indicating the sub HRD parameter type supplied from the time
HRD parameter type setting section 144. The process of calculating the HRD
25 parameter of the time layer will be described later with reference to FIG 18.
[0182]
In step S136, the time layer HRD parameter calculating section 142 supplies
the HRD parameter of the time layer calculated in step S135 to the lossless encoding
section 116 so that the HRD parameter of the time layer is encoded.
30 [0183]
The flag indicating the sub HRD parameter type supplied in step SI34 and
3
SP351652WO00
42/133
the HRD parameter of the time layer supplied in step S134 are encoded in step S125
of FIG. 15.
[0184]
In step S137, the time layer HRD parameter calculating section 142
5 determines whether or not the process has ended on all the time layers. When the
process is determined to have ended on all the time layers in step SI37, the HRD
parameter encoding process ends, and the process returns to step S124 of FIG. 15.
[0185]
When the process is determined not to have ended on any one of the time
10 layers in step S137, the process returns to step SI34, and the subsequent process is
repeated.
[0186]

Next, the process of calculating the HRD parameter in step S132 of FIG. 16
15 will be described with reference to a flowchart of FIG 17.
[0187]
The HRD parameter type flag is supplied to the layer HRD parameter
calculating section 141 through step S131 of FIG. 16. In step S141, the layer HRD
parameter calculating section 141 determines whether or not the HRD parameter type
20 flag supplied from the HRD parameter type setting section 143 is 1.
[0188]
When the HRD parameter type flag is determined to be 1 in step S141, the
process proceeds to step SI42.
[0189]
25 In step S142, the layer HRD parameter calculating section 141 acquires the
information indicating the state of the whole accumulation buffer 132. In step SI 43,
the layer HRD parameter calculating section 141 calculates the HRD parametei- of
the corresponding layer based on the acquired information indicating the state of the
whole accumulation buffer 132.
30 [0190]
When the HRD parameter type flag is determined not to be 1 in step S141,
jj
SP351652WO00
43/133
the process proceeds to step S144.
[0191]
In step S144, the layer HRD parameter calculating section 141 acquires the
information indicating the state of the whole accumulation buffer 132. In step SI 45,
5 the layer HRD parameter calculating section 141 acquires the information indicating
the state of the accumulation buffer (the lower layer) 117 of the base layer image
encoding section I01-I. In step S146, the layer HRD parameter calculating section
141 calculates the HRD parameter of the corresponding layer based on the acquired
information.
10 [0192]
After step SI43 or SI46, the HRD parameter calculation process ends, and
the process returns to step S132 of FIG. 16.
[0193]

15 Next, the process of calculating the HRD parameter of the time layer in step
S135 of FIG. 16 will be described with reference to a flowchart of FIG 18.
[0194]
The sub HRD parameter type flag is supplied to the time layer HRD
parameter calculating section 142 through step S134 of FIG. 16. In step S151, the
20 time layer HRD parameter calculating section 142 determines whether or not the sub
HRD parameter type flag supplied from the time HRD parameter type setting section
144 is 1.
[0195]
When the sub HRD parameter type flag is determined to be 1 in step S151,
25 the process proceeds to step SI52.
[0196]
In step S152, the time layer HRD parameter calculating section 142 acquires
the information indicating the state of the accumulation buffer of the corresponding
time layer of the partial accumulation buffer 131. In step SI 53, the time layer HRD
30 parameter calculating section 142 calculates the HRD parameter of the time layer
based on the acquired information indicating the state of the paitial accumulation
i
SP351652WO00
44/133
buffer 131.
[0197]
When the sub HRD parameter type flag is determined not to be 1 in step
SI51, the process proceeds to step SI54.
5 [0198]
In step SI54, the time layer HRD parameter calculating section 142 acquires
the information indicating the state of the whole accumulation buffer 132. In step
SI55, the time layer HRD parameter calculating section 142 acquires the information
indicating the state of the accumulation buffer of the corresponding time layer of the
10 partial accumulation buffer 131. In step SI56, the time layer HRD parameter
calculating section 142 calculates the HRD parameter of the corresponding time
layer based on the acquired information.
[0199]
After step S153 or S156, the time layer HRD parameter calculation process
15 ends, and the process returns to step SI 35 of FIG. 16.
[0200]

Next, the HRD parameter encoding process of step S124 of FIG. 15 will be
described with reference to a flowchart of FIG 19. FIG 16 illustrates an example
20 of encoding the HRD parameter illustrated in FIGS. 8 and 9.
[0201]
In step SI61, the HRD parameter type setting section 143 sets the HRD
parameter type according to the user's instruction. The HRD parameter type setting
section 143 supplies the flag indicating the set HRD parameter type to the lossless
25 encoding section 116 and the layer HRD parameter calculating section 141.
[0202]
In step S162, the layer HRD parameter calculating section 141 performs the
process of calculating the HRD parameter of the corresponding layer according to
the flag indicating the HRD parameter type supplied from the HRD parameter type
30 setting section 143. Since the process of calculating the HRD parameter is basically
the same as the process described with reference to FIG 17, duplicate description is
SP351652WO00
45/133
omitted.
[0203]
In step S163, the layer HRD parameter calculating section 141 determines
whether or not the HRD parameter calculation process has ended on both of type 1
5 and type2. When the HRD parameter calculation process is determined not to have
ended on either type in step SI63, the process returns to step SI62, and the
subsequent process is repeated.
[0204]
Further, when either of typel and type2 is calculated, the process proceeds
10 to step SI63.
[0205]
When the HRD parameter calculation process is determined to have ended
on both typel and type2 in step SI63, the process proceeds to step SI64.
[0206]
15 In step S164, the layer HRD parameter calculating section 141 supplies the
HRD parameter of the corresponding layer calculated in step SI62 to the lossless
encoding section 116 so that the HRD parameter of the corresponding layer is
encoded.
[0207]
20 The flag indicating the HRD parameter type supplied in step S161 and the
HRD parameter of the layer supplied in step S164 are encoded in step S125 of FIG.
15.
[0208]
In step SI65, the time HRD parameter type setting section 144 sets the sub
25 HRD parameter type according to the user's instruction. The time HRD parameter
type setting section 144 supplies the flag indicating the set sub HRD parameter type
to the lossless encoding section 116 and the time layer HRD parameter calculating
section 142.
[0209]
30 In step SI66, the time layer HRD parameter calculating section 142
performs the process of calculating the HRD parameter of the corresponding time
SP351652WO00
46/133
layer according to the flag indicating the sub HRD parameter type supplied from the
time HRD parameter type setting section 144. Since the process of calculating the
HRD parameter of the time layer is basically the same as the process described with
reference to FIG. 18, duplicate description is omitted.
5 [0210]
In step S167, the layer HRD parameter calculating section 141 determines
whether or not the HRD parameter calculation process has ended on both of typel
and type2. When the HRD parameter calculation process is determined not to have
ended on either type in step SI63, the process returns to step SI66, and the
10 subsequent process is repeated.
[0211]
Further, when either of typel and type2 is calculated, the process proceeds
tostepS168.
[0212]
15 When the HRD parameter calculation process is determined to have ended
on both typel and type2 in step SI67, the process proceeds to step SI68.
[0213]
In step SI68, the time layer HRD parameter calculating section 142 supplies
the HRD parameter of the corresponding time layer calculated in step S166 to the
20 lossless encoding section 116 so that the HRD parameter of the time layer is encoded.
[0214]
The flag indicating the sub HRD parameter type supplied in step SI65 and
the HRD parameter of the time layer supplied in step S168 are encoded in step S125
of FIG. 15.
25 [0215]
In step SI69, the time layer HRD parameter calculating section 142
determines whether or not the process has ended on all the time layers. When the
process is determined to have ended on all the time layers in step SI69, the HRD
parameter encoding process ends, and the process returns to step S124 of FIG. 15.
30 [0216]
When the process is determined not to have ended on any one of the time
SP351652WO00
47/133
layers in step SI69, the process returns to step SI65, and the subsequent process is
repeated.
[0217]
As described above, the HRD parameter type flag indicating whether the
5 HRD parameter is the parameter for performing the decoding process of only a
corresponding layer or the parameter for performing the decoding process of the
corresponding layer and the lower layer is set at the encoding side. Thus, it is
possible to perform a decoding process at a proper timing.
[0218]
10 <2. Second embodiment

Next, decoding of the encoded data (bitstream) that has been subjected to
the scalable video coding as described above will be described. FIG. 20 is a block
diagram illustrating an example of a main configuration of a scalable decoding
15 device corresponding to the scalable encoding device 100 of FIG. 10. For example,
a scalable decoding device 200 illustrated in FIG. 20 performs scalable decoding on
the encoded data obtained by performing the scalable encoding on the image data
through the scalable encoding device 100 according to a method corresponding to the
encoding method.
20 [0219]
The scalable decoding device 200 is configured to include a base layer
image decoding section 201-1, an enhancement layer image decoding section 201-2,
and a decoding control section 202.
[0220]
25 The base layer image decoding section 201-1 is an image decoding section
corresponding to the base layer image encoding section 101-1 of FIG. 10, and
acquires, for example, the base layer encoded data obtained by eneoding the base
layer image information through the base layer image encoding section 101-1. The
base layer image decoding section 201-1 decodes the base layer encoded data
30 without referring to other layers, reconstructs the base layer image information, and
outputs the base layer image information.
48/133
SP351652WO00
[0221]
The enhancement layer image decoding section 201-2 is an image decoding
section corresponding to the enhancement layer image encoding section 101-2, and
acquires, for example, the enhancement layer encoded data obtained by encoding the
5 enhancement layer image information through the enhancement layer image
encoding section 101-2, The enhancement layer image decoding section 201-2
decodes the enhancement layer encoded data. At this time, the enhancement layer
image decoding section 201-2 performs the inter-layer prediction with reference to
information related to decoding of the base layer as necessary.
10 [0222]
Further, the flag indicating the HRD parameter type and the HRD parameter
are added to each piece of encoded data (bitstream) and transmitted. The
enhancement layer image decoding section 201-2 receives the flag indicating the
HRD parameter type, acquires the state information of the accumulation buffer
15 according to the received flag indicating the HRD parameter type, and monitors the
accumulation buffer.
[0223]
When the flag indicating the HRD parameter type is 1, the enhancement
layer image decoding section 201-2 recognizes the HRD parameter as a parameter
20 used to decode only a corresponding layer, acquires the state information of its own
accumulation buffer, and monitors the accumulation buffer. When the flag
indicating the HRD parameter type is 0, the enhancement layer image decoding
section 201-2 recognizes the HRD parameter as a parameter used to perform a
decoding process of a corresponding layer and a lower layer, acquires the state
25 information of the accumulation buffer of the base layer image decoding section 201-
1, and monitors the accumulation buffer. This process is performed on a layer and a
sublayer (time layer). In the base layer image decoding section 201-1, this process
is performed on only a sublayer.
' [0224]
30 Through the decoding, the enhancement layer image decoding section 201-2
decodes the encoded data of the enhancement layer, reconstructs the enhancement
SP351652WO00
49/133
layer image information, and outputs the enhancement layer image information.
[0225]
The base layer image decoding section 201-1 and the enhancement layer
image decoding section 201-2 are appropriately referred to collectively as a "layer
5 image decoding section 201."
[0226]
The decoding control section 202 controls the decoding process of the layer
image decoding sections 201, for example, in view of the reference relation of the
layer image decoding sections 201.
10 [0227]
Tn the example of FIG. 20, one enhancement layer image decoding section
201-2 is illustrated, but when there is an upper layer, enhancement layer image
decoding sections 201-3, 4, ... that encode the upper layer are provided for each of
the upper layers.
15 [0228]

FIG 21 is a block diagram illustrating an example of a main configuration
of the enhancement layer image decoding section 201-2 of FTG. 20. The base layer
image decoding section 201-1 of FIG 20 has basically the same configuration as the
20 enhancement layer image decoding section 201-2 of FIG. 21 except that a type of an
image serving as a target is different. For the sake of description, in the example of
FIG. 21, a configuration of the enhancement layer image decoding section 201-2 will
be described as an example.
[0229]
25 As illustrated in FIG. 21, the enhancement layer image decoding section
201-2 includes an accumulation buffer 211, a lossless decoding section 212, an
inverse quantization section 213, an inverse orthogonal transform section 214, an
operation section 215, a loop filter 216, a screen reordering buffer 217, and a D/A
converting section 218. The enhancement layer image decoding section 201-2
30 fuither includes a frame memory 219, a selecting section 220, an intra prediction
section 221, a motion prediction/compensation section 222, and a selecting section
SP351652WO00
50/133
223. The enhancement layer image decoding section 201-2 further includes an
HRD type decoding section 224.
[0230]
The accumulation buffer 211 is a receiving section that receives the
5 transmitted enhancement layer encoded data. The accumulation buffer 211 receives
and accumulates the transmitted enhancement layer encoded data, and supplies the
encoded data to the lossless decoding section 212 at a certain timing. Information
necessary for decoding of the prediction mode information or the like is added to the
enhancement layer encoded data. The flag indicating the HRD parameter type and
10 the HRD parameter are added to the enhancement layer encoded data as well.
[0231]
When there is a request from the HRD type decoding section 224, the
accumulation buffer 211 supplies information indicating a state of the accumulation
buffer 211. Further, for example, when there is an enhancement layer image
15 decoding section 201-3 of an upper layer as indicated by a dotted line, and there is a
request from its HRD type decoding section 224, the accumulation buffer 211
supplies the information indicating the state of the accumulation buffer 211.
[0232]
The lossless decoding section 212 decodes the information that has been
20 encoded by the lossless encoding section 116 and supplied from the accumulation
buffer 211 according to a scheme corresponding to the encoding scheme of the
lossless encoding section 116. The lossless decoding section 212 supplies
quantized coefficient data of a differential image obtained by the decoding to the
inverse quantization section 213.
25 [0233]
Further, the lossless decoding section 212 appropriately extracts and
acquires the NAL unit including the video parameter set (VPS), the sequence
parameter set (SPS), the picture parameter set (PPS), and the like which are included
in the enhancement layer encoded data. The lossless decoding section 212 extracts
30 the information related to the optimal prediction mode from the information,
determines which of the intra prediction mode and the inter prediction mode has been
/
SP351652WO00
51/133
selected as the optimal prediction mode based on the information, and supplies the
information related to the optimal prediction mode to one of the intra prediction
section 221 and the motion prediction/compensation section 222 that corresponds to
the mode determined to have been selected.
5 [0234]
In other words, for example, in the enhancement layer image decoding
section 201-2, when the intra prediction mode is selected as the optimal prediction
mode, the information related to the optima! prediction mode is supplied to the intra
prediction section 221. Further, for example, in the enhancement layer image
10 decoding section 201-2, when the inter prediction mode is selected as the optimal
prediction mode, the information related to the optimal prediction mode is supplied
to the motion prediction/compensation section 222.
[0235]
Further, the lossless decoding section 212 extracts information necessary for
15 inverse quantization such as the quantization matrix or the quantization parameter
from the NAL unit, and supplies the extracted information to the inverse quantization
section 213. Further, the lossless decoding section 212 extracts the flag indicating
the HRD parameter type and the HRD parameter, for example, from the VPS, and
supplies the extracted flag indicating the HRD parameter type and the HRD
20 parameter to the HRD type decoding section 224.
[0236]
The inverse quantization section 213 inversely quantizes the quantized
coefficient data obtained through the decoding performed by the lossless decoding
section 212 according to a scheme corresponding to the quantization scheme of the
25 quantization section 115. The inverse quantization section 213 is the same
processing section as the inverse quantization section 118. In other words, the
description of the inverse quantization section 213 can be applied to the inverse
quantization section 118 as well. Here, it is necessary to appropriately change and
read a data input/output destination or the like according to a device. The inverse
30 quantization section 213 supplies the obtained coefficient data to the inverse
orthogonal transform section 214.
52/133
SP351652WO00
[0237]
The inverse orthogonal transform section 214 performs the inverse
orthogonal transform on the coefficient data supplied from the inverse quantization
section 213 according to a scheme corresponding to the orthogonal transform scheme
5 of the orthogonal transform section 114. The inverse orthogonal transform section
214 is the same processing section as the inverse orthogonal transform section 119.
In other words, the description of the inverse orthogonal transform section 214 can
be applied to the inverse orthogonal transform section 119 as well. Here, it is
necessary to appropriately change and read a data input/output destination or the like
10 according to a device
[0238]
The inverse orthogonal transform section 214 obtains decoded residual data
corresponding to residual data that is not subjected to the orthogonal transform in the
orthogonal transform section 114 through the inverse orthogonal transform process.
15 The decoded residual data obtained through the inverse orthogonal transform is
supplied to the operation section 215. Further, the predictive image is supplied
from the intra prediction section 221 or the motion prediction/compensation section
222 to the operation section 215 via the selecting section 223.
[0239]
20 The operation section 215 adds the decoded residual data and the predictive
image, and obtains decoded image data corresponding to the image data from which
the predictive image is not subtracted by the operation section 113. The operation
section 215 supplies the decoded image data to the loop filter 216.
[0240]
25 The loop filter 216 appropriately performs the filter process such as the
deblock filter, the adaptive offset filter, or the adaptive loop filter on the supplied
decoded image, and supplies the resultant image to the screen reordering buffer 217
and the frame memory 219. For example, the loop filter 216 removes the block
distortion of the decoded image by performing the deblock filter process on the
30 decoded image. Further, for example, the loop filter 216 improves the image
quality by performing the loop filter process on the deblock filter process result (the
SP351652WO00
53/133
decoded image from which the block distortion has been removed) using the Wiener
filter. The loop filter 216 is the same processing section as the loop filter 121.
[0241]
Further, the decoded image output from the operation section 215 can be
5 supplied to the screen reordering buffer 217 or the flame memory 219 without
intervention of the loop filter 216. In other words, part or all of the filter process
performed by the loop filter 216 can be omitted.
[0242]
The screen reordering buffer 217 reorders the decoded image. In other
10 words, the order of the frames reordered in the encoding order by the screen
reordering buffer 112 is reordered in the original display order. The D/A converting
section 218 performs D/A conversion on the image supplied from the screen
reordering buffer 217, and outputs the converted image to be displayed on a display
(not illustrated).
15 [0243]
The frame memory 219 stores the supplied decoded image, and supplies the
stored decoded image to the selecting section 220 as the reference image at a certain
timing or based on an external request, for example, from the intra prediction section
221, the motion prediction/compensation section 222, or the like.
20 [0244]
The frame memory 219 sets the stored decoded image as information related
to decoding of the enhancement layer, and supplies the information to the
enhancement layer image decoding section 201-2 of an upper layer.
[0245]
25 The selecting section 220 selects the supply destination of the reference
image supplied from the frame memory 219. When an image encoded by the intra
coding is decoded, the selecting section 220 supplies the reference image supplied
from the frame memory 219 to the intra prediction section 221. Further, when an
image encoded by the inter coding is decoded, the selecting section 220 supplies the
30 reference image supplied from the frame memory 219 to the motion
prediction/compensation section 222.
SP351G52WO00
54/133
[0246]
For example, the information indicating the intra prediction mode obtained
by decoding the header information is appropriately supplied from the lossless
decoding section 212 to the intra prediction section 221. The intra prediction
5 section 221 generates the predictive image by performing the intra prediction using
the reference image acquired from the frame memory 219 in the intra prediction
mode used in the intra prediction section 124. The intra prediction section 221
supplies the generated predictive image to the selecting section 223.
[0247]
10 The motion prediction/compensation section 222 acquires information
(optimal prediction mode information, reference image information, and the like)
obtained by decoding the header information from the lossless decoding section 212.
[0248]
The motion prediction/compensation section 222 generates the predictive
15 image by performing the inter prediction using the reference image acquired from the
frame memory 219 in the inter prediction mode indicated by the optimal prediction
mode information acquired from the lossless decoding section 212. Although not
illustrated, in the motion prediction/compensation section 222, the reference image
supplied from the frame memory 219 of the base layer image decoding section 201-1
20 is also referred to as necessary.
[0249]
The selecting section 223 supplies the predictive image supplied from the
intra prediction section 221 or the predictive image supplied from the motion
prediction/compensation section 222 to the operation section 215. Then, the
25 operation section 215 adds the predictive image generated using the motion vector to
the decoded residual data (the differential image information) supplied from the
inverse orthogonal transform section 214 to decode the original image.
[0250]
The HRD type decoding section 224 acquires the information indicating the
30 accumulation state from the accumulation buffer 211 or the accumulation buffer (the
lower layer) 211 of the base layer image decoding section 201-1 according to the flag
SP351652WO00
55/133
indicating the HRD parameter type supplied from the lossless decoding section 212.
The HRD type decoding section 224 monitors the accumulation buffer 211 based on
the acquired information according to the HRD parameter corresponding to the flag
indicating the HRD parameter type.
5 [0251]

FIG. 22 is a block diagram illustrating an example of a configuration of the
accumulation buffer and the HRD type decoding section of FIG. 21.
10 [0252]
In an example of FIG. 22, the accumulation buffer 211 is configured to
include a partial accumulation buffer 231 and a whole accumulation buffer 232.
[0253]
The HRD type decoding section 224 is configured to include a layer HRD
15 parameter monitoring section 241, a time layer HRD parameter monitoring section
242, a HRD parameter type decoding section 243, and a time HRD parameter type
decoding section 244.
[0254]
The partial accumulation buffer 231 is configured with accumulation buffers
20 that accumulate encoded data related to each upper time layer among the encoded
data (codes) accumulated in the whole accumulation buffer 232. The information
indicating the state of each accumulation buffer is supplied to the time layer HRD
parameter monitoring section 242 on request.
[0255]
25 The whole accumulation buffer 232 accumulates the encoded data (codes)
encoded by the enhancement layer image encoding section 101-2. The information
indicating the state of the whole accumulation buffer of the whole accumulation
buffer 232 is supplied to the layer HRD parameter monitoring section 241 and the
time layer HRD parameter monitoring section 242 on request. Further, there are
30 cases in which there is an enhancement layer image decoding section 201-3 of an
upper layer as indicated by a dotted line. In this case, when there is a request from
SP351652WO00
56/133
the HRD type decoding section (upper layer) 224 of the enhancement layer image
decoding section 201-3, the information indicating the state of the whole
accumulation buffer of the whole accumulation buffer 232 is also supplied to the
HRD type decoding section (upper layer) 224.
5 [0256]
The layer HRD parameter monitoring section 241 receives the HRD
parameter supplied from the lossless decoding section 212, and acquires the HRD
parameter corresponding to the flag indicating the HRD parameter type supplied
from the HRD parameter type decoding section 243. The layer HRD parameter
10 monitoring section 241 monitors the accumulation buffer 211 based on the acquired
HRD parameter.
[0257]
In other words, the layer HRD parameter monitoring section 241 acquires
information indicating the state of the whole accumulation buffer 232 and
15 information indicating the state of the accumulation buffer (the lower layer) 211 of
the base layer image decoding section 201-1 according to the flag indicating the
HRD parameter type supplied from the HRD parameter type decoding section 243.
[0258]
In the case of the HRD parameter type in which the flag indicates 1, the
20 information indicating the state of the whole accumulation buffer 232 is acquired.
In the case of the HRD parameter type in which the flag indicates 0, the information
indicating the state of the whole accumulation buffer 232 and the information
indicating the state of the accumulation buffer (the lower layer) 211 of the base layer
image decoding section 201-1 are acquired. Practically, information is acquired
25 from the whole accumulation buffer 232 of the accumulation buffer of the base layer
image decoding section 201-1.
[0259]
The time layer HRD parameter monitoring section 242 receives the time
layer HRD parameter supplied from the lossless decoding section 212, and acquires
30 the time layer HRD parameter corresponding to the flag indicating the sub HRD
parameter type supplied from the time HRD parameter type decoding section 244.
57/133
SP351652WO00
The layer HRD parameter monitoring section 241 monitors the accumulation buffer
211 based on the acquired time layer HRD parameter.
[0260]
In other words, the time layer HRD parameter monitoring section 242
5 acquires the information indicating the state of the whole accumulation buffer 232
and information indicating the state of the accumulation buffer of the corresponding
time layer of the partial accumulation buffer 231 according to the flag indicating the
sub HRD parameter type supplied from the time HRD parameter type decoding
section 244.
10 [0261]
When the flag indicating the sub HRD parameter type is 1, the information
indicating the state of the accumulation buffer of the corresponding time layer of the
partial accumulation buffer 231 is acquired. When the flag indicating the sub HRD
parameter type is 0, the information indicating the state of the whole accumulation
15 buffer 232 and the information indicating the state of the accumulation buffer of the
corresponding time layer of the partial accumulation buffer 231 are acquired.
[0262]
The HRD parameter type decoding section 243 receives the HRD parameter
type flag supplied from the lossless decoding section 212. Then, the HRD
20 parameter type decoding section 243 selects a flag indicating an HRD parameter type
corresponding to a layer configuration of an actual stream or a configuration or a
function of a device among the received flags, and supplies the selected flag to the
layer HRD parameter monitoring section 241.
[0263]
25 The time HRD parameter type decoding section 244 receives the sub HRD
parameter type flag supplied from the lossless decoding section 212. Then, the time
HRD parameter type decoding section 244 selects a flag indicating a sub HRD
parameter type corresponding to a layer configuration of an actual stream or a
configuration or a function of a device among the received flags, and supplies the
30 selected flag to the layer HRD parameter monitoring section 241.
[0264]
SP351652WO00
58/133
In the case of the HRD parameter described above with reference to FIGS. 8
and 9, two types (a type of only a corresponding layer and a type of a corresponding
layer and a lower layer) are described, and thus any one type can be selected by a
configuration or a function of an actual stream or device. On the other hand, in the
5 case of the HRD parameter described above with reference to FIG. 7, only one type is
described, and thus the HRD parameter is ignored when a type of a configuration or
a function of an actual stream or device is different from a described type.
[0265]

10 Next, the flow of the process performed by the scalable decoding device 200
will be described. First, an example of the flow of the decoding process will be
described with reference to a flowchart of FIG. 23.
[0266]
When the decoding process starts, in step S20I, the decoding control section
15 202 of the scalable decoding device 200 decides a layer of a processing target, for
example, in view of the reference relation of an image.
[0267]
In step S202, the base layer image decoding section 201-1 performs a layer
decoding process under control of the decoding control section 202. The layer
20 decoding process will be described later with reference to FIG. 24. When the
process of step S202 ends, the process proceeds to step S203.
[0268]
In step S203, the decoding control section 202 determines whether or not all
the main layers have been processed. When it is determined that there is a non-
25 processed main layer, the process proceeds to step S204.
[0269]
In step S204, the decoding control section 202 sets a next non-processed
main layer as a processing target (current main layer). When the process of step
S204 ends, the process returns to step S202. In step S202, the enhancement layer
30 image decoding section 201-2 performs the layer decoding process under control of
the decoding control section 202. The process of steps S202 to S204 is repeatedly
SP351652WO00
59/133
performed to encode the main layers as described above. The process of step S202
may be processed in parallel by a plurality of layer image decoding sections 201
having no reference relation.
[0270]
5 Then, when all the main layers are determined to have been processed in
step S203, the decoding process ends.
[0271]

Next, an example of the flow of the layer decoding process performed in
10 step S202 of FIG. 23 will be described with reference to a flowchart of FIG. 24. An
example of FIG. 24 will be described in connection with an example of the
enhancement layer image decoding section 201-2.
[0272]
When the layer decoding process starts, in step S211, the accumulation
15 buffer 211 of the enhancement layer image decoding section 201-2 accumulates the
bitstreams of the enhancement layer transmitted from the encoding side.
[0273]
In step S212, the lossless decoding section 212 decodes the bitstream (the
encoded differentia! image information) of the enhancement layer supplied from the
20 accumulation buffer 211. In other words, the I picture, the P picture, and the B
picture encoded by the lossless encoding section 116 are decoded. At this time,
various kinds of information other than the differential image information included in
the bitstream, such as the header information, are also decoded. The flag indicating
the 1-IRD parameter type supplied from the lossless decoding section 212 and the
25 HRD parameter are supplied to the HRD type decoding section 224.
[0274]
In step S213, the HRD type decoding section 224 performs an HRD
parameter decoding process. The HRD parameter decoding process will be
described later with reference to FIG. 26.
30 [0275]
The HRD parameter is decoded in step S2I3, and the accumulation buffer
SP351652WO00
60/133
211 is monitored based on the decoded HRD parameter so that no overflow or
underflow occurs.
[0276]
In step S214, the inverse quantization section 213 inversely quantizes the
5 quantized coefficients obtained in the process of step S212.
[0277]
In step S2I5, the inverse orthogonal transform section 214 performs the
inverse orthogonal transform on a current block (a current TU).
[0278]
10 In step S216, the intra prediction section 221 or the motion
prediction/compensation section 222 perforins the prediction process, and generates
the predictive image. In other words, the prediction process is performed in the
prediction mode that is determined to have been applied at the time of encoding in
the lossless decoding section 212. More specifically, for example, when the intra
15 prediction is applied at the time of encoding, the intra prediction section 221
generates the predictive image in the intra prediction mode recognized to be optimal
at the time of encoding. Further, for example, when the inter prediction is applied
at the time of encoding, the motion prediction/compensation section 222 generates
the predictive image in the inter prediction mode recognized to be optimal at the time
20 ofencoding.
[0279]
In step S217, the operation section 215 adds the predictive image generated
in step S216 to the differential image information generated by the inverse
orthogonal transform process of step S215. As a result, the original image is
25 decoded,
[0280]
In step S218, the loop filter 216 appropriately performs the loop filter
process on the decoded image obtained in step S217.
[0281]
30 In step S219, the screen reordering buffer 217 reorders the image that has
been subjected to the filter process in step S218. In other words, the order of the
SP351652WO00
61/133
frames reordered for encoding through the screen reordering buffer 112 is reordered
in the original display order.
[0282]
In step S220, the D/A con veiling section 218 performs D/A conversion on
5 the image in which the order of the frames is reordered in step S219. The image is
output to a display (not illustrated), and the image is displayed.
[0283]
In step S221, the frame memory 219 stores the image that has been
subjected to the loop filter process in step S218.
10 [0284]
When the process of step S221 ends, the base layer decoding process ends,
and the process returns to FIG. 23.
[0285]

15 Next, an example of the flow of the HRD parameter decoding process
performed in step S213 of FIG. 24 will be described with reference to a flowchart of
FIG 25.
[0286]
The HRD parameter type decoding section 243 receives the flag indicating
20 the HRD parameter type of the corresponding layer in step S231. Then, the HRD
parameter type decoding section 243 supplies, for example, a flag indicating an HRD
parameter type corresponding to a layer configuration of an actual stream among the
received flags to the layer HRD parameter monitoring section 241.
[0287]
25 In step S232, the layer HRD parameter monitoring section 241 receives the
HRD parameter supplied from the lossless decoding section 212, and acquires the
HRD parameter corresponding to the flag indicating the HRD parameter type
supplied from the HRD parameter type decoding section 243.
[0288]
30 In step S233, the time HRD parameter type decoding section 244 receives
the flag indicating the HRD parameter type supplied from the lossless decoding
SP351652WO00
62/133
section 212. Then, the time HRD parameter type decoding section 244 supplies, for
example, a flag indicating a sub HRD parameter type corresponding to a layer
configuration of an actual stream among the received flags to the layer HRD
parameter monitoring section 241.
5 [0289]
In step S234, the time layer HRD parameter monitoring section 242 receives
the time layer HRD parameter supplied from the lossless decoding section 212, and
acquires the time layer HRD parameter corresponding to the sub HRD parameter
type flag supplied from the time HRD parameter type decoding section 244.
10 [0290]
In step S235, the time layer HRD parameter monitoring section 242
determines whether or not the process has ended on all the time layers. When the
process is determined not to have ended on any one of the time layers in step S235,
the process returns to step S233, and the subsequent process is repeated.
15 [0291]
When the process is determined to have ended on all the time layers in step
S235, the process proceeds to step S236. In step S236, the layer HRD parameter
monitoring section 241 and the time layer HRD parameter monitoring section 242
perform an accumulation buffer monitoring process which will be described below.
20 [0292]

Next, the accumulation buffer monitoring process will be described with
reference to a flowchait of FIG. 26. The accumulation buffer monitoring process is
an example using the HRD parameter type flag according to the present technology,
25 and the present technology is not limited to this example. For the sake of
description, a timing at which this process is performed is desci'ibed as being within
the HRD parameter decoding process, but the present technology is not limited to
this example, and this process may be performed, for example, at any timing within
the layer decoding process of FIG. 24.
30 [0293]
In step S251, the layer HRD parameter monitoring section 241 determines
SP351652WO00
63/133
whether or not the flag indicating the HRD parameter type of the corresponding layer
is 1. When the flag indicating the HRD parameter type of the corresponding layer
is determined to be 1 in step S251, the process proceeds to step S252.
[0294]
5 In step S252, the layer HRD parameter monitoring section 241 acquires the
information indicating the state of the whole accumulation buffer 232, and in step
S253, the layer HRD parameter monitoring section 241 monitors the whole
accumulation buffer 232 of the corresponding layer according to the HRD parameter
of the corresponding layer using the acquired information.
10 [0295]
When the flag indicating the HRD parameter type of the corresponding
layer is determined to be 0 in step S251, the process proceeds to step S254.
[0296]
In step S254, the layer HRD parameter monitoring section 241 acquires the
15 information indicating the state of the whole accumulation buffer 232, and in step
S255, the layer HRD parameter monitoring section 241 acquires the information
indicating the state of the whole accumulation buffer 232 of the lower layer.
[0297]
Then, in step S256, the layer HRD parameter monitoring section 241
20 monitors the whole accumulation buffer 232 of the corresponding layer using the
information acquired in steps S254 and S255 according to the HRD parameter of the
corresponding layer.
[0298]
In step S257, the time layer HRD parameter monitoring section 242
25 determines whether or not the flag indicating the HRD parameter type of the
corresponding time layer is 1. When the flag indicating the HRD parameter type of
the corresponding time layer is determined to be 1 in step S257, the process proceeds
to step S258.
[0299]
30 In step S258, the time layer HRD parameter monitoring section 242
acquires information indicating the state of the accumulation buffer of each
SP351652WO00
64/133
corresponding time layer of the partial accumulation buffer 231.
[0300]
In step S259, the time layer HRD parameter monitoring section 242
monitors (each buffer of) the partial accumulation buffer 231 using the information
5 acquired in step S258 according to the HRD parameter of each time layer.
[0301]
When the flag indicating the HRD parameter type of the corresponding time
layer is determined to be 0 in step S257, the process proceeds to step S260,
[0302]
10 In step S260, the time layer HRD parameter monitoring section 242
acquires the information indicating the state of the whole accumulation buffer 232.
[0303]
In step S261, the time layer HRD parameter monitoring section 242
acquires the information indicating the state of the accumulation buffer of each
15 corresponding time layer of the partial accumulation buffer 231.
[0304]
In step S259, the time layer HRD parameter monitoring section 242
monitors (each buffer of) the partial accumulation buffer 231 using the information
acquired in steps S260 and S261 according to the HRD parameter of each time layer.
20 [0305]
As described above, at least one flag is set to indicate whether the HRD
parameter is the parameter for performing decoding of only a corresponding layer or
the parameter for performing the decoding process of a corresponding layer and a
lower layer, and thus it is possible to perform a decoding process at a proper timing.
25 [0306]
This flag may be transmitted to the decoding side as supplemental
enhancement information (SEI).
[0307]
Here, it is difficult to detect whether the decoding process is performed by a
30 single decoding device or a plurality of decoding devices as described above with
reference to FIG. 6. In this regard, the example of setting the information indicating
SP351652WO00
65/133
whether the HRD parameter is the parameter for performing the decoding process of
only a corresponding layer or the parameter for performing the decoding process of a
corresponding layer and a lower layer has been described above.
[0308]
5 However, the current HEVC supports only the HRD parameter of the
example (that is, the example in which the decoding process of the multiple layers is
performed by a single decoding device) of exl 1 illustrated in FIG. 6. In other words,
in the current HEVC, the HRD parameter of the example of ex 11 is set to the video
parameter set (VPS) and transmitted to the decoding side.
10 [0309]
In this regard, in the present technology, a technique of transmitting an HRD
parameter of the example (that is, the example in which the decoding process of the
multiple layers is performed by a plurality of decoding devices) of exl2 illustrated in
FIG. 6 to the decoding side through vps_extension in the scalable HEVC is proposed
15 as a third embodiment.
[0310]
<3. Third embodiment

FIG. 27 is a diagram illustrating an example of syntax of vps_ extension.
20 In an example of FIG. 27, i indicates the number of layersets, and j indicates the
number of layer ids. Further, starting from j=l indicates that there is only a base
layer when j=0, and in this case, the processing methods of exl 1 and exl2 are not
different but all the same.
[0311]
25 In an example of FIG. 27, layer seMird layer info present_flag[i][j] is set
for each layer_id _jncliided_flag[i]|j]. When
layer_setjird_iayer_info_present_Jlag[i][j] is 1, it indicates that the HRD parameter
corresponding to the example of ex 12 is present (in vps_ extension), and in this case,
the HRD parameter corresponding to the example of exl2 is defined in next and
30 subsequent rows.
[0312]
SP351652WO00
66/133
Further, the HRD parameter corresponding to the example of exl2 may be
set in sps (sequence parameter set) extension as illustrated in FIG. 28.
[0313]

5 FIG. 28 is a diagram illustrating an example of syntax of sps_ extension.
[0314]
In an example of FIG. 28, layer set_hrd_layer_info_present_flag is set.
When this flag is 1, it indicates that the HRD parameter corresponding to the
example of exl2 is present (in sps_ extension), and in this case, the HRD parameter
10 corresponding to the example of exl2 is defined in the next row.
[0315]
In the example of FIGS. 27 and 28, the HRD parameter is set for each
Iayer_set. layerset is set in the VPS as illustrated in FIG. 29.
[0316]
15
FIG. 29 is a diagram illustrating an example of syntax of a VPS. Numbers
on the left of each row are row numbers added for description.
[0317]
In a 16th row of FIG. 29, the number of layer_sets is set as
20 vpsnuni Jayersets jninus 1.
[0318]
In 17th to 19th rows, it is described whether or not a layer of idfj] is
included in layerset of [i] as layer_idjncluded_flag[i][j].
layer id included flag[i][j] will be described in detail in semantics illustrated in FIG.
25 30.
[0319]
In a 27th row, the number of hrd parameters is set in a vps as
vpsnumlird_paratneters. In a 29th row, hrdparameters is associated with
layerset as hrd_layer_set_idx[i]. In a 32nd row, the HRD parameter of the
30 example of exl 1 is described as described above.
[0320]
SP351652WO00
67/133
In a 35th row, vps_extension_flag indicating the presence or absence of
vps extension is described.
[0321]

5 FIG. 30 illustrates an example of semantics of layer_id_included „flag[i][j].
[0322]
layer_id included_flag[ i ][ j ] equal to I specifies that the value of
nuh_Jayer_id equal to j is included in the layer identifier list layerSetLayerldListfi ].
Iayer_id_included_flag[ i ][ j ] equal to 0 specifies that the value of nuh_layer_id
10 equal to j is not included in the layer identifier list layerSetLayerIdList[ i ].
The value of numLayersInldListf 0 ] is set equal to 1 and the value of
layerSetLayerIdList[ 0 ][ 0 ] is set equal to 0.
15 For each value of i in the range of 1 to vps_nuni_layer_sets_minusl,
inclusive, the variable numLayersInIdList[ i ] and the layer identifier list
layerSetLayerldListf i ] are derived as follows:
n = 0
for (m = 0; m <= vps_maxjayerjd; m++)
20 if (layer_id_included_flag[ i ][ in ])
layerSetLayerldListf i ][ n++ ] = in
numLayersInIdList[ i ] = n
For eaeh value of i in the range of 1 to vpsnumlayersetsniimtsl,
inclusive, numLayersInIdList[ i ] shall be in the range of 1 to
25 vps_max iayersniinusl + I, inclusive.
When numLayersInldListf iA ] is equal to nuniLayersInIdList[ iB ] for any
iA and iB in the range of 0 to vps num Jayersetsminusl, inclusive, with iA not
equal to iB, the value of layerSetLayerldList[ iA ][ n ] shall not be equal to
layerSetLayerIdList[ iB ][ n ] for at least one value of n in the range of 0 to
30 numLayersInldListf iA ], inclusive.
A layer set is identified by the associated layer identifier list. The i-th layer
68/133
SP351652WO00
set specified by the VPS is associated with the layer identifier list
layerSetLayerldListf i ], for i in the range of 0 to vps_num_layer_sets_minusl,
inclusive.
A layer set consists of all operation points that are associated with the same
5 layer identifier list.
Each operation point is identified by the associated layer identifier list,
denoted as OpLayerldList, which consists of the list of nuh_layer_id values of all
NAL units included in the operation point, in increasing order of nuhjayerjd values,
and a variable OpTid, which is equal to the highest Temporalld of all NAL units
10 included in the operation point. The bitstream subset associated with the operation
point identified by OpLayerldList and OpTid is the output of the sub-bitstream
extraction process as specified in clause 10 with the bitstream, the target highest
Temporalld equal to OpTid, and the target layer identifier list equal to OpLayerldList
as inputs. The OpLayerldList and OpTid that identify an operation point are also
15 referred to as the OpLayerldList and OpTid associated with the operation point,
respectively.
[0323]
Particularly, as surrounded by a frame in FIG. 30, it is described that
layerset consists of all operation points that are associated with the same layer
20 identifier list, and some bitstreams that can be extracted among bitstreams are
associated with operation points identified by OpIayerldList.
[03241
Specifically, layerset is set, for example, as illustrated in FIG. 31.
[0325]
25 For example, when layers 0, 1, and 2 are included in LayerSetfl], and the
layers 0 and 2 are included in LayerSet[2], layerid included_flag is set as follows.
[0326]
layer_id_included_flag[l][0]=l is set, which indicates that the layer 0 is
included in LayerSetfl]. Further, layer_id_Jncluded_flag[I][l]=l is set, which
30 indicates that the layer 1 is included in LayerSet[l], and
layerJd_Jncluded_flag[l][2]=I is set, which indicates that the layer 2 is included in
SP351652WO00
69/133
LayerSetp].
[0327]
layer_id_in eluded„flag[2][0]=l is set, which indicates that the layer 0 is
included in LayerSet[2]. Further, layer_idjncluded_flag[2][l]=0 is set, which
5 indicates that the layer I is not included in LayerSetp], and
layer_id included_flag[2][2]=l is set, which indicates that the layer 2 is included in
LayerSet[2].
[0328]
As described above, the flag indicating that there is the HRD parameter
10 corresponding to the example of ex 11 of FIG. 6 and the HRD parameter
corresponding to the example of ex II are transmitted to the decoding side through
the vps. On the other hand, the flag indicating that there is the HRD parameter
corresponding to the example of exl2 of FIG 6 and the HRD parameter
corresponding to the example of exl2 are transmitted to the decoding side through
15 vps_extension. Thus, the decoding side can perform a decoding process at a proper
timing.
[0329]
Further, the flag indicating that there is the HRD parameter corresponding to
the example of exl2 may also be transmitted to the decoding side as an SEI message.
20 [0330]
<4. Fourth embodiment

Meanwhile, in order to prevent overflow or underflow of a buffer, it is
necessary to apply any one of the following methods to a buffer schedule
25 management.
[0331]
A first method is a method using the parameter transmitted in the HRD
parameter syntax in the first to third embodiments.
[0332]
30 A second method is a method using bufferingperiod SEI and
picture_timing_SEI.
70/133
SP351652WO00
[0333]
A third method is a method using a parameter transmitted in a layer higher
than a video layer such as a time stamp (for example, a PTS or a DTS) in a system
layer,
5 [0334]
Of these methods, the method using buffering_period_SEI according to the
second method will be described below.
[0335]

10 FIG. 32 is a diagram illustrating an example of syntax of
buffering_period_SEI. Numbers on the left of each row are row numbers added for
description.
[0336]
In a 2nd row of FIG. 32, a parameter of buffering_period is set as
15 bp_seq_parameter set id, and can be associated with an SPS.
[0337]
In subsequent rows, a schedule management parameter of an accumulation
buffer by accumulation from a lower layer is set.
[0338]
20 Meanwhile, a technique of defining a parameter for the case of exl2 for
performing a decoding process of decoding respective layers including a layer to be
referred to through separate decoding devices in addition to the case of exll of
performing a decoding process by a single decoding device as a parameter for buffer
management by hdr_parametcrs() in image compression information or sub image
25 compression information as described above with reference to FIG. 6 has been
proposed.
[0339]
hdr_parameters() according to the first method can be transmitted in
association with a VPS or an SPS, and in the case of the former, hdr parameters()
30 can be transmitted for a layer set associated with a VPS as well as a single layer.
[0340]
SP351652WO00
71/133
However, buffering_period SEI illustrated in FIG, 32 can be associated with
only an SPS. Thus, it is difficult to transmit a parameter associated with a plurality
of layer sets as in hdr_parameters() in a VPS.
[0341]
5 In this regard, in the present technology, a schedule management of an
accumulation buffer by buffering_period_SEI is performed by syntax illustrated in
FIGS. 33 to 35.
[0342]
buffering period SET according to the present technology differs from
10 buffering_period_SEI illustrated in FIG 32 in the following points.
[0343]
In other words, a first difference lies in that buffering_period_SEI according
to the present technology can be associated with a VPS as well as an SPS. A
second difference lies in that a parameter can be set for each layer set defined in a
15 VPS when associated with a VPS. A third difference lies in that it is possible to set
a parameter when all layers included in a layer set are decoded by a single decoding
device and a parameter when respective layers are decoded by separate decoding
devices as illustrated in FIG. 6.
[0344]
20 FIGS. 33 to 35 are diagrams illustrating an example of syntax of
buffering period_SEI. Numbers on the left of each row are row numbers added for
description.
[0345]
In an example of FIG 33, associatedparametcr setflag is set in a 2nd row.
25 associated_parameter_set_flag is a flag designating which of a VPS and an SPS is
associated with bufferingperiodSEI. When associated_parameter_set_flag is 0, it
indicates an association with a VPS, and when associated_parameter_set_flag is 1, it
indicates an association with an SPS.
[0346]
30 A parameter when associatedparameter setflag is 0 (VPS) is described in
3rd to 11th rows. bpvideoparameterseMd in a 4th row indicates a
SP351652WO00
72/133
corresponding VPS, and vps_num_bp_parameters in a 5th row indicates the number
of transmitted bp__parameters.
[0347]
bp_layer set_idx in a 7th row indicates a layer set corresponding to each
5 bp_parameter, and syntax of layer_buffering_period of FIGS. 34 and 35 is read with
layer buffer!ng_period in a 10th row.
[0348]
A parameter when assodated_parameter_set_jflag is 1 (SPS) is described as
indicated by else in 12th to 15th rows.
10 [0349]
bp_seq_parameter_set_jd in a 13th row indicates a corresponding SPS, and
syntax of layer_bu ffer in gperiod of FIGS. 34 and 35 is read with
layer_bufTering_period in a 14th row.
[0350]
15 In other words, according to the syntax of FIG. 33, in the case of the VPS,
the syntax of FIG. 34 is read by the number of layer sets, and in the case of the SPS,
the syntax of FIGS. 34 and 35 is read once.
[0351]
As described above, when associatedparameterset flag is 0 (VPS), it is
20 possible to transmit the parameter for buffer management for the layer set designated
by the VPS according to I aye rbuffe ring period SEI syntax of FIGS. 34 and 35.
[0352]
In the transmission of the parameter related to the NAL and the VOL of
FIGS. 34 and 35, layer_specific_parameters_presentflag serving as a flag indicating
25 whether or not a parameter of only a corresponding layer is transmitted is set as
illustrated in 18th and 38th rows. When this flag is 1, it indicates that a parameter
of only a corresponding layer described in 19th to 27th rows and 39th to 47th rows is
transmitted. In other words, when layer specific_parameters_present flag is 1, it is
possible to transmit a parameter for performing a decoding process as in ex 12 in
30 addition to exll of FIG. 6.
[0353]
SP351652WO00
73/133
In the example of FIGS. 34 and 35, buffering_period_SEI is basically the
same as buffering__period_SEI described above with reference to FIG. 32 except for
the above-described points, and in the other rows, a schedule management parameter
of an accumulation buffer by accumulation from a lower layer is set.
5 [0354]
Thus, when schedule management of an accumulation buffer by
buffering_period SEI is performed, it is possible to manage a layer set as well as a
single layer. Further, it is possible to perform a schedule management both when all
layers included in a layer set are decoded by a single decoding device and when
10 respective layers are decoded by separate decoding devices.
[0355]

A scalable encoding device in the case of buffer! ng__period_S EI has
basically the same configuration as in the case of the HRD parameter. Thus, an
15 example of a configuration of a scalable encoding device in the case of
buffering periodSEI will be described with reference to FIG 10.
[0356]
In other words, as in the case of the HRD parameter, the base layer image
encoding section 101-1 acquires image information (base layer image information)
20 of the base layer. The base layer image encoding section 101-1 encodes the base
layer image information without referring to other layers, generates encoded data
(base layer encoded data) of the base layer, and outputs the generated encoded data.
[0357]
As in the case of the HRD parameter, the enhancement layer image
25 encoding section 101-2 acquires image information (enhancement layer image
information) of the enhancement layer. The enhancement layer image encoding
section 101-2 encodes the enhancement layer image information. At this time, the
enhancement layer image encoding section 101-2 performs inter-layer prediction
with reference to information related to encoding of the base layer as necessary.
30 [0358]
Further, buffer management information is supplied from the base layer
74/133
SP351652WO00
image encoding section 101-1 to the enhancement layer image encoding section 101-
2 as necessary. Unlike the case of the HRD parameter, the enhancement layer
image encoding section 101-2 designates an associated parameter set, and designates
a parameter of each layer set or a parameter for a sequence according to the
5 designated parameter set. Then, the enhancement layer image encoding section
101-2 sets the designated parameter with reference to the buffer management
information supplied from the base layer image encoding section 101-1.
[0359]
Further, unlike the case of the HRD parameter, the enhancement layer image
10 encoding section 101-2 sets layer_specific_parameters_present flag, and sets a
parameter of each layer according to a value of
layer spedfic_parameters_present flag. Then, the enhancement layer image
encoding section 101-2 encodes buifering_period SEI and
layer_buffering_period_SEl including the set parameters, and supplies the encoded
15 information to a lossless encoding section 301.
[0360]
The enhancement layer image encoding section 101-2 generates encoded
data (enhancement layer encoded data) of the enhancement layer through the above
encoding, and outputs the generated encoded data.
20 [0361]
As in the case of the HRD parameter, the encoding control section 102
controls the encoding process of the layer image encoding sections 101, for example,
in view of the reference relation of the layer image encoding sections 101.
[0362]
25
FIG. 36 is a block diagram illustrating an example of a main configuration
of the enhancement layer image encoding section 101-2 in the case of
bufferingperiodSEI.
[0363]
30 Further, the base layer image encoding section 101-1 in the case of
buifering_period_SEI has basically the same configuration as the enhancement layer
SP351652WO00
75/133
image encoding section 101-2 of FTG. 36 except that a type of an image serving as a
target is different. For the sake of description, in the example of FIG. 36, a
configuration of the enhancement layer image encoding section 101-2 will be
described as an example.
5 [0364]
The enhancement layer image encoding section 101-2 of FIG. 36 differs
from the enhancement layer image encoding section 101-2 of FIG. 11 in that the
lossless encoding section 116 is replaced with the lossless encoding section 301, the
HRD type setting section 128 is not provided, and a buffering period SEI setting
10 section 302 is added.
[0365]
In other words, the enhancement layer image encoding section 101-2 of FIG
36 includes an A/D converting section 111, a screen reordering buffer 112, an
operation section 113, an orthogonal transform section 114, a quantization section
15 115, a lossless encoding section 301, an accumulation buffer 117, an inverse
quantization section 118, and an inverse orthogonal transform section 119. The
enhancement layer image encoding section 101-2 further includes an operation
section 120, a loop filter 121, a frame memory 122, a selecting section 123, an intra
prediction section 124, a motion prediction/compensation section 125, a predictive
20 image selecting section 126, and a rate control section 127. The enhancement layer
image encoding section 101-2 further includes the buffering period SEI setting
section 302.
[0366]
Like the lossless encoding section 116 of FIG. 11, the lossless encoding
25 section 301 encodes the transform coefficients quantized in the quantization section
115 according to an arbitrary encoding scheme. Since coefficient data is quantized
under control of the rate control section 127, the coding amount becomes a target
value (or approaches a target value) set by the rate control section 127.
[0367]
30 Like the lossless encoding section 116, the lossless encoding section 301
acquires information indicating an intra prediction mode or the like from the intra
76/133
SP351652WO00
prediction section 124, and acquires information indicating an inter prediction mode,
differential motion vector information, or the like from the motion
prediction/compensation section 125.
[0368]
5 Further, unlike the lossless encoding section 116, the lossless encoding
section 301 acquires encoded information (syntax) of layer_buffering^period_SEI
from the buffering period SEI setting section 302 in addition to
buffering_period_SEI.
[0369]
10 Further, like the lossless encoding section 116, the lossless encoding section
301 appropriately generates an NAL unit of the enhancement layer including a
sequence parameter set (SPS), a picture parameter set (PPS), and the like. Like the
lossless encoding section 116, the lossless encoding section 301 encodes various
kinds of information according to an arbitrary encoding scheme, and sets
15 (multiplexes) the encoded information as part of encoded data (also referred to as an
"encoded stream"). The lossless encoding section 301 supplies the encoded data
obtained by the encoding to be accumulated in the accumulation buffer 117.
[0370]
Further, unlike the lossless encoding section 116, for example, when there is
20 an enhancement layer image encoding section 101-3 of an upper layer as indicated
by a dotted line, and there is a request from its buffering period SEI setting section
302, the lossless encoding section 301 supplies management information of the
accumulation buffer 117 to the enhancement layer image encoding section 101-3 of
the upper layer.
25 [0371]
The accumulation buffer 117 temporarily holds the encoded data
(enhancement layer encoded data) supplied from the lossless encoding section 301.
The accumulation buffer 117 outputs the held enhancement layer encoded data to a
recording device (recording medium), a transmission path, or the like (not illustrated)
30 at a subsequent stage at a certain timing. In other words, the accumulation buffer
117 serves as a transmitting section that transmits the encoded data as well.
SP351652WO00
77/133
[0372]
The buffering period SEI setting section 302 designates an associated
parameter set, and designates a parameter of each layer set or a parameter for a
sequence according to the designated parameter set. Further, the buffering period
5 SEI setting section 302 sets the designated parameter with reference to the buffer
management information supplied from the lossless encoding section 301 (the lower
layer) of the base layer image encoding section 101-1. For example, this parameter
is a parameter of accumulation from a lower layer. Further, the buffering period
SEI setting section 302 sets layer specific_parameters_present flag, and sets a
10 parameter of each layer according to the set value of
layer_specific_parameters_present_flag.
[0373]
The buffering period SEI setting section 302 encodes buffer!ng_period SEI
and Iayer_buffering_period SEI including the parameters set as described above, and
15 supplies the encoded information to the lossless encoding section 301.
[0374]

FIG. 37 is a block diagram illustrating an example of a configuration of the
buffering period SEI setting section of FIG. 36.
20 [0375]
In an example of FIG. 37, the buffering period SEI setting section 302 is
configured to include an associated parameter set setting section 311, a layer set
buffer 312, a layer buffering period SEI setting section 313, and a layer parameter
transmission designating section 314.
25 [0376]
The associated parameter set setting section 311 performs a designation
related to associsted parameterset flag according to the user's operation. In other
words, the user designates which of a VPS (flag-0) and an SPS (flag^l) is associated
with bufferingperiodSEI. The associated parameter set setting section 311
30 designates a value of associsted_parameter_set flag, and supplies the designated
value to the lossless encoding section 301 and the layer buffering period SEI setting
SP351652WO00
78/133
section 313.
[0377]
In the case of associsted_parameter_set flag=0, that is, when
buffering_period_SEI is associated with a VPS, information related to a layer set
5 stored in a VPS of enhancement layer image compression information is supplied
and accumulated in the layer set buffer 312 through the lossless encoding section 301.
The layer set buffer 312 accumulates the information related to the layer set, and
supplies the information related to the layer set to the layer buffering period SEI
setting section 313 at a certain timing.
10 [0378]
Further, the buffer management information in the base layer is supplied
from the lossless encoding section 301 of the base layer image encoding section 101-
1 to the layer buffering period SEI setting section 313.
[0379]
15 The layer parameter transmission designating section 314 designates
layer_specific_parameters_present flag according to the user's operation. In other
words, the user designates a value of layer_specific_parameters present flag
indicating whether or not a parameter setting of each layer is performed. The layer
parameter transmission designating section 314 designates a value of
20 layer_specific_parameters_present_flag, and supplies the designated value to the
lossless encoding section 301 and the layer buffering period SEI setting section 313.
[0380]
The layer buffering period SEI setting section 313 performs the encoding
process of buffering_period_SEl and layer_buffering_period_SEI, and supplies
25 encoded information thereof to the lossless encoding section 30 i.
[0381]
Specifically, the layer buffering period SEI setting section 313 sets the
designated parameter according to associsted_parameter_set_flag supplied from the
associated parameter set setting section 3 i 1 with reference to the buffer management
30 information supplied from the lossless encoding section 301 (the lower layer) of the
base layer image encoding section 101-1. In other words, when a value of

Claim 1
An image processing device comprising:
a receiving section configured to receive a bitstream obtained by encoding
5 an image having at least one layer and buffer management parameter information of
each layer indicating at least one of that a parameter for managing a decoder buffer is
a parameter for performing a decoding process of only a corresponding layer and that
the parameter for managing the decoder buffer is a parameter for performing a
decoding process of a corresponding layer and a lower layer; and
10 a decoding section configured to decode the bitstream received by the
receiving section and generate an image.
Claim 2
The image processing device according to claim 1,
15 wherein the layer includes a layer and a sublayer.
Claim 3
The image processing device according to claim 2,
wherein the layer is a view of multi-view coding.
20
Claim 4
The image processing device according to claim 2,
wherein the layer is a layer of scalable video coding.
25 Claim 5
The image processing device according to claim 1,
wherein the buffer management parameter information is described in
supplemental enhancement information (SEI).
30 Claim 6
The image processing device according to claim 5,
10
SP351652WO00
130/133
wherein the buffer management parameter information is described in
buffer in g_period_SEI.
Claim 7
The image processing device according to claim 1,
wherein parameter presence/absence information indicating a presence or
absence of the parameter for managing the decoder buffer serving as the parameter
for performing the decoding process of only the corresponding layer is described in a
vps (video parameter set)_extension.
Claim 8
The image processing device according to claim 1,
wherein the receiving section receives an AVC flag indicating that a layer
lower than the corresponding layer is encoded by MPEG-4 Parti 0 Advanced Video
15 Coding (AVC) and the buffer management parameter information of each layer
indicating that the parameter for managing the decoder buffer is the parameter for
performing the decoding process of only the corresponding layer.
Claim 9
20 An image processing method comprising:
receiving, by an image processing device, a bitstream obtained by encoding
an image having at least one layer and buffer management parameter information of
each layer indicating at least one of that a parameter for managing a decoder buffer is
a parameter for performing a decoding process of only a corresponding layer and that
25 the parameter for managing the decotier buffer is a parameter for performing a
decoding process of a corresponding layer and a lower layer; and
decoding, by the image processing device, the received bitstream and
generating an image.
30 Claim 10
An image processing device comprising:
131/133
SP351652WO00
a setting section configured to set buffer management parameter information
of each layer indicating at least one of that a parameter for managing a decoder
buffer is a parameter for performing a decoding process of only a corresponding
layer and that the parameter for managing the decoder buffer is a parameter for
5 performing a decoding process of a corresponding layer and a lower layer;
an encoding section configured to encode an image having at least one layer
and generate a bitstream; and
a transmitting section configured to transmit the buffer management
parameter information set by the setting section and the bitstream generated by the
10 encoding section.
Claim 11
The image processing device according to claim 10,
wherein the layer includes a layer and a sublayer.
15
Claim 12
The image processing device according to claim 11,
wherein the layer is a view of multi-view coding.
20 Claim 13
The image processing device according to claim 11,
wherein the layer is a layer of scalable video coding.
Claim 14
25 The image processing device according to claim 10,
wherein the buffer management parameter information is described in
supplemental enhancement information (SEI).
Claim 15
30 The image processing device according to claim 14,
wherein the buffer management parameter information is described in
132/133
SP351652WO00
buffering_period_SEI.
Claim 16
The image processing device according to.claim 10,
wherein parameter presence/absence information indicating a presence or
absence of the parameter for managing the decoder buffer serving as the parameter
for performing the decoding process of only the corresponding layer is described in a
vps (video parameter set)_extension.
Claim 17
The image processing device according to claim 10,
wherein the setting section sets an AVC flag indicating that a layer lower
than the corresponding layer is encoded by MPEG-4 PartlO Advanced Video Coding
(AVC) and the buffer management parameter information of each layer indicating
that the parameter for managing the decoder buffer is the parameter for performing
the decoding process of only the corresponding layer.
Claim 18 ,'. '•
An image processing method comprising:
setting, by an image processing device, buffer management parameter
information of each layer indicating at least one of that a parameter for managing a
decoder buffer is a parameter for performing a decoding process of only a
corresponding layer and that the parameter for managing the decoder buffer is a
parameter for performing a decoding process of a corresponding layer and a lower
layer;
encoding, by the image processing deviee, an image having at least one
layer and generating a bitstream; and
transmitting, by the image processing device, the set buffer management
parameter information and the generated bitstream.