IMAGE PROCESSING DEVICE AND METHOD
TECHNICAL FIELD
5 [0001]
The present disclosure relates to an image
processing device and an image processing method, and
more particularly, to an image processing device and an
image processing method capable of improving the coding
10 efficiency in multi-viewpoint coding.
BACKGROUND ART
[0002]
In recent years, image information is handled as
15 digital data, and, for the purpose of transmission and
accumulation of the information having high-efficiency at
that time, devices are widely used which compress and
code images by employing a coding system compressing
image information through an orthogonal transform such as
20 a discrete cosine transform and motion compression by
using the redundancy that is unique to the image
information. As examples of such a coding system, there
are MPEG (Moving Picture Experts Group), H.264, MPEG-4
Part 10 (Advanced Video Coding, hereinafter, referred to
25 as AVC), and the like.
[0003]
Currently, for the purpose of further improving the
coding efficiency to be higher than that of H.264/AVC,
standardization of a coding system called high efficiency
30 video coding (HEVC) has been progressed by a joint
collaboration team - video coding (JCTVC) that is a joint
2
SP351692WO00
standardization organization of the ITU-T and the ISO/IEC.
In addition, as an extension thereof, standardization of
coding of a multi-viewpoint image has been progressed in
parallel therewith, and as a draft for coding a multi-
5 viewpoint image, Non-Patent Document 1 has been issued.
[0004]
Meanwhile, a reference list that is a list in which
candidates for a reference image used for decoding a B
picture are aligned in a preferential order is
10 sequentially generated from the start of reference images
aligned in a preferential order in a temporal list that
is a temporary list.
[0005]
In Non-Patent Document 1, the temporal list is
15 generated such that, after reference images disposed in
the time direction, reference images disposed in the view
direction are arranged. Thus, in order to raise the
preferential order of the reference images disposed in
the view direction within the temporal list, unless a
2 0 reference list command is used, it is difficult to
arrange the reference images disposed in the view
direction in the reference list, and accordingly, it is
difficult to refer to the reference images disposed in
the view direction.
25 [0006]
Thus, in Non-Patent Document 2, it is proposed to
transmit information designating a position at which a
reference image disposed in the view direction is
inserted among a plurality of reference images disposed
30 in the time direction using a slice header when a
temporal list is generated. Accordingly, in
3
SP351692WO00
consideration of a view-direction correlation, a
reference image disposed in the view direction can be
inserted at a preferable position in the temporal list.
5 CITATION LIST
NON-PATENT DOCUMENT
[0007]
Non-Patent Document 1: Gerhard Tech, Krzysztof Wegner,
Ying Chen, Miska Hannuksela, "MV-HEVC Draft Text2",
10 JCT3V-B1004, 2012,11,07
Non-Patent Document 2: Li Zhang, Ying Chen, Jewon Kang,
"AHG5: MV-Hevc software for HTM", JCT3V-B0046, 2012,10,07
SUMMARY OF THE INVENTION
15 PROBLEMS TO BE SOLVED BY THE INVENTION
[0008]
However, in the technology disclosed in Non-Patent
Document 2, also v/hen it is not necessary to change the
insertion position of the reference image disposed in the
20 view direction in the temporal list, the slice header
needs to be transmitted, and there is concern that the
coding efficiency may decrease.
[0009]
The present disclosure is in view of such a
25 situation and is capable of improving the coding
efficiency in hierarchical coding or multi-viewpoint
coding.
SOLUTIONS TO PROBLEMS
30 [0010]
According to an aspect of the present disclosure,
4
SP351692WO00
there is provided an image processing device including: a
decoding unit that generates an image by decoding a bit
stream; a reference list setting unit that sets a
reference list being referred to when the image generated
5 by the decoding unit is predicted by inserting a
reference image that can be referred to in a layer
direction between a plurality of reference images that
can be referred to in a time direction; and a prediction
unit that predicts the image generated by the decoding
10 unit by referring to the reference list set by the
reference list setting unit.
[0011]
The above-described reference list setting unit may
set the reference list by inserting the reference image
15 that can be referred to in the layer direction between a
reference image located before the image in time and a
reference image located after the image in time among the
plurality of reference images that can be referred to in
the time direction.
20 [0012]
The above-described reference list setting unit may
set the reference list by arranging images in a direction
L0 in order of reference images located before the image
in time, the reference image that can be referred to in
25 the layer direction, and reference images located after
the image in time.
[0013]
The above-described reference list setting unit may
set the reference list by arranging images in a direction
30 LI in order of reference images located after the image
in time, the reference image that can be referred to in
5
SP351692WO00
the layer direction, and reference images located before
the image in time.
[0014]
The above-described reference list setting unit may
5 set the reference list by alternately arranging the
reference image that can be referred to in the time
direction and the reference image that can be referred to
in the layer direction.
[0015]
10 The above-described reference list setting unit may
set the reference list by alternately arranging an
element of a list of the reference images that can be
referred to in the time direction set in order of the
reference images located before the image in time and the
15 reference images located after the image in time and an
element of a list of the reference images that can be
referred to in the layer direction, which is configured
by the reference images that can be referred to in the
layer direction, in a direction L0.
20 [0016]
The above-described reference list setting unit may
set the reference list by alternately arranging an
element of a list of the reference images that can be
referred to in the time direction set in order of the
25 reference images located after the image in time and the
reference images located before the image in time and an
element of a list of the reference images that can be
referred to in the layer direction, which is configured
by the reference images that can be referred to in the
30 layer direction, in a direction LI.
[0017]
/
/
6
SP351692WO00
The above-described reference list setting unit mayset
the reference list by inserting the reference images
that can be referred to in the layer direction in a
direction LI with order of the reference image that can
5 be referred to in the layer direction being opposite to
that of the case of a direction LO.
[0018]
The above-described reference list setting unit may
include: a temporal list setting unit that sets a
10 temporal list that is a temporary list used for setting
the reference list by inserting the reference image that
can be referred to in the layer direction between the
plurality of reference images that can be referred to in
the time direction; and a reference image list setting
15 unit that sets the reference list based on the temporal
list set by the temporal list setting unit.
[0019]
According to an aspect of the present disclosure,
there is provided an image processing method implemented
20 in an image processing device. The image processing
method includes: generating an image by decoding a bit
stream; setting a reference list being referred to when
the generated image is predicted by inserting a reference
image that can be referred to in a layer direction
25 between a plurality of reference images that can be
referred to in a time direction; and predicting the
generated image by referring to the set reference list.
[0020]
According to another aspect of the present
30 disclosure, there is provided an image processing device
including: a reference list setting unit that sets a
7
SP351692WO00
reference list being referred to when an image is
predicted by inserting a reference image that can be
referred to in a layer direction between a plurality of
reference images that can be referred to in a time
5 direction; a prediction unit that predicts the image by
referring to the reference list set by the reference list
setting unit; and an encoding unit that generates a bit
stream by performing coding using the image predicted by
the prediction unit.
10 [0021]
The above-described reference list setting unit may
set the reference list by inserting the reference image
that can be referred to in the layer direction between a
reference image located before the image in time and a
15 reference image located after the image in time among the
plurality of reference images that can be referred to in
the time direction.
[0022]
The above-described reference list setting unit may
20 set the reference list by arranging images in a direction
L0 in order of reference images located before the image
in time, the reference image that can be referred to in
the layer direction, and reference images located after
the image in time.
25 [0023]
The above-described reference list setting unit may
set the reference list by arranging images in a direction
LI in order of reference images located after the image
in time, the reference image that can be referred to in
30 the layer direction, and reference images located before
the image in time.
8
SP351692WO00
[0024]
The above-described reference list setting unit may
set the reference list by alternately arranging the
reference image that can be referred to in the time
5 direction and the reference image that can be referred to
in the layer direction.
[0025]
The above-described reference list setting unit may
set the reference list by alternately arranging an
10 element of a list of the reference images that can be
referred to in the time direction set in order of the
reference images located before the image in time and the
reference images located after the image in time and an
element of a list of the reference images that can be
15 referred to in the layer direction, v/hich is configured
by the reference images that can be referred to in the
layer direction, in a direction L0.
[0026]
The above-described reference list setting unit may
20 set the reference list by alternately arranging an
element of a list of the reference images that can be
referred to in the time direction set in order of the
reference images located after the image in time and the
reference images located before the image in time and an
25 element of a list of the reference images that can be
referred to in the layer direction, v/hich is configured
by the reference images that can be referred to in the
layer direction, in a direction Ll.
[0027]
30 The above-described reference list setting unit may
set the reference list by inserting the reference images
9
SP351692WO00
that can be referred to in the layer direction in a
direction LI with order of the reference image that can
be referred to in the layer direction being opposite to
that of the case of a direction LO.
5 [0028]
The above-described reference list setting unit may
include; a temporal list setting unit that sets a
temporal list that is a temporary list used for setting
the reference list by inserting the reference image that
10 can be referred to in the layer direction between the
plurality of reference images that can be referred to in
the time direction; and a reference image list setting
unit that sets the reference list based on the temporal
list set by the temporal list setting unit.
15 [0029]
According to another aspect of the present
disclosure, there is provided an image processing method
implemented in an image processing device. The image
processing method includes: setting a reference list
20 being referred to v/hen an image is predicted by inserting
a reference image that can be referred to in a layer
direction betv/een a plurality of reference images that
can be referred to in a time direction; predicting the
image by referring to the set reference list; and
25 generating a bit stream by performing coding using the
predicted image.
[0030]
According to an aspect of the present disclosure,
an image is generated by decoding a bit stream, and a
30 reference list being referred to v/hen the image generated
is predicted is set by inserting a reference image that
/
10
SP351692WO00
can be referred to in a layer direction between a
plurality of reference images that can be referred to in
a time direction. Then, the generated image is predicted
by referring to the set reference list.
5 [0031]
According to another aspect of the present
disclosure, a reference list being referred to when an
image is predicted is set by inserting a reference image
that can be referred to in a layer direction between a
10 plurality of reference images that can be referred to in
a time direction. Then, the image is predicted by
referring to the set reference list, and a bit stream is
generated by performing coding using the predicted image.
[0032]
15 Here, the above-described image processing device
may be an independent device or an internal block that
configures one image encoding device or image decoding
device.
20 EFFECTS OF THE INVENTION
[0033]
According to an aspect of the present disclosure,
an image can be decoded. Particularly, the coding
efficiency can be improved.
25 [0034]
According to another aspect of the present
disclosure, an image can be coded. Particularly, the
coding efficiency can be improved.
30 BRIEF DESCRIPTION OF DRAWINGS
[0035]
/
/
11
SP351692WO00
Fig. 1 is a block diagram that illustrates an
example of the main configuration of an image encoding
device.
Fig. 2 is an explanatory diagram that illustrates
5 an example of the structure of reference images.
Fig. 3 is a diagram that illustrates an example of
a conventional method of generating a reference list.
Fig. 4 is a diagram that illustrates another
example of a conventional method of generating a
10 reference list.
Fig. 5 is a diagram that illustrates an example of
a method of generating a reference list according to the
present technology.
Fig. 6 is a diagram that illustrates another
15 example of a method of generating a reference list
according to the present technology.
Fig. 7 is a diagram that illustrates a comparison
between the present technology and a conventional
technology.
20 Fig. 8 is a diagram that illustrates the
arrangement order of inter-view images according to the
present technology.
Fig. 9 is a diagram that illustrates a method of
changing the arrangement order of inter-view images
25 according to the present technology.
Fig. 10 is a block diagram that illustrates an
example of the configuration of a reference list
generating unit illustrated in Fig. 1.
Fig. 11 is a flowchart that illustrates an example
30 of the flow of a coding process.
Fig. 12 is a flowchart that illustrates an example
12
SP351692WO00
of the flow of a reference list generating process.
Fig. 13 is a flowchart that illustrates another
example of the flow of the reference list generating
process.
5 Fig. 14 is a block diagram that illustrates an
example of the main configuration of an image decoding
device.
Fig. 15 is a block diagram that illustrates an
example of the configuration of a reference list
10 generating unit illustrated in Fig. 14.
Fig. 16 is a flowchart that illustrates an example
of the process of a decoding process.
Fig. 17 is a flowchart that illustrates an example
of the flow of a reference list generating process.
15 Fig. 18 is a flowchart that illustrates another
example of the flow of the reference list generating
process.
Fig. 19 is a block diagram that illustrates another
example of the main configuration of an image encoding
2 0 device.
Fig. 20 is a block diagram that illustrates another
example of the main configuration of an image decoding
device.
Fig. 21 is a block diagram that illustrates an
25 example of the main configuration of a computer.
Fig. 22 is a block diagram that illustrates an
example of the schematic configuration of a television
apparatus.
Fig. 23 is a block diagram that illustrates an
30 example of the schematic configuration of a mobile phone.
Fig. 24 is a block diagram that illustrates an
13
SP351692WO00
example of the schematic configuration of a
recording/reproducing apparatus.
Fig. 25 is a block diagram that illustrates an
example of the schematic configuration of an imaging
5 apparatus.
Fig. 26 is a block diagram that illustrates an
example of the schematic configuration of a video set.
Fig. 27 is a block diagram that illustrates an
example of the schematic configuration of a video
10 processor.
Fig. 28 is a block diagram that illustrates another
example of the schematic configuration of the video
processor.
15 MODE FOR CARRYING OUT THE INVENTION
[0036]
Hereinafter, embodiments for executing the present
disclosure (hereinafter, referred to as embodiments) will
be described. The description will be presented in the
20 following order.
1. First Embodiment (Image Encoding Device)
2. Second Embodiment (Image Decoding Device)
3. Third Embodiment (Image Encoding Device and
Image Decoding Device of an AVC System)
25 4. Fourth Embodiment (Computer)
5. Application Example
6. Fifth Embodiment (Set/Unit/Module/Processor)
[0037]

30 [Description of Hierarchical Coding]
The image coding systems such as MPEG2 and AVC
14
SP351692WO00
described until now have a scalability function.
Scalable coding (hierarchical coding) is a system in
which an image is configured by multiple layers
(hierarchizing), and coding is performed for each layer.
5 [0038]
In hierarchizing an image, one image is divided
into a plurality of images (layers) by referring to a
predetermined parameter. Basically, each layer is
configured by differential data so as to reduce the
10 redundancy. For example, in a case where one image is
hierarchized into two layers including a base layer and
an enhancement layer, an image having a quality lower
than the original image is acquired by using only the
data of the base layer, and the original image (in other
15 words, a high-quality image) is acquired by composing the
data of the base layer and the data of the enhancement
layer.
[0039]
By hierarchizing the image in this way, images of
20 various qualities can be easily acquired according to the
situation. For example, as in a case where, to a
terminal having a low processing capability such as a
mobile phone, image compression information of only a
base layer is transmitted, and a moving image having low
25 spatial/temporal resolution or a low image quality is
reproduced, and, to a terminal having a high processing
capability such as a television set or a personal
computer, image compression information of an enhancement
layer in addition to the base layer is transmitted, and a
30 moving image having high spatial/temporal resolution or a
high image quality is reproduced, the image compression
15
SP351692WO00
information according to the capability of the terminal
or the network can be transmitted from a server without
performing trans-code processing.
[0040]
5 As a parameter enabling such scalability, for
example, there is spatial resolution (spatial
scalability). In the case of this spatial scalability,
the resolution is different for each layer. In other
words, each picture is hierarchized into two hierarchies
10 of the base layer having spatial resolution lower than
the original image and the enhancement layer that can be
used for acquiring the original image (original spatial
resolution) by being composed with the image of the base
layer. It is apparent that the number of hierarchies is
15 an example, and the picture can be hierarchized into an
arbitrary number of hierarchies.
[0041]
As another parameter having such scalability, for
example, there is temporal resolution (temporal
20 scalability). In the case of this temporal scalability,
the frame rate is different for each layer. In other
words, in this case, since an image is hierarchized into
layers having mutually-different frame rates, by adding a
layer of a high frame rate to a layer of a low frame rate,
25 a moving image having a higher frame rate can be acquired,
and, by adding all the layers, the original moving image
(the original frame rate) can be acquired. Here, the
number of hierarchies is an example, and the image can be
hierarchized for an arbitrary number of hierarchies.
30 [0042]
In addition, as another parameter enabling such
16
SP351692WO00
scalability, for example, a signal to noise ratio (SNR)
may be applied (SNR scalability) . In the case of this
SNR scalability, the SN ratio is different for each layer.
In other words, each picture is hierarchized into two
5 hierarchies including a base layer having an SNR lower
than the original image and an enhancement layer capable
of acquiring the original image (original SNR) by being
composed with an image of the base layer. In other words,
in the base layer image compression information,
10 information relating to an image having a low PSNR is
transmitted, and, by adding enhancement layer image
compression information thereto, an image having a high
PSNR can be rebuilt. It is apparent that the number of
hierarchies is an example, and the image can be
15 hierarchized for an arbitrary number of hierarchies.
[0043]
It is apparent that a parameter having the
scalability may be other than those of the examples
described above. For example, there is bit-depth
20 scalability in which a base layer is configured by an 8-
bit image, and, by adding an enhancement layer thereto,
an image of 10 bits is acquired.
[0044]
In addition, there is chroma scalability in which a
25 base layer is configured by a component image of the
4:2:0 format, and a component image of the 4:2:2 format
is acquired by adding an enhancement layer thereto.
[0045]
Furthermore, as a parameter enabling the
30 scalability, there is a multiview. In this case, each
picture is hierarchized into layers of mutually-different
•I
17
SP351692WO00
views (viewpoints) .
[0046]
The layer described in this embodiment includes the
spatial/ the temporal SNR, the bit depth, the color, the
5 view, and the like of the scalability coding described
above.
[0047]
The term "layer" used in this specification
includes each layer of the scalable (hierarchical) coding
10 described above and each view at the time of considering
multiviews of the multi-viewpoints.
[0048]
Hereinafter, while the view will be described as an
example, by substituting the view with another layer, the
15 present technology can be similarly applied to the other
layers described above.
[0049]
[Configuration Example of Image Encoding Device]
Fig. 1 illustrates of the configuration of an image
2 0 encoding device according to an embodiment as an image
processing device to which the present disclosure is
applied.
[0050]
An image encoding device 100 illustrated in Fig. 1
25 codes image data by using a prediction process. Here, as
the coding system, for example, the HEVC system or the
like is used. In other words, the image encoding device
100 performs the process in units of CUs (coding units).
[0051]
30 In the example illustrated in Fig. 1, the image
encoding device 100 includes: an A/D (analog/digital)
18
SP351692WO00
converter 101; a screen rearrangement buffer 102; a
calculation unit 103; an orthogonal transform unit 104; a
quantization unit 105; a lossless encoding unit 106; and
an accumulation buffer 107. In addition, the image
5 encoding device 100 includes: an inverse quantization
unit 108; an inverse orthogonal transform unit 109; a
calculation unit 110; a deblocking filter 111; a decoded
picture buffer 112; a selection unit 113; an intra
prediction unit 114; a motion parallax
10 prediction/compensation unit 115; a selection unit 116;
and a rate control unit 117.
[0052]
Furthermore, the image encoding device 100
includes: a syntax processing unit 121; a reference list
15 generating unit 122; an adaptable offset filter 123; and
an adaptive loop filter 124.
[0053]
The A/D converter 101 performs A/D conversion of
input image data and outputs resultant image data to the
20 screen rearrangement buffer 102 so as to be stored
therein.
[0054]
The screen rearrangement buffer 102 rearranges
stored images, which are in display order of frames, in
25 order of frames for coding in accordance with a group of
picture (GOP). The screen rearrangement buffer 102
supplies the image of which the order of frames has been
rearranged to the syntax processing unit 121.
[0055]
30 The syntax processing unit 121 sequentially checks
the data of an image read from the screen rearrangement
19
SP351692WO00
buffer 102 and inserts header information, which is
supplied from a previous stage not illustrated in the
figure, into the data of the image. The header
information, for example, includes a video parameter set
5 (VPS), a sequence parameter set (SPS), a picture
parameter set (PPS) , and the like. In addition, the
syntax processing unit 121 adds a slice header (SH) to
the start of a slice.
[0056]
10 The syntax processing unit 121 supplies the image
into which the header information and the like are
inserted to the calculation unit 103, the intra
prediction unit 114, and the motion parallax
prediction/compensation unit 115. In addition, the
15 syntax processing unit 121 supplies each header
information and information such as the slice header to
the reference list generating unit 122.
[0057]
The calculation unit 103 subtracts a predicted
20 image supplied from the intra prediction unit 114 or the
motion parallax prediction/compensation unit 115 from an
image supplied from the syntax processing unit 121
through the selection unit 116 and outputs differential
information thereof to the orthogonal transform unit 104.
25 [0058]
For example, in the case of an image for which
intra coding is performed, the calculation unit 103
subtracts the predicted image supplied from the intra
prediction unit 114 from the image supplied from the
30 syntax processing unit 121. On the other hand, for
example, in the case of an image for which inter coding
20
SP351692WO00
is performed, the calculation unit 103 subtracts the
predicted image supplied from the motion parallax
prediction/compensation unit 115 from the image supplied
from the syntax processing unit 121.
5 [0059]
The orthogonal transform unit 104 performs an
orthogonal transform such as a discrete cosine transform
or a Karhunen-Loeve transform for the differential
information supplied from the calculation unit 103, and
10 supplies a transform coefficient thereof to the
quantization unit 105.
[0060]
The quantization unit 105 performs quantization of
the transform coefficient the orthogonal transform unit
15 104 outputs. The quantization unit 105 supplies the
quantized transform coefficient to the lossless encoding
unit 106.
[0061]
The lossless encoding unit 106 performs lossless
20 coding such as variable-length coding or arithmetic
coding for the quantized transform coefficient.
[0062]
The lossless encoding unit 106 acquires information
representing an intra prediction mode and the like from
25 the intra prediction unit 114 and acquires information
representing an inter prediction mode, motion vector
information, and the like from the motion parallax
prediction/compensation unit 115.
[0063]
3 0 The lossless encoding unit 10 6 codes the quantized
transform coefficient and sets the intra prediction mode
21
SP351692WO00
information, the inter prediction mode information, the
motion vector information, and the quantization parameter
as a part of the header information of the coded data
(multiplexing). The lossless encoding unit 106 supplies
5 the coded data acquired by the coding process to the
accumulation buffer 107 so as to be accumulated therein.
[0064]
For example, the lossless encoding unit 10 6
performs a lossless coding process such as a variable-
10 length coding process or an arithmetic coding process.
As the variable-length coding, for example, there is a
context-adaptive variable length coding (CAVLC) or the
like. As the arithmetic coding, for example, there is
context-adaptive binary arithmetic coding (CABAC) or the
15 like.
[0065]
The accumulation buffer 107 temporarily stores the
coded data supplied from the lossless encoding unit 106
and outputs the stored coded data at predetermined timing,
20 for example, to an image decoding device disposed on a
later stage, a recording device, a transmission line, and
the like not illustrated in the figure as a coded image
that has been coded.
[0066]
25 In addition, the transform coefficient quantized by
the quantization unit 105 is supplied also to the inverse
quantization unit 108. The inverse quantization unit 108
performs inverse quantization of the quantized transform
coefficient by using a method corresponding to the
30 quantization process performed by the quantization unit
105. The inverse quantization unit 108 supplies the
22
SP351692WO00
acquired transform coefficient to the inverse orthogonal
transform unit 109.
[0067]
The inverse orthogonal transform unit 109 performs
5 an inverse orthogonal transform of the supplied transform
coefficient by using a method corresponding to the
orthogonal transform process performed by the orthogonal
transform unit 104. An output {the differential
information that is restored) for which the inverse
10 orthogonal transform has been performed is supplied to
the calculation unit 110.
[0068]
The calculation unit 110 adds the predicted image
supplied from the intra prediction unit 114 or the motion
15 parallax prediction/compensation unit 115 through the
selection unit 116 to a result of the inverse orthogonal
transform that is supplied from the inverse orthogonal
transform unit 109, in other words, the restored
differential information, thereby acquiring a locally-
20 decoded image (decoded image).
[0069]
For example, in a case where the differential
information corresponds to an image for which the intra
coding process is performed, the calculation unit 110
25 adds the predicted image supplied from the intra
prediction unit 114 to the differential information. In
addition, for example, in a case where the differential
information corresponds to an image for which the inter
coding process is performed, the calculation unit 110
30 adds the predicted image supplied from the motion
parallax prediction/compensation unit 115 to the
23
SP351692WO00
differential information.
[0070]
The result of the addition is supplied to the
deblocking filter 111 and the decoded picture buffer 112.
5 [0071]
The deblocking filter 111 appropriately performs a
deblocking filter process, thereby eliminating a block
distortion of the decoded image. The deblocking filter
111 supplies a result of the filter process to the
10 adaptive offset filter 123.
[0072]
The adaptive offset filter 123 performs an adaptive
offset filter (S7AO: Sample adaptive offset) process,
which mainly eliminates ringing, for the image after the
15 deblocking filter process performed by the deblocking
filter 111.
[0073]
Described in more detail, the adaptive offset
filter 123 determines the type of an adaptive offset
20 filter process for each largest coding unit (LCU) that is
a maximal encoding unit and acquires an offset used in
the adaptive offset filter process. The adaptive offset
filter 123 performs the adaptive offset filter process of
the determined type for the image after the adaptive
25 deblocking filter process by using the acquired offset.
Then, the adaptive offset filter 123 supplies the image
after the adaptive offset filter process to the adaptive
loop filter 124.
[0074]
30 In addition, the adaptive offset filter 123
includes a butter in which the offset is stored. The
24
SP351692WO00
adaptive offset filter 12 3 determines whether or not the
offset used in the adaptive deblocking filter process has
already been stored in the buffer for each LCU.
[0075]
5 In a case where it is determined that the offset
used in the adaptive deblocking filter process has
already been stored in the buffer, the adaptive offset
filter 123 sets a storage flag, which represents whether
the offset is stored in the buffer, to a value {here,
10 "1") representing that the offset is stored in the buffer.
[0076]
Then, the adaptive offset filter 123 supplies the
storage flag set to "1", an index that represents a
storage position of the offset in the buffer, and type
15 information that represents the type of the performed
adaptive offset filter process to the lossless encoding
unit 106 for each LCU.
[0077]
On the other hand, in a case where it is determined
20 that the offset used in the adaptive deblocking filter
process has not been stored yet in the buffer, the
adaptive offset filter 123 sequentially stores the offset
in the buffer. In addition, the adaptive offset filter
123 sets the storage flag to a value (here, "0")
25 representing that the offset is not stored in the buffer.
Then, the adaptive offset filter 123 supplies the storage
flag set to "0", the offset, and the type information to
the lossless encoding unit 106 for each LCU.
[0078]
30 The adaptive loop filter 124 performs an adaptive
loop filter (ALF) process for the image after the
25
SP351692WO00
adaptive offset filter process, which is supplied from
the adaptive offset filter 123, for example, for each LCD.
As the adaptive loop filter process, for example, a
process using a two-dimensional Weiner filter is used.
5 It is apparent that a filter other than the Wiener filter
may be used.
[0079]
More specifically, the adaptive loop filter 124,
for each LCD", calculates a filter coefficient used in the
10 adaptive loop filter process such that a residual between
the original image that is the image output from the
syntax processing unit 121 and the image after the
adaptive loop filter process is minimal. Then, the
adaptive loop filter 124 performs an adaptive loop filter
15 process by using the calculated filter coefficient for
the image after the adaptive offset filter process for
each LCU.
[0080]
The adaptive loop filter 124 supplies the image
20 after the adaptive loop filter process to the decoded
picture buffer 112. In addition, the adaptive loop
filter 12 4 supplies the filter coefficient to the
lossless encoding unit 106.
[0081]
25 Here, the adaptive loop filter process is performed
for each LCU, the processing unit of the adaptive loop
filter process is not limited to the LCU. However, by
matching the processing units of the adaptive offset
filter 123 and the adaptive loop filter 124, the process
30 can be efficiently performed,
[0082]
26
SP351692WO00
The decoded picture buffer 112 outputs the
accumulated reference image at predetermined timing to
the intra prediction unit 114 or the motion parallax
prediction/compensation unit 115 through the selection
5 unit 113.
[0083]
For example, in the case of an image for which the
intra coding is performed, the decoded picture buffer 112
supplies the reference image to the intra prediction unit
10 114 through the selection unit 113. On the other hand,
for example, in a case where the inter coding is
performed, the decoded picture buffer 112 supplies the
reference image to the motion parallax
prediction/compensation unit 115 through the selection
15 unit 113.
[0084]
In a case where the reference image supplied from
the decoded picture buffer 112 is an image for which the
intra coding is performed, the selection unit 113
2 0 supplies the reference image to the intra prediction unit
114. On the other hand, in a case where the reference
image supplied from the decoded picture buffer 112 is an
image for which the inter coding is performed, the
selection unit 113 supplies the reference image to the
25 motion parallax prediction/compensation unit 115.
[0085]
The intra prediction unit 114 performs an intra
prediction (in-screen prediction) for generating a
predicted image by using an in-screen pixel value of an
30 input image supplied from the syntax processing unit 121.
The intra prediction unit 114 performs the intra
V
27
SP351692WO00
prediction in a plurality of modes (intra prediction
modes).
[0086]
The intra prediction unit 114 generates predicted
5 images in all the intra prediction modes, evaluates each
predicted image, and selects an optimal mode. When the
optimal intra prediction mode is selected, the intra
prediction unit 114 supplies the predicted image
generated in the optical mode to the calculation unit 103
10 or the calculation unit 110 through the selection unit
116.
[0087]
In addition, as described above, the intra
prediction unit 114 appropriately supplies information
15 such as intra prediction mode information representing
the used intra prediction mode to the lossless encoding
unit 106.
[0088]
The motion parallax prediction/compensation unit
20 115 performs a motion parallax prediction for the image
for which the inter coding is performed by using the
input image supplied from the syntax processing unit 121
and the reference image supplied from the decoded picture
buffer 112 through the selection unit 113. At that time,
25 the motion parallax prediction/compensation unit 115 uses
a reference image of a reference image index that is
represented in a reference list generated by the
reference list generating unit 122. The motion parallax
prediction/compensation unit 115 performs a motion
30 parallax compensation process according to a detected
motion parallax vector, thereby generating a predicted
28
SP351692WO00
image (inter predicted image information).
[0089]
The motion parallax prediction/compensation unit
115 performs the inter prediction process of each of all
5 the inter prediction modes that are candidates, thereby
generating predicted images. The motion parallax
prediction/compensation unit 115 supplies the generated
predicted image to the calculation unit 103 or the
calculation unit 110 through the selection unit 116.
10 [0090]
In addition, the motion parallax
prediction/compensation unit 115 supplies inter
prediction mode information representing the used inter
prediction mode and motion vector information
15 representing the calculated motion vector to the lossless
encoding unit 106.
[0091]
The selection unit 116 supplies the output of the
intra prediction unit 114 to the calculation unit 103 or
20 the calculation unit 110 in the case of an image for
which the intra coding is performed and supplies the
output of the motion parallax prediction/compensation
unit 115 to the calculation unit 103 or the calculation
unit 110 in the case of an image for which the inter
25 coding is performed.
[0092]
The rate control unit 117 controls the rate of the
quantization operation of the quantization unit 105 based
on a compressed image stored in the accumulation buffer
30 107 such that overflow or under flow does not occur.
[0093]
29
SP351692WO00
The reference list generating unit 122 generates a
reference list that is referred to when the motion
parallax prediction/compensation unit 115 predicts a
current image by using information (POC information and
5 view information) of a reference image accumulated in the
decoded picture buffer 112. At that time, the reference
list generating unit 122 generates the reference list by
inserting reference images that can be referred to in the
view direction between a plurality of reference images
10 that can be referred to in the time direction.
[0094]
In other words, the reference list generating unit
122 generates a temporal list that is a temporary list
used when the reference list is generated by inserting
15 reference images that can be referred to in the view
direction betv/een a plurality of reference images that
can be referred to in the time direction. Then, the
reference list generating unit 122 generates a reference
list based on the generated temporal list.
20 [0095]
Here, the reference images that can be referred to
in the time direction are reference images that can be
referred to at different time points, and the reference
images that can be referred to in the view (layer)
25 direction are reference images that can be referred to at
different views (layers).
[0096]
[Structure of Reference Image]
Next, the structure of the reference image
30 according to the HEVC system will be described with
reference to Fig. 2.
30
SP351692WO00
[0097]
In the example illustrated in Fig. 2, in order of
POC from the left side, long-term (LT) reference images,
short-term (before) reference images (Short-term before
5 Curr: STbC) , a current image in v/hich MC" is represented,
and short-term (after) reference images (Short-term after
Curr: STaC) are illustrated. In addition, in order of
the view (parallax) direction from the lower side, the
current image in v/hich "C" is represented, and inter-view
10 (IV) reference images are illustrated. In addition, a
number represented in each reference image represents an
arrangement index of each reference image in the decoded
picture buffer 112.
[0098]
15 In the HEVC system, when an inter prediction is
performed, four kinds of images including the long-term
reference images, the short-term (before) reference
images, the short-term (after) reference images, and the
inter-view reference images illustrated in Fig. 2 can be
20 set as candidates for a reference image.
[0099J
The short-term (before) reference images are shortterm
reference images located before the current image
(POC is smaller), in v/hich "C" is represented, in time
25 within the same view, and the short-term (after)
reference images are short-term reference images located
after the current image (POC is larger) in time within
the same view. In addition, the long-term reference
images are reference images designated as reference
30 images of a long term within the same view. The interview
reference images are reference images of different
31
SP351692WO00
views at the same time.
[0100]
[Conventional Method 1 of Generating Reference List]
Next, a method of generating a default reference
5 list of the HEVC system will be described as a
conventional method 1 with reference to Fig. 3. In an
example illustrated in Fig. 3, as the example in which
four reference images are referred to, a list
(arrangement) (RefPicSetStCurrBefore[i]) of short-term
10 (before) reference images, a list
(RefPicSetStCurrAfter[i]) of short-term (after) reference
images, a list (RefPicSetLtCurr[i]) of long-term
reference images, and a list (RefPicSetlvCurr[i]) (i:
index = 0 to 3) of inter-view reference images in the
15 decoded picture buffer 112 are illustrated.
[0101]
For the short-term reference images, in the syntax
of the slice header or the RPS (Reference Picture Set), a
flag used_by_curr is set to each image having a
20 possibility of being referred to. The RPS is a syntax
used for clearly indicating the state of the decoded
picture buffer for each picture included in the SPS. For
the long-term reference images, in the syntax of the
slice header or the SPS, the flag used__by_curr is set to
25 each image having a possibility of being referred to.
Only each image of which the flag used_by__curr is ul" is
added to the temporal list that is a temporary list used
when the reference list is generated. In addition, since
the flag used_by_curr is not set to inter-view reference
30 images, all the inter-view reference images arranged in
the list of inter-view reference images are added to the
/
32
SP351692WO00
temporal list.
[0102]
In the example represented in Fig. 3, a temporal
list (RefPicListTempO[ridx]) of LO (direction LO) and a
5 temporal list (RefPicListTempl[ridx]) of LI (direction
Ll) that are generated from such lists of reference
images are illustrated.
[0103]
The temporal list (RefPicListTempO[ridx]) of LO is
10 generated by adding images of which the flag used__by_curr
is set to "1" in order of a short-term (before) reference
image/ a short-term (after) reference image, a long-term
reference image, and an inter-view reference image.
Accordingly, the temporal list (RefPicListTempO[ridx]) of
15 L0 is generated in order of short-term (before) reference
images of indexes "0" and "1", short-term (after)
reference images of indexes "0" and "1", a long-term
reference image of an index "0", and inter-view reference
images of indexes "0" to "3".
20 [0104]
The temporal list (RefPicListTempl[ridx]) of Ll is
generated by adding images of which the flag used__by_curr
is set to ul" in order of a short-term (after) reference
image, a short-term (before) reference image, a long-term
25 reference image, and an inter-view reference image.
Accordingly, the temporal list (RefPicListTempl[ridx]} of
Ll is generated in order of short-term (after) reference
images of indexes "0" and wl", short-term (before)
reference images of indexes "0" and "1", a long-term
30 reference image of index "0", and inter-view reference
images of indexes xv0" to "3".
/
33
SP351692WO00
[0105]
In addition, in the example represented in Fig. 3,
a reference list (RefPicListO[rldx]) of LO that is
generated based on the temporal list of LO and a
5 reference list (RefPicListl[rldx]) of LI that is
generated based on the temporal list of LI are
illustrated.
[0106]
Here, as the syntax of the slice header or the SPS,
10 in the temporal list, as the number of referable images,
num_re f_idx_l0_active_minusl and
num_ref_idx__ll_active_minusl are set. From the start of
the temporal list of L0/L1, the number of reference
images is limited thereby, and only the reference images
15 are added to the reference list and can be used as the
reference images.
[0107]
For example, in a case where
num_ref_idx_10_active_minusl is u4", images of the start
20 {the first from the left side) to the fifth of the
temporal list of L0 are added to a reference list
(RefPicListO[rldx]) of L0, whereby the reference list of
L0 is generated. In the case of the example illustrated
in Fig. 3, the reference list of L0 is generated in order
25 of short-term (before) reference images of indexes "0'"'
and "1", short-term (after) reference images of indexes
"0" and "1", and a long-term reference image of index "0"
[0108]
For example, in a case where
30 num_ref_idx_ll_active__minusl is "3", images of the start
to the fourth of the temporal list of Ll are added to a
34
SP351692WO00
reference list (RefPicListl[rldx]) of LI, whereby the
reference list of Ll is generated. In the case of the
example illustrated in Fig. 3, the reference list of Ll
is generated in order of a short-term (after) reference
5 images of indexes "0" and "1" and short-term (before)
reference images of indexes "0" and "1".
[0109]
As above, according to the HEVC system, the
temporal list is generated such that, after reference
10 images disposed in the time direction, reference images
disposed in the view direction are arranged. Thus, it is
difficult to arrange the reference images disposed in the
view direction in the reference list.
[0110]
15 In addition, according to the HEVC system, a change
command for a reference list is defined, and, by
transmitting the change command, images that are freely
rearranged in the temporal list can be inserted into the
reference list. Accordingly, a reference image disposed
20 in the view direction can be referred to. However, since
the change command is transmitted in the syntax, an
additional bit is generated.
[0111]
In other words, in the case of the method described
25 above, while there is no problem when the time
correlation is extremely higher than the view correlation,
in a general sequence, the correlation among views is not
markedly low, and thus, it is necessary to use a
reference list command for performing a parallax
30 prediction.
[0112]
35
SP351692WO00
Thus, in Non-Patent Document 2, it has been
proposed to transmit information designating a position
at which a reference image disposed in the view direction
is to be inserted among a plurality of reference images
5 disposed in the time direction using a slice header when
a temporal list is generated.
[0113]
[Conventional Method 2 of Generating Reference List]
Next, a method of generating a default reference
10 list described in Non-Patent Document 2 will be described
as a conventional method 2 with reference to Fig. 4. In
Fig. 4f only an example of L0 is described.
[0114]
According to the method described in Non-Patent
15 Document 2, until the temporal list is generated, the
process is similar to that of the HEVC system, and
repetitive description thereof will not be presented here.
[0115]
In other words, the temporal list
20 (RefPicListTempO[rIdx]) of L0, similar to the HEVC system,
is generated by adding images of which the flag
used_by_curr is set to "1" in order of a short-term
(before) reference image, a short-term (after) reference
image, a long-term reference image, and an inter-view
25 reference image. Accordingly, the temporal list
(RefPicListTempO[rldx]) of L0 is generated in order of
short-term (before) reference images of indexes "0" and
"1", short-term (after) reference images of indexes "0"
and "1", a long-term reference image of an index "0", and
30 inter-view reference images of indexes "0" to "3".
[0116]
36
SP351692WO00
Here, in the case of the method described in Non-
Patent Document 2, a position
(inter_view_ref_start_position) at which the inter-view
reference image is inserted in the temporal list is set
5 in a slice header extension and is transmitted,
[0117]
For example, according to the position
inter__view_ref_start_position, as denoted by arrow PI, a
second position from the start (first) of the temporal
10 list is designated. In such a case, the temporal list
{RefPicListTempO[rldx]) of L0 is generated in order of a
short-term (before) reference image of index "0", interview
reference images of indexes "0" to "3", a short-term
(before) reference image of index wl", short-term (after)
15 reference images of indexes "0" and "1", and a long-term
reference image of index "0".
[0118]
In such a case, the reference list of L0 is
generated according to num_ref_idx_10_active_minusl = 4
20 in order of a short-term (before) reference image of
index H 0 " and inter-view reference images of indexes "0"
to "3".
[0119]
In addition, for example, according to the position
25 inter_view_ref_start_position, as denoted by arrow P2, a
third position from the start (first) of the temporal
list is designated. In such a case, the temporal list
(RefPicListTempO[rldx]) of L0 is generated in order of
short-term (before) reference images of indexes "0" and
30 "1", inter-view reference images of indexes "0" to "3",
short-term (after) reference images of indexes "0" and
37
SP351692WO00
^1", and a long-term reference image of index "0".
[0120]
In such a case, the reference list of L0 is
generated according to num_ref_idx_10_active_minusl — A
5 in order of short-term (before) reference images of
indexes w 0 " and "1" and inter-view reference images of
indexes u 0 " to "2".
[0121]
In addition, for example, according to the position
10 inter_view_jref_start_jposition, as denoted by arrow P3, a
fourth position from the start (first) of the temporal
list is designated. In such a case, the temporal list
(RefPicListTempO[rldx]) of L0 is generated in order of
short-term (before) reference images of indexes "0" and
15 ul", a short-term (after) reference image of index "0",
inter-view reference images of indexes u 0 " to "3", a
short-term (after) reference image of index "1", and a
long-term reference image of index "0",
[0122]
20 In such a case, the reference list of L0 is
generated according to num_ref_idx_10_active_minusl = 4
in order of short-term (before) reference images of
indexes "0" and "1", a short-term (after) reference image
of index "0", and inter-view reference images of indexes
25 "0" to nl".
[0123]
As above, according to the method disclosed in Non-
Patent Document 2, in the slice header extension,
positions at which inter-view reference images are
30 inserted in the temporal list is transmitted in the
syntax. Accordingly, in consideration of the view
/ '
38
SP351692WO00
correlation/ inter-view reference images can be inserted
at a preferred position in the temporal list.
[0124]
However, in the case of the method disclosed in
5 Non-Patent Document 2, also when the inter-view reference
images do not need to be moved, the syntax needs to be
transmitted.
[0125]
In addition, in such a case, a plurality of the
10 inter-view reference images are inserted to the position
set in the syntax at once. Accordingly, in the temporal
list, when inter-view reference images and short-term
reference images or long-term reference image are desired
to be mixed, the reference list change command needs to
15 be transmitted.
[0126]
Thus, according to the present technology, the
temporary list is generated by inserting reference images
that can be referred to in the view direction among a
20 plurality of reference images that can be referred to in
the time direction.
[0127]
[Method of Generating Reference List According to Present
Technology 1]
25 Next, a method of generating a reference list
according to the present technology 1 will be described
with reference to Fig. 5. In an example illustrated in
Fig. 5, as the example in which four reference images are
referred to, a list (arrangement)
30 (RefPicSetStCurrBefore[i]) of short-term (before)
reference images, a list (RefPicSetStCurrAfter[i]) of
39
SP351692WO00
short-term (after) reference images, a list
(RefPicSetLtCurr[i] ) of long-term reference images, and a
list (RefPicSetlvCurr[i]) (i: index = 0 to 3) of interview
reference images in the decoded picture buffer 112
5 are illustrated.
[0128]
According to the present technology 1, a temporal
list (RefPicListTempO[rldx]) of L0 is generated in order
of short-term (before) reference images of indexes "0"
10 and "1", inter-view reference images of indexes "0" to
"3", short-term (after) reference images of indexes "0"
and "1", and a long-term reference image of index "0".
[0129]
In such a case, the reference list of L0 is
15 generated according to num__ref__idx_10_active_minusl — 4
in order of short-term (before) reference images of
indexes "0" and "1" and inter-view reference images of
indexes w 0 " to "2".
[0130]
20 According to the present technology 1, a temporal
list (RefPicListTempl[rldx]) of Ll is generated in order
of short-term (before) reference images of indexes "0"
and "1", inter-view reference images of indexes "0" to
"3", short-term (after) reference images of indexes "0"
25 and "1", and a long-term reference image of index "0".
[0131]
In such a case, the reference list of Ll is
generated according to num_ref_idx_10_active_minusl = 3
in order of short-term (after) reference images of
30 indexes "0" and ul" and inter-view reference images of
indexes "0" and "1".
40
SP351692WO00
[0132]
As above, according to the method of present
technology 1, in generation of the temporal list, the
inter-view images are inserted between the short-term
5 (before) reference images and the short-term (after)
reference images.
[0133]
Since this is a default process, an additional
syntax does not need to be transmitted. In addition,
10 since the short-term (after) reference images enters the
list with a highest priority level in Ll, by inserting
the inter-view reference images before the short-term
(after) reference images in L0, a duplicate use of the
short-term (after) reference images in L0/L1 can be
15 avoided.
[0134]
However, in the case of L0, when the number of
short-term (before) reference images is large, there are
inter-view images not entering the reference list.
20 Regarding this, the number of short-term (before)
reference images may be limited by using the RPS. In
addition, in the case of Ll, while the order of the
short-term reference images being the target is reversed,
the process may be similarly described.
25 [0135]
In addition, after the short-term (before/after)
reference images, a plurality of inter-view reference
images are inserted at once. Thus, in the temporal list,
when the inter-view reference images and the short-term
3 0 reference images or the long-term reference images are
desired to be mixed, the reference list change command
41
SP351692WO00
needs to be transmitted.
[0136]
In a case where the time correlation is extremely
higher than the correlation between views, there is
5 concern that the coding efficiency may decrease.
[0137]
[Method of Generating Reference List According to Present
Technology 2]
In addition, the method of generating a reference
10 list according to the present technology 2 will be
described with reference to Fig. 6. In an example
illustrated in Fig. 6, as the example in which four
reference images are referred to, a list (arrangement)
(RefPicSetStCurrBefore[i]) of short-term (before)
15 reference images, a list (RefPicSetStCurrAfter[i]) of
short-term (after) reference images, a list
(RefPicSetLtCurr[i]) of long-term reference images, and a
list (RefPicSetlvCurr[i]) (i: index = 0 to 3) of interview
reference images in the decoded picture buffer 112
20 are illustrated.
[0138]
According to the present technology 2, a temporal
list (RefPi cListTempO[rldx]} of L0 is generated in order
of a short-term (before) reference image of index "0", an
25 inter-view reference image of index "0", a short-term
(before) reference image of index "1", an inter-view
reference image of index "1", a short-term (after)
reference image of index u0", an inter-view reference
image of index "2", a short-term (after) reference image
30 of index "1", an inter-view reference image of index 3,
and a long-term reference image of index "0".
42
SP351692WO00
[0139]
In such a case, the reference list of L0 is
generated according to num_ref_idx_10_active_minusl = 4
in order of the short-term (before) reference image of
5 index u0", the inter-view reference image of index "0",
the short-term (before) reference image of index ul", the
inter-view reference image of index "1", and the shortterm
(after) reference image of index "0".
[0140]
10 In addition, according to the present technology 2,
a temporal list (RefPicListTempl[rldx]) of Ll is
generated in order of a short-term (after) reference
image of index "0", an inter-view reference image of
index "0", a short-term (after) reference image of index
15 "1", an inter-view reference image of index "1", a shortterm
(before) reference image of index "0", an inter-view
reference image of index "2", a short-term (before)
reference image of index "1", an inter-view reference
image of index 3, and a long-term reference image of
2 0 index "0".
[0141]
In such a case, the reference list of Ll is
generated according to num_ref_idx_ll active_minusl = 3
in order of a short-term (after) reference image of index
25 "0", an inter-view reference image of index "0", a shortterm
(after) reference image of index "1", and an interview
reference image of index "1".
[0142]
As above, according to the method of the present
30 technology 2, in the generation of the temporal list, a
reference image disposed in the time direction and a
43
SP351692WO00
reference image disposed in the view direction are
alternately arranged.
[0143]
When the height of the time correlation and the
5 height of the correlation between views are alternately
lined up/ an arrangement having high reference efficiency
is formed. In addition, in the case of L0, even when the
number of short-term (before) reference images is large,
it is easy to add inter-view reference images to the
10 reference list as default. In addition, since this is a
default process, an additional syntax does not need to be
transmitted.
[0144]
However, in a case where the time correlation is
15 extremely higher than the correlation between views,
there is concern that the coding efficiency decreases.
[0145]
[Comparison between Conventional Technology and Present
Technology]
20 In an example represented in Fig. 7, a table
comparing the conventional method described above and the
method according to the present technology is illustrated.
[0146]
There are two advantages of the present technology
25 1 over the conventional technology 1. The first
advantage is that the coding efficiency can be improved
for a sequence having a high inter-view correlation
{general case). The inter-view correlation represents a
correlation between inter-view images. The second
30 advantage is that, when the number of short-term (before)
reference images (STbC) and short-term (after) reference
44
SP351692WO00
images (STaC) is small, the number of unique pictures can
be increased in the whole reference list LO/Ll.
[0147]
There are two advantages of the present technology
5 1 over the conventional technology 2. The first
advantage is that an additional syntax does not need to
be transmitted. The second advantage is that the
processing amount is small owing to no rearrangement of
the temporal list.
10 [0148]
There are two advantages of the present technology
2 over the conventional technology 1. The first
advantage is that, when the temporal correlation and the
inter-view correlation are alternately aligned, the
15 coding efficiency can be improved. Here, the temporal
correlation is a correlation between inter images of
different time points. The inter-view correlation is a
correlation between inter-view images. The second
advantage is that inter-view reference images can be
20 arranged at the second position from the start of the
reference list also in the worst case.
[0149]
There are two advantages of the present technology
2 over the conventional technology 2. The first
25 advantage .is that an additional syntax does not need to
be transmitted. The second advantage is that, when the
temporal correlation and the inter-view correlation are
alternately aligned, the coding efficiency can be
improved.
30 [0150]
In addition, the present technology 1 and the
45
SP351692WO00
present technology 2 will be compared with each other.
[0151]
The present technology 2 has two advantages over
the present technology 1. The first advantage is that,
5 when the temporal correlation and the inter-view
correlation are alternately aligned, the coding
efficiency can be improved. The second advantage is that
inter-view reference images can be arranged at the second
position from the start of the reference list also in the
10 worst case.
[0152]
In the description presented above, similar to a
conventional case, while the example has been described
in which the seguence of adding inter-view images to the
15 reference list (temporal list) is the same for LO and LI,
the order may be changed between LO and Ll.
[0153]
[Alignment between Inter-view Images]
Next, the alignment order between inter-view images
20 according to the present technology will be described
with reference to Fig. 8.
[0154]
Conventionally, corresponding inter-view reference
images are added to the reference list (temporal list) in
25 order of j = 0, 1, 2, • • * based on ref_layer__id[i] [j] of
the VPS (Video Parameter Set). This process is
completely the same between L0 and Ll, and, in the
temporal lists of L0/L1, the order in which inter-view
reference images are aligned is the same.
30 [0155]
Thus, according to this technology, the inter-view
46
SP351692WO00
reference images are added to the temporal list of LO in
order of ref_layer_id[i][j] (here/ j = 0, 1, 2, • * • ) . In
addition, the inter-view reference images are added to
the temporal list of Ll in order of ref__layer_id[i][j]
5 (here, j - • • *, 2, 1, 0. ) .
[0156]
More specifically, as illustrated on the left side
in Fig. 8, for example, in a case where a view image of
view_id = 2 is to be coded among four views of view_id =
10 0, 1, 2, and 3, it is assumed that the reference relation
is described in order of view_id = 1, 0, and 3 in the VPS.
[0157]
In this case, according to the present technology,
as illustrated on the right side in Fig. 8, in the
15 temporal list of L0, inter-view reference images are
aligned in order of view_id = 1, 0, and 3 described in
the VPS. In addition, in the temporal list of Ll, interview
reference images are aligned in order of view_id = 3,
0, and 1 that is the reverse order of view__id described
20 in the VPS.
[0158]
By configuring as such, in the temporal lists of
L0/L1, in a case where only up to the (ref_id = n+l)-th
image is referred to, in other words, in a case where
25 only up to the (ref_id = n+l)-th image is added to the
reference list, different inter-view reference images can
be referred to in L0 and Ll.
[0159]
More specifically, in the example illustrated in
30 Fig. 8, the (n+l)-th inter-view reference image is an
inter-view reference image vl of view id = 1 in L0, and,
47
SP351692WO00
the {n+l)-th inter-view reference image is an inter-view
reference image v3 of view_id = 3 in LI, whereby the
(n+l)-th inter-view reference image can be changed
between LO and Ll.
5 [0160]
In this way, since different inter-view reference
images can be referred to in LO and Ll, there is a high
possibility of having higher performance of a parallax
prediction than that of a case where the same image is
10 referred to, and the coding efficiency can be expected to
be improved.
[0161]
In the example represented in Fig. 8, similar to
the conventional example represented in Fig. 3, while the
15 example is illustrated in which, after the reference
images disposed in the time direction, the reference
images disposed in the view direction are added, the
order of the inter-view reference images of the case of
the present technology described above with reference to
20 Figs. 5 and 6 may be employed as well.
[0162]
In a case where the present technology is applied,
according to the technology disclosed in Non-Patent
Document 1, as denoted by a solid line in Fig. 9, there
25 is no change for L0, as denoted by a dotted line
illustrated in Fig. 9, the inter-view images are changed
to be added in reverse order of L0.
[0163]
The reference list generating unit 122, which is
30 illustrated in Fig. 1, performing the above-described
process is configured as illustrated in the following Fig.
48
SP351692WO00
10.
[0164]
[Configuration Example of Reference List Generating Unit]
Fig. 10 is a block diagram that illustrates an
5 example of the configuration of the reference list
generating unit that performs the above-described process.
[0165]
In the example illustrated in Fig. 10, the
reference list generating unit 122 is configured to
10 include: a reference image setting unit 131; a temporal
list generating unit 132; and a reference image list
generating unit 133.
[0166]
The reference image setting unit 131 sets short-
15 term (before) reference images by referring to time
information (in other words, the POC information) of the
reference image supplied from the decoded picture buffer
112 and the flag used__by_curr of the short-term reference
image that is set in the slice header or the RPS supplied
20 from the syntax processing unit 121 and generates a
short-term (before) reference image list
(RefPicSetStCurrBefore[i]). In addition, the reference
image setting unit 131 sets short-term (after) reference
images by referring to time information of the reference
25 image supplied from the decoded picture buffer 112 and
the flag used__by_curr of the short-term reference image
that is set in the slice header or the RPS supplied from
the syntax processing unit 121 and generates a short-term
(after) reference image list(RefPicSetStCurrAfter[i]).
30 [0167]
The reference image setting unit 131 sets long-term
49
SP351692WO00
reference images by referring to time information of the
reference image supplied from the decoded picture buffer
112 and the flag used_by_curr of the long-term reference
image that is set in the slice header or the SPS supplied
5 from the decoded picture buffer 112 and generates a longterm
reference image list (RefPicSetLtCurr[i]). In
addition, the reference image setting unit 131 sets
inter-view reference images {RefPicSetlvCurr[i]) by
referring to view information of the reference image
10 supplied from the decoded picture buffer 112 and
generates a list thereof.
[0168]
The temporal list generating unit 132 combines
lists set by the reference image setting unit 131 in
15 predetermined order according to the present technology
described above, thereby generating temporal lists
(RefPicListTempO[rIdx] and RefPicListTempl[rldx]) of L0
and Ll.
[0169]
20 The reference image list generating unit 133 refers
to the number num_ref__idx_10_active_minusl and the number
nuni_ref_idx__ll__active_minusl set in the slice header or
the SPS supplied from the syntax processing unit 121.
The reference image list generating unit 133 extracts
25 reference images corresponding to the number
num_ref_idx_10_active_minusl and the number
num_ref_idx__ll_active_minusl from the temporal lists of
L0/L1 generated by the temporal list generating unit 132
and adds the extracted reference images, thereby
30 generating reference lists of L0/L1.
[0170]
50
SP351692WO00
The reference lists of L0/L1 generated by the
reference image list generating unit 133 are referred to
by the motion parallax prediction/compensation unit 115.
[0171]
5 [Flow of Coding Process]
Next, the flow of each process performed by the
image encoding device 100 as above will be described.
First, an example of the flow of the coding process will
be described with reference to a flowchart represented in
10 Fig. 11.
[0172]
In Step S101, the A/D converter 101 performs A/D
conversion of an input image. In Step S102, the screen
rearrangement buffer 102 stores the image that is
15 converted from analog to digital and rearranges the image
from the display order of pictures to the coding order.
[0173]
In Step S103, the syntax processing unit 121
sequentially checks data of images read from the screen
20 rearrangement buffer 102 and inserts header information
to the data of each image. The syntax processing unit
121 supplies the image into which the header information
and the like are inserted to the calculation unit 103,
the intra prediction unit 114, and the motion parallax
25 prediction/compensation unit 115. In addition, the
syntax processing unit 121 supplies information such as
the VPS, the SPS (including the RPS), and the slice
header to the reference list generating unit 122.
[0174]
3 0 In Step SI04, the calculation unit 103 calculates a
difference between the image supplied from the syntax
51
SP351692WO00
processing unit 121 and the predicted image. The
predicted image is supplied to the calculation unit 103
through the selection unit 116 from the motion parallax
prediction/compensation unit 115 in the case of the inter
5 prediction or from the intra prediction unit 114 in the
case of the intra prediction.
[0175]
The data amount of the differential data is smaller
than that of the original image data. Accordingly, the
10 data amount can be compressed more than that of a case
where the image is directly coded.
[0176]
In Step SI05, the orthogonal transform unit 104
performs an orthogonal transform of the differential
15 information generated by the process of Step S104. More
specifically, an orthogonal transform such as a discrete
cosine transform or a Karhuren-Loeve transform is
performed, and a transform coefficient is output.
[0177]
2 0 In Step S10 6, the quantization unit 105 quantizes
the orthogonal transform coefficient acquired by the
process of Step S105.
[0178]
The differential information quantized by the
25 process of Step S106 is locally decoded as below. In
Step S107, the inverse quantization unit 108 performs
inverse quantization of the quantized orthogonal
transform coefficient (also referred to as a quantization
coefficient), which is generated by the process of Step
30 S106, according to characteristics corresponding to the
characteristics of the quantization unit 105.
52
SP351692WO00
[0179]
In Step S108, the inverse orthogonal transform unit
109 performs an inverse orthogonal transform of the
orthogonal transform coefficient, which is acquired by
5 the process of Step S107, according to characteristics
corresponding to the characteristics of the orthogonal
transform unit 104.
[0180]
In Step S109, the calculation unit 110 adds the
10 predicted image to the differential information that is
locally decoded, thereby generating a locally-decoded
image (an image corresponding to the input to the
calculation unit 103).
[0181]
15 In Step S110, the deblocking filter 111 performs
filtering for the image supplied from the calculation
unit 110, thereby eliminating a block distortion. Then,
the deblocking filter 111 supplies an image acquired as a
result thereof to the adaptive offset filter 123.
20 [0182]
In Step Sill, the adaptive offset filter 123
performs an adaptive offset filter process for the image
supplied from the deblocking filter 111 for each LCU.
The adaptive offset filter 123 supplies an image acquired
25 as a result thereof to the adaptive loop filter 124. In
addition, the adaptive offset filter 123 supplies the
storage flag, the index or the offset, and the type
information to the lossless encoding unit 106 as offset
filter information for each LCU.
30 [0183]
In Step SI12, the adaptive loop filter 12 4 performs
53
SP351692WO00
an adaptive loop filter process for the image supplied
from the adaptive offset filter 123 for each LCU. The
adaptive loop filter 124 supplies an image acquired as a
result thereof to the decoded picture buffer 112. In
5 addition, the adaptive loop filter 124 supplies the
filter coefficient used in the adaptive loop filter
process to the lossless encoding unit 106.
[0184]
In Step S113, the decoded picture buffer 112 stores
10 the image for which the filter process has been performed.
In addition, an image for which the filter process has
not been performed is supplied also to the decoded
picture buffer 112 from the calculation unit 110 and is
stored therein. The image accumulated in the decoded
15 picture buffer 112 is supplied to the motion parallax
prediction/compensation unit 115 or the intra prediction
unit 114 through the selection unit 113 as a reference
image.
[0185]
20 In Step S114, the intra prediction unit 114
performs an intra prediction process of the intra
prediction mode.
[0186]
In Step S115, the reference list generating unit
25 122 generates a reference list that is referred to when
the motion parallax prediction/compensation unit 115
predicts a current image. The process of generating this
reference list will be described later in detail with
reference to Fig. 12.
30 [0187]
By the process of Step S115, the temporal list that
54
SP351692WO00
is a temporary list used when the reference list is
generated is generated by inserting reference images that
can be referred to in the view direction among a
plurality of reference images that can be referred to in
5 the time direction. Then, a reference list is generated
based on the generated temporal list.
[0188]
In Step S116, the motion parallax
prediction/compensation unit 115 performs an inter motion
10 parallax prediction process in which a motion parallax
prediction of the inter prediction mode or a motion
parallax compensation is performed by using the reference
image of a reference image index represented by the
reference list generated by the process of Step S115.
15 [0189]
In Step S117, the selection unit 116 determines an
optimal prediction mode based on cost function values
output from the intra prediction unit 114 and the motion
parallax prediction/compensation unit 115. In other
20 words, the selection unit 116 selects either the
predicted image generated by the intra prediction unit
114 or the predicted image generated by the motion
parallax prediction/compensation unit 115.
[0190]
25 In addition, selection information representing
which predicted image is selected is supplied to the
intra prediction unit 114 or the motion parallax
prediction/compensation unit 115 of which the predicted
image has been selected. In a case where the predicted
30 image of the optimal intra prediction mode is selected,
the intra prediction unit 114 supplies information (in
55
SP351692WO00
other words, intra prediction mode information)
representing the optimal intra prediction mode to the
lossless encoding unit 106.
[0191]
5 In a case where the predicted image of the optimal
inter prediction mode is selected, the motion parallax
prediction/compensation unit 115 outputs information
representing the optimal inter prediction mode and
information according to the optimal inter prediction
10 mode as is necessary to the lossless encoding unit 106.
As the information according to the optimal inter
prediction mode, motion vector information, parallax
vector information, flag information, reference frame
information, or the like may be provided.
15 [0192]
In Step S118, the lossless encoding unit 106 codes
the transform coefficient quantized by the process of
Step S106. In other words, for a differential image (in
the case of the inter prediction, a secondary
20 differential image), lossless coding such as variablelength
coding or arithmetic coding is performed.
[0193]
In addition, the lossless encoding unit 106 adds
the information relating to the prediction mode of the
25 prediction image that is selected by the process of Step
S117 to the coded data acquired by coding the
differential image. In other words, the lossless
encoding unit 106 adds the intra prediction mode
information supplied from the intra prediction unit 114,
30 the information according to the optimal inter prediction
mode supplied from the motion parallax
56
SP351692WO00
prediction/compensation unit 115, or the like to the
coded data.
[0194]
In Step S119, the accumulation buffer 107
5 accumulates the coded data output from the lossless
encoding unit 106. The coded data accumulated in the
accumulation buffer 107 is appropriately read and is
transmitted to the decoding side through a transmission
line.
10 [0195]
In Step S120, the rate control unit 117 controls
the rate of the quantization operation performed by the
quantization unit 105 based on the compressed image
accumulated in the accumulation buffer 107 by the process
15 of Step S119 such that overflow or underflow does not
occur.
[0196]
When the process of Step S120 ends, the coding
process ends.
20 [0197]
[Flow of Reference List Generating Process]
Next, an example of the flow of the reference list
generating process performed in Step S115 represented in
Fig. 11 will be described with reference to a flowchart
25 represented in Fig. 12. By performing this process, the
reference lists according to the present technology 1
described above with reference to Fig. 5 are generated.
This process is performed only in a case where the
decoded image {in other words, a current image for a
30 prediction) is a P picture or a B picture.
[0198]
57
SP351692WO00
In Step S103 represented in Fig. 11 described above,
information such as the VPS, the SPS (including the RPS)
and the slice header of the decoded image is supplied
from the syntax processing unit 121 to the reference list
5 generating unit 122. The flag used_by_curr of the shortterm
reference image is set in the slice header or the
RPS supplied from the syntax processing unit 121. The
flag used_by_curr of the long-term reference image is set
in the slice header or the SPS.
10 [0199]
In Step S131, the reference image setting unit 131
of the reference list generating unit 122 sets short-term
(before) reference images (STbC) by referring to the time
information (in other words, the POC information) of the
15 reference image supplied from the decoded picture buffer
112 and the flag used_by_curr of the short-term reference
image of the slice header or the RPS and generates a list
RefPicSetStCurrBefore[i].
[0200]
2 0 In Step S132, the reference image setting unit 131
sets short-term (after) reference images (STaC) by
referring to the time information of the reference image
supplied from the decoded picture buffer 112 and the flag
used__by_curr of the short-term reference image of the
25 slice header or the RPS and generates a list
RefPicSetStCurrAfterfi].
[0201]
In Step S133, the reference image setting unit 131
sets long-term reference images (LT) by referring to the
30 time information of the reference image supplied from the
decoded picture buffer 112 and the flag used by curr of
58
SP351692WO00
the long-term reference image of the slice header or the
SPS and generates a list RefPicSetLtCurr[i].
[0202]
In Step S134, the reference image setting unit 131
5 sets inter-view reference images {IV) by referring to the
view information of the reference image supplied from the
decoded picture buffer 112 and generates a list
RefPicSetlvCurr[i].
[0203]
10 In Step S135, the temporal list generating unit 13 2
combines four types of lists set by the reference image
setting unit 131 in order of STbC, IV, STaC, and LT,
thereby generating a temporal list
(RefPicListTempO[rIdx]) of L0.
15 [0204]
In other words, only reference images, of which the
flag used_by_curr is "1" in the lists of four types
described above are combined in order of STbC, IV, STaC,
and LT, whereby a temporal list of L0 is generated. In
20 addition, at that time, all the inter-view reference
images (IV) are added.
[0205]
In Step S136, the temporal list generating unit 132
combines four types of lists set by the reference image
25 setting unit 131 in order of STaC, IV, STbC, and LT,
thereby generating a temporal list
{RefPicListTempl[rIdx]) of LI.
[0206]
In other words, only reference images of which the
30 flag used_by_curr is "1" are combined in order of STaC,
IV, STbC, and LT, whereby a temporal list of LI is
59
SP351692WO00
generated. In addition, at that time, while all the
inter-view reference images (IV) are added, the order, as
described above with reference to Fig. 8, may be
configured to be reverse order of that of the direction
5 LO,
[0207]
The numbers num_ref_idx_10_active_minusl and
num_ref_idx_ll_active_minusl are set in the slice header
or the SPS supplied from the syntax processing unit 121.
10 [0208]
In Step S137, the reference image list generating
unit 133 extracts the first to the
(num__ref_idx_10_active_minusl + l) ~th elements of the
temporal list (RefPicListTempO[rldx]) of L0 and generates
15 a reference list (RefPicListO[rldx]) of L0.
[0209]
In Step S138, the reference image list generating
unit 133 extracts the first to the
(num_ref_idx_ll_active_minusl+l)~th elements of the
20 temporal list (RefPicListTempl[rldx]) ot Ll and generates
a reference list (RefPicListl[rldx]) of Ll.
[0210]
As above, the inter-view images are inserted
between the short-term (before) reference images and the
25 short-term (after) reference images, whereby reference
lists described above with reference to Fig. 5 are
generated.
[0211]
[Another Flow of Reference List Generating Process]
30 Next, an example of the flow of the reference list
generating process performed in Step S115 represented in
60
SP351692WO00
Fig. 11 will be described with reference to a flowchart
represented in Fig. 13. By performing this process, the
reference lists according to the present technology 2
described above with reference to Fig. 6 are generated.
5 This process is performed only in a case where the
decoded image (in other words, a current image for a
prediction) is a P picture or a B picture.
[0212]
In Step S103 represented in Fig. 11 described above,
10 information such as the VPS, the SPS (including the RPS)
and the slice header of the decoded image is supplied
from the syntax processing unit 121 to the reference list
generating unit 122. The flag used_by_curr of the shortterm
reference image is set in the slice header or the
15 RPS supplied from the syntax processing unit 121. The
flag used__by_curr of the long-term reference image is set
in the slice header or the SPS.
[0213]
In Step S151, the reference image setting unit 131
20 of the reference list generating unit 122 sets short-term
(before) reference images (STbC) by referring to the time
information (in other words, the POC information) of the
reference image supplied from the decoded picture buffer
112 and the flag used_by_curr of the short-term reference
25 image of the slice header or the RPS and generates a list
RefPicSetStCurrBefore[i].
[0214]
In Step S152, the reference image setting unit 131
sets short-term (after) reference images (STaC) by
30 referring to the time information of the reference image
supplied from the decoded picture buffer 112 and the flag
/
61
SP351692WO00
used_by_curr of the short-term reference image of the
slice header or the RPS and generates a list
RefPicSetStCurrAfter[i].
[0215]
5 In Step S153, the reference image setting unit 131
sets long-term reference images (LT) by referring to the
time information of the reference image supplied from the
decoded picture buffer 112 and the flag used_by_curr of
the long-term reference image of the slice header or the
10 SPS and generates a list RefPicSetLtCurr[i].
[0216]
In Step S154, the reference image setting unit 131
sets inter-view reference images (IV) by referring to the
view information of the reference image supplied from the
15 decoded picture buffer 112 and generates a list
RefPicSetlvCurr[i].
[0217]
In Step S155, the temporal list generating unit 132
generates two lists including a list 1 acquired by
20 sequentially combining three lists of STbC, STaC, and LT
set by the reference image setting unit 131 and a list 2
configured by the IV. In other words, only reference
images, of which the flag used__by_curr is "1" are
combined in order of STbC, STaC, and LT, whereby the list
25 1 is generated. In addition, the list 2 is generated by
adding all the IV of the list RefPicSetlvCurr[i] to the
list.
[0218]
In Step S15 6, the temporal list generating unit 132
30 extracts one element from the start of each of the lists
1 and 2 generated in Step S155 and alternately aligns the
62
SP351692WO00
extracted elements, thereby generating a temporal list
(RefPicListTempO[ridx]) of LO.
[0219]
In Step S157, the temporal list generating unit 132
5 generates two lists including a list 11 acquired by
sequentially combining three lists STaC, STbC, and LT set
by the reference image setting unit 131 and a list 12
configured by the IV. In other words, only the reference
images of which the flag used_by_curr is "1" are combined
10 in order of STaC, STbC, and LT, whereby the list 11 is
generated. In addition, while the list 12 is generated
by adding all the IV of the list RefPicSetlvCurr[i], at
that time, as described above with reference to Fig. 8,
the inter-view reference images may be added in order
15 that is reverse order of the order of the direction LO.
[0220]
In Step S158, the temporal list generating unit 132
extracts one element from the start of each of the lists
11 and 12 generated in Step S155 and alternately aligns
20 the extracted elements, thereby generating a temporal
list (RefPicListTempl[rIdx]) of LI.
[0221]
The numbers num_ref__idx_10 active_minusl and
num_ref__idx_ll_active_minusl are set in the slice header
25 or the SPS supplied from the syntax processing unit 121.
[0222]
In Step S159, the reference image list generating
unit 133 extracts the first to the
(num_ref_idx_10_active__minusl + l) -th elements of the
30 temporal list (RefPicListTempO[ridx]) of L0 and generates
a reference list (RefPicListO[ridx]) of L0.
63
SP351692WO00
[0223]
In Step S160, the reference image list generating
unit 133 extracts the first to the
(nun\_ref_idx_ll_active_jninusl + l) ~th elements of the
5 temporal list (RefPicListTempl[rldx]) of Ll and generates
a reference list (RefPicListl[rldx]) of Ll.
[0224]
As above, in the generation of the temporal lists,
the reference images disposed in the time direction and
10 the reference images disposed in the view direction are
alternately arranged, whereby the reference lists
described above with reference to Fig. 6 are generated.
[0225]
In addition, while the example of two types of
15 reference lists has been described in the description
presented above, for example, it may be configured such
that the coding side transmits a reference list selection
flag of one bit representing a selected reference list to
the decoding side, and the reference list according to
20 the reference list selection flag is generated on the
decoding side. This reference list selection flag, for
example, may be configured to be transmitted for each
slice.
[0226]
25 For example, as described above with reference to
Fig. 7, a more appropriate list is different depending on
the degree of the temporal correlation or the degree of
the inter-view correlation in the sequence. Thus, by
transmitting the reference list selection flag in this
30 way, for example, the coding efficiency can be improved
more than that of a case where the change command is
64
SP351692WO00
transmitted using an image or that of a case where the
syntax is transmitted each time as disclosed in Non-
Patent Document 2.
[0227]
5 <2. Second Embodiment>
[Image Decoding Device]
Fig. 14 illustrates the configuration of an image
decoding device according to an embodiment as an image
processing device to which the present disclosure is
10 applied. An image decoding device 200 illustrated in Fig.
14 is a decoding device that corresponds to the image
encoding device 100 illustrated in Fig. 1.
[0228]
The coded data coded by the image encoding device
15 100 is assumed to be transmitted to the image decoding
device 200 corresponding to the image encoding device 100
through a predetermined transmission line and is decoded
by using the HEVC system or the like.
[0229]
20 As illustrated in Fig. 14, the image decoding
device 200 includes: an accumulation buffer 201; a
lossless decoding unit 202; an inverse quantization unit
203; an inverse orthogonal transform unit 204; a
calculation unit 205; a deblocking filter 206; a screen
25 rearrangement buffer 207; and a D/A converter 208. In
addition, the image decoding device 200 includes: a
decoded picture buffer 209; a selection unit 210; an
intra prediction unit 211; a motion parallax
prediction/compensation unit 212; and a selection unit
30 213.
[0230]
65
SP351692WO00
Furthermore, the image decoding device 200
includes: a syntax processing unit 221; a reference list
generating unit 222; an adaptive offset filter 223; and
an adaptive loop filter 224.
5 [0231]
The accumulation buffer 201 accumulates coded data
that is transmitted. This coded data is coded by the
image encoding device 100. The syntax processing unit
221 acquires a VPS, an SPS, a slice header, and the like
10 from the coded data read from the accumulation buffer 201
at predetermined timing and supplies each header
information that has been acquired to the lossless
decoding unit 202 together with the coded data. In
addition, the syntax processing unit 221 supplies the
15 header information and the like that have been acquired
to the reference list generating unit 222.
[0232]
The lossless decoding unit 2 02 decodes the coded
data supplied from the syntax processing unit 221 by
20 using a system corresponding to the coding system of the
lossless encoding unit 106 illustrated in Fig. 1. The
inverse quantization unit 203 performs inverse
quantization of coefficient data (quantization
coefficient) acquired by the decoding process performed
25 by the lossless decoding unit 202 by using a system
corresponding to the quantization system of the
quantization unit 105 illustrated in Fig. 1. In other
words, the inverse quantization unit 2 03 performs inverse
quantization of the quantization coefficient by using a
30 method similar to that of the inverse quantization unit
108 illustrated in Fig. 1 by using the quantization
66
SP351692WO00
parameter supplied from the image encoding device 100.
[0233]
The inverse quantization unit 203 supplies the
coefficient data, in other words, the orthogonal
5 transform coefficient for which the inverse quantization
process has been performed to the inverse orthogonal
transform unit 204. The inverse orthogonal transform
unit 204, by using a system corresponding to the
orthogonal transform system of the orthogonal transform
10 unit 104 illustrated in Fig. 1, performs an inverse
orthogonal transform of the orthogonal transform
coefficient, thereby acquiring decoded residual data
corresponding to the residual data before the orthogonal
transform process performed by the image encoding device
15 100.
[0234]
The decoded residual data that is acquired through
the inverse orthogonal transform is supplied to the
calculation unit 205. In addition, a predicted image is
20 supplied to the calculation unit 205 through the
selection unit 213 from the intra prediction unit 211 or
the motion parallax prediction/compensation unit 212.
[0235]
The calculation unit 205 adds the decoded residual
25 data and the predicted image, thereby acquiring decoded
image data corresponding to the image data before the
subtraction of the predicted image that is performed by
the calculation unit 103 of the image encoding device 100.
The calculation unit 205 supplies the decoded image data
30 to the deblocking filter 206.

CLAIMS
1. An image processing device comprising:
a decoding unit that generates an image by decoding
5 a bit stream;
a reference list setting unit that sets a reference
list being referred to when the image generated by the
decoding unit is predicted by inserting a reference image.
that can be referred to in a layer direction between a
10 plurality of reference images that can be referred to in
a time direction; and
a prediction unit that predicts the image generated
by the decoding unit by referring to the reference list
set by the reference list setting unit.
15
2. The image processing device according to claim 1,
wherein the reference list setting unit sets the
reference list by inserting the reference image that can
be referred to in the layer direction between a reference
20 image located before the image in time and a reference
image located after the image in time among the plurality
of reference images that can be referred to in the time
direction.
25 3. The image processing device according to claim 2,
wherein the reference list setting unit sets the
reference list by arranging images in a direction L0 in
order of reference images located before the image in
time, the reference image that can be referred to in the
30 layer direction, and reference images located after the
image in time.
146
SP351692WO00
4. The image processing device according to claim 2,
wherein the reference list setting unit sets the
reference list by arranging images in a direction LI in
5 order of reference images located after the image in time,
the reference image that can be referred to in the layer
direction, and reference images located before the image
in time.
10 5. The image processing device according to claim 1,
wherein the reference list setting unit sets the
reference list by alternately arranging the reference
image that can be referred to in the time direction and
the reference image that can be referred to in the layer
15 direction.
6. The image processing device according to claim 5,
wherein the reference list setting unit sets the
reference list by alternately arranging an element of a
20 list of the reference images that can be referred to in
the time direction set in order of the reference images
located before the image in time and the reference images
located after the image in time and an element of a list
of the reference images that can be referred to in the
25 layer direction, which is configured by the reference
images that can be referred to in the layer direction, in
a direction L0.
7. The image processing device according to claim 5,
30 wherein the reference list setting unit sets the
reference list by alternately arranging an element of a
147
SP351692WO00
list of the reference images that can be referred to in
the time direction set in order of the reference images
located after the image in time and the reference images
located before the image in time and an element of a list
5 of the reference images that can be referred to in the
layer direction, which is configured by the reference
images that can be referred to in the layer direction, in
a direction LI.
10 8. The image processing device according to claim 1,
wherein the reference list setting unit sets the
reference list by inserting the reference images that can
be referred to in the layer direction in a direction Ll
with order of the reference image that can be referred to
15 in the layer direction being opposite to that of the case
of a direction L0.
9. The image processing device according to claim 1,
wherein the reference list setting unit includes:
20 a temporal list setting unit that sets a temporal
list that is a temporary list used for setting the
reference list by inserting the reference image that can
be referred to in the layer direction between the
plurality of reference images that can be referred to in
25 the time direction; and
a reference image list setting unit that sets the
reference list based on the temporal list set by the
temporal list setting unit.
30 10. An image processing method implemented in an image
processing device, the image processing method
148
SP351692WO00
comprising:
generating an image by decoding a bit stream;
setting a reference list being referred to when the
generated image is predicted by inserting a reference
5 image that can be referred to in a layer direction
between a plurality of reference images that can be
referred to in a time direction; and
predicting the generated image by referring to the
set reference list.
10
11. An image processing device comprising:
a reference list setting unit that sets a reference
list being referred to when an image is predicted by
inserting a reference image that can be referred to in a
15 layer direction between a plurality of reference images
that can be referred to in a time direction;
a prediction unit that predicts the image by
referring to the reference list set by the reference list
setting unit; and
20 an encoding unit that generates a bit stream by
performing coding using the image predicted by the
prediction unit.
12. The image processing device according to claim 11,
25 wherein the reference list setting unit sets the
reference list by inserting the reference image that can
be referred to in the layer direction between a reference
image located before the image in time and a reference
image located after the image in time among the plurality
30 of reference images that can be referred to in the time
direction.
149
SP351692WO00
13. The image processing device according to claim 12,
wherein the reference list setting unit sets the
reference list by arranging images in a direction LO in
5 order of reference images located before the image in
time, the reference image that can be referred to in the
layer direction, and reference images located after the
image in time.
10 14. The image processing device according to claim 12,
wherein the reference list setting unit sets the
reference list by arranging images in a direction LI in
order of reference images located after the image in time,
the reference image that can be referred to in the layer
15 direction, and reference images located before the image
in time.
15. The image processing device according to claim 11,
wherein the reference list setting unit sets the
20 reference list by alternately arranging the reference
image that can be referred to in the time direction and
the reference image that can be referred to in the layer
direction.
25 16. The image processing device according to claim 15,
wherein the reference list setting unit sets the
reference list by alternately arranging an element of a
list of the reference images that can be referred to in
the time direction set in order of the reference images
30 located before the image in time and the reference images
located after the image in time and an element of a list
150
SP351692WO00
of the reference images that can be referred to in the
layer direction, which is configured by the reference
images that can be referred to in the layer direction, in
a direction L0.
5
17. The image processing device according to claim 15,
wherein the reference list setting unit sets the
reference list by alternately arranging an element of a
list of the reference images that can be referred to in
10 the time direction set in order of the reference images
located after the image in time and the reference images
located before the image in time and an element of a list
of the reference images that can be referred to in the
layer direction, which is configured by the reference
15 images that can be referred to in the layer direction, in
a direction Ll.
18. The image processing device according to claim 11,
wherein the reference list setting unit sets the
20 reference list by inserting the reference images that can
be referred to in the layer direction in a direction Ll
with order of the reference image that can be referred to
in the layer direction being opposite to that of the case
of a direction L0.
25
19. The image processing device according to claim 11,
wherein the reference list setting unit includes:
a temporal list setting unit that sets a temporal
list that is a temporary list used for setting the
30 reference list by inserting the reference image that can
be referred to in the layer direction between the
151
SP351692WO00
plurality of reference images that can be referred to in
the time direction; and
a-reference image list setting unit that sets the
reference list based on the temporal list set by the
temporal list setting unit.
20. An image processing method implemented in an image
processing device, the image processing method
comprising:
setting a reference list being referred to when an
image is predicted by inserting"a reference image that
can be referred to in a layer direction between a
plurality of reference images that can be referred to in
a time direction;
predicting the iimage by referring to the set
I
reference list; and
generating a bit stream by performing coding using
the predicted image.