DESCRIPTION
IMAGE PROCESSING DEVICE AND METHOD
TECHNICAL FIELD
5 [0001]
The present disclosure relates to an image processing
device and method, and more particularly, an image processing
device and method which are capable of suppressing an increase
in a storage capacity necessary for encoding and decoding.
10
BACKGROUND ART
[0002]
In recent years, for the purpose of digitalizing image
information and transmitting and accumulating information at
15 high efficiency at that time, devices that compress and encode
images by using image information-specific redundancy and
employing a coding scheme that performs compression through
an orthogonal transform such as a discrete cosine transform
and motion compensation have been spread. As such a coding
20 scheme, for example, there is Moving Picture Experts Group
(MPEG).
[0003]
Particularly, MPEG 2 (ISO/IEC 13818-2) is a standard
that is defined as a general-purpose image coding scheme, and
25 covers interlaced scan images, progressive scan images,
standard resolution images, and high definition images. For
example, MPEG 2 has been widely used for a wide range of
applications such as professional use and consumer use . Using
the MPEG 2 compression scheme, for example, in the case of
30 an interlaced scan image of a standard resolution having 720
x 480 pixels, a coding amount (bit rate) of 4 to 8 Mbps is
/
2
SP349405WO00
allocated. Further, using the MPEG 2 compression scheme, for
example, in the case of an interlaced scan image of a high
resolution having 1920 x 1088 pixels, a coding amount (bit
rate) of 18 to 22 Mbps is allocated. Thus, it is possible
5 to implement a high compression rate and an excellent image
quality.
[0004]
MPEG 2 is mainly intended for high definition coding
suitable for broadcasting but does not support a coding scheme
10 having a coding amount (bit rate) lower than that of MPEG 1,
that is, a coding scheme of a high compression rate. With
the spread of mobile terminals, the need for such a coding
scheme is considered to be increased in the future, and thus
an MPEG 4 coding scheme has been standardized. In connection
15 v/ith an image coding scheme, an international standard thereof
has been approved as ISO/IEC 14496-2 on December, 1998.
[0005]
Further, in recent years, standardization of a standard
such as H.2 6L (International Telecommunication Union
20 Telecommunication Standardization Sector Q6/16 Video Coding
Expert Group (ITU-T Q6/16 VCEG) ) for the purpose of image coding
for video conference has been conducted. H.2 6L requires a
larger computation amount for coding and decoding than in an
existing coding scheme such as MPEG 2 or MPEG 4, but is known
25 to implement high coding efficiency. Further, currently, as
one of activities of MPEG 4, standardization of incorporating
even a function that is not supported in H . 26L and implementing
high coding efficiency based on H.2 6L has been performed as
a Joint Model of Enhanced-Compression Video Coding.
30 [0006]
As a standardization schedule, an international
3
SP349405WO00
standard called H. 2 64 andMPEG-4 Parti0 (Advanced Video Coding,
hereinafter, referred to as "AVC") has been established on
March, 2003.
[0007]
5 However, a micro block size of 16 x 16 pixels may not
be optimal for a large image frame such as a Ultra High Definition
(UHD; 4000 x 2000pixels) serving as a target of anext generation
coding scheme.
[0008]
10 In this regard, currently, for the purpose of further
improving coding efficiency as compared to H.2 64/AVC,
standardization of a coding scheme called High Efficiency Video
Coding (HEVC) has been conducted by Joint Collaboration
Team-Video Coding (JCTVC) which is a joint standardization
15 organization of ITU-T and ISO/IEC. A committee draft of a
HEVC standard, that is, a first draft version specification
has been issued on February, 2012 (for example, see Non-Patent
Document 1).
[0009]
20 Meanwhile, in the past, as one of 3D extensions of HEVC,
a scheme of changing a CU level and improving encoding
performance of a non-base view has been reviewed. As one of
tools for this scheme, there is inter-view motion prediction
(IVMP) in which an encoded vector of a different view is used
25 as a candidate of a predictive vector of a non-base viev/ (for
example, see Non-Patent Document 2).
[0010]
For example, in the standard of HEVC Ver.l, there is
tool temporal MVP (TMVP) in which a motion vector of a picture
30 of a different timing can be used as a candidate of a predictive
vector. When this tool is used, a motion vector (MV) of an
4
SP349405WO00
encoded picture is held for the TMVP. The motion vector (MV)
is encoded in minimum 4 x 4 units, but information thereof
is compressed in 16 x 16 units until it is referred to in the
TMVP . This compression lowers prediction accuracy of amotion
5 vector (MV), but the capacity of a memory holding a motion
vector can be reduced to 1/16.
CITATION LIST
NON-PATENT DOCUMENT
10 [0011]
Non-Patent Document 1: Benjamin Bross, Woo-Jin Han,
Jens-Rainer Ohm, Gary J. Sullivan, Thomas Wiegand, "High
efficiency video coding (HEVC) text specification draft 6, "
JCTVC-H1003 ver 20, February 17, 2012
15 Non-Patent Document 2: Yoshiya Yamamoto, Tomohiro Ikai,
Tadashi Uchiumi, "3D-CE5 .h related: Simplification of AMVP, "
JCT2-A0014, Joint Collaborative Team on 3D Video Coding
Extension Development of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC
29/WG 111st Meeting: Stockholm, SE, 16-20 July, 2012
20
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0012]
However, in the 1VMP, such a motion vector compression
25 function is not prepared. In other words, in the case of the
IVMP, a motion vector is referred to with the accuracy at the
time of encoding in another layer . For example, when a motion
vector is encoded in minimum 4 x 4 units, a temporal buffer
holding a motion vector of 4 x 4 accuracy is necessary for
3 0 the IVMP.
[0013]
5
SP349405WO00
In other words, in this temporal buffer, a capacity
capable of storing at least "a motion vector of 4 x 4 accuracy
by one screen" for one view is necessary . Thus, in the temporal
buffer, a capacity that is 16 times as large as when a motion
5 vector for TMVP to be compressed up to 16 * 16 accuracy is
stored (capable of storing 16 pictures) is necessary.
[0014]
In other words, a storage capacity necessary for encoding
and decoding is likely to be increased due to the IVMP.
10 [0015]
The present disclosure was made in light of the foregoing,
and it is desirable to be able to suppress an increase in a
storage capacity necessary for encoding and decoding.
15 SOLUTIONS TO PROBLEMS
[0016]
An aspect of the present technology is an image
processing device, including: a motion compensating unit that
performs motion compensation in decoding of a current layer;
20 and a first compressing unit that compresses a motion vector
of the current layer that is reconstructed by the motion
compensating unit and used for the motion compensation in
decoding of another layer.
[0017]
25 The image processing device may further include a second
compressing unit that compresses the motion vector of the
current layer reconstructed by the motion compensating unit
at a compression rate higher than a compression rate of the
first compressing unit, and the motion compensating unit may
30 perform the motion compensation in the decoding of the current
layer using the motion vector compressed by the second
6
SP349405WO00
compressing unit.
[0018]
The second compressing unit may further compress the
motion vector compressed by the first compressing unit.
5 [0019]
The motion compensating unit may perform the motion
compensation in the decoding of the current layer using the
motion vector compressed by the first compressing unit.
[0020]
10 The image processing device according may further
includes: a receiving unit that receives a flag indicating
whether or not the motion vector of the current layer used
in the motion compensation in the decoding of the other layer;
and a selecting unit that selects the motion vector compressed
15 by the first compressing unit as the motion vector of the current
layer used in the motion compensation in the decoding of the
other layer when the flag received through the receiving unit
indicates that the motion vector is compressed, and selects
the motion vector that is not compressed by the first
20 compressing unit as the motion vector of the current layer
used in the motion compensation in the decoding of the other
layer when the flag received through the receiving unit
indicates that the motion vector is not compressed.
[0021]
25 The motion compensating unit may perform the motion
compensation in the decoding of the current layer using the
motion vector compressed by the first compressing unit
regardless of a value of the flag received through the receiving
unit.
30 [0022]
The first compressing unit may select a motion vector
7
SP349405WO00
serving as a representative value from among a plurality of
motion vectors reconstructed by the motion compensating unit,
and compress the motion vector of the current layer.
[0023]
5 The first compressing unit may calculate a motion vector
serving as a representative value using a plurality of motion
vectors reconstructed by the motion compensating unit, and
compress the motion vector of the current layer.
[0024]
10 The motion compensating unit may perform the motion
compensation using a motion vector reconstructed in the motion
compensation in the decoding of the other layer.
[0025]
Further, an aspect of the present technology is an image
15 processing method of an image processing device, including:
performing, by the image processing device, motion
compensation in decoding of a current layer; compressing, by
the image processing device, a motion vector of the current
layer that is reconstructed by the motion compensation and
20 used in the motion compensation in decoding of another layer.
[0026]
Another aspect of the present technology is an image
processing device, including: a motion
predicting/compensating unit that performs motion prediction
25 and compensation in encoding of a current layer; and a first
compressing unit that compresses amotion vector of the current
layer that is generated by the motion predicting/compensating
unit and used in the motion prediction and compensation in
encoding of another layer.
30 [0027]
The image processing device may further include a second
8
SP349405WO00
compressing unit that compresses the motion vector of the
current layer generated by the motion predicting/compensating
unit at a compression rate higher than a compression rate of
the first compressing unit, and the motion
5 predicting/compensating unit may perform the motion
prediction and compensation in the encoding of the current
layer using the motion vector compressed by the second
compressing unit.
[0028]
10 The second compressing unit may further compress the
motion vector compressed by the first compressing unit.
[0029]
The motion predicting/compensating unit may perform the
motion prediction and compensation in the encoding of the
15 current layer using the motion vector compressed by the first
compressing unit.
[0030]
The image processing device may further includes: a
control unit that controls whether or not the motion vector
20 of the current layer used in the motion prediction and
compensation in the encoding of the other layer is compressed;
a selecting unit that selects any one of the motion vector
that is not compressed by the first compressing unit and the
motion vector compressed by the first compressing unit as the
2 5 motion vector of the current layer used in the motion prediction
and compensation in the encoding of the other layer according
to control of the control unit; .a generating unit that generates
a flag indicating whether or not the motion vector of the current
layer used in the motion prediction and compensation in the
30 encoding of the other layer is compressed according to control
of the control unit; and a transmitting unit that transmits
9
SP349405WO00
the flag generated by the generating unit.
[0031]
The motion predicting/compensating unit may perform the
motion prediction and compensation in the encoding of the
5 current layer using the motion vector compressed by the first
compressing unit regardless of control of the control unit.
[0032]
The first compressing unit may select a motion vector
serving as a representative value from among a plurality of
10 motion vectors generated by the motion
predicting/compensating unit, and compress the motion vector
of the current layer.
[0033]
The first compressing unit may calculate a motion vector
15 serving as a representative value using a plurality of motion
vectors generated by the motion predicting/compensating unit,
and compress the motion vector of the current layer.
[0034]
The motion predicting/compensating unit may perform the
20 motion prediction and compensation using a motion vector
generated in the motion prediction and compensation in the
encoding of the other layer.
[0035]
Further, another aspect of the present technology is
25 an image processing method of an image processing device,
including: performing, by the image processing device, motion
prediction and compensation in encoding of a current layer;
and compressing, by the image processing device, a motion
vector of the current layer that is generated by the motion
30 prediction and compensation and used in the motion prediction
and compensation in encoding of another layer.
10
SP349405WO00
[0036]
In one aspect of the present technology, motion
compensation in decoding of a current layer is performed, and
a motion vector of the current layer that is reconstructed
5 by the motion compensation and used for the motion compensation
in decoding of another layer is compressed.
[0037]
In another aspect of the present technology, motion
prediction and compensation in encoding of a current layer
10 is performed, and a motion vector of the current layer that
is generated by the motion prediction and compensation and
used in the motion prediction and compensation in encoding
of another layer is compressed.
[0038]
15 The image processing device may be an independent device
or may be an internal block configuring a single image coding
device or a single image decoding device.
EFFECTS OF THE INVENTION
20 [0039]
According to the present disclosure, it is possible to
encode and decode an image. Particularly, it is possible to
suppress an increase in a storage capacity necessary for
encoding and decoding.
25
BRIEF DESCRIPTION OF DRAWINGS
[0040]
Fig. 1 is a diagram for describing an exemplary
configuration of a coding unit.
30 Fig. 2 is a diagram illustrating an exemplary multi-view
image coding scheme.
11
SP349405WO00
Fig, 3 is a diagram for describing an example of IVMP.
Fig, 4 is a diagram for describing an example of a motion
vector reference according to a related art.
Fig. 5 is a diagram for describing an example of a motion
5 vector reference according to the present technology.
Fig . 6 is a block diagram illustrating an exemplary main
configuration of a VO image coding device.
Fig. 7 is a block diagram illustrating an exemplary main
configuration of a VI image coding device.
10 Fig . 8 is a block diagram illustrating an exemplary main
configuration of a V2 image coding device.
Fig. 9 is a flowchart for describing an exemplary flov/
of a coding process.
Fig. 10 is a flowchart for describing an exemplary flov/
15 of a VO coding process.
Fig. 11 is a flowchart for describing an exemplary flow
of a VO inter motion prediction process.
Fig. 12 is a flowchart for describing an exemplary flov/
of a VI coding process.
20 Fig. 13 is a flowchart for describing an exemplary flov/
of a VI inter motion prediction process.
Fig. 14 is a flowchart for describing an exemplary flow
of a V2 coding process.
Fig. 15 is a flowchart for describing an exemplary flow
25 of a V2 inter motion prediction process.
Fig. 16 is a block diagram illustrating an exemplary
main configuration of a VO image decoding device.
Fig. 17 is a block diagram illustrating an exemplary
main configuration of a VI image decoding device.
30 Fig. 18 is a block diagram illustrating an exemplary
main configuration of a V2 image decoding device.
12
SP349405WO00
Fig. 19 is a flowchart for describing an exemplary flow
of a decoding process.
Fig. 20 is a flowchart for describing an exemplary flow
of a V0 decoding process.
5 Fig. 21 is a flowchart for describing an exemplary flow
of a V0 motion compensation process.
Fig. 22 is a flowchart for describing an exemplary flow
of a VI decoding process.
Fig. 23 is a flowchart for describing an exemplary flow
10 of a VI motion compensation process.
Fig. 24 is a flowchart for describing an exemplary flow
of a V2 decoding process.
Fig. 25 is a flowchart for describing an exemplary flow
of a V2 motion compensation process.
15 Fig . 2 6 is a diagram for describing an example of a motion
vector reference according to the present technology.
Fig. 27 is a block diagram illustrating an exemplary
main configuration of a V0 image coding device.
Fig. 28 is a block diagram illustrating an exemplary
20 main configuration of a VI image coding device.
Fig. 29 is a block diagram illustrating an exemplary
main configuration of a V2 image coding device.
Fig. 30 is a flowchart for describing an exemplary flow
of a V0 coding process.
25 Fig. 31 is a flowchart for describing an exemplary flow
of a VI coding process.
Fig. 32 is a flowchart for describing an exemplary flow
of a VI inter motion prediction process.
Fig. 33 is a flowchart for describing an exemplary flow
30 of a V2 coding process.
Fig. 34 is a flowchart for describing an exemplary flow
13
SP349405WO00
of a V2 inter motion prediction process.
Fig. 35 is a block diagram illustrating an exemplary
main configuration of a VO image decoding device.
Fig. 36 is a block diagram illustrating an exemplary
5 main configuration of a VI image decoding device.
Fig. 37 is a block diagram illustrating an exemplary
main configuration of a V2 image decoding device.
Fig. 38 is a flowchart for describing an exemplary flow
of a VO decoding process.
10 Fig. 39 is a flowchart for describing an exemplary flow
of a VI decoding process.
Fig. 4 0 is a flowchart for describing an exemplary flow
of a VI motion compensation process.
Fig. 41 is a flowchart for describing an exemplary flow
15 of a V2 decoding process.
Fig. 42 is a flowchart for describing an exemplary flow
of a V2 motion compensation process.
Fig. 4 3 is a diagram for describing an example of a motion
vector reference according to the present technology.
20 Fig . 44 is a diagram for describing an exemplary syntax .
Fig. 45 is a block diagram illustrating an exemplary
main configuration of a VO image coding device.
Fig. 46 is a block diagram illustrating an exemplary
main configuration of a VI image coding device.
25 Fig. 47 is a block diagram illustrating an exemplary
main configuration of a V2 image coding device.
Fig. 48 is a flowchart for describing an exemplary flow
of the coding process.
Fig. 49 is a flowchart for describing an exemplary flow
30 of a VO coding process.
Fig. 50 is a flowchart for describing an exemplary flow
14
SP349405WO00
of a VO coding process, subsequently to Fig. 49.
Fig. 51 is a flowchart for describing an exemplary flow
of a VI coding process.
Fig. 52 is a flowchart for describing an exemplary flow
5 of a VI coding process, subsequently to Fig. 51.
Fig. 53 is a flowchart for describing an exemplary flow
of a VI inter motion prediction process.
Fig. 54 is a flowchart for describing an exemplary flow
of a V2 coding process.
10 Fig. 55 is a flowchart for describing an exemplary flow
of a V2 coding process, subsequently to Fig. 54.
Fig. 56 is a flowchart for describing an exemplary flow
of a V2 inter motion prediction process.
Fig. 57 is a block diagram illustrating an exemplary
15 main configuration of a VO image decoding device.
Fig. 58 is a block diagram illustrating an exemplary
main configuration of a VI image decoding device.
Fig. 59 is a block diagram illustrating an exemplary
main configuration of a V2 image decoding device.
20 Fig. 60 is a flowchart for describing an exemplary flow
of a decoding process.
Fig. 61 is a flowchart for describing an exemplary flow
of a VO decoding process.
Fig. 62 is a flowchart for describing an exemplary flow
25 of a VO decoding process, subseguently to Fig. 61.
Fig. 63 is a flowchart for describing an exemplary flow
of a VI decoding process.
Fig. 64 is a flowchart for describing an exemplary flow
of a VI decoding process, subsequently to Fig. 64.
30 Fig. 65 is a flowchart for describing an exemplary flow
of a VI motion compensation process.
15
SP349405WO00
Fig. 66 is a flowchart for describing an exemplary flov;
of a V2 decoding process.
Fig. 67 is a flowchart for describing an exemplary flov;
of a V2 decoding process, subsequently to Fig, 66.
5 Fig. 68 is a flowchart for describing an exemplary flov/
of a V2 motion compensation process.
Fig. 69 is a diagram illustrating an exemplary scalable
image coding scheme.
Fig. 70 is a diagram for describing an exemplary spatial
10 scalable coding.
Fig . 71 is a diagram for describing an exemplary temporal
scalable coding.
Fig . 72 is a diagram for describing an exemplary scalable
coding of a signal to noise ratio.
15 Fig.73isa diagram for describing an example of thinning
out a motion vector.
Fig. 74 is a diagram for describing another example of
thinning out a motion vector.
Fig. 75 is a diagram for describing exemplary angular
20 prediction.
Fig. 76 is a diagram for describing exemplary planar
prediction.
Fig. 77 is a diagram for describing an exemplary most
probable mode.
25 Fig. 78 is a diagram for describing exemplary mode
dependent intra smoothing (MDIS).
Fig . 79 is a diagram for describing an exemplary boundary
value smoothing process.
Fig . 8 0 is a diagram illustrating a thinning-out example
30 of an intra prediction mode.
Fig. 81 is a block diagram illustrating an exemplary
16
SP349405WO00
main configuration of a scalable coding device.
Fig. 82 is a block diagram illustrating an exemplary
main configuration of a base layer image encoding unit.
Fig. 83 is a block diagram illustrating an exemplary
5 main configuration of an enhancement layer image encoding unit.
Fig. 84 is a block diagram illustrating an exemplary
main configuration of an intra prediction mode providing unit.
Fig. 85 is a flowchart for describing an exemplary flow
of a common information generation process.
10 Fig. 86 is a flowchart for describing an exemplary flow
of a coding process.
Fig. 87 is a flowchart for describing an exemplary flow
of a base layer coding process.
Fig. 88 is a flowchart for describing an exemplary flow
15 of an enhancement layer coding process.
Fig. 89 is a block diagram illustrating an exemplary
main configuration of a scalable decoding device.
Fig. 90 is a block diagram illustrating an exemplary
main configuration of a base layer image decoding unit.
20 Fig. 91 is a block diagram illustrating an exemplary
main configuration of an enhancement layer image decoding unit.
Fig. 92 is a block diagram illustrating an exemplary
main configuration of an intra prediction mode providing unit.
Fig. 93 is a flowchart for describing an exemplary flow
25 of a common information acguisition process.
Fig. 94 is a flowchart for describing an exemplary flow
of a decoding process.
Fig. 95 is a flowchart for describing an exemplary flow
of a base layer decoding process.
30 Fig. 96 is a flowchart for describing an exemplary flow
of an enhancement layer decoding process.
17
SP349405WO00
Fig. 97 is a flowchart for describing an exemplary flow
of a prediction process.
Fig. 98 is a diagram illustrating an exemplary syntax
of a slice header.
5 Fig. 99 is a diagram illustrating an exemplary syntax
of a slice header, subsequently to Fig. 98.
Fig. 100 is a diagram illustrating an exemplary syntax
of a slice header, subsequently to Fig. 99.
Fig. 101 is a diagram illustrating another exemplary
10 syntax of a slice header.
Fig. 102 is a diagram illustrating another exemplary
syntax of a slice header, subsequently to Fig. 101.
Fig. 103 is a diagram illustrating another exemplary
syntax of a slice header, subsequently to Fig. 102.
15 Fig. 104 is a block diagram illustrating an exemplary
main configuration of an image coding device.
Fig. 105 is a block diagram illustrating an exemplary
main configuration of a base layer image encoding unit.
Fig. 106 is a block diagram illustrating an exemplary
2 0 main configuration of an enhancement layer image encoding unit.
Fig. 107 is a block diagram illustrating an exemplary
main configuration of an inter predicting unit.
Fig. 108 is a flowchart for describing an exemplary flow
of an image coding process.
25 Fig . 109 is a flowchart for describing an exemplary flow
of a base layer coding process.
Fig. 110 is a flowchart for describing an exemplary flow
of an enhancement layer coding process.
Fig. Ill is a flowchart for describing an exemplary flow
30 of a base layer motion information reference process.
Fig. 112 is a block diagram illustrating an exemplary
18
SP349405WO00
main configuration of an image decoding device.
Fig. 113 is a block diagram illustrating an exemplary
main configuration of a base layer image decoding unit.
Fig. 114 is a block diagram illustrating an exemplary
5 main configuration of an enhancement layer image decoding unit.
Fig. 115 is a block diagram illustrating an exemplary
main configuration of an inter predicting unit.
Fig. 116 is a flowchart for describing an exemplary flow
of an image decoding process.
10 Fig. 117 is a flowchart for describing an exemplary flow
of a base layer decoding process.
Fig. 118 is a flowchart for describing an exemplary flow
of an enhancement layer decoding process.
Fig. 119 is a block diagram illustrating an exemplary
15 main configuration of a computer.
Fig. 120 is a block diagram illustrating an exemplary
schematic configuration of a television device.
Fig. 121 is a block diagram illustrating an exemplary
schematic configuration of a mobile telephone.
20 Fig. 122 is a block diagram illustrating an exemplary
schematic configuration of a recording/reproducing device.
Fig. 123 is a block diagram illustrating an exemplary
schematic configuration of an imaging device.
Fig. 124 is a block diagram illustrating a utilization
25 example of scalable coding.
Fig. 125 is a block diagram illustrating another
utilization example of scalable coding.
Fig. 126 is a block diagram illustrating still another
utilization example of scalable coding.
30 Fig. 127 is a block diagram illustrating an exemplary
schematic configuration of a video set.
19
SP349405WO00
Fig. 128 is a block diagram illustrating an exemplary
schematic configuration of a video processor.
Fig. 129 is a block diagram illustrating another
exemplary schematic configuration of a video processor.
5 Fig. 130 is an explanatory diagram illustrating a
configuration of a content reproducing system.
Fig. 131 is an explanatory diagram illustrating a data
flow in a content reproducing system.
Fig. 132 is an explanatory diagram illustrating a
10 specific example of MPD.
Fig. 133 is a functional block diagram illustrating a
configuration of a content server of a content reproducing
system,
Fig. 134 is a functional block diagram illustrating a
15 configuration of a content reproducing device of a content
reproducing system.
Fig. 135 is a functional block diagram illustrating a
configuration of a content server of a content reproducing
system.
20 Fig. 136 is a seguence chart illustrating an exemplary
communication process performed by devices of a wireless
communication system.
Fig. 137 is a seguence chart illustrating an exemplary
communication process performed by devices of a wireless
25 communication system.
Fig. 138 is a diagram schematically illustrating an
exemplary configuration of a frame format transceived in a
communication process performed by devices of a wireless
communication system.
30 Fig. 139 is a seguence chart illustrating an exemplary
communication process performed by devices of a wireless
20
SP349405WO00
communication system.
MODE FOR CARRYING OUT THE INVENTION
[0041]
5 Hereinafter, modes (hereinafter/ referred to as
"embodiments") of carrying out the present disclosure will
be described. The description will proceed in the following
order.
0. Overview
10 1. First embodiment (image coding device and image
decoding device)
2. Second embodiment (image coding device and image
decoding device)
3. Third embodiment (image coding device and image
15 decoding device)
4. Fourth embodiment (scalable image coding and
decoding)
5. Overview 2 (intra prediction)
6. Fifth embodiment (scalable image coding device)
20 7. Sixth embodiment (scalable image decoding device)
8. Overview 3 (prediction direction control)
9. Seventh embodiment (image coding device)
10. Eighth embodiment (image decoding device)
11. Ninth embodiment (computer)
25 12. Application examples
13. Application examples of scalable coding
14. Set unit module processor
15. Application examples of content reproducing system
of MPEG-DASH
30 16. Application examples of wireless communication
system of Wi-Fi standard
21
SP349405WO00
[0042]
<0. Overview>

Hereinafter/ the present technology will be described
5 in connection with an application to image coding and decoding
of a High Efficiency Video Coding {HEVC} scheme. In image
coding such as Advanced Video Coding (AVC) or HEVC, motion
prediction using a correlation (between frames) in a time
direction is performed.
10 [0043]

In the AVC scheme/ a hierarchical structure based on
a macroblock and a sub macroblock is defined. However/ a
macroblock of 16 * 16 pixels is not optimal for a large image
15 frame such as a Ultra High Definition (UHD: 4000 * 2000 pixels}
serving as a target of a next generation coding scheme.
[0044]
On the other hand/ in the HEVC scheme/ a coding unit
(CU) is defined as illustrated in Fig. 1.
20 [0045]
A CU is also referred to as a coding tree block (CTB) /
and a partial area of an image of a picture unit undertaking
the same role of a macroblock in the AVC scheme. The latter
is fixed to a size of 16 x 16 pixels/ but the former is not
25 fixed to a certain size and designated in image compression
information in each seguence.
[0046]
For example, a largest coding unit (LCD) and a smallest
coding unit (SCU) of a CU are specified in a seguence parameter
30 set (SPS) included in encoded data to be output.
[0047]
22
SP349405WO00
As split-flag = 1 is set in a range in which each LCU
is not smaller than a SCU, a coding unit can be divided into
CUs having a smaller size. In the example of Fig. 1, a size
of an LCU is 128, and a largest scalable depth is 5. A CU
5 of a size of 2N x 2N is divided into CUs having a size of N
x N serving as the hierarchy that is one-level lower when a
value of split_flag is "1."
[0048]
Further, a CU is divided in prediction units (PUs) that
10 are areas (partial areas of an image of a picture unit) serving
as processing units of intra or inter prediction, and divided
into transform units (TUs) that are areas (partial areas of
an image of a picture unit) serving as processing units of
orthogonal transform. Currently, in the HEVC scheme, in
15 addition to 4 x 4 and 8 x 8, orthogonal transform of 16 * 16
and 32 * 32 can be used.
[0049]
In the case of the coding scheme in which a CU is defined,
and various kinds of processes are performed in units of CUs
20 such as the HEVC scheme, in the AVC scheme, a macroblock can
be considered to correspond to an LCU, and a block (sub block)
can be considered to correspond to a CU. Further, in the AVC
scheme, a motion compensation block can be considered to
correspond to a PU. Here, since a CU has a hierarchical
25 structure, a size of an LCU of a topmost layer is commonly
set to be larger thanamacroblock in the AVC scheme, for example,
such as 128 x 128 pixels.
[0050]
Thus, hereinafter, an LCU is assumed to include a
30 macroblock in the AVC scheme, and a CU is assumed to include
ablock (subblock) in the AVC scheme . mother words, a "block"
23
SP349405WO00
used in the following description indicates an arbitrary
partial area in a picture, and, for example/ a size, shape,
and characteristics of a block are not limited. In other words,
a "block" includes an arbitrary area (a processing unit) such
5 as a TU, a PU, an SCU, a CU, an LCU, a sub block, a macroblock,
or a slice. Of course, a "block" includes any other partial
area (processing unit) as well. When it is necessary to limit
a size, a processing unit, or the like, it will be appropriately
described.
10 [0051]

Meanwhile, in the AVC and HEVC coding schemes, in order
to achieve high coding efficiency, it is important to select
an appropriate prediction mode.
15 [0052]
As an example of such a selection method, there is a
method implemented in reference software (opened at
http://iphome.hhi.de/suehring/tml/index.htm) of
H.264/MPEG-4 AVC called a joint model (JM).
20 [0053]
In the JM, it is possible to select two mode determination
methods of a high complexity mode and a low complexity mode,
which will be described below. In both modes, cost function
values related to respective prediction modes are calculated,
25 and a prediction mode having a smaller cost function value
is selected as an optimal mode for a corresponding block or
macroblock.
[0054]
A cost function in the high complexity mode is
30 represented as in the following Formula (1):
[0055]
24
SP349405WO00
[Mathematical Formula 1]
Cost(Mode e n ) = D + A*R ... (1)
[0056]
Here, Q indicates a universal set of a candidate mode
5 for encoding a corresponding block or macroblock, and D
indicates differential energy between a decoded image and an
input image when encoding is performed in a corresponding
prediction mode. A indicates Lagrange's undetermined
multiplier given as a function of a quantization parameter.
10 R indicates a total coding amount including an orthogonal
transform coefficient when encoding is performed in a
corresponding mode.
[0057]
In other words, in order to perform encoding in the high
15 complexity mode/ it is necessary to perform a temporary
encoding process once by all candidate modes in order to
calculate the parameters D and R, and thus a large computation
amount is required.
[0058]
20 A cost function in the low complexity mode is represented
by the following Formula (2):
[0059]
[Mathematical Formula 2]
Cost(Mode G Q) = D + QP2Quant(QP)*HeaderBit ... (2)
25 [0060]
Here, D indicates differential energy between a
prediction image and an input image unlike the high complexity
mode. QP2Quant (QP) is given as a function of a quantization
parameter QP, and HeaderBit indicates a coding amount related
30 to information belonging to a header such as a motion vector
or a mode including no orthogonal transform coefficient.
25
SP349405WO00
[0061]
In other words, in the low complexity mode, it is
necessary to perform a prediction process for respective
candidate modes, but since up to a decoded image is not necessary,
5 it is unnecessary to perform up to a coding process. Thus,
it is possible to implement coding at a computation amount
smaller than that in the high complexity mode.
[0062]

10 Meanwhile, in the HEVC, it is possible to encode and
decode a moving image including a plurality of layers such
as a multi-view image illustrated in Fig. 2. In a multi-view
moving image, each picture includes a plurality of images
having parallax therebetween . An image group (a moving image)
15 of each point of view is referred to as a view (layer) . In
the case of the example of Fig. 2, a moving image includes
3 views of views 0 to 2 . In other words, a picture of a certain
POC includes an image of a view 0, an image of a view 1, and
an image of a view 2. In Fig. 2, the view 0 (view_id = 0)
20 is also referred to as a base view. Further, the view 1 (view id
= 1) and the view 2 (view__id = 2) are also referred to as non-base
views.
[0063]

25 Further, intheHEVC, when an image is encoded or decoded,
motion prediction and compensation processes are used to
improve the coding efficiency, but in order to further improve
the coding efficiency, in the motion prediction and
compensation processes, a motion vector of a current block
30 serving as a processing target is predicted using a motion
vector of a neighboring block, a differential motion vector
26
SP349405WO00
between the motion vector of the current block and a prediction
motion vector thereof is obtained, and the differential motion
vector is transmitted.
[0064]
5 A motion vector is predicted such that a plurality of
candidates are generated from a motion vector of a neighboring
block, and among the plurality of candidates, an optimal
candidate is selected.
[0065]
10 In the HEVC, it is possible to generate a candidate of
a prediction motion vector using a motion vector of a block
spatially neighboring a current block. A spatially
neighboring block refers to a block in the vicinity of a current
block in the same image (an image of the same view of the same
15 POC) as a current block.
[0066]
Further, in the HEVC, it is possible to generate a
candidate of a prediction motion vector using a motion vector
of a block temporally neighboring a current block (Temporal
20 Motion Vector Prediction (TMVP)). A temporally neighboring
block refers to a block (which is also referred to as a "co located
block") (for example, at the same position as) corresponding
to a current block in an image of a different POC of the same
view of a current block.
25 [0067]
Further, when the multi-view image illustrated in Fig.
2 is encoded, in the HEVC, it is possible to generate a candidate
of a prediction motion vector using a motion vector of a block
neighboring a current block in terms of a view (a layer)
30 (Inter-view motion prediction (IVMP)). A block neighboring
in terms of a view (a layer) refers to a block (which is also
27
SP349405WO00
referred to as a "colocated block") (for example, at the same
position as} corresponding to a current block in an image of
a different view of the same POC as a current block. For example,
as illustrated in Fig. 3, a motion vector of an immediately
5 previously processed view is referred to using a correlation
between views.
[0068]
Since it is possible to obtain more candidates of higher
prediction accuracy as described above, it is possible to
10 obtain a predictive vector of higher prediction accuracy, and
it is possible to further improve the coding efficiency.
[0069]
However, in the TMVP and the IVMP, a motion vector other
than an image of a current view of a current picture is referred
15 to. Thus, in the TMVP and the IVMP, it is necessary to hold
a motion vector of each block obtained at the time of encoding
or decoding until it is not referred to (for example, until
motion prediction and compensation processes or a motion
compensation process of an image of a next view ends). To
20 this end, it is necessary to prepare the capacity of a memory
by an assumed maximum value or more.
[0070]
In the case of the HEVC, it is possible to compress a
motion vector to be held for the TMVP and reduce the accuracy
25 of a motion vector. However, a motion vector to be referred
to in the IVMP is used without being compressed. Thus, a motion
vector for the IVMP has to be held without being compressed.
For this reason, a memory that holds a motion vector for the
IVMP is likely to be increased to be larger than a memory that
30 holds a motion vector for the TMVP . In other words , an increase
in a storage capacity necessary for encoding and decoding is
28
SP349405WO00
likely to be suppressed.
[0071]
Fig. 4 illustrates a specific example. As illustrated
in Fig. 4, in the TMVP, a motion vector (a compressed MV) of
5 a current layer (view) is used, and in the IVMP, a motion vector
(anon-compressedMV) of an immediately previous layer (another
layer) is used.
[0072]
Thus, in this case, in addition to a memory (a V0 MV
10 memory and a VI MV memory) storing a motion vector of a 16
x 16 accuracy for the TMVP, a memory storing a motion vector
for the IVMP is necessary. In addition, since the mot ion vector
is not compressed, the capacity larger than that of a memory
storing a motion vector to be referred to in the TMVP is
15 necessary.
[0073]
As the capacity necessary for the memory is increased,
a design or manufacturing cost is likely to be increased.
Further, power consumption and a load are likely to be
20 increased.
[0074]

In this regard, a motion vector for the IVMP is also
compressed {for example, thinned out at a certain thinning
25 rate).
[0075]
For example, in the case of decoding, motion compensation
for decoding of a current layer is performed, and a motion
vector of a current layer that is reconstructed by the motion
30 compensation and used in motion compensation for decoding of
another layer is compressed (for example, thinned out at a
29
SP349405WO00
certain thinning rate) . In other words, motion compensation
for decoding of a current layer is performed using a motion
vector that is reconstructed and compressed (for example/
thinned out at a certain thinning rate) in motion compensation
5 for decoding of another layer,
[0076]
Further, for example, in the case of encoding, motion
prediction and compensation for encoding of a current layer
are performed, and a motion vector of a current layer that
10 is generated by the motion prediction and compensation and
used in motion prediction and compensation for encoding of
another layer is compressed (for example, thinned out at a
certain thinning rate). In other words, motion prediction
and compensation for encoding of a current layer are performed
15 using a motion vector that is generated and compressed (for
example, thinned out at a certain thinning rate) in motion
prediction and compensation for encoding of another layer.
[0077]
As a result, it is possible to suppress an increase in
2 0 the capacity of a memory necessary for storing the motion vector
for the IVMP. In other words, it is possible to suppress an
increase in a storage capacity necessary for encoding and
decoding.
[0078]
25 <1. First embodiment>

In the motion vector compression, as illustrated in Fig .
5, a motion vector for the IVMP may be compressed (for example,
thinned out at the same thinning rate as the motion vector
30 for the TMVP) at the same compression rate as the motion vector
for the TMVP, In other words, a motion vector for the IVMP
30
SP349405WO00
and a motion vector for the TMVP may have the same accuracy
through the compression. In this case, it is possible to
commonalize the motion vector for the TMVP and the motion vector
for the IVMP, and it is possible to suppress an increase in
5 the capacity of a necessary memory.
[0079]
For example, in the case of the example of Fig. 4, a
decoder (a V0 (Base) decoder, a VI (Depend) decoder, and a
V2 (Depend) decoder) reconstructs a motion vector of a maximum
10 of 4 x 4 accuracy through the decoding, and a motion vector
compressing unit ((V0 to V2) MV compressors) performs
compression at a 16 x 16 accuracy. Thus, a memory ( (V0 to
V2) temporal MV memories) storing the motion vector for the
IVMP needs the capacity capable of storing a motion vector
15 of a 4 x 4 accuracy by one screen.
[0080]
As in the example illustrated in Fig. 5, as the motion
vector for the IVMP is also compressed (for example, thinned
out) at a 16 x 16 accuracy by a motion vector compressing unit
20 ( (V0 to V2) MV compressors) , it is possible to omit a memory
( (V0 to V2) temporal MV memories) capable of storing a motion
vector of a 4 x 4 accuracy by one screen.
[0081]
It is possible to commonalize the compression process
25 (for example, the thinning process) through the
commonalization of the motion vector, and thus it is possible
to suppress an increase in a load caused by the compression
process for the motion vector.
[0082]
30 The compression method will be more specifically
described below.
31
SP349405WO00
[0083]
In the following, a moving image of an encoding or
decoding target is assumed to be a moving image of 3 views
(layers) of views 0 to 2 as illustrated in Fig. 2. Further,
5 pictures of the moving image are processed in a certain order,
and in each picture, an image of a view 0, an image of a view
1, and an image of a view 2 are assumed to be seguentially
processed.
[0084]
10
Fig. 6 is a block diagram illustrating an exemplary main
configuration of a V0 image coding device. Fig. 7 is a block
diagram illustrating an exemplary main configuration of a VI
image coding device. Fig. 8 is a block diagram illustrating
15 an exemplary main configuration of a V2 image coding device.
[0085]
An image coding device 100 {not illustrated) encodes
a moving image including a plurality of layers such as the
multi-view image illustrated in Fig. 2. The image coding
20 device 100 includes a V0 image coding device 100-0 of Fig.
6, a VI image coding device 100-1 of Fig. 7, and a V2 image
coding device 100-3 of Fig. 8 in order to encode views of a
multi-view image. The V0 image coding device 100-0 encodes
an image of a view 0 . The VI image coding device 100-1 encodes
25 an image of a view 1. The V2 image coding device 100-2 encodes
an image of a view 2.
[0086]
The V0 image coding device 100-0 includes an A/D
converting unit 101-0, a screen sorting buffer 102-0, an
30 operation unit 103-0, an orthogonal transforming unit 104-0,
a guantizing unit 105-0, a lossless encoding unit 106-0, an
32
SP349405WO00
accumulation buffer 107-0, an inverse quantizing unit 108-0,
and an inverse orthogonal transforming unit 109-0 as
illustrated in Fig. 6. The V0 image coding device 100-0 further
includesan operation unit 110-0, a loop filter 111-0, a decoded
5 picture buffer 112-0, an intra predicting unit 113-0, amotion
predicting/compensating unit 114-0, a prediction image
selecting unit 115-0, and a rate control unit 116-0.
[0087]
The VI image coding device 100-1 includes an A/D
10 converting unit 101-1, a screen sorting buffer 102-1, an
operation unit 103-1, an orthogonal transforming unit 104-1,
a quantizing unit 105-1, a lossless encoding unit 106-1, an
accumulation buffer 107-1, an inverse quantizing unit 108-1,
and an inverse orthogonal transforming unit 109-1 as
15 illustrated in Fig. 7 . The VI image coding device 100-1 further
includes an operation unit 110-1, a loop filter 111-1, a decoded
picture buffer 112-1, an intra predicting unit 113-1, amotion
predicting/compensating unit 114-1, a prediction image
selecting unit 115-1, and a rate control unit 116-1.
20 [0088]
The V2 image coding device 100-2 includes an A/D
converting unit 101-2, a screen sorting buffer 102-2, an
operation unit 103-2, an orthogonal transforming unit 104-2,
a quantizing unit 105-2, a lossless encoding unit 106-2, an
25 accumulation buffer 107-2, an inverse quantizing unit 108-2,
and an inverse orthogonal transforming unit 109-2 as
illustrated in Fig. 8 . The V2 image coding device 100-2 further
includes an operation unit 110-2, a loop filter 111-2, a decoded
picture buffer 112-2, an intra predicting unit 113-2, amotion
30 predicting/compensating unit 114-2, a prediction image
selecting unit 115-2, and a rate control unit 116-2.
33
SP349405WO00
[0089]
Hereinafter, when theA/Dconvertingunits 101-0 to 101-2
need not be distinguished from one another, they are referred
to simply as an "A/D converting unit 101. " Further, when the
5 screen sorting buff ers 102-0 to 102-2 need not be distinguished
from one another, they are referred to simply as a "screen
sorting buffer 102 . " Further, when the operation units 103-0
to 103-2 need not be distinguished from one another, they are
referred to simply as an "operation unit 103. " Further, when
10 the orthogonal transforming units 104-0 to 104-2 need not be
distinguished from one another, they are referred to simply
as an "orthogonal transforming unit 104." Further, when the
guantizing units 105-0 to 105-2 need not be distinguished from
one another, they are referred to simply as a "guantizing unit
15 105 . " Further, when the lossless encoding units 106-0 to 106-2
need not be distinguished from one another, they are referred
to simply as a "lossless encoding unit 106." Further, when
the accumulation buffers 107-0 to 107-2 need not be
distinguished from one another, they are referred to simply
20 as an "accumulation buffer 107." Further, when the inverse
guantizing units 108-0 to 108-2 need not be distinguished from
one another, they are referred to simply as an "inverse
guantizing unit 108." Further, when the inverse orthogonal
transforming units 109-0 to 109-2 need not be distinguished
25 from one another, they are referred to simply as an "inverse
orthogonal transforming unit 109."
[0090]
Further, when the operation units 110-0 to 110-2 need
not be distinguished from one another, they are referred to
30 simply as an "operation unit 110." Further, when the loop
filters 111-0 to 111-2 need not be distinguished from one
34
SP349405WO00
another, they are referred to simply as a "loop filter 111."
Further, when the decoded picture buffers 112-0 to 112-2 need
not be distinguished from one another, they are referred to
simply as a "decoded picture buffer 112." Further, when the
5 intra predicting units 113-0 to 113-2 need not be distinguished
from one another, they are referred to simply as an "intra
predicting unit 113." Further, when the motion
predicting/compensating units 114-0 to 114-2 need not be
distinguished from one another, they are referred to simply
10 as a "motion predicting/compensating unit 114." Further,
when the prediction image selecting units 115-0 to 115-2 need
not be distinguished from one another, they are referred to
simply as a "prediction image selecting unit 115." Further,
when the rate control units 116-0 to 116-2 need not be
15 distinguished from one another, they are referred to simply
as a "rate control unit 116."
[0091]
The V0 image coding device 100-0 further includes a V0
motion vector compressing unit 121-0 and a V0 motion vector
20 memory 122-0. The VI image coding device 100-1 further
includes a VI motion vector compressing unit 121-1 and a VI
motion vector memory 122-1. The V2 image coding device 100-2
further includes a V2 motion vector compressing unit 121-2
and a V2 motion vector memory 122-2.
25 [0092]
The A/D converting unit 101 performs A/D conversion on
input image data (image data of the viev/ 0 in the case of the
A/D converting unit 101-0, image data of the view 1 in the
case of the A/D converting unit 101-1, and the image data of
30 the viev/ 2 in the case of the A/D converting unit 101-2) , and
supplies converted image data (digital data) to be stored in
35
SP349405WO00
the screen sorting buffer 102.
[0093]
The screen sorting buffer 102 performs sorting on a
stored image of frames arranged in a display order in a frame
5 order for encoding according to a group of picture (GOP) . In
other words, the screen sorting buffer 102 sorts pictures
arranged in a POC order in a processing order. The screen
sorting buffer 102 supplies the image in which a frame order
is sorted to the operation unit 103 . The screen sorting buffer
10 102 also supplies the image in which a frame order is sorted
to the intra predicting unit 113 and the motion
predicting/compensating unit 114.
[0094]
The operation unit 103 subtracts a prediction image
15 supplied from the intra predicting unit 113 or the motion
predicting/compensating unit 114 via the prediction image
selecting unit 115 from an image read from the screen sorting
buffer 102, and outputs differential information thereof to
the orthogonal transforming unit 104, For example, in the
2 0 case of an image on which intra coding is performed, the
operation unit 103 subtracts a prediction image supplied from
the intra predicting unit 113 from an image read from the screen
sorting buffer 102. Further, for example, in the case of an
image on which inter coding is performed, the operation unit
25 103 subtracts a prediction image supplied from the motion
predicting/compensating unit 114 from an image read from the
screen sorting buffer 102.
[0095]
The orthogonal transforming unit 104 performs
30 orthogonal transform such as discrete cosine transform or
Karhunen Loeve transform on the differential information
36
SP349405WO00
supplied from the operation unit 103. The orthogonal
transforming unit 104 supplies transform coefficients to the
quantizing unit 105.
[0096]
5 The quantizing unit 105 guantizes the transform
coefficients supplied from the orthogonal transforming unit
104. The quantizing unit 105 sets a guantization parameter
based on information related to a target value of a coding
amount supplied from the rate control unit 116, and performs
10 the guantization. The quantizing unit 105 supplies the
quantized transf ormcoef f icients to the lossless encodingunit
106.
[0097]
The lossless encoding unit 106 encodes the transform
15 coefficients quantized by the guantizing unit 105 according
to an arbitrary coding scheme. Since coefficient data is
quantized under control of the rate control unit 116, the coding
amount becomes the target value set by the rate control unit
116 (or approximates to the target value).
20 [0098]
The lossless encoding unit 106 acquires, for example,
information indicating an intra (intra-screen) prediction
mode from the intra predicting unit 113, and acguires, for
example, information indicating an inter (inter-screen)
25 prediction mode and differential motion vector information
from the motion predicting/compensating unit 114.
[0099]
The lossless encoding unit 106 encodes various kinds
of information according to an arbitrary coding scheme, and
30 sets (multiplexes) them as part of header information of
encoded data (vzhich is also referred to as an "encoded stream") .
37
SP349405WO00
The lossless encoding unit 106 supplies the encoded data
obtained by the encoding to be accumulated in the accumulation
buffer 107.
[0100]
5 Examples of the coding scheme of the lossless encoding
unit 106 include variable length coding or arithmetic coding.
As the variable length coding, for example, there is
context-adaptive variable length coding (CAVLC) defined in
the H. 264/AVC scheme . As the arithmetic coding, for example,
10 there is context-adaptive binary arithmetic coding (CABAC) .
[0101]
The accumulation buffer 107 temporarily holds the
encoded data supplied from the lossless encoding unit 106.
The accumulation buffer 107 outputs the held encoded data to,
15 for example, a recording device (not illustrated) (a recording
medium) at a subsequent stage or a transmission path at a certain
timing. In other words, the accumulation buffer 107 also
serves as a transmitting unit that transmits the encoded data.
[0102]
20 The transform coefficients quantized by the quantizing
unit 105 are also supplied to the inverse quantizing unit 108 .
The inverse quantizing unit 108 inversely quantizes the
quantized transform coefficients by a method corresponding
to the quantization performed by the quantizing unit 105 . The
25 inverse quantizing unit 108 supplies the obtained transform
coefficients to the inverse orthogonal transforming unit 109.
[0103]
The inverse orthogonal transforming unit 109 performs
inverse orthogonal transform on the transform coefficients
30 supplied from the inverse quantizing unit 108 by a method
corresponding to the orthogonal transform process performed
38
SP349405WO00
by the orthogonal transforming unit 104 . An output {restored
differential information) that has been subjected to the
inverse orthogonal transform is supplied to the operation unit
110.
5 [0104]
The operation unit 110 obtains a locally decoded image
{decoded image) by adding the prediction image received from
the intra predicting unit 113 or the motion
predicting/compensating unit 114 via the prediction image
10 selecting unit 115 to the restored differential information
that is the inverse orthogonal transform result supplied from
the inverse orthogonal transforming unit 109. The decoded
image is supplied to the loop filter 111.
[0105]
15 The loop filter 111 includes a deblock filter, an
adaptive loop filter, or the like/ and performs an appropriate
filter process on the decoded image supplied from the operation
unit 110 . For example/ the loop filter 111 performs the deblock
filter process on the decoded image, and removes block
20 distortion of the decoded image. Further, for example, the
loop filter 111 performs the loop filter process on the deblock
filter process result {the decoded image from which the block
distortion has been removed) using the Wiener Filter, and
improves the image quality.
25 [0106]
The loop filter 111 may performs an arbitrary filter
process on the decoded image. The loop filter 111 may supply
information used in the filter process such as a filter
coefficient to the lossless encoding unit 106 as necessary
30 so that the information is encoded.
[0107]
39
SP349405WO00
The loop filter 111 supplies the filter process result
(hereinafter/ referred to as a "decoded image") to the decoded
picture buffer 112. The loop filter 111 supplies the
reconstructed image supplied from the operation unit 110 to
5 the intra predicting unit 113.
[0108]
The decoded picture buffer 112 stores the decoded image
supplied from the loop filter 111. Further, the decoded
picture buffer 112 stores a view ID and a POC of the image.
10 [0109]
The decoded picture buffer 112 supplies the stored
decoded image (and the view ID and the POC of the image) to
the motion predicting/compensating unit 114 for a
corresponding view at a certain timing or based on a request
15 given from the outside such as the motion
predicting/compensating unit 114.
[0110]
Further, the decoded picture buffer 112 also supplies
the stored decoded image (and the view ID and the POC of the
20 image) to the motion predicting/compensating unit 114 for a
next processed view at a certain timing or based on a request
given from the outside such as the motion
predicting/compensating unit 114. For example, the decoded
picture buffer 112-0 supplies the decoded image to the motion
25 predicting/compensating unit 114-0 and the motion
predicting/compensating unit 114-1. Further, for example,
the decoded picture buffer 112-1 supplies the decoded image
to the motion predicting/compensating unit 114-1 and the mot ion
predicting/compensating unit 114-2. Here, since the view 2
30 is a last processed view, the decoded picture buffer 112-2
supplies the decoded image to the motion
40
SP349405WO00
predicting/compensating unit 114-2.
[0111]
Upon acquisition of an image of a neighboring area (a
neighboring block) positioned around a processing target area
5 (acurrentblock) from the loop filter 111, the intra predicting
unit 113 performs intra prediction of generating a prediction
image basically using a prediction unit (PU) as a processing
unit using pixel values of the image of the neighboring block.
The intra predicting unit 113 performs the intra prediction
10 in a plurality of modes {intra prediction modes) that are
prepared in advance.
[0112]
The intra predicting unit 113 generates prediction
images in all intra prediction modes serving as the candidate,
15 evaluates cost function values of the prediction images using
the input image supplied from the screen sorting buffer 102,
and selects an optimal mode . When the optimal intra prediction
mode is selected, the intra predicting unit 113 supplies the
prediction image generated in the optimal mode to the
20 prediction image selecting unit 115.
[0113]
Further, the intra predicting unit 113 appropriately
supplies intra prediction information including information
related to intra prediction such as an optimal intra prediction
25 mode to the lossless encoding unit 106 so that the intra
prediction information is encoded.
[0114]
The motion predicting/compensating unit 114 performs
motion prediction on a current block to generate amotion vector,
30 and performs a compensation process according to the generated
motion vector to generate a prediction image (inter prediction
41
SP349405WO00
image information) of the current block. In the motion
prediction and compensation processes, the motion
predicting/compensating unit 114 generates a prediction image
in all inter prediction modes serving as a candidate, evaluates
5 a cost function value of each prediction image, and selects
an optimal mode. When the optimal inter prediction mode is
selected, the motion predicting/compensating unit 114
supplies the prediction image generated in the optimal mode
to the prediction image selecting unit 115.
10 [0115]
The motion predicting/compensating unit 114 performs
the above motion prediction and compensation processes using
the decoded image acguired from the decoded picture buffer
112.
15 [0116]
In the case of an initially processed view, the motion
predicting/compensating unit 114 performs the motion
prediction and compensation processes using the decoded image
acguired from the decoded picture buffer 112 storing an image
20 of a corresponding view. For example, the motion
predicting/compensating unit 114-0 performs the motion
prediction and compensation processes using the decoded image
acguired from the decoded picture buffer 112-0.
[0117]
2 5 In the case of a secondly or subseguently processed view,
the motion predicting/compensating unit 114 performs the
motion prediction and compensation processes using the decoded
images that are acguired from the decoded picture buffer 112
storing an image of a corresponding view and the decoded picture
30 buffer 112 storing an image of an immediately previously
processed view. For example, the motion
42
SP349405WO00
predicting/compensating unit 114-1 performs the motion
prediction and compensation processes using the decoded images
acquired from the decoded picture buffer 112-0 and the decoded
picture buffer 112-1. Further, for example, the motion
5 predicting/compensating unit 114-2 performs the motion
prediction and compensation processes using the decoded images
acquired from the decoded picture buffer 112-1 and the decoded
picture buffer 112-2.
[0118]
10 Further, when the inter prediction is employed, the
motion predicting/compensating unit 114 supplies inter
prediction information including information related to inter
prediction such as an optimal inter prediction mode to the
lossless encoding unit 106 so that the inter prediction
15 information is transmitted.
[0119]
The motion predicting/compensating unit 114 transmits
a motion vector of an optimal mode as the inter prediction
information so that the motion compensation is performed at
20 the decoding side. Practically, the motion
predicting/compensating unit 114 transmits a differential
motion vector serving as a difference between a motion vector
and a prediction motion vector instead of a motion vector in
order to further improve the coding efficiency. In other words,
25 the motion predicting/compensating unit 114 predicts a motion
vector of a current block, generates a prediction motion vector,
and generates a differential motion vector serving as a
difference between the motion vector of the current block and
the prediction motion vector. The motion
30 predicting/compensating unit 114 transmits the differential
motion vector to the decoding side as part of the inter
43
SP349405WO00
prediction information.
[0120]
In the prediction of the motion vector, the motion
predicting/compensating unit 114 generates a prediction
5 motion vector of a current block using a motion vector of a
block neighboring a current block. At this time, in order
to further improve the prediction accuracy of the prediction
motion vector and to further improve the coding efficiency,
the motion predicting/compensating unit 114 can generate a
10 plurality of prediction motion vector candidates, obtain the
cost function values for the respective candidates, and select
an optimal candidate from among the candidates based on the
cost function values. In other words, the motion
predicting/compensating unit 114 can generate the prediction
15 motion vector candidates by a plurality of methods.
[0121]
For example, the motion predicting/compensating unit
114 can generate a prediction motion vector of a current block
with reference to a motion vector of a block neighboring a
20 current block spatially, temporally, and in terms of a view.
Further, the motion predicting/compensating unit 114
transmits information (that is, mode information of a
prediction motion vector) indicating a candidate selected as
a prediction motion vector to the decoding side as part of
25 the inter prediction information.
[0122]
The motion predicting/compensating unit 114 acquires
a motion vector of a spatially neighboring block from a motion
vector memory corresponding to a corresponding view. For
30 example, the motion predicting/compensating unit 114-0
acquires amotion vector from the V0 motion vector memory 122-0 .
44
SP349405WO00
Further, forexarnple, the motion predicting/compensating unit
114-1 acguires a motion vector from the VI motion vector memory
122-1. Furthermore, for example, the motion
predicting/compensating unit 114-2 acguires a motion vector
5 from the V2 motion vector memory 122-2.
[0123]
In a secondly or subseguently processed view, the motion
predicting/compensating unit 114 also acguires amotion vector
of a block neighboring in terms of a view from a motion vector
10 memory corresponding to a previously processed view. For
example, the motion predicting/compensating unit 114-1
acquires amotion vector from the VO motion vector memory 122-0.
Further, for example, the motion predicting/compensating unit
114-2 acguires a motion vector from the VI motion vector memory
15 122-1.
[0124]
The motion vector is compressed. In other words, motion
prediction and compensation for encoding of a current layer
is performed using a motion vector that is generated and
20 compressed in motion prediction and compensation for encoding
of another layer. In other words, it is possible to suppress
an increase in the storage capacity necessary for encoding.
[0125]
The motion predicting/compensating unit 114-0 supplies
25 amotion vector of a current block (amotion vector of an optimal
mode} generated in the motion prediction and compensation
processes to the V0 motion vector compressing unit 121-0.
Further, the motion predicting/compensating unit 114-1
supplies the generated motion vector of the current block to
30 the VI motion vector compressing unit 121-1. Furthermore,
the motion predicting/compensating unit 114-2 supplies the
45
SP349405WO00
generated motion vector of the current block to the V2 motion
vector compressing unit 121-2.
[0126]
The prediction image selecting unit 115 selects a supply
5 source of a prediction image to be supplied to the operation
unit 103 and the operation unit 110 . For example, in the case
of the intra coding, the prediction image selecting unit 115
selects the intra predicting unit 113 as the supply source
of the prediction image, and supplies the prediction image
10 supplied from the intra predicting unit 113 to the operation
unit 103 and the operation unit 110. Further, for example,
in the case of the inter coding, the prediction image selecting
unit 115 selects the motion predicting/compensating unit 114
as the supply source of the prediction image, and supplies
15 the prediction image supplied from the motion
predicting/compensating unit 114 to the operation unit 103
and the operation unit 110.
[0127]
The rate control unit 116 controls a rate of a
20 quantization operation of the quantizing unit 105 based on
the coding amount of the encoded data accumulated in the
accumulation buffer 107 such that neither an overflow nor an
underflow occurs.
[0128]
25 The V0 motion vector compressing unit 121-0 performs
compression (which is also referred as "1/16 compression")
on a motion vector of a maximum of 4 x 4 accuracy (which is
also referred to as a "non-compressed V0 motion vector")
acquired from the motion predicting/compensating unit 114-0
30 with the 16 * 16 accuracy, and supplies the compressed motion
vector (which is also referred to as a "1/16-compressed V0
46
SP349405WO00
motion vector") to the VO motion vector memory 122-0.
[0129]
A method of compressing a motion vector is arbitrary.
For example, the VO motion vector compressing unit 121-0 may
5 select a motion vector serving as a representative value from
among a plurality of motion vectors acquired from the motion
predicting/compensating unit 114-0. For example, one motion
vector serving as a representative value may be selected from
among 16 motion vectors (motion vectors of 4 * 4 blocks) of
10 the 4 x 4 accuracy. Through this compression, the accuracy
of the motion vector becomes the 16 x 16 accuracy.
[0130]
Further, a method of selecting the motion vector is
arbitrary . Amotion vector of a block at a position determined
15 by a certain method may be selected, and for example, a motion
vector of a block at a certain position such as a block at
an upper left end may be selected, and a block corresponding
to a position in an image may be selected.
[0131]
20 The number of selected motion vectors is arbitrary and
may be 2 or more.
[0132]
Further, for example, the V0 motion vector compressing
unit 121-0 calculates a representative value through a certain
25 operation using each motion vector. A method of calculating
a representative value is arbitrary. For example, an average
value or a median value of motion vectors of respective blocks
may be used as a representative value. Further, the number
of calculated representative values is arbitrary and may be
30 2 or more.
[0133]
47
SP349405WO00
The 1/16-compressed VO motion vector (the
representative value of the motion vector) obtained as
described above is supplied to and stored in the VO motion
vector memory 122-0. The VO motion vector memory 122-0
5 appropriately supplies the stored 1/16-compressed VO motion
vector to the motion predicting/compensating unit 114-0 as
a motion vector of a temporally neighboring block. Further,
the VO motion vector memory 122-0 appropriately supplies the
stored 1/16-compressed VO motion vector to the motion
10 predicting/compensating unit 114-1 as a motion vector of a
block neighboring in terms of a view.
[0134]
The VI motion vector compressing unit 121-1 performs
1/16 compression on amotion vector of a maximum of 4x4 accuracy
15 (which is also referred to as a "non-compressed Vl motion
vector") acquired from the motion predicting/compensating
unit 114-1, and supplies the compressed motion vector (which
is also referred to as a "1/16-compressed Vl motion vector")
to be stored in the Vl motion vector memory 122-1. The Vl
20 motion vector memory 122-1 appropriately supplies the stored
1/16-compressed Vl motion vector to the motion
predicting/compensating unit 114-1 as a motion vector of a
temporally neighboring block. Further, the Vl motion vector
memory 122-1 appropriately supplies the stored
25 1/16-compressed Vl motion vector to the motion
predicting/compensating unit 114-2 as a motion vector of a
block neighboring in terms of a view.
[0135]
The V2 motion vector compressing unit 121-2 performs
30 1/16 compression on amotion vector of a maximumof 4x4 accuracy
(which is also referred to as a "non-compressed V2 motion
48
SP349405WO00
vector") acquired from the motion predicting/compensating
unit 114-2, and supplies the compressed motion vector (which
is also referred to as a "1/16-compressed V2 motion vector")
to be stored in the V2 motion vector memory 122-2. The V2
5 motion vector memory 122-2 appropriately supplies the stored
1/16-compressed V2 motion vector to the motion
predicting/compensating unit 114-2 as a motion vector of a
temporally neighboring block.
[0136]
10 Further, a method of compressing a motion vector through
the VI motion vector compressing unit 121-1 and the V2 motion
vector compressing unit 121-2 is the same as in the VO motion
vector compressing unit 121-0, and thus a description thereof
is omitted.
15 [0137]
The VO motion vector compressing unit 121-0, the VI
motion vector compressing unit 121-1, and the V2 motion vector
compressing unit 121-2 perform the above motion vector
compression in certain units. For example, the V0 motion
20 vector compressing unit 121-0, the VI motion vector compressing
unit 121-1, and the V2 motion vector compressing unit 121-2
may perform the above motion vector compression in units of
LCUs. The V0 motion vector compressing unit 121-0, the VI
motion vector compressing unit 121-1, and the V2 motion vector
25 compressing unit 121-2 may be the same or differ in the
processing unit. The processing unit may be changed during
a sequence.
[0138]
Further, the motion vector compression methods
30 performed by the V0 motion vector compressing unit 121-0, the
VI motion vector compressing unit 121-1, and the V2 motion
49
SP349405WO00
vector compressing unit 121-2 may be the same as or different
from one another.
[0139]
As described above, the VO motion vector compressing
5 unit 121-0, the VI motion vector compressing unit 121-1, and
the V2 motion vector compressing unit 121-2 can reduce (that
is, compress) an information amount of motion vectors by
reducing the number of motion vectors. Thus, it is possible
to reduce the capacities of the VO motion vector memory 122-0,
10 the VI motion vector memory 122-1, and the V2 motion vector
memory 122-2.
[0140]
Further, as described above, the motion
predicting/compensating unit 114 may refer to the motion
15 vectors stored in the V0 motion vector memory 122-0, the Vl
motion vector memory 122-1, and the V2 motion vector memory
122-2 as the motion vector for the IVMP as well as the motion
vector for the TMVP. As the motion vector for the TMVP and
the motion vector for the IVMP are commonalized as described
20 above, it is possible to reduce the storage capacity necessary
for encoding . Further, it is possible to suppress an increase
in a load caused by compression of amotion vector. Accordingly,
it is possible to implement a reduction in a manufacturing
or developing cost, device downsizing, a reduction in power
25 consumption, and the like for the image coding device 100.
[0141]

Next, a flow of processing performed by the image coding
device 100 will be described. An exemplary flow of a coding
30 process performed by the image coding device 100 will be
described with reference to a flowchart of Fig. 9.
50
SP349405WO00
[0142]
When the coding process starts, in step S1001, the V0
image coding device 100-0 performs a V0 coding process on a
current picture. In step S1002, the VI image coding device
5 100-1 performs a VI coding process on the current picture.
In step S1003, the V2 image coding device 100-2 performs a
V2 coding process on the current picture.
[0143]
In step S1004, the image coding device 100 determines
10 whether or not all pictures have been processed, and v/hen it
is determined that there is a non-processed picture, the
process returns to step S1001, and the subsequent process is
repeated.
[0144]
15 The process of steps S1001 to S1004 is repeatedly
performed for each picture, and v;hen it is determined in step
S1004 that all pictures have been processed, the image coding
device 100 ends the coding process.
[0145]
20
Next, an exemplary flow of the V0 coding process of
encoding the view 0 which is performed in step S1001 of Fig.
9 will be described with reference to a flowchart of Fig. 10.
[0146]
25 In step S1101, the A/D converting unit 101-0 performs
A/D conversion on an input image. In step S1102, the screen
sorting buffer 102-0 stores the A/D converted image, and sorts
respective pictures arranged in a display order in an encoding
order . In step SI 10 3, the intra predicting unit 113-0 performs
30 the intra prediction process of the intra prediction mode.

CLAIMS
1, An image processing device, comprising:
a motion compensating unit that performs motion
5 compensation in decoding of a current layer; and
a first compressing unit that compresses a motion vector
of the current layer that is reconstructed by the motion
compensating unit and used for the motion compensation in
decoding of another layer.
10
2 . The image processing device according to claim 1, further
comprising
a second compressing unit that compresses the motion
vector of the current layer reconstructed by the motion
15 compensating unit at a compression rate higher than a
compression rate of the first compressing unit,
wherein the motion compensating unit performs the motion
compensation in the decoding of the current layer using the
motion vector compressed by the second compressing unit.
20
3. The image processing device according to claim 2,
wherein the second compressing unit further compresses
the motion vector compressed by the first compressing unit,
25 4, The image processing device according to claim 1,
wherein the motion compensating unit performs the motion
compensation in the decoding of the current layer using the
motion vector compressed by the first compressing unit.
30 5 . The imageprocessingdeviceaccording to claim 1, further
comprising:
355
SP349405WO00
a receiving unit that receives a flag indicating whether
or not the motion vector of the current layer used in the motion
compensation in the decoding of the other layer; and
a selecting unit that selects the motion vector
5 compressed by the first compressing unit as the motion vector
of the current layer used in the motion compensation in the
decoding of the other layer when the flag received through
the receiving unit indicates that the motion vector is
compressed, and selects the motion vector that is not
10 compressed by the first compressing unit as the motion vector
of the current layer used in the motion compensation in the
decoding of the other layer when the flag received through
the receiving unit indicates that the motion vector is not
compressed.
15
6. The image processing device according to claim 5,
wherein themotion compensating unitperforms themotion
compensation in the decoding of the current layer using the
motion vector compressed by the first compressing unit
20 regardless of a value of the flag received through the receiving
unit.
7. The image processing device according to claim 1,
wherein the first compressing unit selects a motion
25 vector serving as a representative value from among a plurality
of motion vectors reconstructed by the motion compensating
unit, and compresses the motion vector of the current layer.
8. The image processing device according to claim 1,
30 wherein the first compressing unit calculates a motion
vector serving as a representative value using a plurality
356
SP349405WO00
of motion vectors reconstructed by the motion compensating
unit, and compresses the motion vector of the current layer.
9. The image processing device according to claim 1,
5 wherein the motion compensating unit performs the motion
compensation using a motion vector reconstructed in the motion
compensation in the decoding of the other layer.
10 . An image processing method of an image processing device,
10 comprising:
performing, by the image processing device, motion
compensation in decoding of a current layer;
compressing, by the image processing device, a motion
vector of the current layer that is reconstructed by the motion
15 compensation and used in the motion compensation in decoding
of another layer.
11. An image processing device, comprising:
a motion predicting/compensating unit that performs
20 motion prediction and compensation in encoding of a current
layer; and
a first compressing unit that compresses a motion vector
of the current layer that is generated by the motion
predicting/compensating unit and used in the motion prediction
25 and compensation in encoding of another layer.
12. The image processing device according to claim 11,
further comprising
a second compressing unit that compresses the motion
30 vector of the current layer generated by the motion
predicting/compensating unit at a compression rate higher than
357
SP349405WO00
a compression rate of the first compressing unit,
wherein the motion predicting/compensating unit
performs the motion prediction and compensation in the encoding
of the current layer using the motion vector compressed by
5 the second compressing unit.
13. The image processing device according to claim 12,
wherein the second compressing unit further compresses
the motion vector compressed by the first compressing unit.
10
14. The image processing device according to claim 11,
wherein the motion predicting/compensating unit
performs the motion prediction and compensation in the encoding
of the current layer using the motion vector compressed by
15 the first compressing unit.
15. The image processing device according to claim 11,
further comprising:
a control unit that controls whether or not the motion
20 vector of the current layer used in the motion prediction and
compensation in the encoding of the other layer is compressed;
a selecting unit that selects any one of the motion vector
that is not compressed by the first compressing unit and the
motion vector compressed by the first compressing unit as the
25 motion vector of the current layer used in the motion prediction
and compensation in the encoding of the other layer according
to control of the control unit;
a generating unit that generates a flag indicating
whether or not the motion vector of the current layer used
30 in the motion prediction and compensation in the encoding of
the other layer is compressed according to control of the
control unit; and
a transmitting unit that transmits the flag generated
by the generating unit.
5 16. The image processing device according to claim 15,
wherein the motion predicting/compensating unit
performs the motion prediction and compensation in the encoding
of the current layer using the motion vector compressed by
the first compressing unit regardless of control of the control
10 unit.
17. The image processing device according to claim 11,
wherein the first compressing unit selects a motion
vector serving as a representative value from among a plurality
15 of motion vectors generated by the motion
predicting/compensating unit, and compresses the motion
vector of the current layer.
18. The image processing device according to claim 11,
20 wherein the first compressing unit calculates amotion
vector serving as a representative value using a plurality
of motion vectors generated by the motion
predicting/compensating unit, and compresses the motion
vector of the current layer.
25
19. The image processing device according to claim 11,
wherein the motion predicting/compensating unit
performs the motion prediction and compensation using amotion
vector generated in the motion prediction and compensation
3 0 in the encoding of the other layer.
20. Animageprocessingmethodof an image processing device,
comprising:
performing/ by the image processing device, motion
prediction and compensation in encoding of a current layer;
5 and
compressing/ by the image processing device, a motion
vector of the current layer that is generated by the motion
prediction and compensation and used in the motion prediction
and compensation in encoding of another layer.