r
I 1
,^ SP317970WO00
\ DESCRIPTION
1
IMAGE PROCESSING DEVICE AND METHOD
TECHNICAL FIELD
I 5 [0001]
This disclosure relates to image processing devices
and methods, and more particularly, to an image
processing device and method that can increase encoding
efficiency.
10
BACKGROUND ART
[0002]
In recent years, to handle image information as
digital information and achieve high-efficiency
15 information transmission and accumulation in doing do,
apparatuses compliant with a standard, such as MPEG
(Moving Picture Experts Group) for compressing image
information through orthogonal transforms such as
discrete cosine transforms and motion compensation by
20 using redundancy inherent to image information, have been
spreading among broadcast stations to distribute
information and among general households to receive
information.
[0003]
25 Particularly, MPEG2 (ISO (International
Organization for Standardization)/lEC (International
Electrotechnical Commission) 13818-2) is defined as a
general-purpose image encoding standard, and is
applicable to interlaced images and non-interlaced images,
30 and to standard-resolution images and high-definition
images. Currently, MPEG2 is used in a wide range of
! 1
i
! 2
•
SP317970WO00
I applications for professionals and general consumers.
According to the MPEG2 compression method, a bit rate of
4 to 8 Mbps is assigned to an interlaced image having a
standard resolution of 720 x 480 pixels, and a bit rate
5 of 18 to 22 Mbps is assigned to an interlaced image
having a high-resolution of 1,920 x 1,088 pixels, for
example. In this manner, high compression rates and
excellent image quality can be realized.
[0004]
10 MPEG2 is designed mainly for high-quality image
encoding suited for broadcasting, but is not compatible
with lower bit rates than MPEGl or encoding methods
involving higher compression rates. As mobile terminals
are becoming popular, the demand for such encoding
15 methods is expected to increase in the future, and to
meet the demand, the MPEG4 encoding method has been
standardized. As for image encoding methods, the ISO/IEC
14496-2 standard was approved as an international
standard in December 1998.
20 [0005]
Further, a standard called H.2 6L (ITU-T
(International Telecommunication Union Telecommunication
Standardization Sector) Q6/16 VCEG (Video Coding Expert
Group)), which is originally intended for encoding images
25 for video conferences, is currently being set. Compared
with the conventional encoding methods such as MPEG2 and
MPEG4, H.2 6L requires a larger amount of calculation in
encoding and decoding, but is known to achieve a higher
encoding efficiency. Also, as a part of the MPEG4
30 activity, "Joint Model of Enhanced-Compression Video
Coding" is now being established as a standard for
i 3
I ^. SP317970WO00 w
I achieving a higher encoding efficiency by incorporating
I functions unsupported by H.2 6L into the functions based
on H.2 6L.
[0006]
5 On the standardization schedule, the standard was
approved as an international standard under the name of
H,2 64 and MPEG-4 Part 10 (Advanced Video Coding,
hereinafter referred to as AVC) in March 2003.
[0007]
10 Meanwhile, to improve motion vector encoding using
median predictions according to AVC, there has been a
suggestion to adaptively use "Temporal Predictor" or
"Spatio-Temporal Predictor" as predicted motion vector
information in addition to "Spatial Predictor", which is
15 defined in AVC and is determined through a median
prediction (see Non-Patent Document 1, for example).
[0008]
In an image information encoding device, cost
function values for respective blocks are calculated by
2 0 using the predicted motion vector information about the
respective blocks, and optimum predicted motion vector
information is selected. Through the compressed image
information, flag information indicating the information
as to which predicted motion vector information has been
25 used is transmitted for each block.
[0009]
Meanwhile, the macroblock size of 16 x 16 pixels
might not be optimal for a large frame such as an UHD
(Ultra High Definition: 4000 x 2000 pixels) frame to be
30 encoded by a next-generation encoding method.
[0010]
i 4
I ^. SP317970WO00
In view of this and for the purpose of achieving an
even higher encoding efficiency than that achieved by AVC,
an encoding method called HEVC (High Efficiency Video
Coding) is now being standardized by JCTVC (Joint
5 Collaboration Team - Video Coding), which is a joint
standards organization of ITU-T and ISO/IEC (see Non-
Patent Document 2, for example).
[0011]
I According to the HEVC encoding method, coding units
10 (CUs) are defined as processing units like macroblocks of
AVC. Unlike the macroblocks of AVC, the CUs are not
fixed to the size of 16 x 16 pixels. The size of the CUs
is specified in the compressed image information in each
sequence.
15 [0012]
The CUs form a hierarchical structure including the
largest coding units (LCUs) and the smallest coding units
(SCUs). Roughly speaking, the LCUs can be considered
equivalent to the macroblocks of AVC, and the CUs on the
20 lower hierarchical levels than the LCUs (CUs smaller than
LCUs) can be considered equivalent to the sub macroblocks
of AVC.
[0013]
In motion vector (MV) encoding, MVs are not sent
25 directly to a decoder, but the difference vectors (MVDs)
that are the differences from predicted motion vectors
(PMVs) are subjected to lossless encoding, and are then
sent to the decoder. By a technique called Advanced MV
prediction(AMVP) or MV competition, an index (pmv_index)
30 for identifying PMVs is contained in each stream when
there are two or more candidate PMVs (see Non-Patent
5
•
SP317970WO00
Document 2, for example).
[0014]
The candidate PMVs may be MVs of blocks close to
i each other in a frame of the same time (spatial_pmv), or
5 MVs of blocks that differ from the encoded current frame
in terms of time (temporal_pmv). Where pmv_index is
encoded, the bit rate after the encoding is normally low
if the value of pmv_index is small.
[0015]
10 Therefore, to achieve a high encoding efficiency,
pmv_index having a small value should be assigned to a
candidate PMV having a higher designation frequency.
CITATION LIST
15 NON-PATENT DOCUMENTS
[0016]
Non-Patent Document 1: Joel Jung, Guillaume
Laroche,"Competition-Based Scheme for Motion Vector
Selection and Coding", VCEG-AC06,ITU - Telecommunications
20 Standardization SectorSTUDY GROUP 16 Question 6Video
Coding Experts Group (VCEG)29th Meeting: Klagenfurt,
Austria, 17-18 July, 2006
Non-Patent Document 2: "Test Model under Consideration",
JCTVC-B205, Joint Collaborative Team on Video Coding
25 (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTCl/SC29/WG112nd
Meeting: Geneva, CH, 21-28 July, 2010
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
30 [0017]
However, the frequency at which a candidate PMV is
6
•
SP317970WO00
designated depends on the contents of images. Therefore,
even if code numbers are assigned in such a manner as to
give priority to temporal_pmv or spatial_pmv (or if s
smaller code number is assigned to temporal_pmv or
5 spatial_pmv), there has been a possibility that the
encoding efficiency becomes lower depending on the
contents of images, and the lower encoding efficiency
leads to a problem.
[0018]
10 For example, if s larger code number is assigned to
temporal_pmv (while a smaller code number is assigned to
spatial_pmv), there is a possibility that the bit rate
becomes higher when temporal_pmv is designated as a PMV,
due to discontinuity or the like appearing in motion
15 around the current block.
[0019]
If a larger code number is assigned to spatial_pmv
(while a smaller code number is assigned to temporal_pmv),
there is a possibility that the bit rate becomes higher
2 0 when spatial_pmv is designated as a PMV, due to a stop of
panning of a camera, for example.
[0020]
This disclosure has been made in view of the above
circumstances, and the object thereof is to prevent
25 reductions in encoding efficiency by assigning a smaller
code number to temporal_pmv when the distance between the
current picture and the picture containing temporal_pmv
is short in terms of displaying order, and assigning a
smaller code number to spatial_pmv when the distance
30 between the current picture and the anchor picture is
long in terms of displaying order, taking advantage of
7
^ SP317970WO00
the fact that frames close to each other in terms of
displaying order are highly likely to have similar
motions.
5 SOLUTIONS TO PROBLEMS
[0021]
One aspect of this disclosure is an image
processing device that includes: an assignment control
unit that controls assignment of a binary bit sequence to
10 predicted motion vector select information indicating a
motion vector selected as a predicted motion vector, to
assign a bit sequence having a shorter code length to the
select information about a motion vector having a higher
designation frequency; and a binarization unit that
15 binarizes the predicted motion vector select information
with the bit sequence assigned by the assignment control
unit.
[0022]
The assignment control unit may include a table
20 selection unit that selects a table that designates the
bit sequence to be assigned to the predicted motion
vector select information in accordance with the type of
the motion vector selected as the predicted motion vector,
and the binarization unit can binarize the predicted
25 motion vector select information by using the table
selected by the table selection unit.
[0023]
The assignment control unit may further include a
distance calculation unit that calculates the distance
30 between the current picture and an anchor picture, and
the table selection unit can select the table based on
8
^ . SP317970WO00
the distance calculated by the distance calculation unit.
[0024]
The assignment control unit may further include a
distance threshold acquirement unit that acquires a
5 distance threshold indicating the threshold of the
distance, and the table selection unit can select the
table in accordance with the magnitude relationship
between the distance calculated by the distance
calculation unit and the distance threshold acquired by
10 the distance threshold acquirement unit.
[0025]
The distance threshold acquired by the distance
threshold acquirement unit may be supplied to another
device that decodes encoded data of the predicted motion
15 vector select information.
[0026]
The assignment control unit may further include a
similarity calculation unit that calculates the
similarity between peripheral predicted motion vectors,
20 and the table selection unit can select the table based
on the similarity calculated by the similarity
calculation unit.
[0027]
The assignment control unit may further include a
25 similarity threshold acquirement unit that acquires a
similarity threshold indicating the threshold of the
similarity, and the table selection unit can select the
table in accordance with the magnitude relationship
between the similarity calculated by the similarity
30 calculation unit and the similarity threshold acquired by
the similarity threshold acquirement unit.
i
9
© SP317970WO00
[0028]
The similarity threshold acquired by the similarity
threshold acquirement unit may be supplied to another
device that decodes encoded data of the predicted motion
5 vector select information.
[0029]
The assignment control unit may further include a
cost function value calculation unit that calculates a
cost function value of the predicted motion vector select
10 information, and the table selection unit can select the
table based on the cost function value calculated by the
cost function value calculation unit.
[0030]
Information indicating the result of the table
15 selection performed by the table selection unit may be
supplied to another device that decodes encoded data of
the predicted motion vector select information.
[0031]
The image processing device may further include an
2 0 encoding unit that encodes the binarized data obtained
through the binarization performed by the binarization
unit.
[0032]
The one aspect of this disclosure is also an image
25 processing method for an image processing device. The
image processing method includes: controlling assignment
of a binary bit sequence to predicted motion vector
select information indicating a motion vector selected as
a predicted motion vector, to assign a bit sequence
30 having a shorter code length to the select information
about a motion vector having a higher designation
i
I
i
10
•
SP317970WO00
frequency, the controlling being performed by an
assignment control unit; and binarizing the predicted
motion vector select information with the assigned bit
sequence, the binarizing being performed by a
5 binarization unit.
[0033]
The other aspect of this disclosure is an image
processing device that includes: a decoding unit that
decodes encoded data of predicted motion vector select
10 information indicating a motion vector selected as a
predicted motion vector; and a debinarization unit that
debinarizes the binarized data of the predicted motion
vector select information obtained through the decoding
performed by the decoding unit, with a bit sequence that
15 is assigned in an assigning operation controlled to
assign a bit sequence having a shorter code length to the
select information about a motion vector having a higher
designation frequency.
[0034]
2 0 The image processing device may further including
an assignment control unit that controls the bit sequence
to be assigned to the predicted motion vector select
information based on a parameter supplied from another
device that has encoded the predicted motion vector
25 select information. The debinarization unit can
debinarize the predicted motion vector select information
under the control of the assignment control unit.
[0035] I
The assignment control unit may include a table
30 selection unit that selects a table that designates the
bit sequence to be assigned to the predicted motion
i
I
I
11
© SP317970WO00
vector select information in accordance with the type of
the motion vector selected as the predicted motion vector.
[0036]
The assignment control unit may further include a
5 distance calculation unit that calculates the distance
between the current picture and an anchor picture, and
the table selection unit can select the table in
accordance with the magnitude relationship between the
distance calculated by the distance calculation unit and
10 a distance threshold supplied from the other device that
has encoded the predicted motion vector select
information.
[0037]
The assignment control unit may further include a
15 similarity calculation unit that calculates the
similarity between peripheral predicted motion vectors,
and the table selection unit can select the table in
accordance with the magnitude relationship between the
similarity calculated by the similarity calculation unit
2 0 and a similarity threshold supplied from the other device
that has encoded the predicted motion vector select
information.
[0038]
The debinarization unit may debinarize the
25 binarized data of the predicted motion vector select
information based on information indicating a table
selection result supplied from the other device that has
encoded the predicted motion vector select information.
[0039]
30 The other aspect of this disclosure is also an
image processing method for an image processing device.
!
i
12
^. SP317970WO00
The image processing method includes: decoding encoded
data of predicted motion vector select information
indicating a motion vector selected as a predicted motion
vector, the decoding being performed by a decoding unit;
5 and debinarizing the binarized data of the predicted
motion vector select information obtained through the
decoding, with a bit sequence that is assigned in an
assigning operation controlled to assign a bit sequence
having a shorter code length to the select information
10 about a motion vector having a higher designation
frequency, the debinarizing being performed by a
debinarization unit.
[0040]
In the one aspect of this disclosure, assignment of
15 a binary bit sequence to predicted motion vector select
information indicating a motion vector selected as a
predicted motion vector is controlled so as to assign a
bit sequence having a shorter code length to the select
information about a motion vector having a higher
2 0 designation frequency, and the predicted motion vector
select information is binarized with the assigned bit
sequence.
[0041]
In the other aspect of this disclosure, encoded
25 data of predicted motion vector select information
indicating a motion vector selected as a predicted motion
vector is decoded, and the binarized data of the
predicted motion vector select information obtained
through the decoding is debinarized with a bit sequence
30 that is assigned in an assigning operation controlled so
as to assign a bit sequence having a shorter code length
i
i
J
13
© SP317970WO00
to the select information about a motion vector having a
higher designation frequency.
EFFECTS OF THE INVENTION
5 [0042]
According to this disclosure, images can be
processed. Particularly, encoding efficiency can be
increased.
10 BRIEF DESCRIPTION OF DRAWINGS
[0043]
Fig. 1 is a block diagram showing an image encoding
device that outputs compressed image information
according to the AVC encoding method.
15 Fig. 2 is a block diagram showing an image decoding
device that receives an input of compressed image
information according to the AVC encoding method.
Fig. 3 is a diagram showing an example motion
prediction/compensation operation with decimal pixel
20 precision.
Fig. 4 is a diagram showing example macroblocks.
Fig. 5 is a diagram for explaining an example
situation of a median operation.
Fig. 6 is a diagram for explaining an example
25 Multi-Reference Frame.
Fig. 7 is a diagram for explaining an example
situation in Temporal Direct Mode.
Fig. 8 is a diagram for explaining an example
situation according to a motion vector encoding method
30 suggested in Non-Patent Document 1.
Fig. 9 is a diagram for explaining example
I
i
14
^fc, SP317970WO00
#
structures of coding units.
Fig. 10 is a block diagram showing a typical
example structure of an image encoding device.
Fig. 11 is a diagram for explaining example
5 candidate PMVs for AMVP.
Fig. 12 is a diagram for explaining the
relationship between MV_Tmp and the difference between
the current picture and the anchor picture in terms of
displaying order.
10 Fig. 13 is a diagram for explaining the
relationship between MV_Tmp and the difference between
the current picture and the anchor picture in terms of
displaying order.
Fig. 14 is a block diagram showing typical example
15 structures of the lossless encoding unit and the
assignment control unit shown in Fig. 11.
Fig. 15 is a diagram for explaining an example
pattern of assignment of code numbers and bit sequences.
Fig. 16 is a flowchart for explaining an example
20 flow of an encoding operation.
Fig. 17 is a flowchart for explaining an example
flow of a lossless encoding operation.
Fig. 18 is a flowchart for explaining an example
flow of an inter prediction mode information encoding
25 operation.
Fig. 19 is a block diagram showing a typical
example structure of an image decoding device.
Fig. 2 0 is a block diagram showing typical example
structures of the lossless decoding unit and the
30 assignment control unit shown in Fig. 19.
Fig. 21 is a flowchart for explaining an example
j
/
i
I ^^
[ ^ SP317970WO00
flow of a decoding operation.
Fig. 22 is a flowchart for explaining an example
flow of a lossless decoding operation.
Fig. 2 3 is a flowchart for explaining an example
5 flow of an inter prediction mode information decoding
operation.
Fig. 2 4 is a diagram for explaining another example
pattern of assignment of code niombers and bit sequences.
Fig. 25 is a diagram for explaining yet another
10 example pattern of assignment of code numbers and bit
sequences.
Fig. 2 6 is a block diagram showing typical example
structures of the lossless encoding unit and the
assignment control unit shown in Fig. 11.
15 Fig. 2 7 is a diagram for explaining an example
relationship between peripheral motion vectors.
Fig. 2 8 is a diagram for explaining another example
relationship between peripheral motion vectors.
Fig. 2 9 is a diagram for explaining yet another
20 example relationship between peripheral motion vectors.
Fig. 30 is a diagram for explaining still another
example relationship between peripheral motion vectors.
Fig. 31 is a diagram for explaining yet another
example relationship between peripheral motion vectors.
25 Fig. 32 is a flowchart for explaining another
example flow of the inter prediction mode information
encoding operation.
Fig. 33 is a block diagram showing typical example
structures of the lossless decoding unit and the
30 assignment control unit shown in Fig. 19.
Fig. 34 is a flowchart for explaining another
i
16
,^^ SP317970WO00
#
I
example flow of the inter prediction mode information
decoding operation.
i Fig. 35 is a block diagram showing other typical
I example structures of the lossless encoding unit and the
I 5 assignment control unit shown in Fig. 11.
I Fig. 3 6 is a flowchart for explaining yet another
example flow of the inter prediction mode information
encoding operation.
Fig. 37 is a block diagram showing other typical
10 example structures of the lossless decoding unit and the
assignment control unit shown in Fig. 19.
Fig. 38 is a flowchart for explaining yet another
example flow of the inter prediction mode information
decoding operation.
I 15 Fig. 39 is a block diagram showing a typical
i
I example structure of a personal computer.
I
Fig. 4 0 is a block diagram showing a typical
I example structure of a television receiver.
Fig. 41 is a block diagram showing a typical
2 0 example structure of a portable telephone device.
Fig. 42 is a block diagram showing a typical
example structure of a hard disk recorder.
Fig. 43 is a block diagram showing a typical
example structure of a camera.
25
MODES FOR CARRYING OUT THE INVENTION
[0044]
The following is a description of modes for
carrying out the present technique (hereinafter referred
30 to as the embodiments). Explanation will be made in the
following order.
17 '
•
' SP317970WO00
1. First Embodiment (Image Encoding Device and
Image Decoding Device)
2. Second Embodiment (Image Encoding Device and
Image Decoding Device)
5 3. Third Embodiment (Image Encoding Device and
i Image Decoding Device)
I 4. Fourth Embodiment (Personal Computer)
I
I 5, Fifth Embodiment (Television Receiver)
6. Sixth Embodiment (Portable Telephone Device)
j 10 7. Seventh Embodiment (Hard Disk Recorder)
• 8. Eighth Embodiment (Camera)
[0045]
<1. First Embodiment>
[Image Encoding Device Compatible with the AVC Encoding
15 Method]
Fig. 1 shows the structure of an embodiment of an
1 image encoding device that encodes images by the H.2 64
and MPEG (Moving Picture Experts Group) 4 Part 10
(AVC(Advanced Video Coding)) encoding methods,
20 [0046]
The image encoding device 100 shown in Fig. 1 is a
device that encodes and outputs images by an encoding
method compliant with the AVC standard. As shown in Fig.
1, the image encoding device 100 includes an A/D
25 converter 101, a screen rearrangement buffer 102, an
arithmetic operation unit 103, an orthogonal transform
unit 104, a quantization unit 105, a lossless encoding
unit 106, and an accumulation buffer 107. The image
encoding device 100 also includes an inverse quantization
30 unit 108, an inverse orthogonal transform unit 109, an
arithmetic operation unit 110, a deblocking filter 111, a
1
i
I
18
•
SP317970WO00
frame memory 112, a selection unit 113, an intra
prediction unit 114, a motion prediction/compensation
unit 115, a selection unit 116, and a rate control unit
I 117.
5 [0047]
The A/D converter 101 subjects input image data to
an A/D conversion, and outputs and stores the resultant
image data into the screen rearrangement buffer 102. The
screen rearrangement buffer 102 rearranges the image
10 frames stored in displaying order in accordance with the
GOP (Group of Pictures) structure, so that the frames are
arranged in encoding order.
[0048]
The screen rearrangement buffer 102 supplies the
15 image having the rearranged frame order to the arithmetic
operation unit 103. The screen rearrangement buffer 102
also supplies the image having the rearranged frame order
to the intra prediction unit 114 and the motion
prediction/compensation unit 115.
20 [0049]
The arithmetic operation unit 103 subtracts a
predicted image supplied from the intra prediction unit
114 or the motion prediction/compensation unit 115 via
the selection unit 116, from the image read from the
25 screen rearrangement buffer 102, and outputs the
difference information to the orthogonal transform unit
104.
[0050]
When intra encoding is performed on an image, for
30 example, the arithmetic operation unit 103 subtracts a
predicted image supplied from the intra prediction unit
i
r
! I
i !
i 19
•
' SP317970WO00
114, from the image read from the screen rearrangement
buffer 102. When inter encoding is performed on an image,
for example, the arithmetic operation unit 103 subtracts
a predicted image supplied from the motion
5 prediction/compensation unit 115, from the image read
j from the screen rearrangement buffer 102.
[0051]
The orthogonal transform unit 104 performs an
orthogonal transform operation, such as a discrete cosine
10 transform or a Karhunen-Loeve transform, on the
difference information supplied from the arithmetic
operation unit 103, and supplies the transform
coefficient to the quantization unit 105.
[0052]
15 The quantization unit 105 quantizes the transform
coefficient output from the orthogonal transform unit 104.
Based on target bit rate value information supplied from
the rate control unit 117, the quantization unit 105 sets
a quantization parameter, and performs quantization. The
20 quantization unit 105 supplies the quantized transform
coefficient to the lossless encoding unit 106.
[0053]
The lossless encoding unit 106 performs lossless I
encoding on the quantized transform coefficient through
25 variable-length encoding or arithmetic encoding or the
like. Since the coefficient data has already been
quantized under the control of the rate control unit 117,
I the bit rate becomes equal to the target value (or
approximates the target value) that is set by the rate
30 control unit 117.
[0054]
j
I
I •
i
i 20
•
' SP317970WO00
I The lossless encoding unit 106 obtains information
1
indicating an intra prediction and the like from the
j intra prediction unit 114, and obtains information
; indicating an inter prediction mode, motion vector
5 information, and the like from the motion
prediction/compensation unit 115. The information
indicating an intra prediction (an intra-screen
prediction) will be hereinafter also referred to as intra
prediction mode information. The information indicating
10 an inter prediction (an inter-screen prediction) will be
hereinafter referred to as inter prediction mode
information.
I [0055]
! The lossless encoding unit 106 not only encodes the
I
15 quantized transform coefficient, but also incorporates
(multiplexes) various kinds of information such as a
filter coefficient, the intra prediction mode information,
the inter prediction mode information, and the
quantization parameter, into the header information of
20 encoded data. The lossless encoding unit 106 supplies
and stores the encoded data obtained through the encoding
into the accumulation buffer 107.
[0056]
For example, in the lossless encoding unit 106, a
25 lossless encoding operation such as variable-length
encoding or arithmetic encoding is performed. The
variable-length encoding may be CAVLC (Context-Adaptive
Variable Length Coding) specified in H.264/AVC, for
example. The arithmetic encoding may be CABAC (Context-
30 Adaptive Binary Arithmetic Coding) or the like.
[0057]
I
I
I
II
t
21
•
SP317970WO00
The accumulation buffer 107 temporarily stores the
encoded data supplied from the lossless encoding unit 106,
and outputs the encoded data as an encoded image encoded
by H,2 64/AVC to a recording device or a transmission path
5 (not shown) in a later stage at a predetermined time, for
:
example.
\ [0058]
The transform coefficient quantized at the
quantization unit 105 is also supplied to the inverse
10 quantization unit 108. The inverse quantization unit 108
inversely quantizes the quantized transform coefficient
by a method compatible with the quantization performed by
the quantization unit 105. The inverse quantization unit
108 supplies the obtained transform coefficient to the
15 inverse orthogonal transform unit 109.
[0059]
The inverse orthogonal transform unit 10 9 performs
an inverse orthogonal transform on the supplied transform
coefficient by a method compatible with the orthogonal
20 transform operation performed by the orthogonal transform
unit 104. The output subjected to the inverse orthogonal
transform (the restored difference information) is
supplied to the arithmetic operation unit 110.
[0060]
25 The arithmetic operation unit 110 obtains a locally
decoded image (a decoded image) by adding the predicted
image supplied from the intra prediction unit 114 or the
motion prediction/compensation unit 115 via the selection
unit 116 to the inverse orthogonal transform result
30 supplied from the inverse orthogonal transform unit 109
or the restored difference information.
22
•
SP317970WO00
[0061]
For example, when the difference information is
compatible with an image to be intra-encoded, the I
arithmetic operation unit 110 adds the predicted image i
5 supplied from the intra prediction unit 114 to the
difference information. When the difference information
is compatible with an image to be inter-encoded, the
arithmetic operation unit 110 adds the predicted image
I supplied from the motion prediction/compensation unit 115
10 to the difference information, for example.
I [0062]
The addition result is supplied to the deblocking
filter 111 or the frame memory 112.
[0063]
15 The deblocking filter 111 removes block distortions
I from the decoded image by performing a deblocking
I
i filtering operation where necessary. The deblocking
i
1 filter 111 supplies the filtering operation result to the
I frame memory 112. The decoded image that is output from
20 the arithmetic operation unit 110 can be supplied to the
frame memory 112 without passing through the deblocking
filter 111. That is, the deblocking filtering operation
of the deblocking filter 111 can be skipped.
[0064]
25 The frame memory 112 stores the supplied decoded
image, and outputs the stored decoded image as a
reference image to the intra prediction unit 114 or the
motion prediction/compensation unit 115 via the selection
unit 113 at a predetermined time.
30 [0065]
When intra encoding is performed on an image, for
I !
I 23
© SP317970WO00
I example, the frame memory 112 supplies the reference
i
I image to the intra prediction unit 114 via the selection
I unit 113. When inter encoding is performed on an image,
for example, the frame memory 112 supplies the reference
I 5 image to the motion prediction/compensation unit 115 via
the selection unit 113.
[0066]
When the reference image supplied from the frame
memory 112 is an image to be subjected to intra encoding,
10 the selection unit 113 supplies the reference image to
the intra prediction unit 114. When the reference image
supplied from the frame memory 112 is an image to be
subjected to inter encoding, the selection unit 113
supplies the reference image to the motion
1
I 15 prediction/compensation unit 115.
I
[0067]
The intra prediction unit 114 performs intra
I predictions (intra-screen predictions) to generate a
I predicted image by using the pixel values in the current
I I
20 picture supplied from the frame memory 112 via the
selection unit 113. The intra prediction unit 114
performs intra predictions in more than one mode (intra
prediction modes) that is prepared in advance.
[0068]
25 By the H.2 64 image information encoding method, an
intra 4 x 4 prediction mode, an intra 8 x 8 prediction
mode, and an intra 16 x 16 prediction mode are defined
for luminance signals. As for chrominance signals,
prediction modes for respective macroblocks can be
I
30 defined independently of the luminance signals. In the
I
I intra 4 x 4 prediction mode, one intra prediction mode is
I
24 1
© SP317970WO00
defined for each 4 x 4 luminance block. In the intra 8 x
8 prediction mode, one intra prediction mode is defined
for each 8 x 8 luminance block. In the intra 16 x 16
prediction mode and for the chrominance signals, one
5 prediction mode is defined for each macroblock.
[0069]
The intra prediction unit 114 generates predicted
[ images in all the candidate intra prediction modes,
I evaluates the cost function values of the respective
I
I 10 predicted images by using the input image supplied from
I the screen rearrangement buffer 102, and selects an
i optimum mode. After selecting the optimism intra
prediction mode, the intra prediction unit 114 supplies
the predicted image generated in the optimum intra
15 prediction mode to the arithmetic operation unit 103 and
1 the arithmetic operation unit 110 via the selection unit
I 116.
[0070]
As described above, the intra prediction unit 114
20 also supplies information such as the intra prediction
mode information indicating the adopted intra prediction
mode to the lossless encoding unit 106 where appropriate.
[0071]
Using the input image supplied from the screen
25 rearrangement buffer 102, and the reference image
supplied from the frame memory 112 via the selection unit
113, the motion prediction/compensation unit 115 performs
motion predictions (inter predictions) on an image to be
subjected to inter encoding, and performs a motion
30 compensation operation in accordance with the detected
motion vectors, to generate a predicted image (inter
25
i -i. SP317970WO00
i predicted image information). The motion
! prediction/compensation unit 115 performs such inter
predictions in more than one mode (inter prediction
modes) that is prepared in advance.
5 [0072]
The motion prediction/compensation unit 115
generates predicted images in all the candidate inter
prediction modes, evaluates the cost function values of
the respective predicted images, and selects an optimum
10 mode. The motion prediction/compensation unit 115
supplies the generated predicted image to the arithmetic
I
j operation unit 103 and the arithmetic operation unit 110
via the selection unit 116. 1
I [0073]
i 15 The motion prediction/compensation unit 115
I
I supplies the inter prediction mode information indicating I
the adopted inter prediction mode, and motion vector
I information indicating the calculated motion vectors to
I the lossless encoding unit 106.
I
20 [0074]
When intra encoding is performed on an image, the
selection unit 116 supplies the output of the intra
prediction unit 114 to the arithmetic operation unit 103
and the arithmetic operation unit 110. When inter
25 encoding is performed on an image, the selection unit 116
supplies the output of the motion prediction/compensation
unit 115 to the arithmetic operation unit 103 and the
arithmetic operation unit 110.
[0075]
30 Based on the compressed images accumulated in the
accumulation buffer 107, the rate control unit 117
i
I
* I ^ I
26
•
SP317970WO00
controls the quantization operation rate of the
quantization unit 105 so as not to cause an overflow or
underflow.
[0076]
5 [Image Decoding Device Compatible with the AVC Encoding
Method]
Fig. 2 is a block diagram showing a typical example
i structure of an image decoding device that realizes image
i
compression through orthogonal transforms, such as
10 discrete cosine transforms or Karhunen-Loeve transforms,
and motion compensation. The image decoding device 200
shown in Fig. 2 is a decoding device that is compatible
with the image encoding device 100 shown in Fig. 1.
[0077]
15 Data encoded by the image encoding device 100 is
supplied to the image decoding device 2 00 compatible with
the image encoding device 100 via a passage such as a
transmission path or a recording medium, and is then
decoded.
20 [0078]
As shown in Fig. 2, the image decoding device 200
includes an accumulation buffer 201, a lossless decoding
unit 202, an inverse quantization unit 203, an inverse
orthogonal transform unit 204, an arithmetic operation
25 unit 205, a deblocking filter 206, a screen rearrangement
buffer 207, and a D/A converter 208. The image decoding
device 200 also includes a frame memory 209, a selection
unit 210, an intra prediction unit 211, a motion
prediction/compensation unit 212, and a selection unit
30 213.
[0079]
II
27
•
SP317970WO00
The accumulation buffer 2 01 accumulates transmitted
encoded data. The encoded data has been encoded by the
image encoding device 100. The lossless decoding unit
202 decodes the encoded data read from the accumulation
5 buffer 2 01 at a predetermined time, by a method
compatible with the encoding method used by the lossless
encoding unit 106 shown in Fig. 1.
[0080]
When the current frame is an intra-encoded frame,
10 the header portion of the encoded data stores intra
prediction mode information. The lossless decoding unit
202 also decodes the intra prediction mode information,
and supplies the resultant information to the intra
prediction unit 211. When the current frame is an inter- j
15 encoded frame, on the other hand, the header portion of I
the encoded data stores motion vector information. The j
lossless decoding unit 202 also decodes the motion vector j
information, and supplies the resultant information to I
the motion prediction/compensation unit 212. j
20 [0081]
The inverse quantization unit 203 inversely I
quantizes the coefficient data (the quantized
coefficient) decoded by the lossless decoding unit 202,
by a method compatible with the quantization method used
25 by the quantization unit 105 shown in Fig. 1. That is,
the inverse quantization unit 203 inversely quantizes the
quantized coefficient by the same method as the method
used by the inverse quantization unit 108 shown in Fig. 1.
[0082]
30 The inverse quantization unit 203 supplies the
inversely-quantized coefficient data, or the orthogonal
28
•
SP317970WO00
transform coefficient, to the inverse orthogonal
transform unit 204. The inverse orthogonal transform
unit 204 subjects the orthogonal transform coefficient to
an inverse orthogonal transform by a method compatible
5 with the orthogonal transform method used by the
orthogonal transform unit 104 shown in Fig. 1 (the same
method as the method used by the inverse orthogonal
transform unit 109 shown in Fig. 1), and obtains decoded
residual error data corresponding to the residual error
10 data from the time prior to the orthogonal transform
performed by the image encoding device 100. For example,
a fourth-order inverse orthogonal transform is performed.
[0083]
The decoded residual error data obtained through
15 the inverse orthogonal transform is supplied to the
arithmetic operation unit 205. A predicted image is also
supplied to the arithmetic operation unit 205 from the
intra prediction unit 211 or the motion
prediction/compensation unit 212 via the selection unit
20 213.
[0084]
The arithmetic operation unit 2 05 adds the decoded
residual error data to the predicted image, and obtains
decoded image data corresponding to the image data from
25 the time prior to the predicted image subtraction
performed by the arithmetic operation unit 103 of the
image encoding device 100. The arithmetic operation unit
2 05 supplies the decoded image data to the deblocking
filter 206.
30 [0085]
The deblocking filter 206 removes block distortions
I
j
29 I
•
SP317970WO00
from the supplied decoded images, and supplies the images
to the screen rearrangement buffer 2 07.
I [0086]
The screen rearrangement buffer 2 07 performs image
5 rearrangement. Specifically, the frame sequence
I rearranged in the encoding order by the screen
rearrangement buffer 102 shown in Fig. 1 is rearranged in
the original displaying order. The D/A converter 208
performs a D/A conversion on the images supplied from the
10 screen rearrangement buffer 2 07, and outputs the
converted images to a display (not shown) to display the
images.
[0087]
The output of the deblocking filter 206 is further
15 supplied to the frame memory 209.
[0088]
The frame memory 2 09, the selection unit 210, the
intra prediction unit 211, the motion
prediction/compensation unit 212, and the selection unit
20 213 are equivalent to the frame memory 112, the selection
unit 113, the intra prediction unit 114, the motion
prediction/compensation unit 115, and the selection unit
116 of the image encoding device 100, respectively.
[0089]
25 The selection unit 210 reads, from the frame memory
209, an image to be inter-processed and an image to be
referred to, and supplies the images to the motion
prediction/compensation unit 212. The selection unit 210
also reads an image to be used for intra predictions from
30 the frame memory 209, and supplies the image to the intra
prediction unit 211.
30
•
SP317970WO00
[0090]
Information that has been obtained by decoding the
header information and indicates an intra prediction mode
or the like is supplied, where appropriate, from the
5 lossless decoding unit 202 to the intra prediction unit
211. Based on the information, the intra prediction unit
211 generates a predicted image from the reference image
obtained from the frame memory 209, and supplies the
generated predicted image to the selection unit 213.
10 [0091]
The motion prediction/compensation unit 212 obtains
the information obtained by decoding the header
information (prediction mode information, motion vector
information, reference frame information, a flag,
15 respective parameters, and the like), from the lossless
decoding unit 2 02.
[0092]
Based on the information supplied from the lossless
decoding unit 2 02, the motion prediction/compensation
2 0 unit 212 generates a predicted image from the reference
image obtained from the frame memory 2 09, and supplies
the generated predicted image to the selection unit 213.
[0093]
The selection unit 213 selects a predicted image
25 generated by the motion prediction/compensation unit 212
or the intra prediction unit 211, and supplies the
selected predicted image to the arithmetic operation unit
205.
[0094]
30 [Motion Prediction/Compensation Operation with Decimal
Pixel Precision]
i
31
^^ SP317970WO00
By an encoding method such as MPEG2, motion |
prediction/compensation operations with 1/2 pixel
precision are performed through linear interpolations.
By the AVC encoding method, on the other hand, motion
5 prediction/compensation operations with 1/4 pixel
precision using a 6-tap FIR filter are performed, and
encoding efficiency is increased accordingly,
[0095]
Fig. 3 is a diagram for explaining an example
10 motion prediction/compensation operation with 1/4 pixel
precision specified by the AVC encoding method. In Fig.
3, each square represents a pixel. Among those squares,
each A represents the position of an integer precision
pixel stored in the frame memory 112, b, c, and d
15 represent the positions of 1/2 precision pixels, and ei,
e2, and ea represent the positions of 1/4 precision
pixels.
[0096]
In the following, the function Clip 1() is defined
2 0 as shown in the following equation (1):
[0097]
[Mathematical Formula 1]
r 0;if(a<0)
Clip 1 (a) =" a; otherwise
[max_pix;if(a>max_pix)
. . . (1)
25 [0098]
When an input image has 8-bit precision, for
example, the value of max_pix in the equation (1) is 255.
[0099]
The pixel values in the positions represented by b
32
© SP317970WO00 i
i and d are generated by using a 6-tap FIR filter, as shown i
in the following equations (2) and (3): j
I [0100] I
[Mathematical Formula 2] I
F = A.2-5 • A.1+20 • Ao+20 • Ai-5 • A2+A3
5 I
... (2) I
[Mathematical Formula 3]
I
I b,d = Clipl((F+16)»5) j
... (3)
10 [0101]
The pixel value in the position represented by c is |
I generated by using a 6-tap FIR filter in the horizontal I I
I direction and the vertical direction, as shown in the j
; following equations (4) through (6): j
i 15 [0102]
[Mathematical Formula 4] j
I F=b.2-5-b.i+20-bo+20-bi-5-b2+b3
j ... (4)
or i
I
20 [Mathematical Formula 5] !
F -d.2-5 • d.i+20 • do+20 • di-5 • d2+d3
j ... (5)
i [Mathematical Formula 6] j
c-Clipl((F+512)>>10)
25 ... (6)
[0103] I
I
I
33 « SP317970WO00
The Clip operation is only once performed at last,
after both the horizontal product-sum operation and the
vertical product-sum operation are performed.
[0104]
5 Meanwhile, ei through e^ are generated through
linear interpolations, as shown in the following
equations (7) through (9):
[0105]
[Mathematical Formula 7]
^^ ei = (A+b+l)>>l
... (7)
[Mathematical Formula 8]
e2 = (b+d+l)>>l
... (8)
15 [Mathematical Formula 9]
e3 = (b+c+l)>>l
... (9) [0106]
[Motion Prediction/Compensation Operation]
In MPEG2, each unit in motion
20 prediction/compensation operations is 16 x 16 pixels in a
frame motion compensation mode, and is 16 x 8 pixels in
each of a first field and a second field in a field
motion compensation mode. With such units, motion
prediction/compensation operations are performed.
25 [0107]
In AVC, on the other hand, each one macroblock
formed with 16 x 16 pixels is divided into 16 x 16, 16 x
8, 8 X 16, or 8 X 8 parts, as shown in Fig. 4, and those
34
# ' SP317970WO00
parts can have motion vector information independently of
one another on a sub macroblock basis. Each 8 x 8 part
can be further divided into 8 x 8 , 8 x 4 , 4 x 8 , or 4 x4
sub macroblocks that can have motion vector information
5 independently of one another, as shown in Fig. 4.
[0108]
By the AVC image encoding method, however, there is
a possibility that an enormous amount of motion vector
information is generated if such motion
10 prediction/compensation operations are performed, as in
the case of MPEG2. Encoding the generated motion vector
information without any change might lead to a decrease
in encoding efficiency.
[0109]
15 [Motion Vector Median Prediction]
To solve this problem, the method described below
is used in AVC image encoding, and a decrease in the
amount of encoded motion vector information is realized,
[0110]
2 0 Each straight line shown in Fig. 5 indicates a
boundary between motion compensation blocks. In Fig. 5,
E represents the current motion compensation block to be
encoded, and A through D each represents a motion
compensation block that has already been encoded and is
25 adjacent to E.
[0111]
Where X is A, B, C, D, or E, mvx represents the
motion vector information about a block X.
[0112]
30 By using the motion vector information about the
motion compensation blocks A, B, and C, predicted motion
I
!
35
^ . SP317970WO00
vector information pmvE about the motion compensation
block E is generated through a median operation as shown
in the following equation (10):
[0113]
5 [Mathematical Formula 10]
pmvE = med(mvA, rnvs, mvc)
. . . (10)
[0114]
If the information about the motion compensation
10 block C is "unavailable" because the block C is located
at a corner of the image frame or the like, the
information about the motion compensation block D is used
instead.
[0115]
15 In the compressed image information, the data mvds
to be encoded as the motion vector information about the
motion compensation block E is generated by using pmvE as
shown in the following equation (11).
[0116]
20 [Mathematical Formula 11]
mvdE^mvE- prnve
. . . (11)
[0117]
In an actual operation, processing is performed on
25 the horizontal component and the vertical component of
the motion vector information independently of each other.
[0118]
[Multi-Reference Frame]
In AVC, Multi-Reference Frame method, which is not
36
© ^ SP317970WO00
specified by conventional image encoding methods such as
MPEG2 or H.263, is specified.
[0119]
Referring now to Fig. 6, Multi-Reference Frame
5 specified in AVC is described.
[0120]
In MPEG-2 and H.263, a motion
prediction/compensation operation is performed by
referring to only one reference frame stored in a frame
10 memory in the case of a P-picture. In AVC, however, more
than one reference frame is stored in a memory, and a
different memory can be referred to for each macroblock,
as shown in Fig. 5.
[0121]
15 [Direct Modes]
Although the amount of motion vector information in
a B-picture is very large, there are predetermined modes
called Direct Modes in AVC.
[0122]
20 In Direct Modes, motion vector information is not
stored in compressed image information. In an image
decoding device, the motion vector information about a
current block to be processed is calculated from the
motion vector information about a peripheral block or the
25 motion vector information about a co-located block that
is a block located in the same position as the current
block in a reference frame.
[0123]
Direct Modes includes the two modes: Spatial Direct
30 Mode and Temporal Direct Mode. One of the two modes can
be selected for each slice.
37 I
.». SP317970WO00 I
I [0124]
I In Spatial Direct Mode, the motion vector j
information mvE about the motion compensation block E to 1
be processed is calculated as shown in the following I
5 equation (12): |
[0125]
mvE = pmvE ... (12) j
[0126]
That is, motion vector information that is
10 generated through a median prediction is applied to the
current block.
[0127]
Referring now to Fig. 7, Temporal Direct Mode is
described.
15 [0128]
In Fig. 7, the block located at the address of the
same space as the current block in a LO reference picture
is referred to as a co-located block, and the motion
vector information about the co-located block is
20 represented by mvcoi. Also, TDB represents the distance
on the temporal axis between the current picture and the
LO reference picture, and TDD represents the distance on
the temporal axis between the LO reference picture and an
LI reference picture.
25 [0129]
At this point, the motion vector information mvLo
about LO and the motion vector information mvn about LI
in the current picture are calculated as shown in the
following equations (13) and (14):
30 [0130]
[Mathematical Formula 12]
I
i
i
j
38
# SP317970WO00
T DB
mvLo = Tpf;—mvcoi
. . . (13)
[Mathematical Formula 13]
T D D - T DB
mvLi = — = = r mvcoj
iUx)
5 ... (14)
[0131]
In AVC compressed image information, information
indicating a distance on the temporal axis TD does not
exist, and therefore, the calculations according to the
10 above mentioned equations (12) and (13) are performed by
using POC (Picture Order Count).
[0132]
In AVC compressed image information. Direct Modes
can be defined on a 16 x 16 pixel macroblock basis or an
15 8 x 8 pixel block basis.
[0133]
[Prediction Mode Selection]
To achieve higher encoding efficiency, it is
critical to select an appropriate prediction mode by the
20 AVC encoding method.
[0134]
An example of such a selection method is a method
stored in the H.264/MPEG-4 AVC reference software, called
JM (Joint Model) (available from
25 http://iphome.hhi.de/suehring/tml/index.htm).
[0135]
In JM, a method of determining a mode between two
I
I
i
39 I
^ . SP317970WO00 I
i
i
modes, which are High Complexity Mode and Low Complexity i
i i
j Mode, can be selected as described below. In either of j
I the modes, the cost function value as to each prediction i
mode is calculated, and the prediction mode that i
5 minimizes the cost function value is selected as the j
optimum mode for the current sub macroblock or the
current macroblock. i
[0136] :
A cost function in High Complexity Mode can be
10 calculated as shown in the following equation (15):
[0137]
Cost (Mode e Q ) = D + A,*R ... (15)
[0138]
Here, Q represents the universal set of candidate
15 modes for encoding the current block or macroblock, and D
represents the difference energy between a decoded image
and an input image when encoding is performed in the
prediction mode. X, represents the Lagrange's
undetermined multiplier provided as the quantization
2 0 parameter function. R represents the total bit rate in a
case where encoding is performed in the mode, including
the orthogonal transform coefficient.
[0139]
That is, to perform encoding in High Complexity
25 Mode, a provisional encoding operation needs to be
performed in all the candidate modes to calculate the
above parameters D and R, and therefore, a larger amount
of calculation is required.
[0140]
30 A cost function in Low Complexity Mode can be
calculated as shown in the following equation (16):
40
.». SP317970WO00
W
i
[0141] i
Cost (Mode e Q) = D + QP2Quant(QP) * HeaderBit ... I
(16) i
[0142]
5 Here, D differs from that in High Complexity Mode,
and represents the difference energy between a predicted
image and an input image. QP2Quant(QP) is a function of
a quantization parameter QP, and HeaderBit represents the
bit rate related to information that excludes the
10 orthogonal transform coefficient and belongs to Header,
such as motion vectors and the mode.
[0143]
That is, in Low Complexity Mode, a predicting
operation needs to be performed for each of the candidate
15 modes, but a decoded image is not required. Therefore,
there is no need to perform an encoding operation.
Accordingly, the calculation amount can be smaller than
that in High Complexity Mode.
[0144]
20 [Competition among Motion Vectors]
To improve motion vector encoding using median
predictions as described above with reference to Fig. 5,
Non-Patent Document 1 suggests the method described below.
[0145]
25 That is, in addition to "Spatial Predictor"
determined through a median prediction and defined in AVC,
one of "Temporal Predictor" and "Spatio-Temporal
Predictor" described below can be adaptively used as
predicted motion vector information.
30 [0146]
Specifically, in Fig. 8, "mvcol" represents the
i
j
i
!
i
5
41 I
^ SP317970WO00 w
motion vector information about a co-located block (a
block having the same x-y coordinates as the current
block in a reference image) of the current block, and
"mvtk" (k being one of 0 through 8) represents the motion ;
5 vector information about a peripheral block. The
predicted motion vector information (Predictor) about
each block is defined as shown in the following equations
(17) through (19):
[0147]
10 Temporal Predictor:
[Mathematical Formula 14]
mVtm5 = median{mVcoh mv,o,..., m\^}
. . . (17)
[Mathematical Formula 15]
mvtn,9 = median{mvcoi, mvto,.-., mvts}
ID
. . . (18)
Spatio-Temporal Predictor:
[Mathematical Formula 16]
mvspt=median{mVcoi, mVcoi, mVa, mvb, mVc}
20 ... (19)
[0148]
In the image encoding device 100, the cost function
values for respective blocks are calculated by using the
predicted motion vector information about the respective
25 blocks, and optimum predicted motion vector information
is selected. Through the compressed image information, a
flag indicating the information as to which predicted
motion vector information has been used is transmitted
j
I
i
I
42 i
,^fc, SP317970WO00 I
i
for each block. sj
[0149] I
[Coding Unit]
I
I
The macroblock size of 16 x 16 pixels is not I
5 optimal for large frames such as UHD (Ultra High !
Definition: 4000 x 2000 pixels) frames to be encoded by a |
next-generation encoding method.
[0150] In view of this, AVC specifies a hierarchical
10 structure formed with macr.oblocks and sub macroblocks as
shown in Fig. 4. In HEVC (High Efficiency Video Coding),
however, coding units (CUs) are specified as shown in Fig.
9.
[0151]
15 CUs are also called Coding Tree Blocks (CTBs), and
are partial regions of picture-based images that have the
same roles as those of macroblocks in AVC. While the
size of the latter is limited to the size of 16 x 16
pixels, the size of the former is not limited to a
20 certain size, and may be designated by the compressed
image information in each sequence.
[0152]
For example, in a sequence parameter set (SPS)
contained in encoded data to be output, the largest
25 coding unit (LCU) and the smallest coding unit (SCU) of
the CUs are specified.
[0153]
In each LCU, split-flag = 1 is set within a range
not lower than the SCU size, so that each LCU can be
30 divided into CUs of a smaller size. In the example shown
in Fig. 9, the size of the LCU is 128, and the greatest
43
^i. SP317970WO00
hierarchical depth is 5. When the value of split_flag is
"1", a CU of 2N X 2N in size is divided into CUs of N x N
in size, which is one hierarchical level lower.
[0154]
5 Each of the CUs is further divided into prediction
units (PUs) that are processing-unit regions (partial
regions of picture-based images) for intra or inter
predictions, or are divided into transform units (TUs)
that are processing-unit regions (partial regions of
I 10 picture-based images) for orthogonal transforms. At
present, 16 x 16 and 32 x 32 orthogonal transforms, as |
well as 4 X 4 and 8 x 8 orthogonal transforms, can be j
used in HEVC.
I [0155] !
i I
i 15 In a case where CUs are defined, and each
processing operation is performed on the CU basis in an |
encoding operation as in the above described HEVC, the
macroblocks in AVC can be considered equivalent to the
LCUs. However, a CU has a hierarchical structure as
20 shown in Fig. 9. Therefore, the size of the LCU on the
highest hierarchical level is normally as large as 128 x
128 pixels, which is larger than the size of each
macroblock in AVC, for example.
[0156]
25 [Remarks on This Embodiment]
As described above, various predicted motion
vectors are prepared. However, when predicted motion
vector select information (pmv_index) indicating which
predicted motion vector is to be adopted is encoded, a
30 shorter code sequence to be assigned as binarized data is
desirable, as the bit rate can be reduced with a shorter
I
i
I
i
I
44
^|. SP317970WO00
code sequence. That is, it is preferable to assign a
shorter bit sequence to a predicted motion vector having
i a higher designation frequency.
[0157]
5 However, the frequency varies depending on the I
j contents of images and the like. Therefore, there might
i be a case where temporal_pmv is more frequently
designated in an image, while spatial_pmv is more
frequently designated in another image. In view of this,
10 there is a possibility that bit sequences are not
appropriately assigned to the respective predicted motion
vectors with respect to the designation frequencies of
the respective predicted motion vectors, and the encoding
efficiency becomes lower.
15 [0158]
Therefore, in the following description, a smaller
code number is assigned to temporal_pmv when the current
picture and the anchor picture containing temporal_pmv
are close to each other in terms of displaying order, by
2 0 taking advantage of the fact that there is a high
possibility that motions of frames close to each other in
terms of displaying order are similar. When the current
picture and the anchor picture are large, a smaller code
number is assigned to spatial_pmv. The following is a
25 description of this method.
[0159]
[Image Encoding Device]
Fig. 10 is a block diagram showing a typical
example structure of an image encoding device.
30 [0160]
The image encoding device 300 shown in Fig. 10 is
45 « SP317970WO00
basically the same device as the image encoding device
100 shown in Fig. 1, and encodes image data. As shown in
Fig. 11, the image encoding device 300 includes an A/D
converter 301, a screen rearrangement buffer 302, an
5 arithmetic operation unit 303, an orthogonal transform
unit 304, a quantization unit 305, a lossless encoding
unit 306, and an accumulation buffer 307. The image
encoding device 300 also includes an inverse quantization
unit 308, an inverse orthogonal transform unit 309, an
10 arithmetic operation unit 310, a loop filter 311, a frame
memory 312, a selection unit 313, an intra prediction
unit 314, a motion prediction/compensation unit 315, a
selection unit 316, and a rate control unit 317.
[0161]
15 The image encoding device 300 further includes an
assignment control unit 321.
[0162]
The A/D converter 301 performs an A/D conversion on
input image data. The A/D converter 301 supplies and
20 stores the converted image data (digital data) into the
screen rearrangement buffer 302. The screen j
rearrangement buffer 302 rearranges the image frames j
i
stored in displaying order in accordance with the GOP, so I
that the frames are arranged in encoding order. The j
25 screen rearrangement buffer 302 supplies the image having j
the rearranged frame order to the arithmetic operation
j
unit 303. The screen rearrangement buffer 302 also |
I
supplies the image having the rearranged frame order to !
the intra prediction unit 314 and the motion
]
30 prediction/compensation unit 315. j
[0163] j
•j
46 i
^ . SP317970WO00 I w
The arithmetic operation unit 303 subtracts a j
predicted image supplied from the intra prediction unit !
314 or the motion prediction/compensation unit 315 via 5
the selection unit 316, from the image read from the
5 screen rearrangement buffer 302. The arithmetic
operation unit 303 then outputs the difference
information to the orthogonal transform unit 304.
[0164]
When intra encoding is performed on an image, for
10 example, the arithmetic operation unit 303 subtracts a
predicted image supplied from the intra prediction unit
i
314, from the image read from the screen rearrangement j
buffer 302. When inter encoding is performed on an image,
for example, the arithmetic operation unit 303 subtracts
15 a predicted image supplied from the motion
prediction/compensation unit 315, from the image read
from the screen rearrangement buffer 302.
[0165]
The orthogonal transform unit 304 performs an
20 orthogonal transform operation, such as a discrete cosine
transform or a Karhunen-Loeve transform, on the
difference information supplied from the arithmetic
operation unit 303. This orthogonal transform is
performed by any appropriate method. The orthogonal
25 transform unit 304 supplies the transform coefficient to
the quantization unit 305.
[0166]
The quantization unit 305 quantizes the transform
coefficient supplied from the orthogonal transform unit
30 304. Based on target bit rate value information supplied
from the rate control unit 317, the quantization unit 305
i i
I
1
47 I
.m. SP317970WO00
sets a quantization parameter, and performs quantization.
This quantization is performed by any appropriate method.
I The quantization unit 305 supplies the quantized
transform coefficient to the lossless encoding unit 306.
I
5 [0167] I
The lossless encoding unit 306 encodes the !
I
transform coefficient quantized at the quantization unit j
305 by an appropriate encoding method. Since the j
I
coefficient data has already been quantized under the i
10 control of the rate control unit 317, the bit rate I
becomes equal to the target value (or approximates the I
target value) that is set by the rate control unit 317. |
[0168] j
The lossless encoding unit 306 obtains information
15 indicating an intra prediction mode and the like from the
intra prediction unit 314, and obtains information
indicating an inter prediction mode, motion vector
information, and the like from the motion
prediction/compensation unit 315. The lossless encoding
20 unit 306 further obtains the filter coefficient and the
like used at the loop filter 311.
[0169]
The lossless encoding unit 306 encodes those
various kinds of information by an appropriate encoding
25 method, and incorporates the information into (or
multiplexes the information with) the header information
of the encoded data. The lossless encoding unit 306
supplies the encoded data obtained through the encoding
to the accumulation buffer 307 to accumulate the encoded
30 data.
[0170]
i
! I
i
3
I
48 j
-fc. SP317970WO00 w
The encoding method used by the lossless encoding
unit 306 may be variable-length encoding or arithmetic
I encoding, for example. The variable-length encoding may
be CAVLC (Context-Adaptive Variable Length Coding)
5 specified in H.2 64/AVC, for example. The arithmetic
encoding may be CABAC (Context-Adaptive Binary Arithmetic
Coding), for example. i
i
[0171] j
I The accumulation buffer 307 temporarily holds the
10 encoded data supplied from the lossless encoding unit 306.
The accumulation buffer 307 outputs the encoded data held
therein to a recording device (a recording medium) or a
transmission path or the like (not shown) in a later
stage, for example, at a predetermined time.
15 [0172]
The transform coefficient quantized at the
quantization unit 305 is also supplied to the inverse
quantization unit 308. The inverse quantization unit 308
inversely quantizes the quantized transform coefficient
2 0 by a method compatible with the quantization performed by
the quantization unit 305. The inverse quantization
method may be any method as long as the method is
compatible with the quantization operation performed by
the quantization unit 305. The inverse quantization unit
25 308 supplies the obtained transform coefficient to the
inverse orthogonal transform unit 309.
[0173]
The inverse orthogonal transform unit 309 performs
an inverse orthogonal transform on the transform
30 coefficient supplied from the inverse quantization unit
308, by a method compatible with the orthogonal transform
i
1
i
49
© SP317970WO00 j
1
i
operation performed by the orthogonal transform unit 304. j
P
This inverse orthogonal transform may be performed by any j
?
j method as long as the method is compatible with the ij I orthogonal transform operation performed by the I
5 orthogonal transform unit 304. The output subjected to 'j
the inverse orthogonal transform (the restored difference j
information) is supplied to the arithmetic operation unit
310.
[0174]
10 The arithmetic operation unit 310 obtains a locally
decoded image (a decoded image) by adding the predicted
image supplied from the intra prediction unit 314 or the
motion prediction/compensation unit 315 via the selection
unit 316 to the inverse orthogonal transform result
15 supplied from the inverse orthogonal transform unit 309 !
or the restored difference information. !
[0175] For example, when the difference information
corresponds to an image to be intra-encoded, the
20 arithmetic operation unit 310 adds the predicted image
supplied from the intra prediction unit 314 to the
difference information. When the difference information
corresponds to an image to be inter-encoded, the
arithmetic operation unit 310 adds the predicted image
25 supplied from the motion prediction/compensation unit 315
to the difference information, for example.
[0176]
The addition result (a decoded image) is supplied
to the loop filter 311 or the frame memory 312.
30 [0177]
The loop filter 311 includes a deblocking filter,
a I
I
50
^1^ SP317970WO00
an adaptive loop filter, and the like, and, where
appropriate, performs a filtering operation on the
decoded image supplied from the arithmetic operation unit
310. For example, the loop filter 311 removes block
5 distortions from the decoded image by performing, on the
decoded image, the same deblocking filtering operation as
that performed by the deblocking filter 111. Also, the
loop filter 311 improves image quality by performing a j
loop filtering operation using a Wiener filter on the 10 result of the deblocking filtering operation (the decoded j
image from which block distortions have been removed). j
[0178] i
Alternatively, the loop filter 311 may perform any
appropriate filtering operation on the decoded image.
15 Also, the loop filter 311 can supply the information such ]
as the filter coefficient used in the filtering operation
to the lossless encoding unit 306 to encode the j
information, where necessary. J
[0179] i
20 The loop filter 311 supplies the result of the j
filtering operation (the decoded image after the filtering operation) to the frame memory 312. As ;
described above, the decoded image that is output from ]
the arithmetic operation unit 310 can be supplied to the
25 frame memory 312 without passing through the loop filter 311. That is, the filtering operation by the loop filter
311 may be skipped.
[0180] The frame memory 312 stores the supplied decoded '
30 image, and supplies the stored decoded image as a
reference image to the selection unit 313 at a
:
51
•
SP317970WO00
predetermined time.
[0181]
The selection unit 313 selects a supply destination
of the reference image supplied from the frame memory 312.
5 In the case of an intra prediction, for example, the
selection unit 313 supplies the reference image supplied
from the frame memory 312, to the intra prediction unit
314. In the case of an inter prediction, for example,
the selection unit 313 supplies the reference image
10 supplied from the frame memory 312, to the motion
prediction/compensation unit 315.
[0182]
The intra prediction unit 314 performs intra
predictions (intra-screen predictions) to generate a
15 predicted image by using the pixel values in the current
picture that is the reference image supplied from the
frame memory 312 via the selection unit 313. In the
generation of the predicted image, a PU is used basically
as a unit of processing. The intra prediction unit 314
20 performs intra predictions in more than one mode (intra
prediction modes) that is prepared in advance. The intra
prediction unit 314 can perform the intra predictions not
only in the modes specified by the AVC encoding method
but also in any other appropriate modes.
25 [0183]
The intra prediction unit 314 generates predicted
images in all the candidate intra prediction modes,
evaluates the cost function values of the respective
predicted images by using the input image supplied from
30 the screen rearrangement buffer 102, and selects an
optimum mode. After selecting the optimum intra
52
© SP317970WO00
prediction mode, the intra prediction unit 314 supplies
the predicted image generated in the optimum intra
prediction mode to the selection unit 316.
[0184]
5 As described above, the intra prediction unit 314
also supplies the intra prediction mode information
indicating the adopted intra prediction mode, to the
lossless encoding unit 306 to encode the intra prediction
mode information, where appropriate.
10 [0185]
Using the input image supplied from the screen
rearrangement buffer 302, and the reference image
supplied from the frame memory 312 via the selection unit
313, the motion prediction/compensation unit 315 performs
15 motion predictions (inter predictions), and performs a
motion compensation operation in accordance with the
detected motion vectors, to generate a predicted image
(inter predicted image information). In the motion
predictions, a PU is used basically as a unit of
20 processing. The motion prediction/compensation unit 315
performs such inter predictions in more than one mode
(inter prediction modes) that is prepared in advance.
The motion prediction/compensation unit 315 can perform
the inter predictions not only in the modes specified by
25 the AVC encoding method but also in any other appropriate
modes.
[0186]
The motion prediction/compensation unit 315
generates predicted images in all the candidate inter
30 prediction modes, evaluates the cost function values of
the respective predicted images, and selects an optimum
53
^ SP317970WO00
mode. After selecting the optimiim inter prediction mode,
the motion prediction/compensation unit 315 supplies the
predicted image generated in the optimum intra prediction
mode to the selection unit 316.
5 [0187]
When the information indicating the selected inter
prediction mode and encoded data are decoded, the motion
prediction/compensation unit 315 supplies the necessary
information for performing operations in the inter
10 prediction mode, to the lossless encoding unit 306 to
encode the information.
[0188]
The selection unit 316 selects the supplier of the
predicted image to be supplied to the arithmetic
15 operation unit 303 and the arithmetic operation unit 310.
In the case of intra encoding, for example, the selection
unit 316 selects the intra prediction unit 314 as the
supplier of a predicted image, and supplies the predicted
image supplied from the intra prediction unit 314, to the
20 arithmetic operation unit 303 and the arithmetic
operation unit 310. In the case of inter encoding, for
example, the selection unit 316 selects the motion
prediction/compensation unit 315 as the supplier of a
predicted image, and supplies the predicted image
25 supplied from the motion prediction/compensation unit 315,
to the arithmetic operation unit 303 and the arithmetic
operation unit 310.
[0189]
Based on the bit rate of the encoded data
30 accumulated in the accumulation buffer 307, the rate
control unit 317 controls the quantization operation rate
54
# SP317970WO00
of the quantization unit 305 so as not to cause an
overflow or underflow.
[0190]
The assignment control unit 321 controls assignment
5 of a bit sequence (binarized data) to the predicted
motion vector select information (pmv_index) indicating
the adopted (selected) predicted motion vector at the
lossless encoding unit 306. The assignment control unit
321 controls the assignment of a bit sequence (binarized
10 data) in accordance with the distance between the current
picture and the anchor picture in terms of displaying
order.
[0191]
[Bit Sequence Assignment Control]
15 Next, the assignment control by the assignment
control unit 321 is described.
[0192]
Fig. 11 is a diagram for explaining example
candidate predicted motion vectors (PMVs) in AMVP. In
20 Fig. 11, X is the current block. MV_A, MV_B, MV_C, MV_D,
and MV_E are motion vectors (MVs) around the current
block of the current picture, and are spatial_pmv.
MV_Tmp is the MV of a co-located block located in the
same position as x in an encoded picture, and is
25 temporal_pmv. The anchor picture is normally the encoded
picture that is the closest to the current picture on the
LI side.
[0193]
Fig. 12 is a diagram showing an example motion
30 vector correlation in a case where the POC (Picture Order
Count) difference dPOC between the current picture and
55
^B, SP317970WO00
the anchor picture is 2. In the diagram, t+4 is the POC
of the current picture, and t+6 is the POC of the anchor
picture. The bold line in the current picture represents
the current block. The motion vectors (MVs) around the
5 current block are spatial_pmv. In this diagram, one of
the motion vectors is shown as MV_B. The bold line in
the anchor picture indicates the co-located block. The
motion vector MV_Tmp is temporal_pmv.
[0194]
10 Fig. 13 is a diagram showing an example motion
vector correlation in a case where the POC difference
dPOC between the current picture and the anchor picture
is 1.
[0195]
15 As can be seen from a comparison between the
example shown in Fig. 12 and the example shown in Fig. 13,
temporal_pmv (MV_Tmp) is closer to spatial_pmv (MV_B) in
the example shown in Fig. 13 than in the example shown in
Fig. 12. That is, the smaller the dPOC, the higher the
20 temporal_pmv prediction accuracy.
[0196]
In view of this, when the dPOC is small, the
assignment control unit 321 assigns a short bit sequence
to temporal_pmv, and, when the dPOC is large, the
25 assignment control unit 321 assigns a short bit sequence
to spatial_pmv. By controlling the bit sequence
assignment in the above manner, the assignment control
unit 321 can reduce the bit rate of the predicted motion
vector select information (pmv_index) for designating a
30 candidate PMV having a high possibility to achieve high
prediction accuracy.
56
^ SP317970WO00
[0197]
[Lossless Encoding Unit and Assignment Control
Unit]
Fig, 14 is a block diagram showing typical example
5 structures of the lossless encoding unit 306 and the
assignment control unit 321 shown in Fig. 10.
[0198]
As shown in Fig. 14, the lossless encoding unit 306
includes a motion vector storage unit 331, a predicted
10 motion vector selection unit 332, a difference motion
vector calculation unit 333, a table storage unit 334, a
binarization unit 335, a binarization unit 336, and an
entropy encoding unit 337.
[0199]
15 The assignment control unit 321 includes a distance
threshold acquirement unit 341, a distance calculation
unit 342, and a table selection unit 343.
[0200]
The motion vector storage unit 331 stores the
20 motion vector of the current PU to be processed. The
motion vector of the current PU is contained in optimum
mode information supplied from the motion
prediction/compensation unit 315. In an operation to be
performed for another region in a later stage in terms of
25 time, the motion vector storage unit 331 supplies stored
motion vectors as candidate predicted motion vectors to
the predicted motion vector selection unit 332.
[0201]
The predicted motion vector selection unit 332
30 acquires the motion vectors stored in the motion vector
storage unit 331, as the candidate predicted motion
57
^ SP317970WO00
vectors, and selects, from the candidate predicted motion
vectors, the motion vector closest to the motion vector
of the current PU contained in the optimum mode
information supplied from the motion
5 prediction/compensation unit 315, as the predicted motion
vector. The predicted motion vector selection unit 332
supplies the selected motion vector as the predicted
motion vector to the difference motion vector calculation
unit 333. The predicted motion vector selection unit 332
10 also supplies predicted motion vector select information
indicating which candidate predicted motion vector has
been selected as the predicted motion vector, to the
binarization unit 335.
[0202]
15 The difference motion vector calculation unit 333
calculates a difference motion vector that is the
difference between the predicted motion vector supplied
from the predicted motion vector selection unit 332 and
the motion vector of the current PU contained in the
20 optimum mode information supplied from the motion
prediction/compensation unit 315. The difference motion
vector calculation unit 333 supplies the calculated
difference motion vector to the binarization unit 336.
[0203]
25 The table storage unit 334 stores table information
that designates the bit sequence (binarized data) to be
assigned to the predicted motion vector select
information in accordance with the type of the vector
selected as the predicted motion vector. The table
30 storage unit 334 stores different sets of table
information indicating different methods of bit sequence
i.
58
g^ SP317970WO00
assignment in accordance with the types of vectors.
[0204]
Fig. 15 is a diagram showing an example of the
table information.
5 [0205]
In the table information in Fig. 15, the bit
sequence column shows the values of the bit sequences to
I be assigned. The SI coliamn shows the type of predicted
motion vectors to which the respective bit sequences are
10 to be assigned. The S2 column is the same as the SI
column, but differs from the SI column in the
correspondence relationship with the bit sequences.
[0206]
According to the assignment pattern of 81, for
15 example, when the predicted motion vector is the median
value (Median) (spatial_pmv) of MV_A, MV_B, and MV_C, the
bit sequence "1" of 1 bin in code length is assigned.
When the predicted motion vector is MV_A, the bit
sequence "010" of 3 bins in code length is assigned.
20 Likewise, when the predicted motion vector is MV_B, the
bit sequence "Oil" of 3 bins in code length is assigned.
When the predicted motion vector is MV_C, the bit
sequence "00100" of 5 bins in code length is assigned.
When the predicted motion vector is MV_E, the bit
25 sequence "00101" of 5 bins in code length is assigned.
When the predicted motion vector is MV_D, the bit
sequence "00110" of 5 bins in code length is assigned.
[0207]
When the predicted motion vector is MV_Tmp
30 (temporal_pmv), the bit sequence "00111" of 5 bins in
code length is assigned.
:
59
•
SP317970WO00
[0208]
That is, in the case of SI, a shorter bit sequence
is assigned to spatial_pmv, and a longer bit sequence is
assigned to temporal_pmv.
5 [0209]
According to the assignment pattern of S2, on the
other hand, when the predicted motion vector is MV_Tmp
(temporal_pmv), the bit sequence "1" of 1 bin in code
length is assigned, for example.
10 [0210]
When the predicted motion vector is the median
value (Median) (spatial_pmv) of MV_A, MV_B, and MV_C, the
bit sequence "010" of 3 bins in code length is assigned.
When the predicted motion vector is MV_A, the bit
15 sequence "Oil" of 3 bins in code length is assigned.
Likewise, when the predicted motion vector is MV_B, the
bit sequence "00100" of 5 bins in code length is assigned.
When the predicted motion vector is MV_C, the bit
sequence "00101" of 5 bins in code length is assigned,
2 0 When the predicted motion vector is MV_E, the bit
sequence "00110" of 5 bins in code length is assigned.
When the predicted motion vector is MV_D, the bit
sequence "00111" of 5 bins in code length is assigned.
[0211]
25 That is, in this case, a longer bit sequence is
assigned to spatial_pmv, and a shorter bit sequence is
assigned to temporal_pmv.
[0212]
Referring back to Fig. 14, the table storage unit
30 334 stores different sets of table information showing
different correspondence relationships, such as a table
60
^ SP317970WO00
containing the "bit sequence" column and the "SI" column,
and a table containing the "bit sequence" column and the
"S2" column.
[0213]
5 The code numbers shown in the example in Fig. 15
are virtual information for identifying the respective
bit sequences, and therefore, have any values.
[0214]
Although the table storage unit 334 stores sets of
10 table information in the above description, more than one
pattern should be prepared for assignment of bit
sequences to the types of predicted motion vectors, and
the assignment patterns of the respective bit sequences
may be put into a single set of table information as
15 shown in the example in Fig. 15.
[0215]
The binarization unit 335 shown in Fig. 14 refers
to the table information of the assignment pattern (Sn)
that is stored in the table storage unit 334 and is
20 selected by the table selection unit 343 of the
assignment control unit 321, and, in accordance with the
assignment pattern, binarizes the predicted motion vector
select information (pmv_index) supplied from the
predicted motion vector selection unit 332 (or transforms
25 the predicted motion vector select information into the
bit sequence assigned according to the table information).
After the binarization, the binarization unit 335
supplies the binarized data to the entropy encoding unit
337. !
30 [0216]
The binarization unit 336 binarizes various kinds
I
61
•
SP317970WO00
of information, such as the quantized coefficient data
supplied from the quantization unit 305, the optimum mode
information (such as the intra prediction mode
information) supplied from the intra prediction unit 314,
5 and filter information (containing the filter coefficient
and the like) supplied from the loop filter 311, and
supplies the binarized data to the entropy encoding unit
337.
[0217]
10 The binarization unit 336 also binarizes the
optimum mode information supplied from the motion
prediction/compensation unit 315, and supplies the
binarized data to the entropy encoding unit 337.
[0218]
15 The binarization unit 336 further binarizes the
difference motion vector supplied from the difference
motion vector calculation unit 333, and supplies the
binarized data to the entropy encoding unit 337.
[0219]
20 The binarization unit 336 also binarizes a distance
threshold dPOC_th supplied from the distance threshold
acquirement unit 341 of the assignment control unit 321,
and supplies the binarized data to the entropy encoding
unit 337.
25 [0220]
The entropy encoding unit 337 encodes the
respective sets of binarized data supplied from the
i
binarization unit 335 and the binarization unit 336,
generates information such as header information where
30 necessary, combines the respective sets of information,
and supplies the combined information as a stream to the
62
gj^ SP317970WO00
accumulation buffer 307 to accumulate the information,
[0221]
The distance threshold acquirement unit 341
acquires the distance threshold dPOC_th from outside.
5 This distance threshold dPOC_th is a threshold value for
the assignment control unit 321 to control the bit
sequence assignment pattern in accordance with the POC
difference dPOC between the current picture and the
anchor picture, or the distance between the current
10 picture and the anchor picture in terms of displaying
order.
I [0222]
This distance threshold dPOC_th may be set in any
appropriate manner. For example, the distance threshold
15 dPOC_th may be set in accordance with an instruction from
a user, may be determined in accordance with the
processing capability of the image encoding device 300,
or may be set in accordance with the contents of the
image.
20 [0223]
After acquiring the distance threshold dPOC th,
which has been determined in some manner, from outside,
the distance threshold acquirement unit 341 supplies the
distance threshold dPOC_th to the table selection unit
25 343, so that the distance threshold dPOC_th is used in
selecting a table (or selecting an assignment pattern).
[0224]
The distance threshold acquirement unit 341 also
supplies the acquired distance threshold dPOC_th to the
30 binarization unit 336 of the lossless encoding unit 306, i
so that the distance threshold dPOC_th is supplied to the
;
63
^ SP317970WO00
decoding side.
[0225]
The distance calculation unit 342 counts the POC of
each picture, and calculates the POC difference dPOC
5 between the current picture and the anchor picture. The
distance calculation unit 342 supplies the difference
dPOC to the table selection unit 343.
[0226]
The table selection unit 343 compares the
10 difference (distance) dPOC supplied from the distance
calculation unit 342 with the distance threshold dPOC_th
supplied from the distance threshold acquirement unit 341,
and, in accordance with the comparison result, selects a
table (a bit sequence assignment pattern) Sn.
15 [0227]
In the example case shown in Fig. 15, when the
distance dPOC is longer than the distance threshold
dPOC_th (or when the distance dPOC is equal to or longer
than the distance threshold dPOC_th), spatial_pmv is
20 prioritized, and accordingly, the table selection unit
343 selects SI, so that a shorter bit sequence is
assigned to spatial_pmv. When the distance dPOC is equal
to or shorter than the distance threshold dPOC_th (or
when the distance dPOC is shorter than the distance
25 threshold dPOC_th), temporal_pmv is prioritized, and
accordingly, the table selection unit 343 selects S2, so
that a shorter bit sequence is assigned to temporal_pmv.
[0228] j
The table selection unit 343 supplies the selection
30 result Sn to the binarization unit 335 of the lossless
encoding unit 306. j
I ^^
1 A SP317970WO00
[0229]
As described above, the assignment control unit 321
can appropriately control the bit sequence assignment in
accordance with the distance (dPOC) between the current
5 picture and the anchor picture in terms of displaying
order. That is, the assignment control unit 321 can
assign a bit sequence having a shorter code length to the
selection information about a motion vector having a
higher designation frequency. In other words, the
10 assignment control unit 321 can assign a bit sequence
having a longer code length to the selection information
about a motion vector having a lower designation
frequency. Accordingly, the lossless encoding unit 306
can reduce the bit rate of the predicted motion vector
15 select information (pmv_index). In this manner, the
image encoding device 300 can increase encoding
efficiency.
[0230]
[Encoding Operation Flow]
20 Next, the flow of each operation to be performed by
the above described image encoding device 300 is
described. Referring first to the flowchart shown in Fig.
16, an example flow of an encoding operation is described.
[0231]
25 There are cases where the processing data units in j
the respective steps differ from one another. Therefore, i
i in practice, the procedures of the respective steps might J
be carried out in parallel, or the sequence of the j
procedures might be changed. The same applies to the 30 other operations described later.
[0232] 65
•
SP317970WO00
In step S301, the A/D converter 301 performs an A/D
conversion on input images. In step S302, the screen
rearrangement buffer 302 stores the images subjected to
the A/D conversion, and rearranges the respective
5 pictures in encoding order, instead of displaying order.
[0233]
In step S303, the intra prediction unit 314
performs intra prediction operations in intra prediction
modes. In step S304, the motion prediction/compensation
10 unit 315 performs inter motion prediction operations to
perform motion predictions and motion compensation in
inter prediction modes.
[0234]
In step S305, the selection unit 316 determines an
15 optimum mode based on the respective cost function values
that are output from the intra prediction unit 314 and
the motion prediction/compensation unit 315. That is,
the selection unit 316 selects the predicted image
generated by the intra prediction unit 314 or the
20 predicted image generated by the motion
prediction/compensation unit 315.
[0235]
The selection information indicating which
predicted image has been selected is supplied to the
25 intra prediction unit 314 or the motion
prediction/compensation unit 315, whichever has generated
the selected predicted image. When the predicted image
generated in the optimum intra prediction mode is
selected, the intra prediction unit 314 supplies the
30 intra prediction mode information indicating the optimum
intra prediction mode and the like to the lossless
66
^H SP317970WO00
encoding unit 306. When the predicted image generated in
the optimum inter prediction mode is selected, the motion
prediction/compensation unit 315 outputs the information
indicating the optimum inter prediction mode to the
5 lossless encoding unit 306.
[0236]
In step S306, the arithmetic operation unit 303
calculates the difference between the images rearranged
in the procedure of step S302 and the predicted image
10 selected in the procedure of step S305. The predicted
image is supplied to the arithmetic operation unit 303
via the selection unit 316 from the motion
prediction/compensation unit 315 when an inter prediction
is performed, and from the intra prediction unit 314 when
15 an intra prediction is performed.
[0237]
The data amount of the difference data is smaller
than that of the original image data. Accordingly, the
data amount can be made smaller than in a case where
20 images are directly encoded.
[0238]
In step S307, the orthogonal transform unit 304
performs an orthogonal transform on the difference
information generated in the procedure of step S306.
25 Specifically, an orthogonal transform such as a discrete
cosine transform or a Karhunen-Loeve transform is
performed, and a transform coefficient is output. j
[0239]
In step S308, the quantization unit 305 quantizes
30 the orthogonal transform coefficient obtained in the
procedure of step S307.
67
•
SP317970WO00
[0240]
The difference information quantized in the
procedure of step S308 is locally decoded in the
following manner. In step S309, the inverse quantization
5 unit 308 inversely quantizes the quantized orthogonal
transform coefficient (also referred to as the quantized
coefficient) generated in the procedure of step S308,
using properties compatible with the properties of the
quantization unit 305. In step S310, the inverse
10 orthogonal transform unit 309 performs an inverse
orthogonal transform on the orthogonal transform
coefficient obtained in the procedure of step S307, using
properties compatible with the properties of the
orthogonal transform unit 304.
15 [0241]
In step S311, the arithmetic operation unit 310
adds the predicted image to the locally decoded-'
difference information, and generates a locally decoded
image (an image corresponding to the input to the
20 arithmetic operation unit 303). In step S312, the loop
filter 311 performs a loop filtering operation including
a deblocking filtering operation and an adaptive loop
filtering operation on the locally decoded image obtained
in the procedure of step S311, where appropriate.
25 [0242]
In step S313, the frame memory 312 stores the
decoded image subjected to the loop filtering operation i
in the procedure of step S312. Images that are not
subjected to filtering operations by the loop filter 311
30 are also supplied from the arithmetic operation unit 310,
and are stored into the frame memory 312.
I 68
1 ^n. SP317970WO00
[0243]
In step S314, the lossless encoding unit 306
encodes the transform coefficient quantized in the
procedure of step S308. That is, lossless encoding such
5 as variable-length encoding or arithmetic encoding is
performed on the difference image.
[0244]
The lossless encoding unit 306 also encodes the
quantization parameter calculated in step S308, and adds
10 the encoded quantization parameter to the encoded data.
The lossless encoding unit 306 also encodes the
information about the mode of the predicted image
selected in the procedure of step S305, and adds the
encoded information to the encoded data obtained by
15 encoding the difference image. That is, the lossless
encoding unit 306 also encodes the optimum intra
prediction mode information supplied from the intra
prediction unit 314 or the optimum inter prediction mode
supplied from the motion prediction/compensation unit 315,
2 0 and adds the encoded information to the encoded data.
[0245]
In step S315, the accumulation buffer 307
accumulates the encoded data that is output from the
lossless encoding unit 306. The encoded data accumulated
25 in the accumulation buffer 307 is read where appropriate,
and is transmitted to the decoding side via a
transmission path or a recording medium. I
[0246] 5
In step S316, based on the bit rate (bit generation
30 rate) of the encoded data that is accumulated in the
accumulation buffer 307 in the procedure of step S315,
69
^ SP317970WO00
the rate control unit 317 controls the quantization
operation rate of the quantization unit 305 so as not to
cause an overflow or underflow.
[0247]
5 When the procedure of step S316 is completed, the
encoding operation comes to an end.
[0248]
[Lossless Encoding Operation Flow]
Referring now to the flowchart shown in Fig. 17, an
10 example flow of the lossless encoding operation-performed
in step S314 of Fig. 16 is described.
[0249]
When the lossless encoding operation is started,
the binarization unit 33 6 and the entropy encoding unit
I
15 337 binarize and encode the quantized coefficient data in
step S321, binarize and encode the filter information in
step S3322, and binarize and encode the intra prediction
mode information about the intra-encoded region in step
S323.
20 [0250]
In step S324, the lossless encoding unit 306 and
the assignment control unit 321 binarize and encode the
inter prediction mode information about the inter-encoded
region.
25 [0251]
In step S325, the entropy encoding unit 337 j
combines the respective sets of encoded data into a
stream.
[0252]
30 After the procedure of step S325 is completed, the
entropy encoding unit 337 ends the lossless encoding
70
A SP317970WO00
operation. The operation then returns to step S314 of
Fig. 16, and moves on to step S315.
[0253]
[Flow of the Inter Prediction Mode Information Encoding
5 Operation]
Referring now to the flowchart shown in Fig. 18, an
example flow of the inter prediction mode information
encoding operation performed in step S324 of Fig. 17 is
described.
10 [0254]
When the inter prediction mode information encoding
operation is started, the distance threshold acquirement
unit 341 acquires the distance threshold dPOC_th in step
S331. In step S332, the binarization unit 336 and the
15 entropy encoding unit 337 binarize and encode the
distance threshold dPOC_th acquired in step S331, and
f
store the encoded distance threshold dPOC_th into the |
sequence parameter set (SPS). j
[0255] i
i
20 In step S333, the motion vector storage unit 331 j
1
stores the motion vector of the current PU contained in
the optimum mode information (inter prediction mode
information). In step S334, the predicted motion vector
selection unit 332 acquires the peripheral motion vectors
25 stored in the motion vector storage unit 331, as
candidate predicted motion vectors. In step S335, the
predicted motion vector selection unit 332 selects the
candidate predicted motion vector closest to the motion
vector of the current PU, as the predicted motion vector.
30 [0256]
In step S336, the distance calculation unit 342
:
71
j ^ SP317970WO00
calculates the distance dPOC between the current picture
and the anchor picture. In step S337, the table
selection unit 343 selects the table information
indicating a bit sequence assignment pattern, in
5 accordance with the magnitude relationship between the
distance dPOC determined in step S33 6 and the distance
threshold dPOC_th acquired in step S331.
[0257]
In step S338, the binarization unit 335 refers to
10 the table information that is stored in the table storage I
unit 334 and has been selected in step S337, and j
binarizes the predicted motion vector select information
indicating which candidate predicted motion vector has
been selected in step S335, In step S339, the entropy
15 encoding unit 337 encodes the binarized data of the
predicted motion vector select information binarized in
step S338.
[0258]
In step S340, the difference motion vector
20 calculation unit 333 determines the difference motion
vector that is the difference between the current motion
vector and the predicted motion vector. In step S341,
the binarization unit 33 6 and the entropy encoding unit
337 binarize and encode the difference motion vector
25 acquired in step S340.
[0259] i
In step S342, the binarization unit 336 and the ]
entropy encoding unit 337 binarize and encode other I
optimum mode information.
30 [0260]
After the procedure of step S342 is completed, the
:
1
72
•
SP317970WO00
entropy encoding unit 337 ends the inter prediction mode
information encoding operation. The operation then
returns to step S324 of Fig. 17, and moves on to step
S325.
5 [0261]
As the respective operations are performed in the
above described manner, the assignment control unit 321
can appropriately control the bit sequence assignment in
accordance with the distance (dPOC) between the current
10 picture and the anchor picture in terms of displaying
order. That is, the assignment control unit 321 can
assign a bit sequence having a shorter code length to the
selection information about a motion vector having a
higher designation frequency. In other words, the
15 assignment control unit 321 can assign a bit sequence
having a longer code length to the selection information
about a motion vector having a lower designation
frequency. Accordingly, the lossless encoding unit 306
can reduce the bit rate of the predicted motion vector
20 select information (pmv_index). In this manner, the
image encoding device 300 can increase encoding
efficiency.
[0262]
[Image Decoding Device]
25 Fig. 19 is a block diagram showing a typical i
example structure of an image decoding device. The image
decoding device 400 shown in Fig. 19 is a decoding device
that is compatible with the image encoding device 300 j
shown in Fig. 10. Data encoded by the image encoding
30 device 300 is supplied to the image decoding device 400
via a passage such as a transmission path or a recording
:
73
^ SP317970WO00
mediiom, and is then decoded.
[0263]
As shown in Fig, 19, the image decoding device 400
includes an accumulation buffer 401, a lossless decoding
5 unit 402, an inverse quantization unit 403, an inverse
orthogonal transform unit 404, an arithmetic operation
unit 405, a loop filter 406, a screen rearrangement
buffer 407, and a D/A converter 408. The image decoding
device 400 also includes a frame memory 409, a selection
10 unit 410, an intra prediction unit 411, a motion
prediction/compensation unit 412, and a selection unit
413.
[0264]
The image decoding device 400 further includes an
15 assignment control unit 421.
[0265]
The accumulation buffer 401 accumulates transmitted
encoded data. The encoded data has been encoded by the
image encoding device 300. The lossless decoding unit
20 402 reads the encoded data from the accumulation buffer
401 at a predetermined time, and decodes the encoded data
by a method compatible with the encoding method used by
the lossless encoding unit 306 shown in Fig. 10.
[0266]
25 When the current frame is an intra-encoded frame,
the header portion of the encoded data stores intra
prediction mode information. The lossless decoding unit
402 also decodes the intra prediction mode information, and supplies the resultant information to the intra
30 prediction unit 411. When the current frame is an inter- i
encoded frame, on the other hand, the header portion of
74
^ SP317970WO00
the encoded data stores motion vector information and
inter prediction mode information. The lossless decoding
unit 4 02 also decodes the motion vector information and
the inter prediction mode information, and supplies the
5 resultant information to the motion
prediction/compensation unit 412.
[0267]
The inverse quantization unit 403 inversely
quantizes the coefficient data (the quantized
I 10 coefficient) decoded by the lossless decoding unit 402,
i
by a method compatible with the quantization method used
by the quantization unit 305 shown in Fig. 10. That is,
the inverse quantization unit 403 inversely quantizes the
quantized coefficient by the same method as the method
15 used by the inverse quantization unit 308 shown in Fig.
10.
[0268]
The inverse quantization unit 403 supplies the
inversely-quantized coefficient data, or the orthogonal
20 transform coefficient, to the inverse orthogonal
transform unit 404. The inverse orthogonal transform
unit 4 04 performs an inverse orthogonal transform on the
orthogonal transform coefficient by a method compatible
with the orthogonal transform method used by the
25 orthogonal transform unit 304 shown in Fig. 10 (or by the j
same method as that used by the inverse orthogonal
transform unit 309 shown in Fig. 10) . The inverse
orthogonal transform unit 404 obtains decoded residual
error data equivalent to the residual error data from the
30 time prior to the orthogonal transform in the image
encoding device 300. For example, a fourth-order inverse
75
^ SP317970WO00
orthogonal transform is performed.
[0269]
The decoded residual error data obtained through
the inverse orthogonal transform is supplied to the
5 arithmetic operation unit 405. A predicted image is also
supplied to the arithmetic operation unit 405 from the
intra prediction unit 411 or the motion
prediction/compensation unit 412 via the selection unit
413.
10 [0270]
The arithmetic operation unit 405 adds the decoded
residual error data to the predicted image, and obtains
decoded image data corresponding to the image data from
the time prior to the predicted image subtraction
15 performed by the arithmetic operation unit 303 of the
image encoding device 300. The arithmetic operation unit
4 05 supplies the decoded image data to the loop filter
406.
[0271]
20 The loop filter 406 performs a loop filtering
operation including a deblocking filtering operation and
an adaptive loop filtering operation on the supplied
decoded image, where appropriate, and supplies the
resultant decoded image to the screen rearrangement
25 buffer 407.
[0272]
The loop filter 406 includes a deblocking filter,
an adaptive loop filter, and the like, and, where
appropriate, performs a filtering operation on the
30 decoded image supplied from the arithmetic operation unit
405. For example, the loop filter 406 removes block
I
•'
76
•
SP317970WO00
distortions from the decoded image by performing a
deblocking filtering operation on the decoded image.
Also, the loop filter 40 6 improves image quality by
performing a loop filtering operation using a Wiener
5 filter on the result of the deblocking filtering
operation (the decoded image from which block distortions
have been removed).
[0273]
Alternatively, the loop filter 406 may perform any
10 appropriate filtering operation on the decoded image.
The loop filter 406 may also perform a filtering
operation by using the filter coefficient supplied from
the image encoding device 300 shown in Fig. 10.
[0274]
15 The loop filter 406 supplies the result of the
filtering operation (the decoded image after the
filtering operation) to the screen rearrangement buffer
407 and the frame memory 4 09. The decoded image that is
output from the arithmetic operation unit 405 can be
20 supplied to the screen rearrangement buffer 407 and the
frame memory 409 without passing through the loop filter
406. That is, the filtering operation by the loop filter
40 6 may be skipped.
[0275]
25 The screen rearrangement buffer 407 performs image
rearrangement. Specifically, the frame sequence
rearranged in the encoding order by the screen
rearrangement buffer 302 shown in Fig. 10 is rearranged
in the original displaying order. The D/A converter 408
30 performs a D/A conversion on the images supplied from the
screen rearrangement buffer 407, and outputs the
I
11
•
SP317970WO00
converted images to a display (not shown) to display the
images.
[0276]
The frame memory 4 09 stores the supplied decoded
5 image, and supplies the stored decoded image as a
reference image to the selection unit 410 at a
predetermined time or in response to an external request
from the intra prediction unit 411 or the motion
prediction/compensation unit 412 or the like. i
10 [0277] I
The selection unit 410 selects a supply destination
of the reference image supplied from the frame memory 40 9,
When an intra-encoded image is decoded, the selection
unit 410 supplies the reference image supplied from the
15 frame memory 409, to the intra prediction unit 411. When
an inter-encoded image is decoded, the selection unit 410
supplies the reference image supplied from the frame
memory 409, to the motion prediction/compensation unit
412.
20 [0278]
Information that has been obtained by decoding the
header information and indicates an intra prediction mode
or the like is supplied from the lossless decoding unit
402 to the intra prediction unit 411, where appropriate.
25 The intra prediction unit 411 performs intra predictions
in the intra prediction modes used by the intra
prediction unit 314 shown in Fig. 10, by using the
reference image acquired from the frame memory 409. A
predicted image is thus generated. Like the intra
30 prediction unit 314 shown in Fig. 10, the intra
prediction unit 411 can perform the intra predictions not
78 I
^ SP317970WO00
only in the modes specified by the AVC encoding method
but also in any other appropriate modes.
[0279]
The intra prediction unit 411 supplies the
5 generated predicted image to the selection unit 413.
[0280]
The motion prediction/compensation unit 412 obtains,
from the lossless decoding unit 402, the information
obtained by decoding the header information (optimum mode
10 information, motion vector information, reference frame
information, a flag, respective parameters, and the like).
[0281]
The motion prediction/compensation unit 412
performs inter predictions in the inter prediction modes
15 used by the motion prediction/compensation unit 315 shown
in Fig. 10, by using the reference image acquired from
the frame memory 409. A predicted image is thus
generated. Like the motion prediction/compensation unit
315 shown in Fig. 10, the motion prediction/compensation
20 unit 412 can perform the intra predictions not only in
the modes specified by the AVC encoding method but also
in any other appropriate modes.
[0282]
Like the motion prediction/compensation unit 212,
25 the motion prediction/compensation unit 412 supplies the
generated predicted image to the selection unit 413.
[0283]
The selection unit 413 selects the supplier of the
predicted image to be supplied to the arithmetic
30 operation unit 405. Specifically, the selection unit 413
supplies the predicted image generated by the motion
79 j
^ SP317970WO00
prediction/compensation unit 412 or the intra prediction
unit 411, to the arithmetic operation unit 405.
[0284]
The assignment control unit 421 performs the same
5 bit sequence assignment control as that performed in the
image encoding device 300, so as to correctly debinarize
the binarized data of the predicted motion vector select j
information in the debinarization to be performed in the
lossless decoding unit 402.
I 10 [0285]
[Lossless Decoding Unit and Assignment Control Unit]
i
Fig. 2 0 is a block diagram showing typical example
structures of the lossless decoding unit 402 and the
assignment control unit 421 shown in Fig. 19.
15 [0286]
As shown in Fig. 20, the lossless decoding unit 402
includes an entropy decoding unit 431, a debinarization
unit 432, a table storage unit 433, a debinarization unit
434, a predicted motion vector selection unit 435, a
20 motion vector calculation unit 436, and a motion vector
storage unit 437.
[0287]
The assignment control unit 421 includes a distance
calculation unit 441 and a table selection unit 442.
25 [0288]
The entropy decoding unit 431 decodes a code stream
supplied from the accumulation buffer 4 01, and supplies
the binarized data to the debinarization unit 432 and the
debinarization unit 434.
30 [0289]
The debinarization unit 432 debinarizes the
80
^ SP317970WO00
#
binarized data of various kinds of information supplied
from the image encoding device 300, such as the quantized
coefficient data, the optimum mode information, the
filter information, the difference motion vector, and the
5 distance threshold dPOC th. The debinarization unit 432
supplies the obtained coefficient data to the inverse
quantization unit 403. The debinarization unit 432 also
supplies the obtained optimiom mode information to the
intra prediction unit 411 or the motion
10 prediction/compensation unit 412. The debinarization
unit 432 further supplies the obtained filter information
to the loop filter 406. The debinarization unit 432 also
supplies the obtained difference motion vector to the
motion vector calculation unit 436. The debinarization
15 unit 432 further supplies the obtained distance threshold
dPOC_th to the table selection unit 442 of the assignment
control unit 421.
[0290]
The table storage unit 433 stores table information
2 0 that indicates bit sequence assignment patterns for
predicted motion vector select information, like the
table information stored in the table storage unit 334 of
the image encoding device 300 as shown in Fig. 15, for
example. Like the table storage unit 334, the table
25 storage unit 433 stores more than one set of table
information (SI and S2, for example) indicating different
assignment patterns from each other. The table
information may be put into one table as in the case of
the table storage unit 334.
30 [0291]
The debinarization unit 434 refers to table
81
^ SP317970WO00
information Sn that is stored in the table storage unit j
433 and has been selected by the table selection unit 442 j
of the assignment control unit 421, and, in accordance j
with the assignment pattern, debinarizes the binarized
5 data of the predicted motion vector select information
supplied from the entropy decoding unit 431. The
debinarization unit 434 supplies the predicted motion
vector select information obtained through the
debinarization to the predicted motion vector selection
10 unit 435.
[0292]
The predicted motion vector selection unit 435
acquires the motion vectors (peripheral motion vectors)
that are located around the current PU and are stored in
15 the motion vector storage unit 437, as candidate
predicted motion vectors. From the candidate predicted
motion vectors, the predicted motion vector selection
unit 435 selects the candidate predicted motion vector
indicated by the predicted motion vector select
2 0 information supplied from the debinarization unit 434, as
the predicted motion vector. The predicted motion vector
selection unit 435 supplies the selected predicted motion
vector to the motion vector calculation unit 436.
[0293]
25 The motion vector calculation unit 436 calculates
the motion vector of the current PU by adding the
predicted motion vector supplied from the predicted
motion vector selection unit 435 to the difference motion
vector supplied from the debinarization unit 432. The
30 motion vector calculation unit 436 supplies the
calculated motion vector to the motion
82
^ SP317970WO00 I
I
prediction/compensation unit 412. The motion vector I
calculation unit 436 also supplies and stores the
calculated motion vector into the motion vector storage
unit 437.
5 [0294]
The motion vectors stored in the motion vector
storage unit 437 are used as the peripheral motion j
vectors (the candidate predicted motion vectors) in !
operations to be performed for regions to be processed in I
10 later stages than the current PU in terms of time. |
[0295]
The distance calculation unit 441 of the assignment
control unit 421 counts the POC of each picture, and
calculates the POC difference dPOC between the current
15 picture and the anchor picture. The distance calculation
unit 441 supplies the difference dPOC to the table
selection unit 442.
[0296]
The table selection unit 442 compares the
20 difference dPOC supplied from the distance calculation
unit 441 with the distance threshold dPOC_th that is
supplied from the debinarization unit 432 and has been
used at the time of encoding, and, in accordance with the
comparison result, selects a table (a bit sequence
25 assignment pattern) Sn. The table selection unit 442
supplies the selection result Sn to the debinarization
unit 434 of the lossless encoding unit 306.
[0297]
As described above, as the assignment control unit
30 421 appropriately controls the bit sequence assignment in
accordance with the distance (dPOC) between the current
1^
83
•
SP317970WO00 I
picture and the anchor picture in terms of displaying I
order, the lossless decoding unit 402 can reproduce the j
same bit sequence assignment as that used in the encoding,
and correctly decode the encoded data supplied from the
5 image encoding device 300. In short, the image decoding
device 400 can increase encoding efficiency.
[0298]
[Decoding Operation Flow]
Next, the flow of each operation to be performed by
10 the above described image decoding device 400 is
described. Referring first to the flowchart shown in Fig.
21, an example flow of a decoding operation is described.
[0299]
When the decoding operation is started, the
15 accumulation buffer 401 accumulates transmitted encoded
data in step S401. In step S402, the lossless decoding
unit 402 decodes the encoded data (encoded data generated
by the image encoding device 300 encoding image data)
supplied from the accumulation buffer 401.
20 [0300]
In step S403, the inverse quantization unit 403
inversely quantizes the quantized orthogonal transform
coefficient obtained through the decoding performed by
the lossless decoding unit 402, by a method compatible
25 with the quantization operation performed by the
quantization unit 305 shown in Fig. 10. In step S404,
the inverse orthogonal transform unit 404 performs an
inverse orthogonal transform on the orthogonal transform
coefficient obtained through the inverse quantization
30 performed by the inverse quantization unit 403, by a
method compatible with the orthogonal transform operation
i
I
84
^k SP317970WO00 I
I
; I
i I
I performed by the orthogonal transform unit 304 shown in i
Fig. 10. As a result, the difference information I
i
corresponding to the input to the orthogonal transform
unit 304 (or the output from the arithmetic operation
i 5 unit 303) shown in Fig. 10 is decoded.
[0301]
In step S405, the intra prediction unit 411 and the
motion prediction/compensation unit 412 perform
prediction operations, and generate predicted images.
10 [0302]
In step S406, the selection unit 413 selects a
predicted image generated in the procedure of step S405.
Specifically, the predicted image generated by the intra
prediction unit 411, or the predicted image generated by
15 the motion prediction/compensation unit 412 is supplied
to the selection unit 413. The selection unit 413
selects the supplied predicted image, and supplies the
predicted image to the arithmetic operation unit 405.
[0303]
20 In step S407, the arithmetic operation unit 405
adds the predicted image selected in step S406 to the
difference information obtained in the procedure of step
S404, In this manner, the original image data is decoded.
[0304]
25 In step S408, the loop filter 406 performs
filtering on the decoded image obtained in the procedure
of step S407, where appropriate.
[0305]
In step S409, the screen rearrangement buffer 407
30 rearranges the frames of the decoded images appropriately
subjected to the filtering in step S408. Specifically,
I
I
85
.^ SP317970WO00 |
I i
I the order of frames rearranged for encoding by the screen !
i
rearrangement buffer 302 of the image encoding device 300 i
(Fig. 10) is rearranged in the original displaying order.
I [0306]
5 In step S410, the D/A converter 408 performs a D/A
conversion on the decoded image data having the frames
rearranged in step S409. The decoded image data is
output to a display (not shown), and the images are
displayed.
10 [0307]
In step S411, the frame memory 409 stores the
decoded images appropriately subjected to the filtering
in step S40B.
[0308]
15 After the procedure of step S411 is completed, the
frame memory 4 09 ends the decoding operation.
[0309]
[Lossless Decoding Operation Flow]
Referring now to the flowchart shown in Fig. 22, an
20 example flow of the lossless decoding operation performed
in step S402 of Fig. 21 is described.
[0310]
When the lossless decoding operation is started,
the lossless decoding unit 402 and the assignment control
25 unit 421 decode and debinarize the inter prediction mode
information about the inter-encoded region in step S421.
[0311]
The entropy decoding unit 431 and the
debinarization unit 432 decode and debinarize the intra
30 prediction mode information about the intra-encoded
region in step S422, decode and debinarize the filter
86 j
•
SP317970WO00 I
I
information in step S423, and decode and debinarize the I
coefficient data in step S424. j
[0312] I
After the procedure of step S424 is completed, the !
5 debinarization unit 432 ends the lossless decoding i
operation. The operation then returns to step S402 of j
Fig. 21, and moves on to step S403.
[0313]
[Flow of the Inter Prediction Mode Information
10 Decoding Operation]
Referring now to the flowchart shown in Fig. 23, an
example flow of the inter prediction mode information
decoding operation performed in step S421 of Fig. 22 is
described.
15 [0314]
When the inter prediction mode information decoding
operation is started, the entropy decoding unit 431 and
the debinarization unit 432, in step S431, extract the
encoded data of the distance threshold dPOC_th from the
20 SPS, and decode and debinarize the extracted encoded data.
[0315]
In step S432, the distance calculation unit 441 of
the assignment control unit 421 calculates the distance
dPOC between the current picture and the anchor picture.
25 [0316]
In step S433, the table selection unit 442 of the
assignment control unit 421 compares the distance dPOC
between the current picture and the anchor picture
calculated in step S432 with the threshold distance
30 dPOC_th acquired in step S431, and selects the table
information Sn to be used in debinarization in accordance
87 !
^. SP317970WO00 1
I
i
with the magnitude relationship between the distance dPOC |
and the threshold distance dPOC_th. j
I
[0317] I
In step S434, the entropy decoding unit 431 decodes I
5 the predicted motion vector select information.
[0318]
In step S435, the debinarization unit 434 refers to !
i
the table that is stored in the table storage unit 433 I
!
and has been selected in step S433, and, in accordance j
I
10 with the bit sequence assignment pattern Sn, debmarizes I
i
the binarized data of the predicted motion vector select
information obtained in step S434 (or transforms a bit
sequence into information indicating the motion vector
corresponding to the bit sequence).
15 [0319]
In step S436, the predicted motion vector selection
unit 435 acquires the peripheral motion vectors stored in
the motion vector storage unit 437, as candidate
predicted motion vectors.
20 [0320]
In step S437, the predicted motion vector selection
unit 435 selects, from the candidate predicted motion
vectors obtained in step S436, the motion vector
designated by the predicted motion vector select
25 information obtained through the debinarization performed
in step S435, and sets the selected motion vector as the
predicted motion vector.
[0321]
In step S438, the entropy decoding unit 431 and the
30 debinarization unit 432 decode the encoded data of the
difference motion vector, and debinarize the resultant
88 I
© SP317970WO00 I
I
i
binarized data, to obtain the difference motion vector. I
I
[0322] I I
In step S439, the motion vector calculation unit i
436 calculates the motion vector of the current PU by j
5 adding the predicted motion vector selected in step S437 j
to the difference motion vector obtained in the procedure j
of step S438.
[0323]
In step S440, the motion vector storage unit 437
10 stores the motion vector of the current PU obtained in
the procedure of step S439.
[0324]
In step S441, the entropy decoding unit 431 and the
debinarization unit 432 decode the encoded data of other
15 optimum mode information, and debinarize the resultant
binarized data, to obtain the other optimum mode
information.
[0325]
After the procedure of step S441 is completed, the
20 debinarization unit 432 ends the inter prediction mode
information decoding operation. The operation then
returns to step S421 of Fig. 22, and moves on to step
S422.
[0326]
25 As described above, by performing the respective
operations, the lossless decoding unit 402 can reproduce
the same bit sequence assignment as that used in the
encoding, and correctly decode the encoded data supplied
from the image encoding device 300. In short, the image
30 decoding device 400 can increase encoding efficiency.
[0327]
89 I
^ . SP317970WO00 I w I
I
i
[Other Examples] |
II
Although example bit sequence assignment patterns |
have been described with reference to Fig. 15, other bit |
8
sequence assignment methods may be used. For example, |
5 when the distance dPOC is equal to or shorter than the j
I
distance threshold dPOC th (or when the distance dPOC is I I shorter than the distance threshold dPOC th), S3 in the j
~ I
table information shown in Fig. 24 may be selected, j
instead of S2 in Fig. 15.
10 [0328]
In the case of S3, the bit sequence to be assigned
when the predicted motion vector MV_Tmp (temporal_pmv)
and the bit sequence to be assigned when the predicted
motion vector is the median value (Median) (spatial_pmv)
15 of MV_A, MV_B, and MV_C are switched, compared with those
in SI.
[0329]
It is of course possible to prepare assignment
patterns other than that.
20 [0330]
In Fig. 15, exponential Golomb values are used as
bit sequential values, but any values may be used as the
bit sequential values, as long as those values can be
distinguished from one another. For example, unary codes
25 may be used as in the example table information shown in
Fig. 25.
[0331]
In the above description, the distance threshold
dPOC_th has been described as a variable value, but the
30 distance threshold dPOC_th may be a fixed value that is
determined in advance. For example, the image encoding
I M
S
i
90 1
^ SP317970WO00
device 300 and the image decoding device 400 may share
the distance threshold dPOC_th, which is a fixed value.
I In that case, the transmission and the reception of the
distance threshold dPOC_th are skipped. Alternatively,
5 the image encoding device 300 may supply the fixed
distance threshold dPOC_th to the image decoding device
400 prior to transmission of encoded data. Further, the
image decoding device 400 may supply the fixed distance
threshold dPOC_th to the image encoding device 300 prior
I 10 to encoding. Also, a device other than -the image
encoding device 300 and the image decoding device 400 may
supply the fixed distance threshold dPOC_th to the image
encoding device 300 and the image decoding device 400.
[0332]
15 Also, more than one distance threshold dPOC_th may
be prepared. In that case, three or more sets of table
information (code sequence assignment patterns Sn) are
prepared, and an appropriate pattern is selected in
accordance with the magnitude relationship between the
2 0 distance dPOC and the distance threshold dPOC_th.
[0333]
The distance threshold dPOC_th can be stored in any
position in a stream, other than the SPS. For example,
the distance threshold dPOC_th may be stored in the
25 picture parameter set (PPS) or the slice header. The
distance threshold dPOC_th may also be stored in the SEI
(Supplemental Enhancement Information) or the like.
[0334]
Further, the optimum mode information may be
30 transmitted to the decoding side, independently of
encoded data.
91 !
^t. SP317970WO00 j
[0335] j
Also, the distance threshold dPOC_th may be changed !
for each processing unit. For example, the distance l
threshold dPOC_th may be changed for each picture, may be |
[j
5 changed for each slice, may be changed for each CU, or I
i
may be changed m some other manner. |
[0336] I
i
<2. Second Embodiment> | I [Lossless Encoding Unit and Assignment Control Unit] |
10 In the above description, the bit sequence j
I assignment pattern is controlled in accordance with the
distance dPOC between the current picture and the anchor
picture. However, the present invention is not limited
to that, and the bit sequence assignment pattern may be
15 controlled based on the similarity between peripheral
motion vectors that are motion vectors of regions located
around the current PU.
[0337]
Fig. 2 6 is a block diagram showing typical example
20 structures of the lossless encoding unit 306 and the
assignment control unit 321 in such a case.
[0338]
In the example case illustrated in Fig. 2 6, the
lossless encoding unit 306 has basically the same
25 structure as that in the example case described with
reference to Fig. 14.
[0339]
In this case, however, the assignment control unit
321 includes a similarity threshold acquirement unit 541,
30 a similarity calculation unit 542, and a table selection
unit 543.
92 ^ . SP317970WO00
[0340] As in the case of the distance threshold dPOC th, I
the similarity threshold acquirement unit 541 acquires a I
similarity threshold MV_th from outside. This similarity 1
5 threshold MV_th is the threshold value for the assignment .•
control unit 321 to control the bit sequence assignment 1
i
pattern in accordance with the simxlarity between I
peripheral motion vectors. |
I
[0341] I
i I
I 10 Like the distance threshold dPOC th, this I
I ~ 1
I similarity threshold MV th may also be set in any I 1
appropriate manner. After acquiring the similarity j
I
threshold MV_th, which has been determined in some manner,
from outside, the similarity threshold acquirement unit
15 541 supplies the similarity threshold MV_th to the table
selection unit 543, so that the similarity threshold
MV_th is used in selecting a table (selecting an
assignment pattern). The similarity threshold
acquirement unit 541 also supplies the acquired
20 similarity threshold MV_th to the binarization unit 336
of the lossless encoding unit 306, so that the similarity
threshold MV_th is supplied to the decoding side.
[0342]
The similarity calculation unit 542 acquires, from
25 the motion vector storage unit 331, peripheral motion
vectors that are the motion vectors of regions located
around the current PU, and calculates the similarity
between the peripheral motion vectors by comparing the
peripheral motion vectors. In calculating the similarity,
30 any peripheral motion vectors may be used. The
similarity calculation unit 542 supplies the calculated
93 !
A SP317970WO00 j
similarity to the table selection unit 543. [0343] I
The table selection unit 543 compares the I
similarity supplied from the similarity calculation unit ij
5 542 with the similarity threshold MV_th supplied from the ij
I similarity threshold acquirement unit 541, and, in |
accordance with the comparison result, selects a table (a |
bit sequence assignment pattern) Sn. |
[0344] 1
10 Fig. 27 shows an example case where the upper right I
peripheral motion vector (MV_TR) in the current region is
similar to the lower left peripheral motion vector
(MV_BL) . In this case, the motion vector of the current
region x is expected to be also similar to those
15 peripheral motion vectors. Therefore, in the example
case illustrated in Fig. 15, the table selection unit 543
selects SI.
[0345] i
On the other hand. Fig. 2 8 shows an example case
I 20 where the upper right peripheral motion vector (MV_TR) is
not similar to the lower left peripheral motion vector
(MV_BL) . In this case, the motion vector of the current
region x is determined not to be necessarily similar to
those peripheral motion vectors. Therefore, in the
25 example case illustrated in Fig. 15, the table selection
unit 543 selects S2.
[0346]
There are cases where the region C and the region E
have not been encoded, or are not inter prediction blocks.
30 In view of this, the upper-right peripheral motion vector
and the lower-left peripheral motion vector that are used
I
i
I
I
: f
i
94 I
^ SP317970WO00 |
i I
in calculating a similarity may be peripheral motion j
I I
vectors of other regions, as shown in Figs. 2 9 through 31. j
[0347] I
j
For example, in a case where the peripheral motion j
5 vector MV_E is not available, the similarity between the j
I
\ motion vector MV_A of the region A and the motion vector
I MV_C of the region C may be calculated as shown in Fig.
29. Also, in a case where the peripheral motion vector
MV_C is not available, for example, the similarity
10 between the motion vector MV_E of the region E and the
motion vector MV_B of the region B may be calculated as
shown in Fig. 30. Further, in a case where the
peripheral motion vector MV_E and the peripheral motion
vector MV_C are not available, for example, the
15 similarity between the motion vector MV_A of the region A
and the motion vector MV_B of the region B may be
calculated as shown in Fig. 31. It is of course possible
to use peripheral motion vectors other than the above.
Which vector should be used instead may be acknowledged
2 0 beforehand between the encoding side and the decoding
side, and information indicating which vectors have been
used may be transmitted and received.
[0348]
The similarity between the upper-right peripheral
25 motion vector (MV_TR) and the lower-left peripheral
motion vector (MV_BL) is calculated as the absolute value
of the vector difference |MV_TR-MV_BL|. When the
absolute value is larger than the similarity threshold
MVth, there is a high possibility that temporal_pmv is
30 used, and therefore, a bit sequence of S2 is selected.
[0349]
I
H
ij
I
95 i
^1^ SP317970WO00 I
w
i
I Referring back to Fig. 26, the table selection unit
543 supplies the selection result Sn to the binarization
i unit 335 of the lossless encoding unit 306.
[0350]
i 5 As described above, the assignment control unit 321
can appropriately control the bit sequence assignment in
accordance with the similarity between peripheral motion
vectors. That is, the assignment control unit 321 can
also assign a bit sequence having a shorter code length
10 to the selection information about a motion vector having
a higher designation frequency in this case. In other
words, the assignment control unit 321 can assign a bit
sequence having a longer code length to the selection
information about a motion vector having a lower
15 designation frequency. Accordingly, the lossless
encoding unit 306 can reduce the bit rate of the
predicted motion vector select information (pmv_index).
In this manner, the image encoding device 300 can
increase encoding efficiency.
20 [0351]
[Flow of the Inter Prediction Mode Information
Encoding Operation]
Referring now to the flowchart shown in Fig. 32, an
example flow of the inter prediction mode information
25 encoding operation in this case is described.
[0352]
The operation to be performed in this case is also
basically the same as the operation performed in the case
where the distance threshold is used as described above
30 with reference to the flowchart shown in Fig. 18.
[0353]
i
I I
I
If
I
96 I
•
SP317970WO00 1
Ij
In step S531, however, the similarity threshold |
!l
acquirement unit 541 acquires the similarity threshold ij
it
MV_th, instead of the distance threshold. In step S532, |
1
the binarization unit 336 and the entropy encoding unit i
1
5 337 binarize and encode the similarity threshold MV_th j
•i
acquired in step S531, and store the similarity threshold j
I
MV th into the SPS. 1
— 1
[0354] I
! In step S533, the same procedure as step S333 is
I 10 carried out. In step S534, the similarity calculation
I unit 542 acquires peripheral motion vectors. In step
S535, the same procedure as step S335 is carried out.
[0355]
In step S536, the similarity calculation unit 542
15 calculates the similarity between the peripheral motion
vectors.
[0356]
In step S537, the table selection unit 543 selects
a table in accordance with the magnitude relationship
2 0 between the similarity and the similarity threshold MV_th.
[0357]
In steps S538 through S542, the same procedures as
steps S338 through S342 are carried out.
[0358]
25 After the procedure of step S542 is completed, the
entropy encoding unit 337 ends the inter prediction mode
information encoding operation. The operation then
returns to step S324 of Fig. 17, and moves on to step
S325.
30 [0359]
As the respective operations are performed in the
I
!!
I
97 [i
^ SP317970WO00 I
above described manner, the lossless encoding unit 306 ,;
can also reduce the bit rate of the predicted motion
vector select information (pmv_index) in this case. In :
this manner, the image encoding device 300 can increase ;
5 encoding efficiency. ••
[0360] ;
[Lossless Decoding Unit and Assignment Control •
Unit] j
Next, the image decoding device 400 compatible with |
10 the image encoding device 300 in this case is described. |
[0361] j
Fig. 33 is a block, diagram showing typical example i
i
structures of the lossless decoding unit 402 and the
1
assignment control unit 421 of the image decoding device j
i !
I 15 400 in this case. j
j [0362] j
I In this case, the lossless decoding unit 402 has j
i i
the same structure as that in the case illustrated in Fig. )
f
20, but the assignment control unit 421 includes a
20 similarity calculation unit 641 and a table selection
unit 642.
[0363]
The entropy decoding unit 431 decodes the encoded
data of the similarity threshold MV_th used in the image
25 encoding device 300. The debinarization unit 432
debinarizes the obtained binarized data, and supplies the
obtained similarity threshold MV_th to the table
selection unit 642.
[0364]
30 The similarity calculation unit 641 acquires
peripheral motion vectors stored in the motion vector
I
I
I
I
J
98 •
SP317970WO00 '
storage unit 437, and calculates the similarity between
the peripheral motion vectors. The method of calculating
the similarity is the same as the method used by the j
similarity calculation unit 542. |
1
5 [0365] I
The similarity calculation unit 641 supplies the i
I
calculated similarity to the table selection unit 642, I
s
[0366] I
The table selection unit 642 compares the
10 similarity with the similarity threshold MV_th, and, in
accordance with the magnitude relationship, selects a
table (a bit sequence assignment pattern) Sn, The method
of selecting a table is the same as the method used by
the table selection unit 543.
15 [0367]
The table selection unit 642 supplies the selection
result Sn to the debinarization unit 434 of the lossless
decoding unit 402.
[0368]
2 0 As described above, as the assignment control unit
421 appropriately controls the bit sequence assignment in
accordance with the similarity between peripheral motion
vectors, the lossless decoding unit 402 can reproduce the
same bit sequence assignment as that used in the encoding,
25 and correctly decode the encoded data supplied from the
image encoding device 300. In short, the image decoding
device 400 can increase encoding efficiency.
[0369]
[Flow of the Inter Prediction Mode Information
30 Decoding Operation]
Referring now to the flowchart shown in Fig. 34, an
I
Ij
J
•5
99 }
^ SP317970WO00 I
example flow of the inter prediction mode information 1
1
decoding operation in thxs case is described. |
[0370] j
i
The operation to be performed in this case is also j
I
5 basically the same as the operation performed in the j
example case illustrated in Fig. 23. I
[0371] I
In step S631, however, the entropy decoding unit j
I
431 and the debinarization unit 432 extract the encoded j
I
I 10 data of the similarity threshold MV_th from the SPS, and
decode and debinarize the extracted encoded data.
[0372]
The similarity calculation unit 641 of the
I assignment control unit 421 acquires peripheral motion
I 15 vectors from the motion vector storage unit 437 in step
i 632, and calculates the similarity between the acquired
i
I peripheral motion vectors in step S633.
j [0373]
\ In step S634, the table selection unit 642 selects
2 0 a table in accordance with the magnitude relationship
between the similarity and the similarity threshold MV_th.
[0374]
In steps S635 through S642, the same procedures as
steps S434 through S441 are carried out.
25 [0375]
After the procedure of step S642 is completed, the
debinarization unit 432 ends the inter prediction mode
information decoding operation. The operation then
returns to step S421 of Fig. 22, and moves on to step
30 S422.
[0376]
I
j
I ji
I 1
\ J
I j
I 100 i
I ^ SP317970WO00
I
As described above, by performing the respective j
i operations, the lossless decoding unit 402 can also i
I reproduce the same bit sequence assignment as that used 1
i I
in the encoding, and correctly decode the encoded data
5 supplied from the image encoding device 300 in this case.
i In short, the image decoding device 400 can increase
I encoding efficiency.
i [0377]
[Other Examples]
10 Like the distance threshold dPOC_th, the similarity
threshold MV_th may be a fixed value that is determined
in advance. Also, more than one similarity threshold
MV_th may be prepared. Further, like the distance
threshold dPOC_th, the similarity threshold MV_th can
15 also be stored in any position in a stream, other than
the SPS. Also, like the distance threshold dPOC_th, the
I similarity threshold MV_th can also be changed for each
arbitrary processing unit.
[0378]
2 0 <3. Third Embodiment>
I [Lossless Encoding Unit and Assignment Control Unit]
I
In the above description, a parameter threshold
such as the distance threshold or the similarity
threshold is transmitted from the image encoding device
25 300 to the image decoding device 400, and the same table
selection as that performed in the image encoding device
300 is performed in the image decoding device 400 using
the threshold value. However, the present invention is
not limited to them, and information indicating the table
30 that has been selected in the image encoding device 300
may be provided to the image decoding device 400. In the
i t
!
if
101
^1,. SP317970WO00 1
W
image decoding device 400 in this case, the same table as I
the table that has been selected in the image encoding |
I
device 300 is selected based on the information. %
'i
[0379] I
5 In this case, the image decoding device 400 can ^j
select the same table as that selected in the image 1
encoding device 300, without depending on the table 'i
i ,i
selection method used in the image encoding device 300. j
[0380] '
10 For example, the image encoding device 300 may
control the bit sequence assignment pattern based on the
cost function values of the results of predicted motion
vector selection, as described below. .,
[0381] I
15 Fig. 35 is a block diagram showing typical example ^
structures of the lossless encoding unit 306 and the
assignment control unit 321 in such a case.
[0382]
In this case, the lossless encoding unit 306 also
20 has the same structure as that of the above described
other embodiments, but the assignment control unit 321
calculates the cost function values of the results of the
predicted motion vector selection performed by the
lossless encoding unit 306, and controls the bit sequence
25 assignment pattern based on the cost function values.
[0383]
The assignment control unit 321 includes a cost
function value calculation unit 741, a cost function
value storage unit 742, and a table selection unit 743. ;j
'I
30 [0384] I
The cost function value calculation unit 741 |
i
102
^ SP317970WO00
acquires the predicted motion vector select information
from the predicted motion vector selection unit 332, and
calculates the cost function values of selection results
for table (Sn) selection. The cost function value
5 calculation unit 741 supplies the calculated cost
function values for Sn to the cost function value storage
unit 742.
[0385]
The cost function value storage unit 742 aggregates
10 the supplied cost functions values for Sn in each table
stored in the table storage unit 334, and then stores the
cost function values for Sn. The aggregation result is
used in selecting tables for regions to be processed in
later stages in terms of time than the current PU, which
15 is the current object to be processed.
[0386]
The table selection unit 743 acquires the result of
aggregation of the cost function values for Sn in each of
the tables stored in the cost function value storage unit
20 742. Based on the aggregation result, the table
selection unit 743 selects a table (Sn), and supplies the
selection result to the binarization unit 335.
j [0387]
The binarization unit 335 refers to the table
25 information that is stored in the table storage unit 334
I and has been selected by the table selection unit 743,
I and binarizes the predicted motion vector select
i
information. The binarized data is supplied to the
I
I entropy encoding unit 337, and is then encoded.
I 30 [0388]
I The table selection unit 743 also supplies the
j
I
I
i
I
103
^ SP317970WO00
^^
i
selection result Sn to the binarization unit 336.
Specifically, the table selection result Sn from the
table selection unit 743 is binarized by the binarization
unit 336, is encoded by the entropy encoding unit 337,
5 and is then supplied to the image decoding device 400.
[0389]
As described above, the assignment control unit 321
can appropriately control the bit sequence assignment in
accordance with the cost function values of the results
10 of predicted motion vector selection. That is, the
assignment control unit 321 can also assign a bit
sequence having a shorter code length to the selection
information about a motion vector having a higher
designation frequency in this case. In other words, the
15 assignment control unit 321 can assign a bit sequence
having a longer code length to the selection information
about a motion vector having a lower designation
frequency. Accordingly, the lossless encoding unit 306
can reduce the bit rate of the predicted motion vector
20 select information (pmv_index). In this manner, the
image encoding device 300 can increase encoding
efficiency.
[0390]
[Flow of the Inter Prediction Mode Information
25 Encoding Operation]
Referring now to the flowchart shown in Fig. 3 6, an
example flow of the inter prediction mode information
encoding operation in this case is described.
[0391]
30 The operation to be performed in this case is also
basically the same as the operation performed in the case
104
^ SP317970WO00
where the distance threshold is used as described above
with reference to the flowchart shown in Fig. 18.
[0392]
In steps S731 through S733, the same procedures as
5 steps S333 through S335 are carried out.
[0393]
In step S734, the table selection unit 743 selects
a table based on (the aggregation result in each table
of) the cost function values of the past regions stored
10 in the cost function value storage unit 742.
[0394]
In step S735, the binarization unit 336 and the
entropy encoding unit 337 binarize and encode the table
select information (the result of table selection)
15 obtained in step S734, and stores the table select
information into the SPS.
[0395]
In steps S736 through S739, the same procedures as
steps S338 through S341 are carried out.
20 [0396]
In step S740, the cost function value calculation
unit 741 calculates the cost function values
corresponding to each table with respect to the predicted
motion vector select information. In step S741, the cost
25 function value storage unit 742 aggregates and stores the
cost function values calculated in step S740 for each
table.
[0397]
In step S742, the same procedure as step S342 is
30 carried out. After the procedure of step S742 is
completed, the entropy encoding unit 337 ends the inter
I
j
I
!
105
© SP317970WO00
prediction mode information encoding operation. The
operation then returns to step S324 of Fig. 17, and moves
on to step S325.
[0398]
5 As the respective operations are performed in the
above described manner, the lossless encoding unit 306
can also reduce the bit rate of the predicted motion
vector select information (pmv_index) in this case. In
this manner, the image encoding device 300 can increase
10 encoding efficiency.
[0399]
[Lossless Decoding Unit and Assignment Control
Unit]
Next, the image decoding device 400 compatible with
15 the image encoding device 300 in this case is described.
[0400]
Fig. 37 is a block diagram showing a typical
example structure of the lossless decoding unit 402 of
the image decoding device 400 in this case. In this case,
20 the image decoding device 400 selects a table by using
the table select information Sn supplied from the image
encoding device 300, and therefore, the assignment
control unit 421 is removed.
[0401]
25 As shown in Fig. 37, unlike the lossless decoding
unit 402 in the example case illustrated in Fig. 20, the
lossless decoding unit 402 in this case includes a
debinarization unit 832 in place of the debinarization
unit 432, and a debinarization unit 834 in place of the
30 debinarization unit 434.
[0402]
!
;
;
106
JH SP317970WO00
Like the debinarization unit 432, the
debinarization unit 832 debinarizes the binarized data of
the coefficient data, the optimum mode information, the
filter information, the difference motion vector, and the
5 like, and also debinarizes the binarized data of the
table select information Sn. The debinarization unit 832
supplies the table select information Sn obtained through
the debinarization, to the debinarization unit 834.
[0403]
10 The debinarization unit 834 refers to the table
information that is stored in the table storage unit 433
and is indicated by the table select information Sn
supplied from the debinarization unit 832, and
debinarizes the binarized data that is formed by
15 binarizing the predicted motion vector select information
and is supplied from the entropy decoding unit 431.
[0404]
The debinarization unit 834 supplies the predicted
motion vector select information obtained through the
20 debinarization, to the predicted motion vector selection
unit 435.
[0405]
In this manner, the lossless decoding unit 402
appropriately debinarizes the binarized data of the
25 predicted motion vector select information, by using the
result of table information selection performed in the
image encoding device 300, the result being supplied from
the image encoding device 300. That is, the lossless
decoding unit 402 can also reproduce the same bit
30 sequence assignment as that used in the encoding, and
correctly decode the encoded data supplied from the image
i
107
^-^ SP317970WO00 I
encoding device 300 m this case. In short, the image
decoding device 400 can increase encoding efficiency.
[0406]
[Flow of the Inter Prediction Mode Information
5 Decoding Operation]
Referring now to the flowchart shown in Fig. 38, an
example flow of the inter prediction mode information
decoding operation in this case is described.
[0407]
10 The operation to be performed in this case is also
basically the same as the operation performed in the
example case illustrated in Fig. 23.
[0408]
In step S831, however, the entropy decoding unit
15 431 and the debinarization unit 832 extract the encoded
data of the table select information Sn from the SPS, and
decode and debinarize the extracted encoded data. That
is, a table is selected at this point,
[0409]
20 In steps S832 through S839, the same procedures as
steps S434 through S441 are carried out.
[0410]
After the procedure of step S839 is completed, the
debinarization unit 832 ends the inter prediction mode
25 information decoding operation. The operation then
returns to step S421 of Fig. 22, and moves on to step
S422.
[0411]
As described above, by performing the respective
30 operations, the lossless decoding unit 402 can also
reproduce the same bit sequence assignment as that used
!
108
^ SP317970WO00
in the encoding, and correctly decode the encoded data
supplied from the image encoding device 300 in this case.
In short, the image decoding device 400 can increase j
encoding efficiency. I
5 [0412]
[Other Examples] I
In the above description, the cost function values
are calculated based on the predicted motion vector
select information about the regions that have been
10 processed in the past, and the table information about
the current region to be processed is selected by using
the cost function values. However, the table information
about the current region to be processed may be selected
by using the cost function value of the predicted motion
15 vector select information about the current region to be
processed.
[0413]
Although the table select information Sn is stored
into the SPS in the above description, the table select
2 0 information Sn may be stored in any position in a stream
other than the SPS, like the distance threshold dPOC_th.
[0414]
The method of transmitting the table select
information Sn from the image encoding device 300 to the
25 image decoding device 400 can be applied in the above
described case where the image encoding device 300
selects a table by using the distance threshold dPOC_th,
or in the above described case where the image encoding
device 300 selects a table by using the similarity
30 threshold MV_th. In either case, however, the image
decoding device 400 selects a table based on the table
f
109
© SP317970WO00
select information Sn supplied from the image encoding
device 300, as described above with reference to Fig. 37.
[0415]
In any of the above described cases, the switching
5 of tables can be performed for each arbitrary processing
unit, such as each PU (or each divided part thereof),
each slice, each picture, or each sequence.
[0416]
<4. Fourth Embodiment>
10 [Personal Computer]
The above described series of operations can be
performed by hardware or can be performed by software.
In this case, the operations may be performed by the
computer shown in Fig. 39, for example.
15 [0417]
In Fig. 39, the CPU (Central Processing Unit) 901
of the personal computer 900 performs various kinds of
operations in accordance with a program stored in a ROM
(Read Only Memory) 902 or a program loaded into a RAM
20 (Random Access Memory) 903 from a storage unit 913. The
data necessary for the CPU 901 to perform various kinds
of operations is also stored in the RAM 903 where
necessary.
[0418]
25 The CPU 901, the ROM 902, and the RAM 903 are
connected to one another via a bus 904. An input/output
interface 910 is also connected to the bus 904.
[0419]
The input/output interface 910 has the following
30 components connected thereto: an input unit 911 formed
with a keyboard, a mouse, or the like; an output unit 912
110
j ^ SP317970WO00
formed with a display such as a CRT (Cathode Ray Tube) or
a LCD (Liquid Crystal Display), and a speaker; the
storage unit 913 formed with a hard disk or the like; and
a communication unit 914 formed with a modem. The
5 communication unit 914 performs communications via
networks including the Internet.
[0420]
A drive 915 is also connected to the input/output
interface 910 where necessary, and a removable mediiom 921
10 such as a magnetic disk, an optical disk, a
magnetooptical disk, or a semiconductor memory is mounted
on the drive as appropriate. A computer program read
from such a removable disk is installed in the storage
unit 913 where necessary.
15 [0421]
When the above described series of operations are
performed by software, a program to form the software is
installed from a network or a recording medium.
[0422]
20 As shown in Fig. 39, this recording medium is
formed with the removable medium 921 that is distributed
for delivering the program to users separately from the
device, such as a magnetic disk (including a flexible
disk), an optical disk (including a CD-ROM (Compact Disc
25 - Read Only Memory) or a DVD (Digital Versatile Disc)), a
magnetooptical disk (including an MD (Mini Disc)) , or a
semiconductor memory, which has the program recorded
thereon. Alternatively, the recording mediiom may be
formed with the ROM 902 having the program recorded
30 therein or a hard disk included in the storage unit 913. j
Such a recording medium is incorporated beforehand into
Ill
^ SP317970WO00
the device prior to the delivery to users.
[0423]
The program to be executed by the computer may be a
program for carrying out processes in chronological order
5 in accordance with the sequence described in this
specification, or a program for carrying out processes in
parallel or whenever necessary such as in response to a
call.
[0424]
10 In this specification, the step written in the
program to be recorded in a recording medium includes
operations to be performed in parallel or independently
of one another if not necessarily in chronological order,
as well as operations to be performed in chronological
15 order in accordance with the sequence described herein.
[0425]
In this specification, a "system" means an entire
apparatus formed with two or more devices (apparatuses).
[0426]
2 0 Also, in the above described examples, any
structure described as one device (or one processing
unit) may be divided into two or more devices (or
processing units). Conversely, any structure described
as two or more devices (or processing units) may be
25 combined to form one device (or one processing unit).
Also, it is of course possible to add a structure other
than the above described ones to the structure of any of
the devices (or any of the processing units). Further,
as long as the structure and function of the entire
30 system remain the same, part of the structure of a device
(or a processing unit) may be incorporated into another
1.
^ 112
^ SP317970WO00
• device (or another processing unit). That is, I
embodiments of the present technique are not limited to I
the above described embodiments, and various
modifications may be made to them without departing from
5 the scope of the technique.
[0427]
For example, the above described image encoding
device 300 and the image decoding device 400 can be
applied to any electronic apparatuses. The following is
10 a description of such examples.
[0428]
<5. Fifth Embodiment>
[Television Receiver]
Fig. 4 0 is a block diagram showing a typical
15 example structure of a television receiver using the
image decoding device 400.
[0429]
The television receiver 1000 shown in Fig. 40
includes a terrestrial tuner 1013, a video decoder 1015,
20 a video signal processing circuit 1018, a graphic
generation circuit 1019, a panel drive circuit 1020, and
a display panel 1021.
[0430]
The terrestrial tuner 1013 receives a broadcast
25 wave signal of analog terrestrial broadcasting via an
antenna, and demodulates the signal to obtain a video
signal. The terrestrial tuner 1013 supplies the video
signal to the video decoder 1015. The video decoder 1015
performs a decoding operation on the video signal
30 supplied from the terrestrial tuner 1013, and supplies
the resultant digital component signal to the video
113
gg^ SP317970WO00
signal processing circuit 1018.
[0431]
The video signal processing circuit 1018 performs
predetermined processing such as denoising on the video
5 data supplied from the video decoder 1015, and supplies
the resultant video data to the graphic generation
circuit 1019.
[0432]
The graphic generation circuit 1019 generates video
10 data of a show to be displayed on the display panel 1021,
or image data by performing an operation based on an
application supplied via a network. The graphic
generation circuit 1019 supplies the generated video data
or image data to the panel drive circuit 1020. The
15 graphic generation circuit 1019 also generates video data
(graphics) for displaying a screen to be used by a user
to select an item, and superimposes the video data on the
video data of the show. The resultant video data is
supplied to the panel drive circuit 1020 where
2 0 appropriate.
[0433]
Based on the data supplied from the graphic
generation circuit 1019, the panel drive circuit 1020
drives the display panel 1021, and causes the display
25 panel 1021 to display the video image of the show and
each screen described above.
[0434]
The display panel 1021 is formed with an LCD
(Liquid Crystal Display) or the like, and displays the
30 video image of a show or the like under the control of
the panel drive circuit 1020.
•
114
•
SP317970WO00
[0435] I
The television receiver 1000 also includes an audio j
A/D (Analog/Digital) converter circuit 1014, an audio |
signal processing circuit 1022, an echo I
5 cancellation/voice synthesis circuit 1023, an audio i
amplifier circuit 1024, and a speaker 1025.
[0436]
The terrestrial tuner 1013 obtains not only a video
signal but also an audio signal by demodulating a
10 received broadcast wave signal. The terrestrial tuner
1013 supplies the obtained audio signal to the audio A/D
converter circuit 1014.
[0437]
The audio A/D converter circuit 1014 performs an
15 A/D converting operation on the audio signal supplied
from the terrestrial tuner 1013, and supplies the
resultant digital audio signal to the audio signal
processing circuit 1022.
[0438]
20 The audio signal processing circuit 1022 performs
predetermined processing such as denoising on the audio
data supplied from the audio A/D converter circuit 1014,
and supplies the resultant audio data to the echo
cancellation/voice synthesis circuit 1023.
25 [0439]
The echo cancellation/voice synthesis circuit 1023
supplies the audio data supplied from the audio signal
processing circuit 1022 to the audio amplifier circuit
1024.
30 [0440]
The audio amplifier circuit 1024 performs a D/A
115
1^ SP317970WO00
converting operation and an amplifying operation on the
audio data supplied from the echo cancellation/voice
synthesis circuit 1023. After adjusted to a
predetermined sound volume, the sound is output from the
5 speaker 1025.
[0441]
The television receiver 1000 further includes a
digital tuner 1016 and an MPEG decoder 1017.
[0442]
10 The digital tuner 1016 receives a broadcast wave
signal of digital broadcasting (digital terrestrial
broadcasting or digital BS (Broadcasting Satellite)/CS
(Communications Satellite) broadcasting) via the antenna,
and demodulates the broadcast wave signal, to obtain an
15 MPEG-TS (Moving Picture Experts Group-Transport Stream).
The MPEG-TS is supplied to the MPEG decoder 1017.
[0443]
The MPEG decoder 1017 descrambles the MPEG-TS
supplied from the digital tuner 1016, and extracts the
2 0 stream containing the data of the show to be reproduced
(to be viewed). The MPEG decoder 1017 decodes the audio
packet forming the extracted stream, and supplies the
resultant audio data to the audio signal processing
circuit 1022. The MPEG decoder 1017 also decodes the
25 video packet forming the stream, and supplies the
resultant video data to the video signal processing
circuit 1018. The MPEG decoder 1017 also supplies EPG
(Electronic Program Guide) data extracted from the MPEGTS
to a CPU 1032 via a path (not shown).
30 [0444]
The television receiver 1000 uses the above
•
;
I
116
•
SP317970WO00
described image decoding device 400 as the MPEG decoder
1017, which decodes the video packet as described above.
The MPEG-TS transmitted from a broadcast station or the
like has been encoded by the image encoding device 300.
5 [0445]
As in the case of the image decoding device 400,
the MPEG decoder 1017 appropriately controls bit sequence
assignment in accordance with a predetermined parameter.
Accordingly, the MPEG decoder 1017 can reproduce the same
10 bit sequence assignment as that used in the encoding, and
correctly decode encoded data supplied from the encoding
side. In this manner, the MPEG decoder 1017 can increase
encoding efficiency for encoded data.
[0446] i
15 The video data supplied from the MPEG decoder 1017
is subjected to predetermined processing at the video i
signal processing circuit 1018, as in the case of the video data supplied from the video decoder 1015. At the graphic generation circuit 1019, generated video data and I
20 the like are superimposed on the video data where
appropriate. The resultant video data is supplied to the
display panel 1021 via the panel drive circuit 1020, and
the image is displayed.
[0447] i
25 The audio data supplied from the MPEG decoder 1017 j
is subjected to predetermined processing at the audio
signal processing circuit 1022, as in the case of the
audio data supplied from the audio A/D converter circuit
1014. The resultant audio data is supplied to the audio
30 amplifier circuit 1024 via the echo cancellation/voice
synthesis circuit 1023, and is subjected to a D/A j
117
•
SP317970WO00
converting operation or an amplifying operation. As a
result, a sound that is adjusted to a predetermined sound
level is output from the speaker 1025.
[0448]
5 The television receiver 1000 also includes a
microphone 1026 and an A/D converter circuit 1027.
[0449]
The A/D converter circuit 1027 receives a signal of
a user's voice captured by the microphone 102 6 provided
10 for voice conversations in the television receiver 1000.
The A/D converter circuit 1027 performs an A/D converting
operation on the received audio signal, and supplies the
resultant digital audio data to the echo
cancellation/voice synthesis circuit 1023.
15 [0450]
When audio data of a user (a user A) of the
television receiver 1000 is supplied from the A/D
converter circuit 1027, the echo cancellation/voice
synthesis circuit 1023 performs echo cancellation on the
2 0 audio data of the user A, and combines the audio data
with other audio data or the like. The resultant audio
data is output from the speaker 1025 via the audio
amplifier circuit 1024.
[0451]
25 The television receiver 1000 further includes an
audio codec 1028, an internal bus 1029, an SDRAM
(Synchronous Dynamic Random Access Memory) 1030, a flash
memory 1031, the CPU 1032, a USB (Universal Serial Bus)
I/F 1033, and a network I/F 1034.
30 [0452]
The A/D converter circuit 1027 receives the signal
118
•
SP317970WO00
of the user's voice captured by the microphone 1026
provided for voice conversations in the television
receiver 1000. The A/D converter circuit 1027 performs
an A/D converting operation on the received audio signal,
5 and supplies the resultant digital audio data to the
audio codec 1028.
[0453]
The audio codec 1028 transforms the audio data
supplied from the A/D converter circuit 1027 into data in
10 a predetermined format for transmission via a network,
and supplies the result to the network I/F 1034 via the
internal bus 1029.
[0454]
The network I/F 1034 is connected to a network via
15 a cable attached to a network terminal 1035. The network
I/F 1034 transmits the audio data supplied from the audio :
codec 1028 to another device connected to the network,
for example. The network I/F 1034 also receives, via the
network terminal 1035, audio data transmitted from
2 0 another device connected to the network, and supplies the
audio data to the audio codec 1028 via the internal bus
1029.
[0455]
The audio codec 1028 transforms the audio data
25 supplied from the network I/F 1034 into data in a i
predetermined format, and supplies the result to the echo i
cancellation/voice synthesis circuit 1023. !
[0456]
The echo cancellation/voice synthesis circuit 1023
30 performs echo cancellation on the audio data supplied
from the audio codec 1028, and combines the audio data
119
JH SP317970WO00
with other audio data or the like. The resultant audio
data is output from the speaker 1025 via the audio
amplifier circuit 1024.
[0457]
5 The SDRAM 1030 stores various kinds of data
necessary for the CPU 1032 to perform processing.
[0458]
The flash memory 1031 stores the program to be
executed by the CPU 1032. The program stored in the
10 flash memory 1031 is read by the CPU 1032 at a
predetermined time, such as when the television receiver
1000 is activated. The flash memory 1031 also stores EPG
data obtained through digital broadcasting, data obtained
from a predetermined server via a network, and the like.
15 [0459]
For example, the flash memory 1031 stores a MPEG-TS
containing content data obtained from a predetermined
server via a network, under the control of the CPU 1032.
The flash memory 1031 supplies the MPEG-TS to the MPEG
20 decoder 1017 via the internal bus 1029, under the control
of the CPU 1032, for example.
[0460]
The MPEG decoder 1017 processes the MPEG-TS, as in
the case of the MPEG-TS supplied from the digital tuner
25 1016. In this manner, the television receiver 1000
receives the content data formed with a video image and a
sound via the network, and decodes the content data by
using the MPEG decoder 1017, to display the video image
and output the sound.
30 [0461]
The television receiver 1000 also includes a light
i
120
^ SP317970WO00
receiving unit 1037 that receives an infrared signal
transmitted from a remote controller 1051.
[0462]
The light receiving unit 1037 receives an infrared
5 ray from the remote controller 1051, and performs
demodulation. The light receiving unit 1037 outputs a
control code indicating the contents of a user operation
obtained through the demodulation, to the CPU 1032.
[0463]
10 The CPU 1032 executes the program stored in the
flash memory 1031, and controls the entire operation of
the television receiver 1000 in accordance with the
control code and the like supplied from the light
receiving unit 1037. The respective components of the
15 television receiver 1000 are connected to the CPU 1032
via paths (not shown).
[0464]
The USB I/F 1033 exchanges data with an apparatus
that is located outside the television receiver 1000 and
20 is connected thereto via a USB cable attached to a USB
terminal 1036. The network I/F 1034 is connected to the
network via the cable attached to the network terminal
1035, and also exchanges data other than audio data with
various kinds of devices connected to the network.
25 [0465]
Using the image decoding device 400 as the MPEG
decoder 1017, the television receiver 1000 can increase
the encoding efficiency of broadcast wave signals
received via an antenna or content data obtained via a
30 network.
[0466]
121
•
SP317970WO00
<6. Sixth Embodiment>
[Portable Telephone Device]
Fig. 41 is a block diagram showing a typical
example structure of a portable telephone device using
5 the image encoding device 300 and the image decoding
device 400.
[0467]
The portable telephone device 1100 shown in Fig. 41
includes a main control unit 1150 designed to
10 collectively control respective components, a power
source circuit unit 1151, an operation input control unit
1152, an image encoder 1153, a camera I/F unit 1154, an
LCD control unit 1155, an image decoder 1156, a
multiplexing/separating unit 1157, a
15 recording/reproducing unit 1162, a
modulation/demodulation circuit unit 1158, and an audio
codec 1159. Those components are connected to one
another via a bus 1160.
[0468]
20 The portable telephone device 1100 also includes
operation keys 1119, a CCD (Charge Coupled Device) camera
1116, a liquid crystal display 1118, a storage unit 1123,
a transmission/reception circuit unit 1163, an antenna
1114, a microphone (mike) 1121, and a speaker 1117.
25 [0469]
When a call is ended or the power key is switched
on by a user's operation, the power source circuit unit
1151 puts the portable telephone device 1100 into an
operable state by supplying power from a battery pack to
30 the respective components.
[0470]
122
^ h SP317970WO00 w
Under the control of the main control unit 1150
formed with a CPU, a ROM, a RAM, and the like, the
portable telephone device 1100 performs various kinds of
operations, such as transmission and reception of audio
5 signals, transmission and reception of electronic mail
and image data, image capturing, and data recording, in
various kinds of modes such as a voice communication mode
and a data communication mode.
[0471]
10 • In the portable telephone device 1100 in the voice
communication mode, for example, an audio signal captured
by the microphone (mike) 1121 is transformed into digital
audio data by the audio codec 1159, and the digital audio
data is subjected to spread spectrum processing at the
15 modulation/demodulation circuit unit 1158. The resultant
data is then subjected to a digital-analog converting
operation and a frequency converting operation at the
transmission/reception circuit unit 1163. The portable
telephone device 1100 transmits the transmission signal
20 obtained through the converting operations to a base
station (not shown) via the antenna 1114. The
transmission signal (audio signal) transmitted to the
base station is further supplied to the portable
telephone device at the other end of the communication
25 via a public telephone line network.
[0472]
Also, in the portable telephone device 1100 in the
voice communication mode, for example, a reception signal
received by the antenna 1114 is amplified at the
30 transmission/reception circuit unit 1163, and is further
subjected to a frequency converting operation and an
123
^ SP317970WO00
analog-digital converting operation. The resultant
signal is subjected to inverse spread spectrum processing
at the modulation/demodulation circuit unit 1158, and is
transformed into an analog audio signal by the audio
5 codec 1159. The portable telephone device 1100 outputs,
from the speaker 1117, the analog audio signal obtained
through the conversions.
[0473]
Further, when electronic mail is transmitted in the
10 data.communication mode, for example, the operation input
control unit 1152 of the portable telephone device 1100
receives text data of the electronic mail that is input
by operating the operation keys 1119. The portable
telephone device 1100 processes the text data at the main
15 control unit 1150, and displays the text data as an image
on the liquid crystal display 1118 via the LCD control
unit 1155.
[0474]
In the portable telephone device 1100, the main
20 control unit 1150 generates electronic mail data based on
text data, a user's instruction, or the like received by
the operation input control unit 1152. The portable
telephone device 1100 subjects the electronic mail data
to spread spectrum processing at the
25 modulation/demodulation circuit unit 1158, and to a
digital-analog converting operation and a frequency
converting operation at the transmission/reception
circuit unit 1163. The portable telephone device 1100
transmits the transmission signal obtained through the
30 converting operations to a base station (not shown) via
the antenna 1114. The transmission signal (electronic
i
124
^ SP317970WO00
mail) transmitted to the base station is supplied to a
predetermined address via a network, a mail server, and j
the like.
[0475]
5 When electronic mail is received in the data
communication mode, for example, the
transmission/reception circuit unit 1163 of the portable
telephone device 1100 receives a signal transmitted from
a base station via the antenna 1114, and the signal is
10 amplified and is further subjected to a frequency
converting operation and an analog-digital converting
operation. The portable telephone device 1100 subjects
the received signal to inverse spread spectrum processing
at the modulation/demodulation circuit unit 1158, to
15 restore the original electronic mail data. The portable
telephone device 1100 displays the restored electronic
mail data on the liquid crystal display 1118 via the LCD
control unit 1155. ;
[0476] j
20 The portable telephone device 1100 can also record
(store) the received electronic mail data into the i
storage unit 1123 via the recording/reproducing unit 1162. l
[0477] j
The storage unit 1123 is a rewritable storage
25 medium. The storage unit 1123 may be a semiconductor
memory such as a RAM or an internal flash memory, a hard
disk, or a removable medium such as a magnetic disk, a
magnetooptical disk, an optical disk, a USB memory, or a
memory card. It is of course possible to use a memory
30 other than the above.
[0478]
:
125
^1, SP317970WO00
Further, when image data is transmitted in the data
communication mode, for example, the portable telephone
device 1100 generates the image data at the CCD camera
1116 capturing an image. The CCD camera 1116 includes
5 optical devices such as a lens and a diaphragm, and a CCD
as a photoelectric conversion device. The CCD camera
1116 captures an image of an object, converts the
intensity of received light into an electrical signal,
and generates image data of the image of the object. The
10 CCD camera 1116 encodes the image data at the image
encoder 1153 via the camera I/F unit 1154, to obtain
encoded image data.
[0479]
The portable telephone device 1100 uses the above
15 described image encoding device 300 as the image encoder
1153 that performs the above operation. As in the case
of the image encoding device 300, the image encoder 1153
appropriately controls bit sequence assignment in
accordance with a predetermined parameter. That is, the
20 image encoder 1153 can assign a bit sequence having a
shorter code length to the selection information about a
motion vector having a higher designation frequency. In
this manner, the image encoder 1153 can increase encoding
efficiency for encoded data.
25 [0480]
At the same time as above, in the portable
telephone device 1100, the sound captured by the
microphone (mike) 1121 during the image capturing by the
CCD camera 1116 is analog-digital converted at the audio
30 codec 1159, and is further encoded.
[0481]
126
© SP317970WO00
The multiplexing/separating unit 1157 of the
portable telephone device 1100 multiplexes the encoded
image data supplied from the image encoder 1153 and the
digital audio data supplied from the audio codec 1159 by
5 a predetermined method. The portable telephone device
1100 subjects the resultant multiplexed data to spread
spectrum processing at the modulation/demodulation
circuit unit 1158, and to a digital-analog converting
operation and a frequency converting operation at the
10 transmission/reception circuit unit 1163. The portable
telephone device 1100 transmits the transmission signal
obtained through the converting operations to a base
station (not shown) via the antenna 1114. The
transmission signal (image data) transmitted to the base j
15 station is supplied to the other end of the communication i
via a network or the like. j
[0482]
When image data is not transmitted, the portable j
telephone device 1100 can also display image data |
I 20 generated at the CCD camera 1116 on the liquid crystal i
display 1118 via the LCD control unit 1155, instead of i
I the image encoder 1153. j
[0483]
When the data of a moving image file linked to a
25 simplified homepage or the like is received in the data
communication mode, for example, the
transmission/reception circuit unit 1163 of the portable j
telephone device 1100 receives a signal transmitted from I
a base station via the antenna 1114. The signal is
30 amplified, and is further subjected to a frequency
converting operation and an analog-digital converting
127
^ SP317970WO00 m
operation. The portable telephone device 1100 subjects
the received signal to inverse spread spectrum processing
at the modulation/demodulation circuit unit 1158, to
restore the original multiplexed data. The portable
5 telephone device 1100 divides the multiplexed data into
encoded image data and audio data at the
multiplexing/separating unit 1157.
[0484]
By decoding the encoded image data at the image I
10 decoder 1156, the port-able telephone device 1100 i
generates reproduced moving image data, and displays the i
reproduced moving image data on the liquid crystal
display 1118 via the LCD control unit 1155. In this
manner, the moving image data contained in a moving image
15 file linked to a simplified homepage, for example, is
displayed on the liquid crystal display 1118.
[0485]
The portable telephone device 1100 uses the above
described image decoding device 400 as the image decoder
20 1156 that performs the above operation. As in the case
of the image decoding device 400, the image decoder 1156
appropriately controls bit sequence assignment in I
accordance with a predetermined parameter. Accordingly, i
the image decoder 1156 can reproduce the same bit
25 sequence assignment as that used in the encoding, and
correctly decode encoded data supplied from the encoding
side. In this manner, the image decoder 1156 can
increase encoding efficiency for encoded data.
[0486]
30 At the same time as above, the portable telephone I
device 1100 transforms the digital audio data into an i
128
•
SP317970WO00
analog audio signal at the audio codec 1159, and outputs
the analog audio signal from the speaker 1117. In this
manner, the audio data contained in a moving image file
linked to a simplified homepage, for example, is
5 reproduced.
[0487]
As in the case of electronic mail, the portable
telephone device 1100 can also record (store) received
data linked to a simplified homepage or the like into the
10 storage unit 1123 via the recording/reproducing unit 1162.
[0488]
The main control unit 1150 of the portable
telephone device 1100 can also analyze a two-dimensional j
code obtained by the CCD camera 1116 performing image I
15 capturing, and obtain the information recorded in the I
two-dimensional code. I
[0489]
Further, an infrared communication unit 1181 of the I
portable telephone device 1100 can communicate with an I
20 external apparatus by using infrared rays. j
[0490]
By using the image encoding device 300 as the image
encoder 1153, the portable telephone device 1100 can
increase encoding efficiency for encoded data when image j
25 data generated by the CCD camera 1116 is encoded and is j
then transmitted, for example. j
[0491]
Also, by using the image decoding device 4 00 as the
image decoder 1156, the portable telephone device 1100
30 can increase the encoding efficiency for the data
(encoded data) of a moving image file linked to a
129
^ . SP317970WO00
#
simplified homepage, for example.
[0492]
In the above description, the portable telephone
device 1100 uses the CCD camera 1116. However, instead
5 of the CCD camera 1116, an image sensor (a CMOS image
sensor) using a CMOS (Complementary Metal Oxide
Semiconductor) may be used. In that case, the portable
telephone device 1100 can also capture an image of an
object, and generate the image data of the image of the
10 object, as in the case where the CCD camera 1116 is used.
[0493]
Although the portable telephone device 1100 has I
been described above, the image encoding device 300 and i
the image decoding device 400 can also be applied to any I
15 device in the same manner as in the case of the portable
telephone device 1100, as long as the device has the same
image capturing function and the same communication j
function as the portable telephone 1100. Such a device
may be a PDA (Personal Digital Assistant), a smartphone,
20 an UMPC (Ultra Mobile Personal Computer), a netbook, or a
notebook personal computer, for example.
[0494]
<7. Seventh Embodiment> !
[Hard Disk Recorder] j
25 Fig. 42 is a block diagram showing a typical j
example structure of a hard disk recorder using the image i
encoding device 300 and the image decoding device 400. i
[0495]
The hard disk recorder (a HDD recorder) 1200 shown
30 in Fig. 42 is a device that stores, into an internal hard
disk, the audio data and the video data of a broadcast
130
^ SP317970WO00 m
show contained in a broadcast wave signal (a television
signal) that is transmitted from a satellite or a
terrestrial antenna or the like and is received by a
tuner, and provides the stored data to a user at a time
5 designated by an instruction from the user.
[0496]
The hard disk recorder 1200 can extract audio data
and video data from a broadcast wave signal, for example, I
decode those data where appropriate, and store the data
10 into an internal hard disk. Also, the hard disk recorder
1200 can obtain audio data and video data from another
I device via a network, for example, decode those data i
where appropriate, and store the data into an internal
hard disk.
15 [0497]
Further, the hard disk recorder 1200 can decode
audio data and video data recorded on an internal hard
disk, for example, supply those data to a monitor 1260,
display the image on the screen of the monitor 12 60, and
20 output the sound from the speaker of the monitor 1260, I
Also, the hard disk recorder 1200 can decode audio data I
and video data extracted from a broadcast wave signal j
obtained via a tuner, or audio data and video data |
obtained from another device via a network, for example,
25 supply those data to the monitor 12 60, display the image
on the screen of the monitor 12 60, and output the sound
from the speaker of the monitor 12 60.
[0498]
The hard disk recorder 700 can of course perform
30 operations other than the above.
[0499]
131
© SP317970WO00
As shown in Fig. 42, the hard disk recorder 1200
includes a reception unit 1221, a demodulation unit 1222,
a demultiplexer 1223, an audio decoder 1224, a video I
decoder 1225, and a recorder control unit 122 6. The hard
5 disk recorder 1200 further includes an EPG data memory
1227, a program memory 1228, a work memory 1229, a
display converter 1230, an OSD (On-Screen Display)
control unit 1231, a display control unit 1232, a j
recording/reproducing unit 1233, a D/A converter 1234, j
10 and a communication unit 1235.
[0500]
The display converter 1230 includes a video encoder |
1241. The recording/reproducing unit 1233 includes an |
encoder 1251 and a decoder 1252. I
15 [0501] I
The reception unit 1221 receives an infrared signal I
from a remote controller (not shown), converts the j
infrared signal into an electrical signal, and outputs j
the electrical signal to the recorder control unit 122 6. j
2 0 The recorder control unit 122 6 is formed with a j
microprocessor, for example, and performs various kinds I
of operations in accordance with a program stored in the
program memory 1228. At this point, the recorder control
unit 1226 uses the work memory 1229 where necessary.
25 [0502]
The communication unit 1235 is connected to a j
network, and performs a communication operation with I
another device via the network. For example, under the I
control of the recorder control unit 1226, the j
30 communication unit 1235 communicates with a tuner (not
shown), and outputs a station select control signal
i
132
© SP317970WO00
mainly to the tuner.
[0503]
I The demodulation unit 1222 demodulates a signal
supplied from the tuner, and outputs the signal to the
j
5 demultiplexer 1223. The demultiplexer 1223 divides the
data supplied from the demodulation unit 1222 into audio
data, video data, and EPG data. The demultiplexer 1223 I
outputs the audio data, the video data, and the EPG data i
to the audio decoder 1224, the video decoder 1225, and
id the recorder control unit 1226, respectively.
[0504]
The audio decoder 1224 decodes the input audio data,
and outputs the decoded audio data to the
recording/reproducing unit 1233. The video decoder 1225
15 decodes the input video data, and outputs the decoded
video data to the display converter 1230. The recorder
control unit 122 6 supplies and stores the input EPG data
into the EPG data memory 1227.
[0505]
20 The display converter 1230 encodes video data
supplied from the video decoder 1225 or the recorder
control unit 122 6 into video data compliant with the NTSC
(National Television Standards Committee) standards, for
example, using the video encoder 1241. The encoded video
25 data is output to the recording/reproducing unit 1233.
Also, the display converter 1230 converts the screen size
of video data supplied from the video decoder 1225 or the
recorder control unit 1226 into a size compatible with
the size of the monitor 1260. The video encoder 1241
30 converts the video data into video data compliant with
the NTSC standards. The NTSC video data is converted
f
133
^j, SP317970WO00
into an analog signal, and is output to the display
control unit 1232.
I [0506]
Under the control of the recorder control unit 1226,
5 the display control unit 1232 superimposes an OSD signal
output from the OSD (On-Screen Display) control unit 1231
on the video signal input from the display converter 1230,
and outputs the resultant signal to the display of the
monitor 1260 to display the image.
10 [0507]
Audio data that is output from the audio decoder
1224 and is converted into an analog signal by the D/A
converter 1234 is also supplied to the monitor 1260. The
monitor 12 60 outputs the audio signal from an internal
15 speaker.
[0508]
The recording/reproducing unit 1233 includes a hard
disk as a storage medium for recording video data, audio
data, and the like.
20 [0509]
The recording/reproducing unit 1233 causes the
encoder 1251 to encode audio data supplied from the audio
decoder 1224, for example. The recording/reproducing
unit 1233 also causes the encoder 1251 to encode video
25 data supplied from the video encoder 1241 of the display
converter 1230. The recording/reproducing unit 1233
combines the encoded data of the audio data with the
encoded data of the video data, using a multiplexer. The
recording/reproducing unit 1233 amplifies the combined
30 data through channel coding, and writes the resultant
data on the hard disk via a recording head.
134
^ ^ SP317970WO00 w
I
[0510] I
The recording/reproducing unit 1233 reproduces data j
recorded on the hard disk via a reproduction head, j
i
amplifies the data, and divides the data into audio data j
5 and video data by using a demultiplexer. The |
recording/reproducing unit 1233 decodes the audio data i
I
and the video data by using the decoder 1252. The !
recording/reproducing unit 1233 performs a D/A conversion I
on the decoded audio data, and outputs the resultant data |
10 to the speaker, of the monitor 12 60. The j
recording/reproducing unit 1233 also performs a D/A j
conversion on the decoded video data, and outputs the i
resultant data to the display of the monitor 1260. I
[0511] I
15 Based on a user's instruction indicated by an |
infrared signal that is transmitted from a remote I
i
controller and is received via the reception unit 1221, i
the recorder control unit 122 6 reads the latest EPG data |
from the EPG data memory 1227, and supplies the EPG data I
I
20 to the OSD control unit 1231. The OSD control unit 1231 I
generates image data corresponding to the input EPG data, I
and outputs the image data to the display control unit I
1232. The display control unit 1232 outputs the video
data input from the OSD control unit 1231 to the display
25 of the monitor 1260, to display the image. In this
manner, an EPG (Electronic Program Guide) is displayed on
the display of the monitor 12 60.
[0512] i The hard disk recorder 1200 can also obtain various I
30 kinds of data, such as video data, audio data, and EPG
data, which are supplied from another device via a I
I
I
135
—^ SP317970WO00
network such as the Internet. i
[0513] I
I
I Under the control of the recorder control unit 122 6, 1
the communication unit 1235 obtains encoded data of video i
5 data, audio data, EPG data, and the like from another I
device via a network, and supplies those data to the j
recorder control unit 122 6. For example, the recorder j
I
control unit 122 6 supplies encoded data of obtained video i
data and audio data to the recording/reproducing unit i
i
10 1233, and stores those data into the hard disk. At this j
point, the recorder control unit 122 6 and the I
recording/reproducing unit 1233 may perform an operation I
such as a re-encoding where necessary. I
[0514] I
15 The recorder control unit 122 6 also decodes encoded i
data of obtained video data and audio data, and supplies i
the resultant video data to the display converter 1230, 1
The display converter 1230 processes the video data |
supplied from the recorder control unit 122 6 in the same j
20 manner as processing video data supplied from the video i
decoder 1225, and supplies the resultant data to the I
monitor 1260 via the display control unit 1232, to «
j
display the image.
[0515] I
25 In synchronization with the image display, the j
recorder control unit 122 6 may supply the decoded audio i
data to the monitor 12 60 via the D/A converter 1234, and I
output the sound from the speaker. |
[0516] j
30 Further, the recorder control unit 1226 decodes I
encoded data of obtained EPG data, and supplies the i
I I
136
^^. SP317970WO00
decoded EPG data to the EPG data memory 1227. I
[0517]
I The above described hard disk recorder 1200 uses I
the image decoding device 400 as the video decoder 1225, !
5 the decoder 1252, and the decoder installed in the |
recorder control unit 1226. As in the case of the image |
decoding device 400, the video decoder 1225, the decoder I
1252, and the decoder installed in the recorder control i
unit 122 6 appropriately control bit sequence assignment I
10 in accordance with a predetermined parameter. I
Accordingly, the video decoder 1225, the decoder 1252, I
and the decoder installed in the recorder control unit j
122 6 can reproduce the same bit sequence assignment as |
that used in the encoding. In this manner, the video I
15 decoder 1225, the decoder 1252, and the decoder installed i
in the recorder control unit 122 6 can increase encoding j
efficiency for encoded data. I
[0518] I
Thus, the hard disk recorder 1200 can increase |
20 encoding efficiency for video data (encoded data) |
received by a tuner or the communication unit 1235 and I
fi
video data (encoded data) to be reproduced by the I
recording/reproducing unit 1233. j
[0519] I
25 The hard disk recorder 1200 also uses the image i
encoding device 300 as the encoder 1251. As in the case I
of the image encoding device 300, the encoder 1251 |
appropriately controls bit sequence assignment in |
accordance with a predetermined parameter. That is, the j
30 encoder 1251 can assign a bit sequence having a shorter I
code length to the selection information about a motion I
137
I ^^. SP317970WO00
® I
vector having a higher designation frequency. In this i
manner, the encoder 1251 can increase encoding efficiency I
I for encoded data.
[0520]
5 Thus, the hard disk recorder 1200 can increase
encoding efficiency for data to be recorded on the hard
disk, for example. i
[0521] I
In the above description, the hard disk recorder I
10 1200 that records video data and audio data on a hard
disk has been described. However, any other recording
medium may be used. For example, as in the case of the i
above described hard disk recorder 12 00, the image
encoding device 300 and the image decoding device 400 can i
15 be applied to a recorder that uses a recording medium j
other than a hard disk, such as a flash memory, an j
optical disk, or a videotape. |
[0522] I
<8. Eighth Embodiment> |
i
20 [Camera] |
I
Fig. 43 is a block diagram showing a typical |
I
example structure of a camera using the image encoding i
j
device 300 and the image decoding device 400. |
[0523] I
i
25 The camera 1300 shown in Fig. 43 captures an image j
I
of an object, and displays the image of the object on an j
LCD 1316 or records the image of the object as image data j
on a recording medium 1333. j
[0524] I
30 A lens block 1311 has light (or a video image of an | I object) incident on a CCD/CMOS 1312. The CCD/CMOS 1312 I
!
I
138
^t. SP317970WO00 w
is an image sensor using a CCD or a CMOS. The CCD/CMOS
1312 converts the intensity of the received light into an ;
I electrical signal, and supplies the electrical signal to j
a camera signal processing unit 1313. |
5 [0525] I
The camera signal processing unit 1313 transforms j
the electrical signal supplied from the CCD/CMOS 1312
into a Y, Cr, Cb chrominance signal, and supplies the
!
signal to an image signal processing unit 1314. Under j
10 the control of a controller 1321, the image signal j
processing unit 1314 performs predetermined image |
processing on the image signal supplied from the camera |
k
signal processing unit 1313, and encodes the image signal | 1
by using an encoder 1341. The image signal processing |
I
15 unit 1314 supplies the encoded data generated by encoding
!
the image signal to a decoder 1315. The image signal j
j
processing unit 1314 further obtains display data I
I generated at an on-screen display (OSD) 1320, and i
supplies the display data to the decoder 1315. I
20 [0526] j
In the above operation, the camera signal i
processing unit 1313 uses a DRAM (Dynamic Random Access j
Memory) 1318 connected thereto via a bus 1317, to store the image data and the encoded data or the like generated j
25 by encoding the image data into the DRAM 1318 where i
I
necessary. j
[0527] !
The decoder 1315 decodes the encoded data supplied j
from the image signal processing unit 1314, and supplies j
30 the resultant image data (decoded image data) to the LCD 1316. The decoder 1315 also supplies the display data I I
139
^^ SP317970WO00
supplied from the image signal processing unit 1314 to
the LCD 1316. The LCD 1316 combines the image
corresponding to the decoded image data supplied from the
I
decoder 1315 with the image corresponding to the display |
5 data, and displays the combined image. j
[0528] j
r
Under the control of the controller 1321, the on- I
screen display 1320 outputs the display data of a menu screen or icons formed with symbols, characters, and !
10 figures, to the image signal processing unit 1314 via the bus 1317. 1
[0529] j
Based on a signal indicating contents designated by |
a user using an operation unit 1322, the controller 1321
15 performs various kinds of operations, and controls, via
the bus 1317, the image signal processing unit 1314, the I
DRAM 1318, an external interface 1319, the on-screen
J
display 1320, a media drive 1323, and the like. A flash
•s
ROM 1324 stores programs, data, and the like necessary I
I
20 for the controller 1321 to perform various kinds of |
operations. I
[0530] I
•I I
For example, in place of the image signal J
processing unit 1314 and the decoder 1315, the controller J
I
25 1321 can encode the image data stored in the DRAM 1318, |
and decode the encoded data stored in the DRAM 1318. In 1
j
doing so, the controller 1321 may perform encoding and j
I
decoding operations by using the same methods as the I
encoding and decoding methods used by the image signal .j
30 processing unit 1314 and the decoder 1315, or may perform j
'I
encoding and decoding operations by using methods that |
I
i
;
:
140
^^ SP317970WO00
I
are not compatible with the image signal processing unit j I \ 1314 and the decoder 1315. j
I
[0531] i I When a start of image printing is requested through |
5 the operation unit 1322, for example, the controller 1321 |
reads image data from the DRAM 1318, and supplies the j
ii image data to a printer 1334 connected to the external jl
i
it interface 1319 via the bus 1317, so that the printing is
performed. i
10 [0532] I Further, when image recording is requested through j
i i
I the operation unit 1322, for example, the controller 1321 I
reads encoded data from the DRAM 1318, and supplies and !j
stores the encoded data into the recording medium 1333 |
ti
Ii
15 mounted on the media drive 1323 via the bus 1317. I
i
[0533] I
The recording medium 1333 is a readable and 1I
writable removable medium, such as a magnetic disk, a j
i
magnetooptical disk, an optical disk, or a semiconductor "j
I
2 0 memory. The recording medium 1333 may be any kind of |
removable medium, and may be a tape device, a disk, or a |
it
memory card. It is of course possible to use a non- |
I
contact IC card or the like. |
[0534] I
25 Alternatively, the media drive 1323 and the j
recording medium 1333 may be integrated, and may be j
formed with an immobile storage medium such as an I
i
internal hard disk drive or a SSD (Solid State Drive). I
[0535] I I 30 The external interface 1319 is formed with a USB i
input/output terminal and the like, for example, and is |
i
P •I
i
I
p.
141 I
—. SP317970WO00
tP
i
I
connected to the printer 1334 when image printing is j
performed. Also, a drive 1331 is connected to the
I external interface 1319 where necessary, and a removable
medium 1332 such as a magnetic disk, an optical disk, or j
5 a magnetooptical disk is mounted on the drive 1331 where i
1
appropriate. A computer program that is read from such a j
disk is installed in the flash ROM 1324 where necessary. |
"I
[0536] I
j
Further, the external interface 1319 includes a J
10 network interface connected to a predetermined network j
I
such as a LAN or the Internet. In accordance with an I
instruction from the operation unit 1322, for example, I
the controller 1321 can read encoded data from the DRAM j
1318, and supply the encoded data from the external j
15 interface 1319 to another device connected thereto via a 1
i I
network. Also, the controller 1321 can obtain encoded j
j
data and image data supplied from another device via a j
i
network, and store the data into the DRAM 1318 or supply I
the data to the image signal processing unit 1314 via the j
20 external interface 1319. j
[0537] j
The above camera 1300 uses the image decoding device 400 as the decoder 1315. As in the case of the j
image decoding device 400, the decoder 1315 appropriately j
25 controls bit sequence assignment in accordance with a i
I
I
predetermined parameter. Accordingly, the decoder 1315 |
can reproduce the same bit sequence assignment as that I
i
used in the encoding, and correctly decode encoded data I
supplied from the encoding side. In this manner, the {
30 decoder 1315 can increase encoding efficiency for encoded ij
data. 1
i
i
I
I
1
I
142 I
SP317970WO00 I
\ C i
[0538] j
Thus, the camera 1300 can increase encoding | I
I efficiency for image data generated at the CCD/CMOS 1312, j
I i
encoded data of video data read from the DRAM 1318 or the j
5 recording medium 1333, or encoded data of video data j
1
obtained via a network, for example, I
[0539] i
i
Also, the camera 1300 uses the image encoding [I
•I
device 300 as the encoder 1341. As in the case of the I
10 image encoding device 300, the encoder 1341 appropriately J I controls bit sequence assignment in accordance with a j
predetermined parameter. That is, the encoder 1341 can j
!
assign a bit sequence having a shorter code length to the j
I
selection information about a motion vector having a i
15 higher designation frequency. In this manner, the •
encoder 1341 can increase encoding efficiency for encoded 1
i
data. I
i
[0540] J 1
Thus, the camera 1300 can increase the encoding j
20 efficiency for encoded data to be recorded in the DRAM j
1318 or the recording medium 1333, and for encoded data j
i
to be provided to other devices, for example. 1
[0541] 5
I
I
The decoding method used by the image decoding |
25 device 400 may be applied to decoding operations to be j
performed by the controller 1321. Likewise, the encoding !
method used by the image encoding device 300 may be t
i
applied to encoding operations to be performed by the j
controller 1321. i
30 [0542] I
I
i
Image data to be captured by the camera 1300 may be j
i I
I
I
I k
f i
I i
i 1
143 I
^ ^ SP317970WO00 |
i %
of a moving image, or may be of a still image. I
[0543] j
j It is of course possible to use an image encoding |
i
device and an image decoding device having the present | I 5 technique applied thereto in any devices and systems |
other than the above described.devices. I
[0544]
The present technique can be applied to image i!
I
encoding devices and image decoding devices that are used ij
i
10 when image information (bit streams) compressed through |
orthogonal transforms such as discrete cosine transforms i I
and motion compensation is received via a network medium ij
such as satellite broadcasting, cable television, the ii
i
Internet, or a portable telephone, or when such image I
s
II
15 information is processed in a storage medium such as an |
ii
optical or magnetic disk or a flash memory, as in MPEG, !j
(j
H.2 6x, for example. |
[0545] i
ij
ij
The present technique can also be in the following jl
I
2 0 forms. |
i|
(1) An image processing device including: |
an assignment control unit that controls assignment jl
of a binary bit sequence to predicted motion vector |
j
select information indicating a motion vector selected as i
I
J
25 a predicted motion vector, to assign a bit sequence j| I having a shorter code length to the select information |
about a motion vector having a higher designation |
is
frequency; and |
a binarization unit that binarizes the predicted I
I!
30 motion vector select information with the bit sequence ||
i
assigned by the assignment control unit. |
I
I
144
I SP317970WO00
•I I
(2) The image processing device of (1), wherein 1 J
the assignment control unit includes a table I
I
I selection unit that selects a table that designates the •!
bit sequence to be assigned to the predicted motion il
•! 5 vector select information in accordance with the type of |
the motion vector selected as the predicted motion vector, I
I and I
the binarization unit binarizes the predicted I
motion vector select information by using the table |
10 selected by the table selection unit. ij
(3) The image processing device of (2), wherein |
f!
the assignment control unit further includes a ;i
distance calculation unit that calculates the distance S
between the current picture and an anchor picture, and |
15 the table selection unit selects the table based on li
the distance calculated by the distance calculation unit. I
(4) The image processing device of (3), wherein |
il
the assignment control unit further includes a I
distance threshold acquirement unit that acquires a
2 0 distance threshold indicating the threshold of the i
distance, and j
the table selection unit selects the table in j
"i
accordance with the magnitude relationship between the j
distance calculated by the distance calculation unit and j
25 the distance threshold acquired by the distance threshold acquirement unit. i
1!
(5) The image processing device of (4), wherein the j)
distance threshold acquired by the distance threshold i
acquirement unit is supplied to another device that Ij
i
30 decodes encoded data of the predicted motion vector ij
select information. {j
Ij
1
4
I
I !'
' 145 I
SP317970WO00 ij
I ^ 11
I (6) The image processing device of any of (2) to Ij
i
(5), wherein j
I
1 the assignment control unit further includes a i
similarity calculation unit that calculates the j
5 similarity between peripheral predicted motion vectors, {
and I
the table selection unit selects the table based on |{
the similarity calculated by the similarity calculation j
unit, I
I
10 (7) The image processing device of (6), wherein !
the assignment control unit further includes a i
similarity threshold acquirement unit that acquires a j
similarity threshold indicating the threshold of the j
similarity, and ]
15 the table selection unit selects the table in j
accordance with the magnitude relationship between the j
s
I
similarity calculated by the similarity calculation unit i
and the similarity threshold acquired by the similarity I
threshold acquirement unit. j
2 0 (8) The image processing device of (7), wherein the j
similarity threshold acquired by the similarity threshold acquirement unit is supplied to another device that I
decodes encoded data of the predicted motion vector i
select information. J
25 (9) The image processing device of any of (2) to {
(8), wherein i
I
the assignment control unit further includes a cost j
function value calculation unit that calculates a cost {
i
function value of the predicted motion vector select j
30 information, and j
the table selection unit selects the table based on i
I
•J
146 I
_ SP317970WO00
^ i
the cost function value calculated by the cost function |
value calculation unit. i
ii
\ (10) The image processing device of any of (2) to ij
(9), wherein information indicating the result of the |
fi
5 table selection performed by the table selection unit is I
I
supplied to another device that decodes encoded data of ij
I
the predicted motion vector select information. I
•1
(11) The image processing device of any of (1) to
(10), further including
10 an encoding unit that encodes the binarized data I
obtained through the binarization performed by the
binarization unit. j
i
(12) An image processing method for an image i
processing device, including: j
15 controlling assignment of a binary bit sequence to
predicted motion vector select information indicating a j
I
motion vector selected as a predicted motion vector, to I
!
assign a bit sequence having a shorter code length to the I
j
select information about a motion vector having a higher i
i
2 0 designation frequency, the controlling being performed by j
an assignment control unit; and j
I
binarizing the predicted motion vector select j
I
information with the assigned bit sequence, the i
binarizing being performed by a binarization unit. j
j
25 (13) An image processing device including: j
a decoding unit that decodes encoded data of i
predicted motion vector select information indicating a j
I
I
motion vector selected as a predicted motion vector; and j
I
a debinarization unit that debinarizes the ]
!
30 binarized data of the predicted motion vector select I
information obtained through the decoding performed by j
i I
147 {
SP317970WO00 I
the decoding unit, with a bit sequence that is assigned
in an assigning operation controlled to assign a bit j
I sequence having a shorter code length to the select j
information about a motion vector having a higher 5 designation frequency. j
(14) The image processing device of (13), further j
i
including j
an assignment control unit that controls the bit *
I
sequence to be assigned to the predicted motion vector ij
10 select information based on a parameter supplied from j
another device that has encoded the predicted motion )
I
vector select information, wherein the debinarization unit debinarizes the
predicted motion vector select information under the j
i
15 control of the assignment control unit. |
(15) The image processing device of (14), wherein j
f
the assignment control unit includes a table selection j
unit that selects a table that designates the bit j
sequence to be assigned to the predicted motion vector j
j
20 select information in accordance with the type of the j
i
motion vector selected as the predicted motion vector. j
(16) The image processing device of (15), wherein i
the assignment control unit further includes a j
distance calculation unit that calculates the distance j
25 between the current picture and an anchor picture, and j
the table selection unit selects the table in )
I
accordance with the magnitude relationship between the distance calculated by the distance calculation unit and I
a distance threshold supplied from the other device that j
30 has encoded the predicted motion vector select I
information. j
i
•
148
,^ SP317970WO00
(17) The image processing device of (15) or (16),
wherein
!
the assignment control unit further includes a I i
similarity calculation unit that calculates the j
I
5 similarity between peripheral predicted motion vectors, 1
and I
] the table selection unit selects the table in
I
accordance with the magnitude relationship between the
similarity calculated by the similarity calculation unit |
10 and a similarity threshold supplied from the other device i
that has encoded the predicted motion vector select I
J
information. j
(18) The image processing device of any of (13) to I
I
(17), wherein the debinarization unit debinarizes the | I 15 binarized data of the predicted motion vector select I 'i
'i
information based on information indicating a table ^i
i|
fi selection result supplied from the other device that has I
'4
encoded the predicted motion vector select information. l!
(19) An image processing method for an image Ij
20 processing device, including: I
i
decoding encoded data of predicted motion vector i
;l
select information indicating a motion vector selected as i
ij
si
a predicted motion vector, the decoding being performed ij
by a decoding unit; and il
25 debinarizing the binarized data of the predicted |
motion vector select information obtained through the |j
j
decoding, with a bit sequence that is assigned in an !
assigning operation controlled to assign a bit sequence j
having a shorter code length to the select information ij
ji
30 about a motion vector having a higher designation Ij
frequency, the debinarizing being performed by a jj
I
j
i
149 j
SP317970WO00 t
I debinarization unit. j
REFERENCE SIGNS LIST j
[0546] I
ji
5 300 Image encoding device, 306 Lossless encoding il
unit, 321 Assignment control unit, 331 Motion vector j
I
storage unit, 332 Predicted motion vector selection unit, j
333 Difference motion vector calculation unit, 334 Table i
storage unit, 335 Binarization unit, 336 Binarization i
10 unit, 337 Entropy encoding unit, 341 Distance threshold j
acquirement unit, 342 Distance calculation unit, 343 j
Table selection unit, 400 Image decoding device, 402 I
Lossless decoding unit, 421 Assignment control unit, 431 i
Entropy decoding unit, 432 Debinarization unit, 433 Table
15 storage unit, 434 Debinarization unit, 435 Predicted j
motion vector selection unit, 436 Motion vector j
calculation unit, 437 Motion vector storage unit, 441 i
Distance calculation unit, 442 Table selection unit, 541 j
Similarity threshold acquirement unit, 542 Similarity i
20 calculation unit, 543 Table selection unit, 641
Similarity calculation unit, 642 Table storage unit, 741 j
Cost function value calculation unit, 742 Cost function ;
value storage unit, 743 Table selection unit, 832 i
Debinarization unit, 834 Debinarization unit J
25 j

I 150
SP317970WO00
CLAIMS
I 1. An image processing device comprising:
an assignment control unit configured to control
5 assignment of a binary bit sequence to predicted motion
vector select information indicating a motion vector
selected as a predicted motion vector, to assign a bit
sequence having a shorter code length to select
information about a motion vector having a higher
10 designation frequency; and
a binarization unit configured to binarize the
predicted motion vector select information with the bit
sequence assigned by the assignment control unit.
15 2. The image processing device according to claim 1,
wherein
the assignment control unit comprises a table
selection unit configured to select a table, the table
designating the bit sequence to be assigned to the
20 predicted motion vector select information in accordance
with a type of the motion vector selected as the
predicted motion vector, and
the binarization unit binarizes the predicted
motion vector select information by using the table
25 selected by the table selection unit.
3. The image processing device according to claim 2,
wherein
the assignment control unit further comprises a
30 distance calculation unit configured to calculate a
distance between a current picture and an anchor picture,
;
i.
151
SP3179'70WO00 c
and
the table selection unit selects the table based on
I the distance calculated by the distance calculation unit.
5 4. The image processing device according to claim 3,
wherein
the assignment control unit further comprises a
distance threshold acquirement unit configured to acquire
a distance threshold indicating a threshold of the
10 distance, and
the table selection unit selects the table in
accordance with a magnitude relationship between the
distance calculated by the distance calculation unit and
the distance threshold acquired by the distance threshold
15 acquirement unit.
5. The image processing device according to claim 4,
wherein the distance threshold acquired by the distance
threshold acquirement unit is supplied to another device
2 0 that decodes encoded data of the predicted motion vector
select information.
6. The image processing device according to claim 2,
wherein
25 the assignment control unit further comprises a
similarity calculation unit configured to calculate
similarity between peripheral predicted motion vectors,
and
the table selection unit selects the table based on
30 the similarity calculated by the similarity calculation
unit.
152
SP317970WO00
7. The image processing device according to claim 6,
wherein
the assignment control unit further comprises a
5 similarity threshold acquirement unit configured to
acquire a similarity threshold indicating a threshold of
the similarity, and
the table selection unit selects the table in
accordance with a magnitude relationship between the
10 similarity calculated by the similarity calculation unit
and the similarity threshold acquired by the similarity
threshold acquirement unit.
8. The image processing device according to claim 7,
15 wherein the similarity threshold acquired by the
similarity threshold acquirement unit is supplied to
another device that decodes encoded data of the predicted
motion vector select information.
20 9. The image processing device according to claim 2,
wherein
the assignment control unit further comprises a
cost function value calculation unit configured to
calculate a cost function value of the predicted motion
25 vector select information, and
the table selection unit selects the table based on
the cost function value calculated by the cost function
value calculation unit.
30 10. The image processing device according to claim 2,
wherein information indicating the result of the table
153
-^ SP317970WO00
#
selection performed by the table selection unit is
supplied to another device that decodes encoded data of
the predicted motion vector select information.
5 11. The image processing device according to clam 1,
further comprising
an encoding unit configured to encode the binarized
data obtained through the binarization performed by the
binarization unit.
10 I
12. An image processing method for an image processing I
device, comprising: I
controlling assignment of a binary bit sequence to i
predicted motion vector select information indicating a i
15 motion vector selected as a predicted motion vector, to
assign a bit sequence having a shorter code length to
select information about a motion vector having a higher
designation frequency, the controlling being performed by
an assignment control unit; and
20 binarizing the predicted motion vector select
information with the assigned bit sequence, the
binarizing being performed by a binarization unit.
13. An image processing device comprising:
25 a decoding unit configured to decode encoded data
of predicted motion vector select information indicating
a motion vector selected as a predicted motion vector;
and
a debinarization unit configured to debinarize
30 binarized data of the predicted motion vector select
information obtained through the decoding performed by
154
SP317970WO00
the decoding unit, with a bit sequence that is assigned
in an assigning operation controlled to assign a bit
sequence having a shorter code length to select
information about a motion vector having a higher
5 designation frequency.
14. The image processing device according to claim 13,
further comprising
an assignment control unit configured to control
10 the bit sequence to be assigned to the predicted motion
vector select information based on a parameter supplied
from another device that has encoded the predicted motion
vector select information,
wherein the debinarization unit debinarizes the
15 predicted motion vector select information under the
control of the assignment control unit.
15. The image processing device according to claim 14,
wherein the assignment control unit comprises a table
20 selection unit configured to select a table, the table
designating the bit sequence to be assigned to the
predicted motion vector select information in accordance
with a type of the motion vector selected as the
predicted motion vector.
25
16. The image processing device according to claim 15,
wherein
the assignment control unit further comprises a
distance calculation unit configured to calculate a
i
30 distance between a current picture and an anchor picture, 5
and
i
l
I
155
SP317970WO00
the table selection unit selects the table in
accordance with a magnitude relationship between the
distance calculated by the distance calculation unit and
a distance threshold supplied from another device that
5 has encoded the predicted motion vector select
information.
17. The image processing device according to claim 15,
wherein
10 the assignment control unit further comprises a
similarity calculation unit configured to calculate
similarity between peripheral predicted motion vectors,
and
the table selection unit selects the table in
15 accordance with a magnitude relationship between the
similarity calculated by the similarity calculation unit
and a similarity threshold supplied from another device
that has encoded the predicted motion vector select
information.
20
18. The image processing device according to claim 13,
wherein the debinarization unit debinarizes the binarized
data of the predicted motion vector select information
based on information indicating a table selection result
25 supplied from another device that has encoded the
predicted motion vector select information.
19. An image processing method for an image processing
device, comprising:
30 decoding encoded data of predicted motion vector I
select information indicating a motion vector selected as
156 .
\^ SP317970WO00. w
a predicted motion vector, the decoding being performed
by a decoding unit; and
debinarizing binarized data of the predicted motion
vector select information obtained, through the decoding,
5 with a bit sequence that is assigned in an assigning
operation controlled to assign a bit sequence having a
shorter code length to the select information about a
motion vector having a higher designation frequency, the
debinarizing being performed by a debinarization unit.