1/68
Description
Title of Invention
IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD
5
Technical Field
[OOO 11
The present disclosure relates to an image processing device, and an image
processing method.
10
Background Art
[0002]
Conventionally, a compression technology is widespread that has its object
to effectively transmit or accumulate digital images, and that compresses the amount
15 of information of an image by motion compensation and orthogonal transform such
as discrete cosine transform, for example, by using redundancy unique to the image.
For example, an image encoding device and an image decoding device conforming to
a standard technology such as H.26~ standards developed by ITU-T or MPEG-y
standards developed by MPEG (Moving Picture Experts Group) are widely used in
20 various scenes, such as accumulation and distribution of images by a broadcaster and
reception and accumulation of images by a general user.
[0003]
MPEG2 (ISOIIEC 13818-2) is one of MPEG-y standards defined as a
general-purpose image encoding method. MPEG2 is capable of handling both
25 interlaced scanning images and non-interlaced images, and targets high-definition
images, in addition to digital images in standard resolution. MPEG2 is currently
widely used in a wide range of applications including professional uses and
consumer uses. According to MPEG2, for example, by allocating a bit rate of 4 to 8
Mbps to an interlaced scanning image in standard resolution of 720x480 pixels and a
30 bit rate of 18 to 22 Mbps to an interlaced scanning image in high resolution of
1920x 1088 pixels, both a high compression ratio and a desirable image quality can
be realized.
[0004]
MPEG2 was primarily for high-quality encoding suitable for broadcasting
use, and did not handle a bit rate lower than MPEG1, that is, a high compression
5 ratio. However, with the spread of mobile terminals of recent years, the demand for
an encoding method enabling a high compression ratio is increasing. Accordingly,
standardization of an MPEG4 encoding method was newly promoted. With regard
to an image encoding method which is a part of the MPEG4 encoding method, its
standards were accepted as an international standard (ISOIIEC 14496-2) in
10 December 1998.
[OOOS]
The H.26~ standards (ITU-T Q6/16 VCEG) are standards developed
initially with the aim of performing encoding that is suitable for communications
such as video telephones and video conferences. The H.26~st andards are known to
15 require a large computation amount for encoding and decoding, but to be capable of
realizing a higher compression ratio, compared with the MPEG-y standards.
Furthermore, with Joint Model of Enhanced-Compression Video Coding, which is a
part of the activities of MPEG4, a standard allowing realization of a higher
compression ratio by adopting a new hction while being based on the H.26~
20 standards is developed. This standard was made an international standard under the
names of H.264 and MPEG-4 Part1 0 (Advanced Video Coding; AVC) in March 2003.
[0006]
Furthermore, in recent years, a need is also recognized for an encoding
method with a further increased compression ratio whose aim is to compress a large
25 image of 4000x2000 pixels or to deliver a high-vision image in an environment
where there is a restriction on a transmission capacity, such as the Internet. Thus,
~ Non-Patent Literature 1 proposes to set the size of a macro block to a size larger than
that of MPEG2 or H.264lAVC (for example, to 32x32 pixels) and to perform intra
prediction. Also, Non-Patent Literature 2 proposes to set the size of a macro block
30 to a size larger than that of MPEG2 or H.264JAVC (for example, to 32x32 pixels)
and to perform inter prediction. By performing motion prediction using a macro
block of a greater size, the effect of compression of the data amount by the motion
prediction can be enhanced, and the compression ratio of an image may be fwther
increased.
5 Citation List
Non-Patent Literature
[0007]
Non-Patent Literature 1: Sung-Chang Lim, Hahyun Lee, Jinho Lee,
Jongho Kim, Haechul Choi, Seyoon Jeong, Jin Soo Choi, "Intra coding using
10 extended block size", (ITU-Telecommunications Standardization Sector, Study
Group 16 Question 6, Video Coding Experts Group, 38th Meeting: London,
UWGeneva, CH, 1-8 July, 2009, D)
Non-Patent Literature 2: Qualcornrn Inc, "Video Coding Using Extended
Block1 Sizes" (STUDY GROUP1 6-CONTRIBUTION 123, ITU-
15 Telecommunications Standardization Sector, January 2009)
Summary of Invention
Technical Problem
[OOOS]
20 However, that the largest size of a macro block is extended means that the
size of a line memory required for encoding or decoding of an image also becomes
larger. For example, with H.264/AVCY since a largest macro block (LMB) of 16x 16
pixels may be used, the size of a line memory required for encoding or decoding is
16 pixels. In contrast, with the method proposed in Non-Patent Literature 1 or 2
25 mentioned above, since a largest macro block of 32x32 pixels may be used, a line
memory with a size of 32 pixels is necessary. However, unnecessarily increasing
the size of a line memory is not advantageous from the standpoint of the cost of the
hardware. Also, for hardware with restricted memory resources, it is desirable that
an image encoded at a high compression ratio can be reproduced without extending
30 the line memory as much as possible.
[0009]
Thus, the technology according to the present disclosure intends to provide
an image processing device and an image processing method which enable handling
of a larger block while suppressing the increase in a line memory.
[OO 1 01
5 According to an embodiment of the present disclosure, there is an image
processing device including a shape determination section for determining a shape of
a block set in an image using shape determination information for determining a
shape of a block that is in accordance with a size, a motion vector setting section for
arranging a prediction unit in each block in the image according to the shape
10 determined by the shape determination section, and setting, for each arranged
prediction unit, a motion vector to be used for prediction of an image in the
prediction unit, and an encoding section for encoding the image using the motion
vector set by the motion vector setting section.
[OOll]
15 The image processing device may be typically realized as an image
encoding device that encodes images.
[OO 1 21
Further, the shape determination information may include information about
a boundary value for a block size that separates a square block and a non-square
20 block.
[00 1 31
Further, the shape determination section may determine that a shape of a
block of a larger size than the boundary value is a non-square, and that a shape of a
block of a smaller size than the boundary value is a square.
25 [0014]
Further, a shape of each block in the image may be a square or a rectangle
with a fixed ratio of a long side to a short side.
[00 1 51
Further, a shape of each block in the image is square or a rectangle, and the
30 shape determination information may include ratio information defining a ratio of a
long side to a short side of a block.
[00 161
Further, the shape determination information may include number-of-layers
information defining the number of layers whose ratios of a long side to a short side
of a block are different from one another, and the ratio information may be
5 information defining the ratio for each of the layers.
[00 171
Further, the motion vector setting section may output, for each block,
information indicating a size of each block set in the image and prediction mode
information specifying an arrangement pattern of a prediction unit in the block.
10 [0018]
Further, a shape of each block in the image is a square or a rectangle, and
candidates for arrangement of a prediction unit for a square block and candidates for
arrangement of a prediction unit for a rectangular block may be different.
[00 191
15 Further, the shape determination information may be allowed to be input by
a user.
[0020]
Further, the image processing device may further include transmission
means for transmitting an encoded stream generated by the encoding section and the
20 shape determination information.
[002 11
Further, the encoding section may multiplex the shape determination
information to a sequence parameter set, a picture parameter set or a slice header of
the encoded stream.
25 [0022]
Further, according to an embodiment of the present disclosure, there is
provided an image processing method for processing an image, including
determining a shape of a block set in an image using shape determination
information for determining a shape of a block that is in accordance with a size,
30 arranging a prediction unit in each block in the image according to the determined
shape, and setting, for each arranged prediction unit, a motion vector to be used for
prediction of an image in the prediction unit, and encoding the image using the set
motion vector.
[0023]
Further, according to an embodiment of the present disclosure, there is
5 provided an image processing device including a shape determination section for
determining a shape of each block in an image using shape determination
information for determining a shape of a block that is in accordance with a size and a
size of a block set in the image at a time of encoding of the image, a prediction unit
determination section for determining an arrangement of a prediction unit in each
10 block according to the shape of each block determined by the shape determination
section, a prediction section for generating a predicted image for each block using
the arrangement of a prediction unit determined by the prediction unit determination
section and a motion vector acquired for each prediction unit, and a decoding section
for decoding an encoded stream in which the image is encoded, using the predicted
15 image generated by the prediction section.
[0024]
The image processing device may be typically realized as an image
decoding device that decodes images.
[0025]
20 Further, the shape determination information may include information about
a boundary value for a block size that separates a square block and a non-square
block.
[0026]
Further, a shape of each block in the image may be a square or a rectangle
25 with a fixed ratio of a long side to a short side.
[0027]
Further, a shape of each block in the image is a square or a rectangle, and
the shape determination information may include ratio information defining a ratio of
a long side to a short side of a block.
30 [0028]
Further, the shape determination information may include number-of-layers
information defining the number of layers whose ratios of a long side to a short side
of a block are different from one another, and the ratio information may be
information defining the ratio for each of the layers.
[0029]
5 Further, the shape determination section may acquire the shape
determination information from a sequence parameter set, a picture parameter set or
a slice header multiplexed to the encoded stream.
[0030]
Further, the shape determination section may acquire information for
10 determining a size of each block from a block header included in the encoded stream.
[003 11
Further, a shape of each block in the image is a square or a rectangle, and
candidates for arrangement of a prediction unit for a square block and candidates for
arrangement of a prediction unit for a rectangular block may be different.
15 [0032]
Further, the shape determination information may be information acquired
based on a user input at a device that has encoded the image.
[0033]
Further, the image processing device may further include receiving means
20 for receiving the encoded stream and the shape determination information.
[0034]
Further, according to an embodiment of the present disclosure, there is
provided an image processing method for processing an image, including
determining a shape of each block in an image using shape determination
25 information for determining a shape of a block that is in accordance with a size and a
size of a block set in the image at a time of encoding of the image, determining an
arrangement of a prediction unit in each block according to the determined shape of
each block, generating a predicted image for each block using the determined
arrangement of a prediction unit and a motion vector acquired for each prediction
30 unit, and decoding an encoded stream in which the image is encoded, using the
generated predicted image.
Advantageous Effects of Invention
[003 51
As described above, according to the image processing device and the
5 image processing method of the present disclosure, a larger block can be handled
while suppressing the increase in a line memory.
Brief Description of Drawings
[0036]
10 [Fig. 11 Fig. 1 is a block diagram showing an example of a configuration of an image
encoding device according to an embodiment.
[Fig. 21 Fig. 2 is a block diagram showing an example of a detailed configuration of
a motion estimation section of an image encoding device of an embodiment.
[Fig. 31 Fig. 3 is an explanatory diagram for describing a first embodiment of shape
15 determination information.
[Fig. 41 Fig. 4 is an explanatory diagram showing a first example of a hierarchical
structure of a coded block that can be defined by the first embodiment of the shape
determination information.
[Fig. 51 Fig. 5 is an explanatory diagram showing a second example of the
20 hierarchical structure of a coded block that can be defined by the first embodiment of
the shape determination information.
[Fig. 61 Fig. 6 is an explanatory diagram showing a third example of the hierarchical
structure of a coded block that can be defined by the first embodiment of the shape
determination information.
25 [Fig. 71 Fig. 7 is an explanatory diagram for describing arrangement of prediction
units in a square block.
[Fig. 81 Fig. 8 is an explanatory diagram for describing arrangement of prediction
units in a block having a 2NxN shape.
[Fig. 91 Fig. 9 is an explanatory diagram for describing an estimation range for a
30 motion vector in the first embodiment of the shape determination information.
[Fig. 101 Fig. 10 is an explanatory diagram for describing arrangement of transform
units in a block having a 2NxN shape.
[Fig. 111 Fig. 11 is an explanatory diagram showing an example of a coded block
pattern with respect to a transform unit having a 2NxN shape.
[Fig. 121 Fig. 12 is an explanatory diagram for describing a second embodiment of
5 the shape determination information.
[Fig. 131 Fig. 13 is an explanatory diagram showing an example of a hierarchical
structure of a coded block that may be defined by the second embodiment of the
shape determination information.
[Fig. 141 Fig. 14 is an explanatory diagram for describing arrangement of prediction
10 units in a block having a 4NxN shape.
[Fig. 151 Fig. 15 is an explanatory diagram for describing transform units that may
be set in a block having a 4NxN shape.
[Fig. 161 Fig.16 is an explanatory diagram showing an example of a coded block
pattern of a block having a 4NxN shape.
15 [Fig. 171 Fig. 17 is an explanatory diagram for describing a third embodiment of the
shape determination information.
[Fig. 181 Fig. 18 is an explanatory diagram showing a first example of a hierarchical
structure of a coded block that can be defined by the third embodiment of the shape
determination information.
20 [Fig. 191 Fig. 19 is an explanatory diagram showing a second example of the
hierarchical structure of a coded block that can be defined by the third embodiment
of the shape determination information.
[Fig. 201 Fig. 20 is an explanatory diagram showing a third example of the
hierarchical structure of a coded block that can be defined by the third embodiment
25 of the shape determination information.
[Fig. 211 Fig. 21 is an explanatory diagram showing a fourth example of the
hierarchical structure of a coded block that can be defined by the third embodiment
of the shape determination information.
[Fig. 221 Fig. 22 is a first half of a flow chart showing an example of a flow of a
30 motion estimation process according to an embodiment.
[Fig. 231 Fig. 23 is a second half of the flow chart showing an example of a flow of a
motion estimation process according to an embodiment.
[Fig. 241 Fig. 24 is a block diagram showing an example of a configuration of an
image decoding device according to an embodiment.
[Fig. 251 Fig. 25 is a block diagram showing an example of a detailed configuration
5 of a motion compensation section of an image decoding device according to an
embodiment.
[Fig. 261 Fig. 26 is a flow chart showing an example of a flow of a motion
compensation process according to an embodiment.
[Fig. 271 Fig. 27 is a block diagram showing an example of a schematic
10 configuration of a television.
[Fig. 281 Fig. 28 is a block diagram showing an example of a schematic
configuration of a mobile telephone.
[Fig. 291 Fig. 29 is a block diagram showing an example of a schematic
configuration of a recording/reproduction device.
15 [Fig. 301 Fig. 30 is a block diagram showing an example of a schematic
configuration of an image capturing device.
Description of Embodiments
[0037]
20 Hereinafter, preferred embodiments of the present invention will be
described in detail with reference to the appended drawings. Note that, in this
specification and the drawings, elements that have substantially the same function
and structure are denoted with the same reference signs, and repeated explanation is
omitted.
25 [0038]
Furthermore, the "Description of Embodiments" will be described in the
order mentioned below.
1. Example Configuration of Image Encoding Device According to an
Embodiment
30 2. Example of Shape Determination Information
3. Flow of Process at the Time of Encoding According to an Embodiment
4. Example Configuration of Image Decoding Device According to an
Embodiment
5. Flow of Process at the Time of Decoding According to an Embodiment
6. Example Application
5 7. Summary
100391

First, an example configuration of an image encoding device according to
10 an embodiment will be described using Figs. 1 and 2.
[0040]
[l - 1. Example of Overall Configuration]
Fig. 1 is a block diagram showing an example of a configuration of an
image encoding device 10 according to an embodiment. Referring to Fig. 1, the
15 image encoding device 10 includes an A/D (Analogue to Digital) conversion section
11, a sorting buffer 12, a subtraction section 13, an orthogonal transform section 14,
a quantization section 15, a lossless encoding section 16, an accumulation buffer 17,
a rate control section 18, an inverse quantization section 21, an inverse orthogonal
transform section 22, an addition section 23, a deblocking filter 24, a frame memory
20 25, a selector 26, an intra prediction section 30, a motion estimation section 40, and a
mode selection section 50.
[004 11
The A/D conversion section 11 converts an image signal input in an
analogue format into image data in a digital format, and outputs a series of digital
25 image data to the sorting buffer 12.
[0042]
The sorting buffer 12 sorts the images included in the series of image data
input from the A/D conversion section 11. After sorting the images according to the
a GOP (Group of Pictures) structure according to the encoding process, the sorting
30 buffer 12 outputs the image data which has been sorted to the subtraction section 13,
the intra prediction section 30, and the motion estimation section 40.
*
[0043]
The image data input from the sorting buffer 12 and predicted image data
selected by the mode selection section 50 described later are supplied to the
subtraction section 13. The subtraction section 13 calculates predicted error data
5 which is a difference between the image data input from the sorting buffer 12 and the
predicted image data input from the mode selection section 50, and outputs the
calculated predicted error data to the orthogonal transform section 14.
[0044]
The orthogonal transform section 14 performs orthogonal transform for each
10 transform unit (TU) described later on the predicted error data input from the
subtraction section 13. The orthogonal transform to be performed by the orthogonal
transform section 14 may be discrete cosine transform (DCT) or Karhunen-Loeve
transform, for example. The orthogonal transform section 14 outputs transform
coefficient data acquired by the orthogonal transform process to the quantization
15 section 15.
[0045]
The transform coefficient data input from the orthogonal transform section
14 and a rate control signal from the rate control section 18 described later are
supplied to the quantization section 15. The quantization section 15 quantizes the
20 transform coefficient data, and outputs the transform coefficient data which has been
, quantized (hereinafter, referred to as quantized data) to the lossless encoding section
16 and the inverse quantization section 21. Also, the quantization section 15
switches a quantization parameter (a quantization scale) based on the rate control
signal from the rate control section 18 to thereby change the bit rate of the quantized
25 data to be input to the lossless encoding section 16.
[0046]
The quantized data input from the quantization section 15 and information
described later about intra prediction or inter prediction generated by the intra
prediction section 30 or the motion estimation section 40 and selected by the mode
30 selection section 50 are supplied to the lossless encoding section 16. Also,
definition information about the shape of a block output from the motion estimation
section 40 is supplied to the lossless encoding section 16. The information about
intra prediction may include prediction mode information indicating the optimal intra
prediction mode for each block, for example. Also, as will be described later in
detail, the information about inter prediction may include size specification
5 information specifying the size of a block set in an image, prediction mode
information specifying the arrangement of prediction units in each block, motion
vector information, and reference image information, for example. Furthermore,
the definition information about the shape of a block may include largest size
information, smallest size information, and shape determination information
10 described later.
[0047]
The lossless encoding section 16 generates an encoded stream by
performing a lossless encoding process on the quantized data. The lossless
encoding by the lossless encoding section 16 may be variable-length coding or
15 arithmetic coding, for example. Furthermore, the lossless encoding section 16
multiplexes the information about intra prediction or the information about inter
prediction mentioned above to the header of the encoded stream (for example, a
block header, a slice header or the like). Furthermore, the lossless encoding section
16 multiplexes the definition information supplied from the motion estimation
20 section 40 to, for example, a sequence parameter set, a picture parameter set, or the
slice header of the encoded stream. Then, the lossless encoding section 16 outputs
the generated encoded stream to the accumulation buffer 17.
[0048]
The accumulation buffer 17 temporarily stores the encoded stream input
25 from the lossless encoding section 16 using a storage medium, such as a
semiconductor memory. Then, the accumulation buffer 17 outputs the accumulated
encoded stream at a rate according to the band of a transmission line (or an output
line from the image encoding device 10).
[0049]
30 The rate control section 18 monitors the free space of the accumulation
buffer 17. Then, the rate control section 18 generates a rate control signal
according to the free space on the accumulation buffer 17, and outputs the generated
rate control signal to the quantization section 15. For example, when there is not
much free space on the accumulation buffer 17, the rate control section 18 generates
a rate control signal for lowering the bit rate of the quantized data. Also, for
5 example, when the free space on the accumulation buffer 17 is sufficiently large, the
rate control section 18 generates a rate control signal for increasing the bit rate of the
quantized data.
[0050]
The inverse quantization section 21 performs an inverse quantization
10 process on the quantized data input from the quantization section 15. Then, the
inverse quantization section 21 outputs transform coefficient data acquired by the
inverse quantization process to the inverse orthogonal transform section 22.
[005 11
The inverse orthogonal transform section 22 performs an inverse orthogonal
15 transform process on the transform coefficient data input from the inverse
quantization section 2 1 to thereby restore the predicted error data. Then, the inverse
orthogonal transform section 22 outputs the restored predicted error data to the
addition section 23.
[0052]
20 The addition section 23 adds the restored predicted error data input from the
inverse orthogonal transform section 22 and the predicted image data input from the
mode selection section 50 to thereby generate decoded image data. Then, the
addition section 23 outputs the generated decoded image data to the deblocking filter
24 and the frame memory 25.
25 [0053]
The deblocking filter 24 performs a filtering process for reducing block
distortion occurring at the time of encoding of an image. The deblocking filter 24
filters the decoded image data input from the addition section 23 to remove the block
distortion, and outputs the decoded image data after filtering to the frame memory 25.
30 [0054]
The frame memory 25 stores, using a storage medium, the decoded image
data input from the addition section 23 and the decoded image data after filtering
input from the deblocking filter 24.
[0055]
The selector 26 reads, from the frame memory 25, the decoded image data
5 before filtering that is to be used for the intra prediction, and supplies the decoded
image data which has been read to the intra prediction section 30 as reference image
data. Also, the selector 26 reads, from the frame memory 25, the decoded image
data after filtering to be used for the inter prediction, and supplies the decoded image
data which has been read to the motion estimation section 40 as reference image data.
10 [0056]
The intra prediction section 30 performs an intra prediction process in each
intra prediction mode defined by H.264/AVC, based on the encoding target image
data that is input from the sorting buffer 12 and the decoded image data supplied via
the selector 26. For example, the intra prediction section 30 evaluates the
15 prediction result of each intra prediction mode using a predetermined cost function.
Then, the intra prediction section 30 selects an intra prediction mode by which the
cost function value is the smallest, that is, an intra prediction mode by which the
compression ratio (the encoding efficiency) is the highest, as the optimal intra
prediction mode. Furthermore, the intra prediction section 30 outputs, to the mode
20 selection section 50, prediction mode information indicating the optimal intra
prediction mode, the predicted image data, and the information about intra prediction
such as the cost function value. Moreover, the intra prediction section 30 may
perform the intra prediction process with a larger block than each intra prediction
mode defined by H.264/AVC, based on the encoding target image data input from the
25 sorting buffer 12 and the decoded image data supplied via the selector 26. Also in
this case, the intra prediction section 30 evaluates the prediction result of each intra
prediction mode using a predetermined cost function, and outputs, to the mode
selection section 50, the information about intra prediction for the optimal intra
prediction mode.
30 [0057]
The motion estimation section 40 performs a motion estimation process
using a largest block size larger than that of the macro block defined by H.264/AVC,
based on the encoding target image data input from the sorting buffer 12 and the
decoded image data supplied via the selector 26. For example, the motion
estimation section 40 arranges prediction units in a largest block that can be set and
5 in each block obtained by partitioning the largest block using a plurality of prediction
modes, respectively. Also, the motion estimation section 40 determines a motion
vector for each prediction unit arranged by each prediction mode. Then, the motion
estimation section 40 evaluates each prediction mode using a predetermined cost
function. Then, the motion estimation section 40 selects a block size and a
10 prediction mode by which the cost function value is the smallest, that is, a block size
and a prediction mode by which the compression ratio (the encoding efficiency) is
the highest, as the optimal block size and the optimal prediction mode. The motion
estimation process by the motion estimation section 40 will be described later in
greater detail. The motion estimation section 40 outputs, to the mode selection
15 section 50, the information about inter prediction, such as the size specification
information specifying a block size by which the cost function value is the smallest,
the prediction mode information specifying the arrangement of prediction units, the
motion vector information, the reference image information, the predicted image data,
the cost function value and the like.
20 [0058]
The mode selection section 50 compares the cost function value related to
the intra prediction input from the intra prediction section 30 and the cost function
value related to the inter prediction input from the motion estimation section 40.
Then, the mode selection section 50 selects a prediction method with a smaller cost
25 function value, from the intra prediction and the inter prediction. In the case of
selecting the intra prediction, the mode selection section 50 outputs the information
about intra prediction to the lossless encoding section 16, and also, outputs the
predicted image data to the subtraction section 13. Also, in the case of selecting the
inter prediction, the mode selection section 50 outputs the size specification
30 information about the inter prediction, the prediction mode information, the motion
vector information and the reference image information to the lossless encoding
section 16, and also, outputs the predicted image data to the subtraction section 13.
[0059]
[I-2. Example Configuration of Motion Estimation Section]
Fig. 2 is a block diagram showing an example of a detailed configuration of
5 the motion estimation section 40 of the image encoding device 10 shown in Fig. 1.
Referring to Fig. 2, the motion estimation section 40 includes an information holding
section 41, a shape determination section 42, a motion vector setting section 43, and
a motion compensation section 44.
[0060]
10 The information holding section 41 holds in advance, using a storage
medium such as a semiconductor memory, the largest size information, the smallest
size information, and the shape determination information. The largest size
information is information defining the size of a largest block that can be set in an
image. For example, the size of .a largest block that can be set in an image
15 according to H264lAVC is 16x 16 pixels. On the other hand, according to extended
standards, the largest size information may indicate a value exceeding 16 (for
example, 32, 64, 128, or the like). The smallest size information is information
defining the size of a smallest block that can be set in an image.
[006 11
20 The shape determination information is information defined to determine
the shape of a block in accordance with a size. That is, the shape determination
information is information allowing, in a case a size is specified for a certain block,
to determine the shape of the block according to the shape determination information
and the specified size. A concrete example of the shape determination information
25 will be described later in detail by citing three examples.
[0062]
Additionally, in the present embodiment, the size of a block used for
encoding of an image may be expressed by the number of times of partitioning from
a largest block. For example, if the number of times of partitioning is zero, the size
30 of the block is equal to the size of the largest block. If the number of times of
partitioning from a largest block is one or more, the size of the block is smaller than
the size of the largest block. In the present specification, the number of times of
partitioning of each block from a largest block will be referred to as a "depth" of a
block (or a layer).
[0063]
5 The shape determination section 42 determines the shape of a block that can
be set in an image using the shape determination information described above held in
the information holding section 41. In the present embodiment, the shape of each
block in an image is either a square or a rectangle. Accordingly, the shape
determination section 42 refers to the shape determination information, and, for
10 example, determines that the shape of a block of a certain size is a rectangle and the
shape of a block of another size is a square.
[0064]
Additionally, as shown in Fig. 2, the information held in the information
holding section 41 may be input or updated by a user using the image encoding
15 device 10 (for example, a developer who embeds the image encoding device 10 in an
electronic appliance as an encoder, or the like). That is, a user using the image
encoding device 10 is allowed to specify the largest size information, the smallest
size information, and the shape determination information to be used at the time of
an image encoding process, according to the use of the image encoding device 10.
20 [0065]
For example, the user may specify the shape determination information such
that the shape of a block of largest size will be a rectangle. A block of 32x 16 pixels,
for example, can therefore be used as a block of a size exceeding the largest macro
block size (1 6x 16 pixels) defined by H.264/AVC, for example. In this case, while a
25 line memory of a size the same as that of a conventional method (16 pixels) can be
used because the length of the short side of the block is not changed, the block size is
extended and thus the compression ratio of an image can be further increased. Also,
for example, in the case of embedding the image encoding device 10 in an appliance
capable of securing a line memory of a larger size, the user may specify the shape
30 determination information such that the shape of the block of largest size will be a
square. These pieces of definition information specified by the user and held in the
information holding section 41 are output to the lossless encoding section 16 shown
in Fig. 1
[0066]
The motion vector setting section 43 includes prediction unit arrangement
5 means for arranging a prediction unit in each block in the image, according to the
shape of the block described above determined by the shape determination section 42.
Then, the motion vector setting section 43 sets, for each prediction unit arranged
according to the shape of the block, a motion vector to be used for prediction of an
image in the prediction unit, for example.
10 [0067]
More specifically, for example, the motion vector setting section 43 first
arranges prediction units in a plurality of prediction modes (arrangement patterns of
the prediction units) in a block at a depth zero (a largest block), and decides the
motion vector of each prediction unit. Also, the motion vector setting section 43
15 arranges prediction units in a plurality of prediction modes in a block at a depth one,
and decides a motion vector of each prediction unit. Likewise, the motion vector
setting section 43 arranges prediction units in a plurality of prediction modes in a
corresponding block at each depth down to a block of a smallest size, and decides a
motion vector of each prediction unit. As a result, a plurality of sets of motion
20 vectors each corresponding to a combination of a depth and a prediction mode are
decided. Additionally, when deciding the motion vector of each prediction unit, a
pixel value of an original image in a prediction unit and a pixel value of a reference
image input from the frame memory 25 are used. Next, the motion vector setting
section 43 selects an optimal combination of a depth and a prediction mode by which
25 the cost function value will be the smallest. Then, the motion vector setting section
43 outputs, to the motion compensation section 44, the size specification information
specifying the block size according to the selected combination (information
corresponding to the depth), the prediction mode information, the motion vector
information, the reference image information, and the cost function value.
30 [0068]
Here, in the present embodiment, each block has a square or rectangle shape
according to the size of the block, as described above. Thus, candidates for
arrangement of prediction units for a square block and candidates for arrangement of
prediction units for a rectangular block in the motion vector setting section 43 are
different. Such candidates for arrangement of prediction units will be further
5 described later.
[0069]
The motion compensation section 44 generates predicted image data using
an optimal motion vector set by the motion vector setting section 43 and the
reference image data (decoded image data after filtering) input fiom the frame
10 memory 25. Then, the motion compensation section 44 outputs, to the mode
selection section 50, the generated predicted image data, and the information about
inter prediction such as the size specification information, the prediction mode
information, the motion vector information, the reference image information and the
like.
15 [0070]
<2. Example of Shape Determination Information>
Next, three typical embodiments of the shape determination information
will be described using Figs. 3 to 2 1.
[0071]
20 [2- 1. First Embodiment]
(Explanation on Data)
Fig. 3 is an explanatory diagram for describing a first embodiment of the
shape determination information. Referring to Fig. 3, definition information 4 1 a
that may be held in the information holding section 41 in the first embodiment is
25 shown. The definition information 41 a includes largest size information
"high-layer", smallest size information "low-layer", and shape determination
information "high-square-layer". Additionally, the name of each parameter shown
here is only an example, and other parameter names may also be used.
[0072]
30 The largest size information "high-layer" indicates the size of a largest
block that can be set in an image. The value of the largest size information is 64,32,
16, 8, or 4, for example. The smallest size information "low-layer" indicates the
size of a smallest block that can be set in an image. The value of the smallest size
information is 64, 32, 16, 8, or 4, for example. The largest size information and the
smallest size information are information that can also be used in H264/AVC, for
5 example.
[0073]
The "high-square - layer", which is the shape determination information
according to the present embodiment, indicates a boundary value for the size of a
block that separates a square block and a non-square block. The
10 "high - square - layer" may take a value of a block size within a range defined by
"high-layer" and "low-layer" (64, 32, 16, 8, or 4 in the example of Fig. 3). The
boundary value "high - square - layer" means that a block whose block size exceeds
the value is a non-square and a block whose block size is equal to or below the value
is a square. Accordingly, in the case of "high - square - layer" = "high-layer", the
15 blocks at all the depths (of all the sizes) are squares. On the other hand, in the case
of "high - square-layer" < "low - layer", the blocks at all the depths (of all the sizes)
are non-squares.
[0074]
In the present embodiment, the shape of a non-square block is a rectangle
20 having a fixed ratio of a long side to a short side. In the following explanation, a
case where the ratio of a long side to a short side is 2 to 1 will be described.
However, it is not limited to such an example, and other ratios may also be used as
the ratio of a long side to a short side of a non-square block.
[0075]
2 5 Additionally, the size values of "high - layer", "low - layer", and
"high-square-layer" correspond to the block sizes shown in the lower table in Fig. 3,
for example. That is, in the case a size value (N) is 64, the size of a square block is
128x 128 pixels and the size of a rectangular block is 128x64 pixels. In the case the
size value (N) is 32, the size of the square block is 64x64 pixels and the size of the
30 rectangular block is 64x32 pixels. In the case the size value (N) is 16, the size of
the square block is 32x32 pixels and the size of the rectangular block is 32x 16 pixels.
In the case the size value (N) is 8, the size of the square block is 16x 16 pixels and the
size of the rectangular block is 16x8 pixels. In the case the size value (N) is 4, the
size of the square block is 8x8 pixels and there is no rectangular block. Incidentally,
these size definitions are merely examples, and other size value definitions may also
5 be used.
[0076]
The semantics of the definition information 41a will be shown below.
Additionally, in the following semantics, "HighSQsize" and "LowSQsize" are
variables indicating the upper limit and the lower limit, respectively, of the size of a
10 block having a square shape. Also, "HighNSsize" and "LowNSsize" are variables
indicating the upper limit and the lower limit, respectively, of the size of a block
having a non-square shape:
[0077]
[Math. I]
If high.,,.square-layer = high-layer // there are only Square layers
HighSQsize = high-layer
LoKSQsiae = low-layer
If high-square-layer < lov-layer // there are on1 y Non-Square layers
HighNSsize = high-layer
LowNSsize = low-layer
Else // there will be both Non-Square and Square layers
HighNSsize = high-layer
tawMSsize - high-square-layer x 2
HighSQsiee = highsquare-layer
LowSQsize = tow-layer
(Hierarchical Structure of Coded Block)
Fig. 4 is an explanatory diagram showing a first example of a hierarchical
structure of a coded block that can be defined by the first embodiment. In the
20 example of Fig. 4, the definition information 41a indicates that "high-layer" is 64,
"low-layer" is 4, and "high-square-layer" is 64. In this case, since
"high-square-layer" and "high-layer" are equal, the blocks at all the depths (of all
the sizes) are squares. That is, as shown in Fig. 4, a block of largest size (128x128)
at a depth zero, a block of the second size (64x64) at a depth one, and a block of the
smallest size (8x8) at a depth four are all squares.
[0079]
5 Fig. 5 is an explanatory diagram showing a second example of the
hierarchical structure of a coded block that can be defined by the first embodiment.
In the example of Fig. 5, the definition information 41a indicates that "high-layer" is
64, "low-layer" is 8, and "high-square - layer" is -1. In this case, since
"high-square-layer" is smaller than "low - layer", the blocks at all the depths (of all
10 the sizes) are non-squares. That is, as shown in Fig. 5, a block of largest size
(128x64) at a depth (the number of times of partitioning) zero, a block of the second
size (64x32) at a depth one, and a block of the smallest size (16x8) at a depth three
are all non-squares.
[0080]
15 Fig. 6 is an explanatory diagram showing a third example of the hierarchical
structure of a coded block that can be defined by the first embodiment. In the
example of Fig. 6, the definition information 41a indicates that "high-layer" is 32,
"low - layer" is 4, and "high-square-layer" is 8. In this case, both a non-square
block and a square block can be set according to the block size. That is, as shown
20 in Fig. 6, a block of the largest size (64x32) at a depth zero and a block of the second
size (32x 16) at a depth one are non-squares. Also, a block of the third size (16x 16)
at a depth two and a block of the smallest size (8x8) at a depth three are squares.
[008 11
Additionally, as can be understood from Figs. 4 to 6, in the case of
25 partitioning a block without changing the shape of the block (for example, in the case
the shapes of blocks before and after partitioning are both rectangles or both squares),
a block is partitioned into four with the center lines in the vertical direction and the
horizontal direction as the separations, for example. On the other hand, in the case
a rectangular block is to be partitioned into square blocks across the boundary value
30 indicated by "high-square-layer", the block may be partitioned into two with the line
in the vertical direction as the separation (four, in the case the ratio of the long side to
the short side is four to one), for example.
[0082]
(Prediction Unit: Square Block)
Fig. 7 is an explanatory diagram for describing arrangement of prediction
units in a block having a 2Nx2N shape, that is, a square block. Referring to Fig. 7,
nine types of prediction modes, from Mode 0 to Mode 8, are illustrated.
[0083]
Among these, Mode 0 is a distinct prediction mode that is selected in a case
a skipped macro block mode or a direct mode is used. In Mode 0, one prediction
unit of size 2Nx2N is arranged in the block. Mode 1 is a prediction mode in which
one prediction unit of size 2Nx2N is arranged in the block. Mode 2 is a prediction
mode in which two prediction units of size 2NxN are arranged vertically in the block.
Mode 3 is a prediction mode in which two prediction units of size Nx2N are
arranged horizontally in the block. Mode 4 is a prediction mode in which four
prediction units of size NxN are arranged, two in the vertical direction and two in the
horizontal direction, in the block. These Modes 0 to 4 are prediction modes
adopted by H264lAVC.
[0084]
Mode 5 is a prediction mode in which two prediction units of size 2NxnU
(N12, 3NI2) are arranged vertically in the block. Mode 6 is a prediction mode in
which the arrangement of the prediction units of Mode 5 is flipped vertically. Mode
7 is a prediction mode in which two prediction units of size nL(N12, 3N/2)x2N are
arranged horizontally in the block. Mode 8 is a prediction mode in which the
arrangement of the prediction units of Mode 7 is flipped horizontally. These Modes
5 to 8 may also be used as the candidates for the prediction mode with respect to a
square block.
[0085]
(Prediction Unit: Two-to-One Rectangular Block)
Fig. 8 is an explanatory diagram for describing the arrangement of the
prediction units in a block having a shape of a rectangle whose ratio of a long side to
a short side is 2 : 1. Referring to Fig. 8, nine types of prediction modes, from Mode
0 to Mode 8, are illustrated. Also, a partition mode is shown as Mode 9.
[0086]
Among these, Mode 0 is a distinct prediction mode that is selected in a case
a skipped macro block mode or a direct mode is used. In Mode 0, one prediction
5 unit of size 2NxN is arranged in the block. Mode 1 is a prediction mode in which
one prediction unit of size 2NxN is arranged in the block. Mode 2 is a prediction
mode in which two prediction units of size 2NxN12 are arranged vertically in the
block. Mode 3 is a prediction mode in which two prediction units of size NxN are
arranged horizontally in the block. Mode 4 is a prediction mode in which four
10 prediction units of size NxNI2 are arranged, two in the vertical direction and two in
the horizontal direction, in the block. Mode 5 is a prediction mode in which two
prediction units of size N/2xN and 3NI2xN are arranged horizontally in the block.
Mode 6 is a prediction mode in which the arrangement of the prediction units of
Mode 5 is flipped horizontally. Mode 7 and Mode 8 are prediction modes in which
15 two triangular prediction units which are separated by a diagonal line of the block are
arranged in the block.
[0087]
Mode 9 is a partition mode indicating that a block is to be further partitioned.
In the case the partition mode is indicated as the prediction mode for a block at a
20 certain depth, the block is partitioned into a smaller blocks. Accordingly, the
number of partition modes specified for a certain block of the largest size represents
the number of times of partitioning from the block of the largest size. This means
that the number of partition modes serves as the size specification information
specifying the size of a block set in an image. Additionally, in the explanation in
25 the present specification, the size specification information and the prediction mode
information are distinguished from each other, but the size specification information
may also be treated as a part of the prediction mode information. Additionally, as
described above, in the case of partitionining a block without changing the shape of
the block, the block may be partitioned into four with the center lines in the vertical
30 direction and the horizontal direction as the separations, for example. On the other
hand, in the case a rectangular block is to be partitioned into square blocks across the
boundary value indicated by "high-square-layer", the block may be partitioned with
the line in the vertical direction as the separation, for example.
[0088]
(Estimation Range for Motion Vector)
Fig. 9 is an explanatory diagram for describing an estimation range for a
motion vector in the first embodiment of the shape determination information.
[0089]
Referring to Fig. 9, a tree structure showing an estimation range for a
motion vector in a case of "high-layer" = 64, "low-layer" = 4, and
"high - square - layer" = 8 is shown. A first layer at the highest level of the tree
structure includes nine prediction modes, from Mode 0 to Mode 8, that may be
applied to a non-square block of largest size of 128x64 pixels. Also, a second layer
following the first layer includes nine prediction modes, from Mode 0 to Mode 8,
that may be applied to a non-square block of size of 64x32 pixels. Furthermore, a
third layer following the second layer includes nine prediction modes, from Mode 0
to Mode 8, that may be applied to a non-square block of size of 32x16 pixels. A
fourth layer following the third layer includes prediction modes from Mode 0 to
Mode M that may be applied to a square block of size of 16x16 pixels. A fifth layer
following the fourth layer includes prediction modes from Mode 0 to Mode M that
may be applied to a square block of smallest size of 8x8 pixels.
[0090]
The motion vector setting section 43 of the motion estimation section 40
shown in Fig. 2 estimates a motion vector according to the shape of a block of each
block size determined by the shape determination section 42, taking every prediction
mode of the first layer to the fifth layer included in such tree structure as the target.
Then, the motion vector setting section 43 selects a prediction mode by which the
cost function value is the smallest within an estimation range as a prediction mode
for identifying the arrangement of prediction units to be used for encoding. Such an
optimal prediction mode is uniquely identified by the number of a prediction mode
and the depth of a layer as illustrated in Fig. 7 or 8. The depth indicates the number
of times of partitioning from a block of the largest size, and is information specifying
the size of a block after partitioning. The number of a prediction mode and the
depth are multiplexed, as the prediction mode information and the size specification
information, within a block header of an encoded stream, for example. Also, the
largest size information and the shape determination information are multiplexed
5 within a sequence parameter set, a picture parameter set or a slice header of an
encoded stream, for example. Therefore, a device that decodes an encoded image
can recognize the size, the shape, and the arrangement of the prediction units of a
block used at the time of encoding, by acquiring these pieces of multiplexed
information from the encoded stream.
10 [0091]
(Transform Unit: Two-to-One Rectangular Block)
Fig. 10 is an explanatory diagram for describing arrangement of transform
units (TU), which are units of an orthogonal transform process, in a block having a
2NxN shape.
15 [0092]
Referring to Fig. 10, five arrangement patterns of transform units from type
TO to type T4 are shown. Among these, type TO is a pattern of arranging one
transform unit of size 2NxN in a block having a 2NxN shape. Type T1 is a pattern
of horizontally arranging two transform units of size NxN next to each other in the
20 block. Type T2 is a pattern of arranging four transform units of size NxN/2 in the
block, two arranged next to each other in the vertical direction and two arranged next
to each other in the horizontal direction, Type T3 is a pattern of arranging four
transform units of size NI2xN next to each other in the horizontal direction in the
block. Type T4 is a pattern of arranging eight transform units of size N/2xN/2, two
25 arranged next to each other in the vertical direction and four arranged next to each
other in the horizontal direction. The orthogonal transform section 14 performs
orthogonal transform for each transform unit of any of these five types, for example,
and outputs a transform coefficient data.
[0093]
30 Table 1 shows a mapping of the prediction units (PU) illustrated in Fig. 8
and the transform units (TU) illustrated in Fig. 10. The "0" in Table 1 indicates
that if a prediction unit in the left column is selected, the corresponding transform
unit may be used. The "xu indicates that if a prediction unit in the left column is
selected, the corresponding transform unit may not be used.
[0094]
[Table 11
Table 1. Mapping of Prediction Unit (PU) and Transform Unit (TU) (Ratio: 2 to 1)
[0095]
Additionally, generally, the units of orthogonal transform may be units of
32x32 pixels at the largest to allow execution. Accordingly, it should be noted that
even if "0" is assigned in Table 1, a transform unit that cannot be used may possibly
exist depending on the block size.
[0096]
Fig. 11 is an explanatory diagram showing an example of a coded block
pattern (CBP) with respect to a transform unit having a 2NxN shape. In the case of
performing orthogonal transform with a transform unit having a 2NxN shape,
whereas transform coefficients of 2NxN are generated from a luma signal (Y),
transform coefficients of NxN12 are respectively generated from chroma signals (Cb,
Cr), for example. In this case, the transform coefficients of the luma signal (Y) are
grouped into four groups (YO, Y1, Y2, and Y3), for example, and four bits (bO, bl,
b2, and b3) are assigned to respective groups. Also, one bit (b4 or b5) is assigned
to each of the transform coefficients of the chroma signals (Cb, Cr). Whether a
significant transform coefficient other than a DC value is generated or not can
thereby be indicated by the value of each bit.
[0097]
5 (Summary of First Embodiment)
According to the first embodiment described above, the shape determination
information includes information about a boundary value of a size of a block that
separates a square block from a non-square block. Then, the shape determination
section 42 determines that the shape of a block having a size larger than the boundary
10 value is a non-square and that the shape of a block having a size smaller than the
boundary value is a square. By allowing such stepwise definition of a block shape,
a method that handles larger sizs such as 32x16 pixels, or 64x32 pixels and 32x16
pixels can be adopted while supporting the H.264lAVC method according to which
the largest macro block is a square of 16x 16 pixels, for example.
15 [0098]
Furthermore, the shapes of these blocks of larger sizes are rectangles.
Therefore, by making the size of a line memory required for image processing the
same as the size of the shorter sides, increase in the line memory accompanying the
extension of a block size can be can be prevented.
20 [0099]
Also, the largest size information and the smallest size information
described above are already usable in the H.264/AVC method and the like, and the
only information to be newly defined in the first embodiment is "high-square-layer",
which is the shape determination information. Accordingly, the framework of such
25 a new block size can be easily introduced, and the possibility of the compression
ratio decreasing due to the increase in the header information is small.
[O 1001
12-2. Second Embodiment]
(Explanation on Data)
30 Fig. 12 is an explanatory diagram for describing a second embodiment of
the shape determination information. Referring to Fig. 12, definition information
41b which may be held by the information holding section 41 in the second
embodiment is shown. The definition information 41b includes largest size
information "high-layer", smallest size information "low-layer" and shape
determination information "width-to-height-ratio".
5 [OlOl]
The "width - to - height-ratio", which is the shape determination information
according to the present embodiment, indicates the ratio of a long side to a short side
of a block of largest size. The value of "width-to-height-ratio" may be 1, 2 or 4,
for example. In the case one is specified as the value of "width-to-height-ratio",
10 only a square block can be set in an image. On the other hand, in the case a value
other than one is specified as the value of "width-to-height-ratio", the shape of the
block of largest size will be a rectangle.
[O 1021
In the present embodiment, the shape of a block of largest size is assumed to
15 be a square or a rectangle, and the shape of a block of smaller size is assumed to be a
square. The semantics of the definition information 41 b is shown below:
[0 1031
[Math. 21
If width-ta-height-ratio = 1 I/ there are only Square layers
HighSQsize = high-layer
LowSQsiee = low-layer
Else // there will be both NOR-Square and Square layers
KighNSsize = high-layer
LawNSsize = HigMSsize
HighSQsize = high-layer / 2
LowSQsize = low-layer
20 [0104]
(Hierarchical Structure of Coded Block)
Fig. 13 is an explanatory diagram showing an example of a hierarchical
structure of a coded block that may be defined by the second embodiment. In the
example of Fig. 13, the definition information 4 1 b indicates that "high-layer" is 16,
"low-layer" is 4, and "width-to-height-ratio" is 4. In this case, both a non-square
block and a square block may be set depending on the block size. That is, as shown
in Fig. 13, a block of largest size (64x 16) at a depth zero is a rectangle with the ratio
of a long side to a short size being four (four to one). Also, the shape of each of
5 blocks at a depth of one or more is a rectangle.
[0 1051
(Prediction Unit : Four-to-One Rectangular Block)
Fig. 14 is an explanatory diagram for describing arrangement of prediction
units in a block of a rectangular shape whose ratio of a long side to a short side is
10 four to one. Referring to Fig. 14, nine types of prediction modes, from Mode 0 to
Mode 8, are illustrated. Also, a partition mode is shown as Mode 9.
[0 1061
Among these, Mode 0 is a distinct prediction mode that is selected in a case
a skipped macro block mode or a direct mode is used. In Mode 0, one prediction
15 unit of size 4NxN is arranged in the block. Mode 1 is a prediction mode in which
one prediction unit of size 4NxN is arranged in the block. Mode 2 is a prediction
mode in which two prediction units of size 4NxN12are arranged vertically in the
block. Mode 3 is a prediction mode in which two prediction units of size 2NxN are
arranged horizontally in the block. Mode 4 is a prediction mode in which four
20 prediction units of size NxN are arranged horizontally in the block. Mode 6 is a
prediction mode in which the arrangement of the prediction units of Mode 5 is
flipped horizontally. Mode 7 and Mode 8 are prediction modes in which two
triangular prediction units which are separated by a diagonal line of the block are
arranged in the block.
25 [0107]
Mode 9 is a partition mode indicating that a block is to be further partitioned.
In the example of Fig. 13, the four-to-one rectangular block is partitioned into four
square blocks with three vertical, equally distanced lines as the separations. Also,
in the case rectangular blocks may be set over a plurality of layers, the four-to-one
30 rectangular block may be partitioned into four with the center lines in the vertical
direction and the horizontal direction as the separations, for example.
[0108]
Also in the present embodiment, the motion vector setting section 43 of the
motion estimation section 40 may estimate a motion vector, taking every prediction
mode of every layer included in the hierarchical structure illustrated in Fig. 13, for
5 example, as the target.
[0 1091
(Transform Unit: Four-to-One Rectangular Block)
Fig. 15 is an explanatory diagram for describing arrangement of transform
units (TU), which are units of an orthogonal transform process, in a block having a
10 4NxN shape.
[OllO]
Referring to Fig. 15, five arrangement patterns of transform units from type
TO to type T4 are shown. Among these, type TO is a pattern of arranging one
transform unit of size 4NxN in a block having a 4NxN shape. Type T1 is a pattern
15 of horizontally arranging two transform units of size 2NxN next to each other in the
block. Type T2 is a pattern of arranging four transform units of size 2NxN12 in the
block, two arranged next to each other in the vertical direction and two arranged next
to each other in the horizontal direction,. Type T3 is a pattern of arranging four
transform units of size NxN next to each other in the horizontal direction in the block.
20 Type T4 is a pattern of arranging eight transform units of size NxN12, two arranged
next to each other in the vertical direction and four arranged next to each other in the
horizontal direction. The orthogonal transform section 14 performs orthogonal
transform for each transform unit of any of these five types, for example, and outputs
a transform coefficient data.
25 [Olll]
Table 2 shows a mapping of the prediction units (PU) illustrated in Fig. 14
and the transform units (TU) illustrated in Fig. 15.
[0112]
[Table 21
30 Table 2. Mapping of Prediction Unit (PU) and Transform Unit (TU) (Ratio: 4 to 1)
[0113]
Additionally, as in the case of a two-to one rectangular block, it should be
noted that even if "0" is assigned in Table 2, a transform unit that cannot be used
may possibly exist depending on the block size.
[0114]
Fig. 16 is an explanatory diagram showing an example of a coded block
pattern (CBP) with respect to a transform unit having a 4NxN shape. In the case of
performing orthogonal transform with a transform unit having a 4NxN shape,
whereas transform coefficients of 4NxN are generated from a luma signal (Y),
transform coefficients of 2NxN/2 are respectively generated from chroma signals
(Cb, Cr), for example. In this case, the transform coefficients of the luma signal (Y)
are grouped into four groups (YO, Y1, Y2, and Y3), for example, and four bits (bO,
bl, b2, and b3) are assigned to respective groups. Also, one bit (b4 or b5) is
assigned to each of the transform coefficients of the chroma signals (Cb, Cr).
Whether a significant transform coefficient other than a DC value is generated or not
can thereby be indicated by the value of each bit.
[0115]
(Summary of Second Embodiment)
According to the second embodiment described above, the shape of each
block in an image is a square or a rectangle, and the shape determination information
includes ratio information defining the ratio of a long side to a short side of a block.
The shape of a block of largest size may be determined based on the ratio
information. By allowing such stepwise definition of a block shape, a method that
handles larger sizes such as 32x 16 pixels, 64x 16 pixels and the like can be adopted
while supporting the H.2641AVC method according to which the largest macro block
is a square of 16x 16 pixels, for example.
[0116]
Furthermore, according to the second embodiment, the ratio of a long side
to a short side regarding a rectangular block can be specified by a user, and thus, it is
possible to flexibly meet various needs of the user regarding the size of a line
memory, the compression ratio, and the image quality.
[0117]
Furthermore, also in the second embodiment, the only information to be
newly defined is "width-to - height - ratio", which is the shape determination
information. Accordingly, the framework of such a new block size can be easily
introduced, and the decrease in the compression ratio due to the increase in the
header information can be prevented.
[0118]
Additionally, both "high - square - layer" according to the first embodiment
and "width-to-height-ratio" according to the second embodiment may be used as the
shape determination information. In this case, a user can flexibly specify the
number of layers and the shape of a rectangular block, for example.
[0119]
[2-3 .Third Embodiment]
(Explanation on Data)
Fig. 17 is an explanatory diagram for describing a third embodiment of the
shape determination information. Referring to Fig. 17, definition information 4 1 c
which may be held by the information holding section 41 in the third embodiment is
shown. The definition information 41c includes, in addition to largest size
information "max-high-level", three data items, "number-of-layers",
"width-to-height-ratio[]" and "low~level[]", as the shape determination information.
[O 1201
The largest size information "max-high-level" indicates the size of a largest
block that can be set in an image. The value of the largest size information is, for
example, 128,64, 32, 16 or 8.
5 [0121]
The "number - of - layers" indicates the number of layers with different ratios
of a long side to a short side included in the hierarchical structure of a coded block.
For example, if the shapes of blocks of all the layers are the same in the hierarchical
structure of a coded block, the value of "number-of-layers" is one. Also, as with
10 the hierarchical structure illustrated in Fig. 13, in the case there is a layer where the
ratio of a long side to a short side is four (four to one) and there is a layer where the
ratio of a long side to a short side is one, the value of "number-of-layers" is two.
[O 1 221
The "width-to-height-ratio[]" is a data array indicating the ratio of each of
15 the layers with different ratios of a long side to a short side of a block. The number
of elements of the array is equal to the value of "number - of-layers". For example,
in the case the ratio in the first layer is four and the ratio in the second layer is one, as
in the hierarchical structure illustrated in Fig. 13, the contents of
"width-to-height - ratio[]" are (4, I), that is, width-to-height - ratio[l] is four and
20 width-to_height_ratio[2] is one.
[0 1231
The "low - level[]" is a data array indicating the smallest block size to which
each ratio defined by "width-to-height-ratio[]" is applied. The number of elements
of the array is equal to the value of "number-of-layers".
25 [0124]
The semantics of the definition information 41c is shown below.
Additionally, in the semantics below, High-level[k] and low - level[k] are variables
indicating the level of the layer of the largest size and the level of layer of the
smallest size, respectively, in a k-th layer group with different ratios.
30 [0125]
[Math. 31
high-level Ell = max-high-level
low-level El3 is obtained from the bitstreen
rat io[l] = width_to_hight_ratio[1]
For i = 2.. nuaber-aelayers
High-level [i] = low-level [i-l]
low-level[i] is obtained from the bitstream
ratio[il = width-to-hight-ratio [i]
Next
[0 1261
(Hierarchical Structure of Coded Block)
Fig. 18 is an explanatory diagram showing a first example of the
hierarchical structure of a coded block that can be defined by the third embodiment.
In the example of Fig. 18, the definition information 41c indicates that
"max - high-level" is 16, "number-of-layers" is 2, "width-to-height-ratio[]" is (2, l),
and "low~level[]" is (16,8). In this case, two layer groups with different ratios may
be set according to the block sizes. That is, as shown in Fig. 18, a block of largest
size (32x 16) at a depth zero is a rectangle whose ratio of a long side to a short side is
two. Also, each block of a size at a depth of one or more is a square.
[0 1271
Fig. 19 is an explanatory diagram showing a second example of the
hierarchical structure of a coded block that can be defined by the third embodiment.
In the example of Fig. 19, the definition information 41c indicates that
"rnax-high - level" is 16, "number-of-layers" is 2, "width-to-height-ratio[]" is (4,1),
and "low~level[]" is (16,8). In this case, two layer groups with different ratios may
be set according to the block sizes. That is, as shown in Fig. 19, a block of largest
size (64x 16) at a depth zero is a rectangle whose ratio of a long side to a short side is
four. Also, each block of a size at a depth of one or more is a square.
[0128]
Fig. 20 is an explanatory diagram showing a third example of the
hierarchical structure of a coded block that can be defined by the third embodiment.
In the example of Fig. 20, the definition information 41c indicates that
"max-high-level" is 32, "number-of-layers" is 2, "width-to - height-ratio[]" is (4,1),
and "low~level[]" is (16,8). In this case, two layer groups with different ratios may
be set according to the block sizes. That is, as shown in Fig. 20, a block of largest
size (128x32) at a depth zero and a block of a second size (64x16) at a depth one are
rectangles whose ratios of a long side to a short side are four. Also, each block of a
size at a depth of two or more is a square.
[0 1 291
Fig. 21 is an explanatory diagram showing a fourth example of the
hierarchical structure of a coded block that can be defined by the third embodiment.
In the example of Fig. 21, the definition information 41c indicates that
"max - high-level" is 16, "number-of-layers" is 3, "width-to - height-ratio[]" is
(4,2,1), and "low - level[]" is (16,16,8). In this case, three layer groups with
different ratios may be set according to the block sizes. That is, as shown in Fig. 21,
a block of largest size (64x16) at a depth zero is a rectangle whose ratio of a long
side to a short side is four. A block of a second size (32x16) at a depth one is a
rectangle whose ratio of a long side to a short side is two. Also, each block of a size
at a depth of two or more is a square.
[0130]
Additionally, as can be seen from Figs. 18 to 2 1, in the case of partitioning a
block without changing the shape of the block, the block may be partitioned into four
with the center lines in the vertical direction and the horizontal direction as the
separations, as in the first embodiment. On the other hand, in the case of
partitioning the block so as to shift from one layer group to another layer group with
a different ratio, the block may be partitioned into two or four with a line in the
vertical direction as the separation, for example.
[0131]
Also in the present embodiment, the motion vector setting section 43 of the
motion estimation section 40 may estimate a motion vector, taking every prediction
mode of every layer included in the hierarchical structure as the target.
[0 1321
(Summary of Third Embodiment)
According to the third embodiment described above, the shape of each
block in an image is a square or a rectangle, and the shape determination information
includes number-of-layers information defining the number of layers whose ratios of
5 a long side to a short side of a block are different fiom one another and information
defining the ratio for each layer. By allowing such stepwise definition of a block
shape, blocks of various sizes with various ratios of a long side to a short side may be
handled while supporting the H.2641AVC method according to which the largest
macro block is a square of 16x 16 pixels, for example.
10 [0133]
Furthermore, according to the third embodiment, a plurality of ratios of a
long side to a short side regarding a rectangular block can be specified, and thus, the
flexibility at the time of designing an image processing device while taking into
account the size of a line memory, the compression ratio, the image quality and the
15 like is further increased.
[0 1341
<3. Flow of Process at the Time of Encoding According to an Embodiment>
Next, a flow of a process at the time of encoding will be described using
Figs. 22 and 23. Figs. 22 and 23are flow charts showing an example of a motion
20 estimation process of the motion estimation section 40 of the image encoding device
10 according to the present embodiment.
101351
Referring to Fig. 22, first, the shape determination section 42 acquires the
shape determination information from the information holding section 4 1 (step S 102).
25 The shape determination information acquired here may be "high-square-layer"
illustrated in Fig. 3, "width-to-height-ratio" illustrated in Fig. 12, or
"number-of-layers", "width-to-height-ratio[]" and "low~level[]" illustrated in Fig.
17, for example. The shape determination section 42 also acquires information
such as the largest size information, the smallest size information and the like fiom
30 the information holding section 4 1.
[0136]
Next, the shape determination section 42 sets in an image a block of largest
size that can be set (step S 104).
[0137]
Then, the shape determination section 42 determines, based on the acquired
5 shape determination information and the size of a block that is currently set, whether
or not the shape of the block is a square (step S106). For example, in the case the
definition information 4 la illustrated in Fig. 4 is defined and the size of the block is
128x 128 pixels (N = 64), the shape of the block is determined to be a square. Also,
for example, in the case the definition information 41 a illustrated in Fig. 6 is defined
10 and the size of the block is 64x32 pixels (N = 32), the shape of the block is
determined to be not a square. In the case the shape of the block is determined to be
not a square, the process proceeds to step S108. On the other hand, in the case the
shape of the block is determined to be a square, the process proceeds to step S 112.
[0138]
15 In step S108, the motion vector setting section 43 arranges prediction unit(s)
in the block in each prediction mode for a non-square block, and decides a motion
vector for each arranged prediction unit (step S 108). The prediction modes used
here may be the nine types of prediction modes illustrated in Fig. 8 in a case the
shape of the block is a two-to-one rectangle, and the nine types of prediction modes
20 illustrated in Fig. 14 in a case the shape of the block is a four-to-one rectangle.
Next, the motion vector setting section 43 calculates a cost function value for each
prediction mode (step S 1 10). Additionally, the motion vector decided in step S 108
is a provisional motion vector for selecting the arrangement of prediction unit(s) (that
is, the optimal prediction mode) to be set for generation of a predicted pixel value of
25 the block.
[0139]
In step S 112, the motion vector setting section 43 arranges prediction unit(s)
in the block in each prediction mode for a square block, and decides a motion vector
for each arranged prediction unit (step S112). The prediction modes used here may
30 be the nine types of prediction modes illustrated in Fig. 7, for example. Next, the
motion vector setting section 43 calculates a cost function value for each prediction
mode (step S 1 14). Additionally, the motion vector decided in step S 112 is also a
provisional motion vector for selecting the optimal prediction mode.
[0 1 401
Next, the motion vector setting section 43 determines whether or not the
5 size of the block is the smallest size that can be set (step S116). In the case the size
of the block is not the smallest size here, the process proceeds to step S 11 8. On the
other hand, in the case the size of the block is the smallest size, the process proceeds
to step S 120.
[0141]
10 In step S 11 8, the motion vector setting section 43 partitions the block into a
plurality of blocks of smaller size (step S118). In this case, a partition mode may be
selected as the prediction mode for the layer before partitioning. Then, the process
is repeated from step S 106 to step S 11 6 described above for the block after
partitioning.
15 [0142]
In step S120 of Fig. 23, the motion vector setting section 43 compares the
cost function values calculated for respective prediction modes, and selects the
optimal prediction mode by which the cost function value is the smallest (step S120).
Then, the motion vector setting section 43 outputs, to the motion compensation
20 section 44, prediction mode information indicating the selected optimal prediction
mode, size specification information (for example, the number of times of partition
mode selection representing the number of times of block partitioning),
corresponding motion vector information, and reference image information.
[0 1431
25 Next, the motion compensation section 44 generates a predicted pixel value
according to the optimal prediction mode selected by the motion vector setting
section 43 (step S122). Then, the motion compensation section 44 outputs
predicted image data including the generated predicted pixel value to the mode
selection section 50, and also, outputs the prediction mode information, the size
30 specification information, the motion vector information and the reference image
information to the mode selection section 50 as the information about inter prediction
(step S 124).
[0 1441
<4. Example Configuration of Image Decoding Device According to an
Embodiment>
In this section, an example configuration of an image decoding device
according to an embodiment will be described using Figs. 24 and 25.
[0 1451
[4- 1. Example of Overall Configuration]
Fig. 24 is a block diagram showing an example of a configuration of an
10 image decoding device 60 according to an embodiment. Referring to Fig. 24, the
image decoding device 60 includes an accumulation buffer 61, a lossless decoding
section 62, an inverse quantization section 63, an inverse orthogonal transform
section 64, an addition section 65, a deblocking filter 66, a sorting buffer 67, a D/A
(Digital to Analogue) conversion section 68, a frame memory 69, selectors 70 and 71,
15 an intra prediction section 80, and a motion compensation section 90.
[0 1461
The accumulation buffer 61 temporarily stores an encoded stream input via
a transmission line using a storage medium.
[0 1471
20 The lossless decoding section 62 decodes an encoded stream input from the
accumulation buffer 61 according to the encoding method used at the time of
encoding. Also, the lossless decoding section 62 decodes information multiplexed
to the header region of the encoded stream. Information that is multiplexed to the
header region of the encoded stream may include shape determination information
25 within a sequence parameter set, a picture parameter set or a slice header, for
example. Furthermore, information that is multiplexed to the header region of the
encoded stream may include information about intra prediction and information
about inter prediction in the block header, for example. The lossless decoding
section 62 outputs the information about intra prediction to the intra prediction
30 section 80. Also, the lossless decoding section 62 outputs the shape determination
information and the information about inter prediction to the motion compensation
section 90.
[0 1481
The inverse quantization section 63 inversely quantizes quantized data
which has been decoded by the lossless decoding section 62. The inverse
5 orthogonal transform section 64 generates predicted error data by performing inverse
orthogonal transformation on transform coefficient data input from the inverse
quantization section 63 according to the orthogonal transformation method used at
the time of encoding. Then, the inverse orthogonal transform section 64 outputs the
generated predicted error data to the addition section 65.
10 [0149]
The addition section 65 adds the predicted error data input from the inverse
orthogonal transform section 64 and predicted image data input from the selector 71
to thereby generate decoded image data. Then, the addition section 65 outputs the
generated decoded image data to the deblocking filter 66 and the frame memory 69.
15 [0150]
The deblocking filter 66 removes block distortion by filtering the decoded
image data input from the addition section 65, and outputs the decoded image data
after filtering to the sorting buffer 67 and the frame memory 69.
[0151]
20 The sorting buffer 67 generates a series of image data in a time sequence by
sorting images input from the deblocking filter 66. Then, the sorting buffer 67
outputs the generated image data to the D/A conversion section 68.
[0 1 521
The D/A conversion section 68 converts the image data in a digital format
25 input from the sorting buffer 67 into an image signal in an analogue format. Then,
the D/A conversion section 68 causes an image to be displayed by outputting the
analogue image signal to a display (not shown) connected to the image decoding
device 60, for example.
[0153]
30 The frame memory 69 stores, using a storage medium, the decoded image
data before filtering input from the addition section 65, and the decoded image data
after filtering input from the deblocking filter 66.
[0 1 541
The selector 70 switches the output destination of the image data from the
frame memory 69 between the intra prediction section 80 and the motion
5 compensation section 90 for each block in the image according to mode information
acquired by the lossless decoding section 62. For example, in the case the intra
prediction mode is specified, the selector 70 outputs the decoded image data before
filtering that is supplied from the frame memory 69 to the intra prediction section 80
as reference image data. Also, in the case the inter prediction mode is specified, the
10 selector 70 outputs the decoded image data after filtering that is supplied from the
frame memory 69 to the motion compensation section 90 as the reference image data.
[0155]
The selector 71 switches the output source of predicted image data to be
supplied to the addition section 65 between the intra prediction section 80 and the
15 motion compensation section 90 for each block in the image according to the mode
information acquired by the lossless decoding section 62. For example, in the case
the intra prediction mode is specified, the selector 71 supplies to the addition section
65 the predicted image data output from the intra prediction section 80. In the case
the inter prediction mode is specified, the selector 71 supplies to the addition section
20 65 the predicted image data output from the motion compensation section 90.
[0156]
The intra prediction section 80 performs in-screen prediction of a pixel
value based on the information about intra prediction input from the lossless
decoding section 62 and the reference image data from the frame memory 69, and
25 generates predicted image data. Then, the intra prediction section 80 outputs the
generated predicted image data to the selector 71.
[0 1571
The motion compensation section 90 performs a motion compensation
process based on the information about inter prediction input from the lossless
30 decoding section 62 and the reference image data from the frame memory 69, and
generates predicted image data. Then, the motion compensation section 90 outputs
the generated predicted image data to the selector 71.
[0158]
[4-2. Example Configuration of Motion Compensation Section]
Fig. 25 is a block diagram showing an example of a detailed configuration
5 of the motion compensation section 90 of the image decoding device 60 shown in
Fig. 24. Referring to Fig. 25, the motion compensation section 90 includes a shape
determination section 9 1, a prediction unit determination section 92, and a prediction
section 93.
[0159]
10 The shape determination section 91 determines the shape of each block set
in an image at the time of encoding of the image, using the shape determination and
the size specification information included in the information about inter prediction
input from the lossless encoding section 62. The shape determination information
acquired by the shape determination section 91 may be the information described in
15 the first, second or third embodiment described above, for example. For example,
in the case "high-square-layer"=8 is defined as the shape determination information
and the size of the block specified by the size specification information is 16 in the
first embodiment, the shape determination section 91 may determine that the shape
of the block is a rectangle whose ratio of a long side to a short side is two to one.
20 Also, for example, in the case "width-to-height - ratioV=4 is defined as the shape
determination information and the size of the block specified by the size
specification information is equal to the largest size in the second embodiment, the
shape determination section 91 may determine that the shape of the block is a
rectangle whose ratio of a long side to a short side is four to one. Furthermore, for
25 example, in the case the shape determination information as illustrated in Fig. 21 is
defined and the size of the block specified by the size specification information is
equal to the smallest size in the third embodiment, the shape determination section
91 may determine that the shape of the block is a square. Additionally, the size
specification information may be represented by the number of times of block
30 partitioning from the largest block, as described above.
[0 1601
The prediction unit determination section 92 determines the arrangement of
the prediction unit(s) in each block using the prediction mode information input from
the lossless encoding section 62. Here, the candidates for the arrangement of the
prediction unit(s) for a square block and the candidates for the arrangement of the
prediction unit(s) for a rectangular block are different in the present embodiment, as
illustrated in Figs. 7, 8 and 14. Also, the candidates for the arrangement of the
prediction unit(s) for a rectangular block are different depending on the ratio of a
long side to a short side of the rectangle. Thus, the prediction unit determination
section 92 determines the arrangement of the prediction unit(s) in each block
according to the shape of each block determined by the shape determination section
91. For example, in the case the shape of a block is a square, and the prediction
mode information indicates Mode 2, the arrangement of the prediction units may be
determined to be that of Mode 2 (2NxN) in Fig. 7. Also, for example, in the case
the shape of a block is a two-to-one rectangle, and the prediction mode information
indicates Mode 4, the arrangement of the prediction units may be determined to be
that of Mode 4 (NxN12) in Fig. 8. Furthermore, in the case the shape of a block is a
four-to-one rectangle, and the prediction mode information indicates Mode 3, the
arrangement of the prediction units may be determined to be that of Mode 3 (2NxN)
in Fig. 14.
[0161]
The prediction section 93 generates a predicted pixel value for each block
using the arrangement of the prediction unit(s) determined by the prediction unit
determination section 92, a motion vector for each prediction unit and reference
image information input from the lossless encoding section 62, and reference image
data input from a frame memory. Then, the prediction section 93 outputs the
predicted image data including the generated predicted pixel value to the selector 7 1.
[0 1 621
<5. Flow of Process at the Time of Decoding According to an Embodiment>
Next, a flow of a process at the time of decoding will be described using Fig.
26. Fig. 26 is a flow chart showing an example of a flow of a motion compensation
process of the motion compensation section 90 of the image decoding device 60
according to the present embodiment.
[0 1631
Referring to Fig. 26, first, the shape determination section 91 acquires the
shape determination information and the size specification information included in
5 the information about inter prediction input from the lossless decoding section 62
(step S202). The shape determination information acquired here may be
"high-square - layer" illustrated in Fig. 3, "width - to - height - ratio" illustrated in Fig.
12, or "number-of-layers", "width-to-height-ratio[]" and "low - level[]" illustrated
in Fig. 17, for example.
10 [0164]
Next, the shape determination section 91 determines the size of a block set
at the time of encoding of the image, using the acquired size specification
information (step S204). The size specification information may be acquired as a
partition mode indicating the number of times of block partitioning from the largest
15 block, for example. Such a partition mode may be a part of the prediction mode
information. In this case, the size of a block set at the time of encoding of the
image may be determined based on the largest block size and the number of times of
block partitioning.
[0 1651
20 Then, the shape determination section 91 determines whether or not the
shape of a block set at the time of encoding of the image is a square based on the
acquired shape determination information and the block size (step S206). The
example of determination of a shape by the shape determination section 91 is the
same as the example described above in relation to step S 106 in Fig. 22. In the case
25 the shape of the block is a non-square here, the process proceeds to step S208. On
the other hand, in the case the shape of the block is a square, the process proceeds to
step S210.
[0 1661
In step S208, the prediction unit determination section 92 recognizes the
30 prediction mode indicated by the prediction mode information input from the lossless
encoding section 62 as the prediction mode for a non-square, and determines the
arrangement of the prediction unit(s) for the block in question (step S208). Also, in
step S2 10, the prediction unit determination section 92 recognizes the prediction
mode indicated by the prediction mode information input from the lossless encoding
section 62 as the prediction mode for a square, and determines the arrangement of the
5 prediction unit(s) for the block in question (step S2 10).
[0 1671
Next, the prediction section 93 generates a predicted pixel value for a pixel
in each block based on the arrangement of the prediction unit(s) determined by the
prediction unit determination section 92, using the motion vector for each prediction
10 unit and reference image data (step S212). Then, the prediction section 93 outputs
predicted image data including the generated predicted pixel value to the selector 71
(step S214).
[O 1681
<6. Example Application>
The image encoding device 10 and the image decoding device 60 according
to the embodiment described above may be applied to various electronic appliances
such as a transmitter and a receiver for satellite broadcasting, cable broadcasting such
as cable TV, distribution on the Internet, distribution to terminals via cellular
communication, and the like, a recording device that records images in a medium
20 such as an optical disc, a magnetic disk or a flash memory, a reproduction device that
reproduces images from such storage medium, and the like. Four example
applications will be described below.
[0 1691
[6- 1. First Example Application]
25 Fig. 27 is a block diagram showing an example of a schematic configuration
of a television adopting the embodiment described above. A television 900 includes
an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, an video signal
processing section 905, a display section 906, an audio signal processing section 907,
a speaker 908, an external interface 909, a control section 910, a user interface 91 1,
30 andabus912.
[0 1701
The tuner 902 extracts a signal of a desired channel from broadcast signals
received via the antenna 901, and demodulates the extracted signal. Then, the tuner
902 outputs an encoded bit stream obtained by demodulation to the demultiplexer
903. That is, the tuner 902 serves as transmission means of the televisions 900 for
5 receiving an encoded stream in which an image is encoded and the shape
determination information described above.
[0171]
The demultiplexer 903 separates a video stream and an audio stream of a
program to be viewed from the encoded bit stream, and outputs each stream which
10 has been separated to the decoder 904. Also, the demultiplexer 903 extracts
auxiliary data such as an EPG (Electronic Program Guide) from the encoded bit
stream, and supplies the extracted data to the control section 910. Additionally, the
demultiplexer 903 may perform descrambling in the case the encoded bit stream is
scrambled.
15 [0172]
The decoder 904 decodes the video stream and the audio stream input from
the demultiplexer 903. Then, the decoder 904 outputs video data generated by the
decoding process to the video signal processing section 905. Also, the decoder 904
outputs the audio data generated by the decoding process to the audio signal
20 processing section 907.
[0 1731
The video signal processing section 905 reproduces the video data input
from the decoder 904, and causes the display section 906 to display the video. The
video signal processing section 905 may also cause the display section 906 to display
25 an application screen supplied via a network. Further, the video signal processing
section 905 may perform an additional process such as noise removal, for example,
on the video data according to the setting. Furthermore, the video signal processing
section 905 may generate an image of a GUI (Graphical User Interface) such as a
menu, a button, a cursor or the like, for example, and superimpose the generated
30 image on an output image.
[0 1 741
The display section 906 is driven by a drive signal supplied by the video
signal processing section 905, and displays a video or an image on an video screen of
a display device (for example, a liquid crystal display, a plasma display, an OLED, or
the like).
5 [0175]
The audio signal processing section 907 performs reproduction processes
such as DIA conversion and amplification on the audio data input from the decoder
904, and outputs audio fiom the speaker 908. Also, the audio signal processing
section 907 may perform an additional process such as noise removal on the audio
10 data.
[0 1 761
The external interface 909 is an interface for connecting the television 900
and an external appliance or a network. For example, a video stream or an audio
stream received via the external interface 909 may be decoded by the decoder 904.
15 That is, the external interface 909 also serves as transmission means of the
televisions 900 for receiving an encoded stream in which an image is encoded and
the shape determination information described above.
[O 1 771
The control section 910 includes a processor such as a CPU (Central
20 Processing Unit), and a memory such as an RAM (Random Access Memory), an
ROM (Read Only Memory), or the like. The memory stores a program to be
executed by the CPU, program data, EPG data, data acquired via a network, and the
like. The program stored in the memory is read and executed by the CPU at the
time of activation of the television 900, for example. The CPU controls the
25 operation of the television 900 according to an operation signal input fiom the user
interface 91 1, for example, by executing the program.
[0 1 781
The user interface 911 is connected to the control section 910. The user
interface 91 1 includes a button and a switch used by a user to operate the television
30 900, and a receiving section for a remote control signal, for example. The user
interface 91 1 detects an operation of a user via these structural elements, generates an
operation signal, and outputs the generated operation signal to the control section 910.
[0 1 791
The bus 912 interconnects the tuner 902, the demultiplexer 903, the decoder
904, the video signal processing section 905, the audio signal processing section 907,
5 the external interface 909, and the control section 91 0.
[O 1 801
In the television 900 configured in this manner, the decoder 904 has a
hction of the image decoding device 60 according to the embodiment described
above. Accordingly, in the television 900, various effects can be acquired, such as
10 the increase in the compression ratio due to the extension of a block size for motion
prediction, while suppressing the increase in the line memory.
[0181]
[6-2. Second Example Application]
Fig. 28 is a block diagram showing an example of a schematic configuration
15 of a mobile phone adopting the embodiment described above. A mobile phone 920
includes an antenna 921, a communication section 922, an audio codec 923, a
speaker 924, a microphone 925, a camera section 926, an image processing section
927, a demultiplexing section 928, a recording/reproduction section 929, a display
section 930, a control section 931, an operation section 932, and a bus 933.
20 [0182]
The antenna 921 is connected to the communication section 922. The
speaker 924 and the microphone 925 are connected to the audio codec 923. The
operation section 932 is connected to the control section 931. The bus 933
interconnects the communication section 922, the audio codec 923, the camera
25 section 926, the image processing section 927, the demultiplexing section 928, the
recording/reproduction section 929, the display section 930, and the control section
93 1.
[0183]
The mobile phone 920 performs operation such as transmissiodreception of
30 audio signal, transmissiodreception of emails or image data, image capturing,
recording of data, and the like, in various operation modes including an audio
communication mode, a data communication mode, an image capturing mode, and a
videophone mode.
[0 1 841
In the audio communication mode, an analogue audio signal generated by
5 the microphone 925 is supplied to the audio codec 923. The audio codec 923
converts the analogue audio signal into audio data, and A/D converts and compresses
the converted audio data. Then, the audio codec 923 outputs the compressed audio
data to the communication section 922. The communication section 922 encodes
and modulates the audio data, and generates a transmission signal. Then, the
10 communication section 922 transmits the generated transmission signal to a base
station (not shown) via the antenna 921. Also, the communication section 922
amplifies a wireless signal received via the antenna 921 and converts the frequency
of the wireless signal, and acquires a received signal. Then, the communication
section 922 demodulates and decodes the received signal and generates audio data,
15 and outputs the generated audio data to the audio codec 923. The audio codec 923
extends and D/A converts the audio data, and generates an analogue audio signal.
Then, the audio codec 923 supplies the generated audio signal to the speaker 924 and
causes the audio to be output.
[0185]
20 Also, in the data communication mode, the control section 931 generates
text data that makes up an email, according to an operation of a user via the operation
section 932, for example. Moreover, the control section 931 causes the text to be
displayed on the display section 930. Furthermore, the control section 931
generates email data according to a transmission instruction of the user via the
25 operation section 932, and outputs the generated email data to the communication
section 922. Then, the communication section 922 encodes and modulates the
email data, and generates a transmission signal. Then, the communication section
922 transmits the generated transmission signal to a base station (not shown) via the
antenna 921. Also, the communication section 922 amplifies a wireless signal
30 received via the antenna 921 and converts the frequency of the wireless signal, and
acquires a received signal. Then, the communication section 922 demodulates and
decodes the received signal, restores the email data, and outputs the restored email
data to the control section 93 1. The control section 93 1 causes the display section
930 to display the contents of the email, and also, causes the email data to be stored
in the storage medium of the recording/reproduction section 929.
5 [0186]
The recording/reproduction section 929 includes an arbitrary readable and
writable storage medium. For example, the storage medium may be a built-in
storage medium such as an RAM, a flash memory or the like, or an externally
mounted storage medium such as a hard disk, a magnetic disk, a magneto-optical
10 disk, an optical disc, an USB memory, a memory card, or the like.
[0 1 871
Furthermore, in the image capturing mode, the camera section 926 captures
an image of a subject, generates image data, and outputs the generated image data to
the image processing section 927, for example. The image processing section 927
15 encodes the image data input from the camera section 926, and causes the encoded
stream to be stored in the storage medium of the recording/reproduction section 929.
[0188]
Furthermore, in the videophone mode, the demultiplexing section 928
multiplexes a video stream encoded by the image processing section 927 and an
20 audio stream input from the audio codec 923, and outputs the multiplexed stream to
the communication section 922, for example. The communication section 922
encodes and modulates the stream, and generates a transmission signal. Then, the
communication section 922 transmits the generated transmission signal to a base
station (not shown) via the antenna 921. Also, the communication section 922
25 amplifies a wireless signal received via the antenna 921 and converts the frequency
of the wireless signal, and acquires a received signal. These transmission signal and
received signal may include an encoded bit stream and the shape determination
information described above. Then, the communication section 922 demodulates
and decodes the received signal, restores the stream, and outputs the restored stream
30 to the demultiplexing section 928. The demultiplexing section 928 separates a
video stream and an audio stream from the input stream, and outputs the video
stream to the image processing section 927 and the audio stream to the audio codec
923. The image processing section 927 decodes the video stream, and generates
video data. The video data is supplied to the display section 930, and a series of
images is displayed by the display section 930. The audio codec 923 extends and
5 D/A converts the audio stream, and generates an analogue audio signal. Then, the
audio codec 923 supplies the generated audio signal to the speaker 924 and causes
the audio to be output.
[0 1 891
In the mobile phone 920 configured in this manner, the image processing
10 section 927 has a function of the image encoding device 10 and the image decoding
device 60 according to the embodiment described above. Accordingly, in the
mobile phone 920, various effects can be acquired, such as the increase in the
compression ratio due to the extension of a block size for motion prediction, while
suppressing the increase in the line memory.
15 [0190]
[6-3. Third Example Application]
Fig. 29 is a block diagram showing an example of a schematic configuration
of a recording/reproduction device adopting the embodiment described above. A
recording/reproduction device 940 encodes, and records in a recording medium,
20 audio data and video data of a received broadcast program, for example. The
recording/reproduction device 940 may also encode, and record in the recording
medium, audio data and video data acquired from another device, for example.
Furthermore, the recording/reproduction device 940 reproduces, using a monitor or a
speaker, data recorded in the recording medium, according to an instruction of a user,
25 for example. At this time, the recording/reproduction device 940 decodes the audio
data and the video data.
[0191]
The recording/reproduction device 940 includes a tuner 941, an external
interface 942, an encoder 943, an HDD (Hard Disk Drive) 944, a disc drive 945, a
30 selector 946, a decoder 947, an OSD (On-Screen Display) 948, a control section 949,
and a user interface 950.
[0 1 921
The tuner 941 extracts a signal of a desired channel from broadcast signals
received via an antenna (not shown), and demodulates the extracted signal. Then,
the tuner 941 outputs an encoded bit stream obtained by demodulation to the selector
5 946. That is, the tuner 941 serves as transmission means of the
recording/reproduction device 940.
[0 1931
The external interface 942 is an interface for connecting the
recording/reproduction device 940 and an external appliance or a network. For
10 example, the external interface 942 may be an IEEE 1394 interface, a network
interface, an USB interface, a flash memory interface, or the like. For example,
video data and audio data received by the external interface 942 are input to the
encoder 943. That is, the external interface 942 serves as transmission means of the
recording/reproduction device 940.
15 [0194]
In the case the video data and the audio data input from the external
interface 942 are not encoded, the encoder 943 encodes the video data and the audio
data. Then, the encoder 943 outputs the encoded bit stream to the selector 946.
[0 1951
20 The HDD 944 records in an internal hard disk an encoded bit stream, which
is compressed content data of a video or audio, various programs, and other pieces of
data. Also, the HDD 944 reads these pieces of data from the hard disk at the time
of reproducing a video or audio.
[0 1961
2 5 The disc drive 945 records or reads data in a recording medium that is
mounted. A recording medium that is mounted on the disc drive 945 may be a
DVD disc (a DVD-Video, a DVD-RAM, a DVD-R, a DVD-RW, a DVD+, a
DVD+RW, or the like), a Blu-ray (registered trademark) disc, or the like, for example.
[0 1971
30 The selector 946 selects, at the time of recording a video or audio, an
encoded bit stream input from the tuner 941 or the encoder 943, and outputs the
selected encoded bit stream to the HDD 944 or the disc drive 945. Also, the
selector 946 outputs, at the time of reproducing a video or audio, an encoded bit
stream input from the HDD 944 or the disc drive 945 to the decoder 947.
[0 1981
5 The decoder 947 decodes the encoded bit stream, and generates video data
and audio data. Then, the decoder 947 outputs the generated video data to the OSD
948. Also, the decoder 904 outputs the generated audio data to an external speaker.
[0 1 991
The OSD 948 reproduces the video data input from the decoder 947, and
10 displays a video. Also, the OSD 948 may superimpose an image of a GUI, such as
a menu, a button, a cursor or the like, for example, on a displayed video.
[0200]
The control section 949 includes a processor such as a CPU, and a memory
such as an RAM or an ROM. The memory stores a program to be executed by the
15 CPU, program data, and the like. A program stored in the memory is read and
executed by the CPU at the time of activation of the recording/reproduction device
940, for example. The CPU controls the operation of the recording/reproduction
device 940 according to an operation signal input from the user interface 950, for
example, by executing the program.
20 [0201]
The user interface 950 is connected to the control section 949. The user
interface 950 includes a button and a switch used by a user to operate the
recording/reproduction device 940, and a receiving section for a remote control
signal, for example. The user interface 950 detects an operation of a user via these
25 structural elements, generates an operation signal, and outputs the generated
operation signal to the control section 949.
[0202]
In the recording/reproduction device 940 configured in this manner, the
encoder 943 has a function of the image encoding device 10 according to the
30 embodiment described above. Also, the decoder 947 has a function of the image
decoding device 60 according to the embodiment described above. Accordingly, in
the recording/reproduction device 940, various effects can be acquired, such as the
increase in the compression ratio due to the extension of a block size for motion
prediction, while suppressing the increase in the line memory.
[0203]
5 [6-4. Fourth Example Application]
Fig. 30 is a block diagram showing an example of a schematic configuration
of an image capturing device adopting the embodiment described above. An image
capturing device 960 captures an image of a subject, generates an image, encodes the
image data, and records the image data in a recording medium.
10 [0204]
The image capturing device 960 includes an optical block 961, an image
capturing section 962, a signal processing section 963, an image processing section
964, a display section 965, an external interface 966, a memory 967, a media drive
968, an OSD 969, a control section 970, a user interface 971, and a bus 972.
15 [0205]
The optical block 961 is connected to the image capturing section 962.
The image capturing section 962 is connected to the signal processing section 963.
The display section 965 is connected to the image processing section 964. The user
interface 971 is connected to the control section 970. The bus 972 interconnects the
20 image processing section 964, the external interface 966, the memory 967, the media
drive 968, the OSD 969, and the control section 970.
[0206]
The optical block 961 includes a focus lens, an aperture stop mechanism,
and the like. The optical block 961 forms an optical image of a subject on an image
25 capturing surface of the image capturing section 962. The image capturing section
962 includes an image sensor such as a CCD, a CMOS or the like, and converts by
photoelectric conversion the optical image formed on the image capturing surface
into an image signal which is an electrical signal. Then, the image capturing
section 962 outputs the image signal to the signal processing section 963.
30 [0207]
The signal processing section 963 performs various camera signal processes,
such as knee correction, gamma correction, color correction and the like, on the
image signal input from the image capturing section 962. The signal processing
section 963 outputs the image data after the camera signal process to the image
processing section 964.
5 [0208]
The image processing section 964 encodes the image data input from the
signal processing section 963, and generates encoded data. Then, the image
processing section 964 outputs the generated encoded data to the external interface
966 or the media drive 968. Also, the image processing section 964 decodes
10 encoded data input from the external interface 966 or the media drive 968, and
generates image data. Then, the image processing section 964 outputs the
generated image data to the display section 965. Also, the image processing section
964 may output the image data input from the signal processing section 963 to the
display section 965, and cause the image to be displayed. Furthermore, the image
15 processing section 964 may superimpose data for display acquired from the OSD 969
on an image to be output to the display section 965.
[0209]
The OSD 969 generates an image of a GUI, such as a menu, a button, a
cursor or the like, for example, and outputs the generated image to the image
20 processing section 964.
[02 lo]
The external interface 966 is configured as an USB inputloutput terminal,
for example. The external interface 966 connects the image capturing device 960
and a printer at the time of printing an image, for example. Also, a drive is
25 connected to the external interface 966 as necessary. A removable medium, such as
a magnetic disk, an optical disc or the like, for example, is mounted on the drive, and
a program read from the removable medium may be installed in the image capturing
device 960. Furthermore, the external interface 966 may be configured as a
network interface to be connected to a network such as a LAN, the Internet or the
30 like. That is, the external interface 966 serves as transmission means of the image
capturing device 960.
[0211]
A recording medium to be mounted on the media drive 968 may be an
arbitrary readable and writable removable medium, such as a magnetic disk, a
magneto-optical disk, an optical disc, a semiconductor memory or the like, for
5 example. Also, a recording medium may be fixedly mounted on the media drive
968, configuring a non-transportable storage section such as a built-in hard disk drive
or an SSD (Solid State Drive), for example.
[02 121
The control section 970 includes a processor such as a CPU, and a memory
10 such as an RAM or an ROM. The memory stores a program to be executed by the
CPU, program data, and the like. A program stored in the memory is read and
executed by the CPU at the time of activation of the image capturing device 960, for
example. The CPU controls the operation of the image capturing device 960
according to an operation signal input from the user interface 971, for example, by
15 executing the program.
[02 131
The user interface 971 is connected to the control section 970. The user
interface 971 includes a button, a switch and the like used by a user to operate the
image capturing device 960, for example. The user interface 971 detects an
20 operation of a user via these structural elements, generates an operation signal, and
outputs the generated operation signal to the control section 970.
[02 141
In the image capturing device 960 configured in this manner, the image
processing section 964 has a function of the image encoding device 10 and the image
25 decoding device 60 according to the embodiment described above. Thus, with the
image capturing device 960, various effects can be acquired, such as the increase in
the compression ratio due to the extension of a block size for motion prediction,
while suppressing the increase in the line memory.
[02 151
30 <7. Summary>
Heretofore, the image encoding device 10 and the image decoding device 60
according to an embodiment has been described using Figs. 1 to 30. According to
the present embodiment, when encoding an image, the shape of a block set in the
image is determined using the shape determination information for determining the
shape of a block according to the size, and a motion vector to be used for prediction
5 of an image in each prediction unit is set for each prediction unit arranged according
to the shape. Also, when decoding an image, the shape of each block in an image is
determined using the shape determination information and the block size, and a
predicted image for each block is generated based on the arrangement of prediction
unit(s) according to the shape and a motion vector. Therefore, a block of a shape
10 other than a square can be used for motion prediction. The increase in a line
memory can thereby be suppressed even in a case of extending the block size for
motion prediction to increase the compression ratio, for example.
[02 1 61
Also, according to the present embodiment, the shape of a largest block that
15 can be set in an image can be made a non-square, and at the same time, the shape of a
smaller block can be made a square. Thus, the encoding method proposed in the
present specification can be realized while maintaining the compatibility with an
existing encoding method according to which a largest block is a square.
[02 1 71
20 Furthermore, according to the present embodiment, the shape determination
information can be input by a user. The shape determination information input by a
user is multiplexed to a sequence parameter set, a picture parameter set or a slice
header of an encoded stream, and is transmitted from the encoding side to the
decoding side. The shape of a block can thereby be flexibly defined by a user using
25 the encoding method proposed in the present specification according to various needs
regarding the size of a line memory, the compression ratio, and the image quality,
and the definition can be used by both the encoding side and the decoding side.
[02 1 81
Still further, according to the present embodiment, various prediction modes
30 (candidates for arrangement of prediction units) are provided also for a non-square
block. Thus, motion prediction can be effectively performed according to the
contents of an image also in the case of using a block of a shape other than a square.
Additionally, in the present specification, an example has been mainly
described where the shape determination information is multiplexed within a
5 sequence parameter set, a picture parameter set or a slice header, for example, of an
encoded stream, and the encoded stream is transmitted fiom the encoding side to the
decoding side. However, the method of transmitting the shape determination
information is not limited to such an example. For example, the shape
determination information may be transmitted or recorded as individual data that is
10 associated with an encoded bit stream, without being multiplexed to the encoded bit
stream. The term "associate" here means to enable an image included in a bit
stream (or a part of an image, such as a slice or a block) and information
corresponding to the image to link to each other at the time of decoding. That is,
the shape determination information may be transmitted on a different transmission
15 line fiom the image (or the bit stream). Or, the shape determination information
may be recorded on a different recording medium (or in a different recording area on
the same recording medium) from the image (or the bit stream). Furthermore, the
shape determination information and the image (or the bit stream) may be associated
with each other on the basis of arbitrary units such as a plurality of frames, one frame,
20 a part of a frame or the like, for example.
Heretofore, a preferred embodiment of the present disclosure has been
described in detail while referring to the appended drawings, but the technical scope
of the present disclosure is not limited to such an example. It is apparent that a
25 person having an ordinary skill in the art of the technology of the present disclosure
may make various alterations or modifications within the scope of the technical ideas
described in the claims, and these are, of course, understood to be within the
technical scope of the present disclosure.
Reference Signs List
Image encoding device (Image processing device)
Encoding section
Shape determination section
Motion vector setting section
Image decoding device (Image processing device)
Decoding section
Shape determination section
Prediction unit determination section
Prediction section

SP305849WO00
62/68
CLAIMS
Claim 1
An image processing device comprising:
a shape determination section for determining a shape of a block set in an
5 image using shape determination information for determining a shape of a block that
is in accordance with a size;
a motion vector setting section for arranging a prediction unit in each block
in the image according to the shape determined by the shape determination section,
and setting, for each arranged prediction unit, a motion vector to be used for
10 prediction of an image in the prediction vmit; and
an encoding section for encoding the image using the motion vector set by
the motion vector setting section.
Claim 2
15 The image processing device according to claim 1, wherein the shape
determination information includes information about a boundary value for a block
size that separates a square block and a non-squaf e block.
Claim 3
20 The image processing device according to claim 2, wherein the shape
determination section determines that a shape of a block of a larger size than the
boimdary value is a non-square, and that a shape of a block of a smaller size than the
boundary value is a square.
25 Claim 4
The image processing device according to claim 3, wherein a shape of each
block in the image is a square or a rectangle with a fixed ratio of a long side to a
short side.
30 Claim 5
The image processing device according to claim 1,
SP305849WO00
63/68
wherein a shape of each block in the image is a square or a rectangle, and
wherein the shape determination information includes ratio information
defining a ratio of a long side to a short side of a block.
5 Claim 6
The image processing device according to claim 5,
wherein the shape determination information includes number-of-layers
information defining the number of layers whose ratios of a long side to a short side
of a block are different from one another, and
10 wherein the ratio information is information defining the ratio for each of
the layers.
Claim 7
The image processing device according to claim 1, wherein the motion
15 vector setting section outputs, for each block, information indicating a size of each
block set in the image and prediction mode information specifying an arrangement
pattern of a prediction unit in the block.
Claim 8
20 The image processing device according to claim 1,
wherein a shape of each block in the image is a square or a rectangle, and
wherein candidates for arrangement of a prediction unit for a square block
and candidates for arrangement of a prediction unit for a rectangular block are
different.
25
Claim 9
The image processing device according to claim 1, wherein the shape
determination information is allowed to be input by a user.
30 Claim 10
The image processing device according to claim 1, further comprising
SP305849WO00
64/68
transmission means for transmitting an encoded stream generated by the
encoding section and the shape determination information.
Claim 11
5 The image processing device according to claim 10, wherein the encoding
section multiplexes the shape determination information to a sequence parameter set,
a picture parameter set or a slice header of the encoded stream.
Claim 12
10 An image processing method for processing an image, comprising:
determining a shape of a block set in an image using shape determination
information for determining a shape of a block that is in accordance with a size;
arranging a prediction unit in each block in the image according to the
determined shape, and setting, for each arranged prediction unit, a motion vector to
15 be used for prediction of an image in the prediction unit; and
encoding the image using the set motion vector.
Claim 13
An image processing device comprising:
20 a shape determination section for determining a shape of each block in an
image using shape determination information for determining a shape of a block that
is in accordance with a size and a size of a block set in the image at a time of
encoding of the image;
a prediction unit determination section for determining an arrangement of a
25 prediction unit in each block according to the shape of each block determined by the
shape determination section;
a prediction section for generating a predicted image for each block using
the arrangement of a prediction unit determined by the prediction unit determination
section and a motion vector acquired for each prediction unit; and
30 a decoding section for decoding an encoded stream in which the image is
encoded, using the predicted image generated by the prediction section.
SP305849WO00
65/68
*
Claim 14
The image processing device according to claim 13, wherein the shape
determination information includes information about a boundary value for a block
5 size that separates a square block and a non-square block.
Claim 15
The image processing device according to claim 14, wherein a shape of each
block in the image is a square or a rectangle with a fixed ratio of a long side to a
10 short side.
Claim 16
The image processing device according to claim 13,
wherein a shape of each block in the image is a square or a rectangle, and
15 wherein the shape determination information includes ratio information
defining a ratio of a long side to a short side of a block.
Claim 17
The image processing device according to claim 16,
20 wherein the shape determination information includes number-of-layers
information defining the number of layers whose ratios of a long side to a short side
of a block are different from one another, and
wherein the ratio information is information defining the ratio for each of
the layers.
25
Claim 18
The image processing device according to claim 13, wherein the shape
determination section acquires the shape determination information from a sequence
parameter set, a picture parameter set or a slice header multiplexed to the encoded
30 stream.
SP305849WO00
66/68
Claim 19
The image processing device according to claim 13, wherein the shape
determination section acquires information for determining a size of each block from
a block header included in the encoded stream.
5
Claim 20
The image processing device according to claim 13,
wherein a shape of each block in the image is a square or a rectangle, and
wherein candidates for arrangement of a prediction unit for a square block
10 and candidates for arrangement of a prediction unit for a rectangular block are
different.
Claim 21
The image processing device according to claim 13, wherein the shape
15 determination information is information acquired based on a user input at a device
that has encoded the image.
Claim 22
The image processing device according to claim 13, fiirther comprising
20 receiving means for receiving the encoded stream and the shape
determination information.
Claim 23
An image processing method for processing an image, comprising:
25 determining a shape of each block in an image using shape determination
information for determining a shape of a block that is in accordance with a size and a
size of a block set in the image at a time of encoding of the image;
determining an arrangement of a prediction unit in each block according to
the determined shape of each block;
30 generating a predicted image for each block using the determined
arrangement of a prediction unit and a motion vector acquired for each prediction
, SP305849WO00
^ 67/68 ^ :,
unit; arid
decoding -an encoded stream in which the image is encoded, using the
generated predicted image.
Dated this 02.01.2013 \M)i^^f''^^l^^
[NEMXSRIVASTAVA]
OF REMFRY & SAGAR
ATTORNEY FOR THE APPLICANT[S]
I
!