DESCRIPTION ENCODING DEVICE, ENCODING METHOD, DECODING DEVICE, AND
DECODING METHOD
5 TECHNICAL FIELD [0001]
The present technology relates to an encoding device, an encoding method, a decoding device, and a decoding method, and more particularly, to an encoding 10 device, an encoding method, a decoding device, and a decoding method capable of reducing the amount of calculation of orthogonal transform processing or inverse orthogonal transform processing.
15 BACKGROUND ART
[0002]
In an advanced video coding (AVC) mode, a mode
dependent directional transform (MDDT) that improves
intra prediction encoding is proposed (for example, see 20 Patent Documents 1 and 2). The MDDT is a technology for
improving the encoding efficiency by using a Karhunen-
Loeve transform (KLT) in accordance with an intra
prediction mode instead of using a conventional discrete
cosine transform (DOT) when an orthogonal transformation 25 is performed for a residual error between an intra-
predicted image and an input image.
[0003]
The KLT is such a transform that coefficients after
the transform have no correlation and is a transform in 30 which the dispersion of a first principal component is a
maximum. More bits are assigned to a large coefficient
1

H SP305119WO00
of dispersion that is acquired by such a transform, and fewer bits are assigned to a small coefficient of the dispersion, whereby the encoding efficiency can be improved. 5 [0004]
Further, as described in Patent Document 1, the residual error between an intra-predicted image and an input image depends on an intra prediction mode. Accordingly, in the MDDT, bases of the KLT are learned in
10 accordance with an intra prediction mode in advance, and, by using a base of the KLT that differs in accordance with each intra prediction mode, the residual can be efficiently transformed. [0005]
15 However, the KLT has a calculation amount more than the DCT. Thus, in Patent Document 2, it is proposed to decrease the number of times of multiplication by performing not a non-separable type KLT that has generality but a separable-type KLT having a relatively
20 small amount of calculation for the residual between an intra-predicted image and an input-image.
CITATION LIST
NON-PATENT DOCUMENTS 25 [0006]
Non-Patent Document 1: Marta Karczewicz, "Improved Intra
Coding", VCEG (Video Coding Experts Group) -AF15, USA,
April, 2007
Non-Patent Document 2: Yan Ye, Marta Karczewicz, 30 "Improved Intra Coding", VCEG (Video Coding Experts
Group) -AGllrl, China, October, 2007
2

4^ SP305119WO00
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION [0007] 5 However, as disclosed in Patent Document 2, even in a case where the separable-type KLT is performed, when the block size is large, the amount of calculation is large. More specifically, when the block size is N x N pixels, the order of the amount of calculation of the DCT
10 using a high-speed algorithm is Nlog|N|, and the order of the amount of calculation of the separable-type KLT is N^. Accordingly, as the block size becomes larger, the amount of calculation of the separable-type KLT becomes larger than the amount of calculation of the DCT using a high-
15 speed algorithm. [0008]
As a result, in a case where the separable-type KLT is realized by hardware, the circuit scale of the large scale integration (LSI) increases, and accordingly, the
20 manufacturing cost increases. On the other hand, in a
case where the separable-type KLT is realized by software, the amount of processing increases, and it is difficult to reproduce an image in real time. [0009]
25 The present technology is contrived in view of such situations and enables a decrease in the amount of calculation in orthogonal transform processing or inverse orthogonal transform processing.
30 SOLUTIONS TO PROBLEMS [0010]
3

|g SP305119WO00
A first aspect of the present technology is an encoding device including: a prior orthogonal transformation unit that performs a second orthogonal transform of which a calculation amount is smaller than a 5 calculation amount of a first orthogonal transform for an image; a posterior orthogonal transformation unit that performs the first orthogonal transform for a low-frequency component of the image that is acquired as a result of the second orthogonal transform; and an
10 encoding unit that encodes the low-frequency component
after the first orthogonal transform and a high-frequency component of the image that is acquired as a result of the second orthogonal transform. [0011]
15 An encoding method according to the first aspect of the present technology corresponds to the encoding device according to the first aspect of the present technology. [0012]
According to the first aspect of the present
20 technology, a second orthogonal transform of which the calculation amount is smaller than that of a first orthogonal transform is performed for an image, he first orthogonal transform is performed for a low-frequency component of the image that is acquired as a result of
25 the second orthogonal transform, and the low-frequency component after the first orthogonal transform and a high-frequency component of the image that is acquired as a result of the second orthogonal transform are encoded. [0013]
30 A second aspect of the present technology is a
decoding device including: a decoding unit that acquires
4

UL SP305119WO00
a coefficient of a low-frequency component of an image after a first orthogonal transform that is acquired as a result of performing a second orthogonal transform of which a calculation amount is smaller than a calculation 5 amount of the first orthogonal transform as a result of encoding the image and a result of encoding a high-frequency component of the image that is acquired as a result of performing the second orthogonal transform and decodes a result of encoding of the image; a prior
10 inverse orthogonal transformation unit that performs a first inverse orthogonal transform that corresponds to the first orthogonal transform for the coefficient of the low-frequency component after the first orthogonal transform that is acquired as a result of the decoding;
15 and a posterior inverse orthogonal transformation unit that acquires the image by performing a second inverse orthogonal transform that corresponds to the second orthogonal transform for the low-frequency component that is acquired as a result of the first inverse orthogonal
2 0 transform and the high-frequency component that is acquired as a result of the decoding. [0014]
A decoding method according to the second aspect of the present technology corresponds to the decoding device
25 according to the second aspect of the present technology. [0015]
According to the second aspect of the present technology, a coefficient of a low-frequency component of an image after a first orthogonal transform that is
30 acquired as a result of performing a second orthogonal
transform of which a calculation amount is smaller than a
5

^ SP305119WO00
calculation amount of the first orthogonal transform as a result of encoding the image and a result of encoding a high-frequency component of the image that is acquired as a result of performing the second orthogonal transform 5 are acquired, a result of encoding of the image is decoded, a first inverse orthogonal transform that corresponds to the first orthogonal transform is performed for the coefficient of the low-frequency component after the first orthogonal transform that is
10 acquired as a result of decoding, and a second inverse orthogonal transform that corresponds to the second orthogonal transform is performed for the low-frequency component that is acquired as a result of the first inverse orthogonal transform and the high-frequency
15 component that is acquired as a result of the decoding, whereby the image can be acquired. [0016]
The encoding device according to the first aspect and the decoding device according to the second aspect
20 may be independent devices or may be internal blocks that configure one device.
EFFECTS OF THE INVENTION
[0017] 25 According to the first aspect of the present
technology, the amount of calculation in orthogonal
transform processing can be reduced.
[0018]
In addition, according to the second aspect of the 30 present technology, the amount of calculation in inverse
orthogonal transform processing can be reduced.
6

SP305119WO00
BRIEF DESCRIPTION OF DRAWINGS [0019]
Fig. 1 is a block diagram that illustrates a 5 configuration example of an image encoding device
according to a first embodiment of the present technology.
Fig. 2 is a diagram that illustrates an example of a predicted direction of each intra prediction mode. Fig. 3 is a block diagram that illustrates an 10 example of a detailed configuration of an intra prediction orthogonal transformation unit that is illustrated in Fig. 1.
■Fig. 4 is a diagram that illustrates a DWT performed by a DWT unit. 15 Fig. 5 is a block diagram that illustrates a
detailed configuration example of the intra prediction inverse orthogonal transformation unit that is illustrated in Fig. 1.
Fig. 6 is a diagram that illustrates the sequence 20 of lossless encoding that is performed by the lossless encoding unit illustrated in Fig. 1.
Fig. 7 is a flowchart that illustrates an encoding process of the image encoding device illustrated in Fig. 1. 25 Fig. 8 is a flowchart that illustrates orthogonal transform processing.
Fig. 9 is a flowchart that illustrates inverse orthogonal transform processing.
Fig. 10 is a flowchart that illustrates an intra-30 prediction cost function value calculating process. Fig. 11 is a flowchart that illustrates a cost
7

^ SP305119WO00
function value calculating process.
Fig. 12 is a block diagram that illustrates a configuration example of an image decoding device according to the first embodiment of the present 5 technology.
Fig. 13 is a flowchart that illustrates a decoding process performed by the image decoding device.
Fig. 14 is a flowchart that illustrates a lossless decoding process. 10 Fig. 15 is a block diagram that illustrates a configuration example of an image encoding device according to a second embodiment of the present technology.
Fig. 16 is a block diagram that illustrates an 15 example of a detailed configuration of an intra
prediction orthogonal transformation unit that is illustrated in Fig. 15.
Fig. 17 is a diagram that illustrates a DWT performed by the DWT unit. 20 Fig. 18 is a block diagram that illustrates an example of a detailed configuration of the intra prediction inverse orthogonal transformation unit that is illustrated in Fig. 15.
Fig. 19 is a flowchart that illustrates orthogonal 25 transform processing.
Fig. 20 is a flowchart that illustrates inverse orthogonal transform processing.
Fig. 21 is a block diagram that illustrates a configuration example of an image decoding device 30 according to the second embodiment of the present technology.
8

4l SP305119WO00
Fig. 22 is a diagram that illustrates an example of the size of a macro block.
Fig. 23 illustrates a configuration example of a computer according to an embodiment. 5 Fig. 24 is a block diagram that illustrates an example of a main configuration of a television set.
Fig. 25 is a block diagram that illustrates an example of a main configuration of a cellular phone.
Fig. 26 is a block diagram that illustrates an 10 example of a main configuration of a hard disk recorder.
Fig. 27 is a block diagram that illustrates an example of a main configuration of a camera.
MODE FOR CARRYING OUT THE INVENTION
15 [0020]

[Configuration Example of Image Encoding Device]
Fig. 1 is a block diagram that illustrates a configuration example of an image encoding device
20 according to a first embodiment of the present technology. [0021]
The image encoding device 51 illustrated in Fig. 1 is configured by an A/D conversion unit 61, a screen rearranging buffer 62, a calculation unit 63, an
25 orthogonal transformation unit 64, a quantization unit 65, a lossless encoding unit 66, a storage buffer 67, an inverse quantization unit 68, an inverse orthogonal transformation unit 69, a calculation unit 70, a deblocking filter 71, a frame memory 72, a switch 73, an
30 intra prediction unit 74, a motion
predicting/compensating unit 75, a predicted image
9

ffl SP305119WO00
selecting unit 76, and a rate control unit 77. The image encoding device 51 compresses and encodes an input image in an AVC mode. [0022] 5 More specifically, the A/D conversion unit 61 of the image encoding device 51 performs A/D conversion of input images that are formed in units of frames and outputs resultant images to the screen rearranging buffer 62 for storage. The screen rearranging buffer 62
10 arranges stored images of frames, which are in display order, in order for encoding in accordance with a group of picture (GOP) structure. [0023]
The calculation unit 63, as is needed, subtracts a
15 predicted image supplied from the predicted image
selecting unit 7 6 from an image read from the screen rearranging buffer 62. The calculation unit 63 outputs an image acquired from a result of the subtraction or the image read from the screen rearranging buffer 62 to the
20 orthogonal transformation unit 64 as residual information. In a case where a predicted image is not supplied from the predicted image selecting unit 7 6, the calculation unit 63 outputs the image that has been read from the screen rearranging buffer 62 to the orthogonal■
25 transformation unit 64 without any change as the residual information. [0024]
The orthogonal transformation unit 64 is configured by an inter prediction orthogonal transformation unit 64A
30 and an intra prediction orthogonal transformation unit 64B. In a case where information (hereinafter, referred
10

^ SP305119WO00
to as inter prediction mode information) that represents an optimal inter prediction mode is supplied from the predicted image selecting unit 7 6, the inter prediction orthogonal transformation unit 64A performs orthogonal 5 transform processing such as a DCT or a KLT for the
residual information supplied from the calculation unit 63 as orthogonal transform processing. Then, the orthogonal transformation unit 64 supplies coefficients that are acquired as the result of the process to the
10 quantization unit 65 as transform coefficients. [0025]
In a case where information (hereinafter, referred to as intra-prediction mode information) that represents an optimal intra prediction mode is supplied from the
15 predicted image selecting unit 7 6, the intra prediction orthogonal transformation unit 64B performs orthogonal transform processing for the residual information supplied from the calculation unit 63. [0026]
20 More specifically, the intra prediction orthogonal transformation unit 64B performs a discrete wavelet transform (DWT), which has a calculation amount less than a KLT, for the residual information and extracts a low-frequency component and a high-frequency component. Then,
25 the intra prediction orthogonal transformation unit 64B performs a separable-type KLT of each intra prediction mode for the low-frequency component. The intra prediction orthogonal transformation unit 64B supplies coefficients that are acquired as a result of the
30 separable-type KLT of the optimal intra-prediction mode that is represented by the intra prediction mode
11

0 SP305119WO00
information and a high frequency component to the quantization unit 65 as transform coefficients. [0027]
The quantization unit 65 quantizes the transform 5 coefficients that are supplied from the orthogonal transformation unit 64. The quantized transform coefficients are input to the lossless encoding unit 66. [0028]
The lossless encoding unit 66 acquires the intra
10 prediction mode information from the intra prediction unit 7 4 and acquires the inter prediction mode information, information of a motion vector, and the like from the motion predicting/compensating unit 75. [0029]
15 The lossless encoding unit 66 (encoding unit)
performs lossless encoding such as variable-length coding (for example, context-adaptive variable length coding (CAVLC)) and an arithmetic coding (for example, context-adaptive binary arithmetic coding (CABAC)) for the
20 quantized transform coefficients supplied from the
quantization unit 65 and sets resultant information as a compressed image. In addition, the lossless encoding unit 66 performs lossless encoding of the intra prediction mode information, the inter prediction mode
25 information, the information of a motion vector, and the like and sets acquired resultant information as header information to be added to the compressed image. [0030]
In addition, in a case where the optimal prediction
30 mode of the transform coefficients is the optimal intra prediction mode, the lossless encoding unit 66 performs
12

^ SP305119WO00
lossless encoding of the transform coefficients in order according to the optimal intra prediction mode. The lossless encoding unit 66 supplies and stores the compressed image, to which the header information 5 acquired as the result of the lossless encoding is added, to the storage buffer 67 as compressed information. [0031]
The storage buffer 67 temporarily stores the compressed information that is supplied from the lossless
10 encoding unit 66 and, for example, outputs the compressed information to a recording device, a transmission line, or the like, which is not illustrated in the diagram, disposed on the latter part. [0032]
15 Further, the quantized transform coefficients
output from the quantization unit 65 are also input to the inverse quantization unit 68, inverse-quantization is performed for the quantized transform coefficients, and then, the transform coefficients are supplied to the
20 inverse orthogonal transformation unit 69. [0033]
The inverse orthogonal transformation unit 69 is configured by an inter prediction inverse orthogonal transformation unit 69A and an intra prediction inverse
25 orthogonal transformation unit 69B. In a case where the inter prediction mode information is supplied from the predicted image selecting unit 7 6, the inter prediction inverse orthogonal transformation unit 69A performs an inverse orthogonal transform such as an inverse DCT or an
30 inverse KLT for the transform coefficients that are
supplied from the inverse quantization unit 68 as inverse
13

^ SP305119WO00
orthogonal transform processing. The inter prediction inverse orthogonal transformation unit 69A supplies acquired resultant residual information to the calculation unit 70. 5 [0034]
In a case where the intra prediction mode information is supplied from the predicted image selecting unit 7 6, the intra prediction inverse orthogonal transformation unit 69B performs inverse
10 orthogonal transform processing for the transform coefficients that are supplied from the inverse quantization unit 68. More specifically, the intra prediction inverse orthogonal transformation unit 69B performs an inverse KLT of each intra prediction mode for
15 a coefficient of the low-frequency component out of the transform coefficients. In addition, the intra prediction inverse orthogonal transformation unit 69B performs an inverse DWT for a high-frequency component out of the low-frequency component that is acquired as
20 the result of the inverse KLT of the optimal intra prediction mode that is represented by the intra prediction mode information and the transform coefficients, thereby acquiring residual information. The intra prediction inverse orthogonal transformation
25 unit 69B supplies the residual information to the calculation unit 70. [0035]
The calculation unit 70 adds the residual information supplied from the inverse orthogonal
30 transformation unit 69 and the predicted image supplied
from the predicted image selecting unit 7 6 together as is
14

j| SP305119WO00
needed, thereby acquiring an image that is locally decoded. The calculation unit 7 0 supplies the acquired image to the deblocking filter 71. [0036] 5 The deblocking filter 71 filters the locally-decoded signal that is supplied from the calculation unit 70, thereby eliminating block distortion. The deblocking filter 71 supplies an acquired resultant image to the frame memory 72 for storage. In addition, the image
10 before the deblocking filter process performed by the
deblocking filter 71 is also supplied to the frame memory
72 and is stored.
[0037]
The switch 73 outputs the image stored in the frame
15 memory 72 to the motion predicting/compensating unit 75
or the intra prediction unit 7 4 as a reference image.
[0038]
I The intra prediction unit 74 performs an intra
I prediction process of all the intra prediction modes that
20 are candidates based on the image read from the screen rearranging buffer 62 and the reference image supplied through the switch 73, thereby generating a predicted image. Here, intra prediction modes having different block sizes are described as mutually-different intra
25 prediction modes even when the intra prediction modes are for the same predicted direction. In addition, as the block size, it is assumed that two types of 4 x 4 pixels and 8x8 pixels are handled. [0039]
30 In addition, the intra prediction unit 74
calculates cost function values for all the intra
15

m SP305119WO00
prediction modes that are candidates. Then, the intra prediction unit 74 selects an intra prediction mode of which the cost function value is a minimum as an optimal intra prediction mode. 5 [0040]
This cost function value is also called a rate distortion (RD) cost and is calculated, for example, based on any one technique of a high complexity mode and a low complexity mode as defined in a joint model (JM)
10 that is reference software of the AVC mode. [0041]
More specifically, in a case where the high complexity mode is employed as the technique for calculating a cost function value, provisionally the
15 process up to the lossless encoding is performed for all the intra prediction modes that are candidates, and the cost function value represented in the following Equation (1) is calculated for each intra prediction mode. [0042]
2 0 Equation (1)
Cost(Mode) = D + X-R [0043]
Here, D is a difference (distortion) between an original image and a decoded image, R is the amount of
25 generated codes including up to a transform coefficient,
and A, is a Larange multiplier that is given as a function
of a quantization parameter QP.
[0044]
On the other hand, in a case where the low 30 complexity mode is employed as the technique for
calculating a cost function value, generation of a
16

^ SP305119WO00
decoded image and calculation of a header bit such as intra prediction mode information are performed for all the intra prediction modes that are candidates, and a cost function value represented by the following Equation 5 (2) is calculated for each intra prediction mode. [0045]
Equation (2)
Cost(Mode) = D + QPtoQuant(QP)•Header_Bit [0046]
10 Here, D is a difference (distortion) between an original image and a decoded image, Header_Bit is a header bit for an intra prediction mode, and QPtoQuant is a function that is given as a function of a quantization parameter QP.
15 [0047]
In the low complexity mode, only decoded images are generated for all the intra prediction modes, and it is not necessary to perform lossless encoding, whereby the amount of calculation is small. Here, it is assumed that
2 0 the high complexity mode is employed as the technique for calculating a cost function value. [0048]
The intra prediction unit 74 supplies the intra prediction mode information, the predicted image that is
25 generated in the optimal intra prediction mode, and the
corresponding cost function value to the predicted image
selecting unit 76. When being notified of the selection
of the predicted image that is generated in the optimal
intra prediction mode from the predicted image selecting I
30 unit 7 6, the intra prediction unit 74 supplies the intra I
prediction mode information to the lossless encoding unit
17
■

m SP305119WO00
66. [0049]
The motion predicting/compensating unit 75 predicts motions of all the inter prediction modes that are 5 candidates based on the image supplied from the screen rearranging buffer 62 and the reference image supplied through the switch 73 and generates a motion vector. Then, the motion predicting/compensating unit 75 performs a compensation process for the reference image based on
10 the motion vector and generates a predicted image. At this time, the motion predicting/compensating unit 75 calculates cost function values for all the inter prediction modes that are candidates and determines an inter prediction mode of which the cost function value is
15 a minimum as an optimal inter prediction mode. [0050]
Then, the motion predicting/compensating unit 75 supplies inter prediction mode information, the predicted image that is generated in the optimal inter prediction
20 mode, and the corresponding cost function value to the predicted image selecting unit 76. When being notified of the selection of the predicted image of the optimal inter prediction mode from the predicted image selecting unit 7 6, the motion predicting/compensating unit 75
25 outputs the inter prediction mode information,
information of the corresponding motion vector, and the
like to the lossless encoding unit 66.
[0051]
The predicted image selecting unit 7 6 determines
30 one of the optimal intra prediction mode and the optimal inter prediction mode as an optimal prediction mode based
18

H SP305119WO00
on the cost function values supplied from the intra prediction unit 74 and the motion predicting/compensating unit 75. Then, the predicted image selecting unit 76 supplies the predicted image of the optimal prediction 5 mode to the calculation unit 63 and the calculation unit 70. In addition, the predicted image selecting unit 7 6 notifies the intra prediction unit 74 or the motion predicting/compensating unit 7 5 of the selection of the predicted image of the optimal prediction mode.
10 Furthermore, the predicted image selecting unit 7 6
supplies the inter prediction mode information or the intra prediction mode information that represents the optimal prediction mode to the orthogonal transformation unit 64 and the inverse orthogonal transformation unit 69.
15 [0052]
The rate control unit 77 performs control of the rate of the quantization operation of the quantization unit 65 based on the compressed information stored in the storage buffer 67 such that overflow or underflow does
2 0 not occur. [0053] [Description of Intra Prediction Mode]
Fig. 2 is a diagram that illustrates an example of a predicted direction of each intra prediction mode.
25 [0054]
As illustrated in Fig. 2, in the intra prediction mode of the AVC mode, there are a mode in which eight directions are predicted and a mode in which a DC is predicted. Numbers represented in Fig. 2 are intra
30 prediction mode numbers, and arrows to which the numbers are assigned represent predicted directions of the
!
19

_|^> SP305119WO00
numbers.
[0055]
More specifically, as illustrated in Fig. 2, a
predicted direction of the intra prediction mode of 5 number 0 is the vertical direction, and a predicted
direction of the intra prediction mode of number 1 is the
horizontal direction. An intra prediction mode of number
2 that is not illustrated in the diagram is a mode in
which a DC is predicted. As illustrated in Fig. 2, 10 predicted directions of the intra prediction modes of
numbers 3 to 8 are oblique directions.
[0056]
[Example of Detailed Configuration of Intra Prediction
Orthogonal Transformation Unit] 15 Fig. 3 is a block diagram that illustrates an
example of a detailed configuration of the intra
prediction orthogonal transformation unit 64B that is
illustrated in Fig. 1.
[0057] 2 0 The intra prediction orthogonal transformation unit
64B illustrated in Fig. 3 is configured by a DWT unit 91,
KLT units 92-0 to 92-8, and a selector 93.
[0058]
The DWT unit 91 (prior orthogonal transformation 25 unit) performs a DWT for the residual information that is
supplied from the calculation unit 63 illustrated in Fig.
1, as is needed, and thereby acquiring a low-frequency
component and a high-frequency component. The DWT unit
91 supplies the low-frequency component to the KLT units 30 92-0 to 92-8 and supplies the high-frequency component to
the selector 93. In a case where the KLT does not need
20

^ SP305119WO00
to be performed, the DWT unit 91 supplies the residual information to the KLT units 92-0 to 92-8 without change. [0059]
The KLT units 92-0 to 92-8 (posterior orthogonal 5 transformation units) perform separable-type KLTs for the low-frequency component or the residual information that is supplied from the DWT unit 91 using bases of the KLTs of the intra prediction modes of numbers 0 to 8. Here, the base of the KLT of each intra prediction mode is an
10 optimal value that is acquired through learning in
advance. The KLT units 92-0 to 92-8 supply coefficients that are acquired as results of the separable-type KLTs to the selector 93, [0060]
15 The intra prediction mode information supplied from the predicted image selecting unit 7 6 is supplied to the selector 93. The selector 93 selects a coefficient, which is supplied from one of the KLT units 92-0 to 92-8 that corresponds to the optimal intra prediction mode,
20 out of coefficients supplied from the KLT units 92-0 to 92-8 based on the intra prediction mode information. Then, the selector 93 supplies the selected coefficient of the residual information to the quantization unit 65 (Fig. 1) as a transform coefficient or supplies a
25 coefficient of the low-frequency component and the high-frequency component that is supplied from the DWT unit 91 to the quantization unit 65 as transformation coefficients. [0061]
30 Although the selector 93 is disposed after the KLT units 92-0 to 92-8 in the intra prediction orthogonal
21
■
I

^ SP305119WO00
transformation unit 64B illustrated in Fig. 3, the selector may be disposed before the KLT units 92-0 to 92-8, In such a case, the low-frequency component or the residual information that is output from the DWT unit 91 5 is supplied to only one of the KLT units 92-0 to 92-8 that corresponds to the optimal intra prediction mode. [0062] [Description of DWT]
Fig. 4 is a diagram that illustrates the DWT
10 performed by the DWT unit 91. [0063]
When the DWT is performed for the residual information illustrated in A of Fig. 4 in the horizontal direction and the vertical direction, as illustrated in B
15 of Fig. 4, the horizontal component and the vertical
component are respectively divided into a high-frequency component and a low-frequency component. In Fig. 4, L represents the low-frequency component, and H represents the high-frequency component. In addition, the
20 horizontal component and the vertical component are
continuously represented. These are similarly applied
also to Fig. 17 to be described below.
[0064]
As illustrated in B of Fig. 4, when a DWT is
25 performed for the residual information in the horizontal direction and the vertical direction, the residual information is divided into a component (LL component) in which the horizontal component and the vertical component are low-frequency components, a component (HL component)
30 in which the horizontal component is a high-frequency
component and the vertical component is a low-frequency
22

0 SP305119WO00
■
component, a component (LH component) in which the horizontal component is a high-frequency component and the vertical component is a low-frequency component, and a component (HH component) in which the horizontal 5 component and the vertical component are high frequency components. [0065]
In addition, when a DWT is performed again for the LL component in the horizontal direction and the vertical
10 direction, as illustrated in C of Fig. 4, the horizontal component and the vertical component of the LL component are respectively divided further into a high-frequency component and a low-frequency component. As a result, the LL component is divided into a component (LLLL
15 component) in which the horizontal component and the vertical component are low-frequency components, a component (LLHL component) in which the horizontal component is a high-frequency component and the vertical component is a low frequency component, a component
20 (LLLH) in which the horizontal component is a low-frequency component and the vertical component is a high-frequency component, and a component (LLHH component) in which the horizontal component and the vertical component are high-frequency components. The HL component, the LH
25 component, and the HH component that are components of
the residual information other than the LL component are
unchanged.
[0066]
For example, in a case where the number of DWTs
30 performed by the DWT unit 91 is two, the LLLL component that is formed by the horizontal component and the
23

# SP305119WO00
vertical component that have the lowest frequency is output to the KLT units 92-0 to 92-8 (Fig. 3) as a low frequency component. Accordingly, the size of data that is a target for the separable-type KLT can be 1/16 of the 5 size of data of the residual information. For example, in a case where the block size of the intra prediction corresponds to 4 x 4 pixels, in other words, in a case where the residual information corresponds to 4 x 4 pixels, the size of data that is a target for the
10 separable-type KLT corresponds to 1 x 1 pixel. In
addition, in a case where the block size of the intra prediction corresponds to 8 x 8 pixels, the size of data that is a target for the separable-type KLT corresponds to 2 x 2 pixels.
15 [0067]
As a result, the calculation amount of the orthogonal transform processing for the residual information using a predicted image according to an intra prediction can be reduced. More specifically, when the
20 block size corresponds to N x N pixels, the order of the calculation amount of the DWT unit 91 is about from N to Nlog|N|, and the order of the calculation amounts of the KLT units 92-0 to 92-8 is (N/4)^. Accordingly, the order of the calculation amount of the orthogonal transform
25 processing is about (N/4)^ + Nlog|N|, which is
sufficiently smaller than N^ that is the order of the calculation amount of the separable-type KLT in a case where the block size corresponds to N x N pixels. [0068]
30 The components other than the LLLL component are output to the quantization unit 65 (Fig. 1) as high
24

^ SP305119WO00
frequency components without change.
[0069]
Here, the number of DWTs that are performed is not
limited to two. The DWT is repeatedly performed until 5 the size of a component that is formed by the horizontal
component and the vertical component that have the lowest
frequency is a size that can be processed by the KLT
units 92-0 to 92-8 or less. As the size that can be
processed by the KLT units 92-0 to 92-8 becomes smaller, 10 although the calculation amount of the orthogonal
transform processing is smaller, which is advantageous in
terms of the manufacturing cost or real-time reproduction,
the number of DWTs that are performed increases, and the
coding efficiency decreases. Accordingly, the size that 15 can be processed by the KLT units 92-0 to 92-8 is
determined in consideration of the coding efficiency and
an allowed range of the calculation amount of the
orthogonal transform processing.
[0070] 20 [Detailed Configuration Example of Intra Prediction
Orthogonal Transformation Unit]
Fig. 5 is a block diagram that illustrates a
detailed configuration example of the intra prediction
inverse orthogonal transformation unit 69B that is 25 illustrated in Fig. 1. [0071]
The intra prediction inverse orthogonal
transformation unit 69B illustrated in Fig. 5 is
configured by inverse KLT units 101-0 to 101-8, a 30 selector 102, and an inverse DWT unit 103.
■
[0072]
25
I t

0 SP305119WO00
The inverse KLT units 101-0 to 101-8 perform
separable-type inverse KLTs for coefficients out of
transform coefficients supplied from the inverse
quantization unit 68 illustrated in Fig. 1 using bases of 5 the inverse KLTs of the intra prediction modes of numbers
0 to 8.
[0073]
The intra prediction mode information that is
supplied from the predicted image selecting unit 7 6 is 10 supplied to the selector 102. The selector 102 selects a
post-inverse KLT component, which is supplied from one of
the inverse KLT units 101-0 to 101-8, corresponding to
the optimal intra prediction mode out of post-inverse KLT
components supplied from the inverse KLT units 101-0 to 15 101-8 based on the intra prediction mode information.
Then, the selector 102 supplies the selected post-inverse
KLT component to the inverse DWT unit 103.
[0074]
The inverse DWT unit 103 supplies the component 20 that is supplied from the selector 102 to the calculation
unit 7 0 (Fig. 1) as the residual information.
Alternatively, the inverse DWT unit 103 performs an
inverse DWT that corresponds to the DWT according to the
DWT unit 91 for the component supplied from the selector 25 102 and a high-frequency component out of the transform
coefficients supplied from the inverse quantization unit
68 and supplies an acquired resultant component to the
calculation unit 70 as the residual information.
[0075] 30 In the intra prediction inverse orthogonal
transformation unit 69B illustrated in Fig. 5, although
26 i
i

Upl SP305119WO00
the selector 102 is disposed after the inverse KLT units
101-0 to 101-8, the selector may be disposed before the
inverse KLT units 101-0 to 101-8. In such a case,
coefficients of the transform coefficients supplied from 5 the inverse quantization unit 68 are supplied only to one
of the inverse KLT units 101-0 to 101-8 that corresponds
to the optimal intra prediction mode.
[0076]
[Description of Procedure of Lossless Encoding] 10 Fig. 6 is a diagram that illustrates the sequence
of lossless encoding that is performed by the lossless
encoding unit 66 illustrated in Fig. 1.
[0077]
In the lossless encoding, since the number of 15 consecutive O's is coded, and it is preferable that the
sequence of lossless encoding is the sequence in which
O's are consecutive.
[0078]
Thus, the lossless encoding unit 66 changes the 20 encoding sequence in accordance with the intra prediction
mode such that O's of the transform coefficient after
quantization is consecutive.
[0079]
More specifically, when a DWT is performed for the 25 residual information illustrated in A of Fig. 6, the
residual information, as illustrated in B of Fig. 6, is
divided into bands of four components including an LL
component, an HL component, an LH component, and an HH
component. At this time, as illustrated in B of Fig. 6, 30 the LL component has characteristics of the residual
information. In addition, the HL component represents
27
I:

j^ SP305119WO00
lines extending in the vertical direction, the LH component represents lines extending in the horizontal direction, and the HH component represents lines extending in an oblique direction. In other words, the 5 HL component has a strong correlation in the vertical direction, the LH component has a strong correlation in the horizontal direction, and the HH component has a strong correlation in the oblique direction. [0080]
10 In the intra prediction mode of number 0 in which the predicted direction is the vertical direction, the correlation of the residual information in the vertical direction is strong. In other words, there are many high-frequency components of the residual information in
15 the horizontal direction. Accordingly, in the intra
prediction mode of number 0, there are many cases where
there are many HL components.
[0081]
In the intra prediction mode of number 1 in which
20 the predicted direction is the horizontal direction, the correlation of the residual information in the horizontal direction is strong. In other words, there are many high-frequency components of the residual information in the vertical direction. Accordingly, in the intra
25 prediction mode of number 1, there are many cases where there are many LH components. [0082]
In the intra prediction modes of numbers 3 to 8 in which the predicted directions are oblique directions,
30 the correlations of the residual information in the
oblique directions are strong. In other words, there are
28

g^ SP305119WO00
many high-frequency components of the residual information in the oblique directions. Accordingly, in the intra prediction modes of numbers 3 to 8, there are many cases where there are many HH components. 5 [0083]
Here, in order to allow O's of the transform coefficients after quantization to be consecutive as a target for lossless encoding, it is necessary to set the transform coefficients of components after quantization
10 as targets for lossless encoding in order from a
transform coefficient of a component, of which the value after the DWT is estimated to be larger, to a transform coefficient of which the value after the DWT is estimated to be smaller.
15 [0084]
Accordingly, in a case where the optimal prediction mode is the intra prediction mode having number 0, for example, the lossless encoding unit 66 sets the targets for lossless encoding in order of the LL component, the
20 HL component, the HH component, and the LH component. In addition, in a case where the optimal prediction mode is the intra prediction mode of number 1, for example, the targets for lossless encoding are set in order of the LL component, the LH component, the HH component, and the HL
25 component. Furthermore, in a case where the optimal
prediction mode is any one of the intra prediction modes of numbers 3 to 8, for example, the targets for lossless encoding are set in order of the LL component, the HH component, the HL component, and the LH component. In
30 addition, in a case where the optimal prediction mode is the intra prediction mode of number 2, components are set
29 I
i--
I f i

^ SP305119WO00
as targets for lossless encoding in a predetermined order
set in advance.
[0085]
Also within the LH component, the HL component, and 5 the HH component, blocks are set as targets for lossless encoding in order from a block of which a value after the DWT is estimated to be larger to a block of which a value after the DWT is estimated to be smaller. More specifically, since the LH component has a strong
10 correlation in the horizontal direction, blocks are set as targets for lossless encoding in order of the alignment in the horizontal direction within the LH component. In addition, since the HL component has a strong correlation in the vertical direction, blocks are
15 set as targets for lossless encoding in order of the alignment in the vertical direction within the HL component. Further, since the HH component has a strong correlation in the oblique direction, blocks are set as targets for lossless encoding in order of the alignment
20 in the oblique direction within the HH component. [0086]
Within the LL component, blocks are set as targets for lossless encoding in a predetermined order that is determined in advance. The order within the LL component
25 may be different for each intra prediction mode. [0087]
[Description of Encoding Process of Image Encoding Device]
Fig. 7 is a flowchart that illustrates an encoding
30 process of the image encoding device 51 illustrated in Fig. 1.
30

^ SP305119WO00
[0088]
In Step Sll, the A/D conversion unit 61 perforins A/D conversion of an input image that is in units of frames and supplies a resultant image to the screen 5 rearranging buffer 62. In Step S12, the screen
rearranging buffer 62 stores the image that is supplied from the A/D conversion unit 61, rearranges the images of frames stored in order of display in the order for encoding, and supplies a resultant image to the
10 calculation unit 63. [0089]
In Step S13, the calculation unit 63 subtracts the predicted image supplied from the predicted image selecting unit 7 6 from the image read from the screen
15 rearranging buffer 62 as is necessary, thereby acquiring residual information. Then, the calculation unit 63 outputs the residual information to the orthogonal transformation unit 64. [0090]
20 In Step S14, the orthogonal transformation unit 64 performs orthogonal transform processing for the residual information supplied from the calculation unit 63. More specifically, in a case where the inter prediction mode information is supplied from the predicted image
25 selecting unit 7 6, the inter prediction orthogonal
transformation unit 64A of the orthogonal transformation unit 64 performs orthogonal transform processing for the residual information supplied from the calculation unit 63 and supplies acquired resultant coefficients to the
30 quantization unit 65 as transform coefficients. [0091]
31

^ SP305119WO00
In a case where the intra-prediction mode information is supplied from the predicted image selecting unit 7 6, the intra prediction orthogonal transformation unit 64B performs orthogonal transform 5 processing for the residual information that is supplied from the calculation unit 63. The orthogonal transform processing will be described in detail with reference to Fig. 8 to be described below. [0092]
10 In Step S15, the quantization unit 65 quantizes the transform coefficients that are acquired as a result of the orthogonal transform processing. When the quantization is performed, the rate is controlled as will be described regarding the process of Step S30 to be
15 described below. [0093]
The transform coefficients that are quantized as described above are locally decoded as below. In Step S16, the inverse quantization unit 68 performs inverse-
20 quantization of the transform coefficients that are
quantized by the quantization unit 65 in accordance with characteristics corresponding to the characteristics of the quantization unit 65. In Step S17, the inverse orthogonal transformation unit 69 performs inverse
25 orthogonal transform processing that corresponds to the orthogonal transform processing performed by the orthogonal transformation unit 64 for the transform coefficients for which the inverse-quantization has been performed by using the inverse quantization unit 68.
30 [0094]
More specifically, in a case where the inter
32

^ SP305119WO00
prediction mode information is supplied from the predicted image selecting unit 7 6, the inter prediction inverse orthogonal transformation unit 69A of the inverse orthogonal transformation unit 69 performs inverse 5 orthogonal transform processing for the transform coefficients that are supplied from the inverse quantization unit 68. The inter prediction inverse orthogonal transformation unit 69A supplies acquired resultant residual information to the calculation unit 70.
10 [0095]
In addition, in a case where the intra prediction mode information is supplied from the predicted image selecting unit 7 6, the intra prediction inverse orthogonal transformation unit 69B performs inverse
15 orthogonal transform processing for the transform coefficients that are supplied from the inverse quantization unit 68. The inverse orthogonal transform processing will be described in detail with reference to Fig. 9 to be described below.
20 [0096]
In Step S18, the calculation unit 70 adds the residual information that is supplied from the inverse orthogonal transformation unit 69 to the predicted image that is supplied from the predicted image selecting unit
25 76, thereby generating an image that is locally decoded. Then, the calculation unit 70 supplies the locally-decoded image to the calculation unit 70 and the frame memory 72. [0097]
30 In Step S19, the deblocking filter 71 eliminates
block distortion by filtering the image, which is locally
33

^ SP305119WO00
decoded, supplied from the calculation unit 70. Then, the deblocking filter 71 supplies an acquired resultant image to the frame memory 72. [0098] 5 In Step S2 0, the frame memory 72 stores the image that is supplied from the deblocking filter 71 and the image, which is not filtered by the deblocking filter 71, supplied from the calculation unit 70. [0099]
10 In Step S21, the intra prediction unit 74 performs an intra prediction process. More particularly, the intra prediction unit 7 4 performs an intra prediction process of all the intra prediction modes that are candidates based on the image read from the screen
15 rearranging buffer 62 and the reference image supplied from the frame memory 72 through the switch 73, thereby generating a predicted image. [0100]
In Step S22, the intra prediction unit 74 performs
20 an intra prediction cost function value calculating
process for calculating cost function values of all the intra prediction modes that are candidates. The intra prediction cost function value calculating process will be described in detail with reference to Fig. 10 to be
25 described below. [0101]
In Step S23, the motion predicting/compensating unit 75 performs a motion predicting/compensating process. More specifically, the motion predicting/compensating
30 unit 75 predicts the motions of all the inter prediction modes that are candidates based on the image supplied
■
34

^ SP305119WO00
from the screen rearranging buffer 62 and the reference
image supplied through the switch 73, thereby generating
a motion vector. Then, the motion
predicting/compensating unit 75 performs a compensation 5 process for the reference image based on the motion
vector, thereby generating a predicted image.
[0102]
In Step S24, the motion predicting/compensating
unit 75 calculates the cost function values of all the 10 inter prediction modes that are candidates and determines
an inter prediction mode of which the cost function value
is a minimum as an optimal inter prediction mode. Then,
the motion predicting/compensating unit 75 supplies the
inter prediction mode information, the predicted image 15 generated in the optimal inter prediction mode, and the
corresponding cost function value to the predicted image
selecting unit 76.
[0103]
In Step S25, the predicted image selecting unit 76 20 determines whether the cost function value of the optimal
intra prediction mode that is supplied from the intra
prediction unit 74 is larger than the cost function value
of the optimal inter prediction mode that is supplied
from the motion predicting/compensating unit 75. 25 [0104]
In a case where the cost function value of the
optimal intra prediction mode is determined to be larger
than the cost function value of the optimal inter
prediction mode in Step S25, the process proceeds to Step 30 S26. In Step S26, the predicted image selecting unit 76
determines the optimal intra prediction mode as an
35
i

^ SP305119WO00
optimal prediction mode and selects a predicted image of the optimal intra prediction mode. Then, the predicted image selecting unit 7 6 supplies the predicted image of the optimal intra prediction mode to the calculation unit 5 63 and the calculation unit 70. This predicted image is used in the processes of Steps S13 and S18 described above. In addition, the predicted image selecting unit 7 6 supplies the intra prediction mode information to the orthogonal transformation unit 64 and the inverse
10 orthogonal transformation unit 69 and notifies the intra prediction unit 74 of the selection of the predicted image of the optimal intra prediction mode. The intra prediction unit 74 supplies the intra prediction mode information to the lossless encoding unit 66 in
15 accordance with the notification. Then, the process proceeds to Step S28. [0105]
On the other hand, in a case where the cost function value of the optimal intra prediction mode is
20 determined not to be larger than the cost function value of the optimal inter prediction mode in Step S25, the process proceeds to Step S27. In Step 827, the predicted image selecting unit 7 6 determines the optimal inter prediction mode as an optimal prediction mode and selects
25 a predicted image of the optimal inter prediction mode. Then, the predicted image selecting unit 7 6 supplies the predicted image of the optimal inter prediction mode to the calculation unit 63 and the calculation unit 70. This predicted image is used in the processes of Steps
30 S13 and S18 described above. In addition, the predicted image selecting unit 76 supplies the inter prediction
36

^ SP305119WO00
mode information to the orthogonal transformation unit 64 and the inverse orthogonal transformation unit 69 and notifies the motion predicting/compensating unit 75 of the selection of the predicted image of the inter 5 prediction mode. The motion predicting/compensating unit 75 supplies the inter prediction mode information, information of the corresponding motion vector, and the like to the lossless encoding unit 66 in accordance with the notification. Then, the process proceeds to Step S28.
10 [0106]
In Step S28, the lossless encoding unit 66 performs lossless encoding of the quantized transform coefficients that are supplied from the quantization unit 65, thereby acquiring a compressed image. In a case where the
15 optimal prediction mode of the transform coefficient that is a target for lossless encoding is the optimal intra prediction mode, lossless encoding is performed in the sequence according to the optimal intra prediction mode as described with reference to Fig. 6. In addition, the
20 lossless encoding unit 66 performs lossless encoding of the intra prediction mode information, the inter prediction mode information, the motion vector, and the like and sets acquired resultant information as header information added to the compressed image. The lossless
25 encoding unit 66 supplies the compressed image to which the header information acquired as a result of the lossless encoding is added to the storage buffer 67 as compressed information. [0107]
30 In Step S29, the storage buffer 67 stores the
compressed information that is supplied from the lossless
37

^ SP305119WO00
encoding unit 66. The compressed information that is stored in the storage buffer 67 is appropriately read and is transmitted to the decoding side through transmission lines. 5 [0108]
In Step S30, the rate control unit 77 performs control of the rate of the quantization operation of the quantization unit 65 based on the compressed information stored in the storage buffer 67 such that overflow or 10 underflow does not occur. [0109]
Fig. 8 is a flowchart that illustrates orthogonal transform processing that is performed by the intra prediction orthogonal transformation unit 64B in Step S14 15 illustrated in Fig. 7. [0110]
In Step S41, the DWT unit 91 (Fig. 3) of the intra prediction orthogonal transformation unit 64B determines whether or not the size of the residual information 20 supplied from the calculation unit 63 (Fig. 1) is a size that can be processed by the KLT units 92-0 to 92-8. [0111]
In a case where the size of the residual information is determined not to be a size that can be 25 processed by the KLT units 92-0 to 92-8 in Step S41, the DWT unit 91 performs a DWT for the residual information in Step S42. [0112]
In Step S43, the DWT unit 91 determines whether or 30 not the size of a low-frequency component acquired as a result of the process of Step S42 is a size that can be
I
38 [
f I
\ f
I

djL SP305119WO00
processed by the KLT units 92-0 to 92-8. In a case where the size of the low-frequency component acquired as the result of the DWT is determined not to be a size that can be processed by the KLT units 92-0 to 92-8 in Step S43, 5 the DWT unit 91 performs a DWT in Step S44 for the low-frequency component acquired as the result of the processes of Step S42. Then, the process is returned to Step S43, and the process of Steps S43 and S44 is repeated until the size of the low-frequency component
10 becomes a size that can be processed by the KLT units 92-0 to 92-8. [0113]
In a case where the size of the low-frequency component acquired as the result of the DWT is determined
15 to be a size that can be processed by the KLT units 92-0 to 92-8 in Step S43, the DWT unit 91 supplies the low-frequency component acquired as the result of the DWT to the KLT units 92-0 to 92-8 and supplies a high-frequency component to the selector 93.
20 [0114]
Then, in Step S45, the KLT units 92-0 to 92-8 perform separable-type KLTs for the low-frequency component by using bases of the KLTs of the corresponding intra prediction modes.
25 [0115]
In Step S46, the selector 93 selects a coefficient supplied from one of the KLT units 92-0 to 92-8 that corresponds to the optimal intra prediction mode out of the coefficients supplied from the KLT units 92-0 to 92-8
30 based on the intra prediction mode information and
outputs the selected coefficient to the quantization unit
39

^ SP305119WO00
65 (Fig. 1) . [0116]
In Step S47, the selector 93 outputs the high-frequency component supplied from the DWT unit 91 to the 5 quantization unit 65. [0117]
On the other hand, in a case where the size of the residual information is determined to be a size that can be processed by the KLT units 92-0 to 92-8 in Step S41, 10 the DWT unit 91 supplies the residual information to the KLT units 92-0 to 92-8 without change. Then, in Step S48, the KLT units 92-0 to 92-8 perform separable-type KLTs for the residual information by using the bases of the KLTs of the corresponding intra prediction modes. 15 [0118]
In Step S49, the selector 93 selects a coefficient that is supplied from one of the KLT units 92-0 to 92-8 that corresponds to the optimal intra prediction mode out of the coefficients supplied from the KLT units 92-0 to 20 92-8 based on the intra prediction mode information and outputs the selected coefficient to the quantization unit 65. [0119]
Fig. 9 is a flowchart that illustrates inverse 25 orthogonal transform processing performed by the intra prediction inverse orthogonal transformation unit 69B of Step S17 illustrated in Fig. 7. [0120]
In Step S61, the inverse KLT units 101-0 to 101-8 30 (Fig. 5) of the intra prediction inverse orthogonal
transformation unit 69B perform separable-type KLTs for
40

^ SP305119WO00
the coefficients included in the transform coefficients supplied from the inverse quantization unit 68 (Fig. 1) by using the bases of the KLTs of the corresponding intra prediction modes. 5 [0121]
In Step S62, the selector 102 selects a post-inverse KLT component supplied from one of the inverse KLT units 101-0 to 101-8 that corresponds to the optimal intra prediction mode out of the post-inverse KLT
10 components supplied from the inverse KLT units 101-0 to 101-8 based on the intra prediction mode information supplied from the predicted image selecting unit 76. Then, the selector 102 supplies the selected post-inverse KLT component to the inverse DWT unit 103.
15 [0122]
In Step S63, the inverse DWT unit 103 determines whether or not the size of the post-inverse KLT component supplied from the selector 102 is the block size. In a case where the size of the post-inverse KLT component is
20 determined not to be the block size in Step S63, the
process proceeds to Step S64. In Step S64, the inverse DWT unit 103 performs an inverse DWT for high-frequency-components of the post-inverse KLT component and the transform coefficients supplied from the inverse
25 quantization unit 68. [0123]
In Step S65, the inverse DWT unit 103 determines whether or not the size of the post-inverse DWT component that is acquired by as a result of the process of Step
30 S64 is the block size. In a case where the size of the post-inverse DWT component is determined not to be the
41

jl SP305119WO00
block size in Step S65, the process is returned to Step
S64, and the processes of Steps S64 and S65 are repeated
until the size of the post-inverse DWT component becomes
the block size. 5 [0124]
In a case where the size of the post-inverse DWT
component is determined to be the block size in Step 565,
the process proceeds to Step S66.
[0125] 10 On the other hand, in a case where the size of the
post-inverse KLT component is determined to be the block
size in Step S63, the process proceeds to Step S66.
[0126]
In Step S66, the inverse DWT unit 103 supplies the 15 post-inverse DWT component or the post inverse KLT
component to the calculation unit 70 (Fig. 1) as the
residual information.
[0127]
Fig. 10 is a flowchart that illustrates an intra-20 prediction cost function value calculating process of
Step S22 illustrated in Fig. 7.
[0128]
In Step S81, the intra prediction unit 74 sets a
block sizes that has not been yet set as a block size 25 (hereinafter, referred to as a current block size) of a
block that is a processing target and is a candidate as a
current block size. Here, the block sizes that are
candidates are 4x4 pixels and 8x8 pixels.
[0129] 30 In Step S82, the intra prediction unit 74 sets a
block IDX that is not set as a block IDX (hereinafter,
42

^ SP305119WO00
referred to as a current block IDX) of a processing target block out of blocks IDX of a current block size as a current block IDX. Here, the block IDX is information that specifies each block when a macro block is divided 5 into blocks in accordance with a block size. For example, in a case where the macro block has a size of 128 x 128, and the block size is 8 x 8, there are 16 blocks IDX of 0 to 15. [0130]
10 In Step S83, the intra prediction unit 74 sets the number mode of the intra prediction mode to "0". [0131]
In Step S84, the intra prediction unit 74 performs a cost function value calculating process for calculating
15 a cost function value of the intra prediction mode of the number mode. This cost function calculating process will be described in detail with reference to Fig. 11 to be described below. [0132]
20 In Step S85, the intra prediction unit 74
determines whether or not the number mode is less than the number "9" of the intra prediction modes that are candidates. In a case where the number mode is determined to be less than "9" in Step S85, in other
25 words, in a case where the cost function values of all the intra prediction modes of the block of the current block IDX have not been calculated yet, the process proceeds to Step S8 6. [0133]
30 In Step S86, the intra prediction unit 74
increments the number mode by one, returns the process to
43

^ SP305119WO00
Step S84, and repeats the processes of Steps S84 to S86
until the number mode becomes "9" or more.
[0134]
In a case where the niomber mode is determined not 5 to be less than "9" in Step S85, in other words, in a case where the cost function values of all the intra prediction modes of the block of the current block IDX are calculated, the process proceeds to Step S87. In Step S87, the intra prediction unit 74 determines an
10 intra prediction mode that corresponds to a minimal value out of the cost function values of all the intra prediction modes of the block of the current block IDX as an optimal intra prediction mode of the block of the current block IDX.
15 [0135]
In Step S88, the intra prediction unit 74 determines whether or not all the blocks IDX of the current block size have been set as current blocks IDX. In a case where all the blocks IDX of the current block
2 0 size are determined not to have been set as the current blocks IDX in Step S88, the process is returned to Step S82. Then, until all the blocks IDX of the current block size are set as the current blocks IDX, the process of Steps S82 to S88 is repeated.
25 [0136]
On the other hand, in a case where all the blocks IDX of the current block size are determined to have been set as the current blocks IDX in Step S88, the intra prediction unit 74 determines whether or not all the
30 block sizes are set as the current block sizes in Step S89.
44

i I
^ SP305119WO00
[0137]
In a case where all the block sizes are determined not to have been yet set as the current block sizes in Step S89, the process is returned to Step S81, and the 5 process of Steps S81 to S89 is repeated until all the block sizes are set as the current block sizes. [0138]
On the other hand, in a case where all the block sizes are determined to have been set as the current
10 block sizes in Step S89, in other words, in a case where the optimal intra prediction modes of all the blocks of all the blocks of all the block sizes are determined, the process proceeds to Step S90. [0139]
15 In Step S90, the intra prediction unit 74
determines an intra prediction mode of which the cost function value is a minimum in units of macro blocks as a final optimal intra prediction mode based on the cost function values of the optimal intra prediction modes of
20 all the blocks of all the block sizes. Then, the intra prediction unit 74 supplies intra prediction mode information that represents the optimal intra prediction mode, a predicted image that is generated in the optimal prediction mode, and a corresponding cost function value
25 to the predicted image selecting unit 76. Then, the
process is returned to Step S22 illustrated in Fig. 7 and proceeds to Step S23. [0140]
Fig. 11 is a flowchart that illustrates the cost
30 function value calculating process of Step S84 illustrated in Fig. 10.
45

^ SP305119WO00
[0141]
In Step Sill, the intra prediction unit 74
subtracts a predicted image of an intra prediction mode
of a number mode that corresponds to the current block 5 IDX from an image corresponding to the current block IDX
that is supplied from the screen rearranging buffer 62,
thereby acquiring residual information.
[0142]
In Step S112, the intra prediction unit 74 performs 10 the orthogonal transform processing illustrated in Fig. 8
for the residual information.
[0143]
In Step S113, the intra prediction unit 74
quantizes transform coefficients that are acquired as a 15 result of the orthogonal transform processing.
[0144]
In Step S114, the intra prediction unit 74 performs
lossless encoding of coefficients of the quantized
transform coefficients in a predetermined order. 20 [0145]
In Step S115, the intra prediction unit 74 performs
lossless encoding of high-frequency components (for
example, an LH component, an HL component, and an HH
component) of the quantized transform coefficients in the 25 order according to the intra prediction modes of the
number modes. In addition, in a case where a DWT is not
performed in the orthogonal transform processing of Step
S112, the process of Step S115 is not provided.
[0146] 30 In Step S116, the intra prediction unit 74 performs
inverse quantization of the quantized transform
46

^ SP305119WO00
coefficients that are acquired in the process of Step
S113.
[0147]
In Step S117, the intra prediction unit 74 performs 5 inverse orthogonal transforming illustrated in Fig. 9 for the transform coefficients for which inverse quantization has been performed in Step S116. [0148]
In Step S118, the intra prediction unit 74 adds
10 residual information that is acquired as a result of the process of Step S117 and the predicted image of the intra prediction mode of the number mode that corresponds to the current block IDX together. [0149]
15 In Step S119, the intra prediction unit 74
calculates a cost function value using Equation (1) described above by using the decoded image that is acquired as a result of the process of Step S118, the image supplied from the screen rearranging buffer 62, and
20 the amount of generated codes in the process of Step S115. This cost function value is a cost function value of the intra prediction mode of the number mode of the block of the current block IDX. [0150]
25 As above, in the image encoding device 51, the intra prediction orthogonal transformation unit 64B performs a DWT of which the amount of calculation is less than that of a discrete type KLT before a discrete type KLT and performs the discrete-type KLT only for low-
30 frequency components that are acquired as a result of the DWT. As a result, the amount of calculation in the
47

SP305119WO00
orthogonal transform processing at the time of performing intra prediction is reduced, and the amount of calculation can be reduced even in a case where the block size is large. In addition, at the time of performing 5 the inverse orthogonal transform processing, a discrete-type inverse KLT is performed only for low-frequency components, and an inverse DWT of which the amount of calculation is less than that of an inverse KLT is performed, whereby the amount of calculation of the
10 inverse orthogonal transform processing can be similarly reduced. [0151]
In the current AVC mode, although a maximum block size of the intra prediction is 16 x 16, there is a high
15 possibility that a block size exceeding 16 x 16 is
employed in a next-generation standard. Accordingly, it is very useful to reduce the amount of calculation in the orthogonal transform processing at the time of performing an intra prediction. Since the amount of calculation in
20 the orthogonal transform processing is reduced, in a case where the intra prediction orthogonal transformation unit 64B is realized by hardware, the circuit scale of the LSI is reduced, whereby the manufacturing cost can be reduced. In addition, in a case where the intra prediction
25 orthogonal transformation unit 64B is realized by
software, the amount of processing is reduced, whereby reproduction of an image in real time can be performed more reliably. [0152]
30 [Configuration Example of Image Decoding Device]
Fig. 12 is a block diagram that illustrates a
48

^ SP305119WO00
♦
configuration example of an image decoding device
according to the first embodiment of the present
technology.
[0153] 5 The image decoding device 151 is configured by a
storage buffer 161, a lossless decoding unit 162, an
inverse quantization unit 163, an inverse orthogonal
transformation unit 164, a calculation unit 165, a
deblocking filter 166, a screen rearranging buffer 167, a 10 D/A conversion unit 168, a frame memory 169, a switch 170,
an intra prediction unit 171, a motion compensating unit
172, and a switch 173.
[0154]
The storage buffer 161 stores compressed 15 information that is transmitted from the image encoding
device 51. The lossless decoding unit 162 (decoding
unit) reads and acquires the compressed information from
the storage buffer 161 and performs lossless decoding of
the compressed information using a mode that corresponds 20 to the lossless encoding mode of the lossless encoding
unit 66 that is illustrated in Fig. 1. [0155]
More specifically, the lossless decoding unit 162
performs lossless decoding of the header information 25 included in the compressed information, thereby acquiring
intra prediction mode information, inter prediction mode
information, motion vector information, and the like. In
addition, the lossless decoding unit 162 performs
lossless decoding of a compressed image included in the 30 compressed information. In a case where the intra
prediction mode information is acquired, the lossless
■ 4 9

SP305119WO00
decoding unit 162 performs lossless decoding of the compressed image in the order according to an optimal intra prediction mode that is represented by the intra prediction mode information. 5 [0156]
In addition, the lossless decoding unit 162 supplies quantized transform coefficients that are acquired as a result of performing lossless decoding of the compressed image to the inverse quantization unit 163.
10 The lossless decoding unit 162 supplies the intra
prediction mode information that is acquired as a result of performing the lossless decoding to the inverse orthogonal transformation unit 164 and the intra prediction unit 171 and supplies the inter prediction
15 mode information, the motion vector information, and the like to the motion compensating unit 172. Furthermore, the lossless decoding unit 162 supplies the inter prediction mode information to the inverse orthogonal transformation unit 164.
20 [0157]
The inverse quantization unit 163 is configured similarly to the inverse quantization unit 68 illustrated in Fig. 1 and performs inverse quantization for quantized transform coefficients that are supplied from the
25 lossless decoding unit 16 using a mode that corresponds
to the quantization mode of the inverse quantization unit 65 illustrated in Fig. 1. The inverse quantization unit 163 supplies transform coefficients that are acquired as a result of the inverse quantization to the inverse
30 orthogonal transformation unit 164. [0158]
50

^ SP305119WO00
The inverse orthogonal transformation unit 164 is configured similarly to the inverse orthogonal transformation unit 69 illustrated in Fig. 1. The inverse orthogonal transformation unit 164 performs 5 inverse orthogonal transform processing for the transform coefficients that are supplied from the inverse quantization unit 163 in accordance with the inter prediction mode information or the intra prediction mode information that is supplied from the lossless decoding
10 unit 162. The inverse orthogonal transformation unit 164 supplies residual information that is acquired as a result of the orthogonal transform processing to the calculation unit 165. [0159]
15 The calculation unit 165 adds the residual information supplied from the inverse orthogonal transformation unit 164 to the predicted image supplied from the switch 173 as is needed and decodes the image. The calculation unit 165 supplies the decoded image to
20 the deblocking filter 166. [0160]
The deblocking filter 166 filters the decoded signal that is supplied from the calculation unit 165, thereby eliminating block distortion. The deblocking
25 filter 166 supplies an acquired resultant image to the frame memory 169 for storage and outputs the resultant image to the screen rearranging buffer 167. [0161]
The screen rearranging buffer 167 arranges images
30 supplied from the deblocking filter 166. More
specifically, the order of images that are arranged in
51

^ SP305119WO00
order for encoding by the screen rearranging buffer 62 illustrated m Fig. 1 is arranged in the order of the original display. The D/A conversion unit 168 performs D/A conversion of the images arranged by the screen 5 rearranging buffer 167 and outputs resultant images to a display not illustrated in the diagram for a display. [0162]
The switch 170 reads an image that is stored in the frame memory 169 as a reference image, outputs the read
10 image to the motion compensating unit 172, and supplies the read image to the intra prediction unit 171. [0163]
The intra prediction unit 171 performs an intra prediction process of an optimal intra prediction mode
15 that is represented by the intra prediction mode information based on the intra prediction mode information that is supplied from the lossless decoding unit 162, thereby generating a predicted image. The intra prediction unit 171 outputs the predicted image to
20 the switch 173. [0164]
The motion compensating unit 172 performs a motion compensating process for the reference image that is supplied through the switch 17 0 based on the inter
25 prediction mode information, the motion vector
information, and the like that are supplied from the lossless decoding unit 162, thereby generating a predicted image. The motion compensating unit 172 supplies the predicted image to the switch 173.
30 [0165]
The switch 173 selects the predicted image that is
52

SP305119WO00
generated by the motion compensating unit 172 or the intra prediction unit 171 and supplies the selected image to the calculation unit 165. [0166] 5 [Description of Process of Image Decoding Device]
Fig. 13 is a flowchart that illustrates a decoding ■process performed by the image decoding device 151. [0167]
In Step S131, the storage buffer 161 stores
10 compressed information that is transmitted from the image encoding device 51, In Step S132, the lossless decoding unit 162 reads the compressed information from the storage buffer 161 and performs a lossless decoding process for the compressed information. The lossless
15 decoding process performed in a case where the intra
I prediction mode information is acquired will be described
in detail with reference to Fig. 14 to be described below. [0168]
In addition, the lossless decoding unit 162
20 supplies quantized transform coefficients that are
acquired as a result of the lossless decoding process to the inverse quantization unit 163. The lossless decoding unit 162 supplies intra prediction mode information that is acquired as a result of the lossless decoding process
25 to the inverse orthogonal transformation unit 164 and the intra prediction unit 171 and supplies the inter prediction mode information, the motion vector information, and the like to the motion compensating unit 172. In addition, the lossless decoding unit 162
30 supplies the inter prediction mode information to the inverse orthogonal transformation unit 164.
53

SP305119WO00
[0169]
In Step S133, the inverse quantization unit 163 performs inverse quantization of the quantized transform coefficients that are supplied from the lossless decoding 5 unit 162 so as to have characteristics that correspond to the characteristics of the quantization unit 65 illustrated in Fig. 1. The inverse quantization unit 163 supplies the transform coefficients that are acquired as a result of the inverse quantization to the inverse
10 orthogonal transformation unit 164. [0170]
In Step S134, the inverse orthogonal transformation unit 164 performs inverse orthogonal transform processing, which is similar to the inverse orthogonal transform
15 processing of Step S17 illustrated in Fig. 7, of the
transform coefficients that are supplied from the inverse quantization unit 163. The inverse orthogonal transformation unit 164 supplies residual information that is acquired as a result of the inverse orthogonal
20 transform processing to the calculation unit 165. [0171]
In Step S135, the calculation unit 165 adds the predicted image that is input by the process of Step S141 to be described below and the residual information that
25 is supplied from the inverse orthogonal transformation unit 164 together as is needed and decodes the image. The calculation unit 165 supplies the decoded image to the deblocking filter 166. [0172]
30 In Step S136, the deblocking filter 166 filters the decoded image that is supplied from the calculation unit
54

'jdl^ SP305119WO00
165, thereby eliminating block distortion. The
deblocking filter 166 supplies an acquired resultant
image to the frame memory 169 and the screen rearranging
buffer 167. 5 [0173]
In Step S137, the frame memory 169 stores the image
that is supplied from the deblocking filter 166.
[0174]
In Step S138, the lossless decoding unit 162 10 determines whether or not the compressed image included
in the compressed information is an inter-predicted image,
in other words, whether or not the inter prediction mode
information has been acquired.
[0175] 15 In a case where it is determined that the
compressed image included in the compressed information
is an inter-predicted image in Step 3138, the lossless
decoding unit 162 supplies the inter prediction mode
information, the motion vector, and the like to the 20 motion compensating unit 172.
[0176]
Then, in Step S139, the motion compensating unit
172 performs a motion compensating process for the
reference image supplied through the switch 170 based on 25 the inter prediction mode information, the motion vector
information, and the like supplied from the lossless
decoding unit 162, thereby generating a predicted image.
The motion compensating unit 172 supplies the predicted
image to the switch 173. I
30 [0177]
On the other hand, in Step S138, in a case where it
■ 55

SP305119WO00
is determined that the compressed image included in the
compressed information is not an inter-predicted image,
in other words, in a case where it is determined that
intra prediction mode information has been acquired, the 5 lossless decoding unit 162 supplies the intra prediction
mode information to the intra prediction unit 171.
[0178]
Then, in Step S140, the intra prediction unit 171
performs an intra prediction process of an optimal intra 10 prediction mode that is represented by the intra
prediction mode information based on the intra prediction
mode information that is supplied from the lossless
decoding unit 162, thereby generating a predicted image.
The intra prediction unit 171 outputs the predicted image 15 to the switch 173.
[0179]
In Step S141, the switch 173 selects the predicted
image that is supplied from the motion compensating unit
172 or the intra prediction unit 171 and outputs the 20 selected image to the calculation unit 165. This
predicted image, as described above, is added to the
output of the inverse orthogonal transformation unit 164
in Step S135.
[0180] 25 In Step S142, the screen rearranging buffer 167
arranges images supplied from the deblocking filter 166.
[0181]
In Step S143, the D/A conversion unit 168 performs
D/A conversion of the images arranged by the screen 30 rearranging buffer 167 and outputs resultant images to a
display not illustrated in the diagram for a display.
56

^ SP305119WO00
[0182]
Fig, 14 is a flowchart that illustrates a lossless decoding process in a case where intra prediction mode information is acquired in Step S132 illustrated in Fig. 5 13.
[0183]
In Step S151, the lossless decoding unit 162 performs lossless decoding of coefficients, which are quantized and are losslessly encoded, included in the
10 compressed information that is supplied from the storage buffer 161 and supplies the coefficients to the inverse quantization unit 163. [0184]
In Step S152, the lossless decoding unit 162
15 performs lossless decoding of high-frequency components, which are quantized and losslessly encoded, included in the compressed information in the order according to an optimal intra prediction mode that is represented by the intra prediction information and supplies decoded high-
20 frequency components to the inverse quantization unit 163. In a case where a high-frequency component is not included in the compressed information, the process of Step S152 is not provided. [0185]
25 As above, the image decoding device 151 decodes the compressed information that is acquired from the encoding device 51, and, by performing a discrete-type inverse KLT only for low-frequency components and performing an inverse DWT of which the amount of calculation is less
30 than that of an inverse KLT, the inverse orthogonal
transform processing can be performed. As a result, the
57

^ SP305119WO00
amount of calculation performed in the inverse orthogonal transform processing is reduced, and accordingly, the amount of calculation can be reduced even in a case where the block size is large. 5 [0186]

[Configuration Example of Image Encoding Device]
Fig. 15 is a block diagram that illustrates a configuration example of an image encoding device
10 according to a second embodiment of the present technology. [0187]
In the configuration illustrated in Fig. 15, a same reference sign is assigned to the same configuration as
15 that illustrated in Fig. 1. Duplicate description will not be presented as is appropriate. [0188]
The configuration of the image encoding device 2 00 illustrated in Fig. 15 is mainly different from that
20 illustrated in Fig. 1 that an orthogonal transformation unit 201 is disposed instead of the orthogonal transformation unit 64, and an inverse orthogonal transformation unit 202 is disposed instead of the inverse orthogonal transformation unit 69. In the intra
25 prediction orthogonal transform processing, the image
encoding device 200 does not perform a separable-type KLT in all the intra prediction modes but performs a separable-type KLT in intra prediction modes of which the predicted directions are other than the oblique
30 directions and performs a non-separable type KLT in intra prediction modes of which the predicted directions are
58

^ SP305119WO00
other than the oblique directions. [0189]
More specifically, the orthogonal transformation unit 201 is configured by an inter prediction orthogonal 5 transformation unit 64A and an intra prediction
orthogonal transformation unit 201B. In a case where intra prediction mode information is supplied from a predicted image selecting unit 7 6, the intra prediction orthogonal transformation unit 201B performs orthogonal
10 transform processing on residual information supplied
from a calculation unit 63. More specifically, the intra prediction orthogonal transformation unit 201B extracts a high-frequency component and a low-frequency component by performing a DWT on the residual information. In
15 addition, the intra prediction orthogonal transformation unit 201B may change the niimber of DWTs to be performed based on whether a predicted direction of the intra prediction mode is an oblique direction or a direction other than the oblique direction.
20 [0190]
In a case where the predicted direction of the intra prediction mode is the oblique direction, the intra prediction orthogonal transformation unit 201B performs a non-separable type KLT of each intra prediction mode
25 information on the low-frequency component. On the other hand, in a case where the predicted direction of the intra prediction mode is a direction other than the oblique direction, the intra prediction orthogonal transformation unit 2 01B performs a separable-type KLT of
30 each intra prediction mode information on the low-frequency component. The intra prediction orthogonal
59

^ SP305119WO00
transformation unit 201B supplies coefficients that are acquired as a result of the KLT of an optimal intra prediction mode that is represented by the intra prediction mode information and the high-frequency 5 component to the quantization unit 65 as transform coefficients, [0191]
The inverse orthogonal transformation unit 2 02 is configured by an inter prediction inverse orthogonal
10 transformation unit 69A and an intra prediction inverse orthogonal transformation unit 202B. In a case where the intra prediction mode information is supplied from the predicted image selecting unit 7 6, the intra prediction inverse orthogonal transformation unit 202B performs
15 inverse orthogonal transform processing on transform coefficients that are supplied from the inverse quantization unit 68. More specifically, in a case where the predicted direction of the intra prediction mode is the oblique direction, the intra prediction inverse
20 orthogonal transformation unit 202B performs a non-separable type inverse KLT of each intra prediction mode on coefficients of the low-frequency component out of the transform coefficients. On the other hand, in a case where the predicted direction of the intra prediction
25 mode is a direction other than the oblique direction, the intra prediction inverse orthogonal transformation unit 2 02B performs a separable-type inverse KLT of each intra prediction mode on coefficients of the low-frequency component out of the transform coefficients.
30 [0192]
In addition, the intra prediction inverse
60

^ SP305119WO00
orthogonal transformation unit 202B performs an inverse DWT for a low frequency component that is acquired as a result of performing the inverse KLT of an optimal intra prediction mode that is represented by the intra 5 prediction mode information and a high-frequency component of the transform coefficients, thereby acquiring residual information. The intra prediction inverse orthogonal transformation unit 202B may change the number of inverse DWTs to be performed based on
10 whether a predicted direction of the optimal intra
prediction mode is an oblique direction or a direction other than the oblique direction. The intra prediction inverse orthogonal transformation unit 202B supplies the residual information to the calculation unit 70.
15 [0193]
[Example of Detailed Configuration of Intra Prediction Orthogonal Transformation Unit]
Fig. 16 is a block diagram that illustrates an example of a detailed configuration of the intra
20 prediction orthogonal transformation unit 201B that is illustrated in Fig. 15. [0194]
In the configuration illustrated in Fig. 16, a same reference sign is assigned to a same configuration as
25 that illustrated in Fig. 3. Repetitive description will not be presented as is appropriate. [0195]
The configuration of the intra prediction orthogonal transformation unit 201B illustrated in Fig.
30 16 is mainly different from the configuration illustrated in Fig. 3 in that a DWT unit 221 is newly disposed, KLT
61

^ SP305119WO00
units 222-3 to 222-8 are deposed instead of the KLT units 92-3 to 92-8, and a selector 223 is disposed instead of the selector 93. [0196] 5 In the intra prediction orthogonal transformation unit 201B illustrated in Fig. 16, a DWT unit 91 is a DWT unit that is used for intra prediction modes of numbers 0 to 2, and a low-frequency component that is output from the DWT unit 91 is supplied only to KLT units 92-0 to 92-
10 2.
[0197]
The DWT unit 221 is a DWT unit that is used for intra prediction modes of numbers 3 to 8, and the DWT unit 221 (prior orthogonal transformation unit) performs
15 a DWT on residual information that is supplied from the calculation unit 63 (Fig. 15) as is needed, thereby acquiring a low-frequency component and a high-frequency component. The number of DWTs performed by the DWT unit 221 and the number of DWTs performed by the DWT unit 91
20 may be the same or be different from each other. However, since the non-separable type KLT performed by the KLT units 222-3 to 222-8 has a more calculation amount than the separable-type KLT performed by the KLT units 92-0 to 92-2, in a case where the calculation amounts that can be
25 processed by both KLT units are the same, the DWT unit 221 needs to perform DWTs more than the DWT unit 91. [0198]
The DWT unit 221 supplies low-frequency components to the KLT units 222-3 to 222-8 and supplies high-. 30 frequency components to the selector 223. In addition, in a case where a KLT does not need to be performed, the
62

SP305119WO00 1^
DWT unit 221 supplies the residual information to the KLT units 222-3 to 222-8 without change. [0199]
The KLT units 222-3 to 222-8 (posterior orthogonal 5 transformation units) perform non-separable type KLTs on the low-frequency components or the residual information that is supplied from the DWT unit 221 using bases of the KLTs of the intra prediction modes of numbers 3 to 8. Here, the base of the KLT of each intra prediction mode
10 is an optimal value that is acquired through learning in advance. The KLT units 222-3 to 222-8 supply coefficients that are acquired as results of the non-separable type KLTs to the selector 223. [0200]
15 The intra prediction mode information supplied from the predicted image selecting unit 7 6 is supplied to the selector 223. The selector 223 selects a coefficient, which is supplied from one of the KLT units 92-0 to 92-2 and the KLT units 222-3 to 222-8 that corresponds to the
20 optimal intra prediction mode, out of coefficients
supplied from the KLT units 92-0 to 92-2 and the KLT units 222-3 to 222-8 based on the intra prediction mode information. Then, the selector 223 supplies the selected coefficient of the residual information to the
25 quantization unit 65 (Fig. 15). Alternatively, the selector 223 supplies the coefficients of the low-frequency component and high-frequency components supplied from the DWT unit 91 or the DWT unit 221 that corresponds to the optimal intra prediction mode to the
30 quantization unit 65 as transform coefficients. [0201]
63

^ SP305119WO00
As described above, the intra prediction orthogonal transformation unit 201B performs non-separable type KLTs as KLTs that correspond to the intra prediction modes of niambers 3 to 8 of which the predicted directions are 5 oblique directions, and accordingly, the encoding efficiency can be improved. [0202]
More specifically, in the separable-type KLT, a KLT is performed with a component being divided into the
10 horizontal direction and the vertical direction, and
accordingly, restriction is added to the base of the KLT more than the non-separable type KLT, whereby the .performance is degraded. More specifically, in a case where the predicted direction of the intra prediction
15 mode is an oblique direction, although residual
information having a strong correlation in an oblique direction may be easily generated, it is difficult to consider the correlation in the oblique direction in the separable-type KLT, whereby the performance is degraded.
20 In contrast to this, the intra prediction orthogonal
transformation unit 2 01B performs a non-separable type KLT as a KLT that corresponds to the intra prediction mode of which the predicted direction is an oblique direction, and accordingly, a correlation in the oblique
25 direction is considered, whereby the performance is improved. As a result, the encoding efficiency is improved. [0203]
In addition, in the case of an intra prediction
30 mode in which the predicted direction is a direction
other than an oblique direction, that is, the horizontal
64

^ SP305119WO00
direction or the vertical direction, residual information having a strong correlation in the horizontal direction or the vertical direction may be easily generated, whereby sufficient encoding efficiency can be realized 5 even in the separable-type KLT. Accordingly, in the case of an intra prediction mode in which the predicted direction is other than an oblique direction, the intra prediction orthogonal transformation unit 2 01B performs a separable-type KLT, thereby reducing the amount of 10 calculation. [0204]
Although the selector 223 is disposed after the KLT units 92-0 to 92-2 and the KLT units 222-3 to 222-8 in the intra prediction orthogonal transformation unit 201B 15 illustrated in Fig. 16, the selector may be disposed
before the KLT units 92-0 to 92-2 and the KLT units 222-3 to 222-8. In such a case, the low-frequency component or the residual information that is output from the DWT unit 91 or the DWT unit 221 is supplied to only one of the KLT 20 units 92-0 to 92-2 and the KLT units 222-3 to 222-8 that corresponds to the optimal intra prediction mode. [0205] [Description of DWT]
Fig. 17 is a diagram that illustrates the DWT 25 performed by the DWT unit 221, [0206]
In the example illustrated in Fig 17, the DWT unit 221 performs three DWTs. Here, A of Fig. 17 to C of Fig. 17 are the same as A of Fig. 4 to C of Fig. 4, and thus, 30 the description thereof will not be presented. [0207]
65

^ SP305119WO00
As illustrated in D of Fig. 17, when a DWT is performed for an LLLL component illustrated in C of Fig. 17 in the horizontal direction and the vertical direction, the horizontal component and the vertical component of 5 the LLLL component are respectively divided into a high-frequency component and a low-frequency component. As a result, the LLLL component is divided into a component (LLLLLL component) in which the horizontal component and the vertical component are low-frequency components, a
10 component (LLLLHL component) in which the horizontal
component is a high-frequency component and the vertical component is a low frequency component, a component (LLLLLH) in which the horizontal component is a low-frequency component and the vertical component is a high-
15 frequency component, and a component (LLLLHH component) in which the horizontal component and the vertical component are high-frequency components. The HL component, the LH component, the HH component, the LLHL component, the LLLH component, and the LLHH component
20 that are components of the residual information other than the LLLL component are unchanged. [0208]
For example, as illustrated in Fig. 4, in a case where the number of DWTs performed by the DWT unit 91 is
25 two, the DWT unit 221, as illustrated in Fig. 17,
performs three DWTs, thereby further decreasing the size
of the low-frequency component that is a target for a KLT.
[0209]
[Detailed Configuration Example of Intra Prediction
30 Inverse orthogonal transformation unit]
Fig. 18 is a block diagram that illustrates an
66

g^ SP305119WO00
example of a detailed configuration of the intra prediction inverse orthogonal transformation unit 202B that is illustrated in Fig. 15. [0210] 5 In the configuration illustrated in Fig. 18, a same reference sign is assigned to the same configuration as that illustrated in Fig. 5. Duplicate description will not be presented as is appropriate. [0211]
10 The configuration of the intra prediction inverse orthogonal transformation unit 2 02B illustrated in Fig. 18 is mainly different from the configuration illustrated in Fig. 5 that inverse KLT units 241-3 to 241-8 are disposed instead of the inverse KLT units 101-3 to 101-8,
15 selectors 242 and 243 are disposed instead of the selector 102, and an inverse DWT unit 244 is newly-disposed. [0212]
The inverse KLT units 241-3 to 241-8 of the intra
20 prediction inverse orthogonal transformation unit 202B
illustrated in Fig. 18 perform non-separable type inverse KLTs for coefficients of the transform coefficients supplied from the inverse quantization unit 68 illustrated in Fig. 15 using bases of the inverse KLTs of
25 the intra prediction modes of numbers 3 to 8. [0213]
The intra prediction mode information that is supplied from the predicted image selecting unit 7 6 is supplied to the selector 242. The selector 242 selects a
30 post-inverse KLT component, which is supplied from one of the inverse KLT units 101-0 to 101-2, corresponding to
67

SP305119WO00
m
the optimal intra prediction mode out of post-inverse KLT components supplied from the inverse KLT units 101-0 to 101-2 based on the intra prediction mode information. Then, the selector 242 supplies the selected post-inverse 5 KLT component to the inverse DWT unit 103. [0214]
The intra prediction mode information that is supplied from the predicted image selecting unit 76 is supplied to the selector 243. The selector 243.selects a
10 post-inverse KLT component, which is supplied from one of the inverse KLT units 241-3 to 241-8, corresponding to the optimal intra prediction mode out of post-inverse KLT components supplied from the inverse KLT units 241-3 to 241-8 based on the intra prediction mode information.
15 Then, the selector 243 supplies the selected post-inverse KLT component to the inverse DWT unit 244. [0215]
In the intra prediction inverse orthogonal transformation unit 202B, the inverse DWT unit 103 is an
20 inverse DWT unit used for intra prediction modes of
numbers 0 to 2, and a component that is acquired from one of the inverse KLT units 101-0 to 101-2 is supplied to the inverse DWT unit 103 through the selector 242. [0216]
25 The inverse DWT unit 244 is an inverse DWT unit used for intra prediction modes of numbers 3 to 8, and the inverse DWT unit 244 supplies a component that is supplied from one of the inverse KLT units 241-3 to 241-8 through the selector 243 to the calculation unit 70 (Fig.
30 15) as residual information. Alternatively, the inverse DWT unit 244 performs an inverse DWT that corresponds to
68

^ SP305119WO00
a DWT performed by the DWT unit 221 for the component supplied from the selector 243 and a high-frequency component of the transform coefficients supplied from the inverse quantization unit 68 and supplies an acquired 5 resultant component to the calculation unit 7 0 as the residual information. [0217]
In the intra prediction inverse orthogonal transformation unit 2 02B illustrated in Fig. 18, although
10 the selectors 242 and 243 are disposed after the inverse KLT units 101-0 to 101-2 and the inverse KLT units 241-3 to 241-8, the selector may be disposed before the inverse KLT units 101-0 to 101-2 and the inverse KLT units 241-3 to 241-8. In such a case, coefficients of the transform
15 coefficients supplied from the inverse quantization unit 68 are supplied only to one of the inverse KLT units 101-0 to 101-2 and the inverse KLT units 241-3 to 241-8 that corresponds to the optimal intra prediction mode. [0218]
20 [Description of Process of Image Encoding Device]
The encoding process of the image encoding device 200 illustrated in Fig. 15 is the same as the encoding process illustrated in Fig. 7 except for orthogonal transform processing performed by the intra prediction
25 orthogonal transformation unit 64B in Step S14
illustrated in Fig. 7 and inverse orthogonal transform
processing performed by the intra prediction inverse
orthogonal transformation unit 69B in Step S17. Thus,
only the orthogonal transform processing and the inverse I
30 orthogonal transform processing will be described. [0219]
69

SP305119WO00
Fig. 19 is a flowchart that illustrates orthogonal
transform processing performed by the intra prediction
orthogonal transformation unit 201B of the image encoding
device 200. I
5 [0220]
The processes of Steps S241 to S246 illustrated in Fig. 19 are the same as those of Steps S41 to S44 and S48 illustrated in Fig. 8 except that the processes of Steps S245 and S246 are performed by the KLT units 92-0 to 92-2. 10 [0221]
After the process of Step S245 or Step S246, the DWT unit 221 (Fig. 16) of the intra prediction orthogonal transformation unit 201B determines whether or not the size of the residual information supplied from the 15 calculation unit 63 (Fig. 15) is a size that can be
processed by the KLT units 222-3 to 222-8 in Step S247. [0222]
In a case where the size of the residual , information is determined not to be a size that can be 20 processed by the KLT units 222-3 to 222-8 in Step S247, the DWT unit 221 performs a DWT for the residual information in Step S248. [0223]
In Step S249, the DWT unit 221 determines whether 25 or not the size of a low-frequency component acquired as a result of the process of Step S248 is a size that can be processed by the KLT units 222-3 to 222-8. In a case where the size of the low-frequency component acquired as the result of the DWT is determined not to be a size that 30 can be processed by the KLT units 222-3 to 222-8 in Step S249, the DWT unit 221 performs a DWT for the low-
70,

^ SP305119WO00
frequency component acquired as the result of the process of Step S248 in Step S250. Then, the process is returned to Step S249, and the process of Steps S249 and S250 is repeated until the size of the low-frequency component 5 becomes a size that can be processed by the KLT units 222-3 to 222-8. [0224]
In a case where the size of the low-frequency component acquired as the result of the DWT is determined
10 to be a size that can be processed by the KLT units 222-3 to 222-8 in Step S249, the DWT unit 221 supplies the low-frequency component acquired as the result of the DWT to the KLT units 222-3 to 222-8 and supplies a high-frequency component to the selector 223.
15 [0225]
Then, in Step S252, the KLT units 222-3 to 222-8 perform non-separable type KLTs for the low-frequency component by using bases of the KLTs of the corresponding intra prediction modes. Then, the process proceeds to
20 Step S253. [0226]
On the other hand, in a case where the size of the residual information is determined to be a size that can be processed by the KLT units 222-3 to 222-8 in Step S247,
25 the DWT unit 221 supplies the residual information to the KLT units 222-3 to 222-8 without change. Then, in Step S251, the KLT units 222-3 to 222-8 perform non-separable type KLTs for the residual information by using the bases of the KLTs of the corresponding intra prediction modes,
30 and the process proceeds to Step S253. [0227]
71

SP305119WO00
In Step S253, similarly to the process of Step S46 illustrated in Fig. 8, the selector 223 selects a coefficient that is supplied from one of the KLT units 92-0 to 92-2 or the KLT units 222-3 to 222-8 that 5 corresponds to the optimal intra prediction mode out of the coefficients supplied from the KLT units 92-0 to 92-2 and the KLT units 222-3 to 222-8 based on the intra prediction mode information and outputs the selected coefficient to the quantization unit 65 (Fig. 15).
10 [0228]
In Step S254, the selector 223 determines whether or not the DWT unit 91 or the DWT unit 221 that corresponds to the optimal intra prediction mode has performed a DWT, in other words, whether or not a high-
15 frequency component has been supplied from the DWT unit 91 or the DWT unit 221 that corresponds to the optimal intra prediction mode. [0229]
In a case where it is determined that the DWT unit
20 91 or the DWT unit 221 that corresponds to the optimal intra prediction mode has performed a DWT in Step S254, the process proceeds to Step S255. In Step S255, the selector 223 outputs a high-frequency component supplied from the DWT unit 91 or the DWT unit 221 that corresponds
25 to the optimal intra prediction mode to the quantization unit 65 (Fig. 15) without change and ends the process. [0230]
On the other hand, in a case where it is determined that the DWT unit 91 or the DWT unit 221 that corresponds
30 to the optimal intra prediction mode has not performed a DWT in Step S254, the process immediately ends.
72

SP305119WO00 f
[0231]
Fig. 20 is a flowchart that illustrates inverse orthogonal transform processing of the intra prediction inverse orthogonal transformation unit 202B of the image 5 encoding device 200. [0232]
In Step S261, the inverse KLT units 101-0 to 101-2 (Fig. 18) of the intra prediction inverse orthogonal transformation unit 2 02B perform separable-type inverse
10 KLTs for the coefficients out of the transform
coefficients supplied from the inverse quantization unit 68 (Fig. 15) using bases of the KLTs of the intra prediction modes of numbers 0 to 2. [0233]
15 In Step S262, the inverse KLT units 241-3 to 241-8 perform non-separable type inverse KLTs for the coefficients out of the transform coefficients supplied from the inverse quantization unit 68 using bases of the KLTs of the intra prediction modes of numbers 3 to 8.
20 [0234]
In Step S263, each of the selectors 242 and 243 selects a post-inverse KLT component supplied from one of the inverse KLT units 101-0 to 101-2 or the inverse KLT units 241-3 to 241-8 that corresponds to the optimal
25 intra prediction mode out of the post-inverse KLT
components supplied from the inverse KLT units 101-0 to 101-2 or the inverse KLT units 241-3 to 241-8 based on the intra prediction mode information supplied from the predicted image selecting unit 76. Then, the selectors
30 242 and 243 supply the selected post-inverse KLT
components to the inverse DWT unit 103 or the inverse DWT
73

SP305119WO00
unit 244, and the process proceeds to Step S264. The subsequent process is performed by the inverse DWT unit 103 or the inverse DWT unit 244 to which the post-inverse KLT component is supplied. 5 [0235]
In Step S264, the inverse DWT unit 103 or the inverse DWT unit 244 determines whether or not the size of the post-inverse KLT component supplied from the selector 242 or 243 is the block size. In a case where
10 the size of the post-inverse KLT component is determined not to be the block size in Step S264, the process proceeds to Step S265. In Step S265, the inverse DWT unit 103 performs an inverse DWT for high-frequency components out of the post-inverse KLT component and the
15 transform coefficients supplied from the inverse quantization unit 68. [0236]
In Step S266, the inverse DWT unit 103 or the inverse DWT unit 244 determines whether or not the size
20 of the post-inverse DWT component that is acquired by as a result of the process of Step S264 is the block size. In a case where the size of the post-inverse DWT component is determined not to be the block size in Step S265, the process is returned to Step S265, and the
25 processes of Steps S265 and S266 are repeated until the size of the post-inverse DWT component becomes the block size. [0237]
In a case where the size of the post-inverse DWT
30 component is determined to be the block size in Step S266, the process proceeds to Step S2 67.
74

SP305119WO00 »
[0238]
On the other hand, in a case where the size of the post-inverse KLT component is determined to be the block, size in Step S264, the process proceeds to Step S267. 5 [0239]
In Step S2 67, the inverse DWT unit 103 or the inverse DWT unit 244 supplies the post-inverse DWT component or the post-inverse KLT component to the calculation unit 7 0 as the residual information. 10 [0240]
[Configuration Example of Image Decoding Device]
Fig. 21 is a block diagram that illustrates a configuration example of an image decoding device according to a second embodiment of the present 15 technology. [0241]
In the configuration illustrated in Fig. 21, a same reference sign is assigned to the same configuration as that illustrated in Fig. 12. Duplicate description will 20 not be presented as is appropriate. [0242]
The configuration of the image decoding device 300 illustrated in Fig. 21 is mainly different from the configuration illustrated in Fig. 12 that an inverse 25 orthogonal transformation unit 301 is disposed instead of the inverse orthogonal transformation unit 164. [0243]
The inverse orthogonal transformation unit 301 of the image decoding device 300 is configured similarly to 30 the inverse orthogonal transformation unit 2 02 illustrated in Fig. 15. The inverse orthogonal
75

SP305119WO00 #
transformation unit 301 performs inverse orthogonal transform processing for transform coefficients supplied from the inverse quantization unit 163 in accordance with the inter prediction mode information or the intra 5 prediction mode information supplied from the lossless decoding unit 162. The inverse orthogonal transformation unit 301 supplies the residual information acquired as a result of the inverse orthogonal transform processing to the calculation unit 165.
10 [0244]
[Description of Process of Image Decoding Device]
The decoding process of the image decoding device 300 illustrated in Fig. 21 is the same as the decoding process illustrated in Fig. 13 except for inverse
15 orthogonal transform processing of Step S134 illustrated in Fig. 13, and thus, description of processes other than the inverse orthogonal transform process will not be presented. In the inverse orthogonal transform processing of the image decoding device 300, in a case
2 0 where intra prediction information is supplied from the lossless decoding unit 162, inverse orthogonal transform processing similar to the inverse orthogonal transform processing illustrated in Fig. 20 is performed, and, in a case where inter prediction information is supplied, an
25 inverse orthogonal transform is performed for the transform coefficients supplied from the inverse quantization unit 163. [0245]
In the description presented above, although the
30 predicted directions of the intra prediction modes of numbers 3 to 8 are assumed to be oblique directions, it
76

SP305119WO00
may be assiamed that the predicted directions of the intra prediction modes of numbers 5 and 7 are the vertical direction, and the predicted directions of the intra prediction modes of numbers 6 and 8 are the horizontal 5 direction. [0246]
In the description presented above, two types of 4 X 4 pixels and 8x8 pixels are assumed as the block sizes of the intra prediction, other block sizes such as
10 16 x 16 pixels, 32 x 32 pixels, 64 x 64 pixels, and 128 x 128 pixels may be handled. However, in such a case, since the number of the intra prediction modes changes, the number of the KLT units or the inverse KLT units changes. For example, in a case where the block size is
15 16 X 16 pixels, there are four types of intra prediction modes, and the number of the KLT units or the inverse KLT units is four. [0247]
In the description presented above, although the
20 DWT is performed before the KLT, the orthogonal transform that is performed before the KLT may be any type of orthogonal transform, of which the calculation amount is less than that of the KLT, for example, for which a high¬speed algorithm can be arranged.
25 [0248]
In the present technology, the size of the macro block is arbitrary. The present technology may be applied to a macro block having any size, for example, as illustrated in Fig. 22. For example, the present
30 technology may be applied not only to a macro block of an ordinary 16 x 16 pixels but also to a macro block
77

SP305119WO00
(expanded macro block) that is expanded as 32 x 32 pixels.
[0249]

[Description of Computer to Which Present Technology Is 5 Applied]
Next, the encoding process or the decoding process
described above may be performed by hardware or software.
In a case where the encoding process or the decoding
process is performed by the software, a program that 10 configures the software is installed to a general-purpose
computer or the like.
[0250]
Fig. 23 illustrates a configuration example of a
computer, to which the program that executes a series of 15 processes described above is installed, according to an
embodiment.
[0251]
The program may be recorded in a storage unit 408
or a read only memory (ROM) 402 in advance as a recording 20 medium that is built in the computer.
[0252]
Alternatively, the program may be stored (recorded)
on a removable medium 411. Such a removable medium 411
may be provided as so-called package software. Here, 25 examples of the removable mediiom 411 include a flexible
disk, a compact disc read only memory (CD-ROM), a magneto
optical (MO) disk, a digital versatile disc (DVD), a
magnetic disk, and a semiconductor memory.
[0253] 30 In addition, instead of installing the program to
the computer from a removable medium 411 as described
78

^ SP305119WO00
above through a drive 410, the program may be downloaded into the computer through a communication network or a broadcast network and be installed to the storage unit 408 that is built therein. In other words, the program 5 may be transmitted to the computer in a wired manner, for example, from a download site through a satellite used for digital satellite broadcasting or be transmitted to the computer in a wired manner through a network such as a local area network (LAN) or the Internet.
10 [0254]
The computer includes a central processing unit (CPU) 401 therein, and an input/output interface 405 is connected to the CPU 401 through a bus 404. [0255]
15 When an instruction is input from a user through
the input/output interface 405 by operating an input unit 406, the CPU 401 executes a program that is stored in the ROM 402 in accordance with the instruction. Alternatively, the CPU 401 loads a program that is stored
20 in the storage unit 408 into a random access memory (RAM) 403 and executes the program. [0256]
Accordingly, the CPU 401 performs the process according to the above-described flowchart or the process
25 that is performed by the configuration of the above-described block diagram. Then, the CPU 4 01 outputs a processing result from an output unit 407, for example, through the input/output interface 405, transmits the processing result from a communication unit 409, or
30 records the processing result in the storage unit 408 as is needed.
79

A SP305119WO00
[0257]
Here, the input unit 40 6 is configured by a keyboard, a mouse, a microphone, and the like. In addition, the output unit 407 is configured by a liquid 5 crystal display (LCD), a speaker, and the like. [0258]
Here, in this specification, the process that is performed by a computer in accordance with a program does not need to be performed necessarily in a time series in
10 accordance with the sequence described in the flowchart. In other words, the process that is performed by the computer in accordance with the program includes a process (for example, a parallel process or a process using an object) that is performed in a parallel manner
15 or in an individual manner. [0259]
In addition, the program may be processed by one computer (processor) or may be processed by a plurality of computers in a distributed manner. Furthermore, the
20 program may be transmitted to a remote computer and be executed. [0260]
[Configuration Example of Television Receiver]
25 Fig. 24 is a block diagram that illustrates an
example of a main configuration of a television receiver that uses the image decoding device according to the present technology. [0261]
30 The television receiver 500 illustrated in Fig. 24 includes a terrestrial tuner 513, a video decoder 515, a
80

SP305119WO00
video signal processing circuit 518, a graphic generating circuit 519, a panel driving circuit 520, and a display panel 521. [0262] 5 The terrestrial tuner 513 receives a broadcasting signal of terrestrial analog broadcasting through an antenna and demodulates the received broadcasting signal so as to acquire a video signal and supplies the video signal to the video decoder 515. The video decoder 515
10 performs a decoding process for the video signal that is supplied from the terrestrial tuner 513 and supplies acquired digital component signals to the video signal processing circuit 518. [0263]
15 The video signal processing circuit 518 performs a predetermined process such as a noise eliminating process for the video data that is supplied from the video decoder 515 and supplies acquired video data to the graphic generating circuit 519.
20 [0264]
The graphic generating circuit 519 generates video data of a program to be displayed on the display panel 521, image data according to a process that is based on an application supplied through a network, and the like
25 and supplies the video data or the image data that has been generated to the panel driving circuit 520. In addition, the graphic generating circuit 519 appropriately performs a process of generating video data (graphics) used for displaying a screen that is used by a
30 user for selecting an item or the like and supplying video data that is acquired by superimposing the
81

-
SP305119WO00
generated video data on the video data of the program to
the panel driving circuit 52 0.
[0265]
The panel driving circuit 520 drives the display 5 panel 521 based on data that is supplied from the graphic generating circuit 519, thereby displaying the video of the program or various screens described above on the display panel 521. [0266]
10 The display panel 521 is formed by a liquid crystal display (LCD) or the like and displays a video of a program or the like under the control of the panel driving circuit 52 0. [0267]
15 In addition, the television receiver 500 further
includes an audio analog/digital (A/D) conversion circuit 514, an audio signal processing circuit 522, an echo cancellation/audio synthesizing circuit 523, an audio amplifying circuit 524, and a speaker 525.
20 [0268]
The terrestrial tuner 513 acquires not only a video signal but also an audio signal by demodulating the received broadcasting signal. The terrestrial tuner 513 supplies the acquired audio signal to the audio A/D
25 conversion circuit 514. [0269]
The audio A/D conversion circuit 514 performs an A/D conversion process for the audio signal supplied from the terrestrial tuner 513 and supplies an acquired
30 digital audio signal to the audio signal processing circuit 522.
82

SP305119WO00
#
[0270]
The audio signal processing circuit 522 performs a predetermined process such as a noise eliminating process for the audio data that is supplied from the audio A/D 5 conversion circuit 514 and supplies acquired audio data to the echo cancellation/audio synthesizing circuit 523. [0271]
The echo cancellation/audio synthesizing circuit 523 supplies the audio data that is supplied from the 10 audio signal processing circuit 522 to the audio amplifying circuit 524. [0272]
The audio amplifying circuit 524 performs a D/A conversion process and an amplifying process for the 15 audio data that is supplied from the echo
cancellation/audio synthesizing circuit 523, adjusts resultant data to predetermined volume, and then outputs audio from the speaker 525. [0273] 20 Furthermore, the television receiver 500 includes a digital tuner 516 and an MPEG decoder 517.' [0274]
■
The digital tuner 516 receives a broadcasting signal of digital broadcasting (terrestrial digital
25 broadcasting or broadcasting satellite
(BS)/communications satellite (CS) digital broadcasting) through an antenna and demodulates the received broadcasting signal so as to acquire a moving picture experts group-transport stream (MPEG-TS) and supplies the
30 MPEG-TS to the MPEG decoder 517. [0275]
83

SP305119WO00
The MPEG decoder 517 dissolves the scramble that is applied to the MPEG-TS supplied from the digital tuner 516 and extracts a stream including data of a program that is a reproduction target (viewing target). The MPEG 5 decoder 517 decodes an audio packet that configures the extracted stream, supplies acquired audio data to the audio signal processing circuit 522, decodes a video packet that configures the stream, and supplies acquired video data to the video signal processing circuit 518.
10 In addition, the MPEG decoder 517 supplies electronic program guide (EPG) data extracted from the MPEG-TS to the CPU 532 through a path not illustrated in the diagram. [0276]
The television receiver 500 uses the above-
15 described image decoding device 151 (300) as the MPEG decoder 517 that decodes a video packet as described above. Accordingly, similarly to the case of the image decoding device 151 (300), the MPEG decoder 517 can perform inverse orthogonal transform processing with a
20 small amount of calculation. [0277]
A predetermined process is performed for the video data that is supplied from the MPEG decoder 517, similarly to the video data supplied from the video
25 decoder 515, by the video signal processing circuit 518. Then, the video data generated by the graphic generating circuit 519 and the like appropriately overlap the video data for which the predetermined process has been performed, and resultant video data is supplied to the
30 display panel 521 through the panel driving circuit 520, whereby an image thereof is displayed.
84

SP305119WO00
[0278]
A predetermined process is performed for the audio data that is supplied from the MPEG decoder 517, similarly to the audio data supplied from the audio A/D 5 conversion circuit 514, by the audio signal processing circuit 522. Then, the audio data for which the predetermined process has been performed is supplied to the audio amplifying circuit 524 through the echo cancellation/audio synthesizing circuit 523, and a D/A
10 conversion process and an amplifying process are performed for the audio data. As a result, audio adjusted to predetermined volume is output from the speaker 525. [0279]
15 In addition, the television receiver 500 includes a microphone 526 and an A/D conversion circuit 527. [0280]
The A/D conversion circuit 527 receives a user's audio signal that is received by the microphone 52 6 that
20 is set for audio and is disposed in the television
receiver 500. The A/D conversion circuit 527 performs an A/D conversion process for the received audio signal and supplies acquired digital audio data to the echo cancellation/audio synthesizing circuit 523.
25 [0281]
In a case where audio data of a user (user A) using the television receiver 500 is supplied from the A/D conversion circuit 527, the echo cancellation/audio synthesizing circuit 523 performs echo cancellation for
30 the audio data of user A as a target. Then, after the echo cancellation, the echo cancellation/audio
85

SP305119WO00
•
synthesizing circuit 523 outputs audio data that is acquired by a process of being composed with other audio data or the like from the speaker 525 through the audio amplifying circuit 524. 5 [0282]
Furthermore, the television receiver 500 also includes an audio codec 52 8, an internal bus 52 9, a synchronous dynamic random access memory (SDRAM) 530, a flash memory 531, a CPU 532, a universal serial bus (USB)
10 I/F 533, and a network I/F 534. [0283]
The A/D conversion circuit 527 receives a user's audio signal that is received by the microphone 52 6 that is set for audio and is disposed in the television
15 receiver 500. The A/D conversion circuit 527 performs an A/D conversion process for the received audio signal and supplies acquired digital audio data to the audio codec 528. [0284]
20 The audio codec 528 converts the audio data that is supplied from the A/D conversion circuit 527 into data of a predetermined format for transmission through a network and supplies the converted data to the network I/F 534 through the internal bus 52 9.
25 [0285]
The network I/F 534 is connected to the network through a cable that is installed to a network terminal 535. The network I/F 534, for example, transmits the audio data that is supplied from the audio codec 528 to
30 another device that is connected to the network. In
addition, the network I/F 534, for example, receives the
86

SP305119WO00
audio data transmitted from the another device that is connected thereto through the network through the network terminal 535 and supplies the received audio data to the audio codec 528 through the internal bus 52 9. 5 [0286]
The audio codec 52 8 converts the audio data supplied from the network I/F 534 into data of a predetermined format and supplies the converted data to the echo cancellation/audio synthesizing circuit 523.
10 [0287]
The echo cancellation/audio synthesizing circuit 523 performs echo cancellation for the audio data supplied from the audio codec 528 as a target and outputs audio data that is acquired by a process of being
15 composed with other audio data or the like from the speaker 525 through the audio amplifying circuit 524. [0288]
The SDRAM 530 stores various kinds of data that is necessary for the CPU 532 to perform the process.
20 [0289]
The flash memory 531 stores a program that is executed by the CPU 532. The program that is stored in the flash memory 531 is read by the CPU 532 at predetermined timing such as at the time of start-up of
25 the television receiver 500. In the flash memory 531, the EPG data that is acquired through digital broadcasting, data that is acquired from a predetermined server through the network, and the like are also stored. [0290]
30. For example, in the flash memory 531, MPEG-TS that includes content data acquired from a predetermined
87

SP305119WO00
m
server through the network under the control of the CPU 532 is stored. The flash memory 531 supplies the MPEG-TS to the MPEG decoder 517 through the internal bus 529, for example, under the control of the CPU 532. 5 [0291]
The MPEG decoder 517, similarly to the MPEG-TS that is supplied from the digital tuner 516, processes the MPEG-TS. As above, the television receiver 500 can receive content data that is formed by a video, an audio, 10 and the like through the network, decode the content data using the MPEG decoder 517, and display the video or output the audio. [0292]
In addition, the television receiver 500 also 15 includes a light receiving unit 537 that receives an
infrared signal transmitted from a remote controller 551. [0293]
The light receiving unit 537 receives infrared rays transmitted from the remote controller 551 and outputs a 20 control code that represents the content of a user
operation that is acquired through demodulation to the
CPU 532.
[0294]
The CPU 532 executes the program that is stored in 25 the flash memory 531 and controls the entire operation of the television receiver 500 in accordance with a control code supplied from the light receiving unit 537 and the like. The CPU 532 and each unit of the television receiver 500 are connected to each other through a path 30 not illustrated in the diagram. [0295]
88

SP305119WO00
The USB I/F 533 transmits and receives data between the television receiver 500 and an external device that is connected through a USB cable that is installed to the USB terminal 536. The network I/F 534 is connected to 5 the network through a cable that is installed to the network terminal 535 and transmits and receives data other than audio data to or from various devices connected to the network as well. [0296]
10 By using the image decoding device 151 (300) as the MPEG decoder 517, the television receiver 500 can reduce the calculation amount of the inverse orthogonal transform processing. As a result, the television receiver 500 can acquire a decoded image with a smaller
15 calculation amount based on a broadcasting signal
received through the antenna or content data acquired
through the network and display the decoded image.
[0297]

2 0 [Configuration Example of Cellular Phone]
Fig. 25 is a block diagram that illustrates an example of a main configuration of a cellular phone that uses the image encoding device and the image decoding device according to the present technology.
25 [0298]
The cellular phone 600 illustrated in Fig. 25 includes a main control unit 650 that is configured so as to perform overall control of each unit, a power supply circuit unit 651, an operation input control unit 652, an
30 image encoder 653, a camera I/F unit 654, an LCD control
unit 655, an image decoder 656, a demultiplexing unit 657,
89

SP305119WO00
a recording/reproducing unit 662, a
modulation/demodulation circuit unit 658, and an audio codec 659. These are interconnected through a bus 660. [0299] 5 In addition, the cellular phone 600 includes
operation keys 619, a charge coupled devices (CCD) camera 616, a liquid crystal display 618, a storage unit 623, a transmission/reception circuit unit 663, an antenna 614, a microphone (mike) 621, and a speaker 617.
10 [0300]
When a call-end and power key is turned on by a user's operation, the power supply circuit unit 651 supplies power from a battery pack to each unit, thereby starting up the cellular phone 600 to be in an operable
15 state. [0301]
The cellular phone 600 performs various operations such as transmission and reception of an audio signal, transmission and reception of an electronic mail or image
2 0 data, image capturing, and data recording in various
modes such as a phone call mode and a data communication mode under the control of the main control unit 650 that is formed by a CPU, a ROM, a RAM, and the like. [0302]
25 For example, in the phone call mode, the cellular phone 600 converts an audio signal that is collected by the microphone (mike) 621 into digital audio data using the audio codec 659, performs a spread-spectrum process for the digital audio data using the
30 modulation/demodulation circuit unit 658, and performs a digital-to-analog conversion process and a frequency
90

SP305119WO00
converting process using the transmission/reception circuit unit 663. The cellular phone 600 transmits a transmission signal that is acquired by the conversion processes to a base station not illustrated in the 5 diagram through the antenna 614. The transmission signal (audio signal) transmitted to the base station is supplied to a cellular phone of a phone call opponent through a public telephone line network. [0303]
10 In addition, in the phone call mode, the cellular phone 600 amplifies a reception signal received using the antenna 614 by using the transmission/reception circuit unit 663, further performs a frequency conversion process and an analog-to-digital conversion process for the
15 reception signal, performs a spread-spectriom process
using the modulation/demodulation circuit unit 658, and converts a resultant signal into an analog audio signal using the audio codec 659. The cellular phone 600 outputs the analog audio signal acquired through the
20 conversions from the speaker 617. [0304]
Furthermore, for example, in a case where an electronic mail is transmitted in the data communication mode, the cellular phone 600 receives text data of an
25 electric mail that is input by operating the operation
keys 619 using the operation input control unit 652. The cellular phone 600 processes the text data using the main control unit 650 and displays the processed text data as an image on the liquid crystal display 618 through the
30 LCD control unit 655. [0305]
91
■

SP305119WO00 «
In addition, the cellular phone 600 generates electronic mail data based on text data, a user instruction, and the like received by the operation input control unit 652 using the main control unit 650. The 5 cellular phone 600 performs spread-spectrum process for the electronic mail data using the
modulation/demodulation circuit unit 658 and performs a digital-to-analog conversion process and a frequency conversion process for the electronic mail data using the
10 transmission/reception circuit unit 663. The cellular phone 600 transmits a transmission signal that is acquired by the conversion processes to a base station not illustrated in the diagram through the antenna 614. The transmission signal (electronic mail) transmitted to
15 the base station is supplied to a predetermined
destination through the network, a mail server, and the
like.
[0306]
Furthermore, for example, in a case where an
20 electronic mail is received in the data communication mode, the cellular phone 600 receives a signal transmitted from the base station through the antenna 614 using the transmission/reception circuit unit 663, amplifies the signal, and further performs a frequency
25 conversion process and an analog-to-digital conversion process for the signal. The cellular phone 600 performs a spread-spectrum process for the reception signal using the modulation/demodulation circuit unit 658, thereby restoring the original electronic mail data. The
30 cellular phone 600 displays the restored electronic mail data on the liquid crystal display 618 through the LCD
92

SP305119WO00 #
control unit 655. [0307]
In addition, the cellular phone 600 may record (store) the received electronic mail data in the storage 5 unit 623 through the recording/reproducing unit 662. [0308]
The storage unit 623 is an arbitrary rewritable storage medium. The storage unit 623, for example, may be a semiconductor memory such as a RAM or a built-in
10 type flash memory, a hard disk, or a removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card. It is apparent that the storage unit 623 may be any other type. [0309]
15 In addition, for example, in a case where image data is transmitted in the data communication mode, the cellular phone 600 generates image data using the CCD camera 616 through imaging. The CCD camera 616 includes optical devices such as a lens and a diaphragm and CCDs
20 as photoelectric conversion devices, images a subject, and converts the intensity of received light into an electrical signal, thereby generating image data of an image of the subject. The image data is converted into encoded image data by performing compression and encoding
25 of the image data using a predetermined coding mode such as MPEG2 or MPEG4 by using the image encoder 653 through the camera I/F unit 654. [0310]
The cellular phone 60 0 uses the above-described
30 image encoding device 51 (200) as the image encoder 653 that performs such a process. Accordingly, the image
93

SP305119WO00 «
encoder 653, similarly to the case of the image encoding device 51 (200), can perform the orthogonal transform processing and the inverse orthogonal transform processing with a smaller calculation amount. 5 [0311]
In addition, at the same time, the cellular phone 600 performs analog-to-digital conversion of audio collected using the microphone (mike) 621 while performing image capturing using the CCD camera 616 by
10 using the audio codec 659 and further encodes the audio. [0312]
The cellular phone 600 multiplexes the encoded image data supplied from the image encoder 653 and the digital audio data supplied from the audio codec 659 in a
15 predetermined mode in the demultiplexing unit 657. The
cellular phone 600 performs a spread-spectrum process for the acquired resultant multiplexed data using the modulation/demodulation circuit unit 658 and performs a digital-to-analog conversion process and a frequency
20 conversion process for resultant data using the
transmission/reception circuit unit 663. The cellular phone 600 transmits a transmission signal that is acquired by the conversion processes to a base station not illustrated in the diagram through the antenna 614.
25 The transmission signal (image data) transmitted to the base station is supplied to a communication opponent through a network or the like. [0313]
In a case where the image data is not transmitted,
30 the cellular phone 600 may display image data generated by the CCD camera 616 on the liquid crystal display 618
94

SP305119WO00
not through the image encoder 653 but through the LCD
control unit 655.
[0314]
In addition, for example, in a case where data of a 5 moving image file linked from a simple home page or the like is received in the data communication mode, the cellular phone 600 receives a signal transmitted from the base station using the transmission/reception circuit unit 663 through the antenna 614, amplifies the received
10 signal, and further performs a frequency conversion
process and an analog-to-digital conversion process for the received signal. The cellular phone 600 performs a spread-spectrum process for the received signal using the modulation/demodulation circuit unit 658, thereby
15 restoring the original multiplexed data. The cellular phone 600 separates the multiplexed data using the demultiplexing unit 657 so as to be divided into encoded image data and audio data. [0315]
20 The cellular phone 600 generates reproduction
moving image data by decoding the encoded image data in a decoding mode that corresponds to a predetermined encoding mode such as MPEG2 or MPEG4 by using the image decoder 656 and displays the reproduction moving image
25 data on the liquid crystal display 618 through the LCD control unit 655. Accordingly, for example, the moving image data included in the moving image file linked from the simple home page is displayed on the liquid crystal display 618.
30 [0316]
The cellular phone 600 uses the above-described
95

SP305119WO00
image decoding device 151 (300) as the image decoder 656 that performs such a process. Accordingly, similarly to the case of the image decoding device 151 (300), the image decoder 656 can perform the inverse orthogonal 5 transform processing with a smaller calculation amount. [0317]
At this time, the cellular phone 600, at the same time, converts the digital audio data into an analog audio signal using the audio codec 659 and outputs the
10 analog audio signal from the speaker 617. Accordingly, for example, the audio data included in the moving image file that is linked from a simple home page is reproduced. [0318]
In addition, similarly to the case of the
15 electronic mail, the cellular phone 600 may record
(store) the data linked from a received simple home page or the like in the storage unit 623 through the recording/reproducing unit 662. [0319]
20 The cellular phone 600 may acquire information
recorded in a two-dimensional code by analyzing the two-dimensional code acquired by using the CCD camera 616 through imaging by using the main control unit 650. [0320]
25 In addition, the cellular phone 600 may communicate with an external device through infrared rays using the infrared communication unit 681. [0321]
By using the image encoding device 51 (200) as the
30 image encoder 653, the cellular phone 600 can perform the orthogonal transform processing and the inverse
96

SP305119WO00
orthogonal transform processing with a smaller
calculation amount. As a result, the cellular phone 600
can provide another device with encoded data (image data)
that can be decoded with a smaller calculation amount. 5 [0322]
In addition, by using the image decoding device 151 I
(300) as the image decoder 656, the cellular phone 600
can perform the inverse orthogonal transform processing
with a smaller calculation amount. As a result, the 10 cellular phone 600 can acquire a decoded image with a
smaller calculation amount from a moving image file that
is linked from a simple home page and display the decoded
image.
[0323] 15 In the description presented above, although the
cellular phone 600 has been described to use the CCD
camera 616, an image sensor (CMOS image sensor) using a
complementary metal oxide semiconductor (CMOS) instead of
the CCD camera 616 may be used. Also in such a case, 20 similarly to the case of using the CCD camera 616, the
cellular phone 600 can generate image data of an image of
a subject by imaging the subject.
[0324]
In the description presented above, although the 25 cellular phone 600 has been described, for example, the
image encoding device 51 (200) and the image decoding
device 151 (300) can be applied to any device, similarly
to the case of the cellular phone 600 as long as the
device has an imaging function and a communication 30 function that are the same as those of the cellular phone
600 such as a personal digital assistants (PDA), a smart
97

SP305119WO00 #
phone, an ultra mobile personal computer (UMPC), a netbook, or a notebook personal computer. [0325]
5 [Configuration Example of Hard Disk Recorder]
Fig. 2 6 is a block diagram that illustrates an example of a main configuration of a hard disk recorder that uses the image encoding device and the image decoding device according to the present technology.
10 [0326]
The hard disk recorder (HDD recorder) 700 illustrated in Fig. 2 6 is a device that stores audio data and video data of a broadcasting program included in a broadcasting signal (television signal), which is
15 received by a tuner,', transmitted from a satellite, a
terrestrial antenna, or the like in a hard disk that is built therein and provides a user with the stored data at timing according to a user's instruction. [0327]
2 0 The hard disk recorder 700, for example, may
extract audio data and video data from a broadcasting signal, appropriately decode the audio data and the video data, and store the audio data and the video data that have been decoded in the hard disk that is built therein.
25 In addition, the hard disk recorder 700, for example, may acquire audio data and video data from another device through a network, appropriately decode the audio data and the video data, and store the audio data and the video data that have been decoded in the hard disk that
30 is built therein. [0328]
98

SP305119WO00
In addition, the hard disk recorder 700, for example, decodes the audio data or the video data that is recorded in the built-in hard disk, supplies the audio data or the video data that has been decoded to a monitor 5 7 60, and displays an image thereof on the screen of the monitor 760. Furthermore, the hard disk recorder 700 may output audio from a speaker of the monitor 7 60. [0329]
The hard disk recorder 700, for example, decodes
10 audio data or video data that is extracted from a
broadcasting signal acquired through a tuner or audio data or video data acquired from another device through the network and supplies the audio data or the video data that has been decoded to the monitor 7 60 so as to display
15 the image on the screen of the monitor 7 60. In addition, the hard disk recorder 700 may output the audio from the speaker of the monitor 7 60. [0330]
It is apparent that other operations can be
20 performed. [0331]
As illustrated in Fig. 26, the hard disk recorder 700 includes a reception unit 721, a demodulation unit 722, a demultiplexer 723, an audio decoder 724, a video
25 decoder 725, and a recorder control unit 72 6. In
addition, the hard disk recorder 7 00 includes an EPG data memory 727, a program memory 72 8, a work memory 729, a display converter 730, an on screen display (OSD) control unit 731, a display control unit 732, a
30 recording/reproducing unit 733, a D/A converter 734, and a communication unit 735.
99
■

SP305119WO00
[0332]
The display converter 730 includes a video encoder 741. The recording/reproducing unit 733 includes an encoder 751 and a decoder 752. 5 [0333]
The reception unit 721 receives an infrared signal transmitted from a remote controller (not illustrated in the diagram), converts the infrared signal into an electrical signal, and outputs the electrical signal to
10 the recorder control unit 726. The recorder control unit 726, for example, is configured by a microprocessor or the like and performs various processes in accordance with a program that is stored in the program memory 728. The recorder control unit 72 6, at this time, uses the
15 work memory 729 as is necessary. [0334]
The communication unit 735 is connected to a network and performs a process of communicating with other devices through the network. For example, the
20 communication unit 735 communicates with a tuner (not illustrated in the diagram) under the control of the recorder control unit 72 6 and outputs a channel selection signal mainly to the tuner. [0335]
25 The demodulation unit 722 demodulates a signal supplied from the tuner and outputs the demodulated signal to the demultiplexer 723. The demultiplexer 723 separates the data supplied from the demodulation unit 722 into audio data, video data, and EPG data and outputs
30 the separated data to the audio decoder 724, the video decoder 725, or the recorder control unit 726.
100

SP305119WO00
[0336]
The audio decoder 724 decodes the input audio data, for example, in an MPEG mode and outputs the decoded audio data to the recording/reproducing unit 733. The 5 video decoder 725 decodes the input video data, for
example, in the MPEG mode and outputs the decoded video data to the display converter 730. The recorder control unit 72 6 supplies the input EPG data to the EPG data memory 727 for storage,
10 [0337]
The display converter 730 encodes the video data that is supplied from the video decoder 725 or the recorder control unit 72 6 into video data, for example, of a national television standards committee (NTSC) mode
15 using the video encoder 741 and outputs the encoded video data to the recording/reproducing unit 733. In addition, the display converter 730 converts the screen size of video data that is supplied from the video decoder 725 or the recorder control unit 72 6 into a size that
20 corresponds to the size of the monitor 760. The display converter 730 further converts the video data, of which the screen size has been converted, into a video data of the NTSC mode using the video encoder 741, converts the converted video data into an analog signal, and outputs
25 the analog signal to the display control unit 732. [0338]
The display control unit 732 overlaps an on screen display (OSD) signal output by the OSD control unit 731 with the video signal input from the display converter
30 730 and outputs a resultant signal to the display of the monitor 7 60 for display under the control of the recorder
101

SP305119WO00
control unit 726.
[0339]
In addition, the audio data output by the audio
decoder 724 is converted into an analog signal by the D/A 5 converter 734 and is supplied to the monitor 7 60. The monitor 7 60 outputs the audio signal from the built-in
speaker.
[0340]
The recording/reproducing unit 733 includes a hard 10 disk as a storage medium that records video data, audio
data, and the like therein.
[0341]
The recording/reproducing unit 733, for example,
encodes the audio data supplied from the audio decoder 15 724 in the MPEG mode using the encoder 751. In addition,
the recording/reproducing unit 733 encodes the video data
supplied from the video encoder 741 of the display
converter 730 in the MPEG mode using the encoder 751.
The recording/reproducing unit 733 composes the encoded 20 data of the audio data and the encoded data of the video
data using a multiplexer. The recording/reproducing unit
733 amplifies the composed data through channel coding
and writes the data into the hard disk through a
recording head. 25 [0342]
The recording/reproducing unit 733 reproduces the
data recorded in the hard disk through a reproducing head,
amplifies the data, and separates the data into audio
data and video data using the demultiplexer. The 30 recording/reproducing unit 733 decodes the audio data and
the video data in the MPEG mode using the decoder 752.
102

SP305119WO00
The recording/reproducing unit 733 performs D/A conversion of the decoded audio data and outputs resultant audio data to the speaker of the monitor 760. In addition, the recording/reproducing unit 733 performs 5 D/A conversion of the decoded video data and outputs resultant video data to the display of the monitor 7 60. [0343]
The recorder control unit 726 reads latest EPG data from the EPG data memory 727 based on a user instruction
10 that is represented by an infrared signal, which is
received through the reception unit 721, transmitted from the remote controller and supplies the latest EPG data to the OSD control unit 731. The OSD control unit 731 generates image data that corresponds to the input EPG
15 data and outputs the image data to the display control
unit 732. The display control unit 732 outputs the video data input from the OSD control unit 731 to the display of the monitor 7 60 for display. Accordingly, the electronic program guide (EPG) is displayed on the
20 display of the monitor 760. [0344]
In addition, the hard disk recorder 700 can acquire various kinds of data such as video data, audio data, or EPG data supplied from another device through a network
25 such as the Internet. [0345]
The communication unit 735 acquires encoded data of video data, audio data, EPG data, and the like transmitted from another device through the network under
30 the control of the recorder control unit 72 6 and supplies the acquired encoded data to the recorder control unit
103

SP305119WO00
726. The recorder control unit 72 6, for example, supplies the acquired encoded data of the video data or the audio data to the recording/reproducing unit 733 and stores the encoded data in the hard disk. At this time, 5 the recorder control unit 72 6 and the
recording/reproducing unit 733 may perform a re-encoding process or the like as is necessary. [0346]
In addition, the recorder control unit 726 decodes
10 the acquired encoded data of the video data or the audio data and supplies acquired video data to the display converter 730. The display converter 730, similarly to the video data supplied from the video decoder 725, processes the video data supplied form the recorder
15 control unit 72 6 and supplies the processed video data to the monitor 7 60 through the display control unit 732, whereby an image thereof is displayed. [0347]
Furthermore, in accordance with the display of the
20 image, the recorder control unit 726 may supply the
decoded audio data to the monitor 7 60 through the D/A converter 734 and output the audio from the speaker. [0348]
In addition, the recorder control unit 72 6 decodes
25 the acquired encoded data of the EPG data and supplies the decoded EPG data to the EPG data memory 727. [0349]
The above-described hard disk recorder 700 uses the image decoding device 151 (300) as the video decoder 725,
30 the decoder 752, and the decoder that is built in the
recorder control unit 72 6. Accordingly, similarly to the
104

SP305119WO00 #
case of the image decoding device 151 (300), the video decoder 725, the decoder 752, and the decoder that is built in the recorder control unit 72 6 can perform the inverse orthogonal transform processing with a smaller 5 calculation amount. [0350]
Accordingly, the hard disk recorder 700 can generate a predicted image with high accuracy. As a result, the hard disk recorder 700 can acquire a decoded
10 image with high accuracy based on the encoded data of the video data received through the tuner, the encoded data of the video data read from the hard disk of the recording/reproducing unit 733, and the encoded data of the video data acquired through the network and display
15 the decoded image on the monitor 7 60. [0351]
In addition, the hard disk recorder 700 uses the
image encoding device 51 (200) as the encoder 751. I
Accordingly, the encoder 751, similarly to the case of
20 the image encoding device 51 (200), can perform the orthogonal transform processing and the inverse orthogonal transform processing with a smaller calculation amount. [0352]
25 Accordingly, the hard disk recorder 700, for
example, can improve the encoding efficiency of encoded data to be stored in the hard disk. As a result, the hard disk recorder 700 can use the storage area of the hard disk more efficiently at a higher speed.
30 [0353]
In the description presented above, although the
105

^ SP305119WO00
hard disk recorder 700 that records the video data or the
audio data in the hard disk has been described, it is
apparent that a recording mediiim of any type may be used.
For example, even in the case of a recorder such as a 5 flash memory, an optical disc, or a video tape that uses
a recording medium other than a hard disk, similarly to
the case of the above-described hard disk recorder 700,
the image encoding device 51 (200) and the image decoding
device 151 (300) can be applied. 10 [0354]

[Configuration Example of Camera]
Fig. 27 is a block diagram that illustrates an
example of a main configuration of a camera that uses the 15 image decoding device and the image encoding device
according to the present technology.
[0355]
The camera 8 00 illustrated in Fig. 27 images a
subject and displays an image of the subject on an LCD 20 816 or records the image in a recording medium 833 as
image data.
[0356]
A lens block 811 allows light (in other words, a
video of a subject) to be incident to a CCD/CMOS 812. 25 The CCD/CMOS 812 is an image sensor that uses a CCD or a
CMOS, converts the intensity of received light into an
electrical signal, and supplies the electrical signal to
a camera signal processing unit 813.
[0357] 30 The camera signal processing unit 813 converts
electrical signals supplied from the CCD/CMOS 812 into
106

^ SP305119WO00
chrominance signals of Y, Cr, and Cb and supplies the chrominance signals to an image signal processing unit 814. The image signal processing unit 814 performs predetermined image processing for an image signal 5 supplied from the camera signal processing unit 813 or decodes the image signal, for example, in the MPEG mode using an encoder 841 under the control of a controller 821. The image signal processing unit 814 encodes an image signal and supplies the generated encoded data to a
10 decoder 815. In addition, the image signal processing
unit 814 acquires display data generated on an on-screen display (OSD) 820 and supplies the display data to the decoder 815. [0358]
15 In the above-described process, the camera signal processing unit 813 maintains the image data, encoded data acquired by encoding the image data, or the like in a dynamic random access memory (DRAM) 818 as is necessary by appropriately using the DRAM 818 that is connected
20 through a bus 817. [0359]
The decoder 815 decodes the encoded data supplied from the image signal processing unit 814 and supplies acquired image data (decoded image data) to the LCD 816.
25 In addition, the decoder 815 supplies the display data
supplied from the image signal processing unit 814 to the LCD 816. The LCD 816 appropriately composes the image of the decoded image data supplied from the decoder 815 and the image of the display data and displays the composed
30 image. [0360]
107

SP305119WO00
*
The on screen display 820 outputs display data such as a menu screen that is formed by symbols, characters, or graphics or icons to the image signal processing unit 814 through the bus 817 under the control of the 5 controller 821. [0361]
The controller 821 performs various processes based on a signal that represents a content that is instructed by a user using an operation unit 822 and controls the
10 image signal processing unit 814, the DRAM 818, an external interface 819, the on screen display 820, a medium drive 823, and the like through the bus 817. In a flash ROM 824, a program, data, and the like that are necessary for the controller 821 to perform various
15 processes are stored. [0362]
For example, the controller 821, instead of the image signal processing unit 814 or the decoder 815, may encode the image data that is stored in the DRAM 818 or
20 decode the encoded data that is stored in the DRAM 818. At this time, the controller 821 may perform an encoding/decoding process in the same mode as the encoding/decoding mode of the image signal processing unit 814 or the decoder 815 or may perform the
25 encoding/decoding process in a mode that the image signal processing unit 814 or the decoder 815 does not support. [0363]
For example, in a case where image printing is directed to be started from the operation unit 822, the
30 controller 821 reads image data from the DRAM 818 and supplies the image data to a printer 834 that is
108

SP305119WO00
connected to the external interface 819 through the bus
817 for being printed.
[0364]
In addition, for example, in a case where image 5 recording is directed from the operation unit 822, the controller 821 reads encoded data from the DRAM 818 and supplies the encoded data to the recording medium 833 installed to the medium drive 823 through the bus 817 for storage.
10 [0365]
The recording medium 8.33 is an arbitrary readable and writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disc, or a semiconductor memory. It is apparent that the recording medium 833 may
15 be an arbitrary type as a removable medium and may be a tape device, a disk, or a memory card. Furthermore, the recording medium 833 may be a non-contact IC card or the like. [0366]
20 In addition, the medium drive 823 and the recording mediiim 833 may be integrated so as to be configured as a non-portable storage medium such as a built-in hard disk drive or a solid state drive (SSD). [0367]
25 The external interface 819, for example, is
configured by a USB input/output terminal or the like and is connected to the printer 834 in a case where an image is printed. In addition, a drive 831 is connected to the external interface 819 as is necessary, a removable
30 medium 832 such as a magnetic disk, an optical disc, or a magneto-optical disk is appropriately installed, and a
109

SP305119WO00
computer program read from them is installed in the flash
ROM 824 as is necessary.
[0368]
In addition, the external interface 819 includes a 5 network interface that is connected to a predetermined network such as a LAN or the Internet. The controller 821, for example, may read encoded data from the DRAM 818 in accordance with an instruction transmitted from the operation unit 822 and supply the encoded data to another
10 device that is connected through the network from the
external interface 819. In addition, the controller 821 may acquire encoded data or image data supplied from another device through the network through the external interface 819 and maintain the data in the DRAM 818 or
15 supply the data to the image signal processing unit 814. [0369]
The above-described camera 800 uses the image decoding device 151 (300) as the decoder 815. Accordingly, similarly to the case of the image decoding
20 device 151, the decoder 815 can perform the inverse orthogonal transform processing with a smaller calculation amount. [0370]
Accordingly, the camera 800 can realize a high-
25 speed process and generate a predicted image having high accuracy. As a result, the camera 800, for example, can acquire a decoded image having higher accuracy based on the image data generated by the CCD/CMOS 812, the encoded data of the video data read from the DRAM 818 or the
30 recording medium 833, encoded data of the video data acquired through the network and display the decoded
110

SP305119WO00
image on the LCD 816.
[0371]
In addition, the camera 800 uses the image encoding
device 51 (200) as the encoder 841. Accordingly, the 5 encoder 841, similarly to the case of the image encoding
device 51 (200), can perform the orthogonal transform
processing and the inverse orthogonal transform
processing with a smaller calculation amount.
[0372] 10 Therefore, the camera 800, for example, can improve
the encoding efficiency of encoded data that is recorded
in a hard disk. As a result, the camera 800 can use the
storage area of the DRAM 818 or the recording medium 833
at a high speed more efficiently. 15 [0373]
In addition, a decoding method used in the image
decoding device 151 (300) may be applied to the decoding
process that is performed by the controller 821.
Similarly, an encoding method used in the image encoding 20 device 51 (200) may be applied to the encoding process
that is performed by the controller 821.
[0374]
Furthermore, the image data that is captured by the
camera 800 may be a moving image or a still image. 25 [0375]
It is apparent that the image encoding device 51
(200) and the image decoding device 151 (300) also can be
applied to a device other than the above-described device
or a system. 30 [0376]
In addition, an embodiment of the present
111

SP305119WO00 t
technology is not limited to the above-described embodiments, and various changes can be made therein in a range not departing from the concept of the present technology. 5 [0377]
Furthermore, the present technology may have the following configurations. [0378] (1)
10 There is provided an encoding device including: a prior orthogonal transformation unit that performs a second orthogonal transform of which a calculation amount is smaller than a calculation amount of a first orthogonal transform for an image; a posterior orthogonal
15 transformation unit that performs the first orthogonal
transform for a low-frequency component of the image that is acquired as a result of the second orthogonal transform; and an encoding unit that encodes the low-frequency component after the first orthogonal transform
20 and a high-frequency component of the image that is
acquired as a result of the second orthogonal transform. (2)
The encoding device described in (1) described above, wherein the first orthogonal transform is a
25 separable-type Karhunen-Loeve transform. (3)
The encoding device described in (1) or (2) described above, wherein the posterior orthogonal transformation unit performs a non-separable type
30 Karhunen-Loeve transform for the low-frequency component in a case where a predicted direction of an intra
112

SP305119WO00
prediction mode in an AVC (advanced video coding) mode of the image is an oblique direction and performs a separable-type Karhunen-Loeve transform for the low-frequency component in a case where the predicted 5 direction of the intra prediction mode of the image is not the oblique direction. (4)
The encoding device described in (3) described above, wherein, in a case where the predicted direction
10 of the intra prediction mode of the image is the oblique direction, the prior orthogonal transformation unit performs the second orthogonal transforms more than the second orthogonal transforms that are performed in a case where the predicted direction of the intra prediction
15 mode of the image is not the oblique direction. (5)
The encoding device described in any one of (1) to
(4) described above, wherein the second orthogonal
transform is a wavelet transform.
20 (6)
The encoding device described in any one of (1) to
(5) described above, wherein the encoding unit encodes
the low-frequency component after the first orthogonal
transform and the high-frequency component in an order
25 according to the intra prediction mode in the AVC
(advance video coding) mode of the image.
(7)
There is provided an encoding method performed by
an encoding device including the steps of: performing a 30 second orthogonal transform of which a calculation amount
is smaller than a calculation amount of a first
113

SP305119WO00
orthogonal transform for an image; performing the first orthogonal transform for a low-frequency component of the image that is acquired as a result of the second orthogonal transform; and encoding the low-frequency 5 component after the first orthogonal transform and a
high-frequency component of the image that is acquired as
a result of the second orthogonal transform.
(8)
There is provided a decoding device including: a
10 decoding unit that acquires a coefficient of a low-frequency component of an image after a first orthogonal transform that is acquired as a result of performing a second orthogonal transform of which a calculation amount is smaller than a calculation amount of the first
15 orthogonal transform as a result of encoding the image and a result of encoding a high-frequency component of the image that is acquired as a result of performing the second orthogonal transform and decodes a result of encoding of the image; a prior inverse orthogonal
20 transformation unit that performs a first inverse orthogonal transform that corresponds to the first orthogonal transform for the coefficient of the low-frequency component after the first orthogonal transform that is acquired as a result of the decoding; and a
25 posterior inverse orthogonal transformation unit that acquires the image by performing a second inverse orthogonal transform that corresponds to the second orthogonal transform for the low-frequency component that is acquired as a result of the first inverse orthogonal
30 transform and the high-frequency component that is acquired as a result of the decoding.
114

- SP305119WO00
(9)
The decoding device described in (8) described above, wherein the first inverse orthogonal transform is a separable-type inverse Karhunen-Loeve transform. 5 (10)
The decoding device described in (8) or (9) described above, wherein an intra prediction mode of the image in the AVC (advanced video coding) mode is included in the result of the encoding, and the prior inverse
10 orthogonal transformation unit performs a non-separable
type inverse Karhunen-Loeve transform for the coefficient of the low-frequency component after the first orthogonal transform in a case where a predicted direction of the intra prediction mode is an oblique direction and
15 performs a separable-type inverse Karhunen-Loeve
transform for the coefficient of the low-frequency component after the first orthogonal transform in a case where a predicted direction of the intra prediction mode is not the oblique direction.
20 (11)
The decoding device described in (10) described above, wherein, in a case where the predicted direction of the intra prediction mode is an oblique direction, the prior inverse orthogonal transformation unit performs the
25 second inverse orthogonal transforms more than the second inverse orthogonal transforms that are performed in a case where the predicted direction of the intra prediction mode is not the oblique direction. (12)
30 The decoding device described in any one of (8) to (11) described above, wherein the second inverse
115

^ SP305119WO00
orthogonal transform is an inverse wavelet transform. (13)
The decoding device described in any one of (8) to (12) described above, wherein an intra prediction mode of 5 the image in the AVC (advanced video coding) mode is
included in the result of the encoding, and the decoding unit decodes the coefficient of the low-frequency component after the first orthogonal transform and the high-frequency component in order according to the intra
10 prediction mode. (14)
There is provided a decoding method performed by a decoding device including the steps of: acquiring a coefficient of a low-frequency component of an image
15 after a first orthogonal transform that is acquired as a result of performing a second orthogonal transform of which a calculation amount is smaller than a calculation amount of the first orthogonal transform as a result of encoding the image and a result of encoding a high-
20 frequency component of the image that is acquired as a
result of performing the second orthogonal transform and decoding a result of encoding of the image; performing a first inverse orthogonal transform that corresponds to the first orthogonal transform for the coefficient of the
25 low-frequency component after the first orthogonal
transform that is acquired as a result of the decoding; and acquiring the image by performing a second inverse orthogonal transform that corresponds to the second orthogonal transform for the low-frequency component that
30 is acquired as a result of the first inverse orthogonal transform and the high-frequency component that is
116

SP305119WO00
acquired as a result of the decoding.
REFERENCE SIGNS LIST
[0379] 5 51 Image encoding device
66 Lossless encoding unit
91 DWT unit
92-0 to 92-8 KLT unit
101-0 to 101-8 Inverse KLT units 10 103 Inverse DWT unit
151 Image decoding device
162 Lossless decoding unit
200 Image encoding device
221 DWT unit 15 222-3 to 222-8 KLT unit
241-3 to 241-8 Inverse KLT unit
244 Inverse DWT unit
117

CLAIMS
1. An encoding device comprising:
a prior orthogonal transformation unit that
5 performs a second orthogonal transform of which a
calculation amount is smaller than a calculation amount
of a first orthogonal transform for an image;
a posterior orthogonal transformation unit that
performs the first orthogonal transform for a low-
10 frequency component of the image that is acquired as a
result of the second orthogonal transform; and
an encoding unit that encodes the low-frequency
component after the first orthogonal transform and a
high-frequency component of the image that is acquired as
15 a result of the second orthogonal transform.
2. The encoding device according to claim 1, wherein
the first orthogonal transform is a separable-type
Karhunen-Loeve transform.
20
3. The encoding device according to claim 1, wherein
the posterior orthogonal transformation unit performs a
non-separable type Karhunen-Loeve transform for the lowfrequency
component in a case where a predicted direction
25 of an intra prediction mode in an AVC (advanced video
coding) mode of the image is an oblique direction and
performs a separable-type Karhunen-Loeve transform for
the low-frequency component in a case where the predicted
direction of the intra prediction mode of the image is
30 not the oblique direction.
118
SP305119WO00
4. The encoding device according to claim 3, wherein,
in a case where the predicted direction of the intra
prediction mode of the image is the oblique direction,
the prior orthogonal transformation unit performs the
5 second orthogonal transforms more than the second
orthogonal transforms that are performed in a case where
the predicted direction of the intra prediction mode of
the image is not the oblique direction.
10 5. The encoding device according to claim 1, wherein
the second orthogonal transform is a wavelet transform.
6. The encoding device according to claim 1, wherein
the encoding unit encodes the low-frequency component
15 after the first orthogonal transform and the highfrequency
component in an order according to the intra
prediction mode in the AVC (advance video coding) mode of
the image.
2 0 7. An encoding method performed by an encoding device,
the encoding method comprising the steps of:
performing a second orthogonal transform of which a
calculation amount is smaller than a calculation amount
of a first orthogonal transform for an image;
25 performing the first orthogonal transform for a
low-frequency component of the image that is acquired as
a result of the second orthogonal transform; and
encoding the low-frequency component after the
first orthogonal transform and a high-frequency component
30 of the image that is acquired as a result of the second
orthogonal transform.
119
^ SP305119WO00
8. A decoding device comprising:
a decoding unit that acquires a coefficient of a
low-frequency component of an image after a first
5 orthogonal transform that is acquired as a result of
performing a second orthogonal transform of which a
calculation amount is smaller than a calculation amount
of the first orthogonal transform as a result of encoding
the image and a result of encoding a high-frequency
10 component of the image that is acquired as a result of
performing the second orthogonal transform and decodes a
result of encoding of the image;
a prior inverse orthogonal transformation unit that
performs a first inverse orthogonal transform that
15 corresponds to the first orthogonal transform for the
coefficient of the low-frequency component after the
first orthogonal transform that is acquired as a result
of the decoding; and
a posterior inverse orthogonal transformation unit
20 that acquires the image by performing a second inverse
orthogonal transform that corresponds to the second
orthogonal transform for the low-frequency component that
is acquired as a result of the first inverse orthogonal
transform and the high-frequency component that is
25 acquired as a result of the decoding.
9. The decoding device according to claim 8, wherein
the first inverse orthogonal transform is a separabletype
inverse Karhunen-Loeve transform.
30
10. The decoding device according to claim 8,
120
^ SP305119WO00
wherein an intra prediction mode of the image in
the AVC (advanced video coding) mode is included in the
result of the encoding, and
wherein the prior inverse orthogonal transformation
5 unit performs a non-separable type inverse Karhunen-Loeve
transform for the coefficient of the low-frequency
component after the first orthogonal transform in a case
where a predicted direction of the intra prediction mode
is an oblique direction and performs a separable-type
10 inverse Karhunen-Loeve transform for the coefficient of
the low-frequency component after the first orthogonal
transform in a case where a predicted direction of the
intra prediction mode is not the oblique direction.
15 11. The decoding device according to claim 10, wherein,
in a case where the predicted direction of the intra
prediction mode is an oblique direction, the prior
inverse orthogonal transformation unit performs the
second inverse orthogonal transforms more than the second
20 inverse orthogonal transforms that are performed in a
case where the predicted direction of the intra
prediction mode is not the oblique direction.
12. The decoding device according to claim 8, wherein
25 the second inverse orthogonal transform is an inverse
wavelet transform.
13. The decoding device according to claim 8,
wherein an intra prediction mode of the image in
30 the AVC (advanced video coding) mode is included in the
result of the encoding, and
121
^ SP305119WO00
wherein the decoding unit decodes the coefficient
of the low-frequency component after the first orthogonal
transform and the high-frequency component in order
according to the intra prediction mode.
• \ ^ ' • - • " , • • . • . ^- •. • ' < ' ' . • • - . . '
> 14. A decoding method performed by a decoding device,
. the decoding method comprising the steps of: .
"acquiring a coefficient of a low-frequency .
i component of an image after a first orthogonal transform
' ' • ' • • • ' • ' *.
10 that is acquired as a result of performing a second
orthogonal transform of which a calculation amount is
*
smaller than a, calculation amount of the first orthogonal
transform as a result of encoding the image and a result
of encoding a high-frequency component of the image that
15 is acquired as a result of performing the second /<
orthogonal transform and decoding a result of encoding of •
the image;
performing a first inverse orthogonal transform
that corresponds to the first orthogonal transform for
20 the coefficient of the low-frequency component after the
first orthogojisil transform that is acquired as a result
of the decoding; and ,
acquiring the image by performing a second inverse
orthogonal transform that corresponds to the second
, 25 ..orthogonal transform for the low-frequency component that
is acquired as a result of the first inverse orthogonal
transform and the high-frequency component that is
acquired as a result of the decoding.