IMAGE PROCESSING DEVICE AND METHOD TECHNICAL FIELD 5 [0001] This disclosure relates to an image processing device and method, and more particularly, to an image processing device that can reduce the load of image encoding. 10 BACKGROUND ART [0002] In recent years, to handle image information as digital information and achieve high-efficiency 15 information transmission and accumulation in doing do, apparatuses compliant with a standard, such as MPEG (Moving Picture Experts Group) for compressing image I information through orthogonal transforms such as discrete cosine transforms and motion compensations by 2 0 using redundancy inherent to image information, have been spreading among broadcast stations to distribute information and among general households to receive information. ! i [0003] f 25 Particularly, MPEG2 (ISO (International Organization for Standardization)/IEC (International Electrotechnical Commission) 13818-2) is defined as a general-purpose image encoding standard, and is applicable to interlaced images and non-interlaced images, 30 and to standard-resolution images and high-definition • images. Currently, MPEG2 is used in a wide range of I ;; l i, SP310529WO00 9 applications for professionals and general consumers. According to the MPEG2 compression standard, a bit rate of 4 to 8 Mbps is assigned to an interlaced image having a standard resolution of 720 x 480 pixels, and a bit rate 5 of 18 to 22 Mbps is assigned to an interlaced image having a high resolution of 1920 x 1088 pixels, for example. In this manner, high compression rates and excellent image quality can be realized. [0004] 10 MPEG2 is designed mainly for high-quality image encoding suited for broadcasting, but is not compatible with lower bit rates than MPEGl or encoding standards with higher compression rates. As mobile terminals are becoming popular, the demand for such encoding standards 15 is expected to increase in the future, and to meet the demand, the MPEG4 encoding standard has been set. As for image encoding standards, the ISO/IEC 14496-2 standard was approved as an international standard in December 1998. 20 [0005] ; Further, a standard called H.2 6L (ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Q6/16 VCEG (Video Coding Expert Group)), which is originally intended for encoding images 25 for video conferences, is currently being set. Compared with the conventional encoding methods such as MPEG2 and ; I MPEG4, H.2 6L requires a larger amount of calculation in j encoding and decoding, but is known to achieve a higher j encoding efficiency. Also, as a part of the MPEG4 j 30 activity, "Joint Model of Enhanced-Compression Video • Coding" is now being established as a standard for :: 2 f: SP310529WO00 achieving a higher encoding efficiency by incorporating functions unsupported by H.2 6L into the functions based on H.26L. [0006] 5 On the standardization schedule,,the standard was approved as an international standard under the name of H.2 64 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as AVC) in March 2003. | [0007] i 10 Further, as an extension of that, FRExt (Fidelity Range Extension) involving encoding tools required for professional use, such as RGB, 4:2:2, and 4:4:4, and 8 x 8 DCT and quantization matrixes specified in MPEG2, was set as a standard in February 2 005. This is an encoding 15 method for enabling excellent representation of even film noise contained in movie films by using AVC, and is now used in a wide range of applications such as Blu-Ray discs. [0008] 20 However, there is an increasing demand for encoding at a higher compression rate so as to compress images having a resolution of 4096 x 2048 pixels, which is four times higher than the high-definition image resolution, t or distribute high-definition images in today's 25 circumstances where transmission capacities are limited as in the Internet. Therefore, studies on improvement in I I encoding efficiency are still continued by VCEG under j ITU-T. i I [0009] | 30 When images having an even higher resolution, such 1 as 4000 x 2000 pixels, or existing high-definition images | 3 I SP310529WO00 » are transmitted through a line with a limited bandwidth such as the Internet, a compression rate achieved by AVC is not sufficiently high. In view of this, a group called VCEG (Video Coding Expert Group) under ITU-T is 5 trying to further improve encoding efficiency (see Non- Patent Document 1, for example). [0010] As a method for improving encoding efficiency, Non- Patent Document 1 suggests a method involving an adaptive 10 loop filter (ALF). CITATION LIST NON-PATENT DOCUMENT [0011] 15 Non-Patent Document 1: Takeshi. Chujoh, et al., "Blockbased Adaptive Loop Filter" ITU-T SG16 Q6 VCEG Contribution, AI18, Germany, July, 2008 SUMMARY OF THE INVENTION 2 0 PROBLEMS TO BE SOLVED BY THE INVENTION [0012] However, using the adaptive loop filter suggested in Non-Patent Document 1 for all the pictures and slices in the sequence requires an enormous amount of 25 calculation, and there is a possibility of an increase in the image encoding operation load. [0013] This disclosure has been made in view of those circumstances, and an object thereof is to reduce the 30 load of the adaptive loop filter while restraining increases in image quality deterioration, so as to i 4 I SP310529WO00 ; restrain increases in the image encoding operation load caused by the adaptive loop filtering operation. SOLUTIONS TO PROBLEMS 5 [0014] An aspect of this disclosure is an image processing device that includes: a filter control unit that controls an adaptive filtering operation to be performed on image data, in accordance with whether the image data is to be 10 referred to by other image data; and a filtering operation unit that performs the adaptive filtering operation on the image data under the control of the filter control unit in a motion compensation loop. [0015] 15 When the image data being subjected to the adaptive filtering operation is to be referred to by the other image data in an operation to encode the image data, the filter control unit can control the adaptive filtering operation to be performed. When the image data being 20 subjected to the adaptive filtering operation is not to be referred to by the other image data in the operation to encode the image data, the filter control unit can control the adaptive filtering operation not to be performed. L 25 [0016] The image data may be picture data, and the filter control unit can control the adaptive filtering operation for the image data in accordance with the type of the f j picture. j 30 [0017] ! When the image data is an I-picture, the filter • - 5 ; SP310529WO00 control unit can control the adaptive filtering operation to be performed. When the image data is a P-picture and a B-picture, the filter control unit can control the adaptive filtering operation not to be performed. 5 [0018] When the image data is an I-picture or a P-picture, the filter control unit can control the adaptive filtering operation to be performed. When the image data is a B-picture, the filter control unit can control the 10 adaptive filtering operation not to be performed. [0019] When.the image data is an I-picture and a P-picture in image data containing hierarchical B-pictures, or a Bpicture to be referred to, the filter control unit can 15 control the adaptive filtering operation to be performed. When the image data is a B-picture not to be referred to in the image data containing hierarchical B-pictures, the filter control unit can control the adaptive filtering operation not to be performed. 20 [0020] The image data may be slice data, and the filter control unit can control the adaptive filtering operation for the image data in accordance with the type of the slice. 25 [0021] When the image data is an I-slice, the filter control unit can control the adaptive filtering operation to be performed. When the image data is a P-slice and a B-slice, the filter control unit can control the adaptive 30 filtering operation not to be performed. [0022] 6 SP310529WO00 When the image data is an I-slice or a P-slice, the filter control unit can control the adaptive filtering operation to be performed. When the image data is a Bpicture, the filter control unit can control the adaptive 5 filtering operation not to be performed. [0023] When the image data is an I-slice and a P-slice in image data containing hierarchical B-slices, or a B-slice to be referred to, the filter control unit can control 10 the adaptive filtering operation to be performed. When the image data is a B-slice not to be referred to in the image data containing hierarchical B-pictures, the filter control unit can control the adaptive filtering operation not to be performed. 15 [0024] The image processing device further includes an encoding unit that encodes the image data subjected to J the adaptive filtering operation. The encoding unit can 1 f encode the filter coefficient of the adaptive filtering I | 20 operation and flag information indicating whether to \ I i perform the adaptive filtering operation, and adds the [ i resultant data to the encoded data of the image data. | [ [0025] | The filter control unit can control the tap length i 25 of the filter coefficient of the adaptive filtering j I operation, in accordance with whether the image data is ! I to be referred to by other image data. The filtering t ! operation unit can perform the adaptive filtering j I operation on the image data, using the filter coefficient | 30 having the tap length controlled by the filter control j unit. I 1 7 I SP310529WO00 [0026] When the image data being subjected to the adaptive filtering operation is to be referred to by the other image data in an operation to encode the image data, the 5 filter control unit can perform control to increase the tap length. When the image data being subjected to the adaptive filtering operation is not to be referred to by the other image data in the operation to encode the image data, the filter control unit can perform control to 10 shorten the tap length. [0027] An aspect of this disclosure is an image processing method that includes: controlling an adaptive filtering operation to be performed on image data, in accordance 15 with whether the image data is to be referred to by other image data, the control being performed by a filter control unit of an image processing device; and performing the adaptive filtering operation on the image data in a motion compensation loop, the adaptive I 20 filtering operation being performed by a filtering i | operation unit of the image processing device. [0028] In an aspect of this disclosure, an adaptive filtering operation to be performed on image data is 25 controlled in accordance with the type of each s predetermined unit data of image data, and the adaptive I I i filtering operation is performed on the image data in a j I motion compensation loop. } i 30 EFFECTS OF THE INVENTION 1 [0029] I r 8 ! SP310529WO00 According to this disclosure, images can be processed. Particularly, the load of image encoding operations can be reduced while restraining increases in | image quality deterioration. 5 BRIEF DESCRIPTION OF DRAWINGS [0030] Fig. 1 is a block diagram showing an image encoding device that outputs compressed image information 10 according to the AVC encoding method. Fig. 2 is a block diagram showing an image decoding device that receives an input of compressed image information according to the AVC encoding method. Fig. 3 is a diagram for explaining the operating 15 principles of a deblocking filter. Fig. 4 is a diagram for explaining a method of defining Bs. Fig. 5 is a diagram for explaining the operating principles of a deblocking filter. 20 Fig. 6 is a diagram showing an example of correspondence relationships between indexA and indexB, and values of a and p. Fig. 7 is a diagram showing an example of correspondence relationships among Bs, indexA, and tCO. i: s f 25 Fig. 8 is a block diagram showing an exemplary I structure of part of an image encoding device using an i adaptive loop filter. j i Fig. 9 is a block diagram showing an exemplary ! structure of part of an image decoding device using an ! 30 adaptive loop filter. | Fig. 10 is a block diagram showing a typical j 9 I i i 'I SP310529WO00 exemplary structure of an image encoding device. Fig. 11 is a block diagram showing a typical exemplary structure of an adaptive loop filter. I Fig. 12 is a diagram for explaining an example of I' 5 ON/OFF control performed by an adaptive loop filter. | Fig. 13 is a diagram for explaining another example | of ON/OFF control performed by an adaptive loop filter. I Fig. 14 is a diagram for explaining an example of | the syntax of a slice header. | 10 Fig. 15 is a diagram for explaining an example of I the parameter syntax of an adaptive loop filter. | Fig. 16 is a diagram for explaining an example of | the parameter syntax of an adaptive loop filter, I continued from Fig. 15. | 15 Fig. 17 is a diagram for explaining an example of | the parameter syntax of an adaptive loop filter, | i continued from Fig. 16. ., Fig. 18 is a flowchart for explaining an example i flow of an encoding operation. 2 0 Fig. 19 is a flowchart for explaining an example J flow of an adaptive loop filtering operation. ' Fig. 2 0 is a block diagram showing another 1 exemplary structure of an adaptive loop filter. j Fig. 21 is a flowchart for explaining another ; 25 example flow of an adaptive loop filtering operation. Fig. 22 is a diagram for explaining examples of i macroblocks. I Fig. 23 is a block diagram showing a typical I exemplary structure of a personal computer. 30 Fig. 24 is a block diagram showing a typical [ ji exemplary structure of a television receiver. j j ! io ! t I SP310529WO00 fan. m Fig. 25 is a block diagram showing a typical exemplary structure of a portable telephone device. Fig. 2 6 is a block diagram showing a typical exemplary structure of a hard disk recorder. 5 Fig. 27 is a block diagram showing a typical exemplary structure of a camera. MODE FOR CARRYING OUT THE INVENTION [0031] 10 The following is a description of modes for. carrying out this technique (hereinafter referred to as i embodiments). Explanation will be made in the following order. 1. First Embodiment (Image Encoding Device) 15 2. Second Embodiment (Image Encoding Device) 3. Third Embodiment (Personal Computer) 4. Fourth Embodiment (Television Receiver) I I 5. Fifth Embodiment (Portable Telephone Device) 6. Sixth Embodiment (Hard Disk Recorder) 20 7. Seventh Embodiment (Camera) [0032] <1. First Embodiment> [Image Encoding Device According to the AVC Encoding f Method] [ [ 25 Fig. 1 shows the structure of an embodiment of an [ ! image encoding device that encodes images according to I the AVC encoding method. [ [0033] J The image encoding device 100 shown in Fig. 1 is a j 30 device that encodes and outputs images by an encoding method compliant with the AVC standard. As shown in Fig. 11 : SP310529WO00 1, the image encoding device 100 includes an A/D converter 101, a screen rearrangement buffer 102, an arithmetic operation unit 103, an orthogonal transform unit 104, a quantization unit 105, a lossless encoding 5 unit 106, and an accumulation buffer 107. The image encoding device 100 also includes an inverse quantization unit 108, an inverse orthogonal transform unit 109, an arithmetic operation unit 110, a deblocking filter 111, a frame memory 112, a selection unit 113, an intra 10 prediction unit 114, a motion prediction/compensation unit 115, a selection unit 116, and a rate control unit 117. [0034] The A/D converter 101 subjects input image data to 15 an A/D conversion, and outputs and stores the image data into the screen rearrangement buffer 102. The screen rearrangement buffer 102 rearranges the image frames stored in displaying order in accordance with the GOP (Group of Pictures) structure, so that the frames are j 2 0 arranged in encoding order. The screen rearrangement j buffer 102 supplies the image having the rearranged frame 1 order to the arithmetic operation unit 103. The screen rearrangement buffer 102 also supplies the image having the rearranged frame order to the intra prediction unit j 25 114 and the motion prediction/compensation unit 115. j [0035] I The arithmetic operation unit 103 subtracts a j predicted image supplied from the intra prediction unit I 114 or the motion prediction/compensation unit 115 via ; 30 the selection unit 116, from the image read from the I screen rearrangement buffer 102, and outputs the I 12 i i SP310529WO00 41 difference information to the orthogonal transform unit 104. [0036] For example, when intra encoding is performed on an 5 image, the arithmetic operation unit 103 subtracts a predicted image supplied from the intra prediction unit I 114, from the image read from the screen rearrangement buffer 102. When inter encoding is performed on an image, the arithmetic operation unit 103 subtracts a predicted 10 image supplied from the motion prediction/compensation I unit 115, from the image read from the screen I rearrangement buffer 102. I [0037] I The orthogonal transform unit 104 performs an 1 15 orthogonal transform operation, such as a discrete cosine 1 transform or a Karhunen-Loeve transform, on the I difference information supplied from the arithmetic 1 operation unit 103, and supplies the transform I coefficient to the quantization unit 105. J 20 [0038] I The quantization unit 105 quantizes the transform I coefficient output from the orthogonal transform unit 104. I Based on target bit rate value information supplied from I the rate control unit 117, the quantization unit 105 sets I 25 a quantization parameter, and performs quantization. The I quantization unit 105 supplies the quantized transform 1 coefficient to the lossless encoding unit 106. 1 [0039] I The lossless encoding unit 106 performs lossless 1 30 encoding on the quantized transform coefficient through I variable-length encoding or arithmetic encoding or the j '•i 13 1 SP310529WO00 • like. Since the coefficient data has already been quantized under the control of the rate control unit 117, the bit rate is equal to the target value (or approximates the target value) set by the rate control 5 unit 117. [0040] The lossless encoding unit 106 obtains information indicating an intra prediction or the like from the intra prediction unit 114, and obtains information indicating 10 an inter prediction mode or motion vector information or the like from the motion prediction/compensation unit 115. The information indicating an intra prediction (an intrascreen prediction) will be hereinafter also referred to as intra prediction mode information. The information 15 indicating an inter prediction (an inter-screen prediction) will be hereinafter referred to as inter prediction mode information. [0041] The lossless encoding unit 106 not only encodes the I 20 quantized transform coefficient, but also incorporates j (multiplexes) various kinds of information such as a filter coefficient, the intra prediction mode information, J the inter prediction mode information, and the quantization parameter, into the header information of 25 encoded data. The lossless encoding unit 106 supplies L and stores the encoded data obtained through the encoding I into the accumulation buffer 107. [0042] I. For example, in the lossless encoding unit 106, a 30 lossless encoding operation such as variable-length | encoding or arithmetic encoding is performed. The • • 14 • ; ;' SP310529WO00 • variable-length encoding may be CAVLC (Context-Adaptive Variable Length Coding) specified in H.264/AVC, for example. The arithmetic encoding may be CABAC (Context- Adaptive Binary Arithmetic Coding). 5 [0043] The accumulation buffer 107 temporarily stores the encoded data supplied from the lossless encoding unit 106, and outputs the encoded data as an encoded image encoded by H.2 64/AVC to a recording device or a transmission path 10 (not shown) in a later stage at a predetermined time, for example. [0044] The transform coefficient quantized at the quantization unit 105 is also supplied to the inverse 15 quantization unit 108. The inverse quantization unit 108 inversely quantizes the quantized transform coefficient by a method compatible with the quantization performed by the quantization unit 105. The inverse quantization unit 108 supplies the obtained transform coefficient to the I 20 inverse orthogonal transform unit 109. j I [0045] j The inverse orthogonal transform unit 109 performs an inverse orthogonal transform on the supplied transform coefficient by a method compatible with the orthogonal 25 transform operation performed by the orthogonal transform unit 104. The output subjected to the inverse orthogonal transform (the uncompressed difference information) is supplied to the arithmetic operation unit 110. I [0046] I 30 The arithmetic operation unit 110 obtains a locally | decoded image (a decoded image) by adding the predicted | 15 I SP310529WO00 • image supplied from the intra prediction unit 114 or the motion prediction/compensation unit 115 via the selection unit 116 to the inverse orthogonal transform result supplied from the inverse orthogonal transform unit 109 5 or the uncompressed difference information. [0047] For example, when the difference information is compatible with an image to be intra-encoded, the arithmetic operation unit 110 adds the predicted image 10 supplied from the intra prediction unit 114 to the difference information. When the difference information is compatible with an image to be inter-encoded, the arithmetic operation unit 110 adds the predicted image supplied from the motion prediction/compensation unit 115 15 to the difference information, for example. I [0048] I The addition result is supplied to the deblocking filter 111 or the frame memory 112. [0049] 20 The deblocking filter 111 removes block distortions I from the decoded image by performing a deblocking filtering operation where necessary, and performs a loop filtering operation, where necessary, by using a Wiener filter, for example, to improve image quality. The 25 deblocking filter 111 classifies respective pixels into classes, and performs an appropriate filtering operation on each of the classes. The deblocking filter 111 supplies the filtering operation results to the frame memory 112. [ 30 [0050] The frame memory 112 outputs a stored reference I : 16 SP310529WO00 • image to the intra prediction unit 114 or the motion prediction/compensation unit 115 via the selection unit 113 at a predetermined time. [0051] 5 For example, when intra encoding is performed on an image, the frame memory 112 supplies the reference image to the intra prediction unit 114 via the selection unit 113. When inter encoding is performed on an image, the frame memory 112 supplies the reference image to the 10 motion prediction/compensation unit 115 via the selection I unit 113, for example. J [0052] I When the reference image supplied from the frame memory 112 is an image to be subjected to intra encoding, 15 the selection unit 113 supplies the reference image to the intra prediction unit 114. When the reference image supplied from the frame memory 112 is an image to be subjected to inter encoding, the selection unit 113 supplies the reference image to the motion 20 prediction/compensation unit 115. [0053] I The intra prediction unit 114 performs intra 1 predictions (intra-screen predictions) to generate a 1 predicted image by using the pixel values in the screen. 1 25 The intra prediction unit 114 performs intra predictions I in more than one mode (intra prediction modes). I [0054] 1 By the H.2 64 image information encoding method, an 1 intra 4 x 4 prediction mode, an intra 8 x 8 prediction | 30 mode, and an intra 16 x 16 prediction mode are defined 1 for luminance signals. As for chrominance signals, I 17 I SP310529WO00 prediction modes for respective macroblocks can be defined independently of the luminance signals. In the intra 4 x 4 prediction mode, one intra prediction mode is defined for each 4 x 4 luminance block. In the intra 8 x 5 8 prediction mode, one intra prediction mode is defined for each 8 x 8 luminance block. In the intra 16 x 16 prediction mode and for the chrominance signals, one prediction mode is defined for each macroblock. [0055] I 10 The intra prediction unit 114 generates predicted I images in all the intra prediction modes, evaluates the 1 respective predicted images, and selects an optimum mode. 8 After selecting the optimum intra prediction mode, the I intra prediction unit 114 supplies the predicted image I 15 generated in the optimum intra prediction mode to the I arithmetic operation unit 103 and the arithmetic I operation unit 110 via the selection unit 116. I [0056] I As described above, the intra prediction unit 114 I 20 also supplies information such as the intra prediction 1 mode information indicating the selected intra prediction I mode to the lossless encoding unit 106 where appropriate. I [0057] Using the input image supplied from the screen I 25 rearrangement buffer 102, and a reference image supplied I from the frame memory 112 via the selection unit 113, the I I motion prediction/compensation unit 115 performs a motion 1 prediction on an image to be subjected to inter encoding, 1 and performs a motion compensating operation in | 30 accordance with the detected motion vectors, to generate | a predicted image (inter predicted image information). I 18 I! SP310529WO00 • [0058] The motion prediction/compensation unit 115 performs inter predicting operations in all candidate inter prediction modes, to generate a predicted image. 5 The motion prediction/compensation unit 115 supplies the generated predicted image to the arithmetic operation unit 103 and the arithmetic operation unit 110 via the selection unit 116. [0059] 10 The motion prediction/compensation unit. 115 supplies the inter prediction mode information indicating I the selected inter prediction mode, and motion vector I information indicating the calculated motion vectors to J the lossless encoding unit 106. I 15 [0060] When intra encoding is performed on an image, the selection unit 116 supplies the output of the intra prediction unit 114 to the arithmetic operation unit 103 I and the arithmetic operation unit 110. When inter 2 0 encoding is performed on an image, the selection unit 116 supplies the output of the motion prediction/compensation unit 115 to the arithmetic operation unit 103 and the arithmetic operation unit 110. [0061] 25 Based on the compressed images stored in the accumulation buffer 107, the rate control unit 117 controls the quantizing operation rate of the quantization unit 105 so as not to cause an overflow or underflow. 30 [0062] [Image Decoding Device According to the AVC Encoding 19 I SP310529WO00 Method] Fig. 2 is a block diagram showing a typical exemplary structure of an image decoding device that realizes image compression through orthogonal transforms, 5 such as discrete cosine transforms or Karhunen-Loeve transforms, and motion compensation. The image decoding device 2 00 shown in Fig. 2 is a decoding device that is compatible with the image encoding device 100. [0063] 10 Data encoded by the image encoding device 100 is supplied to the image decoding device 2 00 compatible with the image encoding device 100 via a predetermined transmission path, for example, and is decoded. [0064] 15 As shown in Fig. 2, the image decoding device 200 includes an accumulation buffer 201, a lossless decoding unit 202, an inverse quantization unit 203, an inverse orthogonal transform unit 204, an arithmetic operation unit 2 05, a deblocking filter 2 0 6, a screen rearrangement 20 buffer 207, and a D/A converter 208. The image decoding device 2 00 also includes a frame memory 2 09, a selection unit 210, an intra prediction unit 211, a motion prediction/compensation unit 212, and a selection unit 213. 25 [0065] The accumulation buffer 201 stores transmitted encoded data. The encoded data has been encoded by the image encoding device 100. The lossless decoding unit 202 decodes the encoded data read from the accumulation 30 buffer 2 01 at a predetermined time, by a method compatible with the encoding method used by the lossless | 20 i I SP310529WO00 encoding unit 106 shown in Fig. 1. [0066] When the frame is an intra-encoded frame, the header portion of the encoded data stores intra 5 prediction mode information. The lossless decoding unit 202 also decodes the intra prediction mode information, and supplies the information to the intra prediction unit 211. When the frame is an inter-encoded frame, on the other hand, the header portion of the encoded data stores 10 motion vector information. The lossless decoding unit 202 also decodes the motion vector information, and supplies the information to the motion prediction/compensation unit 212. [0067] 15 The inverse quantization unit 203 inversely quantizes the coefficient data (the quantized coefficient) decoded by the lossless decoding unit 202 by a method compatible with the quantization method used by the quantization unit 105 shown in Fig. 1. That is, the 20 inverse quantization unit 203 inversely quantizes the quantized coefficient by the same method as the method used by the inverse quantization unit 108 shown in Fig. 1. [0068] The inverse quantization unit 203 supplies the 25 inversely-quantized coefficient data, or the orthogonal transform coefficient, to the inverse orthogonal transform unit 204. The inverse orthogonal transform unit 2 04 subjects the orthogonal transform coefficient to an inverse orthogonal transform by a method compatible 30 with the orthogonal transform method used by the orthogonal transform unit 104 shown in Fig. 1 (the same 21 SP310529WO00 method as the method used by the inverse orthogonal transform unit 109 shown in Fig. 1), and obtains decoded residual error data corresponding to the residual error data from the time prior to the orthogonal transform 5 performed by the image encoding device 100. [0069] The decoded residual error data obtained through the inverse orthogonal transform is supplied to the arithmetic operation unit 205. A predicted image is also i 10 supplied to the arithmetic operation unit 205 from the intra prediction unit 211 or the motion prediction/compensation unit 212 via the selection unit 213. [0070] 15 The arithmetic operation unit 205 adds the decoded residual error data to the predicted image, and obtains decoded image data corresponding to the image data from the time prior to the predicted image subtraction performed by the arithmetic operation unit 103 of the 20 image encoding device 100. The arithmetic operation unit ; 205 supplies the decoded image data to the deblocking filter 206. [0071] I The deblocking filter 206 removes block distortions I 25 from the supplied decoded images, and supplies the images 1 to the screen rearrangement buffer 207. [0072] The screen rearrangement buffer 207 performs image rearrangement. Specifically, the frame order rearranged 1 I 30 in the encoding order of by the screen rearrangement buffer 102 of Fig. 1 is rearranged in the original 22 I ] : : SP310529WO00 1 * display order. The D/A converter 208 performs a D/A I conversion on the images supplied from the screen I rearrangement buffer 207, and outputs the converted | images to a display (not shown) to display the images. I 5 [0073] I The output of the deblocking filter 206 is further supplied to the frame memory 2 09. i [0074] I The frame memory 209, the selection unit 210, the 10 intra prediction unit 211, the motion prediction/compensation unit 212, and the selection unit I 213 are equivalent to the frame memory 112, the selection unit 113, the intra prediction unit 114, the motion prediction/compensation unit 115, and the selection unit 15 116 of the image encoding device 100, respectively. I [0075] j The selection unit 210 reads an image to be interprocessed and an image to be referred to from the frame I memory 209, and supplies the images to the motion 20 prediction/compensation unit 212. The selection unit 210 also reads an image to be used for intra predictions from the frame memory 209, and supplies the image to the intra prediction unit 211. [0076] | 25 Information that has been obtained by decoding the ; header information and indicates an intra prediction mode or the like is supplied, where appropriate, from the 1 lossless decoding unit 202 to the intra prediction unit 211. Based on the information, the intra prediction unit 1 30 211 generates a predicted image from the reference image 1 obtained from the frame memory 209, and supplies the 1 23 | SP310529WO00 ft generated predicted image to the selection unit 213. [0077] The motion prediction/compensation unit 212 obtains the information obtained by decoding the header 5 information (prediction mode information, motion vector information, reference frame information, a flag, respective parameters, and the like), from the lossless decoding unit 202. [0078] 10 Based on the information supplied from the lossless decoding unit 202, the motion prediction/compensation unit 212 generates a predicted image from the reference image obtained from the frame memory 209, and supplies the generated predicted image to the selection unit 213. 15 [0079] The selection unit 213 selects a predicted image generated by the motion prediction/compensation unit 212 or the intra prediction unit 211, and supplies the selected predicted image to the arithmetic operation unit 20 205. [0080] [Orthogonal Transforms] Meanwhile, by the AVC encoding method, only 4 x4 orthogonal transforms can be used as orthogonal 25 transforms in Baseline Profile, Extended Profile, and Main Profile. In High Profile and higher, an operation can be switched between a 4 x 4 orthogonal transform and an 8 x 8 orthogonal transform in a screen, as shown in Fig. 3. 30 [0081] I [Deblocking Filter] 1 I 24 1 SP310529WO00 By the AVC encoding method, a deblocking filter is included in each loop, as shown in Figs. 1 and 2. With this arrangement, block distortions can be effectively removed from decoded images, and motion compensation can 5 effectively prevent the block distortions from propagating to images referring to the decoded image. [0082] In the following, the operating principles in each deblocking filter according to the AVC encoding method 10 are described. [0083] As operations of a deblocking filter, the following three operations can be designated in accordance with the two parameters, deblocking_filter_control_present_flag in 15 Picture Parameter Set RBSP and disable_deblocking_filter_idc in Slice Header, which are I contained in compressed image information. I [0084] (a) To be performed on a block boundary or a 2 0 macroblock boundary (b) To be performed only on a macroblock boundary • (c) Not to be performed [0085] As for the quantization parameter QP, QPY is used 25 when the following operation is performed on luminance signals, and QPc is used when the following operation is performed on chrominance signals. In motion vector encoding, intra predictions, and entropy encoding (CAVLC/CABAC), pixel values that belong to a different | i 30 slice are processed as "not available". However, in f deblocking filtering operations, pixel values that belong I I 25 I SP310529WO00 to a different slice but belong to the same picture are processed as "available. [0086] In the following, pixel values yet to be subjected 5 to a deblocking filtering operation are represented by pO through p3 and qO through q3, and processed pixel values are represented by pO' through p3' and qO' through q3', as shown in Fig. 3. [0087] 10 As shown in Fig. 4, prior to a deblocking filtering operation, Bs (Boundary strengths) are defined on the ps and qs shown in Fig. 3. [0088] I Only when the following two conditions (the J 15 expression (1) and the expression (2)) are satisfied, is I a deblocking filtering operation performed on (p2, pi, pO, qO, ql, and q2) in Fig. 3. [0089] Bs > 0 ... (1) 20 Ip0-q0| < a; |pl-p0| < p; | ql-qO I < p ... (2) [0090] Although the default values of a and p in the expression (2) are defined in accordance with QP as shown below, the values can be adjusted by a user in accordance 25 with the two parameters "slice_alpha_c0_offset_div2" and "slice_beta_offset_div2" contained in the slice header in compressed image information (or in encoded data), as shown in Fig. 5. [0091] 30 The indexA and indexB shown in the tables in Figs. I 6A and 6B are defined as shown in the following 1 26 j I SP310529WO00 t expressions (3) through (5) . [0092] [Mathematical Formula 1] qPw = (qPp+qPq+l)»l 5 ... (3) [Mathematical Formula 2] indexA= Clip3(0,51, qPav + FilterOffsetA) ... (4) [Mathematical Formula 3] 10 indexB = Clip3(0,51, qPav + FilterOffsetB) ... (5) [0093] In the above expressions (3) through (5), "FilterOffsetA" and "FilterOffsetB" are the portions to 15 be adjusted by a user. [0094] Different methods are defined as deblocking filtering operations in cases (1) where Bs < 4, and (2) where Bs = 4, as described below. 20 [0095] Where Bs < 4, the pixel values p'O and q'O subjected to the deblocking filtering operation are calculated according to the following expressions (6) through (8). 25 [0096] [Mathematical Formula 4] I 27 | SP310529WO00 A=CUp3(-tc,tcf((((q0-p0)«2) + (ft-q1) + 4)»3)) ... (6) [Mathematical Formula 5] P; = ciipi(Po+A) 5 ... (7) [Mathematical Formula 6] q; = Clipl(q0 + A) ... (8) [0097] j i. 10 Here, tc is calculated as described below. That is, where the value of chromaEdgeFlag is 0, tc is calculated I I according to the expression (9) shown below. In other cases, tc is calculated according to the expression (10) 1 s shown below. 1 !' 15 [0098] [Mathematical Formula 7] tc = tCo+((ap>2)+2) j ... d9) i [Mathematical Formula 18] Po = (p2 + 2 * Pi + 2 • Po + 2 • q o+qi+ 4 ) » 3 ! 15 ... (20) ; 1 [Mathematical Formula 19] j p;=(p2+Pi+Po+qo+2)»2 ... (21) [Mathematical Formula 2 0] j 20 P2 =(2-p3+3.p2 + p1 +p0 +q0 + 4) » 3 j ... (22) 1 [Mathematical Formula 21] Po = (2-Pi + Po+qi+2)»2 ... (23) I 25 [Mathematical Formula 22] I 31 I SP310529WO00 PI = Pi ... (24) [Mathematical Formula 23] P2= P2 5 ... (25) I [0110] | The pixel values q'i (i = 0, . .., 2) subjected to j the deblocking filtering operation are calculated as ( described below. That is, when the value of 1 l 10 chromaEdgeFlag is 0, and the condition shown below (the i expression (26)) is satisfied, q'o, q'i, and q'2 are I calculated according to the expressions (27) through (29) shown below. When the above mentioned condition is not satisfied, q'or q'lr and q'2 are calculated according to i 15 the expressions (30) through (32) shown below. [0111] J [Mathematical Formula 24] I aq>2 1 25 ... (28) j [Mathematical Formula 27] j 32 I SP310529WO00 q;=(2-q3+3-q2 + q] + q0+p4 + 4)»3 ... (29) [Mathematical Formula 28] qi = (2-qi + qo+Pi+2)»2 5 ... (30) [Mathematical Formula 29] ... (31) [Mathematical Formula 30] 10 32 = [Another Example of an Image Encoding Device] Although the ON/OFF control performed by the adaptive loop filter in accordance with the type of an 20 image has been described above, the invention is not limited to that, and the number of taps of an adaptive loop filter may be controlled in accordance with the type of an image. [0213] 25 Specifically, in an adaptive loop filtering operation, the tap length may be changed in accordance with the type of an image, such as a picture type or a slice type. For example, in an adaptive loop filtering operation, a longer tap length may be used for a picture 30 to be referred to, and a shorter tap length may be used for a picture not to be referred to. 59 SP310529WO00 [0214] According to the method disclosed in Non-Patent Document 1, adaptive loop filtering operations are performed for all predetermined tap lengths, such as five 5 taps, seven taps, and nine taps, and the filtering operation result with the optimum tap length is selected in accordance with the costs of the respective operation results. [0215] 10 At this point, the tap lengths may be shortened byperforming filtering operations, with some of the respective coefficients being reduced to zero. For example, in a 9-tap filtering operation, the first coefficient and the ninth coefficient (the coefficients 15 at both ends) are reduced to zero (0), to substantially shorten the tap length (to seven taps). The tap lengths can also be shortened in a 5-tap filtering operation and a 7-tap filtering operation in the same manner as above. The number of coefficients to be reduced to zero is of 20 course arbitrarily determined. Also, it is possible to arbitrarily determine which coefficient(s) is (are) to be reduced zero. [0216] As the tap length in an adaptive loop filtering 25 operation for an image not to be referred to is shortened as described above, the amount of calculation can be reduced. In this case, a filtering operation is performed, though the tap length is shortened. Accordingly, the adverse influence on the image quality 30 of decoded images can be made smaller than that in the first embodiment. That is, image quality deterioration 60 SP310529WO00 in decoded image can be more effectively restrained than in the first embodiment. [0217] Fig. 2 0 is a block diagram showing exemplary 5 structures of the filter control unit and the adaptive loop filter used in that case. [0218] As shown in Fig. 2 0, the image encoding device 500 in this case includes a filter control unit 601 in place 10 of the filter control unit 501, and an adaptive loop filter 602 in place of the adaptive loop filter 502. [0219] While the filter control unit 501 controls switching on/off of the adaptive loop filtering operation 15 of the adaptive loop filter 502 in accordance with the type of the image being subjected to the adaptive loop filtering operation, the filter control unit 601 controls the tap length in the adaptive loop filtering operation of the adaptive loop filter 602 in accordance with the 20 type of the image being subjected to the adaptive loop filtering operation. [0220] More specifically, based on the information indicating a picture type (or a slice type) supplied from 25 the screen rearrangement buffer 102, the filter control unit 601 determines whether the image being subjected to the adaptive loop filtering operation is an "image to be referred to". When the image being subjected to the adaptive loop filtering operation is not an "image to be 30 referred to", the filter control unit 601 controls the operation of the adaptive loop filter 602 so as to 61 SP310529WO00 shorten the tap length. [0221] The filter control unit 601 supplies tap length information designating a tap length to a tap length 5 setting unit 611 of the adaptive loop filter 602. [0222] Under the control of the filter control unit 601, the adaptive loop filter 602 performs the adaptive loop filtering operation with the tap length that has been set 10 in accordance with the type of the image being subjected to the filtering operation. [0223] The adaptive loop filter 602 includes the tap length setting unit 611, a filter coefficient calculation 15 unit 612, and a filtering unit 513. [0224] The tap length setting unit 611 generates coefficient control information that is control information to issue an instruction to calculate a filter 20 coefficient of the tap length designated by the tap length information supplied from the filter control unit 601, and supplies the coefficient control information to the filter coefficient calculation unit 612. [0225] 25 That is, when the image being subjected to the adaptive loop filtering operation is not an "image to be referred to" as described above, the tap length setting unit 611 generates coefficient control information so as to shorten the tap length, and supplies the coefficient 30 control information to the filter coefficient calculation unit 612. In other words, when the image being subjected 62 j 1 SP310529WO00 j } * 1 to the adaptive loop filtering operation is an "image to 1 be referred to", the tap length setting unit 611 generates coefficient control information so as to increase the tap length, and supplies the coefficient 5 control information to the filter coefficient calculation unit 612. [0226] The tap length setting unit 611 includes a zero coefficient setting unit 621. The zero coefficient 10 setting unit 621 sets the value of some of filter coefficients calculated by the filter coefficient calculation unit 612, to zero. That is, the tap length setting unit 611 generates the coefficient control information designating zero as the value of some of the 15 filter coefficients calculated by the filter coefficient calculation unit 612. In this case, as some coefficients are set to zero, a desired tap length is realized. [0227] For example, when the filter coefficient 20 calculation unit 612 calculates filter coefficients of nine taps, the zero coefficient setting unit 621 sets the first coefficient and the ninth coefficient of the nine taps to zero. In this case, the coefficient control information designates seven taps. The filter 25 coefficient calculation unit 612 sets the values of the coefficients designated by the coefficient control information to zero, and calculates the other coefficients. As a result, the filter coefficient calculation unit 612 calculates the filter coefficients 30 of the seven taps. [0228] 63 SP310529WO00 The filter coefficient calculation unit 612 supplies the calculated filter coefficients to the filtering unit 513. In this case, the filter coefficient calculation unit 612 generates an ON/OFF flag having the 5 value of ON, and supplies the ON/OFF flag to the filtering unit 513. [0229] Using the filter coefficients supplied from the filter coefficient calculation unit 612, the filtering 10 unit 513 performs the adaptive loop filtering operation • on the image that has been subjected to the deblocking filtering operation and been supplied from the deblocking filter 111. [0230] 15 In this case, the filtering unit 513 supplies and stores the image subjected to the adaptive loop filtering operation into the frame memory 112. The filter coefficient calculation unit 612 supplies the calculated filter coefficients and the ON/OFF flag having the value 20 of ON to the lossless encoding unit 106, which encodes the filter coefficients and the ON/OFF flag. [0231] The encoding operation in this case is performed in the same manner as in the case described with reference 25 to the flowchart in Fig. 18. [0232] [Flow of the Adaptive Loop Filtering Operation] Referring now to the flowchart in Fig. 21, an example of the flow of the adaptive loop filtering 30 operation to be performed in this case is described. This flowchart is equivalent to the flowchart in Fig. 19. 64 SP310529WO00 m [0233] When the adaptive loop filtering operation is started, the filter control unit 601 determines the type of the image being subjected to the adaptive loop 5 filtering operation in step S631. [0234] In step S632, the filter control unit 601 determines whether the image being subjected to the adaptive loop filtering operation is an image to be 10 referred to. When the result of the type determination in step S631 shows that the image is an image to be referred to, the filter control unit 601 moves on to step S633. In step S633, the tap length setting unit 611 performs control so as to increase the filter coefficient 15 tap length, and the operation moves on to step S635. [0235] When it is determined in step S632 that the image being subjected to the adaptive loop filtering operation is not an image to be referred to, the filter control 20 unit 601 moves on to step S634. In step S634, the tap length setting unit 611 performs control so as to shorten the filter coefficient tap length, and the operation moves on to step S635. [0236] 25 In step S635, based on the image subjected to the deblocking filtering operation and the input image, the filter coefficient calculation unit 612 calculates appropriate filter coefficients. The filter coefficient calculation unit 612 generates the ON/OFF flag having the 30 value of ON. In step S636, the filtering unit 513 performs the adaptive loop filtering operation on the 65 SP310529WO00 « image subjected to the deblocking filtering operation, by using the filter coefficients calculated in step S635. [0237] In step S637, the filtering unit 513 supplies the I 5 ON/OFF flag and the filter coefficients used as described I above to the lossless encoding unit 106, which then j encodes the ON/OFF flag and the filter coefficients. I [0238] After ending the procedure of step S637, the I 10 adaptive loop filter 602 ends the adaptive loop filtering I operation. The operation then returns to step S513 of Fig. 18, and the procedures of step S514 and thereafter are carried out. [0239] 15 In the above manner, the filter control unit 601 can readily control the operation of the adaptive loop filter 602. Also, as the filter control unit 601 controls the tap length in the filtering operation of the adaptive loop filter 602 in accordance with the type of 20 the image, the image encoding device 500 can reduce the encoding operation load while restraining image quality deterioration in decoded images. [0240] In this case, encoded data that has been generated 25 and output by the image encoding device 500 as described above can also be decoded in the same manner as a conventional manner (in the same manner as in a case where encoded data generated by the image encoding device 300 is decoded) by a conventional image decoding device 30 (such as the image decoding device 400 that is disclosed in Non-Patent Document 1 and has been described with 66 1 i SP310529WO00 1 reference to Fig. 9). [0241] That is, using the information added to encoded data, such as the adaptive loop filter flag 5 (adaptive_loop_filter_flag) and the filter coefficients, the loop filter 401 performs the adaptive loop filtering operation, where appropriate, on the image subjected to the deblocking filtering operation by the deblocking filter 206. In this manner, the image decoding device 10 400 can restrain image quality deterioration in decoded images. [0242] [Example of an Extended Macroblock] In H.264/AVC, the macroblock size is 16 x 16 pixels. 15 However, the macroblock size of 16 x 16 pixels is not optimal for an UHD (Ultra High Definition: 4000 x 2000 pixels) frame to be encoded by a next-generation encoding method. In the image encoding device 500, the macroblock size can be 32 x 32 pixels, 64 x 64 pixels, or the like, 20 as shown in Fig. 22. [0243] Fig. 22 is a diagram showing examples of extended macroblock sizes. In the example shown in Fig. 32, the macroblock size is extended to 32 x 32 pixels. 25 [0244] In the top row in Fig. 22, macroblocks each formed with 32 x 32 pixels that are divided into a block (partition) of 32 x 32 pixels, blocks of 32 x 16 pixels, blocks of 16 x 32 pixels, and blocks of 16 x 16 pixels 30 are shown in this order. In the middle row in Fig. 22, blocks each formed with 16 x 16 pixels that are divided 67 SP310529WO00 V into a block of 16 x 16 pixels, blocks of 16 x 8 pixels, blocks of 8 x 16 pixels, and blocks of 8 x 8 pixels are shown in this order. In the bottom row in Fig. 22, blocks each formed with 8 x 8 pixels that are divided 5 into a block of 8 x 8 pixels, blocks of 8 x 4 pixels, blocks of 4 x 8 pixels, and blocks of 4 x 4 pixels are shown in this order. [0245] That is, a macroblock of 32 x 32 pixels can be 10 processed as the block of 32 x 32 pixels, the blocks of 32 x 16 pixels, the blocks of 16 x 32 pixels, or the blocks 16 x 16 pixels shown in the top row in Fig. 22. [0246] Each of the blocks of 16 x 16 pixels shown at the 15 right end of the top row can be processed as the block of 16 x 16 pixels, the blocks of 16 x 8 pixels, the blocks of 8 x 16 pixels, and the blocks of 8 x 8 pixels shown in the middle row, in the same manner as in H.2 64/AVC. [0247] 20 Each of the blocks of 8 x 8 pixels shown at the right end of the middle row can be processed as the block of 8 x 8 pixels, the blocks of 8 x 4 pixels, the blocks of 4 x 8 pixels, and the blocks of 4 x 4 pixels shown in the bottom row, in the same manner as in H.2 64/AVC. 25 [0248] Those blocks can be classified into the following three hierarchical levels. That is, the blocks of 32 x 32 pixels, 32 x 16 pixels, and 16 x 32 pixels shown in the top row in Fig. 22 are referred to as a first 30 hierarchical level. The blocks of 16 x 16 pixels shown at the right end of the top row, and the blocks of 16 x 68 SP310529WO00 m 16 pixels, 16 x 8 pixels, and 8 x 16 pixels shown in the middle row are referred to as a second hierarchical level. The blocks of 8 x 8 pixels shown at the right end of the middle row, and the blocks of 8 x 8 pixels, 8 x 4 pixels, 5 4 x 8 pixels, and 4 x 4 pixels shown in the bottom row are referred to as a third hierarchical level. [0249] The hierarchical structure shown in Fig. 22 is used, so that blocks of 16 x 16 pixels and smaller blocks 10 maintain compatibility with the macroblocks of the current H.264/AVC. As the supersets of those blocks, even larger blocks are defined. [0250] Any macroblock size may of course be used, and 15 larger macroblocks than 64 x 64 pixels may be defined, for example. [0251] <3. Third Embodiments [Personal Computer] 20 The above described series of operations can be performed by hardware or software. In this case, a personal computer shown in Fig. 23 may be formed, for example. [0252] 25 In Fig. 23, the CPU (Central Processing Unit) 701 of the personal computer 700 performs various kinds of operations in accordance with a program stored in a ROM (Read Only Memory) 702 or a program loaded into a RAM (Random Access Memory) 703 from a storage unit 713. The 30 data necessary for the CPU 7 01 to perform various kinds of operations is also stored in the RAM 703 where 69 1 SP310529WO00 necessary. [0253] The CPU 701, the ROM 702, and the RAM 703 are connected to one another via a bus 704. An input/output 5 interface 710 is also connected to the bus 704. [0254] An input unit 711 formed with a keyboard, a mouse, and the like, an output unit 712 formed with a display such as a CRT (Cathode Ray Tube) or an LCD (Liquid 10 Crystal Display) and a speaker or the like, the storage unit 713 formed with a hard disk or the like, and a communication unit 714 formed with a modem or the like are connected to the input/output interface 710. The communication unit 714 performs communicating operations 15 via networks including the Internet. [0255] A drive 715 is also connected to the input/output interface 710 where necessary, and a removable medium 721 such as a magnetic disk, an optical disk, a magneto- 20 optical disk, a semiconductor memory, or the like is mounted on the drive 715 where appropriate. Computer programs read out from those media are installed in the storage unit 713 where necessary. [0256] 25 When the above described series of operations are performed by software, a program to form the software is installed from a network or a recording medium. [0257] This recording medium may be distributed to deliver 30 the program to users, separately from the device, as shown in Fig. 23. For example, this recording medium may 70 SP310529WO00 * be formed with the removable medium 721, such as a magnetic disk (or a flexible disk) having the program recorded thereon, an optical disk (or a CD-ROM (Compact Disc-Read Only Memory or a DVD (Digital Versatile Disc)), 5 a magneto-optical disk (or a MD (Mini Disc)), or a semiconductor memory. Alternatively, this recording medium may be formed with the ROM 702 having the program I recorded thereon, or a hard disk contained in the storage unit 713, or the like. The ROM 702 and the hard disk are 10 incorporated into the device beforehand, and are. distributed to users. [0258] Each program to be executed by the computer may be a program for performing operations in chronological 15 order in accordance with the sequences described in this specification, or may be a program for performing operations where necessary in parallel or when there is a call or the like. [0259] 20 In this specification, the step of writing a program to be recorded on a recording medium includes not only operations to be performed in chronological order in accordance with the disclosed sequences, but also operations to be performed in parallel or independently 25 of one another if not in chronological order. [0260] In this specification, a "system" means an entire apparatus formed with two or more devices (apparatuses). [0261] 30 In the above description, any structure described as one device (or one processing unit) may be divided and 71 1 SP310529WO00 formed as two or more devices (or processing units). Conversely, any structure described as two or more devices (or processing units) may be formed as one device (or one processing unit). Also, a structure that has not 5 been described above may of course be added to the structure of each device (or each processing unit). Further, as long as the structure and operations of the entire system will remain substantially the same, part of the structure of a device (or a processing unit) may be 10 incorporated into the structure of another device (or another processing unit). That is, embodiments of this technique are not limited to the above described embodiments, and various modifications may be made to them without departing from the scope of the technique. 15 [0262] For example, the above described image encoding device and the above described image decoding device can be applied to any electronic apparatuses. In the following, examples of such applications are described. 20 [0263] <4. Fourth Embodiment> [Television Receiver] Fig. 24 is a block diagram showing a typical exemplary structure of a television receiver using the 25 image decoding device 400. [0264] The television receiver 1000 shown in Fig. 24 includes a terrestrial tuner 1013, a video decoder 1015, a video signal processing circuit 1018, a graphic 30 generation circuit 1019, a panel drive circuit 1020, and a display panel 1021. 72 SP310529WO00 [0265] I The terrestrial tuner 1013 receives a broadcast wave signal of analog terrestrial broadcasting via an antenna, demodulates the signal to obtain a video signal. 5 The terrestrial tuner 1013 supplies the video signal to the video decoder 1015. The video decoder 1015 performs a decoding operation on the video signal supplied from the terrestrial tuner 1013, and supplies the resultant digital component signal to the video signal processing 10 circuit 1018, [0266] The video signal processing circuit 1018 performs predetermined processing such as denoising on the video 1 data supplied from the video decoder 1015, and supplies I 15 the resultant video data to the graphic generation I circuit 1019. I [0267] The graphic generation circuit 1019 generates video data of a show to be displayed on the display panel 1021, 20 or image data by performing an operation based on an application supplied via a network. The graphic generation circuit 1019 supplies the generated video data or the image data to the panel drive circuit 1020. The graphic generation circuit 1019 also generates video data 25 (a graphic) for displaying a screen to be used by a user to select an item, and superimposes the video data on the video data of the show. The resultant video data is supplied to the panel drive circuit 102 0 where appropriate. 30 [0268] Based on the data supplied from the graphic 73 SP310529WO00 generation circuit 1019, the panel drive circuit 1020 drives the display panel 1021, and causes the display panel 1021 to display the video image of the show and each screen described above. 5 [0269] The display panel 1021 is formed with an LCD (Liquid Crystal Display) or the like, and displays the video image of a show or the like under the control of the panel drive circuit 1020. 10 [0270] The television receiver 1000 also includes an audio A/D (Analog/Digital) converter circuit 1014, an audio signal processing circuit 1022, an echo cancellation/voice synthesis circuit 1023, an audio 15 amplifier circuit 1024, and a speaker 1025. [0271] The terrestrial tuner 1013 obtains not only a video signal but also an audio signal by demodulating a received broadcast wave signal. The terrestrial tuner 20 1013 supplies the obtained audio signal to the audio A/D converter circuit 1014. [0272] The audio A/D converter circuit 1014 performs an A/D converting operation on the audio signal supplied 25 from the terrestrial tuner 1013, and supplies the resultant digital audio signal to the audio signal processing circuit 1022. [0273] The audio signal processing circuit 1022 performs 30 predetermined processing such as denoising on the audio data supplied from the audio A/D converter circuit 1014, 74 SP310529WO00 * and supplies the resultant audio data to the echo cancellation/voice synthesis circuit 1023. [0274] The echo cancellation/voice synthesis circuit 1023 5 supplies the audio data supplied from the audio signal processing circuit 1022 to the audio amplifier circuit 1024. [0275] The audio amplifier circuit 1024 performs a D/A 10 converting operation and an amplifying operation on the audio data supplied from the echo cancellation/voice synthesis circuit 1023. After adjusted to a predetermined sound volume, the sound is output from the speaker 1025. 15 [0276] The television receiver 1000 further includes a digital tuner 1016 and an MPEG decoder 1017. [0277] The digital tuner 1016 receives a broadcast wave 20 signal of digital broadcasting (digital terrestrial broadcasting or digital BS (Broadcasting Satellite)/CS (Communications Satellite) broadcasting) via the antenna, and demodulates the broadcast wave signal, to obtain an MPEG-TS (Moving Picture Experts Group-Transport Stream) . 25 The MPEG-TS is supplied to the MPEG decoder 1017. [0278] The MPEG decoder 1017 descrambles the MPEG-TS supplied from the digital tuner 1016, and extracts the stream containing the data of the show to be reproduced 30 (to be viewed) . The MPEG decoder 1017 decodes the audio packet forming the extracted stream, and supplies the 75 SP310529WO00 * resultant audio data to the audio signal processing circuit 1022. The MPEG decoder 1017 also decodes the video packet forming the stream, and supplies the resultant video data to the video signal processing 5 circuit 1018. The MPEG decoder 1017 also supplies EPG (Electronic Program Guide) data extracted from the MPEGTS to a CPU 1032 via a path (not shown). [0279] The television receiver 1000 uses the image 10 decoding device 400 as the MPEG decoder 1017, which decodes the video packet as described above. The MPEG-TS transmitted from a broadcast station or the like has been encoded by the image encoding device 500. [0280] 15 Like the image decoding device 400, the MPEG decoder 1017 has the loop filter 401 to perform an adaptive loop filtering operation, where appropriate, on an image that has been subjected to a deblocking filtering operation by the deblocking filter 206, by 20 using information supplied from a broadcast station (the image encoding device 500) , such as an adaptive loop filter flag (adaptive_loop_filter_flag) and a filter coefficient. Accordingly, the MPEG decoder 1017 can perform an adaptive loop filtering operation more suited 25 to the contents of images, and restrain image quality deterioration in decoded images. 1 [0281] I The video data supplied from the MPEG decoder 1017 is subjected to predetermined processing at the video 30 signal processing circuit 1018, as in the case of the video data supplied from the video decoder 1015. At the 76 SP310529WO00 graphic generation circuit 1019, generated video data and the like are superimposed on the video data where appropriate. The resultant video data is supplied to the display panel 1021 via the panel drive circuit 1020, and 5 the image is displayed. [0282] The audio data supplied from the MPEG decoder 1017 is subjected to predetermined processing at the audio signal processing circuit 1022, as in the case of the 10 audio data supplied from the audio A/D converter circuit 1014. The resultant audio data is supplied to the audio amplifier circuit 1024 via the echo cancellation/voice synthesis circuit 1023, and is subjected to a D/A converting operation or an amplifying operation. As a 15 result, a sound that is adjusted to a predetermined sound level is output from the speaker 1025. [0283] The television receiver 1000 also includes a microphone 1026 and an A/D converter circuit 1027. 20 [0284] The A/D converter circuit 1027 receives a signal of a user's voice captured by the microphone 1026 provided for voice conversations in the television receiver 1000. The A/D converter circuit 1027 performs an A/D converting 25 operation on the received audio signal, and supplies the resultant digital audio data to the echo cancellation/voice synthesis circuit 1023. [0285] When audio data of a user (a user A) of the 30 television receiver 1000 is supplied from the A/D converter circuit 1027, the echo cancellation/voice 77 SP310529WO00 synthesis circuit 1023 performs echo cancellation on the audio data of the user A, and combines the audio data with other audio data or the like. The resultant audio data is output from the speaker 1025 via the audio 5 amplifier circuit 1024. [0286] The television receiver 1000 further includes an audio codec 1028, an internal bus 1029, an SDRAM (Synchronous Dynamic Random Access Memory) 1030, a flash 10 memory 1031, the CPU 1032, a USB (Universal Serial Bus) I/F 1033, and a network I/F 1034. [0287] The A/D converter circuit 1027 receives the signal of the user's voice captured by the microphone 1026 15 provided for voice conversations in the television receiver 1000. The A/D converter circuit 1027 performs an A/D converting operation on the received audio signal, and supplies the resultant digital audio data to the audio codec 1028. 20 [0288] The audio codec 1028 transforms the audio data supplied from the A/D converter circuit 1027 into data in a predetermined format for transmission via a network, and supplies the result to the network I/F 1034 via the 25 internal bus 1029. [0289] The network I/F 1034 is connected to a network via a cable attached to a network terminal 1035. The network I/F 1034 transmits the audio data supplied from the audio 30 codec 1028 to another device connected to the network, for example. The network I/F 1034 also receives, via the 78 I SP310529WO00 network terminal 1035, audio data transmitted from another device connected to the network, and supplies the audio data to the audio codec 102 8 via the internal bus 1029. 5 [0290] The audio codec 102 8 transforms the audio data supplied from the network I/F 1034 into data in a predetermined format, and supplies the result to the echo cancellation/voice synthesis circuit 1023. 10 [0291] The echo cancellation/voice synthesis circuit 1023 performs echo cancellation on the audio data supplied from the audio codec 102 8, and combines the audio data with other audio data or the like. The resultant audio 15 data is output from the speaker 1025 via the audio amplifier circuit 1024. [0292] The SDRAM 1030 stores various kinds of data necessary for the CPU 1032 to perform processing. 20 [0293] The flash memory 1031 stores the program to be executed by the CPU 1032. The program stored in the flash memory 1031 is read by the CPU 1032 at a predetermined time, such as when the television receiver 25 1000 is activated. The flash memory 1031 also stores EPG data obtained through digital broadcasting, data obtained from a predetermined server via a network, and the like. j I [0294] For example, the flash memory 1031 stores a MPEG-TS j 30 containing content data obtained from a predetermined ! t i server via a network, under the control of the CPU 1032. j 79 i ! j; i" t SP310529WO00 The flash memory 1031 supplies the MPEG-TS to the MPEG decoder 1017 via the internal bus 1029, under the control of the CPU 1032, for example. [0295] 5 The MPEG decoder 1017 processes the MPEG-TS, as in the case of the MPEG-TS supplied from the digital tuner 1016. In this manner, the television receiver 1000 receives the content data formed with a video image and a sound via the network, and decodes the content data by 10 using the MPEG decoder 1017, to display the video image and output the sound. [0296] The television receiver 1000 also includes a light receiving unit 1037 that receives an infrared signal 15 transmitted from a remote controller 1051. [0297] [ The light receiving unit 1037 receives an infrared ray from the remote controller 1051, and outputs a control code indicating the contents of a user operation s 20 obtained through decoding, to the CPU 1032. [0298] j j The CPU 1032 executes the program stored in the j flash memory 1031, and controls the entire operation of the television receiver 1000 in accordance with the t i 25 control code and the like supplied from the light I receiving unit 1037. The respective components of the television receiver 1000 are connected to the CPU 1032 I via paths (not shown). j [0299] I j 30 The USB I/F 1033 exchanges data with an apparatus | that is located outside the television receiver 1000 and ! i I 80 | jj i SP310529WO00 # is connected to thereto via a USB cable attached to a USB terminal 1036. The network I/F 1034 is connected to the network via the cable attached to the network terminal 1035, and also exchanges data other than audio data with 5 any kinds of devices connected to the network. [0300] Using the image decoding device 400 as the MPEG decoder 1017, the television receiver 1000 can perform an adaptive loop filtering operation more suited to the 10 contents of images on broadcast wave signals received via an antenna or content data obtained via a network, and can restrain deterioration of the subjective image quality of decoded images. [0301] 15 <5. Fifth Embodiment> [Portable Telephone Device] Fig. 25 is a block diagram showing a typical exemplary structure of a portable telephone device using the image encoding device 500 and the image decoding 20 device 400. [0302] The portable telephone device 1100 shown in Fig. 25 includes a main control unit 1150 designed to collectively control respective components, a power 25 source circuit unit 1151, an operation input control unit 1152, an image encoder 1153, a camera I/F unit 1154, an LCD control unit 1155, an image decoder 1156, a multiplexing/separating unit 1157, a j recording/reproducing unit 1162, a 30 modulation/demodulation circuit unit 1158, and an audio codec 1159. Those components are connected to one 81 j SP310529WO00 another via a bus 1160. [0303] The portable telephone device 1100 also includes operation keys 1119, a CCD (Charge Coupled Device) camera 5 1116, a liquid crystal display 1118, a storage unit 1123, a transmission/reception circuit unit 1163, an antenna 1114, a microphone (mike) 1121, and a speaker 1117. [0304] When a call is ended or the power key is switched 10: on by a user's operation, the power source circuit unit. 1151 puts the portable telephone device 1100 into an operable state by supplying power from a battery pack to the respective components. [0305] 15 Under the control of the main control unit 1150 formed with a CPU, a ROM, a RAM, and the like, the portable telephone device 1100 performs various kinds of operations, such as transmission and reception of audio signals, transmission and reception of electronic mail 2 0 and image data, image capturing, and data recording, in various kinds of modes such as a voice communication mode and a data communication mode. [0306] In the portable telephone device 1100 in the voice 25 communication mode, for example, an audio signal captured by the microphone (mike) 1121 is transformed into digital audio data by the audio codec 1159, and the digital audio data is subjected to spread spectrum processing at the modulation/demodulation circuit unit 1158. The resultant 30 data is then subjected to a digital-analog converting : i. operation and a frequency converting operation at the I i I 82 f r | i SP310529WO00 transmission/reception circuit unit 1163. The portable telephone device 1100 transmits the transmission signal obtained through the converting operations to a base station (not shown) via the antenna 1114. The 5 transmission signal (audio signal) transmitted to the base station is further supplied to the portable telephone device at the other end of the communication via a public telephone line network. [0307] 10 In the portable telephone device 1100 in the voice communication mode,' for example, a reception signal received by the antenna 1114 is amplified at the transmission/reception circuit unit 1163, and is further subjected to a frequency converting operation and an 15 analog-digital converting operation. The resultant signal is subjected to inverse spread spectrum processing at the modulation/demodulation circuit unit 1158, and is transformed into an analog audio signal by the audio codec 1159. The portable telephone device 1100 outputs, 20 from the speaker 1117, the-analog audio signal obtained through the conversions. [0308] Further, when electronic mail is transmitted in the data communication mode, for example, the operation input 25 control unit 1152 of the portable telephone device 1100 receives text data of the electronic mail that is input by operating the operation keys 1119. The portable telephone device 1100 processes the text data at the main control unit 1150, and displays the text data as an image 30 on the liquid crystal display 1118 via the LCD control unit 1155. ' I ! 83 I I I SP310529WO00 [0309] In the portable telephone device 1100, the main control unit 1150 generates electronic mail data, based on text data, a user's instruction, or the like received 5 by the operation input control unit 1152. The portable telephone device 1100 subjects the electronic mail data to spread spectrum processing at the modulation/demodulation circuit unit 1158, and to a digital-analog converting operation and a frequency 10 converting operation at the transmission/reception circuit unit 1163. The portable telephone device 1100 transmits the transmission signal obtained through the converting operations to a base station (not shown) via the antenna 1114. The transmission signal (electronic 15 mail) transmitted to the base station is supplied to a predetermined address via a network, a mail server, and the like. [0310] When electronic mail is received in the data 20 communication mode, for example, the transmission/reception circuit unit 1163 of the portable telephone device 1100 receives a signal transmitted from a base station via the antenna 1114, and the signal is amplified and is further subjected to a frequency 25 converting operation and an analog-digital converting operation. The portable telephone device 1100 subjects the received signal to inverse spread spectrum processing at the modulation/demodulation circuit unit 1158, to uncompress the original electronic mail data. The 30 portable telephone device 1100 displays the uncompressed electronic mail data on the liquid crystal display 1118 84 SP310529WO00 via the LCD control unit 1155. [0311] The portable telephone device 1100 can also record (store) the received electronic mail data into the 5 storage unit 1123 via the recording/reproducing unit 1162. [0312] The storage unit 1123 is a rewritable storage medium. The storage unit 1123 may be a semiconductor memory such as a RAM or an internal flash memory, a hard 10 disk, or a removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card. It is of course possible to use a memory other than the above. [0313] 15 Further, when image data is transmitted in the data t communication mode, for example, the portable telephone device 1100 generates the image data at the CCD camera 1116 capturing an image. The CCD camera 1116 includes optical devices such as a lens and a diaphragm, and a CCD 20 as a photoelectric conversion device. The CCD camera 1116 captures an image of an object, converts the intensity of received light into an electrical signal, | i and generates image data of the image of the object. The j. CCD camera 1116 encodes the image data at the image | 25 encoder 1153 via the camera I/F unit 1154, to obtain [ encoded image data. j [0314] j I The portable telephone device 1100 uses the above | described image encoding devxce 500 as the image encoder 30 1153 that performs the above operation. In the same S manner as in the case of the image encoding device 500, i i i l 85 I i i SP310529WO00 the image encoder 1153 has the filter control unit 501 to control the operation of the adaptive loop filter 502 in accordance with types of images. By doing so, the image encoder 1153 can perform an adaptive loop filtering 5 operation more suited to images, and can reduce the encoding operation load while restraining image quality deterioration in decoded images. [0315] At the same time as above, in the portable 10 telephone device 1100, the sound captured by the microphone (mike) 1121 during the image capturing by the CCD camera 1116 is analog-digital converted at the audio codec 1159, and is further encoded. [0316] 15 The multiplexing/separating unit 1157 of the portable telephone device 1100 multiplexes the encoded image data supplied from the image encoder 1153 and the digital audio data supplied from the audio codec 1159 by a predetermined technique. The portable telephone device 20 1100 subjects the resultant multiplexed data to spread spectrum processing at the modulation/demodulation circuit unit 1158, and to a digital-analog converting operation and a frequency converting operation at the transmission/reception circuit unit 1163. The portable 25 telephone device 1100 transmits the transmission signal obtained through the converting operations to a base station (not shown) via the antenna 1114. The transmission signal (image data) transmitted to the base station is supplied to the other end of the communication 30 via a network or the like. [0317] 86 I SP310529WO00 When image data is not transmitted, the portable telephone device 1100 can also display image data generated at the CCD camera 1116 on the liquid crystal display 1118 via the LCD control unit 1155, instead of 5 the image encoder 1153. [0318] When the data of a moving image file linked to a simplified homepage or the like is received in the data communication mode, the transmission/reception circuit 10 unit 1163 of the portable telephone device 1100 receives a signal transmitted from a base station via the antenna 1114. The signal is amplified, and is further subjected to a frequency converting operation and an analog-digital converting operation. The portable telephone device 1100 15 subjects the received signal to inverse spread spectrum processing at the modulation/demodulation circuit unit 1158, to uncompress the original multiplexed data. The portable telephone device 1100 divides the multiplexed data into encoded image data and audio data at the 20 multiplexing/separating unit 1157. [0319] By decoding the encoded image data at the image decoder 1156, the portable telephone device 1100 generates reproduced moving image data, and displays the 25 reproduced moving image data on the liquid crystal display 1118 via the LCD control unit 1155. In this manner, the moving image data contained in a moving image : I file linked to a simplified homepage, for example, is ! displayed on the liquid crystal display 1118. f 30 [0320] I i The portable telephone device 1100 uses the above ! f S- 87 SP310529WO00 • described image decoding device 400 as the image decoder 1156 that performs the above operation. Like the image decoding device 400, the image decoder 1156 performs an adaptive loop filtering operation, where appropriate, on 5 an image that has been subjected to a deblocking filtering operation by the deblocking filter 206, byusing information supplied from the encoding side (the image encoding device 500), such as an adaptive loop filter flag (adaptive_loop_filter_flag) and a filter 10 coefficient. Accordingly, the image decoder 1156 can perform an inverse quantization operation more suited to the contents of images, and restrain image quality deterioration in decoded images. [0321] 15 At the same time as above, the portable telephone i t device 1100 transforms the digital audio data into an j analog audio signal at the audio codec 1159, and outputs 1 ! the analog audio signal from the speaker 1117. In this manner, the audio data contained in a moving image file 20 linked to a simplified homepage, for example, is reproduced. [0322] As in the case of electronic mail, the portable telephone device 1100 can also record (store) received 25 data linked to a simplified homepage or the like into the storage unit 1123 via the recording/reproducing unit 1162. [0323] The main control unit 1150 of the portable j telephone device 1100 can also analyze a two-dimensional 30 code obtained by the CCD camera 1116 performing image I' capturing, to obtain the information recorded in the two- j 1 j 88 | l i SP310529WO00 dimensional code. [0324] Further, an infrared communication unit 1181 of the portable telephone device 1100 can communicate with an 5 external apparatus by using infrared rays. [0325] By using the image encoding device 500 as the image encoder 1153, the portable telephone device 1100 can perform an adaptive loop filtering operation more suited 10 to images, and generate encoded image data so as to reduce the encoding operation load while restraining deterioration of the subjective image quality of decoded images, when image data generated at the CCD camera 1116 | I is encoded and transmitted, for example. I 15 [0326] I Also, by using the image decoding device 400 as the I image decoder 1156, the portable telephone device 1100 j can perform an adaptive loop filtering operation more | suited to images, and restrain deterioration of the j 20 subjective image quality of decoded images, when the data j | (encoded data) of a moving image file linked to a ! i simplified homepage is decoded, for example. | i [0327] [ i In the above description, the portable telephone i 25 device 1100 uses the CCD camera 1116. However, instead I of the CCD camera 1116, an image sensor (a CMOS image j sensor) using a CMOS (Complementary Metal Oxide [ I Semiconductor) may be used. In that case, the portable I i telephone device 1100 can also capture an image of an j i 30 object, and generate the image data of the image of the j object, as in the case where the CCD camera 1116 is used. 1 l I 89 | i i SP310529WO00 [0328] Although the portable telephone device 1100 has been described above, the image encoding device 500 and the image decoding device 400 can also be applied to any 5 device in the same manner as in the case of the portable telephone device 1100, as long as the device has the same image capturing function and the same communication [ function as the portable telephone device 1100. Such a ! device may be a PDA (Personal Digital Assistant), a 10 smartphone, an UMPC (Ultra Mobile Personal Computer), a netbook, or a notebook personal computer, for example. [0329] <6. Sixth Embodiment> [Hard Disk Recorder] 15 Fig. 2 6 is a block diagram showing a typical exemplary structure of a hard disk recorder using the image encoding device 500 and the image decoding device 400. [0330] 20 The hard disk recorder (a HDD recorder) 1200 shown in Fig. 2 6 is a device that stores, into an internal hard disk, the audio data and the video data of a broadcast show contained in a broadcast wave signal (a television signal) that is transmitted from a satellite or a 25 terrestrial antenna or the like and is received by a tuner, and provides the stored data to a user at a time designated by an instruction from the user. [0331] i I I The hard disk recorder 1200 can extract audio data I ! 30 and video data from a broadcast wave signal, for example, [ decode those data where appropriate, and store the data 90 SP310529WO00 into an internal hard disk. Also, the hard disk recorder 12 00 can obtain audio data and video data from another device via a network, for example, decode those data where appropriate, and store the data into an internal 5 hard disk. [0332] Further, the hard disk recorder 1200 can decode audio data and video data recorded on an internal hard disk, for example, supply those data to a monitor 1260, 10 display the image on the screen of the monitor 12 60, and output the sound from the speaker of the monitor 12 60. I Also, the hard disk recorder 1200 can decode audio data j and video data extracted from a broadcast wave signal obtained via a tuner, or audio data and video data 15 obtained from another device via a network, for example, supply those data to the monitor 12 60, display the image on the screen of the monitor 12 60, and output the sound from the speaker of the monitor 12 60. [0333] 20 The hard disk recorder 1200 can of course perform operations other than the above. [0334] As shown in Fig. 2 6, the hard disk recorder 1200 includes a reception unit 1221, a demodulation unit 1222, 25 a demultiplexer 1223, an audio decoder 1224, a video decoder 1225, and a recorder control unit 1226. The hard disk recorder 1200 further includes an EPG data memory 1227, a program memory 1228, a work memory 1229, a display converter 1230, an OSD (On-Screen Display) 30 control unit 1231, a display control unit 1232, a ! [ recording/reproducing unit 1233, a D/A converter 1234, i I • 91 i ! i s l i r SP310529WO00 and a communication unit 1235. [0335] The display converter 1230 includes a video encoder 1241. The recording/reproducing unit 1233 includes an 5 encoder 1251 and a decoder 1252. [0336] The reception unit 1221 receives an infrared signal from a remote controller (not shown), converts the infrared signal into an electrical signal, and outputs 10 the electrical signal to the recorder control unit 122 6. The recorder control unit 122 6 is formed with a microprocessor, for example, and performs various kinds of operations in accordance with a program stored in the program memory 1228. At this point, the recorder control 15 unit 1226 uses the work memory 1229 where necessary. [0337] The communication unit 1235 is connected to a network, and performs a communication operation with another device via the network. For example, under the 20 control of the recorder control unit 1226, the communication unit 1235 communicates with a tuner (not shown), and outputs a station select control signal mainly to the tuner. [0338] 25 The demodulation unit 1222 demodulates a signal supplied from the tuner, and outputs the signal to the demultiplexer 1223. The demultiplexer 1223 divides the data supplied from the demodulation unit 1222 into audio data, video data, and EPG data. The demultiplexer 1223 30 outputs the audio data, the video data, and the EPG data to the audio decoder 1224, the video decoder 1225, and 92 SP310529WO00 the recorder control unit 1226, respectively. [0339] The audio decoder 1224 decodes the input audio data, and outputs the decoded audio data to the 5 recording/reproducing unit 1233. The video decoder 1225 decodes the input video data, and outputs the decoded video data to the display converter 1230. The recorder control unit 122 6 supplies and stores the input EPG data into the EPG data memory 1227. 10 [0340] The display converter 1230 encodes video data supplied from the video decoder 1225 or the recorder control unit 122 6 into video data compliant with the NTSC (National Television Standards Committee) standards, for 15 example, using the video encoder 1241. The encoded video data is output to the recording/reproducing unit 1233. Also, the display converter 1230 converts the screen size of video data supplied from the video decoder 1225 or the recorder control unit 122 6 into a size compatible with 20 the size of the monitor 1260. The video encoder 1241 [ converts the video data into video data compliant with j the NTSC standards. The NTSC video data is converted [ Iinto an analog signal, and is output to the display control unit 1232. ! 25 [0341] I t Under the control of the recorder control unit 1226, ! [ the display control unit 1232 superimposes an OSD signal [ output from the OSD (On-Screen Display) control unit 1231 [ on the video signal input from the display converter 1230, f 30 and outputs the resultant signal to the display of the [ i monitor 12 60 to display the image. j 1 i | 93 i SP310529WO00 [0342] Audio data that is output from the audio decoder 1224 and is converted into an analog signal by the D/A converter 1234 is also supplied to the monitor 1260. The 5 monitor 12 60 outputs the audio signal from an internal speaker. [0343] The recording/reproducing unit 1233 includes a hard disk, as a storage medium for recording video data, audio 10 data, and the like. [0344] The recording/reproducing unit 1233 causes the encoder 1251 to encode audio data supplied from the audio decoder 1224, for example. The recording/reproducing 15 unit 1233 also causes the encoder 1251 to encode video data supplied from the video encoder 1241 of the displayconverter 1230. The recording/reproducing unit 1233 combines the encoded data of the audio data with the encoded data of the video data, using a multiplexer. The 20 recording/reproducing unit 1233 amplifies the combined data through channel coding, and writes the resultant data on the hard disk via a recording head. [0345] The recording/reproducing unit 1233 reproduces data 25 recorded on the hard disk via a reproduction head, amplifies the data, and divides the data into audio data and video data by using a demultiplexer. The recording/reproducing unit 1233 decodes the audio data and the video data by using the decoder 1252. The 30 recording/reproducing unit 1233 performs a D/A conversion on the decoded audio data, and outputs the result to the 94 !• ; r '• I i SP310529WO00 speaker of the monitor 12 60. The recording/reproducing unit 1233 also performs a D/A conversion on the decoded video data, and outputs the result to the display of the monitor 1260. 5 [0346] Based on a user's instruction indicated by an infrared signal that is transmitted from a remote i controller and is received via the reception unit 1221, the recorder control unit 122 6 reads the latest EPG data 10 from the EPG data memory 1227, and supplies the EPG data | to the OSD control unit 1231. The OSD control unit 1231 | i generates image data corresponding to the input EPG data, i and outputs the image data to the display control unit | ! 1232. The display control unit 1232 outputs the video i [ 15 data input from the OSD control unit 1231 to the display j i of the monitor 1260, to display the image. In this manner, an EPG (Electronic Program Guide) is displayed on j j the display of the monitor 12 60. [0347] 20 The hard disk recorder 1200 can also obtain various kinds of data, such as video data, audio data, and EPG data, which are supplied from another device via a | I l network such as the Internet. ; [0348] | 25 Under the control of the recorder control unit 1226, j the communication unit 1235 obtains encoded data of video i i data, audio data, EPG data, and the like from another j j' device via a network, and supplies those data to the j recorder control unit 122 6. For example, the recorder j 30 control unit 122 6 supplies encoded data of obtained video I data and audio data to the recording/reproducing unit j j [ 95 ! j i ! I SP310529WO00 1233, and stores those data into the hard disk. At this point, the recorder control unit 122 6 and the recording/reproducing unit 1233 may perform an operation such as a re-encoding where necessary. 5 [0349] The recorder control unit 1226 also decodes encoded data of obtained video data and audio data, and supplies the resultant video data to the display converter 1230. The display converter 1230 processes the video data 10 supplied from the recorder control unit 122 6 in the same manner as processing video data supplied from the video decoder 1225, and supplies the result to the monitor 1260 via the display control unit 1232, to display the image. [0350] 15 In synchronization with the image display, the recorder control unit 122 6 may supply the decoded audio data to the monitor 12 60 via the D/A converter 1234, and output the sound from the speaker. [0351] 20 Further, the recorder control unit 1226 decodes encoded data of obtained EPG data, and supplies the decoded EPG data to the EPG data memory 1227. [0352] The above described hard disk recorder 1200 uses 25 the image decoding device 400 as the video decoder 1225, the decoder 1252, and the decoder installed in the recorder control unit 122 6. That is, like the image decoding device 400, the video decoder 1225, the decoder 1252, and the decoder installed in the recorder control 30 unit 122 6 perform an adaptive loop filtering operation, where appropriate, on an image that has been subjected to 96 SP310529WO00 a deblocking filtering operation by the deblocking filter 206, by using information supplied from the encoding side (the image encoding device 500), such as an adaptive loop filter flag (adaptive_loop_filter_flag) and a filter 5 coefficient. Accordingly, the video decoder 1225, the decoder 1252, and the decoder installed in the recorder control unit 1226 can perform an adaptive loop filtering operation more suited to images, and restrain image quality deterioration in decoded images. 10 [0353] Thus, the hard disk recorder 1200 can perform an adaptive loop filtering operation more suited to image, on video data (encoded data) received by a tuner or the communication unit 1235 and video data (encoded data) to 15 be reproduced by the recording/reproducing unit 1233, and restrain deterioration of the subjective image quality of decoded images. [0354] The hard disk recorder 1200 also uses the image 20 encoding device 500 as the encoder 1251. Accordingly, in the same manner as in the case of the image encoding device 500, the encoder 1251 has the filter control unit 501 to control the operation of the adaptive loop filter 502 in accordance with types of images. By doing so, the 25 encoder 1251 can perform an adaptive loop filtering operation more suited to images, and can reduce the encoding operation load while restraining image quality deterioration in decoded images. [0355] 30 Accordingly, when encoded data to be recorded on a hard disk is generated, the hard disk recorder 1200 can 97 SP310529WO00 # perform an adaptive loop filtering operation more suited to images, and generate encoded data so as to reduce the encoding operation load while restraining deterioration of the subjective image quality of decoded images. 5 [0356] In the above description, the hard disk recorder 12 00 that records video data and audio data on a hard disk has been described. However, any other recording medium may be used. For example, as in the case of the 10 above described hard disk recorder 1200, the image encoding device 500 and the image decoding device 400 can be applied to a recorder that uses a recording medium other than a hard disk, such as a flash memory, an optical disk, or a videotape. 15 [0357] <7. Seventh Embodiment> [Camera] Fig. 27 is a block diagram showing a typical exemplary structure of a camera using the image encoding 20 device 500 and the image decoding device 400. [0358] The camera 1300 shown in Fig. 27 captures an image of an object, and displays the image of the object on an LCD 1316 or records the image of the object as image data 25 on a recording medium 1333. [0359] A lens block 1311 has light (or a video image of an object) incident on a CCD/CMOS 1312. The CCD/CMOS 1312 is an image sensor using a CCD or a CMOS. The CCD/CMOS 30 1312 converts the intensity of the received light into an electrical signal, and supplies the electrical signal to 98 SP310529WO00 a camera signal processing unit 1313. [0360] The camera signal processing unit 1313 transforms the electrical signal supplied from the CCD/CMOS 1312 5 into a YCrCb chrominance signal, and supplies the signal to an image signal processing unit 1314. Under the control of a controller 1321, the image signal processing 1 unit 1314 performs predetermined image processing on the image signal supplied from the camera signal processing 10 unit 1313, and encodes the image signal by using an encoder 1341. The image signal processing unit 1314 supplies the encoded data generated by encoding the image signal to a decoder 1315. The image signal processing unit 1314 further obtains display data generated at an 15 on-screen display (OSD) 1320, and supplies the display data to the decoder 1315. [0361] In the above operation, the camera signal processing unit 1313 uses a DRAM (Dynamic Random Access 20 Memory) 1318 connected thereto via a bus 1317, to store the image data and the encoded data or the like generated by encoding the image data into the DRAM 1318 where necessary. [0362] 25 The decoder 1315 decodes the encoded data supplied from the image signal processing unit 1314, and supplies the resultant image data (decoded image data) to the LCD 1316. The decoder 1315 also supplies the display data supplied from the image signal processing unit 1314 to 30 the LCD 1316. The LCD 1316 combines the image corresponding to the decoded image data supplied from the 99 SP310529WO00 decoder 1315 with the image corresponding to the display data, and displays the combined image. [0363] Under the control of the controller 1321, the on- 5 screen display 132 0 outputs the display data of a menu screen or icons formed with symbols, characters, or figures, to the image signal processing unit 1314 via the bus 1317. [0364] 10 Based on a signal indicating contents designated by a user using an operation unit 1322, the controller 1321 performs various kinds of operations, and controls, via the bus 1317, the image signal processing unit 1314, the DRAM 1318, an external interface 1319, the on-screen 15 display 1320, a media drive 1323, and the like. A flash ROM 1324 stores programs, data, and the like necessary for the controller 1321 to perform various kinds of operations. [0365] 20 For example, in place of the image signal processing unit 1314 and the decoder 1315, the controller 1321 can encode the image data stored in the DRAM 1318, and decode the encoded data stored in the DRAM 1318. In doing so, the controller 1321 may perform encoding and 25 decoding operations by using the same methods as the encoding and decoding methods used by the image signal processing unit 1314 and the decoder 1315, or may perform encoding and decoding operations by using methods that are not compatible with the image signal processing unit 30 1314 and the decoder 1315. [0366] 100 SP310529WO00 * When a start of image printing is requested through the operation unit 1322, for example, the controller 1321 reads image data from the DRAM 1318, and supplies the image data to a printer 1334 connected to the external 5 interface 1319 via the bus 1317, so that the printing is performed. [0367] Further, when image recording is requested through the operation unit 1322, for example, the controller 1321 10 reads encoded data from the DRAM 1318, and supplies and stores the encoded data into the recording medium 1333 mounted on the media drive 1323 via the bus 1317. [0368] The recording medium 1333 is a readable and 15 writable removable medium, such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. The recording medium 1333 may be any kind of removable medium, and may be a tape device, a disk, or a memory card. It is of course possible to use a non- 20 contact IC card or the like. [0369] Alternatively, the media drive 1323 and the recording medium 1333 may be integrated, and may be formed with an immobile storage medium such as an 25 internal hard disk drive or an SSD (Solid State Drive). [0370] The external interface 1319 is formed with a USB input/output terminal and the like, for example, and is connected to the printer 1334 when image printing is 30 performed. Also, a drive 1331 is connected to the external interface 1319 where necessary, and a removable I 101 I SP310529WO00 medium 1332 such as a magnetic disk, an optical disk, or a magneto-optical disk is mounted on the drive 1331 where appropriate. A computer program that is read from such a disk is installed in the flash ROM 1324 where necessary. 5 [0371] Further, the external interface 1319 includes a network interface connected to a predetermined network such as a LAN or the Internet. In accordance with an instruction from the operation unit 1322, for example, 10 the controller 1321 can read encoded data from the DRAM 1318, and supply the encoded data from the external interface 1319 to another device connected thereto via a network. Also, the controller 1321 can obtain encoded data and image data supplied from another device via a 15 network, and store the data into the DRAM 1318 or supply the data to the image signal processing unit 1314 via the external interface 1319. [0372] The above camera 1300 uses the image decoding 20 device 400 as the decoder 1315. That is, like the image decoding device 400, the decoder 1315 performs an adaptive loop filtering operation, where appropriate, on an image that has been subjected to a deblocking filtering operation by the deblocking filter 206, by 25 using information supplied from the encoding side (the image encoding device 500), such as an adaptive loop filter flag (adaptive_loop_filter_flag) and a filter coefficient. Accordingly, the decoder 1315 can perform an adaptive loop filtering operation more suited to 30 images, and restrain image quality deterioration in decoded images. 102 SP310529WO00 [0373] Accordingly, the camera 1300 can perform an adaptive loop filtering operation more suited to images on image data generated at the CCD/CMOS 1312, encoded 5 data of video data read from the DRAM 1318 or the recording medium 1333, or encoded data of video data obtained via a network, for example, and restrain deterioration of subjective image quality. [0374] 10 Also, the camera 1300 uses the image encoding device 500 as the encoder 1341. In the same manner as in the case of the image encoding device 500, the encoder 1341 has the filter control unit 501 to control the operation of the adaptive loop filter 502 in accordance 15 with types of images. By doing so, the encoder 1341 can perform an adaptive loop filtering operation more suited to images, and can reduce the encoding operation load while restraining image quality deterioration in decoded images. 20 [0375] Accordingly, when encoded data to be recorded on the DRAM 1318 or the recording medium 1333 is generated, or encoded data to be provided to another device is generated, the camera 1300 can perform an adaptive loop 25 filtering operation more suited to images, and reduce the encoding operation load while restraining deterioration of the subjective image quality of decoded images. [0376] The decoding method used by the image decoding 30 device 400 may be applied to decoding operations to be performed by the controller 1321. Likewise, the encoding 103 SP310529WO00 method used by the image encoding device 500 may be applied to encoding operations to be performed by the controller 1321. [0377] 5 Image data to be captured by the camera 1300 may be of a moving image, or may be of a still image. [0378] It is of course possible to apply the image encoding device 500 and the image decoding device 400 to 10 any devices and systems other than the above described devices. [0379] This technique can also be in the following forms. (1) An image processing device that includes: 15 a filter control unit that controls an adaptive filtering operation to be performed on image data, in accordance with whether the image data is to be referred to by other image data; and a filtering operation unit that performs the 20 adaptive filtering operation on the image data under the control of the filter control unit in a motion compensation loop. (2) The image processing device of (1), wherein, when the image data being subjected to the adaptive 25 filtering operation is to be referred to by the other image data in an operation to encode the image data, the filter control unit controls the adaptive filtering operation to be performed, and when the image data being subjected to the adaptive 30 filtering operation is not to be referred to by the other image data in the operation to encode the image data, the 104 SP310529WO00 filter control unit controls the adaptive filtering operation not to be performed. (3) The image processing device of (1) or (2), wherein 5 the image data is picture data, and the filter control unit controls the adaptive filtering operation for the image data in accordance with a type of the picture. (4) The image processing device of (3), wherein the 10 filter control unit controls the. adaptive filtering operation to be performed when the image data is an Ipicture, and controls the adaptive filtering operation not to be performed when the image data is a P-picture and a B-picture. 15 (5) The image processing device of (3), wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an Ipicture or a P-picture, and controls the adaptive filtering operation not to be performed when the image 20 data is a B-picture. (6) The image processing device of (3), wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an illpicture and a P-picture in image data containing 25 hierarchical B-pictures, or a B-picture to be referred to, and controls the adaptive filtering operation not to be performed when the image data is a B-picture not to be referred to in the image data containing hierarchical Bpictures. 30 (7) The image processing device of any of (1) through (6), wherein 105 ! i SP310529WO00 the image data is slice data, and the filter control unit controls the adaptive filtering operation for the image data in accordance with a type of the slice. 5 (8) The image processing device of (7), wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an Islice, and controls the adaptive filtering operation not to be performed when the image data is a P-slice and a B- 10 slice. (9) The image processing device of (7), wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an Islice or a P-slice, and controls the adaptive filtering 15 operation not to be performed when the image data is a Bpicture. (10) The image processing device of (7), wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an I- 20 slice and a P-slice in image data containing hierarchical B-slices, or a B-slice to be referred to, and controls the adaptive filtering operation not to be performed when the image data is a B-slice not to be referred to in the image data containing hierarchical B-pictures. 25 (11) The image processing device of any of (1) through (10), further including an encoding unit that encodes the image data subjected to the adaptive filtering operation, wherein the encoding unit encodes a filter 30 coefficient of the adaptive filtering operation and flag information indicating whether to perform the adaptive 106 SP310529WO00 filtering operation, and adds the resultant data to the encoded data of the image data. (12) The image processing device of any of (1) through (11), wherein 5 the filter control unit controls a tap length of a filter coefficient of the adaptive filtering operation, in accordance with whether the image data is to be referred to by other image data, and the filtering operation unit performs the adaptive 10 filtering operation on the image data, using the filter coefficient having the tap length controlled by the filter control unit. (13) The image processing device of (12), wherein, when the image data being subjected to the adaptive 15 filtering operation is to be referred to by the other image data in an operation to encode the image data, the filter control unit performs control to increase the tap length, and when the image data being subjected to the adaptive 20 filtering operation is not to be referred to by the other image data in the operation to encode the image data, the filter control unit performs control to shorten the tap length. (14) An image processing method that includes: 25 controlling an adaptive filtering operation to be performed on image data, in accordance with whether the image data is to be referred to by other image data, the control being performed by a filter control unit of an image processing device; and 30 performing the adaptive filtering operation on the image data in a motion compensation loop, the adaptive 107 SP310529WO00 filtering operation being performed by a filtering operation unit of the image processing device. REFERENCE SIGNS LIST 5 [0380] 500 Image encoding device, 501 Filter control unit, 502 Adaptive loop filter, 511 ON/OFF unit, 512 Filter coefficient calculation unit, 513 Filtering unit, 601 Filter control unit, 602 Adaptive loop filter, 611 Tap 10 length setting unit, 612 Filter coefficient calculation unit, 621 Zero coefficient setting unit SP310529WO00 CLAIMS 1. An image processing device comprising: a filter control unit configured to control an 5 adaptive filtering operation to be performed on image data, in accordance with whether the image data is to be referred to by other image data; and a filtering operation unit configured to perform the adaptive filtering operation on the image data under 10 the control of the filter control unit in a motion compensation loop, 2. The image processing device according to claim 1, wherein, 15 when the image data being subjected to the adaptive filtering operation is to be referred to by the other image data in an operation to encode the image data, the filter control unit controls the adaptive filtering operation to be performed, and 20 when the image data being subjected to the adaptive filtering operation is not to be referred to by the other image data in the operation to encode the image data, the filter control unit controls the adaptive filtering operation not to be performed. 25 3. The image processing device according to claim 1, wherein the image data is picture data, and the filter control unit controls the adaptive 30 filtering operation for the image data in accordance with a type of the picture. 109 SP310529WO00 4. The image processing device according to claim 3, wherein the filter control unit controls the adaptive filtering operation to be performed when the image data 5 is an I-picture, and controls the adaptive filtering operation not to be performed when the image data is a Ppicture and a B-picture. 5. The image processing device according to claim 3, 10 wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an I-picture or a P-picture, and controls the adaptive filtering operation not to be performed when the image data is a B-picture. 15 6. The image processing device according to claim 3, wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an I-picture and a P-picture in image data containing 20 hierarchical B-pictures, or a B-picture to be referred to, and controls the adaptive filtering operation not to be performed when the image data is a B-picture not to be referred to in the image data containing hierarchical Bpictures. 25 7. The image processing device according to claim 1, wherein the image data is slice data, and the filter control unit controls the adaptive 30 filtering operation for the image data in accordance with a type of the slice. 110 SP310529WO00 8. The image processing device according to claim 1, wherein the filter control unit controls the adaptive filtering operation to be performed when the image data 5 is an I-slice, and controls the adaptive filtering operation not to be performed when the image data is a Pslice and a B-slice. 9. The image processing device according to claim 7, 10 wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an I-slice or a P-slice, and controls the adaptive filtering operation not to be performed when the image data is a B-picture. 15 10. The image processing device according to claim 7, wherein the filter control unit controls the adaptive filtering operation to be performed when the image data is an I-slice and a P-slice in image data containing 20 hierarchical B-slices, or a B-slice to be referred to, and controls the adaptive filtering operation not to be performed when the image data is a B-slice not to be referred to in the image data containing hierarchical Bpictures. 25 11. The image processing device according to claim 1, further comprising an encoding unit configured to encode the image data subjected to the adaptive filtering operation, 30 wherein the encoding unit encodes a filter coefficient of the adaptive filtering operation and flag 111 SP310529WO00 information indicating whether to perform the adaptive filtering operation, and adds the resultant data to the encoded data of the image data. 5 12. The image processing device according to claim 1, wherein the filter control unit controls a tap length of a filter coefficient of the adaptive filtering operation, in accordance with whether the image data is to be 10 referred to by other image data, and the filtering operation unit performs the adaptive filtering operation on the image data, using the filter coefficient having the tap length controlled by the filter control unit. 15 13. The image processing device according to claim 12, wherein, when the image data being subjected to the adaptive filtering operation is to be referred to by the other 20 image data in an operation to encode the image data, the filter control unit performs control to increase the tap length, and when the image data being subjected to the adaptive filtering operation is not to be referred to by the other 25 image data in the operation to encode the image data, the filter control unit performs control to shorten the tap length. 14. An image processing method comprising: 30 controlling an adaptive filtering operation to be performed on image data, in accordance with whether the 112 SP310529WO00 # . image data is to be referred to by other image data, the control being performed by a filter control unit of an image processing device; and '•» ^esforming the adaptive filtering operation on the 5 image data in a motion compensation loop, the adaptive filtering operation being performed by a filtering . operation unit of the image processing device.