Technical field
[0001]
　The present disclosure relates to an image processing apparatus and an image processing method.
BACKGROUND
[0002]
　It is one H. standard specifications of the image coding method In 264 / AVC, in order to suppress deterioration of image quality due to block distortion occurring at the time of encoding the image, the deblocking filter to the block boundary is applied to, for example, each block of 4 × 4 pixels. Amount of processing required for this deblocking filter is great, for example, are also said to occupy 50% of the total throughput of the decoding of the image.
[0003]
　The standardization of the next generation of the image coding method HEVC (High Efficiency Video Coding), in JCTVC-A119 (see Non-Patent Document 1), applying a deblocking filter for each block of 8 × 8 pixels or more Size it has been proposed to. In the proposed approach in JCTVC-A119, that the block size of the minimum unit of the deblocking filter is expanded, parallel execute the filtering process for a plurality of block boundaries in the same direction within one macroblock it is possible to become.
CITATION
Non-patent literature
[0004]
非特許文献1 : K.Ugur　(Nokia),　K.R.Andersson　(LM　Ericsson),　A.Fuldseth　(Tandberg　Telecom),　"JCTVC-A119:Video　coding　technology　proposal　by　Tandberg,　Nokia,　and　Ericsson",　Documents　of　the　first　meeting　of　the　Joint　Collaborative　Team　on　Video　Coding　(JCT-VC),　Dresden,　Germany,　15-23　April,　2010.
Summary of the Invention
Problems that the Invention is to Solve
[0005]
　However, even adopting the method proposed in JCTVC-A119, dependency between the processing of the processing and horizontal block boundary in the vertical direction of the block boundary is left. Specifically, for example, the processing of the vertical border of one macroblock is done waiting for processing in the horizontal boundary of the neighboring macroblocks. The processing of the horizontal boundary of one macroblock is done waiting for processing in the vertical boundaries of the same macro block. Accordingly, even in the above method, it is impossible to parallelize the process of the deblocking filter only a very limited range. Therefore, it is hard to say that issues such as reduction in delay or data rate caused significant amount of processing time of the application of the deblocking filter is satisfactorily resolved.
[0006]
　Therefore, the technology according to the present disclosure allows for a further parallelization of processing in the application of the deblocking filter, is intended to provide an image processing apparatus and an image processing method.
Means for Solving the Problems
[0007]
　According to the present disclosure, it is decoded and the decoding unit for decoding an image from encoded stream, and the horizontal filtering unit applying a deblocking filter to the vertical block boundaries in an image to be decoded by the decoder, by the decoding unit a vertical filtering unit applying a deblocking filter to the horizontal block boundaries in the image, parallel to filter the plurality of vertical block boundaries included in the processing unit includes a plurality of coding units in the horizontal filtering unit, the processing a control unit for parallel filtering a plurality of the horizontal block boundary to the vertical filtering unit included in the unit, the image processing apparatus comprising a are provided.
[0008]
　The image processing device mentioned above may be typically realized as an image decoding apparatus for decoding an image.
[0009]
　Further, according to the present disclosure, and decoding the picture from the encoded stream, and performing horizontal filtering applying the deblocking filter to the vertical block boundaries in an image to be decoded, horizontal in the image to be decoded and performing a vertical filtering of the deblocking filter to the block boundary, a plurality of vertical block boundaries included in the processing unit includes a plurality of coding units are filtered in parallel, a plurality of which are included in the processing unit as the horizontal block boundary are filtered in parallel, image processing method comprising, and controlling said horizontal filtering and the vertical filtering is provided.
[0010]
　Further, according to the present disclosure, a horizontal filtering unit applying a deblocking filter to the vertical block boundaries in an image to be locally decoded when encoding the encoding target image, the deblocking horizontal block boundary in the image a vertical filtering unit applying a filter, parallel to filter the plurality of vertical block boundaries included in the processing unit includes a plurality of coding units in the horizontal filtering unit, a plurality of horizontal block border included in the processing unit the use and the control unit in parallel to the filtering in the vertical filtering unit, the filtered image by the horizontal filtering unit and the vertical filtering unit, an image and an encoding unit for encoding the encoding target image processing apparatus is provided.
[0011]
　The image processing device mentioned above may be typically realized as an image encoding device for encoding an image.
[0012]
　Further, according to the present disclosure, and performing the horizontal filtering applying the deblocking filter to the vertical block boundaries in an image to be locally decoded when encoding the encoding target image, horizontal block border in the image a plurality of horizontal blocks included in and performing vertical filtering of the deblocking filter, a plurality of vertical block boundaries included in the processing unit includes a plurality of coding units are filtered in parallel, to the processing unit as the boundary is filtered in parallel, said and controlling horizontal filtering and the vertical filtering, by using the filtered image by the horizontal filtering and the vertical filtering, encoding the encoding target image When the image processing method comprising is provided.
The invention's effect
[0013]
　As described above, according to the image processing apparatus and image processing method according to the present disclosure, it is possible to further parallelization of processing in the application of the deblocking filter.
BRIEF DESCRIPTION OF THE DRAWINGS
[0014]
Is a block diagram showing an example of the configuration of FIG. 1 the image coding apparatus according to an embodiment.
Is a block diagram showing an example of the configuration of FIG. 2 the image decoding apparatus according to an embodiment.
3 is an explanatory diagram showing an example of pixels adjacent to each other across the border.
FIG. 4 is an explanatory view for explaining reference pixels of the filtering necessity determination process in the existing technique.
FIG. 5 is an explanatory diagram for explaining pixels to be updated by the filtering process.
6 is an explanatory diagram for describing Designation edge for description of the embodiments.
7 is an explanatory view for explaining parallel processing in the existing technique.
8 is a first explanatory diagram for describing dependencies between processes in existing techniques.
9 is a second explanatory diagram for describing dependencies between processes in existing techniques.
FIG. 10 is an explanatory diagram for explaining an example of a sequence of processing in the existing method.
11 is an explanatory diagram for describing an example of a sequence of processing in the first embodiment.
Is a block diagram illustrating an example of FIG. 12 detailed structure of the de-blocking filter according to the first embodiment.
Is a block diagram showing an example of a more detailed configuration of FIG. 13 judging unit.
FIG. 14 is an explanatory diagram for describing an example of the neighboring blocks across the slice boundary.
It is an explanatory diagram for FIG. 15 for the first example of the order of the processing of each slice will be described.
FIG. 16 is an explanatory diagram for explaining a second example of the order of processing of each slice.
FIG. 17 is an explanatory diagram for explaining a first and second example of the determination method that may be implemented in one variation.
FIG. 18 is an explanatory diagram for explaining a third and fourth example of the determination method that may be implemented in one variation.
19 is an explanatory diagram for describing a fifth and sixth example of determination technique may be implemented in one variation.
FIG. 20 is a flowchart illustrating an example of a processing flow by the deblocking filter according to the first embodiment.
21 is a flowchart showing an example of a flow of filtering necessity determining process.
FIG. 22 is an explanatory diagram for explaining an example of a sequence of processing in the second embodiment.
FIG. 23 is a block diagram showing an example of a detailed configuration of the deblocking filter according to a second embodiment.
FIG. 24 is a flowchart showing an example of a flow of a process by the deblocking filter according to the second embodiment.
[25] is an explanatory diagram for explaining a sequence of processing for each LCU.
Is a flowchart illustrating an example of a processing flow for each [FIG 26] LCU.
FIG. 27 is an explanatory diagram for describing a schematic of a third embodiment.
[FIG. 28] is a block diagram showing an example of a detailed configuration of the deblocking filter according to a third embodiment.
It is an explanatory diagram for explaining the determination of the weights for [29] the weighted average.
It is an explanatory diagram for explaining an example of the weight for the [30] the weighted average.
[FIG. 31] is an explanatory diagram for describing the output pixel values from the calculation unit according to the third embodiment.
[FIG. 32] is an explanatory diagram for explaining a first example of order of processing for comparison.
[33] is an explanatory diagram for describing a first example of a sequence of processing realized in the third embodiment.
FIG. 34 is an explanatory diagram for explaining a second example of the order of processing for comparison.
[FIG. 35] is an explanatory diagram for describing a second example of the order of processing realized in the third embodiment.
[FIG. 36] is a flowchart showing an example of a flow of a process by the deblocking filter according to a third embodiment.
[FIG. 37] is a flowchart showing an exemplary flow of a pixel value calculation process shown in FIG. 36.
[FIG. 38] is an explanatory diagram for describing a multi-view codec.
[39] is an explanatory diagram for describing application to multi-view codec of the image encoding process according to an embodiment.
Is an explanatory diagram for describing application to multi-view codecs [40] The image decoding process according to an embodiment.
[FIG. 41] is an explanatory diagram for describing a scalable codec.
[FIG. 42] is an explanatory diagram for describing application to scalable codec of the image encoding process according to an embodiment.
[43] is an explanatory diagram for describing application to scalable codec of an image decoding process according to an embodiment.
[FIG. 44] is a block diagram showing an example of a schematic configuration of a television device.
[FIG. 45] is a block diagram showing an example of a schematic configuration of a mobile phone.
[FIG. 46] is a block diagram showing an example of a schematic configuration of a recording and reproducing apparatus.
[FIG. 47] is a block diagram showing an example of a schematic configuration of an imaging apparatus.
DESCRIPTION OF THE INVENTION
[0015]
　Reference will now be described in detail preferred embodiments of the present disclosure. In the specification and the drawings, components having substantially the same functional configuration are repeated explanation is omitted by putting the same symbols.
[0016]
　Further, explaining the "Description of the Invention" following order.
　　1. Overview of the device
　　　1-1. The image coding apparatus
　　　1-2. The image decoding device
　　2. Description of existing approaches
　　　2-1. The basic configuration of the de-block filter
　　　2-2. It dependencies between processes in existing method
　　3. The first embodiment
　　　3-1. Configuration example of the de-block filter
　　　3-2. Modification of the determination condition
　　　3-3. Processing of flow
　　4. Second embodiment
　　　4-1. Configuration example of the de-block filter
　　　4-2. Processing of flow
　　　4-3. Examples of processing for each LCU
　　5. Third Embodiment
　　　5-1. Outline
　　　5-2. Configuration example of the de-block filter
　　　5-3. Examples of order of processing
　　　5-4. Processing of flow
　　6. It applied to a variety of codecs
　　　6-1. Multi-view codec
　　　6-2. Scalable codec
　　7. Application Example
　　8. Summary
[0017]
　<1. Device Overview>
　First, with reference to FIGS. 1 and 2, an overview of the apparatus as an example of applicability of the technology disclosed herein. The technology disclosed herein, for example, is applicable to the image coding apparatus and image decoding apparatus.
[0018]
　　[1-1. Image encoding apparatus]
　FIG. 1 is a block diagram showing an example of a configuration of an image encoding apparatus 10 according to an embodiment. Referring to FIG. 1, the image encoding apparatus 10, A / D (Analogue to Digital ) conversion unit 11, the reordering buffer 12, a subtraction unit 13, an orthogonal transform unit 14, quantization unit 15, a lossless coding unit 16, storage buffer 17, the rate control unit 18, inverse quantization unit 21, inverse orthogonal transform unit 22, adding unit 23, a deblocking filter 24a, a frame memory 25, a selector 26, an intra prediction unit 30, motion estimation unit 40, and a mode selection It comprises a section 50.
[0019]
　A / D converter 11 converts the image signal input in analog form to the digital image data, and outputs a series of digital image data to the reordering buffer 12.
[0020]
　Sorting buffer 12 rearranges the images included in the series of image data input from the A / D converter 11. Reordering buffer 12, it sorts the images in accordance with the GOP (Group of Pictures) structure according to the coding process, and outputs the image data after rearrangement to subtraction unit 13, the intra prediction unit 30 and the motion estimation section 40 to.
[0021]
　The subtraction unit 13, the predicted image data selected by the mode selection unit 50 to be described the image data, and after the input from the reordering buffer 12 is supplied. Subtraction unit 13 calculates the prediction error data which is a difference between the predicted image data input from the image data and a mode selection unit 50 which is input from the rearrangement buffer 12, the calculated prediction error data to the orthogonal transform section 14 Output.
[0022]
　Orthogonal transform unit 14 performs orthogonal transform on the prediction error data inputted from the subtraction unit 13. Orthogonal transform performed by the orthogonal transform unit 14, for example, discrete cosine transform (Discrete Cosine Transform: DCT) or Karhunen-Loeve transform may the like. Orthogonal transform unit 14, transform coefficient data acquired by the orthogonal transform processing and outputs it to the quantization unit 15.
[0023]
　The quantization unit 15, a rate control signal from the rate control unit 18 to be described transform coefficient data input from the orthogonal transformation unit 14, and after it is supplied. Quantization unit 15, transform coefficient data quantized transform coefficient data after quantization (hereinafter referred to as quantized data) to the to the lossless encoding unit 16 and the inverse quantization unit 21. The quantization unit 15, by switching the quantization parameter (quantization scale) on the basis of the rate control signal from the rate control unit 18, change the bit rate of the quantized data input to the lossless encoding unit 16 make.
[0024]
　The lossless encoding unit 16, the quantized data input from the quantization unit 15, and intra prediction or inter prediction is selected by the generation mode selecting unit 50 by the intra prediction unit 30 or the motion estimation section 40 will be described later information about is supplied. Information about the intra prediction may include, for example, prediction mode information indicating the optimal intra prediction mode for each block. Also, information on inter prediction may include, for example, prediction mode information for the prediction of motion vectors of each block, the difference motion vector information, and the like reference picture information.
[0025]
　Lossless encoding section 16, by performing a lossless encoding process on the quantized data, to generate a coded stream. Lossless encoding by the lossless coding unit 16, for example, variable length coding, or the like arithmetic coding. Also, the lossless encoding unit 16, the information about the information or inter prediction regarding intra-prediction as described above, multiplexed into the header of the encoded stream (e.g., a block header or slice header). The lossless encoding unit 16 outputs the generated encoded stream to the storage buffer 17.
[0026]
　The accumulation buffer 17 temporarily accumulates using a storage medium an encoded stream input from the lossless encoding section 16 such as a semiconductor memory. Then, the accumulation buffer 17, the stored encoded stream, and outputs at a rate corresponding to the bandwidth of the (output lines from or image coding apparatus 10) transmission path.
[0027]
　The rate control section 18 monitors the free space of the storage buffer 17. Then, the rate control unit 18 generates a rate control signal in accordance with the free space in the accumulation buffer 17, and outputs the generated rate control signal to the quantization unit 15. For example, the rate control unit 18, when the free space of the storage buffer 17 is small, and generates a rate control signal for lowering the bit rate of the quantized data. Further, for example, the rate control unit 18, when the free space of the storage buffer 17 is sufficiently large to generate a rate control signal for increasing the bit rate of the quantized data.
[0028]
　Inverse quantization unit 21 performs an inverse quantization process on the quantized data input from the quantization unit 15. Then, the inverse quantization unit 21, transform coefficient data acquired by the inverse quantization processing, and outputs it to the inverse orthogonal transform unit 22.
[0029]
　Inverse orthogonal transform unit 22 performs inverse orthogonal transform processing for transform coefficient data input from the inverse quantization unit 21, restores the prediction error data. Then, the inverse orthogonal transform unit 22 outputs the prediction error data restored to the adder 23.
[0030]
　Addition unit 23, by adding the predicted image data input from the prediction error data and mode selection unit 50 that is restored is inputted from the inverse orthogonal transform unit 22 to generate a decoded image data. The adding unit 23 outputs the generated decoded image data to the deblocking filter 24a and the frame memory 25.
[0031]
　Deblocking filter 24a performs a filtering process for reducing block distortion occurring at the time of encoding the image. For example, the deblocking filter 24a determines necessity of filtering for each block boundaries for the decoded image data input from the addition unit 23 applies the deblocking filter to the boundary is determined that it should apply the filter . The deblocking filter 24a, in addition to the decoded image data from the adder 23, the information used for determining the necessity of filtering (e.g., mode information, the transform coefficient information and motion vector information) is also input . Then, the deblocking filter 24a outputs the decoded image data after filtering that is removal of block distortion to the frame memory 25. The processing by the deblocking filter 24a, will be described in detail later.
[0032]
　The frame memory 25 stores, using a storage medium decoded image data after filtering input decoded image data input from the addition unit 23, and the deblocking filter 24a.
[0033]
　The selector 26 reads the decoded image data before filtering that is used for intra prediction from the frame memory 25, and supplies the intra prediction unit 30 reads decoded image data as reference image data. The selector 26 reads the decoded image data after filtering to be used for inter prediction from the frame memory 25, and supplies the motion search unit 40 reads decoded image data as reference image data.
[0034]
　The intra prediction unit 30, the image data of the input from the rearrangement buffer 12 coded, and based on the decoded image data supplied via the selector 26 performs intra prediction processing of each intra prediction mode. For example, the intra prediction unit 30 is evaluated using the prediction result given cost function by each intra prediction mode. Then, the intra prediction unit 30, an intra prediction mode cost function value is minimized, i.e. the intra prediction mode compression ratio is the highest is selected as the optimal intra prediction mode. Furthermore, the intra prediction unit 30, prediction mode information indicating the optimal intra prediction mode, the prediction image data, and information on intra-prediction, such as the cost function value, and outputs to the mode selection unit 50.
[0035]
　Motion estimation unit 40, image data of the input from the rearrangement buffer 12 coded, and based on the decoded image data supplied via the selector 26, performs inter prediction process (inter-frame prediction process). For example, the motion search unit 40 is evaluated using the prediction result given cost function by each prediction mode. Next, the motion estimation section 40, the prediction mode cost function value is minimized, i.e. the prediction mode compression ratio is the highest is selected as the optimal prediction mode. Also, the motion estimation section 40 generates predicted image data in accordance with the optimum prediction mode. Then, the motion estimation unit 40, prediction mode information indicating the optimum prediction mode selected, the prediction image data, and the information about the inter prediction of cost function values, and outputs to the mode selection unit 50.
[0036]
　Mode selecting unit 50 compares the cost function values ​​for the inter prediction input from the cost function value and the motion estimation section 40 about the intra prediction input from the intra prediction unit 30. The mode selection unit 50, the cost function value of the intra prediction and inter prediction to select fewer prediction method. Mode selector 50, when selecting the intra prediction outputs the information on the intra prediction to the lossless encoding unit 16, and outputs the predicted image data to the subtraction section 13 and addition section 23. The mode selection unit 50, if you select the inter prediction outputs the above-mentioned information about the inter prediction to the lossless encoding unit 16, and outputs the predicted image data to the subtraction section 13 and addition section 23.
[0037]
　　[1-2. The image decoding apparatus]
　FIG. 2 is a block diagram showing an example of a configuration of an image decoding apparatus 60 according to an embodiment. Referring to FIG. 2, the image decoding apparatus 60 includes a storage buffer 61, reversible decoding unit 62, inverse quantization unit 63, inverse orthogonal transform unit 64, an adder 65, a deblocking filter 24b, the reordering buffer 67, D / A comprises (Digital to Analogue) conversion unit 68, frame memory 69, a selector 70 and 71, the intra prediction unit 80 and motion compensation unit 90,.
[0038]
　The accumulation buffer 61 temporarily accumulates using a storage medium an encoded stream input via a transmission path.
[0039]
　Lossless decoding unit 62, an encoded stream input from the accumulation buffer 61, decodes according to the encoding scheme used at the time of encoding. Also, the lossless decoding unit 62 decodes information multiplexed in the header area of ​​the encoded stream. The information multiplexed in the header area of ​​the encoded stream, for example, may include information about the information and inter prediction concerning intra prediction in the block header. Lossless decoding section 62 outputs the information about intra prediction to the intra prediction unit 80. Also, the lossless decoding unit 62 outputs the information about inter prediction to the motion compensation unit 90.
[0040]
　Inverse quantization unit 63 inversely quantizes the quantized data decoded by the lossless decoding section 62. Inverse orthogonal transform unit 64 in accordance with the orthogonal transform method used in encoding, by performing inverse orthogonal transform for the transform coefficient data input from the inverse quantization unit 63 generates a prediction error data. Then, the inverse orthogonal transform unit 64 outputs the generated prediction error data to the addition section 65.
[0041]
　Adding section 65, the prediction error data inputted from the inverse orthogonal transform unit 64, by adding the predicted image data input from the selector 71, and generates the decoded image data. The adding unit 65 outputs the generated decoded image data to the deblocking filter 24b and the frame memory 69.
[0042]
　Deblocking filter 24b performs filtering processing for reducing the block distortion appearing on the decoded image. Deblocking filter 24b, for example, to determine the necessity of filtering for each block boundaries for the decoded image data input from the addition unit 65 applies the deblocking filter to the boundary is determined that it should apply the filter . The deblocking filter 24b, in addition to the decoded image data from the adder unit 65, information used for judging the necessity of filtering is also input. Then, the deblocking filter 24b outputs the sorting buffer 67 and the frame memory 69 the decoded image data after removed filtering of blockiness. The processing by the deblocking filter 24b, will be described in detail later.
[0043]
　Sorting buffer 67, by rearranging the image input from the deblocking filter 24b, when generating a sequence of image data of the series. Then, the reordering buffer 67 outputs the generated image data to the D / A converter 68.
[0044]
　The D / A converter 68 converts the digital image data inputted from the rearrangement buffer 67 into an image signal of analog form. Then, D / A conversion unit 68, for example, by outputting an analog image signal to a display (not shown) connected to an image decoding apparatus 60 to display the image.
[0045]
　The frame memory 69 stores, using a storage medium decoded image data after filtering input before filtering the decoded image data input from the addition unit 65, and the deblocking filter 24b.
[0046]
　The selector 70, in accordance with the mode information acquired by the lossless decoding unit 62, for each block in the image, switch the output destination of image data from the frame memory 69 with the intra-prediction unit 80 and motion compensation unit 90 . For example, the selector 70, when the intra prediction mode is specified, and outputs it to the intra prediction unit 80 the decoded image data before filtering that is supplied from the frame memory 69 as reference image data. The selector 70, when the inter prediction mode is specified, and outputs it to the motion compensation unit 90 the decoded image data after filtering that is supplied from the frame memory 69 as reference image data.
[0047]
　The selector 71, in accordance with the mode information acquired by the lossless decoding unit 62, for each block in the image, the output source of the predicted image data to be supplied to the addition section 65 and the intra prediction unit 80 and motion compensation unit 90 switch between. For example, the selector 71, when the intra prediction mode is specified, and supplies the predicted image data output from the intra prediction unit 80 to the addition unit 65. The selector 71, when the inter prediction mode is specified, and supplies the predicted image data output from the motion compensation unit 90 to the addition unit 65.
[0048]
　Intra prediction unit 80 performs intra prediction of pixel values ​​based on the reference image data from the information and the frame memory 69 relating to the intra prediction input from the lossless decoding section 62 generates a predicted image data. Then, the intra prediction unit 80 outputs the generated predicted image data to the selector 71.
[0049]
　Motion compensation unit 90 performs motion compensation processing on the basis of the reference image data from the information and the frame memory 69 about an inter prediction input from the lossless decoding section 62 generates a predicted image data. Then, the motion compensation unit 90 outputs the generated predicted image data to the selector 71.
[0050]
　<2. Description of existing
　　methods> [2-1. The basic configuration of the deblocking filter]
　Generally, H. Processing by the deblocking filter in existing image encoding method such as 264 / AVC or HEVC includes two types of processing of the filtering necessity determining process and filtering process. Hereinafter, an example of HEVC, describe these two processes.
[0051]
　　　(1) filtering necessity determining process
　filtering necessity determining process is a process of determining whether to apply the deblocking filter per block boundaries in the input image. Boundaries of blocks includes a vertical boundary between blocks adjacent to the left and right, and a horizontal boundary between blocks adjacent in the vertical direction. In JCTVC-A119, a block size of 8 × 8 pixels is the minimum unit of processing. For example, 16 × 16 in the pixel in the macro block is present block of the four 8 × 8 pixels, one for each block (the left) vertical boundary and one (upper) horizontal boundary, i.e. a total of 4 + 4 = eight of the boundary is subject to judgment. In this specification, the term macroblock coding unit in the context of HEVC (CU: Coding Unit) shall also include.
[0052]
　Figure 3 is an explanatory diagram showing an example of a pixel of two adjacent block (neighboring block) Ba and Bb across the boundary. Here it will be explained as an example vertical boundary, of course, where the matters described is applicable equally to the horizontal boundary. In the example of FIG. 3, the pixels in the block Ba, p ij is indicated by the symbol as. i is an index of the index, j row of columns. The index i of the column, from the column close to the vertical boundary in the order (from right to left) 0, 1, 2, 3 are numbered with. The index j of the line, 0, 1, 2, from top to bottom, ..., are numbered and 7. Incidentally, the left half of the block Ba is omitted in FIG. Meanwhile, the pixels in the block Bb, q kj are indicated by the symbol as. k is the index of the index, j row of columns. The index k of the column, from the column close to the vertical boundary in the order (from left to right) 0, 1, 2, 3 are numbered with. Incidentally, it is omitted also in FIG right half of the block Bb.
[0053]
　The vertical boundary between blocks Ba and Bb shown in Fig. 3, is whether or not to apply the deblocking filter may be determined according to the conditions as follows:
[0054]
　Luminance components (Luma) determination condition ... conditions A and B are both applied if true
　　- Condition
　　　A: (A1) blocks Ba or Bb is an intra prediction
　　　mode; (A2) blocks Ba or Bb non with orthogonal transform coefficients of zero; or
　　　(A3) | MVAx-MVBx | ≧ 4 or | MVAy-MVBy | ≧ 4
　　- condition B:
| p 22 -2p 12 + p 02 | + | q 22 -2Q 12 + q 02 | Tasu | the p- 25 -2P 15 Tasu the p- 05 | Tasu | Q 25 -2Q 15 Tasu Q 05 | >
　　　2) (C2) (| the p- 3j -p 0j | + | q 0j -q 3j |) <(beta >> 3)
(C3) | p 0j -q 0j | <((5t C +1) >> 1) here, j is the vertical boundary the line for the horizontal boundary is the index of columns. Further, d = | p 22 -2p 12 + p 02 | + | q 22 -2Q 12 + q 02 | + | p 25 -2p 15
The p-Tasu 05 | Tasu | Q 25 -2Q 15 Tasu Q 05 | it is.
[0060]
　　- Weak fuィsuitable for center Surlyn Corning:
　　　[Delta] = the Clip (-t C , T C , (13 is (Q 0j -p 0j ) +4 (Q 1J -p 1J ) -5 (Q 2J -p 2J ) +16)
　　　>>. 5)) P 0j the Clip = 0-255 (P 0j +
　　　[Delta]) Q 0j = the Clip 0-255 (Q 0j -
　　　[Delta) P 1J = the Clip 0-255 (P 1J + [Delta]
　　　/ 2) Q 1J = the Clip ０－２５５（ｑ １ｊ－Δ／２）
[0061]
　　- strength
　　　filtering: the p- 0J = Clip 0-255 ((the p- 2J Tasu 2P 1J Tasu 2P 0J Tasu 2Q 0J Tasu Q 1J Tasu4) >> 3) Q 0J = Clip 0-255 ((the p- 1J Tasu 2P 0J Tasu 2Q 0J Tasu 2Q 1J Tasu Q 2J Tasu4) 　　　3 >>) the p- 1J = Clip 0-255 ((the p- 2J Tasu the p- 1J Tasu the p- 0J Tasu Q 0J

＋２）>>２）
　　　ｑ １ｊ＝Ｃｌｉｐ ０－２５５（（ｐ ０ｊ＋ｑ ０ｊ＋ｑ １ｊ＋ｑ ２ｊ＋２）>>２）
　　　ｐ ２ｊ＝Ｃｌｉｐ ０－２５５（（２ｐ ３ｊ＋３ｐ ２ｊ＋ｐ １ｊ＋ｐ ０ｊ＋ｑ ０ｊ＋４）>>３）
　　　ｑ ２ｊ＝Ｃｌｉｐ ０－２５５（（ｐ ０ｊ＋ｑ ０ｊ＋ｑ １ｊ＋３ｑ ２ｊ＋２ｑ ３ｊ＋４）>>３）
[0062]
　Incidentally, Clip (a, b, c ) is the process of clipping the value c in the range of a ≦ c ≦ b, Clip 0-255 (c) is a process of clipping the value c in the range of 0 ≦ c ≦ 255, respectively represent.
[0063]
　· Chrominance componentのfuィsuitable for center Surlyn Corning:
　　　[Delta] = the Clip (-t C , T C , ((((Q 0j -p 0j ) << 2) + P 1J -q 1J +4)
　　　>>. 3)) P 0j = the Clip 0-255 (P 0j +
　　　[Delta]) Q 0j = the Clip 0-255 (Q 0j - [Delta)
[0064]
　That is, as indicated by the dotted frame C6 ~ C8 and C1 ~ C3 in FIG. 5, the filtering process for the general vertical boundaries (in particular, strong filtering of the luminance component), 1-3 th blocks and 6 pixel values ​​of 1-8-th row is updated. Similarly, in the filtering process in the horizontal boundary, pixel values ​​of first to third and 6-8-th row of each block is updated.
[0065]
　　[2-2. Dependencies between processing in the existing technique
　, where, for purposes of explanation, as shown in FIG. 6, the upper left vertical boundary of the macroblock MBx (MB0, MB1 ...) having a size of 16 × 16 pixels Vx , 0, the vertical boundary of the central Vx, 1, the lower left vertical boundary Vx, 2, the vertical boundary of the central lower Vx, 3, the horizontal boundary of the upper left Hx, 0, Hx, 1 horizontal boundary in the upper right, the horizontal boundaries of the left middle Hx, 2, the horizontal boundary of the right middle Hx, 3, shall be expressed as. Further, for example, filtering necessity determination process of the boundary Z J Z , the filtering process F Z is intended to refer to as.
[0066]
　In existing method described above, there is no dependency between the processing for a plurality of boundary in the same direction within one macroblock. Therefore, it is possible to perform filtering for a plurality of vertical boundary, for example, within one macroblock parallel run, and a plurality of filtering in the horizontal boundary parallel. As an example, referring to FIG. 7, in the macroblock MB0, 4 single filtering F V0,0 , F V0,1 , F V0,2 and F V0,3 no dependencies between (i.e., duplicate no pixels to be updated Te), it can be seen that perform these parallel.
[0067]
　However, existing method described above, the dependency between the filtering necessity determining process for filtering and horizontal boundaries of the vertical boundary is left. Also, dependencies between the filtering necessity determining process for filtering and vertical boundary of the horizontal boundary is also retained. Thus, for example, filtering necessity determining process when, for the horizontal boundary in a certain macro block for processing the vertical boundary before the horizontal boundary, will be performed after completion of the filtering process in the vertical boundaries . As an example, referring to FIG. 8, in the macroblock MB0, filtering F V0,0 and F V0,1 filtering necessity determining process J to results H0,0 was dependent, filtering F V0,1 result of filtering necessity determining process J in H0,1 is shown to be dependent. Similarly, the filtering necessity determination process in the vertical boundaries of a certain macro block is to be executed after completion of the filtering process in the horizontal boundaries of macro blocks of the next. As an example, referring to FIG. 9, the macro filtering F blocks MB0 H0,1 and F H0,3 the result of the filtering necessity determining process J macroblock MB1 V1,0 and dependent, filtering of macroblocks MB0 F H0,3Filtering necessity determining process J in the results of macro blocks MB1 V1,2 is shown to be dependent.
[0068]
　Existing approaches, because it has a dependency between such processing, for example, even adopting the method proposed in JCTVC-A119, parallel processing of the deblocking filter only a very limited range it is not possible to achieve the reduction.
[0069]
　Figure 10 is an explanatory diagram for explaining an example of order of processing of the deblocking filter in the existing method. Here, as an example, it is assumed that an image having a size of 32 × 32 pixels is inputted to the de-block filter. The input image includes four macroblocks MB0 ~ MB3 each having a size of 16 × 16 pixels.
[0070]
　Referring to FIG. 10, the processing may be performed in parallel are shown respectively in each dotted line frame. For example, in a first step, filtering necessity determining process J for the four vertical boundary macroblocks MB0 V0,0 , J V0,1 , J V0,2 and J V0,3 are executed in parallel. Then, in a second step, the filtering process F for four vertical boundary macroblocks MB0 V0,0 , F V0,1 , F V0,2 and F V0,3 are executed in parallel. Then, after the second step is completed, in a third step, the filtering necessity determining process J for the four horizontal boundaries of macro blocks MB0 H0,0 , J H0,1 , J H0,2 and J H0,3 There are executed in parallel. In the third step, the pixel values after the filtering processing for the vertical boundary in the second step is used for filtering necessity determination process in the horizontal boundary. Then, in the fourth step, the filtering process for the four horizontal boundaries of macro blocks MB0 F H0,0, F H0,1 , F H0,2 and F H0,3 are executed in parallel. Then, after the fourth step has ended, the processing of the macroblock MB1 (Fifth step to eighth step) are sequentially performed. In these fifth step, pixel values after the filtering process in the horizontal boundaries of macro blocks MB0 in the fourth step is used for filtering necessity determination process in the vertical boundaries of the macro block MB1. Then, after the processing for the macroblock MB1 is finished, the processing of macroblock MB2 (9th step to 12th step) are sequentially performed. Then, after the processing for the macroblock MB2 is completed, processing for macroblocks MB3 (13th step to 16th step) are sequentially performed.
[0071]
　In parallel processing in such a limited range, it is difficult to say that issues such as reduction in delay or data rate caused significant amount of processing time of the application of the deblocking filter is satisfactorily resolved. Therefore, the three examples of the deblocking filtering to be described below, to achieve a further parallelization of processing in the application of the deblocking filter.
[0072]
　<3. The first
　　embodiment> [3-1. Configuration Example of a deblocking filter
　in this section, the configuration according to the first embodiment of the deblocking filter 24b of the image decoding apparatus 60 shown in the deblocking filter 24a and FIG. 2 of the image encoding apparatus 10 shown in FIG. 1 illustrating an example. The configuration of the deblocking filter 24a and the deblocking filter 24b may be common. Accordingly, in the subsequent description, particularly when there is no need to distinguish between the two is a generic term for deblocking filter 24a and deblocking filter 24b and the deblocking filter 24.
[0073]
　　(1) dependencies between new processing
　also in the present embodiment, processing by the deblocking filter 24 comprises two types of treatment that the above-described filtering necessity determining process and filtering process. However, deblocking filter 24 in the filtering necessity determining process on one of the vertical boundary and the horizontal boundary, over a plurality of macro-blocks, using the pixel values of the input image to the deblocking filter for the determination. Thereby, for example, when the vertical boundary monkey processed before the horizontal boundaries deblocking filter 24, filtering necessity determination process in the vertical border of a block, a filtering process in the horizontal block boundary of adjacent it can be executed without waiting. Also, when the horizontal boundary monkey processed before the vertical boundaries, the deblocking filter 24, filtering necessity determination process in the horizontal boundary of a block, waiting for the filtering of the vertical boundary of the block next it can be executed without. Therefore, dependency processing between macro blocks is reduced.
[0074]
　By processing the dependencies between macroblocks is reduced, it is possible to parallelize the processing between a plurality of macroblocks in the image. For example, it is possible to perform filtering necessity determination process in the vertical boundaries of all blocks in the input image in parallel. It is also possible to carry out the filtering necessity determination process in the horizontal boundaries of all blocks in the input image in parallel.
[0075]
　Figure 11 is an explanatory diagram for explaining an example of a sequence of feasible process in the present embodiment. Again, as an example, it is assumed that an image having a size of 32 × 32 pixels is inputted to the de-block filter. The input image includes four macroblocks MB0 ~ MB3 each having a size of 16 × 16 pixels.
[0076]
　Referring to FIG. 11, the processing may be performed in parallel are shown respectively in each dotted line frame. In the example of FIG. 10 while it took 16 processing steps to a series of processing, in the example of FIG. 11, the processing of the same number are aggregated into four process steps. That is, in the first step, filtering necessity determining process J for all vertical boundaries of all macroblocks ~ MB3 MB0 V0,0 ~ J V3,3 are executed in parallel. Then, in a second step, the filtering process F for sixteen vertical boundaries of all macroblocks ~ MB3 MB0 V0,0 ~ F V3,3 are executed in parallel. Next, in a third step, the filtering necessity determining process J for all horizontal boundaries of all macroblocks ~ MB3 MB0 H0,0 ~ J H3,3 are executed in parallel. Then, in the fourth step, the filtering process F for sixteen horizontal boundaries of all macroblocks ~ MB3 MB0 H0,0 ~ F H3,3 are executed in parallel. Incidentally, when the horizontal boundary monkey processed before the vertical boundaries, prior to the first step and second step, third step and fourth step can be performed.
[0077]
　11, (multitude of processes are executed in parallel) parallelism by parallel processing over all macroblocks in the image is an example that is maximize. However, not limited to such an example, the processing over a portion of the macro-blocks rather than all the macroblocks in the picture may be parallelized.
[0078]
　　(2) deblocking the detailed configuration of the filter
　12 is a block diagram showing an example of a detailed configuration of the deblocking filter 24 according to the first embodiment for realizing the above-described parallel processing. Referring to FIG. 12, the de-block filter 24 has a vertical decision block 110, horizontal decision block 114, the horizontal filtering block 130, a vertical filtering block 140 and the parallel control unit 150.
[0079]
　　(2-1) Vertical decision block
　vertical decision block 110 includes a plurality of vertical boundary determination unit 112-1 ~ 112-n. Each vertical boundary determination unit 112, determines information used for the input image, and determines the necessity of filtering the deblocking filter 24 is supplied.
[0080]
　Vertical boundary determination unit 112-1 ~ 112-n, using the pixel values ​​of the input image to the deblocking filter 24 over a plurality of macro blocks in the image, determine whether or not to apply the deblocking filter for vertical boundary to. Each vertical boundary determination unit 112, information indicating the determination result for each vertical boundary (e.g., binary information indicating a determination result that "1" is to be applied to deblocking filter), the horizontal filtering block 130 to output to.
[0081]
　　(2-2) horizontal filtering block
　horizontal filtering block 130 includes a plurality of horizontal filtering unit 132-1 ~ 132-n. Each horizontal filtering unit 132, the determination results for each vertical boundary from the input image and the vertical decision block 110 is supplied.
[0082]
　Each horizontal filtering unit 132, when the determination result by the corresponding vertical boundary determination unit 112 indicates that it should apply a filter to the left and right of the pixel of the corresponding vertical boundary to apply the deblock filter for vertical boundary . Each horizontal filtering unit 132, the pixel by a filtered pixel value after filtering, the pixel values ​​of the input image for the other pixels, and outputs to the horizontal decision block 114 and vertical filtering block 140.
[0083]
　　(2-3) horizontal decision block
　horizontal decision block 114 includes a plurality of horizontal boundary determining unit 116-1 ~ 116-n. Each horizontal boundary determination unit 116, the pixel values after the filtering by the horizontal filtering block 130, and a determination information used for determining the necessity of filtering is supplied.
[0084]
　Horizontal boundary determining unit 116-1 ~ 116-n is across a plurality of macro blocks in an image using pixel values ​​after filtering by the horizontal filtering block 130 determines whether or not to apply the deblocking filter for the horizontal boundary . Each horizontal boundary determination unit 116, information indicating the determination result for each horizontal boundary, and outputs to the vertical filtering block 140.
[0085]
　　(2-4) vertical filtering block
　vertical filtering block 140 includes a plurality of vertical filtering unit 142-1 ~ 142-n. Each vertical filtering unit 142, the determination results for each horizontal boundary from the pixel value and the horizontal decision block 114 after filtering by the horizontal filtering block 130 is supplied.
[0086]
　Each vertical filtering unit 142, when the determination result by the corresponding horizontal boundary determination unit 116 indicates that it should apply a filter to the upper and lower pixels in the corresponding horizontal border to apply the deblock filtering in the horizontal boundary . Each vertical filtering unit 142, the pixel by a filtered pixel value after filtering, and other pixel outputs a pixel value supplied from the horizontal filtering block 130. The output from the vertical filtering section 142 may constitute the output image from the deblocking filter 24.
[0087]
　　(3) detailed configuration of the determination section
　13 is a block diagram showing an example of a more detailed configuration of each vertical boundary determination unit 112 and the horizontal boundary determination unit 116. Referring to FIG. 13, the judgment unit includes tap forming unit 121, calculation unit 122, threshold comparator 123, the distortion evaluation unit 124 and a filtering decision unit 125.
[0088]
　Tap forming unit 121 acquires the pixel values ​​of the reference pixels from the pixel values ​​of two adjacent blocks across the boundary of interest in the input image, tap (see for determining the determination condition B of the luminance component as described above building a set) of pixel values. For example, in a case where the size of each block is 8 × 8 pixels, when a boundary of interest is vertical boundary, tap forming unit 121, from the pixel values ​​belonging to third and sixth row of the left and right two blocks to build a tap. Further, when a boundary of interest is horizontal boundary, tap forming unit 121 constructs a tap from the pixel values ​​belonging to the third and sixth columns of the upper and lower two blocks. Calculation unit 122 substitutes the tap constructed by the tap forming unit 121 to the left side of the judgment formula of the determination condition B, and calculating the value of the edge to be compared with the edge determination threshold value beta. Threshold comparator 123, a value calculated by the calculation unit 122 as compared with the edge determination threshold beta, and outputs the comparison result to the filtering decision unit 125.
[0089]
　Distortion evaluation unit 124, mode information supplied as the determination information (MB mode), using the transformation coefficient information and motion vector information, to evaluate the determination condition A of the luminance component as described above. The distortion evaluating unit 124 outputs the evaluation result to the filtering decision unit 125. As for the color difference components, only the determination of the determination condition A1 based on the mode information due to distortion evaluation unit 124 may be performed.
[0090]
　Filtering determination unit 125, based on the comparison result of the determination condition B which is input from the threshold comparator 123, the evaluation result of the determination condition A inputted from the distortion evaluation unit 124, the deblocking filter for the target boundary It determines whether to apply. Then, the filtering decision unit 125 outputs the information indicating the determination result.
[0091]
　　(4) parallel controller
　parallelization control unit 150 shown in Figure 12, the parallelism of the filtering necessity determination process in the vertical decision block 110 and a horizontal decision block 114, and filtering in the horizontal filtering block 130 and the vertical filtering block 140 controlling the parallelism of the processing.
[0092]
　For example, parallel control unit 150, based on the size of the input image, may control the parallelism of the processing of each block. More specifically, the parallelization control unit 150, the size of the input image is to be relatively large, increasing the parallelism of the processing of each block. Thereby, a decrease in the delay or data rate caused the amount of processing increases with the size of the image can be prevented adaptively. Further, for example, parallel controller 150, a sequence parameter set, based on the parameters included in the picture parameter set or a slice header may control the degree of parallelism of the processing of each block. Thereby, it is possible to configure the flexible parallelism depending on the requirements of each user to develop system. For example, the degree of parallelism may be set in accordance with the constraints of implementations, such as cores or threads software processor.
[0093]
　Further, in this embodiment, that allows parallel processing between macro blocks, it does not affect the result be set how the order of processing of each block in the image is finally output means. Therefore, parallelization control unit 150 may control the order of the filtering necessity determination process in the vertical decision block 110 and a horizontal decision block 114, as well as the order of the filtering process in the horizontal filtering block 130 and vertical filtering block 140 for each block.
[0094]
　More specifically, the parallelization control unit 150, in accordance with the filtering process dependencies between macroblocks may control the order of the filtering process. For example, in the existing techniques, an attempt to implement the parallel processing of each slice in the image, there is a case where the processing of dependencies between macroblocks adjacent to each other across the slice boundary causes the processing delay. However, in this embodiment, parallel control unit 150 can be performed before any other macro block filtering process for a macroblock adjacent to a slice boundary.
[0095]
　For example, in FIG. 14, eight macroblocks MB10 ~ MB13 and MB20 ~ MB23 adjacent across the slice boundary is shown. Among them, the macro block MB10 ~ MB13 is, belongs to the slice SL1. Macro block MB20 ~ MB23 is, belongs to the slice SL2. Of these macroblocks, the filtering processing in the horizontal boundaries of macro blocks MB20 of slice SL2 is dependent on the filtering processing in the vertical boundary of the macroblock MB12 in the slice SL1. Likewise, filtering of the horizontal boundary of the macroblock MB21 of slice SL2 is dependent on the filtering processing in the vertical boundary of the macroblock MB13 in the slice SL1.
[0096]
　In such a situation, as in the example of FIG. 15, parallel controller 150, for example, from the processing of the other boundary filtering process in the vertical boundary of the macroblock MB12 and MB13 of filtering the slice SL1 also it is preferentially executed. As a result, the filtering process of the slice SL2, that a large delay filtering processing in the horizontal boundaries of macro blocks MB20 and MB21 occurs is prevented. Also, as in the example of FIG. 16, when the filtering process in the vertical boundaries of all the macroblocks included in the slice SL1 is executed in parallel to the first well, the horizontal boundaries of macro blocks MB20 and MB21 of slice SL2 It does not cause a delay in the filtering process for.
[0097]
　　[3-2. Modification of the determination condition
　in the preceding paragraph, the vertical boundary determination unit 112, as in the existing method shown in FIG. 4, with reference to the third and sixth pixel row of the block, the vertical boundary of each block determining the necessity of filtering for, and has been described. Similarly, each horizontal boundary determination unit 116 refers to the pixel of the third and sixth columns of the block, it determines necessity of filtering in the horizontal boundaries of each block, and has been described. In such a case, without changing the determination condition for filtering necessity determination process implemented in existing equipment, it is possible to easily realize the configuration according to the present embodiment.
[0098]
　However, the vertical boundary determination unit 112 and the horizontal boundary determination unit 116, when deciding, may use different determination condition existing techniques. For example, each vertical boundary determination unit 112 may refer to the pixels of three or more rows of blocks. Each horizontal boundary determining unit 116 may refer to the pixels of three or more rows of blocks. Further, each vertical boundary determination unit 112 and the horizontal boundary determination unit 116 may use a different determination condition to the existing methods. Hereinafter, with reference to FIGS. 17 to 19, illustrating six examples of methods of determination that may be realized in the present embodiment, respectively.
[0099]
　　(1) First Example
　Figure 17 is an explanatory diagram for explaining the first and second embodiment of the determination method, respectively. In the first and second examples, the filtering necessity determination process in the vertical boundaries (in particular, the determination of the determination condition B of the luminance component), the pixels of all the rows L1 ~ L8 from the first blocks to the eighth There is referenced. Also in the filtering necessity determination process in the horizontal boundary, the pixels of all columns from the first blocks to 8th are referenced.
[0100]
　In a first example, the determination condition of the luminance component may be defined as follows:
　- the luminance component (Luma) of the determination condition ... conditions A and B are both applied if true
　　- Condition A:
　　　(A1 ) block Ba or Bb is an intra prediction
　　　mode; (A2) blocks Ba or Bb has an orthogonal transform coefficient of the non-zero; or
　　　(A3) | MVAx-MVBx | ≧ 4 or | MVAy-MVBy | ≧ 4
　　- conditions
　　　B: ID 0 = | the p- 20 -2P 10 Tasu the p- 00 | Tasu | Q 20 -2Q 10 Tasu Q 00 | Tasu | the p- 27 -2P 17 Tasu the p- 07 | Tasu | Q 27 -2Q 17 Tasu Q 07 |
　　　ｉＤ １＝｜ｐ ２１－２ｐ １１＋ｐ ０１｜＋｜ｑ ２１－２ｑ １１＋ｑ ０１｜＋｜ｐ ２６－２ｐ １６＋ｐ ０６｜＋｜ｑ ２６－２ｑ １６＋ｑ ０６｜
　　　ｉＤ ２＝｜ｐ ２２－２ｐ １２＋ｐ ０２｜＋｜ｑ ２２－２ｑ １２＋ｑ ０２｜＋｜ｐ ２５－２ｐ １５＋ｐ ０５｜＋｜ｑ ２５－２ｑ １５＋ｑ ０５｜
　　　ｉＤ ３＝｜ｐ ２３－２ｐ １３＋ｐ ０３｜＋｜ｑ ２３－２ｑ １３＋ｑ ０３｜＋｜ｐ ２４－２ｐ １４＋ｐ ０４｜＋｜ｑ ２４－２ｑ １４＋ｑ ０４｜
　　　ｉＤ ａｖｅ＝（ｉＤ ０＋ｉＤ １＋ｉＤ ２＋ｉＤ ３）>>２
　　　In the case of a, ID Ave > 1
　　　in the case of a, ID Ave > 1
　　　in the case of a, ID Ave [4-1. Configuration Example of a deblocking filter
　in this section, an example of a configuration according to a second embodiment of a deblocking filter 24.
[0124]
　　(1) dependencies between new processing
　in the present embodiment, the deblocking filter 24 deblocking of filtering necessity determination process in the vertical boundaries of each block, the other blocks in the macro block to which the block belongs to run without waiting for the application of the filter. Moreover, the deblocking filter 24, filtering necessity determination process in the horizontal boundaries of each block is performed without waiting for the application of the deblocking filter to the other blocks in the macro block to which the block belongs. Thus, the dependency of the processing in the macroblock is alleviated.
[0125]
　As a result of the process dependency is relaxed as described above, it is possible to parallelize the filtering necessity determination process in the vertical boundary and the horizontal boundary in the macro block.
[0126]
　Figure 22 is an explanatory diagram for describing an example of a sequence of feasible process in the present embodiment. Again, as an example, it is assumed that an image having a size of 32 × 32 pixels is inputted to the de-block filter. The input image includes four macroblocks MB0 ~ MB3 each having a size of 16 × 16 pixels.
[0127]
　Referring to FIG. 22, the processing may be performed in parallel are shown respectively in each dotted line frame. In the example of FIG. 10 while it took 16 processing steps to a series of processing, in the example of FIG. 22, processing the same number, are aggregated into 12 process steps. That is, in the first step, filtering necessity determining process for the four vertical boundary and four horizontal boundaries of macro blocks J MB0 V0,0 ~ J V0,3 and J H0,0 ~ J H0,3 parallel is It is executed. Then, in a second step, the filtering process F for four vertical boundary macroblocks MB0 V0,0 ~ F V0,3 are executed in parallel. Next, in a third step, the filtering necessity determining process J for the four vertical boundary and four horizontal boundaries of macro blocks MB1 V1,0 ~ J V1,3 and J H1,0 ~ J H1,3 is parallel It is executed. Then, in the fourth step, the filtering process for the four vertical boundary macroblock F MB1 V1,0 ~ F V1,3There are executed in parallel. Then, in the fifth step, the filtering process F for four horizontal boundaries of macro blocks MB0 H0,0 ~ F H0,3 are executed in parallel. Then, in the sixth step, the filtering necessity determining process J for the four vertical boundary and four horizontal boundaries of macro blocks MB2 V2,0 ~ J V2,3 and J H2,0 ~ J H2,3 is parallel It is executed. Then, in the seventh step, the filtering process F for four vertical boundary of the macro blocks MB2 V2,0 ~ F V2,3 are executed in parallel. Next, in the eighth step, the filtering process F for four horizontal boundaries of macro blocks MB1 H1,0 ~ F H1,3 are executed in parallel. Then, in the ninth step, the filtering necessity determining process for the four vertical boundary and four horizontal boundaries of macro blocks J MB3 V3,0 ~ J V3,3 and J H3,0 ~ J H3,3There are executed in parallel. Next, in the tenth step, the filtering process F for four vertical boundary macroblocks MB3 V3,0 ~ F V3,3 are executed in parallel. Then, in the eleventh step, the filtering process F for four horizontal boundaries of macro blocks MB2 H2,0 ~ F H2,3 are executed in parallel. Next, in the twelfth step, the filtering process F for four horizontal boundaries of macro blocks MB3 H3,0 ~ F H3,3 are executed in parallel. In this case also, it is possible to perform the process of the deblocking filter 24 for the entire input image with a small number of processing steps than conventional methods.
[0128]
　　(2) detailed configuration of the deblocking filter
　Figure 23 is a block diagram showing an example of a detailed configuration of the deblocking filter 24 according to the second embodiment for realizing the above-described parallel processing. Referring to FIG. 23, the de-block filter 24 has a vertical decision block 210, horizontal decision block 214, the horizontal filtering block 130, a vertical filtering block 140 and the parallel control unit 150.
[0129]
　　(2-1) Vertical decision block
　vertical decision block 210 includes a plurality of vertical boundary determination unit 212-1 ~ 212-n. Each vertical boundary determination unit 212, whether or not to apply the deblocking filter for the vertical boundary of each block is executed without waiting for the application of the deblocking filter to the other blocks in the macro block to which the block belongs. Each vertical boundary determination unit 212, information indicating the determination result for each vertical boundary (e.g., binary information indicating a determination result that "1" is to be applied to deblocking filter), the horizontal filtering block 130 to output to.
[0130]
　　(2-2) horizontal decision block
　horizontal decision block 214 includes a plurality of horizontal boundary determining unit 216-1 ~ 216-n. Each horizontal boundary determining unit 216, whether or not to apply the deblocking filter for the horizontal boundary of each block is executed without waiting for the application of the deblocking filter to the other blocks in the macro block to which the block belongs. Each horizontal boundary determination unit 216, information indicating the determination result for each horizontal boundary, and outputs to the vertical filtering block 140.
[0131]
　Also in this embodiment, each vertical boundary determination unit 212 and the horizontal boundary determination unit 216 refers to the pixel of the same position as the existing technique, it may determine the necessity of filtering for each boundary . Instead, each vertical boundary determination unit 212 and the horizontal boundary determination unit 216, 3-2. According to the procedure described in the modified example of the determination condition, it may determine the necessity of filtering for each boundary.
[0132]
　　[4-2. Process Flow
　FIG. 24 is a flowchart illustrating an example of processing flow by the deblocking filter 24 in the second embodiment. Referring to FIG. 24, first, the vertical boundary determination unit 212-1 ~ 212-n, for all vertical boundaries included in one target macroblock in the input image, parallel determines the necessity of filtering ( step S202). The horizontal boundary determining unit 214-1 ~ 214-n, for all horizontal boundaries included in the target macroblock, in parallel determines the necessity of filtering (step S204). These steps S202 and Step S204 are also performed in parallel.
[0133]
　Next, the horizontal filtering unit 132-1 ~ 132-n, for vertical boundaries deblocking filter within the determined target macroblock which the to be applied in step S202, to apply the deblocking filter in parallel (step S210 ).
[0134]
　The process in next step S220 is performed as an object the target macroblock in the previous loop. For the first target macroblock, the processing of step S220 may be skipped. Vertical filtering unit 142-1 ~ 142-n, for the determined horizontal boundary in step S204 of the previous loop and to apply the deblocking filter applies a deblocking filter in parallel (step S220).
[0135]
　Thereafter, if the unprocessed target macroblock in the input image, the processing of steps S202 ~ S220 are repeated for the new target macroblock (step S230).
[0136]
　On the other hand, if there are no remaining target macroblock Untreated vertical filtering unit 142-1 ~ 142-n are directed to the target macroblock of the last loop, is determined to apply the deblocking filter the horizontal boundary, applying a deblocking filter in parallel (step S240).
[0137]
　Incidentally, also merely an example process flow described herein. That is, the degree of parallelism and order of processing may be changed. Moreover, the degree of parallelism and order of processing may be adaptively controlled by parallelizing the control unit 150.
[0138]
　　[4-3. Examples of Processing of each LCU
　As described above, the technique according to various embodiments described herein may be implemented as a process based on HEVC coding unit (CU). In HEVC, the largest coding unit size is called the LCU (Largest Coding Unit), for example 64 × 64 pixels LCU can be selected. The size of the smallest selectable CU is 8 × 8 pixels. In encoding and decoding of the image, usually starting from the upper left LCU picture (or slice), according to the order of raster scanning, processing is performed for each LCU. Therefore, in this section, an example of the processing of each such LCU in the deblocking filter 24.
[0139]
　Figure 25 is an explanatory view for explaining the sequence of processing for each LCU associated with the second embodiment described above. Here, the size of the LCU 16 × 16 pixels, the size of the CU assumed to be 8 × 8 pixels.
[0140]
　Referring to FIG. 25, in the upper left of the first stage, filtering the LCU until the (n-1) th LCU has ended. Note that the target pixel filtering for the vertical boundary is shaded by diagonal lines, the pixel filtering in the horizontal boundaries are filled.
[0141]
　The second stage and the processing of the third stage of the lower left upper right in FIG. 25 is a process directed to the n-th LCU. First, before the second stage, the filtering necessity determining process for all vertical boundary and all horizontal boundaries belonging to the n-th LCU is performed in parallel. That is, the filtering necessity determination process of the boundary belonging to each CU in the n-th LCU is performed without waiting for the application of the deblocking filter to other CU in the n-th LCU. Then, in a second step, the filtering process in the vertical boundaries is determined that an application of the deblocking filter of the vertical border belonging to the n-th LCU is performed in parallel. Next, filtering processing in the horizontal boundary is determined that an application of the deblocking filter of the horizontal border belonging to the n-th LCU is performed in parallel.
[0142]
　Thereafter, the processing of the fourth stage at the lower right of FIG. 25 is a process directed to the (n + 1) th LCU. After filtering necessity determining process on border belonging to all CU in the (n + 1) th LCU is performed in parallel, in the fourth step, the filtering of the vertical boundary is determined that an application of the deblocking filter processing is performed in parallel.
[0143]
　Here, although the 16 × 16 pixels the size of LCU as an example, the size of the LCU may be a 32 × 32 pixels or 64 × 64 pixels. In particular, if more larger the size of the LCU chosen, since it increases the number of vertical boundary and the horizontal boundary belongs to one LCU, the effect of shortening the processing time by parallelization is more enhanced.
[0144]
　Figure 26 is a flowchart illustrating an example of a processing flow for each LCU by the deblocking filter 24.
[0145]
　Referring to FIG. 26, first, the vertical boundary determination unit 212-1 ~ 212-n is one for all the vertical boundary included in the target LCU, parallel determining (step a necessity of filtering the input image S252). The horizontal boundary determining unit 216-1 ~ 216-n, for all horizontal boundaries included in the target LCU, parallel determines the necessity of filtering (step S254). These steps S252 and step S254 are also performed in parallel.
[0146]
　Next, the horizontal filtering unit 132-1 ~ 132-n, for vertical boundary deblock the filter and should be applied the determined attention LCU in step S252, to apply the deblocking filter in parallel (step S260) .
[0147]
　Next, the vertical filtering unit 142-1 ~ 142-n, for horizontal boundaries deblocking the filter and should be applied the determined attention LCU in step S254, to apply the deblocking filter in parallel (step S270) .
[0148]
　Thereafter, when there remains LCU unprocessed in the input image, the processing of steps S252 ~ S270 are repeated for the new target LCU (step S280). On the other hand, when there are no remaining LCU untreated, the process ends.
[0149]
　<5. Third
　　Embodiment> [5-1. Summary
　In the first and second embodiments, changes the order of the existing process of the deblocking filter, parallelism of the processing is increased. In particular, in the first embodiment, by utilizing a wider pixel values of the input image to the deblocking filter upon judgment of the necessity of filtering, processing dependencies are mitigated. In the third embodiment described in this section, this concept is further extended. That is, in the third embodiment, by filtering the input pixel values to the deblocking filter in the filtering process for both the vertical boundary and the horizontal boundary, achieve further parallelization of process.
[0150]
　Figure 27 is an explanatory diagram for describing a schematic of this embodiment. The lower left of FIG. 27 is shown the graphic representing the input pixel before being processed by the deblocking filter (also referred to as a re-construct the pixel). In this embodiment, the input pixel to the deblocking filter, filtering necessity determination process in the vertical boundary and the horizontal boundary, and is referred to in the filtering process in the vertical boundary and the horizontal boundary. Therefore, the dependency between the two filtering necessity determination process, and dependencies between the two filtering processes is eliminated together.
[0151]
　However, the filtering process for filtering and horizontal boundaries of the vertical boundary, there is a possibility that the value of a pixel overlapping is updated. The positions of the pixels having such a possibility is illustrated by the blacked out pixel 27. Therefore, deblocking filter according to the present embodiment, the pixel to be updated redundantly by two filters operating in parallel, calculate one output pixel values ​​from the two filter outputs.
[0152]
　　[5-2. Configuration Example of a deblock filter]
　FIG 28 is a block diagram showing an example of a detailed configuration of the deblocking filter 24 according to the third embodiment. Referring to FIG. 28, the de-block filter 24 has a line memory 308, decision block 310, the horizontal filtering block 330, a vertical filtering block 340, a parallel control unit 150 and the calculating unit 360.
[0153]
　The line memory 308 stores the pixel values ​​of the input image input to the deblocking filter 24. Pixel value to be stored by line memories 308 is not updated by the filtering process in the horizontal filtering block 330 and vertical filtering block 340. Pixel value to be stored by the line memory 308 is referred to in the filtering necessity determination process by each unit in the decision block 310 described below. The memory provided in the device for a different purpose from the processing of the deblocking filter 24 may be reused (shared) as the line memory 308.
[0154]
　Decision block 310 includes a plurality of vertical boundary determination unit 312-1 ~ 312-n and a plurality of horizontal boundary determining unit 314-1 ~ 314-n. Each vertical boundary determination unit 312 and the horizontal boundary determination unit 314, the pixel values ​​of the input image to the deblocking filter 24 to be stored by line memories 308, and determination information used for determining the necessity of filtering There is supplied.
[0155]
　Each vertical boundary determination unit 312, using the input pixel value to the de-block filter 24, determines whether or not to apply the deblocking filter for each vertical boundary. Each vertical boundary determination unit 312, information indicating the determination result for each vertical boundary, and outputs to the horizontal filtering block 330.
[0156]
　Each horizontal boundary determining unit 314 also uses the input pixel values ​​to the de-block filter 24, determines whether or not to apply the deblocking filter for each horizontal boundary. Determination processing by the horizontal boundary determination unit 314, parallel performed a determination process by the vertical boundary determination unit 312. Each horizontal boundary determination unit 314, information indicating the determination result for each horizontal boundary, and outputs to the vertical filtering block 340.
[0157]
　Also in this embodiment, each vertical boundary determination unit 312 and the horizontal boundary determination unit 314 refers to the pixel of the same position as the existing technique, it may determine the necessity of filtering for each boundary . Instead, each vertical boundary determination unit 312 and the horizontal boundary determination unit 314, 3-2. According to the procedure described in the modified example of the determination condition, it may determine the necessity of filtering for each boundary.
[0158]
　Horizontal filtering block 330 includes a plurality of horizontal filtering unit 332-1 ~ 332-n. Each horizontal filtering unit 332, the determination results for each vertical boundary from the input pixel value and the decision block 310 from the line memory 308 is supplied.
[0159]
　Each horizontal filtering unit 332, when the determination result by the corresponding vertical boundary determination unit 312 indicates that it should apply a filter to the left and right of the pixel of the corresponding vertical boundary to apply the deblock filter for vertical boundary . Each horizontal filtering unit 332, the pixel by a filtered pixel value after filtering, the input pixel values ​​for the other pixels, and outputs to the computing section 360.
[0160]
　Vertical filtering block 340 includes a plurality of vertical filtering unit 342-1 ~ 342-n. Each vertical filtering unit 342, the determination results for each horizontal boundary from the input pixel value and the decision block 310 from the line memory 308 is supplied.
[0161]
　Each vertical filtering unit 342 when the determination result by the corresponding horizontal boundary determination unit 314 indicates that it should apply a filter to the upper and lower pixels in the corresponding horizontal border to apply the deblock filtering in the horizontal boundary . Filtering process by the vertical filtering unit 342-1 ~ 342-n is performed in parallel with the filtering process with horizontal filtering unit 332-1 ~ 332-n. Each vertical filtering unit 342, the pixel by a filtered pixel value after filtering, the input pixel values ​​for the other pixels, and outputs to the computing section 360.
[0162]
　The calculation unit 360, and the output pixel values ​​from the output pixel values ​​and vertical filtering block 340 from the horizontal filtering block 330 is supplied in parallel. Furthermore, the calculation unit 360, the determination result of the vertical boundary determination unit 312 and the horizontal boundary determination unit 314 is supplied. Calculation unit 360, according to the determination result supplied for a pixel to be filtered by both horizontal filtering block 330 and the vertical filtering block 340, the filter output from the filter output and vertical filtering block 340 from the horizontal filtering block 330 based calculating an output pixel value.
[0163]
　For example, in this embodiment, calculation unit 360, the pixels that are duplicate filters, calculates the average of the two filter outputs. Two average filter output calculated by the calculating unit 360 may be a simple average. Instead, the calculation unit 360 may calculate a weighted average of the two filter outputs. Calculation unit 360, for example, the weight of the weighted average for each pixel can be determined according to the distance to the distance and the horizontal boundaries to the vertical boundary of each pixel.
[0164]
　Figure 29 is an explanatory diagram for explaining the determination of the weights for the weighted average by the calculator 360. Referring to FIG. 29, the pixel of interest P is located in one of the overlapping position shown in FIG. 27 Z is shown in black. Target pixel P Z and the nearest vertical boundary V Z distance D between the V is 3 pixels. Target pixel P Z and the nearest horizontal border H Z distance D between the H is two pixels. The distance D H is the distance D V smaller than. In this case, calculation unit 360, the horizontal boundary H Z weights for the output of the deblock filter for the vertical boundary V Z may be determined larger than the weight for the output of the deblock filter for. In the example of FIG. 29, the vertical boundary V Z filter output V for out the horizontal boundary H Z filter output H for out ratio of the weight between the 2: is 3 and determined.
[0165]
　As understood from FIG. 29, as a result of a weighted average of the two filter outputs are computed, one two-dimensional filter having filter taps along the filter taps and vertical direction along the horizontal direction is applied the same output pixel value and the case can be obtained for each pixel of interest. Thus, the filtering process in the vertical boundary and the horizontal boundary even when allowed to parallelization, it is possible to reduce the block distortion appearing in both the vertical boundary and the horizontal boundary properly. As another example, deblocking filter 24 may have a single two-dimensional filter that calculates horizontal filtering, vertical filtering and weighted average simultaneously. However, in that case, since the need to variously changed for each pixel filter coefficients occurs, mounting becomes very complicated. In contrast, if calculating a weighted average after two one-dimensional filter was executed in parallel as in the third embodiment, while taking advantage of the mechanism of existing deblocking filter, substantially two-dimensional filter the manner equivalent processing can be easily realized.
[0166]
　Figure 30 is an explanatory diagram for explaining an example of the weight for the weighted average is determined according to the example of FIG. 29. Referring to FIG. 30, 6 × 6 = 36 pixels located around the single intersection of the vertical boundary and the horizontal boundary (pixels overlapping position described above) are shown. Of these pixels, for the pixels located at the same distance from the vertical boundary and the horizontal boundary, the filter output V out and the filter output H out weight ratio pair between 1 (or 2: 2 or 3: 3) it is. The pixel closer to the vertical boundary, the filter output V out towards the weights to the filter output H out is determined greater than the weight of the (e.g., pixel P 1 ratio of the weight of V out : H out = 3 : 1). On the other hand, the pixel closer to the horizontal boundary, the filter output V out towards the weights to the filter output H out is determined smaller than the weight of the (e.g., pixel P 2 ratio of the weight of V out : H out = 1: 3).
[0167]
　By thus changing the weight of the weighted average according to the distance between each pixel and the boundary, thereby improving image quality by effectively suppress block distortion.
[0168]
　Incidentally, the weight described above is only an example. For example, calculation unit 360, instead of the distance between each pixel and the boundary (or in addition to), according to the strength of the edge of the vertical boundary and the horizontal boundary corresponding to each pixel, weighted for each pixel it may determine a weighted average. The strength of the edges of this case, for example, may be represented by parameters such as the edge of the value calculated by the arithmetic unit 122 shown in FIG. 13. In this case, the weight of the filter output for strong border a more edges can be determined larger than the weight of the filter output for more edges weak boundary. By thus changing the weight of the weighted average according to the strength of the edges, it is possible to improve the adaptive effect of the deblocking filter for the boundary that block distortion has appeared strongly.
[0169]
　Calculation unit 360 for the pixel to be filtered by one of horizontal filtering block 330 and the vertical filtering block 340, selects the actual output from the block subjected to filtering. The calculation unit 360 for the pixels is also not filtered by any of the horizontal filtering block 330 and the vertical filtering block 340 outputs the input pixel values ​​to the deblocking filter 24 as it is. The output pixel values ​​from the calculator 360 in accordance with the filtering necessity of determination results are shown in the table of FIG. 31.
[0170]
　　[5-3. Sample order processing]
　will be described two examples of the sequence of feasible process by the deblocking filter 24 in the present embodiment. Again, as an example, it is assumed that an image having a size of 32 × 32 pixels is inputted to the de-block filter. The input image includes four macroblocks MB0 ~ MB3 each having a size of 16 × 16 pixels.
[0171]
　　(1) First Example
　Firstly, for comparison, showing the sequence of processing when the dependencies remain between the filtering process for filtering and horizontal boundaries of the vertical boundary in Fig. 32. In Figure 32, the first step, filtering necessity determining process J for all four vertical boundary and all horizontal boundaries of macro blocks ~ MB3 MB0 V0,0 ~ J V3,3 and J H0,0 ~ J H3 , 3 are executed in parallel. Then, in a second step, the filtering process F for four sixteen vertical boundary macroblocks ~ MB3 MB0 V0,0 ~ F V3,3 are executed. Next, in a third step, the filtering process F for four sixteen horizontal boundaries of macro blocks ~ MB3 MB0 H0,0 ~ F H3,3 are executed. Thereafter, in the fourth step, the pixel values after the filtering process in the horizontal boundary, is stored in the memory for the output from the deblocking filter 24.
[0172]
　Figure 33 shows a first example of a sequence of processing realized in the present embodiment. In Figure 33, the first step, filtering necessity determining process J for all four vertical boundary and all horizontal boundaries of macro blocks ~ MB3 MB0 V0,0 ~ J V3,3 and J H0,0 ~ J H3 , 3 are executed in parallel. Then, in a second step, the four macro blocks MB0 ~ filtering F for all vertical boundary and all horizontal boundary MB3 V0,0 ~ F V3,3 and F H0,0 ~ F H3,3 parallel that may be executed manner (in practice, only the filtering is determined to be necessary boundary are filtered). Thereafter, in the third step, each pixel value is stored in the memory for the output from the deblocking filter 24. At that time, for the pixels which are filtered by both the horizontal filtering block 330 and the vertical filtering block 340, the weighted average of the two filter outputs may be calculated as an output pixel value.
[0173]
　　(2) Second Example
　either the first example described above is an example to maximize parallelism, also in this embodiment, the deblocking filter 24 may also implement the processing of each macro block.
[0174]
　First, for comparison, shows the order of processing of each macro block in the case where dependencies remain between the filtering process for filtering and horizontal boundaries of the vertical boundary in Fig. 34. The order of processing shown in FIG. 34 is a sequence substantially the same as the processing shown in FIG. 22 in connection with the second embodiment. Not shown in terms of simplicity of explanation in FIG. 22, four processing steps for storing the pixel values ​​of the memory for the output (6, 10, 14 and 16 steps), FIG. 34 It has been explicitly. Process of Figure 34 is composed of sixteen processing steps including these four processing steps.
[0175]
　Figure 35 shows a second example of the order of processing to be implemented in the present embodiment. In Figure 35, the first step, filtering necessity determining process J for the four vertical boundary and four horizontal boundaries of macro blocks MB0 V0,0 ~ J V0,3 and J H0,0 ~ J H0,3 parallel to be executed. Then, in a second step, the filtering process F for four vertical boundaries and 4 horizontal boundaries of macro blocks MB0 V0,0 ~ F V0,3 and F H0,0 ~ F H0,3 are performed in parallel that. Next, in a third step, each pixel value of the macroblock MB0 is stored in the memory for the output from the deblocking filter 24. At that time, for the pixels which are duplicate filtered by two filters, a weighted average of the two filter outputs may be calculated as an output pixel value. Then, the macro block MB1 in the sixth step from the fourth step, the macro block MB2 in the ninth step from the seventh step, the macroblock MB3 is similarly processed in the twelfth step from the tenth step. Process of Figure 35 is composed of twelve process steps less than the processing of FIG. 34.
[0176]
　Thus, in the third embodiment, since the the dependencies between the filtering process for filtering and horizontal boundaries of the vertical boundary it is eliminated, as compared with the first and second embodiment, it is possible to execute the processing in the deblocking filter 24 with fewer processing steps. Incidentally, one of the advantages of having only input pixels to the deblocking filter is referred to in the filtering process, the filtering process for filtering and horizontal boundaries of the vertical boundary in any way to a filter tap dependencies between is that does not occur. Therefore, it is possible to improve the image quality by a filter taps from many pixels than existing methods. For example, existing techniques, as described in relation to FIG. 7, a filter tap three pixels per each side of each boundary was used. However, in this embodiment, for example it is used 5 pixels or more filter taps for each pixel of each boundary, causing no dependencies between process. Further, even when a smaller block size of a processing unit of the deblocking filter, never again dependencies between process occurs.
[0177]
　In the third embodiment, similarly to the first and second embodiments, the degree of parallelism and order of the processing in the deblocking filter 24 may be controlled by the parallel control unit 150.
[0178]
　　[5-4. Process Flow
　FIG. 36 is a flowchart illustrating an example of processing flow by the deblocking filter according to a third embodiment. Further, FIG. 37 is a flowchart showing an exemplary flow of a pixel value calculation process shown in FIG. 36.
[0179]
　Referring to FIG. 36, first, the vertical boundary determination unit 312-1 ~ 312-n for all vertical boundary in the input image (or macro block), parallel determines the necessity of filtering (step S302 ). The horizontal boundary determining unit 314-1 ~ 314-n, for all of the horizontal boundary in the input image (or macro block), parallel determines the necessity of filtering (step S304). These steps S302 and Step S304 are also performed in parallel.
[0180]
　Next, the horizontal filtering unit 332-1 ~ 332-n, for all vertical boundary is determined that an application of the deblocking filter in step S302, to apply the deblocking filter in parallel (step S306). Also, vertical filtering unit 342-1 ~ 342-n, for all of the horizontal boundary is determined that an application of the deblocking filter in step S304, to apply the deblocking filter in parallel (step S308). These steps S306 and Step S308 are also performed in parallel.
[0181]
　Next, the calculation unit 360, the pixel value calculation processing shown in FIG. 37 is performed (step S310). Referring to FIG. 37, the processing from step S314 to step S326 is a loop for each pixel to be processed (step S312).
[0182]
　In step S314, the calculation unit 360 determines whether or filtered by two of both filter for the target pixel is the vertical boundary and the horizontal boundary (step S314). Here, when the pixel of interest is filtered by both of the two filters, the process proceeds to step S322. On the other hand, when the pixel of interest is not been filtered by both of the two filters, the process proceeds to step S316.
[0183]
　In step S316, calculation unit 360 determines whether or filtered by one of two filters for the target pixel is the vertical boundary and the horizontal boundary (step S316). Here, when the pixel of interest is filtered by one of two filters, the process proceeds to step S320. On the other hand, when the pixel of interest is not being filtered by any filters, the process proceeds to step S318.
[0184]
　In step S318, calculation unit 360 acquires an input pixel value to the deblocking filter 24 (step S318). In step S320, calculation unit 360, actually obtains the filter output from the filter by filtering the target pixel (step S320).
[0185]
　In step S322, calculation unit 360, the distance of the value of the weight for calculating the weighted average of the filter outputs from the two filters for the pixel of interest, for example a distance and horizontal boundaries to the vertical boundary of a pixel of interest, or target It determined according to the strength of the edge of the vertical boundary and the horizontal boundary corresponding to the pixel (step S322). The calculation unit 360, using the determined weight and calculating a weighted average of the filter outputs from the two filters (step S324).
[0186]
　Thereafter, the calculation unit 360 obtains in step S318 or step S320, or to store the pixel values ​​of the pixel of interest calculated in step S324 into the memory (step S326). When such processing is performed for all the pixels to be processed, a series of processes shown in FIGS. 36 and 37 is terminated.
[0187]
　<6. Application> to various codecs
　technology according to the present disclosure are applicable to a variety of codecs associated with the encoding and decoding of the image. In this section, an example in which the technology according to the present disclosure is applied respectively to multi-view codec and a scalable codec.
[0188]
　　[6-1. Multiview Codec
　multiview codec is an image coding method for encoding and decoding a so-called multi-view images. Figure 38 is an explanatory diagram for describing a multi-view codec. Referring to FIG. 38, the sequence of frames of the three views taken respectively in the three aspects is illustrated. Each view view ID (view_id) is applied. These one of views of the plurality of views is specified in the base view (base view). Views other than the base view is referred to as a non-base views. In the example of FIG. 38, the view is the view ID is "0" and the base view, two views view ID is "1" or "2" is a non-base view. When encoding the image data of multi-view, by encoding a frame of non-base view based on the coding information for a frame of the base view, the data size of the encoded stream as a whole can be compressed.
[0189]
　In coding and decoding processes in accordance with the multiview codec described above, the deblocking filter can be applied to each view. Upon application of the deblocking filter for each view, according to the technique of the present disclosure, the processing unit includes a plurality of CU of each view, horizontal filtering is parallelized, and vertical filtering may be parallelized. The processing unit may be the like several CU, LCU or picture. Further, parameters for controlling the parallel processing (e.g., parameters described in paragraph 0092), may be set for each view. Further, the parameters set in the base view may be reused in non-base view.
[0190]
　The horizontal filtering and vertical filtering across multiple views may be parallelized. Parameters for controlling the parallel processing (e.g., parameters described in paragraph 0092), it may be common across multiple views. Further, a flag indicating whether or not parameters across multiple views are shared may be designated additionally
[0191]
　Figure 39 is an explanatory diagram for describing application to multi-view codec of the above-described image encoding process. Referring to FIG. 39, the configuration of the multi-view coding apparatus 710 as one example is shown. Multi-view coding apparatus 710 includes a first encoding unit 720, a second encoding unit 730 and a multiplexing unit 740.
[0192]
　The first encoding unit 720, a base view image encoding, to generate an encoded stream of the base view. The second encoding unit 730 encodes the non-base view image to generate a coded stream of non-base view. Multiplexer 740, and encoded stream of the base view generated by the first encoding unit 720, one or more non-base view generated by the second encoding unit 730 and the encoded stream multiplexing, generating a multiplexed stream of multi-view.
[0193]
　The first encoding unit 720 and the second encoding unit 730 illustrated in FIG. 39 has the same structure as the image coding apparatus 10 according to the embodiment described above. Thus, upon application of the deblocking filter for each view, the process unit includes a plurality of CU, parallel the horizontal filtering, and it is possible to parallelize the vertical filtering. These processes control the parameters may be inserted in the header area of ​​the encoded stream of each view, or may be inserted into common header area in a multiplexed stream.
[0194]
　Figure 40 is an explanatory diagram for describing application to multi-view codec of the above-described image decoding processing. Referring to FIG. 40, the configuration of the multi-view decoding apparatus 760 as one example is shown. Multiview decoding apparatus 760 includes a demultiplexer 770, first decoding section 780 and second decoding section 790.
[0195]
　Demultiplexer 770 demultiplexes the encoded stream of encoded streams and one or more non-base view of the base view a multiplexed stream of multi-view. The first decoding unit 780 decodes the base view image from the encoded stream of the base view. Second decoding unit 730 decodes the non-base view image from the encoded stream of the non-base view.
[0196]
　Figure 40 The first decoding unit 780 and the second decoding unit 790 illustrated in has the same structure as the image decoding apparatus 60 according to the embodiment described above. Thus, upon application of the deblocking filter for each view, the process unit includes a plurality of CU, parallel the horizontal filtering, and it is possible to parallelize the vertical filtering. These processes control the parameters may be obtained from the header area of ​​the encoded stream of each view, or may be obtained from common header area in a multiplexed stream.
[0197]
　　[6-2. Scalable Codec
　scalable codec is an image coding method for realizing a so-called hierarchical coding. Figure 41 is an explanatory diagram for describing a scalable codec. Referring to FIG. 41, the spatial resolution, there is shown a sequence of frames of three layers of different temporal resolution or image quality. Each layer, the layer ID (layer_id) is applied. Among the plurality of layers, most resolution (or image quality) low layer, a base layer (base layer). Layer other than the base layer is referred to as an enhancement layer. In the example of FIG. 41, the layer layer ID is "0" is the base layer, two layers, the layer ID is "1" or "2" is an enhancement layer. When encoding the image data of the multi-layer, by encoding a frame of the enhancement layer based on the coding information for a frame of the base layer, the data size of the encoded stream as a whole can be compressed.
[0198]
　In encoding and decoding processing according to a scalable codec described above, the deblocking filter can be applied to each layer. Upon application of the deblocking filter for each layer, according to the technique of the present disclosure, the processing unit includes a plurality of CU of each layer, the horizontal filtering is parallelized, and vertical filtering may be parallelized. The processing unit may be the like several CU, LCU or picture. Further, parameters for controlling the parallel processing (e.g., parameters described in paragraph 0092), may be set for each layer. Further, the parameters set in the base layer, may be reused in the enhancement layer.
[0199]
　The horizontal filtering and vertical filtering across multiple layers may be parallelized. Parameters for controlling the parallel processing (e.g., parameters described in paragraph 0092), may be shared across multiple layers. Further, a flag indicating whether or not the parameter over a plurality of layers is common may be designated additionally
[0200]
　Figure 42 is an explanatory diagram for describing application to scalable codec of the above-described image encoding process. Referring to FIG. 42, the configuration of the scalable coding apparatus 810 as one example is shown. Scalable encoding apparatus 810 includes a first encoding unit 820, a second encoding unit 830 and a multiplexing unit 840.
[0201]
　The first encoding unit 820, a base layer picture is encoded to generate an encoded stream of the base layer. The second encoding unit 830, an enhancement layer picture is encoded to generate an encoded stream of the enhancement layer. Multiplexing unit 840, and encoded stream of the base layer generated by the first encoding unit 820, one or more enhancement layers generated by the second encoding unit 830 and the encoded stream multiplexing, multi generating a multiplexed stream layers.
[0202]
　The first encoding unit 820 and the second encoding unit 830 illustrated in FIG. 42 has the same structure as the image coding apparatus 10 according to the embodiment described above. Thus, upon application of the deblocking filter to each layer, in a processing unit including a plurality of CU, parallel the horizontal filtering, and it is possible to parallelize the vertical filtering. These processes control the parameters may be inserted in the header area of ​​the encoded stream of each layer, or may be inserted into common header area in a multiplexed stream.
[0203]
　Figure 43 is an explanatory diagram for describing application to scalable codec of the above-described image decoding processing. Referring to FIG. 43, the configuration of the scalable decoding apparatus 860 as one example is shown. Scalable decoding apparatus 860 includes demultiplexer 870 includes a first decoding unit 880 and the second decoding unit 890.
[0204]
　Demultiplexer 870 demultiplexes the encoded stream of the base layer of the coded stream and one or more enhancement layers the multiplexed stream of a multi-layer. The first decoding unit 880 decodes the base layer image from the base layer of the coded stream. Second decoding section 830 decodes the enhancement layer image from the enhancement layer encoded stream.
[0205]
　The first decoding unit 880 and the second decoding unit 890 illustrated in FIG. 43 has the same configuration as the image decoding apparatus 60 according to the embodiment described above. Thus, upon application of the deblocking filter to each layer, in a processing unit including a plurality of CU, parallel the horizontal filtering, and it is possible to parallelize the vertical filtering. These processes control the parameters may be obtained from the header area of ​​the encoded stream of each layer, or may be obtained from common header area in a multiplexed stream.
[0206]
　<7. Applications>
　image encoding device 10 and the image decoding apparatus 60 according to the embodiment described above, satellite broadcasting, cable broadcasting such as cable TV, distribution on the Internet, and a transmitter, such as in distribution to the terminal by the cellular communication or receiver, an optical disk, a recording apparatus for recording an image on a medium such as a magnetic disk and a flash memory, or may be applied to various electronic devices such as reproducing apparatus for reproducing images from these storage media. The following describes four applications.
[0207]
　　[7-1. First application example]
　FIG. 44 illustrates an example of a schematic configuration of a television device according to the embodiment described above. Television 900 includes an antenna 901, a tuner 902, a demultiplexer 903, a decoder 904, a video signal processing unit 905, display unit 906, the audio signal processing unit 907, a speaker 908, an external interface 909, the control unit 910, a user interface 911, and a bus 912.
[0208]
　The tuner 902 extracts a signal of a desired channel from broadcast signals received through an antenna 901, and demodulates the extracted signal. Then, the tuner 902 outputs an encoded bit stream obtained by the demodulation to the demultiplexer 903. That is, the tuner 902, the image receives an encoded stream that has been encoded, serves as transmission means of the television 900.
[0209]
　The demultiplexer 903 separates a video stream and an audio stream of a program to be viewed from the encoded bit stream, and outputs each stream separated to the decoder 904. Further, the demultiplexer 903 extracts auxiliary data from the encoded bit stream, such as EPG (Electronic Program Guide), and supplies the extracted data to the control unit 910. Note that the demultiplexer 903, when the coded bit stream is scrambled may perform descrambling.
[0210]
　The decoder 904 decodes the video stream and the audio stream input from the demultiplexer 903. The decoder 904 outputs video data generated by the decoding process to the video signal processing unit 905. The decoder 904 outputs the audio data generated by the decoding process to the audio signal processing unit 907.
[0211]
　Video signal processing unit 905 reproduces the video data input from the decoder 904, and displays the video on the display unit 906. The video signal processing unit 905 may display an application screen supplied via a network to the display unit 906. The video signal processor 905, the video data, according to the setting, for example, an additional process such as noise removal may be performed. Furthermore, the video signal processing unit 905, for example a menu, and generates an image of a GUI such as buttons or a cursor (Graphical User Interface), may be superimposed the generated image in the output image.
[0212]
　Display unit 906 is driven by a drive signal supplied from the video signal processing unit 905, a display device (e.g., a liquid crystal display, a plasma such as a display or OLED) displays an image or images on the image plane of the.
[0213]
　The audio signal processing unit 907, the audio data input from the decoder 904 performs a reproducing process such as D / A conversion and amplification, to output the sound from the speaker 908. The audio signal processing unit 907 may perform additional processing such as noise removal on the audio data.
[0214]
　The external interface 909 is an interface for connecting the television 900 to an external device or network. For example, a video stream or an audio stream received via the external interface 909 may be decoded by the decoder 904. That is, the external interface 909 also receives an encoded stream image is encoded, serves as transmission means of the television 900.
[0215]
　Control unit 910 includes a processor such as a CPU (Central Processing Unit), and memory such as RAM (Random Access Memory) and ROM (Read Only Memory). The memory stores programs, program data, EPG data, and the like data acquired via a network to be executed by the CPU. The program stored in the memory is, for example, is read by the CPU at the time of activation of the television 900, it is executed. The CPU executes a program, for example, in response to an operation signal input from the user interface 911, controls the operation of the television 900.
[0216]
　The user interface 911 is connected to the control unit 910. The user interface 911 may, for example, a user has a like receiver of buttons and switches, and a remote control signal for operating the television 900. The user interface 911 detects a user operation through these components, generates an operation signal, and outputs the generated operation signal to the control unit 910.
[0217]
　Bus 912, a tuner 902, a demultiplexer 903, a decoder 904, connects the video signal processing unit 905, the audio signal processing unit 907, an external interface 909 and the control unit 910 to each other.
[0218]
　In such television 900 which is configured, the decoder 904 has a function of an image decoding apparatus 60 according to the embodiment described above. Thereby, it is possible to speed when decoding the image, the processing to increase the parallelism of processing deblock filter at television apparatus 900.
[0219]
　　[7-2. Second application example]
　FIG. 45 illustrates an example of a schematic configuration of a cellular phone including the above-described embodiments. Mobile phone 920 includes an antenna 921, a communication unit 922, the audio codec 923, a speaker 924, a microphone 925, camera unit 926, image processing unit 927, demultiplexing unit 928, a recording reproduction unit 929, a display unit 930, the control unit 931, the operation part 932, and a bus 933.
[0220]
　Antenna 921 is connected to the communication unit 922. Speaker 924 and microphone 925 are connected to the voice codec 923. Operation unit 932 is connected to the control unit 931. Bus 933, communication unit 922, the audio codec 923, the camera unit 926, image processing unit 927, demultiplexing unit 928, a recording reproduction unit 929, the display unit 930, and the control unit 931 to each other.
[0221]
　Mobile phone 920, voice communication mode, data communication mode, a variety of operating modes including a shooting mode and a video communication mode, transmission and reception of voice signals, e-mail or image data transmitted and received, captured image, and data such as recording of perform the operation.
[0222]
　In voice communication mode, an analog audio signal generated by the microphone 925 is supplied to the audio codec 923. The audio codec 923, the analog audio signal is converted into sound data, and the converted audio data A / D-converted compressed. The audio codec 923 outputs the compressed audio data to the communication unit 922. The communication unit 922 encodes and modulates the audio data, to generate a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station via an antenna 921 (not shown). The communication unit 922 amplifies a radio signal received via the antenna 921 and frequency conversion, and acquires the received signal. Then, the communication unit 922 generates audio data demodulates and decodes the received signal, and outputs the generated audio data to the audio codec 923. The audio codec 923 decompresses the audio data and the D / A conversion to generate an analog audio signal. Then, the audio codec 923 to output voice generated audio signal is supplied to the speaker 924.
[0223]
　Further, in the data communication mode, for example, the control unit 931, in response to an operation by the user via the operation unit 932, generates character data forming the electronic mail. The control unit 931 displays the character on the display unit 930. The control unit 931 generates electronic mail data in accordance with a transmission instruction from the user via the operation unit 932, and outputs the generated electronic mail data to the communication unit 922. The communication unit 922, the e-mail data encoding and modulation to generate a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station via an antenna 921 (not shown). The communication unit 922 amplifies a radio signal received via the antenna 921 and frequency conversion, and acquires the received signal. Then, the communication unit 922 restores the electronic mail data demodulates and decodes the received signal, and outputs the restored electronic mail data to the control unit 931. Control unit 931 causes display of the contents of the e-mail on the display unit 930, and stores the e-mail data in the storage medium of the recording and reproducing unit 929.
[0224]
　Reproducing unit 929 includes an arbitrary storage medium can be read and written. For example, the storage medium may be a built-in storage medium such as RAM or flash memory, a hard disk, magnetic disk, optical magnetic disk, an optical disk, an external attachment type storage medium such as a USB memory or a memory card, it may be.
[0225]
　Further, in the photographing mode, for example, the camera unit 926, the image data generated by imaging a subject, and outputs the generated image data to the image processing unit 927. The image processing unit 927, the image data input from the camera unit 926 encodes, and stores the encoded stream in the storage medium of the recording and reproducing unit 929.
[0226]
　Further, in the video communication mode, for example, the demultiplexer 928, the video stream encoded by the image processing unit 927, multiplexes the audio stream input from the audio codec 923, the communication unit 922 of the multiplexed stream to output to. The communication unit 922 encodes and modulates the stream to generate a transmission signal. Then, the communication unit 922 transmits the generated transmission signal to a base station via an antenna 921 (not shown). The communication unit 922 amplifies a radio signal received via the antenna 921 and frequency conversion, and acquires the received signal. These transmit and receive signals may include encoded bit stream. Then, the communication unit 922 restores the stream demodulates and decodes the received signal, and outputs the restored stream to the demultiplexing unit 928. Demultiplexing unit 928 separates a video stream and an audio stream from the input stream, and outputs the video stream to the image processing unit 927, an audio stream to the audio codec 923. The image processing unit 927 decodes the video stream to generate video data. The video data is supplied to the display unit 930, a series of images is displayed by the display unit 930. The audio codec 923 decompresses the audio stream and D / A conversion to generate an analog audio signal. Then, the audio codec 923 to output voice generated audio signal is supplied to the speaker 924.
[0227]
　In the portable telephone 920 thus constructed, the image processing unit 927 has a function of the image encoding apparatus 10 and the image decoding apparatus 60 according to the embodiment described above. Thus, in encoding and decoding of the image in the mobile phone 920, it is possible to speed up the process by increasing the parallelism of the processing of the deblocking filter.
[0228]
　　[7-3. Third Application Example]
　FIG. 46 illustrates an example of a schematic configuration of a recording and reproducing apparatus according to the embodiment described above. Recording and reproducing apparatus 940, for example, for recording audio data and video data of the received broadcast program encoded to the recording medium. The recording and reproducing apparatus 940, for example, audio data and video data acquired from another apparatus may be recorded in coded and recorded medium. The recording and reproducing apparatus 940, for example, in response to a user instruction to reproduce the data recorded on the recording medium monitor and on a speaker. At this time, the recording and reproduction apparatus 940 decodes the audio data and video data.
[0229]
　Recording reproducing apparatus 940 includes a tuner 941, an external interface 942, an encoder 943, HDD (Hard Disk Drive) 944, a disk drive 945, a selector 946, a decoder 947, OSD (On-Screen Display) 948, the control unit 949 and a user interface, equipped with a 950.
[0230]
　The tuner 941 extracts a signal of a desired channel from broadcast signals received via an antenna (not shown), and demodulates the extracted signal. Then, the tuner 941 outputs a coded bit stream obtained by the demodulation to the selector 946. That is, the tuner 941 serves as transmission means of the recording and reproducing apparatus 940.
[0231]
　The external interface 942 is an interface for connecting the recording and reproducing apparatus 940 and an external device or network. The external interface 942 is, for example, IEEE1394 interface, a network interface, or and the like USB interface or a flash memory interface. For example, video data and audio data received via the external interface 942 are input to the encoder 943. That is, the external interface 942 serves as transmission means in a recording and reproducing apparatus 940.
[0232]
　The encoder 943, when the video data and audio data inputted from the external interface 942 are not encoded, encoding video data and audio data. Then, the encoder 943 outputs the encoded bit stream to the selector 946.
[0233]
　HDD944 records video and coded bit stream content data is compressed, such as voice, various programs and other data in an internal hard disk. Further, HDD 944, upon reproduction of video and audio, reading these data from the hard disk.
[0234]
　Disk drive 945 performs recording and reading of data into a recording medium attached. Recording medium mounted on the disk drive 945, for example, a DVD disc (DVD-Video, DVD-RAM, DVD-R, DVD-RW, DVD + R, DVD + RW, etc.), or Blu-ray may the like (registered trademark) disk .
[0235]
　The selector 946, when the video and audio recording, and select the encoded bit stream input from the tuner 941 or the encoder 943, and outputs the selected encoded bit stream to the HDD944 or the disk drive 945. The selector 946, when reproduction of video and audio, and outputs the encoded bit stream input from the HDD944 or the disk drive 945 to the decoder 947.
[0236]
　The decoder 947 decodes the encoded bit stream to generate video data and audio data. Then, the decoder 947 outputs the generated video data to the OSD 948. The decoder 904 outputs the generated audio data to an external speaker.
[0237]
　OSD948 reproduces the video data input from the decoder 947, and displays an image. Also, OSD 948 is the image to be displayed, for example menu may overlap the GUI image such as a button or a cursor.
[0238]
　Control unit 949 includes a processor such as a CPU, and a memory such as RAM and ROM. The memory stores programs executed by the CPU, program data, and the like. The program stored in the memory is, for example, is read by the CPU at the time of activation of the recording and reproducing apparatus 940, is executed. The CPU executes a program, for example, in response to an operation signal input from the user interface 950, controls the operation of the recording and reproducing apparatus 940.
[0239]
　The user interface 950 is connected to the control unit 949. The user interface 950 includes a button and a switch for the user to operate the recording and reproducing apparatus 940, and the like receiving portion of the remote control signal. The user interface 950 detects a user operation through these components, generates an operation signal, and outputs the generated operation signal to the control unit 949.
[0240]
　In the recording and reproducing apparatus 940 configured in this manner, the encoder 943 has the function of the image encoding apparatus 10 according to the embodiment described above. The decoder 947 has a function of an image decoding apparatus 60 according to the embodiment described above. Thus, in encoding and decoding of the image in the recording and reproducing apparatus 940, it is possible to speed up the process by increasing the parallelism of the processing of the deblocking filter.
[0241]
　　[7-4. Fourth Application Example]
　FIG. 47 illustrates an example of a schematic configuration of an imaging apparatus according to the embodiment described above. Imaging device 960 captures a subject to generate an image, and records the image data to the encoding to the recording medium.
[0242]
　Imaging device 960 includes an optical block 961, an imaging unit 962, the signal processing unit 963, image processing unit 964, display unit 965, external interface 966, a memory 967, media drive 968, OSD 969, a control unit 970, a user interface 971, and bus equipped with a 972.
[0243]
　The optical block 961 is connected to the imaging unit 962. Imaging unit 962 is connected to the signal processing unit 963. Display unit 965 is connected to the image processing unit 964. The user interface 971 is connected to the control unit 970. Bus 972 connects the image processing unit 964, external interface 966, a memory 967, media drive 968, OSD 969, and a control unit 970 to each other.
[0244]
　The optical block 961 has a focus lens and a diaphragm mechanism. The optical block 961 forms an optical image of a subject on the imaging surface of the imaging unit 962. Imaging unit 962 includes an image sensor such as CCD or CMOS, converts the image signal as an electric signal by photoelectric converting an optical image formed on the imaging surface. The imaging unit 962 outputs the image signal to the signal processing section 963.
[0245]
　The signal processing unit 963 performs knee correction, gamma correction, various types of camera signal processing such as color correction on the image signal input from the imaging unit 962. The signal processing unit 963 outputs the image data after the camera signal processing to the image processing unit 964.
[0246]
　The image processing unit 964, the image data input from the signal processing unit 963 encodes, and generates encoded data. Then, the image processing unit 964 outputs the generated encoded data to the external interface 966 or the media drive 968. The image processing unit 964 decodes the encoded data input from the external interface 966 or the media drive 968 to generate image data. Then, the image processing unit 964 outputs the generated image data to the display unit 965. The image processing unit 964 may output the image data input from the signal processing unit 963 to the display unit 965 to display the image. The image processing unit 964, the display data acquired from the OSD 969, may be superimposed on the image to be output to the display unit 965.
[0247]
　OSD969, for example menu, and generates a GUI image such as a button or a cursor, and outputs the generated image to the image processing unit 964.
[0248]
　The external interface 966 is configured as, for example, a USB input and output terminals. External interface 966, for example, when printing images, and connects the image pickup device 960 and the printer. Further, the external interface 966, the drive is connected as necessary. The drive, for example, the mounted removable medium such as a magnetic disk or optical disk, a program read from the removable medium can be installed in the imaging device 960. Furthermore, the external interface 966 may be configured as a network interface connected to a network such as a LAN or the Internet. That is, the external interface 966 serves as transmission means in the imaging device 960.
[0249]
　Recording medium mounted on the media drive 968, for example, a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory may be any readable and writable removable medium. The recording medium in the media drive 968 is fixedly attached, for example, the storage unit of the non-commutative transportable may be configured as internal hard disk drive or SSD (Solid State Drive).
[0250]
　Control unit 970 includes a processor such as a CPU, and a memory such as RAM and ROM. The memory stores programs executed by the CPU, program data, and the like. The program stored in the memory is, for example, is read by the CPU at the time of activation of the image pickup device 960, it is performed. The CPU executes a program, for example, in response to an operation signal input from the user interface 971, controls the operation of the imaging apparatus 960.
[0251]
　The user interface 971 is connected to the control unit 970. The user interface 971, for example, with a button and a switch for the user to operate the imaging apparatus 960. The user interface 971 detects a user operation through these components, generates an operation signal, and outputs the generated operation signal to the control unit 970.
[0252]
　In such an imaging apparatus 960 thus constructed, the image processing unit 964 has a function of the image encoding apparatus 10 and the image decoding apparatus 60 according to the embodiment described above. Thus, in encoding and decoding of the image in the image pickup apparatus 960, it is possible to speed up the process by increasing the parallelism of the processing of the deblocking filter.
[0253]
　<8. Summary>
　Up to this point, with reference to FIGS. 1 to 47 were described in detail three embodiments of the deblocking filter of the image encoding device 10 and the image decoding apparatus 60 according to an embodiment. According to these three embodiments, the processing dependency deblock filter that existed in the existing technique is alleviated. Thereby, it is possible to increase the parallelism of processing in the application of the deblocking filter. As a result, to avoid a reduction in delay or data rate caused significant amount of processing deblocking filter, it is possible to speed up the process. It is also possible to flexibly set the parallelism and order of processing of the deblocking filter according to various conditions such as size limitation or implementations of the image.
[0254]
　Further, according to the first embodiment, when the necessity of determining the filtering of one of the vertical boundary and the horizontal boundary, over a plurality of macroblocks in the image, the pixel values ​​of the input image to the deblocking filter is referred to that. Thus, treatment of dependencies between macroblocks (or inter-coding unit) is reduced. Thus, a plurality of macro-blocks (in the case of most parallelism is increased, all macroblocks in the image) it is possible to parallelize the wide filtering necessity determining process.
[0255]
　Further, according to the second embodiment, the determination whether or not the filtering in the vertical boundary and the horizontal boundary of each block, waiting for the application of the deblocking filter to the other blocks in the macro block to which the block belongs not be executed. Thus, treatment of dependencies between the vertical boundary and the horizontal boundary in the macro block (or encoded within the unit) is reduced. Thus, in a macro block, it is possible to parallelize the filtering necessity determination process in the vertical boundary and the horizontal boundary.
[0256]
　Further, according to the third embodiment, in both of the filtering process in the vertical boundary and the horizontal boundary, the input pixel to the deblocking filter is filtering. According to such a configuration, it is possible to parallelize each other a filtering process in the vertical boundary and the horizontal boundary. Thereby, it is possible to further speed up the process in the deblocking filter. Further, for the pixel to be updated by both of the two filtering processes are executed in parallel, the output pixel value based on the two filter outputs are computed. Thus, when allowed to parallelize the above two filtering process also can reduce the block distortion appearing in both the vertical boundary and the horizontal boundary properly. Further, the output pixel value may be calculated as a weighted average of the two filter outputs. Thereby enhancing the effect of the removal of the block distortion by the deblocking filter, it can be further improved image quality.
[0257]
　In this specification, mainly but filtering in the vertical boundary is described as an example which is performed before the filtering process in the horizontal boundary, the present disclosure when the filtering process in the horizontal boundary is performed first effect described above according to the techniques according to the may be equally enjoyed. The size of the size or macroblock processing unit of the deblocking filter is not limited to the examples described herein, it may be other sizes. Further, as one approach, after omitting the filtering necessity determination process itself, it is possible to parallelize the application of the deblocking filter to the plurality of vertical boundary and a plurality of horizontal boundary.
[0258]
　Further, the method of transmitting to the decoding side information used for parallel processing of the deblocking filter from the encoding side is not limited to the method of multiplexing the header of the encoded stream of these information. For example, such information may not be multiplexed to the encoded bit stream may be transmitted or recorded as separate data associated with the coded bit stream. Here, the term "associated" means (such as a slice or block may be a part of the image) the image included in the bit stream and be adapted to be be linked at the time of decoding the information corresponding to the image means. That is, information may be transmitted in a different transmission path from the image (or bit stream). Further, the information image (or bit stream) may be recorded in a (separate recording areas or the same recording medium) another recording medium to the. Furthermore, the information and the image (or bit stream), for example, a plurality of frames may be associated with each other in arbitrary units, such as a portion of one frame, or frame.
[0259]
　Having described in detail preferred embodiments of the present disclosure with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such an example. It would be appreciated by those skilled in the art of the techniques of this disclosure, within the scope of the technical idea described in the claims, it is intended to cover various modifications, combinations, for even such modifications are intended to fall within the technical scope of the present disclosure.
[0260]
　In this specification, "horizontal filtering" the filtering process in the vertical boundary, and a filtering process in the horizontal boundary is expressed as "vertical filtering". In general, a filter tap filtering in the vertical boundaries are aligned along the horizontal direction, a filter tap filtering in the horizontal boundary arranged along the vertical direction. Therefore, designation of the filtering process described above is employed.
DESCRIPTION OF SYMBOLS
[0261]
　10, 60 image processing apparatus
　112, 212 the first determination unit (vertical boundary determining
　unit) 116, 216 second determination section (horizontal boundary determining
　unit) 132 horizontal filtering section
　142 vertical filtering unit
　150 parallelization control unit

Claims

[1][Corrected]　
　a decoding unit for generating an image by decoding the encoded stream,
　and the horizontal filtering unit applying a deblocking filter to the adjacent block adjacent to the vertical block boundaries in an image that is by re-generate the decoding section ,
　as a unit of parallel processing a plurality of vertical block boundaries between a plurality of blocks, and a control unit for parallel filtering neighboring blocks adjacent to each of the plurality of vertical block boundary to the horizontal filtering unit
　image comprising a processing apparatus.
[2]
[Corrected]　
　wherein, the plurality of the horizontal fill evening to set the ring portion vertical block boundary as a unit of parallel processing, the image processing apparatus according to claim 1.
[3]
[Corrected]　
　wherein, the filtering for each vertical block boundary, without depending on the filtering results for the other vertical block boundaries, is performed in the horizontal filtering unit, the image processing according to claim 2 apparatus.
[4]
[Corrected]　
　wherein, as the block size of each block, thereby using the block size dependency does not occur between the filtering for different vertical block boundary to one another in the horizontal filtering unit, an image according to claim 3 processing apparatus.
[5]
[Corrected]　
　The image processing apparatus includes a vertical filtering unit applying a deblocking filter to the adjacent block adjacent to the horizontal block boundary in the image generated by the decoding unit, further comprising a
　said control unit includes a plurality of as the unit of parallel processing a plurality of horizontal block boundary between blocks, parallel to filter the neighbor blocks adjacent to each of the plurality of horizontal block border to the vertical filtering unit,
　an image processing apparatus according to claim 4 .
[6]
[Corrected]　　
　the control unit, wherein is set to the vertical filtering unit a plurality of the horizontal block boundary as a unit of parallel processing, the image processing apparatus according to claim 5.
[7]
[Corrected]　
　wherein, the filtering for each horizontal block boundary, without depending on the filtering results for the other horizontal block boundary, to be executed by the vertical filtering unit, the image processing according to claim 6 apparatus.
[8]
[Corrected]　
　wherein, as the block size of each block, thereby using the block size dependency does not occur between the filtering for different horizontal block border each other in the vertical filtering unit, an image according to claim 7 processing apparatus.
[9]
[Corrected]　
　wherein, said horizontal filtering unit 8x8 pixels as the block size and is used in the vertical filtering unit, an image processing apparatus according to claim 8.
[10]
[Corrected]　
　the control unit, a plurality of vertical block boundaries after parallel to filtering the horizontal filtering section, parallel to filter the plurality of horizontal block border to the vertical filtering unit, according to claim 9 the image processing apparatus according to.
[11]
[Corrected]　
　The coded stream is hierarchically are coded in units blocked,
　the decoding unit decodes the encoded stream in the units hierarchically blocked,
　claim the image processing apparatus according to 10.
[12]
[Corrected]　
　as the unit of parallel processing a plurality of vertical block boundaries between a plurality of blocks, parallel determination whether to apply the deblocking filter to the adjacent block adjacent to each of the plurality of vertical block boundary parts, further comprising a
　said control unit, in response to said determination result by the determination unit, to perform filtering on the horizontal filtering unit,
　an image processing apparatus according to claim 11.
[13]
[Corrected]　
　The determination unit, parallel as a unit of parallel processing a plurality of horizontal block boundary between a plurality of blocks, whether to apply a deblocking filter to the adjacent block adjacent to each of the plurality of horizontal block border determined, the manner
　the control unit, in response to said determination result by the determination unit, to perform filtering on the vertical filtering unit,
　an image processing apparatus according to claim 12.
[14]
[Corrected]　
　and that by decoding the encoded stream to generate an image,
　and performing horizontal filtering of the deblocking filter to the adjacent block adjacent to vertical block boundaries in an image generated,
　a plurality of blocks as the unit of parallel processing a plurality of vertical block boundaries between, said such that a plurality of adjacent blocks adjacent to each of the vertical block boundary are filtered in parallel, and controlling said horizontal filtering
　image comprising Processing method.
[15]
[Corrected]　
　and horizontal filtering unit applying a deblocking filter to the adjacent block adjacent to the vertical block boundaries in an image to be locally decoded when encoding the encoding target image,
　a plurality of vertical between a plurality of blocks as the unit of parallel processing block boundaries, a control unit for parallel filtering neighboring blocks adjacent to each of the plurality of vertical block boundary to the horizontal filtering unit,
　using the filtered image by the horizontal filtering unit, an encoding unit for encoding the encoding target image
　the image processing apparatus comprising a.
[16]
[Corrected]　
　wherein, the plurality of the to set the horizontal filtering unit vertical block boundary as a unit of parallel processing, the image processing apparatus according to claim 15.
[17]
[Corrected]　
　wherein, the filtering for each vertical block boundary, without depending on the filtering results for the other vertical block boundaries, is performed in the horizontal filtering unit, the image processing according to claim 16 apparatus.
[18]
[Corrected]　
　wherein, as the block size of each block, thereby using the block size dependency does not occur between the filtering for different vertical block boundary to one another in the horizontal filtering unit, an image according to claim 17 processing apparatus.
[19]
[Corrected]　
　The image processing apparatus includes a vertical filtering unit applying a deblocking filter to the adjacent block adjacent to the horizontal block boundary in the image, further comprising a
　said control unit includes a plurality of between the plurality of blocks as the unit of parallel processing horizontal block boundary, parallel to filter the neighbor blocks adjacent to each of the plurality of horizontal block border to the vertical filtering unit,
　an image processing apparatus according to claim 18.
[20]
[Corrected]　
　the control unit, wherein is set to the vertical filtering unit a plurality of the horizontal block boundary as a unit of parallel processing, the image processing apparatus according to claim 19.
[21]
Add　
　the control unit, the filtering for each horizontal block boundary, without depending on the filtering results for the other horizontal block boundary, to be executed by the vertical filtering unit, an image processing apparatus according to claim 20 .
[22]
Add　
　the control unit, as the block size of each block, thereby using the block size dependency does not occur between the filtering for different horizontal block border each other in the vertical filtering unit, the image processing according to claim 21 apparatus.
[23]
Add　
　the control unit, said 8x8 pixels as the block size is used in the horizontal filtering unit and the vertical filtering unit, an image processing apparatus according to claim 22.
[24]
Add　
　the control unit, a plurality of vertical block boundaries after parallel to filtering the horizontal filtering unit, wherein the parallel to filter a plurality of horizontal block boundary to the vertical filtering unit, to claim 23 the image processing apparatus according.
[25]
Add　
　the encoding unit hierarchically the encoding target image in units blocked by encoding
　the encoding target image is locally decoded in the unit that is hierarchically blocked,
　wherein the image processing apparatus according to claim 24.
[26]
Add　
　as a unit of parallel processing a plurality of vertical block boundaries between a plurality of blocks, parallel determination section for determining to apply the deblocking filter to the adjacent block adjacent to each of the plurality of vertical block boundary , further comprising a
　said control unit, in response to said determination result by the determination unit, to perform filtering on the horizontal filtering unit,
　an image processing apparatus according to claim 25.
[27]
Add　
　the determining unit, as the unit of parallel processing a plurality of horizontal block boundary between a plurality of blocks, parallel to either apply the deblocking filter to the adjacent block adjacent to each of the plurality of horizontal block border determined, the
　said control unit, in response to said determination result by the determination unit, to perform filtering on the vertical filtering unit,
　an image processing apparatus according to claim 26.
[28]
Add　
　and performing horizontal filtering of the deblocking filter to the adjacent block adjacent to the vertical block boundaries in an image to be locally decoded when encoding the encoding target image,
　a plurality of between the plurality of blocks as a unit of parallel processing vertical block boundary, adjacent blocks adjacent to each of the plurality of vertical block boundary to be filtered in parallel, and controlling said horizontal filtering,
　filtered by the horizontal filtering using an image, and to encode the encoding target image
　image processing method comprising.