The present disclosure relates to an image processing apparatus and an image processing method.
Background technology
[0002]
　In H.265 / HEVC, which is one of the standard specifications of the video coding method, a deblock filter is applied to the block boundary of the decoded image in order to suppress the deterioration of image quality due to the block distortion that occurs during coding. Will be done. In H.265 / HEVC, there are two types of deblocking filters that can be applied to the luminance component, a weak filter and a strong filter, while only one type of deblocking filter that can be applied to the color difference component is a weak filter.
[0003]
　In addition, JVET (Joint Video Experts Team), a joint standardization organization of ITU-T and ISO / IEC, is currently working on the next generation with the aim of further improving coding efficiency over H.265 / HEVC. Standardization work of FVC (Future Video Coding), which is a video coding method, is underway (see, for example, Non-Patent Document 1).
[0004]
　In the FVC standardization work, in Non-Patent Document 2 below, the deblocking filter that can be applied to the color difference component is changed to two types like the deblocking filter that can be applied to the luminance component, and the strong filter is also applied to the color difference component. Has been proposed as a method to which.
Prior art literature
Non-patent literature
[0005]
Non-Patent Document 1: J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, “Algorithm Description of Joint Exploration Test Model (JEM7)”, JVET-G1001, Joint Video Exploration Team (JVET) ) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11 7th Meeting: Torino, IT, 13-21 July 2017
Non-Patent Document 2: Seung-Hwan Kim, Jie Zhao, Misra Kiran and Andrew Segall, “Improvement of chroma deblocking filter”, JVET-D0108, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11 4th Meeting: Chengdu, CN, 15- 21 October 2016
Outline of the invention
Problems to be solved by the invention
[0006]
　Whether or not the above-mentioned deblock filter is applied is determined by using the boundary strength of the block boundary, and the boundary strength of the block boundary is calculated based on the luminance component information without using the color difference component information. It was done based on. However, when the necessity of applying the deblock filter to the color difference component of the decoded image is determined by using the boundary intensity specified based on the information of the luminance component, the deblock filter cannot be applied properly and the block distortion remains. There was a risk that it would end up.
[0007]
　Therefore, the present disclosure proposes a mechanism capable of applying a deblocking filter more appropriately to the color difference component of the decoded image.
Means to solve problems
[0008]
　According to the present disclosure, a decoding unit that decodes a coded stream to generate a decoded image and a block boundary of the decoded image are targeted and based on a boundary strength calculated using color difference-related parameters related to color difference. Based on the determination unit that determines whether or not the deblock filter is applied to the color difference component of the decoded image and the determination result of whether or not the deblock filter is applied, the color difference component of the pixel located near the block boundary is deblocked. An image processing apparatus comprising a filtering unit for applying a filter is provided.
[0009]
　Further, according to the present disclosure, it is based on decoding a coded stream to generate a decoded image and targeting a block boundary of the decoded image and calculating a boundary strength using color difference-related parameters related to color difference. Then, based on the determination of whether or not the deblock filter is applied to the color difference component of the decoded image and the result of determining whether or not the deblock filter is applied, the color difference component of the pixel located near the block boundary is deblocked. An image processing method including applying a filter is provided.
[0010]
　Further, according to the present disclosure, a deblock filter for the color difference component of the decoded image is targeted based on the boundary strength calculated by using the color difference related parameters related to the color difference, targeting the block boundary of the locally decoded decoded image. A determination unit that determines the necessity of application, a filtering unit that applies a deblock filter to the color difference component of pixels located near the block boundary based on the determination result of the necessity of applying the deblock filter, and the filtering unit. An image processing apparatus including a coding unit that encodes an image using the decoded image to which the deblocking filter is applied.
[0011]
　Further, according to the present disclosure, a deblock filter for the color difference component of the decoded image is targeted based on the boundary strength calculated by using the color difference related parameters related to the color difference, targeting the block boundary of the locally decoded decoded image. Based on the determination of the necessity of application, the determination result of the necessity of application of the deblock filter, the application of the deblock filter to the color difference component of the pixel located near the block boundary, and the deblock filter An image processing method including encoding an image with the applied decoded image is provided.
The invention's effect
[0012]
　As described above, according to the present disclosure, it is possible to apply the deblock filter more appropriately to the color difference component of the decoded image.
[0013]
　It should be noted that the above effects are not necessarily limited, and together with or in place of the above effects, any of the effects shown herein, or any other effect that can be grasped from this specification. May be played.
A brief description of the drawing
[0014]
FIG. 1 is a table for explaining the calculation of bS in HEVC.
FIG. 2 is a table for explaining the calculation of bS in Non-Patent Document 2.
FIG. 3 is an explanatory diagram showing an example of pixels of color difference components (U component and V component) in two blocks Bp and block Bq adjacent to each other with a vertical block boundary BB in between.
FIG. 4 is a table for explaining the calculation of bS in one embodiment of the present disclosure.
FIG. 5 is a block diagram showing an example of a configuration of an image coding device 10 which is an aspect of the image processing device according to the same embodiment.
FIG. 6 is a block diagram showing an example of a configuration of an image decoding device 60, which is an aspect of the image processing device according to the same embodiment.
FIG. 7 is a block diagram showing an example of a detailed configuration of the deblock filter 26 according to the same embodiment.
FIG. 8 is a table showing an example of bS calculated by the boundary strength calculation unit 261.
FIG. 9 is a flowchart showing an example of a processing flow by the deblock filter 26 according to the same embodiment.
FIG. 10 is a flowchart for explaining a flow of a boundary strength calculation process executed by the boundary strength calculation unit 261.
FIG. 11 is a table showing a modified example of bS calculated by the boundary strength calculation unit 261.
FIG. 12A is the first half of a flowchart for explaining an example of the flow of boundary strength calculation processing corresponding to the modification of FIG. 11.
FIG. 12B is the latter half of the flowchart for explaining an example of the flow of the boundary strength calculation process corresponding to the modification of FIG. 11.
[Fig. 13] Fig. 13 is an explanatory diagram showing an example of a hardware configuration.
FIG. 14 is a table showing a modification of the calculation of bS.
Forms for carrying out the invention
[0015]
　Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.
[0016]
　Further, the scope disclosed in the present specification is not limited to the contents of the examples, and the contents of the following references REF1 to REF3 known at the time of filing are also incorporated in the present specification by reference. Is done. In other words, the contents described in the following references REF1 to REF3 are also the basis for judging the support requirements. For example, even if the Quad-Tree Block Structure described in Reference REF2 and the QTBT (Quad Tree Plus Binary Tree) Block Structure described in Reference REF3 are not directly defined in the detailed description of the invention. , Within the scope of this disclosure and shall meet the support requirements of the claims. Similarly, technical terms such as Parsing, Syntax, and Semantics are also within the scope of the present disclosure, even if they are not directly defined in the detailed description of the invention. Yes, it shall meet the support requirements of the claims.
　REF1: Recommendation ITU-T H.264 (04/2017) “Advanced video coding for generic audiovisual services”, April 2017
　REF2: Recommendation ITU-T H.265, (12/2016) “High efficiency video coding”, December 2016
　REF3: J. Chen, E. Alshina, GJ Sullivan, J.-R. Ohm, J. Boyce, "Algorithm Description of Joint Exploration Test Model (JEM7)", JVET-G1001, Joint Video Exploration Team (JVET) of ITU －T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29/WG 11 7th Meeting: Torino, IT, 13-21 July 2017
[0017]
　Further, in the following, unless otherwise specified, a signal in the YUV420 format will be described as an example, and the luminance component may be represented as the Y component, the color difference component may be represented as the U component, and the V component. However, the technique described below can be similarly applied to signals in other formats such as the YUV444 format and the YUV422 format. Further, the representation of the luminance component and the color difference component differs depending on the target signal. For example, the technique described below is similarly applied to a signal in which the luminance component and the color difference component are represented by YCbCr. It is possible to do.
[0018]
　In addition, the following terms used in the present specification are defined as follows.
　The color difference-related parameters mean all the parameters related to the color difference. For example, the color difference-related parameters include the conversion coefficient of the color difference component included in each TU (Transform Unit) and the conversion coefficient of the color difference component such as a flag indicating the presence or absence of the significance coefficient (non-zero conversion coefficient) of the color difference component in each TU. May contain information about. However, the color difference-related parameters are not limited to these examples, and may be various parameters related to the color difference.
　The necessity of applying the deblock filter means whether or not the deblock filter should be applied. For example, determining whether or not a deblocking filter should be applied means determining whether or not a deblocking filter should be applied. Further, the determination result of whether or not the deblock filter is applied is the result of determining whether or not the deblock filter should be applied. For example, the determination result should be applied or should not be applied. It can be information indicating.
　The large block determination means a determination as to whether or not the block to be determined is a large block. In the present specification, the block to be determined may be a block sandwiching a block boundary, as will be described later. Further, the large block determination can be performed by comparing the block size (block size) with a predetermined threshold value. The case where the large block determination is performed and the details of the large block determination will be described later.
[0019]
　The explanations will be given in the following order.
　　1. 1. Outline
　　　1-1. Existing deblock filter
　　　1-2. Outline of the technology related to the present disclosure
　　2. Outline of the device
　　　2-1. Image coding device
　　　2-2. Image decoding device
　　3. Deblock filter
　　　3-1. Configuration example
　　　3-2. Process flow
　　　3-3. Modification example
　　4. Hardware configuration example
　　5. Conclusion
[0020]
　<1. Overview>
　　[1-1. Existing method]
[0021]
　The processing related to the deblock filter in the existing image coding method such as HEVC includes the application necessity determination processing, the filter strength determination processing, and the filtering processing (filter application processing). In the following, the processing applied to the existing deblocking filter will be described using the HEVC deblocking filter as an example. In the following, the deblocking filter for the color difference component of the decoded image (including the image locally decoded at the time of encoding) will be mainly described, and the description of the deblocking filter for the luminance component of the decoded image will be omitted as appropriate.
[0022]
　As the process applied to the deblock filter, the application necessity determination process is first performed. The application necessity determination process is a process of determining whether or not a deblock filter should be applied to the block boundary (Block Boundary) of the decoded image. In HEVC, the block boundary is specified based on the block structure of Quad-Tree Block Structure described in Reference REF2. Specifically, the condition that the edge of the 8 × 8 pixel block (sample grid), which is the smallest block unit, is at least one of the TU (Transform Unit) boundary and the PU (Prediction Unit) boundary is satisfied. Edges are identified as block boundaries in HEVC.
[0023]
　The application necessity determination process is performed based on the boundary strength of the block boundary (Boundary Strength: hereinafter may be referred to as bS). In HEVC, bS is calculated every 4 lines of the specified block boundary. When the block boundary is a vertical boundary, the above line corresponds to a line orthogonal to the vertical boundary. When the block boundary is a horizontal boundary, the line corresponds to a column orthogonal to the horizontal boundary.
[0024]
　FIG. 1 is a table for explaining the calculation of bS in HEVC. As shown in FIG. 1, in HEVC, bS is true of condition A which is a condition related to intra prediction, condition B1 which is a condition related to the significance coefficient of the Y component, and condition B2 which is a condition related to a motion vector (MV) and a reference picture. Calculated based on falsehood (satisfied or unsatisfied). Referring to FIG. 1, bS is set to 2 when condition A is true. Further, bS is set to 1 when the condition A is false and at least one of the condition B1 and the condition B2 is true. Then, when the condition A, the condition B1, and the condition B2 are all false, bS is set to 0. The conditions A, B1 and B2 shown in FIG. 1 are as follows.
[0025]
　-Condition A: At least one of the CUs (Coding Units) including the pixels of the uppermost line of the bS calculation target lines and sandwiching the block boundary is the intra prediction mode.-
　Condition B1: Block The boundary is the TU boundary, and at least one of the two TUs including the pixel of the uppermost line of the bS calculation target lines and sandwiching the block boundary has a significance coefficient of the Y component-
　condition B2: bS. The absolute value of the difference in MV is 1 pixel or more, or the reference picture of motion compensation is different, or the number of MVs between two CUs including the pixel of the top line of the calculation target lines and sandwiching the block boundary. Is different
[0026]
　Further, in HEVC, a deblocking filter for the luminance component (Y component) of the decoded image can be applied to the block boundary set as described above in which the bS is 1 or more. Therefore, in HEVC, the determination result of whether or not the deblock filter should be applied to the luminance component of the decoded image may differ depending on whether or not the conditions B1 and B2 are satisfied.
[0027]
　In HEVC, as a deblocking filter for the luminance component of the decoded image, a strong filter having a large filter strength and a weak filter having a low filter strength are prepared. When bS is 1 or more, the process relating to the deblocking filter for the luminance component of the decoded image is followed by a filter intensity determination process and a filtering process after further application necessity determination process based on further conditions is performed. Details of these processes are described in the above-mentioned reference REF2, and the description thereof is omitted here.
[0028]
　On the other hand, the deblock filter for the color difference components (U component, V component) of the decoded image in HEVC is applied only to the block boundary where bS is 2. Therefore, as shown in FIG. 1, whether or not the conditions B1 and B2 are satisfied does not affect the determination of whether or not the deblock filter is applied to the color difference component of the decoded image in HEVC.
[0029]
　Further, in HEVC, the only deblocking filter that can be applied to the color difference component of the decoded image is the weak filter. Therefore, it is not necessary to determine the filter intensity for the color difference component of the decoded image, and when bS is 2, a weak filter is applied to the color difference component of the decoded image.
[0030]
　By the way, as described in the above-mentioned reference REF3, in the block division by the QTBT Block Structure in FVC, a block having a larger size than the block division by the Quad-Tree Block Structure in HEVC can be selected. When the block size is large in a flat region (a region in which the change in pixel value in the region is small), block distortion is likely to occur. Therefore, in FVC where a block of a larger size can be selected, if the only deblock filter that can be applied to the color difference component of the decoded image is the weak filter as in HEVC, significant block distortion remains in the color difference component. There was a risk that it would end up. In view of such a situation, it is desired to improve the deblocking filter for the color difference component of the decoded image.
[0031]
　For example, in Non-Patent Document 2, the deblocking filter that can be applied to the color difference component can be changed to two types like the deblocking filter that can be applied to the luminance component, and the strong filter can be applied to the color difference component as well. A method has been proposed. Further, Non-Patent Document 2 describes that the deblock filter can be applied to the color difference component of the decoded image not only when the bS is 2 but also when the bS is 1.
[0032]
　FIG. 2 is a table for explaining the calculation of bS in Non-Patent Document 2. As shown in FIG. 2, in Non-Patent Document 2, bS is calculated based on the above-mentioned conditions A, B1 and B2 as in the example of HEVC shown in FIG. However, as described above, in Non-Patent Document 2, the deblock filter can be applied to the color difference component of the decoded image not only when the bS is 2 but also when the bS is 1. Therefore, as shown in FIG. 2, in Non-Patent Document 2, it is necessary to apply the deblock filter to the color difference component (U component, V component) of the decoded image depending on whether the condition B1 and the condition B2 are satisfied. Judgment result may be different.
[0033]
　Hereinafter, the application necessity determination process, the filter strength determination process, and the filtering process regarding the deblock filter that can be applied to the color difference component of the decoded image in Non-Patent Document 2 will be described with reference to FIG. FIG. 3 is an explanatory diagram showing an example of pixels of color difference components (U component and V component) in two blocks Bp and block Bq adjacent to each other with a vertical block boundary BB in between. Although the vertical boundary will be described here as an example, the matters described here can be applied to the horizontal boundary as well. Further, although FIG. 3 shows an example in which the block Bp and the block Bq are 4 × 4 in the color difference component, the matters described here can be similarly applied to blocks of other sizes.
[0034]
　In the example of FIG. 3, the pixels of the color difference component in the block Bp are indicated by the symbols p i and j . i is the column index and j is the row index. The column indexes i are numbered 0, 1, 2, and 3 in order from the column closest to the block boundary BB (from left to right in the figure). The row index j is numbered 0, 1, 2, 3 from top to bottom. On the other hand, the pixels of the color difference component in the block Bq are indicated by the symbols q k and j . k is the column index and j is the row index. The column indexes k are numbered 0, 1, 2, and 3 in order from the column closest to the block boundary BB (from right to left in the figure).
[0035]
　After the bS is calculated as described with reference to FIG. 2, the application necessity determination process and the filter strength determination process are performed using the following three conditions. In the case of the YUV420 format, such processing is performed every two lines in the color difference component. For example, in the example shown in FIG. 3, the determination regarding the line L11 and the line L12 and the determination regarding the line L21 and the line L22 are performed separately. The determination for each line is performed using the pixels of the line to be determined. Hereinafter, the application necessity determination process, the filter strength determination process, and the filtering process will be described by taking the line L11 and the line L12 as examples.
[0036]
　First, in the application necessity determination process, it is determined in order whether or not the following condition C91 and condition C92 are true.
[0037]
　-Condition C91: (bS == 2 || bS == 1 && (block_width> 16 && block_height> 16)))
　-Condition C92: d 16 && block_height> 16)
[0045]
　The block_width and block_height in the above condition C93 are the horizontal size and the vertical size of the block related to the block boundary to be determined, respectively, like the block_width and the block_height in the condition C91.
[0046]
　When the condition C93 is true, a strong filter is applied to the color difference component of the decoded image at the target block boundary, and when the condition C93 is false, the color difference of the decoded image is applied at the target block boundary. A weak filter is applied to the ingredients.
[0047]
　The strong filter applied to the color difference component in Non-Patent Document 2 is the same as the strong filter applied to the luminance component in HEVC, and is represented by the following equations (8) to (13).
[0048]
　p 0 ′ = Clip3 (p 0 -2 * tc, p 0 + 2 * t C , (p 2 + 2 * p 1 + 2 * p 0 + 2 * q 0 + q 1 +4) >> 3)… (8)
　p 1 ′ = Clip3 (p 1 -2 * tc, p 1 + 2 * t C , (p 2 + p 1 + p 0 + q 0 +2) >> 2)… (9)
　p 2 ′ = Clip3 (p 2 -2 * tc, p 2 + 2 * t C, (2 * p 3 + 3 * p 2 + p 1 + p 0 + q 0 +4) >> 3)… (10)
　q 0 ′ = Clip3 (q 0 -2 * tc, q 0 + 2 * t C , (p 1 + 2p 0 + 2q 0 + 2q 1 + q 2 +4) >> 3)… (11)
　q 1 ′ = Clip3 (q 1 -2 * tc, q 1 + 2 * t C , ( p 0 + q 0 + q 1 + q 2 +2) >> 2)… (12)
　q 2 ′ = Clip3 (q 2 -2 * t c , q 2 +2 * t C , (p 0 + q 0 + q 1 + 3 * q 2 +2 * q 3 +4) >> 3)… (13)
[0049]
　In the above equations (8) to (13), p i and q k are pixel values ​​of the color difference component before applying the deblock filter. Further, p i ′ and q k ′ are pixel values ​​of the color difference component after applying the deblock filter. Here, i and k are the indexes of the columns in the blocks Bp and Bq described above, respectively, and the row indexes are omitted in the equations (8) to (13), respectively. In addition, t C is a parameter given according to the quantization parameter. Further, Clip3 (a, b, c) represents a clipping process in which the value c is clipped in the range of a ≦ c ≦ b.
[0050]
　Since the weak filter applied to the color difference component in Non-Patent Document 2 is the same as the weak filter applied to the color difference component in HEVC, the description here is omitted.
[0051]
　The process relating to the deblocking filter that can be applied to the color difference component of the decoded image has been described above in Non-Patent Document 2. According to the above-mentioned method, it is possible to apply the strong filter not only to the luminance component but also to the color difference component depending on the conditions.
[0052]
　However, as explained with reference to FIG. 2, the condition B1 used in the calculation of bS in Non-Patent Document 2 depends on the presence or absence of the significance coefficient of the luminance component (Y component) as in the case of HEVC. Even if other conditions are included, the information on the color difference component (U component, V component) is not used. However, the spatial pattern of the luminance component and the spatial pattern of each of the color difference components do not always match. Therefore, when the necessity of applying the deblock filter for the color difference component is determined according to the condition based on the information of the luminance component, the deblock filter is not properly applied even though the block distortion occurs, and the block is blocked. There was a risk that distortion would remain.
[0053]
　Further, when bS is 1, in order for the condition C91 used for the application necessity determination process in Non-Patent Document 2 to be true, the size in the horizontal direction and the vertical direction of the block relating to the block boundary to be determined are in the vertical direction. Both sizes must be greater than 16. However, as described in reference REF3, the shape of the block (eg, CU) in FVC can be a non-square rectangle as well as a square. Then, the block distortion tends to occur more easily depending on the size in the direction orthogonal to the block boundary than the size in the same direction as the block boundary. Therefore, depending on the shape of the block, the deblock filter may not be properly applied in the application necessity determination process of Non-Patent Document 2, and the block distortion may remain.
[0054]
　Further, the strong filter in Non-Patent Document 2 is the same as the strong filter applied in HEVC. On the other hand, as described above, in FVC, a block having a size larger than that in HEVC can be selected. Therefore, even if the strong filter in Non-Patent Document 2 is applied, the block distortion may not be sufficiently reduced. there were.
[0055]
　　[1-2. Outline of one embodiment of the present disclosure]
　Therefore, one embodiment of the present disclosure has been created with the above circumstances as the first point of view. The image processing apparatus according to the embodiment of the present disclosure needs to apply a deblock filter to the color difference component of the decoded image based on the boundary intensity (bS) calculated using the color difference related parameters related to the color difference of the decoded image. Performs application necessity judgment processing. Hereinafter, an outline of one embodiment of the present disclosure will be described.
[0056]
　FIG. 4 is a table for explaining the calculation of bS in the present embodiment. As shown in FIG. 4, regarding the significance coefficients of the conditions B1-Y and U components, which are the conditions relating to the significance coefficients of the A and Y components, which are the conditions relating to the intra prediction. It is calculated based on the condition B1-V, which is a condition, and the condition B2, which is a condition related to the MV and the reference picture.
[0057]
　Referring to FIG. 4, bS is set to 16 when condition A is true. If the condition A is false and the condition B2 is true, bS is set to 1. Then, when the condition A and the condition B2 are false and any one of the condition B1-Y, the condition B1-U, and the condition B1-V is true, bS is a value between 2 and 14. Is set to. Then, when the conditions A, B1-Y, B1-U, B1-V, and B2 are all false, bS is set to 0. The conditions A, B1-Y, and B2 shown in FIG. 4 are the same as the conditions A, B1, and B2 described with reference to FIG. 1, respectively. Further, the method of calculating bS according to the present embodiment will be described in more detail later.
[0058]
　Further, in the conditions B1-U and B1-V shown in FIG. 4, instead of the presence / absence of the significance coefficient of the Y component in the condition B1-Y, the presence / absence of the significance coefficient of the U component and the significance coefficient of the V component, respectively, It corresponds to the condition that the presence or absence is used for the judgment, and is expressed as follows. The authenticity of the following conditions B1-U and B1-V can be determined based on a flag (an example of a color difference-related parameter) indicating the presence or absence of a significance coefficient of the color difference component in each TU.
[0059]
　-Condition B1-U: The block boundary is the TU boundary, and at least one of the two TUs including the pixel of the uppermost line of the bS calculation target lines and sandwiching the block boundary has the significance coefficient of the U component. Existence
　-Condition B1-V: The block boundary is the TU boundary, and the significance of the V component in at least one of the two TUs including the pixel of the uppermost line of the lines to be calculated for bS and sandwiching the block boundary. There is a coefficient
[0060]
　In the present embodiment, it is determined whether or not the deblock filter is applied to the color difference component of the decoded image based on the bS calculated by using the conditions B1-U and B1-V related to the color difference as described above. .. With such a configuration, it becomes possible to apply the deblock filter more appropriately to the color difference component.
[0061]
　Further, in the present embodiment, as will be described later, it is determined whether or not the deblock filter is applied to the color difference component of the decoded image based on the size in the direction orthogonal to the block boundary. With such a configuration, the deblocking filter can be applied more appropriately even when the shape of the block is a non-square rectangle.
[0062]
　Further, in the present embodiment, as will be described later, a strong filter having a higher intensity (stronger low-pass characteristic) than the strong filter in Non-Patent Document 2 can be applied to the color difference component of the decoded image. Further, in order to apply such a strong filter more appropriately, in the present embodiment, the filter strength is determined by a method different from the filter strength determination process in Non-Patent Document 2. With such a configuration, block distortion can be further reduced.
[0063]
　The outline of one embodiment of the present disclosure has been described above. Hereinafter, the configuration and operation of the present embodiment for realizing the above-mentioned effects will be described in detail in order.
[0064]
　<2. Schematic configuration of the device>
　First, the schematic configuration of the device as an example to which the technique disclosed in the present specification can be applied will be described with reference to FIGS. 5 and 6. The techniques disclosed herein are applicable, for example, to image coding and decoding devices.
[0065]
　[2-1. Image Coding Device]
　FIG. 5 is a block diagram showing an example of the configuration of the image coding device 10 which is one aspect of the image processing device according to the embodiment of the present disclosure.
[0066]
　Referring to FIG. 5, the image coding apparatus 10 includes a sorting buffer 11, a control unit 12, a subtraction unit 13, an orthogonal transform unit 14, a quantization unit 15, a lossless coding unit 16, a storage buffer 17, and an inverse quantization unit. It includes 21, an inverse orthogonal transform unit 22, an addition unit 23, an in-loop filter 24, a frame memory 30, a switch 31, a mode setting unit 32, an intra prediction unit 40, and an inter prediction unit 50.
[0067]
　The sorting buffer 11 sorts a series of images constituting the video to be encoded according to the GOP (Group of Pictures) structure to be encoded. The sorting buffer 11 outputs the sorted images to the control unit 12, the subtraction unit 13, the intra prediction unit 40, and the inter prediction unit 50.
[0068]
　The control unit 12 divides the image into blocks of the processing unit based on the block size of the external or predetermined processing unit. The CU of the Quad-Tree Block Structure or the QTBT (Quad Tree Plus Binary Tree) Block Structure may be formed as a processing unit by the block division by the control unit 12. Further, the control unit 12 determines the parameters related to the coding process based on, for example, RDO (Rate-Distortion Optimization). The determined parameters are supplied to each part.
[0069]
　The subtraction unit 13 calculates a prediction error, which is the difference between the image input from the sorting buffer 11 and the prediction image, and outputs the calculated prediction error to the orthogonal transform unit 14.
[0070]
　The orthogonal transform unit 14 executes the orthogonal transform process for each of the one or more conversion blocks (TUs) set in each region. The orthogonal transform here may be, for example, a discrete cosine transform or a discrete sine transform. More specifically, the orthogonal transform unit 14 converts the prediction error input from the subtraction unit 13 from the image signal in the spatial domain to the conversion coefficient in the frequency domain for each conversion block. Then, the orthogonal transform unit 14 outputs the conversion coefficient to the quantization unit 15.
[0071]
　Further, the orthogonal transform unit 14 generates a flag indicating the presence / absence of a significance coefficient in each TU for each component (for each Y component, U component, and V component) based on the conversion coefficient obtained by the orthogonal transform, and has a reversible code. It may be output to the conversion unit 16 and the in-loop filter 24. The flag indicating the presence / absence of the significance coefficient of the U component in each TU and the flag indicating the presence / absence of the significance coefficient of the V component in each TU, which are generated by the orthogonal transform unit 14, are included in the color difference-related parameters.
[0072]
　The quantization unit 15 is supplied with a conversion coefficient input from the orthogonal transform unit 14 and a rate control signal from the rate control unit 18 to be described later. The quantization unit 15 quantizes the conversion coefficient and outputs the quantized conversion coefficient (hereinafter, also referred to as quantization data) to the reversible coding unit 16 and the inverse quantization unit 21. Further, the quantization unit 15 changes the bit rate of the quantization data input to the lossless coding unit 16 by switching the quantization scale based on the rate control signal from the rate control unit 18.
[0073]
　The lossless coding unit 16 generates a coded stream by coding the quantization data input from the quantization unit 15. Further, the reversible coding unit 16 encodes various parameters referred to by the decoder and inserts the coding parameters into the coding stream. The parameters encoded by the lossless coding unit 16 may include the parameters determined by the control unit 12 described above.
[0074]
　Further, the parameters encoded by the lossless coding unit 16 may include color difference related parameters. The color difference-related parameters encoded by the reversible coding unit 16 are, for example, a flag indicating the presence or absence of the significance coefficient of the U component in each TU input from the orthogonal transform unit 14 as described above, and the significance of the V component in each TU. Includes a flag indicating the presence or absence of a coefficient. The lossless coding unit 16 outputs the generated coded stream to the storage buffer 17.
[0075]
　The storage buffer 17 temporarily stores the coded stream input from the reversible coding unit 16 using a storage medium such as a semiconductor memory. Then, the storage buffer 17 outputs the stored coded stream to a transmission unit (for example, a communication interface or a connection interface with a peripheral device) (not shown) at a rate corresponding to the band of the transmission line.
[0076]
　The rate control unit 18 monitors the free space of the storage buffer 17. Then, the rate control unit 18 generates a rate control signal according to the free capacity of the storage buffer 17, and outputs the generated rate control signal to the quantization unit 15. For example, the rate control unit 18 generates a rate control signal for lowering the bit rate of the quantized data when the free space of the storage buffer 17 is small. Further, for example, the rate control unit 18 generates a rate control signal for increasing the bit rate of the quantized data when the free capacity of the storage buffer 17 is sufficiently large.
[0077]
　The inverse quantization unit 21, the inverse orthogonal transform unit 22, and the addition unit 23 form a local decoder. The local decoder has a role of locally decoding the decoded image from the encoded data.
[0078]
　The inverse quantization unit 21 dequantizes the quantization data with the same quantization parameters as those used by the quantization unit 15 and restores the conversion coefficient. Then, the inverse quantization unit 21 outputs the restored conversion coefficient to the inverse orthogonal conversion unit 22.
[0079]
　The inverse orthogonal transform unit 22 restores the prediction error by executing the inverse orthogonal transform process on the conversion coefficient input from the inverse quantization unit 21. Then, the inverse orthogonal transform unit 22 outputs the restored prediction error to the addition unit 23.
[0080]
　The addition unit 23 adds the restored prediction error input from the inverse orthogonal transform unit 22 and the prediction image input from the intra prediction unit 40 or the inter prediction unit 50 to obtain a decoded image (reconstruct image). Generate. Then, the addition unit 23 outputs the generated decoded image to the in-loop filter 24 and the frame memory 30.
[0081]
　The in-loop filter 24 applies a series of in-loop filters for the purpose of improving the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" of reference REF3, four in-loop filters are applied in the order of bilateral filter, deblock filter, adaptive offset filter, and adaptive loop filter. You may. The in-loop filter 24 shown in FIG. 5 includes, for example, a bilateral filter 25, a deblocking filter 26a, an adaptive offset filter 27, and an adaptive loop filter 28, and the above four in-loop filters can be applied in order. However, the in-loop filter 24 is not limited to such a configuration, and which of the four in-loop filters is applied and in what order may be appropriately selected. The deblock filter 26a will be described in detail later.
[0082]
　The in-loop filter 24 outputs the decoded image to which the in-loop filter is applied to the frame memory 30.
[0083]
　The frame memory 30 stores the decoded image before filtering input from the addition unit 23 and the decoded image to which the in-loop filter input from the in-loop filter 24 is applied by using the storage medium.
[0084]
　The switch 31 reads the decoded image before filtering used for the intra prediction from the frame memory 30, and supplies the read decoded image as a reference image to the intra prediction unit 40. Further, the switch 31 reads the filtered decoded image used for the inter-prediction from the frame memory 30, and supplies the read decoded image as a reference image to the inter-prediction unit 50.
[0085]
　The mode setting unit 32 sets the prediction coding mode for each block based on the comparison of the costs input from the intra prediction unit 40 and the inter prediction unit 50. The mode setting unit 32 outputs the prediction image generated by the intra prediction unit 40 to the subtraction unit 13 and the addition unit 23 for the block for which the intra prediction mode is set, and outputs the information related to the intra prediction to the reversible coding unit 16. Output. Further, the mode setting unit 32 outputs the prediction image generated by the inter-prediction unit 50 to the subtraction unit 13 and the addition unit 23 for the block for which the inter-prediction mode is set, and outputs the information related to the inter-prediction to the reversible coding unit. Output to 16.
[0086]
　The intra prediction unit 40 executes the intra prediction process based on the original image and the decoded image. For example, the intra prediction unit 40 evaluates the cost based on the prediction error and the amount of code generated for each of the prediction mode candidates included in the search range. Next, the intra prediction unit 40 selects the prediction mode that minimizes the cost as the optimum prediction mode. In addition, the intra prediction unit 40 generates a prediction image according to the selected optimum prediction mode. Then, the intra prediction unit 40 outputs information on the intra prediction including the prediction mode information indicating the optimum prediction mode, the corresponding cost, and the prediction image to the mode setting unit 32.
[0087]
　The inter-prediction unit 50 executes inter-prediction processing (motion compensation) based on the original image and the decoded image. For example, the inter-prediction unit 50 evaluates the cost based on the prediction error and the amount of code generated for each of the prediction mode candidates included in a certain search range. Next, the inter-prediction unit 50 selects the prediction mode with the lowest cost, that is, the prediction mode with the highest compression ratio, as the optimum prediction mode. Further, the inter-prediction unit 50 generates a prediction image according to the selected optimum prediction mode. Then, the inter-prediction unit 50 outputs information about the inter-prediction, the corresponding cost, and the predicted image to the mode setting unit 32.
[0088]
　　[2-2. Image Decoding Device]
　Next, decoding of the data encoded as described above will be described. FIG. 6 is a block diagram showing an example of the configuration of the image decoding device 60, which is one aspect of the image processing device according to the present embodiment. With reference to FIG. 6, the storage buffer 61, the reversible decoding unit 62, the inverse quantization unit 63, the inverse orthogonal transformation unit 64, the addition unit 65, the in-loop filter 66, the sorting buffer 72, and the D / A (Digital to Analogue) conversion. A unit 73, a frame memory 80, selectors 81a and 81b, an intra prediction unit 90, and an inter prediction unit 100 are provided.
[0089]
　The storage buffer 61 temporarily stores a coded stream received from the image coding device 10 via a transmission unit (for example, a communication interface or a connection interface with a peripheral device) (not shown) using a storage medium.
[0090]
　The reversible decoding unit 62 decodes the coded stream input from the storage buffer 61 according to the coding method used at the time of coding to generate quantization data. The lossless decoding unit 62 outputs the generated quantization data to the inverse quantization unit 63.
[0091]
　Further, the reversible decoding unit 62 parses various parameters from the coded stream. The parameters parsed by the reversible decoding unit 62 may include, for example, information about intra-prediction and information about inter-prediction. The reversible decoding unit 62 outputs information regarding the intra prediction to the intra prediction unit 90. Further, the reversible decoding unit 62 outputs information regarding the inter-prediction to the inter-prediction unit 100.
[0092]
　Further, the parameters parsed by the reversible decoding unit 62 may include color difference related parameters. The reversible decoding unit 62 outputs the color difference-related parameters to the in-loop filter 66. The color difference-related parameters parsed by the reversible decoding unit 62 include, for example, a flag indicating the presence / absence of the significance coefficient of the U component in each TU described above, and a flag indicating the presence / absence of the significance coefficient of the V component in each TU.
[0093]
　The inverse quantization unit 63 dequantizes the quantization data input from the reversible decoding unit 62 in the same quantization step as that used at the time of coding, and restores the conversion coefficient. The inverse quantization unit 63 outputs the restored conversion coefficient to the inverse orthogonal conversion unit 64.
[0094]
　The inverse orthogonal transform unit 64 generates a prediction error by performing an inverse orthogonal transform on the conversion coefficient input from the inverse quantization unit 63 according to the orthogonal transform method used at the time of coding. The inverse orthogonal transform unit 64 outputs the generated prediction error to the addition unit 65.
[0095]
　The addition unit 65 generates a decoded image by adding the prediction error input from the inverse orthogonal transform unit 64 and the prediction image input from the selector 71b. Then, the addition unit 65 outputs the generated decoded image to the in-loop filter 66 and the frame memory 80.
[0096]
　The in-loop filter 66 applies a series of in-loop filters for the purpose of improving the image quality of the decoded image. For example, as described in "2.5. In-loop filtering" of reference REF3, four in-loop filters are applied in the order of bilateral filter, deblock filter, adaptive offset filter, and adaptive loop filter. You may. The in-loop filter 66 shown in FIG. 6 includes, for example, a bilateral filter 67, a deblock filter 26b, an adaptive offset filter 69, and an adaptive loop filter 70, and the above four in-loop filters can be applied in order. However, the in-loop filter 66 is not limited to such a configuration, and which of the four in-loop filters is applied and in what order may be appropriately selected. The deblock filter 26b will be described in detail later.
[0097]
　The in-loop filter 66 outputs the decoded image to which the in-loop filter is applied to the sorting buffer 72 and the frame memory 80.
[0098]
　The sorting buffer 72 sorts the images input from the in-loop filter 66 to generate a series of time-series images. Then, the sorting buffer 72 outputs the generated image to the D / A conversion unit 73.
[0099]
　The D / A conversion unit 73 converts the digital format image input from the sorting buffer 72 into an analog format image signal. Then, the D / A conversion unit 73 displays an image by outputting an analog image signal to a display (not shown) connected to the image decoding device 60, for example.
[0100]
　The frame memory 80 stores the decoded image before filtering input from the addition unit 65 and the decoded image to which the in-loop filter input from the in-loop filter 66 is applied by using the storage medium.
[0101]
　The selector 81a switches the output destination of the image from the frame memory 80 between the intra prediction unit 90 and the inter prediction unit 100 for each block in the image according to the prediction mode information acquired by the reversible decoding unit 62. .. For example, when the intra prediction mode is specified, the selector 81a outputs the decoded image before filtering supplied from the frame memory 80 to the intra prediction unit 90 as a reference image. When the inter-prediction mode is specified, the selector 81a outputs the filtered decoded image as a reference image to the inter-prediction unit 100.
[0102]
　The selector 81b switches the output source of the prediction image to be supplied to the addition unit 65 between the intra prediction unit 90 and the inter prediction unit 100 according to the prediction mode information acquired by the reversible decoding unit 62. For example, the selector 81b supplies the prediction image output from the intra prediction unit 90 to the addition unit 65 when the intra prediction mode is specified. Further, when the inter-prediction mode is specified, the selector 81b supplies the prediction image output from the inter-prediction unit 100 to the addition unit 65.
[0103]
　The intra prediction unit 90 performs intra prediction processing based on the information regarding the intra prediction input from the reversible decoding unit 62 and the reference image from the frame memory 80, and generates a prediction image. Then, the intra prediction unit 90 outputs the generated prediction image to the selector 81b.
[0104]
　The inter-prediction unit 100 performs inter-prediction processing based on the information regarding the inter-prediction input from the reversible decoding unit 62 and the reference image from the frame memory 80, and generates a prediction image. Then, the inter prediction unit 100 outputs the generated prediction image to the selector 81b.
[0105]
　<3. Deblock filter>
　　[3-1. Configuration Example of
　Deblock Filter] This section describes an example of the configuration of the deblock filter 26a of the image coding device 10 shown in FIG. 5 and the deblock filter 26b of the image decoding device 60 shown in FIG. The configurations of the deblock filter 26a and the deblock filter 26b may be the same. Therefore, in the following description, the deblocking filter 26a and the deblocking filter 26b are collectively referred to as the deblocking filter 26 when it is not necessary to distinguish between the two.
[0106]
　As described above, the deblock filter 26 according to the present embodiment determines whether or not the deblock filter is applied to the color difference component of the decoded image based on the bS calculated using the color difference-related parameters related to the color difference. Further, as described above, the deblock filter 26 according to the present embodiment determines whether or not the deblock filter is applied to the color difference component of the decoded image based on the size in the direction orthogonal to the block boundary. Further, as described above, in the deblock filter 26 according to the present embodiment, a strong filter having a higher intensity (stronger low-pass characteristic) than the strong filter in Non-Patent Document 2 can be applied to the color difference component of the decoded image. .. Further, in order to apply such a strong filter more appropriately, in the present embodiment, the filter strength is determined by a method different from the filter strength determination process in Non-Patent Document 2. In the following, the function of the deblocking filter 26 relating to the deblocking filter applied mainly to the color difference component of the decoded image will be described, and the function of the deblocking filter 26 relating to the deblocking filter applied to the luminance component will be described. Will be omitted as appropriate.
[0107]
　FIG. 7 is a block diagram showing an example of a detailed configuration of the deblock filter 26 according to the present embodiment. Referring to FIG. 7, the deblock filter 26 has a boundary strength calculation unit 261, a determination unit 263, and a filtering unit 269.
[0108]
　(1) Boundary strength calculation unit The
　boundary strength calculation unit 261 calculates bS (boundary strength) using the color difference-related parameters related to the color difference, targeting the block boundary of the decoded image. When a signal in the YUV420 format is the target, the boundary intensity calculation unit 261 calculates bS in units of 4 lines in the luminance component of the decoded image, that is, in units of 2 lines in the color difference component of the decoded image.
[0109]
　The color difference-related parameters used by the boundary strength calculation unit 261 in the calculation of bS in the present embodiment include a flag indicating the presence / absence of the significance coefficient of the U component in each TU and a flag indicating the presence / absence of the significance coefficient of the V component in each TU. .. As shown in FIG. 7, in the boundary strength calculation unit 261, a flag indicating the presence or absence of the significance coefficient of each component (Y component, U component, V component) in each TU is set in the orthogonal transform unit 14 or the reversible decoding unit 62. Entered from.
[0110]
　The boundary strength calculation unit 261 calculates bS based on the conditions A, B1-Y, B1-U, B1-V, and B2 described with reference to FIG. That is, the boundary strength calculation unit 261 calculates bS based on whether or not the significance coefficient of the color difference component exists in the TU that sandwiches the block boundary that is the calculation target of bS. Further, the boundary strength calculation unit 261 according to the present embodiment independently determines whether or not the significance coefficient of each component of the Y component, the U component, and the V component exists in the TU sandwiching the block boundary for which the bS is calculated. By determining, bS can be calculated. With such a configuration, bS suitable for the U component and the V component is calculated and more appropriate than calculating the bS based on whether or not the significance coefficient of the Y component exists as described with reference to FIG. It is possible to apply a deblock filter to.
[0111]
　The calculation of bS by the boundary strength calculation unit 261 will be described in more detail with reference to FIG. FIG. 8 is a table showing an example of bS calculated by the boundary strength calculation unit 261. The bS calculated by the boundary strength calculation unit 261 can be represented by a plurality of bits. In the example shown in FIG. 8, bS is represented by 5 bits. Further, the bS may be calculated so that the plurality of bits include at least one bit corresponding to each component of the Y component, the U component, and the V component. With this configuration, when the determination unit 263, which will be described later, determines the necessity of applying the deblock filter based on bS, the determination is easily performed by referring to the bS bit corresponding to each component to be determined. It becomes possible.
[0112]
　Further, the boundary strength calculation unit 261 may calculate bS so that each bit included in bS corresponds to the truth of each condition. In the example shown in FIG. 8, bS is calculated so that when each condition is true, the bit corresponding to the condition is 1, and when each condition is false, the bit corresponding to the condition is 0. NS. Further, in the example shown in FIG. 8, bS is represented by 5 bits, the 5th bit of bS is the condition A related to intra-prediction, the 4th bit of bS is the condition B1-Y related to the significance coefficient of the Y component, and bS. The third bit is the condition B1-U related to the significance coefficient of the U component, the second bit of bS is the condition B1-V related to the significance coefficient of the V component, and the first bit of bS is the condition B2 related to MV and the reference picture, respectively. It corresponds. However, the correspondence between each bit of bS and each condition is not limited to the example shown in FIG. For example, the order of the 4th bit, the 3rd bit, and the 2nd bit of bS corresponding to each component of the Y component, the U component, and the V component may be exchanged.
[0113]
　(2) As
　shown in FIG. 7, the determination unit determination unit 263 is applied to the application necessity determination unit 265 for determining the necessity of applying the deblock filter to the color difference component of the decoded image and the color difference component of the decoded image. It includes a filter strength determination unit 267 for determining the filter strength of the deblock filter. Hereinafter, the functions of the application necessity determination unit 265 and the filter strength determination unit 267 will be sequentially described.
[0114]
　In the following description, the necessity of applying the deblock filter to the color difference component of the decoded image and the determination of the filter intensity will be mainly described, and the determination of the luminance component will be omitted as appropriate. Further, the application necessity determination unit 265 and the filter strength determination unit 267 according to the present embodiment independently determine the application necessity and the filter strength of the deblock filter for each of the U component and the V component.
[0115]
　The application necessity determination unit 265 targets the block boundary of the decoded image, and applies the deblock filter to the color difference component of the decoded image based on the bS (boundary intensity) calculated as described above by the boundary strength calculation unit 261. Judge the necessity.
[0116]
　Further, the application necessity determination unit 265 may further determine whether or not the deblock filter is applied to the color difference component of the decoded image based on the block size of the block sandwiching the block boundary. In the following, the determination based on the block size may be referred to as a large block determination. Further, the application necessity determination unit 265 does not have to always perform a large block determination for all block boundaries, and may determine whether or not to perform a large block determination according to bS. The case where the large block determination is performed and the details of the large block determination will be described later.
[0117]
　The application necessity determination unit 265 according to the present embodiment determines the application necessity of the deblock filter by the determination of the following condition C1 and the determination of the condition C2.
[0118]
　-Condition C1: (bS == 16 || (Condition C11 && Condition C12))
　-Condition C2: d 16) || (EDGE_HOR && block_height> 16)
[0126]
　In the above condition C12, EDGE_VER means that the block boundary to be determined is a vertical boundary, and EDGE_HOR means that the block boundary to be determined is a horizontal boundary.
[0127]
　Further, since the above-mentioned condition C2 is the same as the above-mentioned condition C92, the description here will be omitted. The above-mentioned determination of the condition C2 is performed when the condition C1 is true, and when the condition C1 is false, the determination of the condition C2 is not performed and the deblock filter must be applied. It is judged. The determination of the condition C2 requires the process of calculating the variable d as in the above equations (1) to (7), and the processing amount is larger than the determination of the condition C1. Therefore, the determination of the condition C2 is performed after the condition C1. By doing so, it is possible to suppress the processing amount.
[0128]
　Further, the filter strength determination unit 267 further determines the filter strength of the deblock filter applied to the color difference component of the decoded image after the necessity of applying the deblock filter is determined by the conditions C1 and C2 as described above. .. As will be described later, the deblocking filter that can be applied in the present embodiment may be two types, a weak filter having a weaker strength and a strong filter having a stronger strength. Then, the filtering unit 269 described later applies either a weak filter or a strong filter according to the filter strength determined by the filter strength determining unit 267.
[0129]
　The filter strength determination unit 267 determines the filter strength when it is determined that the deblock filter is applied. By determining the filter strength after the determination of the necessity of applying the deblock filter, it is possible to suppress the processing related to the determination of the filter strength.
[0130]
　Further, the filter strength determination unit 267 determines the filter strength based on the waveform of the color difference component of the pixel located near the block boundary. Hereinafter, the determination based on the waveform will be described. The filter strength determination unit 267 determines the filter strength under the condition C3 based on the following waveform.
[0131]
　- conditions C3 :( conditions C31 && conditions C32 && conditions C33) -
　conditions C31: | the p- 3 -P 0 | Tasu | Q 3 -Q 0 | <(Beta >> 3)
　- conditions C32: | the p- 2 -2 * the p- 1 + p 0 | + | q 2 -2 * q 1 + q 0 | <(beta >> 2)
　-Condition C33: | p 0 -q 0 | <((t c * 5 + 1) >> 1)
[0132]
　The filter strength determination unit 267 determines the above condition C3 for the pixels included in the two lines among the pixels located near the block boundary. The condition C31, the condition C32, and the condition C33 used in the above C3 are determined for each line. Note that the p i , q k , p i ′, q k ′, beta, and t C under the condition C31, the condition C32, and the condition C33 have already been described above, and thus the description thereof will be omitted here.
[0133]
　Condition C31, condition C32, and condition C33 are conditions for determining using the pixels included in each line. More specifically, the condition C31 is a condition relating to the flatness in the block of the color difference component of the pixel included in each line. Further, the condition C32 is a condition relating to the determination of the continuity within the block of the color difference component of the pixels included in each line. Further, the condition C33 is a condition relating to the gap (difference) between blocks of the color difference component of the pixel included in each line, and more specifically, the gap between blocks is determined by using the pixel value adjacent to the block boundary. It is a condition to do.
[0134]
　When the condition C31 is true, the flatness of the waveform of the color difference component is high in each block. Further, when the condition C32 is true, the waveform of the color difference component has high continuity in each block. Further, when the condition C32 is true, the waveform of the color difference component has a large gap at the block boundary.
[0135]
　As described above, the condition C3 is determined to be true when all of the above conditions C31, C32, and C33 are true. Further, the filter strength determination unit 267 determines the above condition C3 for each line. However, as described above, the filter strength is determined in units of 2 lines. That is, the strong filter is applied to the two lines when the above condition C3 is true in both of the two consecutive lines, and the weak filter is applied to the two lines when the condition C3 is false. The filter strength is determined.
[0136]
　(3) Filtering unit The
　filtering unit 269 applies the deblocking filter to the color difference component of the pixel located near the block boundary based on the determination result of the deblocking filter application necessity by the application necessity determination unit 265. Further, as described above, the filtering unit 269 applies a weak filter or a strong filter as a deblocking filter according to the filter strength determined by the filter strength determining unit 267.
[0137]
　The weak filter applied to the color difference component by the filtering unit 269 according to the present embodiment is the same as the weak filter applied to the color difference component of the decoded image in, for example, Non-Patent Document 2 and HEVC described above. good. On the other hand, the strong filter applied to the color difference component in the present embodiment is different from the strong filter applied to the color difference component in Non-Patent Document 2 (strong filter applied to the luminance component in HEVC). It's okay. Hereinafter, an example of a strong filter applied to the color difference component in the present embodiment will be described.
[0138]
　The coefficient of the strong filter applied to the color difference component in the present embodiment may be 2 at the center position of the application range of the strong filter and 1 at other positions. Further, the filtering unit 269 sets the application target range of the strong filter from the block boundary to 3 pixels on both sides, and sets the 3 pixels on both sides of the center position of the application target range as reference pixels, and sets the color difference component of the pixels included in the application target range. A strong filter may be applied. For example, a strong filter with p 0 at the center of the applicable range is expressed by the following equation (14).
[0139]
　p 0 ′ = Clip3 (p 0- w * t C , p 0 + w * t C , ((p 3 + p 2 + p 1 + 2 * p 0 + q 0 + q 1 + q 2 + 4) ＞ > 3))… (14)
[0140]
　In the above equation (14), w is a weight that can be set as appropriate, and may be set to, for example, 1 or 2. Further, Clip3 (a, b, c) represents a clipping process in which the value c is clipped in the range of a ≦ c ≦ b as described above.
[0141]
　By applying such a strong filter, it is possible to apply a deblocking filter stronger than the strong filter applied to the color difference component in Non-Patent Document 2 described above.
[0142]
　By the way, when the center position of the application target range of the strong filter is the second pixel or the third pixel from the block boundary, the reference pixel includes a pixel separated by 5 pixels or more from the block boundary. However, pixels separated by 5 pixels or more from the block boundary are not used for determining the filter strength and may not be suitable for use as reference pixels. Therefore, the filtering unit 269 may use the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel by padding instead of the pixel separated by 5 pixels or more from the block boundary.
[0143]
　For example, a strong filter with p 1 at the center of the applicable range is expressed by the following equation (15).
　p 1 ′ = Clip3 (p 1- w * t C , p 1 + w * t C , ((p 4 + p 3 + p 2 + 2 * p 1 + p 0 + q 0 + q 1 + 4) ＞ ＞ 3))
　　　= Clip3 (p 1- w * t C , p 1 + w * t C , ((p 3 + p 3 + p 2 + 2 * p) 1 + p 0 + q 0 + q 1 +4) >> 3))
　　　= Clip3 (p 1- w * t C , p 1 + w * t C , ((2 * p 3 + p 2 + 2 * p) 1 + p 0 + q 0 + q 1 +4) >> 3))… (15)
[0144]
　Similarly, a strong filter having p 2 at the center position of the applicable range is expressed by the following equation (16).
　p 2 ′ = Clip3 (p 2- w * t C , p 2 + w * t C , ((p 5 + p 4 + p 3 + 2 * p 2 + p 1 + p 0 + q 0 +4) ＞ ＞ 3))
　　　= Clip3 (p 2- w * t C , p 2 + w * t C , ((p 3 + p 3 + p 3 + 2 * p) 2 + p 1 + p 0 + q 0 +4) >> 3)))
　　　= Clip3 (p 2- w * t C , p 2 + w * t C , ((3 * p 3 + 2 * p 2 + p) 1 + p 0 + q 0 +4) >> 3))… (16)
[0145]
　Similarly, the strong filters having q 0 to q 3 at the center position of the applicable range are expressed by the following equations (17) to (19), respectively.
　q 0 ′ = Clip3 (q 0 -w * t C , q 0 + w * t C , ((p 2 + p 1 + p 0 + 2 * q 0 + q 1 + q 2 + q 3 +4) ＞ ＞ 3))… (17)
　q 1 ′ = Clip3 (q 1 -w * t C , q 1 + w * t C , ((p 1)+ p 0 + q 0 + 2 * q 1 + q 2 + 2 * q 3 +4) >> 3))… (18)
　q 2 ′ = Clip3 (q 2 -w * t C , q 2 + w * t C , ((p 0 + q 0 + q 1 + 2 * q 2 + 3 * q 3 +4) >> 3))… (19)
[0146]
　　[3-2. Process flow]
　The configuration example of the deblock filter 26 according to the present embodiment has been described above. Subsequently, the flow of processing by the deblock filter 26 according to the present embodiment will be described. FIG. 9 is a flowchart showing an example of the processing flow by the deblock filter 26 according to the present embodiment. In the following, among the processes by the deblock filter 26, the processes related to the features of the present embodiment will be described, and the description of other processes will be omitted as appropriate.
[0147]
　First, the boundary strength calculation unit 261 calculates bS (boundary strength) (S10). Here, the method of calculating bS will be described in more detail with reference to FIG. FIG. 10 is a flowchart for explaining the flow of the boundary strength calculation process (S10) executed by the boundary strength calculation unit 261.
[0148]
　First, the boundary strength calculation unit 261 initializes bS to 0 (S102). Subsequently, the boundary strength calculation unit 261 determines the authenticity of the condition A, which is a condition related to the intra prediction (S104). If condition A is true (YES in S104), bS is set to 16 (S106).
[0149]
　On the other hand, when the condition A is false (NO in S104), the boundary strength calculation unit 261 determines the authenticity of the condition B2, which is a condition relating to the motion vector (MV) and the reference picture (S108). If condition B2 is true (YES in S108), bS is set to 1 (S110).
[0150]
　On the other hand, when the condition B2 is false (NO in S108), the boundary strength calculation unit 261 determines the authenticity of the conditions B1-Y, which is a condition regarding the presence or absence of the significance coefficient of the Y component (S112). If the condition B1-Y is true (YES in S112), the process proceeds to step S116 after 8 is added to bS (S114). On the other hand, when the condition B1-Y is false (NO in S112), the process proceeds to step S116 as it is.
[0151]
　In step S116, the boundary strength calculation unit 261 determines the authenticity of the conditions B1-U, which is a condition regarding the presence or absence of the significance coefficient of the U component. If the condition B1-U is true (YES in S116), the process proceeds to step S120 after 4 is added to bS (S118). On the other hand, when the condition B1-U is false (NO in S116), the process proceeds to step S120 as it is.
[0152]
　In step S120, the boundary strength calculation unit 261 determines the authenticity of the conditions B1-V, which is a condition regarding the presence or absence of the significance coefficient of the V component. When the condition B1-V is true (YES in S120), the boundary strength calculation process (S10) ends after 2 is added to bS (S122). When the condition B1-V is false (NO in S120), the boundary strength calculation process (S10) ends as it is.
[0153]
　Returning to FIG. 9, the description of the processing flow by the deblock filter 26 will be continued. In step S20, the application necessity determination unit 265 of the determination unit 263 determines the authenticity of the above-mentioned condition C1. If the condition C1 is false (NO in S20), the process ends.
[0154]
　On the other hand, when the condition C1 is true (YES in S20), the application necessity determination unit 265 determines whether the condition C2 is true or false (S30). If the condition C2 is false (NO in S30), the process ends.
[0155]
　On the other hand, when the condition C2 is true (YES in S30), the filter strength determination unit 267 of the determination unit 263 determines the filter strength by determining the authenticity of the condition C3 described above (S40). When the condition C3 is true (YES in S40), the filtering unit 269 applies a strong filter to the color difference component of the pixel located near the block boundary (S50). On the other hand, when the condition C3 is false (NO in S40), the filtering unit 269 applies a weak filter to the color difference component of the pixel located near the block boundary (S60).
[0156]
　The flow of processing by the deblock filter 26 according to the present embodiment has been described above. In the case of the YUV420 format, for example, the above-mentioned processing described with reference to FIGS. 9 and 10 can be performed in units of 4 lines in the luminance component of the decoded image, that is, in units of 2 lines in the color difference component of the decoded image.
[0157]
　　[3-3. Modification example]
　FIG. 11 is a table showing a modification of bS calculated by the boundary strength calculation unit 261. Similar to the example of FIG. 8, bS is again represented by 5 bits. Further, the plurality of bits of bS include at least one bit corresponding to each component of the Y component, the U component, and the V component. The 5th bit (most significant bit) of bS is the condition A related to intra-prediction, the 4th bit of bS is the condition B1-Y related to the significance coefficient of the Y component, and the 3rd bit of bS is the condition B1 related to the significance coefficient of the U component. -U and the second bit of bS correspond to the condition B1-V relating to the significance coefficient of the V component, and the first bit (most significant bit) of bS corresponds to the condition B2 relating to MV and the reference picture.
[0158]
　Unlike the example of FIG. 8, in this modified example, the first bit of bS is set to 1 especially when the condition B1-Y is false (hence, bS is less than 8). As a result, in the example of FIG. 8, bS takes any one of 10 values ​​of 0, 1, 2, 4, 6, 8, 10, 12, 14 or 16, whereas in this modified example, it takes one of ten values. , BS takes any of 13 values ​​of 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14 or 16.
[0159]
　12A and 12B are flowcharts for explaining an example of the flow of the boundary strength calculation process corresponding to the modification of FIG. 11. It is assumed that bS has been initialized to 0 before the start of processing. Referring to FIG. 12A, first, the boundary strength calculation unit 261 determines the authenticity of the condition A, which is a condition related to the intra prediction (S150). If the condition A is true, bS is set to 16 (S152), and the boundary strength calculation process ends.
[0160]
　When the condition A is false, the boundary strength calculation unit 261 determines the truth of the conditions B1-Y, which is a condition regarding the presence or absence of the significance coefficient of the Y component (S154). If the condition B1-Y is true, 8 is added to bS (S156), and if the condition B1-Y is false, the addition of 8 is skipped (S158). Next, the authenticity of the condition B1-U, which is a condition regarding the presence or absence of the significance coefficient of the U component, is determined (S160), and if the condition B1-U is true, 4 is added to bS (S162), and the condition If B1-U is false, the addition of 4 is skipped (S164). Next, the authenticity of the condition B1-V, which is a condition regarding the presence or absence of the significance coefficient of the V component, is determined (S166), and if the condition B1-V is true, 2 is added to bS (S168), and the condition If B1-V is false, the addition of 2 is skipped (S170).
[0161]
　Referring to FIG. 12B, the subsequent processing branches depending on whether the condition B1-Y, which is a condition regarding the presence or absence of the significance coefficient of the Y component, is true or false (S172). When the conditions B1-Y are true, the fourth bit of bS is equal to 1, so bS is set to a value of 8 or more (14, 12, 10 or 8) (S174), which will be described later. The boundary strength calculation process ends without executing the MV check.
[0162]
　On the other hand, when the conditions B1-Y are false, the boundary strength calculation unit 261 executes the MV check (S176). The MV check here means a determination of authenticity of the condition B2, which is a condition relating to the motion vector and the reference picture. If the condition B2 is true, 1 is added to bS (S178), and if the condition B2 is false, the addition of 1 is skipped (S180).
[0163]
　<4. Hardware configuration example> The
　above-mentioned series of processes can be executed by hardware or software. When a series of processes are executed by software, the programs constituting the software are installed on the computer. Here, the computer includes a computer embedded in dedicated hardware, a general-purpose personal computer capable of executing various functions by installing various programs, and the like.
[0164]
　FIG. 13 is a block diagram showing a configuration example of the hardware of a computer that executes the above-mentioned series of processes programmatically.
[0165]
　In the computer 800 shown in FIG. 13, the CPU (Central Processing Unit) 801 and the ROM (Read Only Memory) 802 and the RAM (Random Access Memory) 803 are connected to each other via the bus 804.
[0166]
　The input / output interface 810 is also connected to the bus 804. An input unit 811, an output unit 812, a storage unit 813, a communication unit 814, and a drive 815 are connected to the input / output interface 810.
[0167]
　The input unit 811 includes, for example, a keyboard, a mouse, a microphone, a touch panel, an input terminal, and the like. The output unit 812 includes, for example, a display, a speaker, an output terminal, and the like. The storage unit 813 includes, for example, a hard disk, a RAM disk, a non-volatile memory, or the like. The communication unit 814 is composed of, for example, a network interface. The drive 815 drives a removable medium 821 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0168]
　In the computer configured as described above, the CPU 801 loads the program stored in the storage unit 813 into the RAM 803 via the input / output interface 810 and the bus 804 and executes the above-described series. Is processed. The RAM 803 also appropriately stores data and the like necessary for the CPU 801 to execute various processes.
[0169]
　The program executed by the computer (CPU801) can be recorded and applied to the removable media 821 as a package media or the like, for example. In that case, the program can be installed in the storage unit 813 via the input / output interface 810 by attaching the removable media 821 to the drive 815.
[0170]
　The program can also be provided via wired or wireless transmission media such as local area networks, the Internet, and digital satellite broadcasting. In that case, the program can be received by the communication unit 814 and installed in the storage unit 813.
[0171]
　In addition, this program can be pre-installed in the ROM 802 or the storage unit 813.
[0172]
　<5. Conclusion> As
　described above, according to the embodiment of the present disclosure, it is possible to apply the deblock filter more appropriately to the color difference component of the decoded image.
[0173]
　Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is clear that a person having ordinary knowledge in the technical field of the present disclosure can come up with various modifications or modifications within the scope of the technical ideas described in the claims. Of course, it is understood that the above also belongs to the technical scope of the present disclosure.
[0174]
　(Color Difference Related Parameters)
　For example, in the above embodiment, an example in which a flag indicating the presence or absence of the significance coefficient of the color difference component in each TU is used as the color difference related parameter has been described, but the present technology is not limited to such an example. For example, the conversion coefficient itself of the color difference component may be included in the color difference related parameters. In such a case, the boundary strength calculation unit 261 may calculate bS by determining the presence or absence of the significance coefficient of the color difference component in each TU from the conversion coefficient of the color difference component. Further, in relation to the above embodiment, an example in which the value of bS differs depending not only whether or not the conditions B1-Y, B1-U or B1-V are satisfied, but also whether or not the condition B2 is satisfied. It is shown in FIG. However, as in the alternative example shown in FIG. 14, for example, in order to suppress an increase in processing cost, the determination of whether the condition B2 is satisfied for both the color difference components U and V may be omitted.
[0175]
　(Threshold value for large block determination) In the
　above embodiment, an example in which the threshold value used for the large block determination is 16 has been described, but the present technology is not limited to such an example, and may be set to 8 or 32. Further, in the case of the YUV444 format, a threshold value equal to or higher than the threshold value used in the YUV420 format may be used in the large block determination.
[0176]
　(Strong Filter) In the
　above embodiment, an example in which the strong filter represented by the formulas (15) to (19) is applied to the color difference component has been described, but the strong filter applied in the present technology is an example thereof. Not limited. The strong filter applied to the color difference component may be a filter having a stronger filter strength than the weak filter. For example, a strong filter applied to a color difference component in Non-Patent Document 2 (a strong filter applied to a luminance component in HEVC) may be applied to a color difference component in the present technology.
[0177]
　(Application target of
　this technology ) This technology can be applied to any image coding / decoding method. That is, as long as it does not contradict the above-mentioned technology, the specifications of various processes related to image coding / decoding such as conversion (inverse transformation), quantization (inverse quantization), coding (decoding), and prediction are arbitrary. It is not limited to the example. In addition, some of these processes may be omitted as long as they do not contradict the present technology described above.
[0178]
　(Block) Further
　, in the present specification, a "block" (not a block indicating a processing unit) used in the description as a partial area of ​​an image (picture) or a processing unit is an arbitrary partial area in the picture unless otherwise specified. The size, shape, characteristics, etc. are not limited. For example, the “block” includes TB (Transform Block), TU (Transform Unit), PB (Prediction Block), PU (Prediction Unit), SCU (Smallest Coding Unit), and CU described in the above-mentioned references REF1 to REF3. (Coding Unit), LCU (Largest Coding Unit), CTB (Coding Tree Block), CTU (Coding Tree Unit), conversion block, subblock, macroblock, tile, slice, etc. It shall be included.
[0179]
　(Processing unit) The
　data unit in which the various information described above is set and the data unit targeted by the various processing are arbitrary and are not limited to the above-mentioned examples. For example, these information and processes are TU (Transform Unit), TB (Transform Block), PU (Prediction Unit), PB (Prediction Block), CU (Coding Unit), LCU (Largest Coding Unit), and subblock, respectively. , Blocks, tiles, slices, pictures, sequences, or components, or data in those data units may be targeted. Of course, this data unit can be set for each information or process, and it is not necessary that the data unit of all the information or process is unified. The storage location of these information is arbitrary, and may be stored in the header, parameter set, or the like of the above-mentioned data unit. Further, it may be stored in a plurality of places.
[0180]
　Further, in the above embodiment, the deblocking filter processing for the color difference component is performed in units of two lines, but the present technology is not limited to such an example. For example, in the case of the YUV444 format, the deblocking filter processing for the color difference component may be performed in units of 4 lines. In such a case, the application necessity determination unit 265 may make a determination by referring to the first line and the third line in the determination of the above-mentioned condition C3.
[0181]
　(Control Information) The control information
　related to the present technology described above may be transmitted from the coding side to the decoding side. For example, control information (for example, enabled_flag) that controls whether or not the application of the present technology described above is permitted (or prohibited) may be transmitted. Further, for example, control information indicating an object to which the present technology is applied (or an object to which the present technology is not applied) may be transmitted. For example, control information may be transmitted that specifies the block size (upper limit, / lower limit, or both) to which the present technology is applied (or permitted or prohibited), frames, components, layers, and the like.
[0182]
　(Block size information) When
　specifying the size of the block to which the present technology is applied, the block size may be specified not only directly but also indirectly. For example, the block size may be specified using the identification information that identifies the size. Further, for example, the block size may be specified by the ratio or difference with the size of the reference block (for example, LCU, SCU, etc.). For example, when transmitting information for specifying a block size as a syntax element or the like, the information for indirectly specifying the size as described above may be used as the information. By doing so, the amount of information of the information can be reduced, and the coding efficiency may be improved. Further, the designation of the block size also includes the designation of the range of the block size (for example, the designation of the range of the allowable block size).
[0183]
　(Others) In
　the present specification, the "flag" is information for identifying a plurality of states, and is not only information used for identifying two states of true (1) or false (0), but also. Information that can identify three or more states is also included. Therefore, the value that this "flag" can take may be, for example, 2 values ​​of 1/0 or 3 or more values. That is, the number of bits constituting this "flag" is arbitrary, and may be 1 bit or a plurality of bits. Further, (including flags) identification information, the identification information not only by enclosing the bit stream, identification for the information to be certain criteria
for the form to include the difference information by information in the bit stream are also contemplated, hereby In the document, the "flag" and "identification information" include not only the information but also the difference information with respect to the reference information.
[0184]
　Further, various information (metadata, etc.) regarding the coded data (bit stream) may be transmitted or recorded in any form as long as it is associated with the coded data. Here, the term "associate" means, for example, to make the other data available (linkable) when processing one data. That is, the data associated with each other may be combined as one data or may be individual data. For example, the information associated with the coded data (image) may be transmitted on a transmission path different from the coded data (image). Further, for example, the information associated with the coded data (image) may be recorded on a recording medium (or another recording area of ​​the same recording medium) different from the coded data (image). good. Note that this "association" may be a part of the data, not the entire data. For example, an image and information corresponding to the image may be associated with each other in an arbitrary unit such as a plurality of frames, one frame, or a part within the frame.
[0185]
　In addition, in this specification, "synthesize", "multiplex", "add", "integrate", "include", "store", "insert", "insert", "insert". A term such as "" means combining a plurality of objects into one, for example, combining encoded data and metadata into one data, and means one method of "associating" described above.
[0186]
　This technology provides any configuration that constitutes a device or system, for example, a processor as a system LSI (Large Scale Integration), a module that uses a plurality of processors, a unit that uses a plurality of modules, and other functions in the unit. It can also be implemented as an added set or the like (that is, a part of the configuration of the device).
[0187]
　In the present specification, the system means a set of a plurality of components (devices, modules (parts), etc.), and it does not matter whether all the components are in the same housing. Therefore, a plurality of devices housed in separate housings and connected via a network, and a device in which a plurality of modules are housed in one housing are both systems. ..
[0188]
　Further, for example, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). On the contrary, the configurations described above as a plurality of devices (or processing units) may be collectively configured as one device (or processing unit). Further, of course, a configuration other than the above may be added to the configuration of each device (or each processing unit). Further, if the configuration and operation of the entire system are substantially the same, a part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit). .. Further, for example, the present technology can have a cloud computing configuration in which one function is shared and jointly processed by a plurality of devices via a network.
[0189]
　Further, for example, each step described in the above-mentioned flowchart can be executed by one device or can be shared and executed by a plurality of devices. Further, when a plurality of processes are included in one step, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices. In other words, a plurality of processes included in one step can be executed as processes of a plurality of steps. On the contrary, the processes described as a plurality of steps can be collectively executed as one step.
[0190]
　In the program executed by the computer, the processing of the steps for describing the program may be executed in chronological order in the order described in the present specification, or may be called in parallel or called. It may be executed individually at a required timing such as time. That is, as long as there is no contradiction, the processing of each step may be executed in an order different from the above-mentioned order. Further, the processing of the step for writing this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.
[0191]
　In addition, the effects described herein are merely explanatory or exemplary and are not limited. That is, the techniques according to the present disclosure may exhibit other effects apparent to those skilled in the art from the description herein, in addition to or in place of the above effects.
[0192]
　The following configurations also belong to the technical scope of the present disclosure.
(1) The
　decoding is based on a decoding unit that decodes a coded stream to generate a decoded
　image and a boundary strength calculated by targeting a block boundary of the decoded image and using color difference-related parameters related to color difference. A deblock filter is applied
　to the color difference component of pixels located near the block boundary based on the determination unit that determines the necessity of applying the deblock filter to the color difference component of the image and the determination result of the necessity of applying the deblock filter. An
　image processing device including a filtering unit for processing.
(2) The
　color difference-related parameter includes information on the conversion coefficient of the color difference component, and the
　boundary strength is whether or not the significance coefficient of the color difference component exists in the block sandwiching the block boundary for which the boundary strength is calculated. The image processing apparatus according to (1) above, which is calculated based on the coefficient.
(3) The
　boundary strength is calculated by independently determining whether or not a significance coefficient of each component exists in a block sandwiching the block boundary for which the boundary strength is calculated. The image processing apparatus described.
(4) The
　boundary strength is represented by a
　plurality of bits, and the plurality of bits include at least one bit corresponding to each component, according to any one of the above (1) to (3). The image processing apparatus described.
(5)
　The determination unit determines whether or not the deblock filter is applied based on the determination based on the boundary strength and the large block determination using the block size of the blocks sandwiching the block boundary. The image processing apparatus according to any one of the above.
(6)
　The image processing apparatus according to (5), wherein the determination unit performs the large block determination when the boundary strength has a value related to inter-prediction.
(7)
　The image processing apparatus according to (5) or (6), wherein the determination unit performs the large block determination based on the size of the block sandwiching the block boundary in a direction orthogonal to the block boundary.
(8)
　The image according to (7) above, wherein the determination unit makes the large block determination based on whether or not the size of the block sandwiching the block boundary in the direction orthogonal to the block boundary is larger than 16. Processing equipment.
(9)
　Any one of the above (1) to (8), wherein the filtering unit applies a weak filter or a strong filter as the deblock filter to the color difference component of the pixel located near the block boundary. The image processing apparatus according to.
(10)
　The image processing apparatus according to (9) above, wherein the coefficient of the strong filter is 2 at the center position of the applicable range of the strong filter and 1 at other positions.
(11)
　In the filtering unit, the application range of the strong filter is from the block boundary to 3 pixels on both sides, and the 3 pixels on both sides of the center position of the application range are reference pixels, and the color difference of the pixels included in the application range. The image processing apparatus according to (9) or (10) above, wherein the strong filter is applied to the components.
(12) The
　filtering unit uses the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel instead of the pixel separated by 5 pixels or more from the block boundary. The image processing apparatus according to.
(13) The item according to any one of (10) to (12 )
　above, wherein the filtering unit performs clipping processing based on the parameter t C specified based on the quantization parameter and applies the strong filter. Image processing equipment.
(14) The
　determination unit further determines the filter strength of the deblock filter applied to the color difference component of the pixel located near the block boundary, and the
　filtering unit further determines the filter strength determined by the determination unit. The image processing apparatus according to any one of (9) to (13), wherein the weak filter or the strong filter is applied to a color difference component of a pixel located near the block boundary depending on the intensity.
(15)
　The image processing apparatus according to (14), wherein the determination unit determines the filter strength after determining whether or not the deblock filter is applicable.
(16)
　The image processing apparatus according to (15), wherein the determination unit determines the filter intensity based on the waveform of the color difference component of the pixel located near the block boundary.
(17) The
　determination unit determines the filter based on a condition relating to flatness, a condition relating to continuity, and a condition relating to a gap of the color difference component of the pixels included in the two lines among the pixels located near the block boundary. The intensity is determined, and the
　filtering unit applies the strong filter when it is determined that all of the conditions relating to the flatness, the condition relating to the continuity, and the condition relating to the gap are satisfied, and at least one of them is applied. The image processing apparatus according to (16), wherein the weak filter is applied when it is determined that one of the conditions is not satisfied.
(18) The
　decoded image is generated based on decoding the coded stream to generate a
　decoded image, and based on the boundary strength calculated by targeting the block boundary of the decoded image and using the color difference-related parameters related to the color difference. and determining the deblocking filter applies necessity for the color difference components of,
　based on the deblocking filter applies necessity of determination results, applying the deblocking filter to the chrominance components of pixels located in the vicinity of the block boundary And,
　including image processing methods.
(19)
　A determination unit that determines whether or not a deblock filter is applied to the color difference component of the decoded image based on the boundary strength calculated using the color difference-related parameters related to the color difference, targeting the block boundary of the locally decoded decoded image. If,
　based on the deblocking filter applies necessity determination result, a filtering unit applying a deblocking filter to the chrominance components of pixels located in the vicinity of the block boundary,
　the deblocking filter is applied by the filtering unit An
　image processing device including a coding unit that encodes an image using the decoded image .
(20)
　Targeting the block boundary of a locally decoded decoded image, it is determined whether or not a deblock filter is applied to the color difference component of the decoded image based on the boundary strength calculated using the color difference-related parameters related to the color difference. and that,
　based on the deblocking filter applies necessity determination result, and applying a deblocking filter to the chrominance components of pixels located in the vicinity of the block boundary,
　the decoding the deblocking filter is applied An image
　processing method that includes encoding an image using an image .
Code description
[0193]
　10 Image coding device
　16 Lossless coding unit
　26 Deblock filter
　60 Image decoding device
　62 Lossless decoding unit
　261 Boundary strength calculation unit
　263 Judgment unit
　265 Applicability judgment unit
　267 Filter strength determination unit
　269 ​​Filtering unit
The scope of the claims
[Claim 1]

　The color difference of the decoded image is based on the boundary strength calculated using the color difference-related parameters related to the color difference, targeting the block boundary of the decoded image 　and the decoding unit that decodes the coded stream to generate the decoded image. A determination unit that determines
　whether or not a deblock filter is applied to a component, and a filtering unit that applies a deblock filter to a color difference component of a pixel located near the block boundary based on the determination result of whether or not the deblock filter is applied. And,
　an image processing device.
[Claim 2]
　The color difference-related parameter includes information on the conversion coefficient of the color difference component, and the
　boundary strength is based on whether or not the significance coefficient of the color difference component exists in the block sandwiching the block boundary for which the boundary strength is calculated. The image processing apparatus according to claim 1, which is calculated by the above method.
[Claim 3]
　The image processing according to claim 2, wherein the boundary strength is calculated by independently determining whether or not a significance coefficient of each component exists in a block sandwiching the block boundary for which the boundary strength is calculated. Device.
[Claim 4]

　The image processing apparatus according to claim 1, 　wherein the boundary strength is represented by a plurality of bits, and the plurality of bits include at least one bit corresponding to each component.
[Claim 5]
　The image processing according to claim 1, wherein the determination unit determines whether or not the deblock filter is applied based on the determination based on the boundary strength and the large block determination using the block size of the blocks sandwiching the block boundary. Device.
[Claim 6]
　The image processing apparatus according to claim 5, wherein the determination unit performs the large block determination when the boundary strength has a value related to inter-prediction.
[Claim 7]
　The image processing apparatus according to claim 5, wherein the determination unit performs the large block determination based on the size of the block sandwiching the block boundary in a direction orthogonal to the block boundary.
[Claim 8]
　The image processing apparatus according to claim 7, wherein the determination unit performs the large block determination based on whether or not the size of the block sandwiching the block boundary in the direction orthogonal to the block boundary is larger than 16.
[Claim 9]
　The image processing apparatus according to claim 1, wherein the filtering unit applies a weak filter or a strong filter as the deblock filter to a color difference component of a pixel located near the block boundary.
[Claim 10]
　The image processing apparatus according to claim 9, wherein the coefficient of the strong filter is 2 at the central position of the applicable range of the strong filter and 1 at other positions.
[Claim 11]
　In the filtering unit, the application range of the strong filter is from the block boundary to 3 pixels on both sides, and the 3 pixels on both sides of the center position of the application range are reference pixels, and the color difference of the pixels included in the application range. The image processing apparatus according to claim 9, wherein the strong filter is applied to the components.
[Claim 12]
　The eleventh aspect of claim 11, wherein the filtering unit uses the pixel value of the fourth pixel from the block boundary as the pixel value of the reference pixel by padding instead of the pixel separated by 5 pixels or more from the block boundary. Image processing device.
[Claim 13]
　The image processing apparatus according to claim 10, wherein the filtering unit performs clipping processing based on the parameter tC specified based on the quantization parameter and applies the strong filter.
[Claim 14]
　The determination unit further determines the filter strength of the deblock filter applied to the color difference component of the pixel located near the block boundary, and the
　filtering unit responds to the filter strength determined by the determination unit. The image processing apparatus according to claim 9, wherein the weak filter or the strong filter is applied to a color difference component of a pixel located in the vicinity of the block boundary.
[Claim 15]
　The image processing apparatus according to claim 14, wherein the determination unit determines the filter strength after determining whether or not the deblock filter is applied.
[Claim 16]
　The image processing apparatus according to claim 15, wherein the determination unit determines the filter intensity based on the waveform of the color difference component of the pixel located near the block boundary.
[Claim 17]
　The determination unit determines the filter strength based on the condition regarding the flatness of the color difference component of the pixels included in the two lines, the condition regarding the continuity, and the condition regarding the gap among the pixels located near the block boundary. Then, the
　filtering unit applies the strong filter when it is determined that all of the conditions relating to the flatness, the condition relating to the continuity, and the condition relating to the gap are satisfied, and at least one condition is satisfied. The image processing apparatus according to claim 16, wherein the weak filter is applied when it is determined that the condition is not satisfied.
[Claim 18]

　The color difference component of the decoded image is based on 　decoding the coded stream to generate a decoded image and the boundary strength calculated using the color difference related parameters related to the color difference for the block boundary of the decoded image. and determining the deblocking filter applies necessity for,
　and that based on the deblocking filter applies necessity of the determination result, to apply a deblocking filter to the chrominance components of pixels located in the vicinity of the block boundary,
　the Image processing method including.
[Claim 19]
　A determination unit that determines whether or not a deblock filter is applied to the color difference component of the decoded image based on the boundary strength calculated using the color difference-related parameters related to the color difference, targeting the block boundary of the locally decoded decoded image. If,
　based on the deblocking filter applies necessity determination result, a filtering unit applying a deblocking filter to the chrominance components of pixels located in the vicinity of the block boundary,
　the deblocking filter is applied by the filtering unit An
　image processing device including a coding unit that encodes an image using the decoded image .
[Claim 20]
　For the block boundary of the locally decoded decoded image, it is determined whether or not the deblock filter is applied to the color difference component of the decoded image based on the boundary strength calculated by using the color difference related parameters related to the color difference. ,
　based on the deblocking filter applies necessity determination result, and applying a deblocking filter to the chrominance components of pixels located in the vicinity of the block boundary,
　using the decoded image the deblocking filter is applied An
　image processing method that includes encoding an image .