Title of invention: Video signal processing method and apparatus
Technical field
[One]
The present invention relates to a video signal processing method and apparatus.
Background
[2]
Recently, demand for high-resolution and high-quality images such as high definition (HD) images and ultra high definition (UHD) images is increasing in various application fields. The higher the resolution and quality of the video data, the higher the amount of data is compared to the existing video data. Therefore, when the video data is transmitted using a medium such as a wired/wireless broadband line or stored using an existing storage medium, the transmission cost and The storage cost will increase. High-efficiency image compression techniques can be used to solve these problems that occur as image data becomes high-resolution and high-quality.
[3]
An inter-screen prediction technology that predicts pixel values ​​included in the current picture from a picture before or after the current picture using video compression technology, an intra-screen prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, Various technologies exist, such as an entropy encoding technology that allocates a short code to a value with a high frequency of appearance and a long code to a value with a low frequency of appearance, and by using such an image compression technology, image data can be effectively compressed and transmitted or stored.
[4]
Meanwhile, as the demand for high-resolution images increases, the demand for 3D image contents as a new image service is also increasing. Discussions are underway on video compression techniques for effectively providing 3D image contents of high resolution and ultra high resolution.
Detailed description of the invention
Technical challenge
[5]
An object of the present invention is to provide a method and apparatus for encoding/decoding residual data based on a plurality of transformations in encoding/decoding a video signal.
[6]
An object of the present invention is to provide a method and apparatus for performing a second order transformation on a partial region of a block on which the first order transformation has been performed.
[7]
The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.
Means of solving the task
[8]
A video signal decoding method and apparatus according to the present invention obtains residual coefficients of a current block, inverse quantizes the residual coefficients, performs a second-order inverse transform on the inverse quantized residual coefficients, and performs a second-order inverse transform. It is possible to perform a first-order inverse transformation on the result. In this case, the second-order inverse transformation may be performed on a partial region of the current block.
[9]
A video signal encoding method and apparatus according to the present invention performs a first-order transformation on a residual sample, a second-order transformation on the first-order transformation coefficients generated as a result of the first-order transformation, and the second-order transformation Second-order transform coefficients generated as a result of may be quantized and the quantized transform coefficients may be encoded. In this case, the second-order transformation may be performed on a partial region of the current block.
[10]
In the video signal encoding/decoding method and apparatus according to the present invention, the second-order transform/inverse transform may be performed using a one-dimensional matrix generated by arranging a transform matrix and residual data in one dimension.
[11]
In the video signal encoding/decoding method and apparatus according to the present invention, the number of rows or columns of the transform matrix may be smaller than the number of samples included in the partial region.
[12]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the size or shape of the partial region may be determined based on the size or shape of the current block.
[13]
In the method and apparatus for encoding/decoding a video signal according to the present invention, a transform type for the first-order transform/inverse transform may be determined based on index information signaled through a bitstream.
[14]
In the video signal encoding/decoding method and apparatus according to the present invention, the index information specifies one of a plurality of transform sets, and a first transform type candidate included in the transform set is transformed in the horizontal direction of the current block. A second transform type candidate included in the transform set may be determined as a type, and may be determined as a vertical transform type of the current block.
[15]
In the method and apparatus for encoding/decoding a video signal according to the present invention, a transform type for the first-order transform/inverse transform may be determined based on a result of comparing a width of the current block and a height of the current block.
[16]
The features briefly summarized above with respect to the present invention are merely exemplary aspects of the detailed description of the present invention to be described later, and do not limit the scope of the present invention.
Effects of the Invention
[17]
According to the present invention, it is possible to improve the encoding/decoding efficiency of residual data by performing transformation on the residual data multiple times.
[18]
According to the present invention, encoding/decoding efficiency can be improved by performing second-order transformation on a partial region of a block on which the first-order transformation has been performed.
[19]
The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.
Brief description of the drawing
[20]
1 is a block diagram showing an image encoding apparatus according to an embodiment of the present invention.
[21]
2 is a block diagram showing an image decoding apparatus according to an embodiment of the present invention.
[22]
3 is a diagram illustrating partition mode candidates that can be applied to the coding block when the coding block is encoded by inter prediction.
[23]
4 illustrates an example of hierarchically partitioning a coding block based on a tree structure as an embodiment to which the invention is applied.
[24]
5 is a diagram showing a partition type in which partitioning based on a binary tree is allowed as an embodiment to which the present invention is applied.
[25]
6 shows a triple-tree division type.
[26]
7 is a diagram showing an example in which only a specific type of binary tree-based division is allowed.
[27]
FIG. 8 is a diagram for explaining an example in which information related to an allowable number of binary tree divisions is encoded/decoded as an embodiment to which the present invention is applied.
[28]
9 is a flowchart illustrating a process of encoding a residual sample according to an embodiment to which the present invention is applied.
[29]
11 is a diagram showing whether a vertical direction transform set and a horizontal direction transform set are the same for 33 intra prediction modes.
[30]
12 is a diagram showing a performing area of ​​a quadratic transformation.
Mode for carrying out the invention
[31]
In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements.
[32]
Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.
[33]
When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.
[34]
The terms used in the present application are used only to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements, numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility.
[35]
Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same elements in the drawings, and duplicate descriptions for the same elements are omitted.
[36]
[37]
1 is a block diagram showing an image encoding apparatus according to an embodiment of the present invention.
[38]
Referring to FIG. 1, the image encoding apparatus 100 includes a picture splitter 110, a prediction unit 120, 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, and an entropy encoder ( 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.
[39]
Each of the components shown in FIG. 1 is shown independently to represent different characteristic functions in an image encoding apparatus, and does not mean that each component is formed of separate hardware or a single software component. That is, each constituent part is listed and included as a respective constituent part for convenience of explanation, and at least two constituent parts of each constituent part are combined to form one constituent part, or one constituent part may be divided into a plurality of constituent parts to perform a function. Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention unless departing from the essence of the present invention.
[40]
In addition, some of the components are not essential components that perform essential functions in the present invention, but may be optional components only for improving performance. The present invention can be implemented by including only components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.
[41]
The picture dividing unit 110 may divide the input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitter 110 divides a picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, a prediction unit, and a transformation unit based on a predetermined criterion (for example, a cost function). Select to encode the picture.
[42]
For example, one picture may be split into a plurality of coding units. In order to split the coding units in a picture, a recursive tree structure such as a quad tree structure can be used. Encoding that is split into other coding units based on one image or the largest coding unit as a root. A unit may be divided with as many child nodes as the number of divided coding units. Coding units that are no longer split according to certain restrictions become leaf nodes. That is, when it is assumed that only square splitting is possible for one coding unit, one coding unit may be split into up to four different coding units.
[43]
Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit that performs encoding or a unit that performs decoding.
[44]
The prediction unit may be split in a shape such as at least one square or rectangle of the same size within one coding unit, or one prediction unit among prediction units split within one coding unit is another prediction. It may be divided to have a shape and/or size different from the unit.
[45]
When a prediction unit that performs intra prediction based on a coding unit is not a minimum coding unit, intra prediction may be performed without dividing into a plurality of prediction units NxN.
[46]
The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It is possible to determine whether to use inter prediction or to perform intra prediction for the prediction unit, and determine specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. In this case, a processing unit in which prediction is performed may be different from a processing unit in which a prediction method and specific content are determined. For example, a prediction method and a prediction mode are determined in a prediction unit, and prediction may be performed in a transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoder 165 together with a residual value and transmitted to a decoder. In the case of using a specific encoding mode, it is possible to encode an original block as it is and transmit it to a decoder without generating a prediction block through the prediction units 120 and 125.
[47]
The inter prediction unit 120 may predict a prediction unit based on information of at least one picture of a picture before or after a current picture, and in some cases, predict based on information of a partial region in the current picture that has been encoded. You can also predict units. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.
[48]
The reference picture interpolation unit may receive reference picture information from the memory 155 and may generate pixel information of an integer number of pixels or less from the reference picture. In the case of a luminance pixel, a DCT-based 8-tap interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/4 pixels. In the case of a color difference signal, a DCT-based interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.
[49]
The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. Various methods, such as a full search-based block matching algorithm (FBMA), a three step search (TSS), and a new three-step search algorithm (NTS), can be used as a method for calculating a motion vector. The motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixels. The motion prediction unit may predict the current prediction unit by differently predicting the motion. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, and an intra block copy method may be used as the motion prediction method.
[50]
The intra predictor 125 may generate a prediction unit based on reference pixel information around a current block, which is pixel information in the current picture. If the neighboring block of the current prediction unit is a block that has performed inter prediction and the reference pixel is a pixel that has performed inter prediction, the reference pixel included in the block that has performed inter prediction is a reference pixel of the block that has performed intra prediction Can be used in place of information. That is, when the reference pixel is not available, information on the reference pixel that is not available may be replaced with at least one reference pixel among the available reference pixels.
[51]
In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which directional information is not used when prediction is performed. A mode for predicting luminance information and a mode for predicting color difference information may be different, and intra prediction mode information or predicted luminance signal information used to predict luminance information may be used to predict color difference information.
[52]
When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is based on a pixel on the left, a pixel on the top left, and a pixel on the top of the prediction unit. Can be done. However, when the size of the prediction unit and the size of the transformation unit are different when performing intra prediction, intra prediction may be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN splitting may be used for only the smallest coding unit.
[53]
The intra prediction method may generate a prediction block after applying an AIS (Adaptive Intra Smoothing) filter to a reference pixel according to a prediction mode. The types of AIS filters applied to the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, the current prediction unit and the surrounding prediction units are used using predetermined flag information. Information indicating that the prediction mode of is the same can be transmitted, and if the prediction modes of the current prediction unit and the neighboring prediction units are different, entropy encoding is performed to encode prediction mode information of the current block.
[54]
Also, a residual block including a prediction unit that performs prediction based on a prediction unit generated by the prediction units 120 and 125 and residual information that is a difference value from the original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130.
[55]
In the transform unit 130, the original block and the residual block including residual information of the prediction unit generated through the prediction units 120 and 125 are converted to DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT. It can be converted using the same conversion method Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information of a prediction unit used to generate the residual block.
[56]
The quantization unit 135 may quantize values ​​converted into the frequency domain by the transform unit 130. Quantization coefficients may vary depending on the block or the importance of the image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.
[57]
The reordering unit 160 may rearrange coefficient values ​​on the quantized residual values.
[58]
The rearrangement unit 160 may change the two-dimensional block shape coefficient into a one-dimensional vector shape through a coefficient scanning method. For example, the rearrangement unit 160 may scan from a DC coefficient to a coefficient in a high frequency region using a Zig-Zag Scan method, and change it into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scan, a vertical scan that scans a two-dimensional block shape coefficient in a column direction and a horizontal scan that scans a two-dimensional block shape coefficient in a row direction may be used. That is, according to the size of the transform unit and the intra prediction mode, it is possible to determine which scan method is to be used among zig-zag scan, vertical direction scan, and horizontal direction scan.
[59]
The entropy encoding unit 165 may perform entropy encoding based on values ​​calculated by the rearrangement unit 160. Entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).
[60]
The entropy encoder 165 includes residual value coefficient information and block type information of a coding unit, prediction mode information, division unit information, prediction unit information and transmission unit information, and motion from the reordering unit 160 and the prediction units 120 and 125. Various information, such as vector information, reference frame information, block interpolation information, and filtering information, can be encoded.
[61]
The entropy encoder 165 may entropy-encode a coefficient value of a coding unit input from the reordering unit 160.
[62]
The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize values ​​quantized by the quantization unit 135 and inverse transform the values ​​transformed by the transform unit 130. The residual generated by the inverse quantization unit 140 and the inverse transform unit 145 is reconstructed by combining the prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125 Blocks (Reconstructed Block) can be created.
[63]
The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).
[64]
The deblocking filter can remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be processed in parallel when performing vertical filtering and horizontal filtering.
[65]
The offset correction unit may correct an offset from the original image in pixel units of the deblocking image. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a certain number of areas, and then the area to be offset is determined and the offset is applied to the area, or offset by considering the edge information of each pixel. You can use the method of applying.
[66]
Adaptive Loop Filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into predetermined groups, one filter to be applied to the corresponding group may be determined, and filtering may be performed differentially for each group. Information related to whether to apply the ALF may be transmitted for each coding unit (CU) of the luminance signal, and the shape and filter coefficient of an ALF filter to be applied may vary according to each block. In addition, the same type (fixed type) ALF filter may be applied regardless of the characteristics of the block to be applied.
[67]
The memory 155 may store the reconstructed block or picture calculated through the filter unit 150, and the stored reconstructed block or picture may be provided to the prediction units 120 and 125 when performing inter prediction.
[68]
[69]
2 is a block diagram showing an image decoding apparatus according to an embodiment of the present invention.
[70]
2, the image decoder 200 includes an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, and a filter unit 240) and a memory 245 may be included.
[71]
When an image bitstream is input from the image encoder, the input bitstream may be decoded in a procedure opposite to that of the image encoder.
[72]
The entropy decoder 210 may perform entropy decoding in a procedure opposite to that of performing entropy encoding in an entropy encoder of an image encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied in response to the method performed by the image encoder.
[73]
The entropy decoder 210 may decode information related to intra prediction and inter prediction performed by the encoder.
[74]
The rearrangement unit 215 may perform rearrangement based on a method of rearranging the bitstream entropy-decoded by the entropy decoder 210 by the encoder. Coefficients expressed in the form of a one-dimensional vector may be reconstructed into coefficients in the form of a two-dimensional block and rearranged. The reordering unit 215 may perform reordering through a method of receiving information related to coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the corresponding encoder.
[75]
The inverse quantization unit 220 may perform inverse quantization based on a quantization parameter provided by an encoder and a coefficient value of a rearranged block.
[76]
The inverse transform unit 225 may perform an inverse transform, that is, an inverse DCT, an inverse DST, and an inverse KLT, for transforms, that is, DCT, DST, and KLT, performed by the transform unit on the quantization result performed by the image encoder. Inverse transformation may be performed based on a transmission unit determined by the image encoder. The inverse transform unit 225 of the image decoder may selectively perform a transformation technique (eg, DCT, DST, KLT) according to a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.
[77]
The prediction units 230 and 235 may generate a prediction block based on the prediction block generation-related information provided from the entropy decoder 210 and the previously decoded block or picture information provided from the memory 245.
[78]
As described above, if the size of the prediction unit and the size of the transformation unit are the same when intra prediction is performed in the same manner as the operation of the image encoder, a pixel on the left side of the prediction unit, a pixel on the top left side, and a pixel on the top side. If the size of the prediction unit and the size of the transform unit are different when performing intra prediction, but the size of the prediction unit and the size of the transform unit are different when performing intra prediction, intra prediction is performed using a reference pixel based on the transform unit. I can. In addition, intra prediction using NxN splitting for only the smallest coding unit may be used.
[79]
The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, etc., and classifies the prediction unit from the current coding unit, and predicts It can be determined whether the unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information necessary for inter prediction of the current prediction unit provided by the video encoder to predict the current based on information included in at least one picture of a previous picture or a subsequent picture of the current picture containing the current prediction unit. Inter prediction for a unit can be performed. Alternatively, inter prediction may be performed based on information on a partial region previously-restored in the current picture including the current prediction unit.
[80]
In order to perform inter prediction, the motion prediction method of the prediction unit included in the coding unit based on the coding unit is among the skip mode, merge mode, AMVP mode, and intra block copy mode. You can determine whether or not this is any way.
[81]
The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit that has performed intra prediction, intra prediction may be performed based on intra prediction mode information of a prediction unit provided from an image encoder. The intra prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on a reference pixel of the current block, and may determine whether to apply the filter according to the prediction mode of the current prediction unit and apply it. AIS filtering may be performed on a reference pixel of a current block by using the prediction mode and AIS filter information of the prediction unit provided by the video encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.
[82]
When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating a reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. When the prediction mode of the current prediction unit is a prediction mode in which a prediction block is generated without interpolating a reference pixel, the reference pixel may not be interpolated. The DC filter may generate a prediction block through filtering when the prediction mode of the current block is the DC mode.
[83]
The reconstructed block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.
[84]
Information on whether a deblocking filter is applied to a corresponding block or picture from the video encoder, and when a deblocking filter is applied, information on whether a strong filter or a weak filter is applied may be provided. In the deblocking filter of the image decoder, information related to the deblocking filter provided from the image encoder may be provided, and the image decoder may perform deblocking filtering on a corresponding block.
[85]
The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and information on the offset value.
[86]
The ALF may be applied to a coding unit based on information on whether to apply ALF and information on ALF coefficients provided from the encoder. Such ALF information may be provided by being included in a specific parameter set.
[87]
The memory 245 may store the reconstructed picture or block so that it can be used as a reference picture or a reference block, and may also provide the reconstructed picture to an output unit.
[88]
As described above, in an embodiment of the present invention, for convenience of description, a coding unit is used as a term, but it may be a unit that performs not only encoding but also decoding.
[89]
In addition, the current block represents a block to be encoded/decoded, and according to an encoding/decoding step, a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), or a prediction block (Or a prediction unit) or the like. In this specification,'unit' denotes a basic unit for performing a specific encoding/decoding process, and'block' may denote a sample array of a predetermined size. Unless otherwise specified,'block' and'unit' may be used interchangeably. For example, in an embodiment to be described later, it may be understood that the coding block (coding block) and the coding unit (coding unit) have the same meaning as each other.
[90]
[91]
One picture may be divided into square or non-square basic blocks and encoded/decoded. In this case, the basic block may be referred to as a coding tree unit. The coding tree unit may be defined as a coding unit having the largest size allowed in a sequence or slice. Information indicating whether the coding tree unit is square or non-square or information related to the size of the coding tree unit may be signaled through a sequence parameter set, a picture parameter set, or a slice header. The coding tree unit can be divided into smaller sized partitions. In this case, when the partition generated by dividing the coding tree unit is referred to as depth 1, the partition generated by dividing the partition having depth 1 may be defined as depth 2. That is, a partition generated by dividing a partition having a depth k in a coding tree unit may be defined as having a depth k+1.
[92]
A partition of an arbitrary size generated as the coding tree unit is divided may be defined as a coding unit. The coding unit may be recursively divided, or may be divided into basic units for performing prediction, quantization, transformation, or in-loop filtering. For example, a partition of an arbitrary size generated as the coding unit is divided may be defined as a coding unit, or as a transform unit or a prediction unit, which is a basic unit for performing prediction, quantization, transformation, or in-loop filtering.
[93]
Alternatively, a prediction block having the same size as the coding block or smaller than the coding block may be determined through prediction partitioning of the coding block. For predictive partitioning of a coding block, any one of partition mode (Part_mode) candidates indicating a partitioning type of the coding block may be specified. Information for determining a partition index indicating any one of the partition mode candidates may be signaled through a bitstream. Alternatively, the partition index of the coding block may be determined based on at least one of the size, shape, or coding mode of the coding block. The size or shape of the prediction block may be determined based on the partition mode specified by the partition index. The partition mode candidate may include an asymmetric partition type (eg, nLx2N, nRx2N, 2NxnU, 2NxnD). The number or type of asymmetric partition mode candidates that can be used by the coding block may be determined based on at least one of the size, shape, or coding mode of the coding block.
[94]
3 is a diagram illustrating partition mode candidates that can be applied to the coding block when the coding block is encoded by inter prediction.
[95]
When the coding block is coded by inter prediction, any one of the eight partition mode candidates shown in FIG. 3 may be applied to the coding block.
[96]
On the other hand, when the coding block is encoded by intra prediction, only square partition division can be applied to the coding block. That is, when the coding block is encoded by intra prediction, the partition mode PART_2Nx2N or PART_NxN may be applied to the coding block.
[97]
PART_NxN can be applied when a coding block has a minimum size. Here, the minimum size of the coding block may be predefined by an encoder and a decoder. Alternatively, information on the minimum size of the coding block may be signaled through the bitstream. As an example, the minimum size of the coding block may be signaled through a slice header. Accordingly, the minimum size of the coding block may be determined differently for each slice.
[98]
As another example, the partition mode candidates that can be used by the coding block may be differently determined according to at least one of the size or shape of the coding block. For example, the number or type of partition mode candidates that the coding block can use may be differently determined according to at least one of the size or shape of the coding block.
[99]
Alternatively, the type or number of asymmetric partition mode candidates that can be used by the coding block may be determined based on the size or shape of the coding block. The number or type of asymmetric partition mode candidates that the coding block can use may be differently determined according to at least one of the size or shape of the coding block. For example, when a coding block has an amorphous shape having a width greater than a height, at least one of PART_2NxN, PART_2NxnU, and PART_2NxnD may not be used as a partition mode candidate of the coding block. When the coding block has an amorphous shape whose height is greater than the width, at least one of PART_Nx2N, PART_nLx2N, and PART_nRx2N may not be used as a partition mode candidate of the coding block.
[100]
In general, the size of the prediction block may range from 64x64 to 4x4. However, when the coding block is encoded by inter prediction, in order to reduce a memory bandwidth when performing motion compensation, the prediction block may not have a 4x4 size.
[101]
It is also possible to recursively partition the coding block based on the partition mode. That is, based on the partition mode determined by the partition index, the coding block may be partitioned, and each partition generated as a result of the partitioning of the coding block may be defined as a coding block.
[102]
Hereinafter, a method of dividing the coding unit will be described in more detail. In an embodiment described below, the coding unit may mean a coding tree unit or a coding unit included in a coding tree unit. In addition, a'partition' generated as a coding block is divided may mean a'coding block'. The partitioning method described below may be applied to partitioning a coding block into a plurality of prediction blocks or a plurality of transform blocks.
[103]
The coding unit can be divided by at least one line. In this case, the angle of the line dividing the coding unit may be a value within the range of 0 degrees to 360 degrees. For example, an angle of a horizontal line may be 0 degrees, an angle of a vertical line may be 90 degrees, an angle of a diagonal line in the upper right direction may be 45 degrees, and an angle of a diagonal line in the upper left corner may be 135 degrees.
[104]
When the coding unit is divided by a plurality of lines, all of the plurality of lines may have the same angle. Alternatively, at least one of the plurality of lines may have a different angle from the other lines. Alternatively, the coding tree unit or a plurality of lines dividing the coding unit may have a predefined angle difference (eg, 90 degrees).
[105]
Information about a line dividing a coding unit may be determined by a partition mode. Alternatively, information on at least one of the number, direction, angle, or position of a line within a block may be encoded.
[106]
For convenience of description, in an embodiment to be described later, it is assumed that the coding unit is divided into a plurality of coding units by using at least one of a vertical line or a horizontal line.
[107]
The number of vertical lines or horizontal lines for partitioning the coding unit may be at least one or more. For example, the coding unit may be divided into two partitions using one vertical line or one horizontal line. Alternatively, the coding unit may be divided into three partitions by using two vertical lines or two horizontal lines. Alternatively, one vertical line and one horizontal line may be used to divide the coding unit into four partitions whose width and height are 1/2 smaller than that of the coding unit.
[108]
When the coding unit is divided into a plurality of partitions using at least one vertical line or at least one horizontal line, the partitions may have a uniform size. Alternatively, one partition may have a different size from the other partitions, or each partition may have a different size. For example, when the coding unit is divided into two horizontal lines or two vertical lines, the coding unit may be divided into three partitions. In this case, the width ratio or height ratio of the three partitions may be n:2n:n, 2n:n:n, or n:n:2n.
[109]
In embodiments to be described later, the division of the coding unit into four partitions will be referred to as quad-tree-based division. In addition, the division of the coding unit into two partitions is referred to as binary tree-based division. In addition, division of the coding unit into three partitions will be referred to as a triple tree-based division.
[110]
In the drawings to be described later, it will be shown that one vertical line and/or one horizontal line is used to divide the coding unit, but using a greater number of vertical lines and/or a greater number of horizontal lines than that shown, It will be said that dividing a coding unit into a larger number of partitions than shown or a smaller number of partitions than shown is also included in the scope of the present invention.
[111]
4 illustrates an example of hierarchically partitioning a coding block based on a tree structure as an embodiment to which the present invention is applied.
[112]
The input video signal is decoded in units of a predetermined block, and a basic unit for decoding the input video signal in this way is called a coding block. The coding block may be a unit that performs intra/inter prediction, transform, and quantization. In addition, a prediction mode (eg, an intra prediction mode or an inter prediction mode) is determined for each coding block, and prediction blocks included in the coding block may share the determined prediction mode. The coding block may be a square or non-square block having an arbitrary size in the range of 8×8 to 64×64, and may be a square or non-square block having a size of 128×128, 256×256 or higher.
[113]
Specifically, the coding block may be hierarchically partitioned based on at least one of a quad tree partitioning method, a binary tree partitioning method, or a triple tree partitioning method. The quad-tree-based division may mean a method in which a 2Nx2N coding block is divided into four NxN coding blocks. The binary tree-based partitioning may mean a method in which one coding block is divided into two coding blocks. The triple tree-based partitioning may mean a method in which one coding block is divided into three coding blocks. Even if division based on a binary tree or a triple tree is performed, a coding block having a square shape may exist in a lower depth.
[114]
Partitions created due to binary tree-based partitioning may be symmetric or asymmetric. Further, the coding block divided based on the binary tree may be a square block or a non-square block (eg, a rectangle).
[115]
5 is a diagram showing a partitioning form of a coding block based on binary tree partitioning. The partition type of a coding block based on binary tree division is a symmetric type such as 2NxN (horizontal asymmetric coding unit) or Nx2N (vertical amorphous coding unit), or asymmetric type such as nLx2N, nRx2N, 2NxnU, or 2NxnD. It may include an (asymmetric) type. Only one of a symmetric type or an asymmetric type may be allowed as a division type of a coding block.
[116]
The triple tree division type may include at least one of a type of dividing a coding block into two vertical lines or a type of dividing a coding block into two horizontal lines. Three non-square partitions can be created by triple tree partitioning.
[117]
6 shows a triple-tree division type.
[118]
The triple-tree division type may include a type of dividing a coding block into two horizontal lines or a type of dividing a coding block into two vertical lines. The width or height ratio of partitions generated as a result of dividing the coding block may be n:2n:n, 2n:n:n, or n:n:2n.
[119]
The position of the partition having the largest width or height among the three partitions may be predefined in the encoder and decoder. Alternatively, information indicating a partition having the largest width or height among the three partitions may be signaled through a bitstream.
[120]
It is possible to allow only the division of the square shape or the asymmetric shape of the coding unit. In this case, dividing the coding unit into square-shaped partitions corresponds to quad-tree CU partitioning, and dividing the coding unit into symmetrical non-square partitions corresponds to binary tree partitioning. have. Dividing the coding tree unit into square partitions and symmetric non-square partitions may correspond to Quad Tree and Binary Tree CU Partitioning (QTBT).
[121]
Partitioning based on a binary tree or a triple tree may be performed on a coding block for which partitioning based on a quad tree is no longer performed. A coding block generated as a result of dividing based on a binary tree or a triple tree may be divided into smaller coding blocks. In this case, the coding block may be set so that at least one of quad-tree division, triple-tree division, and binary tree division is not applied to the coding block. Alternatively, binary tree division in a predetermined direction or triple tree division in a predetermined direction may not be allowed in the coding block. For example, quad-tree division and triple-tree division may not be allowed in a coding block generated as a result of division based on a binary tree or a triple tree. Only binary tree division may be allowed in the coding block.
[122]
Alternatively, only the coding block having the largest size among the three coding blocks generated as a result of the triple tree-based division may be divided into coding blocks having a smaller size. Alternatively, binary tree-based division or triple tree-based division may be allowed only for a coding block having the largest size among the three coding blocks generated as a result of the triple tree-based division.
[123]
The division type of the lower depth partition may be determined dependently on the division type of the upper depth partition. For example, when an upper partition and a lower partition are partitioned based on a binary tree, only a binary tree based partition having the same type as the binary tree partition type of the upper depth partition may be allowed in the lower depth partition. For example, when the binary tree division type of the upper depth partition is a 2NxN type, the binary tree division type of the lower depth partition may also be set to the 2NxN type. Alternatively, when the binary tree division type of the upper depth partition is an Nx2N type, the division type of the lower depth partition may also be set to an Nx2N type.
[124]
Alternatively, the partition with the largest size among the partitions generated as a result of partitioning based on the triple tree may be configured not to allow binary tree partitioning in the same direction as the partitioning direction of the upper depth partition or triple-tree partitioning in the same direction as the partitioning direction of the upper depth partition. have.
[125]
Alternatively, the partition type of the lower depth partition may be determined in consideration of the partition type of the upper depth partition and the partition type of the neighboring lower depth partition. Specifically, if the upper depth partition is partitioned based on the binary tree, the partitioning type of the lower depth partition may be determined so that the same result as the partitioning of the upper depth partition based on the quad tree does not occur. As an example, when the partition type of the upper depth partition is 2NxN and the partition type of the neighboring lower depth partition is Nx2N, the current partition type of the lower depth partition cannot be set to Nx2N. This is because, when the current sub-depth partition has an Nx2N partition type, the same result as that of dividing the upper depth partition into an NxN type quad tree occurs. When the partition type of the upper depth partition is Nx2N and the partition type of the neighboring lower depth partition is 2NxN, the current partition type of the lower depth partition cannot be set to 2NxN. That is, when the binary tree division type of the upper depth partition and the binary tree division type of the neighboring lower depth partition are different, the current binary tree division type of the lower depth partition may be set to be the same as the binary tree division type of the upper depth partition.
[126]
Alternatively, the binary tree division type of the lower depth partition may be set to be different from the binary tree division type of the upper depth partition.
[127]
In units of sequence, slice, or coding unit, an allowable binary tree division type can be determined. For example, a binary tree division type allowed for a coding tree unit may be limited to a 2NxN or Nx2N type. The allowable split type may be predefined in the encoder or decoder. Alternatively, information on an allowable or disallowed partition may be encoded and signaled through a bitstream.
[128]
7 is a diagram showing an example in which only a specific type of binary tree-based division is allowed.
[129]
FIG. 7A shows an example in which only Nx2N-type binary tree-based division is allowed, and FIG. 7B shows an example in which only 2NxN-type binary tree-based division is allowed.
[130]
In order to represent various types of division, information on quadtree division, information on binary tree division, or information on triple tree division may be used. The information on quad-tree division may include at least one of information indicating whether quad-tree-based division is performed or information on a size/depth of a coding block in which quad-tree-based division is allowed. Information on binary tree division includes information indicating whether or not binary tree-based division is performed, information indicating whether binary tree-based division is vertical or horizontal, and coding blocks in which binary tree-based division is allowed. It may include at least one of information on the size/depth of and information on the size/depth of a coding block in which binary tree-based division is not allowed. The information on triple-tree partitioning includes information indicating whether triple-tree-based partitioning is performed, information indicating whether the triple-tree-based partitioning is in the vertical direction or the horizontal direction, and a coding block in which triple-tree-based partitioning is allowed. It may include at least one of information on the size/depth of the triple tree or information on the size/depth of a coding block in which the triple tree-based division is not allowed. The information on the size of the coding block may indicate a minimum value or a maximum value of at least one of a width, a height, a product of a width and a height, or a width and a height ratio of the coding block.
[131]
For example, when the width or height of the coding block is less than the minimum size allowed for binary tree division, or the division depth of the coding block is greater than the maximum depth allowed for binary tree division, the coding block is based on a binary tree. Splitting may not be allowed.
[132]
For example, when the width or height of the coding block is less than the minimum size allowed for triple tree splitting, or the splitting depth of the coding block is greater than the maximum depth allowed for triple tree splitting, the coding block Splitting may not be allowed.
[133]
Information on the partitioning allowance condition based on a binary tree or a triple tree may be signaled through a bitstream. The information may be encoded in units of a sequence, picture, or fragment image. The fragment image may mean at least one of a slice, a tile group, a tile, a brick, a coding block, a prediction block, or a transform block.
[134]
For example, through the bitstream, the syntax'max_mtt_depth_idx_minus1' indicating the maximum depth in which binary tree/triple tree division is allowed may be encoded/decoded through the bitstream. In this case, max_mtt_depth_idx_minus1+1 may indicate the maximum depth in which binary tree/triple tree division is allowed.
[135]
As an example, at least one of the number of times the binary tree/triple tree division is allowed, the maximum depth that the binary tree/triple tree division is allowed, or the number of depths that the binary tree/triple tree division is allowed is signaled at the sequence or slice level. I can. Accordingly, at least one of the number of binary tree/triple tree division times, the maximum depth allowed for binary tree/triple tree division, or the number of depths allowed for binary tree/triple tree division of the first slice and the second slice may be different. I can. For example, in the first slice, binary tree/triple tree division may be allowed in only one depth, whereas in the second slice, binary tree/triple tree division may be allowed in two depths.
[136]
Referring to the example shown in FIG. 8, in FIG. 8, it is shown that binary tree division is performed on a coding unit having a depth of 2 and a coding unit having a depth of 3. Accordingly, information indicating the number of times (2 times) that the binary tree division in the coding tree unit is performed, information indicating the maximum depth (depth 3) of the partition generated by the binary tree division in the coding tree unit At least one of information indicating the number of partition depths (2, depth 2 and depth 3) to which the division is applied may be encoded/decoded through a bitstream.
[137]
Alternatively, the number of times the binary tree/triple tree division is allowed in the encoder and the decoder, the depth at which the binary tree/triple tree division is allowed, or the number of depths in which the binary tree/triple tree division is allowed may be predefined. Alternatively, based on at least one of the index of the sequence or slice or the size/type of the coding unit, the number of times the binary tree/triple tree division is allowed, the depth at which the binary tree/triple tree division is allowed, or the binary tree/triple tree division is The number of allowed depths may be determined. For example, in a first slice, a binary tree/triple tree division may be allowed in one depth, and a binary tree/triple tree division may be allowed in two depths in a second slice.
[138]
As another example, depending on the temporal level identifier (TemporalID) of a slice or picture, at least one of the number of times a binary tree is allowed to be split, a depth where a binary tree is allowed to be split, or the number of depths where a binary tree is allowed to be split may be differently set. Here, the temporal level identifier (TemporalID) is used to identify each of a plurality of layers of an image having at least one scalability of view, spatial, temporal, or quality. will be.
[139]
As shown in FIG. 4, the first coding block 300 having a split depth of k may be divided into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having half the width and height of the first coding block, and the dividing depth of the second coding block may be increased to k+1.
[140]
The second coding block 310 having a splitting depth of k+1 may be split into a plurality of third coding blocks having a splitting depth of k+2. The division of the second coding block 310 may be performed by selectively using either a quart tree or a binary tree according to a division method. Here, the partitioning method may be determined based on at least one of information indicating partitioning based on a quad tree or information indicating partitioning based on a binary tree.
[141]
When the second coding block 310 is divided based on a quart tree, the second coding block 310 is divided into four third coding blocks 310a having half the width and height of the second coding block, and the third coding block 310a is The splitting depth can be increased to k+2. On the other hand, when the second coding block 310 is divided based on a binary tree, the second coding block 310 may be divided into two third coding blocks. In this case, each of the two third coding blocks is an amorphous block in which one of the width and height of the second coding block is half the size, and the split depth may be increased to k+2. The second coding block may be determined as a horizontal or vertical amorphous block according to the division direction, and the division direction may be determined based on information on whether the binary tree-based division is in the vertical direction or the horizontal direction.
[142]
Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer divided based on a quad tree or a binary tree, and in this case, the corresponding coding block may be used as a prediction block or a transform block.
[143]
Like the division of the second coding block 310, the third coding block 310a may be determined as a terminal coding block or may be additionally divided based on a quad tree or a binary tree.
[144]
Meanwhile, the third coding block 310b divided based on a binary tree may be further divided into a coding block 310b-2 in a vertical direction or a coding block 310b-3 in a horizontal direction based on the binary tree, and the corresponding coding The division depth of a block can be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block 310b-1 that is no longer divided based on a binary tree, and in this case, the corresponding coding block 310b-1 is used as a prediction block or a transform block. I can. However, in the above-described partitioning process, information on the size/depth of a coding block in which quad-tree-based division is allowed, information on the size/depth of a coding block in which binary tree-based division is allowed, or binary tree-based division is allowed. It may be limitedly performed based on at least one of information on the size/depth of a coding block that is not not used.
[145]
The size candidates that a coding block can have may be limited to a predetermined number, or a size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block within a sequence or a size of a coding block within a picture may be limited to have any one of 256x256, 128x128, or 32x32. Information indicating the size of a coding block in a sequence or picture may be signaled through a sequence header or a picture header.
[146]
As a result of the division based on the quad tree and the binary tree, the coding unit may take a square or a rectangle of any size.
[147]
As shown in FIG. 4, the first coding block 300 having a split depth of k may be divided into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having half the width and height of the first coding block, and the dividing depth of the second coding block may be increased to k+1.
[148]
The second coding block 310 having a splitting depth of k+1 may be split into a plurality of third coding blocks having a splitting depth of k+2. The division of the second coding block 310 may be performed by selectively using either a quart tree or a binary tree according to a division method. Here, the partitioning method may be determined based on at least one of information indicating partitioning based on a quad tree or information indicating partitioning based on a binary tree.
[149]
When the second coding block 310 is divided based on a quart tree, the second coding block 310 is divided into four third coding blocks 310a having half the width and height of the second coding block, and the third coding block 310a is The splitting depth can be increased to k+2. On the other hand, when the second coding block 310 is divided based on a binary tree, the second coding block 310 may be divided into two third coding blocks. In this case, each of the two third coding blocks is an amorphous block in which one of the width and height of the second coding block is half the size, and the split depth may be increased to k+2. The second coding block may be determined as a horizontal or vertical amorphous block according to the division direction, and the division direction may be determined based on information on whether the binary tree-based division is in the vertical direction or the horizontal direction.
[150]
Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer divided based on a quad tree or a binary tree, and in this case, the corresponding coding block may be used as a prediction block or a transform block.
[151]
Like the division of the second coding block 310, the third coding block 310a may be determined as a terminal coding block or may be additionally divided based on a quad tree or a binary tree.
[152]
Meanwhile, the third coding block 310b divided based on a binary tree may be further divided into a coding block 310b-2 in a vertical direction or a coding block 310b-3 in a horizontal direction based on the binary tree, and the corresponding coding The division depth of a block can be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block 310b-1 that is no longer divided based on a binary tree, and in this case, the corresponding coding block 310b-1 is used as a prediction block or a transform block. I can. However, in the above-described partitioning process, information on the size/depth of a coding block in which quad-tree-based division is allowed, information on the size/depth of a coding block in which binary tree-based division is allowed, or binary tree-based division is allowed. It may be limitedly performed based on at least one of information on the size/depth of a coding block that is not not used.
[153]
The size candidates that a coding block can have may be limited to a predetermined number, or a size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block within a sequence or a size of a coding block within a picture may be limited to have any one of 256x256, 128x128, or 32x32. Information indicating the size of a coding block in a sequence or picture may be signaled through a sequence header or a picture header.
[154]
As a result of the division based on the quad tree and the binary tree, the coding unit may take a square or a rectangle of any size.
[155]
As a result of partitioning based on a quad tree, a binary tree, or a triple tree, a coding block that is no longer partitioned may be used as a prediction block or a transform block. That is, it can be used as a coding block, a prediction block, or a transform block generated as a result of quad-tree partitioning or binary tree partitioning. For example, a prediction image may be generated in units of coding blocks, and a residual signal, which is a difference between the original image and the predicted image, may be transformed in units of coding blocks. In order to generate a prediction image in units of coding blocks, motion information may be determined based on a coding block or an intra prediction mode may be determined based on a coding block. Accordingly, the coding block may be encoded using at least one of skip mode, intra prediction, and inter prediction.
[156]
Alternatively, a plurality of coding blocks generated by dividing the coding block may be configured to share at least one of motion information, merge candidate, reference sample, reference sample line, and intra prediction mode. For example, when the coding block is divided into a triple tree, the partitions generated by dividing the coding block may select at least one of motion information, merge candidate, reference sample, reference sample line, or intra prediction mode according to the size or shape of the coding block. You can share. Alternatively, only some of the plurality of coding blocks may share the information, and the residual coding block may be set not to share the information.
[157]
As another example, it is possible to divide the coding block and use a prediction block or a transform block having a size smaller than that of the coding block.
[158]
Hereinafter, a method of performing a transform on a coding block or a transform block generated by dividing a coding block will be described in detail. In an embodiment to be described later, the current block may represent a transform block or a coding block as a basic unit in which transform and quantization are performed.
[159]
[160]
9 is a flowchart illustrating a process of encoding a residual sample according to an embodiment to which the present invention is applied.
[161]
The residual sample can be obtained by subtracting the predicted sample from the original sample.
[162]
When the residual sample is obtained, it may be determined whether to skip the transformation for the obtained residual sample (S910). The encoder may encode information indicating whether the transformation for the current block is skipped. As an example,'transform_skip_flag' indicates whether transformation for the current block is skipped. A value of transform_skip_flag of 1 indicates that the transform for the current block is skipped, and a value of transform_skip_flag of 0 indicates that the transform for the current block is not skipped.
[163]
The encoder may encode information for determining the size of a block in which transform skip is allowed. For example,'log2_transform_skip_max_size_minus2' represents the maximum size of a block in which transform skip is allowed. The encoder may encode the transform_skip_flag only when the current block is smaller than or equal to the maximum size allowed for transform skipping.
[164]
Alternatively, it may be determined whether to skip the transformation for each of the horizontal direction and the vertical direction. The encoder may encode at least one of information on whether to skip the transformation in the horizontal direction or information indicating whether to skip the transformation in the vertical direction. For example, at least one of'hor_transform_skip_flag' indicating whether to skip transformation in the horizontal direction or'ver_transform_skip_flag' indicating whether transformation in the vertical direction is skipped may be encoded.
[165]
When it is determined that the transform skip is not applied to the current block, first-order transform may be performed on the current block (S920). The first transformation may be performed based on a transformation core such as DCT or DST. The transform core may be determined based on the transform type of the current block. The encoder may encode information for determining a transform type of the current block. Alternatively, the transform type of the current block may be implicitly derived according to a predetermined rule in the encoder and the decoder. For example, based on at least one of a size of a current block, a shape of the current block, an encoding mode of the current block, or an intra prediction mode of the current block, a transform type of the current block may be determined.
[166]
A quadratic transformation may be performed on the transform coefficient converted by the first order (S930). Only when the first-order transformation is performed on the current block, the second-order transformation may be selectively performed. Whether to perform the quadratic transformation may be determined based on at least one of the size, shape, transformation type, encoding mode, and intra prediction mode of the current block. Alternatively, the encoder may encode information indicating whether to perform the second-order transformation.
[167]
The quadratic transformation may be performed on a partial area of ​​the current block. The location and/or size of the region where the quadratic transformation is performed may be predefined in an encoder and a decoder. Alternatively, information indicating the performing region of the quadratic transformation may be encoded in the bitstream. Alternatively, based on at least one of the size, shape, encoding mode, and intra prediction mode of the current block, the performing region of the quadratic transformation may be determined.
[168]
When it is determined that the transformation for the current block is skipped, the residual sample of the current block may be scaled (S940). The scaling may be performed using a scaling parameter. The scaling parameter may be determined based on the width, height, or size of the current block.
[169]
A transform coefficient generated by transforming the residual coefficient or a scaled residual coefficient generated by scaling the residual coefficient may be quantized (S950). The residual coefficients generated as a result of quantization may be arranged in one dimension, and the residual coefficients may be encoded (S960). The order of arranging the residual coefficients may follow at least one of a diagonal scan, a zigzag scan, an up-right scan, a vertical scan, or a horizontal scan.
[170]
10 is a flowchart illustrating a process of decoding a residual sample according to an embodiment to which the present invention is applied.
[171]
First, a residual coefficient of the current block may be obtained (S1010). The decoder may obtain a residual coefficient through coefficient scanning. The coefficient scanning may be performed based on at least one of a diagonal scan, a zigzag scan, an up-right scan, a vertical scan, or a horizontal scan. The scan order of the current block may be determined based on at least one of a size, a shape, an encoding mode, and an intra prediction mode of the current block. As a result of the coefficient scanning, a residual coefficient in the form of a 2D block may be obtained.
[172]
Inverse quantization may be performed on the residual coefficient of the current block (S1020).
[173]
It may be determined whether to skip the inverse transform in the inverse quantized residual coefficient of the current block (S1030). Information indicating whether the inverse transform for the current block is skipped may be signaled through the bitstream. As an example,'transform_skip_flag' indicates whether the inverse transform for the current block is skipped. A value of transform_skip_flag of 1 indicates that the inverse transform for the current block is skipped, and a value of transform_skip_flag of 0 indicates that the inverse transform of the current block is not skipped.
[174]
Information for determining the size of a block in which skipping of the inverse transformation is permitted may be signaled through a bitstream. As an example,'log2_transform_skip_max_size_minus2' indicates the maximum size of a block in which skipping of inverse transform is allowed. The transform_skip_flag may be encoded only when the current block is smaller than or equal to a maximum size in which inverse transform skip is allowed. When encoding of transform_skip_flag is omitted, the value of transform_skip_flag may be derived as a predefined value. It may be a predefined value of 0. Accordingly, when the size of the current block is larger than the maximum size in which the inverse transform skip is allowed, inverse transform may be performed on the current block.
[175]
Alternatively, it may be determined whether to skip the inverse transform for each of the horizontal and vertical directions. At least one of information indicating whether to skip the inverse transformation in the horizontal direction or information indicating whether to skip the inverse transformation in the vertical direction may be signaled through the bitstream. For example, at least one of'hor_transform_skip_flag' indicating whether to skip the inverse transformation in the horizontal direction or'ver_transform_skip_flag' indicating whether to skip the inverse transformation in the vertical direction may be signaled through the bitstream.
[176]
The hor_transform_skip_flag and the ver_transform_skip_flag may be encoded only when the current block is smaller than a maximum size in which the inverse transform skip is allowed. As an example, hor_transform_skip_flag may be encoded only when the width of the current block is smaller than the maximum size in which the inverse transform skip is allowed. In addition, ver_transform_skip_flag may be encoded only when the height of the current block is smaller than the maximum size in which the inverse transform skip is allowed.
[177]
Alternatively, it may be determined whether to encode the hor_transform_skip_flag and the ver_transform_skip_flag based on the shape of the current block. As an example, hor_transform_skip_flag may be encoded only when the current block is an amorphous block having a width greater than a height. In addition, ver_transform_skip_flag may be encoded only when the height of the current block is an amorphous block larger than the width.
[178]
When encoding of the information is omitted, the value of the information may be derived as a predefined value. It may be a predefined value of 0.
[179]
When it is determined that the inverse transform is performed on the current block, the inverse quantized residual coefficient of the current block may be inverse transformed (S1040 and S1050). The inverse transform may be performed in the order of a second-order inverse transform and a first-order inverse transform. The second-order inverse transform may be performed on a partial region of the current block. The transform matrix for the second-order inverse transform may be determined based on the size of a region to which the second-order inverse transform is applied.
[180]
The second-order inverse transformation can be selectively performed. That is, it is possible to perform the first-order inverse transformation, omitting the second-order inverse transformation. Whether or not the second-order inverse transform is performed may be determined based on at least one of the size and shape of the current block, a transform type encoding mode, or an intra prediction mode. Alternatively, information indicating whether the second-order inverse transformation is performed may be signaled through the bitstream.
[181]
The first inverse transform may be performed based on a transform core such as DCT or DST. The transform core may be determined based on the transform type of the current block. The encoder may encode information for determining a transform type of the current block. Alternatively, the encoder and the decoder may determine the transform type of the current block according to a predetermined rule. For example, based on at least one of a size of a current block, a shape of the current block, an encoding mode of the current block, or an intra prediction mode of the current block, a transform type of the current block may be determined.
[182]
As a result of performing the inverse transformation, a residual sample of the current block may be obtained.
[183]
When it is determined that the inverse transform is skipped in the current block, the inverse quantized residual coefficient of the current block may be scaled (S1060). The scaling may be performed using a scaling parameter. The scaling parameter may be determined based on the width, height, or size of the current block. The residual coefficient of the current block may be obtained by scaling the inverse quantized residual coefficient based on the scaling parameter.
[184]
[185]
Transformation in at least one of a vertical direction or a horizontal direction may be skipped. When the transform skip is applied only in the horizontal direction, the transform may not be performed in the horizontal direction, and the transform may be performed in the vertical direction. When the transformation skip is applied only in the vertical direction, the transformation is not performed in the vertical direction, and the transformation may be performed in the horizontal direction.
[186]
Depending on the division type of the upper node block, it may be determined whether or not a transform skip technique for the current block can be used. For example, when the current block is generated through binary tree-based partitioning, a transform skip technique may not be allowed for the current block. That is, when the current block is generated through binary tree-based division, a residual sample of the current block may be obtained by transforming the inverse quantized residual coefficient. When the current block is generated through binary tree-based division, encoding/decoding of information indicating whether transformation is skipped (e.g., at least one of transform_skip_flag, hor_transform_skip_flag, or ver_transform_skip_flag) may be omitted.
[187]
Alternatively, when the current block is generated through division based on a binary tree, a transform skip technique may be allowed only in a horizontal direction or a vertical direction. The direction in which the transform skip technique is not allowed may be determined based on at least one of information signaled through the bitstream, the size of the current block, the shape of the current block, or an intra prediction mode of the current block. As an example, when the current block has an amorphous shape whose width is greater than the height, the transform skip technique may be allowed only in the vertical direction. That is, when the current block has a 2NxN type, transformation may be performed in a horizontal direction and may be selectively performed in a vertical direction. Alternatively, when the current block is an irregular shape whose height is greater than the width, the transform skip technique may be allowed only in the horizontal direction. That is, when the current block is in the form of Nx2N, transformation may be performed in a vertical direction, and transformation may be selectively performed in a horizontal direction. Contrary to the above example, if the current block is a non-square block whose width is greater than the height, the transform skip technique is allowed only in the horizontal direction, and if the current block is a non-square block whose height is greater than the width, it is converted only in the vertical direction. Skip techniques may be allowed.
[188]
Alternatively, whether to skip transformation of the current block may be determined according to the division type of the upper node block. As an example, when the current block is generated by dividing a binary tree, transformation in a horizontal direction or a vertical direction may be omitted. That is, if the current block is generated by binary tree-based partitioning, the current block is horizontally or vertically directed without encoding/decoding information indicating whether transformation of the current block is skipped (e.g., transform_skip_flag, hor_transform_skip_flag, ver_transform_skip_flag). Transformation for at least one of them may be skipped.
[189]
[190]
Hereinafter, a first-order transform/inverse transform and a second-order transform/inverse transform will be described in detail. Hereinafter, matters described focusing on the first-order transformation and the second-order transformation may be extendedly applied to the first-order inverse transformation and the second-order inverse transformation.
[191]
The first transformation may be performed based on a transformation core such as DCT or DST. As an example, Equation 1 represents a transformation matrix A 4 based on DST-VII . The inverse transformation based on DST-VII may be performed based on A 4 T.
[192]
[Equation 1]

[193]
Equation 2 shows a transform matrix T 8 based on DCT-II . Inverse transformation based on DCT-II may be performed based on T 8 T.
[194]
[Equation 2]

[195]
The transform core of the current block may be determined based on the transform type of the current block. The transform type of the current block may be explicitly determined based on information signaled through the bitstream. The information may be index information indicating a transformation type for a horizontal direction and a transformation type for a vertical direction. Table 1 shows an example in which a horizontal direction transformation type and a vertical direction transformation type are determined based on the index information.
[196]
[Table 1]
Transform set Index HorTrType VerTrType
0 DST-VII DST-VII
One DCT-II DCT-II
2 DCT-VII DCT-II
3 DCT-II DCT-VII
4 DCT-VII DCT-VII
[197]
In Table 1, HorTrType represents a conversion type in the horizontal direction, and VerTrType represents a conversion type in the vertical direction.
[198]
Information indicating whether index information for determining a transformation type in the horizontal direction and the vertical direction is signaled may be encoded. The information may be signaled through a picture parameter set or a sequence parameter set. When the index information is not signaled, a transformation type for a horizontal direction and a vertical direction may be implicitly determined based on at least one of a size, a shape, or an intra prediction mode of a current block.
[199]
For example, when the current block is encoded by inter prediction, transformation types in the horizontal direction and the vertical direction may be determined based on the size or shape of the current block. Specifically, when the current block is an irregular shape having a width greater than the height and the width of the current block is included in a predefined range, the horizontal direction transformation type of the current block may be determined as DCT-II. If the above condition is not satisfied, the horizontal direction conversion type of the current block may be determined as DST-VII. In addition, when the current block is an irregular shape whose height is greater than the width and the height of the current block is included in a predefined range, the vertical direction transformation type of the current block may be determined as DCT-II. If the above condition is not satisfied, the vertical direction transformation type of the current block may be determined as DST-VII. The predefined range may be defined as N or more and/or M or less. N and M are different natural numbers and may be 4, 8, 16 or 32.
[200]
When the current block is encoded by intra prediction, a horizontal direction transformation type and a vertical direction transformation type may be determined based on at least one of a size, shape, or intra prediction mode of the current block. For example, when the intra prediction mode of the current block is the DC mode, the horizontal direction transformation type and the vertical direction transformation type of the current block may be determined as DST-VII.
[201]
When the intra prediction mode of the current block is the Planar mode, the horizontal direction transformation type and the vertical direction transformation type of the current block may be determined based on whether the width and height of the current block are included in a predefined range, respectively. For example, when the width of the current block is included in the predefined range, the horizontal direction transformation type of the current block may be determined as DCT-II. On the other hand, when the width of the current block is not included in the predefined range, the horizontal direction conversion type of the current block may be determined as DST-VII. When the height of the current block is included in the predefined range, the vertical direction transformation type of the current block may be determined as DCT-II. On the other hand, when the height of the current block is not included in the predefined range, the vertical direction transformation type of the current block may be determined as DST-VII.
[202]
When the intra prediction mode of the current block is a directional mode in the first direction, the horizontal direction transform type of the current block may be determined as DST-VII. When the intra prediction mode of the current block is the first directional mode, the vertical direction transformation type of the current block may be determined as DST-VII or DCT-II depending on whether the height of the current block is included in a predefined range. .
[203]
When the intra prediction mode of the current block is the directional mode in the second direction, the horizontal direction transformation type of the current block may be determined as DST-VII or DCT-II depending on whether the width of the current block is included in a predefined range. have. When the intra prediction mode of the current block is the second directional mode, the vertical direction transform type of the current block may be determined as DST-VII. The first direction and the second direction may be different from each other.
[204]
[205]
Alternatively, the transform type of the current block may be determined based on at least one of the size, shape, encoding mode, and intra prediction mode of the current block, but the transform type in the vertical direction and the transform type in the horizontal direction may be set to be the same.
[206]
For example, when the current block is a 4x4 block coded in an intra mode, a DST (specifically, DST-VII) based transformation matrix may be used. If the current block does not satisfy the above condition, a DCT (specifically, DCT-II) based transformation matrix may be used.
[207]
Alternatively, the transform type of the current block may be determined based on at least one of the intra prediction mode of the current block or the number of samples included in the current block. The number of samples for determining the transform type may have a value predefined in the encoder and the decoder. Alternatively, information for determining the number of samples may be signaled through a bitstream. The information may be signaled through a block, slice header, or picture parameter set.
[208]
When the number of samples represents 16, if the current block is encoded in the intra prediction mode and the number of samples included in the current block is 16 or less, a DST-based transform matrix may be applied to the current block. For example, a DST-based transform matrix may be applied to a block having a size of 4x4, 2x8, or 8x2 encoded by intra prediction. On the other hand, when the current block does not satisfy the above condition, a DCT-based transform matrix may be used.
[209]
The conditions for determining the transform type may be set differently for each sequence, slice, or block. For example, in slice 0, when a transform block encoded in an intra mode has a size of 4x4, DST may be selected. On the other hand, in slice 1, when the transform block encoded in the intra mode has a size smaller than 8x8 or 8x8, DST may be selected.
[210]
As another example, based on the transform set, the transform type of the current block may be determined. Index information specifying the transform type of the current block may be signaled through the bitstream. Alternatively, the transform type of the current block may be determined based on at least one of the size, shape, coding mode, and intra prediction mode of the current block. A transform set can be determined for a coding block or a transform block. Accordingly, the coding blocks or transform sets between transform blocks may be different. Alternatively, a plurality of transform blocks may share one transform set. The plurality of transform blocks may be included in a coding block having a predetermined size or a predetermined shape.
[211]
Table 2 illustrates a plurality of transform sets.
[212]
[Table 2]
Transform set Index Transform candidate 0 Transform candidate 1
0 DST-VII DCT-II
One DST-VII DST-I
2 DST-VII DCT-VIII
[213]
The transform set may include at least one transform type candidate. At least one of the number of transform type candidates included in each of the plurality of transform sets or types of transform type candidates may be different. That is, at least one of the number of transform type candidates included in the transform set or the type of transform type candidates may be different from other transform sets.
[214]
Table 1 exemplifies that each transform set includes two transform type candidates. It is also possible to define a transform set including one, three, four or more transform type candidates. Alternatively, transformation type candidate 0 and transformation type candidate 1 may define the same transformation set.
[215]
Alternatively, the maximum number of transform type candidates included in the transform set may be signaled through the bitstream. The number of transform type candidates included in each transform set may be less than or equal to the maximum number signaled through the bitstream. The information may be signaled through a slice header or a sequence header.
[216]
Any one of the transform type candidates included in the transform set may be determined as the transform type of the current block. The transform type of the current block may be determined based on at least one of the size, shape, encoding mode, and intra prediction mode of the current block. The intra prediction mode represents an intra prediction mode of a prediction block or a coding block corresponding to a transform block.
[217]
For example, when the current block is a block of 4x4 size encoded in the intra mode, a transform type candidate 0 in the transform set may be determined as the transform type of the current block. On the other hand, when the current condition does not satisfy the above condition, the transform type candidate 1 in the transform set may be determined as the transform type of the current block.
[218]
As a specific example, when the transform set index 0 is determined as the transform set of the current block, if the current block is a 4x4 size block encoded in the intra mode, transform may be performed based on DST-VII. On the other hand, when the current block does not satisfy the above condition, conversion may be performed based on DCT-II. When transform set index #2 is determined as the transform set of the current block, if the current block is a 4x4 or 8x8-sized block encoded in an intra mode, transform may be performed based on DST-VII. If the current block does not satisfy the above condition, conversion may be performed based on DCT-VIII.
[219]
The selection condition for the transform type candidate may be determined based on at least one of the size, shape, coding mode, and intra prediction mode of the coding block. For example, when the size of the coding block is less than or equal to 32x32, the transform type candidate 0 may be applied only when the transform block is encoded in the intra mode and the size of the transform block is 4x4. On the other hand, when the size of the coding block is larger than 32x32, the transform type candidate 0 can be applied only when the transform block is encoded in the intra mode and the size of the transform block is 8x8 or less.
[220]
Alternatively, information for specifying any one of a plurality of transform type candidates may be signaled through a bitstream.
[221]
Transformation skip may be set as a transform type candidate. Whether transform skip can be used as a transform type candidate may be determined based on whether transform skip is allowed in the current block. Depending on whether or not the transform skip can be used as a transform skip candidate, at least one of the types of transform type candidates included in the transform set or the number of transform type candidates may be differently set.
[222]
Table 3 shows a plurality of transform sets in which transform skip is set as a transform skip candidate.
[223]
[Table 3]
Transform set Index Transform candidate 0 Transform candidate 1 Transform candidate 2
0 DST-VII DCT-II Transform skip
One DST-VII DST-I Transform skip
2 DST-VII DCT-VIII Transform skip
[224]
When transform_skip_enabled_flag indicating whether or not to allow transform skip within a given picture is 1, as shown in Table 3, a transform set further including transform skip as a transform type candidate may be used. On the other hand, when transform_skip_enabled_flag is 0, as shown in Table 2, a transform set that does not include transform skip as a transform type candidate may be used.
[225]
The transformation type for the horizontal direction and the transformation type for the vertical direction of the current block may be set equally. For example, when the transform type of the current block is determined, the determined transform type may be applied to both the determined horizontal transform and the vertical transform. Alternatively, a transformation type for a horizontal direction and a transformation type for a vertical direction of the current block may be set differently. For example, when a transform set of the current block is determined, a transform type candidate 0 may be determined as a transform type for a horizontal direction, and a transform type candidate 1 may be determined as a transform type for a vertical direction.
[226]
As another example, a transform set for a horizontal direction and a transform set for a vertical direction may be separately determined. To this end, a first transform set index for specifying a transform set in the horizontal direction and a second transform set index for specifying a transform set in the vertical direction may be signaled through the bitstream. Any one of the transform type candidates included in the first transform set specified by the first transform set index may be determined as a transform type in the horizontal direction. Any one of the transform type candidates included in the second transform set specified by the second transform set index may be determined as a transform type in the vertical direction.
[227]
As an example, it is assumed that the transform set corresponding to the transform set index 0 is determined as a transform set in the horizontal direction, and the transform set corresponding to the transform set index 1 is determined as the transform set in the vertical direction. When the current block is not encoded by intra prediction or the size of the current block is not 4x4, the transform type candidate 1 may be determined as the transform type of the current block. That is, as for the transformation type of the vertical transformation, transformation type candidate 1 (ie, DCT-II) included in transformation set index 0 is determined as the transformation type in the horizontal direction, and transformation type candidate 1 included in transformation set index 1 (ie, DST-I) may be determined as a vertical transformation type.
[228]
The first transform set index and the second transform set index may represent the same transform set. Alternatively, the second transform set index may specify one of residual transform sets excluding the transform set specified by the first transform set index.
[229]
Whether the transform set in the horizontal direction and the transform set in the vertical direction are the same may be determined based on at least one of the size, shape, encoding mode, and intra prediction mode of the current block. For convenience of explanation, a transform set for a horizontal transform is defined as a horizontal direction transform set, and a transform set for a vertical transform is defined as a vertical transform set. For example, when the intra prediction mode of the current block is an intra prediction mode similar to a horizontal direction or an intra prediction mode similar to a vertical direction, the horizontal direction transform set and the vertical direction transform set may be different from each other. Here, the intra prediction mode similar to the horizontal direction may mean an intra prediction mode in which a mode value difference from the horizontal direction or the horizontal direction intra prediction mode is within a predefined value. In addition, the intra prediction mode similar to the vertical direction may mean an intra prediction mode in which a difference in a mode value from the intra prediction mode in the vertical direction or the vertical direction is within a predefined value.
[230]
On the other hand, when the intra prediction mode of the current block is a non-directional mode or a directional mode that does not satisfy the above condition, the horizontal direction transform set and the vertical direction transform set may be the same. When the horizontal direction transformation set and the vertical direction transformation set are set to be the same, at least one of the first index information for specifying the horizontal direction transformation set or the second index information for specifying the vertical direction transformation set may be omitted. have.
[231]
Alternatively, when the intra prediction mode of the current block is a non-directional mode, it is possible to set the horizontal direction transform set and the horizontal transform transform set of the current block to be different from each other.
[232]
11 is a diagram showing whether a vertical direction transform set and a horizontal direction transform set are the same for 33 intra prediction modes.
[233]
In the example shown in FIG. 11, when the intra prediction mode of the current block is included in the range 7-13 or 23-29, the horizontal direction transform set and the vertical direction transform set are shown to be different from each other. On the other hand, when the intra prediction mode of the current block is a directional mode not included in the range, it is shown that the horizontal direction transform set and the vertical direction transform set are the same.
[234]
When a block having the same intra prediction mode as the current block in a predetermined block exists, the transform set of the current block may be set to be the same as the transform set of the block having the same intra prediction mode as the current block. Here, the predetermined block may be a coding block, a coding tree block, or a block having a predetermined size.
[235]
As an example, it is assumed that the intra prediction mode corresponding to the first transform block in the coding block is in the vertical direction (eg, mode number 26), the horizontal direction transform set is index 2, and the vertical direction transform set is index 0. Here, the order of the transform blocks may be determined according to the scan order of the coding blocks.
[236]
When a transform block having a vertical intra prediction mode exists in the coding block, a transform set index value may not be signaled to the newly scanned transform block. Instead, a transform set of a transform block having an intra prediction mode in a vertical direction may be applied to the transform set of the newly scanned transform block. That is, the horizontal direction transform set and the vertical direction transform set of the newly scanned transform block may be determined as index 2 and index 0, respectively.
[237]
As another example, when a block having an intra prediction mode similar to a current block in a predetermined block exists, a transform set of the current block may be set to be the same as a transform set of a block having an intra prediction mode similar to the current block. Here, the intra prediction mode similar to the current block may refer to a reference intra prediction mode or an intra prediction mode in which a mode value difference from the reference intra prediction mode is less than or equal to a predefined value. The reference intra prediction mode may include an intra prediction mode in a horizontal direction or an intra prediction mode in a vertical direction.
[238]
As an example, it is assumed that the intra prediction mode corresponding to the first transform block in the coding block is in the vertical direction (eg, mode number 26), the horizontal direction transform set is index 2, and the vertical direction transform set is index 0.
[239]
When a transform block having an intra prediction mode similar to the vertical direction in the coding block (eg, mode number 27) exists, a transform set index value may not be signaled to the newly scanned transform block. Instead, a transform set of a transform block having an intra prediction mode similar to that of the current block may be applied to the transform set of the newly scanned transform block. That is, the horizontal direction transform set of the newly scanned transform block may be determined as index 2, and the vertical direction transform set may be determined as index 0.
[240]
Based on the intra prediction mode of the current block, a transform set for the horizontal direction and/or a transform set for the vertical direction may be determined. Table 4 shows a horizontal direction transform set and a vertical direction transform set according to the intra prediction mode of the current block.
[241]
[Table 4]
Intra Mode 0 One 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
H 2 One 0 One 0 One 0 One 0 0 0 0 0 One 0 One 0 One
V One One 0 One 0 One 0 One 2 2 2 2 2 One 0 One 0 One
Intra Mode 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
H 0 One 0 One 0 One 2 2 2 2 2 One 0 One 0 One 0
V 0 One 0 One 0 One 0 0 0 0 0 One 0 One 0 One 0
[242]
When the current block is encoded by inter prediction, the current block may use a predefined transform set. The predefined transform set may be a transform set having index 0.
[243]
Alternatively, when the coding block is encoded by inter prediction, a transform set may be selected for the coding block, and transform blocks within the coding block may use transform type candidates included in the transform set of the coding block. In this case, the transform type of each transform block may be determined by the size and shape of the transform block, or information for identifying the transform type selected by each transform block may be signaled through a bitstream.
[244]
Selecting a transform type of the current block from among a plurality of transform type candidates may be defined as an adaptive multiple transform (AMT). Whether the adaptive multi-transformation is applied to the current block may be determined based on at least one of a size, a shape, an encoding mode, and an intra prediction mode of the current block. Information for determining at least one of a size or a shape of a block in which adaptive multiple transformation is allowed may be signaled through a bitstream. The size information may include at least one of a maximum size or a minimum size in which multiple transformation is allowed. Information for determining and/or the size of a block in which multiple transformation is allowed may be signaled through at least one of a block, a slice header, or a sequence header.
[245]
[246]
Next, let's look at the quadratic transformation.
[247]
The encoder may perform first-order transformation on the current block and then perform second-order transformation. After performing the second-order inverse transform on the current block, the decoder may perform the first-order inverse transform.
[248]
Whether the second-order transform is applied to the current block may be determined based on at least one of a size, a shape, a transform type, and a quantization parameter size of the current block. For example, when at least one of the width or height of the current block is smaller than a predefined value, second-order transformation may not be performed on the current block.
[249]
The second-order transformation may be performed on the entire area of ​​the current block or a partial area of ​​the current block. 12 is a diagram showing a performing area of ​​a quadratic transformation.
[250]
While the first transformation is performed on the entire area of ​​the current block, the second transformation may be performed on a partial area of ​​the current block.
[251]
The location and/or size of the region where the quadratic transformation is performed may be predefined in an encoder and a decoder. For example, the area may have a size of MxN, and a location of the upper left corner may coincide with the upper left corner of the current block. M and N can be 2, 4, 8 or 16. M and N may have the same value or different values. As an example, a quadratic transformation may be applied to a block having a size of 4x4.
[252]
Alternatively, the location and/or size of the region where the quadratic transformation is performed may be determined based on at least one of a size, a shape, a transform type, a quantization parameter size, or whether a residual flipping is performed. . For example, when at least one of the height or width of the current block is equal to or smaller than the threshold value, quadratic transformation may be performed on a 4x4 block. When the height and width of the current block are greater than the threshold value, quadratic transformation may be performed on the 8x8 block. Here, the threshold value may be a natural number such as 4, 8, or 16.
[253]
Alternatively, information for determining the location and/or size of the region in which the quadratic transformation is performed may be signaled through the bitstream. The information may be signaled through slices, pictures, or sequences.
[254]
The second transform may be a separable transform or a non-separable transform. The separable transformation and the non-separable transformation may be classified according to whether the transformation for the horizontal direction and the transformation for the vertical direction are performed separately when performing the second order transformation.
[255]
Specifically, the separable transformation means that transformation in the horizontal direction and transformation in the vertical direction are separately performed for a block of MxN size. Non-separable transform means that horizontal and vertical transforms are not separated for MxN-sized blocks.
[256]
The non-separable transform may be performed based on a one-dimensional matrix in which a transform matrix T and transform coefficients included in a block having a size of MxN are arranged in one dimension. Specifically, a (MxN)x1 matrix is ​​created by arranging transform coefficients included in a block of MxN size in one dimension, and a transform coefficient of size Kx1 can be generated by multiplying the transform matrix T of size Kx(MxN) by the one-dimensional matrix. have. The decoder can perform a second-order inverse transform using the transform matrix T T.
[257]
The transform matrix T may be determined based on the quadratic transform type. Information for determining the second-order transformation type may be signaled through a bitstream. Alternatively, the second-order transform type may be determined based on at least one of a size, a width, an encoding mode, and an intra prediction mode of the current block.
[258]
Equation 3 shows transform coefficients included in a 4x4 block, and Equation 4 shows an example in which transform coefficients included in a 4x4 block are arranged in one dimension.
[259]
[Equation 3]

[260]
[Equation 4]

[261]
A 16x1 matrix

can be obtained by arranging transform coefficients included in a 4x4 block in a line . The order of arrangement of transform coefficients may follow the scan order of the current block. That is, the arrangement order of the transform coefficients may follow at least one of a vertical scan, a horizontal scan, a raster scan, a zigzag scan, or a diagonal scan.
[262]
Equation 5

shows an example in which a quadratic transformation is performed based on a 1D matrix and a transformation matrix T.
[263]
[Equation 5]

[264]
When the size of the transform matrix T is 16x16, a matrix

having a size of 16x1 may be generated as a result of the quadratic transform .
[265]
Performing non-separable transform on only some regions of the transform block may be referred to as sub-non-separable transform. Equation 6 shows the sub-non-separable transform.
[266]
[Equation 6]

[267]
T'denotes a transform matrix, and Xi' denotes a matrix in which transform coefficients included in a sub-block are arranged in one dimension. Fi' represents a non-separable transform coefficient generated as a result of performing the non-separable transform. The non-separable transform coefficient of a region where non-separable transform is not performed may be set to a predefined value. The predefined value may be an integer including 0.
[268]
The size of each matrix may be determined based on the size of a subblock on which non-separable transformation is performed. As an example, it is assumed that non-separable transform is performed only in a region corresponding to a 2x2 size subblock among a 4x4 size transform block. Xi' represents a 4x1 matrix in which transform coefficients included in a 2x2 subblock are arranged in one dimension, and T'represents a 16x4 transform matrix. Fi' denotes a 16x1 transform coefficient matrix generated as a result of performing the non-separable transform.
[269]
As an example, it is assumed that non-separable transform is performed only in a region corresponding to an 8x4 size subblock among 8x8 size transform blocks. Xi' represents a 32x1 matrix in which transform coefficients included in an 8x4 subblock are arranged in one dimension, and T'represents a 64x32 transform matrix. Fi' represents a transform coefficient matrix having a size of 64x1 generated as a result of performing the non-separable transform.
[270]
As an example, it is assumed that non-separable transform is performed only in a region corresponding to a 4x4 size subblock among 8x8 size transform blocks. Xi' represents a 16x1 matrix in which transform coefficients included in a 4x4 subblock are arranged in one dimension, and T'represents a 64x16 transform matrix. Fi' represents a transform coefficient matrix having a size of 64x1 generated as a result of performing the non-separable transform.
[271]
The area in which the quadratic transformation is performed does not necessarily have to be square. For example, a 2x8 or 8x2 amorphous area or a polygonal area in which three blocks of 4x4 size are combined may be determined as a second-order transform performing area.
[272]
The size or area of ​​the area where the quadratic transformation is performed may be determined according to the size, shape, encoding mode, or intra prediction mode of the current block. For example, when the current block is a square, the area in which the second order transformation is performed may be set to a square. When the current block is a non-square, the area in which the quadratic transformation is performed may be set to a non-square.
[273]
The non-separable transformation may consist of a plurality of sub-non-separable transformations. Equation 7 shows a plurality of non-separable transforms.
[274]
[Equation 7]

[275]
In Equation 7, X'n represents a matrix in which transform coefficients included in the N-th subblock are arranged in one dimension. A non-separable transform coefficient Fn' for the N-th sub-block may be generated through sub-non-separable transform.
[276]
The set of non-separable transform coefficients of each sub-block may be set as the non-separable transform coefficient of the current block. Equation 8 shows the non-separable transform coefficient F of the current block.
[277]
[Equation 8]

[278]
The size of T of the transform matrix may be determined based on the size of a region to which the quadratic transform is applied. For example, when a quadratic transform is applied to a 4x4 block, a 16x16 transform matrix may be used, and when a quadratic transform is applied to an 8x8 block, a 64x64 transform matrix may be used.
[279]
Alternatively, a reduced transform matrix in which the number of rows or columns is smaller than the number of samples included in a region to which the quadratic transformation is applied may be used. For example, for a 4x4 block, quadratic transformation may be performed using a 4x16 reduced transform matrix instead of a 16x16 transform matrix. When the reduced transform matrix is ​​used, four transform coefficients may be output as a result of performing the quadratic transform. The four samples may be mapped to a block having a size of 2x2 at the upper left of the block, and a transform coefficient of the residual region may be set to 0.
[280]
Alternatively, for an 8x8 block, quadratic transformation may be performed using a 16x64 reduced transform matrix instead of a 64x64 transform matrix. When the reduced transform matrix is ​​used, 16 transform coefficients may be output as a result of performing the quadratic transform. The 16 samples may be mapped to a block having a size of 4x4 at the upper left of the block, and a transform coefficient of the residual region may be set to 0.
[281]
Information for determining the size of the reduced transform matrix may be signaled through the bitstream. The information may indicate a reducing parameter. The number of columns or rows of the reduced transformation matrix may be determined by scaling the number of samples included in the performing region of the quadratic transformation using a reduction factor.
[282]
Whether or not the reduced transform matrix is ​​applied may be determined based on at least one of the size, shape, coding mode, intra prediction mode, and transform mode of the current block. For example, when at least one condition of the current block and/or the height is greater than or equal to a threshold value or when the current block is encoded by intra prediction, a quadratic transformation based on the reduced transform matrix may be allowed.
[283]
Alternatively, information indicating whether or not the reduced transform matrix is ​​allowed may be signaled through the bitstream.
[284]
In the above-described embodiment, it has been described that residual coefficients are encoded in the order of first-order transformation, second-order transformation, and quantization. Unlike the described example, the residual coefficients may be encoded in the order of first-order transformation, quantization, and second-order transformation.
[285]
[286]
It is within the scope of the present invention to apply the embodiments described centering on the decoding process or the encoding process to the encoding process or the decoding process. It is also within the scope of the present invention to change the embodiments described in a predetermined order in a different order from those described.
[287]
The above-described embodiment has been described based on a series of steps or flow charts, but this does not limit the time series order of the invention, and may be performed simultaneously or in a different order as necessary. In addition, each of the components (eg, units, modules, etc.) constituting the block diagram in the above-described embodiment may be implemented as a hardware device or software, or a plurality of components are combined to form a single hardware device or software. It can also be implemented. The above-described embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-optical media such as floptical disks. media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.
Industrial availability
[288]
The present invention can be applied to an electronic device capable of encoding/decoding an image.
Claims
[Claim 1]
Obtaining residual coefficients of the current block; Inverse quantizing the residual coefficients; Performing a second-order inverse transform on the inverse quantized residual coefficients; And performing a first-order inverse transform on a result of performing the second-order inverse transform, wherein the second-order inverse transform is performed for a partial region of the current block.
[Claim 2]
The image decoding method of claim 1, wherein the second-order inverse transform is performed using a transform matrix and a one-dimensional matrix generated by arranging inverse quantized residual coefficients included in the partial region in one dimension.
[Claim 3]
The image decoding method of claim 2, wherein the number of rows or columns of the transformation matrix is ​​smaller than the number of samples included in the partial region.
[Claim 4]
The method of claim 1, wherein the size or shape of the partial area is determined based on the size or shape of the current block.
[Claim 5]
The method of claim 1, wherein a transform type for the first-order inverse transform is determined based on index information signaled through a bitstream.
[Claim 6]
The method of claim 5, wherein the index information specifies any one of a plurality of transform sets, and a first transform type candidate included in the transform set is determined as a horizontal transform type of the current block, and the transform set The included second transform type candidate is determined as a vertical transform type of the current block.
[Claim 7]
The method of claim 1, wherein a transform type for the first-order inverse transform is determined based on a result of comparing a width of the current block and a height of the current block.
[Claim 8]
Performing a first order transformation on the residual samples; Performing a second order transformation on the first order transform coefficients generated as a result of the first order transformation; Quantizing quadratic transform coefficients generated as a result of the quadratic transformation; And encoding quantized transform coefficients, wherein the second-order transform is performed on a partial region of the current block.
[Claim 9]
The image encoding method of claim 8, wherein the second-order transform is performed using a transform matrix and a one-dimensional matrix generated by arranging the first-order transform coefficients included in the partial region in one dimension.
[Claim 10]
The image encoding method of claim 9, wherein the number of rows or columns of the transformation matrix is ​​smaller than the number of samples included in the partial region.
[Claim 11]
The method of claim 8, wherein the size or shape of the partial area is determined based on the size or shape of the current block.
[Claim 12]
9. The method of claim 8, wherein index information indicating a transform type for the first-order transform is encoded in a bitstream.
[Claim 13]
The method of claim 12, wherein the index information specifies any one of a plurality of transform sets, and a first transform type candidate included in the transform set is determined as a horizontal transform type of the current block, and the transform set The included second transform type candidate is determined as a vertical transform type of the current block.
[Claim 14]
The method of claim 8, wherein a transform type for the first-order transform is determined based on a result of comparing a width of the current block and a height of the current block.
[Claim 15]
A decoding unit that decodes residual coefficients of the current block; An inverse quantization unit for inverse quantization of the residual coefficients; And an inverse transform unit performing a second-order inverse transform on the inverse quantized residual coefficients, and performing a first-order inverse transform on a result of the second-order inverse transform, wherein the second-order inverse transform targets a partial region of the current block A video decoding apparatus, characterized in that performed.