Title of invention: Video signal processing method and apparatus
Technical field
[One]
The present invention relates to a video signal processing method and apparatus.
Background
[2]
Recently, demand for high-resolution and high-quality images such as high definition (HD) images and ultra high definition (UHD) images is increasing in various application fields. The higher the resolution and quality of the video data, the higher the amount of data is compared to the existing video data. Therefore, when the video data is transmitted using a medium such as a wired/wireless broadband line or stored using an existing storage medium, the transmission cost and The storage cost will increase. High-efficiency image compression techniques can be used to solve these problems that occur as image data becomes high-resolution and high-quality.
[3]
An inter-screen prediction technology that predicts pixel values ​​included in the current picture from a picture before or after the current picture using video compression technology, an intra-screen prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, Various technologies exist, such as an entropy encoding technology that allocates a short code to a value with a high frequency of appearance and a long code to a value with a low frequency of appearance, and by using such an image compression technology, image data can be effectively compressed and transmitted or stored.
[4]
Meanwhile, as the demand for high-resolution images increases, the demand for 3D image contents as a new image service is also increasing. Discussions are underway on video compression techniques for effectively providing 3D image contents of high resolution and ultra high resolution.
Detailed description of the invention
Technical challenge
[5]
An object of the present invention is to provide a method and apparatus capable of efficiently performing inter prediction on an encoding/decoding target block in encoding/decoding a video signal.
[6]
An object of the present invention is to provide a method and apparatus for performing motion compensation using a plurality of merge candidate lists in encoding/decoding a video signal.
[7]
An object of the present invention is to provide a method and apparatus for efficiently encoding/decoding a merge index in encoding/decoding a video signal.
[8]
The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems that are not mentioned will be clearly understood by those of ordinary skill in the technical field to which the present invention belongs from the following description. I will be able to.
Means of solving the task
[9]
In the video signal decoding method and apparatus according to the present invention, merge candidates are derived from neighboring blocks adjacent to a current block, a first merge candidate list including the merge candidates is generated, and included in the first merge candidate list. Information for specifying any one of the merge candidates may be decoded, and motion information of the current block may be derived from a merge candidate to which an index determined by the information is assigned. In this case, when the number of merge candidates included in the first merge candidate list is smaller than a predetermined value, a merge candidate included in the second merge candidate list may be added to the first merge candidate list.
[10]
The method and apparatus for encoding a video signal according to the present invention induce merge candidates from neighboring blocks adjacent to a current block, generate a first merge candidate list including the merge candidates, and include in the first merge candidate list. Information for specifying any one of the merge candidates may be encoded, and motion information of the current block may be derived from a merge candidate to which an index determined by the information is assigned. In this case, when the number of merge candidates included in the first merge candidate list is smaller than a predetermined value, a merge candidate included in the second merge candidate list may be added to the first merge candidate list.
[11]
In the video signal encoding/decoding method and apparatus according to the present invention, the information may include an index prefix and an index suffix.
[12]
In the method and apparatus for encoding/decoding a video signal according to the present invention, when a value of the index prefix is ​​smaller than a threshold value, the index may be set equal to the index prefix.
[13]
In the method and apparatus for encoding/decoding a video signal according to the present invention, when the value of the index prefix is ​​equal to or greater than the threshold value, the index is obtained by adding the value of the index suffix to a value derived based on the index prefix. Can be induced.
[14]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the threshold value may be determined based on the number of merge candidates included in the first merge candidate list.
[15]
In the video signal encoding/decoding method and apparatus according to the present invention, the second merge candidate list may include merge candidates derived from blocks not adjacent to the current block.
[16]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the non-adjacent block may be placed on the same line as a block adjacent to the current block.
[17]
The features briefly summarized above with respect to the present invention are merely exemplary aspects of the detailed description of the present invention to be described later, and do not limit the scope of the present invention.
Effects of the Invention
[18]
According to the present invention, the inter prediction efficiency can be improved by performing motion compensation using a plurality of merge candidate lists.
[19]
According to the present invention, inter prediction efficiency can be improved by obtaining motion information based on a plurality of merge candidates.
[20]
According to the present invention, it is possible to provide an efficient encoding/decoding method of a merge index.
[21]
The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned can be clearly understood by those of ordinary skill in the art from the following description. will be.
Brief description of the drawing
[22]
1 is a block diagram showing an image encoding apparatus according to an embodiment of the present invention.
[23]
2 is a block diagram showing an image decoding apparatus according to an embodiment of the present invention.
[24]
3 is a diagram illustrating partition mode candidates that can be applied to the coding block when the coding block is encoded by inter prediction.
[25]
4 illustrates an example of hierarchically partitioning a coding block based on a tree structure as an embodiment to which the invention is applied.
[26]
5 is a diagram showing a partition type in which partitioning based on a binary tree is allowed as an embodiment to which the present invention is applied.
[27]
6 shows a triple-tree division type.
[28]
7 is a diagram showing an example in which only a specific type of binary tree-based division is allowed.
[29]
FIG. 8 is a diagram for explaining an example in which information related to an allowable number of binary tree divisions is encoded/decoded as an embodiment to which the present invention is applied.
[30]
9 is a flowchart illustrating an inter prediction method according to an embodiment to which the present invention is applied.
[31]
10 is a diagram illustrating a process of deriving motion information of a current block when a merge mode is applied to a current block.
[32]
11 is a diagram illustrating an example of a spatial neighboring block.
[33]
12 is a diagram for describing an example of deriving a motion vector of a temporal merge candidate.
[34]
13 is a diagram illustrating locations of candidate blocks that can be used as collocated blocks.
[35]
14 is a diagram illustrating a process of deriving motion information of a current block when the AMVP mode is applied to the current block.
[36]
15 is a diagram illustrating an example of deriving a merge candidate from a second merge candidate block when the first merge candidate block is not available.
[37]
16 is a diagram illustrating an example of deriving a merge candidate from a second merge candidate block located on the same line as a first merge candidate block.
[38]
17 to 20 are diagrams illustrating a search order of merge candidate blocks.
[39]
21 is a diagram illustrating an example in which a merge candidate of an amorphous block is derived based on a square block.
[40]
22 is a diagram illustrating an example of deriving a merge candidate based on an upper node block.
[41]
23 is a diagram for describing an example in which availability of a spatial neighboring block is determined based on a merge induction region.
[42]
24 is a diagram illustrating an example in which a merge candidate is derived based on a merge induction region.
Mode for carrying out the invention
[43]
In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements.
[44]
Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from other components. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.
[45]
When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.
[46]
The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements, numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance the possibility.
[47]
Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same elements in the drawings, and duplicate descriptions for the same elements are omitted.
[48]
[49]
1 is a block diagram showing an image encoding apparatus according to an embodiment of the present invention.
[50]
Referring to FIG. 1, the image encoding apparatus 100 includes a picture splitter 110, a prediction unit 120, 125, a transform unit 130, a quantization unit 135, a rearrangement unit 160, and an entropy encoder ( 165, an inverse quantization unit 140, an inverse transform unit 145, a filter unit 150, and a memory 155.
[51]
Each of the components shown in FIG. 1 is shown independently to represent different characteristic functions in an image encoding apparatus, and does not mean that each component is formed of separate hardware or a single software component. That is, each constituent part is listed and included as a respective constituent part for convenience of explanation, and at least two constituent parts of each constituent part are combined to form one constituent part, or one constituent part may be divided into a plurality of constituent parts to perform a function. Integrated embodiments and separate embodiments of the components are also included in the scope of the present invention unless departing from the essence of the present invention.
[52]
In addition, some of the components are not essential components that perform essential functions in the present invention, but may be optional components only for improving performance. The present invention can be implemented by including only components essential to implement the essence of the present invention excluding components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.
[53]
The picture dividing unit 110 may divide the input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitter 110 divides a picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, a prediction unit, and a transformation unit based on a predetermined criterion (for example, a cost function). Select to encode the picture.
[54]
For example, one picture may be split into a plurality of coding units. In order to split the coding units in a picture, a recursive tree structure such as a quad tree structure can be used. Encoding that is split into other coding units based on one image or the largest coding unit as a root. A unit may be divided with as many child nodes as the number of divided coding units. Coding units that are no longer split according to certain restrictions become leaf nodes. That is, when it is assumed that only square splitting is possible for one coding unit, one coding unit may be split into up to four different coding units.
[55]
Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit that performs encoding or a unit that performs decoding.
[56]
The prediction unit may be split in a shape such as at least one square or rectangle of the same size within one coding unit, or one prediction unit among prediction units split within one coding unit is another prediction. It may be divided to have a shape and/or size different from the unit.
[57]
When a prediction unit that performs intra prediction based on a coding unit is not a minimum coding unit, intra prediction may be performed without dividing into a plurality of prediction units NxN.
[58]
The prediction units 120 and 125 may include an inter prediction unit 120 that performs inter prediction and an intra prediction unit 125 that performs intra prediction. It is possible to determine whether to use inter prediction or to perform intra prediction for the prediction unit, and determine specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. In this case, a processing unit in which prediction is performed may be different from a processing unit in which a prediction method and specific content are determined. For example, a prediction method and a prediction mode are determined in a prediction unit, and prediction may be performed in a transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130. In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoder 165 together with a residual value and transmitted to a decoder. In the case of using a specific encoding mode, it is possible to encode an original block as it is and transmit it to a decoder without generating a prediction block through the prediction units 120 and 125.
[59]
The inter prediction unit 120 may predict a prediction unit based on information of at least one picture of a picture before or after a current picture, and in some cases, predict based on information of a partial region in the current picture that has been encoded. You can also predict units. The inter prediction unit 120 may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.
[60]
The reference picture interpolation unit may receive reference picture information from the memory 155 and may generate pixel information of an integer number of pixels or less from the reference picture. In the case of a luminance pixel, a DCT-based 8-tap interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/4 pixels. In the case of a color difference signal, a DCT-based interpolation filter with different filter coefficients may be used to generate pixel information of an integer pixel or less in units of 1/8 pixels.
[61]
The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolation unit. Various methods, such as a full search-based block matching algorithm (FBMA), a three step search (TSS), and a new three-step search algorithm (NTS), can be used as a method for calculating a motion vector. The motion vector may have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixels. The motion prediction unit may predict the current prediction unit by differently predicting the motion. Various methods such as a skip method, a merge method, an advanced motion vector prediction (AMVP) method, and an intra block copy method may be used as the motion prediction method.
[62]
The intra predictor 125 may generate a prediction unit based on reference pixel information around a current block, which is pixel information in the current picture. If the neighboring block of the current prediction unit is a block that has performed inter prediction and the reference pixel is a pixel that has performed inter prediction, the reference pixel included in the block that has performed inter prediction is a reference pixel of the block that has performed intra prediction Can be used in place of information. That is, when the reference pixel is not available, information on the reference pixel that is not available may be replaced with at least one reference pixel among the available reference pixels.
[63]
In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which directional information is not used when prediction is performed. A mode for predicting luminance information and a mode for predicting color difference information may be different, and intra prediction mode information or predicted luminance signal information used to predict luminance information may be used to predict color difference information.
[64]
When performing intra prediction, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is based on a pixel on the left, a pixel on the top left, and a pixel on the top of the prediction unit. Can be done. However, when the size of the prediction unit and the size of the transformation unit are different when performing intra prediction, intra prediction may be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN splitting may be used for only the smallest coding unit.
[65]
The intra prediction method may generate a prediction block after applying an AIS (Adaptive Intra Smoothing) filter to a reference pixel according to a prediction mode. The types of AIS filters applied to the reference pixels may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When predicting the prediction mode of the current prediction unit using the mode information predicted from the surrounding prediction units, if the intra prediction modes of the current prediction unit and the surrounding prediction units are the same, the current prediction unit and the surrounding prediction units are used using predetermined flag information. Information indicating that the prediction mode of is the same can be transmitted, and if the prediction modes of the current prediction unit and the neighboring prediction units are different, entropy encoding is performed to encode prediction mode information of the current block.
[66]
Also, a residual block including a prediction unit that performs prediction based on a prediction unit generated by the prediction units 120 and 125 and residual information that is a difference value from the original block of the prediction unit may be generated. The generated residual block may be input to the transform unit 130.
[67]
In the transform unit 130, the original block and the residual block including residual information of the prediction unit generated through the prediction units 120 and 125 are converted to DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT. It can be converted using the same conversion method Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information of a prediction unit used to generate the residual block.
[68]
The quantization unit 135 may quantize values ​​converted into the frequency domain by the transform unit 130. Quantization coefficients may vary depending on the block or the importance of the image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160.
[69]
The reordering unit 160 may rearrange coefficient values ​​on the quantized residual values.
[70]
The rearrangement unit 160 may change the two-dimensional block shape coefficient into a one-dimensional vector shape through a coefficient scanning method. For example, the rearrangement unit 160 may scan from a DC coefficient to a coefficient in a high frequency region using a Zig-Zag Scan method, and change it into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of zig-zag scan, a vertical scan that scans a two-dimensional block shape coefficient in a column direction and a horizontal scan that scans a two-dimensional block shape coefficient in a row direction may be used. That is, according to the size of the transform unit and the intra prediction mode, it is possible to determine which scan method is to be used among zig-zag scan, vertical direction scan, and horizontal direction scan.
[71]
The entropy encoding unit 165 may perform entropy encoding based on values ​​calculated by the rearrangement unit 160. Entropy coding may use various coding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC).
[72]
The entropy encoder 165 includes residual value coefficient information and block type information of a coding unit, prediction mode information, division unit information, prediction unit information and transmission unit information, and motion from the reordering unit 160 and the prediction units 120 and 125. Various information, such as vector information, reference frame information, block interpolation information, and filtering information, can be encoded.
[73]
The entropy encoder 165 may entropy-encode a coefficient value of a coding unit input from the reordering unit 160.
[74]
The inverse quantization unit 140 and the inverse transform unit 145 inverse quantize values ​​quantized by the quantization unit 135 and inverse transform the values ​​transformed by the transform unit 130. The residual generated by the inverse quantization unit 140 and the inverse transform unit 145 is reconstructed by combining the prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction units 120 and 125 Blocks (Reconstructed Block) can be created.
[75]
The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).
[76]
The deblocking filter can remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on the pixels included in several columns or rows included in the block. When applying a deblocking filter to a block, a strong filter or a weak filter may be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be processed in parallel when performing vertical filtering and horizontal filtering.
[77]
The offset correction unit may correct an offset from the original image in pixel units of the deblocking image. In order to perform offset correction for a specific picture, the pixels included in the image are divided into a certain number of areas, and then the area to be offset is determined and the offset is applied to the area, or offset by considering the edge information of each pixel. You can use the method of applying.
[78]
Adaptive Loop Filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into predetermined groups, one filter to be applied to the corresponding group may be determined, and filtering may be performed differentially for each group. Information related to whether to apply the ALF may be transmitted for each coding unit (CU) of the luminance signal, and the shape and filter coefficient of an ALF filter to be applied may vary according to each block. In addition, the same type (fixed type) ALF filter may be applied regardless of the characteristics of the block to be applied.
[79]
The memory 155 may store the reconstructed block or picture calculated through the filter unit 150, and the stored reconstructed block or picture may be provided to the prediction units 120 and 125 when performing inter prediction.
[80]
[81]
2 is a block diagram showing an image decoding apparatus according to an embodiment of the present invention.
[82]
2, the image decoder 200 includes an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, and a filter unit 240) and a memory 245 may be included.
[83]
When an image bitstream is input from the image encoder, the input bitstream may be decoded in a procedure opposite to that of the image encoder.
[84]
The entropy decoder 210 may perform entropy decoding in a procedure opposite to that of performing entropy encoding in an entropy encoder of an image encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied in response to the method performed by the image encoder.
[85]
The entropy decoder 210 may decode information related to intra prediction and inter prediction performed by the encoder.
[86]
The rearrangement unit 215 may perform rearrangement based on a method of rearranging the bitstream entropy-decoded by the entropy decoder 210 by the encoder. Coefficients expressed in the form of a one-dimensional vector may be reconstructed into coefficients in the form of a two-dimensional block and rearranged. The reordering unit 215 may perform reordering through a method of receiving information related to coefficient scanning performed by the encoder and performing reverse scanning based on the scanning order performed by the corresponding encoder.
[87]
The inverse quantization unit 220 may perform inverse quantization based on a quantization parameter provided by an encoder and a coefficient value of a rearranged block.
[88]
The inverse transform unit 225 may perform an inverse transform, that is, an inverse DCT, an inverse DST, and an inverse KLT, for transforms, that is, DCT, DST, and KLT, performed by the transform unit on the quantization result performed by the image encoder. Inverse transformation may be performed based on a transmission unit determined by the image encoder. The inverse transform unit 225 of the image decoder may selectively perform a transformation technique (eg, DCT, DST, KLT) according to a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.
[89]
The prediction units 230 and 235 may generate a prediction block based on the prediction block generation-related information provided from the entropy decoder 210 and the previously decoded block or picture information provided from the memory 245.
[90]
As described above, if the size of the prediction unit and the size of the transformation unit are the same when intra prediction is performed in the same manner as the operation of the image encoder, a pixel on the left side of the prediction unit, a pixel on the top left side, and a pixel on the top side. If the size of the prediction unit and the size of the transform unit are different when performing intra prediction, but the size of the prediction unit and the size of the transform unit are different when performing intra prediction, intra prediction is performed using a reference pixel based on the transform unit. I can. In addition, intra prediction using NxN splitting for only the smallest coding unit may be used.
[91]
The prediction units 230 and 235 may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determining unit receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, motion prediction related information of the inter prediction method, etc., and classifies the prediction unit from the current coding unit, and predicts It can be determined whether the unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information necessary for inter prediction of the current prediction unit provided by the video encoder to predict the current based on information included in at least one picture of a previous picture or a subsequent picture of the current picture containing the current prediction unit. Inter prediction for a unit can be performed. Alternatively, inter prediction may be performed based on information on a partial region previously-restored in the current picture including the current prediction unit.
[92]
In order to perform inter prediction, the motion prediction method of the prediction unit included in the coding unit based on the coding unit is among the skip mode, merge mode, AMVP mode, and intra block copy mode. You can determine whether or not this is any way.
[93]
The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit that has performed intra prediction, intra prediction may be performed based on intra prediction mode information of a prediction unit provided from an image encoder. The intra prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on a reference pixel of the current block, and may determine whether to apply the filter according to the prediction mode of the current prediction unit and apply it. AIS filtering may be performed on a reference pixel of a current block by using the prediction mode and AIS filter information of the prediction unit provided by the video encoder. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.
[94]
When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating a reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. When the prediction mode of the current prediction unit is a prediction mode in which a prediction block is generated without interpolating a reference pixel, the reference pixel may not be interpolated. The DC filter may generate a prediction block through filtering when the prediction mode of the current block is the DC mode.
[95]
The reconstructed block or picture may be provided to the filter unit 240. The filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.
[96]
Information on whether a deblocking filter is applied to a corresponding block or picture from the video encoder, and when a deblocking filter is applied, information on whether a strong filter or a weak filter is applied may be provided. In the deblocking filter of the image decoder, information related to the deblocking filter provided from the image encoder may be provided, and the image decoder may perform deblocking filtering on a corresponding block.
[97]
The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and information on the offset value.
[98]
The ALF may be applied to a coding unit based on information on whether to apply ALF and information on ALF coefficients provided from the encoder. Such ALF information may be provided by being included in a specific parameter set.
[99]
The memory 245 may store the reconstructed picture or block so that it can be used as a reference picture or a reference block, and may also provide the reconstructed picture to an output unit.
[100]
As described above, in an embodiment of the present invention, for convenience of description, a coding unit is used as a term, but it may be a unit that performs not only encoding but also decoding.
[101]
In addition, the current block represents a block to be encoded/decoded, and according to an encoding/decoding step, a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), or a prediction block (Or a prediction unit) or the like. In this specification,'unit' denotes a basic unit for performing a specific encoding/decoding process, and'block' may denote a sample array of a predetermined size. Unless otherwise specified,'block' and'unit' may be used interchangeably. For example, in an embodiment to be described later, it may be understood that the coding block (coding block) and the coding unit (coding unit) have the same meaning as each other.
[102]
[103]
One picture may be divided into square or non-square basic blocks and encoded/decoded. In this case, the basic block may be referred to as a coding tree unit. The coding tree unit may be defined as a coding unit having the largest size allowed in a sequence or slice. Information indicating whether the coding tree unit is square or non-square or information related to the size of the coding tree unit may be signaled through a sequence parameter set, a picture parameter set, or a slice header. The coding tree unit can be divided into smaller sized partitions. In this case, when the partition generated by dividing the coding tree unit is referred to as depth 1, the partition generated by dividing the partition having depth 1 may be defined as depth 2. That is, a partition generated by dividing a partition having a depth k in a coding tree unit may be defined as having a depth k+1.
[104]
A partition of an arbitrary size generated as the coding tree unit is divided may be defined as a coding unit. The coding unit may be recursively divided, or may be divided into basic units for performing prediction, quantization, transformation, or in-loop filtering. For example, a partition of an arbitrary size generated as the coding unit is divided may be defined as a coding unit, or as a transform unit or a prediction unit, which is a basic unit for performing prediction, quantization, transformation, or in-loop filtering.
[105]
Alternatively, a prediction block having the same size as the coding block or smaller than the coding block may be determined through prediction partitioning of the coding block. For predictive partitioning of a coding block, any one of partition mode (Part_mode) candidates indicating a partitioning type of the coding block may be specified. Information for determining a partition index indicating any one of the partition mode candidates may be signaled through a bitstream. Alternatively, the partition index of the coding block may be determined based on at least one of the size, shape, or coding mode of the coding block. The size or shape of the prediction block may be determined based on the partition mode specified by the partition index. The partition mode candidate may include an asymmetric partition type (eg, nLx2N, nRx2N, 2NxnU, 2NxnD). The number or type of asymmetric partition mode candidates that can be used by the coding block may be determined based on at least one of the size, shape, or coding mode of the coding block.
[106]
3 is a diagram illustrating partition mode candidates that can be applied to the coding block when the coding block is encoded by inter prediction.
[107]
When the coding block is coded by inter prediction, any one of the eight partition mode candidates shown in FIG. 3 may be applied to the coding block.
[108]
On the other hand, when the coding block is encoded by intra prediction, only square partition division can be applied to the coding block. That is, when the coding block is encoded by intra prediction, the partition mode PART_2Nx2N or PART_NxN may be applied to the coding block.
[109]
PART_NxN can be applied when a coding block has a minimum size. Here, the minimum size of the coding block may be predefined by an encoder and a decoder. Alternatively, information on the minimum size of the coding block may be signaled through the bitstream. As an example, the minimum size of the coding block may be signaled through a slice header. Accordingly, the minimum size of the coding block may be determined differently for each slice.
[110]
As another example, the partition mode candidates that can be used by the coding block may be differently determined according to at least one of the size or shape of the coding block. For example, the number or type of partition mode candidates that the coding block can use may be differently determined according to at least one of the size or shape of the coding block.
[111]
Alternatively, the type or number of asymmetric partition mode candidates that can be used by the coding block may be determined based on the size or shape of the coding block. The number or type of asymmetric partition mode candidates that the coding block can use may be differently determined according to at least one of the size or shape of the coding block. For example, when a coding block has an amorphous shape having a width greater than a height, at least one of PART_2NxN, PART_2NxnU, and PART_2NxnD may not be used as a partition mode candidate of the coding block. When the coding block has an amorphous shape whose height is greater than the width, at least one of PART_Nx2N, PART_nLx2N, and PART_nRx2N may not be used as a partition mode candidate of the coding block.
[112]
In general, the size of the prediction block may range from 64x64 to 4x4. However, when the coding block is encoded by inter prediction, in order to reduce a memory bandwidth when performing motion compensation, the prediction block may not have a 4x4 size.
[113]
It is also possible to recursively partition the coding block based on the partition mode. That is, based on the partition mode determined by the partition index, the coding block may be partitioned, and each partition generated as a result of the partitioning of the coding block may be defined as a coding block.
[114]
Hereinafter, a method of dividing the coding unit will be described in more detail. In an embodiment described below, the coding unit may mean a coding tree unit or a coding unit included in a coding tree unit. In addition, a'partition' generated as a coding block is divided may mean a'coding block'. The partitioning method described below may be applied to partitioning a coding block into a plurality of prediction blocks or a plurality of transform blocks.
[115]
The coding unit can be divided by at least one line. In this case, the angle of the line dividing the coding unit may be a value within the range of 0 degrees to 360 degrees. For example, an angle of a horizontal line may be 0 degrees, an angle of a vertical line may be 90 degrees, an angle of a diagonal line in the upper right direction may be 45 degrees, and an angle of a diagonal line in the upper left corner may be 135 degrees.
[116]
When the coding unit is divided by a plurality of lines, all of the plurality of lines may have the same angle. Alternatively, at least one of the plurality of lines may have a different angle from the other lines. Alternatively, the coding tree unit or a plurality of lines dividing the coding unit may have a predefined angle difference (eg, 90 degrees).
[117]
Information about a line dividing a coding unit may be determined by a partition mode. Alternatively, information on at least one of the number, direction, angle, or position of a line within a block may be encoded.
[118]
For convenience of description, in an embodiment to be described later, it is assumed that the coding unit is divided into a plurality of coding units by using at least one of a vertical line or a horizontal line.
[119]
The number of vertical lines or horizontal lines for partitioning the coding unit may be at least one or more. For example, the coding unit may be divided into two partitions using one vertical line or one horizontal line. Alternatively, the coding unit may be divided into three partitions by using two vertical lines or two horizontal lines. Alternatively, one vertical line and one horizontal line may be used to divide the coding unit into four partitions whose width and height are 1/2 smaller than that of the coding unit.
[120]
When the coding unit is divided into a plurality of partitions using at least one vertical line or at least one horizontal line, the partitions may have a uniform size. Alternatively, one partition may have a different size from the other partitions, or each partition may have a different size. For example, when the coding unit is divided into two horizontal lines or two vertical lines, the coding unit may be divided into three partitions. In this case, the width ratio or height ratio of the three partitions may be n:2n:n, 2n:n:n, or n:n:2n.
[121]
In embodiments to be described later, the division of the coding unit into four partitions will be referred to as quad-tree-based division. In addition, the division of the coding unit into two partitions is referred to as binary tree-based division. In addition, division of the coding unit into three partitions will be referred to as a triple tree-based division.
[122]
In the drawings to be described later, it will be shown that one vertical line and/or one horizontal line is used to divide the coding unit, but using a greater number of vertical lines and/or a greater number of horizontal lines than that shown, It will be said that dividing a coding unit into a larger number of partitions than shown or a smaller number of partitions than shown is also included in the scope of the present invention.
[123]
4 illustrates an example of hierarchically partitioning a coding block based on a tree structure as an embodiment to which the present invention is applied.
[124]
The input video signal is decoded in units of a predetermined block, and a basic unit for decoding the input video signal in this way is called a coding block. The coding block may be a unit that performs intra/inter prediction, transform, and quantization. In addition, a prediction mode (eg, an intra prediction mode or an inter prediction mode) is determined for each coding block, and prediction blocks included in the coding block may share the determined prediction mode. The coding block may be a square or non-square block having an arbitrary size in the range of 8×8 to 64×64, and may be a square or non-square block having a size of 128×128, 256×256 or higher.
[125]
Specifically, the coding block may be hierarchically partitioned based on at least one of a quad tree partitioning method, a binary tree partitioning method, or a triple tree partitioning method. The quad-tree-based division may mean a method in which a 2Nx2N coding block is divided into four NxN coding blocks. The binary tree-based partitioning may mean a method in which one coding block is divided into two coding blocks. The triple tree-based partitioning may mean a method in which one coding block is divided into three coding blocks. Even if division based on a binary tree or a triple tree is performed, a coding block having a square shape may exist in a lower depth.
[126]
Partitions created due to binary tree-based partitioning may be symmetric or asymmetric. Further, the coding block divided based on the binary tree may be a square block or a non-square block (eg, a rectangle).
[127]
5 is a diagram showing a partitioning form of a coding block based on binary tree partitioning. The partition type of a coding block based on binary tree division is a symmetric type such as 2NxN (horizontal asymmetric coding unit) or Nx2N (vertical amorphous coding unit), or asymmetric type such as nLx2N, nRx2N, 2NxnU, or 2NxnD. It may include an (asymmetric) type. Only one of a symmetric type or an asymmetric type may be allowed as a division type of a coding block.
[128]
The triple tree division type may include at least one of a type of dividing a coding block into two vertical lines or a type of dividing a coding block into two horizontal lines. Three non-square partitions can be created by triple tree partitioning.
[129]
6 shows a triple-tree division type.
[130]
The triple-tree division type may include a type of dividing a coding block into two horizontal lines or a type of dividing a coding block into two vertical lines. The width or height ratio of partitions generated as a result of dividing the coding block may be n:2n:n, 2n:n:n, or n:n:2n.
[131]
The position of the partition having the largest width or height among the three partitions may be predefined in the encoder and decoder. Alternatively, information indicating a partition having the largest width or height among the three partitions may be signaled through a bitstream.
[132]
It is possible to allow only the division of the square shape or the asymmetric shape of the coding unit. In this case, dividing the coding unit into square-shaped partitions corresponds to quad-tree CU partitioning, and dividing the coding unit into symmetrical non-square partitions corresponds to binary tree partitioning. have. Dividing the coding tree unit into square partitions and symmetric non-square partitions may correspond to Quad Tree and Binary Tree CU Partitioning (QTBT).
[133]
Partitioning based on a binary tree or a triple tree may be performed on a coding block for which partitioning based on a quad tree is no longer performed. A coding block generated as a result of dividing based on a binary tree or a triple tree may be divided into smaller coding blocks. In this case, the coding block may be set so that at least one of quad-tree division, triple-tree division, and binary tree division is not applied to the coding block. Alternatively, binary tree division in a predetermined direction or triple tree division in a predetermined direction may not be allowed in the coding block. For example, quad-tree division and triple-tree division may not be allowed in a coding block generated as a result of division based on a binary tree or a triple tree. Only binary tree division may be allowed in the coding block.
[134]
Alternatively, only the coding block having the largest size among the three coding blocks generated as a result of the triple tree-based division may be divided into coding blocks having a smaller size. Alternatively, binary tree-based division or triple tree-based division may be allowed only for a coding block having the largest size among the three coding blocks generated as a result of the triple tree-based division.
[135]
The division type of the lower depth partition may be determined dependently on the division type of the upper depth partition. For example, when an upper partition and a lower partition are partitioned based on a binary tree, only a binary tree based partition having the same type as the binary tree partition type of the upper depth partition may be allowed in the lower depth partition. For example, when the binary tree division type of the upper depth partition is a 2NxN type, the binary tree division type of the lower depth partition may also be set to the 2NxN type. Alternatively, when the binary tree division type of the upper depth partition is an Nx2N type, the division type of the lower depth partition may also be set to an Nx2N type.
[136]
Alternatively, the partition with the largest size among the partitions generated as a result of partitioning based on the triple tree may be configured not to allow binary tree partitioning in the same direction as the partitioning direction of the upper depth partition or triple-tree partitioning in the same direction as the partitioning direction of the upper depth partition. have.
[137]
Alternatively, the partition type of the lower depth partition may be determined in consideration of the partition type of the upper depth partition and the partition type of the neighboring lower depth partition. Specifically, if the upper depth partition is partitioned based on the binary tree, the partitioning type of the lower depth partition may be determined so that the same result as the partitioning of the upper depth partition based on the quad tree does not occur. As an example, when the partition type of the upper depth partition is 2NxN and the partition type of the neighboring lower depth partition is Nx2N, the current partition type of the lower depth partition cannot be set to Nx2N. This is because, when the current sub-depth partition has an Nx2N partition type, the same result as that of dividing the upper depth partition into an NxN type quad tree occurs. When the partition type of the upper depth partition is Nx2N and the partition type of the neighboring lower depth partition is 2NxN, the current partition type of the lower depth partition cannot be set to 2NxN. That is, when the binary tree division type of the upper depth partition and the binary tree division type of the neighboring lower depth partition are different, the current binary tree division type of the lower depth partition may be set to be the same as the binary tree division type of the upper depth partition.
[138]
Alternatively, the binary tree division type of the lower depth partition may be set to be different from the binary tree division type of the upper depth partition.
[139]
In units of sequence, slice, or coding unit, an allowable binary tree division type can be determined. For example, a binary tree division type allowed for a coding tree unit may be limited to a 2NxN or Nx2N type. The allowable split type may be predefined in the encoder or decoder. Alternatively, information on an allowable or disallowed partition may be encoded and signaled through a bitstream.
[140]
7 is a diagram showing an example in which only a specific type of binary tree-based division is allowed.
[141]
FIG. 7A shows an example in which only Nx2N-type binary tree-based division is allowed, and FIG. 7B shows an example in which only 2NxN-type binary tree-based division is allowed.
[142]
In order to represent various types of division, information on quadtree division, information on binary tree division, or information on triple tree division may be used. The information on quad-tree division may include at least one of information indicating whether quad-tree-based division is performed or information on a size/depth of a coding block in which quad-tree-based division is allowed. Information on binary tree division includes information indicating whether or not binary tree-based division is performed, information indicating whether binary tree-based division is vertical or horizontal, and coding blocks in which binary tree-based division is allowed. It may include at least one of information on the size/depth of and information on the size/depth of a coding block in which binary tree-based division is not allowed. The information on triple-tree partitioning includes information indicating whether triple-tree-based partitioning is performed, information indicating whether the triple-tree-based partitioning is in the vertical direction or the horizontal direction, and a coding block in which triple-tree-based partitioning is allowed. It may include at least one of information on the size/depth of the triple tree or information on the size/depth of a coding block in which the triple tree-based division is not allowed. The information on the size of the coding block may indicate a minimum value or a maximum value of at least one of a width, a height, a product of a width and a height, or a width and a height ratio of the coding block.
[143]
For example, when the width or height of the coding block is less than the minimum size allowed for binary tree division, or the division depth of the coding block is greater than the maximum depth allowed for binary tree division, the coding block is based on a binary tree. Splitting may not be allowed.
[144]
For example, when the width or height of the coding block is less than the minimum size allowed for triple tree splitting, or the splitting depth of the coding block is greater than the maximum depth allowed for triple tree splitting, the coding block Splitting may not be allowed.
[145]
Information on the partitioning allowance condition based on a binary tree or a triple tree may be signaled through a bitstream. The information may be encoded in units of a sequence, picture, or fragment image. The fragment image may mean at least one of a slice, a tile group, a tile, a brick, a coding block, a prediction block, or a transform block.
[146]
For example, through the bitstream, the syntax'max_mtt_depth_idx_minus1' indicating the maximum depth in which binary tree/triple tree division is allowed may be encoded/decoded through the bitstream. In this case, max_mtt_depth_idx_minus1+1 may indicate the maximum depth in which binary tree/triple tree division is allowed.
[147]
As an example, at least one of the number of times the binary tree/triple tree division is allowed, the maximum depth that the binary tree/triple tree division is allowed, or the number of depths that the binary tree/triple tree division is allowed is signaled at the sequence or slice level. I can. Accordingly, at least one of the number of binary tree/triple tree division times, the maximum depth allowed for binary tree/triple tree division, or the number of depths allowed for binary tree/triple tree division of the first slice and the second slice may be different. I can. For example, in the first slice, binary tree/triple tree division may be allowed in only one depth, whereas in the second slice, binary tree/triple tree division may be allowed in two depths.
[148]
Referring to the example shown in FIG. 8, in FIG. 8, it is shown that binary tree division is performed on a coding unit having a depth of 2 and a coding unit having a depth of 3. Accordingly, information indicating the number of times (2 times) that the binary tree division in the coding tree unit is performed, information indicating the maximum depth (depth 3) of the partition generated by the binary tree division in the coding tree unit At least one of information indicating the number of partition depths (2, depth 2 and depth 3) to which the division is applied may be encoded/decoded through a bitstream.
[149]
Alternatively, the number of times the binary tree/triple tree division is allowed in the encoder and the decoder, the depth at which the binary tree/triple tree division is allowed, or the number of depths in which the binary tree/triple tree division is allowed may be predefined. Alternatively, based on at least one of the index of the sequence or slice or the size/type of the coding unit, the number of times the binary tree/triple tree division is allowed, the depth at which the binary tree/triple tree division is allowed, or the binary tree/triple tree division is The number of allowed depths may be determined. For example, in a first slice, a binary tree/triple tree division may be allowed in one depth, and a binary tree/triple tree division may be allowed in two depths in a second slice.
[150]
As another example, depending on the temporal level identifier (TemporalID) of a slice or picture, at least one of the number of times a binary tree is allowed to be split, a depth where a binary tree is allowed to be split, or the number of depths where a binary tree is allowed to be split may be differently set. Here, the temporal level identifier (TemporalID) is used to identify each of a plurality of layers of an image having at least one scalability of view, spatial, temporal, or quality. will be.
[151]
As shown in FIG. 4, the first coding block 300 having a split depth of k may be divided into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having half the width and height of the first coding block, and the dividing depth of the second coding block may be increased to k+1.
[152]
The second coding block 310 having a splitting depth of k+1 may be split into a plurality of third coding blocks having a splitting depth of k+2. The division of the second coding block 310 may be performed by selectively using either a quart tree or a binary tree according to a division method. Here, the partitioning method may be determined based on at least one of information indicating partitioning based on a quad tree or information indicating partitioning based on a binary tree.
[153]
When the second coding block 310 is divided based on a quart tree, the second coding block 310 is divided into four third coding blocks 310a having half the width and height of the second coding block, and the third coding block 310a is The splitting depth can be increased to k+2. On the other hand, when the second coding block 310 is divided based on a binary tree, the second coding block 310 may be divided into two third coding blocks. In this case, each of the two third coding blocks is an amorphous block in which one of the width and height of the second coding block is half the size, and the split depth may be increased to k+2. The second coding block may be determined as a horizontal or vertical amorphous block according to the division direction, and the division direction may be determined based on information on whether the binary tree-based division is in the vertical direction or the horizontal direction.
[154]
Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer divided based on a quad tree or a binary tree, and in this case, the corresponding coding block may be used as a prediction block or a transform block.
[155]
Like the division of the second coding block 310, the third coding block 310a may be determined as a terminal coding block or may be additionally divided based on a quad tree or a binary tree.
[156]
Meanwhile, the third coding block 310b divided based on a binary tree may be further divided into a coding block 310b-2 in a vertical direction or a coding block 310b-3 in a horizontal direction based on the binary tree, and the corresponding coding The division depth of a block can be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block 310b-1 that is no longer divided based on a binary tree, and in this case, the corresponding coding block 310b-1 is used as a prediction block or a transform block. I can. However, in the above-described partitioning process, information on the size/depth of a coding block in which quad-tree-based division is allowed, information on the size/depth of a coding block in which binary tree-based division is allowed, or binary tree-based division is allowed. It may be limitedly performed based on at least one of information on the size/depth of a coding block that is not not used.
[157]
The size candidates that a coding block can have may be limited to a predetermined number, or a size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block within a sequence or a size of a coding block within a picture may be limited to have any one of 256x256, 128x128, or 32x32. Information indicating the size of a coding block in a sequence or picture may be signaled through a sequence header or a picture header.
[158]
As a result of the division based on the quad tree and the binary tree, the coding unit may take a square or a rectangle of any size.
[159]
As shown in FIG. 4, the first coding block 300 having a split depth of k may be divided into a plurality of second coding blocks based on a quad tree. For example, the second coding blocks 310 to 340 are square blocks having half the width and height of the first coding block, and the dividing depth of the second coding block may be increased to k+1.
[160]
The second coding block 310 having a splitting depth of k+1 may be split into a plurality of third coding blocks having a splitting depth of k+2. The division of the second coding block 310 may be performed by selectively using either a quart tree or a binary tree according to a division method. Here, the partitioning method may be determined based on at least one of information indicating partitioning based on a quad tree or information indicating partitioning based on a binary tree.
[161]
When the second coding block 310 is divided based on a quart tree, the second coding block 310 is divided into four third coding blocks 310a having half the width and height of the second coding block, and the third coding block 310a is The splitting depth can be increased to k+2. On the other hand, when the second coding block 310 is divided based on a binary tree, the second coding block 310 may be divided into two third coding blocks. In this case, each of the two third coding blocks is an amorphous block in which one of the width and height of the second coding block is half the size, and the split depth may be increased to k+2. The second coding block may be determined as a horizontal or vertical amorphous block according to the division direction, and the division direction may be determined based on information on whether the binary tree-based division is in the vertical direction or the horizontal direction.
[162]
Meanwhile, the second coding block 310 may be determined as a terminal coding block that is no longer divided based on a quad tree or a binary tree, and in this case, the corresponding coding block may be used as a prediction block or a transform block.
[163]
Like the division of the second coding block 310, the third coding block 310a may be determined as a terminal coding block or may be additionally divided based on a quad tree or a binary tree.
[164]
Meanwhile, the third coding block 310b divided based on a binary tree may be further divided into a coding block 310b-2 in a vertical direction or a coding block 310b-3 in a horizontal direction based on the binary tree, and the corresponding coding The division depth of a block can be increased to k+3. Alternatively, the third coding block 310b may be determined as a terminal coding block 310b-1 that is no longer divided based on a binary tree, and in this case, the corresponding coding block 310b-1 is used as a prediction block or a transform block. I can. However, in the above-described partitioning process, information on the size/depth of a coding block in which quad-tree-based division is allowed, information on the size/depth of a coding block in which binary tree-based division is allowed, or binary tree-based division is allowed. It may be limitedly performed based on at least one of information on the size/depth of a coding block that is not not used.
[165]
The size candidates that a coding block can have may be limited to a predetermined number, or a size of a coding block within a predetermined unit may have a fixed value. For example, the size of a coding block within a sequence or a size of a coding block within a picture may be limited to have any one of 256x256, 128x128, or 32x32. Information indicating the size of a coding block in a sequence or picture may be signaled through a sequence header or a picture header.
[166]
As a result of the division based on the quad tree and the binary tree, the coding unit may take a square or a rectangle of any size.
[167]
Transform skip may be set not to be used in a coding unit generated as a result of division based on binary tree or division based on triple tree. Alternatively, the non-square coding unit may be set so that the transform skip is applicable only in at least one of a vertical direction or a horizontal direction. For example, when transform skip is applied in the horizontal direction, it indicates that only scaling is performed without transform/inverse transform in the horizontal direction, and transform/inverse transform using DCT or DST is performed in the vertical direction. When the transform skip is applied in the vertical direction, it indicates that only scaling is performed without transform/inverse transform in the vertical direction and transform/inverse transform using DCT or DST is performed in the horizontal direction.
[168]
Information on whether to skip the inverse transformation in the horizontal direction or information indicating whether to skip the inverse transformation in the vertical direction may be signaled through a bitstream. As an example, the information indicating whether to skip the inverse transformation in the horizontal direction is a 1-bit flag and is'hor_transform_skip_flag', and the information indicating whether to skip the inverse transformation in the vertical direction is a 1-bit flag, and'ver_transform_skip_flag' 'Can be.
[169]
The encoder may determine whether to encode'hor_transform_skip_flag' or'ver_transform_skip_flag' according to the size and/or shape of the current block. For example, when the current block is in the form of Nx2N, hor_transform_skip_flag may be encoded, and encoding of ver_transform_skip_flag may be omitted. When the current block has a 2NxN type, ver_transform_skip_flag may be encoded and hor_transform_skip_flag may be omitted.
[170]
Alternatively, based on the size and/or shape of the current block, whether to skip the transformation in the horizontal direction or the transformation in the vertical direction may be determined. For example, when the current block is in the form of Nx2N, transform skip may be applied in the horizontal direction and transform/inverse transform may be performed in the vertical direction. When the current block has a 2NxN type, a transform skip may be applied in a vertical direction and transform/inverse transform may be performed in a horizontal direction. Transformation/inverse transformation may be performed based on at least one of DCT and DST.
[171]
As a result of partitioning based on a quad tree, a binary tree, or a triple tree, a coding block that is no longer partitioned may be used as a prediction block or a transform block. That is, it can be used as a coding block, a prediction block, or a transform block generated as a result of quad-tree partitioning or binary tree partitioning. For example, a prediction image may be generated in units of coding blocks, and a residual signal, which is a difference between the original image and the predicted image, may be transformed in units of coding blocks. In order to generate a prediction image in units of coding blocks, motion information may be determined based on a coding block or an intra prediction mode may be determined based on a coding block. Accordingly, the coding block may be encoded using at least one of skip mode, intra prediction, and inter prediction.
[172]
Alternatively, a plurality of coding blocks generated by dividing the coding block may be configured to share at least one of motion information, merge candidate, reference sample, reference sample line, and intra prediction mode. For example, when the coding block is divided into a triple tree, the partitions generated by dividing the coding block may select at least one of motion information, merge candidate, reference sample, reference sample line, or intra prediction mode according to the size or shape of the coding block. You can share. Alternatively, only some of the plurality of coding blocks may share the information, and the residual coding block may be set not to share the information.
[173]
As another example, it is possible to divide the coding block and use a prediction block or a transform block having a size smaller than that of the coding block.
[174]
Hereinafter, a method of performing inter prediction on a coding block or a prediction block generated by dividing a coding block will be described in detail.
[175]
[176]
9 is a flowchart illustrating an inter prediction method according to an embodiment to which the present invention is applied.
[177]
Referring to FIG. 9, motion information of a current block may be determined (S910). The motion information of the current block may include at least one of a motion vector of the current block, a reference picture index of the current block, an inter prediction direction of the current block, or a weighted prediction weight. The weighted prediction weight represents a weight applied to the L0 reference block and a weight applied to the L1 reference block.
[178]
A motion vector of the current block may be determined based on information signaled through the bitstream. The motion vector precision indicates a display unit of a motion vector of a current block. For example, the motion vector precision of the current block may be determined by at least one of integer pel, ½ pel, ¼ pel, or ⅛ pel. The motion vector precision may be determined in a picture unit, a slice unit, a tile group unit, a tile unit, or a block unit. A block may represent a coding tree unit, a coding unit, a prediction unit, or a transform unit.
[179]
The motion information of the current block may be obtained based on at least one of information signaled through a bitstream or motion information of a neighboring block adjacent to the current block.
[180]
10 is a diagram illustrating a process of deriving motion information of a current block when a merge mode is applied to a current block.
[181]
The merge mode represents a method of inducing motion information of a current block from neighboring blocks.
[182]
When the merge mode is applied to the current block, a spatial merge candidate may be derived from a spatial neighboring block of the current block (S1010). The spatial neighboring block may include at least one of a block adjacent to an upper boundary of the current block, a left boundary of the current block, or a corner of the current block (eg, at least one of an upper left corner, an upper right corner, or a lower left corner). .
[183]
11 is a diagram illustrating an example of a spatial neighboring block.
[184]
As in the example shown in FIG. 11, the spatial neighboring block is a neighboring block (A 1 ) neighboring to the left of the current block, a neighboring block (B 1 ) neighboring to the top of the current block, and a lower left corner of the current block. It may include at least one of an adjacent neighboring block A 0 , a neighboring block B 0 adjacent to an upper right corner of the current block, and a neighboring block B 2 adjacent to an upper left corner of the current block . As an example, it is assumed that the location of the upper left corner sample of the current block is (0, 0), the width of the current block is W, and the height of the current block is H. Block A 1 may contain a sample at position (-1, H-1). Block B 1 may include a sample at position (W-1, -1). Block A 0 may contain a sample at position (-1, H). Block B 0 may contain a sample at position (W, -1). Block B 2 may include a sample at position (-1, -1).
[185]
By further extending the embodiment of FIG. 11, a spatial merge candidate may be derived from a block adjacent to the upper left sample of the current block and a block adjacent to the upper center sample. As an example, a block neighboring the upper left sample of the current block may include at least one of a block including a sample at a position (0, -1) or a block including a sample at a position (-1, 0). Alternatively, a spatial merge candidate may be derived from at least one of a block adjacent to the upper center sample of the current block or a block adjacent to the left center sample. As an example, a block adjacent to the upper center sample of the current block may include a sample at the (W/2, -1) position. A block adjacent to the left center sample of the current block may include a sample at the (-1, H/2) position.
[186]
Positions of the upper neighboring block and/or the left neighboring block used to derive the spatial merge candidate may be determined based on the size and/or shape of the current block. For example, when the size of the current block is greater than or equal to the threshold value, a spatial merge candidate may be derived from a block adjacent to the upper center sample of the current block and a block adjacent to the left center sample. On the other hand, when the size of the current block is smaller than the threshold value, a spatial merge candidate may be derived from a block adjacent to the upper right sample of the current block and a block adjacent to the lower left sample. Here, the size of the current block may be expressed based on at least one of a width, a height, a sum of a width and a height, a product of a width and a height, or a ratio of a width and height of the current block. The threshold value may be an integer of 2, 4, 8, 16, 32, or 128.
[187]
The availability of the expanded spatial neighboring block may be determined according to the shape of the current block. For example, if the current block is an amorphous block whose width is greater than the height, a block adjacent to the upper left sample of the current block, a block adjacent to the center left sample, or a block adjacent to the lower left sample of the current block is used It can be determined to be impossible. On the other hand, when the current block is a block whose height is greater than the width, it may be determined that a block adjacent to the upper left sample of the current block, a block adjacent to the upper center sample, or a block adjacent to the upper right sample of the current block is unavailable. .
[188]
The motion information of the spatial merge candidate may be set the same as the motion information of the spatial neighboring block.
[189]
The spatial merge candidate may be determined by searching for neighboring blocks in a predetermined order. As an example, in the example shown in FIG. 11, a search for determining a spatial merge candidate may be performed in the order of A 1 , B 1 , B 0 , A 0 and B 2 blocks. In this case, the B 2 block may be used when at least one of the remaining blocks (ie, A 1 , B 1 , B 0 and A 0 ) does not exist or at least one of the remaining blocks is encoded in an intra prediction mode.
[190]
The search order of the spatial merge candidate may be predefined in the encoder/decoder. Alternatively, the search order of spatial merge candidates may be adaptively determined according to the size or shape of the current block. Alternatively, the search order of the spatial merge candidate may be determined based on information signaled through the bitstream.
[191]
A temporal merge candidate may be derived from a temporal neighboring block of the current block (S1020). The temporal neighboring block may mean a co-located block (collocated block) included in a collocated picture. The collocated picture has a different temporal order (Picture Order Count, POC) than the current picture including the current block. The collocated picture may be determined as a picture having a predefined index in the reference picture list or a picture having the smallest difference in output order (POC) from the current picture. Alternatively, a collocated picture may be determined based on information signaled from the bitstream. The information signaled from the bitstream is at least one of information indicating a reference picture list (eg, an L0 reference picture list or an L1 reference picture list) including a collocated picture and/or an index indicating a collocated picture in the reference picture list. It may include. Information for determining a collocated picture may be signaled in at least one of a picture parameter set, a slice header, or a block level.
[192]
The motion information of the temporal merge candidate may be determined based on motion information of the collocated block. For example, the motion vector of the temporal merge candidate may be determined based on the motion vector of the collocated block. For example, the motion vector of the temporal merge candidate may be set to be the same as the motion vector of the collocated block. Alternatively, the motion vector of the temporal merge candidate is based on an output order (POC) difference between the current picture and the reference picture of the current block and/or the output order (POC) difference between the collocated picture and the reference picture of the collocated picture. Thus, it can be derived by scaling the motion vector of the collocated block.
[193]
12 is a diagram for describing an example of deriving a motion vector of a temporal merge candidate.
[194]
In the example shown in FIG. 12, tb denotes the POC difference between the current picture (curr_pic) and the reference picture (curr_ref) of the current picture, and td denotes the difference between the collocated picture (col_pic) and the reference picture of the collocated block ( col_ref) POC difference. The motion vector of the temporal merge candidate may be derived by scaling the motion vector of the collocated block (col_PU) based on tb and/or td.
[195]
Alternatively, in consideration of the availability of the collocated block, both a motion vector of the collocated block and a motion vector obtained by scaling the collocated block may be used as a motion vector of a temporal merge candidate. As an example, a motion vector of a collocated block may be set as a motion vector of a first temporal merge candidate, and a value obtained by scaling a motion vector of a collocated block may be set as a motion vector of a second temporal merge candidate.
[196]
The inter prediction direction of the temporal merge candidate may be set to be the same as the inter prediction direction of the temporal neighboring block. However, the reference picture index of the temporal merge candidate may have a fixed value. As an example, the reference picture index of the temporal merge candidate may be set to '0'. Alternatively, a reference picture index of the temporal merge candidate may be adaptively determined based on at least one of a reference picture index of a spatial merge candidate and a reference picture index of a current picture.
[197]
The collocated block may be determined as an arbitrary block in a block having the same position and size as the current block in the collocated picture or a block adjacent to a block having the same position and size as the current block.
[198]
13 is a diagram illustrating locations of candidate blocks that can be used as collocated blocks.
[199]
The candidate block may include at least one of a block adjacent to an upper left corner position of the current block in the collocated picture, a block adjacent to a center sample position of the current block, or a block adjacent to a lower left corner position of the current block.
[200]
As an example, the candidate block is a block (TL) including the upper left sample position of the current block in the collocated picture, the block BR ​​including the lower right sample position of the current block, and adjacent to the lower right corner of the current block. Block (H), block (C3) including the center sample location of the current block, or a block adjacent to the center sample of the current block (e.g., including a sample location spaced apart from the center sample of the current block by (-1, -1) Block) and may include at least one of (C0).
[201]
In addition to the example shown in FIG. 13, a block including a position of a neighboring block adjacent to a predetermined boundary of a current block in the collocated picture may be selected as the collocated block.
[202]
The number of temporal merge candidates may be one or more. As an example, one or more temporal merge candidates may be derived based on one or more collocated blocks.
[203]
Information on the maximum number of temporal merge candidates may be encoded and signaled by an encoder. Alternatively, the maximum number of temporal merge candidates may be derived based on the maximum number of merge candidates that may be included in the merge candidate list and/or the maximum number of spatial merge candidates. Alternatively, the maximum number of temporal merge candidates may be determined based on the number of available collocated blocks.
[204]
The availability of candidate blocks may be determined according to a predetermined priority, and at least one collocated block may be determined based on the determination and the maximum number of temporal merge candidates. For example, if the block C3 including the center sample position of the current block and the block H adjacent to the lower right corner of the current block are candidate blocks, one of the C3 block and the H block is a collocated block. You can decide. When the H block is available, the H block may be determined as a collocated block. On the other hand, when the H block is not available (e.g., when the H block is encoded by intra prediction, when the H block is not available, or when the H block is located outside the largest coding unit (LCU)) Case, etc.), the C3 block may be determined as a collocated block.
[205]
As another example, when at least one of a plurality of blocks adjacent to the lower right corner of the current block in the collocated picture is not available (e.g., H block and/or BR block), the unavailable block is replaced with another I can. The other block replacing the unusable block is at least one of a block adjacent to the center sample position of the current block in the collocated picture (eg, C0 and/or C3) or a block adjacent to the upper left corner position of the current block (eg, TL). It can contain one.
[206]
Even when at least one of the plurality of blocks adjacent to the center sample position of the current block in the collocated picture is not available, or when at least one of the plurality of blocks adjacent to the upper left corner position of the current block in the collocated picture is not available However, it can be used by replacing an unavailable block with another available block.
[207]
Thereafter, a merge candidate list including the spatial merge candidate and the temporal merge candidate may be generated (S1030). In constructing the merge candidate list, a merge candidate having the same motion information as the previously added merge candidate may be deleted from the merge candidate list.
[208]
Information on the maximum number of merge candidates may be signaled through a bitstream. For example, information indicating the maximum number of merge candidates may be signaled through a sequence parameter or a picture parameter. For example, when the maximum number of merge candidates is 6, 6 may be selected by adding a spatial merge candidate and a temporal merge candidate. For example, five of five spatial merge candidates may be selected, and one of two temporal merge candidates may be selected.
[209]
Alternatively, the maximum number of merge candidates may be predefined in an encoder and a decoder. For example, the maximum number of merge candidates may be 2, 3, 4, 5, or 6. Alternatively, the maximum number of merge candidates may be determined based on at least one of whether to perform merge with MVD (MMVD), whether to perform mixed prediction, or whether to perform triangular partitioning.
[210]
If the number of merge candidates included in the merge candidate list is less than the maximum number of merge candidates, the merge candidates included in the second merge candidate list may be added to the merge candidate list.
[211]
The second merge candidate list may include a merge candidate derived based on motion information of a block encoded/decoded by inter prediction before the current block. As an example, when motion compensation for a block whose encoding mode is inter prediction is performed, a merge candidate derived based on motion information of the block may be added to the second merge candidate list. When encoding/decoding of the current block is completed, motion information of the current block may be added to the second merge candidate list for inter prediction of the next block.
[212]
The second merge candidate list may be initialized in units of CTUs, tiles, or slices. The maximum number of merge candidates that the second merge candidate list may include may be predefined in an encoder and a decoder. Alternatively, information indicating the maximum number of merge candidates that the second merge candidate list may include may be signaled through a bitstream.
[213]
Indexes of merge candidates included in the second merge candidate list may be determined based on an order of addition to the second merge candidate list. As an example, the index allocated to the merge candidate added to the N-th second merge candidate list may have a value smaller than the index allocated to the merge candidate added to the N+1-th second merge candidate list. For example, the index of the N+1th merge candidate may be set to a value greater than the index of the Nth merge candidate. Alternatively, the index of the N-th merge candidate may be set as the index of the N+1-th merge candidate, and a value of the index of the N-th merge candidate may be subtracted by one.
[214]
Alternatively, the index allocated to the merge candidate added to the N-th second merge candidate list may have a value greater than the index allocated to the merge candidate added to the N+1-th second merge candidate list. For example, the index of the Nth merge candidate may be set as the index of the N+1th merge candidate, and the value of the index of the Nth merge candidate may be increased by 1.
[215]
Whether to add a merge candidate derived from the block to the second merge candidate list based on whether the motion information of the block on which motion compensation has been performed and the motion information of the merge candidate included in the second merge candidate list are the same You can decide. For example, when a merge candidate identical to the motion information of the block is included in the second merge candidate list, a merge candidate derived based on the motion information of the block may not be added to the second merge candidate list. Alternatively, if a merge candidate identical to the motion information of the block is included in the second merge candidate list, the merge candidate is deleted from the second merge candidate list, and a merge candidate derived based on the motion information of the block is removed. 2 Can be added to the merge candidate list.
[216]
When the number of merge candidates included in the second merge candidate list is the same as the maximum number of merge candidates, the merge candidate with the lowest index or the merge candidate with the highest index is deleted from the second merge candidate list, and motion information of the block is used. The merge candidate derived by may be added to the second merge candidate list. That is, after the oldest merge candidate among the merge candidates included in the second merge candidate list is deleted, a merge candidate derived based on motion information of the block may be added to the second merge candidate list.
[217]
When the number of merge candidates included in the merge candidate list has not yet reached the maximum number of merge candidates, a combined merge candidate combining two or more merge candidates or a merge candidate having a (0,0) zero motion vector It can be included in the merge candidate list.
[218]
Alternatively, an average merge candidate obtained by averaging motion vectors of two or more merge candidates may be added to the merge candidate list. The average merge candidate may be derived by averaging motion vectors of two or more merge candidates included in the merge candidate list. For example, when a first merge candidate and a second merge candidate are added to the merge candidate list, an average merge candidate may be obtained by averaging the motion vectors of the first merge candidate and the motion vectors of the second merge candidate. Specifically, the L0 motion vector of the average merge candidate is derived by averaging the L0 motion vector of the first merge candidate and the L0 motion vector of the second merge candidate, and the L1 motion vector of the average merge candidate is the L1 motion vector of the first merge candidate. And L1 motion vectors of the second merge candidate may be averaged. When bidirectional prediction is applied to one of the first merge candidate and the second merge candidate, and unidirectional prediction is applied to the other, the motion vector of the bidirectional merge candidate may be set as the L0 motion vector or the L1 motion vector of the average merge candidate as it is. have. As an example, when the first merge candidate performs L0 direction and L1 direction prediction, while the second merge candidate performs L0 direction prediction, the L0 motion vector of the average merge candidate is the L0 motion vector and the second merge candidate. While derived by averaging the L0 motion vectors of the two merge candidates, the L1 motion vector of the average merge candidate may be derived as the L1 motion vector of the first merge candidate.
[219]
When the reference pictures of the first merge candidate and the second merge candidate are different, a motion vector of the first merge candidate or the second merge candidate is considered in consideration of the distance (ie, POC difference) between the current picture and the reference picture of each merge candidate. Can be scaled. For example, after scaling the motion vector of the second merge candidate, the average merge candidate may be derived by averaging the motion vector of the first merge candidate and the scaled motion vector of the second merge candidate. At this time, priority is set based on the size of the reference picture index of each merge candidate, the distance between the current block and the reference picture of each merge candidate, or whether bidirectional prediction is applied, Scaling can be applied to the motion vector of the candidate.
[220]
The reference picture index of the average merge candidate may be set to indicate a reference picture of a specific position in the reference picture list. As an example, the reference picture index of the average merge candidate may indicate the first or last reference picture in the reference picture list. Alternatively, the reference picture index of the average merge candidate may be set equal to the reference picture index of the first merge candidate or the second merge candidate. For example, when the reference picture indexes of the first merge candidate and the second merge candidate are the same, the reference picture index of the average merge candidate may be set to be the same as the reference picture indexes of the first merge candidate and the second merge candidate. When the reference picture indexes of the first merge candidate and the second merge candidate are different, based on the size of the reference picture index of each merge candidate, the distance between the current block and the reference pictures of each merge candidate, or whether bidirectional prediction is applied, etc. By setting the priority, a reference picture index of a merge candidate having a high (or low) priority may be set as a reference picture index of an average merge candidate. For example, when bidirectional prediction is applied to the first merge candidate and unidirectional prediction is applied to the second merge candidate, the reference picture index of the first merge candidate to which bidirectional prediction is applied may be determined as the reference picture index of the average merge candidate.
[221]
A combination order for generating an average merge candidate may be determined based on a priority between combinations of merge candidates. The priority may be predefined by an encoder and a decoder. Alternatively, a combination order may be determined based on whether the merge candidate is predicted in both directions. For example, a combination of merge candidates encoded by bidirectional prediction may be set to have a higher priority than a combination of merge candidates encoded by unidirectional prediction. Alternatively, a combination order may be determined based on the reference picture of the merge candidate. For example, a combination of merge candidates having the same reference picture may have a higher priority than a combination of merge candidates having different reference pictures.
[222]
The merge candidate may be included in the merge candidate list according to a predefined priority. The higher the priority, the smaller the index allocated to the merge candidate may be. As an example, the spatial merge candidate may be added to the merge candidate list before the temporal merge candidate. In addition, the spatial merge candidates are the spatial merge candidate of the left neighboring block, the spatial merge candidate of the upper neighboring block, the spatial merge candidate of the block adjacent to the upper right corner, the spatial merge candidate of the block adjacent to the lower left corner, and the spatial merge candidate of the upper left corner. The blocks may be added to the merge candidate list in the order of spatial merge candidates. Alternatively, it is possible to set a spatial merge candidate derived from a neighboring block (B2 of FIG. 11) adjacent to the upper left corner of the current block to be added to the merge candidate list in a lower order than the temporal merge candidate.
[223]
As another example, priority among merge candidates may be determined according to the size or shape of the current block. For example, when the current block has a rectangular shape whose width is greater than the height, the spatial merge candidate of the left neighboring block may be added to the merge candidate list before the spatial merge candidate of the upper neighboring block. On the other hand, when the current block has a rectangular shape whose height is greater than the width, the spatial merge candidate of the upper neighboring block may be added to the merge candidate list before the spatial merge candidate of the left neighboring block.
[224]
As another example, the priority between merge candidates may be determined according to motion information of each of the merge candidates. For example, a merge candidate having bidirectional motion information may have a higher priority than a merge candidate having unidirectional motion information. Accordingly, a merge candidate having bidirectional motion information may be added to the merge candidate list before a merge candidate having unidirectional motion information.
[225]
As another example, after generating a merge candidate list according to a predefined priority, the merge candidates may be rearranged. Rearrangement may be performed based on motion information of merge candidates. As an example, rearrangement may be performed based on at least one of whether the merge candidate has bidirectional motion information, the size of the motion vector, the motion vector precision, or a temporal order (POC) between the current picture and the reference picture of the merge candidate. . Specifically, rearrangement may be performed to have a higher priority than a merge candidate having a unidirectional merge candidate than after a merge having bidirectional motion information. Alternatively, rearrangement may be performed so that a merge candidate having a prime motion vector precision has a higher priority than a merge candidate having an integer motion vector precision.
[226]
When the merge candidate list is generated, at least one of the merge candidates included in the merge candidate list may be specified based on the merge candidate index (S1040).
[227]
The motion information of the current block may be set equal to the motion information of the merge candidate specified by the merge candidate index (S1050). For example, when a spatial merge candidate is selected by the merge candidate index, motion information of a current block may be set to be the same as motion information of a spatial neighboring block. Alternatively, when a temporal merge candidate is selected by the merge candidate index, motion information of a current block may be set to be the same as motion information of a temporal neighboring block.
[228]
14 is a diagram illustrating a process of deriving motion information of a current block when the AMVP mode is applied to the current block.
[229]
When the AMVP mode is applied to the current block, at least one of the inter prediction direction or the reference picture index of the current block may be decoded from the bitstream (S1410). That is, when the AMVP mode is applied, at least one of the inter prediction direction or the reference picture index of the current block may be determined based on information encoded through the bitstream.
[230]
A spatial motion vector candidate may be determined based on a motion vector of a spatial neighboring block of the current block (S1420). The spatial motion vector candidate may include at least one of a first spatial motion vector candidate derived from an upper neighboring block of the current block or a second spatial motion vector candidate derived from a left neighboring block of the current block. Here, the upper neighboring block includes at least one of blocks adjacent to the upper or upper right corner of the current block, and the left neighboring block of the current block includes at least one of blocks adjacent to the left or lower left corner of the current block. I can. A block adjacent to the upper left corner of the current block may be treated as an upper neighboring block or a left neighboring block.
[231]
Alternatively, a spatial motion vector candidate may be derived from a spatial non-neighbor block that is not adjacent to the current block. For example, a block located on the same vertical line as a block adjacent to the top, top right corner, or top left corner of the current block, and a block located on the same horizontal line as a block adjacent to the left, bottom left corner, or top left corner of the current block Alternatively, a spatial motion vector candidate of the current block may be derived by using at least one of blocks located on the same diagonal as a block adjacent to the corner of the current block. When a spatial neighboring block is not available, a spatial motion vector candidate can be derived using a spatial non-neighboring block.
[232]
As another example, two or more spatial motion vector candidates may be derived using spatial neighboring blocks and spatial non-neighboring blocks. As an example, a first spatial motion vector candidate and a second spatial motion vector candidate are derived based on neighboring blocks adjacent to the current block, while not neighboring the current block, but based on neighboring blocks adjacent to the neighboring blocks Thus, a third spatial motion vector candidate and/or a fourth spatial motion vector candidate may be derived.
[233]
When the reference pictures between the current block and the spatial neighboring block are different, the spatial motion vector may be obtained by scaling the motion vector of the spatial neighboring block. A temporal motion vector candidate may be determined based on a motion vector of a temporal neighboring block of the current block (S1430). When the reference pictures between the current block and the temporal neighboring block are different, the temporal motion vector may be obtained by scaling the motion vector of the temporal neighboring block. In this case, only when the number of spatial motion vector candidates is less than or equal to a predetermined number, a temporal motion vector candidate may be derived.
[234]
A motion vector candidate list including the spatial motion vector candidate and the temporal motion vector candidate may be generated (S1440).
[235]
When the motion vector candidate list is generated, at least one of the motion vector candidates included in the motion vector candidate list may be specified based on information specifying at least one of the motion vector candidate list (S1450).
[236]
A motion vector candidate of the current block may be obtained by setting the motion vector candidate specified by the information as a motion vector predicted value of the current block and adding the motion vector difference value to the motion vector predicted value (S1460). In this case, the motion vector difference value may be parsed through a bitstream.
[237]
When motion information of the current block is acquired, motion compensation for the current block may be performed based on the acquired motion information (S920). Specifically, motion compensation for the current block may be performed based on the inter prediction direction of the current block, a reference picture index, and a motion vector. The inter prediction direction indicates whether to predict the L0 direction, whether to predict the L1 direction, or whether to predict the bidirectional direction. When the current block is encoded by bidirectional prediction, a prediction block of the current block may be obtained based on a weighted sum operation or an average operation of the L0 reference block and the L1 reference block.
[238]
When a prediction sample is obtained as a result of performing motion compensation, a current block may be reconstructed based on the generated prediction sample. Specifically, a reconstructed sample may be obtained by adding the prediction sample and the residual sample of the current block.
[239]
[240]
As in the above-described example, a merge candidate of the current block may be derived based on motion information of a block encoded/decoded by inter prediction before the current block. As an example, a merge candidate of the current block may be derived based on motion information of a neighboring block at a predefined position adjacent to the current block. The neighboring block is a block adjacent to the left of the current block, a block adjacent to the top of the current block, a block adjacent to the upper left corner of the current block, a block adjacent to the upper right corner of the current block, or the lower left of the current block. It may include at least one of blocks adjacent to the corner.
[241]
A merge candidate of the current block may be derived based on motion information of blocks other than the neighboring block. For convenience of explanation, a neighboring block at a predefined position adjacent to the current block is referred to as a first merge candidate block, and a block at a different position from the first merge candidate block is referred to as a second merge candidate block. .
[242]
The second merge candidate block may include at least one of a block encoded/decoded by inter prediction before the current block, a block adjacent to the first merge candidate block, or a block located on the same line as the first merge candidate block. . FIG. 15 shows a second merge candidate block adjacent to the first merge candidate block, and FIG. 16 shows a second merge candidate block positioned on the same line as the first merge candidate block.
[243]
When the first merge candidate block is not available, a merge candidate derived based on motion information of the second merge candidate block may be added to the merge candidate list. Or, even though at least one of the spatial merge candidate or the temporal merge candidate is added to the merge candidate list, if the number of merge candidates included in the merge candidate list is less than the maximum number of merge candidates, based on the motion information of the second merge candidate block. The derived merge candidate can be added to the merge candidate list.
[244]
15 is a diagram illustrating an example of deriving a merge candidate from a second merge candidate block when the first merge candidate block is not available.
[245]
If the first merge candidate block AN (here, N is 0-4) is not available, based on the motion information of the second merge candidate block BM (here, M is 0-6), a merge candidate of the current block is derived can do. That is, a merge candidate of the current block may be derived by replacing the unavailable first merge candidate block with the second merge candidate block.
[246]
A block positioned in a predetermined direction from the first merge candidate block among blocks adjacent to the first merge candidate block may be set as the second merge candidate block. The predefined direction may represent a left direction, a right direction, an upper direction, a lower direction, or a diagonal direction. A predefined direction may be set for each of the first merge candidate blocks. As an example, a predefined direction of a first merge candidate block adjacent to the left side of the current block may be a left direction. A predefined direction of the first merge candidate block adjacent to the top of the current block may be the top direction. The predefined direction of the first merge candidate block adjacent to the corner of the current block may include at least one of a left direction, an upper direction, or a diagonal direction.
[247]
For example, when A0 adjacent to the left of the current block is not available, a merge candidate of the current block may be derived based on B0 adjacent to A0. If A1 adjacent to the top of the current block is not available, a merge candidate of the current block may be derived based on B1 adjacent to A1. If A2 adjacent to the upper right corner of the current block is not available, a merge candidate of the current block may be derived based on B2 adjacent to A2. If A3 adjacent to the lower left corner of the current block is not available, a merge candidate of the current block may be derived based on B3 adjacent to A3. If A4 adjacent to the upper left corner of the current block is not available, a merge candidate of the current block may be derived based on at least one of B4 to B6 adjacent to A4.
[248]
The illustrated example of FIG. 15 is only for explaining an embodiment of the present invention, and does not limit the present invention. The location of the second merge candidate block may be set differently from the example illustrated in FIG. 15. As an example, the second merge candidate block adjacent to the first merge candidate block adjacent to the left of the current block may be located in the upper direction or the lower direction of the first merge candidate block. Alternatively, the second merge candidate block adjacent to the first merge candidate block adjacent to the top of the current block may be located in the left direction or the right direction of the first merge candidate block.
[249]
16 is a diagram illustrating an example of deriving a merge candidate from a second merge candidate block located on the same line as a first merge candidate block.
[250]
A block positioned on the same line as the first merge candidate block is a block positioned on the same horizontal line as the first merge candidate block, a block positioned on the same vertical line as the first merge candidate block, or the same block as the first merge candidate block. It may include at least one of blocks located on the diagonal. The y-coordinate positions of blocks located on the same horizontal line are the same. The x-coordinate positions of blocks located on the same vertical line are the same. The difference value of the x-coordinate position of blocks located on the same diagonal line is the same as the difference value of the y-coordinate position.
[251]
It is assumed that the location of the upper left sample of the current block is (0, 0), and the width and height of the current block are W and H, respectively. In FIG. 18, second merge candidate blocks ( For example, the location of B4, C6) has been shown to be determined. In addition, in FIG. 18, second merge candidate blocks positioned on the same horizontal line as the first merge candidate block based on the lowest block to the left of the coding block (eg, block A0 including (-1, H-1) coordinates) It is shown that the location of (eg, B1, C1) is determined.
[252]
As another example, the positions of the second merge candidate blocks are the leftmost block (e.g., a block including (0, -1) coordinates) above the coding block or a block (e.g., (W/2, -1) It may be determined based on a block including coordinates). In addition, the positions of the second merge candidate blocks are the topmost block on the left of the coding block (eg, a block including (-1, 0) coordinates) or a block located at the left center of the coding block (eg, (-1, H/2) ) May be determined based on a block including coordinates).
[253]
As another example, when there are a plurality of upper neighboring blocks adjacent to the upper end of the current block, the second merge candidate block may be determined using all or part of the plurality of upper neighboring blocks. For example, by using a block at a specific position among a plurality of upper neighboring blocks (eg, at least one of an upper neighboring block located at the leftmost, an upper neighboring block located at the rightmost, or an upper neighboring block located at the center) 2 Can determine the merge candidate block. The number of upper neighbor blocks used to determine the second merge candidate block among the plurality of upper neighbor blocks may be 1, 2, 3 or more. In addition, when there are a plurality of left neighboring blocks adjacent to the left of the current block, the second merge candidate block may be determined using all or part of the plurality of left neighboring blocks. As an example, a second block of a specific position among a plurality of left neighboring blocks (eg, at least one of a left neighboring block positioned at the bottom, a left neighboring block positioned at the top, or a left neighboring block positioned at the center) is used. A merge candidate block can be determined. The number of left neighboring blocks used to determine the second merge candidate block among the plurality of left neighboring blocks may be 1, 2, 3 or more.
[254]
According to the size and/or shape of the current block, the positions and/or the number of the upper neighboring block and/or the left neighboring block used to determine the second merge candidate block may be differently determined. For example, when the size of the current block is greater than or equal to the threshold value, the second merge candidate block may be determined based on the upper center block and/or the left center block. On the other hand, when the size of the current block is smaller than the threshold value, the second merge candidate block may be determined based on the upper rightmost block and/or the leftmost lowermost block. The threshold value may be an integer of 8, 16, 32, 64 or 128.
[255]
A first merge candidate list and a second merge candidate list may be configured, and motion compensation of the current block may be performed based on at least one of the first merge candidate list and the second merge candidate list.
[256]
The first merge candidate list includes at least one of a spatial merge candidate derived based on motion information of a neighboring block at a predefined position adjacent to the current block or a temporal merge candidate derived based on motion information of a collocated block. Can include.
[257]
The second merge candidate list may include a merge candidate derived based on motion information of the second merge candidate block.
[258]
In one embodiment of the present invention, the first merge candidate list is configured to include a merge candidate derived from the first merge candidate block, and the second merge candidate list is configured to include a merge candidate derived from the second merge candidate block Can be. As an example, in the example shown in FIG. 15, merge candidates derived from blocks A0 to A4 may be added to a first merge candidate list, and merge candidates derived from blocks B0 to B6 may be added to a second merge candidate list. . As an example, in the example shown in FIG. 16, merge candidates derived from blocks A0 to A4 are added to the first merge candidate list, and merge candidates derived from blocks B0 to B5 and C0 to C7 are added to the second merge candidate list. Can be added.
[259]
Alternatively, the second merge candidate list may include a merge candidate derived based on motion information of a block encoded/decoded by inter prediction before the current block. As an example, when motion compensation for a block whose encoding mode is inter prediction is performed, a merge candidate derived based on motion information of the block may be added to the second merge candidate list. When encoding/decoding of the current block is completed, motion information of the current block may be added to the second merge candidate list for inter prediction of the next block.
[260]
Indexes of merge candidates included in the second merge candidate list may be determined based on an order of addition to the second merge candidate list. As an example, the index allocated to the merge candidate added to the N-th second merge candidate list may have a value smaller than the index allocated to the merge candidate added to the N+1-th second merge candidate list. For example, the index of the N+1th merge candidate may be set to a value greater than the index of the Nth merge candidate. Alternatively, the index of the N-th merge candidate may be set as the index of the N+1-th merge candidate, and a value of the index of the N-th merge candidate may be subtracted by one.
[261]
Alternatively, the index allocated to the merge candidate added to the N-th second merge candidate list may have a value greater than the index allocated to the merge candidate added to the N+1-th second merge candidate list. For example, the index of the Nth merge candidate may be set as the index of the N+1th merge candidate, and the value of the index of the Nth merge candidate may be increased by 1.
[262]
Whether to add a merge candidate derived from the block to the second merge candidate list based on whether the motion information of the block on which motion compensation has been performed and the motion information of the merge candidate included in the second merge candidate list are the same You can decide. For example, when a merge candidate identical to the motion information of the block is included in the second merge candidate list, a merge candidate derived based on the motion information of the block may not be added to the second merge candidate list. Alternatively, if a merge candidate identical to the motion information of the block is included in the second merge candidate list, the merge candidate is deleted from the second merge candidate list, and a merge candidate derived based on the motion information of the block is removed. 2 Can be added to the merge candidate list.
[263]
When the number of merge candidates included in the second merge candidate list is the same as the maximum number of merge candidates, the merge candidate with the lowest index or the merge candidate with the highest index is deleted from the second merge candidate list, and motion information of the block is used. The merge candidate derived by may be added to the second merge candidate list. That is, after the oldest merge candidate among the merge candidates included in the second merge candidate list is deleted, a merge candidate derived based on motion information of the block may be added to the second merge candidate list.
[264]
The second merge candidate list may be initialized in units of CTUs, tiles, or slices. That is, a block included in a CTU different from the current block, a different tile, or a different slice may be set to be unavailable as the second merge candidate block. The maximum number of merge candidates that the second merge candidate list may include may be predefined in an encoder and a decoder. Alternatively, information indicating the maximum number of merge candidates that the second merge candidate list may include may be signaled through a bitstream.
[265]
One of the first merge candidate list and the second merge candidate list may be selected, and inter prediction of the current block may be performed using the selected merge candidate list. Specifically, based on the index information, any one of merge candidates included in the merge candidate list may be specified, and motion information of the current block may be obtained from the selected merge candidate.
[266]
Information specifying either of the first merge candidate list and the second merge candidate list may be signaled through a bitstream. The decoder may select one of a first merge candidate list and a second merge candidate list based on the information.
[267]
Alternatively, a merge candidate list having a larger number of available merge candidates may be selected from among the first merge candidate list and the second merge candidate list.
[268]
Alternatively, one of the first merge candidate list and the second merge candidate list may be selected based on at least one of the size, shape, or split depth of the current block.
[269]
Alternatively, a merge candidate list configured by adding (or appending) the other one to one of the first merge candidate list and the second merge candidate list may be used.
[270]
For example, inter prediction may be performed based on a merge candidate list including at least one merge candidate included in the first merge candidate list and at least one merge candidate included in the second merge candidate list.
[271]
As an example, a merge candidate included in the second merge candidate list may be added to the first merge candidate list. Alternatively, a merge candidate included in the first merge candidate list may be added to the second merge candidate.
[272]
When the number of merge candidates included in the first merge candidate list is less than the maximum number, or when the first merge candidate block is not available, the merge candidate included in the second merge candidate list may be added to the first merge candidate list. have.
[273]
Alternatively, when the first merge candidate block is not available, a merge candidate derived from a block adjacent to the first merge candidate block among merge candidates included in the second merge candidate list may be added to the first merge candidate list. Referring to FIG. 15, when A0 is not available, a merge candidate derived based on motion information of B0 among merge candidates included in the second merge candidate list may be added to the first merge candidate list. If A1 is not available, a merge candidate derived based on motion information of B1 among merge candidates included in the second merge candidate list may be added to the first merge candidate list. When A2 is not available, a merge candidate derived based on motion information of B2 among merge candidates included in the second merge candidate list may be added to the first merge candidate list. When A3 is not available, a merge candidate derived based on motion information of B3 among merge candidates included in the second merge candidate list may be added to the first merge candidate list. When A4 is not available, a merge candidate derived based on motion information of B4, B5, or B6 among the merge candidates included in the second merge candidate list may be added to the first merge candidate list.
[274]
Alternatively, a merge candidate to be added to the first merge candidate list may be determined according to priorities of merge candidates included in the second merge candidate list. The priority may be determined based on an index value allocated to each merge candidate. For example, when the number of merge candidates included in the first merge candidate list is less than the maximum number or when the first merge candidate block is not available, the merge candidate having the smallest index value among the merge candidates included in the second merge candidate list Alternatively, the merge candidate having the largest index value may be added to the first merge candidate list.
[275]
If a merge candidate with the highest priority and the same motion information among the merge candidates included in the second merge candidate list is included in the first merge candidate list, the merge candidate with the highest priority is the first merge candidate list May not be added to. And, a merge candidate having the next priority (e.g., a merge candidate assigned with an index value greater than 1 than the index value assigned to the merge candidate with the highest priority or 1 than the index value assigned to the merge candidate having the highest priority It may be determined whether the merge candidate to which this small index value is assigned) can be added to the first merge candidate list.
[276]
Alternatively, a merge candidate list including both a merge candidate derived based on motion information of the first merge candidate block and a merge candidate derived based on motion information of the second merge candidate block may be generated. The merge candidate list may be a combination of a first merge candidate list and a second merge candidate list.
[277]
For example, a merge candidate list may be generated by searching for a first merge candidate block and a second merge candidate block according to a predetermined search order.
[278]
17 to 20 are diagrams illustrating a search order of merge candidate blocks.
[279]
In FIGS. 17 to 20, a search order of a merge candidate is shown as follows.
[280]
A0 → A1 → A2 → A3 → A4 → B0 → B1 → B2 → B3 → B4 → (B5) → (B6)
[281]
B5 and B6 may be searched only when the B4 block is not available or when the number of merge candidates included in the merge candidate list is less than or equal to a preset number.
[282]
A search order different from the examples of FIGS. 17 to 20 may be set.
[283]
A combined merge candidate list including at least one merge candidate included in the first merge candidate list and at least one merge candidate included in the second merge candidate list may be generated. As an example, the combined merge candidate list may include N among merge candidates included in the first merge candidate list and M among merge candidates included in the second merge candidate list. N and M can represent the same number or different numbers. Alternatively, at least one of N or M may be determined based on at least one of the number of merge candidates included in the first merge candidate list or the number of merge candidates included in the second merge candidate list. Alternatively, information for determining at least one of N or M may be signaled through a bitstream. Either of N or M may be derived by subtracting the other from the maximum number of merge candidates of the combined merge candidate list.
[284]
Merge candidates added to the combined merge candidate list may be determined according to a predefined priority. The predefined priority may be determined based on indexes allocated to merge candidates.
[285]
Alternatively, a merge candidate to be added to the combined merge candidate list may be determined based on the association between the merge candidates. For example, if A0 included in the first merge candidate list is added to the combined merge candidate list, the merge candidate (eg, B0) adjacent to A0 may not be added to the combined merge list.
[286]
When the number of merge candidates included in the first merge candidate list is less than N, among merge candidates included in the second merge candidate list, more than M merge candidates may be added to the combined merge candidate list. For example, when N is 4 and M is 2, 4 of the merge candidates included in the first merge candidate list are added to the combined merge candidate list, and two of the merge candidates included in the second merge candidate list are added. It can be added to the combined merge candidate list. If the number of merge candidates included in the first merge candidate list is less than four, two or more merge candidates among the merge candidates included in the second merge candidate list may be added to the combined merge candidate list. If the number of merge candidates included in the second merge candidate list is less than 2, four or more of the merge candidates included in the first merge candidate list may be added to the combined merge candidate list.
[287]
That is, the value of N or M may be adjusted according to the number of merge candidates included in each merge candidate list. By adjusting the value of N or M, the total number of merge candidates included in the combined merge candidate list may be fixed. When the total number of merge candidates included in the combined merge candidate list is less than the maximum number of merge candidates, a combined merge candidate, an average merge candidate, or a zero motion vector candidate may be added.
[288]
[289]
Motion compensation of the current block may be performed using at least one of merge candidates included in the first merge candidate list and the second merge candidate list. The encoder may encode index information for specifying any one of a plurality of merge candidates. For example,'merge_idx' may be to specify any one of a plurality of merge candidates. As an example, Table 1 shows merge indexes of merge candidates derived from first merge candidate blocks and second merge candidate blocks shown in FIG. 16.
[290]
[Table 1]
Candidate for merge Merge index (merge_idx)
A1 0
A2 One
A3 2
A4 3
B1 4
B2 5
B3 6
B4 7
B5 8
C1 9
C2 10
C3 11
C4 12
C5 13
C6 14
C7 15
[291]
However, as the number of merge candidates included in the merge candidate list increases, the codeword for encoding the merge index becomes longer, resulting in a problem that the encoding/decoding efficiency decreases. In order to reduce the length of the codeword, a merge index may be determined using a prefix and a suffix. As an example, the merge index may be determined using merge_idx_prefix indicating the prefix of the merge index and merge_idx_suffix indicating the suffix of the merge index.
[292]
Table 2 shows a merge index prefix value and a merge index suffix value for each merge index, and Table 3 shows a process of determining a merge index based on the merge index prefix value and the merge index suffix value.
[293]
[Table 2]
Candidate for merge Merge index (merge_idx) Merge index prefix Merge index suffix
A1 0 0 -
A2 One One -
A3 2 2 -
A4 3 3 -
B1 4 4 0
B2 5 4 One
B3 6 4 2
B4 7 4 3
B5 8 4 4
C1 9 5 0
C2 10 5 One
C3 11 5 2
C4 12 5 3
C5 13 5 4
[294]
[Table 3]
if merge_idx_prefix <4
merge_idx = merge_idx_prefix
else
merge_idx = (merge_idx_prefix-3) << 2 + merge_idx_suffix
[295]
As shown in Tables 2 and 3, when the merge index prefix value is smaller than the threshold value, the merge index may be set equal to the value of the merge index prefix. On the other hand, when the value of the merge index prefix is ​​greater than the threshold value, the merge index may be determined by subtracting the reference value from the merge index prefix and adding the merge index suffix to the shifted value of the result. The reference value may be a threshold value or a value obtained by subtracting 1 from the threshold value.
[296]
In Tables 2 and 3, the threshold value is exemplified as 4. The threshold value may be determined based on at least one of the number of merge candidates included in the merge candidate list, the number of second merge candidate blocks, or the number of lines including second merge candidate blocks. Alternatively, the threshold value may be predefined by an encoder and a decoder.
[297]
Whether a prefix and a suffix are used to determine the merge index may be determined according to the number of merge candidates included in the merge candidate list or the maximum number of merge candidates that can be included in the merge candidate list. For example, when the maximum number of merge candidates that the merge candidate list can include is equal to or greater than a threshold value, a merge index prefix and a merge index suffix for determining a merge index may be signaled. On the other hand, when the maximum number of merge candidates is less than the threshold value, the merge index may be signaled.
[298]
[299]
A rectangular block may be divided into a plurality of triangular blocks. A merge candidate of triangular blocks may be derived based on a rectangular block block including triangular blocks. Triangular blocks can share the same merge candidate.
[300]
A merge index may be signaled for each of the triangular blocks. In this case, the triangular blocks may be set not to use the same merge candidate. As an example, a merge candidate used in the first triangular shape block cannot be used as a merge candidate in the second triangular shape block. Accordingly, the merge index of the second triangular shape block may specify any one of the remaining merge candidates excluding the merge candidate selected from the first triangular shape block.
[301]
[302]
A merge candidate may be derived based on a block having a predetermined shape or a predetermined size or larger. When the current block does not have a predetermined shape, or when the size of the current block is smaller than a predetermined size, a merge candidate of the current block may be derived based on a predetermined shape including the current block or a block having a predetermined size or larger. The predetermined shape may be square or non-square.
[303]
When the predetermined shape is a square shape, a merge candidate for a coding unit of an amorphous shape may be derived based on a coding unit of a square shape including a coding unit of the amorphous shape.
[304]
21 is a diagram illustrating an example in which a merge candidate of an amorphous block is derived based on a square block.
[305]
The merge candidate of the non-square block may be derived based on a square block including the non-square block. For example, merge candidates of the amorphous coding block 0 and the amorphous coding block 1 may be derived based on a square block including the coding block 0 and the coding block 1. That is, the location of the spatial neighboring block may be determined based on the location, width/height, or size of the square block. The merge candidates of the coding block 0 and the coding block 1 may be derived based on at least one of spatial neighboring blocks A0, A1, A2, A3, and A4 adjacent to the square block.
[306]
The temporal merge candidate may also be determined based on a square-shaped block. That is, a temporal neighboring block may be determined based on the position, width/height, or size of a square block. For example, the merge candidates of the coding block 0 and the coding block 1 may be derived based on a temporal neighboring block determined based on a square block.
[307]
Alternatively, one of a spatial merge candidate and a temporal merge candidate may be derived based on a square block, and another merge candidate may be derived based on a non-forward block. As an example, a spatial merge candidate of coding block 0 may be derived based on a forward block, whereas a temporal merge candidate of coding block 0 may be derived based on coding block 0.
[308]
A plurality of blocks included in a block having a predetermined shape or a predetermined size or larger may share a merge candidate. As an example, in the example shown in FIG. 21, at least one of a spatial merge candidate or a temporal merge candidate of the coding block 0 and the coding block 1 may be the same.
[309]
The predetermined shape may be an amorphous shape such as 2NxN or Nx2N. When the predetermined shape is non-square, the merge candidate of the current block may be derived based on the non-square block including the current block. For example, when the current block has a 2Nxn type (where n is 1/2N), a merge candidate of the current block may be derived based on a 2NxN type amorphous block. Alternatively, when the current block is an nx2N type, a merge candidate for the current block may be derived based on an Nx2N type amorphous block.
[310]
Information indicating a predetermined shape or a predetermined size may be signaled through a bitstream. For example, information indicating either amorphous form or a square form may be signaled through a bitstream.
[311]
Alternatively, an encoder and a decoder may determine a predetermined shape or a predetermined size according to a predefined rule.
[312]
[313]
When the child node does not satisfy the predetermined condition, a merge candidate of the child node may be derived based on the parent node that satisfies the predetermined condition. Here, the predetermined condition is at least one of whether the block is a block generated as a result of quad-tree division, the size of the block, the shape of the block, whether it is outside the picture boundary, or whether the depth difference between the child node and the parent node is equal to or greater than a predetermined value It can contain one.
[314]
As an example, the predetermined condition may include whether the block is generated as a result of quad-tree division and whether the block is a square coding block having a predetermined size or more. If the current block is generated by binary tree division or triple tree division, a merge candidate of the current block may be derived based on a higher node block including the current block and satisfying the predetermined condition. When there is no upper node block satisfying the predetermined condition, a current block, a block having a predetermined size or more including the current block, or an upper node block including the current block and having a depth difference of 1 from the current block is selected. A merge candidate of the current block can be derived based on the criteria.
[315]
22 is a diagram illustrating an example of deriving a merge candidate based on an upper node block.
[316]
Block 0 and block 1 are generated by dividing a square block based on a binary tree. The merge candidates of block 0 and block 1 may be derived based on a neighboring block (ie, at least one of A0, A1, A2, A3, or A4) determined based on an upper node block including block 0 and block 1. As a result, block 0 and block 1 may use the same spatial merge candidate.
[317]
An upper node block including blocks 2 and 3 and block 4 may be generated by dividing a square block based on a binary tree. Also, blocks 2 and 3 may be generated by dividing a block having an amorphous form based on a binary tree. The merge candidates of blocks 2, 3, and 4 having an amorphous form may be derived based on an upper node block including them. That is, a merge candidate based on a neighboring block (e.g., at least one of B0, B1, B2, B3, or B4) determined based on the position, width/height, or size of a square block including block 2, block 3, and block 4 Can induce As a result, blocks 2, 3, and 4 may use the same spatial merge candidate.
[318]
A temporal merge candidate for a non-square type of blow can be derived based on the upper node block. For example, temporal merge candidates for block 0 and block 1 may be derived based on a square block including blocks 0 and 1. Temporal merge candidates for blocks 2, 3, and 4 may be derived based on a square block including blocks 2, 3, and 4. Also, the same temporal merge candidate derived from a temporal neighboring block determined based on a quad tree block unit may be used.
[319]
Lower node blocks included in the upper node block may share at least one of a spatial merge candidate and a temporal merge candidate. For example, lower node blocks included in the upper node block may use the same merge candidate list.
[320]
Alternatively, at least one of the spatial merge candidate and the temporal merge candidate may be derived based on a lower node block, and the other may be derived based on an upper node block. For example, spatial merge candidates for block 0 and block 1 may be derived based on an upper node block. On the other hand, a temporal merge candidate for block 0 may be derived based on block 0, and a temporal merge candidate for block 1 may be derived based on block 1.
[321]
Alternatively, when the number of samples included in the lower node block is smaller than the predefined number, the merge candidate may be derived based on the upper node block including samples greater than or equal to the predefined number. For example, when at least one of the lower node blocks generated based on at least one of quad tree division, binary tree division, or triple tree division is smaller than a preset size, at least one of the lower node blocks is a non-forward block. If the upper node block does not deviate from the picture boundary, or if the width or height of the upper node block is equal to or greater than a predefined value, at least one condition is satisfied, a predetermined number of samples (e.g., 64, 128, 256). Lower node blocks included in the upper node block may share merge candidates derived based on the higher node block.
[322]
A merge candidate may be derived based on one of the lower node blocks, and other lower node blocks may be set to use the merge candidate. The lower node blocks may be included in a block having a predetermined shape or a predetermined size or larger. For example, the lower node blocks may share a merge candidate list derived based on any one of the lower node blocks. Information on a lower node block that is a criterion for derivation of a merge candidate may be signaled through a bitstream. The information may be index information indicating any one of lower node blocks. Alternatively, a lower node block as a criterion for deriving a merge candidate may be determined based on at least one of a location, size, shape, or scan order of the lower node blocks.
[323]
Information indicating whether lower node blocks share the merge candidate list derived based on the higher node block may be signaled through a bitstream. Based on the information, it may be determined whether a merge candidate of a block not of a predetermined shape or a block having a size smaller than a predetermined size is derived based on an upper node block including the block. Alternatively, it may be determined whether to derive a merge candidate based on an upper node block according to a rule predefined in the encoder and the decoder.
[324]
When a neighboring block adjacent to the current block exists in a predefined area, it may be determined that the neighboring block cannot be used as a spatial merge candidate. The predefined area may be a parallel processing area defined for parallel processing between blocks. The parallel processing region may also be referred to as a merge estimation region (MER). For example, when a neighboring block adjacent to the current block is included in the same merge induction region as the current block, it may be determined that the neighboring block is unavailable. In order to determine whether the current block and the neighboring block are included in the same merge induction region, a shift operation may be performed. Specifically, the current block and the neighboring block are included in the same merge induction region based on whether the value obtained by shifting the position of the upper left reference sample of the current block and the value obtained by shifting the position of the upper left reference sample of the neighboring block are the same. It can be determined whether or not.
[325]
23 is a diagram for describing an example in which availability of a spatial neighboring block is determined based on a merge induction region.
[326]
In FIG. 23, it is shown that the merge induction region has an Nx2N shape.
[327]
A merge candidate of block 1 may be derived based on a spatial neighboring block adjacent to block 1. Spatial neighboring blocks may include B0, B1, B2, B3, and B4. In this case, it may be determined that spatial neighboring blocks B0 and B3 included in the same merge induction region as block 1 are not available as merge candidates. Accordingly, the merge candidate of block 1 may be derived from at least one of spatial neighboring blocks B1, B2, and B4 excluding spatial neighboring blocks B0 and B3.
[328]
The merge candidate of block 3 may derive the merge candidate of block 3 based on a spatial neighboring block adjacent to block 3. The spatial neighboring block may include C0, C1, C2, C3, and C4. In this case, it may be determined that the spatial neighboring block C0 included in the merge induction region identical to that of block 3 is not available as a merge candidate. Accordingly, the merge candidate of block 3 may be derived from at least one of spatial neighboring blocks C1, C2, C3, and C4 excluding the spatial neighboring block C0.
[329]
A merge candidate of a block included in the merge induction area may be derived based on at least one of the location, size, width, or height of the merge induction area. For example, a merge candidate of a plurality of blocks included in the merge induction region may be derived from at least one of a spatial neighboring block or a temporal neighboring block determined based on at least one of a location, size, width, or height of the merge induction region. . Blocks included in the merge induction region may share the same merge candidate.
[330]
24 is a diagram illustrating an example in which a merge candidate is derived based on a merge induction region.
[331]
When a plurality of coding units are included in the merge induction region, merge candidates of the plurality of coding units may be derived based on the merge induction region. That is, by treating the merge induction region as a coding unit, a merge candidate may be derived based on the location, size, or width/height of the merge induction region.
[332]
For example, a merge candidate of coding unit 0 (CU0) and coding unit 1 (CU1) having a size of (n/2)xN (where n is N/2) included in a merge induction region having a size of (N/2)xN May be derived based on the merge induction region. That is, the merge candidates of the coding unit 0 and the coding unit 1 may be derived from at least one of neighboring blocks C0, C1, C2 C3, and C4 adjacent to the merge induction region.
[333]
For example, merge candidates of nxn-sized coding unit 2 (CU2), coding unit 3 (CU3), coding unit 4 (CU4), and coding unit 5 (CU5) included in the merge induction region of NxN size are based on the merge induction region. Can be induced by That is, the merge candidates of Coding Unit 2, Coding Unit 3, Coding Unit 4, and Coding Unit 5 may be derived from at least one of neighboring blocks C0, C1, C2, C3, or C4 adjacent to the merge induction region.
[334]
The shape of the merge induction region may be square or non-square. For example, a square coding unit (or prediction unit) or an amorphous coding unit (or prediction unit) may be determined as the merge induction region. The width and height ratio of the merge induction area can be limited so as not to deviate from a predetermined range. For example, the merge induction region may not have an amorphous shape in which the width and height ratio exceeds 2 or the amorphous shape in which the width and height ratio is less than 1/2. That is, the amorphous merge induction region may be in the form of 2NxN or Nx2N. Information on the width and height ratio restrictions may be signaled through a bitstream. Alternatively, the limit of the width and height ratio may be predefined in an encoder and a decoder.
[335]
At least one of information indicating the shape of the merge induction region or information indicating the size of the merge induction region may be signaled through a bitstream. For example, at least one of information indicating the shape of the merge induction region or information indicating the size of the merge induction region may be signaled through a slice header, a tile group header, a picture parameter, or a sequence parameter.
[336]
The shape of the merge induction region or the size of the merge induction region may be updated in units of a sequence, picture, slice, tile group, tile, or block (CTU). When the shape of the merge induction region or the size of the merge induction region is different from that of the previous unit, information indicating the new shape of the merge induction region or the new size of the merge induction region may be signaled through the bitstream.
[337]
At least one or more blocks may be included in the merge induction region. The blocks included in the merge induction region may be square or non-square. The maximum number or minimum number of blocks that the merge induction region can contain may be determined. For example, the merge induction region may include three, four, or more CUs. The determination may be determined based on information signaled through the bitstream. Alternatively, the maximum number or minimum number of blocks that can be included in the merge induction region may be predefined in the encoder and decoder.
[338]
Parallel processing of the blocks may be allowed in at least one of a case where the number of blocks included in the merge induction region is smaller than the maximum number or larger than the minimum number. For example, when the number of blocks included in the merge induction region is less than the maximum number or when the number of blocks included in the merge induction region is greater than the minimum number, a merge candidate of the blocks may be derived based on the merge induction region. I can. When the number of blocks included in the merge induction area is greater than the maximum number or when the number of blocks included in the merge induction area is less than the minimum number, the merge candidate of each block is the size, position, width or It can be derived based on height.
[339]
Information indicating the shape of the merge induction region may include a 1-bit flag. For example, the syntax'isrectagular_mer_flag' may indicate that the merge candidate region is a square or amorphous. A value of isrectagular_mer_flag of 1 indicates that the merge induction region is non-square, and a value of isrectagular_mer_flag of 0 indicates that the merge induction region is of a square shape.
[340]
When the information indicates that the merge induction region is non-square, information indicating at least one of the width, height or width and height ratio of the merge induction region may be signaled through a bitstream. Based on this, the size and/or shape of the merge induction region may be determined.
[341]
[342]
It is within the scope of the present invention to apply the embodiments described centering on the decoding process or the encoding process to the encoding process or the decoding process. It is also within the scope of the present invention to change the embodiments described in a predetermined order in a different order from those described.
[343]
The above-described embodiment has been described based on a series of steps or flow charts, but this does not limit the time series order of the invention, and may be performed simultaneously or in a different order as necessary. In addition, each of the components (eg, units, modules, etc.) constituting the block diagram in the above-described embodiment may be implemented as a hardware device or software, or a plurality of components are combined to form a single hardware device or software. It can also be implemented. The above-described embodiments may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic-optical media such as floptical disks. media), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The hardware device may be configured to operate as one or more software modules to perform processing according to the present invention, and vice versa.
Industrial availability
[344]
The present invention can be applied to an electronic device capable of encoding/decoding an image.
Claims
[Claim 1]
Deriving merge candidates from neighboring blocks adjacent to the current block; Generating a first merge candidate list including the merge candidates, when the number of merge candidates included in the first merge candidate list is less than a predetermined value, the merge candidate included in the second merge candidate list is the first Added to merge candidate list; Decoding information for specifying any one of the merge candidates included in the first merge candidate list; And deriving motion information of the current block from a merge candidate to which an index determined by the information is assigned.
[Claim 2]
The method of claim 1, wherein the information includes an index prefix and an index suffix.
[Claim 3]
The method of claim 2, wherein when the value of the index prefix is ​​smaller than a threshold value, the index is set equal to the index prefix.
[Claim 4]
The image decoding of claim 3, wherein when the value of the index prefix is ​​equal to or greater than the threshold value, the index is derived by adding a value of the index suffix to a value derived based on the index prefix. Way.
[Claim 5]
The image decoding method of claim 3, wherein the threshold value is determined based on the number of merge candidates included in the first merge candidate list.
[Claim 6]
The video decoding method of claim 1, wherein the second merge candidate list includes merge candidates derived from blocks not adjacent to the current block.
[Claim 7]
The video decoding method of claim 6, wherein the non-adjacent block is placed on the same line as a block adjacent to the current block.
[Claim 8]
Deriving merge candidates from neighboring blocks adjacent to the current block; Generating a first merge candidate list including the merge candidates, when the number of merge candidates included in the first merge candidate list is less than a predetermined value, the merge candidate included in the second merge candidate list is the first Added to merge candidate list; Encoding information for specifying any one of the merge candidates included in the first merge candidate list; And deriving motion information of the current block from a merge candidate to which an index determined by the information is assigned.
[Claim 9]
The method of claim 8, wherein the information includes an index prefix and an index suffix.
[Claim 10]
10. The method of claim 9, wherein when the value of the index prefix is ​​smaller than a threshold value, the index is set equal to the index prefix.
[Claim 11]
The image encoding method of claim 10, wherein when the value of the index prefix is ​​equal to or greater than the threshold value, the index is derived by adding a value of the index suffix to a value derived based on the index prefix. Way.
[Claim 12]
11. The method of claim 10, wherein the threshold value is determined based on the number of merge candidates included in the first merge candidate list.
[Claim 13]
The image encoding method of claim 8, wherein the second merge candidate list includes merge candidates derived from blocks not adjacent to the current block.
[Claim 14]
14. The method of claim 13, wherein the non-adjacent block is placed on the same line as a block adjacent to the current block.
[Claim 15]
A decoding unit that decodes information specifying any one of merge candidates included in the first merge candidate list; And deriving merge candidates from neighboring blocks adjacent to the current block, generating the first merge candidate list including the merge candidates, and obtaining motion information of the current block from a merge candidate to which an index determined by the information is assigned. Including an inter prediction unit to induce, wherein when the number of merge candidates included in the first merge candidate list is less than a predetermined value, adding a merge candidate included in the second merge candidate list to the first merge candidate list A video decoding device, characterized in that.