Title of Invention: Video signal processing method and apparatus
technical field
[One]
The present invention relates to a video signal processing method and apparatus.
background
[2]
Recently, the demand for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images is increasing in various application fields. As the image data becomes higher resolution and higher quality, the amount of data relatively increases compared to the existing image data. The storage cost will increase. High-efficiency image compression techniques can be used to solve these problems that occur as image data becomes high-resolution and high-quality.
[3]
Inter-screen prediction technology that predicts pixel values ​​included in the current picture from pictures before or after the current picture with image compression technology, intra-picture prediction technology that predicts pixel values ​​included in the current picture using pixel information in the current picture, Various techniques exist, such as entropy encoding technology in which a short code is assigned to a value with a high frequency of occurrence and a long code is assigned to a value with a low frequency of occurrence.
[4]
Meanwhile, as the demand for high-resolution images increases, the demand for stereoscopic image content as a new image service is also increasing. A video compression technique for effectively providing high-resolution and ultra-high-resolution stereoscopic image content is being discussed.
DETAILED DESCRIPTION OF THE INVENTION
technical challenge
[5]
An object of the present invention is to provide a method and apparatus for intra prediction in encoding/decoding a video signal.
[6]
An object of the present invention is to provide a method and apparatus for intra prediction based on a palette mode in encoding/decoding a video signal.
[7]
The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. will be able
means of solving the problem
[8]
A video signal encoding/decoding method and apparatus according to the present invention configures a palette table for a palette mode of a current block, determines a palette index in units of pixels of the current block, and based on the palette table and the palette index , it is possible to predict the pixel of the current block.
[9]
In the video signal encoding/decoding method and apparatus according to the present invention, the palette mode may be selectively used based on a first flag indicating whether the palette mode is used.
[10]
In the video signal encoding/decoding method and apparatus according to the present invention, the first flag indicating whether the palette mode is used is adaptively encoded based on at least one of a prediction mode of the current block or a size of the current block /can be decrypted.
[11]
In the video signal encoding/decoding method and apparatus according to the present invention, the palette table of the current block is composed of at least one palette entry and an index identifying each palette entry, and the palette entry of the palette table is the current block It may be configured using a second flag indicating whether to reuse a palette entry belonging to the palette table and the previous palette table of the previously decoded previous block.
[12]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the palette table is initialized in units of CTU rows, and when the CTU to which the current block belongs is the first CTU of the current CTU row, the CTU to which the current block belongs The palette table of may be initialized using the palette table of the first CTU of the previous CTU row.
[13]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the palette index of the current block may be determined using at least one of an index mode and a copy mode.
[14]
In the video signal encoding/decoding method and apparatus according to the present invention, the index mode is a mode using encoded/decoded palette index information to specify the palette index of the current block, and the copy mode is in a predetermined scan order. It may be a mode using the palette index of neighboring pixels according to the
[15]
In a computer-readable recording medium for storing a bitstream encoded by the video encoding method according to the present invention, the video encoding method includes: constructing a palette table for a palette mode of a current block; a pixel of the current block Determining the palette index in units and based on the palette table and the palette index, may include predicting the pixel of the current block.
Effects of the Invention
[16]
According to the present invention, by configuring the palette table of the current block based on the previous palette table, it is possible to improve the encoding/decoding efficiency of the palette mode.
[17]
According to the present invention, the encoding/decoding efficiency of the palette mode can be improved by adaptively using the scan order of the palette mode.
[18]
According to the present invention, it is possible to improve the encoding/decoding efficiency of the palette index for each pixel of the current block.
[19]
The effects obtainable in the present invention are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present invention belongs from the following description. will be.
Brief description of the drawing
[20]
1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.
[21]
2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present invention.
[22]
3 to 5 are diagrams for explaining the concept of the palette mode according to the present disclosure.
[23]
6 illustrates a method of performing intra prediction based on a palette mode according to the present disclosure.
[24]
7 to 11 show a method of configuring a pallet table according to the present disclosure.
[25]
12 illustrates a method of signaling a palette prediction flag in the form of a binary vector based on run length encoding as an embodiment to which the present invention is applied.
[26]
13 to 18 illustrate a method of encoding/decoding a palette index according to a scan order according to the present disclosure.
Best mode for carrying out the invention
[27]
A video signal encoding/decoding method and apparatus according to the present invention configures a palette table for a palette mode of a current block, determines a palette index in units of pixels of the current block, and based on the palette table and the palette index , it is possible to predict the pixel of the current block.
[28]
In the video signal encoding/decoding method and apparatus according to the present invention, the palette mode may be selectively used based on a first flag indicating whether the palette mode is used.
[29]
In the video signal encoding/decoding method and apparatus according to the present invention, the first flag indicating whether the palette mode is used is adaptively encoded based on at least one of a prediction mode of the current block or a size of the current block /can be decrypted.
[30]
In the video signal encoding/decoding method and apparatus according to the present invention, the palette table of the current block is composed of at least one palette entry and an index identifying each palette entry, and the palette entry of the palette table is the current block It may be configured using a second flag indicating whether to reuse a palette entry belonging to the palette table and the previous palette table of the previously decoded previous block.
[31]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the palette table is initialized in units of CTU rows, and when the CTU to which the current block belongs is the first CTU of the current CTU row, the CTU to which the current block belongs The palette table of may be initialized using the palette table of the first CTU of the previous CTU row.
[32]
In the method and apparatus for encoding/decoding a video signal according to the present invention, the palette index of the current block may be determined using at least one of an index mode and a copy mode.
[33]
In the video signal encoding/decoding method and apparatus according to the present invention, the index mode is a mode using encoded/decoded palette index information to specify the palette index of the current block, and the copy mode is in a predetermined scan order. It may be a mode using the palette index of neighboring pixels according to the
[34]
In a computer-readable recording medium for storing a bitstream encoded by the video encoding method according to the present invention, the video encoding method includes: constructing a palette table for a palette mode of a current block; a pixel of the current block Determining the palette index in units and based on the palette table and the palette index, may include predicting the pixel of the current block.
Modes for carrying out the invention
[35]
Since the present invention can have various changes and can have various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents and substitutes included in the spirit and scope of the present invention. In describing each figure, like reference numerals have been used for like elements.
[36]
Terms such as first, second, etc. may be used to describe various elements, but the elements should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.
[37]
When a component is referred to as being “connected” or “connected to” another component, it is understood that the other component may be directly connected or connected to the other component, but other components may exist in between. it should be On the other hand, when it is said that a certain element is "directly connected" or "directly connected" to another element, it should be understood that no other element is present in the middle.
[38]
The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as “comprise” or “have” are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other features It is to be understood that this does not preclude the possibility of addition or existence of numbers, steps, operations, components, parts, or combinations thereof.
[39]
Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings. Hereinafter, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.
[40]
[41]
1 is a block diagram illustrating an image encoding apparatus according to an embodiment of the present invention.
[42]
Referring to FIG. 1 , the image encoding apparatus 100 includes a picture division unit 110 , prediction units 120 and 125 , a transform unit 130 , a quantization unit 135 , a rearrangement unit 160 , and an entropy encoding unit ( 165 ), an inverse quantization unit 140 , an inverse transform unit 145 , a filter unit 150 , and a memory 155 .
[43]
Each of the constituent units shown in FIG. 1 is independently illustrated to represent different characteristic functions in the image encoding apparatus, and does not mean that each constituent unit is composed of separate hardware or one software constituent unit. That is, each component is listed as each component for convenience of description, and at least two components of each component are combined to form one component, or one component can be divided into a plurality of components to perform a function, and each of these components Integrated embodiments and separate embodiments of components are also included in the scope of the present invention without departing from the essence of the present invention.
[44]
In addition, some of the components are not essential components for performing essential functions in the present invention, but may be optional components for merely improving performance. The present invention can be implemented by including only essential components to implement the essence of the present invention, except for components used for performance improvement, and a structure including only essential components excluding optional components used for performance improvement Also included in the scope of the present invention.
[45]
The picture divider 110 may divide the input picture into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). The picture splitter 110 divides one picture into a combination of a plurality of coding units, prediction units, and transformation units, and combines one coding unit, prediction unit, and transformation unit based on a predetermined criterion (eg, a cost function). can be selected to encode the picture.
[46]
For example, one picture may be divided into a plurality of coding units. In order to split a coding unit in a picture, a recursive tree structure such as a quad tree structure may be used. A coding in which one image or a largest coding unit is used as a root and is divided into other coding units. A unit may be divided having as many child nodes as the number of divided coding units. A coding unit that is no longer split according to certain restrictions becomes a leaf node. That is, if it is assumed that only square splitting is possible for one coding unit, one coding unit may be split into up to four different coding units.
[47]
Hereinafter, in an embodiment of the present invention, a coding unit may be used as a unit for performing encoding or may be used as a meaning for a unit for performing decoding.
[48]
A prediction unit may be split in the form of at least one square or rectangle of the same size within one coding unit, and one prediction unit among the split prediction units within one coding unit is a prediction of another. It may be divided to have a shape and/or size different from that of the unit.
[49]
When a prediction unit for performing intra prediction based on a coding unit is generated, if it is not the smallest coding unit, intra prediction may be performed without dividing the prediction unit into a plurality of prediction units NxN.
[50]
The prediction units 120 and 125 may include an inter prediction unit 120 performing inter prediction and an intra prediction unit 125 performing intra prediction. Whether to use inter prediction or to perform intra prediction for a prediction unit may be determined, and specific information (eg, intra prediction mode, motion vector, reference picture, etc.) according to each prediction method may be determined. In this case, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific content are determined may be different. For example, a prediction method and a prediction mode may be determined in a prediction unit, and prediction may be performed in a transformation unit. A residual value (residual block) between the generated prediction block and the original block may be input to the transform unit 130 . Also, prediction mode information, motion vector information, etc. used for prediction may be encoded by the entropy encoder 165 together with a residual value and transmitted to a decoding apparatus. When a specific encoding mode is used, it is also possible to encode the original block as it is without generating a prediction block through the predictors 120 and 125 and transmit it to the decoder.
[51]
The inter prediction unit 120 may predict a prediction unit based on information on at least one of a picture before or after a picture of the current picture, and in some cases, prediction based on information of a partial region in the current picture for which encoding has been completed Units can also be predicted. The inter prediction unit 120 may include a reference picture interpolator, a motion prediction unit, and a motion compensator.
[52]
The reference picture interpolator may receive reference picture information from the memory 155 and generate pixel information of integer pixels or less in the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/4 pixels may be used. In the case of a color difference signal, a DCT-based 4-tap interpolation filter in which filter coefficients are different to generate pixel information of integer pixels or less in units of 1/8 pixels may be used.
[53]
The motion prediction unit may perform motion prediction based on the reference picture interpolated by the reference picture interpolator. As a method for calculating the motion vector, various methods such as Full search-based Block Matching Algorithm (FBMA), Three Step Search (TSS), and New Three-Step Search Algorithm (NTS) may be used. The motion vector may have a motion vector value of 1/2 or 1/4 pixel unit based on the interpolated pixel. The motion prediction unit may predict the current prediction unit by using a different motion prediction method. Various methods, such as a skip method, a merge method, an AMVP (Advanced Motion Vector Prediction) method, an intra block copy method, etc., may be used as the motion prediction method.
[54]
The intra prediction unit 125 may generate a prediction unit based on reference pixel information around the current block, which is pixel information in the current picture. When a neighboring block of the current prediction unit is a block on which inter prediction is performed, and thus a reference pixel is a pixel on which inter prediction is performed, a reference pixel included in the block on which inter prediction is performed is a reference pixel of the block on which intra prediction is performed. information can be used instead. That is, when the reference pixel is not available, the unavailable reference pixel information may be replaced with at least one reference pixel among the available reference pixels.
[55]
In intra prediction, the prediction mode may have a directional prediction mode in which reference pixel information is used according to a prediction direction and a non-directional mode in which directional information is not used when prediction is performed. A mode for predicting luminance information and a mode for predicting chrominance information may be different, and intra prediction mode information used for predicting luminance information or predicted luminance signal information may be utilized to predict chrominance information.
[56]
When intra prediction is performed, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit based on the pixel present at the left side, the pixel present at the upper left corner, and the pixel present at the upper side of the prediction unit can be performed. However, when the size of the prediction unit is different from the size of the transformation unit when intra prediction is performed, intra prediction may be performed using a reference pixel based on the transformation unit. In addition, intra prediction using NxN splitting may be used only for the smallest coding unit.
[57]
Also, the intra prediction unit 125 may perform intra prediction based on the palette mode, which will be described in detail with reference to FIGS. 3 to 18 .
[58]
The intra prediction method may generate a prediction block after applying an adaptive intra smoothing (AIS) filter to a reference pixel according to a prediction mode. The type of AIS filter applied to the reference pixel may be different. In order to perform the intra prediction method, the intra prediction mode of the current prediction unit may be predicted from the intra prediction mode of the prediction unit existing around the current prediction unit. When the prediction mode of the current prediction unit is predicted using mode information predicted from the neighboring prediction unit, if the intra prediction mode of the current prediction unit and the neighboring prediction unit are the same, the current prediction unit and the neighboring prediction unit are used using predetermined flag information It is possible to transmit information indicating that the prediction modes of , and if the prediction modes of the current prediction unit and the neighboring prediction units are different from each other, entropy encoding may be performed to encode prediction mode information of the current block.
[59]
In addition, a residual block including residual information that is a difference value from the original block of the prediction unit and the prediction unit in which prediction is performed based on the prediction unit generated by the prediction units 120 and 125 may be generated. The generated residual block may be input to the transform unit 130 .
[60]
The transform unit 130 converts the original block and the residual block including residual information of the prediction units generated by the prediction units 120 and 125 to DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT and It can be converted using the same conversion method. Whether to apply DCT, DST, or KLT to transform the residual block may be determined based on intra prediction mode information of a prediction unit used to generate the residual block.
[61]
The quantization unit 135 may quantize values ​​transformed in the frequency domain by the transform unit 130 . The quantization coefficient may change according to blocks or the importance of an image. The value calculated by the quantization unit 135 may be provided to the inverse quantization unit 140 and the rearrangement unit 160 .
[62]
The reordering unit 160 may rearrange the coefficient values ​​on the quantized residual values.
[63]
The rearranging unit 160 may change the two-dimensional block form coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearranging unit 160 may scan from DC coefficients to coefficients in a high frequency region using a zig-zag scan method and may change it into a one-dimensional vector form. A vertical scan for scanning a two-dimensional block shape coefficient in a column direction and a horizontal scan for scanning a two-dimensional block shape coefficient in a row direction may be used instead of the zig-zag scan according to the size of the transform unit and the intra prediction mode. That is, it may be determined whether any of the zig-zag scan, the vertical scan, and the horizontal scan is used according to the size of the transform unit and the intra prediction mode.
[64]
The entropy encoding unit 165 may perform entropy encoding based on the values ​​calculated by the reordering unit 160 . For entropy encoding, various encoding methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be used.
[65]
The entropy encoding unit 165 receives the residual value coefficient information and block type information, prediction mode information, division unit information, prediction unit information and transmission unit information, motion of the coding unit from the reordering unit 160 and the prediction units 120 and 125 . Various information such as vector information, reference frame information, interpolation information of a block, and filtering information may be encoded.
[66]
The entropy encoder 165 may entropy-encode the coefficient values ​​of the coding units input from the reordering unit 160 .
[67]
The inverse quantizer 140 and the inverse transform unit 145 inversely quantize the values ​​quantized by the quantizer 135 and inversely transform the values ​​transformed by the transform unit 130 . The residual values ​​generated by the inverse quantizer 140 and the inverse transform unit 145 are combined with the prediction units predicted through the motion estimation unit, the motion compensator, and the intra prediction unit included in the prediction units 120 and 125 and restored. You can create a Reconstructed Block.
[68]
The filter unit 150 may include at least one of a deblocking filter, an offset correcting unit, and an adaptive loop filter (ALF).
[69]
The deblocking filter may remove block distortion caused by the boundary between blocks in the reconstructed picture. In order to determine whether to perform deblocking, it may be determined whether to apply the deblocking filter to the current block based on pixels included in several columns or rows included in the block. When a deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. In addition, in applying the deblocking filter, horizontal filtering and vertical filtering may be concurrently processed when performing vertical filtering and horizontal filtering.
[70]
The offset correcting unit may correct an offset from the original image in units of pixels with respect to the image on which the deblocking has been performed. In order to perform offset correction on a specific picture, a method of dividing pixels included in an image into a certain number of regions, determining the region to be offset and applying the offset to the region, or taking edge information of each pixel into consideration can be used to apply
[71]
Adaptive loop filtering (ALF) may be performed based on a value obtained by comparing the filtered reconstructed image and the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the corresponding group is determined, and filtering can be performed differentially for each group. As for information on whether to apply ALF, the luminance signal may be transmitted for each coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary according to each block. Also, the ALF filter of the same type (fixed type) may be applied regardless of the characteristics of the block to be applied.
[72]
The memory 155 may store the reconstructed block or picture calculated through the filter unit 150 , and the stored reconstructed block or picture may be provided to the predictors 120 and 125 when inter prediction is performed.
[73]
[74]
2 is a block diagram illustrating an image decoding apparatus according to an embodiment of the present invention.
[75]
2, the image decoding apparatus 200 includes an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, and a filter unit ( 240) and a memory 245 may be included.
[76]
When an image bitstream is input by the image encoding apparatus, the input bitstream may be decoded by a procedure opposite to that of the image encoding apparatus.
[77]
The entropy decoding unit 210 may perform entropy decoding in a procedure opposite to that performed by the entropy encoding unit of the image encoding apparatus. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) may be applied corresponding to the method performed by the image encoding apparatus.
[78]
The entropy decoding unit 210 may decode information related to intra prediction and inter prediction performed by the encoding apparatus.
[79]
The reordering unit 215 may perform rearrangement based on a method of rearranging the entropy-decoded bitstream by the entropy decoding unit 210 by the encoder. Coefficients expressed in a one-dimensional vector form may be restored and rearranged in a two-dimensional block form. The reordering unit 215 may receive information related to coefficient scanning performed by the encoder and perform the reordering by performing a reverse scanning method based on the scanning order performed by the corresponding encoder.
[80]
The inverse quantizer 220 may perform inverse quantization based on the quantization parameter provided by the encoding apparatus and the reordered coefficient values ​​of the blocks.
[81]
The inverse transform unit 225 may perform inverse transforms, ie, inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, ie, DCT, DST, and KLT, on the quantization result performed by the image encoding apparatus. Inverse transform may be performed based on a transmission unit determined by the image encoding apparatus. The inverse transform unit 225 of the image decoding apparatus may selectively perform a transformation technique (eg, DCT, DST, KLT) according to a plurality of pieces of information such as a prediction method, a size of a current block, and a prediction direction.
[82]
The prediction units 230 and 235 may generate a prediction block based on the prediction block generation related information provided from the entropy decoding unit 210 and previously decoded block or picture information provided from the memory 245 .
[83]
As described above, when intra prediction is performed in the same manner as in the operation in the image encoding apparatus, when the size of the prediction unit and the size of the transformation unit are the same, the pixel present at the left side of the prediction unit, the pixel present at the upper left side, and the upper side Intra prediction is performed on the prediction unit based on existing pixels, but when the size of the prediction unit is different from the size of the transformation unit when intra prediction is performed, intra prediction is performed using the reference pixel based on the transformation unit can do. In addition, intra prediction using NxN splitting may be used only for the smallest coding unit.
[84]
The prediction units 230 and 235 may include a prediction unit determiner, an inter prediction unit, and an intra prediction unit. The prediction unit determiner receives various information such as prediction unit information input from the entropy decoder 210, prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, and divides the prediction unit from the current coding unit, and predicts It may be determined whether the unit performs inter prediction or intra prediction. The inter prediction unit 230 uses information required for inter prediction of the current prediction unit provided from the image encoding apparatus based on information included in at least one picture before or after the current picture including the current prediction unit. Inter prediction may be performed on the prediction unit. Alternatively, inter prediction may be performed based on information of a pre-restored partial region in the current picture including the current prediction unit.
[85]
In order to perform inter prediction, a motion prediction method of a prediction unit included in a corresponding coding unit based on a coding unit is selected from among skip mode, merge mode, AMVP mode, and intra block copy mode. You can decide which way to go.
[86]
The intra prediction unit 235 may generate a prediction block based on pixel information in the current picture. When the prediction unit is a prediction unit on which intra prediction is performed, intra prediction may be performed based on intra prediction mode information of the prediction unit provided by the image encoding apparatus. Alternatively, the intra prediction unit 235 may perform intra prediction based on the palette mode, which will be described in detail with reference to FIGS. 3 to 18 . The intra prediction unit 235 may include an adaptive intra smoothing (AIS) filter, a reference pixel interpolator, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and may be applied by determining whether to apply the filter according to the prediction mode of the current prediction unit. AIS filtering may be performed on the reference pixel of the current block by using the prediction mode and AIS filter information of the prediction unit provided by the image encoding apparatus. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.
[87]
When the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on a pixel value obtained by interpolating the reference pixel, the reference pixel interpolator may interpolate the reference pixel to generate a reference pixel of a pixel unit having an integer value or less. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter may generate the prediction block through filtering when the prediction mode of the current block is the DC mode.
[88]
The reconstructed block or picture may be provided to the filter unit 240 . The filter unit 240 may include a deblocking filter, an offset correcting unit, and an ALF.
[89]
Information on whether a deblocking filter is applied to a corresponding block or picture and information on whether a strong filter or a weak filter is applied when the deblocking filter is applied may be provided from the image encoding apparatus. The deblocking filter of the image decoding apparatus may receive deblocking filter related information provided from the image encoding apparatus, and the image decoding apparatus may perform deblocking filtering on the corresponding block.
[90]
The offset correction unit may perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding and information on the offset value.
[91]
ALF may be applied to a coding unit based on information on whether ALF is applied, ALF coefficient information, etc. provided from the encoding apparatus. Such ALF information may be provided by being included in a specific parameter set.
[92]
The memory 245 may store the reconstructed picture or block to be used as a reference picture or reference block, and may also provide the reconstructed picture to an output unit.
[93]
As described above, hereinafter, in the embodiment of the present invention, a coding unit is used as a term for a coding unit for convenience of description, but it may also be a unit for performing decoding as well as coding.
[94]
In addition, the current block denotes an encoding/decoding target block, and depending on the encoding/decoding step, a coding tree block (or coding tree unit), a coding block (or a coding unit), a transform block (or a transform unit), or a prediction block (or prediction unit) and the like. In this specification, a 'unit' may indicate a basic unit for performing a specific encoding/decoding process, and a 'block' may indicate a pixel array of a predetermined size. Unless otherwise specified, 'block' and 'unit' may be used interchangeably. For example, in the embodiments to be described later, it may be understood that the coding block (coding block) and the coding unit (coding unit) have the same meaning.
[95]
[96]
3 to 5 are diagrams for explaining the concept of the palette mode (palette mode) according to the present disclosure.
[97]
In the palette mode, pixels occurring a lot in a block to be encoded (hereinafter referred to as a current block) are displayed with a specific index, and then the specific index is encoded instead of the pixel and transmitted to the decoding device. A flag indicating whether the palette mode is permitted may be encoded and transmitted to the decoding apparatus. In this case, the flag may be coded only when the size of the current block is less than or equal to a pre-set size. The preset size may be determined based on the slice type of the slice to which the current block belongs, the encoding mode or the prediction mode of the current block. For example, when the current block belongs to the I slice, the palette mode may be used only when the size of the current block is 4x4. When the current block belongs to the B or P slice, the palette mode can be used only when the size of the current block is larger than 4x4 and smaller than 64x64.
[98]
3 is an example of a process of generating a pallet table. For convenience of explanation, it is assumed that the size of the current block is 4x4. First, a histogram of 16 pixels existing in the current block is shown in FIG. 3 . In FIG. 3 , the horizontal axis denotes a pixel value (for example, a value from 0 to 255 in the case of a pixel quantized to 8 bits), and the vertical axis denotes the frequency of pixel values. Thereafter, a quantization zone is set based on pixels that occur frequently. Pixels existing in the quantization zone are replaced with the pixel with the highest frequency, and one index is assigned to the pixel with the highest frequency. Information indicating the size of the quantization zone may be encoded and transmitted to a decoding apparatus. Alternatively, the size of the quantization zone may be determined based on at least one of the size, shape, and bit depth of the current block.
[99]
In FIG. 3 , in each quantization zone, a3, a8, a10, and a11 denote pixels having the highest frequency, respectively, and the remaining pixels denote other pixels. In addition, pixels not included in the quantization zone are expressed as escape values, and in this case, in addition to encoding by index, they are additionally quantized and encoded.
[100]
Figure 4 shows an example for the palette table set in Figure 3.
[101]
In Figure 4, each row of the palette table is expressed as a palette entry, and a different index is assigned to each entry. That is, the size of the palette table may mean the number of entries.
[102]
An entry is formed using pixels a3, a8, a10, and a11 having the highest frequency in each quantization zone, and an index is assigned to each entry. If an escape value exists, an escape can be placed in the last entry and an index can be assigned. That is, the last index in the palette can mean an escape value.
[103]
5 is an example of a process in which pixels in a block are allocated as indexes using a set palette table. In Figure 5, the allocated indexes are expressed as palette indexes.
[104]
Pixels existing in the block are replaced with indexes according to the set palette table, and the indexes are encoded and transmitted to the decoding device. In addition, when indicated as escape values ​​(a5, a15 in FIG. 5), quantized a5' and a15' in addition to the index are encoded. In addition, the used palette table is also encoded and transmitted to the decoding device.
[105]
[106]
6 illustrates a method of performing intra prediction based on a palette mode according to the present disclosure.
[107]
The palette mode may be applied on a block-by-block basis (eg, a coding unit or a prediction unit), and for this, flag information (pred_mode_plt_flag) indicating whether to use the palette mode may be signaled on a block-by-block basis. That is, when the value of the flag is 1, the palette mode is applied to the current block, and when the value of the flag is 0, the palette mode is not applied to the current block.
[108]
The flag may be adaptively encoded/decoded based on at least one of a prediction mode of the current block and a size of the current block. For example, the flag may be encoded/decoded only when the prediction mode of the current block is the intra mode. The flag may be encoded/decoded only when the prediction mode of the current block is not a skip mode. The flag may be encoded/decoded only when at least one of a width or a height of the current block is less than or equal to a predetermined first threshold size. Here, the first threshold size is a value pre-defined in the encoding/decoding apparatus, and may be any one of 16, 32, or 64. The flag may be encoded/decoded only when the product of the width and height of the current block is greater than a predetermined second threshold size. Here, the second threshold size is a value pre-defined in the encoding/decoding apparatus, and may be any one of 16, 32, or 64. However, the first threshold size and the second threshold size may be different values. If any one of the above conditions is not satisfied, the flag is not encoded/decoded, and in this case, the value of the flag may be set to 0.
[109]
Referring to FIG. 6 , it is possible to configure a palette table for the palette mode of the current block ( S600 ).
[110]
The palette table may consist of at least one palette entry and a palette index identifying each palette entry. The palette table of the current block may be determined using the palette table of the previous block (hereinafter referred to as the previous palette table). Here, the previous block may mean a block coded or decoded before the current block.
[111]
Specifically, the palette entry of the current block may include at least one of a predicted palette entry or a signaled palette entry. The current block may use all or part of the palette entries used by the previous block, and among the palette entries used in the previous block, the palette entry reused in the current block is called a predicted palette entry.
[112]
The current block can use all of the palette entries in the previous palette table. Alternatively, the current block may use some of the palette entries of the previous palette table, and for this, a flag (PalettePredictorEntryReuseFlag, hereinafter referred to as a palette prediction flag) for specifying whether to reuse the palette entry may be used. The value of the palette prediction flag is assigned to each palette entry of the previous palette table, and the palette prediction flag (PalettePredictorEntryReuseFlag[i]) determines whether the palette entry corresponding to the palette index i in the previous palette table is reused in the palette table of the current block. can indicate whether For example, when the value of the palette prediction flag is 1, the palette entry corresponding to the palette index i in the previous palette table is reused in the palette table of the current block, and when the value of the palette prediction flag is 0, it is not reused No. A palette table of the current block may be constructed by extracting a palette entry having a value of the palette prediction flag of 1 from the previous palette table, and arranging them sequentially.
[113]
Meanwhile, the palette table of the current block may be initialized in units of a predetermined area. Here, the predetermined region may mean a parallel processing region or a CTU row of the current picture. If the current block belongs to the first CTU of the CTU row, the palette table of the current block may be initialized with the palette table of the neighboring CTU of the CTU to which the current block belongs. Here, the neighboring CTU may mean a CTU located above the CTU to which the current block belongs. That is, the palette table for the first CTU of the N-th CTU row may be initialized based on the palette table for the first CTU of the (N-1)-th CTU row. The initialized palette table may be updated based on the palette table of the previous block belonging to the same CTU row. The above-described embodiment is merely an example, and a method of configuring the palette table of the current block will be described in detail with reference to FIGS. 7 to 11 .
[114]
Meanwhile, the palette prediction flag may be signaled in the form of an encoded/decoded flag for each palette entry. Alternatively, the palette prediction flag may be encoded/decoded in the form of a binary vector based on run length encoding. That is, in the palette prediction flag array that specifies whether to reuse the previous palette entry, a syntax palette_predictor_run specifying the number of palette prediction flags that is 0 between non-zero palette prediction flags may be encoded/decoded. This will be described in detail with reference to FIG. 12 .
[115]
In addition, the palette table of the current block may further include a palette entry signaled through a bitstream, wherein the signaled palette entry is a palette entry that is not included in the previous palette table among the palette entries used by the current block can mean The signaled palette entry may be added after the predicted palette entry of the palette table.
[116]
Referring to FIG. 6 , a palette index may be determined in units of pixels of the current block ( S610 ).
[117]
The current block may determine the palette index using at least one of an index mode (INDEX MODE) or a copy mode (COPY MODE).
[118]
Here, the index mode (INDEX MODE) may mean a method of encoding the palette index information (palette_idx_idc) in the encoding apparatus to specify the palette index used in the current block. The decoding apparatus may derive the palette index of the current pixel based on the encoded palette index information. The palette index information has a value between 0 and (MaxPaletteIndex-1), where MaxPaletteIndex may mean the size of the palette table of the current block or the number of palette entries constituting the palette table. In the index mode, the value of the palette index information signaled through the bitstream may be allocated as the palette index of the current pixel.
[119]
The copy mode (COPY MODE) may refer to a method of determining the palette index of the current pixel by using the palette index of the neighboring pixel according to a predetermined scan order. Here, as the scan order according to the present disclosure, horizontal scan, vertical scan, diagonal scan, etc. may be used, and any one of them may be selectively used. To this end, a predetermined flag or index may be encoded/decoded. For example, the encoding apparatus encodes the flag as 0 when horizontal scan is applied as the scan order of the current block, and codes the flag as 1 when vertical scan is applied as the scan order of the current block can do. The decoding apparatus may adaptively determine the scan order of the current block according to the coded flag. However, the present invention is not limited thereto, and a method of encoding/decoding a palette index according to a scan order will be described in detail with reference to FIGS. 13 to 18 .
[120]
In the copy mode, the palette index of the current pixel may be predicted based on the palette index of the neighboring pixel, or the palette index of the neighboring pixel may be copied and set as the palette index of the current pixel as it is. Here, the neighboring pixel may mean a pixel adjacent to the top, bottom, left, or right of the current pixel. In particular, the neighboring pixel may be located on the same horizontal line or the same vertical line as the current pixel.
[121]
For example, the copy mode is the first copy mode in which the palette index used by the pixel adjacent to the top or bottom of the current pixel is identically used as the palette index of the current pixel, the palette index used by the pixel adjacent to the left or right of the current pixel At least one of a second copy mode that uses the same as the palette index of the current pixel, or a third copy mode that uses the palette index used by diagonally adjacent pixels of the current pixel equally as the palette index of the current pixel can
[122]
Meanwhile, any one of the above-described first to third copy modes may be selectively used according to the scan order of the current block. For example, the first copy mode may be applied when the scan order of the current block is a vertical scan, and the second copy mode may be applied when the scan order of the current block is a horizontal scan.
[123]
Also, the scan start position of the current block is not limited to the upper-left pixel of the current block, and other corner pixels of the current block (eg, lower-left pixel, upper-right pixel, and lower-right pixel) may be used as the scan start position. . Therefore, according to the scan order and scan start position of the current block, the same palette index as the pixel adjacent to the top or left may be used as described above, or the same palette index as the pixel adjacent to the bottom or right may be used.
[124]
Any one of the above-described index mode and copy mode may be selectively used. For example, the encoding apparatus may encode a flag run_copy_flag indicating whether the copy mode is used. Here, if the copy mode is used, the encoding apparatus may encode the flag as 1, otherwise (ie, if the index mode is used), the encoding apparatus may encode the flag as 0.
[125]
6, based on the palette table and the palette index, it is possible to predict the pixel of the current block (S620).
[126]
Specifically, it is possible to extract a palette entry having a palette index of the same value as the palette index from the palette table of the current block, and predict/restore the pixels of the current block using this. For example, the value of the palette entry extracted from the palette table may be set as the predicted value or the restored value of the pixel of the current block.
[127]
However, when the palette index indicates the last palette entry among the palette entries in the palette table of the current block, the pixel may be inferred as being encoded in the escape mode (ESCAPE MODE). Here, the escape mode does not use the palette entry of the pre-configured palette table, but instead predicts / restores the pixel based on the additionally signaled palette escape value It can mean a method. . Accordingly, a pixel having a palette index equal to (the number of palette entries - 1) may be predicted/restored using the additionally signaled palette escape value.
[128]
The above-described embodiment is merely an example, and for various methods of configuring the pallet table, it will be looked at in detail with reference to FIGS. 7 to 11 .
[129]
[130]
7 to 11 show a method of configuring a pallet table according to the present disclosure.
[131]
When the current block is encoded in the palette mode, the same palette table used in the encoding apparatus must also exist in the decoding apparatus. Therefore, it is necessary to encode the palette table in the encoding device. Therefore, it is possible to encode the number of palette entries existing in the palette table, and to encode the pixel value assigned to each entry. However, in this method, as the size of the block increases and the number of entries increases, the amount of encoded bits rapidly increases. Therefore, if the palette mode was used in the previous block, the amount of bits required to encode the palette table can be greatly reduced by generating the palette table of the current block based on the palette table used in the previous block. Here, the previous block means a block that has been encoded/decoded before the current block. Specifically, at least one of a flag indicating whether to configure the palette table of the current block based on the previous palette table or a palette prediction flag indicating whether to add an entry included in the palette table of the previous block to the palette table of the current block is available.
[132]
7 is a method of reducing the bit amount of a palette table to be currently encoded by using a palette prediction flag.
[133]
In FIG. 7 , the palette table A may mean a palette table existing in a block encoded using a palette mode before the current block. In the palette table A, it is possible to specify whether or not to be used as it is in the current palette table by using a palette prediction flag for each entry. For example, if the palette prediction flag is 1, it may mean that the corresponding entry is used as it is in the current palette table, and if it is 0, it may mean that the corresponding entry is not used in the current palette table. The index allocated to the entries predicted from the palette table A may be set to be the same as the index allocated to the palette table A. Alternatively, the index of each entry in the ascending/descending order of the indices allocated to each entry in the palette table A may be reassigned.
[134]
In the example of FIG. 7, the first entry, the third entry, and the fifth entry are used in the current palette table, so you can put them in order from the first entry to the third entry of the current palette table, and configure a new entry only from the fourth entry to the fifth entry. have. In this case, the palette prediction flag may be encoded first, and the number of remaining entries (two in the example of FIG. 7 : the fourth entry and the fifth entry of the current palette table) may be encoded. After that, the remaining entries may be encoded as much as the number of remaining entries. By transmitting the information to the decoding device, the decoding device also generates the same palette table as that of the coding device and enables prediction/restore of the current block.
[135]
At this time, the size of the current palette table (the number of entries) and the size of the previous palette table may be different. 8 is an example of a case in which the size of the previous palette table is larger than the size of the current palette table. In this case, the size of the current palette table may be encoded first. For example, through a bitstream, at least one of information indicating the number of entries included in the current palette table or information indicating a difference value with the size of the previous palette table may be encoded and transmitted to the decoding apparatus.
[136]
The palette prediction flag is sequentially encoded for each entry included in the previous table, but when the number of palette prediction flags having a value of 1 reaches the size of the current palette table, encoding of the palette prediction flag is omitted for subsequent entries can do. In the case of the last entry (pixel: a8) of the palette table B in FIG. 8, the corresponding palette prediction flag may not be coded.
[137]
Alternatively, the number of entries that can be imported (hereinafter, referred to as the maximum number of predictions) may be limited by using the palette prediction flag. As an example, information on the maximum number of predictions may be signaled through a bitstream. Alternatively, the maximum predicted number may be determined based on at least one of the size of the palette table, the size/form of the current block, or the size of the previous palette table.
[138]
As an example, only a certain percentage of the size of the current palette table may be obtained by using the palette prediction flag to bring an entry from the previous palette table, and the remaining percentage may be unconditionally generated from the current palette table. For example, if the size of the current palette table is 6 and the ratio is set to 50%, up to 3 entries from the previous palette table are fetched using the palette prediction flag, and the remaining 3 entries can be created from the current palette table unconditionally. have. Accordingly, when the number of entries having the value of the palette prediction flag of 1 reaches three, encoding of the palette prediction flag may be omitted for subsequent entries.
[139]
Alternatively, it is possible to predict entries to be included in the current palette table from a plurality of previous palette tables. As an example, an entry is brought into the current palette table by using a prediction flag for each of the entries included in the first previous palette table, but if the number of palette prediction flags with a value of 1 is smaller than the size of the current palette table, the first A method of continuously allocating a palette prediction flag using a second previous palette table that is earlier than the first previous palette table is also possible.
[140]
9 is an example of a case where the size of the previous palette table is smaller than the size of the current palette table, and at the same time, the ratio of entries generated using the palette prediction flag is set to 50%.
[141]
Since the size of the current palette table is 6, the number of entries created using the palette prediction flag is 3. Therefore, using the previous palette tables, the palette prediction flags are allocated until the number of palette prediction flags with 1 becomes three. Previous palette tables A to C in FIG. 9 are examples of bringing palette tables existing in blocks encoded in the palette mode before the current block in the encoding order of the blocks. At this time, when importing an entry from the previous palette tables, the duplicate entry does not assign a palette prediction flag. In the case of a0 of the previous palette table B in FIG. 9, the previous palette table B does not allocate an additional palette prediction flag because it is indicated as a palette prediction flag in the previous palette table A. And in the case of a5 of the previous palette table C, the previous palette table C does not allocate an additional palette prediction flag because it has already been marked as a palette prediction flag in the previous palette table B.
[142]
In addition, the encoding device and the decoding device may use the number of the referenced previous palette table as a fixed value or may transmit it through a higher header.
[143]
Alternatively, in consideration of the size of the previous palette table, it may be determined whether it is possible to refer to the current palette table creation. As an example, only when the size of the previous palette table is greater than or equal to a threshold value, or when the size of the previous palette table is the same as the current palette table, it may be determined to be referable when the current palette table is created.
[144]
Alternatively, the encoding order of the palette prediction flag may be determined by considering the indexes of the entries included in the first previous palette table and the entries included in the second previous palette table. As an example, after encoding the palette prediction flag for the entry having index 0 included in the first previous palette table, the palette prediction flag may be encoded for the entry having index 0 included in the second previous palette table. Then, after encoding the palette prediction flag for the entry with index 1 included in the first previous palette table, it is possible to encode the palette prediction flag for the entry with index 1 included in the second previous palette table.
[145]
Alternatively, the palette table candidate list may be configured, and at least one of a plurality of previous palette table candidates included in the palette table candidate list may be used when encoding the current palette table. 10 is a method of reducing the bit amount of a palette table to be currently encoded by using a palette prediction flag. In FIG. 10 , RT denotes a pixel located at the upper right of the block, and LB denotes a pixel located at the bottom left of the block. For example, in FIG. 10 , at least one of five neighboring blocks, that is, blocks including pixels A to E, may be referred to. Thereafter, the referenced block may be encoded as an index and transmitted to a decoding apparatus. Alternatively, only blocks at positions pre-defined in the encoding/decoding apparatus may be referred to among the blocks each including pixels A to E described above. Here, the pre-defined position may be the upper block (B) or the left block (A). In this case, encoding of an index specifying the referenced block may be omitted.
[146]
Using only the palette entry of the block corresponding to the index, it is possible to initialize / configure the palette table for the current block.
[147]
Alternatively, if the palette table of the current block cannot be filled more than the reference value using only the palette table of the referenced block, the palette table to be currently encoded may be filled similarly to the method of FIG. 9 by additionally designating a block based on an additional index. In this case, the encoding/decoding apparatus may refer to a pre-promised fixed number of blocks, and information specifying the number of referenced blocks may be transmitted through an upper header. Alternatively, a method in which the encoding/decoding apparatus refers to the same fixed number of neighboring blocks according to the size/form of the block and the size of the palette table is also possible. Alternatively, in addition to the position of FIG. 10, a method of fetching the palette table from the corresponding block by designating M blocks encoded in the palette mode before the current block in the encoding order as indexes is also possible. Alternatively, it is also possible to designate a block included in a collocated picture as an index and bring the palette table from the block.
[148]
Alternatively, a method of referring to a palette table previously used in a block specified by the BV using a block vector (BV) is also possible.
[149]
11 is an example of a method of setting BV. After setting a horizontal search range and a vertical search range in the restoration area around the current block, an area most similar to the current block is searched within the set search range. Thereafter, the region determined to be the most similar is determined, and if there is a region encoded in the palette mode in the region, a palette entry can be obtained from the corresponding palette table in a manner similar to FIG. 9 . At this time, the number of pallet tables used may be one or may be several.
[150]
The determined BV is encoded and then transmitted to a decoding device. After that, the decoding apparatus may also use the same BV to find the region most similar to the current block, then bring it using the palette table of the region and set the palette table in the same way as in the encoding apparatus.
[151]
Alternatively, the BV may be encoded based on the BV of the neighboring block. For example, if an encoding method using BV is used around the current block, the corresponding BV may be merged into the current block and used. In this case, the location referring to the BV may include at least one of the blocks shown in FIG. 10 or the collated block included in the collocated picture. In a manner similar to that of FIG. 10, a position to refer to the BV is set, the position to which the BV is referenced is indicated by an index, and it is encoded and transmitted to the decoding apparatus. Alternatively, the priority may be determined according to the location without indicating the index. For example, in FIG. 10 , after prioritization is determined in the order of A->B->C->D->E, a method of fetching a BV from a position where it is first determined that a BV exists and using it in the current block is also possible.
[152]
Alternatively, the BV of the neighboring block may be set as the predicted value of the BV, an index for identifying the neighboring block and a difference value between the BV and the predicted value may be encoded and transmitted to the decoding apparatus.
[153]
Alternatively, a method of constructing a palette table candidate list is also possible. All used palette tables are stored in the candidate list starting from the block existing in the first position of the image until just before the current block. Alternatively, after setting the number N of tables to be stored in the candidate list, N palette tables are stored in the candidate list. That is, when the encoding of the block is completed, the palette table of the encoded block may be stored in the candidate list. In this case, if the same palette table candidate as the palette table to be added to the candidate list exists, the palette table may not be added to the candidate list. Alternatively, the palette table may be added to the candidate list, and the same palette table candidate as the palette table may be deleted from the candidate list.
[154]
At this time, the method in which the palette table candidates in the candidate list are stored has a higher priority as it is closer to the current block, and may have a lower priority as it is further away from the current block. Alternatively, the priority may be set according to the size or reference frequency of the palette table. According to this priority, when the number of stored tables exceeds N, it may be deleted from the candidate list starting from the palette table with a lower priority.
[155]
Alternatively, in the parallel processing structure, it is also possible to configure a palette table list separately for each area to be processed in parallel. Alternatively, it is also possible to separately configure the palette table list for each CTU row of the area. In this case, if the palette table list is separately for each area in which parallel processing is performed, the number of palette tables stored in the palette table list may be very small in the initial part of the area. Therefore, it is also possible to fill a preset initial palette table without filling the palette table from the beginning for each area in which parallel processing is performed. For example, as shown in FIG. 6 , the initial palette table may be the palette table of the first CTU of the previous CTU row. Alternatively, the pre-set initial palette table may be a palette table derived from the entire image rather than a palette table derived for each block as shown in FIG. 3 . In this case, each entry value of the palette table derived from the entire image may be encoded through a higher header along with the number of entries. Alternatively, when constructing the initial palette table, it is also possible to set a quantized value as an entry value according to the representation bit of a pixel. For example, if an 8-bit pixel is quantized into 5 (5 entries), 0 to 255 can be divided into 5 zones and set as entries and encoded using the representative values ​​of each zone. Alternatively, if 0 to 255 are equally quantized, only information indicating that quantization has been performed equally and information indicating how many quantizations have been performed may be encoded through the upper header.
[156]
Alternatively, a method of configuring the entries included in the palette table as a palette entry candidate list is also possible. Entries included in the palette table of the encoded block can be added to the entry candidate list. In this case, among the entries included in the palette table, only entries having an index smaller than the threshold may be included in the entry candidate list. When the number of entries included in the palette table of the current block is less than the maximum number, the palette table may be configured with reference to the candidate entries included in the palette entry candidate list.
[157]
In the case of the palette prediction flag, a run length encoding method may be used. A continuous sequence of identical data is called a run, and the continuous length is expressed as a run length. For example, if there is a string aaaaaabbccccccc, 6 a, 2 B, and 7 c, so it can be expressed as 6a2b7c. Such an encoding method is called a run-length encoding method. When the palette prediction flags are encoded using run-length encoding, the number of 0's, the number of 1's, etc. may be expressed. Alternatively, run-length encoding may be performed only on 0, and conversely, run-length encoding may be performed on only 1 as well.
[158]
[159]
12 illustrates a method of signaling a palette prediction flag in the form of a binary vector based on run length encoding as an embodiment to which the present invention is applied.
[160]
In this embodiment, it is assumed that the palette table of the previous block uses 8 palette entries with palette indexes of 0 to 7.
[161]
The image encoding apparatus determines whether the corresponding palette entry is reused as the palette entry of the current block for each of the palette entries 0 to 7 of the previous block, and if the palette entry is reused as the palette entry of the current block, the corresponding palette The value of the palette prediction flag for the entry can be set to 1, otherwise, it can be set to 0, respectively. For example, as shown in FIG. 12, among the palette entries of the previous block, the palette entries 0, 1, 3, and 7 are reused as the palette entries of the current block, and the remaining palette entries are not reused, A binary vector represented by 11010001 may be generated.
[162]
Then, the binary vector encodes at least one of the number of 1s (that is, the number of palette entries reused as the palette entry of the current block among the palette entries of the previous block) or the number of 0s preceding 1 in the binary vector. Signaling may be performed by a decoding device. For example, since the number of 1's in the binary vector is 4, 4 can be encoded as the number of palette entries of the previous block that is reused as the palette entry of the current block. In addition, the number of 0s preceding 1 in the binary vector, that is, 0, 0, 1, 3 may be sequentially encoded.
[163]
The decoding apparatus receives at least one of information about the number of palette entries of a previous block reused as a palette entry of the current block from the encoding apparatus or information about the number of zeros before 1 in the binary vector (palette_entry_run), and this It can be used to compose the palette table of the current block.
[164]
For example, the decoding apparatus sequentially extracts information (palette_entry_run), that is, 0, 0, 1, 3 about the number of 0's before 1 in the binary vector from the bitstream, and using this, the palette entry of the previous block A binary vector indicating whether to reuse or not, that is, 11010001 may be restored. When a value of 1 occurs in the process of restoring a binary vector, the palette entry of the previous block corresponding to the value of 1 may be inserted into the palette table of the current block. Through this process, the palette table of the current block can be configured by selectively reusing some palette entries from the palette table of the previous block.
[165]
[166]
13 to 18 illustrate a method of encoding/decoding a palette index according to a scan order according to the present disclosure.
[167]
After encoding the palette table, the palette index assigned to each pixel of the current block must also be encoded. 13 is an example of a scan order performed in a current block.
[168]
The main purpose of the scan sequence shown in FIG. 13 is to perform scanning in consideration of directionality. As shown in FIG. 13(a) , if characteristics of pixels existing in the current block have similar values ​​in the horizontal or vertical direction, the possibility of collecting the same indexes increases when scanning is performed as shown in FIG. 13(a). Alternatively, if characteristics of pixels existing in a block have similar values ​​in the z-direction or diagonal direction as shown in FIG. 13(b) , the probability of collecting the same indexes increases if scanning is performed as shown in FIG. 13(b).
[169]
The encoding apparatus may indicate which scan method is used as an index, encode it, and transmit it to the decoding apparatus. Alternatively, the scan order may be determined according to the size and shape of the current block. After collecting indices having the same value in this scan method, run-length encoding is performed to increase encoding efficiency.
[170]
Alternatively, a fixed scan method may be used, but run-length encoding may be performed after the current block is rotated. The encoding apparatus may encode information indicating whether the current block is rotated and transmit it to the decoding apparatus. Alternatively, whether to rotate the current block may be determined according to the size and shape of the current block.
[171]
In addition, information indicating whether an escape value exists for each block can be encoded. If there is an escape value, it is possible to indicate that the pixel at the corresponding position is an escape value by using an index at an arbitrary fixed position such as the last index or the first index. At this time, the size of the palette table derived as shown in FIG. 3 is used as it is, but only when there is an escape value, it is also possible to increase the size of the palette table by one and allocate an index. Alternatively, it is also possible to display information indicating whether or not it is an escape value for every pixel in the block, and use the index of the palette table only when it is not an escape value. When encoding escape values, both lossy and lossless encoding are available. When lossless encoding information is added, if lossless encoding is indicated by the lossless encoding information when encoding escape values, the escape values ​​are quantized and encoded, and then transmitted to the decoding device. In this case, information (eg, quantization parameter) indicating to what extent the escape value is to be quantized can be additionally encoded, and the quantized escape value can also be encoded. If it is indicated that lossless encoding is performed, the escape value may be encoded as it is without quantization and transmitted to the decoding device.
[172]
14 is an example of encoding the palette index in the current block. In this case, it is assumed that horizontal scanning is applied for convenience of explanation. The information to be encoded and transmitted to the decoding device requires initial indices at which Len length encoding starts, and the run length following the initial index. In FIG. 14, except for the escape value, the initial indexes are 0, 1, 0, 2, 3, 2, 3, 2, 2, 1, 0 in order. And the run length according to each initial index is 6, 4, 3, 5, 10, 1, 4, 4, 3, 3, 9 except for the start index. In the case of the escape value, like other indices, it can be coded using the initial index and run length. Alternatively, encoding may be performed using information indicating whether an escape value is an escape value for each pixel position. For example, only when it is determined that it is not an escape value, the initial index and run length are encoded, and when it is determined that it is an escape value, the escape value can be directly encoded without using the initial index and run length.
[173]
Alternatively, indexes from previous lines may be copied. 15 is an example of a case in which indices are copied from previous lines.
[174]
When the initial index 3 is encoded, the same indexes exist immediately above it. In this case, before encoding the initial index, information indicating whether conventional run-length encoding is used or whether a pixel included in a previous line is copied may be encoded first. The pixels included in the previous line may be located in the upper row, the lower row, the left column, the right column, or the upper left corner according to the scan order. After that, when it is determined based on the information that the copy has been copied from the previous line, it is possible to encode only the run length including the initial index without encoding the initial index.
[175]
For example, if the conventional method is used, information indicating that it has not been copied from the previous line and initial index 3 may be encoded, and run length 4 may be encoded. If the method of copying from the previous line is applied, only information indicating that it has been copied from the previous line and run length 5 can be encoded.
[176]
At this time, information indicating whether a copy has been copied from the previous line may be indexed and displayed as being able to be copied from multiple lines. For example, if the index is 0, the conventional run-length encoding method is used without using this technique, if 1, the method of copying from the immediately preceding line is used, and if the index is 2, the method of copying from a line 2 lines away is used. can be used In this method, if the initial index and run length to be encoded are present at the same horizontal position, a method of copying the vertical position with only the index may be used.
[177]
If it is not the same horizontal position, it is also possible to express from which area the copy is coming by using a vector. 16 is an example of the vector.
[178]
In this case, the encoding/decoding apparatus may determine and use the same rule for the starting point and the ending point of the vector. In FIG. 16 , a vector is expressed as a negative number in a left or upward direction based on the current starting point, and a positive number in a right or downward direction. However, in the case of a horizontal scan, the y-component vector may always have a negative number in the scan order, so the sign may not be coded for the y-component. As another example, in the case of a vertical scan, the sign of the x component may not be coded because the x component vector always comes out as a negative number in the scan order.
[179]
Alternatively, redundancy may be removed between the conventional continuous run-length coding schemes. For example, indices existing in the block of FIG. 15 are 0 0 0 0 0 0 0 1 1 1 1 1 ... in the scan order. was expressed as These indices are initial index 0, run length 6, initial index 1, run length 4... can be expressed as Since the number of pixels having the same value as the initial index is expressed by the run length, the Nth initial index may have a different value from the previous initial index. For example, the meaning that the initial index 1 comes out means that the initial index of the previous order number is not 1. By using this, index values ​​for residual indices other than the previous initial index may be reassigned, and run-length encoding may be performed. For example, an index having an original value smaller than the original value of the previous initial index is maintained as it is, and a value obtained by subtracting 1 from the original value is reassigned to an index having an original value greater than the original value of the previous initial index. Here, the original value indicates an index value before reallocation, not a reallocation index value. For example, if the previous initial index was 1, index 0 maintains the index value as it is, whereas indexes 1 to 3 may be allocated to indices 2 to 4 having an index greater than 1, respectively.
[180]
If this is applied to the example above, the initial index 0, run length 6, initial index 1, run length 4… The method expressed as initial index 0, run length 6, initial index 0, run length 4… can be changed to
[181]
After decoding the second initial index 0, the decoding apparatus may restore the original value of the initial index by increasing it as opposed to the encoding apparatus through a comparison process with the previous initial index. For example, if the value of the initial index is smaller than the original value of the previous initial index, the value of the initial index may be set as the original value of the initial index as it is. On the other hand, when the value of the initial index is equal to or greater than the original value of the previous initial index, a value obtained by adding 1 to the value of the initial index may be set as the original value of the initial index.
[182]
Removing redundancy by reassigning the value of the initial index can be used in the same way as copying from the previous line. When the initial index is encoded, if the previous initial index and the corresponding run length are copied from the previous line, the value of the current initial index and the same position in the previous line must be unconditionally different from the current initial index. If they were the same, the run length would have been expressed by adding the current initial index to the method copied from the previous line, which is the encoding method of the previous initial index. Therefore, it can be encoded by reducing the value as well.
[183]
17 is an example of a method of simultaneously applying in-screen prediction and palette mode. In FIG. 17 , an index and a corresponding pixel are indicated for each position.
[184]
For example, information indicating that in-screen prediction has been used is assigned to index 0 of the palette table. Thereafter, the pixel position indicated by index 0 is assigned a value obtained by performing intra prediction using reconstructed pixels existing around the current block. After encoding information indicating whether any of the method using the conventional palette mode and the method in which intra prediction is mixed for each block is used, if it is determined that the mixed method is used, which intra prediction is used You may decide to use an index. Depending on the number of used intra prediction modes, the mode itself may be encoded as it is, and it is also possible to encode using the Most Probable Mode (MPM). Alternatively, it is also possible to encode the intra prediction mode using the default intra mode. The default mode may include at least one of Planar, DC, Horizontal, and Vertical.
[185]
18 is an example of a case in which a block searched using the BV described in FIG. 11 and a palette mode are mixed.
[186]
For example, information indicating that it is a pixel using BV is assigned to index 0 of the palette table. Thereafter, the pixels at the same position in the block searched using the BV are allocated to the position of the index 0 to the pixel position indicated by the index 0. After encoding information indicating whether a method using a conventional palette mode and a mixed method using BV is used for each block, if it is determined that the mixed method is used, information related to BV is transmitted to the decoding device do. When deriving the BV, as shown in FIG. 10 , a method of designating which BV is used among BVs in the neighboring block as an index is possible, or a method of directly encoding the BV and transmitting it to the decoding apparatus is also possible. Alternatively, it is also possible to set priorities in the order of A->B->C->D->E of FIG. 10 , and then retrieve the BV from the position where it is determined that the BV exists first and use it in the current block. In this case, it is not necessary to encode the information related to the BV.
[187]
An index indicating intra prediction or an index indicating BV use may be assigned to a pre-defined position in the palette table. As an example, as illustrated in FIGS. 17 and 18 , the indexes may be arranged in the first part of the palette table, and unlike the illustrated example, the indexes may be arranged at the end of the palette table. Alternatively, a value assigned to the index may be determined based on at least one of a value/angle of an intra prediction mode, a size of a BV, a size/shape of a block, and an intra prediction mode of a neighboring block. Alternatively, when encoding the escape value, the method using the above intra prediction or the method using BV may be used. For example, depending on the prediction mode used in the picture, a value from a neighboring restored pixel may be taken and replaced with an escape value, or a value from the same position in a block searched using BV may be taken and replaced with an escape value. Alternatively, it is also possible to use the above values ​​as predicted values ​​instead of replacing them with escape values, and to encode and transmit only the difference values. The difference value may be encoded as it is, or a method of encoding after performing quantization is also possible.
[188]
[189]
The names of syntaxes used in the above-described embodiments are merely named for convenience of description.
[190]
It is included in the scope of the present invention to apply the decoding process or the embodiments described based on the encoding process to the encoding process or the decoding process. It is also within the scope of the present invention to change the embodiments described in a certain order in an order different from that described.
[191]
Although the above-described embodiment has been described based on a series of steps or a flowchart, this does not limit the time-series order of the invention, and may be performed simultaneously or in a different order, if necessary. In addition, each of the components (eg, unit, module, etc.) constituting the block diagram in the above-described embodiment may be implemented as a hardware device or software, or a plurality of components may be combined to form one hardware device or software. may be implemented. The above-described embodiment may be implemented in the form of program instructions that can be executed through various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Examples of the computer-readable recording medium include a hard disk, a magnetic medium such as a floppy disk and a magnetic tape, an optical recording medium such as a CD-ROM and DVD, and a magneto-optical medium such as a floppy disk. media), and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The hardware device may be configured to operate as one or more software modules for carrying out the processing according to the present invention, and vice versa.
Industrial Applicability
[192]
The present invention can be used to encode or decode an image.
Claims
[Claim 1]
Constructing a palette table for the palette mode of the current block; determining a palette index in units of pixels of the current block; and predicting a pixel of the current block based on the palette table and the palette index.
[Claim 2]
The method of claim 1 , wherein the palette mode is selectively used based on a first flag indicating whether the palette mode is used.
[Claim 3]
The method of claim 2 , wherein the first flag indicating whether the palette mode is used is adaptively decoded based on at least one of a prediction mode of the current block and a size of the current block.
[Claim 4]
According to claim 1, wherein the palette table of the current block consists of at least one palette entry and an index identifying each palette entry, the palette entry of the palette table, the palette table of the previous block decoded before the current block and a second flag indicating whether or not to reuse a palette entry belonging to the previous palette table, the video decoding method.
[Claim 5]
The method of claim 4, wherein the palette table is initialized in units of CTU rows, and when the CTU to which the current block belongs is the first CTU of the current CTU row, the palette table of the CTU to which the current block belongs is the first of the previous CTU row. A video decoding method that is initialized using a palette table of CTU.
[Claim 6]
According to claim 1, wherein the palette index of the current block is determined using at least one of an index mode or a copy mode, the index is the mode is the palette index information for specifying the palette index of the current block is signaled mode, and the copy mode is a mode that uses palette indexes of neighboring pixels according to a predetermined scan order.
[Claim 7]
Constructing a palette table for the palette mode of the current block; determining a palette index in units of pixels of the current block; and predicting a pixel of the current block based on the palette table and the palette index.
[Claim 8]
The method of claim 7 , further comprising encoding a first flag indicating whether the palette mode is used.
[Claim 9]
The method of claim 8 , wherein the first flag indicating whether the palette mode is used is adaptively encoded based on at least one of a prediction mode of the current block and a size of the current block.
[Claim 10]
The palette table of claim 7, wherein the palette table of the current block includes at least one palette entry and an index identifying each palette entry, and the palette entry of the palette table is encoded before the current block. and a second flag indicating whether to reuse a palette entry belonging to the previous palette table, the video encoding method.
[Claim 11]
The method according to claim 10, wherein the palette table is initialized in units of CTU rows, and when the CTU to which the current block belongs is the first CTU of the current CTU row, the palette table of the CTU to which the current block belongs is the first of the previous CTU row. A video encoding method, initialized using a palette table of the CTU.
[Claim 12]
The method of claim 7, wherein the palette index of the current block is determined using at least one of an index mode or a copy mode, and the mode is a mode for encoding palette index information for specifying the palette index of the current block. and, the copy mode is a mode using palette indexes of neighboring pixels according to a predetermined scan order.
[Claim 13]
A computer-readable recording medium for storing a bitstream encoded by a video encoding method, the video encoding method comprising: constructing a palette table for a palette mode of a current block; determining a palette index in units of pixels of the current block; and predicting a pixel of the current block based on the palette table and the palette index.