Technical Field
[0001]
The present invention relates to a video encoding technique, and particularly to a video encoding technique which makes a prediction with reference to a reconstructed image and performs data compression by quantization.
Background Art
[0002]
A typical video encoding device executes an encoding process that conforms to a predetermined video coding scheme to generate coded data, i.e. a bitstream. In ISO/IEC 14496-10 Advanced Video Coding (AVC) described in Non Patent Literature (NPL) 1 as a representative example of the predetermined video coding scheme, each frame is divided into blocks of 16×16 pixel size called MBs (Macro Blocks), and each MB is further divided into blocks of 4×4 pixel size, setting MB as the minimum unit of encoding. FIG. 23 shows an example of block division in the case where the color format of a frame is the YCbCr 4:2:0 format and the spatial resolution is QCIF (Quarter Common Intermediate Format).
[0003]
Each of the divided image blocks is input sequentially to the video encoding device and encoded. FIG. 24 is a block diagram showing an example of the structure of the typical video encoding device for generating a bitstream that conforms to AVC. Referring to FIG. 24, the structure and operation of the typical video encoding device is described below.

The video encoding device shown in FIG. 24 includes a frequency transformer 101, a quantizer 102, a variable-length encoder 103, a quantization controller 104, an inverse quantizer 105, an inverse frequency transformer 106, a frame memory 107, an intra-frame predictor 108, an inter-frame predictor 109, and a prediction selector 110.
[0005]
An input image to the video encoding device is input to the frequency transformer 101 as a prediction error image, after a prediction image supplied from the intra-frame predictor 108 or the inter-frame predictor 109 through the prediction selector 110 is subtracted from the input image.
[0006]
The frequency transformer 101 transforms the input prediction error image from a spatial domain to a frequency domain, and outputs the result as a coefficient image.
[0007]
The quantizer 102 quantizes the coefficient image supplied from the frequency transformer 101 using a quantization step size, supplied from the quantization controller 104, controlling the granularity of quantization, and outputs the result as a quantized coefficient image.
[0008]
The variable-length encoder 103 entropy-encodes the quantized coefficient image supplied from the quantizer 102. The variable-length encoder 103 also encodes the above quantization step size supplied from the quantization controller 104 and an image prediction parameter supplied from the prediction selector 110. These pieces of coded data are multiplexed and output from the video encoding device as a bitstream.
[0009]
Here, an encoding process for the quantization step

size at the variable-length encoder 103 is described with reference to FIG. 25. In the variable-length encoder 103, a quantization step size encoder for encoding the quantization step size includes a quantization step size buffer 10311 and an entropy encoder 10312 as shown in FIG. 25.
[0010]
The quantization step size buffer 10311 holds a quantization step size Q(i-1) assigned to the previous image block encoded immediately before an image block to be encoded.
[0011]
As shown in the following equation (1), the previous quantization step size Q(i-1) supplied from the
quantization step size buffer 10311 is subtracted from an input quantization step size Q(i), and the result is input to the entropy encoder 10312 as a difference quantization step size dQ(i).
[0012]
dQ(i) = Q(i) - Q(i-1) … (1)
[0013]
The entropy encoder 10312 entropy-encodes the input difference quantization step size dQ(i), and outputs the result as code corresponding to the quantization step size.
[0014]
The above has described the encoding process for the quantization step size.
[0015]
The quantization controller 104 determines a quantization step size for the current input image block. In general, the quantization controller 104 monitors the output code rate of the variable-length encoder 103 to increase the quantization step size so as to reduce the output code rate for the image block concerned, or,

conversely, to decrease the quantization step size so as to increase the output code rate for the image block concerned. The increase or decrease in quantization step size enables the video encoding device to encode an input moving image by a target rate. The determined quantization step size is supplied to the quantizer 102 and the variable-length encoder 103.
[0016]
The quantized coefficient image output from the quantizer 102 is inverse-quantized by the inverse quantizer 105 to obtain a coefficient image to be used for prediction in encoding subsequent image blocks. The coefficient image output from the inverse quantizer 105 is set back to the spatial domain by the inverse frequency transformer 106 to obtain a prediction error image. The prediction image is added to the prediction error image, and the result is input to the frame memory 107 and the intra-frame predictor 108 as a reconstructed image.
[0017]
The frame memory 107 stores reconstructed images of encoded image frames input in the past. The image frames stored in the frame memory 107 are called reference frames.
[0018]
The intra-frame predictor 108 refers to reconstructed images of image blocks encoded in the past within the image frame being currently encoded to generate a prediction image.
[0019]
The inter-frame predictor 109 refers to reference frames supplied from the frame memory 107 to generate a prediction image.
[0020]
The prediction selector 110 compares the prediction image supplied from the intra-frame predictor 108 with the

prediction image supplied from the inter-frame predictor 109, selects and outputs one prediction image closer to the input image. The prediction selector 110 also outputs information (called an image prediction parameter) on a prediction method used by the intra-frame predictor 108 or the inter-frame predictor 109, and supplies the information to the variable-length encoder 103.
[0021]
According to the processing mentioned above, the typical video encoding device compressively encodes the input moving image to generate a bitstream.
[0022]
The output bitstream is transmitted to a video decoding device. The video decoding device executes a decoding process so that the bitstream will be decompressed as a moving image. FIG. 26 shows an example of the structure of a typical video decoding device that decodes the bitstream output from the typical video encoding device to obtain decoded video. Referring to FIG. 26, the structure and operation of the typical video decoding device is described below.
[0023]
The video decoding device shown in FIG. 26 includes a variable-length decoder 201, an inverse quantizer 202, an inverse frequency transformer 203, a frame memory 204, an intra-frame predictor 205, an inter-frame predictor 206, and a prediction selector 207.
[0024]
The variable-length decoder 201 variable-length-decodes the input bitstream to obtain a quantization step size that controls the granularity of inverse quantization, the quantized coefficient image, and the image prediction parameter. The quantization step size and the quantized coefficient image mentioned above are supplied to the

inverse quantizer 202. The image prediction parameter is supplied to the prediction selector 207.
[0025]
The inverse quantizer 202 inverse-quantizes the input quantized coefficient image based on the input quantization step size, and outputs the result as a coefficient image.
[0026]
The inverse frequency transformer 203 transforms the coefficient image, supplied from the inverse quantizer 202, from the frequency domain to the spatial domain, and outputs the result as a prediction error image. A prediction image supplied from the prediction selector 207 is added to the prediction error image to obtain a decoded image. The decoded image is not only output from the video decoding device as an output image, but also input to the frame memory 204 and the intra-frame predictor 205.
[0027]
The frame memory 204 stores image frames decoded in the past. The image frames stored in the frame memory 204 are called reference frames.
[0028]
Based on the image prediction parameter supplied from the variable-length decoder 201, the intra-frame predictor
205 refers to reconstructed images of image blocks decoded
in the past within the image frame being currently decoded
to generate a prediction image.
[0029]
Based on the image prediction parameter supplied from the variable-length decoder 201, the inter-frame predictor
206 refers to reference frames supplied from the frame
memory 204 to generate a prediction image.
[0030]
The prediction selector 207 selects either of the prediction images supplied from the intra-frame predictor

205 and the inter-frame predictor 206 based on the image prediction parameter supplied from the variable-length decoder 201.
[0031]
Here, a decoding process for the quantization step size at the variable-length decoder 201 is described with reference to FIG. 27. In the variable-length decoder 201, a quantization step size decoder for decoding the quantization step size includes an entropy decoder 20111 and a quantization step size buffer 20112 as shown in FIG. 27.
[0032]
The entropy decoder 20111 entropy-decodes input code, and outputs a difference quantization step size dQ(i).
[0033]
The quantization step size buffer 20112 holds the previous quantization step size Q(i-1).
[0034]
As shown in the following equation (2), Q(i-1) supplied from the quantization step size buffer 20112 is added to the difference quantization step size dQ(i) generated by the entropy decoder 20111. The added value is not only output as a quantization step size Q(i), but also input to the quantization step size buffer 20112.
[0035]
Q(i) = Q(i-1) + dQ(i) … (2)
[0036]
The above has described the decoding process for the quantization step size.
[0037]
According to the processing mentioned above, the typical video decoding device decodes the bitstream to generate a moving image.
[0038]

In the meantime, in order to maintain the subjective quality of the moving image to be compressed by the encoding process, the quantization controller 104 in the typical video encoding device is generally analyzes either or both of the input image and the prediction error image, as well as analyzing the output code rate, to determine a quantization step size according to the human visual sensitivity. In other words, the quantization controller 104 performs visual-sensitivity-based adaptive quantization. Specifically, when the human visual sensitivity to the current image to be encoded is determined to be high, the quantization step size is set small, while when the visual sensitivity is determined to be low, the quantization step size is set large. Since such control can assign a larger code rate to a low visual sensitivity region, the subjective quality is improved.
[0039]
As a visual-sensitivity-based adaptive quantization technique, for example, adaptive quantization based on the texture complexity of an input image used in MPEG-2 Test Model 5 (TM5) is known. The texture complexity is typically called activity. Patent Literature (PTL) 1 proposes an adaptive quantization system using the activity of a prediction image in conjunction with the activity of an input image. PTL 2 proposes an adaptive quantization system based on an activity that takes edge portions into account.
[0040]
When the visual-sensitivity-based adaptive quantization technique is used, it will cause a problem if the quantization step size is often changed within an image frame. In the typical video encoding device for generating a bitstream that confirms to the AVC scheme, a difference from a quantization step size for an image block encoded

just before an image block to be encoded is entropy-encoded in encoding the quantization step size. Therefore, as the change in quantization step size in the encoding sequence direction becomes large, the rate required to encode the quantization step size increases. As a result, the code rate assigned to encoding of the coefficient image is relatively reduced, and hence the image quality is degraded.
[0041]
Since the encoding sequence direction is independent of the continuity of the visual sensitivity on the screen, the visual-sensitivity-based adaptive quantization technique inevitably increases the code rate required to encode the quantization step size. Therefore, even using the visual-sensitivity-based adaptive quantization technique in the typical video encoding device, the image degradation associated with the increase in the code rate for the quantization step size may cancel out the subjective quality improved by the adaptive quantization technique, i.e., there arises a problem that a sufficient improvement in image quality cannot be achieved.
[0042]
To address this problem, PTL 3 discloses a technique for adaptively setting a range of quantization to zero, i.e. a dead zone according to the visual sensitivity in the spatial domain and the frequency domain instead of adaptively setting the quantization step size according to the visual sensitivity. In the system described in PTL 3, a dead zone for a transform coefficient determined to be low in terms of the visual sensitivity is more widened than a dead zone for a transform coefficient determined to be high in terms of the visual sensitivity. Such control enables visual-sensitivity-based adaptive quantization without changing the quantization step size.

Citation List Patent Literatures
[0043]
PTL 1: Japanese Patent No. 2646921 PTL 2: Japanese Patent No. 4529919 PTL 3: Japanese Patent No. 4613909
Non Patent Literatures
[0044]
NPL 1: ISO/IEC 14496-10 Advanced Video Coding NPL 2: “WD1: Working Draft 1 of High-Efficiency Video Coding,” Document JCTVC-C403, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 3rd Meeting at Guangzhou, China, October 2010
Summary of Invention Technical Problem
[0045]
However, when the technique described in PTL 3 is used, quantization adaptive to the visual sensitivity cannot be performed on transform coefficients that do not fall within a dead zone. In other words, even when the visual sensitivity is determined to be low, the rate of coefficient code for the transform coefficients that do not fall within the dead zone cannot be reduced. Further, when the quantization step size is enlarged, the transform coefficient values after being subjected to quantization are concentrated near zero, while when the dead zone is widened, the transform coefficients that do not fall within the dead zone are not concentrated near zero even after being subjected to quantization. In other words, when the dead zone is widened, the entropy-encoding efficiency is insufficient compared with the case where the quantization step size is enlarged. For these reasons, it can be said

that there is a problem in typical encoding technology that the assignment of the code rate to a high visual sensitivity region cannot be increased sufficiently.
[0046]
The present invention has been made in view of the above problems, and it is a first object thereof to provide a video encoding device and a video encoding method capable of changing the quantization step size frequently while suppressing an increase in code rate to achieve high-quality moving image encoding. It is a second object of the present invention to provide a video decoding device and a video decoding method capable of regenerating a high-quality moving image.
Solution to Problem
[0047] A video encoding device according to the present invention for dividing input image data into blocks of a predetermined size, and applying quantization to each divided image block to execute a compressive encoding process, comprises quantization step size encoding means for encoding a quantization step size that controls a granularity of the quantization, wherein the quantization step size encoding means predicts the quantization step size that controls the granularity of the quantization by using a quantization step size assigned to a neighboring image block already encoded.
[0048] A video decoding device according to the present invention for decoding image blocks using inverse quantization of input compressed video data to execute a process of generating image data as a set of image blocks, comprises quantization step size decoding means for decoding a quantization step size that controls a

granularity of the inverse quantization, wherein the quantization step size decoding means predicts the
quantization step size that controls the granularity of the inverse quantization by using a quantization step size assigned to a neighboring image block already decoded.
[0049]
A video encoding method according to the present invention for dividing input image data into blocks of a predetermined size, and applying quantization to each divided image block to execute a compressive encoding process, comprises predicting a quantization step size that controls a granularity of the quantization by using a quantization step size assigned to a neighboring image block already encoded.
[0050]
A video decoding method according to the present invention for decoding image blocks using inverse quantization of input compressed video data to execute a process of generating image data as a set of image blocks, comprises predicting a quantization step size that controls a granularity of the inverse quantization by using a quantization step size assigned to a neighboring image block already decoded.
Advantageous Effects of Invention
[0051]
According to the present invention, even when the quantization step size is changed frequently within an image frame, the video encoding device can suppress an increase in code rate associated therewith. In other words, the quantization step size can be encoded by a smaller code rate. This resolves the problem that the subjective quality improved by the visual-sensitivity-based adaptive quantization is canceled out, that is, high-quality moving

image encoding can be achieved. Further, according to the present invention, since the video decoding device can decode the quantization step size frequently changed by receiving only a small code rate, a high-quality moving image can be regenerated by the small code rate.
Brief Description of Drawings
[0052]
[FIG. 1] It depicts a block diagram showing a quantization step size encoder in a video encoding device in a first exemplary embodiment of the present invention.
[FIG. 2] It depicts an explanatory diagram showing an example of an image block to be encoded and neighboring image blocks.
[FIG. 3] It depicts a block diagram showing a quantization step size decoder in a video decoding device in a second exemplary embodiment of the present invention.
[FIG. 4] It depicts a block diagram showing a quantization step size encoder in a video encoding device in a third exemplary embodiment of the present invention.
[FIG. 5] It depicts a block diagram showing a quantization step size decoder in a video decoding device in a fourth exemplary embodiment of the present invention.
[FIG. 6] It depicts an explanatory diagram showing prediction directions of intra-frame prediction.
[FIG. 7] It depicts an explanatory diagram showing an example of inter-frame prediction.
[FIG. 8] It depicts an explanatory diagram showing an example of prediction of a quantization step size using a motion vector of inter-frame prediction in the video encoding device in the third exemplary embodiment of the present invention.
[FIG. 9] It depicts a block diagram showing the structure of another video encoding device according to the

present invention.
[FIG. 10] It depicts a block diagram showing a characteristic component in another video encoding device according to the present invention.
[FIG. 11] It depicts an explanatory diagram of a list showing an example of multiplexing of quantization step size prediction parameters.
[FIG. 12] It depicts a block diagram showing the structure of another decoding device according to the present invention.
[FIG. 13] It depicts a block diagram showing a characteristic component in another video decoding device according to the present invention.
[FIG. 14] It depicts a block diagram showing a quantization step size encoder in a seventh exemplary embodiment of the present invention.
[FIG. 15] It depicts a block diagram showing a quantization step size decoder in a video decoding device in an eighth exemplary embodiment of the present invention.
[FIG. 16] It depicts a block diagram showing a configuration example of an information processing system capable of implementing the functions of a video encoding device and a video decoding device according to the present invention.
[FIG. 17] It depicts a block diagram showing characteristic components in a video encoding device according to the present invention.
[FIG. 18] It depicts a block diagram showing characteristic components in another video encoding device according to the present invention.
[FIG. 19] It depicts a block diagram showing characteristic components in a video decoding device according to the present invention.
[FIG. 20] It depicts a block diagram showing

characteristic components in another video decoding device according to the present invention.
[FIG. 21] It depicts a flowchart showing characteristic steps in a video encoding method according to the present invention.
[FIG. 22] It depicts a flowchart showing characteristic steps in a video decoding method according to the present invention.
[FIG. 23] It depicts an explanatory diagram showing an example of block division.
[FIG. 24] It depicts a block diagram showing an example of the structure of a video encoding device.
[FIG. 25] It depicts a block diagram showing a quantization step size encoder in a typical video encoding device.
[FIG. 26] It depicts a block diagram showing an example of the structure of a video decoding device.
[FIG. 27] It depicts a block diagram showing a quantization step size encoder in a typical video decoding device.
Description of Embodiments
[0053]
Exemplary embodiments of the present invention are described below with reference to the accompanying drawings. [0054]
Exemplary Embodiment 1
Like the video encoding device shown in FIG. 24, a video encoding device in a first exemplary embodiment of the present invention includes the frequency transformer 101, the quantizer 102, the variable-length encoder 103, the quantization controller 104, the inverse quantizer 105, the inverse frequency transformer 106, the frame memory 107, the intra-frame predictor 108, the inter-frame predictor

109, and the prediction selector 110. However, the structure of a quantization step size encoder included in the variable-length encoder 103 is different from the structure shown in FIG. 25.
[0055]
FIG. 1 is a block diagram showing a quantization step size encoder in the video encoding device in the first exemplary embodiment of the present invention. In
comparison with the quantization step size encoder shown in FIG. 25, the quantization step size encoder in the exemplary embodiment is different in including a predicted quantization step size generator 10313 as shown in FIG. 1.
[0056]
The quantization step size buffer 10311 stores and holds quantization step sizes assigned to image blocks encoded in the past.
[0057]
The predicted quantization step size generator 10313 retrieves quantization step sizes assigned to neighboring image blocks encoded in the past from the quantization step size buffer to generate a predicted quantization step size.
[0058]
The predicted quantization step size supplied from the predicted quantization step size generator 10313 is subtracted from the input quantization step size, and the result is input to the entropy encoder 10312 as a difference quantization step size.
[0059]
The entropy encoder 10312 entropy-encodes the input difference quantization step size and outputs the result as code corresponding to the quantization step size.
[0060]
Such a structure can reduce the code rate required to encode the quantization step size, and hence high-quality

moving image encoding can be achieved. The reason is that the absolute amount for the difference quantization step size input to the entropy encoder 10312 can be reduced because the predicted quantization step size generator 10313 generates the predicted quantization step size using the quantization step sizes of neighboring image blocks independent of the encoding sequence. The reason why the absolute amount for the difference quantization step size input to the entropy encoder 10312 can be reduced if the predicted quantization step size is generated using the quantization step sizes of the neighboring image blocks is because there is generally correlation between neighboring pixels in a moving image and hence the degree of similarity of quantization step sizes assigned to neighboring image blocks having high correlation with each other is high when visual-sensitivity-based adaptive quantization is used.
[0061]
A specific operation of the quantization step size encoder in the video encoding device in the first exemplary embodiment is described below by using a specific example.
[0062]
In this example, it is assumed that the image block size as the unit of encoding is a fixed size. It is also assumed that three image blocks respectively adjacent leftwardly, upwardly, and diagonally right upward within the same image frame are used as neighboring image blocks used for prediction of the quantization step size.
[0063]
Suppose that the current image block to be encoded is denoted by X, and three neighboring image blocks A, B, and C are located respectively adjacent leftwardly, upwardly, and diagonally right upward to the image block X as shown in FIG. 2. In this case, if the quantization step size in any block Z is denoted by Q(Z) and the predicted

quantization step size is denoted by pQ(Z), the predicted quantization step size generator 10313 determines the predicted quantization step size pQ(X) by the following equation (3).
[0064]
pQ(X) = Median(Q(A), Q(B), Q(C)) … (3)
Note that Median(x, y, z) is a function for determining an intermediate value from three values of x, y, z.
[0065]
The entropy encoder 10312 encodes a difference quantization step size dQ(X) obtained by the following equation (4) using signed Exp-Golomb (Exponential-Golomb) code as one of entropy codes, and outputs the result as code corresponding to a quantization step size for the image block concerned.
[0066]
dQ(X) = Q(X) - pQ(X) … (4)
[0067]
In this example, the three image blocks adjacent leftwardly, upwardly, and diagonally right upward within the same image frame are used as the neighboring image blocks used for prediction of the quantization step size. However, the neighboring image blocks are not limited thereto. For example, image blocks adjacent leftwardly, upwardly, and diagonally left upward may be used to determine the predicted quantization step size by the following equation (5).
[0068]
pQ(X) = Median(Q(A), Q(B), Q(D)) … (5)
[0069]
The number of image blocks used for prediction may be any number rather than three, and a mean value or the like rather than the intermediate value may be used as the calculation used for prediction may use. The image blocks

used for prediction are not necessarily to be adjacent to the image block to be encoded. The image blocks used for prediction may be separated by a predetermined distance from the image block to be encoded. Further, the image blocks used for prediction are not limited to image blocks located in the spatial neighborhood, i.e. within the same image frame, they may be image blocks within any other image frame already encoded.
[0070]
Further, in this example, it is assumed that the image block to be encoded and the neighboring image blocks are of the same fixed size. However, the present invention is not limited to the case of the fixed size, and the block size as the unit of encoding may be a variable size.
[0071]
Further, in this example, encoding is performed based on the Exp-Golomb code to encode the difference between the quantization step size of the image block to be encoded and the predicted quantization step size. However, the present invention is not limited to use of the Exp-Golomb code, and encoding may be performed based on any other entropy code. For example, encoding based on Huffman code or arithmetic code may be performed.
[0072] The above has described the video encoding device in the first exemplary embodiment of the present invention. [0073]
Exemplary Embodiment 2
Like the video decoding device shown in FIG. 26, a video decoding device in a second exemplary embodiment of the present invention includes the variable-length decoder 201, the inverse quantizer 202, the inverse frequency transformer 203, the frame memory 204, the intra-frame predictor 205, the inter-frame predictor 206.