[DESCRIPTION]
[Title of Invention]
VIDEO CODING METHOD, VIDEO DECODING METHOD, VIDEO CODING APPARATUS, VIDEO DECODING APPARATUS, AND CORRESPONDING PROGRAM AND INTEGRATED CIRCUIT
[Technical Field] [0001]
The present invention relates to filters for video coding, in particular to filters applied to a signal reconstructed from a prediction signal and a quantized prediction error signal in the context of differential pulse code modulation. [Background Art]
[0002]
For the compression of video data, a plurality of video coding standards has been developed. Such video standards are, for instance, ITU-T standards denoted with H.26x and ISO/IEC standards denoted with MPEG-x. The most up-to-date and advanced video coding standard is currently the standard denoted as H.264/MPEG-4 AVC.
[0003]
All of these standards are based on hybrid video coding, which consists of the following main stages:
(a) Dividing each individual video frame into two-dimensional blocks of pixels in order to subject each video frame to data compression at a block level.
(b) Decorrelating spatiotemporal video information by applying a spatio-temporal prediction scheme to each block and by transforming the prediction error from the spatial domain into the frequency domain so as to obtain coefficients.
(c) Reducing the overall amount of data by quantizing the resulting coefficients.
(d) Compressing the remaining data by coding the quantized coefficients and prediction parameters by means of an entropy coding algorithm.
[0004]
Hence, state-of-the-art video standards employ a Differential F'ulse Code Modulation (DPC|v|) approach whlrli only transmits differences between blocks of an Input video se«|uenc(( and their predictions based on previously coded blocl [0005]
FIG. 1 is an exemplary block diagram of a conventional video coding apparatus, which is in accordance with the H.264/AVC standard. The video coding apparatus includes a subtracter 110 for determining differences between a current block of a video image (input signal) and a prediction signal of the current block which is obtained based on previously coded blocks ("the locally decoded image") stored in a memory 140. The memory unit 140 thus operates as a delay unit that allows a comparison between current signal values and a prediction signal generated from previous signal values. A transform and quantization unit 120 transforms the resulting prediction error from the spatial domain to the frequency domain and quantizes the obtained coefficients. An entropy coding unit 190 performs entropy coding on the quantized coefficients. [0006]
The locally decoded image Is provided by a decoding unit incorporated into video coding apparatus. The decoding unit performs the coding steps in reverse manner. An inverse quantization and inverse transform unit 130 inversely quantizes the quantized coefficients and applies an inverse transform to the inversely quantized coefficients. In an adder 135, the decoded differences are added to the prediction signal to form the locally decoded image. [0007]
Due to the quantization, quantization noise Is superimposed to the reconstructed video signal. Due to the blockwise coding, the superimposed noise often has blocking characteristics, which may be subjectively annoying. In order to reduce the blocking characteristics, a deblocking filter 137 Is applied to every reconstructed macroblock. The deblocking filter 137 is applied on the reconstructed signal, which is tlie sum of the prediction signal and the quantized prediction error signal. The deblocked signal Is the decoded signal, which is generally displayed. The deblocking filter 137 in H.264/AVC has the capability of local adaptation. In the case of a high degree of blocking noise, a strong low paws filter Is applied whereas for a low degree of blocking noise, a weak low pass filter Is applied. The strength of the low pass filter Is dotermlned by the prfidlrlinn •signal and by the quantized prediction error signal. This deblocking filter 137 has the following two benefits: [0008]
1. Block edges are smoothed resulting In an enhanced
subjective quality of decoded images.
[0009]
2. Since the filtered macroblock is used for the motion
compensated prediction of further images, the filtering may result in
smaller prediction errors and thus In increased coding efficiency.
[0010]
Intra-coded macroblocks are a specific case In this context;
they are filtered before display, but Intra prediction Is carried out
using the unfiltered reconstructed macroblocks.
[0011]
As an example for deblocking at a vertical block boundary, a linear deblocking filter with four coefficients is provided. This filter is applied to input samples p2, Pi, Po, qo/ qi and q, wherein po and qo are two adjacent pixels at the block boundary, pi and qi pixels adjacent to po and qo, and so on. The filter output po, new and qo, new is then defined as po, new = (p2 - (pi << 1) + (po + qo + 1) >> 1) >> 1 and qo, new = (q2 - (qi << 1) + (qo + Po + 1) >> 1) >> 1. [0012]
The type of prediction that is employed by video coding apparatus 100 depends on whether the macroblocks are coded in "Intra" or "Inter" mode. In "Intra" mode, the video coding standard H.264/AVC uses a prediction scheme based on already coded macroblocks of the same image in order to predict subsequent macroblocks. In "Inter" mode, motion compensated prediction between corresponding blocks of several consecutive frames is employed. [0013]
Only Intra-coded images (I-type images) can be decoded without reference to any previously decoded image. The I-type images provide error resilience for the coded video sequence. Further, entry points into bit streams of coded data are provided by the I-type images in order to enable a random access, that is, to access I-type Images within the sequence of coded video Images. A switch between Intra-mode, that Is, processing by Intra-frame prediction unit 150, and Inter-mode, that Is, processing by motion compensated prediction unit 160, Is controlled by Intra/lnter switch 180. [0014]
In "Inter" mode, a macroblock Is predicted from corresponding blocks of previous frames by employing motion compensated prediction. The estimation Is accomplished by a motion estimation unit 170, receiving the current Input signal and thw locally decoded image. Motion estimation yields two-dimensional motion vectors, representing displacements between the current block and the corresponding block In already decoded frames. Based on the estimated motion, a motion compensated prediction unit 160 provides a prediction signal.
[0015]
In order to optimize prediction accuracy, motion vectors may be determined at sub-pel resolution, for example, half-pel or quarter-pel resolution. A motion vector with sub-pel resolution may point to a position, within an already decoded frame. In which no pixel value is available, that is, a sub-pel position. Hence, spatial interpolation of pixel values is needed in order to perform motion compensation. This is achieved by interpolation filter 162. According to the H.264/A\/C standard, a 6-tap Wiener interpolation filter with fixed filter coefficients and a bilinear filter are applied in order to obtain pixel values for sub-pel positions. [0016]
For both the "Intra" and the "Inter" coding mode, the differences between the current input signal and the prediction signal are transformed and quantized by a transform and quantization unit 120 resulting in quantized coefficients. Generally, an orthogonal transform such as a two-dimensional Discrete Cosine transform (DCT) or an integer version thereof is employed. [0017]
The coefficients are quantized in order to reduce the amount of data that has to be coded. The step of quantization is controlled by quantization tables that specify the precision and therewith the number of bits that are used to code each quantized coefficient. Lower frequency components are usually more Important for picture quality than high frequency components so that more bits are spent for coding the low frequency components than for the higher ones. [0018]
After quantization, the two-dimensional array of quantized coefficients has to be converted into a one-dlmenslonal string to pass it to the entropy encoder. This conversion is done by scanning the array in a predetermined sequence. The thus obtained one-dlmenslonal sequence of quantized coefficients Is compressed and coded by an entropy coding unit 190 by means of a variable length code (VLC). The resulting bit stream is multiplexed with the motion information and stored on a recording medium or transmitted to the video decoding apparatus side. [0019]
In order to enhance the subjective picture qunllly, a so-called post filter may be applied at the video dt'cocllntj apparalu;:. side. The standard H.264/AVC allows sending of post filter information for such a post filter by the use of an SEl (Supplemental Enhancement Information) message. The post filter information may be either the filter coefficients Ihemselves or crosscorielation Information which can be used to cnlculate the filter coefficients. The post filter information is determined on the video coding apparatus side by means of a post filter design unit 138, which compares the locally decoded signal and the original input signal. The output of the post filter design unit 138 is also fed to the entropy coding unit 190 in order to be coded and inserted into the coded signal. [0020]
To reconstruct the coded images at the video decoding apparatus side, the coding processing Is applied in reverse manner. FIG. 2 is a schematic block diagram illustrating the configuration of the corresponding video decoding apparatus. [0021]
In the video decoding apparatus In FIG. 2, first the entropy coding of coefficients and motion data are reversed in an entropy decoding unit 191. This step also involves an Inverse scanning in order to convert the string of decoded coefficients into a two-dimensional array of data as it is required for the inverse transform. The decoded block of quantized coefficients is then submitted to an inverse quantization and inverse transform unit 121 and the decoded motion data Is sent to a motion compensated prediction unit 160. Depending on the actual value of the motion vector, interpolation of pixel values may be needed in order to perform the motion compensated prediction. This interpolation is performed by interpolation filter 162. [0022]

The result of the inverse transform is the quantized prediction error in the spatial domain, which is added by adder 135 to the prediction signal stemming from the motion compensated prediction unit 160 in inter mode or stemming from an intra-frame prediction unit 150 in intra mode. The reconstructed image may be passed through a deblocking filter 137 and the resulting decoded signal is stored in memory 140 to be applied to an intra-frame prediction unit 150 and a motion compensated prediction unit 160. [0023]
The entropy decoding unit 191 is also retrieving the post filter information determined by the encoder. The post filter unit 139 employs this information in order to set up a post filter that Is applied to the decoded signal in order to further enhance the picture quality. [0024]
The above described conventional post filter Is an example of an adaptive flltfu, namely of a filter that Is adapted to the properties of the video signal that is to be coded. Iany Impleniuntailons of such an adaptive filter are based on Wiener filters, that Is, on linear optimum time-discrete filters. In the context of video coding, the quantization error can be considered as noise supijrimposed on the original video input signal. In tlie context of video coding, the quantization error can be connldfirnd as nolSR Rupftrlmpnfied on the original video Input signal. [0025]
One of the advantages of the Wiener filter approach is that the filter coefficients can be determined from the auto-correlatlon of the corrupted signal (decoded video signal) and the cross-correlation between the corrupted signal and the desired signal (original video
input signal). More specifically, If R denotes the MxM
auto-correlation matrix of the corrupted signal and p denotes the Mxl cross-correlation vector between the corrupted signal and the desired signal, wherein M is the length of the Wiener filter, then the Mxl vector of optimum Wiener filter coefficient w Is given as [0026] [Math.]

Expression 1
[0027]
wherein R" is the inverse of the auto-correlation matrix R. [0028]
From reference European Patent Application Publication No. 1841230 (Patent Literature 1), for instance, it is known to apply adaptive filters within the prediction loop, namely to the reconstructed signal directly before or Instead of the deblocking filter 137, to the output of the deblocking filter, or to the prediction signal. The purpose of these conventional video filters is to minimize the mean squared prediction error and/or the mean squared reconstruction error and/or to deblock the image so as to enhance subjective picture quality. [Citation List] [Patent Literature 1] [0029]
European Patent Application Publication No. 1841230
[Summary of Invention] [Technical Problem] [0030]
However, this filtering Is effective only when a prediction signal and a quantized prediction error signal have the same statistical properties. When a prediction signal and a quantized prediction error slynal have different statistical properties such as different superimposed noise, this filtering format cannot support such difference in the statistical properties, and thus the filtering design is disadvantageous. [0031]
Some reasons for such difforence In th« ataUstlcal properties between the prediction signal and the quantized prediction error signal are conceivable. First, quantization noise is superimposed only on the prediction error during the quantization, but not on the
prediction signal Itstjlf at a curtalri point of time, The prediction sicinal may Include quantization noise at a different point of time, and therefore iriay have different statistical propertlets. Second, the edges ol blocks In motion compensated prediction may be different from the edges of blocks in prediction error coding, and the Image may have different blocking characteristics In the prediction signal compared to the image of the quantized prediction error signal. [0032]
The aim of the present invention is to provide methods for video coding with increased coding efficiency and corresponding apparatuses. [Solution to Problem] [0033]
A video coding method according to an embodiment of the present Invention Is for coding a signal to be coded which represents a video. More specifically, the video coding method Includes: generating a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded; quantizing a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients; inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating first filter information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generating second filter information, based on statistical properties of only the quantized prediction' error signal among the prediction signal and the quantized prediction error signal; and performing entropy coding on the quantized coefficients generated In the quantizing and the first and second filter Information generated In the generating of filter Information so as to generate a coded signal. [0034]
With this configuration, It is possible to Individually remove the noise to be superimposed on the prediction signal and the quantized prediction error signal. As a result. It is possible to Increase the coding efficiency.

100351
In addition, in the generating of filter information, the first and second filter information may be generated so as to minimize a difference between (I) the signal to be coded, and (II) a reconstructed signal obtained by adding (II-I) a filtered prediction signal obtained by filtering the prediction signal based on the first filter information, and (Il-ii) a filtered quantized prediction error signal obtained by filtering the quantized prediction error signal based on the second filter Information. This makes It possible to enhance the subjective picture quality of the decoded Image. [0036]
In addition, the generating of a prediction signal may include: filtering the prediction signal based on the first filter information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating a reconstructed signal by adding the filtered prediction signal and the filtered quantized prediction error signal; and generating a second prediction signal predictive of a signal to be coded subsequent to the coding of the signal to be coded, based on the reconstructed signal. As in the configuration. It is also good to apply a loop filter to the first and second filter Information. [0037]
As an embodiment, in the filtering, deblocking of the reconstructed signal is performed so as to reduce block distortion. As another Embodiment, interpolation filtering Is performed prior to the motion compensated prediction In the filtering, [0038]
In addition, in the generating of filter information, third filter information may be generated based on statistical properties of a reconstructed signal obtained by adding the prediction signal and the quantized prediction error signal, and In the performing of entropy coding, entropy coding may be performed on the quantized prediction error signal, and the first, second, and third filter information so as to generate tho coded signal. As In the configuration, It is also good to Cjenerale third filter Information to be used by a ronventlonal post filter, In addition to the first and second filter Information.
[0039:1
A video decoding method according to tin OMil)odirnnnl ol Ihu present invention is for generating a decoded signal by decoding a coded signal which represents a video. More specifically, the video decoding method includes: performing entropy decoding on th«! coded signal to obtain quantized coefficients and first nnd second filter Information; Inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating a prediction signal predictive of the decoded signal, based on a decoded signal decoded prior to the decoding of the coded signal; and filtering the prediction signal based on the first filter Information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating the decoded signal by adding the filtered prediction signal and the filtered quantized prediction error signal. [0040]
With this configuration, it is possible to Individually remove the noise to be superimposed on the prediction signal and the quantized prediction error signal. Thereby, it is possible to enhance the subjective picture quality. [0041]
In addition, the generating of a prediction signal may include: filtering the prediction signal based on the first filter Information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating a reconstructed signal by adding the filtered prediction signal and the filtered quantized prediction error signal; and generating a second prediction signal predictive of a signal to be coded subsequent to the coding of the signal to be coded, based on the reconstructed signal. As in the configuration, it Is also good to apply a loop filter to the first and second filter information without being limited to a post filter.

[0042]
A video coding apparatus according to an embodiment of the present invention codes a signal to be coded which represents a video. More specifically, the video coding apparatus Includes: a prediction signal generation unit configured to generate a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded; a quantization unit configured to quantize a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients; an inverse quantization unit configured to inversely quantize the quantized coefficients to generate a quantized prediction error signal; a filter information generation unit configured to generate first filter Information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generate second filter Information, based on statistical properties of only the quantized prediction error signal among the prediction signal and the quantized prediction error signal; and an entropy coding unit configured to perform entropy coding on the quantized coefficients generated by the quantization unit and the first and second filter information generated by the filter Information generation unit so as to generate a coded signal. [0043]
A video decoding apparatus according to an embodiment of the present invention generates a decoded signal by decoding a coded signal which represents a video. More specifically, the video decoding apparatus Includes: an entropy decoding unit configured to perform entropy decoding on the coded signal to obtain quantized coefficients and first and second filter Infarmntlon; an Invtuse quantization unit configured to Inversely quontl/e thu quantized coefficients to generate a quantized prediction error signal; a prediction signal generation unit configured to generate a prediction signal predictive of the decoded signal, baHod on a decoded Hignal decoded prior to the decoding of the coded signal; tmd a filtering unit configured to filter the prediction signal based Dn the first filter I information to generate a filtered prediction signal, to filter the
quantized prediction error filgiuil based on the second filter Inlorniatlon to generate a filtered ijuantlzed prodU Hon error Silgnal, and generate the decoded signal by adding the filtered prediction signal and the filtered quantized prediction error siunal. [0044]
A program according to an embodiment of the present invention causes a computer to code a signal to be coded which represents a video. More specifically, the program Includes: generating a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded; quantizing a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients; inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating first filter information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generating second filter information, based on statistical properties of only the quantized prediction error signal among the prediction signal and the quantized prediction error signal; and performing entropy coding on the quantized coefficients generated in the quantizing and the first and second filter information generated In the generating of filter information so as to generate a coded signal. [0045]
A program according to an embodiment of the present invention causes a computer to decode a coded signal which represents a video. More specifically, the program includes: performing entropy decoding on the coded signal to obtain quantized coefficients and first and second filter information; Inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating a prediction signal predictive of the decoded signal, based on a decoded signal decoded prior to the decoding of the coded signal; and filtering the prediction signal based on the first filter Information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating the decoded signal by adding the filtered prediction signal and the
filtered quantized prediction error signal,
[0046]
An Integrated circuit according to an enribodlment of the present Invention codes a signal to be coded which represents a video. More specifically, the Integrated circuit Includes: a prediction signal generation unit configured to geniirnte a prediction wltinol predlcllvu of the signal to be coded, based on a coded slgnol i odud inlor tu I he coding of the signal to be coded; a quantization unit configured to quantize a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quanllzod coefficients; an Inverse quantization unit configured to Inversely quantize the quantl/.esd coefficients to generate An Integrated circuit according to an embodiment of the present Invention generates a decoded signal by decoding a coded signal which represents a video. More specifically, the Integrated circuit includes: an entropy decoding unit configured to perform entropy decoding on the coded signal to obtain quantized coefficients and first and second filter information; an inverse quantization unit configured to inversely quantize the quantized coefficients to generate a quantized prediction error signal; a prediction signal generation unit configured to generate a prediction signal predictive of the decoded signal, based on a decoded signal decoded prior to the decoding of the coded signal; and a filtering unit configured to filter the prediction signal based on the first filter Information to generate a filtered prediction signal, to filter the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generate the decoded signal by adding the filtered prediction signal and the filtered quantized prediction error signal. [0048]
It is to be noted that the present invention can be implemented not only as video coding methods (apparatuses) and video decoding methods (apparatuses), but also as integrated circuits which implement these functions and as programs causing computers to execute these functions. Furthermore, these programs can be distributed through recording media such as CD-ROMs and recording media such as the Internet as a matter of course. [0049]
This is achieved by the features as set forth In the Independent Claims. [OOBOJ
Preferred Embodiments are the subject matter of dependent claims.
[0051]
It is the particular approach of the present invention to replace a conventional adaptive filter operating on the reconstructed signal by two filters operating on the prediction signal and the quantized prediction error signal, respectively, which are individually adapted to the statistical properties of their respective Input signals. [0052]
According to a first aspect of the present Invention, a method of coding a video signal Is provided. The method Is based on differential pulse code modulation and Includes the steps of computing a prediction signal and an error signal from the video signal to be coded, and is characterized by obtaining first filter information Indicating a first filter adapted to statistical properties of the prediction signal, obtaining second filter Information Indicating a second filter adapted to statistical properties of the error signal, and coding the first filter information and the second filter Information. [0053]
According to a further aspect of the present Invention, a method of decoding a video signal Is provided. The method Is based on differential pulse code modulation and includes the steps of obtaining a prediction signal and an error signal, and computing a reconstructed signal from the prediction signal and the error signal, and is characterized by decoding first filter Information and second filter Information, setting a first filter and a second filter in accordance with the first filter information and the second filter Information, respectively, and filtering the prediction signal and the error signal by the first filter and the second filter, respectively. Here, wherein the reconstructed signal is computed by adding the filtered prediction signal and the filtered error signal. [0054]
According to a further aspect of the present invention, an apparatus for coding a video signal Is provided. The apparatus includes a differential pulse code modulation unit for computing a prediction signal and an error signal from the video signal to be coded and is characterized by a filter design unit for obtaining first filter information indicating a first filter adapted to statistical properties of the prediction signal and for obtaining second filter information indicating a second filter adapted to statistical properties of the error signal, and an encoder for coding the first filter information and the second filter Information. [0055]
According to a further aspect of the present Invention, an apparatus for decoding a video signal Is provided. The apparatus Is based on differential pulst? code modulnllon and Includes n reconstruction unit for obtaining a prediction signal find an error signal and for computing a reconstructed signal from the prediction signal and the error signal, and Is characterized by a decoder for decoding first filter Information and second filter Ijiformfltjon, a first filter set In accordance with the first filter Information for filtering the prediction signal, and a second flltur set In accordance with the second filter information for filtering the error sli)nol. Herri, the rejconstriK.tion unit Is adapted lo compute the roconfitructod algruil by adding Ihc filtered |)redlctlon slgniil and the flllered error signal. [00 J, 61
In a preferred Embodiment of the coding method or apparatus, the first filter information and the second filter Information are obtained by minimizing a measure Indicating a difference between the video signal to be coded and a reconstructed signal, the reconstructed signal being the sum of the prediction signal filtered by the first filter and the error signal filtered by the second filter. [0057]
In another preferred Embodiment of the coding method or apparatus, the first filter information is obtained by analyzing statistical properties of the prediction signal and the video signal to be coded, and the second filter Information is obtained by analyzing statistical properties of the error signal and the video signal to be coded. In another preferred Embodiment of the coding method or apparatus, the present invention is applied to post filtering, wherein the inventive post filter is operating on the prediction signal, the error signal, and the decoded signal. To this end, the prediction signal and the error signal are added, a predetermined filter Is applied to the result of the addition, third filter information indicating a third filter adapted to statistical properties of a signal resulting from the step of applying the predetermined filter Is obtained, and the third filter information is also coded. Preferably, the predetermined filter is a deblocking filter. [0058]
In a corresponding preferred Embodiment of the decoding method or apparatus, the prediction signal and the error signal are added, a predetermined filter is applied to the results of the addition, third filter information is decoded, a third filter is set in accordance with the third filter information, and the third filter Is applied to an output signal of the predetermined filter. Here, the reconstructed signal is computed by adding the filtered prediction signal, the filtered error signal, and an output signal of the third filter.

[0059]
In another preferred Embodiment of the coding method or apparatus, the present invention is applied to post filtering, wherein the Inventive post filter is operating on the prediction signal, the error signal, and the output signal of a conventional post filter. To this end, the prediction signal and the error signal are added, a predetermined filter Is applUid to the results of the adding step, post filter Infornuitlon based on statistical properties of M signal reBultlntl from the stup of applying the predetermined filter Is obtained, a p(i8t fllttir Is set In accordance with the post filter Information, the post filter Is applied to a signal resulting from the step of applying the priideterrnined filter, third filter information indicating a third filter adapted to slfitlstlcai properties of a signal resulting from the step ol applying the post filter Is obtained, and the third llltei Information and the post filter information are also coded. (00601
In a corresponding preferred Embodiment of the decoding method or apparatus, the prediction signal and the error signal are added, a predetermined filter is applied to the results of the addition, post filter information and third filter Information are decoded, a post filter and a third filter are set in accordance with the post filter Information and the third filter information, respectively, the post filter is applied to an output signal of the predetermined filter, and the third filter is applied to an output signal of the post filter. Here, the reconstructed signal Is computed by adding the filtered prediction signal, the filtered error signal, and an output signal of the third filter. [0061]
In a preferred Embodiment of both the coding and decoding methods or apparatuses, the present invention Is applied as a loop filter. To this end, the filtered prediction signal and the filtered error signal are added, and the result of the addition is delayed. Here, the prediction signal is obtained from the delayed results. [0062]
In another preferred Embodiment of both the coding and decoding methods or apparatuses, the present invention is applied to adaptive interpolation filtering for motion compensated prediction with sub-pel precision. To this end, the prediction signal is delayed, wherein the first filter is applied to the delayed prediction signal, the error signal is delayed, wherein the second filter is applied to the delayed error signal, and the filtered delayed prediction signal and the filtered delayed error signal are added, wherein the prediction signal is obtained from the result of the addition. [0063]
Preferably, the first filter information, the second filter information, and the third filter Information Include filter coefficients of the first filter, the second filter, and the third filter, respectively. In this manner, the computational load for the decoder can be reduced, because the decoder can easily set up the respective filters without any further computation. [0064]
Preferably, the first filter Information and the second filter Information Include an Indication of statistical properties of the prediction signal and the error signal, respectively. In this manner, tliti compultiiional IOIKI for the encoder can be reduced, becausu Mn' decoder Is tal
Preferably, the first filter and (he second flltei are Wiener filters. Wiener filters are well-studied examples of linear optimum filters that can readily be determined from statistical properties of their Input signal and the desired output signal.
[Advantageous Effects of Invention] [0066]
According to the present Invention, prediction signals and quantized prediction error signals are analyzed in separate filtering processes. This mal

[Brief Description of Drawings]
[DRAWINGS]
[0067]
[FIG. 1]
FIG. 1 is a block diagram of a conventional video coding apparatus, [FIG. 2]
FIG. 2 is a block diagrann of a conventional video decoding apparatus. [FIG. 3A]
FIG. 3 is a block diagram of a video coding apparatus according to Embodiment 1 of the present invention. [FIG. 3B]
FIG. 3 is a flow chart indicating operations of the video coding apparatus shown in FIG. 3A. [FIG. 4A]
FIG. 4 is a block diagram of a video decoding apparatus according to Embodiment 1 of the present invention. [FIG. 4B]
FIG. 4 is a flow chart indicating operations of the video decoding apparatus shown in FIG. 4A. [FIG. 5]
FIG. 5 Is a block diagram of a video coding apparatus according to Embodiment 2 of the present invention. [FIG. 6]
FIG. 6 Is a block diagram of a video decoding apparatus according to Embodiment 2 of the present Invention. [FIG. 7]
FIG. 7 Is i\ block diagram of a video coding nppnratuB according to Embodiment 3 of the present Invention. [FIG. 8]
FIG. 8 Is a block diagram of a video docodlng apparatus according to Embodiment 3 of the present InvHntlon. [FIG. 9]

FIG. 9 is a block diagram of a video coding apiiaiiitus according to Embodiment 4 of the present Invention. [(■IG. :I01
riG. 10 Is a block diaijrain of a vldtio decoding ap|)orcilii,'i according to limbodlment 4 of Ihe present Invention. [TIG. IIA)
riG. I. lA is a diagram showing exemplary rato distortion curves for IPEG test sequences. [FIG. IIB]
FIG. 1 IB is a diagram showing other exemplary rate distortion curves for IPEG test sequences.
[Description of Embodiments] [0068]
The H.264/A\/C standard specifies two filter schemes (interpolation filter and deblocking filter) as well as the possibility to transmit post filter hints to a video decoding apparatus according to a specific post-filter scheme (post-filter hint SEI message). The following Embodiments of the present invention will show how each of the three schemes is improved by applying this Invention.
(Embodiment 1) [0069]
FIG. 3A is a block diagram of a video coding apparatus 100 according to Embodiment 1 of the present invention. FIG. 38 is a flow chart indicating operations of the video coding apparatus 100 in FIG. SA.The block diagram of FIG. 3A is similar to that of the conventional video coding apparatus shown in FIG. 1, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0070]
The video coding apparatus 100 shown In FIG. 3A Includes a subtractor 110, a transform and quantization unit 120, an Inverse quantization and inverse transform unit 130, a post filter design unit (filter Information generation unit) 138', an entropy coding unit 190,
and a prediction signal generation unit 200. This video coding apparatus 100 codes a signal to be coded which represents a video, and outputs the coded signal. [0071]
The subtractor 110 generates a prediction error signal by subtracting the prediction signal from the signal to be coded (Sll). The transform and quantization unit 120 generates quantized coefficients by quantizing the prudiction error (SI2). The Inverse quantization and inverse transform unit 130 quantized coefficients to generate a quantized |)re(l|r:tlt)n oi ror slyntil (S13). The post filter design unit 138' generates first filter Information based on the signal to be coded and l;ho prediction signal, and generates the second filter Information based on the filgnnl to be coded and the quantized prediction nrror slgntil (Sll). Subsequently, the post filter design unit 138' outputs post filter informaion Including the first and second filter Information. The entropy coding unit 190 performs entropy coding on the quantized coefficients, post filter information, and later-described motion data to generate a coded signal (S15). The prediction signal generation unit 200 generates a prediction signal for the signal to be coded, based on a signal coded prior to coding of the signal to be coded (Input signal) (S16). [0072]
It is to be noted that the prediction signal generation unit 200 includes an adder 135, a deblocking filter 137, a memory 140, an intra-frame prediction unit 150, a motion compensated prediction unit 160, an interpolation filter 162, a motion estimation unit 170, and an intra/inter switch 180. This prediction signal generation unit 200 forms a loop starting with an operation by a subtractor 110. For this, the filters (such as the deblocking filter 137 and the Interpolation filter 162) Included in the prediction signal generation unit 200 are also referred to as loop filters. [0073]
The prediction signal generation process (S16) in FIG. 3B is described In detail. The adder 135 generates a reconstructed signal by adding the quantized prediction error signal and the prediction

signal. The deblocking filter 137 generates a decoded signal by removing block distortion from the reconstructed signal. The memory 140 functions as a delay device for temporarily storing a decoded signal. The intra-frame prediction unit 150 generates a prediction signal by performing intra-frame prediction on the decoded signal. The interpolation filter 162 spatially interpolates the pixel values of the decoded signal prior to motion compensated prediction. The motion estimation unit 170 performs motion estimation based on the decoded signal and the next signal to be coded so as to generate motion data. The motion compensated prediction unit 160 performs motion compensated prediction based on the decoded signal and motion data to generate a prediction signal. The intra/inter switch 180 selects, as a prediction mode, either the "intra" mode or the "inter" mode. Consequently, the prediction signal outputted from the intra/inter switch 180 is the prediction signal for the next signal to be coded. [0074]
The video coding apparatus 100 according to Embodiment 1 of the present Invention differs from the conventional video coding appfuiaturj In \\m configuration of the post flltur denlgn unit :I3M'. In contrast to the conventional post filter design iinll: lilH, tho new |)OMt filter design unit 138' receives the prediction signal and the quantized prediction error signal in addition to the signni to lie codnd. In this manner, the new post filter design unit 138' may take different statistical properties of the prediction slQiml and the quantlzod prediction error signal Into account when determining optimum filter Information for filtering the prediction signal and the quantized prediction error signal. [0075]
Specifically, the new post filter design unit 138' may compute auto-correlation functions of the prediction signal and the quantized prediction error signal, the cross-correlation function between the prediction signal and the input signal, as well as the cross-correlation function between the quantized prediction error signal and the Input signal. In order to determine an optimum filter Information for each of the prediction signal and the quantized prediction error signal. In other words, the new post filter design unit 138' may apply a Wiener approach in order to determine filter information that minimizes a difference, that is, the mean squared error, between the signal to be coded and the output of the new post filter 139'. The result of this determination is fed as a post filter Information to the entropy coding unit 190 in order to be Inserted Into the coded signal. It is to be noted that the post filter information may Include a set of filter coefficients for each of the two filters, the coefficients of the auto-correlation matrices and the cross-correlation vectors, or any other information that allows the video decoding apparatus 101 to set up appropriate filters. In this context, it is also noted that the auto-correlation functions of the prediction signal and the quantized prediction error signal may also be computed on the video decoding apparatus lOl side so that only the cross-correlation functions between these signals and the input signal need to be transmitted. [0076]
As a specific example, the computation of filter coefficients of the new post filter 139' is detailed below. [0077]
The output of the new post filter 139' may be expressed as the following Expression 2. [0078]

[0079]
wherein Wi,..., WM are M filter coefficients of the first filter 139-1 applied to M prediction samples P\,.>,, PMJ «nd WMM,.-; WMIN are N filter coefficients of the second filter 139-2 appllMd tu N snmpUiH ei,..., eM of the quantized prediction signal. [0080]
Applying the WIener-Hopf equation, the flltei* coefficients that

minimize tlie mean squared error between the deslrod slgncil s and the filtered signal s' can be determined by the foliowinii Exprusslon 3,

[0082]
wherein E[-] denotes the expected value. Hence, the filter coefficients Wi,..., WM+N are determined by the auto-correlation of the prediction signal, the auto-correlation of the quantized prediction error signal, the cross-correlation of the prediction signal and the quantized prediction error signal, the cross-correlation of the prediction signal and the desired signal, and the cross-correlation of the quantized prediction error signal and the desired signal. In the case where the prediction signal and the quantized prediction error signal are statistically independent, above Expression 3 is blocl<~diagonal and the number of filter information Items of the later-described first and second filters 139-1 and 139-2 are determined by the auto-correlation of the prediction signal (quantized prediction error signal) and the cross-correlation of the prediction signal (quantized prediction error signal) and the desired signal only. [0083]
According to Expression 3, It is possible to derive filter coefficients that minimize noise depending on two signals p and e having different statistical properties. In particular, it Is also possible to reduce the amount of computation required to derive a filter assuming that there is no correlation (it Is to be noted that this filter can exert performance higher than that of the conventional filter which does not separate p and e from each other). Otherwise, It Is also possible to obtain the maximum noise reduction performance by
deriving a filter coefficient directly from Expression 3 based on the
correlation.
[0084]
The present invention is not limited to the above method of computing the filter coefficients. Instead, a numerical optimization may be performed in order to determine filter coefficients that minimize a predetermined measure indicating a difference between the flltiM'ed signal and the desired signal. Such a measure m
In a preferred Embodiment, the new posl fllt(
In this preferred Embodiment, the output of the new post filter 139' may be expressed In analogy to Expression 2 as the following Expression 4. [0087]
wherein WM+N+I,-, WM+N+L are L filter coefficients of the third filter 139-3 applied to L samples of the decoded signal. [0089]
Similar as above, the filter coefficients that minimize the mean squared error between the desired signal s and the filtered signal s' can be determined by the following Expression 5.

[0091]
wherein E [xiyi] denotes the sub-mntrix consisting of all cross-correjlatlon terms between x and y, [l)0!)i?]
FIG. 4A Is r\ bUnU dlMgrrim of a video rlRroflJiii)
The video decoding apparatus 101 shown In FIG. 4A Includes an entropy decoding unit 191, an Inverse quantization and Inverse transform unit 121, an adder 135, a deblocking filter 137, a memory 140, an Intra-frame prediction unit 150, a motion compensated prediction unit 160, an interpolation filter 162, an Intra/lnter switch 180, and a post filter 139'.

This video decoding apparatus 101 decodes a coded signal coded by tine video coding apparatus 100 shown in FIG. 3A to generate a decoded signal. It is to be noted that the functional blocl The entropy decoding unit 191 performs entropy decoding on the coded signal (input signal) outputted from the video coding apparatus 100 to obtain quantized coefficients, post filter information, and motion data (S21). The Inverse quantization and Inverse transform unit 121 inversely quantizes the quantized coefficients to generate a quantized prediction error signal (S22). The post filter 139' applies filtering on the respective prediction signal and quantized prediction error signal based on the post filter information (S23). Iore specifically, the post filter 139' includes a first filter 139-1 which filters the prediction signal based on the first filter information to generate a filtered prediction signal, a second filter 139-2 which filters the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and an adder 139-0 which adds the filtered prediction signal and the filtered quantized prediction error signal to generate a decoded signal. On the other hand, the prediction signal generation unit 200' generates a prediction signal for the next signal to be decoded, based on the decoded signal (S24). [0095]
The prediction signal generation unit 200' Includes an adder 135, a deblocking filter 137, a memory 140, an intra-frame prediction unit 150, a motion compensated prediction unit 160, an Interpolation filter 162, and an Intra/lnter switch 180. The different point from the prediction signal generation unit 200 shown In FIG. 3A Is obtaining motion dtila from the entropy decoding unit 191 without using the motion estimation unit 170. [0096]
The video decoding apparatus 101 according to Embodiment 1 o( the presenl: Invention differs from tin? Lonvoiitloiinl vlchso ilocodlnu apparatus in the configuration of the post filter :l lU. The new post filter 139' Includes at least two Independent filters, nno Is ii first filter 139-1 for filtering the prediction signal that is output from either the intra-frame prediction unit 150 or I hi! mollon i ompnnytitjid pi utile lion unit 160, and the other Is a se( imd filter 139-2 for fllteriny the quantized prediction error signal I hat is output from the inverse quantization and transform unit lil. The output of these filters is combined by an adder 139-0, and provided as the final output signal of the video decoding apparatus 101. [0097]
In a manner similar to the conventional post filter 139, the new post filter 139' receives post filter information derived from the Input signal by means of the entropy decoding unit 191. In contrast to the conventional video decoding apparatus, the post filter information is employed to set up at least two different filters, namely the first filter 139-1 for the prediction signal and the second filter 139-2 for the quantized prediction error signal. [0098]
As Indicated by dashed lines In FIG. 4A, the new post filter 139' may also Include a third filter 139-3 that is operating on the decoded signal output by the deblocl
As described above in connection with the video coding apparatus 100 in FIG. 3A, the post filter information may include a set of filter coefficients for each of these filters, the coefficients of the relevant auto-correlation matrices and cross-correlation vectors, or any other information that allows the video decoding apparatus 101 to set up appropriate filters. [0100]

In accordance with Embodiment 1 of the present invention, post filtering of decoded video information is performed by means of at least two independently optimized filters operating on the prediction signal, the quantized prediction error signal, and - optionally - on the output of the deblocking filter. The outputs of the first filter 193-1 and the second filter 193-2 are added in order to form a final output signal of the video decoding apparatus 101. In addition, In the case of addltioncJlly setting the third filter 193-3, It Is also good to average lluj refillII: of adding the outputs of the first and second filters 193-1 and 193-/' iind the output of the third filter 193-3. in this manner, (ixich ul th(;»Bi;! fUteifj niiiy be adnpttid to the stallMtk:«l piopurtiea t)f llu' sUjnal on which the filter 0|:»erates. In HIIM maiiiUM, the reconstruction error can be reduced more efficiently than with a single filter operating on the reconstructed signal only.
[01011 (lirnbodlrnenl: 2)
FIG. b Is a block diagram of a video coding apparatus 100 according to limbodlment 2 of the present Invention. The block diagram of FIG. 5 Is similar to that of the conventional video coding apparatus shown In FIG. 3A, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0102]
The video coding apparatus 100 according to Embodiment 2 of the present Invention differs from the conventional video coding apparatus in having the post filter 139 and the additional second post filter design unit 138'. In addition, the video coding apparatus 100 further differs from the video coding apparatus 100 shown In FIG. 3A In having the first post filter design unit 138 and the post filter 139. Further, the second post filter design unit 138' generates the third filter Information based on the statistical properties of the output signal of the post filter 139, in addition to exerting the function of the post filter design unit 138' shown In FIG. 3A. [0103]

In a manner similar to tlie conventional video coding apparatus, tlie first post filter design unit 138 in FIG. 5 determines filter information optimum for the first post filter 139 that is to be applied to the decoded signal output by the deblocking filter 137 of the video decoding apparatus 101. Such a post filter 139 is also part of the video coding apparatus 100 according to Embodiment 2. The output of this post filter 139 is fed to the additional second post filter design unit 138', which also receives the prediction signal, the quantized prediction error signal, and the input signal. Based on the statistical properties of its input signals, the additional second post filter design unit 138' determines filter information for three independent filters operating on the prediction signal, the quantized prediction error signal, and the output signal of the post filter 139. The thus determined filter information is fed to the entropy coding unit 190 in order to be inserted into the coded signal. [0104]
Specifically, the additional second post filter design unit 138' may compute auto-correlation functions of the prediction signal, the quantized prediction error signal, and the output signal from the post filter 139, as well as the cross-correlation functions between these signals and the Input signal, In order to determine optimum filter InrormalIon Inr tuirh of Ihe pr(»dlcllf)n signni, thw qiuuill/od prodlrilon error signal, and the output signal from the puHi filler l:i9. A lenult of this determination Is fed as additional post flltoi llifurm«tlon to the entropy coding unit 190 In order to be inserted Into tho coded signal. The post filter Information may Include a set of flltfar coefficients for each of the three filters, the coafflclenti nf llm niitxi-coirwlBllon matrices ond the cross-correlnl:lon vectors, or any other Information that allows the video decoding apparatus 10.1 to sut up appropriate filters. In this context, It Is also noted that the auto-correlatlon functions of the prediction signal, the quantized prediction error signal, and the output signal from the post filter 139 may also be computed on the video decoding apparatus 101 side so that qnly the cross-correlation functions between these signals and the Input signal need to be transmitted.

[01051
FIG. 6 Is a block diagram of a video decoding apparatus 1.01 according to Embodiment 2 of tiie present invention. The block diagram of FIG. 6 Is similar to that of the video decoding apparatus 101 shown in FIG. 4A, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0106]
The video decoding apparatus 101 according to Embodiment 2 of the present invention differs from the conventional video decoding apparatus In having the additional second post filter 139', which receives the prediction signal, the quantized prediction error signal and the output signal from the first post filter 139. In addition, the video decoding apparatus 101 further differs from the video decoding apparatus 101 shown in FIG. 4A in having the first post filter 139. [0107]
The additional second post filter 139' includes three independent filters, one Is a first filter 139-1 for filtering the prediction signal that is output from the Intra-frame prediction unit 150 or the motion compensated prediction unit 160, another Is a second filter 139-2 for filtering the quantized prediction error signal that is output from the inverse quantization and transform unit 121, and the other is a third filter 139-3 for filtering the post-filtered signal that is output from the first post filter 139. Subsequently, the result of adding the outputs of the first and second filters 139-1 and 139-2 and the output of the third filter 139-3 are averaged, and are provided as a final output signal of the decoding apparatus 101. [0108]
As explained above in connection with the video coding apparatus 100 in FIG. 5, the post filter information may Include a set of filter cotjfflcients for each of these filters, the coefficients of the relevant auto-correlation matrices and cross-correlation vectors, or any other Information that allows the video decoding apparatus to sot up appropriate filters. [0109]
(conventional) post filter. The output of the soparale flllurs Is added In order to form the final outpjit signul uf [\n\ VIJIUQ iliiirKllny apparatus 10:1, liach of these lilttJi u may be wdapteil to the etcitlstlcal propurtlofi of llie Bignal on which llui filter operates. In this numiier, the reconsi ruction error can be reduced more efficiently than with a single filter operating on the reconstructed signal only. Moreover, the coded signal generated by the video coding apparatus tOO of this Embodiment Is compatible to conventional video decoding apparatuses which may simply Ignore the additional post filter Information. The video decoding apparatuses 101 that are aware of the additional post filtering, however, may employ this information for setting up the additional post filter In order to further enhance picture quality.
[0110] (Embodiment 3)
FIG. 7 Is a blocl< diagram of a video coding apparatus 100 according to Embodiment 3 of the present Invention. The block diagram of FIG. 7 is similar to that of the video coding apparatus 100 shown In FIG. 3A, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0111]
The video coding apparatus 100 according to Embodiment 3 of the present invention differs from the conventional video coding apparatus and the video coding apparatus 100 shown In FIG. 3A In that the adder 135 and the deblocking filter 137 are replaced by a new loop filter 137' and a corresponding new loop filter design unit 138'. However, although the post filter design unit 138' shown In FIG. 3A and the post filter design unit 138" are different from each other In terms of whether the resulting filter Information is used by the post filter 139" or the loop filter 137', the filter information generation
process Itself is the same for both.
[0112]
The new loop filter design unit 138' receives the input signal, the prediction signal, and the quantized prediction error signal as an input. In a manner similar to the new post filter design unit 138' of Embodiment 1, the new loop filter design unit 138" analyzes the statistical properties of its input signals and feeds new loop filter Information to the new loop filter 137' and the entropy coding unit 190. 10113]
Thii niiw loop flltcM-137' IncUidos a first filter 137"! oporating on the predk:tloirslyi»nl, n iiecond I'lltfH 137-V, optirnilni) on Ihe (|iMinll/uil prediction error signal, and an adder 137-0 lor tuldlng the Nlterud signals. The first and the second filters 137-1 and 137-2 are set up In accordance with the new loop filter information leculvtjd from the new loop filter design unit 138" as described ahavo In cnirlMnctlon with the above described Eimbodlmonts. [OIHj
Specifically, the first filter 137-1 and the second filter 137-2 iTiciy be optimized so as to minimize a difference mejisure, such as the mean Hqunred error, between the «;ignal to be coded and the reconstructed signal, that is, the output of the new loop filter 137'. [0115]
Depending on how the filters 137-1 and 137-2 are set up, the new loop filter 137' is operating as a proper loop filter (both filters are operating on all pixels of a given block) or as a deblocking filter (both filters are predominantly operating on pixels close to a block boundary). Especially In the former case, the video coding apparatus 100 may additionally be equipped with a conventional deblocking filter (not shown) which receives input from the new loop filter 137' and feeds Its output to the memory unit 140 and the post filter design unit 138. [0116]
FIG. 8 is a block diagram of a video decoding apparatus 101 according to Embodiment 3 of the present Invention. The block diagram of FIG. 8 Is similar to that of the video decoding apparatus 101 shown in FIG. 4A, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0117]
The video decoding apparatus 101 according to Embodiment 3 of the present invention differs from the conventional video decoding apparatus and the video decoding apparatus 101 shown in FIG. 4A In having the new loop filter 137' which replaces the adder 135 and the conventional deblocking filter 137. [0118]
The configuration of the new loop filter 137', however, Is identical to the configuration of the corresponding video coding apparatus 100 shown in FIG. 7. Specifically, the new loop.filter 137' Includes a first filter 137-1 operating on the prediction signal, a second filter 137-2 operating on the quantized prediction error signal, and an adder 137-0 for adding the filtered signals. The first and the second filters 137-1 and 137-2 are set up In accordance with the new loop filter Information received from the entropy decoding unit 191 as explained In conjunction with the above-described Embodiments. As r\ df'srrlptinn ahfarly given nf Ihe vld«o rodlnfi app«r«liis 100 according lo Embodiment 3, the new loop filter 137' of the video decoding iipparaluB 101 muy \w provided dowfslrtswm ol Ihi conventional deblocking filter (not shown). [0119]
In accordance with Embodiment 3 of tho pronent Invention, an adaptive loop filter 137' Is provided which lricluc|«ii two jointly or Independently optimized flltarB opnintlng on the pradlctlon signal and the ciMaiitlzed prediction error sigiinl, respecilvely. In this manner, the loop filter 137' may take different statistical plopertles of those slgnalB Into account. Therefore, the reconstruction error can be reduced more efficiently than with a single filter operating on the reconstructed signal only. Moreover, due to the reduction of the reconstruction error, coding efficiency is Increased.

i:o:i.2o:i
(Embodiment 4)
FIG. 9 is a block diagram of a video coding apparatus 100 according to Embodiment 4 of the present invention. Tlie blocl< diagram of FIG. 9 is similar to that of the video coding apparatus 100 shown in FIG. 3A, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0121]
In Embodiment 4, the present invention is applied to adaptive interpolation filtering for motion compensated prediction. The video coding apparatus 100 according to this Embodiment differs from the conventional video coding apparatus and the video coding apparatus 100 shown in FIG. 3A in that the interpolation filter 162 is replaced by a new interpolation filter and design unit 162' and that there are two memory units 140 and 140' in order to store the prediction signal and the quantized prediction error signal separately, as well as two deblocking filters 137 and 137' in order to perform deblocking of the prediction signal and the quantized prediction error signal independently of each other. [0122]
In addition, it is noted that since the dynamic range of the quantized prediction error is usually larger than the dynamic range of the prediction signal, the memory 140' for storing the quantized prediction error may be of higher bit depth than the memory 140 for storing the prediction signal. In certain situations, It may be beneficial to decrease the dynamic range of the quantized prediction error signal in order to get a lower bit depth. This may be done by an additional quantization of the quantized prediction error signal. [0123]
The new interpolation filter and design unit 162' of FIG. 9 Includtjs a first filter 162-1 operating on the prediction signal delayed by the mennory unit 140, a second filter 162-2 operating on the quantized prwdiction orror signal (lulayed by the rnemnry unit 140',
of this optimization process is fed as now inlerpoiation filter information to the entropy coding unit 100 In iirrler In Ini liiKoiliHl inlo th
l-IG. |() Is a block diagram of a video decoding apparatus 101 according to Embodiment 4 of the present Invention. The block diagram of I'lG. 10 is similar to that of the video decoding apparatus 101 shown In FIG. 4A, wherein like components are denoted by like reference numerals. A repetition of the detailed description of these components will be omitted. [0125]
The video decoding apparatus 101 according to Embodiment 4 of the present invention differs from the conventional video decoding apparatus in that the interpolation filter 162 is replaced by a new interpolation filter 162" and that there are two memory units 140 and 140' In order to store the prediction signal and the quantized predirtion error signal separately, as well as two deblocking filters 137 and 137" in order to perform deblocking of the prediction signal and the quantized prediction error signal independently of each other. [0126]
Similar to the configuration of the corresponding video coding apparatus 100 shown in FIG. 9, the video decoding apparatus 101 is provided with a new loop filter 162", which Includes a first filter 162-1 operating on the prediction signal, a second filter 162-2 operating on the quantized prediction error signal, and an adder 162-0 for adding the filtered signals. The first and the second filters 162-1 and 162-2 are set up in accordance with the new Interpolation filter information received from the entropy decoding unit 191 as described above In conjunction with the previous Embodiments. [0127]

In accordance with Embodiment 4 of the present invention, an adaptive interpolation filter for motion compensated prediction at sub-pel precision is provided. The inventive loop filter 162" includes two jointly or independently optimized filters operating on the (delayed) prediction signal and the (delayed) quantized prediction error signal, respectively. In this manner, the loop filter 162" may take different statistical properties of these signals Into account. Therefore, the prediction error can be reduced more efficiently than with a conventional adaptive interpolation filter 162 operating on the (delayed) decoded signal only. Hence, coding efficiency can be Increased accordingly. [(.)12H1
(ICi. I IA find I IB show rat«i dlslortlon riirvoti for two dlffmenl MPEG test BtuiuoiicnH, llluHlriitlnii IJui «dv«nlayo(iub ni'fmA cil I lid present Invention. The rate distortion ciirvun tiiti l)uaeJ on Embodiment 1, having an additional post filter with MwN«L=l. In the diagrams of FIG. IIA and 1113, the Y-PSNR of the luminance component Is plotted versus the required bll ruin for two rllffiiuent IIPEG test sequences. Thf' Inviinted schenle Is compared tu H.264/AVC using a non-separable Iwo-dlmonHloiial Wiener post filter with SxS coefficients. Intra-frame coding Is applied for comparison. An IncreHRci of the Y-PSNR of up to n.l dR ran hn observed with this Invention. 10129]
In the above Embodiments of the present Inventldn, the Inventive video filter Includes filters operating on the prediction signal and the quantized prediction error signal, respectively. However, each pair of signal selected out of the set of the prediction signal, the error signal, and the reconstructed signal (the sum of the prediction signal and the error signal) Is linearly independent. Therefore, the present invention may also be applied to a combination of the reconstructed signal and the quantized prediction error signal or a combination of the reconstructed signal and the prediction signal. [0130]
Further, the present Invention is not restricted to based on whether a filter is fixed or adaptive. Any filter of the invented filter scheme may contain either fixed or adaptive filters. In the case of adaptive filters, filter information such as coefficients Is coded and sent to the video decoding apparatus as side information. In the context of the H.264/AVC coding scheme, the filter information such as coefficients is transmitted in the slice header, in the picture parameter set, or in the sequence parameter set. To perform the adaptive post filter schemes, the post filter Information can also be sent as an SEI (Supplemental Enhancement Information) to the video decoding apparatus. [0131]
It is also to be noted that the present Invention can be applied to any conventional coding scheme based on differential pulse code modulation, including coding schemes for video and non-video that Is audio data. Moreover, the present Invention may also be applied to the spatial scalable video coding scheme of H.264/AVC, In which reconstructed images of a low spatial resolution, consisting of a prediction signal and a quantized prediction error signal, are upsampled and interpolated with one filter resulting in images of a high spatial resolution. In the context of spatial scalable video coding, the prediction signal and the quantized prediction error signal could he considered separately, for example, by dlfffront Plltpr coefficients for tuich signal. It Is also possible to npply the present Invention In the contttxt of high frequency modeling. In this context, thu priiHent Invtnitlon offers In UHH the staHdtlca of Ihw predlclluii signal and of the quantized prediction error slyuwl liKllvldiifllly for tf)o filtering and for the adjustment of the higher order statistics. [0132]
Further, the present Invention Is not ruiiUictflll ta Wlisn«r filters, but may also be applied to other wdaptlve flllwrii slich as non-lliiBar adaptIve) flltere baaed on a Volturia series oxparnslon, filters that optlrnl/0 a measure other than the mean squared cirror between the corrupted signal and the desired signal, and so on. [0133]
Finally, the various Embodiments of the present Invention may
be toinliinecl, such that, for Instance, two lnck|)endently opthnl/ed (illeis are employed for post filtering in accordance with Embodiment 1 while other two Independently optimized filters are employed as a loop filter in accordance with Embodiment 3. Similar combinations of the other Embodiments are also feasible. [0134]
Summarizing, the present Invention relates to optimum filters in the context of video coding based on differential pulse code modulation. It is the particular approach of the present Invention to provide two filters that operate Independently of each other on the prediction signal and the quantized prediction error signal, rather than a single filter operating on the reconstructed signal that Is the sum of the prediction signal and the error signal. In this manner, different statistical properties of the prediction signal and the error signal can be taken into account, resulting in increased coding efficiency due to lower prediction errors and/or reconstruction errors.
(Other Variations) [0135]
The present invention has been described based on the above Embodiments 1 to 4, but It is to be noted that the present Invention Is not limited to these Embodiments 1 to 4 as a matter of course. The following are also included in the scope of the present Invention. [0136]
Specifically, each of the above-described apparatuses Is a computer system Including a micro processor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, and a mouse. A computer program is stored in the RAM or a hard disk unit. Each of the apparatuses exerts its function by operating according to the computer program. Here, the computer program is configured with combined instruction codes indicating Instructions to the computer in order to achieve predetermined functions. [0137]
Some or all of the structural element! which make up each of the app«rfltuses may be configurwd In form of a single system LSI operating according to tlie computer program. [0138]
Some or all of the structural elemunlH which rnnko up tinch of th«i! apparatuses may be configured In form of «n IC card or a modulti which l!i i:iiliirhablu/{l(:U:achable io/frnm each of tho apparcituaea, I lut IC ctiiril or module Is a computer system configured to Include a mlcrciprocesBor, a ROM, and a RAM. The IC card or module may Include the super multi-functional LSI, The IC card or module achieves Its functions by operating according to the computer program. This IC card or module may be tamper-resistant. [0139]
The present invention may be implemented as the methods. In addition, each of these methods may be a compute program causing a computer to execute the method, and may be a digital signal representing the computer program. [0140]
In addition, the present Invention may be implemented as computer programs or digital signals recorded on computer-readable recording media such as flexible discs, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, BDs (Blu-ray Discs), and semiconductor memories. In addition, the present invention may be the digital signals recorded on these recording media. [0141]
In addition, the present Invention may be intended to transmit the computer programs and digital signals via electrical communication lines, wireless or wired communication lines, networks represented by the Internet, data broadcasting or the like. [0142]
In addition, the present invention may be Implemented as a computer system including a memory which stores the computer program and a micro processor which operates according to the computer program.
[0143]
In addition, the program or digitai signal may be recorded on the recording media and then transferred, or may be transferred via the networl< or the iil
It is aiso good to combine the above-described Embodiments and Variations. [0145]
The Embodiments of the present Invention have been described with reference to the drfiwings, but llie prusuni Invention Is not ilmitod in the lliuslnited EmbodlmiMils. It should he appreclnled tlitit varlouB inodll'Icalions and varlullonr. iire iiouHlbln In Ihti llliiHlrtiltMl Embodiments within the same oi equivalent in:o\\i iit Hio piuHoiil invention.
[Industrial Applicability] [0146]
The present Invention Is njlvantageoualy alipllcablf! to vld«o codini;! mnlhods (apparatuses) and video donodlng mel-hnds (apparatuaea).
[Reference Signs List] [0147]
100 Video coding apparatus
101 Video decoding apparatus 110 Subtracter
120 Transform and quantization unit
121, 130 Inverse quantization and inverse transform unit
135, 135', 137-0, 139-0, 162-0 Adder
137,137' Deblocking filter
137', 162" Loop filter
137-1, 137-2, 139-1, 139-2, 139-3, 162-1, 162-2 Filter
138, 138' Post filter design unit 138" Loop filter design unit
139, 139' Post filter
140, 140' Memory
150 Intra-frame prediction unit
160 lotion compensated prediction unit
162 Interpolation filter
162' Loop filter and design unit
170 Motion estimation unit
180 Intra/Inter switch
190 Entropy coding unit
191 Entropy decoding unit
200, 200' Prediction signal generation unit

CLAIMS :

[Claim 1]
A video coding method of coding a signal to be coded which represents a video, said video coding method comprising:
generating a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded;
quantizing a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients;
inversely quantizing the quantized coefficients to generate a quantized prediction error signal;
generating first filter information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generating second filter information, based on statistical properties of only the quantized prediction error signal among the prediction signal and the quantized prediction error signal; and
performing entropy coding on the quantized coefficients generated in said quantizing and the first and second filter information generated in said generating of filter information so as to generate a coded signal.

[Claim 2]
The video coding method according to Claim 1, wherein, in said generating of filter information, the first and second filter information are generated so as to minimize a difference between (I) the signal to be coded, and (II) a reconstructed signal obtained by adding (Il-i) a filtered prediction signal obtained by filtering the prediction signal based on the first filter Information, and (Il-ii) a filtered quantized prediction error signal obtained by filtering the quantized prediction error signal based on the second filter Information.

[Claim 3]
The video coding method according to Claim 1, wherein, said generating of a prediction signal includes: filtering the prediction signal based on the first filter information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating a reconstructed signal by adding the filtered prediction signal and the filtered quantized prediction error signal; and generating a second prediction signal predictive of a signal to be coded subsequent to the coding of the signal to be coded, based on the reconstructed signal.

[Claim 4]
The video coding method according to Claim 3, wherein, in said filtering, deblocking of the reconstructed signal is performed so as to reduce block distortion.

[Claim 5]
The video coding method according to Claim 3, wherein, in said filtering, interpolation filtering is performed prior to motion compensated prediction.

[Claim 6]
The video coding method according to Claim 1, wherein: In said generating of filter Information, third filter Information Is generated based on statistical properties of a reconstructed signal obtained by adding the prediction signal and the quantized prediction error signal; and in said performing of entropy coding, entropy coding Is performed on the quantized prediction error signal, and the first, second, and third filter information so as to generate the coded signal.

[Claim 7]
A video decoding method of generating a decoded signal by decoding a coded signal which represents a video, said video decoding method comprising:
performing entropy decoding on the coded signal to obtain quantized coefficients and first and second filter information; inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating a prediction signal predictive of the decoded signal, based on a decoded signal decoded prior to the decoding of the coded signal; and filtering the prediction signal based on the first filter information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating the decoded signal by adding the filtered prediction signal and the filtered quantized prediction error signal.

[Claim 8]
The video decoding method according to Claim 7,
Wherein, In said generating of a prediction signal Includes;
filtering the prediction signal baaed on the first filter Information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating a reconstructed signal by adding the filtered prediction signal and the filtered quantized prediction error signal; and generating a second prediction signal predictive of a signal to be decoded subsequent to the decoding of the coded signal, based on the reconstructed signal.

[Claim 9]
A video coding apparatus which codes a signal to be coded which represents a video, said video coding apparatus comprising:
a prediction signal generation unit configured to generate a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded; a quantization unit configured to quantize a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients; an Inverse quantization unit configured to Inversely quantize the quantized coefficients to generate a quantized prediction error signal;
a filter information generation unit configured to generate first filter information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generate second filter Information, based on statistical properties of only the quantized prediction error signal among the prediction signal and the quantized prediction error signal; and an entropy coding unit configured to perform entropy coding on the quantized coefficients generated by said quantization unit and the first and second filter information generated by said filter Information generation unit so as to generate a coded signal.

[Claim 10]
A video decoding apparatus which generates a decoded signal by decoding a coded signal which represents a video, said video decoding apparatus comprising:
an entropy decoding unit configured to perform entropy decoding on the coded signal to obtain quantized coefficients and first and second filter Information;
an Inverse quantization unit configured to Inversely quantize the quantized coefficients to generate a quantized prediction error signal;
a prediction signal generation unit configured to generate a prediction signal predictive of the decoded signal, bused on a decoded ' signal decoded prior to the decoding of the coded signal ; and a filtering unit configured to filter the prediction signal based on the first filter information to generate a filtered prediction signal, to filter the quantized prediction error signal based oil the second filter information lo generate a filtered quantized prediction error signal, and generate the decoded signal by adding the filtered prediction signal and the filtered quantized prediction error signal.

[Claim 11]
A program causing a computer to code a signal to be coded which represents a video, said program comprising: generating a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded;
quantizing a prediction error obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients; inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating first filter information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generating second filter information, based on statistical properties of only the quantized prediction error signal among the prediction signal and the quantized prediction error signal; and performing entropy coding on the quantized coefficients generated in said quantizing and the first and second filter information generated in said generating of filter information so as to generate a coded signal.

[Claim 12]
A program causing a computer to decode a coded signal which represents a video, said program comprising; performing entropy decoding on the coded signal to obtain
quantized coefficients and first and second filter information; inversely quantizing the quantized coefficients to generate a quantized prediction error signal; generating a prediction signal predictive of the decoded signal, based on a decoded signal decoded prior to the decoding of the coded signal; and filtering the prediction signal based on the first filter information to generate a filtered prediction signal, filtering the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generating the decoded signal by adding the filtered prediction signal and filtered quantized prediction error signal.

[Claim 13|
An integrated circuit which codes a signal the be coded which represents a video, said Integrated circuit comprising;
a prediction signal generation unit configured to generate a prediction signal predictive of the signal to be coded, based on a coded signal coded prior to the coding of the signal to be coded ; a quantization unit configured to quantization prediction obtained by subtracting the prediction signal from the signal to be coded to generate quantized coefficients;
an inverse quantization unit configured to Inversely quantize the quantized coefficients to generate a quantized prediction error signal;
a filter Information generation unit configured to generate first filter Information, based on statistical properties of only the prediction signal among the prediction signal and the quantized prediction error signal, and generate second filter Information, based on statistical properties of only the quantized prediction error signal among the prediction signal and the quantized prediction error signal; and
an entropy coding unit configured to perform entropy coding on the quantized coefficients generated by said quantization unit and the first and second fitter information generated by said filter Information generation unit so as to generate a coded signal.

[Claim 14]
An Integrated circuit which generates a decoded signal by decoding a coded signal which represents a video, said Integrated circuit comprising: an entropy decoding unit configured to perform entropy decoding on the coded signal to obtain quantized coefficients and first and second filter information; an inverse quantization unit configured to inversely quantize the quantized coefficients to generate a quantized prediction error signal; a prediction signal generation unit configured to generate a prediction signal predictive of the decoded signal, based on a decoded signal decoded prior to the decoding of the coded signal; and a filtering unit configured to filter the prediction signal based on the first filter information to generate a filtered prediction signal, to filter the quantized prediction error signal based on the second filter information to generate a filtered quantized prediction error signal, and generate the decoded signal by adding the filtered prediction signal and the filtered quantized prediction error signal.