Sign In to Follow Application
View All Documents & Correspondence

Coding Using Matrix Based Intra Prediction And Secondary Transforms

Abstract: There are disclosed techniques for matrix-based predicting a block of a picture efficiently. An embodiment relates to an apparatus for decoding a predetermined block (18) of a picture using intra-prediction, configured to select (602), based on the data stream, a predetermined intra prediction mode (604) out of a plurality (600) of intra-prediction modes which comprises a first set (508) of intra-prediction modes comprising a DC intra prediction mode (506) and angular prediction modes (500), and a second set (520) of matrix-based intra-prediction modes (510) according to each of which a matrix-vector product (512) between a vector (514) derived from reference samples (17) in a neighbourhood of the predetermined block and a prediction matrix (516) associated with the respective matrix-based intra-prediction mode is used to obtain a prediction vector (518), on the basis of which samples of the predetermined block are predicted. The apparatus is configured to derive a prediction signal (606) for the predetermined block using the predetermined intra-prediction mode and select (608) a subset (610) of one or more secondary transforms out of a set (612) of secondary transforms in a manner dependent on the predetermined intra prediction mode so that the subset (610) is nonempty in case of the predetermined intra prediction mode being contained in the first set (508) of intra-prediction modes and the predetermined intra prediction mode being contained in the second set (520) of matrix-based intra-prediction modes (510). Additionally the apparatus is configured to derive (614), from the data stream, a transformed version (616) of a prediction residual for the predetermined block (18), which is related to a spatial domain version (618) of the prediction residual of the predetermined block via a transform (T) defined by a concatenation of a primary transform (Tp) and a predetermined secondary transform (Ts) out of the subset (610) of secondary transforms applied onto a subset (622) of coefficients (620) of the primary transform, in case of the predetermined intra prediction mode being contained in the first set (508) of intraprediction modes and in case of the predetermined intra-prediction mode being contained in the second set (520) of matrix-based intra-prediction modes (510). The apparatus is configured to reconstruct (624) the predetermined block using the prediction signal and the prediction residual for the predetermined block (18).

Get Free WhatsApp Updates!
Notices, Deadlines & Correspondence

Patent Information

Application #
Filing Date
20 September 2024
Publication Number
40/2024
Publication Type
INA
Invention Field
COMMUNICATION
Status
Email
Parent Application

Applicants

Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V.
Hansastraße 27c, 80686 München (DE)

Inventors

1. PFAFF, Jonathan
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
2. HINZ, Tobias
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
3. HELLE, Philipp
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
4. MERKLE, Philipp
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
5. STALLENBERGER, Björn
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
6. SCHÄFER, Michael
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
7. BROSS, Benjamin
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
8. WINKEN, Martin
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
9. SIEKMANN, Mischa
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
10. SCHWARZ, Heiko
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
11. MARPE, Detlev
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)
12. WIEGAND, Thomas
c/o Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI, Einsteinufer 37, 10587 Berlin (DE)

Specification

Description:As attached. , Claims:1. A method for decoding a picture from a data stream, the method comprising:
for a block of the picture, selecting, an intra-prediction mode from a first set of intra-prediction modes or a second set of intra-prediction modes based on an indication included in the data stream, wherein the first set of intra-prediction modes comprises at least one angular prediction mode, a planar intra-prediction mode, and a DC intra-prediction mode, and wherein the second set of intra-prediction modes comprises at least one matrix based intra-prediction mode;
deriving a prediction signal for the block using the selected intra-prediction mode;
selecting a subset of secondary transforms out of a set of secondary transforms including multiple low-frequency non-separable secondary transforms (LFNST) based on the selected intra-prediction mode, wherein the selected subset of secondary transforms includes at least one of the multiple LFNST;
deriving, from the data stream, a prediction residual for the block;
transforming the prediction residual using a LFNST from the selected subset; and
reconstructing the block using the prediction signal and the transformed prediction residual for the block,
wherein the same subset of secondary transforms is selected for the planar intra-prediction mode and each of the at least one matrix based intra-prediction mode.

2. The method as claimed in Claim 1, wherein deriving the prediction signal when the intra-prediction mode is selected from the second set of intra-prediction modes, comprises:
deriving a prediction vector generated from a matrix-vector product between a vector derived from reference samples in a neighborhood of the block and a prediction matrix associated with the selected intra-prediction mode, and
upsampling the prediction vector to obtain the prediction signal of the block.

3. The method as claimed in Claim 1, wherein selecting the subset of secondary transforms is based on:
the selected intra-prediction mode, and
a size of the block.

4. The method as claimed in Claim 1, further comprising:
decoding an indication indicating a LFNST from the selected subset of secondary transforms; and
selecting the LFNST, for transforming the prediction residual, from the subset based on the indication specifying one of two LFNST.

5. The method as claimed in Claim 1, wherein transforming the prediction residual comprises:
applying the LFNST (Ts) onto a subset of coefficients of a primary transform (Tp) to obtain a transform; and
transforming the prediction residual using the transform.

6. The method as claimed in Claim 5, wherein the primary transform (Tp) is a separable 2D transform.

7. The method as claimed in Claim 1, wherein selecting the intra-prediction mode for the block of the picture comprises:
decoding a set of syntax elements from the data stream that indicate whether the block is to be predicted using one of the first set of intra-prediction modes;
in response to a determination that the set of syntax element indicates that the block is to be predicted using one of the first set of intra-prediction modes, generating a list of most probable intra-prediction modes (MPM) based on intra-prediction modes used in blocks neighboring the block, and selecting the intra-prediction mode from the list; and
in response to a determination that the set of syntax element indicates that the block is to be predicted using one of the second set of intra-prediction modes, selecting one of the at least one matrix based intra-prediction mode from the second set of intra-prediction modes.

8. The method as claimed in Claim 1, wherein transforming the prediction residual, comprises:
using a transform defined by a concatenation of a primary transfer (Tp) and a secondary transform (Ts) selected from the subset of secondary transforms, the secondary transform (Ts) corresponds to the LFNST.
9. The method as claimed in Claim 1, the method comprises:
after selecting the intra-prediction mode from the second set of intra-prediction modes, selecting a matrix based intra-prediction mode from the at least one matrix based intra-prediction mode for deriving the prediction signal;

10. The method as claimed in Claim 9, wherein deriving the prediction signal using the matrix based intra-prediction mode comprises:
generating a prediction vector from a matrix-vector product between a vector derived from reference samples in a neighborhood of the block and a prediction matrix associated with the selected matrix-based intra-prediction mode, according to which samples of the block are predicted.

11. An apparatus for decoding a picture from a data stream, the apparatus comprising:
a processor configured to:
for a block of the picture, select, an intra-prediction mode from a first set of intra-prediction modes or a second set of intra-prediction modes based on an indication included in the data stream, wherein the first set of intra-prediction modes comprises at least one angular prediction mode, a planar intra-prediction mode, and a DC intra-prediction mode , and wherein the second set of intra-prediction modes comprises at least one matrix based intra-prediction mode;
derive a prediction signal for the block using the selected intra-prediction mode;
select a subset of secondary transforms out of a set of secondary transforms including multiple low-frequency non-separable secondary transforms (LFNST) based on the selected intra-prediction mode, wherein the selected subset of secondary transforms includes at least one of the multiple LFNST;
derive, from the data stream, a prediction residual for the block;
transform the prediction residual using a LFNST from the selected subset; and
reconstruct the block using the prediction signal and the transformed prediction residual for the block,
wherein the same subset of secondary transforms is selected for the planar intra-prediction mode and each of the at least one matrix based intra-prediction mode.

12. The apparatus as claimed in Claim 11, wherein to derive the prediction signal when the intra-prediction mode is selected from the second set of intra-prediction modes, deriving the prediction signal, the processor is configured to:
derive a prediction vector generated from a matrix-vector product between a vector derived from reference samples in a neighborhood of the block and a prediction matrix associated with the selected intra-prediction mode, and
up sample the prediction vector to obtain the prediction signal of the block.

13. The apparatus as claimed in Claim 11, wherein the processor is further configured to to select the subset of secondary transforms based on:
the selected intra-prediction mode, and
a size of the block.

14. The apparatus as claimed in Claim 11, wherein the processor is further configured to:
decode an indication indicating a LFNST from the selected subset of secondary transforms; and
select the LFNST, for transforming the prediction residual, from the subset based on the indication specifying one of two LFNST.

15. The apparatus as claimed in Claim 11, wherein to transform the prediction residual, the processor is configured to:
apply the LFNST (Ts) onto a subset of coefficients of a primary transform (Tp) to obtain a transform; and
transform the prediction residual using the transform.

16. The apparatus as claimed in Claim 15, wherein the primary transform (Tp) is a separable 2D transform.

17. The apparatus as claimed in Claim 11, wherein to select the intra-prediction mode for the block of the picture, the processor is configured to:
decode a set of syntax elements from the data stream that indicate whether the block is to be predicted using one of the first set of intra-prediction modes;
in response to a determination that the set of syntax element indicates that the block is to be predicted using one of the first set of intra-prediction modes, generate a list of most probable intra-prediction modes (MPM) based on intra-prediction modes used in blocks neighboring the block, and select the intra-prediction mode from the list; and
in response to a determination that the set of syntax element indicates that the block is to be predicted using one of the second set of intra-prediction modes, select one of the at least one matrix based intra-prediction mode from the second set of intra-prediction modes.

18. The apparatus as claimed in Claim 11, wherein to transform the prediction residual, the processor is configured to:
use a transform defined by a concatenation of a primary transfer (Tp) and a secondary transform (Ts) selected from the subset of secondary transforms, the secondary transform (Ts) corresponds to the LFNST.

19. The apparatus as claimed in Claim 11, wherein after selecting the intra-prediction mode from the second set of intra-prediction modes, the processor is configured to:
select a matrix based intra-prediction mode from the at least one matrix based intra-prediction mode for deriving the prediction signal.

20. The apparatus as claimed in Claim 19, wherein to derive the prediction signal using the matrix based intra-prediction mode the processor is configured to:
generate a prediction vector from a matrix-vector product between a vector derived from reference samples in a neighborhood of the block and a prediction matrix associated with the selected matrix-based intra-prediction mode, according to which samples of the block are predicted.

21. A method for encoding a picture from a data stream, the method comprising:
generating, for a block of the picture, an indication of an intra-prediction mode selected from a first set of intra-prediction modes or a second set of intra-prediction modes, wherein the first set of intra-prediction modes comprises at least one angular prediction mode, a planar intra-prediction mode, and a DC intra-prediction mode, and wherein the second set of intra-prediction modes comprises at least one matrix based intra-prediction mode;
encoding a prediction signal for the block using the selected intra-prediction mode;
selecting a subset of secondary transforms out of a set of secondary transforms including multiple low-frequency non-separable secondary transforms (LFNST) based on the selected intra-prediction mode, wherein the selected subset of secondary transforms includes at least one of the multiple LFNST;
deriving a prediction residual for the block;
transforming the prediction residual using a LFNST from the selected subset; and
encoding into the data stream, the transformed prediction residual for the block;
wherein the same subset of secondary transforms is selected for the planar intra-prediction mode and each of the at least one matrix based intra-prediction mode.

22. The method as claimed in Claim 21, further comprising:
in response to the intra-prediction mode being selected from the second set of intra-prediction modes, deriving a prediction vector generated from a matrix-vector product between a vector derived from reference samples in a neighborhood of the block and a prediction matrix associated with the selected intra-prediction mode; and
generating the prediction signal based on the prediction vector.

23. The method as claimed in Claim 21, further comprising:
selecting the subset of secondary transforms is based on:
the selected intra-prediction mode, and
a size of the block.

24. An apparatus for encoding a picture from a data stream, the apparatus comprising:
a processor configured to:
generate, for a block of the picture, an indication of an intra-prediction mode selected from a first set of intra-prediction modes or a second set of intra-prediction modes, wherein the first set of intra-prediction modes comprises at least one angular prediction mode, a planar intra-prediction mode, and a DC intra-prediction mode, and wherein the second set of intra-prediction modes comprises at least one matrix based intra-prediction mode;
encode a prediction signal for the block using the selected intra-prediction mode;
select a subset of secondary transforms out of a set of secondary transforms including multiple low-frequency non-separable secondary transforms (LFNST) based on the selected intra-prediction mode, wherein the selected subset of secondary transforms includes at least one of the multiple LFNST;
derive a prediction residual for the block;
transform the prediction residual using a LFNST from the selected subset; and
encode into the data stream, the transformed prediction residual for the block;
wherein the same subset of secondary transforms is selected for the planar intra-prediction mode and each of the at least one matrix based intra-prediction mode.

25. The apparatus as claimed in Claim 24, wherein in response to the intra-prediction mode being selected from the second set of intra-prediction modes, the processor is configured to:
derive a prediction vector generated from a matrix-vector product between a vector derived from reference samples in a neighborhood of the block and a prediction matrix associated with the selected intra-prediction mode; and
generate the prediction signal based on the prediction vector.

26. The apparatus as claimed in Claim 24, wherein the processor is configured to select the subset of secondary transforms is based on:
the selected intra-prediction mode, and
a size of the block.

Documents

Application Documents

# Name Date
1 202438071209-TRANSLATIOIN OF PRIOIRTY DOCUMENTS ETC. [20-09-2024(online)].pdf 2024-09-20
2 202438071209-STATEMENT OF UNDERTAKING (FORM 3) [20-09-2024(online)].pdf 2024-09-20
3 202438071209-REQUEST FOR EXAMINATION (FORM-18) [20-09-2024(online)].pdf 2024-09-20
4 202438071209-PROOF OF RIGHT [20-09-2024(online)].pdf 2024-09-20
5 202438071209-POWER OF AUTHORITY [20-09-2024(online)].pdf 2024-09-20
6 202438071209-FORM 18 [20-09-2024(online)].pdf 2024-09-20
7 202438071209-FORM 1 [20-09-2024(online)].pdf 2024-09-20
8 202438071209-DRAWINGS [20-09-2024(online)].pdf 2024-09-20
9 202438071209-DECLARATION OF INVENTORSHIP (FORM 5) [20-09-2024(online)].pdf 2024-09-20
10 202438071209-COMPLETE SPECIFICATION [20-09-2024(online)].pdf 2024-09-20
11 202438071209-FORM 3 [04-03-2025(online)].pdf 2025-03-04