Video Data Stream, Video Encoder, Apparatus and Methods for HRD Timing Fixes, and Further Additions for Scalable and Mergeable Bitstreams

Description:

The present invention relates to video encoding and video decoding and, in particular, to a video encoder, to a video decoder, to methods for encoding and decoding and to a video data stream for realizing advanced video coding concepts.

H.265/HEVC (HEVC = High Efficiency Video Coding) is a video codec which already provides tools for elevating or even enabling parallel processing at an encoder and/or at a decoder. For example, HEVC supports a sub-division of pictures into an array of tiles which are encoded independently from each other. Another concept supported by HEVC pertains to WPP, according to which CTU-rows or CTU-lines of the pictures may be processed in parallel from left to right, eg in stripes, provided that some minimum CTU offset is obeyed in the processing of consecutive CTU lines (CTU = coding tree unit). It would be favorable, however, to have a video codec at hand which supports parallel processing capabilities of video encoders and/or video decoders even more efficiently.

Typically, in video coding, a coding process of picture samples requires smaller partitions, where samples are divided into some rectangular areas for joint processing such as prediction or transform coding. Therefore, a picture is partitioned into blocks of a particular size that is constant during encoding of the video sequence. In H.264/AVC standard fixed-size blocks of 16x16 samples, so called macroblocks, are used (AVC = Advanced Video Coding).

In the state-of-the-art HEVC standard (see [1]), there are Coded Tree Blocks (CTB) or Coding Tree Units (CTU) of a maximum size of 64 x 64 samples. In the further description of HEVC, for such a kind of blocks, the more common term CTU is used.

CTUs are processed in raster scan order, starting with the top-left CTU, processing CTUs in the picture line-wise, down to the bottom-right CTU.

The coded CTU data is organized into a kind of container called slice. Originally, in former video coding standards, slice means a segment comprising one or more consecutive CTUs of a picture. Slices are employed for a segmentation of coded data. From another point of view, the complete picture can also be defined as one big segment and hence, historically, the term slice is still applied. Besides the coded picture samples, slices also include additional information related to the coding process of the slice itself which is placed into a so-called slice header.

According to the state-of-the-art, a VCL (video coding layer) also includes techniques for fragmentation and spatial partitioning. Such partitioning may, eg, be applied in video coding for various reasons, among which are processing load-balancing in parallelization, CTU size matching in network transmission, error-mitigation etc.

A bitstream as specified in video coding standards has HRD conformance associated information. This conformance consists of a hypothetical reference decoder (HRD) including a buffer model that assumes that NAL units enter a Coded Picture Buffer (CPB) before the decoder and are removed therefrom at a particular time ensuring that the CPB size is not exceeded (buffer overrun ) or that the NAL units do not arrive later than when they need to be removed (buffer underrun). Further, the model consists of a Decoded Picture Buffer (DPB) out of which decoded pictures are output when they are no longer needed for prediction and whose size is likewise constrained in many implementations. The timing information for the HRD is conveyed in the bitstream through so-called SEI messages, in particular,

A bitstream as specified in video coding standards has HRD (Hypothetical Reference Decoder) conformance associated information. This conformance consists of a hypothetical buffer model that assumes that NAL units enter a Coded Picture Buffer (CPB) and are removed therefrom at a particular time ensuring that the CPB size is not exceeded (buffer overrun) or that the NAL units do not arrive later than when they need to be removed (buffer underrun).

When a bitstream is a scalable bitstream, pruning can be carried out to obtain a sub bitstream that is also a conforming bitstream. For instance, when there is an Output Layer Set (OLS) containing 3 layers with resolution scalability (eg, 480p base layer, 720p 1 st enhancement layer and 1080p 2 nd enhancement layer) referred to as B 3 hereafter, two sub-

bitstreams can be obtained: one with two layers (480p and 720p) B2 and another sub bitstream with one layer (480p) Bi. Likewise, OLS can be used for temporal scalability, where B 3 , B 2 and Bi have the same resolution but different frame rates.

Obviously such bitstreams B 3 , B 2 and Bi have different HRD conformance as their required CPB size, Bitrate and timing information can be different.

Different CPB sizes and Bitrates are indicated in the VPS as a characteristic of the defined output layer sets (3 in the described example). Different timing information are provided by so-called Nesting SEI messages. Nesting SEI messages can contain nested Buffering Period SEIs and Picture Timing SEIs that apply to a sub-bitstream that can be obtained by bitstream pruning (bitstream extraction). Then when this operation (extraction or pruning is carried out), the Buffering Period SEI message and Picture Timing SEI message of eg the input bitstream B3 are removed from the bitstream and also the NAL units belonging to the 2 ndenhancement layer. Besides, the Buffering Period SEI message and Picture Timing SEI message corresponding to the bitstream B2 that are carried in the Nesting SEI message are placed at the bitstream out of the nesting SEI message substituting thus the removed ones.

The object of the present invention is to provide improved concepts for video encoding and video decoding.

The object of the present invention is solved by the subject-matter of the independent claims.

Preferred embodiments are provided in the dependent claims.

According to an embodiment, a video data stream having a video encoded thereinto is provided. The video data stream comprises an indication that indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream comprehends an indication that indicates whether or not one or more scalable nesting

supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

Furthermore, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The apparatus is to process the input bitstream to obtain a sub-bitstream. An indication indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

Moreover, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised an indication that indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream .

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method includes processing the input bitstream to obtain a sub-bitstream. The indication indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

Moreover, computer programs for implementing one of the above-described methods when being executed on a computer or signal processor are provided.

Moreover, according to an embodiment, a video data stream having a video encoded thereinto, is provided. An indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video data stream.

Furthermore, according to an embodiment, a video data stream having a video encoded thereinto, is provided. A first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from the one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream. And/or, a second indication within the video data stream indicates whether or not

the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

Moreover, according to an embodiment, a video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information, is provided. If the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, this indicates that depending on the scalable nesting supplemental enhancement information message: All of the one or more non-scalable-nested timing information supplemental enhancement information messages are to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information (eg, the timing information is picture timing information or buffering period information or decoding unit information).

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video data stream.

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. A first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from the one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream. And/or, a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is

provided. The video encoder is to generate the video data stream such that the video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information. If the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, this indicates that depending on the scalable nesting supplemental enhancement information message: All of the one or more non-scalable-nested timing information supplemental enhancement information messages are to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information (eg, the timing information is picture timing information or buffering period information or decoding unit information). Gold:

Moreover, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The apparatus is to process the input bitstream to obtain a sub-bitstream. An indication within the video data stream indicates whether or not timing information for a sub bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video data stream.

Furthermore, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The apparatus is to process the input bitstream to obtain a sub-bitstream. A first indication within the video data stream indicates whether or not timing information for a sub bitstream is to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream. And/or, a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

Moreover, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The apparatus is to process the input bitstream to obtain a sub-bitstream. The video data stream comprises one or more non-scalable-nested supplemental enhancement information messages comprising timing information. If the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, the apparatus is to substitute, depending on the scalable nesting supplemental enhancement information message: All of the one or more non-scalable-nested timing information supplemental enhancement information messages by the scalable nesting supplemental enhancement information message comprising the timing information (eg , the timing information is picture timing information or buffering period information or decoding unit information). Or, a subset comprising at least one of the one or more non-scalable-nested timing information supplemental enhancement information messages by the scalable nesting supplemental enhancement information message comprising the timing information (eg, the timing information is the picture timing information or the buffering period information or the decoding unit information).

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprises generating the video data stream such that an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video datastream.

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method includes processing the input bitstream to obtain a sub-bitstream. An indication within the video data stream indicates whether or not timing information for a sub bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video data stream.

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. A first indication within the video data stream indicates whether or not timing information for a sub bitstream is to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream. And/or, a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method includes processing the input bitstream to obtain a sub-bitstream. First

indication within the video data stream indicates whether or not timing information for a sub bitstream is to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream. And/or, a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information. If the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, this indicates that at least one of the one or more non-scalable-nested picture timing supplemental enhancement information messages is to be substituted by the scalable nesting supplemental enhancement information information message comprising the timing information.

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method includes processing the input bitstream to obtain a sub-bitstream. The video data stream comprises one or more non-scalable-nested supplemental enhancement information messages comprising timing information. If the video data stream comprised a scalable nesting supplemental enhancement information message comprising the timing information, the method comprised substituting at least one of the one or more non-scalable-nested picture timing supplemental enhancement information messages by the scalable nesting supplemental enhancement information message comprising the timing information.

Moreover, computer programs for implementing one of the above-described methods when being executed on a computer or signal processor are provided.

Moreover, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Furthermore, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the video data stream comprehends an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Moreover, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprehends three or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

Furthermore, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is

provided. The video encoder is to generate the video data stream such that the video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the video encoder is to generate the video data stream such that the video data stream included an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream includes a majority of access units. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Furthermore, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The input bitstream includes a majority of access units. The apparatus is to process the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Moreover, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The input bitstream includes a majority of access units. The apparatus is to process the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units. If the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set,

Furthermore, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The input bitstream comprises a plurality of access units, the apparatus is to process the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Moreover, according to an embodiment, an apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is

provided. The input bitstream includes a majority of access units. The apparatus is to process the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Moreover, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised a plurality of access units. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the method comprehends generating the video data stream such that the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised a plurality of access units. For each access unit of the plurality of access units, if the access unit comprises two or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Moreover, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method included generating the video data stream such that the video data stream

included in a majority of access units. For each access unit of the plurality of access units, if the access unit comprehends three or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the method comprehends generating the video data stream such that the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised a plurality of access units. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The input bitstream includes a majority of access units. The method includes processing the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is

provided. The input bitstream includes a majority of access units. The method includes processing the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprehends two or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the video data stream comprehends an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The input bitstream includes a majority of access units. The method includes processing the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The input bitstream includes a majority of access units. The method includes processing the access units of the input bitstream to obtain a sub-bitstream. For each access unit of the plurality of access units, if the access unit comprises three or more scalable nested supplemental enhancement information messages which comprises buffer period information and/or picture timing information for an output layer set,

Moreover, computer programs for implementing one of the above-described methods when being executed on a computer or signal processor are provided.

Furthermore, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a majority of access units. The video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream; or the video data stream comprises a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream.

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream includes a majority of access units. Furthermore, the video encoder is to generate the video data stream such that the video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream; or the video encoder is to generate the video data stream such that the video data stream comprises a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream.

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream included sub-layer specific frame rate information for a sub-layer; and/or the video encoder is to generate the video data stream such that the video data stream included sub-layer specific frame display duration information for the sub-layer.

Moreover, according to an embodiment, a video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto. The video data stream includes a majority of access units. The video decoder is to decode the video data stream to decode the video. The video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream, wherein the video decoder is to decode the video using the spread factor; or the video data stream comprises a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream, wherein the video decoder is to decode the video using the clock sub-tick value.

Furthermore, according to an embodiment, a video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The video decoder is to decode the video data stream to decode the video. The video data stream comprised sub-layer specific frame rate information for a sub-layer, and/or wherein the video data stream comprised sub-layer specific frame display duration information for the sub-layer. The decoder is to determine a spread factor using the sub layer specific frame rate information for the sub-layer and/or using the sub-layer specific frame display duration information.

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised a plurality of access units. Moreover, the method comprises generating the video data stream such that the video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream; or the method comprehending generating the video data stream such that the video data stream comprehending a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream.

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto. The video data stream includes a majority of access units. The method includes decoding the video data stream to decode the video. The video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream, wherein the method comprises decoding the video using the spread factor; or the video data stream comprised a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream, wherein the method comprised decoding the video using the clock sub-tick value.

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto. The method includes decoding the video data stream to decode the video. The video data stream comprised sub-layer specific frame rate information for a sub-layer, and/or wherein the video data stream comprised sub-layer specific frame display duration information for the sub-layer. The method includes determining a spread factor using the sub-layer specific frame rate information for the sub-layer and/or using the sub-layer specific frame display duration information.

Moreover, computer programs for implementing one of the above-described methods when being executed on a computer or signal processor are provided.

According to an embodiment, a video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The video decoder is to decode the video data stream to decode the video. To decode the video, the video decoder is to estimate a coded picture buffer size for a subpicture depending on information within the video data stream indicating a current coded picture buffer size information.

Moreover, according to an embodiment, a video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The video decoder is to decode the video data stream to decode the video. To decode the video, the video decoder is to estimate a bitrate for a subpicture depending on information within the video data stream indicating current coded video sequence bitrate information.

Furthermore, according to an embodiment, a video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The video decoder is to decode the video data stream to decode the video. To decode the video, the video decoder is to receive a coded picture buffer size for a subpicture being encoded within the video data stream, and is to use the coded picture buffer size for the subpicture to decode the video; and/or to decode the video, the video decoder is to receive a bitrate for a subpicture being encoded within the video data stream, and is to use the bitrate for the subpicture to decode the video.

Moreover, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a syntax element cpb size value minusl[ i ] [ j ] and a syntax element cpb size scale. However, the video data stream includes a syntax element bit_rate_vaiue_minusi [ 1 ] [ j ] and a syntax element bit_rate_scaie.

Furthermore, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream comprised an indication indicating whether or not a coded picture buffer size for a subpicture shall be estimated using current coded picture buffer size information, and/or the video data stream comprised an indication indicating whether or not a bitrate for a subpicture shall be estimated using current coded video sequence bitrate information.

Moreover, according to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream comprised an indication indicating whether or not a coded picture buffer size for a subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated, and/or the video data stream comprised an indication indicating whether or not a bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated.

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream comprises a syntax element cpb_size_vaiue_minusi[ i ] [ j ] and a syntax element cpb_size_scaie. However, the video encoder is to generate the video data stream such that the video data stream includes a syntax element bit rate value minusl[ i ] [ j ] and a syntax element bit rate scale.

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a coded picture buffer size for a subpicture shall be estimated using current coded picture buffer size information, and/or the video encoder is to generate the video data stream such that the video data stream includes an indication indicating whether or not a bitrate for a subpicture shall be estimated using current coded video sequence bitrate information.

Furthermore, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a coded picture buffer size for a subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated, and/or the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated .

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that, if the video data stream comprises common decoding unit removal timing information and a plurality of extractable sub-bitstreams, each of the plurality of sub-bitstreams being specific to an output layer set, each output layer set-specific hypothetical reference decoder parameter syntax structure in a video parameter set or in a sequence parameter set or in a supplemental enhancement information message of the video data stream comprised either a spread factor or an absolute value of a tick divisor for scaling the common decoding unit removal timings.

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method includes decoding the video data stream to decode the video. To decode the video, the method includes estimating a coded picture buffer size for a subpicture depending on information within the video data stream indicating a current coded picture buffer size information.

Moreover, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method includes decoding the video data stream to decode the video. To decode the video, the method includes estimating a bitrate for a subpicture depending on information within the video data stream indicating current coded video sequence bitrate information.

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The method comprises decoding the video data stream to decode the video. To decode the video, the method comprises receiving a coded picture buffer size for a subpicture being encoded within the video data stream, and comprises using the coded picture buffer size for the subpicture to decode the video; and/or to decode the video, the method comprises receiving a bitrate for a subpicture being encoded within the video data stream, and comprises using the bitrate for the subpicture to decode the video.

Moreover, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised a syntax element cpb_size_vaiue_minusi[ i ] [ j ] and a syntax element cpb_size_scaie; or, the method comprised generating the video data stream such that the video data stream comprised a syntax element bit_rate_vaiue_minusi[ i ] [ j ] and a syntax element bit_rate_scaie.

Furthermore, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method included generating the video data stream such that the video data stream

comprised an indication indicating whether or not a coded picture buffer size for a subpicture shall be estimated using current coded picture buffer size information, and/or the method comprised generating the video data stream such that the video data stream comprised an indication indicating whether or not a bitrate for a subpicture shall be estimated using current coded video sequence bitrate information.

Moreover, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised an indication indicating whether or not a coded picture buffer size for a subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated, and/or the method comprehending generating the video data stream such that the video data stream comprehending an indication indicating whether or not a bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated.

Furthermore, computer programs for implementing one of the above-described methods when being executed on a computer or signal processor are provided.

According to an embodiment, a video data stream having a video encoded thereinto, is provided. The video data stream includes a majority of access units. Furthermore, the video data stream includes delta time information for each of two or more decoding units of an access unit of the plurality of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit.

Moreover, according to an embodiment, a video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The video encoder is to generate the video data stream such that the video data stream includes a majority of access units. Furthermore, the video encoder is to generate the video data stream such that the video data stream including delta time information for each of two or more decoding units of an access unit of the majority of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit.

Furthermore, according to an embodiment, a video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The video data stream includes a majority of access units. The video decoder is to decode the video data stream to decode the video. Moreover, the video data stream includes delta time information for each of two or more decoding units of an access unit of the plurality of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit,

Moreover, according to an embodiment, a method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto, is provided. The method comprised generating the video data stream such that the video data stream comprised a plurality of access units. Moreover, the method comprises generating the video data stream such that the video data stream comprises delta time information for each of two or more decoding units of an access unit of the plurality of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit.

Furthermore, according to an embodiment, a method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto, is provided. The video data stream includes a majority of access units. The method includes decoding the video data stream to decode the video. Furthermore, the video data stream includes delta time information for each of two or more decoding units of an access unit of the plurality of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit,

Moreover, computer programs for implementing one of the above-described methods when being executed on a computer or signal processor are provided.

Fig. 1 illustrates a video encoder for encoding a video into a video data stream according to an embodiment.

Fig. 2 illustrates an apparatus for receiving an input video data stream according to an embodiment.

Fig. 3 illustrates a video decoder for receiving a video data stream having a video stored therein according to an embodiment.

Fig. 4 illustrates a change of removal times for the case that there are three decoding units per access unit.

Fig. 5 illustrates two layers and the removal times of access units and decoding units of both layers.

Fig. 6 illustrations to video encoder.

Fig. 7 illustrations to video decoder.

Fig. 8 illustrates the relationship between the reconstructed signal, eg, the reconstructed picture, on the one hand, and the combination of the prediction residual signal as signaled in the data stream, and the prediction signal, on the other hand.

The following description of the figures starts with a presentation of a description of an encoder and a decoder of a block-based predictive codec for coding pictures of a video in order to form an example for a coding framework into which embodiments of the present invention may be built in. The respective encoder and decoder are described with respect to Fig. 6 to Fig. 8. Subsequently the description of embodiments of the concept of the present invention is presented along with a description as to how such concepts could be built into the encoder and decoder of Fig. 6 and Fig. 7, respectively, although the embodiments described with FIG. 1 to Fig. 3 and following, may also be used to form encoders and decoders not operating according to the coding framework underlying the encoder and decoder of Fig. 6 and Fig. 7.

Fig. 6 shows a video encoder, an apparatus for predictively coding a picture 12 into a data stream 14 exemplarily using transform-based residual coding. The apparatus, or encoder, is indicated using reference sign 10. 7 shows a corresponding video decoder 20, eg, an apparatus 20 configured to predictively decode the picture 12' from the data stream 14 also using transform-based residual decoding, wherein the apostrophe has been used to indicate that the picture 12' as reconstructed by the decoder 20 deviates from picture 12 originally encoded by apparatus 10 in terms of coding loss introduced by a quantization of the prediction residual signal. Fig. 6 and Fig. 7 exemplarily use transform based prediction residual coding, although embodiments of the present application are not restricted to this kind of prediction residual coding.

6 and Fig. 7, too, as will be outlined hereafter.

The encoder 10 is configured to subject the prediction residual signal to spatial-to-spectral transformation and to encode the prediction residual signal, thus obtained, into the data stream 14. Likewise, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and subject the prediction residual signal thus obtained to spectral-to-spatial transformation.

encoder 10 may optionally comprised an entropy coder 34 which entropy codes the prediction residual signal as transformed and quantized into data stream 14. The prediction signal 26 is generated by a prediction stage 36 of encoder 10 on the basis of the prediction residual signal 24” encoded into, and decodable from, data stream 14. To this end, the prediction stage 36 may internally, as is shown in Fig. 6, compris a dequantizer 38 which dequantizes prediction residual signal 24” so as to gain spectral-domain prediction residual signal 24”', which corresponds to signal 24' except for quantization loss, followed by an inverse transformer 40 which subjects the latter prediction residual signal 24”' to an inverse transformation, eg, a spectral-to-spatial transformation, to obtain prediction residual signal 24””, which corresponds to the original prediction residual signal 24 except for quantization loss. A combiner 42 of the prediction stage 36 then recombines, such as by addition, the prediction signal 26 and the prediction residual signal 24”” so as to obtain a reconstructed signal 46, eg, a

Reconstructed signal 46 may correspond to signal 12'. A prediction module 44 of prediction stage 36 then generates the prediction signal 26 on the basis of signal 46 by using, for instance, spatial prediction, eg, intra-picture prediction, and/or temporal prediction, eg, inter-picture prediction.

Likewise, decoder 20, as shown in Fig. 7, may be internally composed of components corresponding to, and interconnected in a manner corresponding to, prediction stage 36. In particular, entropy decoder 50 of decoder 20 may entropy decode the quantized spectral-domain prediction residual signal 24” from the data stream, whereupon dequantizer 52, inverse transformer 54, combiner 56 and prediction module 58, interconnected and cooperating in the manner described above with respect to the modules of prediction stage 36, recover the reconstructed signal on the basis of prediction residual signal 24” so that, as shown in Fig. 7, the output of combiner 56 results in the reconstructed signal, namely picture 12'.

Although not specifically described above, it is readily clear that the encoder 10 may set some coding parameters including, for instance, prediction modes, motion parameters and the like, according to some optimization scheme such as, for instance, in a manner optimizing some rate and distortion related criterion, eg, coding cost. For example, encoder 10 and decoder 20 and the corresponding modules 44, 58, respectively, may support different prediction modes such as intra-coding modes and inter-coding modes. The granularity at which encoder and decoder switch between these prediction mode types may correspond to a subdivision of picture 12 and 12', respectively, into coding segments or coding blocks. In units of these coding segments, for instance, the picture may be subdivided into blocks being intra-coded and blocks being inter-coded. Intra-coded blocks are predicted on the basis of a spatial, already coded/decoded neighborhood of the respective block as is outlined in more detail below. Several intra-coding modes may exist and be selected for a respective intra-coded segment including directional or angular intra coding modes according to which the respective segment is filled by extrapolating the sample values ​​of the neighborhood along a certain direction which is specific for the respective directional intra-coding mode, into the respective intra-coded segment. The intra coding modes may, for instance, also comprise one or more further modes such as a DC coding mode, according to which the prediction for the respective intra-coded block assigns a DC value to all samples within the respective intra-coded segment, and/or a planar intra coding mode according to which the prediction of the respective block is approximated or determined to be a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample positions of the respective intra-coded block with driving tilt and offset of the plane defined by the two-dimensional linear function on the basis of the neighboring samples. Compared thereto, inter-coded blocks may be predicted, for instance, temporally. For inter-coded blocks, motion vectors may be signaled within the data stream, the motion vectors indicating the spatial displacement of the portion of a previously coded picture of the video to which picture 12 belongs, at which the previously coded/decoded picture is sampled in order to obtain the prediction signal for the respective inter-coded block. This means, in addition to the residual signal coding understood by data stream 14, such as the entropy-coded transform coefficient levels representing the quantized spectral-domain prediction residual signal 24”, data stream 14 may have encoded thereinto coding mode parameters for assigning the coding modes to the various blocks, prediction parameters for some of the blocks, such as motion parameters for inter-coded segments, and optional further parameters such as parameters for controlling and signaling the subdivision of picture 12 and 12', respectively, into the segments. The decoder 20 uses these parameters to subdivide the picture in the same manner as the encoder did, to assign the same prediction modes to the segments, and to perform the same prediction to result in the same prediction signal.

Fig. 8 illustrates the relationship between the reconstructed signal, eg, the reconstructed picture 12', on the one hand, and the combination of the prediction residual signal 24”” as signaled in the data stream 14, and the prediction signal 26, on the other hand. As already denoted above, the combination may be an addition. The prediction signal 26 is illustrated in Fig. 8 as a subdivision of the picture area into intra-coded blocks which are illustratively indicated using hatching, and inter-coded blocks which are illustratively indicated not-hatched. The subdivision may be any subdivision, such as a regular subdivision of the picture area into rows and columns of square blocks or non-square blocks, or a multi-tree subdivision of picture 12 from a tree root block into a plurality of leaf blocks of varying size, such as a quadtree subdivision or the like, wherein a mixture thereof is illustrated in Fig. 8 in which the picture area is first subdivided into rows and columns of tree root blocks which are then further subdivided in accordance with a recursive multi-tree subdivisioning into one or more leaf blocks.

Again, data stream 14 may have an intra-coding mode coded thereinto for intra-coded blocks 80, which assigns one of several supported intra-coding modes to the respective intra-coded block 80. For inter-coded blocks 82, the data stream 14 may have one or more motion parameters coded thereinto. Generally speaking, inter-coded blocks 82 are not

restricted to being temporally coded. Alternatively, inter-coded blocks 82 may be any block predicted from previously coded portions beyond the current picture 12 itself, such as previously coded pictures of a video to which picture 12 belongs, or picture of another view or an hierarchically lower layer in the case of encoder and decoder being scalable encoders and decoders, respectively.

The prediction residual signal 24”” in Fig. 8 is also illustrated as a subdivision of the picture area into blocks 84. These blocks might be called transform blocks in order to distinguish same from the coding blocks 80 and 82. In effect, Fig. 8 illustrates that encoder 10 and decoder 20 may use two different subdivisions of picture 12 and picture 12', respectively, into blocks, namely one subdivision into coding blocks 80 and 82, respectively, and another subdivision into transform blocks 84. Both subdivisions might be the same, eg, each coding block 80 and 82, may concurrently form a transform block 84, but Fig. 8 illustrates the case where, for instance, a subdivision into transform blocks 84 forms an extension of the subdivision into coding blocks 80, or a combination thereof or any other sort of blocking. Just as an aside, it is noted that blocks 80, 82 and 84 are not restricted to being of quadratic, rectangular or any other shape.

Fig. 8 further illustrates that the combination of the prediction signal 26 and the prediction residual signal 24”” directly results in the reconstructed signal 12'. However, it should be noted that more than one prediction signal 26 may be combined with the prediction residual signal 24”” to result into picture 12' in accordance with alternative embodiments.

In Fig. 8, the transform blocks 84 shall have the following significance. Transformer 28 and inverse transformer 54 perform their transformations in units of these transform blocks 84. For instance, many codecs use some sort of DST or DCT for all transform blocks 84. Some codecs allow for skipping the transformation so that, for some of the transform blocks 84,

the prediction residual signal is coded in the spatial domain directly. However, in accordance with the embodiments described below, encoder 10 and decoder 20 are configured in such a manner that they support several transforms. For example, the transforms supported by encoder 10 and decoder 20 could include:

o DCT-II (or DCT-III), where DCT stands for Discrete Cosine Transform

o DST-IV, where DST stands for Discrete Sine Transform

o DCT-IV

o DST-VII

o Identity Transformation (IT)

Naturally, while transformer 28 would support all of the forward transform versions of these transforms, the decoder 20 or inverse transformer 54 would support the corresponding backward or inverse versions thereof:

o Reverse DCT-II (or Reverse DCT-III)

o Inverse DST-IV

o Reverse DCT-IV

o Inverse DST-VII

o Identity Transformation (IT)

The subsequent description provides more details on which transforms could be supported by encoder 10 and decoder 20. In any case, it should be noted that the set of supported transforms may comprise merely one transform such as one spectral-to-spatial or spatial-to -spectral transform.

As already outlined above, Fig. 6 to Fig. 8 have been presented as an example where the inventive concept described further below may be implemented in order to form specific examples for encoders and decoders according to the present application. Insofar, the encoder and decoder of Fig. 6 and Fig. 7, respectively, may represent possible implementations of the encoders and decoders described herein below. Fig. 6 and Fig. 7 are, however, only examples. An encoder according to embodiments of the present application may, however, perform block-based encoding of a picture 12 using the concept outlined in more detail below and being different from the encoder of Fig. 6 such as, for instance, in that same is no video encoder, but a still picture encoder, in that same does not support inter-prediction, or in that the sub-division into blocks 80 is performed in a manner different than exemplified in Fig. 8. Likewise, decoders according to embodiments of the present application may perform block-based decoding of picture 12' from data stream 14 using the coding concept further outlined below, but may differ, for instance, from the decoder 20 of Fig. 7 in that same is no video decoder, but a still picture decoder, in that same does not support intra-prediction, or in that same sub-divides picture 12' into blocks in a manner different than described with respect to Fig. 8 and/or in that same does not derive the prediction residual from the data stream 14 in transform domain, but in spatial domain, for instance. decoders according to embodiments of the present application may perform block-based decoding of picture 12' from data stream 14 using the coding concept further outlined below, but may differ, for instance, from the decoder 20 of Fig. 7 in that same is no video decoder, but a still picture decoder, in that same does not support intra-prediction, or in that same sub-divides picture 12' into blocks in a manner different than described with respect to Fig. 8 and/or in that same does not derive the prediction residual from the data stream 14 in transform domain, but in spatial domain, for instance. decoders according to embodiments of the present application may perform block-based decoding of picture 12' from data stream 14 using the coding concept further outlined below, but may differ, for instance, from the decoder 20 of Fig. 7 in that same is no video decoder, but a still picture decoder, in that same does not support intra-prediction, or in that same sub-divides picture 12' into blocks in a manner different than described with respect to Fig. 8 and/or in that same does not derive the prediction residual from the data stream 14 in transform domain, but in spatial domain, for instance. in that same does not support intra-prediction, or in that same sub-divides picture 12' into blocks in a manner different than described with respect to Fig. 8 and/or in that same does not derive the prediction residual from the data stream 14 in transform domain, but in spatial domain, for instance. in that same does not support intra-prediction, or in that same sub-divides picture 12' into blocks in a manner different than described with respect to Fig. 8 and/or in that same does not derive the prediction residual from the data stream 14 in transform domain, but in spatial domain, for instance.

Fig. 1 illustrates a video encoder 100 for encoding a video into a video data stream according to an embodiment. The video encoder 100 is configured to generate the video data stream.

Fig. 2 illustrates an apparatus 200 for receiving an input video data stream according to an embodiment. The input video data stream has a video encoded thereinto. The apparatus 200 is configured to generate an output video data stream from the input video data stream.

Fig. 3 illustrates a video decoder 300 for receiving a video data stream having a video stored therein according to an embodiment. The video decoder 300 is configured to decode the video from the video data stream.

Moreover, a system according to an embodiment is provided. The system includes the apparatus of Fig. 2 and the video decoder of Fig. 3. The video decoder (300) of Fig. 3 is configured to receive the output video data stream of the apparatus (200) of Fig. 2. The video decoder 300 of Fig. 3 is configured to decode the video from the output video data stream of the apparatus 200 of Fig. 2.

In an embodiment, the system may, eg, further comprise a video encoder 100 of Fig. 1. The apparatus 200 of Fig. 2 may, eg, be configured to receive the video data stream from the video encoder 100 of Fig. 1 as the input video data stream.

Claims

1. A video data stream having a video encoded thereinto, wherein the video data stream comprises:

an indication that indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

2. A video data stream according to claim 1,

wherein the indication is a parameter set flag,

wherein the video data stream comprises the parameter set flag that indicates whether or not the one or more scalable nesting supplemental enhancement information messages comprising timing information for each of the one or more output layer sets are present within the video data stream.

3. A video data stream according to claim 2,

wherein a sequence parameter set of the video data stream including the parameter set flag.

4. A video data stream according to claim 3,

wherein the parameter set flag is an sps_ols_nest_timing_present_flag.

5. A video data stream according to claim 1,

wherein the video data stream comprises a further supplemental enhancement information message which comprises a parameter set flag,

wherein the parameter set flag indicates whether or not the one or more scalable nesting supplemental enhancement information messages for each of the one or more output layer sets are present within the video data stream.

6. A video data stream according to one of claims 1 to 5,

wherein the timing information includes at least one of picture timing information and buffering period information and decoding unit information.

7. A video data stream according to one of claims 1 to 6,

wherein the timing information is timing information for a Hypothetical Reference Decoder.

8. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises an indication that indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

9. A video encoder according to claim 8,

wherein the indication is a parameter set flag,

wherein the video encoder is to generate the video data stream such that the video data stream comprehends the parameter set flag that indicates whether or not the one or more scalable nesting supplemental enhancement information messages comprising timing information for each of the one or more output layer sets are present within the video data stream.

10. A video encoder according to claim 9,

wherein the video encoder is to generate the video data stream such that a sequence parameter set of the video data stream including the parameter set flag.

11. A video encoder according to claim 10,

wherein the video encoder is to generate the video data stream such that the parameter set flag is an sps_ols_nest_timing_present_flag.

12. A video encoder according to claim 8,

wherein the video encoder is to generate the video data stream such that the video data stream comprehends a further supplemental enhancement information message which comprehends a parameter set flag,

wherein the video encoder is to generate the video data stream such that the parameter set flag indicates whether or not the one or more scalable nesting supplemental enhancement information messages for each of the one or more output layer sets are present within the video data stream.

13. A video encoder according to one of claims 8 to 12,

wherein the timing information includes at least one of picture timing information and buffering period information and decoding unit information.

14. A video encoder according to one of claims 8 to 13,

wherein the timing information is timing information for a Hypothetical Reference Decoder.

15. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein an indication indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

16. An apparatus according to claim 15,

wherein the indication is a parameter set flag,

wherein the apparatus is to process the video data stream comprising the parameter set flag that indicates whether or not the one or more scalable nesting supplemental enhancement information messages timing information for each of the one or more output layer sets are present within the video data stream .

17. An apparatus according to claim 16,

wherein a sequence parameter set of the video data stream including the parameter set flag.

18. An apparatus according to claim 17,

wherein the parameter set flag is an sps_ols_nest_timing_present_flag.

19. An apparatus according to claim 15,

wherein the video data stream comprises a further supplemental enhancement information message which comprises a parameter set flag,

wherein the parameter set flag indicates whether or not the one or more scalable nesting supplemental enhancement information messages for each of the one or more output layer sets are present within the video data stream,

wherein the apparatus is to process the further supplemental enhancement information message.

20. An apparatus according to one of claims 15 to 19,

wherein the timing information includes at least one of picture timing information and buffering period information and decoding unit information.

21. An apparatus according to one of claims 15 to 20,

wherein, if the one or more scalable nesting supplemental enhancement information messages comprehended timing information which comprehended picture timing information for each of the one or more output layer sets are present within the video data stream, the apparatus is to substitute picture timing information of a non -scalable- nested picture timing supplemental enhancement information message,

wherein, if the one or more scalable nesting supplemental enhancement information messages comprised timing information which comprised buffering period information for each of the one or more output layer sets are present within the video data stream, the apparatus is to substitute buffering period information of a non -scalable- nested buffering period supplemental enhancement information message,

wherein, if the one or more scalable nesting supplemental enhancement information messages comprised timing information which comprised decoding unit information for each of the one or more output layer sets are present within the video data stream, the apparatus is to substitute decoding unit information of a non -scalable- nested decoding unit supplemental enhancement information message.

22. An apparatus according to one of claims 15 to 21,

wherein the timing information is timing information for a Hypothetical Reference Decoder.

23. An apparatus according to one of claims 15 to 22,

wherein the apparatus is to decode the sub-bitstream to decode the video.

24. A system for encoding a video into a video data stream, and for decoding the video, wherein the system includes:

a video encoder according to one of claims 8 to 14, and

an apparatus according to claim 23,

wherein the video encoder is to encode the video into the video data stream, such that the video data stream has the video encoded thereinto,

wherein the apparatus is to receive the video data stream as an input bitstream,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein the apparatus is to decode the sub-bitstream to decode the video.

25. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the method comprised generating the video data stream such that the video data stream comprised an indication that indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

26. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes processing the input bitstream to obtain a sub bitstream,

wherein the indication indicates whether or not one or more scalable nesting supplemental enhancement information messages comprising timing information for each of one or more output layer sets are present within the video data stream.

27. A computer program for implementing the method of claim 25 or 26 when being executed on a computer or signal processor.

28. A video data stream having a video encoded thereinto,

wherein an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable- nested picture timing supplemental enhancement information messages of the video data stream.

29. A video data stream according to claim 28,

wherein, if the indication indicates that the timing information for the sub-bitstream is not to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages, this indicates that the one or more non-scalable-nested picture timing supplemental enhancement information messages are to be substituted by one or more scalable nesting picture timing supplemental enhancement information messages.

30. A video data stream according to claim 28 or 29,

wherein the indication is a flag,

wherein one of the one or more the scalable nesting supplemental enhancement information messages comprised the flag that indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non- scalable-nested picture timing supplemental enhancement information messages .

31. A video data stream according to claim 30,

where the flag is a use_orig_pic_timing_flag.

32. A video data stream according to claim 28 or 29,

wherein the video data stream includes a video parameter set,

wherein the indication is a flag,

wherein the video parameter set comprises the flag that indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages.

33. A video data stream according to claim 32,

where the flag is a same_pic_timing_within_ols_flag.

34. A video data stream according to claim 32,

wherein the flag is a general_same_pic_timing_in_all_ols_flag.

35. A video data stream according to one of claims 28 to 34,

wherein, if the indication indicates that the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages, at least one of one or more scalable nesting supplemental enhancement information messages occurs before the one or more non-scalable-nested picture timing supplemental enhancement information messages within an access unit of the bitstream.

36. A video data stream according to claim 28 or 29,

wherein the indication is a constraint flag,

wherein the one or more non-scalable-nested picture timing supplemental enhancement information messages comprehend the constraint flag that indicates whether or not the one or more non-scalable-nested picture timing supplemental enhancement information messages apply to the at least one of the one or more sub-bitstreams.

37. A video data stream according to one of claims 28 to 36,

wherein the indication is a first indication,

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a third indication within the video data stream indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

38. A video data stream according to one of claims 28 to 37,

wherein the indication within the video data stream plus indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein the indication within the video data stream plus indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

39. A video data stream having a video encoded thereinto,

wherein a first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from the one or more non-scalable- nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

40. A video data stream having a video encoded thereinto,

wherein the video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information,

wherein, if the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, this indicates that depending on the scalable nesting supplemental enhancement information message

all of the one or more non-scalable-nested timing information supplemental enhancement information messages are to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information; gold

a subset comprising at least one of the one or more non-scalable-nested timing information enhancement information messages is to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information.

41. A video data stream according to claim 40,

wherein the timing information is the picture timing information or the buffering period information or the decoding unit information.

42. A video data stream according to claim 40 or 41 ,

wherein the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non- scalable-nested picture timing supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting picture timing supplemental enhancement information message; gold

wherein the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non- scalable-nested buffering period supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting buffering period supplemental enhancement information message; gold

wherein the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non- scalable-nested decoding unit supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting decoding unit supplemental enhancement information message.

43. A video data stream according to one of claims 40 to 42,

wherein, if the video data stream comprised the scalable nesting supplemental enhancement information message comprising the timing information, the scalable nesting supplemental enhancement information message comprising the timing information occurs before the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information within an access unit of the bitstream.

44. A video data stream according to one of claims 28 to 43,

wherein the timing information is timing information for a Hypothetical Reference Decoder.

45. A video data stream according to one of claims 28 to 44,

wherein the sub-bitstream depends on an output layer set and/or depends on a sub layer, and/or depends on a subpicture, and/or depends on a subset of a decoding unit.

46. ​​A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video data stream.

47. A video encoder according to claim 46,

wherein, if the indication indicates that the timing information for the sub-bitstream is not to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages, this indicates that the one or more non-scalable-nested picture timing supplemental enhancement information messages are to be substituted by one or more scalable nesting picture timing supplemental enhancement information messages.

48. A video encoder according to claim 46 or 47,

wherein the indication is a flag,

wherein the video encoder is to generate the video data stream such that one of the one or more the scalable nesting supplemental enhancement information messages comprised the flag that indicates whether or not the timing information for the sub bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages.

49. A video encoder according to claim 48,

where the flag is a use_orig_pic_timing_flag.

50. A video encoder according to claim 46 or 47,

wherein the video data stream includes a video parameter set,

wherein the indication is a flag,

wherein the video encoder is to generate the video data stream such that the video parameter set comprises the flag that indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages.

51. A video encoder according to claim 50,

where the flag is a same_pic_timing_within_ols_flag.

52. A video encoder according to claim 50,

wherein the flag is a general_same_pic_timing_in_all_ols_flag.

53. A video encoder according to one of claims 46 to 52,

wherein the indication indicates that each of the one or more non-scalable-nested picture timing supplemental enhancement information messages in each access unit of one or more access units applies to an access unit for any output layer set in the video data stream and that no scalable-nested picture timing supplemental enhancement information messages are present, or indicates that the non-scalable- nested picture timing supplemental enhancement information message in each access unit of the one or more access units might or might not apply to the access unit for any output layer set in the video data stream and that scalable-nested picture timing supplemental enhancement information messages might be present.

54. A video encoder according to one of claims 46 to 53,

wherein, if the indication indicates that the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages, at least one of one or more scalable nesting supplemental enhancement information messages occurs before the one or more non-scalable-nested picture timing supplemental enhancement information messages within an access unit of the bitstream.

55. A video encoder according to claim 46 or 47,

wherein the indication is a constraint flag,

wherein the video encoder is to generate the video data stream such that the one or more non-scalable-nested picture timing supplemental enhancement information messages understood the constraint flag that indicates whether or not the one or more non-scalable-nested picture timing supplemental enhancement information messages apply to the at least one of the one or more sub-bitstreams.

56. A video encoder according to one of claims 46 to 55,

wherein the indication is a first indication,

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a third indication within the video data stream indicates whether or not the timing information for the sub-bitstream is further to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

57. A video encoder according to one of claims 46 to 55,

wherein the indication within the video data stream plus indicates whether or not the timing information for the sub-bitstream is further to be obtained from the one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein the indication within the video data stream plus indicates whether or not the timing information for the sub-bitstream is further to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

58. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein a first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from the one or more non-scalable- nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

59. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprehending one or more non-scalable-nested supplemental enhancement information comprising messages timing information,

wherein, if the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, this indicates that depending on the scalable nesting supplemental enhancement information message

all of the one or more non-scalable-nested timing information supplemental enhancement information messages are to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information; gold

a subset comprising at least one of the one or more non-scalable-nested timing information supplemental enhancement information messages is to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information.

60. A video encoder according to claim 59,

wherein the timing information is the picture timing information or the buffering period information or the decoding unit information.

61. A video encoder according to claim 59 or 60,

wherein the video encoder is to generate the video data stream such that the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non-scalable-nested picture timing supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting picture timing supplemental enhancement information message; gold

wherein the video encoder is to generate the video data stream such that the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non-scalable-nested buffering period supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting buffering period supplemental enhancement information message; gold

wherein the video encoder is to generate the video data stream such that the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non-scalable-nested decoding unit supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting decoding unit supplemental enhancement information message.

62. A video encoder according to one of claims 59 to 61,

wherein, if the video data stream comprises the scalable nesting supplemental enhancement information message comprising the timing information, the video encoder is to generate the video data stream such that the scalable nesting supplemental enhancement information message comprising the timing information occurs before the one or more non -scalable-nested supplemental enhancement

information messages comprising the timing information within an access unit of the bitstream.

63. A video encoder according to one of claims 46 to 62,

wherein the timing information is timing information for a Hypothetical Reference Decoder.

64. A video encoder according to one of claims 46 to 63,

wherein the sub-bitstream depends on an output layer set and/or depends on a sub layer, and/or depends on a subpicture, and/or depends on a subset of a decoding unit.

65. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable- nested picture timing supplemental enhancement information messages of the video data stream.

66. An apparatus according to claim 65,

wherein, if the indication indicates that the timing information for the sub-bitstream is not to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages, the apparatus is to substitute the one or more non-scalable- nested picture timing supplemental enhancement information messages by one or more scalable nesting picture timing supplemental enhancement information messages.

67. An apparatus according to claim 65 or 66,

wherein the indication is a flag,

wherein one of the one or more the scalable nesting supplemental enhancement information messages comprised the flag which indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non- scalable-nested picture timing supplemental enhancement information messages .

68. An apparatus according to claim 67,

where the flag is a use_orig_pic_timing_flag.

69. An apparatus according to claim 65 or 66,

wherein the video data stream includes a video parameter set,

wherein the indication is a flag,

wherein the video parameter set comprises the flag that indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages.

70. An apparatus according to claim 69,

where the flag is a same_pic_timing_within_ols_flag.

71. An apparatus according to claim 69,

wherein the flag is a general_same_pic_timing_in_all_ols_flag.

72. An apparatus according to one of claims 65 to 71,

wherein the indication indicates that each of the one or more non-scalable-nested picture timing supplemental enhancement information messages in each access unit of one or more access units applies to an access unit for any output layer set in the video data stream and that no scalable-nested picture timing supplemental enhancement information messages are present, or indicates that the non-scalable- nested picture timing supplemental enhancement information message in each access unit of the one or more access units might or might not apply to the access unit for any output layer set in the video data stream and that scalable-nested picture timing supplemental enhancement information messages might be present.

73. An apparatus according to claim 72,

wherein, if the indication indicates that the non-scalable-nested picture timing supplemental enhancement information message in each access unit of the one or more access units might or might not apply to the access unit for any output layer set in the video data stream and that scalable-nested picture timing supplemental enhancement information messages might be present, the apparatus is configured to remove from the input bitstream or from the sub-bitstream all supplemental enhancement information network abstraction layer units that comprehend a non- scalable-nested supplemental enhancement information messages with picture timing content.

74. An apparatus according to one of claims 65 to 73,

wherein, if the indication indicates that the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested picture timing supplemental enhancement information messages, the apparatus is to process at least one of one or more scalable nesting supplemental enhancement information messages, which occurs before the one or more non-scalable-nested picture timing supplemental enhancement information messages within an access unit of the bitstream, before the apparatus is to process the one or more non-scalable-nested picture timing supplemental enhancement information messages within the access unit.

75. An apparatus according to claim 65 or 66,

wherein the indication is a constraint flag,

wherein the apparatus is to process the one or more non-scalable-nested picture timing supplemental enhancement information messages which comprehend the constraint flag that indicates whether or not the one or more non-scalable-nested picture timing supplemental enhancement information messages apply to the at least one of the one or more sub-bitstreams.

76. An apparatus according to one of claims 65 to 75,

wherein the indication is a first indication,

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a third indication within the video data stream indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

77. An apparatus according to one of claims 65 to 75,

wherein the indication within the video data stream plus indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein the indication within the video data stream plus indicates whether or not the timing information for the sub-bitstream is further to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

78. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein a first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable- nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

79. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein the video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information,

wherein, if the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, the apparatus is to substitute, depending on the scalable nesting supplemental enhancement information message,

all of the one or more non-scalable-nested timing information supplemental enhancement information messages by the scalable nesting supplemental enhancement information message comprising the timing information; gold

a subset comprising at least one of the one or more non-scalable-nested timing information supplemental enhancement information messages by the scalable nesting supplemental enhancement information message comprising the timing information.

80. An apparatus according to claim 79,

wherein the timing information is the picture timing information or the buffering period information or the decoding unit information.

81. An apparatus according to claim 79 or 80,

wherein the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non-scalable-nested picture timing supplemental enhancement information messages,

and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting picture timing supplemental enhancement information message; gold

wherein the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non- scalable-nested buffering period supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting buffering period supplemental enhancement information message; gold

wherein the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information are one or more non- scalable-nested decoding unit supplemental enhancement information messages, and the scalable nesting supplemental enhancement information message comprising the timing information is a scalable nesting decoding unit supplemental enhancement information message.

82. An apparatus according to one of claims 79 to 81,

wherein, if the video data stream comprises the scalable nesting supplemental enhancement information message comprising the timing information,

the scalable nesting supplemental enhancement information message comprising the timing information occurs before the one or more non- scalable-nested supplemental enhancement information messages the comprising timing information within an access unit of the bitstream, and

the apparatus is to process the scalable nesting supplemental enhancement information message comprising the timing information before the one or more non-scalable-nested supplemental enhancement information messages comprising the timing information.

83. An apparatus according to one of claims 65 to 82,

wherein the timing information is timing information for a Hypothetical Reference Decoder.

84. An apparatus according to one of claims 65 to 83,

wherein the sub-bitstream depends on an output layer set and/or depends on a sub layer, and/or depends on a subpicture, and/or depends on a subset of a decoding unit.

85. An apparatus according to one of claims 65 to 84,

wherein the apparatus is to decode the sub-bitstream to decode the video.

86. A system for encoding a video into a video data stream, and for decoding the video, wherein the system comprises:

a video encoder according to one of claims 46 to 64, and

an apparatus according to claim 85,

wherein the video encoder is to encode the video into the video data stream, such that the video data stream has the video encoded thereinto,

wherein the apparatus is to receive the video data stream as an input bitstream,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein the apparatus is to decode the sub-bitstream to decode the video.

87. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the method comprises generating the video data stream such that an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable-nested picture timing supplemental enhancement information messages of the video data stream.

88. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes processing the input bitstream to obtain a sub bitstream,

wherein an indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable- nested picture timing supplemental enhancement information messages of the video data stream.

89. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein a first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable- nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

90. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes processing the input bitstream to obtain a sub bitstream,

wherein a first indication within the video data stream indicates whether or not timing information for a sub-bitstream is to be obtained from one or more non-scalable- nested buffering period supplemental enhancement information messages of the video data stream; and/or

wherein a second indication within the video data stream indicates whether or not the timing information for the sub-bitstream is to be obtained from the one or more non-scalable-nested decoding unit supplemental enhancement information messages of the video data stream.

91. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the method comprised generating the video data stream such that the video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information,

wherein, if the video data stream comprises a scalable nesting supplemental enhancement information message comprising the timing information, this indicates that at least one of the one or more non-scalable-nested picture timing supplemental enhancement information messages is to be substituted by the scalable nesting supplemental enhancement information message comprising the timing information.

92. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes processing the input bitstream to obtain a sub bitstream,

wherein the video data stream comprised one or more non-scalable-nested supplemental enhancement information messages comprising timing information,

wherein, if the video data stream comprised a scalable nesting supplemental enhancement information message comprising the timing information, the method comprised substituting at least one of the one or more non-scalable-nested picture timing supplemental enhancement information messages by the scalable nesting supplemental enhancement information message comprising the timing information.

93. A computer program for implementing the method of one of claims 87 to 92 when being executed on a computer or signal processor.

94. A video data stream having a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

95. A video data stream having a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video data stream comprised an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

96. A video data stream having a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

97. A video data stream having a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprehends three or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the video data stream comprehends an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

98. A video data stream according to one of claims 94 to 97,

wherein all scalable nested supplemental enhancement information messages in an access unit with a same value of an identifier, identifying an output layer set, carry the same buffer period information and/or the same picture timing information.

99. A video data stream according to claim 98,

wherein, for a particular access unit of the plurality of access units, any picture timing supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries the same picture timing information, and/or

wherein, for a particular access unit of the plurality of access units, any buffer period supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries the same buffer period information, and/or

wherein, for a particular access unit of the plurality of access units, any decoding unit supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries a same decoding unit information.

100. A video data stream according to claim 98 or 99,

wherein, for two scalable-nested supplemental enhancement information messages of a particular payload type in an access unit with the same value of an identifier, identifying an output layer set, carry a same content.

101. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video encoder is to generate the video data stream such that the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

102. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

103. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video encoder is to generate the video data stream such that the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

104. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

105. A video encoder according to one of claims 101 to 104,

wherein all scalable nested supplemental enhancement information messages in an access unit with a same value of an identifier, identifying an output layer set, carry the same buffer period information and/or the same picture timing information.

106. A video encoder according to claim 105,

wherein, for a particular access unit of the plurality of access units, any picture timing supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries the same picture timing information, and/or

wherein, for a particular access unit of the plurality of access units, any buffer period supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries the same buffer period information, and/or

wherein, for a particular access unit of the plurality of access units, any decoding unit supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries a same decoding unit information.

107. A video encoder according to claim 105 or 106,

wherein, for two scalable-nested supplemental enhancement information messages of a particular payload type in an access unit with the same value of an identifier, identifying an output layer set, carry a same content.

108. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the apparatus is to process the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

109. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the apparatus is to process the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video data stream comprised an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

110. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the apparatus is to process the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

111. An apparatus for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the apparatus is to process the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprehends three or more scalable nested supplemental enhancement information messages which comprehend buffer period information and/or picture timing information for an output layer set, then the video data stream comprehends an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

112. An apparatus according to one of claims 108 to 111,

wherein all scalable nested supplemental enhancement information messages in an access unit with a same value of an identifier, identifying an output layer set, carry the same buffer period information and/or the same picture timing information.

113. An apparatus according to claim 112,

wherein, for a particular access unit of the plurality of access units, any picture timing supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries the same picture timing information, and/or

wherein, for a particular access unit of the plurality of access units, any buffer period supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries the same buffer period information, and/or

wherein, for a particular access unit of the plurality of access units, any decoding unit supplemental enhancement information message that applies to a set of layers and sub-layers for said output layer set, carries a same decoding unit information.

114. An apparatus according to claim 112 or 113,

wherein, for two scalable-nested supplemental enhancement information messages of a particular payload type in an access unit with the same value of an identifier, identifying an output layer set, carry a same content.

115. An apparatus according to one of claims 112 to 114,

wherein, if the apparatus has found a buffer period scalable nesting supplemental enhancement information message for an output layer set, the apparatus is to use a content of said buffer period scalable nesting supplemental enhancement information for said output layer set, without searching for further buffer period scalable nesting supplemental enhancement information messages for said output layer set, and/or

wherein, if the apparatus has found a picture timing scalable nesting supplemental enhancement information message for an output layer set, the apparatus is to use a content of said picture timing scalable nesting supplemental enhancement information for said output layer set, without searching for further picture timing scalable nesting supplemental enhancement information messages for said output layer set.

116. An apparatus according to one of claims 108 to 115,

wherein the apparatus is to decode the sub-bitstream to decode the video.

117. A system for encoding a video into a video data stream, and for decoding the video, wherein the system comprises:

a video encoder according to one of claims 101 to 107, and

an apparatus according to claim 116,

wherein the video encoder is to encode the video into the video data stream, such that the video data stream has the video encoded thereinto,

wherein the apparatus is to receive the video data stream as an input bitstream,

wherein the apparatus is to process the input bitstream to obtain a sub-bitstream,

wherein the apparatus is to decode the sub-bitstream to decode the video.

118. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

the method comprised generating the video data stream such that the video data stream comprised a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the method comprehends generating the video data stream such that the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

119. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

the method comprised generating the video data stream such that the video data stream comprised a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the method comprised generating the video data stream such that the video data stream comprised an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

120. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

the method comprised generating the video data stream such that the video data stream comprised a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the method comprehends generating the video data stream such that the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

121. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

the method comprised generating the video data stream such that the video data stream comprised a plurality of access units,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the method comprehending generating the video data stream such that the video data stream comprehending an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

122. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the method includes processing the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

123. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the method includes processing the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised two or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video data stream comprised an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set is equal in all of the two or more scalable nested supplemental enhancement information messages of the access unit.

124. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the method includes processing the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

125. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the input bitstream comprises a plurality of access units,

wherein the method includes processing the access units of the input bitstream to obtain a sub-bitstream,

wherein, for each access unit of the plurality of access units,

if the access unit comprised three or more scalable nested supplemental enhancement information messages which comprised buffer period information and/or picture timing information for an output layer set,

then the video data stream comprised an indication indicating whether or not the buffer period information and/or the picture timing information for the output layer set appears only in one of the three or more scalable nested supplemental enhancement information messages and in another one of the three or more scalable nested supplemental enhancement information messages that immediately succeeds said one of the three or more scalable nested supplemental enhancement information messages.

126. A computer program for implementing the method of one of claims 118 to 125 when being executed on a computer or signal processor.

127. A video data stream having a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein the video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream; gold

wherein the video data stream comprises a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream.

128. A video data stream according to claim 127,

wherein each of the sub-bitstreams depends on an output layer set and/or depends on a sub-layer, and/or depends on a sub-picture.

129. A video data stream according to claim 127 or 128,

wherein a decoding unit removal time depends on an access unit removal time and on the spread factor.

130. A video data stream according to claim 129,

wherein the video data stream comprises a temporal distance, wherein the temporal distance is to be multiplied by a derived clock sub-tick value, wherein the derived clock sub-tick value is derivable using the spread factor.

131. A video data stream according to one of claims 127 to 130,

wherein the spread factor is one of a plurality of spread factors,

wherein each of the plurality of spread factors is assigned to a sublayer of a plurality of sublayers,

wherein the video data stream includes the majority of spread factors.

132. A video data stream according to one of claims 127 to 130,

wherein the clock sub-tick value depends on a clock tick and further depends on the spread factor.

133. A video data stream according to claim 132,

wherein the clock sub-tick value is defined according to:

ClockSubTick=

= ClockTick ÷ ( tick_divisor_minus2 + 2 ) * ( tick_divisor_factor_minusl[ HTid ] + 1 ) wherein ClockSubTick is the clock sub-tick value,

where ClockTick is the clock tick,

wherein tick_divisor_minus2 is an additional tick divisor, and

wherein tick_divisor_factor_minusl[ HTid ] indicates the spread factor of the video data stream.

134. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a plurality of access units,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a spread factor which depends on a number of sub bitstreams of the video data stream; gold

wherein the video encoder is to generate the video data stream such that the video data stream comprises a clock sub-tick value which depends on a highest sub bitstream among the sub-bitstreams of the video data stream.

135. A video encoder according to claim 134,

wherein each of the sub-bitstreams depends on an output layer set and/or depends on a sub-layer, and/or depends on a sub-picture.

136. A video encoder according to claim 134 or 135,

wherein the video encoder is to generate the video data stream such that a decoding unit removal time depends on an access unit removal time and on the spread factor.

137. A video encoder according to claim 136,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a temporal distance, wherein the temporal distance is to be multiplied by a derived clock sub-tick value, wherein the derived clock sub-tick value is derivable using the spread factor.

138. A video encoder according to one of claims 134 to 137,

wherein the spread factor is one of a plurality of spread factors,

wherein each of the plurality of spread factors is assigned to a sublayer of a plurality of sublayers,

wherein the video encoder is to generate the video data stream such that the video data stream comprises the plurality of spread factors.

139. A video encoder according to one of claims 134 to 137,

wherein the clock sub-tick value depends on a clock tick and further depends on the spread factor.

140. A video encoder according to claim 139,

wherein the clock sub-tick value is defined according to:

ClockSubTick=

= ClockTick ÷ ( tick_divisor_minus2 + 2 ) * ( tick_divisor_factor_minusl[ HTid ] + 1 ) wherein ClockSubTick is the clock sub-tick value,

where ClockTick is the clock tick,

wherein tick_divisor_minus2 is an additional tick divisor, and

wherein tick_divisor_factor_minusl[ HTid ] indicates the spread factor of the video data stream.

141. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprised sub-layer specific frame rate information for a sub-layer, and/or

wherein the video encoder is to generate the video data stream such that the video data stream comprised sub-layer specific frame display duration information for the sub-layer.

142. A video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein the video decoder is to decode the video data stream to decode the video,

wherein the video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream, wherein the video decoder is to decode the video using the spread factor; gold

among the video data stream comprised a clock sub-tick value which depends on a highest sub-bitstream the sub-bitstreams of the video data stream, wherein the video decoder is to decode the video using the clock sub-tick value.

143. A video decoder according to claim 142,

wherein each of the sub-bitstreams depends on an output layer set and/or depends on a sub-layer, and/or depends on a sub-picture.

144. A video decoder according to claim 142 or 143,

wherein a decoding unit removal time depends on an access unit removal time and on the spread factor.

145. A video decoder according to claim 144,

wherein the video data stream comprised a temporal distance,

wherein the video decoder is to multiply the temporal distance by a derived clock sub-tick value,

wherein the video decoder is to derive the derived clock sub-tick value using the spread factor.

146. A video decoder according to one of claims 142 to 145,

wherein the spread factor is one of a plurality of spread factors,

wherein each of the plurality of spread factors is assigned to a sublayer of a plurality of sublayers,

wherein the video data stream includes the majority of spread factors,

wherein the video decoder is to decode the video using the plurality of spread factors.

147. A video decoder stream according to one of claims 142 to 145,

wherein the clock sub-tick value depends on a clock tick and further depends on the spread factor.

148. A video decoder according to claim 147,

wherein the clock sub-tick value is defined according to:

ClockSubTick=

= ClockTick ÷ ( tick_divisor_minus2 + 2 ) * ( tick_divisor_factor_minusl[ HTid ] + 1 ) wherein ClockSubTick is the clock sub-tick value,

where ClockTick is the clock tick,

wherein tick_divisor_minus2 is an additional tick divisor, and

wherein tick_divisor_factor_minusl[ HTid ] indicates the spread factor of the video data stream.

149. A video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video decoder is to decode the video data stream to decode the video,

wherein the video data stream comprised sub-layer specific frame rate information for a sub-layer, and/or wherein the video data stream comprised sub-layer specific frame display duration information for the sub-layer,

wherein the decoder is to determine a spread factor using the sub-layer specific frame rate information for the sub-layer and/or using the sub-layer specific frame display duration information.

150. A system for encoding a video into a video data stream, and for decoding the video, wherein the system comprises:

a video encoder according to one of claims 134 to 141, and

a video decoder according to one of claims 142 to 149,

wherein the video encoder is to encode the video into the video data stream, such that the video data stream has the video encoded thereinto,

wherein the video decoder is to receive the video data stream and to decode the video data stream to decode the video.

151. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

the method comprised generating the video data stream such that the video data stream comprised a plurality of access units,

wherein the method comprises generating the video data stream such that the video data stream comprises a spread factor which depends on a number of sub bitstreams of the video data stream; gold

wherein the method comprehends generating the video data stream such that the video data stream comprehends a clock sub-tick value which depends on a highest sub bitstream among the sub-bitstreams of the video data stream.

152. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein the method includes decoding the video data stream to decode the video,

wherein the video data stream comprises a spread factor which depends on a number of sub-bitstreams of the video data stream, wherein the method comprises decoding the video using the spread factor; gold

wherein the video data stream comprises a clock sub-tick value which depends on a highest sub-bitstream among the sub-bitstreams of the video data stream, wherein the method comprises decoding the video using the clock sub-tick value.

153. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes decoding the video data stream to decode the video,

wherein the video data stream comprised sub-layer specific frame rate information for a sub-layer, and/or wherein the video data stream comprised sub-layer specific frame display duration information for the sub-layer,

wherein the method includes determining a spread factor using the sub-layer specific frame rate information for the sub-layer and/or using the sub-layer specific frame display duration information.

154. A computer program for implementing the method of one of claims 151 to 153 when being executed on a computer or signal processor.

155. A video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video decoder is to decode the video data stream to decode the video,

wherein, to decode the video, the video decoder is to estimate a coded picture buffer size for a subpicture depending on information within the video data stream indicating a current coded picture buffer size information.

156. A video decoder according to claim 155,

wherein the video data stream includes a signalled coded picture buffer size for a reference level.

157. A video decoder according to claim 155,

wherein, if a current level is equal to a reference level, the video decoder is to determine the coded picture buffer size using the coded picture buffer size for the reference level.

158. A video decoder according to one of claims 155 to 157,

wherein the video decoder is to estimate the coded picture buffer size depending on a syntax element cpb_size_vaiue_minusi[ i ] [ j ] of the video data stream.

159. A video decoder according to one of claims 155 to 158,

wherein the video decoder is to estimate the coded picture buffer size depending on a syntax element cpb_size_scaie of the video data stream.

160. A video decoder according to one of claims 155 to 159,

wherein the video decoder is to estimate the coded picture buffer size for the subpicture being a video coding layer coded picture buffer size, and

is to estimate a further coded picture buffer size for the subpicture being a network abstraction layer coded picture buffer size.

161. A video decoder according to claim 160,

wherein the video decoder is to estimate the video coding layer coded picture buffer size and/or the network abstraction layer coded picture buffer size depending on a reference level fraction value.

162. A video decoder according to claim 161,

wherein the video decoder is to estimate the video coding layer coded picture buffer size according to:

SubPicCbp Size V cl [ s ]=

= Floor(( cpb_size_value_minusl[ i ][ j ] + 1 )

* 2 ( 4 + c p b s ize sca i e ) * R e £LevelFraction[ i ][ j ] ÷ 256)

wherein the apparatus is to estimate the network abstraction layer coded picture buffer size according to:

SubPicCbpSizeNal[s] =

= Floor(( cpb_size_value_minusl[ i ][ j ] + 1 )

* 2 ( 4 + c p b s ize sca i e ) * R e fLevelFraction[ i ][ j ] ÷ 256),

where RefLevelFraction is the reference level fraction value.

163. A video decoder according to claim 161 ,

wherein the video decoder is to estimate the video coding layer coded picture buffer size according to:

SubpicCpb Size V cl [ i ][ j ][ k ] =

Floor( CpbVclF actor * MaxCPB * 01sRe£LevelFraction[ i ][ j ][ k ] ÷ 256) SubpicCpb SizeNal[ i ][ j ][ k ] =

Floor( CpbNalFactor * MaxCPB * 01sRefLevelFraction[ i ][ j ][ k ] ÷ 256)

wherein i and j and k are indices,

where 01sRefLevelFraction[ i ][ j ][ k ] is a real number.

164. A video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video decoder is to decode the video data stream to decode the video,

wherein, to decode the video, the video decoder is to estimate a bitrate for a subpicture depending on information within the video data stream indicating current coded video sequence bitrate information.

165. A video decoder according to claim 164,

wherein the video data stream comprises an indication indicating whether or not the bitrate for the subpicture shall be estimated using the current coded video sequence bitrate information,

wherein, if the indication of the video data stream indicates that the bitrate for the subpicture shall be estimated using the current coded video sequence bitrate information, the video decoder estimates the bitrate using the current coded video sequence bitrate information,

wherein, if the indication of the video data stream indicates that the bitrate for the subpicture shall be estimated without using the current coded video sequence bitrate information, the video decoder estimates the bitrate using a predefined value or a worst-case value without using the current coded video sequence bitrate information.

166. A video decoder according to claim 164 or 165,

wherein the current coded video sequence bitrate information is a signalled bitrate for a reference level,

wherein the video data stream comprises the signaled bitrate for the reference level, wherein, if a current level is equal to the reference level, the video decoder is to determine the bitrate for the subpicture using the signalled bitrate for the reference level.

167. A video decoder according to one of claims 164 to 166,

wherein the video decoder is to estimate the bitrate for the subpicture depending on a syntax element bit_rate_vaiue_minusi[ i ][ j ] of the video data stream.

168. A video decoder according to one of claims 164 to 167,

wherein the video decoder is to estimate the bitrate for the subpicture depending on a syntax element bit_rate_scaie of the video data stream.

169. A video decoder according to one of claims 164 to 168,

wherein the video decoder is to estimate the bitrate for the subpicture being a video coding layer bitrate for the subpicture, and

is to estimate a further coded picture buffer size for the subpicture being a network abstraction layer bitrate for the subpicture.

170. A video decoder according to claim 169,

wherein the video decoder is to estimate the video coding layer coded picture buffer size and/or the network abstraction layer coded picture buffer size depending on a reference level fraction value.

171. A video decoder according to one of claim 170,

wherein the video decoder is to estimate the video coding layer bitrate for the subpicture according to:

SubPicBitRateVcl[s] =

= Floor(( bit_rate_value_minusl[ i ][ j ] + 1 )

* 2 ( - b u rate scale ) * R e £LevelFraction[ i ][ j ] ÷ 256)

wherein the apparatus is to estimate the network abstraction layer bitrate for the subpicture according to:

SubPicBitRateNal[s] =

= Floor(( bit_rate_value_minusl[ i ][ j ] + 1 )

* 2 ( - bu rate scale ) * R e £LevelFraction[ i ][ j ] ÷ 256),

where RefLevelFraction is the reference level fraction value.

172. A video decoder according to claim 170,

wherein the video decoder is to estimate the video coding layer bitrate for the subpicture according to:

SubpicBitRateVcl[ i ][ j ][ k ] =

Floor( CpbVclFactor * ValBR * 01sRefLevelFraction[ 0 ][ j ][ k ] ÷ 256)

SubpicBitRateNal[ i ][ j ][ k ] =

Floor( CpbNalFactor * ValBR * 01sRefLevelFraction[ 0 ][ j ][ k ] ÷ 256) wherein i and j and k are indices,

where 01sRefLevelFraction[ 0 ][ j ][ k ] is a real number.

173. A video decoder according to claim 163 or 172,

wherein i indicates an index of a particular indicated reference level,

wherein j indicates an index of a particular subpicture of the pictures of an access unit in the video data stream, and

wherein k indicates an index of a maximum temporal sublayer that the video data stream includes and/or that the video decoder operates on.

174. A video decoder according to claim 163 or 172 or 173,

wherein OlsRefLevelFraction[ i ][ j ][ k ] depends on a variable sli_non_subpic_layers_fraction[ i ][ k ] which indicates an i-th fraction of bitstream level limits associated with layers in targetCvss that have sps_num_subpics_minus1 equal to 0 when Htid is equal to k .

175. A video decoder according to claim 174,

Wherein, if vps_max_layers_minus1 is equal to 0 or when no layer in the bitstream has sps_num_subpics_minus1 equal to 0, sli_non_subpic_layers_fraction[ i ][ k ] is 0, and

wherein, if k is less than sli_max_sublayers_minus1 and sli_non_subpic_layers_fraction[ i ][ k ] is not present, sli_non_subpic_layers_fraction[ i ][ k ] = sli_non_subpic_layers_fraction[ i ][ k + 1 ].

176. A video decoder according to one of claims 163 and 172 to 175,

wherein

OlsRefl_evelFraction[ i ][ j ][ k ] =

= sli_non_subpic_layers_fraction[ i ][ k ] + ( n - sli_non_subpic_layers_fraction[ i ][ k ] ) ÷ n * ( sli_ref_level_fraction_minus1 [ i ][ j ][ k ] + 1 ).

where n indicates a positive integer number.

177. A video decoder according to claim 176,

wherein n=256; or n=128; or n=512; or n=1024; gold n = 2048; or n = 4096.

178. A video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video decoder is to decode the video data stream to decode the video,

wherein, to decode the video, the video decoder is to receive a coded picture buffer size for a subpicture being encoded within the video data stream, and is to use the coded picture buffer size for the subpicture to decode the video; and/or

wherein, to decode the video, the video decoder is to receive a bitrate for a subpicture being encoded within the video data stream, and is to use the bitrate for the subpicture to decode the video.

179. A video decoder according to claim 178,

wherein the video data stream comprises an indication indicating whether or not the coded picture buffer size for the subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated,

wherein, if the indication of the video data stream indicates that the coded picture buffer size for the subpicture shall be estimated, the video decoder estimates the coded picture buffer size for the subpicture,

wherein, if the indication of the video data stream indicates that the coded picture buffer size for the subpicture is encoded within the video data stream, the video decoder uses the coded picture buffer size for the subpicture encoded within the video data stream.

180. A video decoder according to claim 178 or 179,

wherein the video data stream comprises an indication indicating whether or not the bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated,

wherein, if the indication of the video data stream indicates that the bitrate for the subpicture shall be estimated, the video decoder estimates the bitrate for the subpicture,

wherein, if the indication of the video data stream indicates that the bitrate for the subpicture is encoded within the video data stream, the video decoder uses the coded picture buffer size for the subpicture encoded within the video data stream. 181. A video decoder according to one of claims 155 to 180,

wherein each of a plurality of extractable sub-bitstreams are specific to an output layer set, wherein the subpicture is assigned to at least one extractable sub bitstream of the plurality of extractable sub-bitstreams,

wherein, if the video data stream includes common decoding unit removal timing information and the majority of extractable sub-bistreams, each output layer set- specific hypothetical reference decoder parameter syntax structure in a video parameter set or in a sequence parameter set or in a supplemental enhancement information message of the video data stream including either a spread factor or an absolute value of a tick divisor for scaling the common decoding unit removal timings,

wherein the video decoder is to process the spread factor or the absolute value of the tick divisor.

182. A video data stream having a video encoded thereinto,

wherein the video data stream includes a syntax element cpb size value minusl[ i ][ j ] and a syntax element cpb size scale;or

wherein the video data stream includes a syntax element bit rate value minusl[ i ][ j ] and a syntax element bit rate scale.

183. A video data stream having a video encoded thereinto,

wherein the video data stream comprises an indication indicating whether or not a coded picture buffer size for a subpicture shall be estimated using current coded picture buffer size information, and/or

wherein the video data stream comprises an indication indicating whether or not a bitrate for a subpicture shall be estimated using current coded video sequence bitrate information.

184. A video data stream according to claim 182 or 183,

wherein the current coded picture buffer size information is a signalled coded picture buffer size for a reference level, wherein the video data stream comprises the signalled coded picture buffer size for the reference level; and/or

wherein the current coded video sequence bitrate information is a signalled bitrate for a reference level, wherein the video data stream comprises the signalled bitrate for the reference level.

185. A video data stream having a video encoded thereinto,

wherein the video data stream includes an indication indicating whether or not a coded picture buffer size for a subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated, and/or

wherein the video data stream comprises an indication indicating whether or not a bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated.

186. A video data stream according to one of claims 182 to 185,

wherein each of a plurality of extractable sub-bitstreams are specific to an output layer set, wherein the subpicture is assigned to at least one extractable sub bitstream of the plurality of extractable sub-bitstreams,

wherein, if the video data stream includes common decoding unit removal timing information and the majority of extractable sub-bistreams, each output layer set- specific hypothetical reference decoder parameter syntax structure in a video parameter set or in a sequence parameter set or in a supplemental enhancement information message of the video data stream including either a spread factor or an absolute value of a tick divisor for scaling the common decoding unit removal timings.

187. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a syntax element cpb_size_vaiue_minusi[ i ] [ j ] and a syntax element cpb size scale;or

wherein the video encoder is to generate the video data stream such that the video data stream comprises a syntax element bit_rate_vaiue_minusi[ i ] [ j ] and a syntax element bit_rate_scaie.

188. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a coded picture buffer size for a subpicture shall be estimated using current coded picture buffer size information, and/or

wherein the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a bitrate for a subpicture shall be estimated using current coded video sequence bitrate information.

189. A video encoder according to claim 187 or 188,

wherein the video encoder is to generate the video data stream such that the current coded picture buffer size information is a signalled coded picture buffer size for a reference level, wherein the video data stream comprises the signalled coded picture buffer size for the reference level; and/or

wherein the video encoder is to generate the video data stream such that the current coded video sequence bitrate information is a signalled bitrate for a reference level, wherein the video data stream comprehends the signalled bitrate for the reference level.

190. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a coded picture buffer size for a subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated, and/or

wherein the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not a bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated.

191. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that, if the video data stream comprehends common decoding unit removal timing information and a plurality of extractable sub-bistreams, each of the plurality of sub-bitstreams being specific to an output layer set , each output layer set-specific hypothetical reference decoder parameter syntax structure in a video parameter set or in a sequence parameter set or in a supplemental enhancement information message of the video data stream comprised either a spread factor or an absolute value of a tick divisor for scaling the common decoding unit removal timings.

192. A system for encoding a video into a video data stream, and for decoding the video, wherein the system comprises:

a video encoder according to one of claims 187 to 191 , and

a video decoder according to one of claims 155 to 181,

wherein the video encoder is to encode the video into the video data stream, such that the video data stream has the video encoded thereinto,

wherein the video decoder is to receive the video data stream and to decode the video data stream to decode the video.

193. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes decoding the video data stream to decode the video,

wherein, to decode the video, the method comprehends estimating a coded picture buffer size for a subpicture depending on information within the video data stream indicating a current coded picture buffer size information.

194. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes decoding the video data stream to decode the video,

wherein, to decode the video, the method comprehends estimating a bitrate for a subpicture depending on information within the video data stream indicating current coded video sequence bitrate information.

195. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the method includes decoding the video data stream to decode the video,

wherein, to decode the video, the method comprehends receiving a coded picture buffer size for a subpicture being encoded within the video data stream, and comprehends using the coded picture buffer size for the subpicture to decode the video; and/or

wherein, to decode the video, the method comprehends receiving a bitrate for a subpicture being encoded within the video data stream, and comprehends using the bitrate for the subpicture to decode the video.

196. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the method comprised generating the video data stream such that the video data stream comprised a syntax element cpb_size_vaiue_minusi[ i ] [ j ] and a syntax element cpb size scale;or

wherein the method comprised generating the video data stream such that the video data stream comprised a syntax element bit_rate_vaiue_minusi[ i ] [ j ] and a syntax element bit_rate_scaie.

197. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the method comprised generating the video data stream such that the video data stream comprised an indication indicating whether or not a coded picture buffer size for a subpicture shall be estimated using current coded picture buffer size information, and/or

wherein the method comprised generating the video data stream such that the video data stream comprised an indication indicating whether or not a bitrate for a subpicture shall be estimated using current coded video sequence bitrate information.

198. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the method comprised generating the video data stream such that the video data stream comprised an indication indicating whether or not a coded picture buffer size for a subpicture is encoded within the video data stream or whether the coded picture buffer size for the subpicture shall be estimated , and/or

wherein the method comprehends generating the video data stream such that the video data stream comprehends an indication indicating whether or not a bitrate for the subpicture is encoded within the video data stream or whether the bitrate for the subpicture shall be estimated.

199. A computer program for implementing the method of one of claims 193 to 198 when being executed on a computer or signal processor.

200. A video data stream having a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein the video data stream includes delta time information for each of two or more decoding units of an access unit of the majority of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit.

201. A video data stream according to claim 200,

wherein the video data stream includes a picture timing supplemental enhancement information,

wherein the picture timing supplemental enhancement information includes the delta time information for the two or more decoding units of said access unit.

202. A video data stream according to claim 200 or 201,

wherein the delta time information indicates a removal time difference between two decoding units of the two or more decoding units of said access unit.

203. A video data stream according to claim 202,

wherein a last decoding unit among the two or more decoding units of said access unit has a removal time being equal to the removal time of said access unit.

204. A video data stream according to claim 202 or 203,

wherein said access unit includes three or more decoding units,

wherein the removal time difference is equal for each pair of two consecutive decoding units of the three or more decoding units of said access unit.

205. A video data stream according to claim 204,

wherein the picture timing supplemental enhancement information applies to a sub bitstream derived from the video data stream, where a number of decoding units remains constant.

206. A video data stream according to claim 205,

wherein a frame time interval is signaled in a parameter set of the video data stream, in the HRD parameters in a sequence parameter set.

207. A video data stream according to claim 205,

wherein a frame time interval is derivable as a difference in removal times of two consecutive access units at highest temporal level.

208. A video data stream according to one of claims 205 to 207,

wherein each decoding unit of the two or more decoding units of said access unit comprises one video coding layer network abstraction layer unit.

209. A video data stream according to claim 204,

wherein the picture timing supplemental enhancement information applies to a sub bitstream derived from the video data stream, where a different number of decoding units is present.

210. A video data stream according to one of claims 200 to 209,

wherein a frame time interval is derivable as

(elemental_duration_in_tc_minus1[maxTiD] +1 ) multiplied by ClockTicks.

211. A video data stream according to one of claims 204 to 210,

wherein the video data stream includes an indication indicating whether or not in a number of decoding units is variable for the video data stream.

212. A video data stream according to one of claims 204 to 210,

wherein the video data stream does not include an indication indicating whether or not in a number of decoding units is variable for the video data stream.

213. A video data stream according to claim 212,

wherein the number of decoding units within an access unit depends on a frame time interval and on a common delay increment.

214. A video data stream according to claim 200,

wherein the video data stream comprises a decoding unit information supplemental enhancement information message for a decoding unit of the two or more decoding units of said access unit,

wherein the decoding unit information supplemental enhancement information message for said decoding unit includes the delta time information for said decoding unit.

215. A video data stream according to one of claims 200 to 214,

wherein the video data stream comprises a minimum picture duration flag of a video parameter set or of a sequence parameter set of the video data stream, wherein the minimum picture duration flag indicates whether or not frame time interval information is present when there is no constant framerate .

216. A video encoder for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a plurality of access units,

wherein the video encoder is to generate the video data stream such that the video data stream including delta time information for each of two or more decoding units of an access unit of the majority of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit.

217. A video encoder according to claim 216,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a picture timing supplemental enhancement information,

wherein the video encoder is to generate the video data stream such that the picture timing supplemental enhancement information comprises the delta time information for the two or more decoding units of said access unit.

218. A video encoder according to claim 216 or 217,

wherein the video encoder is to generate the video data stream such that the delta time information indicates a removal time difference between two decoding units of the two or more decoding units of said access unit.

219. A video encoder according to claim 218,

wherein a last decoding unit among the two or more decoding units of said access unit has a removal time being equal to the removal time of said access unit.

220. A video encoder according to claim 218 or 219,

wherein the video encoder is to generate the video data stream such that said access unit comprises three or more decoding units,

wherein the removal time difference is equal for each pair of two consecutive decoding units of the three or more decoding units of said access unit.

221. A video encoder according to claim 220,

wherein the picture timing supplemental enhancement information applies to a sub bitstream derived from the video data stream, where a number of decoding units remains constant.

222. A video encoder according to claim 221 ,

wherein the video encoder is to generate the video data stream such that a frame time interval is signalled in a parameter set of the video data stream, in the HRD parameters in a sequence parameter set.

223. A video encoder according to claim 221 ,

wherein the video encoder is to generate the video data stream such that a frame time interval is derivable as a difference in removal times of two consecutive access units at highest temporal level.

224. A video encoder according to one of claims 221 to 223,

wherein the video encoder is to generate the video data stream such that each decoding unit of the two or more decoding units of said access unit comprised one video coding layer network abstraction layer unit.

225. A video encoder according to claim 220,

wherein the video encoder is to generate the video data stream such that the picture timing supplemental enhancement information applies to a sub-bitstream derived from the video data stream, where a different number of decoding units is present.

226. A video encoder according to one of claims 216 to 225,

wherein a frame time interval is derivable as

(elemental_duration_in_tc_minus1[maxTiD] +1 ) multiplied by ClockTicks.

227. A video encoder according to one of claims 220 to 226,

wherein the video encoder is to generate the video data stream such that the video data stream comprises an indication indicating whether or not in a number of decoding units is variable for the video data stream.

228. A video encoder according to one of claims 220 to 226,

wherein the video encoder is to generate the video data stream such that the video data stream does not include an indication indicating whether or not in a number of decoding units is variable for the video data stream.

229. A video encoder according to claim 228,

wherein the video encoder is to generate the video data stream such that the number of decoding units within an access unit depends on a frame time interval and on a common delay increment.

230. A video encoder according to claim 216,

wherein the video encoder is to generate the video data stream such that the video data stream comprises a decoding unit information supplemental enhancement information message for a decoding unit of the two or more decoding units of said access unit,

wherein the video encoder is to generate the video data stream such that the decoding unit information supplemental enhancement information message for said decoding unit including the delta time information for said decoding unit.

231. A video encoder according to one of claims 216 to 230,

wherein the video data stream comprises a minimum picture duration flag of a video parameter set or of a sequence parameter set of the video data stream, wherein the minimum picture duration flag indicates whether or not frame time interval information is present when there is no constant framerate .

232. A video decoder for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein the video decoder is to decode the video data stream to decode the video,

wherein the video data stream includes delta time information for each of two or more decoding units of an access unit of the majority of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit, wherein the video decoder is to decode the video data stream using the delta time information for each of two or more decoding units of said access unit .

233. A video decoder according to claim 232,

wherein the video data stream includes a picture timing supplemental enhancement information,

wherein the picture timing supplemental enhancement information includes the delta time information for the two or more decoding units of said access unit.

234. A video decoder according to claim 232 or 233,

wherein the delta time information indicates a removal time difference between two decoding units of the two or more decoding units of said access unit, wherein the video decoder is to decode the video data stream using the removal time difference between said two decoding units.

235. A video decoder according to claim 234,

wherein a last decoding unit among the two or more decoding units of said access unit has a removal time being equal to the removal time of said access unit, wherein the video decoder is to decode the video data stream using the removal time of said access unit .

236. A video decoder according to claim 234 or 235,

wherein said access unit includes three or more decoding units,

wherein the removal time difference is equal for each pair of two consecutive decoding units of the three or more decoding units of said access unit.

237. A video decoder according to claim 236,

wherein the picture timing supplemental enhancement information applies to a sub bitstream derived from the video data stream, where a number of decoding units remains constant.

238. A video decoder according to claim 237,

wherein a frame time interval is signaled in a parameter set of the video data stream, in the HRD parameters in a sequence parameter set, wherein the video decoder is to decode the video data stream using the frame time interval.

239. A video decoder according to claim 237,

wherein a frame time interval is derivable as a difference in removal times of two consecutive access units at highest temporal level, wherein the video decoder is to decode the video data stream using the frame time interval.

240. A video decoder according to one of claims 237 to 239,

wherein each decoding unit of the two or more decoding units of said access unit comprises one video coding layer network abstraction layer unit.

241. A video decoder according to claim 236,

wherein the picture timing supplemental enhancement information applies to a sub bitstream derived from the video data stream, where a different number of decoding units is present.

242. A video decoder according to one of claims 232 to 241,

wherein the video decoder is to derive the frame time interval according to:

(elemental_duration_in_tc_minus1[maxTiD] +1 ) multiplied by ClockTicks.

243. A video decoder according to one of claims 236 to 242,

wherein the video data stream comprises an indication indicating whether or not in a number of decoding units is variable for the video data stream, wherein the video decoder is to decode the video data stream by processing the indication.

244. A video decoder according to one of claims 226 to 242,

wherein the video data stream does not include an indication indicating whether or not in a number of decoding units is variable for the video data stream.

245. A video decoder according to claim 244,

wherein the number of decoding units within an access unit depends on a frame time interval and on a common delay increment.

246. A video decoder according to claim 232,

wherein the video data stream comprises a decoding unit information supplemental enhancement information message for a decoding unit of the two or more decoding units of said access unit,

wherein the decoding unit information supplemental enhancement information message for said decoding unit comprises the delta time information for said decoding unit, wherein the video decoder is to decode the video data stream using said delta time information for said decoding unit.

247. A video decoder according to one of claims 232 to 246,

wherein the video data stream comprises a minimum picture duration flag of a video parameter set or of a sequence parameter set of the video data stream, wherein the minimum picture duration flag indicates whether or not frame time interval information is present when there is no constant framerate .

248. A system for encoding a video into a video data stream, and for decoding the video, wherein the system comprises:

a video encoder according to one of claims 216 to 231 , and

a video decoder according to one of claims 232 to 247,

wherein the video encoder is to encode the video into the video data stream, such that the video data stream has the video encoded thereinto,

wherein the video decoder is to receive the video data stream and to decode the video data stream to decode the video.

249. A method for encoding a video into a video data stream, such that the video data stream has the video encoded thereinto,

the method comprised generating the video data stream such that the video data stream comprised a plurality of access units,

wherein the method comprised generating the video data stream such that the video data stream comprised delta time information for each of two or more decoding units of an access unit of the plurality of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit.

250. A method for receiving a video data stream as an input bitstream, wherein the video data stream has a video encoded thereinto,

wherein the video data stream comprises a plurality of access units,

wherein the method includes decoding the video data stream to decode the video,

wherein the video data stream includes delta time information for each of two or more decoding units of an access unit of the majority of access units, wherein a decoding unit removal time for each decoding unit of the two or more decoding units of said access unit depends on an access unit removal time for the access unit and depends on the delta time information for said decoding unit, wherein the method comprises decoding the video data stream using the delta time information for each of two or more decoding units of said access unit.

251. A computer program for implementing the method of claim 249 or 250 when being executed on a computer or signal processor.