[Name of Document] Specification
[Title of Invention] IMAGE PROCESSING DEVICE AND METHOD
[Technical Field]
[0001]
The present disclosure relates to an image processing
device and an image processing method, and more specifically,
to an image processing device and an image processing method,
capable of reducing a processing burden required when a stream
used at the time, of transmission or a stream used at the time
of storing data in a file is generated.
• [Background Art]
[0002]
In H.264/AVC that is one of standard specifications of
image coding system, two kinds of parameter sets called a
sequence parameter set (SPS) and a picture parameter set (PPS)
that are used for storing parameters for coding and decoding
an image are defined. The image data of each slice is
classified into VCL (Video Coding Layer) NAL (Network
Abstraction Layer) units, and these parameter sets are
classified into non-VCL NAL units . Generally, in'a case where
- a~coded~stream "that Is'coded' using the H.264/AVC system isstored
in a file including a header region and a data region,
the SPS and the PPS are inserted into the header region, and
image data is inserted into the data region.
[0003] '
In a standardization operation of HEVC (High Efficiency
Video Coding) that is a next-generation image coding system '
subsequent to H.264/AVC, introduction of an adaptation
parameter set (APS) that is a new parameter set other than
the SPS and the PPS has been proposed (see Non-Patent Documents
1 and 2 described below).
IPO p. & JUKI 27 -6 4-.;2 Q-..1- 5 16 "- 4.5
4 •
SP350513WO00
[0004]
The APS is also one kind of parameter sets. Thus,
according to an existing technique, similarly to the SPSand
the PPS, the APS is also inserted into the header reqion of
5 a file. Examples of a file format including a header region,
and a data region include an MPEG-4 Part 14 (ISO/IEC 14496-14 :
2003, hereinafter, referred to as MP4) format and an MPEG-4
Part 15 (ISO/IEC 14496-15: 2004, AVC file) format.
[Prior Art Documents]
10 [Non-Patent Document]
[0005]
[Non-Patent Document 1] Stephan Wenger, Jill Boyce, Yu-Wen
Huang, Chia^Yang Tsai, PingWu, Ming Li, ."Adaptation Parameter
Set (APS)", JCTVC-F747r3, July 2011
15 [Non-Patent Document 2] Benjamin Bross, Woo-Jin Han,
Jens-Rainer Ohm, Gary J. Sullivan, Thomas Wiegand, "High
efficiency video coding (HEVC) text specification draft 6",
JCTVC-H1003 ver20, February 2012
[Summary of Invention]
20 [Problems to be Solved by the Invention]
[0006]
However, in a case where a stream used for transmission
or a stream used for storing data-in a file is generated, there
is a format in which data such as start codes and.a filler
25 data is not allowed to be included in the stream. Thus, when
the data of a stream is read, it causes the processing load
to be increased.
[0007]
The present disclosure is configured in consideration
30 of such situations and reduces the processing burden required
when a stream used at the. time of transmission or a stream
T.P© pEtHI- 2'7 - ,@ 4 - 2-.B-,i5 I-JB "- 45
5
SP350513WO00
used at the time of storing data in a file is generated.
[Means for Solving the Problemsl
[0008]
According to a first aspect of the present disclosure,
5 there is provided an image processing device including: a
setting unit that sets a startcode and filler data for a file
including a bitstream acquired by coding an image; and a control
unit that performs control of the setting unit such that the
startcode and the filler data set by the setting unit configure
10 a file with a characteristic of a parameter managing a decoder
buffer being maintained in media data of the file.
[0009]
The parameter managing the decoder buffer is a parameter
included in VUI (Video Usability Information), a parameter
15 included in buffering period SEI (Supplemental Enhancement
Information), or a parameter included in picture timing SEI.
[0010]
The setting unit may set the filler data as VCL data.
[0011]
20 According to the first aspect of the present disclosure,
there -is provided"an image processing method using an image
processing device. The image processing method includes:
• setting a startcode and filler data for a file including a
' bitstream acquired by coding an image; and performing control
-25 of the setting of the startcode and the filler data such that
the.startcode and the filler data that are set configure a
file with a characteristic of a parameter managing a decoder
buffer being maintained in media data of the file.
[0012]
30 According to a second aspect of the presentdisclosure,"
there is provided an image processing device including: a
4
6
SP350513WO00
reception unit that receives a file including a bitstream
acquired by coding an image and being controlled and set such
that a startcode and filler data configure the file with a
characteristic of a parameter managing'a decoder buffer being
5 maintained in media data of the file; and a decoding unit that
reads the startcode and the filler dat from the file received
by the reception unit and decodes the bitstream by using the
parameter managing' the decoder buffer.
[0013]
10 The parameter managing the decoder buff er is a parameter
included in VUI (Video Usability Information), a parameter
included in buffering period SEI (Supplemental Enhancement
Information), or a parameter included in picture timing SEI.
[0014]
15 The filler data is set as VCL data.
[0015]
.According to the second aspect of the present disclosure,
there is provided an image processing method using an image
processing device. The image processing method includes:
20 receiving a file including a bitstream acquired by coding an
;——" i-mage-and being-controlled and set such that a startcode and
filler data configure the file with a characteristic of a
parameter managing a decoder buffer being maintained in media
data of the file; and reading the startcode and the filler
25- dat from the received file and decoding the bitstream by using
the parameter managing the decoder buffer.
[0016]
According to athird aspect of the present disclosure,
there is provided an image processing device including: a
30 setting unit that sets a characteristic of a parameter managing
a,decoder buffer in a case where a startcode and filler data
3P® 'BBhM-£ 2,7-.9 4-2Bi.S 16:45
•7
SP350513WO00
are removed from a bitstream in media data of a file including
the bitstream acquired by coding an image; and a generation
unit that generates the file by using the characteristic set
by the setting unit.
5 [0017]
The setting unit may set an identification parameter
identifying that the characteristic of the parameter managing
the decoder buffer is. changed.
[0018],
10 The setting unit may set the identification parameter
as an optional box in a sample entry of the file.
[0019]
According to the third aspect of the present disclosure,
there is provided an image processing method using an image
15 processing device. The image processing method includes:
setting a characteristic of a parameter managing a decoder
buffer in a case where a startcode and filler data are removed
from a bitstream in media data of a file including the bitstream
acquired by coding an image; and generating the file by using
20 the set characteristic.
[002-0]--
According to a fourth aspect of the present disclosure,
there is provided an image processing device including: a
reception unit that receives a file generated using a set
25 characteristic of a parameter managing a decoder buffer that
is set in a case where a startcode and filler data are removed
from media data of a file including a bitstream acquired by
coding an image; and a decoding unit that reads the parameter
managing the decoder buffer from the file received by the
30 reception unit and decodes the bitstream by using the read
parameter.
T P Q B£t!4X" 2-7-94-2^15 ,.1B : 45
SP350513WO00
[0021]
In the file, an identification parameter identifying
that the characteristic of the parameter managing the decoder
buffer is changed is set.
5 [0022]
According to the fourth aspect of the pre sent'disclosure,
there is provided an image processing method using an image
processing device. The image processing method includes:
receiving a file generated using a set characteristic of a
10 parameter managing a decoder buffer that is set in a case where
a startcode and filler data are removed from media data of
a file including a bitstream acquired by coding an image; and
reading the parameter managing the decoder buffer from the
received file and decoding the bitstream by using the read
15 parameter.
[0023]
According to the first aspect of the present disclosure,
a startcode and filler data are set for a file including a
bitstream acquired by coding an image. Then, the setting of
20 the startcode and the filler data are controlled such that
" "•" the. startcode and' the filler data that are set configure a
file with a characteristic of a parameter managing a decoder
buffer being maintained in media data of the file.
[0024]
25 According to the secondaspect of the present disclosure,
a file including a bitstream acquired by coding an image and
being controlled and set such that a startcode and filler data
configure the file with a characteristic of a parameter
managing a decoder buffer being maintained in media data of
• 30 the file is received. Then, the startcode and the filler dat
are read from the received file, and the bitstream is decoded
TPfi pEtHI 2.7 -04- 2-0 I- 5 lg : 4 5
9
SP350513WO00
by using the parameter managing the decoder buffer.
[0025]
According to the third aspect of the present disclosure,
a characteristic of a parameter managing a decoder buffer is
5 set in a case where a startcode and filler data are removed
from a bitstream inmedia data of a file including the bitstream
acquired by coding an image. Then, the file is. generated by
using the set characteristic.
[0026]
10 According to the fourth aspect of the present disclosure,
a file generated using a set characteristic of a parameter
managing a decoder buffer that is set in a case where a startcode
and filler data are removed from media data of a file including
a bitstream acquired by coding an image is received. Then,
15 the parameter managing the decoder buffer is read from the
received file, and the bitstream is decoded by using the read
parameter.
[0027] • .
Here, the above-described image processing device may
20 be an independent device or an internal block that configures
- .- gn. £magC coding apparatus or an image decoding apparatus.
[Effects of the Invention]
• 1002 8]
According to first and third aspects of the present
25 disclosure, an image can be coded. Particularly, the
processing burden can be reduced when a stream used at the
time of transmission of a bitstream or a stream used at the
time of storing data in a file is generated.
[0029]
30 In addition, according to second and fourth aspects of
the present disclosure, an image canbe decoded. Particularly,
10
SP350513WO00
the processing burden can be reduced when a stream used at
the time of transmission or a stream used at the time of storing
data in'a.file is decoded.
[Brief Description of Drawings]
[0030]'
Fig. 1 is a block diagram that illustrates an example
of the configuration of a coding apparatus, to which the present
technology is applied, according to a first embodiment.
Fig. 2 is a block diagram that illustrates an example
of the configuration of an encoder.
Fig. 3 is a block diagram that illustrates an example
of the configuration of a.coding apparatus, to which the'present
technology is applied, according to a first embodiment.
Fig. 4 is a block diagram that illustrates an example
of the configuration of a decoder.
Fig. 5 is a diagram that illustrates an example of a
file format.
Fig. 6 is a block diagram that illustrates an example
of the configuration of a coding apparatus, to which the present
technology is.applied, according to a second embodiment.
rFl~gT~7-is a fflcxTFlfiagram €hat illustrates an example
'of the configuration of a decoding apparatus, to which the
present technology is applied, according to the second
embodiment.•
[Mode for Carrying Out the Invention]
[0031] ' '
• Hereinafter, embodiments for performing"the present
disclosure (hereinafter, referred to as embodiments) will be
described. The description will-be presented in the following
order. , "
1. First Embodiment
! 12
I SP350513WO00
generating unit 2 transmits the coded stream to a decoding
apparatus to be described later.
[0037]
[Configuration Example of Encoder]
5 Fig. 2 is a block diagram that illustrates an example
of the configuration of the encoder 1 illustrated in Fig. 1.
•\ [0038]
The encoder 1 illustrated in Fig. 2 is configured to
include: ah A/D converter 11; a screen rearrangement buffer
10 12; a calculation unit 13; an orthogonal transform unit 14;
a quantization unit 15; a lossless encoding unit 16; an
accumulation buffer 17; an inverse quantization unit 18; an
inverse orthogonal transform unit 19; an addition unit 20;
a deblocking filter 21; a frame memory 22; a switch 23; an
15 intra prediction unit 24; a motion prediction/compensation
unit 25; a predicted image selecting unit 26; and a rate control
unit 27.
[0039]
In addition, between the deblocking filter 21 and the
20 frame memory 22, an adaptive offset filter 41 and an adaptive
~ l"oop-f i-l'ter—42-are_provided^
[0040]
More specifically, the A/D converter 11 of the encoder
1 performs an A/D conversion of an image, which is configured
25 in units of frames, input as an input signal and outputs the
converted image to the screen rearrangement buffer 12 so as
to be stored therein. The screen rearrangement buffer'12
rearranges the stored image, which is configured in units of
frames in the display order, in order for coding in accordance
30 with a GOP (Group of Picture) structure and outputs the
rearranged image to the calculation unit 13, the intra
3P'B .PE.U.KI .17-94-1915 ±&:a* '
13
SP350513WO00
prediction unit 24, and the motion prediction/compensation
unit 25.
[0041]•
The calculation unit 13 performs coding by calculating
5 a difference between a predicted image supplied from the
predicted image selecting unit 26 and a current coding image
output from the screen rearrangement buffer 12'. More
specifically, the calculation unit 13 subtracts a predicted
image supplied from the predicted image selecting unit 26 from
10 a current coding image output from the screen rearrangement
buffer 12. The calculation unit 13 outputs an image acquired
as a result thereof to the orthogonal transform unit 14 as
differential information. In addition, in a case where a
predicted image is not supplied from the predicted image
15 selecting unit 26, the calculation unit 13 outputs the image
read from the screen rearrangement buffer 12 to the orthogonal
transform unit 14 as it is as differential information.
[0042.]
The orthogonal transform unit 14 performs an orthogonal
20 transform for the differential information supplied from the
•— -calculation unit'13 and supplies coefficients acquired as a
result of the orthogonal transform to the quantization unit
15.
[0043]
25 'The quantization unit 15 quantizes the coefficients
supplied from the orthogonal transform unit 14 . The quantized
coefficients are input to the lossless encoding unit 16.
[0044]
The lossless encoding unit 16 acquires information
30 (hereinafter, referred to as intra prediction mode
information) representing an optimal intra prediction mode
T p © B E != H!• 2 7 -04-2-0 i- 5 i B "- 4 5
i
14 ,
SP350513WO0Q
from the intra prediction unit 24 . In addition, the lossless
encoding unit 16 acquires information representing the optimal
inter prediction mode (hereinafter, referred to as inter
prediction mode information), a motion vector, information
5 used for specifying a reference image, and the like from the
motion prediction/compensation unit 25. Furthermore, the
lossless encoding unit 16 acquires stored flags, an index or
an offset, and type information from the adaptive offset filter
41 as offset filter information and acquires filter
10 coefficients from the adaptive loop filter 42.
'[0045]
The lossless encoding unit 16 performs lossless coding
such as a variable length coding (for example, CAVLC
(Context-Adaptive Variable Length Coding) or the like),
15 arithmetic coding (for example, CABAC (Context-Adaptive
Binary Arithmetic Coding) or the like) for the quantized
coefficients supplied from the quantization unit 15.
[.0046] .
In addition, the lossless encoding unit 16 performs
20 lossless coding of the. intra prediction .mode information or
_ the-i-n-te-r-p-redi'etion'mode-informat±on~th^"mt)t"ibn vector, Che
information specifying'a reference image, the offset filter
information, the filter coefficients, and the like as coding
information relating to coding. The lossless encoding unit
25 16 supplies the coding information and the coefficients that
are coded in a lossless manner to the accumulation buffer 17
as coded data so as to be accumulated therein. -Here, the coding
information that is coded in a lossless manner may be regarded
as header information of the coefficients that are coded in
30 a lossless manner.
[0047]
15
SP350513WO00
The accumulation buffer 17 temporarily stores the coded
data supplied from the lossless encoding unit 16 . In addition,
the accumulation buffer 17 supplies the stored coded data to
the file generating unit 2 illustrated in Fig. 1.
5 [0048]
In addition, the quantized coefficients output from the
quantization unit 15 are input also to the inverse quantization
unit 18, are inversely quantized, and then, are supplied to
the inverse orthogonal transform unit 19.
10 [0049]
The inverse orthogonal transform unit 19 performs an
inverse orthogonal transform for the coefficients supplied
from the inverse quantization unit 18 and supplies differential
information acquired as a result thereof to the addition unit
15 20.
'[0050]
The addition unit 20 adds the differential information
as a current decoding image supplied from the inverse
orthogonal transform unit 19 and the predicted image supplied
20 from the predicted image selecting unit 26 together, thereby
acquiring a locally-decoded image. Tn addition, in~a case
where a predicted image is not supplied from the predicted
image selecting unit 26, the addition unit 20 sets the
differential information supplied from the inverse orthogonal
25 transform unit 19 as a locally-decoded image. The addition
unit 20 supplies the locally-decoded image to the deblocking
filter 21 and supplies the locally-decoded image to the frame
memory 22 so as to be accumulated therein.
[0051]
30 The deblocking filter 21 filters the image, which is
locally decoded, supplied from the addition unit 20, thereby
T-PQ DE-JkHJE 27 -Q 4~ 2 © i f IS "- 45
16
SP350513WO00
removing a block distortion. The deblocking filter 21
supplies an image acquired as a result thereof to the adaptive
offset filter 41.
[0052]
• 5 • The adaptive of f set filter 41 performs an adaptive of f set
filter (SAO: Sample adaptive of f set) process of mainly removing
ringing for the image for which the adaptive deblocking filter
process has been performed by the deblocking- filter 21.
[0053]
10 Described in more detail, the adaptive offset filter
41 determines the type of the adaptive offset filter process
for each LCU (Largest Coding Unit) that is a maximal encoding
unit and acquires an offset that is used for the adaptive offset
filter process. The adaptive offset filter 41 performs the
15 adaptive offset filter process of the determined type for the
image acquired after the adaptive deblocking filter process
by using the acquired of f set. Then, the adaptive of f set filter
41 supplies the image acquired after the adaptive offset filter
process to the adaptive loop filter 42.
20 [0054]
- In addition, the adaptive offset filter 41 includes a
buffer that stores an offset. The adaptive offset filter 41
determines whether or not the offset used for the adaptive
deblocking filter process has already been stored in the buffer
25 for each LCU.
[0055]
In a case where the offset used for the adaptive
deblocking filter process is determined to have already been
stored in the buffer, the- adaptive offset filter 41 sets a
30 storage flag representing whether the offset is stored in thebuffer
to a value (here, .1) representing that the offset is
|P0 p,^k-BI- .2-7 -R4- 2-0!\5 1, g, : 4 5
17
SP350513WO00
stored in the buffer.
[0056]
• Then, the adaptive offset filter 41 supplies the storage
flag set to ".1", the index representing a storage position
5 of an offset in the buffer, and the type information
representing the type of the performed adaptive offset filter
process to the lossless encoding unit 16 for each LCU.
[0057]
On the other hand, in a case where the offset used for
10 the adaptive deblocking filter process is not stored in the
buffer yet, the adaptive offset filter 41 sequentially stores
the offset in the buffer. In addition, the adaptive offset
filter 41 sets the storage flag to a value (here, "0")
representing that the of f set is not stored in the buffer . Then,
15 the adaptive offset filter 41 supplies the storage flag set
to "0", the offset, and the type information to the lossless
encoding unit 16 for each LCU.
[0058]
The adaptive loop filter 42, for example, performs an
20 adaptive loop filter (ALF) process for the image, which is
acquired'after the "adaptive offset filter process, supplied
from the adaptive offset filter 41 for each LCU. As the
adaptive loop filter process, for example, a process using
a two-dimensional Wiener filter is used. It is apparent that
25 a filter other than the Wiener filter may be used.
[0059]
More specifically, the adaptive loop filter 42
calculates filter coefficients used for the adaptive loop
filter process for each LCU such that a difference between
30 the original image that is an image output from the screen
rearrangement buffer 12 and an image acquired after the
18
SP350513WO00
adaptive loop filter process is minimized. Then, the adaptive
loop filter 42 performs an adaptive loop filter process for
the image acquired after the adaptive offset filter process
by using the calculated filter coefficients for each LCU.
5 [0060]
The adaptive loop filter 42 supplies the image acquired
after the adaptive loop filter process to the frame memory
• 22. In addition, the adaptive loop filter 42 supplies the
filter coefficients to the lossless encoding unit 16.
10 [0061] '
Here, while the adaptive loop filter process is performed
. for each LCU, the processing unit of the.adaptive loop filterprocess
is not limited to the LCU. However, by matching the
processing units of the adaptive offset filter 41 and the
15 adaptive loop filter 42, the process can be efficiently
performed.
[0062]
The image accumulated in the frame memory 22 is output
to the intra prediction unit 24 or the motion
20 prediction/compensation unit 25 through the switch 23 as a
* reference image. - - .'.
[0063]
The intra prediction unit 2 4 performs an intra prediction
process of each of all the intra prediction modes that are
25 candidates in units of tiles and slices by using the reference
image, which has not been filtered by the deblocking filter
21, read from the frame memory 22 through the switch 23 -.'
[0064] .
In addition, the intra prediction unit 24 calculates
30 cost function values (to be described later in detail) for
all the intra prediction modes that are candidates based on
_.IP9 ftlkHJ' 27 -.©• 4 - 2B.i5 I %.: 4 5
*
19
SP350513WO00
the image read from the screen rearrangement buffer 12 and
the predicted image'generated as a result of the intra
prediction process. Then, the intra prediction unit 24
determines an intra prediction mode of which the cost function
5 value is smallest as an optimal intra prediction mode.
[0065]
The intra prediction unit 24 supplies the predicted image
generated in the optimal intra prediction mode and a
corresponding cost function value to the predicted image
10 selecting unit 26. In a case where the intra prediction unit
2 4 is notified of the selection of the predicted image generated
in the optimal intra prediction mode from the predicted image
selecting unit 26, the intra prediction unit 24 supplies the
intra prediction mode information to the lossless encoding
. 15 unit 16.
[0066]
The cost function value is also called an RD (Rate
Distortion) cost and, for example, is calculated using a
technique of one of a high complexity mode and a low complexity
20 mode as defined in a JM (Joint Model) that is reference software
in-the H.264/AVC system.
[0067]
More specifically, in a case where the high complexity
mode is employed as the technique for calculating the cost
25 function value, for all the prediction modes that are
candidates, the process up to the lossless coding is
temporarily performed, and a cost function value represented
in the following Equation (1) is calculated for each prediction
mode.
30 [0068] '
Cost(Mode) = D + X-R ••• (1)
ft
20
SP350513WO00-
[0069]
Here, D is a difference between the original image and
a decoded image, R is a generated coding amount including up
to the coefficients of the orthogonal transform, and X is a
5 Lagrange multiplier that is given as a function of the
quantization parameter Q'P.
[0070]
On the other hand, in a case where the low complexity
mode is employed as the technique for calculating- the cost
10 function value, for each of all the prediction modes that are
candidates, a decoded image is generated, and a header bit
such as information'representing a prediction mode is
calculated, and the cost function value represented in the
following Equation (2) is calculated for each predictionmode .
15 [0071]
Cost(Mode) = D + QPtoQuant (QP)-Header_Bit ••• (2)
[0072]
Here, Dis a difference (distortion) between the original
image and a decoded image, Header_Bit is a header bit for a
20 predictionmode, and QPtoQuant is a function given as a function
of the quantization parameter QP.
[0073]
In the low complexity mode, only decoded images may be
generated for all the prediction modes, and lossless coding
25 does not need to be performed, whereby the calculation amount
is reduced.
[0074]
The motion prediction/compensation unit 25 performs a
motion prediction/compensation processes in each of all the
30 inter prediction modes-that are candidates in units of tile's
and.slices. More specifically, the motion
• IPO BftHJ. .2-7-JP4-.3©>5 .m'-&-3
21
SP350513WO00
prediction/compensation unit 25 detects a motion vector of
each of all the inter prediction modes that are candidates
in units of tiles and slices based on the image supplied from
the screen rearrangement buffer 12 and the filtered reference
image read from the frame memory 22 through the switch 23.
Then, the motion prediction/compensation unit 25 performs a
compensation process for the reference image based on the
motion vector in units of tiles and slices, thereby generating
a predicted image.
[0075]
At this time, the motion prediction/compensation unit
25 calculates a cost function value for each of all the'inter
prediction modes that are candidates based on the image
supplied from the screen rearrangement buffer 12 and the
predicted image and determines an inter prediction mode of
which the cost function value is smallest as the optimal inter
prediction mode. Then, the motion prediction/compensation
unit 25 supplies the cost function value of the optimal inter
prediction mode and a corresponding predicted image to the
predicted image selecting unit 26. In addition, in a case
where the motion prediction/compensation unit 25 is notified
of the selection of the predicted image generated in the optimal
inter prediction mode from the predicted image selecting unit
26, the motion prediction/compensation unit 25 outputs the
inter prediction mode information, a corresponding motion
vector, the information specifying the reference image, and
the like to the lossless encoding unit 16.•
[0076]
The predicted image selecting unit 26 determines one
of the optimal•intra prediction mode and the optimal inter
prediction mode that has a smaller cost function value as the
W,T 2. 7 - ft A - 1 U 1. R % R : & 5
22
. SP350513WO00
optimal prediction mode based on the cost function values
supplied from the intra prediction unit 24 and the motion
prediction/compensation unit 25. Then, the predicted image
selecting unit 26 supplies the predicted image of the optimal
5 prediction mode to the calculation unit 13 and the addition
unit 20. In addition, the predicted image selecting unit 26
notifies the intra prediction unit 24 or the motion
prediction/compensation unit 25 of the selection of the
predicted image of the optimal prediction mode.
10- [0077]
The rate control unit 27 performs control of the
quantization operation performed by the quantization unit 15
based on the coded data accumulated in the accumulation buffer
17 such that an overflow or an underflow does not occur.
15 [0078]
[Background and Problem]
In an AVC file format defined in ISO/IEC 14496-15, it
is not allowed to include the start codes and the filler data.
in a bitstream (Elementary Stream) . However, the processing
20 load at the time of handling data of an ISO base media file
'—format-, -an-MPEG-2-TS-(.Transport Sjtream) , and the like is caused
to increase.
[0079]
In addition, in order to generate (convert) an MPEG-2
25 TS from the bitstream (Elementary Stream), there are cases
' where the value of a parameter managing the decoder buffer
becomes different by removing the start codes and the filler
data (dummy data used for adjusting the data size).
Accordingly, for example', it is necessary to reset buffering
30 period SEI (Supplemental enhancement information),'which
leads to an increase in the processing load at" the time of
SPS .BEtHI 27-04-2915 16:45
*
23
SP350513WO00
converting the bitstream.
[0080]
[Operation of File Generating Unit]
Thus, in a case where a stream used at.the time of
5 transmission or a stream used at the time of storing data in
a file is generated, the file generating unit 2 sets startcode
and filler data for a file including a bitstream acquired by
coding an image and performs controls of the setting of the
startcode and the filler data such that the startcode and the
10 filler data, which have been set, configure a file with the
characteristic of the parameter managing the decoder buffer
being maintained in media data of the file.
[0081]
In addition, in a case where the startcode and the filler
15 data are removed f rommedia data of a file including a bitstream
acquired by coding an image, the file generating unit 2 sets
a characteristic of the parameter managing the decoder buffer
and generates a file by using the set-.characteristic.
[0082]
20 Here, the parameter managing the decoder buffer is an
HRD'pa'ramete'r (Hypc7the"fi"cari~Re"ference Decoder Parameter) , £n
other words, a parameter that manages a virtual reference
decoder. This parameter that manages the decoder buffer is
a parameter included in the VUI, a parameter included in the
25 buffering period SEI, or a parameter included in the picture
timing SEI.-
[0083],
By configuring as above, in a case where a stream at
the time of transmission or at the time of.storing data in
30 a file is generated (converted) , the start codes and the filler
data do not need to be reset, and the processing burden can
t P Q JBE.kH.1 37-£4-£&>5 1, & : 4 5
24
SP350513WO00
be reduced..
[0084]
In addition, the filler data may be responded as below.
In' other words, in the AVC or HEVC system, the filler data
5 is non-VCL to be in a table of NAL unit, type codes and NAL
unit type classes. More specifically, when nal_unit_type is
31, Name of nal_unit_type is FD_NUT, and Content of NAL unit
and RBSP syntax structure is Filler data and f iller_data_rbsp () ,
NAL nuit type class is non-VCL. By changing this non-VCL into
10. VCL data, it can be suppressed that the filler data is removed
when the bitstream is'converted into a transmission stream.
[0085]
More specifically, the file generating unit 2 performs
setting of the startcode and the filler data as below.
15 [0086]
• Startcode and filler data
Therefore, this contribution proposes to allow to
include startcode and filler data in the elementary streams
for new brands ony.
20 In the original version of ISO/IEC 14496-15, SPS/PPS
— can-'t-be— included—in^-the-elementary-st ream-—In—ISO/IEC
14496-16 PDAM2, it is allowed to include SPS/PPS now. If it
is allowed to include startcode and filler data in the
elementary stream, then the burden for bitstream converter
25 is significantly reduced.. Rewriting.of HRD parameters, e.g..
buffering period SEI and picture timing SEI, is not necessary
in this case. . . '•
In order to ensure backward compatibility, this change
should be applied to new brands, (new AVC brand and HEVC)-
30 [0087]
• Indication that HRD parameters are wrong
XPft E*Efc~HX. 27 - ©4 - 2-S4 5 1R : 4 5
25
SP350513WO00
When startcode, filler data, etc were removed from the
elementary stream to store ISOMFF, HRD parameters, e.g.
buffering period SEI, picture timing SEI, etc are not correct.
However, decoder cannot know if HRD parameters in the
5 elementary streamis correct. There shouldbe an idto indicate
the HRD parameters in the elementary stream maybe wrong.
[0088]
• In HEVC or AVC change filler data as VCL data
Currently in HEVC and AVC,- filler data is categorized
10 as non-VCL. That mean the bit for filler data is counted as
• non-VCL (header information) . However, if we specify filler
dataas VCLdata, thebit for filler is countedas videobitstream
itself. In this case, filler data is not necessary to remove
from the elementary stream and HRD parameters are not changed.
15 The table for HEVC is attached.
[0089]
Furthermore, in a case where the parameter that manages
the decoder buffer is different (incorrect), the decoder cannot
determine whether the parameter managing the decoder buffer
20 is the same (correct). Thus, in a case where the startcode
and the filler data are removed from the bitstream, an
identification parameter used for identifying whether the
characteristic of the parameter managing the decoder buffer
is changed is set. As an example, a case will be represented
25 in which the identification parameter is stored in an optional
box of the file format.
For example, in case of the AVC, such id can be defined
as an optional box in AVCSampleEntry as follows. (The same
approach can be applied to other AVC sample entries and
30 HEVCSampleEntry.)
AVCSampleEntry() extends VisualSampleEntry(type') {
JrPB BEIrHJE 2-7 - B4-.2-B-.l-5 1 6 : 4 .5
*
26
SP350513WO00
//type' is 'avcl' or 'avc3'
AVCConf'igurationBox config;
MPEG4BitRateBox(); //Optional
MPEG4ExtensiohDescriptorsBox (); //Optional
5 HRDConformanceBox(); //Optional
}
Add HRDConformanceBox into AVCSampleEntry and
AVC2SVVSampleEntry in 5.4.2.1 ' . '
Add HRDConformanceBox into AVCSVCSampleEntry,
10 AVC2SVCSampleEntry and SVCSampleEntry in 6.5.3.1.
Add HRDConformanceBox into AVCMVCSampleEntry,
AVC2MVCSampleEntry and MVCSampleEntry in' section7.6.3.3<
Add HRDConformanceBox into HEVC SampleEntry in section
8.4.1.1.
15 HRDConformanceBox can be defined as follows.
class HRDConformanceBox extends Box('hrdc').{
'HRDConformanceData() HRDConformance;
} •
aligned(8) class HRDConformanceData {
20 unsigned int(7) reserved =0;
~: uns"igned*int (1)- HRDConformanceFlag; . ......
unsigned int (24) reserved;
Box[] any_box;//Optional
' • _ } • '
.25 [0090],
The identification parameter, for example, is set as
below.
HRDConformanceFlag is the identifier if HRD
characteristics was changedby removing startcode, filler data,
30 etc from the elementary stream. When HRDConformanceFlag is
equal to 1, the HRD parameters, e.g. buffering period SEI,
IP.© BEtMl ' 37 -B4 -.3.®if is : 45
4
27
SP350513WO00
picture timing SEI, was changed by removing startcode, filler
data, etc from the elementary stream.
[0091]
[Configuration Example of Decoding Apparatus according
5 to First Embodiment]
Fig. 3 is a block diagram that illustrates an example
of the configuration of a decoding apparatus, which decodes
a coded stream transmitted from the coding apparatus
illustrated in Fig. 1, according to the first embodiment as
10 an image processing device to which the present technology
is applied.
[0092]
The decoding apparatus illustrated in Fig. 3 is
configured by a file reading unit 51 and a decoder 52.
15 [0093]
The file reading unit 51 of the decoding apparatus
receives a coded stream transmitted from the coding apparatus
illustrated in Fig. 1 and reads an SPS, a PPS, VUI, SEI, coded
data, and the like from the received coded stream. The file
20 reading unit 51 supplies the coded data to the decoder 52:
In addition, the file reading unit 51 supplies the SPS, the
PPS, the VUI, the SEI, and the like to the decoder 52 as is
necessary.
[0094]
25 More specifically, the file reading unit 51 receives
a file that is controlled to be set such that the startcode
and the filler data configure the file with the characteristic
of the parameter managing the decoder buffer being maintained
in media data of the file that includes a bitstream acquired
30 by coding an image, reads the startcode and the filler dat
from the received file, and causes the decoder 52 to decode
IP a DEtr.HJ 1?-|4-J0t5 ig:zii;
28
SP350513WO00
the bitstream by using the parameter managing the decoder
buffer.
[0095]
In a case where the startcode and the filler data are
5 removed from the media data of the file including the bitstream
acquired by coding'an image, the file reading unit 51 receives
a file that is generated by using a characteristic of the
parameter managing the decoder buffer that has been set, reads
the parameter managing the decoder buffer from- the received
10 file, and causes the decoder 52 to decode the bitstream'by
using the read parameter.
[0096]
The decoder 52 decodes the coded data supplied from the
file reading unit 51 by using the HEVC system by referring
15 to the SPS, the PPS, the VUI, the SEI, and the like supplied
from the file reading unit 51 as is necessary. The decoder
52 supplies an image acquired as a result of the decoding process
to a later stage as an output signal.'
[0097]
20 [Configuration Example of Decoding Unit]
Fig. 4'is a block diaqram that illustrates an example
of the configuration of the decoder 52 of Fig. 3.
[0098]
The decoder 52 illustrated in Fig. 4 is.configured by:
25 an accumulation buffer 101; a lossless decoding unit 102; an
inverse quantization unit 103; an inverse orthogonal transform
unit 104; an addition unit 105; a deblocking filter 106; a
screen rearrangement buffer 107; a D/A converter 108; a frame
memory 109; a switch 110; an intra prediction unit 111; amotion
30 . compensation unit 112; and a switch 113.
[0099]
2P.9. B.i:.kiil ..i.j -B4-jfi £ i & ;«
«
29
SP350513WO00
In addition, between the deblocking filter 106 and the
screen rearrangement buffer 107 and the frame memory 109, an
adaptive offset filter 141 and an adaptive loop filter 142
are provided.
5 [0100] • •
The 'accumulation buffer 101 of the decoding unit -53
receives the coded data from the file reading unit 51
illustrated in Fig . 3 and accumulates the received coded data .
The accumulation buffer 101 supplies the accumulated coded
10 data to the lossless decoding unit 102.
[0101]
The lossless decoding unit 102 performs lossless
decoding such as variable-length decoding or arithmetic
decoding for the coded data supplied from the accumulation
15 buffer 101, thereby acquiring quantized coefficients and
coding information. The lossless decoding unit 102 supplies
the quantized coefficients to the inverse quantization unit
103. In addition, the lossless decoding unit 102 supplies
the intra prediction mode information and the like as coding
20 information to the intra prediction unit 111 and supplies the
'mot-ion-vectorr'the'i'nformation'used'for spec~i"fying a reference
image, the inter prediction mode information, and the like
to the motion compensation unit 112. Furthermore, the
•lossless decoding unit 102 supplies the intra prediction mode
25 information or the inter prediction mode information as coding
information to the switch 113.
[0102]
The lossless decoding unit 102 supplies offset filter
information as coding information to the adaptive offset filter
30 141 and supplies the filter coefficients to the adaptive loop
filter 142.
JPQ D.PtrH-1: 2-7 " 0 4 - .2- ©Jr 5 _i.6- '- §3
/
30
SP350513WO00
[0103]
The inverse quantization unit 103, the inverse
orthogonal transform unit 104, the addition unit 105, the
deblocking filter 106, the frame memory 109, the switch 110,
5 the intra prediction unit 111, and the motion compensation
unit 112 respectively perform processes similar to those of
the inverse quantization unit 18, the inverse orthogonal
transform unit 19, the addition unit 20, the deblocking filter
21, the frame memory 22, the switch 23, the intra prediction
10 unit 24, and the motion prediction/compensation unit 25
' illustrated in Fig. 2, whereby an image is decoded.
[0104]
More specifically, the inverse quantization unit 103
performs inverse quantization of quantized coefficients
15 supplied from the lossless decoding unit 102 and supplies
coefficients acquired as a result thereof to the inverse
orthogonal transform unit 104.
[0105]
The inverse orthogonal transform unit 104 performs an
20 inverse orthogonal transform for the coefficients supplied
_ . —from—the -inverse-quanti"za"don-unit~r03—ahd supplies
differential information acquired as a result thereof to the
addition unit 105.
[0106] ' .
25 The addition unit 105 adds the differential information
as a current decoding image supplied from the inverse '
orthogonal transform unit 104 and a predicted image supplied
from the switch 113 together, thereby performing a decoding
process. The addition unit 105 supplies an image acquired
30 as a result of the decoding process to the deblocking filter
106 and the frame memory 109. In addition, in a case where
T- P B B E k- B.l 2 1 ~ .P 4 - ..# ® 1- 5 _i B "• 45
31
' SP350513WO00
a predicted image is not supplied from the switch 113, the
addition unit 105 supplies an image that is the differential
information supplied from the inverse orthogonal transform
unit 104 as an image acquired as a result of the decoding process
5 to the deblocking filter 106 and supplies the image to the
frame memory 109 so as to be accumulated therein.
[0107]
The deblocking filter 106 filters an image supplied from
the addition unit 105, thereby removing a block distortion.
10- The deblocking filter 106 supplies an image acquired as a result
thereof to the adaptive offset filter 141.
[0108]
The adaptive offset filter 141 includes a buffer that
sequentially stores offsets supplied from the lossless
15 decoding unit 102. In addition, the adaptive offset filter
141 performs an adaptive offset filter process for an image
acquired after the adaptive deblocking filter process
performed by the deblocking filter 106 for each LCU based on
the, offset filter information supplied from the lossless
20 decoding unit 102.
. - [0109]-- . . -- - -
More specifically, in a case where the storage flag
included in the offset filter information is "0", the adaptive
offset filter 141 performs an adaptive offset filter process
25 of a type that is represented by the type information by using
an offset included in the offset filter information for the
image acquired after the deblocking filter process in units
of LCUs.
[0110]
30 On the other hand, in a case where the storage flag
included in the offset filter information is "1", the adaptive
T. p ft ft £ L- H1: .2- 7 - 9 4 - 2 9 i 5 jfe & '• 4 5
32
SP350513WO00
offset filter 141, for the image acquired after the deblocking
filter process in units of LCUs, reads an offset that is stored
at a position' represented by an index included in the offset
filter information. Then, the adaptive offset filter 141
5 performs an adaptive offset filter process of a type that is
represented by the type information by using the read offset.
The adaptive offset filter 141 supplies an image acquired after
the adaptive offset filter process to the adaptive loop filter
142. • ••
10 [0111]
The adaptive loop filter 142 performs an adaptive loop
filter process for the image supplied from the adaptive offset
filter 141 for each LCU by using a filter coefficient supplied
from the lossless decoding unit 102 . The adaptive loop filter
15 142 supplies-an image acquired as a result thereof to the frame
memory 109 and the screen rearrangement buffer 107.
[0112]
The image accumulated in the frame memory 109 is read
as a reference image through the switch 110 and is supplied
20 to the motion compensation unit 112 or the intra prediction
unit 111.
[0113]
The screen rearrangement buffer 107 stores the image
supplied from the deblocking filter 106 in units of frames.
25 The screen rearrangement buffer 107 rearranges the stored image,
which is configured in units of frames, arranged in the coding
order in the original display order and supplies the rearranged
image to the D/A converter 108.
[0114]
30 The D/A converter 108 performs a D/A conversion of the
image configured in units of frames supplied from the screen
IPO Q p, u n I; 2- 7 -04- 2 @.i 5- ..i © *- #5
33
SP350513WO00
rearrangement buffer 107 and outputs the converted image to
a later stage not illustrated in the figure as an output signal.
[0115]
The intra prediction unit 111 performs an intra
5 prediction process of an intra prediction mode represented
by the intra prediction mode information that is supplied from
the lossless decoding unit 102 by using the reference image,
which has not been filtered by the deblocking filter 106, read
from the frame memory 109 through the switch 110 in units of
10 tiles and slices. The intra prediction unit 111 supplies a
predicted image generated as a result thereof to the switch
113.
[0116]
The motion compensation unit 112 reads the reference
15 image, which has been filtered by the deblocking filter 106,
from the frame memory 109 through the switch 110 based on the
information used for specifying a reference image that is
supplied from the lossless decoding unit 102 in units of tiles
and slices. The motion compensation unit 112 performs amotion
20 ' compensation process of an optimal inter prediction mode
-—represented by-the-inter'predi'cti'on"mode-i'h~forma"t'i"on-by using
the motion vector and the reference image. The motion
compensation unit 112 supplies a predicted image generated
as a result thereof to the switch 113.
25 [0117]
In a case where the intra prediction mode information
is supplied from the lossless decoding unit 102, the switch
113 supplies the predicted image supplied from the intra
prediction unit 111 to the-addition unit 105. On the other
30 hand, in a case where the inter prediction mode information
is supplied from the lossless decoding unit 102, the switch
m
34
SP350513WO00
113 supplies the predicted image supplied from the motion
compensation unit 112 to the addition unit 105.
[0118].
<2. Second Embodiment>
5 In the present technology, for example, the APS may be
handled not as a non-VCL NAL' unit but as a VCL NAL unit. When
the APS is defined as a VCL NAL unit, the APS, for example,
in the MP4 format, is inserted into not a moovbox corresponding'
to the header region but an mdat box corresponding to the data
10 region. Then, the decoder sequentially acquires the APS from
the mdat box without storing the APS in a parameter memory.
[0119] ;
[Example.of File Format]
Fig. 5 is an explanatory diagram that illustrates an
15 example of a file format. By referring to Fig. 5, a video
file 151 is generated according to the MP4 format and includes
a moov box 161 and an mdat box 165. For.the simplification
of description, boxes not directly relating to a technology
that relates to the present disclosure are not illustrated.
20 [0120]
The moov box 161 is a box that corresponds to a header
.region of the video file 151. The moov box 161 includes an
stblboxl62. The stbl box 162 is a container box that includes
boxes for various kinds of header information such as an stsd
25 box 163 and an stsc box 164. The stsd box 163 is a box that
maintains a parameter relating to actual data stored in the
mdat box 165. The stsd box 163 includes an hevl box 163a.
In the hevl box 163a, data that is classified into a non-VCL
NAL unit is stored. The mdat box 165 is a box that corresponds
30 to the data region of the video file 151. The mdat box 165
includes, one or more chunks 165a. In the chunk of image data
m
35
SP350513WO00
coded using the HEVC system, data classified into a VCL NAL
unit is stored.
[0121]
In the example illustrated in Fig. 5, the chunk 165a
5 disposed inside the mdat box 165, in addition to the image
data coded using the HEVC system, includes one or more APS's.
used when each image is decoded. The hevl box 163a includes
the SPS and the PPS but does not include the APS . By classifying
. 'the APS as a VCL NAL unit, a file format in which an APS is
10 inserted between coded image data inside the chunk included
in the data region can be formed.
[0122]
[Configuration Example of Encoder]
Fig. 6 is a block diagram that illustrates an example
15 of the configuration of an image coding apparatus 201 according
to an embodiment for generating the video file illustrated
in Fig. 5. As illustrated in Fig . 6, the image coding apparatus
201 includes: an encoding unit 211; a VCL buff er 212 ; a non-VCL
buffer 213; a file generating unit 214; and a control unit'
20 215.
[0123]
The encoding unit 211 illustrated in Fig. 6 corresponds
to the encoder 1 illustrated in Fig. 1. In addition, the VCL
buffer 212, the non-VCL buffer 213, the file generating unit
25 214, and the control unit 215 illustrated in Fig. 6 correspond
to the file generating unit 2 illustrated in Fig. 1.
[0124]
The encoding unit 211 is an encoder that operates
according to the HEVC system. The encoding unit 211
30 .sequentially acquires images to be coded from a video source
such as a camera or a television tuner connected to the image
TPO RPkHI 27 -04-2-S.1-5 >£ "• 4.5
*
36
SP350513WO00
coding apparatus 201. Then, the.encoding unit 211 codes the
acquired images by using parameters included in the SPS, the
PPS, and the APS, thereby generating a bitstream of image data .
In addition, the encoding unit 211 generates an SPS and a PPS
5 as a non-VCL NAL unit. On the other hand, the encoding unit
211 generates the AP'S and the bitstream of the image data as
a VCL NAL unit. The encoding unit 211 outputs the APS and
the bitstream of the image data to the file generating unit
214 through the VCL buffer 212. In addition, the encoding
10 unit 211 outputs the SPS and the PPS to the file generating
unit 214 through the non-VCL buffer 213. The VCL buffer 212
buffers the VCL NAL unit. The non-VCL buffer 213 buffers the
non-VCL NAL unit. The file generating unit 214 generates a
video file 151 that stores a series of coded image data. More
15 specifically, the file generating unit 214 inserts the APS
and the bitstream of the image data into a data region (for^
example, the mdat box) of the video file 151 in decoding order
as a VCL NAL unit. In addition, the file generating unit 214
inserts the SPS and the PPS into a header region (for example,
20 the moov box) of the video file 151 as a non-VCL NAL unit.
- The control unit 215 controls the coding process performed
by the image coding apparatus 201.
[0125]
In addition, the control unit 215 may control the
25 generation of a coded stream by using a virtual decoder model
called an HRD (Hypothetical Reference Decoder: virtual'
reference decoder) such that the buffer of the decoder does
not fail. In the HEVC system, as conformance points (check
points for standard conformance) to be satisfied by a coded
30 ' stream, two kinds of conformance points including Type 1 and
Type 2 are defined. The conformance point of Type 1 is applied
IPft ,p.Et-H;l 27-J9 4 - 2 0 i 5 l g : 45
37
SP350513WO00
to a VCL NAL unit and a filter data NAL unit but is not applied
to a non-VCL NAL unit. The conformance point of Type 2 is
applied to a VCL NAL unit, a filter data NAL unit, and a non-VCL
NAL unit. In this embodiment, the APS is defined not as the
non-VCL NAL unit but as the VCL NAL unit. Thus, the control
unit 215 may perform control of the generation of a coded stream
so that not only the bitstream of the image data but also the
APS satisfies the conformance point of Type 1.
[0126] .
[Configuration Example of Decoder]
Fig. 7 is a block diagram that illustrates an example
of the configuration of an image decoding apparatus according
to an embodiment for decoding an image from the video file
illustrated in Fig. 4. As illustrated in Fig. 7, the image
decoding apparatus 251 includes: a VCL buffer 261; a non-VCL
buffer 262, a parameter memory 263; a decoding unit 264; and
a control unit 265.
[0127]
Here, the VCL buffer 261, the non-VCL buffer 262, the
parameter memory 263, and the control unit 265 illustrated
in-Fig-r—7-correspond~to~the~f ile reading unit 51 illustrated
in Fig. 3. In addition, the decoding unit 264 illustrated
in Fig. 7 corresponds to the decoder 52 illustrated in Fig.
3.
[0128]
The VCL buffer 261 buffers a bitstream of image data
and an APS read from the data region (for example, the mdat
box) of a file . The non-VCL buffer 262 buffers parameter sets
read from the header region (for example, the moov box) of
a file. The parameter memory 263 stores parameter sets
disposed within the header region of the file acquired through
% a 7 - e 4 - 2Bi-5 IB "• 4 5
• 38
SP350513WO00
the non-VCL buffer 262 altogether. The decoding unit 264 is
a decoder that operates according to the HEVC system. The
decoding unit 264 decodes images from bitstreams that are
sequentially acquired from the data region of the file through
5 the VCL buffer 261. When an image is decoded, the decoding
unit 2.64 uses parameters disposed inside the APS that are
sequentially acquired from the data region of the file in
addition to parameters disposed inside the SPS and the PPS
stored in the parameter memory 263. The control unit 265
10 controls the decoding process performed by the image decoding
apparatus 251.
[0129]
In addition, the file format is not limited to the MP4
file format or the AVC file format. In a case where an object
15 and an advantage according to the present technology are the
same, the present technology can be similarly applied to
another file format, a stream used at the time of transmission,
. or a stream used at the time of being stored in a file.
[0130]
.20 In addition, the present disclosure, for example, may
be appl-i-ed to--an-image-codi'ng"appa'ra~tus and an image decoding
apparatus that aire used when image information (bitstream)
compressed using an orthogonal transform such as a discrete
cosine transform andmotion compensation, like the HEVC system
25 or the like, is received through a network medium such as
. satellite broadcast, cable television, the Internet, or a
mobile phone. Furthermore, the present disclosure may be
applied to an image coding apparatus and an image decoding
apparatus that are used when information is processed on a
30 storage medium such as an optical disc, a magnetic disk, or
a flash memory.
XP @ P-E'!= H I - 2- 7 - ft «# ~ 2-0-.1.5 J-".& : 4.^
*
39
SP350513WO00
[0131] .
A series of the processes described above can be
performed either by hardware or by software. In .a case where
the series of the processes is performedby software, a program
5 configuring the software is installed to a computer. Here,
Lhe computer includes a computer that is built in dedicated
hardware, a computer such as a general-purpose personal
computer that can execute various functions by installing
various programs thereto, and the like.
10 [0132]
In addition, the program executed by the computer may
be a program that executes the processes in a time series along
the sequence described in this specification or a program that
executes the processes in a parallel manner or at necessary
15 timing such as at the timing of being called.
'[0133]
Furthermore, in this specification, a step describing
the program recorded on a recording medium includes not only
a process performed in a time series along the described
20 sequence but also a process that is performed in.a parallel
manner-or-an—individual~manner"without-being necessarily
processed in a time series.
[0134]
In addition, in this specification, a system represents
25 a whole apparatus that is configured by a plurality of devices
(apparatuses).
[0135]
Furthermore, a configuration described above as one
device (or processingunit) maybe divided so as to be configured
30 as a plurality of devices (or processing units). To the
contrary, a configuration described above as a plurality of
IPO Q i t H j '2-7 - \©4~ 2.©>5 &$• "• 45
40
SP350513WO00
devices (or processing units) may be arranged to be configured
as one device (or processing unit). In addition, a
configuration that has not been described above may be added
to the configuration of each device .(or each processing.unit) .
5 As long as the overall configuration and the overall operation
of the system are substantially the same, a part of the
configuration of a specific device (or a specific processing
unit) may be configured to be included in a configuration of
another device (or another processing unit) . In other words,
, 10 the present technology is not limited to the embodiments
described above, and various changes can be made therein in
a range not departing from the concept of the present
technology.
[0136]
15 In other words, the present technology is not limited
to the embodiments described above, and various changes can
be made therein in a range not departing from the concept of
the present technology.
[0137]
20 For example, the present technology may take a
configuration of cloud computing In'which one function is
divided and processed cooperatively by a plurality of
apparatuses through -a network.
[0138]
25 In addition, each step described in each flowchart
described above may be either executed by one apparatus or
executed by a plurality of apparatuses in a shared manner.
[0139]
Furthermore, in a case where a plurality of processes
30 are included in one step, the plurality of processes included
in the one step may be either executed by one apparatus or
executed by a plurality of apparatuses in a shared manner.
[0140]
In this specification, an example has been described
in which various kinds of information such as the startcode,
the filler data, the parameter managing the decoder buffer,
and the identification data are multiplexed in a coded stream
and is transmitted from the coding side to the decoding side.
However, a technique for transmitting such information is not
limited to such a technique. For example, such information
may be transmitted or recorded as individual data associated
with a coded bitstream without being multiplexed in the coded
stream. Here, the term "associated" represents that an image
(it may be a part of an image such as a slice, block, or the
like) included in a bitstream and information corresponding
to the image are acquired with being linked to each other at
the time of decoding the image and the information. In other
words, the information may be transmitted in a transmission
line other than that for the image (or the bitstream). In
addition, the information may be recorded on a recoding medium
other than that for the image (or the bitstream) (or a different
recurding region of the same recording medium) . Furthermore,
the information and the image (or the bitstream) , for example,
may be associated with each other in units of arbitrary parts
such as multiple frames, one frame, or a part of the frame.
[0141]'
While preferred embodiments of the present disclosure
have been described in detail with reference to the
accompanying drawings, the present disclosure is not limited
to such examples . It is apparent that a person having ordinary
knowledge in the technical field of the present disclosure
can devise various changes or modifications within the scope
of the technical idea described in the claims, and, naturally,
it is understood that such changes and modifications belong
to the. technical.scope of the present disclosure.
[Description'of Reference Symbols]
[0142]
1 Encoder,
2 File generating unit
51 File reading unit
52 Decoder

Claims
[Claim 1]
An image processing device comprising:
a setting unit that sets a startcode and filler data
for a file including a bitstream acquired by coding an image;
and
a control unit that performs control of the setting unit
such that the startcode and the filler data set by the setting
unit configure a file with a characteristic of a parameter
managing a decoder buffer being maintained in media data of
the file.
[Claim 2]
The image processing device according to claim 1, wherein
the parameter managing the decoder buffer is a parameter
included in VUI (Video Usability Information), a parameter
included in buffering period SEI (Supplemental Enhancement
Information), or a parameter included in picture timing SEI.
[Claim 3]
The image processing device according to claim 1, wherein
the setting unit sets the filler data as VCL data.
-[Claim"" 4]"
An image processing method using an image processing
device, the image processing method comprising:
setting a startcode and filler data for a file including
a bitstream acquired by coding an image; and
performing control of the setting of the startcode and
the filler data such that the startcode and the filler data
that are set configure a file with a characteristic of a
parameter managing a decoder buffer being maintained in media
data of the file.
[Claim 5]
An image processing device comprising:
a reception unit that receives a" file including a
bitstream acquired by coding an image and being controlled
and set such that a startcode and filler data configure the
file with a characteristic of a parameter managing a decoder
buffer being maintained in media data of the file; and
a decoding unit that reads the startcode and the filler
dat from the file received by the reception unit and decodes '<
the bitstream by using the parameter managing the decoder
buffer. .,
[Claim 6] :
The image processing device according to claim 5, wherein
the parameter managing the decoder buffer is a parameter
included in VUI (Video Usability Information), a parameter
included in buffering period SEI (Supplemental Enhancement
Information), or a parameter included in picture timing SEI .
[Claim 7]
The image processing device according to claim 5, wherein
the filler data is- set as VCL data.-
[Claim 8]
An image processing method using an image processing
device, the image processing method comprising:
receiving a file including abitstreamacquiredby coding
an image and being controlled and set such that a startcode
and filler data configure the file with a characteristic of
a parameter managing a decoder buffer being maintained in media
data of the file; and
reading the startcode and the filler data from the
received file and decoding the bitstreamby using the parameter
managing the decoder buffer.
[Claim 9]
An image processing device comprising:
a setting unit that sets a characteristic of a parameter
managing a decoder buffer in a case where a startcode and filler
data are removed from a bitstream in media data of a file
including the bitstream acquired by coding an image; and
a generation unit-that generates the file by using the
characteristic set by the setting unit.
[Claim 10]- ' <,
The image processing device according to claim 9, wherein
the setting unit sets an identification parameter identifying
that the characteristic of the parameter managing the decb.der
buffer is changed.
[Claim 11]
The image processing device according to claim 10,
wherein the setting unit sets the identification parameter
as an optional box in a sample entry of the file.
[Claim 12]
An image processing method using an image processing
device, the image processing method comprising:
setting a characteristic of a parameter managing, a
decoder buffer in a case where a startcode and filler data
are removed from a bitstream in media data of a file including
the bitstream acquired by coding an image; and
generating the file by using the set characteristic.
[Claim 13]
An image processing device comprising:
a reception unit that receives a file'generated using
a set characteristic of a parameter managing a decoder buffer
that is set in a case where a startcode and filler data are
removed frommedia data of a file including a bitstreamacqiiired
by coding an image;, and
a decoding unit that reads the parameter managing the
decoder buffer from the file received by the reception unit
and decodes the bitstream by using the read parameter.
[Claim 14] • '
The image processing device according to claim 13,
wherein an identification parameter identifying that the
characteristic of the parameter managing the decoder buffer
is changed is set in the file.
[Claim 15]
An image processing method using an image processing
device, the image processing method comprising:
receiving a file generated using a set characteristic
of a parameter managing a decoder buffer that is set in a case
where a startcode and filler data are removed from media data
of a file including a bitstream acquired by coding an image;
and
reading the parameter managing the decoder buffer from
the received file and decoding the bitstream by using the read
parameter.