The present disclosure relates to an infonnation processing apparatns and an
information processing method, and, more particularly, to an information processing
10 apparatus and an information processing method which can recognize performance
required for decoding more accurately.
15
Background Art
(0002]
In recent years, with the aim of fmther improvement of coding efficiency
compared to that ofMPEG-4 Part10 (Advanced Video Coding, hereinafter, described
as "AVC"), joint collaboration team - video coding (JCTVC) which is a joint
standardizing body of international telecommtmication union telecommunication
standardization sector (ITU-T) and international organization for standardization I
20 international electrotechnical connmssron (ISOIIEC) has proceeded with
standardization of a coding scheme called high efficiency video coding (HEVC) (see,
for example, Non-Patent Literature 1 ).
[0003]
HEVC enables decoding of only a regiOn required to be decoded by
25 application by utilizing a tile (Tile) structure. To indicate that a tile region can be
independently decoded, in a second version and thereafter of HEVC (including MVHEVC,
SHVC, Range Ext., or the like), this is supported by Motion-constrained tile
sets SEI.
30
(0004]
By the way, as a content distribution technique utilizing hypertext transfer
protocol (HTTP), there is moving pictnre experts group - dynamic adaptive
5
SP353455WOO 1
2/75
streaming over HITP (MPEG-DASH) (see, for example, Non-Patent Literature 2).
With MPEG-DASH, bit streams of image data encoded using a coding scheme such
as HEVC described above are distributed after being formed into a predetermined
file fonnat such as, for example, an MP4 file format.
Citation List
Non-Patent Literature
[0005)
Non-Patent Literature I: Benjamin Bross, Woo-Jin Han, Jens-Rainer Ohm,
10 Gary J. Sullivan, Ye-Kui Wang, 1l10mas Wiegand, "High Efficiency Video Coding
(HEVC) text specification draft 10 (for FDIS & Last Call)'', JCTVC-L1003_v34,
Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG 16 WP 3 and
ISO!IEC JTC 1/SC 29/WG 11 12th Meeting: Geneva, CH, 14-23 Jan. 2013
Non-Patent Literature 2: MPEG-DASH (Dynamic Adaptive Streaming over
15 HTTP) (URL:http://mpeg.chiariglione.org/standards/mpeg-dash/media-presentationdescription-
and-segment-fonnats/text-isoiec-23009-12012-dam-l)
Summary of Invention
Technical Problem
20 [0006)
However, both in a bit stream and in a file format, only a value in the whole
stream or a value in units of a layer (Layer) are defined as a level (Level) which is
used as a reference for determining whether or not a decoder can decode a stream or
infonnation relating to capacity of a buffer (Buffer).
25 [0007)
Therefore, also in application for decoding only part of the whole image,
whether or not decoding is possible is determined ass1m1ing load in the case where
the whole screen is decoded, which may involve a risk that a decoder with an
unnecessarily high level (Level) is required. Further, there is also a risk that
30 application which can be distributed may be unnecessarily limited.
[0008)
5
SP353455W001
3/75
The present disclosure has been made in view of such circumstances, and is
intended to enable recognition of performance required for decoding more accurately.
Solution to Problem
[0009]
According to an aspect of the present teclmology, there is provided an
information processing apparatus including: a file generating tmit configured to
generate a file of an MP4 file fonnat, in which information specifying a plurality of
NAL units corresponding to a partial image which can be independently decoded in a
whole image is stored in moov and the encoded partial image is stored in mdat; and a
10 translllitting tmit configured to transmit the file generated by the file generating tmit.
[0010]
The information specifying the plurality of NAL tmits can include
information indicating a group of the plurality of NAL tmits corresponding to the
partial image.
15 [0011]
20
The infonnation specifying the plurality of NAL tmits can include
information indicating the number of the plurality ofNAL tmits corresponding to the
partial image.
[0012]
The information specifying the plurality of NAL tmits can include
infonnation indicating a first NAL unit among the NAL units corresponding to the
partial image.
[0013]
The infonnation specifying the plurality of NAL units can include
25 information indicating all the NAL units corresponding to the pariial image.
[0014]
30
The information specifying the plurality of NAL units can be defined using
VisualSampleGroupEntry in the moov.
[0015]
In the file, a plurality of pieces of information for mapping the plurality of
NAL units can be further stored.
SP353155W001
4/75
(0016]
The partial image can be Tile in high efficiency video coding (HEVC).
(0017]
According to an aspect of the present technology, there is provided an
5 information processing method including: generating a file of an MP4 file format, in
which information specifYing a plurality of NAL units corresponding to a partial
image which can be independently decoded in a whole image is stored in moov and
the encoded partial image is stored in mdat; and transmitting the generated file.
(0018]
10 According to another aspect of the present technology, there is provided an
information processing apparatus including: a file reproducing unit configured to
reproduce a file of an MP4 file format, in which information specifying a plurality of
NAL units corresponding to a pmtial image which can be independently decoded in a
whole image is stored in moov and the encoded partial image is stored in mdat.
15 (0019]
20
The information specifying the plurality of NAL units can include
information indicating a group of the plurality of NAL units corresponding to the
partial image.
(0020]
The information specifYing the plurality of NAL tmits can include
infonnation indicating the number of the plurality ofNAL tmits corresponding to the
partial image.
(0021]
The information specifYing the plurality of NAL tmits can include
25 information indicating a first NAL tu1it among the NAL tmits corresponding to the
partial image.
(0022]
The infom1ation specifying the plurality of NAL units can include
information indicating all the NAL units corresponding to the partial image.
30 (0023]
The information specifYing the plurality of NAL units can be defined using
VisualSampleGroupEntry in the moov.
[0024]
51~
SP353455W001
In the file, a plurality of pieces of information for mapping the plurality of
NAL units can be further stored.
5 [0025]
The paltial image can be Tile in high efficiency video coding (HEVC).
[0026]
In the file, information indicating a location of the partial image in the
whole image, information indicating a size of the partial image alld the infonnation
10 specifying the plurality of NAL tmits Call be stored in VisualSampleGroupEntry.
The file reproducing tmit Call select a pattial image which is desired to be reproduced
based on the information indicating the location of the partial image in the whole
image and the information indicating the size of the partial image, and acquire data
of the partial image which is desired to be reproduced based on the information
15 specifying the plurality ofNAL units and generate a bit strealn.
[0027]
In the file, information indicating a location of the partial image in the
whole image, information indicating a size of the partial image and the information
specifying the plurality of NAL tmits can be stored in VisuaiSampleGroupEntry.
20 The file reproducing unit can select a region which is desired to be reproduced based
on the information indicating the location of the partial image in the whole image
and the information indicating the size of the partial image, and acquire data of a
partial image corresponding to the region which is desired to be reproduced based on
the information specifying the plurality ofNAL tmits and generate a bit stream.
25 [0028]
In the file, information indicating a location of the partial image in the
whole image, information indicating a size of the partial image at1d the information
specifying the plurality of NAL units can be stored in TileRegionGroupEntry. The
file reproducing tmit can select a tile which is desired to be reproduced based on the
30 information indicating the location of the partial image in the whole image alld the
information indicating the size of the partial image, acquire a track corresponding to
5
SP353455WOOJ
6/75
the selected tile which is desired to be reproduced based on the information
specifying the plurality of NAL units, and generate a bit stream of a partial image
corresponding to the acquired track.
(0029]
In the file, information indicating a location of the paltial image in the
whole image, information indicating a size of the pattial image and the information
specifying the plurality of NAL mtits can be stored in TileRegionGroupEntry. The
file reproducing tmit can select a region which is desired to be reproduced based on
the information indicating the location of the partial image in the whole image and
10 the information indicating the size of the partial image, acqnire a plurality of tracks
corresponding to the selected region which is desired to be reproduced based on the
information specifying the plurality of NAL lUlits, and generate a bit stream of a
partial image corresponding to the acquired plurality of tracks.
(0030]
15 A decoding unit confignred to decode a bit stream of the partial tmage
reprodnced and generated by the file reproducing lUlit can be further included.
(0031]
According to another aspect of the present teclmology, there is provided an
information processing method including: reproducing a file of an IvlP4 file format,
20 in which information specifying a plurality of NAL units corresponding to a partial
image which Call be independently decoded in a whole image is stored in moov alld
the encoded partial image is stored in mdat.
(0032]
In an aspect of the present teclmology, a file of all IvlP4 file format, in which
25 information specifying a plurality of NAL units corresponding to a partial image
which Call be independently decoded in a whole image is stored in moov alld the
encoded partial image is stored in mdat, is generated, alld the generated file is
trallsmi tted.
30
(0033]
In allother aspect of the present technology, a file of an MP4 file format, in
which infonnation specifying a plurality of NAL tmits corresponding to a partial
SP353155W001
7/75
image which can be independently decoded in a whole image is stored in moov and
the encoded partial image is stored in mdat, is reproduced.
Advantageous Effects of Invention
5 [0034]
10
According to the present disclosure, it is possible to encode mid decode an
1mage. Particularly, it is possible to recognize petformance required for decoding
more accurately.
Brief Description of Drawings
[0035]
[FIG 1] FIG 1 is a diagra111 for explaining a11 exmnple of application for petforming
partial display.
[FIG 2] FIG 2 is a diagram for explaining another example of the application for
15 performing partial display.
[FIG 3] FIG 3 is a diagram for explaining an example of definition of a subsample.
[FIG 4] FIG 4 is a diagran1 for explaining outline of an MP4 file format.
[FIG 5] FIG 5 is a diagram for explaining outline of an MP4 file format.
[FIG 6] FIG 6 is a diagram for explaining outline of a11 MP4 file format.
20 [FIG 7] FIG 7 is a diagram illustrating all extension example of a sample table box.
[FIG 8] FIG 8 is a diagram illustrating an example of a subsa111ple hint information
box.
[FIG 9] FIG 9 is a diagram illustrating an example of semm1tics of independent.
[FIG I 0] FIG 10 is a diagram illustrating an example of a group of subsamples.
25 [FIG 1 1) FIG 11 is a diagram illustrating another example of the subsample hint
information box.
[FIG 12] FIG 12 is a diagram illustrating another example of the subsample hint
information box.
[FIG 13] FIG 13 is a diagram illustrating still a11other example of the subsample hint
30 infmmation box.
[FIG 14] FIG 14 is a diagram for explaining outline of a11 MP4 file format.
SP353455WOO 1
8/75
[FIG 15] FIG 15 is a diagram for explaining outline of an MP4 file format.
[FIG 16] FIG 16 is a diagram illustrating an extension example of a sample table
box.
[FIG 17] FIG 17 is a diagram illustrating an extension example of a visual sample
5 group entty.
[FIG 18] FIG 18 is a diagram illustrating an example of a subsample index.
[FIG 19] FIG 19 is a diagram illustrating another extension example of the visual
sample group entty.
[FIG 20] FIG 20 is a diagram illustrating still another extension example of the
10 visual sample group entty.
[FIG 21] FIG 21 is a block diagram illustrating an example ofmcts.
[FIG 22] FIG 22 is a diagram illustrating an example of syntax ofMCTS SEI.
[FIG 23] FIG 23 is a diagram illustrating an extension example of an MP4 file
fonnat.
15 [FIG 24] FIG 24 is a diagram illustrating an extension example of a visual sample
group entry.
[FIG 25] FIG 25 is a diagram illustrating another extension example of the visual
sample group entry.
[FIG 26] FIG 26 is a diagram illustrating still another extension example of the
20 visual sample group entry.
[FIG 27] FIG 27 is a diagram for explaining a configuration example of an MP4 file
of a tile image.
[FIG 28] FIG 28 is a diagram for explaining an HEVC tile decoder configuration
record.
25 [FIG 29] FIG 29 is a diagram illustrating an example of syntax of temporal MCTS
SEI.
[FIG 30] FIG 30 is a diagram for explaining an HEVC tile decoder configuration
record.
[FIG 31] FIG 31 is a diagram for explaining an HEVC tile decoder configuration
30 record.
[FIG 32] FIG 32 is a diagram for explaining an HEVC tile decoder configuration
SP353455W001
9/75
record.
[FIG 33) FIG 33 is a diagram for explaining an HEVC tile decoder configuration
record.
[FIG 34) FIG 34 is a diagram for explaining an HEVC tile decoder configuration
5 record.
[FIG 35) FIG 35 is a diagram for explaining an HEVC tile decoder configuration
record.
[FIG 36) FIG 36 is a diagram for explaining a configuration example of an MP4 file
of a tile image.
10 [FIG 37) FIG 37 is a diagram for explaining an HEVC tile extension box.
[FIG 38) FIG 38 is a diagram for explaining a configuration example of an MP4 file
of a tile image.
[FIG 39) FIG 39 is a block diagram illustrating an example of main components of
an image encoding apparatus.
15 [FIG 40) FIG 40 is a block diagram illustrating an example of main components of
an image decoding apparatus.
[FIG 41) FIG 41 is a flowchart for explaining an example of flow of image encoding
processmg.
[FIG 42) FIG 42 is a flowchart for explaining an example of flow of image decoding
20 processmg.
[FIG 43) FIG 43 is a flowchart for explaining an example of flow of processing of
determining whether or not reproduction is possible.
[FIG 44) FIG 44 is a flowchart for explaining an example of flow of reproduction
processmg.
25 [FIG 45) FIG 45 is a flowchart for explaining an example of the flow of the
reproduction processing.
[FIG 46) FIG 46 is a flowchatt for explaining an example of the flow of the
reproduction processing.
[FIG 47) FIG 47 is a flowchart for explaining an example of the flow of the
30 reproduction processing.
[FIG 48) FIG 48 is a flowchart for explaining an example of the flow of the
5
SP353455W001
10/75
reproduction processing.
[FIG 49] FIG 49 is a block diagram illustrating an example of main components of a
computer.
Description of Embodiments
[0036]
Embodiments for implementing the present disclosure (hereinafter, referred
to as "embodiments") will be described below in the following order:
I. First embodiment (hint information of subsample)
10 2. Second embodiment (MP4 file)
3. Third embodiment (image encoding apparatus)
4. Fourth embodiment (image decoding apparatus)
5. Fifth embodiment (computer)
[0037]
15 <1. First embodiment>

In recent years, an apparatus has been spread which digitally handles image
information, and, at that time, performs compression coding on an image by utilizing
redundancy specific to image infonnation and employing a coding scheme for
20 compressing through orthogonal transfonn such as discrete cosine transform and
motion compensation with the aim of high efficient transmission and accumulation
of information. Tlris coding scheme includes, for example, moving picture experts
group (MPEG).
25
[0038]
Particularly, MPEG2 (ISO!IEC 13818-2) which is defined as a versatile
image coding scheme, is standard which suppm1s both an interlaced scan image and
a sequential scan image, and a standard resolution image and a high resolution image.
For example, l\1PEG2 is currently widely used in a wide range of application
intended for professional use and for consumer use. Use of a MPEG2 compression
30 scheme enables assignment of a coding amount (bit rate) of 4 to 8 Mbps if, for
example, an image is an interlaced scan image with standard resolution having
SP353455WOOI
.. 11/75 --
720x480 pixels. Further, use of the MPEG2 compressiOn scheme enables
assignment of a coding amount (bit rate) of 18 to 22 Mbps if, for example, an image
is an interlaced scan image with high resolution having 1920x 1088 pixels. By this
means, it is possible to realize a high compression rate and favorable image quality.
5 (0039]
While MPEG2 is intended for high image quality coding mainly adapted to
broadcasting, MPEG2 does not support a coding scheme with a lower coding ammmt
(bit rate), that is, a higher compression rate than MPEGl. It is expected that there
will be a growing need for such a coding scheme in accordance with spread of a
10 mobile terminal in the future, and a MPEG4 coding scheme is standardized to
address this. As to an image coding scheme, the standard was approved as
international standard ofiSO/IEC 14496-2 in December, 1998.
(0040]
Further, in recent years, initially, with the rum of image coding for
15 teleconference, stru1dardization of H.26L (international te1ecolllllltmication union
telecommunication standardization sector (ITU-T)) Q6/16 VCEG (Video Coding
Expert Group)) has proceeded. It is known that while H.26L requires a more
operation amount for encoding and decoding than coding schemes in related art such
as MPEG2 and MPEG4, H.26L cru1 realize higher coding efficiency. Further,
20 currently, as part of activity for MPEG4, standardization for realizing higher coding
efficiency based on H.26L while introducing ftmctions which are not supported in
H.26L has been performed as Joint Model of Enhanced-Compression Video Coding.
(0041]
As schedule for standardization, this higher coding efficiency based on
25 H.26L was intemationally standardized in March, 2003 as H.264 ru1d MPEG-4 Part
10 (Advanced Video Coding, hereinafter, abbreviated as "AVC").
(0042]
Further, as extension of this H.264/ AVC, struldardization of coding tools
such as RGB, 4:2:2 and 4:4:4 which are required for professional use and
30 staildardization of fidelity rru1ge extension (FRExt) including 8x8DCT and a
quru1tization matrix which have been specified in MPEG-2 were completed in
SP353455W001
12175
February, 2005. By this means, a coding scheme which is capable of favorably
expressing also film noise included in a movie using H.264/AVC is realized, and
used in a wide range of application such as Blu-RayDisc (trademark).
(0043]
5 However, in recent years, there is a growing need for coding at a further
higher compression rate such as compression of an image of approximately
4000x2000 pixels which is four times of a high vision image, and distribution of a
high vision image in an environment with limited transmission capacity, such as
lntemet. Therefore, the above-described VCEG under ITU-T has continued to
10 study improvement of coding efficiency.
[0044]
Thus, with the aim of further improvement of coding efficiency compared to
that of "AVC", joint collaboration team - video coding (JCTVC) which is a joint
standardizing body of ITU-T and international organization for standardization I
15 international electrotechnical commission (ISO/IEC) is currently proceeding \vith
standardization of a coding scheme called high efficiency video coding (HEVC).
Concerning HEVC standard, Committee draft which is a draft specification was
issued in January, 2013 (see, for example, Non-Patent Literature 1).
(0045]
20
HEVC enables decoding of only a regwn required to be decoded by
application by utilizing a tile (Tile) structure. To indicate that a tile region can be
independently decoded, in a second version and thereafter of HEVC (including MVHEVC,
SHVC, Range Ext., or the like), this is supported by Motion-constrained tile
25 sets SEI.
(0046]

By the way, as a content distribution technique utilizing hypertext transfer
protocol (HTTP), there is moving picture experts group - dynamic adaptive
30 streaming over HTTP (MPEG-DASH) (see, for example, Non-Patent Literature 2).
With MPEG-DASH, bit streams of image data encoded using a coding scheme such
SP353455W001
13/75
as HEVC described above are distributed after being formed into a predetermined
file fonnat such as, for example, MP4.
[0047]
However, in content distribution such as DASH, only reproduction
5 (decoding) of the whole image is assumed, and reproduction (decoding) of a partial
image which is part of the whole image instead of the whole image is not assumed.
[0048]
More specifically, only a value in the whole stream or a value in tmits of
layer (Layer), that is, a value for the whole image is defined as a level (Level) which
10 is used as a reference for detennining whether a decoder can decode a stream and
information relating to buffer (Buffer) capacity both in a coding scheme such as
HEVC and in a file fonnat such as MP4, and there is no infonnation for reproducing
only a partial image.
[0049]
15 Titerefore, for example, even when only a partial image (partial tile) is
decoded (that is, only a partial image is reproduced) by utilizing a tile structure
supported in the coding scheme such as HEVC described above, whether or not
decoding is possible is determined assuming load in the case where the whole screen
is decoded, which may involve a risk that a decoder with an tU111ecessarily higher
20 level (Level) is required. Further, there is also a risk that application which can be
distributed may be unnecessarily limited.
[0050]

Examples of application for reproducing a partial unage include, for
25 example, the following.
[0051]
Application is assumed in which, in a system in which a server distributes
an image to a tenninal, as illustrated in, for example, FIG 1, one screen is divided
into a plurality of pieces and distributed while a display region is switched. Further,
30 as illustrated in FIG 2, application for selecting a partial region to be displayed (to be
distributed) to select an aspect ratio and resolution of an image is assumed.
SP353155W001
14/75
[0052]
In the case of application in FIG 1, the whole image can be divided into a
plurality of pieces in tmits of tile (Tile), and, at the terminal, a partial image including
one or a plurality of tiles is cut out from the whole image and displayed. A size of a
5 partial image (the mmtber of tiles) which can be displayed is detennined by, for
example, performance (processing capacity or a size of a display (display
resolution)) of the terminal, or the like. Further, a location of a pattial image to be
displayed in the whole image can be designated by a user, or the like. Therefore, a
partial image at a desired location in the whole image Catl be displayed at the
10 terminal. That is, a user of the terminal can focus on a desired portion in the whole
tmage.
[0053]
In the case of application in FIG 2, while the application in FIG 2 is
basically the same as the application in FIG 1, a tile is set so that an aspect ratio or
15 resolution of an image to be displayed can be selected, and the size of each tile is not
fixed. In a similar manner to a case in FIG I, at the terminal, a partial image
including one or a plurality of tiles is cut out from the whole image attd displayed
according to an instruction from the user, or the like. In this manner, only by
selecting a tile to be displayed, it is possible to make resolution of the image to be
20 displayed HD, make the size a cinema size or an extended size.
[0054]
The resolution which can be displayed is determined by, for example,
performance (processing capacity or a size of a display (display resolution)) of the
terminal, or the like.
25 [0055]
Because such adaptive provision (reproduction) of a pattial image according
to petfotmance of the terminal, designation by a user, or the like, is not assumed in
related art, even when a partial image which Catl be independently decoded is
decoded, whether or not decoding is possible is determined assmning load in the case
30 where the whole screen is decoded, which may involve a risk that a decoder with atl
unnecessaty high level (Level) is required. Further, there is also a risk that
SP353455W001
15/75
application which can be distributed may be unnecessarily limited.
[0056]

Therefore, subsample infommtion including hint infonnation used as a
5 reference for decoding processing of a subsample which is a partial region which can
be independently decoded is generated, a file including encoded data of image data is
generated, and the generated snbsample information is arranged in management
information of the encoded data in the file.
10
[0057]
By this means, the terminal can recogn1ze performance required for
decoding a partial region accordi1)g to the subsan1ple information (hint information)
and can determine whether or not a decoder of the terminal can perform decoding
processing of the partial region (snbsample) more accnrately. That is, it is possible
to recognize perfom1ance reqnired for decoding more accnrately. It is therefore
15 possible to select a decoder which has performance more appropriate for the image
data. By this means, it is possible to suppress occurrence of a case where a decoder
with an unnecessary high level (Level) is applied to decoding load of the image data.
Fnrther, it is possible to prevent application which can be distributed from being
um1ecessarily limited.
20 [0058]
Further, header information of encoded data (bit streams) of the partial
region (subsample) is updated to infonnation as to the partial region (subsample)
from information of the whole image. Infonnation for updating is included in a file
and transmitted. By this means, the tenninal can update the header information of
25 the bit streams to information as to the partial region (subsample) and can pass the
information to the decoder. Therefore, the decoder can dete1mine whether or not
the decoder can decode the bit streams more accurately based on the header
infom1ation.
[0059]
30
An example where the present technique IS applied to a case where an
SP353455WOOJ
16/75
encoding and decoding scheme is HEVC and a file format is MP4 will be described
below.
(0060]

5 In the following description, it is assumed that a sample ofMP4 is an access
unit (AU) of HEVC. Further, it is assumed that AU includes a plurality of tiles
(Tile). In a sample table (Sample Table), management is performed in tmits of
sample (Sample).
(0061]
10 Further, a subsample is a constituent element of the sample, and is defined
for each codec (Codec), for example, as illustrated in FIG 3. While the present
teclmique can be applied even if the subsample is any one of these, in the following
description, the present teclmique will be described using an example where a tile
(Tile) is a subsample (tile-based sub-sample).
15 (0062]

Outline of an MP4 file format will be described next. As illustrated in a
left part of FIG 4, an MP4 file (MP4 file) which conforms to MPEG-DASH includes
ftyp, moov and mdat. In moov, management infmmation is stored in a sample table
20 box (Sample Table Box (stbl) for each sample (for example, picture).
(0063]
Further, as illustrated in FIG 4, in a sample table box (Sample Table Box), a
sample description box (Sample Description Box), a time to sample box (Time To
Sample Box), a sample size box (Sample Size Box), a sample to chw1k box (Sample
25 to Chunk Box), a chtmk offset box (Clnmk Offset Box), and a subsample information
box (Subsample Infonnation Box) are provided.
(0064]
In the sample description box, information relating to codec, an image size,
or the like, is stored. For example, a parameter set (video parameter set (VPS
30 (Video Parameter Set)), a sequence parameter set (SPS (Sequence Paramter Set)), a
supplemental enhancement information (SEI (Supplemental Enhancement
SP353455W001
17/75
Information)), a picture parameter set (PPS (Picture Parameter Set)), or the like, of
bit streams of HEVC are stored in an HEVC decoder configuration record (HEVC
Decoder Configuration Record) of an HEVC sample entry (HEVC sample entry)
within the sample description box as codec (Codec) information.
5 [0065)
Further, in the time to sample box, information relating to time of the
sample is stored. In the sample size box, information relating to a size of the
sample is stored. In the sample to chunk box, information relating to a location of
data of the sample is stored. In the clnmk offset box, information relating to offset
10 of data is stored. In the subsample information box, information relating to a
subsample is stored.
[0066)
Further, as illustrated in FIG 4, data of each sample (picture) of HEVC is
stored in mdat as AV data.
15 [0067)
As illustrated in FIG 5, in the time to sample box, the sample size box, the
sample to clnmk box and the chunk offset box, access information to a sample is
stored, while, in the subsample information box, access information to a subsample is
stored. This access information to the subsample includes a size of each subsample
20 (Subsample Size) and additional information (Subsample additional information).
[0068)
For example, as illustrated in FIG 5, when the sample (picture) includes
four subsamples (tiles), in the subsample infommtion box, access information to tile
1 (Tilel) to tile 4 (Tile4) is respectively stored.
25 [0069)
A description example of the subsample information box is illustrated in FIG
6. As illustrated in FIG 6, a size of each tile (subsample_size) is described, and,
further, a reserved field (reserved=O) indicating location information of each tile is
described.
30 [0070)

SP353455W001
T817)
In the present teclmique, the above-described sample table box is extended
so as to be able to be applied in all cases of a case where a tile pattern is fixed for all
samples (such as pictures), a case where the tile pattem is variable in all the samples,
and a case where the tile pattem is fixed for a predetermined period such as, for
5 example, an IDR interval (a case where a tile pattem is variable for each
predetermined period).
(0071]

For example, in the sample table box, subsample information including hint
10 information used as a reference for decoding processing of a subsample which is a
partial region which can be independently decoded is provided as a new box of the
sample table box.
(0072]
The example is illustrated in FIG 7. As illustrated in FIG 7, in the sample
15 table box, a subsample hint information box (Subsample Hint Information Box) 11 is
newly provided.
(0073]
The subsample hint information box II is subsample information including
hint information used as a reference for decoding processing of a subsample which is
20 a partial region which can be independently decoded, and is a different box from the
subsample information box, or the like. By separating the subsample hint
information box 11 which is information for reproducing a partial image from the
subsample information box which is information for reproducing the whole image in
this manner, the whole box can be ignored (not referred to) in normal reproducing for
25 displaying the whole image, so that it is possible to facilitate control.
(0074]
A description example of the subsample hint information box II is
illustrated in a right part of FIG 7. As illustrated in this example, in the subsample
hint information box 11, information such as a hint data type (hint_data_type), a
30 san1ple count (sample_ com1t) and hint data (hint_ data) is stored.
(0075]
SP353155W001
19/75
The hint data type is information indicating a type of hint information of a
subsample stored in this box. The sample count is infonnation indicating the
number of continuous samples associated with this infommtion. The hint data is
hint information of a subsample. Information different for each type of hind data is
5 stored.
[0076]

A specific example where subsample information including hint information
used as a reference for decoding processing of a subsamp1e which is a partial region
10 which can be independently decoded is provided will be described next.
[0077]

In Example 1, the subsample hint information box 11 in which information
required for decoding (decode) is stored is used for each subs ample (tile) which can
15 be accessed by the subsample information box which is extended and defined as
described above.
[0078]
A subsample and hint information are associated with each other using a
table index (table-index) in the subsample infonnation box.
20 [0079]
25
A description example of the subsample hint infonnation box 11 is
illustrated in FIG 8. The subsample hint infonnation box 11-1 illustrated in FIG 8
is a description example of Example l.
[0080]
As illustrated in FIG 8, in the subsample hint information box 11-1, for
example, "sspf' indicating that the inf01mation is profile information for each
subsample is described as a hint data type (hint_data_type="sspf').
[0081]
Fmiher, in the subsample hint information box 11-1, for example,
30 information of any example among examples illustrated in a square 12 is described
as hint information (hint_data). For example, as in Example (A-1), information
SP353455W001
2017S
indicating a profile level required for decoding the subsample (tile) (that is, level
indicating a degree of load of decoding processing of the subsample)
(general_lebel_idc) may be described. Further, for example, as in Example (A-2),
flag information (independent) indicating whether or not the subsample (tile) can be
5 decoded independently from other subsamples (tiles) may be further described.
[0082]
FIG 9.
[0083]
An example of semantics of this independent (independent) is illustrated in
10 Further, for example, as in Example (B-1), infonnation to be replaced when
a sequence parameter set (SPS) of bit streams of the subsample (tile) is updated (that
is, header information of encoded data of the subsample) (na!UnitLength, na!Unit)
may be described. Further, for example, as in Example (B-2), flag infonnation
(independent) indicating whether or not the subsample (tile) can be decoded
15 independently from other subsamples (tiles) may be further described.
[0084]

As reproduction (provision) of a partial image, as in examples illustrated in
FIG 1 and FIG 2, there is a case where a plurality of samples are targeted.
20 Therefore, while, in Example I, hint information is stored for each subsample (tile),
in Example 2, such a plurality of subsamples are arranged into a group, and
information required for decoding the group is also provided.
[008S]
For example; in the case of A in FIG I 0, only tile 3 (Tile3) is provided as a
25 group 14-l among the whole image l3 including tile 1 (Tile1) to tile S (TileS).
30
Further, in the case of B in FIG 10, tile 2 (Tile2) to tile 4 (Tile4) are provided as a
group 14-2. Further, in the case of C in FIG 10, all the tiles in the whole image l3
(that is, tile l (Tile!) to tileS (TileS)) are provided as a group 14-3.
[0086]
Also in Example 2, the subsample hint information box 11 in which
information required for decoding (decode) is stored is used for each subsample (tile)
SP353155W001
21/75
which can be accessed by the subsample information box which is extended and
defined as described above.
[0087]
However, in the case of Example 2, information for grouping a plurality of
5 subsamples and infonnation required for decoding each group of tiles (infmmation
which is normally not required) are respectively provided as subsample hint
information boxes 11. That is, these pieces of information are stored in boxes
different from each other. By this means, it is possible to update a sequence
parameter set (SPS) of encoded data of the tile group only using the box storing the
10 infonnation required for decoding each grouped tile group.
[0088]
FIG 11 and FIG 12 illustrate description examples of the subsample hint
information boxes 11. A subsample hint infonnation box 11-2-1 illustrated in FIG
11 is a description example of information for grouping a plurality of subsamples in
15 Example 2.
[0089]
As illustrated in FIG 11, in the subsample hint information box 11-2-1, for
example, "ssgp" indicating that the information is group information of the
subsample is described as a hint data type (hint_data_type="ssgp").
20 [0090]
Further, in the subsample hint information box 11-2-1, a group index
(group index) which is identification information indicating a group to which the
subsample belongs is described. For example, in the case of the example of FIG 10,
infonnation as indicated in a square 15 is described as the group index.
25 [0091]
30
A subsample hint information box 11-2-2 illustrated in FIG 12 is a
description example of the information required for decoding each group of tiles in
Example2.
[0092]
As illustrated in FIG 12, in the subsamp1e hint information box 11-2-2, for
example, "sgpf' indicating that the information is infonnation required for decoding
SP353455W001
22/75
each tile group (hint_data_type="sgpf') is described as the hint data type.
[0093]
Further, in the subsample hint information box 11-2-2, for example,
information of any example among examples as illustrated in the square 12 is
5 described as hint information (hint_ data). That is, in this case, a level indicating a
degree of load of decoding processing of the group of subsamples and header
information of encoded data of the group of subsamples can be described as the
subsample information.
10
[0094]
It should be noted that in place of independent (independent), a motion
constrained tile set ID (motion constrained tile set ID) can be stored.
[0095]

In the case of Example 2, while a plurality of subsample hint information
15 boxes 11 are required, in Example 3, these boxes are put together in one box. In
Example 3, subsample hint information boxes 11 are provided for each group of
subsamples (tiles), and an index table of the subsamples is created in the subsample
hint infonnation boxes 11.
20
[0096]
A description example of the subsample hint information box 11 is
illustrated in FIG !3. The subsample hint information box 11-3 illustrated in FIG
13 is a description example of Example 3.
[0097]
As illustrated in FIG 13, in the subsample hint information box 11-3, for
25 example, "sgpf' indicating that the information is profile information for each group
ofsubsamples is described as a hint data type (hint_data_type="sgpf').
[0098]
Further, in the subsample hint information box 11-3, for example, any
example among the examples as illustrated in the square 12 is described as hint
30 information (hint_ data).
[0099]
SP353455W001
23/75
Fmiher, in the subsample hint information box 11-3, a subsample index
(subsample_index) which is identification information indicating a subsample
belonging to the group is described. For example, in the case of an example of FIG
10, information as indicated in a square 16 is described as this subsample index.
5 [0100]
It should be noted that an entry cOtmt (entry_cOtmt) indicates how many
times hint information changes within the sequence, and a sample cOtmt
(sample_cOtmt) indicates during how many samples (pictures) the same hint
infonnation continues.
10 [0101]
1l1at is, in the subsample hint information box 11-3, identification
information of a subsample belonging to the group and hint information of the group
can be included as the subsample information. Further, as hint infonnation of the
group, a level indicating a degree of load of decoding processing of the group or
15 header information of encoded data of the group can be included.
[0102]

While, in the above description, an example where the sample hint
information box 11 is provided has been described, the present technique in which
20 subsample information including hint information used as a reference for decoding
processing of a subsample which is a patiial region which can be independently
decoded is included in a file, is not limited to this method. For example, by
extending the sample group description box and the sample to group box in an MP4
file format, subsample information including hint infommtion used as a reference for
25 decoding processing of a subsample which is a partial region which can be
independently decoded can be included in a file.
[0103]
As illustrated in FIG 14, in a sample table box of an MP4 file, the sample
group description box (Sample Group Description Box) 21 and the sample to group
30 box (Sample To Group Box) 22 can be provided.
[0104]
SP353455W001
As illustrated in FIG 15, in the sample group description box 21, as a visual
sample group entry (VisualSampleGroupEntry) 23, information other than basic
information of a sample table such as information relating to codec (codec) and
access information is stored.
5 [0105]
In the sample to group box 22, information associating the visual sample
group entry 23 with respective samples is stored.
[0106]
By this means, it is possible to put information which would be redundant if
10 described for each sample, into one, so that it is possible to reduce an information
amount.
[0107]

In Example 4, subsample information including hint information used as a
15 reference for decoding processing of a subsample which is a partial region which can
be independently decoded is included in a file using this sample group description
box 21 and the sample to group box 22.
[0108]
The example is illustrated in FIG 16. A subsample hint infonnation box
20 11-4 illustrated in FIG 16 is a description example of Example 3. In Example 4,
among these description, a hint data type, and information enclosed in a square at a
lower side including hint data are stored in the sample group description box 21 as a
visual sample group entry 23. Further, among the subsample hint information box
11-4, information enclosed by a square in an upper side including an enlly count is
25 stored in the sample to group box 22 as information associating the visual sample
group entry 23 with the samples.
[0109]
That is, in the case of this example, it is possible to perform index reference
from the sample to group box 22 by storing only a pattem to be utilized. Further, it
30 is possible to compress a table of hint data, so that it is possible to reduce an
infonnation ammmt.
SP353455W001
--25/75
[OliO]
It should be noted that while, in the above description, a case has been
described where the method of Example 4 is applied to information of the subsample
hint information box of Example 3, the method of Example 4 can be also applied to
5 infommtion of the subsample hint information box in Example I and Example 2.
That is, information of the subsample hint information box described in Example I
and Example 2 can be also stored in the sample group description box 21 and the
sample to group box 22 in a similar manner to the case of Example 3 described
above.
10 [Olll]

A visual sample group entty 23-1 of FIG 17 indicates an example of the
visual sample group entty 23 extended to store profile information of the subsample
(SubSamleProfileinformation Entty extends VisualSampleGroupEntty ('sspi')) in the
15 case where an entcy is constituted in a similar mmmer to the group in the example of
FIG 10. In this visual sample group entty, hint information (hint_data) a11d
identification information of a group to which the entty corresponds (GroupiD) are
set for each entcy. In this case, as illustrated in FIG 17, for exmnple, information of
any example among examples as illustrated in the square 12 is described as this hint
20 infommtion (hint_ data).
[0112]
Further, in this case, a subsample index which is identification information
of a subsample (tile) belonging to each entcy is listed as in the example within a
square 31. ·
25 [0113]

An example of entries different from those in FIG 10 is illustrated in FIG
18. In the case of the example of FIG 18, subsamples (tiles) are respectively set as
entries (groups). That is, in the case of A in FIG 18, tile 1 (Tile1) is provided as an
30 entry 14-4, in the case ofB in FIG 18, tile 2 (Tile2) is provided as an entty 14-5, a11d
in the case of C in FIG 18, tile 3 (Tile3) is provided as an entcy 14-6. While not
SP353455W001
26/75
illustrated, in a similar manner, tile 4 (Tile4) and tile 5 (TileS) are respectively
provided as different entries.
(0114]
As in the example of FIG 18, when a plurality of subsample (tiles) are not
5 fonned into a group, in other words, when the subsamples (tiles) are respectively set
as entries, a visual sample group entry 23 extended to store profile information of the
10
subsample (SubSampleaProfilelnforamtionEntry
VisualSampleGroupEntry'('sspi')) is as illustrated in the example of FIG 19.
(0115]
extends
In this case, as in the visual sample group entry 23-2 in FIG 19, for example,
infmmation of any example among examples as illustrated in the square 12 is
described as hint information (hint_ data) for each entry. Further, a subsample index
which is identification information of a subsample (tile) belonging to each entry is
listed as in the example in a square 32. That is, in this case, one subsample index is
15 assigned to each entJ.y.
(0116]

While, in the above description, a case has been described where, as in the
visual sample group entJ.y 23-2 in FIG 20, a subsample index (subsamplejndex) is
20 included in the visual sample group entJ.y, for example, it is also possible to store flag
information (independent) indicating that the subsample (tile) can be decoded
independently from other subsamples (tiles) or information such as a reserved field
(reserved = 0) along with the subsample index for each subsample as described
below.
25 (0117]
(0118]
unsigned int(2) independent
bit(6) reserved =0;
By this means, because it is possible to know dependency for each
30 subsample, the information can be used as auxiliary information for a system to
pe1fonn parallel decoding, or the like.
SP353455WOOI
27175
[0119]
Further, while, in the above description, a case has been described where, as
in the visual sample group entiy 23-2 in FIG 20, information of any example among
examples as illustrated within the square 12 is described as hint information
5 (hint_ data), it is also possible to fmther describe other hint information. For
example, it is also possible to store information relating to a size of grouped
subsamples (tiles), such as a width (Width) and a height (height), or store
information relating to a location of grouped subsamples (tiles), such as offset in a
horizontal direction (H_offset) and offset in a vertical direction (V _offset).
10 [0120]
By storing such information, a system can easily acquire size information of
the grouped subsamples without calculating the information from the subsample
information box (subsample information box).
[0121]
15 Furthe1; it is also possible to describe mcts_id which is information for
identifying a partial region which can be independently decoded in bit streams of
HEVC as hint information (hint_ data).
[0122]
In MCTS SEI (Motion constrained tile set Supplemental Enhancement
20 Information) of HEVC, a partial region which can be independently decoded is set
for each rectangle set. For example, when it is assmned that a shaded area in an
upper left pa1t of A in FIG 21 is a partial region which can be independently decoded,
the partial region is set for each set as illustrated in B in FIG 21 in MCTS SEI. It
should be noted that as in the example illustrated in C in FIG 21, tiles included in the
'
25 set may overlap with tiles in other sets. The number of pixels in the partial region
can be calculated from the number of pixels of each set as illustrated in, for example,
the example ofD in FIG 21.
[0123]
An example of syntax of MCTS SEI is illustrated in FIG 22. As illustrated
30 in FIG 22, respective identification information (mcts id) is· assigned to a partial
region as illustrated in FIG 21. By describing this identification information
SP353455W001
28/75
(mcts_id) as hint information (hint_ data), it is possible to easily associate the patiial
region ofHEVC with the subsamples in the MP4 file format, so that it is possible to
easily update (for example, replace) header information of, for example, the
sequence parameter set (SPS) in the system.
5 [0124]

By the way, in.the case of Example 4, a case has been described where the
visual sample group entry is extended and a subsample belonging to each entry is
described using a subsample index. In this case, information of the subsample
10 infonnation box is utilized as access information to tiles in the sample. That is, in
this case, it is necessary to interpret the meaning of the subsample in a layer in which
the MP4 file is to be parsed.
[0125]
In Example 5, a method will be proposed as a method for accessing tiles
15 within the smnple, which can be a substitute for such a method. That is, in place of
the subsample information box, a configuration of a NAL unit (NAL unit) of bit
stremns constituting the sample is described in the visual sample group entry (Map
Group Entry). For example, the visual sample group entry is extended, and NAL
units of bit stremns of HEVC are grouped (HEVCNALUUMapGroupEntry ).
20 [0126]
25
When the NAL tmits are grouped in this manner, it is possible to support the
grouped NAL units with the same processing flow in a layer in which the MP4 file is
to be parsed regardless of meaning of grouping.
[0127]
The example is illustrated in FIG 23. As illustrated in FIG 23, for
example, a visual sample group entty in which NAL Wlits within the smnple are to be
mapped (for example, grouped for each tile) is prepared, and, for example, in the
visual sample group entry in which video related information such as hint
information is stored, the visual sample group entry in which the NAL tmits within
30 the sample are to be mapped is referred to.
[0128]
SP353455W001
29/75
Visual sample group entries are associated with each other usmg
identification information of the group (GroupiD). It should be noted that when
map patterns of the NAL units are all the same, a sample to group box is not required.
[0129]
5 An example of syntax of the visual sample group entry
(HEVCNALUMapGroupEntiy() extends VisualSampleGroupEntiy('hcnm')) in
which the NAL tmits in the sample are mapped is illustrated in an upper left part of
FIG 24. As illustrated in FIG 24, in this visual sample group entiy, identification
infonnation of the group (GroupiD) to which each NAL tmit (NAL unit) belongs is
10 set.
[0130]
For example, as illustrated in a lower left part of FIG 24, there are five
subsamplse (tiles (file)) within the sample, and each subsample is constituted with
two NAL units. In this case, association between the NAL unit and the GroupiD
15 (map pattern of the NAL unit) is as in the example illustrated in a right side of FIG
24. Therefore, the sample is constituted with 10 NAL units. NALU _count of
syntax of the visual sample group entry (HEVCNALUMapGroupEntry() extends
VisualSampleGroupEntry('hcnm')) illustrated in an upper left part of FIG 24
indicates the number ofNAL m1it. Further, as illusti·ated in a lower left part of FIG
20 24, it is possible to define from which NAL_unit each tile (Tile) starts. In this
example, Tile] starts from NALl, Tile2 starts from NAL3, Tile3 starts from NALS,
Tile4 starts from NAL 7, and TileS starts from NAL9. It is also possible to define
such information indicating from which NAL _unit each Tile statis as syntax of the
visual sample group entry (HEVCNALUMapGroupEntiy() extends
25 VisualSampleGroupEntry('hcnm')) illustrated in the upper left part of FIG 24.
[0131]
An example of the visual sample group entiy
(SubSatnpleProfileinfonnationEntry extends VisualSatnpleGroupEntry('sspi '))
described in Example 4, in which profile information of the subsample is stored, is
30 illustrated in FIG 25. In tllis visual sample group entiy, hint information
(hint_ data) atld identification infonnation of the group (GrouplD) corresponding to
the entry are set for each entry.
[0132]
30/75 --
SP353455WOO I
As illustrated in FIG 25, as this hint information (hint_ data), for example,
information of any example among examples as illustrated in the square 12 as
5 described in Example I to Example 4 is set.
[0133]
However, in FIG 25, as illustrated in the square 12, as the hint infom1ation,
Example (C) is added in addition to Examples (A-I), (A-2), (B-1) and (B-2). Here,
information relating to a location of the grouped subsamples (tiles), such as offset in
10 a horizontal direction (H _offset) and offset in a vertical direction (V offset) is stored.
15
Further, information relating to a size of the grouped subsamples (tiles), such as a
width (Width) and a height (height) is stored. These are the same as the hint
information described with reference to FIG 20.
[0134]
Further, in the identification information of the group (GroupiD), any of
identification information of the group (GroupiD) used for mapping NAL units is set
in the visual sample group entry (HEVCNALUMapGroupEntryO extends
Visua!SampleGroupEntry('hcnm')) in which NAL units within the sample are
mapped described using the example of FIG 24. That is, this identification
20 infonnation of the group (GroupiD) indicates a group of tiles as in the example of
FIG 10.
[0135]
It should be noted that, as in the example of FIG 18, also when a plurality of
subsamples (tiles) are not grouped, information similar to that in the example of FIG
25 25 is set as in the example illustrated in FIG 26 in the visual sample group entry in
which profile infonnation of the subsample IS stored
(SubSampleProfileinformationEnhy extends VisualSampleGroupEntry(' sspi ')). In
this case, the identification information of the group (GroupiD) indicates each tile.
[0136]
30 By extending the visual sample group entry as described above and setting a
configuration of the NAL unit, when the NAL units are grouped, it is possible to
SP353455W001
··· - --:n175
suppmt the grouped NAL units with the same processing flow in a layer in which the
MP4 file is parsed regardless of meaning of grouping.
[0137)
It should be noted that BOX in which GROUP by MAP GROUP is fonned
5 into TILE GROUP may be defined as BOX different from SSPI (SubSample Profile
Information). That is, identification information of the group (GroupiD) by
HEVCNALMapEntty corresponding to each entty may be defined in the visual
sample group entry extended so as to store a tile group map entry
(TileGroupMapEntry extends VisualSampleGroupEntty('tgpm')), which 1s a
10 different box from the visual sample group entry
(SubSampleProfileinformationEntry extends VisualSampleGroupEntty('sspi ')) in
which profile information of the subsamples defining hint infonnation (hint_ data)
corresponding to the entry is stored.
15
[0138)
In the case of the example of FIG 25, syntax ofTGPMmay be, for example,
set as follows:
[0139)
Class TileGroupMapEntry extends VisualSampleGroupEntry ('tgpm') {
unsigned int(16) entry_ count;
20 for (i=O; i < entty_cmmt; i++) {
25
}
}
[0140)
unsigned int(16) TileGroupiD;
unsigned int(16) group_ count;
for (j=O; j Group1D=3
TileGrouplD=2=>GroupiD=2, 3, 4
TileGrouplD=3=>GroupiD=I, 2, 3, 4, 5
SP353455WOOI
5 Syntax of SSPI may be, for example, set as follows as a box different from
10
the box.
[0143]
class SubSampleProfilelnformationEntry extends VisualSampleGroupEntry ('sspi ') {
unsigned int(16) entry_cow1t;
}
}
for (i=O; i < entry_count; i++) {
unsigned int(16) TileGroupiD;
unsigned int(xx) hint_ data;
15 [0144]
20
It should be noted that content of the hint information (hint_ data) in this
case is the same· as that in the above-described examples (information of any
example among examples as illustrated within the square 12 is set).
[0145]
In this manner, by grouping GroupiD in a box different from a box of sspi
defining hint information, or the like, it is possible to reduce dependency of grouping
on other infonnation, so that grouping can be performed independently from other
information. It is therefore possible to realize more versatile (flexible) grouping,
and utilize the group of the GrouplD (TileGroupiD) in more versatile purpose of use.
25 For example, it is possible to easily realize hierarchized and tmhierarchized
definition of the groupiD without the need of taking into accmmt a configuration of
other information such as hint information.
[0146]
Further, while the example of flag (flag) definition of the subsamples have
30 been described with reference to FIG 3, in the case of Example 5 described above,
continuous NAL units grouped by HEVC NAL Map Group Entry may be further
5
SP353455W001
33/75
additionally defined. For example, as described below, in flagc=S, the continuous
NAL units grouped by HEVC NAL Map Group Entty may be additionally defined as
sub-sample.
[0147]
S:O:NAL-unit-based sub-samples. A sub-sample contains one or more
contiguous NAL units. A sub-sample is mapped to GroupiD, grouping in HEVC
NAL Map Group Entry.
[0148]
By this means, it is possible to realize byte access to an arbitraty Group
10 based on NAL units.
[0149]
<2. Second embodiment>

An example of the MP4 file will be described next. FIG 27 is a diagram
15 for explaining a configuration example of the MP4 file of the tile image. This MP4
file is obtained by forming bit streams having a tile (Tile) structure into one file. As
illustrated in FIG 27, tllis MP4 file has five tracks of Track 1 to TrackS.
[01SO]
Track! has an HEVC sample entry (HEVC sample entry) within the sample
20 description box, and its codec type is hvcl indicating that the bit streams are normal
HEVC bit streams. This sample entry (Sample Entry) has an HEVC decoder
configuration record (HEVC Decoder Configuration Record) QwcC box) in which
configuration information required for decoding HEVC is stored. Header
infonnation such as a video parameter set (Video Parameter Set (VPS)), a sequence
25 parameter set (Sequence Parameter Set (SPS)) and a picture parameter set (Picture
Parameter Set (PPS)) is also stored in tllis hvcC box. SEI may be arbitrarily stored
in this hvcC box.
[01S1]
Track! has extractor (extractor) Track2 to TrackS for referring to a tile
30 (slice) of each track2 to trackS. The extractor (extractor) Track2 refers to slice
(Tile!) of Track2, and the extractor (extractor) Track3 refers to slice (Tile2) of
SP353455W001
34/7S
Track3. In a similar manner, the extractor (extractor) Track4 refers to slice (Tile3)
ofTrack4, and the extractor (extractor) TrackS refers to slice (Tile4) of TrackS.
[01S2]
In track2 to trackS, slice which is actual data of each Tile1· to Tile4 is stored.
5 Further, while track2 to trackS have sample entries (Sample Entty), because only
HEVC Tile is (independently) stored, a codec type is set as hvtl. The sample
entries (Sample Entry) of these track2 to trackS have an HEVC decoder
configuration record (HEVC Decoder Configuration Record) (hvtC box) required for
decoding only the HEVC Tile. It should be noted that it is also possible to store
10 grouped tiles (Tile) in one track.
[01S3]
track2 to trackS also have tile region group entries (TileRegionGroupEntry).
In these tile region group entries (TileRegionGroupEntry) of track2 to trackS, values
of five parameters including GroupiD which is identification information for
15 identifying each tile, H_offset indicating a location (offset) in a horiwntal direction
of the tile, V offset indicating a location (offset) in a vertical direction of the tile,
H_width indicating a size (width) in the horizontal direction of the tile, and V height
indicating a size (height) in the vertical direction of the tile are defined. Further,
these may be defined for the grouped tiles (Tile).
20 [01S4]
For example, in the tile region group entry (TileRegionGroupEntry) of
Track2 (tile 1 (Tilel)), Group1D=1, H_offset=O, V_offset=O, H_width=960 and
V height=S40 are defined. Further, for example, in the tile region group entry
(TileRegionGroupEntry) of Track3 (tile 2 (Tile2)), Group1D=2, H_offset=960,
25 V offset=O, H_width=960 and V height=S40 are defined. Further, for example, in
the tile region group entry (TileRegionGroupEntty) of Track4 (tile 3 (Tile3)),
GroupiD=3, H_offset=O, V offset=540, H_width=960 and V height=S40 are defiend.
Further, for example, in the tile region group entty (TileRegionGroupEntiy) of
TrackS (tile 4 (Tile4)), GroupiD=4, H_offset=960, V offset=S40, H_width=960 and
30 V height=S40 are defined. In this case, the whole image (1920x1080) includes
four tiles (960xS40) including.two tiles in a vertical direction and two tiles in a
SP353455WOO!
3S/7S
horizontal direction.
[OlSS]
Further, in track2 to trackS, "pmt= I" is defined as track reference (Track
Reference). This indicates that these track2 to trackS refer to Track I. That is,
5 when any (any tile) of Track2 to TrackS is reproduced using this track reference,
infom1ation ofTrackl (such as a parameter set) is referred to.
10
15
20
[OIS6]

[01S7]
Basic syntax of an HEVC decoder configuration record (HEVC Decoder
Configuration Record) (hvtC box) in which configuration information required for
decoding only an HEVC Tile of the sample entry (Sample Entry) in FIG 27 is stored
is set as follows.
[01S8]
[OIS9]
Class HEVCConfigurationBox extends Box('hvtc'){
HEVCTileDecoderConfigurationRecord()HEVCTileConfig;
}

A specific example of syntax of the HEVCTileDecoderConfigurationRecord
will be described next. FIG 28 is a diagram for explaining the HEVC tile decoder
configuration record. The HEVC tile decoder configuration record in Example 1 is
constituted as illustrated in FIG 28. configuration Version, mcts_tier_flag and
mcts level_idc are extensionally added.
25 [0160]
configuration Version indicates vers10n of the HEVC tile decoder
configuration record. In HEVC, two types of profiles of tier having different
Bitrate are defined for the same image size. That is, the two types are main tier and
high tier. mcts tier flag is a flag indicating which of the tier the tier is.
30 mcts_level_idc is a level indicating a degree of load of decoding processing of the
partial region as decoding load definition information which defines the degree of
SP353455W001
36/75
load of decoding processing of the partial regwn which can be independently
decoded in temporal MCTS SEI (Temporal Motion constratined tile set
Supplemental Enhancement Information).
[0161]
5 Here, temporal motion constrained tile set SEI
(temporal_motion_constrained_tile_sets SEI) will be described. FIG 29 IS a
diagram illustrating an example of syntax of temporal MCTS SEI. In syntax of
temporal MCTS SEI illustrated in FIG 29, various information including
mcts_tier_flag, mcts_1evel_idc, max_mcts_tier_flag and max_mcts_level_id is stored.
10 [0162]
It should be noted that temporal MCTS SEI is substantially the same as
MCTS SEI except the name. Fmiher, mcts_level_idc in FIG 28 and FIG 29 is
substantially the same as generals_level_idc illustrated in the square 12 in FIG 8,
FIG 12, FIG 13, FIG 17. FIG 19, FIG, 20, FIG 25, FIG 26, or the like, except the
15 name.
[0163]
In the exan1ple of FIG 28, among paramters having the same data structure
as that of HEVC decodec configuration record and stored m
temporal_motion_contrained_tile_sets SEI, only parameters required for determining
20 decoding of HEVC Tile, that is, mcts_tier_flag and mcts_level_idc are set. In
HEVCTileDecoderConfigurationRecord in FIG 28, zero is set for other items
because the other items are not required. Altematively, the same value as that of
HEVC decoder configuration record is set.
25
[0164]
That is, in this Example 1, nothing is substantially stored in numOfArrays.
[0165]

FIG 30 is a diagram for explaining an HEVC tile decoder configuration
record. The HEVC tile decoder configuration record of Example 2 is constituted as
30 illustrated in FIG 30.
[0166]
SP353455WOOI
Example 2 of FIG 30 has basically the same configuration as that of
Example 1 of FIG 28, except that, while, in Example 1, nothing is substantially
stored in numOfArrays, in Example 2, temporal_motion constrained_tile_sets SEI
corresponding to HEVC Title is stored in numOfArrays.
5 [0167]

FIG 31 is a diagram for explaining an HEVC tile decoder configuration
record. The HEVC tile decoder configuration record of Example 3 is constituted as
illustrated in FIG 31.
10 [0168]
As illustrated in FIG 31, in Example 3, unlike with the examples in FIG 28
and FIG 30, instead of the HEVC tile decoder configuration record being extended,
the pattem of the HEVC tile decoder configuration record is tmiquely specified.
That is, in Example 3, in addition to configuration Version, among parameters stored
15 m temporal_motion_constrained_tile_sets SEI, only mcts_tier_flag and
mcts_level_idc which are parameters required for determining decoding of HEVC
Tile are stored.
[0169]

20 FIG 32 is a diagram for explaining an HEVC tile decoder configurati01~
record. The HEVC tile decoder configuration record of Example 4 is constituted as
illustrated in FIG 32.
[0170]
As illustrated in FIG 32, in Example 4, in addition to configuration Version,
25 only mcts_tier_flag, mcts_level_idc, max_mcts_tier_flag and max_mcts_level_id are
stored as parameters required for determining decoding of HEVC Tile stored in
temporal_ motion_ constrained_ tile_ sets SEI. max_ mcts _tier_ flag and
max mcts _leveljd, which are respectively indicate maximum values of
mcts_tier_flag and mcts_level_idc, are profile infonnation required for reproducing
30 other tiles. That is, these are information for maximum Tile in the whole stream.
In an example of FIG 36 which will be described later, this is stored in another box
(hvte box).
[0171)
. 38/75
SP353455W001
When a size of Tile is different in the case where there are a plurality of Tile
in the same stream, there is a case where mcts_tier_flag and mcts_level_idc required
5 for each Tile are different from each other. In this case, max_mcts_tier_flag and
max_mcts_level_idc which are maximum values are stored with mcts_tier_flag and
mcts_level_idc for each Tile, and used as hint information of decoding processing.
[0172)
10

FIG 33 is a diagram for explaining an HEVC tile decoder configuration
record. The HEVC tile decoder configuration record of Example 5 is constituted as
illustrated in FIG 33.
[0173)
As illustrated m FIG 33, m Example 5,
15 temporal_motion_constrained_tile_sets SEI corresponding to HEVC Tile required
for determining decoding of HEVC Tile is stored in nalUnit. Therefore, m
NAL_unit_type, a NALtmit type indicating that the type isSEI is stored.
20
[0174)

FIG 34 is a diagram for explaining an HEVC tile decoder configuration
record. The HEVC tile decoder configuration record of Example 6 is constituted as
illustrated in FIG 34.
[0175)
As illustrated in FIG 34, Example 6 has a configuration in which Example 3
25 in FIG 31 and Example 5 in FIG 33 are synthesized. That is, part from a row of
configuration Version to a row of mcts_level_idc in Example 3 and part in a row of
lengthSizeMinusOne and thereafter in Example 5 are synthesized. In other words,
in addition to configurationVersion, mcts_tier_flag and mcts_level_idc which are
paramters required for detennining decoding of HEVC Tile stored in
30 temporal_motion_constrained_tile_sets SEI, and
temporal_motion_constrained_tile_sets SEI corresponding to HEVC Tile required
39/75
for determining decoding ofHEVC Tile are stored.
[0176]
SP353455WOO 1

FIG 35 is a diagram for explaining an HEVC tile decoder configuration
5 record. The HEVC tile decoder configuration record of Example 7 is constituted as
illustrated in FIG 3 5.
[0177]
As illustrated in FIG 3 5, Example 7 has a configuration in which Example 4
in FIG 32 and Example 5 in FIG 33 are synthesized. That is, part from a row of
10 configuration Version to a row ofmax_mcts_level_idc in Example 4 and part in a row
of lengthSizeMinusOne and thereafter in Example 5 are synthesized. In other
words, in addition to configurationVersion, mcts_tier_flag, mcts_level_idc,
max_mcts_tier_flag, and max_mcts_level_id which are paramters required for
determining decoding of HEVC Tile stored 111
15 temporal_ motion_ constrained_ tile_ sets SEI, and
temporal_motion_constrained_tile_sets SEI corresponding to HEVC Tile required
for determining decoding of HEVC Tile are stored.
[0178]

20 FIG 36 is a diagram for explaining an example of a structure of an MP4 file
of a tile image which is different from FIG 27. The MP4 file in FIG 36 has
basically the same structure as that ofMP4 file in FIG 27, while the MP4 file in FIG
36 is different from the MP4 file in FIG 27 in that a sample entry ofTrackl has hvte
box in addition to hvcCbox.
25 [0179]
30
In the MP4 file in FIG 36, max_mcts_tier_flag and max_mcts_level_id
which are information for maximum Tile in the whole stream described in FIG 32
are stored in hvte box of Track! which is a base track.
[0 180]
FIG 37 is a diagram for explaining an HEVC tile extension box. A in FIG
37 indicates extension of the visual sample group entry of the sample ent1y of Track!
SP353455W001
40/7S
m FIG 36, and m addition to HEVCConfigurationBox (hvcC box),
HEVCTileExtensionBox() (hvte box) is added. Bin FIG 37 indicates syntax of the
HEVCTileExtensionBox(). As illustrated in B of FIG 37, max_mcts_tier_flag and
max mcts level id are stored. - - -
5 (0181]
Because max_mcts_tier_flag and max_mcts_level_id are infonnation for
maximum Tile in the whole stream, max_ mcts tier_ flag and max mcts level_id are
not stored in Tile Track (Track2 to TrackS) and stored in Track! which is a base track.
By this means, it is possible to easily acquire a maximum value required for
10 independently decoding arbitrary Tile.
(0182]

FIG 38 is a diagram for explaining an example of a structure of an MP4 file
of a tile image. The MP4 file in FIG 38 has basically the same structure as that of
15 the MP4 file in FIG 27, except that, while, in the MP4 file in FIG 27, configuration
information required for decoding only HEVC Tile is stored in hvtCbox of the
sample entry of Track2 to Track S, in the MP4 file in FIG 38, not only
temporal_motion_constrained_tile_sets SEI is stored in hvtCbox, but also VPS, SPS
and PPS required for decoding only HEVC Tile are further stored in hvtCbox as in
20 the examples illustrated in FIG 30, FIG 33, FIG 34 and FIG 3S.
(0183]
By this means, when any of Tile2 to TileS is reproduced, it is not necessary
to access information of Track! which is a base track to obtain a parameter set
thereof.
25 (0184]
<3. Third embodiment>

An apparatus for implementing the present technique as described above
and a method thereof will be described next. FIG 39 is a diagram illustrating an
30 image encoding apparatus which is one aspect of an image processing apparatus to
which the present technique is applied. An image encoding apparatus I 00
SP353455WOOI
41/75
illustrated in FIG 39 is an apparatus for encoding input image data using an HEVC
coding scheme and forming a file using an MP4 file format.
(0 185)
As illustrated in FIG 39, the image encoding apparatus 100 has an image
5 encoding unit 10 I, a subsample infonnation generating unit I 02 and an MP4 file
generating unit I 03.
(0186)
The image encoding unit 101 divides the whole image of the input image
data into partial images referred to as tiles (Tile), sets each tile as a tile (Tile)
10 supported by an HEVC coding scheme and encodes the image data of the whole
image for each tile. That is, bit streams (HEVC bit streams) in the HEVC coding
scheme which can be independently decoded for each tile are generated.
(0187)
The image encoding unit I 0 I supplies the obtained HEVC bit streams to the
15 MP4 file generating tmit 103. Further, the image encoding unit 101 supplies tile
(Tile) information relating to how the whole image is divided and profile information
of each tile to the subsample information generating unit 102.
(0188)
The subs ample information generating unit 102 generates subs ample
20 information (extended subsample infonnation) using the tile information and the
profile information supplied from the image encoding unit 101, the subsample
information supporting the MP4 file format, and including the tile information and
the profile information, that is, including hint information used as a reference for
decoding processing of a subsample which is a partial region which can be
25 independently decoded in the whole image. The subsample information generating
unit 102 supplies the generated extended subsample infonnation to the MP4 file
generating unit I 03.
(0189)
The MP4 file generating tmit I 03 forms the HEVC bit streams supplied
30 from the image encoding tmit I OJ into a file using the MP4 file format, and stores the
extended subsample information supplied from the subsample information
SP353455WOOI
42/75
generating tmit 102 in management information for managing the HEVC bit streams,
of the file including the HEVC bit streams.
[0190]
At this time, the MP4 file generating unit 103 arranges the extended
5 subsample information as a subsample hint information box (Subsample Hint
Information Box) which is different from a subsample information box (Subsample
Information Box) of a sample table box (Sample Table Box) of the above-described
management information.
10
[0191]
Alternatively, the MP4 file generating tmit 103 arranges the extended
subsample information in a sample group description box (Sample Group
Description Box) of the sample table box (Sample Table Box) of the above-described
management information as a visual sample group enl!y (VisuaiSampleGroupEntry),
and arranges information designating a sample to which the subsample information is
15 to be applied in a sample to group box (Sample To Group Box).
[0192]
It should be noted that content of the subs ample infonnation is the same as
any of the examples described in the first embodiment. For example, in the case of
Example 5, the extended subsample information generated by the subsample
20 information generating unit 102 includes not only hint information used as a
reference for decoding processing of a subsample which is a partial region which can
be independently decoded in the whole image, but also map group information (Map
Group Entry) for grouping NAL unit.
25
30
[0193]
The MP4 file generating tmit 103 then outputs the MP4 file generated as
described above and transfers the MP4 file to, for example, a decoding side via a
network, a recording medium, or the like, or an information processing apparatus, or
the like.
[0194]
By this means, the image encoding apparatus 1 00 can recognize
performance required for decoding more accurately.
SP353455W001
43/75--
[0195]

Flow of each processing executed by the above-described image encoding
apparatus 100 will be described next. An example of flow of image encoding
5 processing will be described with reference to a flowchart in FIG 40.
[0196]
When the image encoding processing is started, the image encoding 1mit
101 encodes image data in step 8101 so that the image data can be decoded for each
subsample (tile).
10 [0197]
h1 step 8102, the image encoding 1mit 101 extracts infmmation relating to
the tile, such as, for example, tile information and profile information of each tile.
[0198]
ill step 8103, the subsample infmmation generating unit 102 generates
15 extended subsample information including hint information of the tile using the
infonnation relating to the tile extracted in step 8102.
[0199]
In step 8104, the MP4 file generating unit 103 generates an MP4 file using
the extended subsample infonnation so that hint information used as a reference for
20 decoding processing of a subsample which is a pattial region which can be
independently decoded is included in the sample table box of moov.
25
30
[0200]
When the processing in step 8104 ends, the image encoding processing ends.
[0201]
By executing the image encoding processing as described above, the image
encoding apparatus 100 can recognize performance required for decoding more
accurate! y.
[0202]

Decoding of encoded data encoded as described above will be described
SP353455WOO 1
44/75
next. FIG 41 is a block diagram illustrating mam components of an image
decoding apparatus corresponding to the image encoding apparatus 100, which is one
aspect of the image processing apparatus to which the present teclmique is applied.
An image decoding apparatus 200 illustrated in FIG 41 decodes the encoded data
5 generated by the image encoding apparatus 100 using a decoding scheme
corresponding to the encoding scheme. That is, the image decoding apparatus 200
extracts HEVC bit streams from the MP4 file, decodes the HEVC bit streams, and
outputs image data. At this time, the image decoding apparatus 200 can decode a
partial image including one or a plurality of tiles (Tile) independently from other
10 parts, for example, by utilizing a tile structure supported by HEVC. At that time,
the image decoding apparatus 200 can determine whether or not decoding is possible
based on hint information used as a reference for decoding processing of a subsample
which is a partial region which can be independently decoded.
15
[0203]
As illustrated in FIG 41, the image decoding apparatus 200 has an MP4 file
reproducing unit 201, a subsample information processing tmit 202 and an image
decoding unit 203.
[0204]
The MP4 file reproducing unit 201 performs processing of determining
20 whether or not reproduction of the inputted MP4 file is possible, pe1forms processing
of reproducing the MP4 file and performs error processmg. The MP4 file
reproducing unit 201 pe1forms these processing by utilizing the subsample
infonnation processing unit 202, generates HEVC bit streams of a partial image (of
course, it is possible to use the whole image) including one or a plurality of tiles and
25 supplies the HEVC bit streams to the image decoding unit 203.
[0205]
The subsample information processing unit 202 performs processing as to
the subsample information during the processing of determining whether or not
reproduction is possible and reproduction processing. It should be noted that
30 content of the subsample information is the same as any of the examples described in
the first embodiment. For example, in the case of Example 5, the extended
SP353455W001
45/75
subsample information generated by the subsample infonnation generating unit 102
includes not only hint information used as a reference for decoding processing of a
subsample which is a pattial region which can be independently decoded in the
whole image, but also map group information (Map Group Entry) for grouping NAL
5 tmit.
10
[0206)
The image decoding unit 203 decodes the HEVC bit streams generated in
the MP4 file reproducing unit 201 and outputs the image data.
[0207)
By this means, the image decoding apparatus 200 can recogmze
performance required for decoding more accurately.
[0208)

Flow of each processing executed by the image decoding apparatus 200 as
15 described above will be described next. First, an example of the flow of the image
decoding processing will be described with reference to a flowchart in FlG 42.
[0209)
When the image decoding processing is slatted, the MP4 file reproducing
unit 201 atld the subsample information processing unit 202 of the image decoding
20 apparatus 200 determines whether or not the partial image designated by the user, or
the like, Catl be reproduced for the HEVC bit streams included in the inputted MP4
file in step S20 1.
[0210)
In step S202, the MP4 file reproducing unit 201 determines whether or not
25 to perform reproduction based on the processing result in step S201. When it is
determined to perform reproduction, the processing proceeds to step S203.
[0211)
In step S203, the MP4 file reproducing tmit 201 and the subsample
information processing unit 202 perform reproduction processing.
30 [0212)
In step S204, the image decoding unit 203 decodes the encoded data (HEVC
5
10
SP353455W001
46/75
bit streams) of the partial image including one or a plurality of tiles, obtained through
the processing in step S203 and outputs image data of the partial image including one
or a plurality of tiles.
[0213]
When the processing in step S204 ends, the image decoding processing ends.
[0214]
Further, in step S202, when it is determined not to perform reproduction, the
processing proceeds to step S205.
[0215]
In step S205, the MP4 file reproducing unit 201 performs error processing
which is predetermined processing in the case where decoding cannot be performed
nonnally. 1l1is error processing may be any processing. For example, decoding
may be forcibly terminated (including intem1pted and suspending), or an alarm may
be issued to the user using an image, sound, or the like. Further, for example,
15 another encoded stream with a lower level may be acquired to perform decoding
agam. Still further, for example, the encoded stream may be forcibly decoded while
allowing disturbance to be generated in the decoded image.
[0216]
When the processing in step S205 ends, the image decoding processing ends.
20 [0217]

An example of flow of the processmg of determining whether or not
reproduction is possible, executed in step S201 in FIG 42 will be described next with
25 reference to a flowchart in FIG 43.
[0218]
When the processing of determining whether or not reproduction is possible
is started, in step S221, the MP4 file reproducing unit 201 acquires a sequence
parameter set (SPS) of HEVC bit streams included in the MP4 file from an HEVC
30 sample entry of the sample description box in the sample table box of the MP4 file.
[0219]
SP353455WOO l
. 47/75
In step S222, the l'v1P4 file reproducing unit 201 determines whether or not
the profile information included in the sequence parameter set (SPS) is profile
infonnation corresponding to the image decoding unit 203. That is, the MP4 file
reproducing wtit 201 determines whether or not the image decoding tmit 203 can
5 decode the HEVC bit streams of the whole image included in the l'v1P4 file based on
infonnation included in the sequence parameter set (SPS).
[0220]
If it is determined that decoding is possible, the processing proceeds to step
S223. In step S223, the l'v1P4 file reproducing unit 201 performs control so that
10 reproduction is performed. That is, it is determined to perform reproduction in step
S202 in FIG 42.
[0221]
When the processing in step S223 ends, the processing of determining
whether or not reproduction is possible ends, and the processing returns to FIG 42.
15 [0222]
Further, in step S222, when it is determined that the profile information does
not correspond to the image decoding tmit 203, that is, when it is determined that the
image decoding writ 203 cannot decode the HEVC bit streams of the whole image
included in the l'v1P4 file, the processing proceeds to step S224. In step S224, the
20 subsample information processing unit 202 acquires hint information of tiles
(subsamples) from the sample table box of the MP4 file.
[0223]
In step S225, the subsample information processing unit 202 determines
whether or not the profile infonnation included in the hint information acquired in
25 step S224 is profile infonnation corresponding to the image decoding unit 203.
That is, the subsample information processing unit 202 determines whether or not the
image decoding wtit 203 can decode the HEVC bit streams of a partial image
including one or a plurality of tiles, included in the l'v1P4 file.
[0224]
30 If it is determined that decoding is possible, the processing retums to step
S223. That is, also in this case, the MP4 file reproducing unit 201 performs control
so that reproduction is pe1formed.
[0225]
48/75
SP353455W001
Further, in step S225, when it is determined that the profile information does
not correspond to the image decoding unit 203, that is, when it is determined that the
5 image decoding lUiit 203 cam10t decode the HEVC bit streams of the partial image
included in the MP4 file, the processing proceeds to step S226.
[0226]
In step S226, the MP4 file reproducing unit 201 performs control so that
error processing is performed as described above.
10 [0227]
When the processing in step S226 ends, the processing of determining
whether or not reproduction is possible ends, and the processing returns to FIG 42.
[0228]

15 An example of flow of reproduction processing executed in step S203 in
FIG 42 will be described next with reference to a flowchmt in FIG 44.
[0229]
When the reproduction processing is started, the MP4 file reproducing unit
20 l acquires a sequence parameter set (SPS) ar1d a picture parameter set from the
20 MP4 file in step S241.
[0230]
25
In step S242, the subsample information processing unit 202 acquires hint
information of tiles to be reproduced from the MP4 file.
[0231]
In step S243, the MP4 file reproducing lUiit 201 updates the sequence
parameter set (SPS) acquired in step S241 using the hint information of the tiles
obtained in step S242. For example, when the hint information is Exan1ple (A-1) or
(A-2) in the square 12, the lvlP4 file reproducing unit 201 rewrites information of the
sequence parameter set (SPS) using information (such as a profile level) included in
30 the hint information. Further, for example, when the hint information is Example
(B-1) or (B-2) in the square 12, the MP4 file reproducing lUiit 201 replaces the
SP353155W001
49/75
infmmation included in the hint information with the sequence parameter set (8P8).
[0232]
In step 8244, the subsample information processing unit 202 acquires data
of tiles to be reproduced from the MP4 file. At this time, in the case of Example I
5 to Example 4, the subsample infonnation processing unit 202 accesses data of NAL
tmit constituting the subsamples (tiles) with reference to the subsample information
box and acquires the data as in the example in FIG 6. Further, in the case of
Example 5, the subsample information processing unit 202 accesses data of NAL
w1it constituting the subsamples (tiles) with reference to association between NAL
10 tmit and GroupiD (map pattern of NAL unit) (Map Group Entty), set in the visual
sample group entty (HEVCNALUMapGroupEntty() extends
Visual8mapleGroupEntty('hcnm')) in which NAL tmits are mapped in the sample
and acquires the data as in the example in FIG 24.
15
[0233]
In step 8245, the MP4 file reproducing unit 201 generates bit streams of
tiles (pmiial image) to be reproduced using the data of the tiles obtained in step 8244,
the sequence parameter set updated in step 8243, or the like.
[0234]
When the processing in step 8245 ends, the processing returns to FIG 42.
20 That is, the generated bit streams of the partial image are decoded.
[0235]
25
By executing each processmg as described above, the image decoding
apparatus 200 can recogllize performance required for decoding more accurately.
[0236]

Another example of flow of reproduction processing executed in step 8203
in FIG 42 will be described next with reference to a flowchart in FIG 45.
[0237]
When the reproduction processing is started, the MP4 file reproducing unit
30 201 acquires information relating to a location of the tile (H_offset, V offset) m1d
information relating to a size of the tile (Width, Height) from the visual sample group
entry in step 8261.
[0238]
8P353455W001
In step 8262, the l'v!P4 file reproducing unit 201 selects a tile which is
desired to be reproduced based on the acquired information relating to the location of
5 the tile and information relating to the size of the tile.
[0239]
In step 8263, the subsample information processmg unit 202 acquires
information relating to the map pattern ofNAL unit cmTesponding to the selected tile
(NALU _ cmmt, groupiD) from the visual sample group entry.
10 [0240]
15
In step S264, the subsample information processing unit 202 acquires data
of the tile based on the information relating to the map pattern of NAL unit acquired
in step S263.
[0241]
In step S265, the l'v!P4 file reproducing unit 201 reproduces bit streams of
the tile acquired in step S264.
[0242]
When the processing in step S265 ends, the processing retums to FIG 42.
That is, bit streams of the reproduced partial image are decoded.
20 [0243]
By executing each processmg as described above, the image decoding
apparatus 200 can recognize performance required for decoding more accurately.
[0244]

25 Another example of flow of reproduction processing executed in step S203
in FIG 42 will be described next with reference to a flowchart in FIG 46.
[0245]
When the reproduction processing is started, the l'v!P4 file reproducing unit
201 acquires information relating to a location of the tile (H_offset, V offset) and
30 infonnation relating to a size of the tile (Width, Height) from the visual sample group
entry in step S28l.
SP353455WOOI
51/75
[0246]
In step S282, the MP4 file reproducing unit 201 selects a region which is
desired to be reproduced based on the acquired information relating to the location of
the tile and infonnation relating to the size of the tile.
5 [0247]
10
In step S283, the subsample inf01mation processing unit 202 acquires a
plurality of GroupiDs based on TileGroupiD corresponding to the region which is
desired to be reproduced from the visual group entty.
[0248]
In step S284, the subsample information processmg unit 202 acqmres
information relating to the map pattern of NAL unit corresponding to tiles of the
selected plurality of Group IDs (NALU cmmt, groupiD) from the visual sample
group entry.
[0249]
15 In step S285, the subsample information processing mtit 202 acquires data
of each tile based on the information relating to the map pattern of NAL unit
acquired in step S284.
[0250]
In step S286, the MP4 file reproducing tmit 201 reproduces bit streams of
20 each tile acquired in step S285.
[0251]
25
When the processing in step S286 ends, the processing returns to FIG 42.
That is, bit streams of the reproduced partial image are decoded.
[0252]
By executing each processing as described above, the image decoding
apparatus 200 can recognize petformance required for decoding more accurately.
[0253]

Another example of flow of reproduction processing executed in step S203
30 in FIG 42 will be described next with reference to a flowchmt in FIG 47.
[0254]
SP353455W001
52/75
When the reproduction processing is started, the MP4 file reproducing unit
201 acquires infonnation relating to a location of the tile (H_offset, V offset) and
infonnation relating to a size of the tile (Width, Height) from the tile region group
entry in step 83 01.
5 [0255]
10
In step 8302, the MP4 file reproducing unit 201 selects a tile which is
desired to be reproduced based on the acquired information relating to the location of
the tile and information relating to the size of the tile.
[0256]
In step 8303, the subsample information processing unit 202 acquires tracks
corresponding to the tile selected in step 8302.
[0257]
In step 8304, the MP4 file reproducing unit 201 reproduces bit streams of
the tiles corresponding to a plurality of tracks acquired in step 8303.
15 [0258]
When the processing in step 8304 ends, the processing returns to FIG 42.
That is, bit streams of the reproduced partial image are decoded.
[0259]
By executing each processmg as described above, the image decoding
20 apparatus 200 can recognize performance required for decoding more accurately.
[0260]

Another example of flow of reproduction processing executed in step 8203
in FIG 42 will be described next with reference to a flowchart in FIG 48.
25 [0261]
When the reproduction processing is started, the MP4 file reproducing unit
201 acquires information relating to a location of the tile (H_offset, V offset) atld
information relating to a size of the tile (Width, Height) from the tile region group
entry in step 8321.
30 [0262]
In step 8322, the MP4 file reproducing 1mit 201 selects a region which is
SP353455WOOI
53/75
desired to be reproduced based on the acquired information relating to the location of
the tile and information relating to the size of the tile.
[0263]
In step S323, the subsample infmmation processing unit 202 acquires a
5 plurality of GroupiDs from the tile region group ently based on TileGroupiD
corresponding to the region which is desired to be reproduced.
[0264]
In step S324, the subsample information processing unit 202 acquires tracks
corresponding to the plurality of tiles selected in step S323.
10 [0265]
In step S325, the :MP4 file reproducing tmit 201 reproduces bit streams of
the plurality of tiles corresponding to a plurality of tracks acquired in step S324.
[0266]
When the processing in step S324 ends, the processing returns to FIG 42.
15 That is, bit streams of the reproduced partial image are decoded.
20
[0267]
By executing each processmg as described above, the image decoding
apparatus 200 can recognize petfonnance required for decoding more accurately.
[0268]
The present teclmique can be applied to any image encoding apparatus and
image decoding apparatus which can encode or decode a partial image.
[0269]
Further, the present teclmique can be applied to an image encoding
apparatus and an image decoding apparatus used when image information (bit
25 streams) compressed through orthogonal transform such as discrete cosine transfom1
and motion compensation, for example, in :MPEQ H.26x, or the like, is received via
a network medium such as satellite broadcasting, cable television, Internet and a
mobile telephone. Further, the present teclmique can be applied to an image
encoding apparatus and an image decoding apparatus used when processing is
30 performed on a storage medium such as an optical disc, a magnetic disc and a flash
memory.
[0270]
<5. Fifth embodiment>

54/75
SP353455W001
The above-described series of processes can also be performed by hardware
5 and can also be performed by software. When the series of processes is performed
by software, a program of the software is installed in a coinputer. Here, the
computer includes a computer embedded in dedicated hardware and, for example, a
general personal computer capable of various functions through installation of
vanous programs.
10 [0271]
FIG 49 is a block diagram illustrating an example of a hardware
configuration of the computer perfonning the above-described series of processes
according to a program.
[0272]
15 In a computer 900 illustrated in FIG 49, a central processing unit (CPU) 901,
a read-only memory (ROM) 902, and a random access memory (RAM) 903 are
connected mutually via a bus 904.
[0273]
An input and output interface 910 is also connected to the bus 904. An
20 input unit 911, an output unit 912, a storage unit 913, a commwlication unit 914, and
a drive 915 are connected to the input and output interface 910.
[0274]
The input unit 911 is formed by, for example, a keyboard, a mouse, a
microphone, a touch panel, or an input ternlinal. The output unit 912 is formed by,
25 for example, a display, a speaker, or an output terminal. The storage tmit 913 is
formed by, for example, a hard disk, a RAM disk, or a non-volatile memory. The
communication tmit 914 is formed by, for exan1ple, a network interface. The drive
915 drives a removable medium 921 such as a magnetic disk, an optical disc, a
magneto-optical disc, or a semiconductor memory.
30 [0275]
In the computer having the above-described configuration, for example, the
SP353455W001
55/75
CPU 901 perfonns the above-described processes by loading a program stored in the
storage unit 913 to the RAM 903 via the input and output interface 910 and the bus
904 and executing the program. The RAM 903 also appropriately stores data
necessary for the CPU 901 to perform various processes.
5 [0276]
For example, a program executed by the computer (the CPU 901) can be
recorded in the removable medium 921 such as a package medium to be applied. In
this case, by mmmting the removable medium 921 on the drive 915, the program can
be installed in the storage unit 913 via the input and output intetface 910.
10 [0277]
TI1e program can also be supplied via a wired or wireless transmission
medium such as a local area network, the Internet, or digital satellite broadcast. In
this case, the program can be received by the communication unit 914 to be installed
in the storage ru1it 913.
15 [0278]
Further, the program can also be installed in advance in the ROM 902 or the
storage unit 913.
[0279]
Programs executed by the computer may be programs which are processed
20 chronologically in the order described in the present specification or may be
programs which are processed at necessary timings, for example, in parallel or when
called.
[0280]
In the present specification, steps describing a program recorded in a
25 recording medium include not only processes which are performed chronologically
in the described order but also processes which are performed in parallel or
individually but not chronologically.
[0281]
1n the present specification, a system means a set of a plurality of
30 constituent elements (devices, modules (components), and the like) and all of the
constituent elements may be included or may not be included in the same casing.
SP353155W001
· ·s6ns-
Accordingly, a plurality of devices accommodated in separate casings and connected
via networks and a single device in which a plurality of modules are accommodated
in a single casing are all systems.
[0282]
5 A configuration described above as a single device (or processing unit) may
be divided and configured as a plurality of devices (or processing lmits). In contrast,
a configuration described above as a plurality of devices (or processing units) may be
collected and configured as a single device (or processing unit). Configurations
other than the above-described configurations may, of course, be added to the
10 configurations of the devices (or the processing lmits ). Further, as long as
configurations or operations are substantially the same in the entire system, parts of
the configurations of certain devices (or processing units) may be included in the
configurations of the other devices (or other processing units).
15
[0283]
The preferred embodiments of the present disclosure have been described
above with reference to the accompanying drawings, whilst the present disclosure is
not limited to the above examples, of course. A person skilled in the art may find
various alterations and modifications within the scope of the appended claims, and it
should be understood that they will natmally come under the teclmical scope of the
20 present disclosure.
[0284]
For example, in the present technology, it is possible to realize a cloud
computing configuration in which a single function is shared and processed jointly
by a plurality of devices via networks.
25 [0285]
Each step described in the above-described flowcharts can be performed by
a single device and can also be shared and performed by a plurality of devices.
[0286]
When a plurality of processes are included in a single step, the plurality of
30 processes included in the single step can be perfonned by a single device and can
also be shared and performed by a plurality of devices.
SP353455WOOI
-------s 717 5
[0287]
The image encoding apparatus and image decoding apparatus according to
the above-described embodiments can be applied to various electronic devices such
as a transmitter or a receiver in delivery of satellite broadcast, a wired broadcast such
5 as a cable TV, or the Internet and delivery to a terminal by cellular communication, a
recording device recording an image in a medium such as an optical disc, a magnetic
disk, or a flash memory, or a reproduction device reproducing an image from the
storage medium.
10
[0288]
The present technology is not limited thereto, and can be implemented as
any configuration mounted in the devices or devices constituting the systems, for
example, processors in the form of system large scale integration (LSI), modules that
use a plurality of processors, units that use a plurality of modules, sets obtained by
further adding other ftmctions to the units (i.e., a partial configuration of the devices),
15 and the like.
[0289]
In the present specification, the examples in which the various kinds of
information are multiplexed in the encoded stream and are transmitted from the
encoding side to the decoding side have been described. However, the methods of
20 transmitting the information are not limited to the examples. For example, the
information may be transmitted or recorded as separate pieces of data associated with
the encoded bit stream without being multiplexed in the encoded bit stream. Here,
the term "associated" means that an image (which may be a part of an image, such as
a slice or a block) included in a bit stream and information corresponding to the
25 image can be linked at the time of decoding. That is, the information may be
transmitted along a different transmission path from the image (or bit stream). TI1e
information may be recorded in a different recording medium (or a different
recording area of the same recording medium) from the image (or bit stream).
Further, the information and the image (or bit stream) may be mutually associated,
30 for example, in any tmit such as a plurality of frames, a single frame, or a part of a
frame.
5
10
15
SP353455W001
58/75
[0290]
Additionally, the present technology may also be configured as below.
(1)
An image encoding apparatus including:
an encoding unit configured to encode image data;
a subsample information generating unit configured to generate subsample
infonnation of an image of the image data, the subsample information including hint
infmmation used as a reference for decoding processing of a subsample, which is a
partial region which can be independently decoded; and
a file generating unit configured to generate a file including encoded data of
the image data generated by the encoding unit and management information of the
encoded data, and arrange the subsample information generated by the subsample
information generating unit in the management information.
(2)
The image encoding apparatus according to any of(1) and (3) to (15),
wherein the subsample information generating tmit generates subsample
information including a hint data type indicating a type of hint infonnation included
in the subsample information, a sample count indicating the number of continuous
samples associated with the subsample information, and the hint infonnation.
20 (3)
25
The image encoding apparatus according to any of(1), (2), and (4) to (15),
wherein the hint information includes a level indicating a degree of load of
decoding processing of the subsample.
(4)
The image encoding apparatus according to any of(1) to (3) and (5) to (15),
wherein the hint information includes header information of the encoded
data of the subsample.
(5)
The image encoding apparatus according to any of(1) to (4) and (6) to (15),
30 wherein the subsample information generating tmit generates subsample
information including a hint data type indicating a type of hint information included
5
SP353455W001
59/75
in the subsample information, a sample count indicating the number of continuous
samples associated with the subsample information, and identification infmmation of
a group to which the subsample belongs.
(6)
The image encoding apparatus according to any of ( 1) to (5) and (7) to (15),
wherein the subsample infonnation generating unit further generates
subsample information including a hint data type indicating a type of hint
infonnation included in the subsample information, a sample count indicating the
number of continuous samples associated with the subsample information, and hint
10 infommtion including a level indicating a degree of load of decoding processing of a
group of a subs ample.
(7)
The image encoding apparatus according to any of(!) to (6) and (8) to (15),
wherein the subsample infonnation generating unit further generates
15 subsample information including a hint data type indicating a type of hint
infonnation included in the subsample information, a sample count indicating the
number of continuous samples associated with the snbsample infonnation, and hint
infmmation including header information of encoded data of a group of a subsample.
(8)
20 The image encoding apparatus according to any of (1) to (7) and (9) to (15),
wherein the subsample information generating unit generates subsample
infonnation including a hint data type indicating a type of hint information included
in the subsample information, a sample cmmt indicating the number of continuous
samples associated with the subsample information, identification information of a
25 subsample belonging to a group, and hint information of the group.
(9)
The image encoding apparatus according to any of ( 1) to (8) and (10) to (15),
wherein the hint information includes a level indicating a degree of load of
decoding processing of the group.
30 (10)
The image encoding apparatus according to any of (1) to (9) and (11) to (15),
SP353455W001
60/75
wherein the hint information includes header information of encoded data of
the group.
(11)
The image encoding apparatus according to any of (1) to (10) and (12) to
5 (15),
wherein the subsample infmmation generating unit generates subsample
information including infom1ation indicating a size and a location of a subsamp1e.
(12)
The image encoding apparatus according to any of (1) to (11) and (13) to
10 (15),
15
20
25
wherein the subsample infonnation generating unit generates subsample
infonnation including information indicating that a subsample can be independently
decoded.
(13)
The image encoding apparatus according to any of(!) to (12), (14), and (15),
wherein the subsample infonnation generating tmit generates subsample
infonnation including infonnation for grouping NAL units constituting a sample.
(14)
The image encoding apparatus according to any of(!) to (13) and (15),
wherein tl1e file generating unit arranges the subsample information as a
subsample hint information box (SubsampleHint Infonnation Box) different from a
subsample information box (Subsample Infom1ation Box) of a sample table box
(Sample Table Box) of the management information.
(15)
The image encoding apparatus according to any of (1) to (!4),
wherein the file generating unit arranges the subsample information in a
san1ple group description box (Sample Group Description Box) of a sample table box
(Sample Table Box) of the management information as a visual sample group entry
(VisualSampleGroupEntry), and arranges information designating a sample to which
30 the subsample information is applied in a sample to group box (Sample To Group
Box).
SP353455WOOJ
61/75
(16)
The image encoding apparatus according to any of (1) to (15), wherein the
file generating unit arranges the subsample information required for decoding only
the partial image in a sample en!ty (Sample Entry) of the sample group description
5 box (Sample Group Description Box) of the sample table box (Sample Table Box) of
the management information of a track having the partial image.
(17)
The image encoding apparatt1s according to any of (1) to (16), wherein the
subsamp1e information is a parameter required for determining decoding of HEVC
10 Tile stored m a temporal motion constrained tile set SEI
(temporal_motion_constrained_tile_sets SEI) in the same data structure as a data
stt·ucture of an HEVC decodec configuration record.
(18)
The image encoding apparatus according to any of (1) to (17), wherein the
15 parameter includes mcts tear flag (mcts_tear_flag) and mcts level ide
(mcts_leve1_idc ).
(19)
The image encoding appara!t1s according to any of (1) to (18), wherein the
partial image information further includes temporal motion constrained tile set SEI
20 (temporal_motion_constrained_tile_sets SEI) corresponding to HEVC Tile.
(20)
The image encoding apparatus according to any of (1) to (19), wherein the
subsample information further includes max mcts tear flag (max_mcts_tear_flag) and
max mcts level ide (max_mcts_levvel_idc).
25 (21)
The image encoding apparatus according to any of (1) to (20), further
including temporal motion constrained tile set SEI
(temporal_motion_constrained_tile_sets SEI) corresponding to HEVC Tile required
for determining decoding ofHEVC Tile.
30 (22)
The image encoding apparatus according to any of (1) to (21), wherein the
5
SP353455W001
6217s-··
max mcts tear flag (max_mcts_tear_flag) and the max mcts level ide
(max_mcts_level_idc) are arranged in a base track.
(23)
An image encoding method including:
encoding image data;
generating subsample information including hint information used as a
reference for decoding processing of a subsample which is a partial region which can
be independently decoded, of an image of the image data; and
generating a file including genereated encoded data of the image data and
10 management information of the encoded data and arranging the generated subsample
information in the management information.
(24)
An image decoding apparatus inc! uding:
an acquiring unit configured to acquire a file including encoded data of
15 image data and management information of the encoded data in which subsample
infonuation of an image of the image data is arranged, the subsample information
including hint information used as a reference for decoding processing of a
subsample which is a partial region which can be independently decoded;
a subsample information analyzing tmit configured to analyze the subsample
20 information included in the file acquired by the acquiring unit;
a control unit configmed to control decoding of the encoded data based on
an analysis result of the subsample information by the subsample information
analyzing tmit;
a encoded data generating unit configured to generate encoded data of the
25 subsample from the encoded data included in the file acquired by the acquiring tmit
according to control by the controltmit; and
a decoding unit configured to decode the encoded data of the subsample
generated by the encoded data generating unit according to control by the control
tmit.
30 (25)
The image decoding apparatus according to (24), wherein the control unit
5
10
15
SP353455WOOI
determines whether the decoding unit can decode the encoded data of the subs ample
based on the hint information of the subsample infommtion and pe1forms control to
decode the encoded data of the subsample when the encoded data can be decoded.
(26)
The image decoding apparatus according to (24) or (25), wherein the
encoded data generating unit updates header information of the encoded data of the
subsample based on the subsample infommtion.
(27)
An image decoding method including:
acquiring a file including encoded data of image data and management
information of the encoded data in which subsample information of an image of the
image data is arranged, the subsample information including hint information used as
a reference for decoding processing of a subsample which is a partial region which
can be independently decoded;
analyzing the subsample information included in the acquired file;
controlling decoding of the encoded data based on an analysis result of the
subsample infmmation;
generating encoded data of the subsample from the encoded data included in
the acquired file according to the control; and
20 decoding the generated encoded data of the subsample according to the
control.
(31)
An information processing apparatus including:
a file generating unit configured to generate a file of an MP4 file format, in
25 which infommtion specifying a plurality of NAL units corresponding to a partial
image which can be independently decoded in a whole image is stored in moov and
the encoded partial image is stored in mdat; and
a transmitting unit configured to transmit the file generated by the file
generating unit.
30 (32)
The information processing apparatus according to (31 ),
5
10
15
20
SP353455WOOJ
64/75
wherein the information specifYing the plurality of NAL units includes
information indicating a group of the plurality of NAL tmits corresponding to the
partial image.
(33)
The information processing apparatus according to (31) or (32),
wherein the information specifYing the plurality of NAL units includes
information indicating the number of the plurality ofNAL tmits corresponding to the
partial image.
(34)
The information processing apparatus according to any of (31) to (33),
wherein the information specifYing the plurality of NAL tmits includes
information indicating a first NAL unit among the NAL units corresponding to the
partial image.
(35)
The information processing apparatus according to any of (31) to (34),
wherein the information specifYing the plurality of NAL units includes
information indicating all the NAL units corresponding to the partial image.
(36)
The information processing apparatus according to any of (31) to (3 5),
wherein the information specifYing the plurality of NAL units is defined
using VisualSampleGroupEntry in the moov.
(37)
The information processing apparatus according to any of (31) to (36),
wherein, in the file, a plurality of pieces of information for mapping the
25 plurality ofNAL units are further stored.
(38)
(39)
30
The information processing apparatus according to any of (31) to (37),
wherein the partial image is Tile in high efficiency video coding (HEVC).
An information processing method including:
generating a file of an MP4 file format, in which information specifYing a
SP353455W001
65/75
plurality of NAL units corresponding to a partial image which can be independently
decoded in a whole image is stored in moov and the encoded partial image is stored
in mdat; and
transmitting the generated file.
5 (40)
An information processing apparatus including:
a file reproducing unit configured to reproduce a file of an MP4 file format,
in which information specifying a plurality of NAL units corresponding to a partial
image which can be independently decoded in a whole image is stored in moov and
10 the encoded partial image is stored in mdat.
(41)
The information processing apparatus according to (40),
wherein the information specifying the plurality of NAL w1its includes
infonnation indicating a group of the plurality of NAL units corresponding to the
15 partial image.
(42)
1l1e information processing apparatus according to (40) or (41),
wherein the information specifying the plurality of NAL units includes
information indicating the number of the plurality ofNAL w1its corresponding to the
20 partial image.
(43)
The information processing apparatus according to any of (40) to (42),
wherein the information specifying the plurality of NAL units includes
information indicating a first NAL unit among the NAL units corresponding to the
25 pattial image.
(44)
The information processing apparatus according to any of (40) to ( 43),
wherein the infonnation specifying the plurality of NAL units includes
infonnation indicating all the NAL units corresponding to the pa1tial image.
30 (45)
The information processing apparatus according to atly of ( 40) to ( 44),
5
SP353455W001
66/75
wherein the information specifying the plurality of NAL tmits is defined
using VisualSampleGroupEntry in the moov.
(46)
The information processing apparatus according to any of(40) to (45),
wherein, in the file, a plurality of pieces of information for mapping the
plurality ofNAL units are further stored.
(47)
The information processing apparatus according to any of(40) to (46),
wherein the pmtial image is Tile in high efficiency video coding (HEVC).
10 (48)
The information processing apparatus according to My of (40) to (47),
wherein, in the file, information indicating a location of the partial image in
the whole image, information indicating a size of the partial image a~td the
infonnation specifying the plurality of NAL units are stored m
15 VisualSampleGroupEntry, and
20
25
30
wherein the file reproducing tmit
selects a partial image which is desired to be reproduced based on
the information indicating the location of the partial image in the whole image a~td
the information indicating the size of the partial image, a~td
acquires data of the partial image which is desired to be reproduced
based on the information specifying the plurality of NAL units m1d generates a bit
stream.
(49)
The information processing apparatus according to any of (40) to (48),
wherein, in the file, information indicating a location of the pmtial image in
the whole image, information indicating a size of the partial image and the
information specifying the plurality of NAL units are stored m
VisualSampleGroupEntry, and
wherein the file reproducing unit
selects a region which is desired to be reproduced based on the
infonnation indicating the location of the partial image in the whole image a11d the
SP353455W001
67175
information indicating the size of the partial image, and
acquires data of a partial image corresponding to the region which
is desired to be reproduced based on the information specifying the plurality ofNAL
w1its and generates a bit stream.
5 (50)
The information processing apparatus according to any of(40) to (49),
wherein, in the file, information indicating a location of the partial image in
the whole image, infonnation indicating a size of the partial image and the
infom1ation specifying the plurality of NAL units are stored m
10 TileRegionGroupEntry, and
wherein the file reproducing unit
selects a tile which is desired to be reproduced based on the
information indicating the location of the partial image in the whole image and the
information indicating the size of the partial image,
15 acquires a track corresponding to the selected tile which is desired
20
25
to be reproduced based on the information specifying the plurality ofNAL Wlits, and
generates a bit stream of a partial image corresponding to the
acquired track.
(51)
The information processing apparatus according to any of(40) to (50),
wherein, in the file, information indicating a location of the partial image in
the whole image, information indicating a size of the partial image and the
infmmation specifying the plurality of NAL tmits are stored m
TileRegionGroupEntry, and
wherein the file reproducing unit
selects a region which is desired to be reproduced based on the
information indicating the location of the partial image in the whole image and the
information indicating the size of the pmiial image,
acquires a plurality of tracks corresponding to the selected region
30 which is desired to be reproduced based on the information specifying the plurality
ofNAL units, m1d
SP353455W001
68/75
generates a bit stream of a partial Image corresponding to the
acquired plurality of tracks.
(52)
The information processing apparatus according to any of (40) to (51),
5 further including:
10
15
20
25
a decoding unit configured to decode a bit stream of the partial Image
reproduced and generated by the file reproducing unit.
(53)
An infonnation processing method including:
reproducing a file of an MP4 file format, in which information specifying a
plurality of NAL units corresponding to a partial image which can be independently
decoded in a whole image is stored in moov and the encoded partial image is stored
in mdat.
Reference Signs List
[0291)
100 image encoding apparatus
101 image encoding unit
102 subsample information generating unit
103 MP4 file generating unit
200 image decoding apparatus
201 MP4 file reproducing unit
202 subsample information processing unit
203 image decoding unit
SP353455WOOI

CLAIMS
Claim 1 (Amended)
An image decoding apparatus comprising:
an acquiring tmit configured to acquire subsample information from a file
5 including encoded data of image data and management information of the encoded
data, in which the subsample information of an image of the image data is arranged,
the subsample information including hint information used as a reference for
decoding processing of a subsample, which is a partial region which can be
independently decoded;
10
15
an encoded data generating unit configured to generate encoded data of the
subsample from the encoded data included in the file based on the subsample
infonnation acquired by the acquiring ltllit; and
a decoding unit configured to decode the encoded data of the subsample
generated by the encoded data generating tmit.
Claim 2 (Amended)
The image decoding apparatus according to claim 1,
wherein the acquiring ltllit acquires subsample information including a hint
data type indicating a type of hint information included in the subsample information,
20 a sample count indicating the number of continuous san1ples associated with the
subsample information, and the hint information.
25
30
Claim 3 (Amended)
The image decoding apparatus according to claim 2,
wherein the hint information includes a level indicating a degree of load of
decoding processing of the subsample.
Claim 4 (Amended)
The image decoding apparatus according to claim 2,
wherein the hint information includes header information of the encoded
data of the subsample.
SP353455W001
Claim 5 (Amended)
The image decoding apparatus according to claim 1,
wherein the acqui1ing unit acquires subsample information including a hint
5 data type indicating a type of hint information included in the subsample information,
a sample count indicating the number of continuous samples associated with the
subsample infonnation, and identification information of a group to which the
subsample belongs.
10 Claim 6 (Amended)
The image decoding apparatus according to claim 5,
wherein the acquiring Wlit further acquires subsample infonnation including
a hint data type indicating a type of hint information inc! uded in the subsample
information, a sample count indicating the number of continuous samples associated
15 with the subsample information, and hint information including a level indicating a
20
25
degree of load of decoding processing of a group of a subsample.
Claim 7 (Amended)
The image decoding apparatus according to claim 5,
wherein the acquiring unit fmiher acquires subsample inf01mation including
a hint data type indicating a type of hint information included in the subsample
information, a sample count indicating the number of continuous samples associated
with the subsample information, and hint information including header information
of encoded data of a group of a subsample.
Claim 8 (Amended)
TI1e image decoding apparatus according to claim 1,
wherein the acquiring unit acquires subsample information including a hint
data type indicating a type of hint information included in the subsample inf01mation,
30 a sample count indicating the number of continuous samples associated with the
subsample information, identification information of a subsample belonging to a
5
10
15
20
25
30
SP353455W001
group, and hint information of the group.
Claim 9 (Amended)
The image decoding apparatus according to claim 8,
wherein the hint information includes a level indicating a degree of load of
decoding processing of the group.
Claim 10 (Amended)
The image decoding apparatus according to claim 8,
wherein the hint infonnation includes header information of encoded data of
the group.
Claim II (Amended)
The image decoding apparatus according to claim I,
wherein the acquiring wtit acquires subsample infonnation including
infonnation indicating a size and a location of a subsample.
Claim 12 (Amended)
The image decoding apparatus according to claim I,
wherein the acquiring tmit acquires subsample infonnation including
infonnation indicating that a subsantple can be independently decoded.
Claim 13 (Amended)
The image decoding apparatus according to claim I,
wherein the acquiring unit acquires subsample information including
infomtation for grouping NAL wtits constituting a sample.
Claim 14 (Amended)
The image decoding apparatus according to claim I,
wherein the acquiring unit acquires the subsample infonnation arranged as a
subsample hint infonnation box (SubsampleHint Information Box) different from a
5
SP353455W001
subsample information box (Subsample Information Box) of a sample table box
(Sample Table Box) of the management information of the file.
Claim 15 (Amended)
The image decoding apparatus according to claim 1,
wherein the acquiring unit acquires the subsample information arranged as a
visual sample group entry (Visua!SampleGroupEntry) in a sample group description
box (Sample Group Description Box) of a sample table box (Sample Table Box) of
the management information of the file and information designating a sample to
10 which the subsample information is applied arranged in a sample to group box
15
(Sample To Group Box).
Claim 16 (Amended)
The image decoding apparatus according to claim 1,
wherein the encoded data generating w1it generates, when the decoding unit
can decode the encoded data of the subsample, the encoded data of the subsample
based on the hint information of the subsample information acquired by the acquiring
unit.
20 Claim 17 (Amended)
25
30
The image decoding apparatus according to claim 1,
wherein the encoded data generating ru1it updates header information of the
encoded data of the subsample based on the subsample information acquired by the
acquiring unit.
Claim 18 (Amended)
The image decoding apparatus according to claim 1,
wherein the subsample is a tile (Tile) in high efficiency video coding
(HEY C).
Claim 19 (Amended)
I I
.1~ --~.co::: ._.::-~·-'--'--:·::__l.;:~ . ____ c.:.-._-- ·-'--------"
SP353455W001
An image decoding method comprising:
acquiring subsample information from a file including encoded data of
nnage data and management infonnation of the encoded data, in which ·the
subsample information of an image of the image data is arranged, the subsample
5 information including hint information used as a reference for decoding processing
10
of a subsample, which is a partial region which can be independently decoded;
generating encoded data of the subsample from the encoded data included in
the file based on the acquired subsample information; and
decoding the g:merated encoded data of the subsample.