Description
Title oflnvention
INFORMATION PROCESSING DEVICE AND METHOD
Technical Field
[0001]
The present disclosure relates to an information processmg device and
method, and more particularly, to an information processing device and method of
10 adaptively supplying data of a partial image.
Background Art
[0002]
In recent years, as a content delivety technology using HyperText Transfer
15 Protocol (HTTP), Moving Picture Experts Group-Dynamic Adaptive Streaming over
HTTP (MPEG-DASH) was standardized (for example, see Non-Patent Literature I).
MPEG-DASH adopts an adaptive bitrate streaming (ABS) technology in which a
plurality of pieces of encoded data in which the same content is expressed at
different bit rates are stored in a content server and a client selects and reproduces
20 one piece of encoded data among the plurality of pieces of encoded data according to
a network bandwidth.
[0003]
Incidentally, selecting a partial image which is a pati of an image instead of
the entire image and delivering it adaptively has been considered. For example,
25 delivering a partial image which is a part selected in the entire image on a terminal
side receiving image data, or controlling the size of the partial image to be delivered
according to the performance of the terminal (for example, a processing ability of a
CPU or the like or the size of a display), a transmission path, a load situation of a
server, or the like has been considered.
30
Citation List
Non-Patent Literature
[0004]
21161
SP353396WOOO
Non-Patent Literature I: MPEG-DASH (Dynamic Adaptive Streaming
over HTTP) (URL:http://mpeg.chiariglione.org/standards/mpeg-dash/media-
5 presentation-description-and-segment-formats/text -isoiec-23 009-120 12-dam-1)
10
15
20
Technical Problem
[0005]
Summary ofinvention
However, the MPEG-DASH standard of the related art relates only to the
concept of switching bit rates (Bitrates ), and no selection of any partial image or
supply of the data performed using tile structures described above, that is, adaptive
supply of data of partial images, may be performed.
[0006]
It is desirable to provide a technology for adaptively supplying data of
partial images.
Solution to Problem
[0007]
According to an aspect of the present technology, there is provided an
information processing device including: a partial image information generation unit
configured to generate partial image information which is information regarding each
partial image which is a part of an entire image; and a metadata generation unit
configured to generate metadata to be used for supply of a bit stream of the entire
25 image and supply of a bit stream of the partial image using the partial nnage
information generated by the partial image information generation unit.
[0008]
The partial image information may include positional information indicating
a position of the pa1iial image in the entire image.
30 [0009]
The positional information may indicate a position of an upper left of the
partial image.
(0010]
3/161
SP353386WOOO
The metadata generation unit may store the partial image information
regarding the plurality of patiial images in mutually different adaptation sets
5 (AdaptationSet) of the metadata and assign the bit streams of the plurality of partial
images to the mutually different adaptation sets.
(OOII]
The metadata generation unit may store the patiial image information
regarding the plurality of partial images in mutually different adaptation sets
10 (AdaptationSet) of the metadata and assign a plurality of files for which one bit
stream including the plurality of partial images is divided for each patiial image to
the mutually different adaptation sets.
[0012]
The metadata generation unit may store the partial tmage information
15 regarding the plurality of partial images in mutually different sub-representations
(Sub-Representation) belonging to one representation (Representation) belonging to
one adaptation set (AdaptationSet) of the metadata and assign the bit streams of the
plurality of partial images to the mutually different sub-representations.
20
(0013]
The partial image information generation unit may further generate
information indicating that information regarding the bit stream is present under the
sub-representation (Sub-Representation).
(0014]
Each of the bit streams of the plurality of partial images may be stored in a
25 TRACK of one MP4 file.
(0015]
30
The metadata generation unit may further generate information indicating a
location of data of the one MP4 file.
[0016]
The patiial image information may further include information regarding a
size of the entire image.
SP353396WOOO
4/161
[0017]
The partial image information may further include group identification
information identifying a group which is a group to which the partial images belong
and which is a group of the partial images displayable as one image.
5 [0018]
10
The information processing device may further includes an encoding unit
configured to encode image data of the entire image and the partial images and
generate a bit stream.
[0019]
The information processing device may further includes a screen division
processing unit configured to generate image data of each partial image from image
data of the entire image.
[0020]
The partial image information may include information indicating the
15 number of partial images forming the entire image, identification information
indicating that sizes of the partial images are equal, and information indicating a
position and a size of each partial image when the sizes of the partial images are not
equal.
20
[0021]
According to an aspect of the present technology, there is provided an
information processing method including: generating ·partial image information
which is information regarding each pa1iial image which is a part of an entire image;
and generating metadata to be used for supply of a bit stream of the entire image and
supply of a bit stream of the partial image using the generated pa1tial image
25 information.
[0022]
According to another aspect of the present technology, there is provided an
information processing device including: an analysis unit configured to analyze
metadata including pmiial image information which is information regarding each
30 partial image which is a part of an entire image and used for supply of a bit stream of
the entire image and supply of a bit stream of the pa1iial image and to obtain the
.H,
SP353396WOOO
5/161
patiial image information; a selection unit configured to select a bit stream of a
desired partial image using the partial image infotmation obtained by the analysis
unit; and a bit stream acquisition unit configured to acquire the bit stream selected by
the selection unit.
5 [0023]
The partial image information may include positional information indicating
a position of the partial image in the entire image.
[0024]
The positional information may indicate a position of an upper left of the
10 partial image.
[0025]
The analysis unit may analyze the metadata in which the partial image
information regarding the plurality of partial images is stored in mutually different
adaptation sets (AdaptationSet) and the bit streams of the plurality of partial images
15 are assigned to the mutually different adaptation sets.
[0026]
The analysis unit may analyze the metadata in which the partial image
information regarding the plurality of patiial images is stored in mutually different
adaptation sets (AdaptationSet) and a plurality of files for which one bit stream
20 including the plurality of partial images is divided for each partial image are assigned
to the mutually adaptation sets.
[0027]
The analysis unit may analyze the metadata in which the partial image
information regarding the plurality of partial images is stored in mutually different
25 sub-representations (Sub-Representation) belonging to one representation
(Representation) belonging to one adaptation set (AdaptationSet) and the bit streams
of the plurality of partial images are assigned to the mutually different subrepresentations.
30
[0028]
The partial image information may include information indicating that
information regarding the bit stream is present under the sub-representation (SubRepresentation).
[0029)
6/161
SP353396WOOO
Each of the bit streams of the plurality of pmiial images may be stored in a
TRACK of one MP4 file.
5 [0030)
The metadata may include information indicating a location of data of the
one MP4 file.
[0031)
The partial image information may fu1iher include information regarding a
10 size of the entire image.
[0032)
The partial image information may further include group identification
information identifying a group which is a group to which the partial images belong
and which is a group of the partial images displayable as one image.
15 [0033)
The information processing device may further include a decoding unit
configured to decode the bit stream acquired by the bit stream acquisition unit.
[0034)
The information processmg device may further includes a screen
20 combination processing unit configured to generate image data of the entire image
from image data of the partial image obtained by the decoding unit decoding the bit
stream.
[0035)
The partial image information may include information indicating the
25 number of partial images forming the entire image, identification information
indicating that sizes of the partial images are equal, and information indicating a
position and a size of each partial image when the sizes of the partial images are not
equal.
30
[0036)
According to another aspect of the present technology, there is provided an
information processing method including: analyzing metadata including partial
SP353396WOOO
7/161
image information which is information regarding each patiial image which is a pati
of an entire image and nsed for supply of a bit stream of the entire image and supply
of a bit stream of the partial image and obtaining the partial image information;
selecting a bit stream of a desired partial image using the obtained partial image
5 information; and acquiring the selected bit stream.
[0037]
According to an aspect of the present technology, partial image information
which is information regarding each partial image which is a part of an entire image
is generated; and metadata to be used for supply of a bit stream of the entire image
10 and supply of a bit stream of the partial image is generated using the generated
partial image information.
[0038]
According to another aspect of the present technology,
metadata is analyzed, the metadata including partial image information
15 which is information regarding each partial image which is a part of an entire image
and used for supply of a bit stream of the entire image and supply of a bit stream of
the partial image and obtaining the partial image information; a bit stream of a
desired partial image is selected using the obtained partial image information; and
the selected bit stream is acquired.
20
25
Advantageous Effects oflnvention
[0039]
According to an embodiment of the present disclosure, information can be
processed. In particular, it is possible to adaptively supply data of partial images.
BriefDescription ofDrawings
[0040]
[FIG. I] FIG. 1 is a diagram for describing the overview ofMPEG-DASH.
[FIG. 2] FIG. 2 is a diagram illustrating an example of the configuration of an MPD.
30 [FIG. 3] FIG. 3 is a diagram for describing temporal separation of content.
[FIG. 4] FIG. 4 is a diagram illustrating an example of a layered structure below a
SP35339GWOOO
. 8/161
period in the MPD.
[FIG. S] FIG. S is a diagram for describing an example of the configuration of an
MPD file on a time axis.
[FIG. 6] FIG. 6 is a diagram illustrating examples of bit streams oftile images.
5 [FIG. 7] FIG. 7 is a diagram for describing examples ofMP4 files oftile images.
[FIG. 8] FIG. 8 is a diagram for describing other examples of the MP4 files of the tile
images.
[FIG. 9] FIG. 9 is a diagram for describing examples of division sizes.
[FIG. 10] FIG. 10 is a diagram for describing an example of extended data.
10 [FIG. 11] FIG. 11 is a block diagram illustrating an example of a main configuration
of a delivery system.
[FIG. 12] FIG. 12 is a block diagram illustrating an example of a main configuration
of a delivery data generation device.
[FIG. 13] FIG. 13 is a block diagram illustrating an example of a main configuration
15 of a tem1inal device.
(FIG. 14] FIG. 14 is a flowchart for describing a flow example of a delive1y data
generation process.
[FIG. IS] FIG. IS is a flowchart for describing a flow example of a tile type MPD file
generation process.
20 (FIG. 16] FIG. 16 is a flowchart for describing a flow example of a delivery data
reproduction process.
[FIG. 17] FIG. 17 is a diagram illustrating an example of an MPD extension method.
[FIG. 18] FIG. 18 is a diagram illustrating an example of an extended MPD.
[FIG. 19] FIG. 19 is a diagram illustrating an example of an MPD extension method.
25 (FIG. 20] FIG. 20 is a diagram illustrating an example of an MPD extension method.
[FIG. 21] FIG. 21 is a diagram illnstrating an example of an extended MPD.
(FIG. 22] FIG. 22 is a diagram illustrating an example of an MPD extension method.
[FIG. 23] FIG. 23 is a diagram illustrating an example of an MPD extension method.
[FIG. 24] FIG. 24 is a diagram illustrating an example of an extended MPD.
30 [FIG. 2S] FIG. 2S is a diagram illustrating an example of an MPD extension method.
[FIG. 26] FIG. 26 is a diagram illustrating an example of an extended MPD.
SP35339GWOOO
9/161
[FIG 27] FIG 27 is a diagram illustrating an example of an MPD extension method.
[FIG 28] FIG 28 is a diagram illustrating an example of an extended MPD.
[FIG. 29] FIG. 29 is a diagram illustrating an example of an MPD extension method.
[FIG 30] FIG 30 is a diagram illustrating an example of an extended MPD.
5 [FIG 31] FIG. 31 is a diagram illustrating an example of an MPD extension method.
[FIG. 32] FIG. 32 is a diagram illustrating an example of an MPD extension method.
[FIG 33] FIG 33 is a diagram illustrating an example of an extended MPD.
[FIG 34] FIG 34 is a diagram illustrating an example of an MPD extension method.
[FIG. 35] FIG. 35 is a diagram illustrating an example of an application using tile
10 image delivery.
[FIG 36] FIG 36 is a diagram illustrating another example of an application using
the tile image delivery.
[FIG. 37] FIG. 37 is a diagram illustrating another example of an application using
the tile image delivery.
15 [FIG. 38] FIG. 38 is a diagram illustrating an example of an MPD extension method.
[FIG 39] FIG 39 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
[FIG. 40] FIG. 40 is a diagram illustrating an example of the configuration of an
MPD.
20 [FIG. 41] FIG. 41 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
[FIG 42] FIG. 42 is a diagram illustrating an example of the configuration of an
MPD.
[FIG. 43] FIG. 43 is a diagram for describing an example of the configuration of an
25 MP4 file of a tile image.
[FIG. 44] FIG. 44 is a diagram illustrating an example of the configuration of an
MPD.
[FIG. 45] FIG. 45 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
30 [FIG 46] FIG 46 is a diagram illustrating an example of the configuration of an
MPD.
SP353396WOOO
10/161
[FIG 47] FIG 47 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
[FIG 48] FIG 48 is a diagram illustrating an example of the configuration of an
MPD.
5 [FIG 49] FIG 49 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
[FIG 50] FIG 50 is a diagram illustrating an example of the configuration of an
MPD.
[FIG 51] FIG 51 is a diagram for describing an example of the configuration of an
10 MP4 file of a tile image.
[FIG 52] FIG 52 is a diagram illustrating an example of the configuration of an
MPD.
[FIG 53] FIG 53 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
15 [FIG 54] FIG 54 is a diagram illustrating an example of the configuration of an
MPD.
[FIG 55] FIG 55 is a diagram for describing an example of the configuration of an
MP4 file of a tile image.
[FIG 56] FIG 56 is a diagram illustrating an example of the configuration of an
20 MPD.
[FIG 57] FIG 57 is a block diagram illustrating an example of a main configuration
of a computer.
[FIG 58] FIG 58 is a diagram illustrating an example of a multi-view image coding
scheme.
25 [FIG 59] FIG 59 is a diagram illustrating an example of a configuration of a multiview
image encoding device.
[FIG 60] FIG 60 is a diagram illustrating an example of a configuration cif a multiview
image decoding device.
[FIG 61] FIG 61 is a diagram illustrating an example of a layered image encoding
30 scheme.
[FIG 62] FIG 62 is a diagram for describing an example of spatial scalable coding.
SP35339GWOOO
11/161
[FIG 63] FIG 63 is a diagram for describing an example of temporal scalable coding.
[FIG 64] FIG 64 is a diagram for describing an example of scalable coding of a
signal-to-noise ratio.
[FIG 65] FIG 65 is a diagram illustrating an example of a configuration of a layered
5 image encoding device.
[FIG 66] FIG 66 is a diagram illustrating an example of a configuration of a layered
image decoding device.
[FIG 67] FIG 67 is a block diagram illustrating an example of a schematic
configuration of a television device.
10 [FIG 68] FIG 68 is a block diagram illustrating an example of a schematic
configuration of a mobile telephone.
[FIG 69] FIG 69 is a block diagram illustrating an example of a schematic
configuration of a video set.
[FIG 70] FIG 70 is a block diagram illustrating an example of a schematic
15 configuration of a video processor.
[FIG 71] FIG 71 is a block diagram illustrating another example of the schematic
configuration of the video processor.
Description of Embodiments
20 [0041]
25
Hereinafter, modes (hereinafter referred to as embodiments) for carrying out
the present disclosure will be described. The description will be made in the
following order,
1. First embodiment (MPD extension)
2. Second embodiment (delivery system)
3. Third embodiment (specific example ofMPD extension)
4. Fourth embodiment (other example ofMPD extension)
5. Fifth embodiment (MP4 file and other example ofMPD extension)
6. Sixth embodiment (computer)
30 7. Seventh embodiment (multi-view image encoding device and multi-view
image decoding device)
SP35339GWOOO
12/161
8. Eighth embodiment (layered image encoding device and layered image
decoding device)
9. Ninth embodiment (application examples)
10. Tenth embodiment (set, unit, module, and processor)
5 [0042]

Conventionally, as a content delivery technology using HyperText Transfer
Protocol (HTTP), there is Moving Picture Experts Group-Dynamic Adaptive
10 Streaming over HTTP (MPEG-DASH) as described in Non-Patent Literature I.
MPEG-DASH adopts an adaptive bitrate streaming (ABS) technology in which a
plurality of pieces of encoded data in which the same content is expressed at
different bit rates are stored in a content server and a client selects and reproduces
one piece of encoded data among the plurality of pieces of encoded data according to
15 a network bandwidth.
[0043]
A procedure of transmission of content by DASH will be described with
reference to FIG I. First, in a moving image reproduction terminal of a side
acquiring the content, software for controlling streaming data selects a media
20 presentation description (MPD) file of desired content and acquires the MPD file
from a web server. The MPD is metadata for managing content such as a moving
image or audio to be delivered.
[0044]
When the MPD is acquired, the software for controlling streaming data of
25 the moving image reproduction terminal analyzes the MPD and performs control
such that data (a DASH segment) of the desired content appropriate for the quality of
a communication line, the performance of the moving image reproduction terminal,
or the like is acquired from the web server. Client software for HTTP access
acquires the DASH segment using HTTP from the web server under the control.
30 The content acquired in this way is reproduced by moving image reproduction
software.
SP353396WOOO
13/161
[0045]
The MPD has, for example, the configuration illustrated in FIG 2. In the
analysis (parsing) of the MPD, a client selects an optimum representation from
attributes of representations (Representation) included in periods (Period) of the
5 MPD (Media Presentation in FIG 2).
[0046]
The client reads the beginning segment (Segment) of the selected
representation (Representation) to acquire and process an initialization segment
(Initialization Segment). Subsequently, the client acquires and reproduces
10 subsequent segments (Segment).
[0047]
A relation among the period (Period), the representation (Representation),
and the segment (Segment) in the MPD is illustrated in FIG 3. That is, one piece of
media content can be managed for each period (Period) which is a unit of data in a
15 time direction and each period (Period) can be managed for each segment (Segment)
which is a unit of data in the time direction. For each period (Period), a plurality of
representations (Representation) with different attributes such as bit rates can be
configured.
20
[0048]
Accordingly, a file of the MPD (also referred to as an MPD file) has the
layered stmcture illustrated in FIG 4 below the period (Period). When the stmcture
of the MPD is arranged on the time axis, the MPD has the structure illustrated in the
example of FIG 5. For the same segment (Segment), a plurality of representations
(Representation) are present as is apparent from the example of FIG 5. The client
25 can acquire proper stream data according to a communication environment, a
30
decoding ability of the client, or the like by adaptively selecting any of the
representations to reproduce the stream data.
[0049]

In DASH of the related art, delivery of data of an entire image is adaptive! y
controlled, but selecting a partial image which is a part of an image instead of the
SP35339GWOOO
I 4/161
entire Image and delivering it adaptively has been considered. For example,
delivering a pmiial image which is a part selected in the entire image on a terminal
side receiving image data, or controlling the size of the pmiial image to be delivered
according to the performance of the terminal (for example, a processing ability of a
5 CPU or the like or the size of a display), a transmission path, a load situation of a
server, or the like has been considered.
[0050]
In order to perform the adaptive delivery of a paiiial image, the concept of a
tile (Tile) has been used. A tile (Tile) is a partial region obtained by dividing an
10 entire image in a pre-decided layout (a size, a shape, a number, or the like).
Hereinafter, an image of one tile is referred to as a tile image. When an entire
image is tiled in advance in this way, adaptive delivery of a partial image can be
facilitated merely by selecting a tile image to be delivered. In this case, a partial
image is configured by a single tile image or a plurality of tile images.
15 [0051]
When the partial image is delivered using HTTP as in DASH, image data is
encoded and a bit stream of the image data is filed to be delivered (published as a
file). When an entire image has the above-described tile structure, image data is
encoded independently for each tile image. At this time, as in the example
20 illustrated in FIG. 6A, each piece of encoded data of each tile may be configured in
one bit stream.
[0052]
In the example of FIG. 6A, an entire image with a 640x480 size, an entire
image with a !980x1080 size, and each of tile images (four partial images) with a
25 960x540 size obtained by dividing the entire image into two in the vertical and
horizontal directions are prepared as images for delive1y. Data of the entire image
with the 640x480 size is encoded and considered to be one bit stream (bitstreaml)
and the data of the entire image with the 1980xl080 size is also encoded and
considered to be one bit stream (bitstream2). In addition to the data, data of each
30 tile image with the 960x540 size is independently encoded and considered to be one
bit stream (bitstream3 to bitstream6).
SP35339GWOOO
15/161
[0053]
In each bit stream, header information such as a video parameter set (VPS),
a sequence parameter set (SPS), supplemental enhancement information (SEI), and a
picture parameter set (PPS) is added, and the bit stream of the image data is arranged
5 for each slice (Slice).
[0054]
By setting such a stmcture, a tile image to be delivered can be selected by
selecting the bit stream to be delivered selected from bitstream3 to bitstream6. In
the case of the example of FIG 6A, each tile image can be delivered as the entire
10 1mage.
[0055]
Incidentally, for example, in a coding scheme such as high efficiency video
coding (HEVC), a stmcture called tiles (Tile) into which an entire image is divided is
supported, and thus encoding can be independently performed for each tile. For
15 example, decoding can be performed so that only the image of some of the tiles is
obtained. That is, the decoding can be performed so that only a partial image which
is a part of the entire image is obtained.
[0056]
As in the example illustrated in FIG 6B, encoded data of a plurality of tile
20 images can also be configured as one bit stream (bitstream7) using the function of
such a coding scheme. That is, in this case, the tiles (Tile) for delivery described
above are handled as tiles (Tile) supported by the coding scheme to be encoded. In
this case, in the bit stream, the data of the tiles is arranged as slices (Slice).
25
[0057]

As described above, the bit stream for delivery is filed in accordance with,
for example, an MP4 file format. In this case, the bit stream of each tile can be set
to be a separate file, as in the example illustrated in FIG 7. The bit stream of each
tile is managed in units called tracks (Track). Further, header (Header) information
30 regarding each tile and a base track (Base Track) in which reference to each track is
described are provided and filed as a different file ii-01n the bit stream of each tile.
SP35339GWOOO
16/161
When all of the tiles are decoded, the base track is reproduced. When each tile is
decoded, the base track is referred to in the header information.
[0058]
As in the example illustrated in FIG 8, the bit streams of the tiles can also
5 be collected and configured in one file. At this time, data of the tiles can also be
collected and managed in one track as in FIG 8A and the tiles can also be managed
as mutually different tracks as in FIG 8B. In this case, as in the case of FIG 7, the
header (Header) information regarding each tile and the base track (Base Track) in
which the reference to each track is described are provided.
10 [0059]

The tiles (Tile) may be tiles obtained by equally dividing an entire image as
in the example of FIG 9A or may be tiles obtained by unequally dividing an entire
image as in the example of FIG 9B. That is, the image sizes of the tile images
15 forming the entire image may be the same as or different from one another.
[0060]

As an example of an application using such a tile (Tile) strnctnre, for
example, an application controlling the size of a partial image to be displayed can be
20 considered.
[0061]
An entire image 10 illustrated in FIG 9A is assumed to be tiled and divided
into a plnrality of tile images 11 with the same size. For example, when the image
is displayed by a motile device 21 with a display of a small size, an application
25 displays partial images 12 which are 4 tile images of 2x2. For example, when the
image is displayed by a television signal receiver (TV) 22 with a display of a large
size, an application displays partial images 13 which are 30 tile images of 6x5. In
this way, an application controlling the image sizes of partial images displayed
according to the performance or the like of a terminal displaying an image is
30 considered.
[0062]
SP35339GWOOO
17/161
In the case of the example of FIG 9B, the image sizes of the tile images are
unequal. The application can display an image with an HD resolution by displaying
an image of a tile 3 (Tile 3), can display an image with a cinema resolution by
displaying images of tile 2 (Tile 2) to tile 4 (Tile 4), and can further display an image
5 with a further extended size (EXT) by displaying images of tile I (Tile!) to tile 5
(TileS). In this way, the application controlling a resolution or an aspect ratio of a
display image by controlling the image sizes of partial images to be displayed ts
considered.
10
[0063]
Since it is not necessary to deliver an image of an unnecessaty portion that
will not be displayed by adaptively controlling the sizes of partial images to be
delivered (by controlling the number of tile images to be delivered), as described
above, according to the image sizes of the partial images to be displayed in such an
application, a load of a server, a terminal, a transmission path, or the like can be
15 adaptively controlled, and thus it is possible to suppress an increase in an
unnecessary load.
[0064]

However, the MPEG-DASH standard of the related art relates only to the
20 concept of switching bit rates (Bitrates), and no selection of any partial image or
supply of the data performed using tile stmctures described above, that is, adaptive
supply of data of partial images, may be performed.
[0065]
Accordingly, partial image information which is information regarding a
25 patiial image which is a part of an entire image is generated as extended data of the
MPD, and an extended MPD which is extended to include metadata used for supply
of a bit stream of the entire image and supply of a bit stream of the partial image, that
is, the partial image information, is generated using the generated partial image
information.
30 [0066]
The partial image to be supplied may be any partial image as long as the
SP353396WOOO
18/161
partial image is a pmt of the entire image, and the shape, size, etc. are arbitrary. For
example, the partial image may be a part which can be encoded independently from
other portions. Hereinafter, however, to facilitate the description, the partial image
is assumed to be an image in units of tiles described above. That is, the partial
5 image is assumed to be formed by a single tile image or a plurality oftile images.
[0067]
The MPD has a layered stmcture, for example, layers of an adaptation set
(AdaptationSet), a representation (Representation), a sub-representation (SubRepresentation),
and a sub-segment (Sub-Segment). Any of these layers may be
10 extended.
[0068]
For example, a description for a tile (Tile) is defined utilizing a descriptor
type element (DescriptorType element) of the MPD. For example, a description for
a tile called a viewpoint (Viewpoint) is defined as in FIG. lOA.
15 [0069]
The viewpoint is an element which is present in the adaptation set
(AdaptationSet). The viewpoint is a description that defines what the view is. For
example, the viewpoint defines whether the view is a right (R) image or a left (L)
image of a stereo image.
20 [0070]
That is, when the adaptation set is extended, an element of the related art is
used (extended). By using the element of the related at1, it is possible to suppress a
reduction in affinity to an MPD of the related art (it is possible to suppress an
increase in a description which may not be analyzed by a decoder of the related art).
25 On the other hand, when the representation (Representation) or the subrepresentation
(Sub-Representation) is extended, a new element is defined.
[0071]
In the element of the viewpoint described above, a schema (schemeidUri)
for storing the partial image information is defined. In the case of the example of
30 FIG. lOA, (urn:mpeg:DASH:tile:2013) is defined as a schema for a tile. The
extension of the schema is performed when any of the adaptation set, the
SP353396WOOO
19/161
representation, and the sub-representation is extended.
[0072]
Further, values of schema (urn:mpeg:DASH:tile:2013) for the new tile are
defined. In the values, the above-described partial image information is defined.
5 For example, a view type((!) viewtype) indicating what an image indicated by the
element is, information ((2) the width and the height of an entire image) regarding
the size of the entire image, information ((3) the x coordinate and they coordinate of
the image indicated by the element) indicating the position of a partial image in the
entire image, and group identification information (( 4) TilegroupiD) identifying a
10 group to which the partial image belongs and which is a group of the partial images
displayable as one image are defined as the values.
[0073]
The view type (viewtype) is information indicating, for example, whether
the image is a tile image, as illustrated in FIG. lOB. For example, a value when the
15 image is an entire image is assumed to be "0," a value when the image is a tile image
and a bit stream is divided for each tile as in the example of FIG. 6A is assumed to be
"I," and a value when the image is a tile image and data of all the tiles is collected in
one bit stream as in the example of FIG. 6B is assumed to be "2." These values and
states (definitions of the values) indicated by the values are decided in advance. Of
20 course, the method of defining these values is arbitrary and an example other than
this example may be used. By referring to these values, it is possible to easily
comprehend whether it is necessary to refer to another element (that is, whether
another tile is present). In patticular, when the image is an entire image, it is
possible to easily comprehend that it is not necessary to refer to another element
25 merely by referring to this value.
[0074]
The information (the width and the height of the entire image) regarding the
size of the entire image is information indicating the size (the horizontal width and
the height) of an image in which all of the tile images belonging to the same group as
30 the image (the tile image) are unified, ,as illustrated in FIG. lOB. In the case of an
MPD of the related art, it is assumed that the sizes of images of bit streams are the
SP353396WOOO
20/161
same as the size of a display image. When the partial images are supplied, as
described above, the sizes of the images of the bit streams are different from the size
of the display image in some cases. For example, when a plurality of tile images of
mutually different bit streams are unified to be displayed, the size of the display
5 image can be larger than the sizes of the images of the bits streams in some cases.
In order to cope with such cases, the size of an image in which all of the tile images
belonging to the same group as the image (the tile image) are unified is indicated.
That is, by referring to this value, it is possible to easily comprehend a maximum
processing load when all of the tile images belonging to the same group of the image
10 (the tile image) are decoded. In the case of the example of FIG. lOB, the size
(1920xl080) of an image in which 4 (2x2) tile images with a 960x540 size are
unified is indicated as information regarding the size of the entire image.
[0075]
The information (the x coordinate and the y coordinate of the image
15 indicated by the element) indicating the position of the partial image in the entire
image is information indicating where the image in which all of the tile images
belonging to the same group as the image (tile image) are unified is located, as
illustrated in FIG. lOB. Expression of the position (indicating with which value) is
arbitrary. For example, the position may be expressed with the coordinates of the
20 upper left of the image. For example, the position may be expressed with another
piece of information such as identification information regarding the tile or the
coordinates of another location other than the upper left. By referring to this value,
it is possible to easily comprehend the position of the image when the images (tile
images) are unified (combined). That is, by referring this value of each tile image
25 to be unified (combined), it is possible to easily comprehend how each tile image is
arranged and unified (combined).
[0076]
The group identification information (TilegroupiD) is identification
information indicating a group of the tile images to which the image belongs, as
30 illustrated in FIG. 1 OB. The same value can be assigned to the tile images of the
same group. In contrast, different values can be assigned to respective groups. In
SP35339GWOOO
21/161
the case of the example of FIG I OB, since the tile images of tile I (Tile I) to tile 4
(Tile 4) can be unified, the same value can be assigned as group identification
information to the tile images. By referring to this value, it is possible to easily
comprehend which tile images can be unified (combined). In other words, it is
5 possible to easily identify other tile images to be unified (combined) with the image
at the time of display.
[0077]
The group identification information (TilegroupiD) may be defined not as
the value of the viewpoint but as an attribute of another element, for example, as
10 follows.
[0078]

[0079]
In the adaptation set, an attribute called a group is already present. In the
15 foregoing example, a meaning can be assigned as a set (Tilegroup) of tiles (Tile) to
the group.
20
[0080]

[0081]
On the other hand, an attribute called group is not present in the
representation or the sub-representation. That is, when the representation or the
sub-representation is extended, a new attribute called (group) is set.
[0082]
The above-described extension method can also be applied when a bit
25 stream is filed (in particular, MP4 filing) as in the example of FIG 7 or 8. In this
case, since the header information or the like of the bit stream assigned to other
tracks is assigned to the base track (Base Track), positional information regarding the
segment is not necessmy. For this reason, in the description (viewpoint)
corresponding to the base track, a value which is not the actual coordinates may be
30 defined as information regarding the position of the image. For example, NULL,
empty, space, or the like may be set. For example, a considerably large value or a
SP35339GWOOO
221161
negative value may be set as the coordinates. Of course, identification (a flag or the
like) indicating the base track may be separately provided.
[0083]
In the case of an MPD of the related att, segments (Segment) are necessarily
5 present under the representation (Representation). That is, a URL of an MP4 file is
described in segments immediately under the representation. The subrepresentation
(Sub-Representation) is, for example, information that is used to
reproduce only trickplay or music and designates data of a part in the MP4 file of the
segment immediately under the representation.
10 [0084]
When the MPD is extended so that the partial image information can be
included, the MPD may be extended so that segments are present under the subrepresentation
(Sub-Representation). That is, a tile image may be assigned to the
sub-representation so that the URL of the MP4 file can be refetTed to.
15 [0085]
More specifically, tags of a base URL ( ), a segment base
( ), a segment list ( ), a segment template
(), and the like are additionally defined in the sub-representation.
[0086]
20 In this case, however, it is necessary to generate segment information
indicating that the information regarding the bit stream is present under the subrepresentation
(Sub-Representation) as the partial image information and store the
segment information m the MPD. For example, a flag
(@SegmentlnSubRepresentation: true or false) indicating whether the information
25 regarding the bit stream is present under the sub-representation is defined as the
segment information.
[0087]
In this way, the representation can be configured by the sub-representations
of the plurality of tile images. By realizing such a stmcture, separation from the
30 representation of the related art is possible.
[0088]
5
SP35339GWOOO
23/161
In the case of an MPD of the related mi, a segment (Segment) expresses a
concept of time, and thus the segments of the same time are not permitted to be
present in one representation (Representation).
(0089]
When the MPD is extended so that the partial image information is included,
the MPD may be extended so that a plurality of segments of the same time can be
present in one representation by assigning the tile images to the segments.
(0090]
In this case, however, it is necessary to generate multi-segment information
10 indicating that the plurality of segments to which the tile images of the same time are
assigned are present as partial image information under the representation and store
the multi-segment information 111 the MPD. For example, a flag
(@multiSegmentlnRepresentation: true or false) indicating whether the plurality of
pieces of information regarding the bit streams of the same time are present under the
15 representation is defined as the multi-segment information.
(0091]
In this way, separation from the segments of the related art is possible.
(0092]
The segment can be designated only in access units (AU) in the related art,
20 but the sub-segment (Sub-Segment) assigning an ssix box extended so that data in
units of tiles can be designated may be defined under the segment to which an MP4
file storing a bit stream of a single tile image or a plurality of tile images is assigned.
That is, under segment to which an MP4 file is assigned, one sub-segment or a
plurality of sub-segments including an ssix designating the tile corresponding to the
25 segment from the MP4 file may be present.
(0093]
In this way, a unit smaller than a sample 111 the sub-segment can be
expressed.
[0094]
30 To this end, it is necessary to allow segment information to be false
(@SegmentlnSubRepresentation = false) and define viewpoint (Viewpoint) in the
5
SP353396WOOO
24/161
segment for implication. That is, from the two pieces of information, it is possible
to comprehend that the tile image is expressed in accordance with the sub-segment
(that the MP4 file is extended).
[0095]
Dedicated flag information may be separately defined to clarify that the tile
Image is expressed in accordance with the sub-segment (that the MP4 file is
extended).
[0096]
The partial image information is not limited to the above-described
'
10 examples, but any partial image information can be used. For example, in the
values, information other than the information (a view type ((!) viewtype), the
information ((2) the width and the height of an entire image) regarding the size of the
entire image, the information ((3) the x coordinate and the y coordinate of the image
indicated by the element) indicating the position of a partial image in the entire
15 image, and the group identification information ((4) TilegroupiD) identifYing a group
to which the partial image belongs and which is a group of the partial images
displayable as one image) indicated in the above-described example may be defined.
Further, flag information other than the above-described flag information may be
defined as pmiial information.
20 [0097]
25
By generating the paiiial image information, as described above, and
extending the MPD (metadata) using the partial image information, it is possible to
realize the adaptive supply of the data of the partial image using the metadata.
[0098]
<2. Second embodiment>

Next, a device realizing the above-described present technology and a
method therefor will be described. FIG. 11 is a diagram illustrating a delivery
system which is a kind of the system to which the present technology is applied. A
30 delivery system 100 illustrated in FIG. 11 is a system that can adaptively deliver data
of a partial image which is a pmi of an entire image.
5
SP353396WOOO
25/161
[0099]
As illustrated in FIG II, the delivery system 100 includes a delivery data
generation device 101, a delivery server 102, and a terminal device 103.
[01 00]
The delivery data generation device 101 generates, for example, files of
content such as an image and audio delivered by the delive1y server I 02 and MPD
files of the files and supplies the content files and the MPD files to the delivery
server I 02. The delive1y server I 02 publishes the content files and the MPD files
supplied from the delive~y data generation device I 0 I on a network 104 and
10 pe!forms adaptive delivery of partial images.
[0101]
The terminal device I 03 accesses the delivery se1ver 102 via the network
104 and acquires the MPD file of desired content published by the delivery server
102.
15 [0102]
The terminal device I 03 accesses the delivery server I 02 via the network
I 04 according to the MPD file, adaptively selects a proper content file corresponding
to the MPD file, and acquires the content file by an HTTP protocol. The terminal
device 103 reproduces the acquired content file.
20 [0103]

FIG 12 is a block diagram illustrating an example of a main configuration
of the delivery data generation device I 01. As illustrated in FIG 12, the delivery
data generation device 101 includes a screen division processing unit 121, an image
25 encoding unit 122, a file generation unit 123, a tile type image information
generation unit 124, an MPD generation unit 125, and a server upload processing
unit 126.
[0104]
The screen division processmg unit 121 edits (processes) image data
30 supplied from the outside to divide the entire image of the image data for each tile
and generates the image data of the tile images. The screen division processing unit
SP35339GWOOO
261161
121 supplies the image data of each tile generated in this way to the image encoding
unit 122. The screen division processing unit 121 supplies, for example,
information regarding the tile stmcture such as the size, the position, or the like of
each tile to the tile type image information generation unit 124.
5 (0105)
The image encoding unit 122 encodes the image data of each tile supplied
from the screen division processing unit 121 to generate a bit stream. As illustrated
in FIG 12, the image encoding unit 122 includes a plurality of encoding processing
units such as an encoding processing unit 131, an encoding processing unit 132, an
10 encoding processing unit 133, etc. and can encode the image data of each tile of the
supplied tiles in parallel. As described with reference to FIG 6 and the like, the
image encoding nnit 122 can generate any number of bit streams from one piece of
image data. The image encoding unit 122 can also collect the plurality of pieces of
image data into one bit stream. For example, the image encoding unit 122 can also
15 generate the bit stream for each tile image and can also collect the plurality of tile
images into one bit stream. The image encoding unit 122 supplies the generated bit
stream to the file generation unit 123.
(0106)
The encoding method of the image encoding unit 122 is arbitrary. The
20 encoding processing units perform the same encoding method or may perform
mutually different encoding methods.
(01 07)
The file generation unit 123 files the supplied bit stream in accordance with
a predetermined format such as an MP4 file format to generate the content file. As
25 described with reference to FIGS. 7 and 8 and the like, the file generation unit 123
can file one bit stream into any number of files. The file generation unit 123 can
also collect the plurality of bit streams into one file. The file generation unit 123
supplies the generated .content file to the MPD generation unit 125. The file
generation unit 123 supplies information regarding the filing such as how to file each
30 bit stream to the tile type image information generation unit 124.
(0108)
SP353386WOOO
27/161
The file generation unit 123 can perform the filing in accordance with any
format.
[01 09]
The tile type image information generation unit 124 generates tile type
5 image information (that is, pa!1ial image information) to match the MPD to the tile
structure based on the information regarding the tile structure supplied from the
screen division processing unit 121, the information regarding the filing supplied
from the file generation unit 123, or the like. The tile type image information (the
paiiial image information) is information including the content described in the first
10 embodiment and is generated as, for example, the values of the viewpoint or the flag
information. The tile type image information generation unit 124 supplies the
generated tile type image information to the MPD generation unit 125.
[0110]
The MPD generation unit 125 generates the MPD regarding the content file
15 supplied from the file generation unit 123, extends the MPD using the tile type image
information (the pal1ial image information) supplied from the tile type image
information generation unit 124, and generates the tile type MPD corresponding to
the tile structure. The MPD generation unit 125 supplies the file (MPD file) of the
generated tile type MPD and the content file to the server upload processing unit 126.
20 [Olll]
25
The server upload processing unit 126 uploads the supplied MPD file or
content file to the delivery server 102 (FIG 11) to publish the MPD file or the
content file.
[0112]
The delivery data generation device 101 generates the tile type MPD
con·esponding to the tile structure in this way, and thus the delivery server 102 can
adaptively deliver (supply) the data of the pal1ial images which are based on the
DASH standard. That is, the delivery system 100 can realize the adaptive supply of
the data of the pal1ial images.
30 [0113]
The above-described processing units may be configured as independent
._!
SP353396WOOO
281161
devices. In patticular, the tile type image information generation unit 124 or the
MPD generation unit 125 may be configured as independent devices. That is, the
configuration related to the generation of the content file is not requisite and only the
generation of the tile type image information (the pattial image information) may be
5 performed. For example, the tile type image information (the partial image
information) may also be generated based on information supplied fi·om another
device. For example, the generated tile type image information (the partial image
information) may be supplied to another device.
[0114]
10 Only the generation of the tile type MPD may be performed. For example,
the tile type MPD corresponding to the content file generated in another device may
be generated using the tile type image information (the pattial image information)
supplied from the other device. The generated MPD file may also be supplied to
another device.
15 [0115]
As in a tile type MPD generation unit 141, the tile type image information
generation unit 124 and the MPD generation unit 125 may be integrated. For
example, the tile type MPD generation unit 141 may be configured as one
independent device.
20 [0116]

FIG. 13 is a block diagram illustrating an example of a main configuration
of the terminal device 103. As illustrated in FIG. 13, the terminal device 103
includes an MPD acquisition unit !51, a parsing processing unit !52, a tile image
25 selection unit 153, a file acquisition unit 154, an image decoding unit 155, a tile
image combination unit 156, and a display unit 157.
[0117]
The MPD acquisition unit 151 acquires the MPD file of desired content
from the delivery server I 02 via the network 104 based on, for example, an
30 instruction of a control program or a user of the terminal device 103. The MPD
acquisition unit 151 supplies the acquired MPD file to the parsing processing unit
152.
[0118]
SP353396WOOO
29/161
The parsing processing unit 152 analyzes (parses) the supplied MPD file.
The parsing processing unit 152 also analyzes (parses) the tile type image
5 information (the pmtial image information) included in the MPD file. The parsing
processing unit 152 supplies an analysis result to the tile image selection unit 153.
[0119]
When the tile image selection unit 153 acquires tile image designation
information which is supplied ii"01n the outside and used to designate a pmtial image
10 (an image formed from a single tile image or a plurality of tile images) to be
reproduced, the tile image selection unit 153 selects the tile image designated by the
tile image designation information among the tile images included in the tile type
image information based on the analysis result of the MPD file (the tile type image
information) in the parsing processing unit 152. The tile image selection unit 153
15 supplies the URL (delivery address) of the file of the selected tile image to the file
acquisition unit 154.
[0120]
The file acquisition unit 154 accesses the delivery address of the delivery
server 102 supplied from the tile image selection unit 153 via the network 104 to
20 acquire the desired content file. The file acquisition unit 154 acquires the bit stream
from the acquired content file and supplies the bit stream to the image decoding unit
!55.
[0121]
The image decoding unit 155 decodes the bit stream supplied from the file
25 acquisition unit 154 to obtain the image data of the tile image. As illustrated in FIG.
13, the image decoding unit 155 includes a plurality of decoding processing units
such as a decoding processing unit 161, a decoding processing unit 162, a decoding
processing unit 163, etc. and can decode the plurality of supplied bit streams in
parallel. The image decoding unit 155 supplies the image data of the tile image
30 obtained by decoding the bit stream to the tile image combination unit 156.
[0122]
H
. '
SP353396WOOO
30/161
The image decoding unit 155 can perform the decoding in accordance with
any decoding method that corresponds to the encoding method of the image
encoding unit 122. Accordingly, each decoding processing unit may also perform
the decoding in accordance with the same method or may also perform the decoding
5 in accordance with mutually different methods.
[0123]
When the image data of the plurality of tile images belonging to the same
group is supplied from the image decoding unit 155, the tile image combination unit
156 combines (unifies) the tile images and combines the image data so that one
10 image is formed. That is, the tile image combination unit !56 generates the image
data of an image for display. When the images are not combined (for example,
when a single tile image is displayed or when a plurality of tile images are already
formed as one bit stream at the time of delivery), the supplied images are considered
to be images for display. The tile image combination unit 156 supplies the image
15 data for display to the display unit 157.
[0124]
20
The display unit 157 reproduces the supplied image data for display and
displays the image for display on a display.
[0125]
As described above, the terminal device 103 can con·ectly analyze the tile
type MPD corresponding to the tile structure and can gain the adaptive delivery
(supply) of the data of the partial image by the delivery server 102 which is based on
the DASH standard. That is, the data of the partial image can be correctly acquired
from the delivery server 102 and can be reproduced. That is, the delivery system
25 100 can realize the adaptive supply of the data of the partial image.
[0126]
As described above, the terminal device 103 can display the image with a
different image size Jiom the image size at the time of the delivery. That is, the
terminal device 103 can control the data delivery more adaptively according to a load
30 situation or the like of the delivery server 102 or the network 104. For example,
since whether to acquire the entire image or acquire the tile image can be controlled,
SP35339GWOOO
31/161
the number of acquired content files can be appropriately increased or decreased
without changing the size of the display image. Therefore, it is possible to
appropriately perform control such as distribution or concentration of a delivery
source or a path.
5 [0127]
The above-described processing units may be configured as independent
devices. In pmiicular, the parsing processing unit !52 or the tile image selection
unit 153 may be configured as independent devices. That is, the configuration
related to the acquisition or reproduction (decoding) of the content file is not
10 requisite and only the analysis of the tile type MPD or the tile type Image
information (the partial image information) may be performed. For example, the
MPD file acquired from the delivery server I 02 by another device may be analyzed.
For example, the analysis result may be supplied to another device.
15
[0128]
As in the tile type image information processmg unit 171, the parsing
processing unit 152 and the tile image selection unit 153 may be integrated. For
example, the tile type image information processing unit 171 may be configured as
one independent device.
[0129]
20 The image data for display output from the tile image combination unit !56
may be supplied to another device or may be recorded on a recording medium. At
this time, the image data may be encoded.
[0130]

25 Next, the flow of each process performed by each device of the delivery
system 100 described above will be described. First, an example of the flow of a
delivery data generation process by the delivery data generation device 10 1 will be
described with reference to the flowcha1i ofFIG 14.
[0131]
30 When the delivery data generation process starts, the screen division
processing unit 121 of the delivery data generation device 101 edits (processes) the
SP353396WOOO
32/161
image data so that a screen (that is, an entire image) is divided into tiles in step S101.
[0132]
In step S 102, the image encoding unit 122 encodes the image data of each
tile image generated in step S 1 0 I .
5 [0133]
In step S103, the file generation unit 123 files the encoded data (bit stream)
generated in step S 102 (that is, generates the content file).
[0134]
In step S104, the tile type MPD generation unit 141 generates the file of the
10 tile type MPD according to the processing result such as the division of step S 101 or
the filing of step S 103.
[0135]
In step S 105, the server upload processing unit 126 uploads the MPD file
and the content file generated in this way to the delivery server 102.
15 [0136]
When the process of step S 105 ends, the delivery data generation process
ends.
[0137]

20 Next, an example of the flow of a tile type MPD file generation process
performed in step S 104 of FIG. 14 will be described with reference to the flowchart
ofFIG. 15.
[0138]
When the tile type MPD file generation process starts, the tile type image
25 information generation unit 124 sets the schema (for example,
urn:mpeg:DASH:tile:2013) of the tile type image information, for example, in the
element of the viewpoint in step S 121.
[0139]
In step S 122, the tile type image information generation unit 124 sets a view
30 type (viewtype) in the value of the schema as the tile type image information.
[0140)
.:--1
'_l
5
SP35339GWOOO
33/161
In step S123, the tile type image information generation unit 124 sets the
size (width and height) of the entire image in the value of the schema as the tile type
image information.
(0 141]
In step S124, the tile type image information generation unit 124 sets the
position (x and y) of the tile image in the value of the schema as the tile type image
information.
(0142]
In step S125, the tile type image information generation unit 124 sets the
10 group identification information (TilegroupiD) in the value of the schema as the tile
type image information.
(0143]
In step S126, the tile type image information generation unit 124 sets the
segment information (@SegmentinSubRepresentation), as necessa1y, as the tile type
15 image information. For example, when the MPD is extended so that the segment is
present under the sub-representation (Sub-Representation), the tile type image
information generation unit 124 generates the segment information indicating that
the information regarding the bit stream is present under the sub-representation (SubRepresentation).
20 (0144]
In step S127, the tile type image information generation unit 124 sets, the
multi-segment information (@multiSegmentinRepresentation), as necessary, as the
tile type image information. For example, when the tile images are assigned to the
segments and the MPD is extended so that the plurality of segments of the same time
25 are present in one representation, the tile type image information generation unit 124
generates the multi-segment information indicating that the plurality of segments to
which the tile images of the same time are assigned are present under the
representation.
30
[0145]
When the process of step S 127 ends, the tile type MPD file generation
process ends and the process returns to FIG. 14.
SP35339GWOOO
34/161
[0146]
By performing the above-described processes, the delivery data generation
device 101 can allow the delivery server 102 to adaptively deliver (supply) the data
of the partial images which are based on the DASH standard. That is, it is possible
5 to realize the adaptive supply of the data of the partial images.
[0147]

Next, an example of the flow of the delivery data reproduction process
performed by the terminal device I 03 will be described with reference to the
10 flowchart of FIG. 16.
[0148]
When the delivery data reproduction process starts, the MPD acquisition
unit 151 acquires the MPD file corresponding to the desired content from the
delivery server I 02 in step S 141.
15 [0149]
In step Sl42, the parsing processing unit !52 analyzes (parses) the MPD file
acquired in step S 141.
[0150]
In step Sl43, the parsing processing unit !52 analyzes (parses) the tile type
20 image information (the pmtial image information) included in the MPD file.
[0 151]
In step Sl44, the tile image selection unit 153 selects the tile images
designated by the tile image designation information supplied from the outside
among the tile images indicated in the tile type image information.
25 [0152]
In step S 145, the file acquisition unit 154 acquires the file of the tile images
selected in step S144.
[0153]
In step S 146, the image decoding unit 155 decodes the bit stream of the tile
30 images included in the file acquired in step.Sl45.
[0154]
5
SP35339GWOOO
35/161
In step S147, the tile image combination unit 156 edits (processes) the
image data of the tile images obtained by decoding the bit stream in step Sl46 so that
the tile images are combined, as necessary.
[0155]
In step Sl48, the display unit 157 displays the image for display such as the
combined image of the tile images obtained in step Sl47 on a display.
[0156]
When the process of step S 148 ends, the delivery data reproduction process
ends.
10 [0157]
As described above, by executing the delivery data reproduction process,
the terminal device I 03 can correctly analyze the tile type MPD corresponding to the
tile stmcture and can gain the adaptive delivery (supply) of the data of the pa1tial
image by the delivery server 102 which is based on the DASH standard. That is,
15 the data of the partial image can be coiTectly acquired from the delivery server 102
and can be reproduced. That is, it is possible to realize the adaptive supply of the
data of the partial image.
[0158]
The above-described adaptive delivery (supply) of the pmtial images can be
20 used together with the delivery (supply) of the entire image. That is, for example,
the server may adaptively deliver the entire image or any partial image according to a
request or the like from the terminal.
[0159]
<3. Third embodiment>
25
Next, a specific example of an MPD extension method will be described.
[0160)

A main configuration example of the extended MPD is illustrated in FIG. 17.
30 In the case ofthe example of FIG. 17, the encoded data of each tile of the image data
to be delivered is configured in one bit stream (MP4 file) (bitstream3 .mp4 to
SP353396WOOO
36/161
bits(ream6.mp4). In the MPD, the adaptation set (AdaptationSet) is extended and
the bit stream (MP4 file) of each tile image is defined in mutually different
adaptation sets. A viewpoint (Viewpoint) which is a description for a tile is defined
in the adaptation set and the URL of the bit stream (MP4 file) of the tile
5 corresponding to the viewpoint is set in the segment (Segment) under the
representation (Representation) under the adaptation set.
[0161]
That is, the partial image information regarding the plurality of partial
images belonging to the same group is stored in the mutually different adaptation sets,
10 and the bit streams of the plurality of patiial images are assigned to the mutually
different adaptation sets.
[0162]
In the case of this example, as illustrated in FIG. 17, the adaptation sets of
the tile images ananged with the adaptation set of the entire images (bitstreaml.mp4
15 and bitstream2.mp4) can be provided, and thus the delivery of the entire images and
the adaptive delivery of the partial images can be managed in a unified manner.
[0163]
In DASH of the related art, for example, images with different displayed
content such as R and L images of a stereo image are defined in mutually different
20 adaptation sets in many cases. In this example, the tile images are defined in the
mutually different adaptation sets in imitation of such a way. Therefore, even in the
delivery control of the partial images, it is possible to realize a natural way close to
the related art. Therefore, development can be facilitated.
25
[0164]
In the example of FIG. 17, the entire images with different resolutions are
defined in the same adaptation set, but these entire images may be defined in
mutually different adaptation sets.
[0165]
A specific description example of the MPD of this example is illustrated in
30 FIG. 18.
[0166]
SP35339GWOOO
37/161

Another configuration example of the extended MPD is illustrated in FIG
19. In the case of the example of FIG 19, all of the encoded data of the tiles of the
image data to be delivered is configured in one bit stream (MP4) (bitstream3.mp4 to
5 bitstream6.mp4). In the MPD, the adaptation set (AdaptationSet) is extended and
the bit stream (MP4 file) of each tile image is defined in a different adaptation set
from the adaptation set in which the entire image is defined. However, unlike the
case of, the bit streams (MP4 files) of the tile images are defined in the
same adaptation set.
10 [0167]
A viewpoint (Viewpoint) which is a description for a tile is defined in a
representation (Representation) under the adaptation set and the URL of the bit
stream (MP4 file) of the tile corresponding to the viewpoint is set in the segment
(Segment) under the representation.
15 [0168]
That is, the partial Image information regarding the plurality of partial
Images belonging to the same group is stored in the mutually different
representations belonging to one adaptation set of metadata, and the bit streams of
the plurality of partial images are assigned to the mutually different representations.
20 [0169]
In the case of this example, as illustrated in FIG 19, the adaptation sets of
the tile images arranged with the adaptation set of the entire images can be provided,
and thus the delivery of the entire images and the adaptive delivery of the partial
images can be managed in a unified manner.
25 [0170]
30
In the example of FIG 19, the entire Images (bitstreaml.mp4 and
bitstream2.mp4) with different resolutions are defined in the same adaptation set, but
these entire images may be defined in mutually different adaptation sets:
[0171]

Another configuration example of the extended MPD is illustrated in FIG
SP353396WOOO
38/161
20. In the case of the example of FIG. 20, the encoded data of the tiles of the image
data to be delivered is collected in one bit stream. The bit stream is filed as an MP4
file for each tile (bitstream7 Tilel.mp4 to bitstream7 Tile4.mp4). As described
with reference to FIG. 7, a base track in which the header information or the like of
5 the tiles is collected is filed separately from the bit streams of the tiles
(bitstream7 base.mp4).
[0172]
In the MPD, the adaptation set (AdaptationSet) is extended and the bit
streams (MP4 files) (bitstream7 Tilel.mp4 to bitstream7 Tile4.mp4) of the tile
10 images are defined in mutually different adaptation sets.
[0173]
A viewpoint (Viewpoint) which is a description for a tile is defined in the
adaptation set and the URL of the bit stream (MP4 file) of the tile corresponding to
the viewpoint is set in the segment (Segment) under the representation
15 (Representation) under the adaptation set.
[0174]
In the x and y coordinates defined in the value of the viewpoint of the bit
stream (MP4 file) (bitstream7 base.mp4) of the base track, as described in the first
embodiment, apparently different values such as NULL or the like from normal
20 coordinates are set. In the value of the view type defined in the value of each
viewpoint, a value ("2" in the case of the example ofFIG. 20) indicating a tile (Tile)
in which the encoding scheme such as HEVC or the like is supported is set.
[0175]
That is, the partial image information regarding the plurality of partial
25 images belonging to the same group is stored in the mutually different adaptation sets
of the metadata, and the plurality of files for which one bit stream including the
plurality of partial images is divided for each partial image are assigned to the
mutually different adaptation sets.
[0176]
30 A specific description example of the MPD of this example is illustrated in
FIG. 21.
SP35339GWOOO
39/161
[0177]

Another configuration example of the extended MPD is illustrated in FIG.
22. In the case of the example of FIG. 22, the extension method is the same as that
5 of . The tiles are set such that the sizes are unequal, as illustrated in
FIG. 22 ( CO!Tesponding to FIG. 9B). In this case, an image with a desired size can
be obtained by adding tiles, as shown with quadrangles.
[0178]
In the case of this example, each piece of encoded data of each tile of the
10 image data to be delivered is configured in one bit stream (MP4 file) (tilel.mp4 to
tile5.mp4). Therefore, no base track is present as in .
[0179]
That is, partial image information regarding control information included in
the bit stream is fiuiher generated, the partial image information regarding the
15 control information is stored in a different adaptation set fi·om the partial image
information regarding each partial image, and a file of the control information is
assigned to the adaptation set.
[0180]

20 Another configuration example of the extended MPD is illustrated in FIG.
23. In the case of the example of FIG. 23, each piece of encoded data of each tile of
the image data to be delivered is configured in one bit stream (MP4 file)
(bitstream3.mp4 to bitstream6.mp4). In the MPD, the representation
(Representation) is extended and the bit streams (MP4 files) of the tile images are
25 defined in mutually different representations under the same adaptation set as the bit
streams (MP4 files) (bitstreaml.mp4 and bitstream2.mp4) of the entire images.
[0 181]
A viewpoint (Viewpoint) which is a description for a tile is defined in the
representation and the URL of the bit stream (MP4 file) of the tile corresponding to
30 the viewpoint is set in the segment (Segment) under the representation.
[0182]
SP353396WOOO
40/161
That is, the pmiial unage information regarding the plurality of patiial
images belonging to the same group is stored in the mutually different
representations belonging to the same adaptation set of the entire images of the
metadata and the bit streams of the plurality of partial images are assigned to the
5 mutually different representations.
[0183]
That is, in the case of this example, as illustrated in FIG. 23, the
representations of the tile images arranged with the representations of the entire
images (bitstreaml.mp4 and bitstream2.mp4) can be provided, and thus the delivery
10 of the entire images and the adaptive delivery of the partial images can be managed
in a unified manner.
[0184]
A specific description example of the MPD of this example is illustrated in
FIG. 24.
15 [0185]

Another configuration example of the extended MPD is illustrated in FIG.
25. In the case of the example of FIG. 25, the encoded data of the tiles of the image
data to be delivered is collected in one bit stream. The bit stream is filed as an MP4
20 file for each tile (bitstream7 Tile l.mp4 to bitstream7 Tile4.mp4). As described
with reference to FIG. 7, a base track in which the header information or the like of
the tiles is collected is filed separately from the bit streams of the tiles
(bitstream7 base.mp4).
[0186]
25 In the MPD, the representation (Representation) is extended, and the bit
streams (MP4 files) (bitstream7 Tile l.mp4 to bitstreams7 Tile4.mp4) of the tile
images are defined in the mutually different representations under the same
adaptation sets.
[0187]
30 A viewpoint (Viewpoint) which is a description for a tile is defined in the
representation and the URL of the bit stream (MP4 file) of the tile corresponding to
SP35339GWOOO
411161
the viewpoint is set in the segment (Segment) under the representation.
[0188]
In the x and y coordinates defined in the value of the viewpoint of the bit
stream (MP4 file) (bitstream7 base.mp4) of the base track, as described in the first
5 embodiment, apparently different values such as NULL or the like from normal
coordinates are set. In the value of the view type defined in the value of each
viewpoint, a value ("2" in the case of the example of FIG 25) indicating a tile (Tile)
in which the encoding scheme such as HEVC or the like is supported is set.
[0189]
10 That is, partial image information regarding control information included in
one bit stream including the plurality of partial images belonging to the same group
is fiuiher generated, the partial image information regarding the plurality of partial
images is stored in the mutually different representations belonging to one adaptation
set of the metadata, the plurality of files for which the bit stream is divided for each
15 partial image are assigned to the mutually different representations, the partial image
information regarding the control information is stored in the different representation
from the partial image information regarding each partial image, and the file of the
control information is assigned to the representation.
[0190]
20 A specific description example of the MPD of this example is illustrated in
FIG 26.
[0191]

Another configuration example of the extended MPD is illustrated in FIG
25 27. In the case of the example of FIG 27, each piece of encoded data of each tile of
the image data to be delivered is configured in one bit stream (MP4 file)
(bitstream3.mp4 to bitstream6.mp4). In the MPD, the sub-representation (SubRepresentation)
is extended and the bit streams (MP4 files) of the tile images are
defined in mutually different sub-representations under the same adaptation set as the
30 bit streams (MP4 files) (bitstreaml.mp4 and bitstream2.mp4) of the entire images
and under different representations fi·om the bit streams (MP4 files) of the entire
'!
SP35339GWOOO
42/161
1mages.
[0192]
· A viewpoint (Viewpoint) which is a description for a tile is defined in the
sub-representation and the URL of the bit stream (MP4 file) of the tile corresponding
5 to the viewpoint is set in the segment (Segment) under the sub-representation.
[0193]
In the representation in which the bit stream (MP4 file) of each tile image is
defined, segment information (@SegmentinSubRepresentation =true) indicating that
information regarding the bit stream is present under the sub-representation is
10 defined.
[0194]
That is, the partial image information regarding the plurality of partial
Images belonging to the same group is stored in mutually different subrepresentations
belonging to one representation belonging to one adaptation set of the
15 metadata, and the bit streams of the plurality of partial images are assigned to the
mutually different sub-representations.
[0195]
That is, in the case of this example, as illustrated in FIG 27, the
representations of the tile images arranged with the representations of the entire
20 images (bitstreaml.mp4 and bitstream2.mp4) can be provided, and thus the delivery
of the entire images and the adaptive delivery of the partial images can be managed
in a unified manner.
[0196]
25 FIG 28.
[0197]
A specific description example of the MPD of this example is illustrated in

Another configuration example of the extended MPD is illustrated in FIG
29. In the case of the example of FIG 29, the encoded data of the tiles of the image
30 data to be delivered is collected in one bit stream. The bit stream is filed as an MP4
file for each tile (bitstream7_Tilel.mp4 to bitstream7_Tile4.mp4). As described
N
'
5
10
SP35339GWOOO
431161
with reference to FIG. 7, a base track in which the header information or the like of
the tiles is collected is filed separately from the bit streams of the tiles
(bitstream7 base.mp4).
[0198]
In the MPD, the sub-representation (Sub-Representation) is extended, and
the bit streams (MP4 files) (bitstream7 Tilel.mp4 to bitstream7 Tile4.mp4) of the
tile images are defined in the mutually different sub-representations under the same
representation (Representation) under the same adaptation set (AdaptationSet).
[0199]
A viewpoint (Viewpoint) which is a description for a tile is defined in the
sub-representation and the URL of the bit stream (MP4 file) of the tile corresponding
to the viewpoint is set in the segment (Segment) under the sub-representation.
[0200]
The viewpoint of a base track is defined in the representation above the sub-
15 representation and the URL of the bit stream (MP4 file) (bitstream7 base.mp4) of
the base track is set in the segment under the representation. In the representation
in which the bit stream (MP4 file) of each tile image is defined, segment information
(@SegmentinSubRepresentation =true) indicating that information regarding the bit
stream is present under the sub-representation is defined. Further, the segment
20 information ( @SegmentlnSubRepresentation = true) indicating that the information
regarding the bit stream is present under the sub-representation may be defined with
another constituent element (for example, AdaptationSet) of the MPD illustrated in
FIG. 4.
25
[0201]
In the x and y coordinates defined in the value of the viewpoint of the bit
stream (MP4 file) (bitstream7_base.mp4) of the base track, as described in the first
embodiment, apparently different values such as NULL or the like from normal
coordinates are set. In the value of the view type defined in the value of each
viewpoint, a value ("2" in the case of the example of FIG. 29) indicating a tile (Tile)
30 in which the encoding scheme such as HEVC or the like is supported is set.
[0202]
SP353396WOOO
44/161
That is, the partial image information regarding the control information
included in one bit stream including the plurality of partial images belonging to the
same group and the segment information indicating that the information regarding
the bit stream is present under the sub-representation (Sub-Representation) are
5 fmiher generated, the segment information and the partial image information of the
control information are stored in one representation belonging to one adaptation set
of the metadata, a file of the control information is assigned to the representation, the
partial image information regarding the plurality of partial images is stored in the
mutually different sub-representations belonging to the representation, and the
10 plurality of flies in which the bit stream is divided for each partial image are assigned
to the mutually different sub-representations.
(0203]
A specific description example of the MPD of this example is illustrated in
FIG. 30.
15 (0204]

Another configuration example of the extended MPD is illustrated in FIG.
31. In the case of the example of FIG. 31, the encoded data of the tiles of the image
data to be delivered is collected in one bit stream. The bit stream is filed as one
20 MP4 file as in the example of FIG. 8 (bitstream7.mp4).
(0205]
In the MPD, the sub-representation (Sub-Representation) is extended and
the bit stream (MP4 file) (bitstream7.mp4) of the tile image is defined under the
representation (Representation) under the adaptation set (AdaptationSet). In the
25 representation above the sub-representation, the viewpoint (Viewpoint)
corresponding to the bit stream (MP4 file) (bitstream7.mp4) of the tile image is
defined and the segment information (@SegmentinSubRepresentation = true)
indicating that information regarding the bit stream is present under the subrepresentation
is further defined.
30 [0206]
In the sub-representation under the representation, the viewpoint of each tile
SP353396WOOO
45/161
is set and the location of the data of each tile in (bitstream7.mp4) is designated with a
byte in the segment under the representation.
[0207]
That is, the segment information indicating that the information regarding
. 5 the bit stream is present under the sub-representation and the partial image
information of the control information included in one bit stream including the
plurality of partial images belonging to the same group are further generated, the
partial image information of the control information and the segment information are
stored in one representation belonging to one adaptation set of the metadata, the bit
10 stream is assigned to the representation, the pmtial image information regarding the
plurality of partial images is stored in mutually different sub-representations
belonging to the representation, and the information indicating the location of the
data of the partial images in the bit stream is assigned to the mutually different subrepresentations.
15 [0208]

Another configuration example of the extended MPD is illustrated in FIG
32. In the case of the example of FIG 32, the encoded data of the tiles of the image
data to be delivered is configured in one bit stream (MP4 file) (bitstream3.mp4 to
20 bitstream6.mp4). In the MPD, the segments (Segment) are extended and the
plurality of segments (Segment) are defined under the representations under the
adaptation set.
[0209]
In the representation, the viewpoint of a combined image of all the tile
25 images IS defined and the multi-segment information
(@multiSegmentinRepresentation =true) indicating that the plurality of segments to
which the tile images of the same time are assigned are present is d~fined under the
representation. Further, the segment information (@SegmentlnSubRepresentation
=true) indicating that the information regarding the bit stream is present under the
30 sub-representation may be defined with another constituent element (for example,
AdaptationSet) of the MPD illustrated in FIG 4.
SP35339GWOOO
46/161
[0210]
The bit streams (MP4 files) of the tile images are defined in mutually
different segments under the different representation from the bit streams (MP4 files)
of the entire images and under the same adaptation set as the bit streams (MP4 files)
5 (bitstreaml.mp4 and bitstream2.mp4) of the entire images.
[0211]
The viewpoint (Viewpoint) which is a description for a tile is defined in the
segment (Segment) and the URL of the bit stream (MP4 file) of the tile
con·esponding to the viewpoint is set in each segment (Segment).
10 [0212]
That is, the multi-segment information indicating that the plurality of pieces
of information regarding the bit streams of the same time are present under the
representation is further generated, the multi-segment information is stored in one
representation belonging to one adaptation set of the metadata, the partial image
15 information regarding the plurality of partial images belonging to the same group is
stored in the mutually different segments belonging to the representation, and the bit
streams of the plurality of partial images are assigned to the mutually different
segments.
20
[0213]
That is, as illustrated in FIG. 32, in the case of this example, the
representation of the tile image arranged with the representations of the entire images
(bitstreaml.mp4 and bitstream2.mp4) can be provided, and thus the delivery of the
entire images and the adaptive delivery of the partial images can be managed in a
unified manner.
25 [0214]
A specific description example of the MPD of this example is illustrated in
FIG. 33.
[0215]

30 Another configuration example of the extended MPD is illustrated in FIG.
34. In the case of the example of FIG. 34, the encoded data of the tiles of the image
SP35339GWOOO
47/161
data to be delivered is collectively configured in one bit stream (MP4 file)
(bitstream7.mp4). In the MPD, the sub-segments (Sub-Segment) are extended and
the plurality of sub-segments (Sub-Segment) are defined under the segment under the
representation under the adaptation set.
5 [0216]
10
In the representation, segment information (@SegmentinSubRepresentation
= false) indicating that the information regarding the bit stream is not present under
the sub-representation is defined.
[0217]
In the segment, the viewpoint of the combined image of all the tile images is
defined and the data of each tile image is shown in accordance with the ssix in the
sub-segment under the segment.
[0218]
That is, the segment information indicating that the information regarding
15 the bit stream is not present under the sub-representation and the partial image
information regarding one bit stream including the plurality of partial images
belonging to the same group are further generated, the segment information is stored
in one representation belonging to one adaptation set of the metadata, the partial
image information is stored in one segment belonging to the representation, the bit
20 stream is assigned to the segment, and the information indicating the location of the
data of each partial image in the bit stream is assigned to the mutually different subsegments
belonging to the segment.
[0219]
Of course, the MPD extension method is arbitrary and methods other than
25 the above-described methods may be used.
[0220]

Next, another example of the application using the adaptive delivery
(supply) of the tile images described above will be described.
30 [0221]
For example, in a system illustrated on the left of FIG 35, a mobile device
5
SP353396WOOO
48/161
221 is assumed to acquire a pariial image 212 with a 1920x1080 size formed by four
tile images 211 of an entire image 210 from a server 220 using a 3G line and
reproduce the partial image 212.
[0222]
To switch display on a television signal receiver (TV) 222, information
regarding a reproduction environment (network bandwidth), a reproduction ability
(resolution and a decoder ability), or the like of the TV 222 of a switching destination
is acquired from the TV 222. The method of acquiring the information is arbitrary.
For example, the mobile device 221 may acquire the information by performing
10 direct communication with the TV 222. Alternatively, the mobile device 221 may
acquire the information via the server 220.
[0223]
The mobile device 221 selects optimum tile images for the TV 222 of the
switching destination from the information regarding the MPD. In the case of the
15 example of FIG 35, a partial image 213 formed by the 5x5 tile images 211 is
selected.
[0224]
The TV222 of the switching destination acquires a bit stream of the tile
images selected in this way and reproduces the bit stream.
20 [0225]
25
30
The above-described selection or acquisition of the optimum stream may be
performed by the mobile device 221 to be pushed to the TV 222 of the switching
destination, or such selection or acquisition may be performed by the TV 222.
[0226]

For example, in a system illustrated on the left of FIG 36, a mobile device
221 is assumed to reproduce a part of an entire image (state 221A of the mobile
device)
[0227]
To reproduce another region by shifting a region during the reproduction, a
user of the mobile device 221 shifts the region with his or her finger on a touch panel
SP35339GWOOO
49/161
to move an image (as indicated by an arrow 233) so that a direction desired to be
reproduced is displayed on a screen. For example, when the user desires to display
an upper right region (pmiial image 232) of the currently displayed region (pmiial
image 231) as indicated by an arrow 234, the user traces his or her finger in the lower
5 left direction from the upper right of the screen.
[0228]
When such a user input is performed, the mobile device 221 calculates a
movement destination of the image based on the input finger motion or the like and
selects a stream of tile images to be displayed from the information regarding the
10 MPD.
15
20
25
[0229]
Then, the mobile device 221 acquires the selected bit stream from the server
220 and performs the reproduction and display (state 221B of the mobile device).
[0230]
The selection of the tile images may be performed by an application
executed in the mobile device 221, or the direction of the movement destination of
the image acquired 11-om the finger motion may be sent to the server 220 and the
images may be selected by the server 220.
[0231]
To actually move the images, a display region may be switched abmptly or
the display region may be gradually shifted and switched to perform smooth
switching.
[0232]
<4. Fourth embodiment>

FIG 37 is a diagram illustrating another example of an application using the
tile image delivery.
[0233]
In order to enable users to select favorite programs among programs of a
30 plurality of channels of broadcast or the like, for example, a menu is generated by
encoding images of the plurality of channels as one image (HD). A combined
SP353396WOOO
50/161
image combined so that such different images are arranged is defined as a mosaic
video.
[0234]
For example, in the case of devices having large displays as in television
5 signal receivers, users can easily comprehend content of each program from a mosaic
video in which programs of all channels are combined, select a desired program, and
display the program.
[0235]
However, in the case of mobile devices, displays of the mobile device are
10 small, and thus can display only images with small image sizes (low resolutions),
such as images in HD or lower. That is, only images with 1920x 1080 can be
delivered to such mobile devices.
[0236]
However, in such small image sizes, regions in which a program of each
15 channel of a mosaic video is displayed are too small, and thus it is difficult for users
to comprehend content of each program from such a mosaic video and select a
desired program.
[0237]
Accordingly, when a technology for adaptively supplying data of a pattial
20 image is applied, as described above, and a user selects and zooms in on a location at
which a program in which he or she is interested is pictured in a mosaic video, the
image is configured to be switched to another HD image in which images of fewer
programs are displayed. The user can easily display only a desired program by
repeating such zooming (image switching).
25 [0238]
In the case of the example of FIG 37, tiles within a range indicated by an
elliptical circle are assumed to be acquired and displayed by a mobile device. In the
leftmost mosaic video, an entire mosaic video can be displayed. At this time,
images of programs corresponding to 16 channels are displayed. In this state,
30 display regions (A to P) of the programs are too small, and thus it is difficult for a
user to select a desired program. Accordingly, when the user performs selection,
SP353396WOOO
51/161
for example, by tapping an upper left pmiion, a delivered file (bit stream) is switched
and an upper left tile image with an image size of 1920x I 080 in the mosaic video is
displayed, as illustrated in the middle of FIG 37. In the tile image, 4 programs (A,
B, E, and F) are displayed. That is, the number of displayed programs is reduced
5 and the display region per program is spread.
[0239]
Further, when the user performs selection, for example, by tapping an upper
left portion of the mosaic video, the delivered file (bit stream) is switched and an
upper left tile image with an image size of 1920xl080 in the mosaic video is
10 displayed, as illustrated on the right in FIG. 37. In the tile image, 1 program (A) is
displayed. That is, the number of displayed programs is further reduced and the
display region per program is spread.
[0240]
The switching of the delivered data described above is realized by extending
15 the DASH standard, as described above. That is, for example, the structure of the
mosaic video forming one screen is defined in the MPD so that the mosaic video can
be used as a user interface (UIIUX).
[0241]
For example, a relation between a screen stmcture and positional
20 information selected by the user is obtained and a stream to be subsequently switched
is selected. Coordinates touched on the screen by the user and coordinates on the
mosaic video are obtained and a mosaic video of a subsequent Layer (extension) in
which the coordinate position is included is obtained to be switched.
25
30
[0242]
New schemeidUri (urn:mpeg:DASH:mosaic:2013) is defined nsmg an
element (Viewpoint element) of the viewpoint. For example, the following
information is defined in content (patiial image information) of the value of the new
schemeidUri.
[0243]
· the number of mosaic images forming one screen
· a flag indicating that the sizes of the mosaic images are equal
SP35339GWOOO
521161
· the coordinates of the origin of the upper left of each mosaic image and
information regarding a width and a height when the sizes are not equal
[0244]
More specifically, a viewpoint is defined as follows. Then, the MPD is
5 extended using such pariial image information.
[0245]

10 [0246]
The element of the viewpoint is an element corresponding to the mosaic
video (urn:mpeg:DASH:mosaic:2013). In order to adaptively supply the data of the
partial image, as described above, it is necessary to further define the element of the
viewpoint for a tile, as illustrated in FIG lOA. That is, the element of the viewpoint
15 for mosaic video described above is positioned as an extension element of the
elements of the viewpoint for a tile.
[0247]
For example, in the case of a state in which a plurality of programs are
displayed, as illustrated on the upper side of FIG 38, it is necessary to define both of
20 the element of the viewpoint for a tile and the element of the viewpoint for mosaic
video in the adaptation set.
[0248]
On the other hand, in the case of a state in which only one program is
displayed, as illustrated on the lower side of FIG 38, as a narrowing result of the
25 programs by the user, the mosaic video is not fanned, and thus it is not necessary to
define the element of the viewpoint for mosaic video. However, in order to indicate
the entire image (Full video), it is necessary to define the element of the viewpoint
for a tile.
30
[0249]
When the sizes of the tile images are equal in the value of the element of the
viewpoint for mosaic video described above, the positional information regarding the
:J
5
SP353396WOOO
53/161
image is handled optionally. Writing may not be performed. When the writing is
performed, it is necessary to write all of the images. Further, information other than
the above-described information may be defined as a value.
[0250]
<5. Fifth embodiment>
< Examples of configuration of MP4 file and extension examples of MPD
corresponding thereto>
The examples of the configuration of the MP4 file have been described with
reference to FIGS. 7 and 8 in the first embodiment. However, an embodiment of
10 the present disclosure is not limited to the examples of the configuration of the MP4
file. Hereinafter, examples of the configuration of the MP4 file and examples of the
15
configuration (extension examples) of the MPD corresponding thereto will be
described.
[0251]

FIG 39 is a diagram illustrating an example of the configuration of an MP4
file obtained by filing the bit stream (bitstream7) having, for example, the tile (Tile)
structure illustrated in FIG 6B. In the case of the example of FIG 39, as in the
example of FIG 8A, bit streams of tiles are collected and considered as one file and
20 the data of the tiles is further managed as one track.
[0252]
Parameter sets such as a video parameter set (VPS), a sequence parameter
set (SPS), and a picture parameter set (PPS) are managed for a sample by a sample
entry (Sample Entry). Each tile is defined by a tile region group entry
25 (TileRegionGroupEntry) in a sample group description (Sample Group Description).
As illustrated in FIG 39, the values of 5 parameters, GroupiD which is identification
information identifying the tile, H_offset indicating the position (offset) of the tile in
the horizontal direction, V offset indicating the position (offset) of the tile in the
vertical direction, H width indicating the size (width) of the tile in the horizontal
30 direction, and V height indicating the size (height) of the tile in the vertical direction,
are defined as the tile region group entry (TileRegionGroupEntry).
SP353396WOOO
541161
[0253]
For example, in the tile region group entry (TileRegionGroupEntry) of tile 1
(Tile 1 ), GroupiD= I, H offset=O, V _ offset=O, H width=960, and V _height=540 are
defined. For example, in the tile region group ent1y (TileRegionGroupEnt1y) of tile
5 2 (Tile 2), GroupiD=2, H offset=960, V offset=O, H width=960, and V height=540
are defined. For example, in the tile region group entry (TileRegionGroupEntry) of
tile 3 (Tile 3), GroupiD=3, H_offset=O, V_offset=540, H_width=960, and
V height=540 are defined. For example, in the tile region group entry
(TileRegionGroupEntry) of tile 4 (Tile 4), GroupiD=4, H_offset=960, V offset=540,
10 H width=960, and V height=540 are defined. In this case, an entire image
(1920x1080) is formed by 4 tiles (960x540), 2 vertical tiles x 2 horizontal tiles.
[0254]
15
20
The file name of this MP4 file is assumed to be bitstream.mp4.
[0255)

To manage the MP4 file of the bit stream having the tile structure in the
example of FIG. 39, for example, an MPD of an MPEG-DASH standard of the
related art is extended, as in FIG. 40.
[0256)
In the case of the example of FIG. 40, an entire image and each tile are
defined in mutually different adaptation sets (AdaptationSet). In the topmost
adaptation set in the drawing defined in the entire image, as illustrated in FIG. 40, a
supplemental property (SupplementaiProperty) is defined as a description for a tile
instead of the viewpoint (Viewpoint) described in the first embodiment.
25 [0257]
The supplemental property (SupplementalProperty) is an element of the
related art. By using the element of the related art, it is possible to suppress a
reduction in affinity to an MPD of the related art (it is possible to suppress an
increase in a description in which a decoder of the related art is not analyzable).
30 The supplemental property is defined in the adaptation set in which the bit stream
decodable even in a decoder of the related art is defined. For example, in the case
SP35339GWOOO
55/161
of FIG 40, the supplemental property is defined in the adaptation set defined 111
regard to an entire image which can be decoded even in the decoder of the related art.
[0258]
For example, the supplemental property is extended and defined as follows.
5 [0259]

[0260]
That is, in the element of the supplemental property, schema (schemeidUri)
10 for storing image information is defined. In the case of the example of FIG. 40,
"urn:mpeg:dash:srd:2013" is defined as the schema.
[0261]
The value of the schema is defined. "source id" is identification
information indicating whether a content source of the adaptation set is the same as a
15 content source of another adaptation set. In the case of FIG 40, since the content
source of each adaptation set is common (bitstream.mp4), "I" is defined as "source
id."
[0262]
"x, y" is information indicating the position (x and y coordinates of the
20 upper left) of the tile defined by the adaptation set. In the case of FIG 40, since the
adaptation set defines the entire image, "0, 0" is defined as "x, y."
[0263]
"width, height" is information indicating the size (the width and the height)
of the tile defined by the adaptation set. In the case of FIG 40, since the adaptation
25 set defines the entire image, "1920, 1080" is defined as "width, height."
[0264]
"width_all, height_ all" is information indicating the size (the width and the
height) of the entire image. In the case of FIG 40, "1920, 1080" is defined as
"width_ all, height_ all."
30 [0265]
"stream type" is identification information indicating whether the adaptation
5
SP353396WOOO
56/161
set defines an entire bit stream or a pmi of the bit stream. In the case of FIG 40,
"0" indicating that the adaptation set defines the entire bit stream is defined as
"stream type."
[0266]
That is, in the case of the topmost adaptation set in the drawing in the
example of FIG 40, the supplemental propeiiy is defined as follows, for example.
[0267]

10 [0268]
As illustrated in FIG 40, in the adaptation set in which tile 1 (Tile 1) is
defined and which is the second adaptation set from the top of the drawing, an
essential property (Essentia!Property) is defined instead of the viewpoint (Viewpoint)
described as the description for a tile in the first embodiment.
15 [0269]
The essential prope1ty (Essentia!Property) is an element of the related art.
By using the element of the related art, it is possible to suppress a reduction in
affinity to an MPD of the related art (it is possible to suppress an increase in a
description in which a decoder of the related art is not analyzable). The essential
20 prope1ty is defined in the adaptation set in which the bit stream undecodable in a
decoder of the related art is defined. For example, in the case of FIG 40, the
essential property is defined in tlie adaptation set defined in regard to each tile image
which cannot be decoded in the decoder of the related art.
25
[0270]
That is, only a decoder which is capable of interpreting the essential
property decodes the bit stream managed by the adaptation set and a decoder which
is not capable of interpreting the essential property skips the adaptation set.
[0271]
For example, the essential prope1ty is extended as follows and is defined.
30 That is, the essential property is defined as in the supplemental property
(SupplementalProperty).
5
SP3533DGWOOO
57/161
[0272]

[0273]
In the case of the second adaptation set from the top of the drawing in the
example of FIG. 40, "urn:mpeg:dash:srd:2013" is defined as the schema. Further,
"!"is defined as "source id" of the value of the schema, "0, 0" is defined as "x, y,"
"960, 540" is defined as "width, height," "1920, 1080" is defined as "width_all,
height all," and "1" indicating the adaptation set defines a part of the bit stream is
10 defined as "stream type."
[0274]
When the value of"stream type" is "1," that is, when a pati of the bit stream
ts defined in the adaptation set, the essential property is further extended as
information indicating the part of the bit stream. For example, when the (Tile) of
15 HEVC is included in an MP4 file managed in the adaptation set, the adaptation set
corresponding to the tile corresponds to the part of the bit stream. In this case, the
essential property in regard to the part of the bit stream is further extended and
defined as follows, for example.
20
[0275]

[0276]
In this case, in the element of the essential propetiy, a schema
(schemeidUri) for storing information indicating a part of the file is defined. In the
25 case of the second adaptation set from the top of the drawing in the example of FIG.
40, "urn:mpeg:dash:hevc:2013" is defined as the schema.
[0277]
A value of the schema is defined. "Sub-Sample-Type" is information
indicating by which information a part of the bit stream to which the adaptation set
30 corresponds is configured. For example, when the value of the information is "0,"
it is indicated that the part of the bit stream is configured by Nal based. For
SP35339GWOOO
58/161
example, when the value of the information is "I," it is indicated that the pmi of the
bit stream is configured by Decoding-unit-based. Further, for example, when the
value of the information is "2," it is indicated that the. pmi of the bit stream is
configured by Tile-based. For example, when the value of the information is "3," it
5 is indicated that the part of the bit stream is configured by CTU-row-based. Further,
for example, when the value of the information is "4," it is indicated that the part of
the bit stream is configured by slice-based. In the case of the second adaptation set
from the top of the drawing in the example of FIG. 40, "2" is defined as "SubSample-
Type."
10 [0278]
"Sub-Sample-is-extracted" is information indicating whether a part of the
bit stream to which the adaptation set corresponds is divided (extracted) into tracks.
For example, when the value of the information is "0," it is indicated that the part of
the bit stream is not divided (false). When the value of the information is "1," it is
15 indicated that the part of the bit stream is divided into the tracks (!Iue). In the case
of the second adaptation set from the top of the drawing in the example of FIG. 40,
the number of tracks is I (not divided), as described with reference to FIG. 39, and
"0" is defined as "Sub-Sample-is-extracted."
20
[0279]
"ID" is identification information. When "2" is defined as "Sub-SampleType,"
that is, in the case of Tile, GrouplD of The tile region group entry
(TileRegionGroupEn!Iy) of the MP4 file is defined. In the case of the second
adaptation set from the top of the drawing in the example of FIG. 40, the pmt of the
bit stream is data of tile I (Tile 1), and thus "I" is defined as "ID."
25 [0280]
30
That is, in the case of the second adaptation set from the top of the drawing
in the example of FIG. 40, the essential property is defined as follows, for example.
[0281]

[0282]
Similarly, in the case of the third adaptation set from the top of the drawing
in the example of FIG 40, the essential prope~ty is defined as follows, for example.
5 [0283]
0,2">

[0286]

[0288]

[0304]
Even in the case of the example of FIG 42, in the adaptation set which is the
second adaptation set from the top of the drawing and in which tile I (Tile I) is
5 defined, the essential property (EssentialProperty) is defined as a description for a
tile, instead of the viewpoint (Viewpoint) described in the first embodiment. The
essential property in regard to a pmt of the bit stream is further extended and defined.
[0305]
That is, as illustrated in FIG 42, the essential property of the second
10 adaptation set from the top of the drawing is defined as follows, for example.
[0306]

[0307]
In this case, since a part of the bit stream to which the adaptation set
corresponds is divided (extracted) into tracks (that is, a plurality of tracks are
formed), "1 (true)" is defined as "Sub-Sample-is-extracted."
20 [0308]
25
Similarly, the essential property of the third adaptation set from the top of
the drawing in the example of FIG 42 is defined as follows, for example.
[0309]
I, 2">
[0310]

[0314]

In this case, the representation (Representation) belonging to the adaptation
20 set IS extended and information indicating dependency between files (tiles) is
additionally defined.
[0329]
In the representation belonging to the topmost adaptation set from the upper
side of the drawing, as illustrated in FIG 44, for example, the following information
25 is defined.
[0330]

[0331]
In the segment (Segment) belonging to the representation,
30 bitstream base.mp4 is defined.
[0332]
t!
SP35339GWOOO
68/161
Even in the case of the example of FIG. 44, in the adaptation set which is the
second adaptation set from the top of the drawing and in which tile 1 (Tile 1) is
defined, the essential property (EssentialProperty) is defined as a description for a
tile, instead of the viewpoint (Viewpoint) described in the first embodiment. The
5 essential property in regard to a part of the bit stream is further extended and defined.
[0333]
10
That is, as illustrated in FIG. 44, the essential property of the second
adaptation set from the top of the drawing is defined as follows, for example.
[0334]
1' 1 ">
[0335]

[0339]
In the segment (Segment) belonging to the representation,
bitstream_tilel.mp4 is defined.
30 [0340]
Similarly, the essential prope1iy of the third adaptation set from the top of
5
SP353396WOOO
69/161
the drawing in the example of FIG 44 is defined as follows, for example.
[0341]
1 ">
[0342]

[0344]
In the segment (Segment) belonging to the representation,
bitstream_tile2.mp4 is defined.
15 [0345]
20
Similarly, the essential property of the fourth adaptation set from the top of
the drawing in the example of FIG 44 is defined as follows, for example.
[0346]
1">
[0347]

[0349]
In the segment (Segment) belonging to the representation,
30 bitstream tile3 .mp4 is defined.
[0350]
5
SP353396WOOO
701161
Similarly, the essential propeiiy of the bottommost adaptation of the
drawing in the example of FIG 44 is defined as follows, for example.
(0351]
1">
[0352]

(0354]
In the segment (Segment) belonging to the representation,
15 bitstream tile4.mp4 is defined.
(0355]

The generation of the extended MPD can be performed as in the case of the
20 first embodiment. For example, when the delivery data generation device 101 (FIG
12) performs the delivery data generation process (FIG 14) and the tile type MPD
generation unit 141 (the tile type image information generation unit 124) (FIG 12)
performs the tile type MPD file generation process (FIG 15), the extended MPD can
be generated (the MPD is extended). Accordingly, even in this case, the delivery
25 data generation device 101 can adaptively deliver (supply) the data of the partial
image to the delivery server 102 based on the DASH standard. That is, it is
possible to realize the adaptive supply of the data of the partial image.
(0356]
The reproduction of the delivery data using the extended MPD can also be
30 performed as in the case of the first embodiment. For example, the terminal device
103 (FIG 13) can correctly analyze the extended MPD by performing the delivery
SP35339GWOOO
71/161
data generation process (FIG. 16) and gain the adaptive delivery (supply) of the data
of the partial image by the delivery server 102 which is based on the DASH standard.
That is, it is possible to correctly acquire the data of the partial image fi·om the
delivery server 102 and reproduce the data of the partial image. That is, it is
5 possible to realize the adaptive supply of the data of the patiial image.
[0357]

FIG. 45 is a diagram illustrating an example of the configuration of an MP4
10 file obtained by filing the bit stream (bitstream7) having, for example, the tile (Tile)
structure illustrated in FIG. 6B. In the case of the example of FIG. 45, as in the
example of FIG. 41, the bit streams of the tiles are collected and considered as one
file and the data of the tiles is further managed as one track.
15
[0358]
In the case of the example of FIG. 41, the reference relation of the data
between the tracks is defined using the extractor. In the case of the example of FIG.
45, however, the reference relation is defined using track reference (Track Reference).
[0359]
The track reference (Track Reference) is information indicating a reference
20 relation (which track refers to which track (or from which track reference is made))
between tracks. That is, the track reference is information in units of tracks and is
defined once for 1 track. "dpnd" is information that defines a track (that is, a
reference source) referring to the track and "prnt" is information that defines a track
(that is, a reference destination) referred to by the track.
25 [0360]
For example, in the case of the example of FIG. 45, "dpnd=2, 3, 4, 5" is
defined as the track reference (Track Reference) in track 1 (Track 1). This indicates
that track 1 is referred to by tracks 2 to 5. Similarly, in track 2 (Track 2) to track 5
(Track 5), "prnt=J" is defined as the track reference (Track Reference). This
30 indicates that these tracks refer to track 1. That is, this indicates that the
information (the parameter sets and the like) regarding track I is referred to in
SP353396WOOO
72/161
accordance with the track reference when any (any tile) of tracks 2 to 5 is reproduced.
[0361]
As described above, the degree of freedom for setting the reference relation
is improved since the extractor is defined for each sample. However, when the
5 reference relation is fixed, redundancy of the extractor increases, and thus there is a
possibility of the amount of information being mmecessarily increasing. For
example, when the sizes or shapes of the tiles are uniform in the bit stream, one time
suffices for the reference relation.
10
15
[0362]
On the other hand, the track reference (Track Reference) is defined only
once for 1 track, as described above. Accordingly, by using the track reference, it is
possible to reduce the definition redundancy of the reference relation and suppress an
increase in the amount of unnecessary information.
[0363]
In the case of this example, track 1 (Track 1) is present for storing the
parameter sets and the reproduction of track 1 (reproduction of an entire image
(1920x 1 080)) may not be performed. However, by reproducing the actual data of
tracks 2 to 5 in the order of the track reference, it is possible to reproduce the entire
nnage.
20 [0364]
25
As 111 the case of FIG 39, the tile regton group entry
(TileRegionGroupEntry) is defined in each of track 2 (Track 2) to track 5 (Track 5).
That is, one tile is defined in each track.
[0365]
The file name of this MP4 file is assumed to be bitstream.mp4.
[0366]

In the MPD of this case, the supplemental propetiy (Supplementa!Property)
or the essential property (EssentialProperty) of the adaptation set (AdaptationSet) is
30 also extended, as in the above-described case of the reference by the extractor. An
example of this is illustrated in FIG 46.
:-_l
5
SP35339GWOOO
73/161
[0367]
That is, as illustrated in FIG 46, in this case, the MP4 file can be managed
by the MPD as in the example of FIG 42.
[0368]

The generation of the extended MPD can be performed as in the case of the
first embodiment. For example, when the delivery data generation device 101 (FIG
12) performs the delivery data generation process (FIG 14) and the tile type MPD
10 generation unit 141 (the tile type image information generation unit 124) (FIG 12)
performs the tile type MPD file generation process (FIG 15), the extended MPD can
be generated (the MPD is extended). Accordingly, even in this case, the delivery
data generation device 101 can adaptively deliver (supply) the data of the partial
image to the delivery server I 02 based on the DASH standard. That is, it is
15 possible to realize the adaptive supply of the data of the partial image.
[0369]
The reproduction of the delivery data using the extended MPD can also be
pe~formed as in the case of the first embodiment. For example, the terminal device
I 03 (FIG 13) can correctly analyze the extended MPD by performing the delivery
20 data generation process (FIG 16) and gain the adaptive delivery (supply) of the data
of the partial image by the delive1y server 102 which is based on the DASH standard.
That is, it is possible to correctly acquire the data of the partial image from the
delivery server I 02 and reproduce the data of the partial image. That is, it is
possible to realize the adaptive supply of the data of the partial image.
25 [0370]

FIG 47 is a diagram illustrating an example of the configuration of an MP4
file obtained by filing the bit stream (bitstream7) having, for example, the tile (Tile)
30 structure illustrated in FIG 6B. In the case of the example of FIG 47, as in the
example of FIG 43, the bit streams of tiles are managed as mutually different files.
SP353396WOOO
741161
Since the tracks of the files are mutually different, the bit streams of the tiles can also
be said to be managed as mutually different tracks.
[0371]
The topmost MP4 file (MP4 File) (that is, track I (Track !)) in FIG 47
5 stores (manages) the parameter sets and the like (the VPS, the SPS, the PPS, the SEI,
and the like).
[0372]
The second to fifth MP4 files (MP4 File) (that is, track 2 (Track 2) to track 5
(Track)) from the top of FIG 47 store (manage) the data of tile I (Tile I) to tile 4
10 (Tile 4). By reproducing any MP4 file (that is, any track) an1ong the files, it is
possible to reproduce the image of any tile.
[0373]
In the case of the example of FIG 43, the reference relation of the data
between the tracks is defined using the extractor. In the case of the example of FIG
15 47, however, the reference relation is defined using track reference (Track Reference)
in a way similar to the case of FIG 45.
[0374]
For example, in the case of the example of FIG 47, "dpnd=2, 3, 4, 5" is
defined as the track reference (Track Reference) in track 1 (Track 1). This indicates
20 that track 1 is referred to by tracks 2 to 5. Similarly, in track 2 (Track 2) to track 5
(Track 5), "pmt= 1" is defined as the track reference (Track Reference). This
indicates that these tracks refer to track I. That is, this indicates that the
information (the parameter sets and the like) regarding track I is referred to in
accordance with the track reference when any (any tile) of tracks 2 to 5 is reproduced.
25 [0375]
30
As 111 the case of FIG 39, the tile region group entry
(TileRegionGroupEntry) is defined in each of track 2 (Track 2) to track 5 (Track 5).
That is, one tile is defined in each track.
[0376]
Even in the case of this example, as described above, the track reference is
used as the information indicating the reference relation. Accordingly, it is possible
SP353396WOOO
75/161
to reduce the definition redundancy of the reference relation and suppress the
increase in the amount of unnecessary information.
[0377]
The file names of the MP4 files in FIG 47 are assumed to be
5 bitstream_ base.mp4, bitstream_ tile l.mp4, bitstream_tile2.mp4, bitstream_ tile3 .mp4,
and bitstream_tile4.mp4 in order from the top.
[0378]

10 In the MPD of this case, the supplemental property (Supplementa!Property)
or the essential property (Essentia!Prope1ty) of the adaptation set (AdaptationSet) is
also extended, as in the above-described case of the reference by the extractor. An
example of this is illustrated in FIG 48.
[0379]
15 That is, as illustrated in FIG 48, in this case, the MP4 file can be managed
20
by the MPD as in the example of FIG 44.
[0380]

The generation of the extended MPD can be performed as in the case of the
first embodiment. For example, when the delivery data generation device I 0 I (FIG
12) performs the delivery data generation process (FIG 14) and the tile type MPD
generation unit 141 (the tile type image information generation unit 124) (FIG 12)
performs the tile type MPD file generation process (FIG 15), the extended MPD can
25 be generated (the MPD is extended). Accordingly, even in this case, the delivery
data generation device 101 can adaptively deliver (supply) the data of the partial
30
image to the delive1y server 102 based on the DASH standard. That is, it is
possible to realize the adaptive supply of the data of the partial image.
[0381]
The reproduction of the delivery data using the extended MPD can also be
performed as in the case of the first embodiment. For example, the terminal device
SP353396WOOO
76/161
I 03 (FIG 13) can correctly analyze the extended MPD by performing the delivery
data generation process (FIG 16) and gain the adaptive delivery (supply) of the data
of the patiial image by the delivery server 102 which is based on the DASH standard.
That is, it is possible to correctly acquire the data of the partial image from the
5 delivery server 102 and reproduce the data of the partial image. That is, it is
possible to realize the adaptive supply of the data of the partial image.
[0382]

10 FIG 49 is a diagram illustrating an example of the configuration of an MP4
file obtained by filing the bit stream (bitstream7) having, for example, the tile (Tile)
structure illustrated in FIG 6B. In the case of the example of FIG 49, as in the
examples of FIG 41 and FIG 45, the bit streams of the tiles are collected and
considered as one file and the data of the tiles is further managed as one track.
15 [0383]
In the case of the example of FIG 41, however, the reference relation of the
data between the tracks is defined using the extractor. In the case of the example of
FIG 45, the reference relation of the data between the tracks is defined using the
track reference. In the case of the example of FIG 49, however, the reference
20 relation is defined using both of the extractor and the track reference.
[0384]
More specifically, track 1 (Track 1) refers to the information regarding track
2 (Track 2) to track 5 (Track 5) using the extractor as in the case of FIG 41. Further,
track 2 (Track 2) to track 5 (Track 5) refer to the information regarding track I
25 (Track 1) using the track reference as in the case of FIG 45.
[0385]
That is, in track I (Track I), as illustrated in FIG 49, the parameter sets such
as the video parameter set (VPS), the sequence parameter set (SPS), and the picture
parameter set (PPS), the actual data such as the SEI, the extractor for referring to the
30 data of the tiles of tracks 2 to 5, and the like are stored.
[0386]
SP35339GWOOO
77/161
In track 2 (Track 2) to track 5 (Track 5), as illustrated in FIG 49, "prnt=l" is
defined as the track reference (Track Reference). This indicates that these tracks
refer to track 1. That is, this indicates that the information (the parameter sets and
the like) regarding track 1 is referred to in accordance with the track reference when
5 any (any tile) of tracks 2 to 5 is reproduced.
[0387]
In this way, it is possible to perform the reproduction of track· 1
(reproduction of the entire image (1920x1080)) as in the case of FIG 41, while
reducing the redundancy as in the case of FIG 45.
10 [0388]
15
As m the case of FIG 39, the tile reg10n group entry
(TileRegionGroupEntty) is defined in each of track 2 (Track 2) to track 5 (Track 5).
That is, one tile is defined in each track.
[0389]
The file name of this MP4 file is assumed to be bitstream.mp4.
[0390]

Even in the MPD of this case, as in the case of the reference by the extractor
20 (FIG 42) or the case of the reference by the track reference (FIG 46), as described
above, the supplemental property (SupplementalProperty) or the essential property
(EssentialProperty) of the adaptation set (AdaptationSet) is extended. The example
is illustrated in FIG 50.
25
30
[0391]
That is, as illustrated in FIG 50, in this case, the MP4 file can be managed
by the MPD as in the examples of FIG 42 and FIG. 46.
[0392]

The generation of the extended MPD can be petformed as in the case of the
first embodiment. For example, when the delivery data generation device 101 (FIG
SP35339GWOOO
78/161
12) performs the delivery data generation process (FIG 14) and the tile type MPD
generation unit 141 (the tile type image information generation unit 124) (FIG 12)
performs the tile type MPD file generation process (FIG 15), the extended MPD can
be generated (the MPD is extended). Accordingly, even in this case, the delivery
5 data generation device 101 can adaptively deliver (supply) the data of the partial
image to the delivery server 102 based on the DASH standard. That is, it is
possible to realize the adaptive supply of the data of the partial image.
(0393]
The reproduction of the delivery data using the extended MPD can also be
10 performed as in the case of the first embodiment. For example, the terminal device
I 03 (FIG 13) can correctly analyze the extended MPD by performing the delivery
data generation process (FIG 16) and gain the adaptive delivery (supply) of the data
of the partial image by the delivery server 102 which is based on the DASH standard.
That is, it is possible to correctly acquire the data of the partial image from the
15 delivery server I 02 and reproduce the data of the partial image. That is, it is
possible to realize the adaptive supply of the data of the partial image.
(0394]

20 FIG 51 is a diagram illustrating an example of the configuration of an MP4
file obtained by filing the bit stream (bitstream7) having, for example, the tile (Tile)
structure illustrated in FIG 6B. In the case of the example ofFIG 51, as in the
examples of FIG 43 and FIG 47, the bit streams of tiles are managed as mutually
different files. Since the tracks of the files are mutually different, the bit streams of
25 the tiles can also be said to be managed as mutually different tracks.
(0395]
In the case of the example of FIG 43, however, the reference relation of the
data between the tracks is defined using the extractor. In the case of the example of
FIG. 47, the reference relation of the data between the tracks is defined using the
30 track reference. In the case of the example of FIG 51, however, the reference
relation is defined using both of the extractor and the track reference.
SP35339GWOOO
79/161
[0396]
More specifically, the topmost MP4 file (track I (Track I)) in FIG 51 refers
to the information regarding the second to fifth MP4 files (track 2 (Track 2) to track
5 (Track 5)) fi·om the top of FIG 51 using the extractor as in the case of FIG 43.
5 Further, the second to fifth MP4 files (track 2 (Track 2) to track 5 (Track 5)) from the
top of FIG 51 refer to the information regarding the topmost MP4 file (track I (Track
I)) in FIG 51 using the track reference as in the case of FIG 47.
[0397]
In the topmost MP4 file (track 1 ), as illustrated in FIG 51, the parameter
10 sets such as the video parameter set (VPS), the sequence parameter set (SPS), and the
picture parameter set (PPS), the actual data such as the SEI, extractors (Track 2,
Track 3, Track 4, and Track 5) of the bit streams of the tiles, and the like are stored.
The parameter sets, the extractors, and the like are managed for each sample by the
sample entry (Sample Entry).
15 [0398]
As illustrated in FIG 51, in the second to fifth MP4 files (track 2 (Track 2)
to track 5 (Track 5)) from the top, "prnt= 1" is defined as the track reference (Track
Reference). This indicates that these tracks refer to track I. That is, this indicates
that the information (the parameter sets and the like) regarding track 1 is referred to
20 in accordance with the track reference when any (any tile) of tracks 2 to 5 is
reproduced.
[0399]
In this way, it is possible to perform the reproduction of the topmost MP4
file (track 1) in FIG 51 (reproduction of the entire image (1920xl080)) as in the case
25 of FIG 43 while reducing the redundancy as in the case ofFIG 47.
[0400]
As in the case of FIG 39, in each of the second to fifth MP4 files (track 2
(Track 2) to track 5 (Track 5)) from the top, the tile region group entry
(TileRegionGroupEntry) is defined. That is, one tile is defined in each track.
30 [0401]
The file names of the MP4 files m FIG 51 are assumed to be

AMENDED CLAIMS
Claim 1 (currently amended)
An information processing device comprising:
a partial image information generation unit configured to generate partial
5 image information which is information regarding each partial image which is a part
of an entire image and includes group identification information identifying a group
which is a group to which the partial images belong and which is a group of the
partial images displayable as one image; and
a metadata generation unit configured to generate metadata to be used for
10 supply of a bit stream of the entire image and supply of a bit stream of the patiial
image using the partial image information generated by the partial image information
generation unit.
Claim 2
The information processing device according to claim I,
wherein the partial image information includes positional information
indicating a position of the partial image in the entire image.
Claim 3
The information processing device according to claim 2,
wherein the positional information indicates a position of an upper left of
the partial image.
Claim 4
The information processing device according to claim I,
wherein the metadata generation unit stores the partial image information
regarding the plurality of patiial images in mutually different adaptation sets
(AdaptationSet) of the metadata and assigns the bit streams of the plurality of partial
images to the mutually different adaptation sets.
Claim 5
The information processing device according to claim I,
wherein the metadata generation unit stores the patiial image information
regarding the plurality of partial images in mutually different adaptation sets
(AdaptationSet) of the metadata and assigns a plurality of files for which one bit
5 stream including the plurality of partial images is divided for each partial image to
the mutually different adaptation sets.

Claim 6
The information processing device according to claim I,
wherein the metadata generation unit stores the partial image information
regarding the plurality of partial images in mutually different sub-representations
(Sub-Representation) belonging to one representation (Representation) belonging to
one adaptation set (AdaptationSet) of the metadata and assigns the bit streams of the
plurality of partial images to the mutually different sub-representations.
Claim 7
The information processing device according to claim 6,
wherein the partial image information generation unit further generates
information indicating that information regarding the bit stream is present under the
20 sub-representation (Sub-Representation).
Claim 8
The information processing device according to claim 6,
wherein each of the bit streams of the plurality of partial images is stored in
25 a TRACK of one MP4 file.
Claim 9
The information processing device according to claim 8,
wherein the metadata generation unit fbrther generates information
30 indicating a location of data of the one MP4 file.
Claim 10 (currently amended)
The information processing device according to claim 1,
wherein the partial image information includes information regarding a size
of the entire image.
Claim 11 (currently amended)
The information processing device according to claim 1,
wherein the patiial image is a tile (Tile) in high efficiency video coding
(HEVC).
Claim 12 (currently amended)
The information processing device according to claim 1,
wherein the partial image information includes a view type ( viewtype)
indicating whether an image is the partial image.
Claim 13 (currently amended)
The information processing device according to claim 1, further comprising:
an encoding unit configured to encode image data of the entire image and
the partial image and generate a bit stream.
Claim 14 (currently amended)
The information processing device according to claim 1, finiher comprising:
a screen division processing unit configured to generate image data of each
partial image from image data of the entire image.
Claim 15 (currently amended)
The information processing device according to claim I,
wherein the partial image information includes information indicating the
number of partial images forming the entire image, identification information
30 indicating that sizes of the partial images are equal, and information indicating a
position and a size of each partial image when the sizes of the partial images are not
H
equal.
Claim 16 (cmTently amended)
An information processing method comprising:
generating partial image information which is information regarding each
partial image which is a part of an entire image and includes group identification
information identifying a group which is a group to which the partial images belong
and which is a group of the partial images displayable as one image; and
generating meta data to be used for supply of a bit stream of the entire image
10 and supply of a bit stream of the partial image using the generated paiiial image
15
information.
Claim 17 (currently amended)
An information processing device comprising:
an analysis unit configured to analyze metadata that includes partial image
information which is information regarding each paiiial image which is a part of an
entire image and includes group identification information identifying a group which
is a group to which the partial images belong and which is a group of the partial
images displayable as one image and that is ~sed for supply of a bit stream of the
20 entire image and supply of a bit stream of the partial image, and to obtain the partial
image information;
a selection unit configured to select the bit stream of a desired partial image
using the partial image information obtained by the analysis unit; and
a bit stream acquisition unit configured to acquire the bit stream selected by
25 the selection unit.
Claim 18 (currently amended)
The information processing device according to claim 17,
wherein the partial image information includes positional information
30 indicating a position of the pmiial image in the entire image.
5
SP353396WOOO
Claim 19 (currently amended)
The information processing device according to claim 18,
wherein the positional information indicates a position of an upper left of
the partial image.
Claim 20 (currently amended)
The information processing device according to claim 17,
wherein the analysis unit analyzes the metadata in which the partial image
information regarding the plurality of partial images is stored in mutually different
10 adaptation sets (AdaptationSet) and the bit streams of the plurality ofpatiial images
are assigned to the mutually different adaptation sets.
15
20
Claim 21 (currently amended)
The information processing device according to claim 17,
wherein the analysis unit analyzes the metadata in which the partial image
information regarding the plurality of partial images is stored in mutually different
adaptation sets (AdaptationSet) and a plurality of files for which one bit stream
including the plurality of partial images is divided for each partial image are assigned
to the mutually different adaptation sets.
Claim 22 (currently amended)
The information processing device according to claim 17,
wherein the analysis unit analyzes the metadata in which the partial image
information regarding the plurality of patiial images is stored in mutually different
25 sub-representations (Sub-Representation) belonging to one representation
(Representation) belonging to one adaptation set (AdaptationSet) and the bit streams
of the plurality of partial images are assigned to the mutually different subrepresentations.
30 Claim 23 (currently amended)
The information processing device according to claim 22,
SP35339GWOOO
wherein the pmiial image information includes information indicating that
information regarding the bit stream is present under the sub-representation (SubRepresentation).
5 Claim 24 (currently amended)
The information processing device according to claim 22,
wherein each of the bit streams of the plurality of partial images is stored in
a TRACK of one MP4 file.
10 Claim 25 (currently amended)
The information processing device according to claim24,
wherein the metadata includes information indicating a location of data of
the one MP4 file.
15 Claim 26 (currently amended)
The information processing device according to claim 17,
wherein the partial image information includes information regarding a size
of the entire image.
20 Claim 27 ( cunently amended)
The information processing device according to claim 17,
wherein the partial image is a tile (Tile) in high efficiency video coding
(HEVC).
25 Claim 28 (currently amended)
The information processing device according to claim 17,
wherein the partial image information includes a view type (viewtype)
indicating whether an image is the partial image.
30 Claim 29 (currently amended)
The information processing device according to claim 17, futiher
SP353396WOOO
compnsmg:
a decoding unit configured to decode the bit streams acquired by the bit
stream acquisition unit.
5 Claim 30 (currently amended)
The information processing device according to claim 29, further
comprising:
a screen combination processing unit configured to generate image data of
the entire image from image data of the partial image obtained by the decoding unit
10 decoding the bit stream.
Claim 31 (new)
The information processing device according to claim 17,
wherein the partial image information includes information indicating the
15 number of partial images forming the entire image, identification information
indicating that sizes of the partial images are equal, and information indicating a
position and a size of each partial image when the sizes of the partial images are not
equal.
20 Claim 32 (new)
An information processing method comprising:
analyzing metadata that includes partial· image information which IS
information regarding each partial image which is a part of an entire image and
includes group identification information identifying a group which is a group to
25 which the pmiial images belong and which is a group of the partial images
displayable as one image and that is used for supply of a bit stream of the entire
image and supply of a bit stream of the partial image, and obtaining the pa1tial image
information;
selecting the bit stream of a desired pmiial image using the obtained pmtial
30 image information; and
acquiring the selected bit stream.