FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEPTION
DEVICE, AND RECEPTION METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-0075, Japan
The following specification particularly describes the invention and the manner in
which it is to be performed.
2
DESCRIPTION
TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEIVING
DEVICE, AND RECEIVING METHOD
5 Technical Field
[0001] The present technology relates to a
transmission device, a transmission method, a receiving
device, and a receiving method, and specifically
relates to a transmission device and the like that
10 provides a multi-image-arrangement distribution
service.
Background Art
[0002] Technologies of multi-window display, in
which a plurality of streams via broadcasting or via a
15 network are multi-decoded and the plurality of multidecoded
images are displayed on one display, is
realized or proposed.
[0003] For example, when a plurality of images are
displayed in the so-called PinP (Picture in Picture)
20 style, a user can watch the image of a program of one
channel displayed on a large main-window and the image
of a program of another channel displayed on a small
sub-window at the same time. Further, for example, when
the images of programs of different channels are
25 displayed on the right-half and the left-half of a
display side-by-side, a user can watch the images of
3
the programs of two channels simultaneously.
[0004] For example, Patent Document 1 discloses a
technology in which, when a multi-window-display
instruction is input under a normal-display status, the
display status is promptly switched 5 to multi-window
display.
[0005] Patent Document 1: Japanese Patent
Application Laid-open No. 2000-13712
Summary of Invention
10 Problem to be solved by the Invention
[0006] For example, a transmitting side may transmit
image data of a plurality of images of a multi-image
arrangement, and a receiving side may process the data
of the plurality of images, obtain a multi-image15
arranged image, and display the multi-image-arranged
image. In this case, the receiving side needs to
arrange the plurality of images appropriately.
[0007] It is an object of the present technology to
successfully provide a multi-image-arrangement
20 distribution service.
Means for solving the Problem
[0008] According to a concept of the present
technology, there is provided a transmission device,
including:
25 a transmitting unit that transmits a container of
a predetermined format containing a predetermined
4
number of video streams containing encoded image data
of a plurality of images of a multi-image-arrangement
distribution service; and
an information-inserting unit that inserts
position information in the video-5 stream layer, the
position information showing arrangement positions of
the plurality of images.
[0009] According to the present technology, a
transmitting unit transmits a container of a
10 predetermined format containing a predetermined number
of video streams containing encoded image data of a
plurality of images of a multi-image-arrangement
distribution service. For example, the container may be
a transport stream (MPEG-2 TS) used as a digital
15 broadcast standard. Alternatively, for example, the
container may be a container having another format such
as MMT or MP4 used for distribution via the Internet.
An information-inserting unit inserts position
information in the video-stream layer, the position
20 information showing arrangement positions of the
plurality of images.
[0010] As described above, according to the present
technology, position information is inserted in the
video-stream layer, the position information showing
5
arrangement positions of the plurality of images. So
the receiving side is capable of arranging the
plurality of images appropriately and easily.
[0011] Note that according to the present technology,
for example, the information-inserting 5 unit may further
insert offset information in the video-stream layer,
the offset information being used to adjust display
timing of the plurality of images. Therefore the
receiving side is capable of adjusting display timing
10 of the plurality of images in a multi-image-arranged
image, and matching display-start timing of the
plurality of images.
[0012] Further, according to the present technology,
for example, the information-inserting unit may further
15 insert type information in the video-stream layer, the
type information showing the multi-image-arrangement
type. Therefore the receiving side is capable of
recognizing a multi-image-arrangement type
appropriately, and arranging the plurality of images
20 appropriately and easily. Further, according to the
present technology, for example, the informationinserting
unit may further insert lifetime information
in the video-stream layer, the lifetime information
showing display-allowed time of the plurality of images
6
is controlled or not. Thanks to this information, for
example, the receiving side is capable of automatically
controlling deletion of image data stored in a virtual
display buffer.
[0013] Further, according to the present 5 technology,
for example, the information-inserting unit may further
insert type information in the video-stream layer, the
type information showing a rotation type of the
plurality of images during display. Therefore it is
10 possible to display a plurality of images in the
rotation status that the service side wants to provide.
[0014] Further, according to the present technology,
for example, the information-inserting unit may further
insert identification information in the container
15 layer, the identification information identifying the
multi-image-arrangement distribution service. Therefore
the receiving side is capable of easily recognizing the
multi-image-arrangement distribution service, and
preparing for that processing. In this case, for
20 example, the information-inserting unit may further
insert identification information in the container
layer, the identification information identifying if
the multi-image-arrangement distribution service is a
moving/still-image mixed service or not.
7
[0015] Further, according to the present technology,
for example, the information-inserting unit may further
insert information on a total number of the video
streams relating to the multi-image-arrangement
distribution service in the container 5 layer. Therefore
the receiving side is capable of knowing the total
number of the video streams to be processed easily, and
processing them appropriately.
[0016] Further, according to the present technology,
10 for example, the information-inserting unit may further
insert type information in the container layer, the
type information showing if the each video stream is a
moving-image stream, a still-image stream, or a mixed
stream, the mixed stream containing still images and
15 moving images mixed. Therefore the receiving side is
capable of knowing the total number of the video
streams to be processed easily, and processing them
appropriately.
[0017] Further, according to a concept of the
20 present technology, there is provided a receiving
device, including:
a receiving unit that receives a predetermined
number of video streams containing encoded image data
of a plurality of images of a multi-image-arrangement
8
distribution service, position information being
inserted in the predetermined number of video streams,
the position information showing arrangement positions
of the plurality of images;
a decoder unit that decodes 5 the predetermined
number of video streams, and obtains image data of the
plurality of images; and
a processing unit that arranges image data of the
plurality of decoded images based on the position
10 information, and obtains image data of a multi-imagearranged
image.
[0018] According to the present technology, a
receiving unit receives a predetermined number of video
streams containing encoded image data of a plurality of
15 images of a multi-image-arrangement distribution
service, position information being inserted in the
predetermined number of video streams, the position
information showing arrangement positions of the
plurality of images. A decoder unit decodes the
20 predetermined number of video streams, and obtains
image data of the plurality of images.
[0019] Further, a processing unit arranges image
data of the plurality of decoded images based on the
position information, and obtains image data of a
9
multi-image-arranged image. For example, the receiving
device may further include: a decoded buffer that
temporarily stores the image data of the plurality of
images obtained by the decoder unit; and a virtual
display buffer that temporarily stores 5 the image data
of the multi-image arrangement, in which the processing
unit may read the image data of the plurality of images
from the decoded buffer in series, and write the image
data of the plurality of images in areas corresponding
10 to the arrangement positions of the position
information of the virtual display buffer.
[0020] As described above, according to the present
technology, image data of a plurality of decoded images
is arranged based on position information, and image
15 data of a multi-image-arranged image is obtained.
Therefore it is possible to obtain image data of a
multi-image-arranged image effectively and
appropriately.
[0021] Note that according to the present technology,
20 for example, the receiving device may further include a
display controller that reads image data corresponding
to part or all of display areas of the multi-imagearranged
image from the virtual display buffer, and
displays images in the part or all of image areas. In
10
this case, for example, the receiving device may
further include a user operation unit that informs the
display controller of the display area. Therefore, for
example, it is possible to change images displayed on a
display, and to display a large multi-5 image-arranged
image to be scrolled based on user-operation, for
example, on a display small in size (resolution).
Further, in this case, for example, the receiving
device may further include a user operation unit that
10 informs the display controller of the number of
displays, images of the display area being to be
displayed on the displays.
[0022] Further, according to the present technology,
for example, the receiving device may further include a
15 request transmitting unit that transmits, to a
transmitting side, requests to stop and resume
transmission of the predetermined number of video
streams. Therefore, for example, a user can watch a
multi-image-arranged image containing still images at
20 arbitrary timing without overflowing a buffer in which
image data is temporarily stored.
[0023] Further, according to another concept of the
present technology, there is provided a transmission
device, including:
11
a metafile generator that generates a metafile
containing information to be used by a client terminal
to obtain a predetermined number of video streams, the
predetermined number of video streams containing
encoded image data of a plurality of 5 images of a multiimage-
arrangement distribution service, a distribution
server being capable of distributing the multi-imagearrangement
distribution service via a network; and
a metafile transmitting unit that transmits the
10 generated metafile to the client terminal via the
network in response to a transmission request from the
client terminal, in which
position information is inserted in the videostream
layer, the position information showing
15 arrangement positions of the plurality of images, and
the metafile generator generates the metafile
containing information showing to constantly read the
predetermined number of video streams.
[0024] According to the present technology, a
20 metafile generator generates a metafile containing
information to be used by a client terminal to obtain a
predetermined number of video streams. The
predetermined number of video streams contains encoded
image data of a plurality of images of a multi-image12
arrangement distribution service, a distribution server
being capable of distributing the multi-imagearrangement
distribution service via a network. In this
case, position information is inserted in the videostream
layer, the position information 5 showing
arrangement positions of the plurality of images.
[0025] The metafile transmitting unit transmits a
metafile to a client terminal via a network in response
to a transmission request from the client terminal. The
10 metafile generator generates the metafile containing
information showing to constantly read the
predetermined number of video streams.
[0026] As described above, according to the present
technology, the metafile contains information showing
15 to constantly read the predetermined number of video
streams. Therefore the receiving side constantly reads
a predetermined number of video streams based on the
information of the metafile. So it is possible to
obtain image data of a multi-image-arranged image
20 successfully.
[0027] Note that according to the present technology,
for example, the metafile generator may generate the
metafile further containing identification information,
the identification information identifying if the
13
predetermined number of video streams are still-image
streams or moving-image streams. Therefore the client
terminal is capable of determining if a predetermined
number of video streams are still-image streams or
moving-image 5 streams easily.
Effect of the Invention
[0028] According to the present technology, it is
possible to provide the multi-image-arrangement
distribution service successfully. Note that effects
10 are not necessarily limited to those described here,
and may be any effect described in the present
disclosure.
Brief Description of Drawings
[0029] [Fig. 1] A block diagram showing a
15 structural example of a distribution system of an
embodiment.
[Fig. 2] A diagram showing an example of
horizontal/vertical two-dimensional multi-image
arrangement.
20 [Fig. 3] A diagram showing an example of a horizontal
one-dimensional multi-image arrangement.
[Fig. 4] A diagram showing an example of a group
picture containing still images.
[Fig. 5] A diagram showing an example of a group
25 picture containing moving images.
14
[Fig. 6] A diagram showing an example of a group
picture containing moving images.
[Fig. 7] A diagram showing an example of a group
picture of still images and moving images.
[Fig. 8] A diagram showing an example 5 of distribution
of still images (Still pictures) service by using a
single stream.
[Fig. 9] A diagram showing an example of distribution
of still images (Still pictures) service by using a
10 plurality of streams.
[Fig. 10] A diagram showing an example of distribution
of mixed service of still images (Still pictures) and
moving images (Moving pictures) by using a single
stream.
15 [Fig. 11] A block diagram showing a structural example
of a transmission-data generator, which generates a
transport stream.
[Fig. 12] A diagram showing a structural example of a
transport stream in distribution using a single stream.
20 [Fig. 13] A diagram showing a structural example of a
transport stream in distribution using a plurality of
streams.
[Fig. 14] A diagram showing a structural example
(Syntax) of a group picture tag SEI (Group_picture_tag
25 SEI).
[Fig. 15] A diagram showing content (Semantics) of main
15
information of the structural example of a group
picture tag SEI.
[Fig. 16] A diagram showing an example rotation during
display based on "display_coordinate".
[Fig. 17] A diagram showing a calculation 5 example of
the display timing CTPs of the images in a group
picture.
[Fig. 18] A diagram showing an example of transition of
various kinds of information on a group picture tag SEI
10 in time series.
[Fig. 19] A diagram showing a structural example
(Syntax) of a multi-stream-service descriptor
(multi_stream_service_descriptor).
[Fig. 20] A diagram showing a content (Semantics) of
15 main information of the structural example of a multistream-
service descriptor.
[Fig. 21] A diagram showing a structural example
(Syntax) of the mixed stream descriptor
(mixed_stream_descriptor), and content (Semantics) of
20 main information of the structural example.
[Fig. 22] A diagram showing a structural example
(Syntax) of the picture lifetime descriptor
(Picture_lifetime_descriptor), and content (Semantics)
of the main information of the structural example.
25 [Fig. 23] A diagram showing an example of a relation
between display areas and images on a virtual display.
16
[Fig. 24] A diagram showing an example of the position
and the size of the display area on the virtual
display.
[Fig. 25] A diagram showing an example of a relation
between the display area and the image 5 on the virtual
display.
[Fig. 26] A diagram showing an example of the position
and the size of the display area on the virtual
display.
10 [Fig. 27] A diagram showing a structural example of
displays for displaying a panoramic image.
[Fig. 28] Diagrams each showing an example of
determining a display area when a group picture
contains seven images continuous in the horizontal
15 direction.
[Fig. 29] A block diagram showing a structural example
of a stream distribution system based on DASH.
[Fig. 30] A diagram showing a layer structure of an MPD
file.
20 [Fig. 31] A diagram showing an example of the relation
of the structures, which are arranged in a layered
manner in the MPD file.
[Fig. 32] A diagram showing an example of a flow of
generating a DASH segment and a DASH MPD file based on
25 content.
[Fig. 33] A diagram showing a structural example of an
17
IPTV client of the stream distribution system.
[Fig. 34] A diagram showing a general streamdistribution-
system series based on DASH.
[Fig. 35] A diagram showing a structural example of
Fragmented MP4 stream containing 5 a video stream.
[Fig. 36] A diagram showing a description example of an
MPD file of the multi-image-arrangement distribution
service.
[Fig. 37] A diagram showing a description example of an
10 MPD file of the multi-image-arrangement distribution
service.
Mode(s) for Carrying Out the Invention
[0030] Hereinafter, an embodiment of the invention
(hereinafter referred to as "embodiment") will be
15 described. Note that description will be made in the
following order.
1. Embodiment
2. Modification example
[0031] <1. Embodiment>
20 [Structural example of distribution system]
Fig. 1 shows a structural example of the
distribution system 10. In the distribution system 10,
the broadcast station 110, the network distribution
server 120, and the NTP (Network Time Protocol)
25 distribution server 130 are arranged at the
transmitting side, and the receiver 200 is arranged at
18
the receiving side.
[0032] The broadcast station 110 provides linear
services of still images and moving images via an RF
transmission path, or download services of still images
and moving images. The broadcast station 5 110 transmits
containers of MPEG2-TS (hereinafter simply referred to
as "transport stream"), for example, as broadcastdistributive
multiplexed data. In this embodiment, the
transport stream includes a predetermined number of
10 video streams, each of which includes encoded image
data of a plurality of images (pictures) of a multiimage-
arrangement distribution service.
[0033] Examples of the multi-image arrangement
include: the horizontal/vertical two-dimensional multi15
image arrangement, in which a plurality of images are
two-dimensionally arranged in the horizontal/vertical
directions; the horizontal one-dimensional multi-image
arrangement, in which a plurality of images are onedimensionally
arranged in the horizontal direction; the
20 vertical one-dimensional multi-image arrangement, in
which a plurality of images are one-dimensionally
arranged in the vertical direction; and the like.
[0034] In the video-stream layer, in addition to the
position information showing the arrangement positions
25 of a plurality of images, offset information for
adjusting display timing of the plurality of images,
19
type information showing the type of the multi-image
arrangement, type information showing the rotation type
when displaying the plurality of images, and the like
are inserted. For example, if image data is encoded
based on AVC (Advanced Video Coding), 5 HEVC (High
Efficiency Video Coding), or the like, such information
is inserted in a newly-defined SEI message.
[0035] Fig. 2 shows an example of the
horizontal/vertical two-dimensional multi-image
10 arrangement. In this example, four images are arranged
in the horizontal direction, and four images are
arranged in the vertical direction. A multi-imagearranged
image (hereinafter referred to as "group
picture" as necessary) containing the 4x4 images has a
15 resolution corresponding to that of a virtual display.
For example, if one image has the HD resolution, then
the group picture has the 8K resolution.
[0036] Each of "0000" to "1111" shows 4-bit data,
i.e., position information of each image. In this case,
20 the former 2 bits show any one position of the quarter
of the virtual display. Further, the latter 2 bits show
any one position of the smaller quarter of each
quarter. Further, "Picture rendering type" is type
information showing a multi-image-arrangement type.
25 "Picture rendering type=011" means the
horizontal/vertical two-dimensional multi-image
20
arrangement.
[0037] Fig. 3 shows an example of the horizontal
one-dimensional multi-image arrangement. In this
example, five images are arranged in the horizontal
direction. The group picture containing 5 the five images
has a resolution corresponding to that of the virtual
display. For example, if one image has the HD
resolution, the group picture has the resolution five
times as large as the HD resolution in the horizontal
10 direction.
[0038] "1110", "1111", "0000", "0001", or "0010"
shows 4-bit data of position information of each image.
For example, in the horizontal one-dimensional multiimage
arrangement, the position information of the
15 front image is "0000", the position information of the
N-th image at the right of the front is "0000+N", and
the position information of the N-th image at the left
of the front is "0000-N".
[0039] Note that description of an example of the
20 vertical one-dimensional multi-image arrangement will
be omitted, which is similar to the example of the
horizontal one-dimensional multi-image arrangement.
[0040] Further, in the transport-stream layer,
identification information that means the multi-image25
arrangement distribution service, identification
information that identifies if the distribution service
21
is moving/still-image mixed service or not, the total
number of video streams relating to the multi-imagearrangement
distribution service, and the like are
inserted. Such information is described in a newlydefined
descriptor, which is inserted 5 below a program
map table (PMT), for example.
[0041] One video stream contains encoded image data
of one image or a plurality of images. Further, a video
stream is a moving-image stream, a still-image stream,
10 or a moving/still-image mixed stream.
[0042] Fig. 4 shows an example of a group picture of
still images. In this example, the group picture
contains 2x2 images (still images). Further, in this
example, one video stream transmits encoded image data
15 of the images time-divisionally. In this case, the
receiving side decodes encoded data of the images in
series, and the image data of each image is written in
an area of a virtual display buffer based on its
position information.
20 [0043] Fig. 5 shows an example of a group picture of
moving images. In this example, the group picture
contains 2x2 images (moving images). Further, in this
example, one video stream transmits encoded image data
of the images time-divisionally. In this case, the
25 receiving side decodes encoded data of the images in
series, and the image data of each image is written in
22
an area of a virtual display buffer based on its
position information.
[0044] Fig. 6 shows an example of a group picture of
moving images. In this example, the group picture
contains 2x2 images (moving images). 5 Further, in this
example, four video streams transmit encoded image data
of the images. In this case, the receiving side decodes
encoded data of the images, and the image data of each
image is written in an area of a virtual display buffer
10 based on its position information.
[0045] Fig. 7 shows an example of a group picture of
still images and moving images. In this example, the
group picture contains 2x2 images (still images, moving
images). Further, in this example, one video stream
15 transmits encoded image data of the images. In this
case, the image 3 (Picture 3) contains both still
images and moving images. In this case, the receiving
side decodes encoded data of the images, and the image
data of each image is written in an area of a virtual
20 display buffer based on its position information.
[0046] Fig. 8 shows an example of distribution of
still-image (Still picture) service by using a single
stream. In this example, a group picture contains 2x2
images. In Fig. 8, a rectangular unit with a circled
25 number shows an access unit. This example shows HEVC
encoding. Note that each of the circled numbers "1" to
23
"4" means that encoded image data of each of the image
1 (Picture 1) to the image 4 (Picture 4) is contained.
[0047] Firstly, the encoded image data of the still
images of the image 1 (Picture 1) to the image 4
(Picture 4) are transmitted in 5 series. Next, the
updated encoded image data of the image 3 (Picture 3)
is transmitted. Next, further, the updated encoded
image data of the image 3 (Picture 3) is transmitted.
Next, the updated encoded image data of the image 1
10 (Picture 1) to the image 4 (Picture 4) are transmitted
in series.
[0048] Note that "group_picture_start" means the
start of a group picture to be updated and displayed at
a certain point of time. Meanwhile, "group_picture_end"
15 means the end of a group picture to be updated and
displayed at a certain point of time. Such information
is inserted in the video-stream layer as described
below. The same applies to the following examples.
[0049] Fig. 9 shows an example of distribution of
20 still-image (Still picture) service by using multistreams.
In this example, a group picture contains 2x2
images. In Fig. 8, a rectangular unit with a circled
number shows an access unit. Note that each of the
circled numbers "1" to "4" means that encoded image
25 data of each of the image 1 (Picture 1) to the image 4
(Picture 4) is contained.
24
[0050] Firstly, the encoded image data of the image
1 (Picture 1) to the image 4 (Picture 4) are
transmitted in series by using the video streams 1 to
4, respectively. Next, the updated encoded image data
of the image 3 (Picture 3) is transmitted 5 by using the
video stream 3. Next, further, the updated encoded
image data of the image 3 (Picture 3) is transmitted by
using the video stream 3. Next, the updated encoded
image data of the image 1 (Picture 1) to the image 4
10 (Picture 4) are transmitted in series by using the
video streams 1 to 4, respectively.
[0051] Fig. 10 shows an example of distribution of
mixed service of still images (Still pictures) and
moving images (Moving pictures) by using a single
15 stream. In this example, a group picture contains 2x2
images. In Fig. 8, a rectangular unit with a circled
number shows an access unit. Note that each of the
circled numbers "1" to "4" means that encoded image
data of each of the image 1 (Picture 1) to the image 4
20 (Picture 4) is contained.
[0052] Firstly, the encoded image data of the still
images of the image 1 (Picture 1) to the image 4
(Picture 4) are transmitted in series. Next, the
encoded image data of the moving image of the image 3
25 (Picture 3) is transmitted. Next, the updated encoded
image data of the still images of the image 1 (Picture
25
1) to the image 4 (Picture 4) are transmitted in
series.
[0053] Fig. 11 shows a structural example of the
transmission-data generator 110, which generates the
transport stream, of the broadcast 5 station 110. The
transmission-data generator 110 includes the
moving/still-image output unit 112, the video encoder
113, and the multiplexer 114.
[0054] The moving/still-image output unit 112
10 outputs image data of a plurality of images of the
multi-image-arrangement distribution service. The
moving/still-image output unit 112 includes, for
example, a camera that captures an image of an object
and outputs image data, an image-data reader that reads
15 image data in a recording medium and outputs the image
data, or the like. In the still-image service (for
example, see Fig. 4), image data of a plurality of
still images is output. In the moving-image service
(for example, see Fig. 5 and Fig. 6), image data of a
20 plurality of moving images is output. Further, in the
moving/still-image mixed service (for example, see Fig.
7), image data of a plurality of moving images and
still images is output.
[0055] The video encoder 113 encodes image data of a
25 plurality of images output from the moving/still-image
output unit 112 based on AVC, HEVC, or the like, for
26
example, and obtains encoded image data. Further, the
video encoder 113 generates a predetermined number of
video streams (video elementary streams) containing
encoded image data by using a stream formatter at the
latter stage. In this case, one video 5 stream contains
encoded image data of one image of the multi-image
arrangement, or may contain encoded image data of a
plurality of images of the multi-image arrangement.
[0056] Here, the video encoder 113 inserts, in the
10 video-stream layer, in addition to the position
information showing the arrangement positions of a
plurality of images, offset information for adjusting
display timing of the plurality of images, type
information showing the type of the multi-image
15 arrangement, type information showing the rotation type
when displaying the plurality of images, and the like.
If image data is encoded based on AVC (Advanced Video
Coding), HEVC (High Efficiency Video Coding), or the
like, such information is inserted in a newly-defined
20 SEI message. Such information will be described later
in detail.
[0057] The multiplexer 114 generates packets of the
video streams generated by the video encoder 113,
multiplexes the packets, and generates a transport
25 stream. Here, the multiplexer 114 inserts, in the
container layer (system layer), identification
27
information that means the multi-image-arrangement
distribution service, identification information that
identifies if the distribution service is moving/stillimage
mixed service or not, the total number of video
streams relating to the multi-5 image-arrangement
distribution service, and the like. Such information is
described in a newly-defined descriptor, which is
inserted below a program map table (PMT), for example.
Such information will be described later in detail.
10 [0058] Behaviors of the transmission-data generator
110 of Fig. 11 will be roughly described. The
moving/still-image output unit 112 outputs image data
of a plurality of images of the multi-image-arrangement
distribution service. The image data is supplied to the
15 video encoder 113. The video encoder 113 encodes the
image data based on AVC, HEVC, or the like, for
example, and obtains encoded image data. Further, the
video encoder 113 generates a predetermined number of
video streams containing the encoded image data.
20 [0059] At this time, the video encoder 113 inserts,
in the video-stream layer, by using a newly-defined SEI
message for example, in addition to the position
information showing the arrangement positions of a
plurality of images, offset information for adjusting
25 display timing of the plurality of images, type
information showing the type of the multi-image
28
arrangement, type information showing the rotation type
when displaying the plurality of images, and the like.
[0060] Further, the predetermined number of video
streams, which are output from the video encoder 113,
are supplied to the multiplexer 114. 5 The multiplexer
114 generates packets of the video streams, multiplexes
the packets, and generates a transport stream. At this
time, the multiplexer 114 inserts, in the transportstream
layer, by using a newly-defined descriptor,
10 identification information that means the multi-imagearrangement
distribution service, identification
information that identifies if the distribution service
is moving/still-image mixed service or not, the total
number of video streams relating to the multi-image15
arrangement distribution service, and the like.
[0061] [TS structure, SEI message structure, and
descriptor structure]
Fig. 12 shows a structural example of a transport
stream in distribution using a single stream. A
20 transport stream contains one video stream. In other
words, according to the structural example, there is a
PES packet "video PES1" of a video stream containing
encoded image data. The video stream contains encoded
image data of a plurality of images of the multi-image25
arrangement distribution service.
[0062] In the video stream, a group picture tag SEI
29
(Group_picture_tag SEI) message is inserted for each
access unit of encoded image data of each image. By
using the SEI message, as described above, in the
video-stream layer, in addition to the position
information showing the arrangement 5 positions of a
plurality of images, offset information for adjusting
display timing of the plurality of images, type
information showing the type of the multi-image
arrangement, type information showing the rotation type
10 when displaying the plurality of images, and the like
are inserted.
[0063] Further, a transport stream TS contains a PMT
(Program Map Table) as PSI (Program Specific
Information). The PSI is information describing
15 programs, to which elementary streams contained in the
transport stream belong.
[0064] The PMT contains a program loop that
describes overall information on the programs. A multistream-
service descriptor
20 (multi_stream_service_descriptor) is inserted below the
program loop. Further, the PMT contains an elementary
loop having information on the elementary streams.
According to the structural example, the PMT contains a
video elementary loop (video ES1 loop).
25 [0065] In the video elementary loop, corresponding
to the video stream (video PES1), information such as a
30
stream type and a packet identifier (PID) is arranged,
and a descriptor that describes information on the
video stream is also arranged. As such descriptors, a
mixed stream descriptor (mixed_stream_descriptor) and a
picture lifetime 5 descriptor
(Picture_lifetime_descriptor) are inserted.
[0066] By using the descriptors inserted below the
PMT, as described above, identification information
that means the multi-image-arrangement distribution
10 service, identification information that identifies if
the distribution service is moving/still-image mixed
service or not, the total number of video streams
relating to the multi-image-arrangement distribution
service, and the like are inserted.
15 [0067] Fig. 13 shows a structural example of a
transport stream in distribution using a plurality of
streams, i.e., four streams in this case. A transport
stream contains four video streams. In other words,
according to the structural example, there are PES
20 packets "video PES1, video PES2, video PES3, and video
PES4" of a video stream containing encoded image data.
The video stream contains encoded image data of a
plurality of images of the multi-image-arrangement
distribution service.
25 [0068] In the video streams, a group picture tag SEI
(Group_picture_tag SEI) message is inserted for each
31
access unit of encoded image data of each image. By
using the SEI message, as described above, in the
video-stream layer, in addition to the position
information showing the arrangement positions of a
plurality of images, offset information 5 for adjusting
display timing of the plurality of images, type
information showing the type of the multi-image
arrangement, type information showing the rotation type
when displaying the plurality of images, and the like
10 are inserted.
[0069] Further, a transport stream TS contains a PMT
(Program Map Table) as PSI (Program Specific
Information). The PSI is information describing
programs, to which elementary streams contained in the
15 transport stream belong.
[0070] The PMT contains a program loop that
describes overall information on the programs. A multistream-
service descriptor
(multi_stream_service_descriptor) is inserted below the
20 program loop. Further, the PMT contains an elementary
loop having information on the elementary streams.
According to the structural example, the PMT contains
four video elementary loops (video ES1 loop, video ES2
loop, video ES3 loop, and video ES4 loop).
25 [0071] In the video elementary loops, corresponding
to each video stream, information such as a stream type
32
and a packet identifier (PID) is arranged, and a
descriptor that describes information on the video
stream is also arranged. As such descriptors, a mixed
stream descriptor (mixed_stream_descriptor) and a
picture lifetime 5 descriptor
(Picture_lifetime_descriptor) are inserted.
[0072] By using the descriptors inserted below the
PMT, as described above, identification information
that means the multi-image-arrangement distribution
10 service, identification information that identifies if
the distribution service is moving/still-image mixed
service or not, the total number of video streams
relating to the multi-image-arrangement distribution
service, and the like are inserted.
15 [0073] Fig. 14 shows a structural example (Syntax)
of a group picture tag SEI (Group_picture_tag SEI).
Further, Fig. 15 shows content (Semantics) of main
information of the structural example. The 8-bit field
"group_picture_id" is association information for
20 assembled display including a plurality of pictures
including still images. In other words, the images of a
group picture (multi-image-arranged image) have the
same "group_picture_id".
[0074] The 3-bit field "picture_rendering_type"
25 shows an arrangement type, i.e., a multi-imagearrangement
type, of the images in the group picture.
33
"001" means the horizontal one-dimensional multi-image
arrangement, "010" means the vertical one-dimensional
multi-image arrangement, and "011" means the
horizontal/vertical two-dimensional multi-image
arrangement. Thanks to this information, 5 the receiving
side is capable of recognizing the multi-imagearrangement
type appropriately, and arranging a
plurality of images appropriately.
[0075] The 1-bit field "lifetime_controlled" shows
10 if the display-allowed time is restricted or not. "1"
means that the display-allowed time is restricted. In
this case, display is allowed by NTP time determined
based on the system layer (container layer). "0" means
that the display-allowed time is not restricted, i.e.,
15 that display is allowed for an indefinite period.
Thanks to this information, for example, the receiving
side is capable of automatically controls the virtual
display buffer to delete image data stored therein.
[0076] The 4-bit field "location_ID" shows the
20 arrangement positions of the images in the group
picture (see Fig. 2 and Fig. 3). The 8-bit field
"version_number" is an ascending value showing update
of the images. Based on this information, for example,
it is possible to update the still images in the group
25 picture in series. The 4-bit field "number_of_pictures"
is the total number of the images in the group picture,
34
i.e., the maximum number of simultaneously-displayed
images having the same "group_picture_id".
[0077] The 3-bit field "display_coordinate" shows a
rotation type during display. "010" means clockwise 90-
degree rotation display of each image. 5 "011" means
anticlockwise 90-degree rotation display of each image.
"110" means clockwise 90-degree rotation display of the
whole group picture. "111" means anticlockwise 90-
degree rotation display of the whole group picture.
10 Thanks to this information, it is possible to display a
plurality of images in a group picture in the rotation
status that the service side wants to provide.
[0078] Fig. 16 shows an example of rotation during
display based on "display_coordinate". This example
15 shows the horizontal/vertical two-dimensional multiimage
arrangement, in which a group picture contains
2x2 images. Fig. 16(a) shows an example in which
"display_coordinate" is "110". The displayed image is
obtained by rotating the whole group picture, i.e., the
20 decoded image, clockwise by 90 degrees. Further, Fig.
16(b) shows an example in which "display_coordinate" is
"010". The displayed image is obtained by rotating the
respective images (pictures), i.e., the decoded image,
clockwise by 90 degrees.
25 [0079] With reference to Fig. 14 again, the 1-bit
field "group_picture_start" shows the start of a group
35
picture to be updated and displayed at a certain point
of time. The 1-bit field "group_picture_end" shows the
end of a group picture to be updated and displayed at a
certain point of time. The 8-bit field "PTD
(Presentation Time Distance)" is an offset 5 value, which
is used to simultaneously display the images in a group
picture. In other words, the "PTD" is offset
information, which is used to adjust display timing of
a plurality of images in a group picture.
10 [0080] The display timing CTP(i) of a picture(i) in
a group picture may be obtained based on PTD(i) as
shown in the following mathematical formula (1). Here,
PTD(max) is the PTD of the image having the latest
display timing (PTS) out of the images having the same
15 "group_picture_id" between "group_picture_start" and
"group_picture_end".
CTP(i)
=PTS(i)+(PTD(max)-PTD(i))x system clock accuracy
(90 KHz) ...(1)
20 [0081] Fig. 17 shows a calculation example of the
display timing CTPs of the images in a group picture.
This example is, as shown in Fig. 17(a), an example of
the horizontal/vertical two-dimensional multi-image
arrangement, in which a group picture contains 2x2
25 images (Picture 1, Picture 2, Picture 3, and Picture
4). In this case, as shown in Fig. 17(b), for example,
36
the PTSs of the images are P1, P2, P3, and p4, and 1,
2, 3, and 4 are set as the PTDs of the images.
PTD(max)=4 is satisfied.
[0082] In this case, "P1+(4-1)x1/90000" is obtained
as the display timing CTP 1 of the image 5 1 (Picture 1).
Further, "P2+(4-2)x1/90000" is obtained as the display
timing CTP 2 of the image 2 (Picture 2). Further,
"P3+(4-3)x1/90000" is obtained as the display timing
CTP 3 of the image 3 (Picture 3). Further, "P4+(4-
10 4)x1/90000=P4" is obtained as the display timing CTP 4
of the image 4 (Picture 4). Note that the display
timing of the images in a group picture are shifted by
setting different CTP values.
[0083] Fig. 18 shows an example of transition of
15 various kinds of information on a group picture tag SEI
in time series. In Fig. 18, a rectangular unit with a
circled number shows an access unit. Note that each of
the circled numbers "1" to "4" means that encoded image
data of each of the image 1 (Picture 1) to the image 4
20 (Picture 4) is contained (see Fig. 17(a)).
[0084] Firstly, the encoded image data of the still
images of the image 1 (Picture 1) to the image 4
(Picture 4), having "1" as "group_picture_id", are
transmitted in series. In this case, the encoded image
25 data of the image 1 (Picture 1) to image 4 (Picture 4)
contains "1" to "4" as "location_id", "1" as
37
"version_number", and "0" as "display_coordinate".
Further, in this case, the encoded image data of the
image 1 (Picture 1) contains "group_picture_start"
showing the start of the group-picture service, and the
encoded image data of the image 4 (5 Picture 4) contains
"group_picture_end" showing the end of the grouppicture
service.
[0085] Next, updated encoded image data of the image
2 (Picture 2) is transmitted. In this case, the encoded
10 image data of the image 2 (Picture 2) contains "2" as
"location_id", "2" as "version_number", and "0" as
"display_coordinate". Further, in this case, the
encoded image data of the image 2 (Picture 2) contains
"group_picture_start" showing the start of the group15
picture service, and "group_picture_end" showing the
end of the group-picture service.
[0086] Next, further updated encoded image data of
the image 2 (Picture 2) is transmitted. In this case,
the encoded image data of the image 2 (Picture 2)
20 contains "2" as "location_id", "3" as "version_number",
and "0" as "display_coordinate". Further, in this case,
the encoded image data of the image 2 (Picture 2)
contains "group_picture_start" showing the start of the
group-picture service, and "group_picture_end" showing
25 the end of the group-picture service.
[0087] Next, the updated encoded image data of the
38
still images of the image 1 (Picture 1) to the image 4
(Picture 4) are transmitted in series. In this case,
the encoded image data of the image 1 (Picture 1) to
image 4 (Picture 4) contains "1" to "4" as
"location_id", "4" as "version_5 number", and "1" as
"display_coordinate". Further, in this case, the
encoded image data of the image 1 (Picture 1) contains
"group_picture_start" showing the start of the grouppicture
service, and the encoded image data of the
10 image 4 (Picture 4) contains "group_picture_end"
showing the end of the group-picture service.
[0088] Fig. 19 shows a structural example (Syntax)
of a multi-stream-service descriptor
(multi_stream_service_descriptor). Further, Fig. 20
15 shows content (Semantics) of main information of the
structural example.
[0089] The 8-bit field
"multi_stream_service_descriptor_tag" shows a
descriptor type, i.e., a multi-stream-service
20 descriptor in this example. The 8-bit field
"multi_stream_service_descriptor_length" shows the
length (size) of the descriptor, i.e., the number of
the following bytes as the descriptor length. The 1-bit
field "multiple_stream_service_flag" is a flag showing
25 the multiple-distribution service, i.e., the multiimage-
arrangement distribution service. "1" shows the
39
multiple-distribution service, and "0" shows the nonmultiple-
distribution service. Thanks to this
information, the receiving side is capable of
recognizing the multi-image-arrangement distribution
service easily, and preparing for 5 that processing.
[0090] The 1-bit field
"moving_picture_still_picture_mixed_service" is a flag
showing the still/moving-image mixed service. "1" means
the still/moving-image mixed service, and "0" means the
10 unmixed service. The 4-bit field "number_of_streams"
shows the total number of streams relating to the
multiple-distribution service. "0001" to "1111" mean 1
stream to 15 streams, respectively. Thanks to this
information, the receiving side is capable of knowing
15 the total number of the video streams to be processed
easily, and processing them appropriately.
[0091] Fig. 21(a) shows a structural example
(Syntax) of the mixed stream descriptor
(mixed_stream_descriptor). Further, Fig. 21(b) shows
20 content (Semantics) of main information of the
structural example.
[0092] The 8-bit field "mixed_stream_descriptor tag"
shows a descriptor type, i.e., the mixed stream
descriptor in this example. The 8-bit field
25 "mixed_stream_descriptor length" shows the descriptor
length (size), and the number of the following bytes as
40
the descriptor length. The 2-bit field
"service_stream_type" shows the stream type. "01" means
the moving-image stream, "10" means the still-image
stream, and "11" means the still/moving-image mixed
stream. Thanks to this information, 5 the receiving side
is capable of knowing the total number of the video
streams to be processed easily, and processing them
appropriately.
[0093] Fig. 22(a) shows a structural example
10 (Syntax) of the picture lifetime descriptor
(Picture_lifetime_descriptor). Further, Fig. 22(b)
shows content (Semantics) of the main information of
the structural example.
[0094] The 8-bit field "Picture_lifetime_descriptor
15 tag" shows the descriptor type, i.e., the picture
lifetime descriptor in this example. The 8-bit field
"Picture_lifetime_descriptor length" shows the
descriptor length (size), i.e., the number of the
following bytes as the descriptor length. The 16-bit
20 field "number_of_group_pictures" shows the number of
group pictures identified by "group_picture_id". The
64-bit field "NTP" shows the clock value determined by
the NTP (Network Time Protocol), and can be set for
each group picture.
25 [0095] With reference to Fig. 1 again, the network
distribution server 120 provides linear services
41
providing still images and moving images or download
services providing still images and moving images via a
communication network. The network distribution server
120 distributes containers of MP4, for example, as IP
distribution data. Similar to the 5 above-mentioned
transport streams from the broadcast station 110, the
MP4 contains a predetermined number of video streams
containing encoded image data of a plurality of images
(pictures) of the multi-image-arrangement distribution
10 service. In response to a reproduction command from the
receiving side, the network distribution server 120
transmits a transmission stream, in which MP4 IP
packets are sequentially arranged, to the receiving
side via the communication network.
15 [0096] In the video-stream layer, in addition to the
position information showing the arrangement positions
of a plurality of images, offset information for
adjusting display timing of the plurality of images,
type information showing the type of the multi-image
20 arrangement, type information showing the rotation type
when displaying the plurality of images, and the like
are inserted. Further, in the container layer (system
layer), identification information that means the
multi-image-arrangement distribution service,
25 identification information that identifies if the
distribution service is moving/still-image mixed
42
service or not, the total number of video streams
relating to the multi-image-arrangement distribution
service, and the like are inserted.
[0097] With reference to Fig. 1 again, the receiver
200 includes the tuner 201, the demultiplexer 5 202, the
compressed data buffer (decode buffer) 203, the decoder
204, and the uncompressed data buffer (decoded buffer)
205. Further, the receiver 200 includes the virtual
display buffer (VP buffer) 206, the scaler 207, the
10 display (screen device) 208, the gateway/network router
209, the CPU (Central Processing Unit) 210, and the
user operation unit 211.
[0098] The CPU 210 controls the behaviors of the
respective units of the receiver 200. A user is capable
15 of inputting various instructions via the user
operation unit 211. The user operation unit 211
includes a remote control unit, a touch panel unit in
which instructions are input based on proximity/touch,
a mouse, a keyboard, a gesture input unit configured to
20 detect input instructions by using a camera, a sound
input unit in which sound instructions are input, and
the like.
[0099] The tuner 201 receives broadcast waves
transmitted from the transmitting side via an RF
25 transmission path, RF-demodulates the broadcast waves,
and obtains transport streams as broadcast-distributive
43
multiplexed data. According to this embodiment, as
described above, the transport stream contains a
predetermined number of video streams containing
encoded image data of a plurality of images (pictures)
of the multi-image-arrangement distribution 5 service.
[0100] The gateway/network router 209 receives a
series of IP packets transmitted from the distribution
server 120 via the communication network, depackets the
series of IP packets, and obtains MP4 as IP
10 distribution data. According to this embodiment,
similar to the above-mentioned transport stream, MP4
contains a predetermined number of video streams
containing encoded image data of a plurality of images
(pictures) of the multi-image-arrangement distribution
15 service.
[0101] The demultiplexer 202 extracts a
predetermined number of video streams from the
transport stream (broadcast-distributive multiplexed
data) obtained by the tuner 201 or MP4 (IP distribution
20 data) obtained by the gateway/network router 209, and
temporarily stores the video streams in the compressed
data buffer 203. At this time, the demultiplexer 202
extracts various information inserted in the container
layer, and transmits the information to the CPU 210. As
25 described above, in this information, identification
information showing the multi-image-arrangement
44
distribution service, identification information
identifying if the distribution service is the
moving/still-image mixed service or not, the total
number of video streams relating to the multi-imagearrangement
distribution service, 5 and the like are
inserted.
[0102] Note that the external storage medium 212,
e.g., an HDD, is connected to the demultiplexer 202,
and the demultiplexer 202 is capable of recording and
10 reproducing the transport stream (broadcastdistributive
multiplexed data) obtained by the tuner
201 or MP4 (IP distribution data) obtained by the
gateway/network router 209. The demultiplexer 202 is
capable of processing reproduced data obtained from the
15 storage medium 212 in the similar manner.
[0103] The decoder 204 fetches a predetermined
number of video streams stored in the compressed data
buffer 203 at a predetermined decoding-timing, decodes
the video streams, obtains image data of the images in
20 a group picture (multi-image-arranged image), and
temporarily stores the video streams in the
uncompressed data buffer 205. At this time, the decoder
204 extracts various information inserted in a videostream
layer, and transmits the information to the CPU
25 210. In this information, as described above, in
addition to the position information showing the
45
arrangement positions of images, offset information for
adjusting display timing of the plurality of images,
type information showing the type of the multi-image
arrangement, type information showing the rotation type
when displaying the images, and the 5 like are inserted.
[0104] The virtual display buffer 206 obtains image
data of the images in the group picture stored in the
uncompressed data buffer 205, and temporarily stores
the image data in areas corresponding to the images,
10 respectively. Data is read from the uncompressed data
buffer 205 to the virtual display buffer 206 at timing
depending on PTS (Presentation Time Stamp), for example
(see CTP of Fig. 17).
[0105] The scaler 207 reads image data of part or
15 all of the area (display area) of a group picture from
the virtual display buffer 206, scales the image data
as necessary, and transmits the image to the displays
208. The image data is scaled in order to, for example,
adjust the resolution of image data of a display area
20 read from the virtual display buffer 206 to the
resolution of the displays 208.
[0106] Read timing (display timing) from the virtual
display buffer 206, and in addition, the position and
size of display areas may be determined via the user
25 operation unit 211. In this case, the number of
displays 208 for displaying images may be determined
46
via the user operation unit 211. Note that a plurality
of displays 208 should be prepared to do so, although
Fig. 1 shows only one displays 208.
[0107] Fig. 23 shows an example of a relation
between display areas and images on a 5 virtual display.
In the example of Fig. 23, a group picture containing
2x2 images P1 to P4 is in the horizontal/vertical twodimensional
multi-image arrangement. The size of the
display area in the virtual display is set based on
10 user-operation, and the position of the display area
may be changed based on user-operation as necessary, as
shown in Fig. 23. In this case, only the image
corresponding to the display area is displayed on the
displays 208 out of the images P1 to P4 on the virtual
15 display.
[0108] Note that, in this case, under control of the
CPU 210, the position and the size of the current
display area on the virtual display may be displayed on
the displays 208 or a not-shown different display
20 device. Thanks to this display, a user may know the
position and the size of the display area on the
virtual display easily, and may determine the position
and the size of the display area appropriately and
effectively.
25 [0109] Fig. 24(a) shows an example of the position
and the size of the display area on the virtual
47
display. Fig. 24(b) shows an example of the position
indicator 250 displayed on the displays 208
corresponding thereto. The position indicator 250
includes the diagram 250a showing the entire virtual
display, and the rectangular diagram 5 250b showing the
current display area, which is arranged on that diagram
corresponding to the position and the size of the
display area.
[0110] Further, Fig. 25 shows an example of a
10 relation between the display area and the image on the
virtual display. In the example of Fig. 25, a group
picture containing five images P1 to P5 is in the
horizontal one-dimensional multi-image arrangement. The
size of the display area in the virtual display is set
15 based on user-operation, and the position of the
display area may be changed based on user-operation as
necessary, as shown in Fig. 25. In this case, only the
image corresponding to the display area is displayed on
the displays 208 out of the images P1 to P5 on the
20 virtual display.
[0111] Fig. 26(a) shows an example of the position
and the size of the display area on the virtual
display. Fig. 26(b) shows an example of the position
indicator 260 displayed on the displays 208
25 corresponding thereto. The position indicator 260
includes the diagram 260a showing the entire virtual
48
display, and the bar diagram 260b showing the center
position of the current display area, which is arranged
on that diagram corresponding to the center position of
the display area. Also in this case, the position and
the size of the display area may be 5 shown by using a
rectangular diagram.
[0112] Fig. 27 shows a structural example of
displays for displaying a panoramic image. This display
is structured by connecting a plurality of, i.e., three
10 in this example, flat (shown in dashed-dotted lines) or
curved (shown in solid lines) displays 208 in the
horizontal direction. In this case, for example, three
continuing images are selectively displayed in the
horizontal direction on the three displays 208, and a
15 panoramic image is thus displayed.
[0113] Fig. 28 shows examples of determining a
display area when a group picture contains seven images
continuous in the horizontal direction. Fig. 28(a)
shows an example of determining a position of a display
20 area having the size of, for example, three images,
which is used to display part of a panoramic image on
the three displays 208 (for example, see Fig. 27). Fig.
28(b) shows an example of determining a position of a
display area having the size of, for example, one
25 image, which is used to display part of a panoramic
image on the one display 208.
49
[0114] With reference to Fig. 1 again, if the images
in a group picture are still images, the still images
are updated based on user-operation. In this case,
image data of images-to-be-updated is read from the
virtual display buffer 206 based on 5 "location_id" and
"version_number", and the displayed images are updated.
[0115] For example, the above-mentioned distribution
example of Fig. 18 will be considered. Let's say, in
this case, firstly, image data of the four images
10 having "1" as "version_number" and "1" to "4" as
"location_id" is read from the virtual display buffer
206, and a group picture containing still images is
displayed. In this situation, if a user inputs update
operation, image data of an image having "2" as
15 "version_number" and "2" as "location_id" is read from
the virtual display buffer 206, and the displayed image
is updated.
[0116] Further, in this situation, if a user inputs
update operation, image data of an image having "3" as
20 "version_number" and "2" as "location_id" is read from
the virtual display buffer 206, and the displayed image
is updated again. Further, in this situation, if a user
inputs update operation, image data of four images
having "4" as "version_number" and "1" to "4" as
25 "location_id" is read from the virtual display buffer
206, and the four displayed images are updated.
50
[0117] If images in a group picture are still
images, and if "0" is set as "lifetime_controlled" even
after image data is read from the virtual display
buffer 206 and displayed on the displays 208, the image
data is remained in the virtual 5 display buffer 206
until a user inputs a release instruction. Meanwhile,
if "1" is set as "lifetime_controlled", after the NTP
time determined in a system layer (container layer)
passes, image data of the corresponding group picture
10 is automatically deleted from the virtual display
buffer 206.
[0118] If part or all of a plurality of images in a
group picture are moving images, the image data of the
moving image is read from the uncompressed data buffer
15 205 into the virtual display buffer 206 at timing
depending on the PTS, after that, is read within a
predetermined time period, and is transmitted to the
displays 208 via the scaler 207. Also in this case,
only image data of the display area, whose size and
20 position are determined by a user, is read as
necessary, and the scaler 207 converts the size
(converts resolution).
[0119] Further, a user inputs an image-rotation
operation via the user operation unit 211, and the
25 displayed images on the displays 208, i.e., the whole
group picture or images in the group picture, may thus
51
be rotated. Note that rotation of the images is
controlled basically based on the information
"display_coordinate". As a result, it is possible to
display images in the rotation status that the service
side wants to provide. However, 5 it is possible to
further rotate images based on operation by a user.
[0120] Further, a user may operate the user
operation unit 211 to transmit a reproduction command,
in addition, a pause command, a resume command, a skip
10 command, a reproduction-end command, and the like to
the network distribution server 120. Further, for
example, before the virtual display buffer 206
overflows, the CPU 210, for example, automatically
transmits a pause command to the network distribution
15 server 120, and, after that, transmits a resume command
if the virtual display buffer 206 has an enough
remaining capacity.
[0121] The behaviors of the receiver 200 will be
roughly described. Firstly the receiver 200 receiving
20 broadcast waves will be described. The tuner 201
receives the still-image linear service and the movingimage
linear service, and obtains a transport stream
(broadcast-distributive multiplexed data). The
transport stream is transmitted to the demultiplexer
25 202.
[0122] The demultiplexer 202 extracts a
52
predetermined number of video streams from the
transport stream (broadcast-distributive multiplexed
data), and temporarily stores the video streams in the
compressed data buffer 203. At this time, the
demultiplexer 202 extracts various information 5 inserted
in a container layer, and transmits the information to
the CPU 210. Note that the demultiplexer 202 processes
a recorded/reproduced transport stream (broadcastdistributive
multiplexed data) of the storage medium
10 212 in the similar manner.
[0123] The decoder 204 fetches a predetermined
number of video streams stored in the compressed data
buffer 203 at predetermined decoding-timing, decodes
the video streams, obtains image data of the images in
15 a group picture (multi-image-arranged image), and
temporarily stores the image data in the uncompressed
data buffer 205. At this time, the decoder 204 extracts
various information inserted in a video-stream layer,
and transmits the information to the CPU 210.
20 [0124] The virtual display buffer 206 obtains image
data of the images in the group picture stored in the
uncompressed data buffer 205, and temporarily stores
the image data in areas corresponding to the images,
respectively. Data is read from the uncompressed data
25 buffer 205 to the virtual display buffer 206 at timing
depending on PTS (Presentation Time Stamp), for
53
example.
[0125] As described above, out of image data of the
images in a group picture stored in the virtual display
buffer 206, an area, whose position and size are
determined by a user, is read at arbitrary 5 timing, the
scaler 207 converts the size (resolution), and, after
that, it is transmitted to the displays 208. As a
result, the display area, which is determined by the
user, out of the group picture is displayed on the
10 displays 208.
[0126] In this case, moving image data of the linear
service is displayed after a delay of a fixed time
after receiving the service. Meanwhile, since the
degree of freedom of viewing time of the still-image
15 service is larger depending on preference of users, a
user inputs instructions to update displayed content on
an irregular basis. Because of this, the amount of
delay of the receiver 200 is not constant irrespective
of the display time period.
20 [0127] The virtual display buffer 206 absorbs a
difference between the delay time and previously
encoded and distributed timing of the received stream.
In the linear service, before the virtual display
buffer 206 overflows, the receiver 200 determines to
25 record the received stream in the storage medium 212,
to suspend receiving of the service, or the like.
54
[0128] Further, when the receiver 200 receives the
download service of moving images or still images via
broadcast waves, files obtained via the tuner 201 are
once stored in the storage medium 212, or the
demultiplexer 202 stores the files 5 in the storage
medium 212 and the decoder 204 decodes the files
simultaneously, the virtual display buffer 206
generates a group picture, and the group picture is
displayed.
10 [0129] Next, how the receiver 200 receives network
distribution will be described. The gateway/network
router 209 receives the still-image linear service and
the moving-image linear service, and obtains MP4 (IP
distribution data). The MP4 is transmitted to the
15 demultiplexer 202.
[0130] The demultiplexer 202 extracts a
predetermined number of video streams from the MP4 (IP
distribution data), and the MP4 is temporarily stored
in the compressed data buffer 203. At this time, the
20 demultiplexer 202 extracts various information inserted
in a container layer, and transmits the information to
the CPU 210. Note that the demultiplexer 202 processes
recorded/reproduced MP4 (IP distribution data) of the
storage medium 212 in the similar manner.
25 [0131] The decoder 204 fetches a predetermined
number of video streams stored in the compressed data
55
buffer 203 at predetermined decoding-timing, decodes
the video streams, obtains image data of the images in
a group picture (multi-image-arranged image), and
temporarily stores the image data in the uncompressed
data buffer 205. At this time, the decoder 5 204 extracts
various information inserted in a video-stream layer,
and transmits the information to the CPU 210.
[0132] The virtual display buffer 206 obtains image
data of the images in the group picture stored in the
10 uncompressed data buffer 205, and temporarily stores
the image data in areas corresponding to the images,
respectively. Data is read from the uncompressed data
buffer 205 to the virtual display buffer 206 at timing
depending on PTS (Presentation Time Stamp), for
15 example.
[0133] As described above, out of image data of the
images in a group picture stored in the virtual display
buffer 206, an area, whose position and size are
determined by a user, is read at arbitrary timing, the
20 scaler 207 converts the size (resolution), and, after
that, it is transmitted to the displays 208. As a
result, the display area, which is determined by the
user, out of the group picture is displayed on the
displays 208.
25 [0134] In this case, moving image data of the linear
service is displayed after a delay of a fixed time
56
after receiving the service. Meanwhile, since the
degree of freedom of viewing time of the still-image
service is larger depending on preference of users, a
user inputs instructions to update displayed content on
an irregular basis. Because of this, 5 the amount of
delay of the receiver 200 is not constant irrespective
of the display time period.
[0135] The virtual display buffer 206 absorbs a
difference between the delay time and previously
10 encoded and distributed timing of the received stream.
In the linear service (multicast), before the virtual
display buffer 206 overflows, the receiver 200
determines to record the received stream in the storage
medium 212, to suspend receiving of the service, or the
15 like. Alternatively, the receiver 200 starts VOD
(unicast service) immediately, and transmits
reproduction-control commands (reproduce start, pause,
resume, stop, and the like) to a VOD distribution
server.
20 [0136] When the receiver 200 receives the download
service of moving images or still images via network
distribution, files obtained via the gateway/network
router 209 are once stored in the storage medium 212,
or the demultiplexer 202 stores the files in the
25 storage medium 212 and the decoder 204 decodes the
files simultaneously, the virtual display buffer 206
57
generates a group picture, and the group picture is
displayed.
[0137] At this time, the receiver 200 suspends
receiving download before the virtual display buffer
206 overflows, and sets the receiver 5 so as to resume
download later. Similar to receiving of the linear
service, if the virtual display buffer 206 cannot
absorb the network distribution service, the storage
medium 212 absorbs instead, or the receiver 200 starts
10 VOD (unicast service) and transmits reproductioncontrol
commands (reproduce start, pause, resume, stop,
and the like) to the VOD distribution server.
[0138] As described above, according to the
distribution system 10 of Fig. 1, when transmitting,
15 via an RF transmission path or a communication network
to a receiving side, a container of a predetermined
format containing a predetermined number of video
streams containing encoded image data of a plurality of
images of a multi-image-arrangement distribution
20 service, position information (location_id) showing
arrangement positions of a plurality of images is
inserted in a video-stream layer. As a result, for
example, the receiving side is capable of arranging a
plurality of images in a group picture (multi-image58
arranged image) effectively and appropriately.
[0139] Further, according to the distribution system
10 of Fig. 1, the receiver 200 arranges image data of a
plurality of images in a decoded group picture based on
position information, and obtains 5 image data of the
group picture. As a result, for example, it is possible
to obtain image data of a group picture effectively and
appropriately.
[0140] <2. Modification example>
10 Note that the present technology is applicable to
a distribution system based on MPEG-DASH. Fig. 29 shows
a structural example of the distribution system 10A
based on MPEG-DASH. In the distribution system 10A, the
N number of IPTV clients 13-1, 13-2, ..., 13-N are
15 connected to the DASH segment streamer 11 and the DASH
MPD server 12 via the CDN (Content Delivery Network) 14.
[0141] The DASH segment streamer 11 generates a
DASH-based stream segment (hereinafter referred to as
"DASH segment") based on media data (video data, audio
20 data, subtitle data, and the like) of predetermined
content, and transmits the segments in response to an
HTTP request from an IPTV client. The DASH segment
streamer 11 is a web server.
[0142] Further, in response to a request for a
59
segment of a predetermined stream transmitted from the
IPTV client 13 (13-1, 13-2, ..., 13-N) via the CDN 14,
the DASH segment streamer 11 transmits the segment of
the stream to the IPTV client 13, which has sent the
request, 5 via the CDN 14.
[0143] The DASH MPD server 12 is a server that
generates an MPD file, which is used to obtain the DASH
segment generated by the DASH segment streamer 11. The
DASH MPD server 12 generates an MPD file based on
10 content metadata from a content management server (not
shown in Fig. 29) and based on an address (url) of the
segment generated by the DASH segment streamer 11.
[0144] According to the MPD format, attributes of
video, audio, and other streams are described by using
15 elements called representation. For example, an MPD
file is divided into representations each containing a
predetermined number of video streams containing
encoded data of a plurality of images of the multiimage-
arrangement distribution service, and the
20 attributes are described. Further, in an MPD file,
information to constantly read the predetermined number
of video streams is described. If the IPTV client 13
receives the multi-image-arrangement distribution
service, the IPTV client 13 obtains a predetermined
60
number of video streams with reference to the
description of the MPD file.
[0145] As shown in Fig. 30, the MPD file has a layer
structure. In the MPD file, information such as the
compression format, the encoding speed, 5 the image size,
and the language of a moving image stored in the DASH
segment streamer 11 is described in a layered manner by
using the XML format. The MPD file contains, in a
layered manner, structures such as period, adaptation
10 set, representation, segment info, initialization
segment, and media segment.
[0146] A period structure contains information on a
program (data such as pair of synchronous moving images
and sound). Further, an adaptation-set structure in the
15 period structure generates a group of selection of a
stream (representation group). Further, a
representation structure in the adaptation-set
structure, holds information such as the encoding speed
of a moving image and a sound, and the sound volume of
20 the moving image.
[0147] Further, a segment-info structure in the
representation structure holds information about a
segment of a moving image or sound. Further, an
initialization-segment structure in the segment-info
61
structure holds initialization information such as a
data-compression format. Further, a media-segment
structure in the segment-info structure holds
information such as an address, which is used to obtain
the segment of a moving 5 image or sound.
[0148] Fig. 31 shows an example of the relation of
the structures, which are arranged in a layered manner
in the above-mentioned MPD file. As shown in Fig. 31(a),
the media presentation, i.e., the entire MPD file,
10 contains a plurality of periods divided based on time
intervals. For example, the first period starts at 0
second, the next period starts at 100 seconds, and the
like.
[0149] As shown in Fig. 31(b), a period contains a
15 plurality of representations. The plurality of
representations are divided into groups based on the
adaptation set, i.e., representation groups each
containing a predetermined number of video streams
containing encoded data of a plurality of images of the
20 multi-image-arrangement distribution service, for
example.
[0150] As shown in Fig. 31(c), a representation
contains segment info. As shown in Fig. 31(d), the
segment info contains an initialization segment and a
62
plurality of media segments, in each of which
information on each segment is described, the segment
being obtained by further dividing a period into
smaller pieces. The media segment contains information
such as an address (url), which is 5 used to obtain a
video or audio segment data or the like actually.
[0151] Fig. 32 shows an example of a flow of
generating a DASH segment and a DASH MPD file based on
content. The content management server 15 transmits
10 content to the DASH segment streamer 11. The DASH
segment streamer 11 generates a DASH segment of each
data stream based on video data, audio data, and the
like of the content.
[0152] Further, the DASH segment streamer 11
15 transmits information on the address (url) of the DASH
segment of each generated data stream to the DASH MPD
server 12. The content management server 15 transmits
the metadata of the content to the DASH MPD server 12.
The DASH MPD server 12 generates a DASH MPD file based
20 on the address information of the DASH segment of each
data stream and based on the metadata of the content.
[0153] Fig. 33 shows a structural example of the
IPTV client 13 (13-1 to 13-N). The IPTV client 13
includes the streaming data controller 131, the HTTP
63
access unit 132, and the moving image reproducer 133.
The streaming data controller 131 obtains an MPD file
from the DASH MPD server 12, and analyzes the content
of the MPD file.
[0154] The HTTP access unit 132 5 requests the DASH
segment streamer 11 to transmit a segment of a moving
image or sound used to reproduce a moving image. Here,
if the HTTP access unit 132 receives the multi-imagearrangement
distribution service, the HTTP access unit
10 132 requests, with reference to the description of the
MPD file, to transmit a segment of a predetermined
number of video streams containing encoded data of a
plurality of images of the multi-image-arrangement
distribution service.
15 [0155] The HTTP access unit 132 transmits the
received segment of a moving image or sound to the
moving image reproducer 133. The moving image
reproducer 133 decodes segments transmitted from the
HTTP access unit 132, obtains image data of images in a
20 group picture (multi-image-arranged image), sound data
in association with the image data, and the like, and
reproduces images and sound. Note that the units of the
IPTV client 13 is processed by using software, for
example.
64
[0156] Fig. 34 shows a general stream-distributionsystem
series based on DASH. All the DASH MPD files and
DASH segments are distributed via the CDN (Content
Delivery Network) 14. In the CDN 14, a plurality of
cache servers (DASH cache servers) 5 have a network
arrangement.
[0157] The cache server receives an HTTP request
from the IPTV client 13 to obtain MPD files. If the
cache server has MPD files in a local MPD cache, the
10 cache server returns an HTTP response to the IPTV
client 13. Further, if the cache server does not have
MPD files in the local MPD cache, the cache server
transfers the request to the DASH MPD server 12 or an
upper cache server. Further, the cache server receives
15 an HTTP response, which means that the MPD files are
stored, transfers the HTTP response to the IPTV client
13, and caches the MPD files.
[0158] Further, the cache server receives an HTTP
request from the IPTV client 13 to obtain DASH segments.
20 If the cache server has DASH segments in a local
segment cache, the cache server returns an HTTP
response to the IPTV client 13. Further, if the cache
server does not have DASH segments in the local segment
cache, the cache server transfers the request to the
65
DASH segment streamer 11 or an upper cache server.
Further, the cache server receives an HTTP response,
which means that the DASH segments are stored,
transfers the HTTP response to the IPTV client 13, and
caches 5 the DASH segments.
[0159] In the CDN 14, an IPTV client 13-1 transmits
an HTTP request firstly, the cache server on the route
temporarily caches DASH segments to be distributed to
the IPTV client 13-1, and the cached DASH segments are
10 distributed to a downstream IPTV client 13-2 in
response to an HTTP request. As a result, it is
possible to increase the distribution efficiency of
HTTP streaming to a large number of IPTV clients.
[0160] The CDN 14 includes, in addition to a
15 plurality of cache servers, a predetermined number of
cache management servers. The cache management server
generates a cache control policy based on an index of
the cache of the DASH segment of each video data stream
in an MPD file, and distributes the cache control
20 policy to each cache server. Each cache server caches
the DASH segment of each video data stream based on the
cache control policy.
[0161] Fig. 35 shows a structural example of
Fragmented MP4 stream. Fragmented MP4 stream of a video
66
contains Fragmented MP4s, which are obtained by
packeting a video stream. A predetermined pictures of a
video stream is inserted in "mdat" section of
Fragmented MP4. Similar to the above-mentioned
embodiment, a group picture tag SEI 5 (Group_picture_tag
SEI) message is inserted in each access unit of the
video stream. Further, similar to the above-mentioned
embodiment, for example, the IPTV client 13 arranges
images effectively and appropriately based on position
10 information ("location_id") on the images in a group
picture.
[0162] Note that, sometimes, a transport stream is
transmitted as it is based on DASH. In this case, a
transport stream, in which a multi-stream-service
15 descriptor (multi_stream_service_descriptor), a mixed
stream descriptor (mixed_stream_descriptor), and a
picture lifetime descriptor
(Picture_lifetime_descriptor) are inserted, is
transmitted as it is.
20 [0163] Each of Fig. 36 and Fig. 37 shows a
description example of an MPD file of the multi-imagearrangement
distribution service. In the "Adaptation
Set" layer of Fig. 36, the description
"parallelStream="4"" means to read four streams
67
constantly. Further, the description "frameRate="0""
means that the frame rate is 0, i.e., a still-image
stream. Further, as shown in Fig. 37, there are four
representations corresponding to the four streams.
[0164] As described above, 5 in the multi-imagearrangement
distribution service, to constantly read a
predetermined number of video streams, which contain
encoded data of a plurality of images of the multiimage-
arrangement distribution service, is described in
10 an MPD file. As a result, the IPTV client 13 is capable
of obtaining image data of the images in a group
picture, i.e., the images of the group picture,
successfully.
[0165] Further, the present technology may employ
15 the following structures.
(1) A transmission device, including:
a transmitting unit that transmits a container of
a predetermined format containing a predetermined
number of video streams containing encoded image data
20 of a plurality of images of a multi-image-arrangement
distribution service; and
an information-inserting unit that inserts
position information in the video-stream layer, the
position information showing arrangement positions of
68
the plurality of images.
(2) The transmission device according to (1), in which
the information-inserting unit further inserts
offset information in the video-stream layer, the
offset information being used to adjust 5 display timing
of the plurality of images.
(3) The transmission device according to (1) or (2),
in which
the information-inserting unit further inserts
10 type information in the video-stream layer, the type
information showing the multi-image-arrangement type.
(4) The transmission device according to any one of
(1) to (3), in which
the information-inserting unit further inserts
15 type information in the video-stream layer, the type
information showing a rotation type of the plurality of
images during display.
(5) The transmission device according to any one of
(1) to (4), in which
20 the information-inserting unit further inserts
lifetime information in the video-stream layer, the
lifetime information showing display-allowed time of
the plurality of images is controlled or not.
(6) The transmission device according to any one of
69
(1) to (5), in which
the information-inserting unit further inserts
identification information in the container layer, the
identification information identifying the multi-imagearrangement
distribution 5 service.
(7) The transmission device according to (6), in which
the information-inserting unit further inserts
identification information in the container layer, the
identification information identifying if the multi10
image-arrangement distribution service is a
moving/still-image mixed service or not.
(8) The transmission device according to any one of
(1) to (7), in which
the information-inserting unit further inserts
15 information on a total number of the video streams
relating to the multi-image-arrangement distribution
service in the container layer.
(9) The transmission device according to any one of
(1) to (8), in which
20 the information-inserting unit further inserts
type information in the container layer, the type
information showing if the each video stream is a
moving-image stream, a still-image stream, or a mixed
stream, the mixed stream containing still images and
70
moving images mixed.
(10) A transmission method, including:
a transmitting step of transmitting a container of
a predetermined format containing a predetermined
number of video streams containing 5 encoded data of a
plurality of images of a multi-image-arrangement
distribution service; and
an information inserting step of inserting
position information in the video-stream layer, the
10 position information showing arrangement positions of
the plurality of images.
(11) A receiving device, including:
a receiving unit that receives a predetermined
number of video streams containing encoded image data
15 of a plurality of images of a multi-image-arrangement
distribution service, position information being
inserted in the predetermined number of video streams,
the position information showing arrangement positions
of the plurality of images;
20 a decoder unit that decodes the predetermined
number of video streams, and obtains image data of the
plurality of images; and
a processing unit that arranges image data of the
plurality of decoded images based on the position
71
information, and obtains image data of a multi-imagearranged
image.
(12) The receiving device according to (11), further
including:
a decoded buffer that temporarily 5 stores the image
data of the plurality of images obtained by the decoder
unit; and
a virtual display buffer that temporarily stores
the image data of the multi-image arrangement, in which
10 the processing unit reads the image data of the
plurality of images from the decoded buffer in series,
and writes the image data of the plurality of images in
areas corresponding to the arrangement positions of the
position information of the virtual display buffer.
15 (13) The receiving device according to (12), further
including:
a display controller that reads image data
corresponding to part or all of display areas of the
multi-image-arranged image from the virtual display
20 buffer, and displays images in the part or all of image
areas.
(14) The receiving device according to (13), further
including:
a user operation unit that informs the display
72
controller of the display area.
(15) The receiving device according to (14), further
including:
a user operation unit that informs the display
controller of the number of displays, 5 images of the
display area being to be displayed on the displays.
(16) The receiving device according to any one of (11)
to (15), further including:
a request transmitting unit that transmits, to a
10 transmitting side, requests to stop and resume
transmission of the predetermined number of video
streams.
(17) A receiving method, including:
a receiving step of receiving a container of a
15 predetermined format containing a predetermined number
of video streams containing encoded data of a plurality
of images of a multi-image-arrangement distribution
service, position information being inserted in the
predetermined number of video streams, the position
20 information showing arrangement positions of the
plurality of images;
a decoding step of decoding the predetermined
number of video streams, and obtaining image data of
the plurality of images; and
73
a processing step of arranging image data of the
plurality of decoded images based on the position
information, and obtaining image data of a multi-imagearranged
image.
(18) A transmission 5 device, including:
a metafile generator that generates a metafile
containing information to be used by a client terminal
to obtain a predetermined number of video streams, the
predetermined number of video streams containing
10 encoded image data of a plurality of images of a multiimage-
arrangement distribution service, a distribution
server being capable of distributing the multi-imagearrangement
distribution service via a network; and
a metafile transmitting unit that transmits the
15 generated metafile to the client terminal via the
network in response to a transmission request from the
client terminal, in which
position information is inserted in the videostream
layer, the position information showing
20 arrangement positions of the plurality of images, and
the metafile generator generates the metafile
containing information showing to constantly read the
predetermined number of video streams.
(19) The transmission device according to (18), in
74
which
the metafile generator generates the metafile
further containing identification information, the
identification information identifying if the
predetermined number of video streams 5 are still-image
streams or moving-image streams.
[0166] According to the main features of the present
technology, when transmitting, via an RF transmission
path or a communication network, a container of a
10 predetermined format containing a predetermined number
of video streams containing encoded image data of a
plurality of images of a multi-image-arrangement
distribution service, position information
(location_id) showing arrangement positions of a
15 plurality of images is inserted in a video-stream
layer. As a result, for example, the receiving side is
capable of arranging a plurality of images in a group
picture (multi-image-arranged image) effectively and
appropriately (see Fig. 18).
20 Description of Symbols
[0167] 10, 10A distribution system
11 DASH segment streamer
12 DASH MPD server
13, 13-1 to 13-N IPTV client
25 14 CDN
15 content management server
75
110 broadcast station
112 moving/still-image output unit
113 video encoder
114 multiplexer
5 200 receiver
201 tuner
202 demultiplexer
203 compressed data buffer
204 decoder
10 205 uncompressed data buffer
206 virtual display buffer
207 scaler
208 display
209 gateway/network router
15 210 CPU
211 user operation unit
212 storage medium
76
Claims
[1] A transmission device, comprising:
a transmitting unit that transmits a container of
a predetermined format containing a predetermined
number of video streams containing 5 encoded image data
of a plurality of images of a multi-image-arrangement
distribution service; and
an information-inserting unit that inserts
position information in the video-stream layer, the
10 position information showing arrangement positions of
the plurality of images.
[2] The transmission device according to claim 1,
wherein
the information-inserting unit further inserts
15 offset information in the video-stream layer, the
offset information being used to adjust display timing
of the plurality of images.
[3] The transmission device according to claim 1,
wherein
20 the information-inserting unit further inserts
type information in the video-stream layer, the type
information showing the multi-image-arrangement type.
[4] The transmission device according to claim 1,
wherein
77
the information-inserting unit further inserts
type information in the video-stream layer, the type
information showing a rotation type of the plurality of
images during display.
[5] The transmission device according 5 to claim 1,
wherein
the information-inserting unit further inserts
lifetime information in the video-stream layer, the
lifetime information showing display-allowed time of
10 the plurality of images is controlled or not.
[6] The transmission device according to claim 1,
wherein
the information-inserting unit further inserts
identification information in the container layer, the
15 identification information identifying the multi-imagearrangement
distribution service.
[7] The transmission device according to claim 6,
wherein
the information-inserting unit further inserts
20 identification information in the container layer, the
identification information identifying if the multiimage-
arrangement distribution service is a
moving/still-image mixed service or not.
[8] The transmission device according to claim 1,
78
wherein
the information-inserting unit further inserts
information on a total number of the video streams
relating to the multi-image-arrangement distribution
service in the 5 container layer.
[9] The transmission device according to claim 1,
wherein
the information-inserting unit further inserts
type information in the container layer, the type
10 information showing if the each video stream is a
moving-image stream, a still-image stream, or a mixed
stream, the mixed stream containing still images and
moving images mixed.
[10] A transmission method, comprising:
15 a transmitting step of transmitting a container of
a predetermined format containing a predetermined
number of video streams containing encoded data of a
plurality of images of a multi-image-arrangement
distribution service; and
20 an information inserting step of inserting
position information in the video-stream layer, the
position information showing arrangement positions of
the plurality of images.
[11] A receiving device, comprising:
79
a receiving unit that receives a predetermined
number of video streams containing encoded image data
of a plurality of images of a multi-image-arrangement
distribution service, position information being
inserted in the predetermined number 5 of video streams,
the position information showing arrangement positions
of the plurality of images;
a decoder unit that decodes the predetermined
number of video streams, and obtains image data of the
10 plurality of images; and
a processing unit that arranges image data of the
plurality of decoded images based on the position
information, and obtains image data of a multi-imagearranged
image.
15 [12] The receiving device according to claim 11,
further comprising:
a decoded buffer that temporarily stores the image
data of the plurality of images obtained by the decoder
unit; and
20 a virtual display buffer that temporarily stores
the image data of the multi-image arrangement, wherein
the processing unit reads the image data of the
plurality of images from the decoded buffer in series,
and writes the image data of the plurality of images in
80
areas corresponding to the arrangement positions of the
position information of the virtual display buffer.
[13] The receiving device according to claim 12,
further comprising:
a display controller that 5 reads image data
corresponding to part or all of display areas of the
multi-image-arranged image from the virtual display
buffer, and displays images in the part or all of image
areas.
10 [14] The receiving device according to claim 13,
further comprising:
a user operation unit that informs the display
controller of the display area.
[15] The receiving device according to claim 14,
15 further comprising:
a user operation unit that informs the display
controller of the number of displays, images of the
display area being to be displayed on the displays.
[16] The receiving device according to claim 11,
20 further comprising:
a request transmitting unit that transmits, to a
transmitting side, requests to stop and resume
transmission of the predetermined number of video
streams.
81
[17] A receiving method, comprising:
a receiving step of receiving a container of a
predetermined format containing a predetermined number
of video streams containing encoded data of a plurality
of images of a multi-image-arrangement 5 distribution
service, position information being inserted in the
predetermined number of video streams, the position
information showing arrangement positions of the
plurality of images;
10 a decoding step of decoding the predetermined
number of video streams, and obtaining image data of
the plurality of images; and
a processing step of arranging image data of the
plurality of decoded images based on the position
15 information, and obtaining image data of a multi-imagearranged
image.
[18] A transmission device, comprising:
a metafile generator that generates a metafile
containing information to be used by a client terminal
20 to obtain a predetermined number of video streams, the
predetermined number of video streams containing
encoded image data of a plurality of images of a multiimage-
arrangement distribution service, a distribution
server being capable of distributing the multi-image82
arrangement distribution service via a network; and
a metafile transmitting unit that transmits the
generated metafile to the client terminal via the
network in response to a transmission request from the
client 5 terminal, wherein
position information is inserted in the videostream
layer, the position information showing
arrangement positions of the plurality of images, and
the metafile generator generates the metafile
10 containing information showing to constantly read the
predetermined number of video streams.
[19] The transmission device according to claim 18,
wherein
the metafile generator generates the metafile
15 further containing identification information, the
identification information identifying if the
predetermined number of video streams are still-image
streams or moving-image streams.
20
83
ABSTRACT
“TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEPTION
DEVICE, AND RECEPTION METHOD”
The objective of the present invention is to enable a service for the distribution of a
multi-image arrangement to be performed in a favorable manner. A transmitting side
transmits a container in a prescribed format, said container including encoded image
data for multiple images pertaining to a multi-image arrangement distribution service,
and having a prescribed number of video streams. Position information indicating the
arrangement position of the multiple images is inserted into the video stream layer. A
receiving side receives and decodes the prescribed number of video streams, thereby
obtaining image data for the multiple images. The image data for the multiple images
is arranged on the basis of the position information, thereby obtaining image data for
a group picture (a multi-image arrangement image).