FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
&
THE PATENTS RULES, 2003
COMPLETE SPECIFICATION
(See section 10, rule 13)
“TRANSMITTING DEVICE, TRANSMISSION METHOD, RECEIVING
DEVICE, AND RECEIVING METHOD”
SONY CORPORATION, of 1-7-1, Konan, Minato-ku, Tokyo 108-
0075, Japan
The following specification particularly describes the
invention and the manner in which it is to be performed.
2
DESCRIPTION
TRANSMISSION DEVICE, TRANSMISSION METHOD, RECEPTION DEVICE,
AND RECEPTION METHOD
5 TECHNICAL FIELD
[0001]
The present technology relates to a transmission device,
a transmission method, a reception device, and a reception
method, and more particularly, relates to a transmission device
10 for transmitting a plurality of types of audio data, and the
like.
BACKGROUND ART
[0002]
15 In related art, as a three-dimensional (3D) sound
technology, there is a proposed technology for mapping encoded
sample data to a speaker existing at an arbitrary location
to render on the basis of metadata (for example, see Patent
Document 1).
20
CITATION LIST
PATENT DOCUMENT
[0003]
Patent Document 1: Japanese Translation of PCT Publication
25 No. 2014-520491
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0004]
30 For example, sound reproduction with an improved
realistic feeling is realized in a reception side by
3
transmitting object data composed of encoded sample data and
metadata together with channel data of 5.1 channel, 7.1 channel,
or the like. In related art, it has been proposed to transmit
an audio stream including encoded data which is obtained by
5 encoding channel data and object data by using an MPEG-H 3D
Audio (3D audio) encoding method to the reception side.
[0005]
The 3D audio encoding method and an encoding method such
as MPEG4 AAC are not compatible in those stream structures.
10 Thus, when a 3D audio service is provided as maintaining
compatibility with a related audio receiver, a simulcast may
be considered. However, the transmission band cannot be
efficiently used when same content is transmitted by different
encoding methods.
15 [0006]
An object of the present technology is to provide a new
service as maintaining compatibility with a related audio
receiver without deteriorating an efficient usage of a
transmission band.
20
SOLUTIONS TO PROBLEMS
[0007]
A concept of the present technology lies in
a transmission device including:
25 an encoding unit configured to generate a predetermined
number of audio streams including first encoded data and second
encoded data which is related to the first encoded data; and
a transmission unit configured to transmit a container
in a predetermined format including the generated
30 predetermined number of audio streams,
wherein the encoding unit generates the predetermined
4
number of audio streams so that the second encoded data is
discarded in a receiver which is not compatible with the second
encoded data.
[0008]
5 According to the present technology, the encoding unit
generates a predetermined number of audio streams having first
encoded data and second encoded data which is related to the
first encoded data. Here, the predetermined number of audio
streams are generated so that the second encoded data is
10 discarded in a receiver which is not compatible with the second
encoded data.
[0009]
For example, an encoding method of the first encoded
data and an encoding method of the second encoded data may
15 be different. In this case, for example, the first encoded
data may be channel encoded data and the second encoded data
may be object encoded data. In addition, in this case, for
example, the encoding method of the first encoded data may
be MPEG4 AAC and the encoding method of the second encoded
20 data may be MPEG-H 3D Audio.
[0010]
The transmission unit transmits a container in a
predetermined format including the generated predetermined
number of audio streams. For example, the container may be
25 a transport stream (MPEG-2 TS), which is used in a digital
broadcasting standard. Further, for example, the container
may be a container of MP4, which is used in distribution through
the Internet, or a container in other formats.
[0011]
30 As described above, according to the present technology,
a predetermined number of audio streams having first encoded
5
data and second encoded data which is related to the first
encoded data are transmitted, and the predetermined number
of audio streams are generated so that the second encoded data
is discarded in a receiver which is not compatible with the
5 second encoded data. Thus, a new service can be provided as
maintaining the compatibility with a related audio receiver
without deteriorating the efficient usage of the transmission
band.
[0012]
10 Note that, in the present technology, for example, the
encoding unit may generate the audio streams having the first
encoded data and embed the second encoded data in a user data
area of the audio streams. In this case, in the related audio
receiver, the second encoded data embedded in the user data
15 area is read and discarded.
[0013]
In this case, for example, an information insertion unit
configured to insert, in a layer of the container,
identification information identifying that there is the
20 second encoded data, which is related to the first encoded
data, embedded in the user data area of the audio streams having
the first encoded data and included in the container may further
be included. With this configuration, in the reception side,
it can be easily recognized that there is second encoded data
25 embedded in the user data area of the audio streams before
performing a decode process of the audio streams.
[0014]
In addition, in this case, for example, the first encoded
data may be channel encoded data and the second encoded data
30 may be object encoded data, and the object encoded data of
a predetermined number of groups may be embedded in the user
6
data area of the audio stream, an information insertion unit
configured to insert, in a layer of the container, attribute
information that indicates an attribute of each piece of the
object encoded data of the predetermined number of groups may
5 further be included. With this configuration, in the
reception side, it can be easily recognized that an attribute
of each object encoded data of a predetermined number of groups
before decoding the object encoded data, so that the only object
encoded data of a necessary group can be selectively decoded
10 and used and this can reduce the processing load.
[0015]
In addition, in the present technology, for example,
the encoding unit may generate a first audio stream including
the first encoded data and generate a predetermined number
15 of second audio streams including the second encoded data.
In this case, in a related audio receiver, a predetermined
number of second audio streams are excluded from the target
of decoding. Or, in this system, it is also possible that
the first encoded data of 5.1 channel is encoded by using an
20 AAC system and data of 2 channel obtained from the data of
5.1 channel and the encoded object data are encoded as second
encoded data by using an MPEG-H system. In this case, a
receiver, which is not compatible with the second encoding
method, decodes only the first encoded data.
25 [0016]
In this case, for example, object encoded data of a
predetermined number of groups may be included in the
predetermined number of second audio streams, an information
insertion unit configured to insert, in a layer of the container,
30 attribute information that indicates an attribute of each piece
of object encoded data of the predetermined number of groups
7
may further be included. With this configuration, in the
reception side, it can be easily recognized an attribute of
each piece of object encoded data of the predetermined number
of groups before decoding the object encoded data, and only
5 the object encoded data of a necessary group can be selectively
decoded and used so that the processing load can be reduced.
[0017]
Then, in this case, for example, the information
insertion unit may be made to further insert, to the layer
10 of the container, stream correspondence relation information
that indicates to which second audio stream the object encoded
data of the predetermined number of groups and the channel
encoded data and object encoded data of the predetermined
number of groups is included respectively. For example, the
15 stream correspondence relation information may be made as
information that indicates a correspondence relation between
a group identifier identifying each piece of encoded data of
the plurality of groups and a stream identifier identifying
each stream of the predetermined number of audio streams. In
20 this case, for example, the information insertion unit may
be made to further insert, in the layer of the container, stream
identifier information that indicates each stream identifier
of the predetermined number of audio streams. With this
configuration, the reception side can easily recognize object
25 encoded data of a necessary group or a second audio stream
that includes the channel encoded data and object encoded data
of the predetermined number of groups so that the processing
load can be reduced.
[0018]
30 In addition, another concept of the present technology
lies in
8
A reception device including
a reception unit configured to receive a container in
a predetermined format including a predetermined number of
audio streams having first encoded data and second encoded
5 data which is related to the first encoded data,
wherein the predetermined number of audio streams are
generated so that the second encoded data is discarded in a
receiver which is not compatible with the second encoded data,
the reception device further including a processing unit
10 configured to extract the first encoded data and the second
encoded data from the predetermined number of audio streams
included in the container and process the extracted data.
[0019]
According to the present technology, the reception unit
15 receives a container in a predetermined format including a
predetermined number of audio streams having first encoded
data and second encoded data which is related to the first
encoded data. Here, the predetermined number of audio streams
are generated so that the second encoded data is discarded
20 in a receiver which is not compatible with the second encoded
data. Then, by the processing unit, the first encoded data
and second encoded data are extracted from the predetermined
number of audio streams and processed.
[0020]
25 For example, an encoding method of the first encoded
data and an encoding method of the second encoded data may
be different. In addition, for example, the first encoded
data may be channel encoded data and the second encoded data
may be object encoded data.
30 [0021]
For example, the container may be made to include an
9
audio stream that has the first encoded data and the second
encoded data embedded in a user data area thereof. In addition,
for example, the container may include a first audio stream
including the first encoded data and a predetermined number
5 of second audio streams including the second encoded data.
[0022]
In this manner, according to the present technology,
the first encoded data and second encoded data are extracted
from the predetermined number of audio streams and processed.
10 Therefore, high quality sound reproduction by a new service
using the second encoded data in addition to the first encoded
data can be realized.
EFFECTS OF THE INVENTION
15 [0023]
According to the present technology, a new service can
be provided as maintaining compatibility with a related audio
receiver without deteriorating an efficient usage of a
transmission band. It is noted that the effect described in
20 this specification is just an example and does not set any
limitation, and there may be additional effects.
BRIEF DESCRIPTION OF DRAWINGS
[0024]
25 Fig. 1 is a block diagram illustrating a configuration
example of a transceiving system as an embodiment.
Figs. 2(a) and 2(b) are diagrams for explaining
transmission audio stream configurations (stream
configuration (1) and stream configuration (2)).
30 Fig. 3 is a block diagram illustrating a configuration
example of a stream generation unit in a service transmitter
10
in a case that the transmission audio stream configuration
is the stream configuration (1).
Fig. 4 is a diagram illustrating a configuration example
of object encoded data that composes 3D audio transmission
5 data.
Fig. 5 is a diagram illustrating a correspondence
relation between groups and attributes or the like in a case
that the transmission audio stream configuration is the stream
configuration (1).
10 Fig. 6 is a diagram illustrating an MPEG4 AAC audio frame
structure.
Fig. 7 is a diagram illustrating a data stream element
(DSE) configuration to which metadata is inserted.
Figs. 8(a) and 8(b) are diagrams illustrating a
15 configuration of "metadata ()" and major information of the
configuration.
Fig. 9 is a diagram illustrating an audio frame structure
of MPEG-H 3D Audio.
Figs. 10(a) and 10(b) are diagrams illustrating packet
20 configuration examples of object encoded data.
Fig. 11 is a diagram illustrating a structure example
of an ancillary data descriptor.
Fig. 12 is a diagram illustrating a correspondence
relation between current bits and data types of an 8-bit field
25 of "ancillary_data_identifier."
Fig. 13 is a diagram illustrating a configuration example
of a 3D audio stream structure descriptor.
Fig. 14 illustrates major information content of the
configuration example of the 3D audio stream structure
30 descriptor.
Fig. 15 is a diagram illustrating types of content, which
11
is defined in "contentKind."
Fig. 16 is a diagram illustrating a configuration example
of a transport stream in a case that the configuration of the
transmission audio stream is the stream configuration (1).
5 Fig. 17 is a block diagram illustrating a configuration
example of a stream generation unit of a service transmitter
in a case that the configuration of the transmission audio
stream is the stream configuration (2).
Fig. 18 is a diagram illustrating a configuration example
10 (divided into two) of object encoded data composing 3D audio
transmission data.
Fig. 19 is a diagram illustrating a correspondence
relation between groups and attributes in a case that the
configuration of the transmission audio stream is the stream
15 configuration (2).
Figs. 20(a) and 20(b) are diagrams illustrating a
structure example of 3D audio stream ID descriptor.
Fig. 21 is a diagram illustrating a configuration example
of a transport stream in a case that the configuration of the
20 transmission audio stream is the stream configuration (2).
Fig. 22 is a block diagram illustrating a configuration
example of a service receiver.
Figs. 23(a) and 23(b) are diagrams for explaining
configurations of received audio streams (stream
25 configuration (1) and stream configuration (2)).
Fig. 24 is a diagram schematically illustrating a decode
process in a case that the configuration of the received audio
stream is the stream configuration (1).
Fig. 25 is a diagram schematically illustrating a decode
30 process in a case that the configuration of the received audio
stream is the stream configuration (2).
12
Fig. 26 is a diagram illustrating a structure of an AC3
frame (AC3 Synchronization Frame).
Fig. 27 is a diagram illustrating a configuration example
of AC3 auxiliary data (Auxiliary Data).
5 Figs. 28(a) and 28(b) are diagrams illustrating a
structure of a layer of an AC4 simple transport (Simple
Transport).
Figs. 29(a) and 29(b) are diagrams illustrating outline
configurations of a TOC (ac4_toc()) and a substream
10 (ac4_substream_data()).
Fig. 30 is a diagram illustrating a configuration example
of "umd_info()" in the TOC (ac4_toc()).
Fig. 31 is a diagram illustrating a configuration example
of "umd_payloads_substream())" in the substream
15 (ac4_substream_data()).
MODE FOR CARRYING OUT THE INVENTION
[0025]
In the following, modes (hereinafter, referred to as
20 "embodiment") for carrying out the invention will be described.
It is noted that the descriptions will be given in the following
order.
1. Embodiment
2. Modified Examples
25 [0026]
<1. Embodiment>
[Configuration Example of Transceiving System]
Fig. 1 illustrates a configuration example of a
transceiving system 10 as an embodiment. The transceiving
30 system 10 includes a service transmitter 100 and a service
receiver 200. The service transmitter 100 transmits a
13
transport stream TS through a broadcast wave or a packet through
a network. The transport stream TS includes a video stream
and a predetermined number, which is one or more, of audio
stream.
5 [0027]
The predetermined number of audio streams include
channel encoded data and a predetermined number of groups of
object encoded data. The predetermined number of audio
streams are generated so that the object encoded data is
10 discarded when a receiver is not compatible with the object
encoded data.
[0028]
In a first method, as illustrated in a stream
configuration (1) of Fig. 2(a), an audio stream (main stream)
15 including channel encoded data which is encoded with MPEG4
AAC is generated and a predetermined number of groups of object
encoded data which is encoded with MPEG-H 3D Audio is embedded
in a user data area of the audio stream.
[0029]
20 In a second method, as illustrated in a stream
configuration (2) of Fig. 2(b), an audio stream (main stream)
including channel encoded data which is encoded with MPEG4
AAC is generated and a predetermined number of audio streams
(substreams 1 to N) including a predetermined number of groups
25 of object encoded data which is encoded with MPEG-H 3D Audio
are generated.
[0030]
The service receiver 200 receives, from the service
transmitter 100, a transport stream TS transmitted using a
30 broadcast wave or a packet though a network. As described
above, the transport stream TS includes a predetermined number
14
of audio streams including channel encoded data and a
predetermined number of groups of object encoded data in
addition to a video stream. The service receiver 200 performs
a decode process on the video stream and obtains a video output.
5 [0031]
Further, when the service receiver 200 is compatible
with the object encoded data, the service receiver 200 extracts
channel encoded data and object encoded data from the
predetermined number of audi streams and performs the decode
10 process to obtain an audio output corresponding to the video
output. On the other hand, when the service receiver 200 is
not compatible with the object encoded data, the service
receiver 200 extracts only channel encoded data from the
predetermined number of audi streams and performs a decode
15 process to obtain an audio output corresponding to the video
output.
[0032]
[Stream Generation Unit Of Service Transmitter]
(A Case That The Stream Configuration (1) Is Employed)
20 Firstly, a case that the audio stream is in the stream
configuration (1) of Fig. 2(a) will be described. Fig. 3
illustrates a configuration example of a stream generation
unit 110A included in the service transmitter 100 in the above
case.
25 [0033]
The stream generation unit 110 includes a video encoder
112, an audio channel encoder 113, an audio object encoder
114, and a TS formatter 115. The video encoder 112 inputs
video data SV, encodes the video data SV, and generates a video
30 stream.
[0034]
15
The audio object encoder 114 inputs object data that
composes audio data SA and generates an audio stream (object
encoded data) by encoding the object data with MPEG-H 3D Audio.
The audio channel encoder 113 inputs channel data that composes
5 the audio data SA, generates an audio stream by encoding the
channel data with MPEG4 AAC, and also embeds the audio stream
generated in the audio object encoder 114 in a user data area
of the audio stream.
[0035]
10 Fig. 4 illustrates a configuration example of the object
encoded data. In this configuration example, two pieces of
object encoded data are included. The two pieces of object
encoded data are encoded data of an immersive audio object
(IAO) and a speech dialog object (SDO).
15 [0036]
Immersive audio object encoded data is object encoded
data for an immersive sound and includes encoded sample data
SCE1 and metadata EXE_El (Object metadata) 1 for rendering
by mapping the encoded sample data SCE1 with a speaker existing
20 at an arbitrary location.
[0037]
Speech dialogue object encoded data is object encoded
data for a spoken language. In this example, there is speech
dialogue object encoded data respectively corresponding to
25 first and second languages. The speech dialogue object
encoded data corresponding to the first language includes
encoded sample data SCE2 and metadata EXE_El (Object metadata)
2 for rendering by mapping the encoded sample data SCE2 with
a speaker existing at an arbitrary location. Further, the
30 speech dialogue object encoded data corresponding to the second
language includes encoded sample data SCE3 and metadata EXE_El
16
(Object metadata) 3 for rendering by mapping the encoded sample
data SCE3 with a speaker existing at an arbitrary location.
[0038]
The object encoded data is distinguished by using a
5 concept of groups (Group) according to the type of data.
According to the illustrated example, the immersive audio
object encoded data is set as Group 1, the speech dialogue
object encoded data corresponding to the first language is
set as Group 2, and the speech dialogue object encoded data
10 corresponding to the second language is set as Group 3.
[0039]
Further, the data which can be selected between groups
in a reception side is registered in a switch group (SW Group)
and encoded. Then, those groups can be grouped as a preset
15 group (preset Group) and reproduced according to a use case.
In the illustrated example, Group 1 and Group 2 are grouped
as Preset Group 1, and Group 1 and Group 3 are grouped as Preset
Group 2.
[0040]
20 Fig. 5 illustrates a correspondence relation or the like
between groups and attributes. Here, a group ID (group ID)
is an identifier to identify a group. An attribute (attribute)
represents an attribute of encoded data of each group. A switch
group ID (switch Group ID) is an identifier to identify a
25 switching group. A reset group ID (preset Group ID) is an
identifier to identify a preset group. A stream ID (sub Stream
ID) is an identifier to identify a stream. A kind (Kind)
represents a kind of content of each group.
[0041]
30 The illustrated correspondence relation indicates that
the encoded data of Group 1 is object encoded data for an
17
immersive sound (immersive audio object encoded data),
composes a switch group, and is embedded in a user data area
of the audio stream including channel encoded data.
[0042]
5 Further, the illustrated correspondence relation
indicates that the encoded data of Group 2 is object encoded
data for a spoken language (speech dialogue object encoded
data) of the first language, composes Switch Group 1, and is
embedded in a user data area of the audio stream including
10 channel encoded data. Further, the illustrated
correspondence relation indicates that the encoded data of
Group 3 is object encoded data for a spoken language (speech
dialogue object encoded data) of the second language, composes
Switch Group 1, and is embedded in a user data area of the
15 audio stream including channel encoded data.
[0043]
Further, the illustrated correspondence relation
indicates that Preset Group 1 includes Group 1 and Group 2.
In addition, the illustrated correspondence relation
20 indicates that Preset Group 2 includes Group 1 and Group 3.
[0044]
Fig. 6 illustrates an audio frame structure of MPEG4
AAC. The audio frame includes a plurality of elements. At
the beginning of each element (element), there is a three-bit
25 identifier (ID) of "id_syn_ele" and an element content can
be identified.
[0045]
The audio frame includes elements such as a single channel
element (SCE), a channel pair element (CPE), a low frequency
30 element (LFE), a data stream element (DSE), a program config
element (PCE), and a fill element (FIL). The elements of SCE,
18
CPE, and LFE include encoded sample data that composes channel
encoded data. For example, in a case of channel encoded data
of 5.1 channel, there included a single SCE, two CPEs, and
a single LFE.
5 [0046]
The element of PCE includes a number of channel elements
and a downmix (down_mix) factor. The element of FIL is used
to define extension (extension) information. In the element
of DSE, user data can be placed and "id_syn_ele" of this element
10 is "0x4." In DSE, object encoded data is embedded.
[0047]
Fig. 7 illustrates a configuration (Syntax) of DSE (Data
Stream Element()). A 4-bit field of "element_instance_tag"
represents a type of data in DSE; however, this value may be
15 set to "0" when the DSE is used as common user data. The field
of "data_byte_align_flag" is set to "1" so that the bytes of
the entire DSE are aligned. A value of "count" or "esc_count"
which represents a number of its added bytes is properly set
according to a user data size. The "count" and "esc_count"
20 can count up to 510 bytes. In other words, the size of the
data placed in a single DSE is 510 bytes at a maximum. To
"data_stream_byte" field, "metadata ()" is inserted.
[0048]
Fig. 8(a) illustrates a configuration (Syntax) of
25 "metadata ()" and Fig. 8(b) illustrates content (semantics)
of main information in the configuration. An 8-bit field of
"metadata_type" indicates a type of metadata. For example,
"0x10" represents object encode data of the MPEG-H system
(MPEG-H 3D Audio).
30 [0049]
An 8-bit field of "count" indicates a count number of
19
metadata in ascending chronological order. As described
above, the size of data placed in a single DSE is up to 510
bytes; however, the size of object encoded data may be larger
than 510 bytes. In such a case, more than one DSEs are used
5 and the count number indicated by "count" is made to represent
a link of those DSEs. In an area of "data_byte," object encoded
data is placed.
[0050]
Fig. 9 illustrates an audio frame structure of MPEG-H
10 3D Audio. This audio frame is composed of a plurality of MPEG
audio stream packets (mpeg Audio Stream Packet). Each MPEG
audio stream packet is composed of a header (Header) and a
payload (Payload).
[0051]
15 The header includes information such as a packet type
(Packet Type), a packet label (Packet Label), and a packet
length (Packet Length). In the payload, information defined
by the packet type in the header is placed. The payload
information includes "SYNC" corresponding to a synchronizing
20 start code, "Frame" which is actual data, and "Config" which
represents a configuration of "Frame."
[0052]
According to the present embodiment, "Frame" includes
object encoded data that composes 3D audio transmission data.
25 The channel encoded data composing the 3D audio transmission
data is included in the audio frame of MPEG4 AAC as described
above. The object encoded data is composed of encoded sample
data of single channel element (SCE) and metadata for rendering
by mapping the encoded sample data with a speaker existing
30 at an arbitrary location (see Fig. 4). The metadata is included
as an extension element (Ext_element).
20
[0053]
Fig. 10(a) illustrates a packet configuration example
of the object encoded data. In this example, object encoded
data of a single group is included. The information of "#obj=1"
5 included in "Config" indicates an existence of "Frame"
including the object encoded data of a single group.
[0054]
The information of "GroupID[0]=1" registered in
"AudioSceneInfo()" in "Config" indicates that "Frame"
10 including the encoded data of Group 1 is placed. Here, a value
of a packet label (PL) is made to be a same value in "Config"
and each "Frame" corresponding thereto. Here, "Frame"
including the encoded data of Group 1 is composed of "Frame"
including metadata as an extension element (Ext_element) and
15 "Frame" including encoded sample data of the single channel
element (SCE).
[0055]
Fig. 10(b) illustrates another packet configuration
example of the object encoded data. In this example, object
20 encoded data of two groups is included. The information of
"#obj=2" included in "Config" indicates that there is "Frame"
that has object encoded data of two groups.
[0056]
The information of "GroupID[1]=2, GroupID[2]=3,
25 SW_GRPID[0]=1" registered in "AudioSceneInfo()" in this order
in "Config" indicates that "Frame" having encoded data of Group
2 and "Frame" having encoded data having Group 3 are placed
in this order and these groups compose Switch Group 1. Here,
a value of a packet label (PL) is set as a same value in "Config"
30 and each "Frame" corresponding thereto.
[0057]
21
Here, "Frame" having the encoded data of Group 2 is
composed of "Frame" including metadata as an extension element
(Ext_element) and "Frame" including encoded sample data of
a single channel element (SCE). Similarly, "Frame" having
5 the encoded data of Group 3 is composed of "Frame" including
metadata as an extension element (Ext_element) and "Frame"
including encoded sample data of a single channel element
(SCE).
[0058]
10 Referring back to Fig. 3, the TS formatter 115 packetizes
a video stream output from the video encoder 112 and an audio
stream output from the audio channel encoder 113 as a PES packet,
further multiplexes by packetizing the data as a transport
packet, and obtains a transport stream TS as a multiplexed
15 stream.
[0059]
Further, the TS formatter 115 inserts identification
information that identifies that the object encoded data
related to the channel encoded data included in the audio stream
20 is embedded to the user data area of the audio stream in a
layer of a container, which is in coverage of a program map
table (PMT) according to the present embodiment. The TS
formatter 115 inserts the identification information to an
audio elementary stream loop corresponding to the audio stream
25 by using an existing ancillary data descriptor
(Ancillary_data_descriptor).
[0060]
Fig. 11 illustrates a structure example (Syntax) of the
ancillary data descriptor. An 8-bit field of
30 "descriptor_tag" indicates a descriptor type. In this case,
the field indicates an ancillary data descriptor. An 8-bit
22
field of "descriptor_length" indicates a length (size) of a
descriptor and indicates a number of following bytes as the
length of the descriptor.
[0061]
5 An 8-bit field of "ancillary_data_identifier" indicates
what kind of data is embedded in the user data area of the
audio stream. In this case, when each bit is set to "1," it
is indicated that data of a type corresponding to the bit is
embedded. Fig. 12 illustrates a correspondence relation
10 between bits and data types in a current condition. According
to the present embodiment, object encoded data (Object data)
is newly defined to Bit 7 as a data type and, when "1" is set
to Bit 7, it is identified that object encoded data is embedded
in the user data area of the audio stream.
15 [0062]
Further, the TS formatter 115 inserts attribute
information that indicates respective attributes of object
encoded data of the predetermined number of groups in the layer
of the container, which is in coverage of the program map table
20 (PMT) according to the present embodiment. The TS formatter
115 inserts the attribute information or the like to the audio
elementary stream loop corresponding to the audio stream by
using a 3D audio stream configuration descriptor
(3Daudio_stream_config_descriptor).
25 [0063]
Fig. 13 illustrates a structure example (Syntax) of the
3D audio stream configuration descriptor. Further, Fig. 14
illustrates content (Semantics) of main information in the
structure example. An 8-bit field of "descriptor_tag"
30 indicates a descriptor type. In this example, the 3D audio
stream configuration descriptor is indicated. An 8-bit field
23
of "descriptor_length" indicates a length (size) of the
descriptor and a number of following bytes are indicated as
the descriptor length.
[0064]
5 An 8-bit field of "NumOfGroups, N" indicates a number
of groups. An 8-bit field of "NumOfPresetGroups, P" indicates
a number of preset groups. An 8-bit field of "groupID," an
8-bit field of "attribute_of_groupID," an 8-bit field of
"SwitchGroupID," and an 8-bit field of "audio_streamID" are
10 repeated as many times as the number of groups.
[0065]
A field of "groupID" indicates an identifier of a group.
A field of "attribute_of_groupID" indicates an attribute of
object encoded data of the group. A field of "SwitchGroupID"
15 is an identifier indicating to which switch group the group
belongs. "0" indicates that the group does not belong to any
switch group. Values other than "0" indicate a switch group
to which the group belongs. An 8-bit field of "contentKind"
indicates a type of content of the group. "audio_streamID"
20 is an identifier indicating an audio stream in which the group
is included. Fig. 15 indicates a type of content defined by
"contentKind."
[0066]
Further, an 8-bit field of "presetGroupID" and an 8-bit
25 field of "NumOfGroups_in_preset, R" are repeated as many times
as the number of preset groups. A field of "presetGroupID"
is an identifier indicating grouped groups as a preset. A
field of "NumOfGroups_in_preset, R" indicates a number of
groups which belongs to the preset group. Then, in every preset
30 group, an 8-bit field of "groupID" is repeated as many times
as the number of the groups which belong to the present group
24
and the groups which belong to the preset group are indicated.
[0067]
Fig. 16 illustrates a configuration example of the
transport stream TS. In this configuration example, there
5 is "video PES" which is a PES packet of a video stream identified
by PID1. Further, in this configuration example, there is
"audio PES" which is a PES packet of an audio stream identified
by PID2. The PES packet is composed of a PES header
(PES_header) and a PES payload (PES_payload).
10 [0068]
Here, in the "audio PES" which is a PES packet of an
audio stream, MPEG4 AAC channel encoded data is included and
MPEG-H 3D Audio object encoded data is embedded in the user
data area thereof.
15 [0069]
Further, in the transport stream TS, the program map
table (PMT) is included, as program specific information (PSI).
The PSI is information that describes to which program each
elementary stream included in the transport stream belongs.
20 In the PMT, there is a program loop (Program loop) that describes
information related to the entire program.
[0070]
Further, in the PMT, there is an elementary stream loop
having information related to each elementary stream. In this
25 configuration example, there is a video elementary stream loop
(video ES loop) corresponding to a video stream as well as
an audio elementary stream loop (audio ES loop) corresponding
to an audio stream.
[0071]
30 In the video elementary stream loop (video ES loop),
corresponding to the video stream, there provided is
25
information such as a stream type, a packet identifier (PID),
or the like as well as a descriptor that describes information
related to the video stream. A value of "Stream_type" of the
video stream is set as "0x24" and PID information indicates
5 PID1 applied to "video PES" which is a PES packet of a video
stream as described above. As one of the descriptors, HEVC
descriptor is placed.
[0072]
In the audio elementary stream loop (audio ES loop),
10 corresponding to the audio stream, there provided is
information such as a stream type, a packet identifier (PID)
or the like as well as a descriptor that describes information
related to the audio stream. A value of "Stream_type" of the
audio stream is set to "0x11" and the PID information indicates
15 PID2 applied to "audio PES" which is a PES packet of an audio
stream as described above. In the audio elementary stream
loop, both of the above described ancillary data descriptor
and 3D audio stream configuration descriptor are provided.
[0073]
20 Operation of the stream generation unit 110A indicated
in Fig. 3 is briefly explained. The video data SV is supplied
to the video encoder 112. In the video encoder 112, the video
data SV is encoded and a video stream including the encoded
video data is included. The video stream is provided to the
25 TS formatter 115.
[0074]
The object data composing the audio data SA is supplied
to the audio object encoder 114. In the audio object encoder
114, MPEG-H 3D Audio encoding is performed on the object data
30 and an audio stream (object encoded data) is generated. This
audio stream is supplied to the audio channel encoder 113.
26
[0075]
The channel data composing the audio data SA is supplied
to the audio channel encoder 113. In the audio channel encoder
113, MPEG4 AAC encoding is performed on the channel data and
5 an audio stream (channel encoded data) is generated. In this
case, in the audio channel encoder 113, the audio stream (object
encoded data) generated in the audio object encoder 114 is
embedded in the user data area.
[0076]
10 The video stream generated in the video encoder 112 is
supplied to the TS formatter 115. Further, the audio stream
generated in the audio channel encoder 113 is supplied to the
TS formatter 115. In the TS formatter 115, streams provided
from each encoder are packetized as PES packets, then
15 packetized as transport packets and multiplexed, and a
transport stream TS as a multiplexed stream is obtained.
[0077]
Further, in the TS formatter 115, an ancillary data
descriptor is inserted in the audio elementary stream loop.
20 This descriptor includes identification information that
identifies that there is object encoded data embedded in the
user data area of the audio stream.
[0078]
Further, in the TS formatter 115, a 3D audio stream
25 configuration descriptor is inserted in the audio elementary
stream loop. This descriptor includes attribute information
that indicates attribute of each piece of object encoded data
of the predetermined number of groups.
[0079]
30 (A Case That The Stream Configuration (2) Is Employed)
Next, a case that the audio stream is in the stream
27
configuration (2) of Fig. 2(b) will be described. Fig. 17
illustrates a configuration example of a stream generation
unit 110B included in the service transmitter 100 in the above
case.
5 [0080]
The stream generation unit 110B includes a video encoder
122, an audio channel encoder 123, audio object encoders 124-1
to 124-N, and a TS formatter 125. The video encoder 122 inputs
video data SV and encodes the video data SV to generate a video
10 stream.
[0081]
The audio channel encoder 123 inputs channel data
composing audio data SA and encodes the channel data with MPEG4
AAC to generate an audio stream (channel encoded data) as a
15 main stream. The audio object encoders 124-1 to 124-N
respectively input object data composing the audio data SA
and encode the object data with MPEG-H 3D Audio to generate
audio streams (object encoded data) as substreams.
[0082]
20 For example, in a case of N=2, the audio object encoder
124-1 generates substream 1 and the audio object encoder 124-2
generates substream 2. For example, as illustrated in Fig.
18, in the configuration example of the object encoded data
composed of two pieces of object encoded data, the substream
25 1 includes an immersive audio object (IAO) and the substream
2 includes encoded data of a speech dialog object (SDO).
[0083]
Fig. 19 illustrates a correspondence relation between
groups and attributes. Here, a group ID (group ID) is an
30 identifier to identify a group. An attribute (attribute)
indicates an attribute of encoded data of each group. A switch
28
group ID (switch Group ID) is an identifier to identify groups
which are switchable to each other. A preset group ID (preset
Group ID) is an identifier to identify a preset group. A stream
ID (Stream ID) is an identifier to identify a stream. A kind
5 (Kind) indicates the type of content of each group.
[0084]
The illustrated correspondence relation illustrates
that the encoded data belonging to Group 1 is object encoded
data (immersive audio object encoded data) for an immersive
10 sound, does not compose a switch group, and is included in
substream 1.
[0085]
Further, the illustrated correspondence relation
illustrates that the encoded data belonging to Group 2 is object
15 encoded data (speech dialogue object encoded data) for a spoken
language of the first language, composes Switch Group 1, and
is included in substream 2. Further, the illustrated
correspondence relation illustrates that the encoded data
belonging to Group 3 is object encoded data (speech dialogue
20 object encoded data) for a spoken language of the second
language, composes Switch Group 1, and is included in substream
2.
[0086]
Further, the illustrated correspondence relation
25 illustrates that Preset Group 1 includes Group 1 and Group
2. Further, the illustrated correspondence relation
illustrates that Preset Group 2 includes Group 1 and Group
3.
[0087]
30 Referring back to Fig. 17, the TS formatter 125 packetizes
the video stream output from the video encoder 112, the audio
29
stream output from the audio channel encoder 123, and further
the audio streams output from the audio object encoders 124-1
to 124-N as PES packets, multiplexes the data as transport
packets, and obtains a transport stream TS as a multiplexed
5 stream.
[0088]
Further, in the coverage of the layer of the container,
which is in the coverage of the program map table (PMT) in
this embodiment, the TS formatter 125 inserts attribute
10 information indicating each attribute of object encoded data
in the predetermined number of groups and stream correspondence
relation information indicating to which substream the object
encoded data in the predetermined number of groups belong.
The TS formatter 125 inserts these pieces of information to
15 the audio elementary stream loop corresponding to one or more
substream among the predetermined number of substreams by using
the 3D audio stream configuration descriptor
(3Daudio_stream_config_descriptor) (see Fig. 13).
[0089]
20 Further, in the coverage of the layer of the container,
which is in the coverage of the program map table (PMT) in
this embodiment, the TS formatter 125 inserts stream identifier
information indicating each stream identifier of the
predetermined number of substreams. The TS formatter 125
25 inserts the information to the audio elementary stream loops
respectively corresponding to the predetermined number of
substreams by using the 3D audio stream ID descriptor
(3Daudio_substreamID_descriptor).
[0090]
30 Fig. 20(a) illustrates a structure example (Syntax) of
a 3D audio stream ID descriptor. Further, Fig. 20(b)
30
illustrates content (Semantics) of main information in the
structure example.
[0091]
An 8-bit field of "descriptor_tag" illustrates a
5 descriptor type. In this example, a 3D audio stream ID
descriptor is indicated. An 8-bit field of
"descriptor_length" indicates a length (size) of the
descriptor and a number of following bytes are indicated as
the descriptor length. An 8-bit field of "audio_streamID"
10 indicates an identifier of a substream.
[0092]
Fig. 21 illustrates a configuration example of a
transport stream TS. In this configuration example, there
is a PES packet "video PES" of a video stream identified by
15 PID1. Further, in this configuration example, there are PES
packets "audio PES" of two audio streams identified by PID2
and PID3 respectively. The PES packet is composed of a PES
header (PES_header) and a PES payload (PES_payload). In the
PES header, time stamps of DTS and PTS are inserted. The
20 synchronization between the devices can be maintained in the
entire system by applying the time stamps and matching the
time stamps of PID2 and PID3 when multiplexing, for example.
[0093]
In the PES packet "audio PES" of the audio stream (main
25 stream) identified by PID2, channel encoded data of MPEG4 AAC
is included. On the other hand, in the PES packet "audio PES"
of the audio stream (substream) identified by PID3, object
encoded data of the MPEG-H 3D Audio is included.
[0094]
30 Further, in the transport stream TS, a program map table
(PMT) is included as program specific information (PSI). The
31
PSI is information that describes to which program each
elementary stream included in the transport stream belongs.
In the PMT, there is a program loop (Program loop) that describes
information related to the entire program.
5 [0095]
Further, in the PMT, there is an elementary stream loop
including information related to each elementary stream. In
this configuration example, there is a video elementary stream
loop (video ES loop) corresponding to the video stream as well
10 as audio elementary stream loops (audio ES loop) corresponding
to the two audio streams.
[0096]
In the video elementary stream loop (video ES loop),
corresponding to the video stream, information such as a stream
15 type and a packet identifier (PID) is placed and a descriptor
that describes information related to the video stream is also
placed. A value of "Stream_type" of the video stream is set
to "0x24," the PID information is assumed to indicate PID1
that is allocated to the PES packet "video PES" of the video
20 stream as described above. An HEVC descriptor is also placed
as a descriptor.
[0097]
In the audio elementary stream loop (audio ES loop)
corresponding to the audio stream (main stream), information
25 such as a stream type and a packet identifier (PID) is placed
and a descriptor that describes information related to the
audio stream is also placed, corresponding to the audio stream.
A value of "Stream_type" of the audio stream is set as "0x11,"
and the PID information is assumed to indicate PID2 which is
30 applied to the PES packet "audio PES" of the audio stream (main
stream) as described above.
32
[0098]
Further, in the audio elementary stream loop (audio ES
loop) corresponding to the audio stream (substream),
information such as a stream type and a packet identifier (PID)
5 is placed and a descriptor that describes information related
to the audio stream is also placed, corresponding to the audio
stream. A value of "Stream_type" of the audio stream is set
to "0x2D," the PID information is assumed to indicate PID3
applied to the PES packet "audio PES" of the audio stream (main
10 stream) as described above. As the descriptor, the above
described 3D audio stream configuration descriptor and 3D audio
stream ID descriptor are placed.
[0099]
An operation of the stream generation unit 110B
15 illustrated in Fig. 17 will be briefly explained. The video
data SV is provided to the video encoder 122. In the video
encoder 122, the video data SV is encoded and a video stream
including the encoded video data is generated.
[0100]
20 The channel data composing the audio data SA is supplied
to the audio channel encoder 123. In the audio channel encoder
123, the channel data is encoded with MPEG4 AAC and an audio
stream (channel encoded data) as a main stream is generated.
[0101]
25 Further, the object data composing the audio data SA
is supplied to the audio object encoders 124-1 to 124-N. The
audio object encoders 124-1 to 124-N respectively encode the
object data with MPEG-H 3D Audio and generate audio streams
(object encoded data) as substreams.
30 [0102]
The video stream generated in the video encoder 122 is
33
supplied to the TS formatter 125. Further, the audio stream
(main stream) generated in the audio channel encoder 113 is
supplied to the TS formatter 125. Further, the audio streams
(substreams) generated in the audio object encoders 124-1 to
124-5 N are provided to the TS formatter 125. In the TS formatter
125, the streams provided from each encoder are packetized
as PES packets and further multiplexed as transport packets,
and a transport stream TS as a multiplexed stream is obtained.
[0103]
10 Further, the TS formatter 115 inserts a 3D audio stream
configuration descriptor in the audio elementary stream loop
corresponding to at least one or more substream in the
predetermined number of substreams. In the 3D audio stream
configuration descriptor, attribute information indicating
15 an attribute of respective pieces of object encoded data of
the predetermined number of groups, stream correspondence
relation information to which substream each piece of object
encoded data of the predetermined number of groups belongs,
or the like are included.
20 [0104]
Further, in the TS formatter 115, in the audio elementary
stream loop corresponding to the substream, that is, in the
audio elementary stream loops respectively corresponding to
predetermined number of substreams, a 3D audio stream ID
25 descriptor is inserted. In this descriptor, stream
identifier information indicating each stream identifier of
the predetermined number of audio streams is included.
[0105]
[Configuration Example Of Service Receiver]
30 Fig. 22 illustrates a configuration example of the
service receiver 200. The service receiver 200 includes a
34
reception unit 201, a TS analyzing unit 202, a video decoder
203, a video processing circuit 204, a panel drive circuit
205, and a display panel 206. Further, the service receiver
200 includes multiplexing buffers 211-1 to 211-M, a combiner
5 212, a 3D audio decoder 213, a sound output processing circuit
214, and a speaker system 215. Further, the service receiver
200 includes a CPU 221, a flash ROM 222, a DRAM 223, an internal
bus 224, a remote control reception unit 225, and a remote
control transmitter 226.
10 [0106]
The CPU 221 controls operation of each unit in the service
receiver 200. The flash ROM 222 stores control software and
keeps data. The DRAM 223 composes a work area of the CPU 221.
The CPU 221 starts software by developing the software or data
15 read from the flash ROM 222 in the DRAM 223 and controls each
unit in the service receiver 200.
[0107]
The remote control reception unit 225 receives a remote
control signal (remote control code) transmitted from the
20 remote control transmitter 226 and supplies the signal to the
CPU 221. On the basis of the remote control code, the CPU
221 controls each unit in the service receiver 200. The CPU
221, the flash ROM 222, and the DRAM 223 are connected to the
internal bus 224.
25 [0108]
The reception unit 201 receives a transport stream TS,
which is transmitted from the service transmitter 100 by using
a broadcast wave or a packet through a network. The transport
stream TS includes a predetermined number of audio streams
30 in addition to a video stream.
[0109]
35
Figs. 23(a) and 23(b) illustrate examples of an audio
stream to be received. Fig. 23(a) illustrates an example of
a case of the stream configuration (1). In this case, there
is only a main stream that includes channel encoded data, which
5 is encoded with MPEG4 AAC, and object encoded data of a
predetermined number of groups, which is encoded with MPEG-H
3D Audio, is embedded in a user data area thereof. The main
stream is identified by PID2.
[0110]
10 Fig. 23(b) illustrates an example of a case of the stream
configuration (2). In this case, there is a main stream that
includes channel encoded data encoded with MPEG4 AAC and there
are a predetermined number of substreams, one substream in
this example, including object encoded data of the
15 predetermined number of groups, which is encoded with MPEG-H
3D Audio. The main stream is identified with PID2 and the
substream is identified with PID3. Here, it is noted that,
in the stream configuration, the main stream may be identified
with PID3 and the substream may be identified with PID2.
20 [0111]
The TS analyzing unit 202 extracts a packet of a video
stream from the transport stream TS and transmits the packet
of the video stream to the video decoder 203. The video decoder
203 reconfigures a video stream from a packet of the video
25 extracted in the TS analyzing unit 202 and obtains uncompressed
image data by performing a decode process.
[0112]
The video processing circuit 204 performs a scaling
process and an image quality adjustment process on the video
30 data obtained in the video decoder 203 and obtains video data
for displaying. The panel drive circuit 205 drives the display
36
panel 206 on the basis of the image data for displaying obtained
in the video processing circuit 204. The display panel 206
is composed of, for example, a liquid crystal display (LCD)
or an organic electroluminescence display (organic EL
5 display).
[0113]
Further, the TS analyzing unit 202 extracts various
information such as descriptor information from the transport
stream TS and transmits the information to the CPU 221. In
10 the case of the stream configuration (1), the various
information includes information of an ancillary data
descriptor (Ancillary_data_descriptor) and a 3D audio stream
configuration descriptor
(3Daudio_stream_config_descriptor) (see Fig. 16). Based on
15 the descriptor information, the CPU 221 can recognize that
object encoded data is embedded in the user data area of the
main stream included in the channel encoded data, and
recognizes an attribute or the like of the object encoded data
of each group.
20 [0114]
Further, in the case of the stream configuration (2),
the various information includes information of a 3D audio
stream configuration descriptor
(3Daudio_stream_config_descriptor) and a 3D audio stream ID
25 descriptor (3Daudio_substreamID_descriptor) (see Fig. 21).
Based on the descriptor information, the CPU 221 recognizes
an attribute of the object encoded data of each group and which
substream the object encoded data of each group is included,
or the like.
30 [0115]
Further, under the control by the CPU 221, the TS
37
analyzing unit 202 selectively extracts a predetermined number
of audio streams included in the transport stream TS by using
a PID filter. In other words, in the case of the stream
configuration (1), the main stream is extracted. On the other
5 hand, in the case of the stream configuration (2), the main
stream is extracted and the predetermined number of substreams
are extracted.
[0116]
The multiplexing buffers 211-1 to 211-M respectively
10 import audio streams (only the main stream, or the main stream
and substream) extracted in the TS analyzing unit 202. Here,
the number M of the multiplexing buffers 211-1 to 211-M is
assumed to be a necessary and sufficient number and, in an
actual operation, the number of buffers as many as the number
15 of audio streams extracted in the TS analyzing unit 202 is
used.
[0117]
The combiner 212 reads, for each audio frame, an audio
stream from the multiplexing buffer to which each audio stream
20 to be extracted by the TS analyzing unit 202 is imported among
the multiplexing buffers 211-1 to 211-M, and transmits the
audio stream to the 3D audio decoder 213.
[0118]
Under the control by the CPU 221, the 3D audio decoder
25 213 extracts channel encoded data and object encoded data,
performs a decode process, and obtains audio data to drive
each speaker of the speaker system 215. In this case, in the
case of the stream configuration (1), channel encoded data
is extracted from the main stream and object encoded data is
30 extracted from the user data area. On the other hand, in a
case of the stream configuration (2), channel encoded data
38
is extracted from the main stream and object encoded data is
extracted from the substream.
[0119]
When decoding the channel encoded data, the 3D audio
5 decoder 213 performs a process of downmixing and upmixing for
the speaker configuration of the speaker system 215 according
to need and obtains audio data to drive each speaker. Further,
when decoding the object encoded data, the 3D audio decoder
213 calculates speaker rendering (a mixing ratio for each
10 speaker) on the basis of the object information (metadata),
and mixes the audio data of the object with the audio data
to drive each speaker according to the calculation result.
[0120]
The sound output processing circuit 214 performs a
15 necessary process such as a D/A conversion, amplification,
or the like on the audio data, which is obtained in the 3D
audio decoder 213 and used to drive each speaker, and supplies
the data to the speaker system 215. The speaker system 215
includes a plurality of speakers of a plurality of channels
20 such as 2 channel, 5.1 channel, 7.1 channel, 22.2 channel,
and the like.
[0121]
An operation of the service receiver 200 illustrated
in Fig. 22 will be briefly explained. The reception unit 201
25 receives a transport stream TS from the service transmitter
100, which is transmitted by using a broadcast wave or a packet
through a network. The transport stream TS includes a
predetermined number of audio streams in addition to a video
stream.
30 [0122]
For example, in the case of the stream configuration
39
(1), as an audio stream, there is only a main stream which
includes channel encoded data encoded with MPEG4 AAC and, in
the user data area thereof, a predetermined number of groups
of object encoded data encoded with MPEG-H 3D Audio is embedded.
5 [0123]
Further, for example, in the case of the stream
configuration (2), as an audio stream, there is a main stream
including channel encoded data, which is encoded with MPEG4
AAC, and there are a predetermined number of substreams
10 including object encoded data, which is encoded with MPEG-H
3D Audio, of a predetermined number of groups.
[0124]
In the TS analyzing unit 202, a packet of a video stream
is extracted from the transport stream TS and supplied to the
15 video decoder 203. In the video decoder 203, a video stream
is reconfigured from the packet of video extracted in the TS
analyzing unit 202 and a decode process is performed to obtain
uncompressed video data. The video data is supplied to the
video processing circuit 204.
20 [0125]
The video processing circuit 204 performs a scaling
process, an image quality adjustment process or the like on
the video data obtained in the video decoder 203 and obtains
video data for displaying. The video data for displaying is
25 supplied to the panel drive circuit 205. On the basis of the
video data for displaying, the panel drive circuit 205 drives
the display panel 206. With this configuration, on the display
panel 206, an image corresponding to the video data for
displaying is displayed.
30 [0126]
Further, in the TS analyzing unit 202, various
40
information such as descriptor information is extracted from
the transport stream TS and transmitted to the CPU 221. In
the case of the stream configuration (1), the various
information also includes information of an ancillary data
5 descriptor and a 3D audio stream configuration descriptor (see
Fig. 16). Based on the descriptor information, the CPU 221
recognizes that the object encoded data is embedded in the
user data area of the main stream including the channel encoded
data and also recognizes an attribute of object encoded data
10 of each group.
[0127]
Further, in the case of the stream configuration (2),
the various information also includes information of a 3D audio
stream configuration descriptor and a 3D audio stream ID
15 descriptor (see Fig. 21). Based on the descriptor information,
the CPU 221 recognizes the attribute of the object encoded
data of each group, or to which substream the object encoded
data of each group is included.
[0128]
20 Under the control by the CPU 221, in the TS analyzing
unit 202, a predetermined number of audio streams included
in the transport stream TS are selectively extracted by using
a PID filter. In other words, in the case of the stream
configuration (1), the main stream is extracted. On the other
25 hand, in the case of the stream configuration (2), the main
stream is extracted and a predetermined number of substreams
are also extracted.
[0129]
In the multiplexing buffers 211-1 to 211-M, the audio
30 stream (only the main stream, or the main stream and substream)
extracted in the TS analyzing unit 202 is imported. In the
41
combiner 212, from each multiplexing buffer in which the audio
stream is imported, the audio stream is read from each audio
frame and supplied to the 3D audio decoder 213.
[0130]
5 Under the control by the CPU 221, in the 3D audio decoder
213, the channel encoded data and object encoded data are
extracted, a decode process is performed, and audio data to
drive each speaker of the speaker system 215 is obtained. Here,
in the case of the stream configuration (1), the channel encoded
10 data is extracted from the main stream and the object encoded
data is also extracted from the user data area thereof. On
the other hand, in the case of the stream configuration (2),
the channel encoded data is extracted from the main stream
and the object encoded data is extracted from the substream.
15 [0131]
Here, when the channel encoded data is decoded, a process
of downmixing or upmixing for the speaker configuration of
the speaker system 215 is performed according to need and audio
data for driving each speaker is obtained. Further, when the
20 object encoded data is decoded, speaker rendering (a mixing
ratio for each speaker) is calculated on the basis of object
information (metadata), and, according to the calculated
result, audio data of the object is mixed to the audio data
for driving each speaker.
25 [0132]
The audio data for driving each speaker obtained in the
3D audio decoder 213 is supplied to the sound output processing
circuit 214. In the sound output processing circuit 214, a
necessary process such as a D/A conversion, amplification,
30 or the like is performed on the audio data for driving each
speaker. Then, the processed audio data is supplied to the
42
speaker system 215. With this configuration, a sound output
corresponding to the display image on the display panel 206
is obtained from the speaker system 215.
[0133]
5 Fig. 24 schematically illustrates an audio decode
process in a case of the stream configuration (1). A transport
stream TS as a multiplexed stream is input to the TS analyzing
unit 202. In the TS analyzing unit 202, a system layer analysis
is performed and descriptor information (information of an
10 ancillary data descriptor and a 3D audio stream configuration
descriptor) is supplied to the CPU 221.
[0134]
On the basis of the descriptor information, the CPU 221
recognizes that the object encoded data is embedded to the
15 user data area of the main stream including the channel encoded
data and also recognizes the attribute of the object encoded
data of each group. Under the control by the CPU 221, in the
TS analyzing unit 202, a packet of the main stream is selectively
extracted by using a PID filter and imported to the multiplexing
20 buffer 211 (211-1 to 211-M).
[0135]
In the audio channel decoder of the 3D audio decoder
213, a process is performed on the main stream imported to
the multiplexing buffer 211. In other words, in the audio
25 channel decoder, a DSE in which object encoded data is placed
is extracted from the main stream and transmitted to the CPU
221. Here, in an audio channel decoder of a related receiver,
the compatibility is maintained since the DSE is read and
discarded.
30 [0136]
Further, in the audio channel decoder, channel encoded
43
data is extracted from the main stream and a decode process
is performed so that audio data for driving each speaker is
obtained. In this case, information of the number of channels
is transmitted between the audio channel decoder and the CPU
5 221 and a process of downmixing and upmixing for the speaker
configuration of the speaker system 215 is performed according
to need.
[0137]
In the CPU 221, a DSE analysis is performed and the object
10 encoded data placed therein is transmitted to an audio object
decoder of the 3D audio decoder 213. In the audio object
decoder, the object encoded data is decoded, and metadata and
audio data of the object are obtained.
[0138]
15 The audio data for driving each speaker obtained in the
audio channel encoder is supplied to the mixing/rendering unit.
Further, the metadata and audio data of the object obtained
in the audio object decoder are also supplied to the
mixing/rendering unit.
20 [0139]
On the basis of the metadata of the object, in the
mixing/rendering unit, a decode output is performed by
calculating mapping of the audio data of the object to a speech
space with respect to a speaker output target, and additively
25 combining the calculation result to channel data.
[0140]
Fig. 25 schematically illustrates an audio decode
process in the case of the stream configuration (2). A
transport stream TS as a multiplexed stream is input to the
30 TS analyzing unit 202. In the TS analyzing unit 202, a system
layer analysis is performed and descriptor information
44
(information of a 3D audio stream configuration descriptor
and a 3D audio stream ID descriptor) is supplied to the CPU
221.
[0141]
5 On the basis of the descriptor information, the CPU 221
recognizes the attribute of the object encoded data of each
group and also recognizes to which substream the object encoded
data of each group is included, from the descriptor information.
Under the control by the CPU 221, in the TS analyzing unit
10 202, packets of a main stream and a predetermined number of
substreams are selectively extracted by using a PID filter
and imported to the multiplexing buffer 211 (211-1 to 211-M).
Here, in a related receiver, packets of the substreams are
not extracted by using a PID filter and only a main stream
15 is extracted so that the compatibility is maintained.
[0142]
In the audio channel decoder of the 3D audio decoder
213, channel encoded data is extracted from the main stream
imported to the multiplexing buffer 211 and a decode process
20 is performed so that audio data for driving each speaker can
be obtained. In this case, information of the number of
channels is transmitted between the audio channel decoder and
the CPU 221 and a process of downmixing and upmixing for the
speaker configuration of the speaker system 215 is performed
25 according to need.
[0143]
Further, in the audio object decoder of the 3D audio
decoder 213, necessary object encoded data of a predetermined
number of groups is extracted from the predetermined number
30 of substreams imported to the multiplexing buffer 211 on the
basis of user's selection or the like and a decode process
45
is performed so that metadata and audio data of the object
can be obtained.
[0144]
The audio data for driving each speaker obtained in the
5 audio channel encoder is supplied to the mixing/rendering unit.
Further, the metadata and audio data of the object obtained
in the audio object decoder are supplied to the
mixing/rendering unit.
[0145]
10 On the basis of the metadata of the object, in the
mixing/rendering unit, a decode output is performed by
calculating mapping of the audio data of the object to a speech
space with respect to the speaker output target and additively
combining the calculation result to the channel data.
15 [0146]
As described above, in the transceiving system 10
illustrated in Fig. 1, the service transmitter 100 transmits
a predetermined number of audio streams including channel
encoded data and object encoded data that compose the 3D audio
20 transmission data, and the predetermined number of audio
streams are generated so that the object encoded data is
discarded in a receiver that is not compatible with the object
encoded data. Thus, without deteriorating an efficient usage
of the transmission band, a new 3D audio service can be provided
25 as maintaining the compatibility with a related audio receiver.
[0147]
<2. Modification Examples>
Here, according to the above described embodiment, an
example that the channel encoded data encoding method is MPEG4
30 AAC has been described; however, other encoding methods such
as AC3 and AC4 for example can also be considered in a similar
46
manner. Fig. 26 illustrates a structure of an AC3 frame (AC3
Synchronization Frame). The channel data is encoded so that
a total size of "Audblock 5," "mantissa data," "AUX," and "CRC"
does not exceed three eighths of the entire size. In a case
5 of AC3, metadata MD is inserted to the area of "AUX." Fig.
27 illustrates a configuration (syntax) of auxiliary data
(Auxiliary Data) of AC3.
[0148]
When "auxdatae" is "1," the "aux data" is made to be
10 enabled, and the data in the size which is indicated by the
14 bits (in a bit unit) of "auxdatal" is defined in "auxbits."
The size of "auxbits" in this case is written in "nauxbits."
In a case of the stream configuration (1), "metadata ()"
illustrated in above Fig. 8(a) is inserted in the field of
15 "auxbits," and object encoded data is placed in the field of
"data_byte."
[0149]
Fig. 28(a) illustrates a structure of a layer of an AC4
simple transport (Simple Transport). AC4 is one of AC3 audio
20 encoding format for the next generation. There are a field
of a syncword (syncWord), a field of a frame length (frame
Length), a field of "RawAc4Frame" as an encoded data field,
and a CRC field. As illustrated in Fig. 28(b), in the field
of "RawAc4Frame," there is a field of Table Of Content (TOC)
25 in the beginning and there are fields of a predetermined number
of substreams (Substream) thereafter.
[0150]
As illustrated in Fig. 29(b), in the substream
(ac4_substream_data()), there is a metadata area (metadata)
30 and a field of "umd_payloads_substream()" is provided therein.
In the case of the stream configuration (1), object encoded
47
data is placed in the field of "umd_payloads_substream()."
[0151]
Here, as illustrated in Fig. 29(a), there is a field
of "ac4_presentation_info()" in TOC (ac4_toc()), and further
5 there is a field of "umd_info()" therein, which indicates that
there is metadata inserted in the field of
"umd_payloads_substream()).
[0152]
Fig. 30 illustrates a configuration (syntax) of
10 "umd_info()." A field of "umd_version" indicates a version
number of a umd syntax. "K_id" indicates that arbitrary
information is contained as '0x6.' The combination of the
version number and the value of "k_id" is defined to indicate
that there is metadata inserted in the payload of
15 "umd_payloads_substream()."
[0153]
Fig. 31 illustrates a configuration (syntax) of
"umd_payloads_substream()." A 5-bit field of
"umd_payload_id" is an ID value indicating that
20 "object_data_byte" is contained and the value is assumed to
be a value other than "0." A 16-bit field of "umd_payload_size"
indicates a number of bits subsequent to the field. An 8-bit
field of "userdata_synccode" is a start code of metadata and
indicates content of the metadata. For example, "0x10"
25 indicates that it is object encode data of the MPEG-H system
(MPEG-H 3D Audio). In the area of "object_data_byte," the
object encoded data is placed.
[0154]
Further, the above described embodiment describes an
30 example that the channel encoded data encoding method is MPEG4
AAC, the object encoded data encoding method is MPEG-H 3D Audio,
48
and the encoding methods of the channel encoded data and object
encoded data are different. However, it may be considered
a case that the encoding methods of the two types of encoded
data are the same method. For example, there may be a case
5 that the channel encoded data encoding method is AC4 and the
object encoded data encoding method is also AC4.
[0155]
Further, the above described embodiment describes an
example that first encoded data is channel encoded data and
10 the second encoded data which is related to the first encoded
data is object encoded data. However, the combination of the
first encoded data and the second encoded data is not limited
to this example. The present technology can similarly be
applied to a case of performing various scalable expansions,
15 which are, for example, an expansion of channel number, a
sampling rate expansion.
[0156]
(Example Of Expansion Of Channel Number)
Encoded data of related 5.1 channel is transmitted as
20 the first encoded data, and encoded data of added channel is
transmitted as the second encoded data. A related decoder
decodes only an element of 5.1 channel and a decoder compatible
with the additional channel decodes all elements.
[0157]
25 (Sampling Rate Expansion)
Encoded data of audio sample data with a related audio
sampling rate is transmitted as the first encoded data, and
encoded data of audio sample data with a higher sampling rate
is transmitted as the second encoded data. A related decoder
30 decodes only related sampling rate data, and a decoder
compatible with a higher sampling rate decodes all data.
49
[0158]
Further, the above described embodiment describes an
example that the container is a transport stream (MPEG-2 TS).
However, the present technology can also be applied to a system
5 in which data is delivered by a container in MP4 or in other
formats in a similar manner. For example, the system is an
MPEG-DASH based stream deliver system or a transceiving system
that handles an MPEG media transport (MMT) structure
transmission stream.
10 [0159]
Further, the above described embodiment describes an
example that the first encoded data is channel encoded data,
and the second encoded data is object encoded data. However,
it may be considered a case that the second encoded data is
15 another type of channel encoded data or includes object encoded
data and channel encoded data.
[0160]
Here, the present technology may employ the following
configurations.
20 (1)
A transmission device including:
an encoding unit configured to generate a predetermined
number of audio streams including first encoded data and second
encoded data which is related to the first encoded data; and
25 a transmission unit configured to transmit a container
in a predetermined format including the generated
predetermined number of audio streams,
wherein the encoding unit generates the predetermined
number of audio streams so that the second encoded data is
30 discarded in a receiver which is not compatible with the second
encoded data.
50
(2)
The transmission device according to (1), wherein an
encoding method of the first encoded data and an encoding method
of the second encoded data are different.
5 (3)
The transmission device according to (2), wherein the
first encoded data is channel encoded data and the second
encoded data is object encoded data.
(4)
10 The transmission device according to (3), wherein the
encoding method of the first encoded data is MPEG4 AAC and
the encoding method of the second encoded data is MPEG-H 3D
Audio.
(5)
15 The transmission device according to any of (1) to (4),
wherein the encoding unit generates the audio streams having
the first encoded data and embeds the second encoded data in
a user data area of the audio streams.
(6)
20 The transmission device according to (5), further
including
an information insertion unit configured to insert, in
a layer of the container, identification information
identifying that there is the second encoded data, which is
25 related to the first encoded data, embedded in the user data
area of the audio streams having the first encoded data and
included in the container.
(7)
The transmission device according to (5) or (6), wherein
30 the first encoded data is channel encoded data and the
second encoded data is object encoded data, and
51
the object encoded data of a predetermined number of
groups is embedded in the user data area of the audio stream,
the transmission device further including an
information insertion unit configured to insert, in a layer
5 of the container, attribute information that indicates an
attribute of each piece of the object encoded data of the
predetermined number of groups.
(8)
The transmission device according to any of (1) to (4),
10 wherein the encoding unit generates a first audio stream
including the first encoded data and generates a predetermined
number of second audio streams including the second encoded
data.
(9)
15 The transmission device according to (8),
wherein object encoded data of a predetermined number
of groups is included in the predetermined number of second
audio streams,
the transmission device further including an
20 information insertion unit configured to insert, in a layer
of the container, attribute information that indicates an
attribute of each piece of object encoded data of the
predetermined number of groups.
(10)
25 The transmission device according to (9), wherein the
information insertion unit further inserts, in the layer of
the container, stream correspondence relation information
that indicates in which of the second audio streams each piece
of the object encoded data of the predetermined number of groups
30 is included, respectively.
(11)
52
The transmission device according to (10), wherein the
stream correspondence relation information is information
that indicates a correspondence relation between a group
identifier identifying each piece of the object encoded data
5 of the predetermined number of groups and a stream identifier
identifying each of the predetermined number of second audio
streams.
(12)
The transmission device according to (11), wherein the
10 information insertion unit further inserts, in the layer of
the container, stream identifier information that indicates
each stream identifier of the predetermined number of second
audio streams.
(13)
15 A transmission method including:
an encoding step of generating a predetermined number
of audio streams including first encoded data and second
encoded data which is related to the first encoded data; and
a transmission step of transmitting, by a transmission
20 unit, a container in a predetermined format including the
generated predetermined number of audio streams,
wherein, in the encoding step, the predetermined number
of audio streams are generated so that the second encoded data
is discarded in a receiver which is not compatible with the
25 second encoded data.
(14)
A reception device including
a reception unit configured to receive a container in
a predetermined format including a predetermined number of
30 audio streams having first encoded data and second encoded
data which is related to the first encoded data,
53
wherein the predetermined number of audio streams are
generated so that the second encoded data is discarded in a
receiver which is not compatible with the second encoded data,
the reception device further including a processing unit
5 configured to extract the first encoded data and the second
encoded data from the predetermined number of audio streams
included in the container and process the extracted data.
(15)
The reception device according to (14), wherein an
10 encoding method of the first encoded data and an encoding method
of the second encoded data are different.
(16)
The reception device according to (14) or (15), wherein
the first encoded data is channel encoded data and the second
15 encoded data is object encoded data.
(17)
The reception device according to any of (14) to (16),
wherein the container includes the audio streams having the
first encoded data and the second encoded data embedded in
20 a user data area thereof.
(18)
The reception device according to any of (14) to (16),
wherein the container includes a first audio stream including
the first encoded data and a predetermined number of second
25 audio streams including the second encoded data.
(19)
A reception method including
a reception step of receiving, by a reception unit, a
container in a predetermined format including a predetermined
30 number of audio streams having first encoded data and second
encoded data which is related to the first encoded data,
54
wherein the predetermined number of audio streams are
generated so that the second encoded data is discarded in a
receiver which is not compatible with the second encoded data,
the reception method further including a processing step
5 of extracting the first encoded data and the second encoded
data from the predetermined number of audio streams included
in the container and processing the extracted data.
[0161]
A major characteristic of the present technology is that
10 a new 3D audio service can be provided as maintaining the
compatibility with a related audio receiver without
deteriorating the efficient usage of the transmission band
by transmitting an audio stream that includes channel encoded
data and object encoded data embedded in a user data area thereof,
15 or by transmitting an audio stream including channel encoded
data together with an audio stream including object encoded
data (see Fig. 2).
REFERENCE SIGNS LIST
20 [0162]
10 Transceiving system
100 Service transmitter
110A, 110B Stream generation unit
112, 122 Video encoder
25 113, 123 Audio channel encoder
114, 124-1 to 124-N Audio object encoder
115, 125 TS formatter
114 Multiplexor
200 Service receiver
30 201 Reception unit
202 TS analyzing unit
55
203 Video decoder
204 Video processing circuit
205 Panel drive circuit
206 Display panel
5 211-1 to 211-M Multiplexing buffer
212 Combiner
213 3D audio decoder
214 Sound output processing circuit
215 Speaker system
10 221 CPU
222 Flash ROM
223 DRAM
224 Internal bus
225 Remote control reception unit
15 226 Remote control transmitter
56
CLAIMS
1. A transmission device comprising:
an encoding unit configured to generate a predetermined
5 number of audio streams including first encoded data and second
encoded data which is related to the first encoded data; and
a transmission unit configured to transmit a container
in a predetermined format including the generated
predetermined number of audio streams,
10 wherein the encoding unit generates the predetermined
number of audio streams so that the second encoded data is
discarded in a receiver which is not compatible with the second
encoded data.
15 2. The transmission device according to claim 1, wherein
an encoding method of the first encoded data and an encoding
method of the second encoded data are different.
3. The transmission device according to claim 2, wherein
20 the first encoded data is channel encoded data and the second
encoded data is object encoded data.
4. The transmission device according to claim 3, wherein
the encoding method of the first encoded data is MPEG4 AAC
25 and the encoding method of the second encoded data is MPEG-H
3D Audio.
5. The transmission device according to claim 1, wherein
the encoding unit generates the audio streams having the first
30 encoded data and embeds the second encoded data in a user data
area of the audio streams.
57
6. The transmission device according to claim 5, further
comprising
an information insertion unit configured to insert, in
5 a layer of the container, identification information
identifying that there is the second encoded data, which is
related to the first encoded data, embedded in the user data
area of the audio streams having the first encoded data and
included in the container.
10
7. The transmission device according to claim 5, wherein
the first encoded data is channel encoded data and the
second encoded data is object encoded data, and
the object encoded data of a predetermined number of
15 groups is embedded in the user data area of the audio stream,
the transmission device further comprising an
information insertion unit configured to insert, in a layer
of the container, attribute information that indicates an
attribute of each piece of the object encoded data of the
20 predetermined number of groups.
8. The transmission device according to claim 1, wherein
the encoding unit generates a first audio stream including
the first encoded data and generates a predetermined number
25 of second audio streams including the second encoded data.
9. The transmission device according to claim 8,
wherein object encoded data of a predetermined number
of groups is included in the predetermined number of second
30 audio streams,
the transmission device further comprising an
58
information insertion unit configured to insert, in a layer
of the container, attribute information that indicates an
attribute of each piece of object encoded data of the
predetermined number of groups.
5
10. The transmission device according to claim 9, wherein
the information insertion unit further inserts, in the layer
of the container, stream correspondence relation information
that indicates in which of the second audio streams each piece
10 of the object encoded data of the predetermined number of groups
is included, respectively.
11. The transmission device according to claim 10, wherein
the stream correspondence relation information is information
15 that indicates a correspondence relation between a group
identifier identifying each piece of the object encoded data
of the predetermined number of groups and a stream identifier
identifying each of the predetermined number of second audio
streams.
20
12. The transmission device according to claim 11, wherein
the information insertion unit further inserts, in the layer
of the container, stream identifier information that indicates
each stream identifier of the predetermined number of second
25 audio streams.
13. A transmission method comprising:
an encoding step of generating a predetermined number
of audio streams including first encoded data and second
30 encoded data which is related to the first encoded data; and
a transmission step of transmitting, by a transmission
59
unit, a container in a predetermined format including the
generated predetermined number of audio streams,
wherein, in the encoding step, the predetermined number
of audio streams are generated so that the second encoded data
5 is discarded in a receiver which is not compatible with the
second encoded data.
14. A reception device comprising
a reception unit configured to receive a container in
10 a predetermined format including a predetermined number of
audio streams having first encoded data and second encoded
data which is related to the first encoded data,
wherein the predetermined number of audio streams are
generated so that the second encoded data is discarded in a
15 receiver which is not compatible with the second encoded data,
the reception device further comprising a processing
unit configured to extract the first encoded data and the second
encoded data from the predetermined number of audio streams
included in the container and process the extracted data.
20
15. The reception device according to claim 14, wherein an
encoding method of the first encoded data and an encoding method
of the second encoded data are different.
25 16. The reception device according to claim 14, wherein the
first encoded data is channel encoded data and the second
encoded data is object encoded data.
17. The reception device according to claim 14, wherein the
30 container includes the audio streams having the first encoded
data and the second encoded data embedded in a user data area
60
thereof.
18. The reception device according to claim 14, wherein the
container includes a first audio stream including the first
5 encoded data and a predetermined number of second audio streams
including the second encoded data.
19. A reception method comprising
a reception step of receiving, by a reception unit, a
10 container in a predetermined format including a predetermined
number of audio streams having first encoded data and second
encoded data which is related to the first encoded data,
wherein the predetermined number of audio streams are
generated so that the second encoded data is discarded in a
15 receiver which is not compatible with the second encoded data,
the reception method further comprising a processing
step of extracting the first encoded data and the second encoded
data from the predetermined number of audio streams included
in the container and processing the extracted data.