1
FORM 2
THE PATENTS ACT, 1970
(39 of 1970)
& The Patent Rules, 2003
COMPLETE SPECIFICATION
1. TITLE OF THE INVENTION:
DECODER, ENCODER AND METHOD FOR INFORMED LOUDNESS ESTIMATION
IN OBJECT-BASED AUDIO CODING SYSTEMS
2. APPLICANT:
Name: FRAUNHOFER-GESELLSCHAFT ZUR FÖRDERUNG DER ANGEWANDTEN
FORSCHUNG E.V.
Nationality: Germany
Address: Hansastraße 27c, 80686 München, Germany.
3. PREAMBLE TO THE DESCRIPTION:
The following specification particularly describes the invention and the manner in which it
is to be performed:
2
5 Description
The present invention relates to audio signal encoding, processing and decoding, and, in
particular, to a decoder, an encoder and method for informed loudness estimation in
object-based audio coding systems.
10
Recently, parametric techniques for bitrate-efficient transmission/storage of audio scenes
comprising multiple audio object signals have been proposed in the field of audio coding
[BCC, JSC, SAOC, SAOC1, SAOC2] and informed source separation [ISS1, ISS2, ISS3,
ISS4, ISS5, ISS6]. These techniques aim at reconstructing a desired output audio scene
15 or audio source object based on additional side information describing the
transmitted/stored audio scene and/or source objects in the audio scene. This
reconstruction takes place in the decoder using an informed source separation scheme.
The reconstructed objects may be combined to produce the output audio scene.
Depending on the way the objects are combined, the perceptual loudness of the output
20 scene may vary.
In TV and radio broadcast, the volume levels of the audio tracks of various programs may
be normalized based on various aspects, such as the peak signal level or the loudness
level. Depending on the dynamic properties of the signals, two signals with the same peak
25 level may have a widely differing level of perceived loudness. Now switching between
programs or channels the differences in the signal loudness are very annoying and have
been to be a major source for end-user complaints in broadcast.
In the prior art, it has been proposed to normalize all the programs on all channels
30 similarly to a common reference level using a measure based on perceptual signal
loudness. One such recommendation in Europe is the EBU Recommendation R128 [EBU]
(later referred to as R128).
The recommendation says that the “program loudness”, e.g., the average loudness over
35 one program (or one commercial, or some other meaningful program entity) should equal
a specified level (with small allowed deviations). When more and more broadcasters
comply with this recommendation and the required normalization, the differences in the
average loudness between programs and channels should be minimized.
3
Loudness estimation can be performed in several ways. There exist several mathematical
models for estimating the perceptual loudness of an audio signal. The EBU
recommendation R128 relies on the model presented in ITU-R BS.1770 (later referred to
as BS.1770) (see [ITU]) for the loudness 5 estimation.
As stated before, e.g., according to the EBU Recommendation R128, the program
loudness, e.g., the average loudness over one program should equal a specified level with
small allowed deviations. However, this leads to significant problems when audio
10 rendering is conducted, unsolved until now in the prior art. Conducting audio rendering on
the decoder side has a significant effect on the overall/total loudness of the received audio
input signal. However, despite scene rendering is conducted, the total loudness of the
received audio signal shall remain the same.
15 Currently, no specific decoder-side solution exists for this problem.
EP 2 146 522 A1 ([EP]), relates to concepts for generating audio output signals using
object based metadata. At least one audio output signal is generated representing a
superposition of at least two different audio object signals, but does not provide a solution
20 for this problem.
WO 2008/035275 A2 ([BRE]) describes an audio system comprising an encoder which
encodes audio objects in an encoding unit that generates a down-mix audio signal and
parametric data representing the plurality of audio objects. The down-mix audio signal and
25 parametric data is transmitted to a decoder which comprises a decoding unit which
generates approximate replicas of the audio objects and a rendering unit which generates
an output signal from the audio objects. The decoder furthermore contains a processor for
generating encoding modification data which is sent to the encoder. The encoder then
modifies the encoding of the audio objects, and in particular modifies the parametric data,
30 in response to the encoding modification data. The approach allows manipulation of the
audio objects to be controlled by the decoder but performed fully or partly by the encoder.
Thus, the manipulation may be performed on the actual independent audio objects rather
than on approximate replicas thereby providing improved performance.
4
EP 2 146 522 A1 ([SCH]) discloses an apparatus for generating at least one audio output
signal representing a superposition of at least two different audio objects comprises a
processor for processing an audio input signal to provide an object representation of the
audio input signal, where this object representation can be generated by a parametrically
guided approximation of original objects using an object downmix 5 signal. An object
manipulator individually manipulates objects using audio object based metadata referring
to the individual audio objects to obtain manipulated audio objects. The manipulated audio
objects are mixed using an object mixer for finally obtaining an audio output signal having
one or several channel signals depending on a specific rendering setup.
10
WO 2008/046531 A1 ([ENG]) describes an audio object coder for generating an encoded
object signal using a plurality of audio objects includes a downmix information generator
for generating downmix information indicating a distribution of the plurality of audio objects
into at least two downmix channels, an audio object parameter generator for generating
15 object parameters for the audio objects, and an output interface for generating the
imported audio output signal using the downmix information and the object parameters.
An audio synthesizer uses the downmix information for generating output data usable for
creating a plurality of output channels of the predefined audio output configuration.
20 It would be desirable to have an accurate estimate of the output average loudness or the
change in the average loudness without a delay and when the program does not change
or the rendering scene is not changed, the average loudness estimate should also remain
static.
25 The object of the present invention is to provide improved audio signal encoding,
processing and decoding concepts. The object of the present invention is solved by a
decoder according to claim 1, by an encoder according to claim 15, by a system according
to claim 18, by a method according to claim 19, by a method according to claim 20 and by
a computer program according to claim 21.
30
An informed way for estimating the loudness of the output in an object-based audio coding
system is provided. The provided concepts rely on information on the loudness of the
objects in the audio mixture to be provided to the decoder. The decoder uses this
information along with the rendering information for estimating the loudness of the output
35 signal. This allows then, for example, to estimate the loudness difference between the
default downmix and the rendered output. It is then possible to compensate for the
difference to obtain approximately constant loudness in the output regardless of the
rendering information. The loudness estimation in the decoder takes place in a fully
5
parametric manner, and it is computationally very light and accurate in comparison to
signal-based loudness estimation concepts.
Concepts for obtaining information on the loudness of the specific output scene using
purely parametric concepts are provided, which then allows for loudness 5 processing
without explicit signal-based loudness estimation in the decoder. Moreover, the specific
technology of Spatial Audio Object Coding (SAOC) standardized by MPEG [SAOC] is
described, but the provided concepts can be used in conjunction with other audio object
coding technologies, too.
10
A decoder for generating an audio output signal comprising one or more audio output
channels is provided. The decoder comprises a receiving interface for receiving an audio
input signal comprising a plurality of audio object signals, for receiving loudness
information on the audio object signals, and for receiving rendering information indicating
15 whether one or more of the audio object signals shall be amplified or attenuated.
Moreover, the decoder comprises a signal processor for generating the one or more audio
output channels of the audio output signal. The signal processor is configured to
determine a loudness compensation value depending on the loudness information and
depending on the rendering information. Furthermore, the signal processor is configured
20 to generate the one or more audio output channels of the audio output signal from the
audio input signal depending on the rendering information and depending on the loudness
compensation value.
According to an embodiment, the signal processor may be configured to generate the one
25 or more audio output channels of the audio output signal from the audio input signal
depending on the rendering information and depending on the loudness compensation
value, such that a loudness of the audio output signal is equal to a loudness of the audio
input signal, or such that the loudness of the audio output signal is closer to the loudness
of the audio input signal than a loudness of a modified audio signal that would result from
30 modifying the audio input signal by amplifying or attenuating the audio object signals of
the audio input signal according to the rendering information.
According to another embodiment, each of the audio object signals of the audio input
signal may be assigned to exactly one group of two or more groups, wherein each of the
35 two or more groups may comprise one or more of the audio object signals of the audio
input signal. In such an embodiment, the receiving interface may be configured to receive
a loudness value for each group of the two or more groups as the loudness information,
wherein said loudness value indicates an original total loudness of the one or more audio
6
object signals of said group. Furthermore, the receiving interface may be configured to
receive the rendering information indicating for at least one group of the two or more
groups whether the one or more audio object signals of said group shall be amplified or
attenuated by indicating a modified total loudness of the one or more audio object signals
of said group. Moreover, in such an embodiment, the signal processor 5 may be configured
to determine the loudness compensation value depending on the modified total loudness
of each of said at least one group of the two or more groups and depending on the original
total loudness of each of the two or more groups. Furthermore, the signal processor may
be configured to generate the one or more audio output channels of the audio output
10 signal from the audio input signal depending on the modified total loudness of each of said
at least one group of the two or more groups and depending on the loudness
compensation value.
In particular embodiments, at least one group of the two or more groups may comprise
15 two or more of the audio object signals.
Moreover an encoder is provided. The encoder comprises an object-based encoding unit
for encoding a plurality of audio object signals to obtain an encoded audio signal
comprising the plurality of audio object signals. Furthermore, the encoder comprises an
20 object loudness encoding unit for encoding loudness information on the audio object
signals. The loudness information comprises one or more loudness values, wherein each
of the one or more loudness values depends on one or more of the audio object signals.
According to an embodiment, each of the audio object signals of the encoded audio signal
25 may be assigned to exactly one group of two or more groups, wherein each of the two or
more groups comprises one or more of the audio object signals of the encoded audio
signal. The object loudness encoding unit may be configured to determine the one or
more loudness values of the loudness information by determining a loudness value for
each group of the two or more groups, wherein said loudness value of said group
30 indicates an original total loudness of the one or more audio object signals of said group.
Furthermore, a system is provided. The system comprises an encoder according to one of
the above-described embodiments for encoding a plurality of audio object signals to
obtain an encoded audio signal comprising the plurality of audio object signals, and for
35 encoding loudness information on the audio object signals. Moreover, the system
comprises a decoder according to one of the above-described embodiments for
generating an audio output signal comprising one or more audio output channels. The
decoder is configured to receive the encoded audio signal as an audio input signal and
7
the loudness information. Moreover, the decoder is configured to further receive rendering
information. Furthermore, the decoder is configured to determine a loudness
compensation value depending on the loudness information and depending on the
rendering information. Moreover, the decoder is configured to generate the one or more
audio output channels of the audio output signal from the audio input signal 5 depending on
the rendering information and depending on the loudness compensation value.
Moreover, a method for generating an audio output signal comprising one or more audio
output channels is provided. The method comprises:
10
- Receiving an audio input signal comprising a plurality of audio object signals.
- Receiving loudness information on the audio object signals.
15 - Receiving rendering information indicating whether one or more of the audio object
signals shall be amplified or attenuated.
- Determining a loudness compensation value depending on the loudness
information and depending on the rendering information. And:
20
- Generating the one or more audio output channels of the audio output signal from
the audio input signal depending on the rendering information and depending on
the loudness compensation value.
25 Furthermore, a method for encoding is provided. The method comprises:
- Encoding an audio input signal comprising a plurality of audio object signals. And:
- Encoding loudness information on the audio object signals, wherein the loudness
30 information comprises one or more loudness values, wherein each of the one or
more loudness values depends on one or more of the audio object signals.
Moreover, a computer program for implementing the above-described method when being
executed on a computer or signal processor is provided.
35
Preferred embodiments are provided in the dependent claims.
8
In the following, embodiments of the present invention are described in more detail with
reference to the figures, in which:
Fig. 1 illustrates a decoder for generating an audio output signal comprising one
or more audio output channels according 5 to an embodiment,
Fig. 2 illustrates an encoder according to an embodiment,
Fig. 3 illustrates a system according to an embodiment,
10
Fig. 4 illustrates a Spatial Audio Object Coding system comprising an SAOC
encoder and a SAOC decoder,
Fig. 5 illustrates an SAOC decoder comprising a side information decoder, an
15 object separator and a renderer,
Fig. 6 illustrates a behavior of output signal loudness estimates on a loudness
change,
20 Fig. 7 depicts informed loudness estimation according to an embodiment,
illustrating components of an encoder and a decoder according to an
embodiment,
Fig. 8 illustrates an encoder according to another embodiment,
25
Fig. 9 illustrates an encoder and a decoder according to an embodiment related
to the SAOC-Dialog Enhancement, which comprises bypass channels,
Fig. 10 depicts a first illustration of a measured loudness change and the result of
30 using the provided concepts for estimating the change in the loudness in a
parametrical manner,
Fig. 11 depicts a second illustration of a measured loudness change and the result
of using the provided concepts for estimating the change in the loudness in
35 a parametrical manner, and
Fig. 12 illustrates another embodiment for conducting loudness compensation.
9
Before preferred embodiments are described in detail, loudness estimation, Spatial Audio
Object Coding (SAOC) and Dialogue Enhancement (DE) are described.
At first, loudness estimation is described.
5
As already stated before, the EBU recommendation R128 relies on the model presented
in ITU-R BS.1770 for the loudness estimation. This measure will be used as an example,
but the described concepts below can be applied also for other loudness measures.
10 The operation of the loudness estimation according to BS.1770 is relatively simple and it
is based on the following main steps [ITU]:
- The input signal i x (or signals in the case of multi-channel signal) is filtered with a Kfilter
(a combination of a shelving and a high-pass filters) to obtain the signal(s) i y .
15
- The mean squared energy i z of the signal i y is calculated.
- In the case of multi-channel signal, channel weighting i G is applied, and the
weighted signals are summed. The loudness of the signal is then defined to be
20
10 10 log i i
i
L  c  G z ,
with the constant value c  0.691. The output is then expressed in the units of
25 “LKFS” (Loudness, K-weighted, relative to Full Scale) which scales similarly to the
decibel scale.
In the above formula, Gi may, for example, be equal to 1 for some of the channels, while
Gi may, for example, be 1.41 for some other channels. For example, if a left channel, a
30 right channel, a center channel, a left surround channel and a right surround channel is
considered, the respective weights Gi may, for example, be 1 for the left, right and center
channel, and may, for example, be 1.41 for the left surround channel and the right
surround channel, see [ITU].
35 It can be seen, that the loudness value L is closely related to the logarithm of the signal
energy.
10
In the following, Spatial Audio Object Coding is described.
Object-based audio coding concepts allow for much flexibility in the decoder side of the
chain. An example of an object-based audio coding concept is Spatial Audio Object
5 Coding (SAOC).
Fig. 4 illustrates a Spatial Audio Object Coding (SAOC) system comprising an SAOC
encoder 410 and an SAOC decoder 420.
10 The SAOC encoder 410 receives N audio object signals S1, …, SN as the input. Moreover,
the SAOC encoder 410 further receives instructions “Mixing information D” how these
objects should be combined to obtain a downmix signal comprising M downmix channels
X1, …, XM. The SAOC encoder 410 extracts some side information from the objects and
from the downmixing process, and this side information is transmitted and/or stored along
15 with the downmix signals.
A major property of an SAOC system is that the downmix signal X comprising the
downmix channels X1, …, XM forms a semantically meaningful signal. In other words, it is
possible to listen to the downmix signal. If, for example, the receiver does not have the
20 SAOC decoder functionality, the receiver can nonetheless always provide the downmix
signal as the output.
Fig. 5 illustrates an SAOC decoder comprising a side information decoder 510, an object
separator 520 and a renderer 530. The SAOC decoder illustrated by Fig. 5 receives, e.g.,
25 from an SAOC encoder, the downmix signal and the side information. The downmix signal
can be considered as an audio input signal comprising the audio object signals, as the
audio object signals are mixed within the downmix signal (the audio object signals are
mixed within the one or more downmix channels of the downmix signal).
30 The SAOC decoder may, e.g., then attempt to (virtually) reconstruct the original objects,
e.g., by employing the object separator 520, e.g., using the decoded side information.
These (virtual) object reconstructions 1
ˆ ,..., ˆN S S , e.g., the reconstructed audio object
signals, are then combined based on the rendering information, e.g., a rendering matrix R,
to produce K audio output channels Y1, …, YK of an audio output signal Y.
35
11
In SAOC, often, audio object signals are, for example, reconstructed, e.g., by employing
covariance information, e.g., a signal covariance matrix E, that is transmitted from the
SAOC encoder to the SAOC decoder.
For example, the following formula may be employed to reconstruct 5 the audio object
signals on the decoder side:
S ˆ
= GX with G  E DH (D E DH) –1
10 wherein
N number of audio object signals,
Nsamples number of considered samples of an audio object signal
15
M number of downmix channels,
X downmix audio signal, size M x NSamples,
20 D downmixing matrix, size M x N
E signal covariance matrix, size N x N defined as E = X XH
S ˆ
parametrically reconstructed N audio object signals, size N x NSamples
25
(∙)H self-adjoint (Hermitian) operator which represents the conjugate transpose
of (∙)
Then, a rendering matrix R may be applied on the reconstructed audio object signals S ˆ to
30 obtain the audio output channels of the audio output signal Y, e.g., according to the
formula:
Y = RS ˆ
35 wherein
K number of the audio output channels Y1, …, YK of the audio output signal Y.
12
R rendering matrix of size K x N
Y audio output signal comprising the K audio output channels,
size K x NSamples
5
In Fig. 5, the process of object reconstruction, e.g., conducted by the object separator
520, is referred to with the notion “virtual”, or “optional”, as it may not necessarily need to
take place, but the desired functionality can be obtained by combining the reconstruction
and the rendering steps in the parametric domain (i.e., combining the equations).
10
In other words, instead of reconstructing the audio object signals using the mixing
information D and the covariance information E first, and then applying the rendering
information R on the reconstructed audio object signals to obtain the audio output
channels Y1, …, YK, both steps may be conducted in a single step, so that the audio
15 output channels Y1, …, YK are directly generated from the downmix channels.
For example, the following formula may be employed:
Y = RGX with G  E DH (D E DH) –1 .
20
In principle, the rendering information R may request any combination of the original audio
object signals. In practice, however, the object reconstructions may comprise
reconstruction errors and the requested output scene may not necessarily be reached. As
a rough general rule covering many practical cases, the more the requested output scene
25 differs from the downmix signal, the more there will be audible reconstruction errors.
In the following, dialogue enhancement (DE) is described. The SAOC technology may for
example by employed to realize the scenario. It should be noted, that even though the
name “Dialogue enhancement” suggests focusing on dialogue-oriented signals, the same
30 principle can be used with other signal types, too.
In the DE-scenario, the degrees of freedom in the system are limited from the general
case.
For example, the audio object signals 1,..., N 35 S S  S are grouped (and possibly mixed) into
two meta-objects of a foreground object (FGO) FGO S and a background object (BGO)
BGO S .
13
Moreover, the output scene 1,..., K Y Y  Y resembles the downmix signal 1,..., M X X  X .
More specifically, both signals have the same dimensionalities, i.e., K  M , and the enduser
can only control the relative mixing levels of the two meta-objects FGO and BGO. To
be more exact, the downmix signal is obtained by mixing the FGO and BGO with some
5 scalar weights
FGO FGO BGO BGO X  h S  h S ,
and the output scene is obtained similarly with some scalar weighting of the FGO and
10 BGO:
FGO FGO BGO BGO Y  g S  g S .
Depending on the relative values of the mixing weights, the balance between the FGO
15 and BGO may change. For example, with the setting
FGO FGO
BGO BGO
g h
g h
 

 
it is possible to increase the relative level of the FGO in the mixture. If the FGO is the
20 dialogue, this setting provides dialogue enhancement functionality.
As a use-case example, the BGO can be the stadium noises and other background sound
during a sports event and the FGO is the voice of the commentator. The DE-functionality
allows the end-user to amplify or attenuate the level of the commentator in relation to the
25 background.
Embodiments are based on the finding that utilizing the SAOC-technology (or similar) in a
broadcast scenario allows providing the end-user extended signal manipulation
functionality. More functionality than only changing the channel and adjusting the playback
30 volume is provided.
One possibility to employ the DE-technology is briefly described above. If the broadcast
signal, being the downmix signal for SAOC, is normalized in level, e.g., according to
R128, the different programs have similar average loudness when no (SAOC-)processing
35 is applied (or the rendering description is the same as the downmixing description).
However, when some (SAOC-)processing is applied, the output signal differs from the
14
default downmix signal and the loudness of the output signal may be different from the
loudness of the default downmix signal. From the point of view of the end-user, this may
lead into a situation in which the output signal loudness between channels or programs
may again have the un-desirable jumps or differences. In other words, the benefits of the
normalization applied by the broadcaster are 5 partially lost.
This problem is not specific for SAOC or for the DE-scenario only, but may occur also with
other audio coding concepts that allow the end-user to interact with the content. However,
in many cases it does not cause any harm if the output signal has a different loudness
10 than the default downmix.
As stated before, a total loudness of an audio input signal program should equal a
specified level with small allowed deviations. However, as already outlined, this leads to
significant problems when audio rendering is conducted, as rendering may have a
15 significant effect on the overall/total loudness of the received audio input signal. However,
despite scene rendering is conducted, the total loudness of the received audio signal shall
remain the same.
One approach would be to estimate the loudness of a signal while it is being played, and
20 with an appropriate temporal integration concept, the estimate may converge to the true
average loudness after some time. The time required for the convergence is however
problematic from the point of view of the end-user. When the loudness estimate changes
even when no changes are applied on the signal, the loudness change compensation
should also react and change its behavior. This would lead into an output signal with
25 temporally varying average loudness, which can be perceived as rather annoying.
Fig. 6 illustrates a behavior of output signal loudness estimates on a loudness change.
Inter alia, a signal-based output signal loudness estimate is depicted, which illustrates the
effect of a solution as just described. The estimate approaches the correct estimate quite
30 slowly. Instead of a signal-based output signal loudness estimate, an informed output
signal loudness estimate, that immediately determines the output signal loudness
correctly would be preferable.
In particular, in Fig. 6, the user input, e.g., the level of the dialogue object, changes at time
35 instant T by increasing in value. The true output signal level, and correspondingly the
loudness, changes at the same time instant. When the output signal loudness estimation
is performed from the output signal with some temporal integration time, the estimate will
change gradually and reach the correct value after a certain delay. During this delay, the
15
estimate values are changing and cannot reliably be used for further processing the
output signal, e.g., for loudness level correction.
As already stated, it would be desirable to have an accurate estimate of the output
average loudness or the change in the average loudness without a 5 delay and when the
program does not change or the rendering scene is not changed, the average loudness
estimate should also remain static. In other words, when some loudness change
compensation is applied, the compensation parameter should change only when either
the program changes or there is some user interaction.
10
The desired behavior is illustrated in the lowest illustration of Fig. 6 (informed output
signal loudness estimate). The estimate of the output signal loudness shall change
immediately when the user input changes.
15 Fig. 2 illustrates an encoder according to an embodiment.
The encoder comprises an object-based encoding unit 210 for encoding a plurality of
audio object signals to obtain an encoded audio signal comprising the plurality of audio
object signals.
20
Furthermore, the encoder comprises an object loudness encoding unit 220 for encoding
loudness information on the audio object signals. The loudness information comprises one
or more loudness values, wherein each of the one or more loudness values depends on
one or more of the audio object signals.
25
According to an embodiment, each of the audio object signals of the encoded audio signal
is assigned to exactly one group of two or more groups, wherein each of the two or more
groups comprises one or more of the audio object signals of the encoded audio signal.
The object loudness encoding unit 220 is configured to determine the one or more
30 loudness values of the loudness information by determining a loudness value for each
group of the two or more groups, wherein said loudness value of said group indicates an
original total loudness of the one or more audio object signals of said group.
Fig. 1 illustrates a decoder for generating an audio output signal comprising one or more
35 audio output channels according to an embodiment.
The decoder comprises a receiving interface 110 for receiving an audio input signal
comprising a plurality of audio object signals, for receiving loudness information on the
16
audio object signals, and for receiving rendering information indicating whether one or
more of the audio object signals shall be amplified or attenuated.
Moreover, the decoder comprises a signal processor 120 for generating the one or more
audio output channels of the audio output signal. The signal processor 5 120 is configured
to determine a loudness compensation value depending on the loudness information and
depending on the rendering information. Furthermore, the signal processor 120 is
configured to generate the one or more audio output channels of the audio output signal
from the audio input signal depending on the rendering information and depending on the
10 loudness compensation value.
According to an embodiment, the signal processor 110 is configured to generate the one
or more audio output channels of the audio output signal from the audio input signal
depending on the rendering information and depending on the loudness compensation
15 value, such that a loudness of the audio output signal is equal to a loudness of the audio
input signal, or such that the loudness of the audio output signal is closer to the loudness
of the audio input signal than a loudness of a modified audio signal that would result from
modifying the audio input signal by amplifying or attenuating the audio object signals of
the audio input signal according to the rendering information.
20
According to another embodiment, each of the audio object signals of the audio input
signal is assigned to exactly one group of two or more groups, wherein each of the two or
more groups comprises one or more of the audio object signals of the audio input signal.
25 In such an embodiment, the receiving interface 110 is configured to receive a loudness
value for each group of the two or more groups as the loudness information, wherein said
loudness value indicates an original total loudness of the one or more audio object signals
of said group. Furthermore, the receiving interface 110 is configured to receive the
rendering information indicating for at least one group of the two or more groups whether
30 the one or more audio object signals of said group shall be amplified or attenuated by
indicating a modified total loudness of the one or more audio object signals of said group.
Moreover, in such an embodiment, the signal processor 120 is configured to determine
the loudness compensation value depending on the modified total loudness of each of
said at least one group of the two or more groups and depending on the original total
35 loudness of each of the two or more groups. Furthermore, the signal processor 120 is
configured to generate the one or more audio output channels of the audio output signal
from the audio input signal depending on the modified total loudness of each of said at
17
least one group of the two or more groups and depending on the loudness compensation
value.
In particular embodiments, at least one group of the two or more groups comprises two or
more of the audio 5 object signals.
A direct relationship exists between the energy ei of an audio object signal i and the
loudness Li of the audio object signal i according to the formulae:
10 10 log , i i L  c  e
 /10 10 Li c
i e   10
wherein c is a constant value.
Embodiments are based on the following findings: Different audio object signals of the
15 audio input signal may have a different loudness and thus a different energy. If, e.g, a
user wants to increase the loudness of one of the audio object signals, the rendering
information may be correspondingly adjusted, and the increase of the loudness of this
audio object signal increases the energy of this audio object. This would lead to an
increased loudness of the audio output signal. To keep the total loudness constant, a
20 loudness compensation has to be conducted. In other words, the modified audio signal
that would result from applying the rendering information on the audio input signal would
have to be adjusted. However, the exact effect of the amplification of one of the audio
object signals on the total loudness of the modified audio signal depends on the original
loudness of the amplified audio object signal, e.g., of the audio object signal, the loudness
25 of which is increased. If the original loudness of this object corresponds to an energy, that
was quite low, the effect on the total loudness of the audio input signal will be minor. If,
however, the original loudness of this object corresponds to an energy, that was quite
high, the effect on the total loudness of the audio input signal will be significant.
30 Two examples may be considered. In both examples, an audio input signal comprises two
audio object signal, and in both examples, by applying the rendering information, the
energy of a first one of the audio object signals is increased by 50 %.
In the first example, the first audio object signal contributes 20 % and the second audio
35 object signal contributes 80 % to the total energy of the audio input signal. However, in the
second example, the first audio object, the first audio object signal contributes 40 % and
the second audio object signal contributes 60 % to the total energy of the audio input
18
signal. In both examples these contributions are derivable from the loudness information
on the audio object signals, as a direct relationship exists between loudness and energy.
In the first example, an increase of 50 % of the energy of the first audio object results in
that a modified audio signal that is generated by applying the rendering 5 information on the
audio input signal has a total energy 1.5 x 20 % + 80 % = 110 % of the energy of the
audio input signal.
In the second example, an increase of 50 % of the energy of the first audio object results
10 in that the modified audio signal that is generated by applying the rendering information on
the audio input signal has a total energy 1.5 x 40 % + 60 % = 120 % of the energy of the
audio input signal.
Thus, after applying the rendering information on the audio input signal, in the first
15 example, the total energy of the modified audio signal has to be reduced by only 9 %
( 10 / 110 ) to obtain equal energy in both the audio input signal and the audio output
signal, while in the second example, the total energy of the modified audio signal has to
be reduced by 17 % ( 20 / 120 ). For this purpose, a loudness compensation value may be
calculated.
20
For example, the loudness compensation value may be a scalar that is applied on all
audio output channels of the audio output signal.
According to an embodiment, the signal processor is configured to generate the modified
25 audio signal by modifying the audio input signal by amplifying or attenuating the audio
object signals of the audio input signal according to the rendering information. Moreover,
the signal processor is configured to generate the audio output signal by applying the
loudness compensation value on the modified audio signal, such that the loudness of the
audio output signal is equal to the loudness of the audio input signal, or such that the
30 loudness of the audio output signal is closer to the loudness of the audio input signal than
the loudness of the modified audio signal.
For example, in the first example above, the loudness compensation value lcv, may, for
example, be set to a value lcv = 10/11, and a multiplication factor of 10/11 may be applied
35 on all channels that result from rendering the audio input channels according to the
rendering information.
19
Accordingly, for example, in the second example above, the loudness compensation value
lcv, may, for example, be set to a value lcv = 10/12 = 5/6, and a multiplication factor of 5/6
may be applied on all channels that result from rendering the audio input channels
according to the rendering information.
5
In other embodiments, each of the audio object signals may be assigned to one of a
plurality of groups, and a loudness value may be transmitted for each of the groups
indicating a total loudness value of the audio object signals of said group. If the rendering
information specifies that the energy of one of the groups is attenuated or amplified, e.g.,
10 amplified by 50 % as above, a total energy increase may be calculated and a loudness
compensation value may be determined as described above.
For example, according to an embodiment, each of the audio object signals of the audio
input signal is assigned to exactly one group of exactly two groups as the two or more
15 groups. Each of the audio object signals of the audio input signal is either assigned to a
foreground object group of the exactly two groups or to a background object group of the
exactly to groups. The receiving interface 110 is configured to receive the original total
loudness of the one or more audio object signals of the foreground object group.
Moreover, the receiving interface 110 is configured to receive the original total loudness of
20 the one or more audio object signals of the background object group. Furthermore, the
receiving interface 110 is configured to receive the rendering information indicating for at
least one group of the exactly two groups whether the one or more audio object signals of
each of said at least one group shall be amplified or attenuated by indicating a modified
total loudness of the one or more audio object signals of said group.
25
In such an embodiment, the signal processor 120 is configured to determine the loudness
compensation value depending on the modified total loudness of each of said at least one
group, depending on the original total loudness of the one or more audio object signals of
the foreground object group, and depending on the original total loudness of the one or
30 more audio object signals of the background object group. Moreover, the signal processor
120 is configured to generate the one or more audio output channels of the audio output
signal from the audio input signal depending on the modified total loudness of each of said
at least one group and depending on the loudness compensation value.
35 According to some embodiments, each of the audio object signals is assigned to one of
three or more groups, and the receiving interface may be configured to receive a loudness
value for each of the three or more groups indicating the total loudness of the audio object
signals of said group.
20
According to an embodiment, to determine the total loudness value of two or more audio
object signals, for example, the energy value corresponding to the loudness value is
determined for each audio object signal, the energy values of all loudness values are
summed up to obtain an energy sum, and the loudness value corresponding 5 to the energy
sum is determined as the total loudness value of the two or more audio object signals. For
example, the formulae
10 10 log , i i L  c  e
 /10 10 Li c
i e   10
may be employed.
In some embodiments, loudness values are transmitted for each of the audio object
15 signals, or each of the audio object signals is assigned to one or two or more groups,
wherein for each of the groups, a loudness value is transmitted.
However, in some embodiments, for one or more audio object signals or for one or more
of the groups comprising audio object signals, no loudness value is transmitted. Instead,
20 the decoder may, for example, assume that these audio object signals or groups of audio
object signals, for which no loudness value is transmitted, have a predefined loudness
value. The decoder may, e.g., base all further determinations on this predefined loudness
value.
25 According to an embodiment, the receiving interface 110 is configured to receive a
downmix signal comprising one or more downmix channels as the audio input signal,
wherein the one or more downmix channels comprise the audio object signals, and
wherein the number of the audio object signals is smaller than the number of the one or
more downmix channels. The receiving interface 110 is configured to receive downmix
30 information indicating how the audio object signals are mixed within the one or more
downmix channels. Moreover, the signal processor 120 is configured to generate the one
or more audio output channels of the audio output signal from the audio input signal
depending on the downmix information, depending on the rendering information and
depending on the loudness compensation value. In a particular embodiment, the signal
35 processor 120 may, for example, be configured to calculate the loudness compensation
value depending on the downmix information.
21
For example, the downmix information may be a downmix matrix. In embodiments, the
decoder may be an SAOC decoder. In such embodiments, the receiving interface 110
may, e.g., be further configured to receive covariance information, e.g., a covariance
matrix as described above.
5
With respect to the rendering information indicating whether one or more of the audio
object signals shall be amplified or attenuated, it should be noted that for example,
information that indicates how one or more of the audio object signals shall be amplified or
attenuated, is rendering information. For example, a rendering matrix R, e.g., a rendering
10 matrix of SAOC, is rendering information.
Fig. 3 illustrates a system according to an embodiment.
The system comprises an encoder 310 according to one of the above-described
15 embodiments for encoding a plurality of audio object signals to obtain an encoded audio
signal comprising the plurality of audio object signals.
Moreover, the system comprises a decoder 320 according to one of the above-described
embodiments for generating an audio output signal comprising one or more audio output
20 channels. The decoder is configured to receive the encoded audio signal as an audio
input signal and the loudness information. Moreover, the decoder 320 is configured to
further receive rendering information. Furthermore, the decoder 320 is configured to
determine a loudness compensation value depending on the loudness information and
depending on the rendering information. Moreover, the decoder 320 is configured to
25 generate the one or more audio output channels of the audio output signal from the audio
input signal depending on the rendering information and depending on the loudness
compensation value.
Fig. 7 illustrates informed loudness estimation according to an embodiment. On the left of
30 transport stream 730, components of an object-based audio coding encoder are
illustrated. In particular, an object-based encoding unit 710 (“object-based audio encoder”)
and an object loudness encoding unit 720 is illustrated (“object loudness estimation”).
The transport stream 730 itself comprises loudness information L, downmixing
35 information D and the output of the object-based audio encoder 710 B.
On the right of transport stream 730, components of a signal processor of an object-based
audio coding decoder are illustrated. The receiving interface of the decoder is not
22
illustrated. An output loudness estimator 740 and an object-based audio decoding unit
750 is depicted. The output loudness estimator 740 may be configured to determine the
loudness compensation value. The object-based audio decoding unit 750 may be
configured to determine a modified audio signal from an audio signal, being input to the
decoder, by applying the rendering information R. Applying the loudness 5 compensation
value on the modified audio signal to compensate a total loudness change caused by the
rendering is not shown in Fig. 7.
The input to the encoder consists of the input objects S in the minimum. The system
10 estimates the loudness of each object (or some other loudness-related information, such
as the object energies), e.g., by the object loudness encoding unit 720, and this
information L is transmitted and/or stored. (It is also possible, the loudness of the objects
is provided as an input to the system, and the estimation step within the system can be
omitted).
15
In the embodiment of Fig. 7, the decoder receives at least the object loudness information
and, e.g., the rendering information R describing the mixing of the objects into the output
signal. Based on these, e.g., the output loudness estimator 740, estimates the loudness of
the output signal and provides this information as its output.
20
The downmixing information D may be provided as the rendering information, in which
case the loudness estimation provides an estimate of the downmix signal loudness. It is
also possible to provide the downmixing information as an input to the object loudness
estimation, and to transmit and/or store it along the object loudness information. The
25 output loudness estimation can then estimate simultaneously the loudness of the downmix
signal and the rendered output and provide these two values or their difference as the
output loudness information. The difference value (or its inverse) describes the required
compensation that should be applied on the rendered output signal for making its
loudness similar to the loudness of the downmix signal. The object loudness information
30 can additionally include information regarding the correlation coefficients between various
objects and this correlation information can be used in the output loudness estimation for
a more accurate estimate.
In the following, a preferred embodiment for dialogue enhancement application is
35 described.
23
In the dialogue enhancement application, as described above, the input audio object
signals are grouped and partially downmixed to form two meta-objects, FGO and BGO,
which can then be trivially summed for obtaining the final downmix signal.
Following the description of SAOC [SAOC], N input object signals are 5 represented as a
matrix S of the size N x NSamples, and the downmixing information as a matrix D of the size
M x N. The downmix signals can then be obtained as X = DS.
The downmixing information D can now be divided into two parts
10
FGO BGO D  D D
for the meta-objects.
15 As each column of the matrix D corresponds to an original audio object signal, the two
component downmix matrices can be obtained by setting the columns, which correspond
to the other meta-object into zero (assuming that no original object may be present in both
meta-objects). In other words, the columns corresponding to the meta-object BGO are set
to zero in FGO D , and vice versa.
20
These new downmixing matrices describe the way the two meta-objects can be obtained
from the input objects, namely:
FGO FGO S  D S and BGO BGO S  D S ,
25
and the actual downmixing is simplified to
FGO BGO X  S S .
30 It can be also considered that the object (e.g., SAOC) decoder attempts to reconstruct the
meta-objects:
FGO FGO S  S and BGO BGO S  S ,
35 and the DE-specific rendering can be written as a combination of these two meta-object
reconstructions:
24
FGO FGO BGO Y  g S  g SBGO  gFGOSFGO  gBGOSBGO   .
The object loudness estimation receives the two meta-objects FGO S and BGO S as the input
and estimates the loudness of each of them: FGO L being the (total/overall) loudness of
FGO S , and BGO L being the (total/overall) loudness of BGO 5 S . These loudness values are
transmitted and/or stored.
As an alternative, using one of the meta-objects, e.g., the FGO, as reference, it is possible
to calculate the loudness difference of these two objects, e.g., as
10
FGO BGO FGO L  L  L .
This single value is then transmitted and/or stored.
15 Fig. 8 illustrates an encoder according to another embodiment. The encoder of Fig. 8
comprises an object downmixer 811 and an object side information estimator 812.
Furthermore, the encoder of Fig. 8 further comprises an object loudness encoding unit
820. Moreover, the encoder of Fig. 8 comprises a meta audio object mixer 805.
20 The encoder of Fig. 8 uses intermediate audio meta-objects as an input to the object
loudness estimation. In embodiments, the encoder of Fig. 8 may be configured to
generate two audio meta-objects. In other embodiments, the encoder of Fig. 8 may be
configured to generate three or more audio meta-objects.
25 Inter alia, the provided concepts provide the new feature that the encoder may, e.g.,
estimates the average loudness of all input objects. The objects may, e.g., be mixed into a
downmix signal that is transmitted. The provided concepts moreover provide the new
feature that the object loudness and the downmixing information may, e.g., be included in
the object-coding side information that is transmitted.
30
The decoder may, e.g., use the object-coding side information for (virtual) separation of
the objects and re-combines the objects using the rendering information.
Furthermore, the provided concepts provide the new feature that either the downmixing
35 information can be used to estimate the loudness of the default downmix signal, the
rendering information and the received object loudness can be used for estimating the
average loudness of the output signal, and/or the loudness change can be estimated from
25
these two values. Or, the downmixing and rendering information can be used to estimate
the loudness change from the default downmix, another new feature of the provided
concepts.
Furthermore, the provided concepts provide the new feature that the 5 decoder output can
be modified to compensate for the change in the loudness so that the average loudness of
the modified signal matches the average loudness of the default downmix.
A specific embodiment related to SAOC-DE is illustrated in Fig. 9. The system receives
10 the input audio object signals, the downmixing information, and the information of the
grouping of the objects to meta-objects. Based on these, the meta audio object mixer 905
forms the two meta-objects FGO S and BGO S . It is possible, that the portion of the signal
that is processed with SAOC, does not constitute the entire signal. For example, in a 5.1
channel configuration, SAOC may be deployed on a sub-set of channels, like on the front
15 channel (left, right, and center), while the other channels (left surround, right surround,
and low-frequency effects) are routed around, (by-passing) the SAOC and delivered as
such. These channels not processed by SAOC are denoted with BYPASS X . The possible
by-pass channels need to be provided for the encoder for more accurate estimation of the
loudness information.
20
The by-pass channels may be handled in various ways.
For example, the by-pass channels may, e.g., form an independent meta-object. This
allows defining the rendering so that all three meta-objects are scaled independently.
25
Or, for example, the by-pass channels may, e.g., be combined with one of the other two
meta-objects. The rendering settings of that meta-object control also the by-pass channel
portion. For example, in the dialogue enhancement scenario, it may be meaningful to
combine the by-pass channels with the background meta-object: BGO BGO BYPASS X  S  X .
30
Or, for example, the by-pass channels may, e.g., be ignored.
According to embodiments, the object-based encoding unit 210 of the encoder is
configured to receive the audio object signals, wherein each of the audio object signals is
35 assigned to exactly one of exactly two groups, wherein each of the exactly two groups
comprises one or more of the audio object signals. Moreover, the object-based encoding
unit 210 is configured to downmix the audio object signals, being comprised by the exactly
two groups, to obtain a downmix signal comprising one or more downmix audio channels
26
as the encoded audio signal, wherein the number of the one or more downmix channels is
smaller than the number of the audio object signals being comprised by the exactly two
groups. The object loudness encoding unit 220 is assigned to receive one or more further
by-pass audio object signals, wherein each of the one or more further by-pass audio
object signals is assigned to a third group, wherein each of the one 5 or more further bypass
audio object signals is not comprised by the first group and is not comprised by the
second group, wherein the object-based encoding unit 210 is configured to not downmix
the one or more further by-pass audio object signals within the downmix signal.
10 In an embodiment, the object loudness encoding unit 220 is configured to determine a first
loudness value, a second loudness value and a third loudness value of the loudness
information, the first loudness value indicating a total loudness of the one or more audio
object signals of the first group, the second loudness value indicating a total loudness of
the one or more audio object signals of the second group, and the third loudness value
15 indicating a total loudness of the one or more further by-pass audio object signals of the
third group. In an another embodiment, the object loudness encoding unit 220 is
configured to determine a first loudness value and a second loudness value of the
loudness information, the first loudness value indicating a total loudness of the one or
more audio object signals of the first group, and the second loudness value indicating a
20 total loudness of the one or more audio object signals of the second group and of the one
or more further by-pass audio object signals of the third group.
According to an embodiment, the receiving interface 110 of the decoder is configured to
receive the downmix signal. Moreover, the receiving interface 110 is configured to receive
25 one or more further by-pass audio object signals, wherein the one or more further by-pass
audio object signals are not mixed within the downmix signal. Furthermore, the receiving
interface 110 is configured to receive the loudness information indicating information on
the loudness of the audio object signals which are mixed within the downmix signal and
indicating information on the loudness of the one or more further by-pass audio object
30 signals which are not mixed within the downmix signal. Moreover, the signal processor
120 is configured to determine the loudness compensation value depending on the
information on the loudness of the audio object signals which are mixed within the
downmix signal, and depending on the information on the loudness of the one or more
further by-pass audio object signals which are not mixed within the downmix signal.
35
Fig. 9 illustrates an encoder and a decoder according to an embodiment related to the
SAOC-DE, which comprises by-pass channels. Inter alia, the encoder of Fig. 9 comprises
an SAOC encoder 902.
27
In the embodiment of Fig. 9, the possible combining of the by-pass channels with the
other meta-objects takes place in the two “bypass inclusion” blocks 913, 914, producing
the meta-objects FGO X and BGO X with the defined parts from the by-pass channels
5 included.
The perceptual loudness BYPASS L , FGO L , and BGO L of both of these meta-objects are
estimated in the loudness estimation units 921, 922, 923. This loudness information is
then transformed into an appropriate encoding in a meta-object loudness information
10 estimator 925 and then transmitted and/or stored.
The actual SAOC en- and decoder operate as expected extracting the object side
information from the objects, creating the downmix signal X , and transmitting and/or
storing the information to the decoder. The possible by-pass channels are transmitted
15 and/or stored along the other information to the decoder.
The SAOC-DE decoder 945 receives a gain value “Dialog gain” as a user-input. Based
on this input and the received downmixing information, the SAOC decoder 945
determines the rendering information. The SAOC decoder 945 then produces the
20 rendered output scene as the signal Y . In addition to that, it produces a gain factor (and a
delay value) that should be applied on the possible by-pass signals BYPASS X .
The “bypass inclusion” unit 955 receives this information along with the rendered output
scene and the by-pass signals and creates the full output scene signal. The SAOC
25 decoder 945 produces also a set of meta-object gain values, the amount of these
depending on the meta-object grouping and desired loudness information form.
The gain values are provided to the mixture loudness estimator 960 which also receives
the meta-object loudness information from the encoder.
30
The mixture loudness estimator 960 is then able to determine the desired loudness
information, which may include, but is not limited to, the loudness of the downmix signal,
the loudness of the rendered output scene, and/or the difference in the loudness between
the downmix signal and the rendered output scene.
35
In some embodiments, the loudness information itself is enough, while in other
embodiments, it is desirable to process the full output depending on the determined
loudness information. This processing may, for example, be compensation of any possible
28
difference in the loudness between the downmix signal and the rendered output scene.
Such a processing, e.g., by a loudness processing unit 970, would make sense in the
broadcast scenario, as it would reduce the changes in the perceived signal loudness
regardless of the user interaction (setting of the input “dialog gain”).
5
The loudness-related processing in this specific embodiment comprises the a plurality of
new features. Inter alia, the FGO, BGO, and the possible by-pass channels are pre-mixed
into the final channel configuration so that the downmixing can be done by simply adding
the two pre-mixed signals together (e.g., downmix matrix coefficients of 1), which
10 constitutes a new feature. Moreover, as a further new feature, the average loudness of the
FGO and BGO are estimated, and the difference is calculated. Furthermore, the objects
are mixed into a downmix signal that is transmitted. Moreover, as a further new feature,
the loudness difference information is included to the side information that is transmitted.
(new) Furthermore, the decoder uses the side information for (virtual) separation of the
15 objects and re-combines the objects using the rendering information which is based on
the downmixing information and the user input modification gain. Moreover, as another
new feature, the decoder uses the modification gain and the transmitted loudness
information for estimating the change in the average loudness of the system output
compared to the default downmix.
20
In the following, a formal description of embodiments is provided.
Assuming that the object loudness values behave similar to the logarithm of energy values
when summing the objects, i.e., the loudness values must be transformed into linear
25 domain, added there, and finally transformed back to the logarithmic domain. Motivating
this through the definition of BS.1770 loudness measure will now be presented (for
simplicity, the number of channels is set to one, but the same principle can be applied on
multi-channel signals with appropriate summing over channels).
The loudness of the ith K-filtered signal i z with the mean-squared energy i 30 e is defined as
10 10 log , i i L  c  e
wherein c is an offset constant. For example, c may be –0.691. From this follows that the
35 energy of the signal can be determined from the loudness with
 /10 10 Li c
i e   .
29
The energy of the sum of N uncorrelated signals
1
N
SUM i
i
z z

  is then
 /10
1 1
10 i
N N
L c
SUM i
i i
e e 
 
  ,
5
and the loudness of this sum signal is then
 /10
10 10
1
10 log 10 log 10 i
N
L c
SUM SUM
i
L c e c 

     .
If the signals are not uncorrelated, the correlation coefficients i, j 10 C must be taken into
account when approximating the energy of the sum signal as
,
1 1
N N
SUM i j
i j
e e
 
  ,
wherein the cross-energy i, j e between the ith and 15 jth objects is defined as
   
 
, ,
/10 /10
,
2 /10
,
10 10
10
i j
i j
i j i j i j
L c L c
i j
L L c
i j
e C e e
C
C
 
 



,
wherein , 1 1 i j   C  is the correlation coefficient between the two objects i and j .
20 When two objects are uncorrelated, the correlation coefficient equals to 0, and when the
two objects are identical, the correlation coefficient equals to 1.
Further extending the model with mixing weights i g to be applied on the signals in the
mixing process, i.e.,
1
N
SUM i i
i
z g z

  , the energy of the sum signal will be
25
,
1 1
N N
SUM i j i j
i j
e g g e
 
  ,
and the loudness of the mixture signal can be obtained from this, as earlier, with
30
10 LSUM  c 10log eSUM .
The difference between the loudness of two signals can be estimated as
5
 ,  i j L i j  L  L .
If the definition of loudness is now used as earlier, this can be written as
 
    10 10
10
,
10 log 10 log
10 log
i j
i j
i
j
L i j L L
c e c e
e
e
  
   

10 ,
which can be observed to be a function of signal energies. If it is now desired to estimate
the loudness difference between two mixtures
1
N
A i i
i
z g z

  and
1
N
B i i
i
z h z

15 
with possibly differing mixing weights i g and i h , this can be estimated with
 
 
 
 
 
2 /10
2 /10
/10
/10
10
,
1 1
10
,
1 1
,
1 1
10
,
1 1
,
1 1
10
,
1 1
, 10 log
10 log
10
10 log
10
10
10 log
10
Li Lj c
Li L j c
Li Lj
Li L j
A
B
N N
i j i j
i j
N N
i j i j
i j
N N
i j i j
i j
N N
i j i j
i j
N N
i j i j
i j
N N
i j i j
i j
e
L A B
e
g g e
h h e
g g C
h h C
g g C
h h C
 
 


 
 
 
 
 
 
 









.
31
In the case the objects are uncorrelated (Ci, j  0,i  j and , 1, i j C  i  j ), the
difference estimate becomes
 
 
 
/10
/10
/10
/10
2
1
10
2
1
2
1
10
2
1
10
, 10 log
10
10
10 log
10
Li c
Li c
Li
Li
N
i
i
N
i
i
N
i
i
N
i
i
g
L A B
h
g
h






 





5 .
In the following, differential encoding is considered.
It is possible to encode the per-object loudness values as differences from the loudness of
10 a selected reference object:
i i REF K  L  L ,
wherein REF L is the loudness of the reference object. This encoding is beneficial if no
15 absolute loudness values are needed as the result, because it is now necessary to
transmit one value less, and the loudness difference estimation can be written as
 
 
 
/10
/10
,
1 1
10
,
1 1
10
, 10 log
10
Ki K j
Ki K j
N N
i j i j
i j
N N
i j i j
i j
g g C
L A B
h h C


 
 
 


,
20 or in the case of uncorrelated objects
 
/10
/10
2
1
10
2
1
10
, 10 log
10
Ki
Ki
N
i
i
N
i
i
g
L A B
h


 


.
In the following, a dialogue enhancement scenario is considered.
25
32
Considering again the application scenario of dialogue enhancement. The freedom of
defining the rendering information in the decoder is limited only into changing the levels of
the two meta-objects. Let us furthermore assume that the two meta-objects are
uncorrelated, i.e., , 0 FGO BGO C  . If the downmixing weights of the meta-objects are FGO h
and BGO h , and they are rendered with the gains FGO f and BGO f , the 5 loudness of the
output relative to the default downmix is
 
   
   
/10 /10
/10 /10
/10 /10
/10 /10
2 2
10 2 2
2 2
10 2 2
10 10
, 10 log
10 10
10 10
10 log
10 10
LFGO c LBGO c
LFGO c LBGO c
LFGO LBGO
LFGO LBGO
FGO BGO
FGO BGO
FGO BGO
FGO BGO
f f
L A B
h h
f f
h h
 
 

 




.
10 This is then also the required compensation if it is desired to have the same loudness in
the output as in the default downmix.
ΔL(A, B) may be considered as a loudness compensation value, that may be transmitted
by the signal processor 120 of the decoder. Δ_______L(A, B) can also be named as a loudness
15 change value and thus the actual compensation value can be an inverse value. Or is it ok
to use the “loudness compensation factor” name for it, too? Thus, the loudness
compensation value lcv mentioned earlier in this document would correspond the value
gDelta below.
For example,
 , 
10 20
L A B
g

 20  1 / ΔL(A, B) may be applied as a multiplication factor on
each channel of a modified audio signal that results from applying the rendering
information on the audio input signal. This equation for gDelta works in the linear domain. In
the logarithmic domain, the equation would be different such as 1 / ΔL(A, B) and applied
accordingly.
25
If the downmixing process is simplified such that the two meta-objects can be mixed with
unity weights for obtaining the downmix signal, i.e., 1 FGO BGO h  h  , and now the
rendering gains for these two objects are denoted with FGO g and BGO g . This simplifies the
equation for the loudness change into
30
33
 
   
   
/10 /10
/10 /10
/10 /10
/10 /10
2 2
10
2 2
10
10 10
, 10 log
10 10
10 10
10 log
10 10
LFGO c LBGO c
LFGO c LBGO c
LFGO LBGO
LFGO LBGO
FGO BGO
FGO BGO
g g
L A B
g g
 
 

 




.
Again, ΔL(A, B) may be considered as a loudness compensation value that is determined
by the signal processor 120.
5
In general, gFGO may be considered as a rendering gain for the foreground object FGO
(foreground object group), and gBGO may be considered as a rendering gain for the
background object BGO (background object group).
10 As mentioned earlier, it is possible to transmit loudness differences instead of absolute
loudness. Let us define the reference loudness as the loudness of the FGO meta-object
REF FGO L  L , i.e., 0 FGO FGO REF K  L  L  and BGO BGO REF BGO FGO K  L  L  L  L . Now, the
loudness change is
 
/10
/10
2 2
10
10
, 10 log
1 10
KBGO
KBGO
FGO BGO g g
L A B

 

15 .
It may also be, as the case in the SAOC-DE is, that two meta-objects do not have
individual scaling factors, but one of the objects is left un-modified, while the other is
attenuated to obtain the correct mixing ratio between the objects. In this rendering setting,
20 the output will be lower in loudness than the default mixture, and the change in the
loudness is
 
/10
/10
2 2
10
ˆ ˆ 10
, 10 log
1 10
KBGO
KBGO
FGO BGO g g
L A B

 

,
25 with
1 , if
ˆ
, if
FGO BGO
FGO FGO
FGO BGO
BGO
g g
g g
g g
g
 
    
, and
, if
ˆ
1 , if
BGO
BGO FGO
BGO FGO
BGO FGO
g
g g
g g
g g
   
  
.
This form is already rather simple, and is rather agnostic regarding the loudness measure
30 used. The only real requirement is, that the loudness values should sum in the
34
exponential domain. It is possible to transmit/store values of signal energies instead of
loudness values, as the two have close connection.
In each of the above formulae, ΔL(A, B) may be considered as a loudness compensation
value, that may be transmitted by the signal processor 120 5 of the decoder.
In the following, example cases are considered. The accuracy of the provided concepts is
illustrated through two example signals. Both signals have a 5.1 downmix with the
surround and LFE channels by-passed from the SAOC processing.
10
Two main approaches are used: one (“3-term”) with three meta-objects: FGO, BGO, and
by-pass channels, e.g.,
FGO BGO BYPASS X  X  X  X ,
15
And another one (“2-term”) with two meta-objects, e.g.:
FGO BGO X  X  X .
20 In the 2-term approach, the by-pass channels may, e.g., be mixed together with the BGO
for the meta-object loudness estimation. The loudness of both (or all three) objects as well
as the loudness of the downmix signal are estimated, and the values are stored.
The rendering instructions are of form
25
ˆ ˆ ˆ FGO FGO BGO BGO BGO BYPASS Y  g X  g X  g X
and
ˆ ˆ FGO FGO BGO BGO 30 Y  g X  g X
for the two approaches respectively.
35
The gain values are, e.g., determined according to:
1 , if 1
ˆ
, otherwise
FGO
FGO
FGO
g
g
g
 


, and
1 , if 1
ˆ
1 , otherwise
FGO
FGO
BGO
g g g
  

,
wherein the FGO gain FGO 5 g is varied between -24 to +24 dB.
The output scenario is rendered, the loudness is measured, and the attenuation from the
loudness of the downmix signal is calculated.
10 This result is displayed in Fig. 10 and Fig. 11 with the blue line with circle markers. Fig. 10
depicts a first illustration and Fig. 11 depicts a second illustration of a measured loudness
change and the result of using the provided concepts for estimating the change in the
loudness in a purely parametrical manner.
15 Next, the attenuation from the downmix is estimated parametrically employing the stored
meta-object loudness values and the downmixing and rendering information. The estimate
using the loudness of three meta-objects is illustrated with the green line with square
markers, and the estimate using the loudness of two meta-objects is illustrated with the
red line with star markers.
20
It can be seen from the figures, that the 2- and 3-term approaches provide practically
identical results, and they both approximate the measured value quite well.
The provided concepts exhibit a plurality of advantages. For example, the provided
25 concepts allow estimating the loudness of a mixture signal from the loudness of the
component signals forming the mixture. The benefit of this is that the component signal
loudness can be estimated once, and the loudness estimate of the mixture signal can be
obtained parametrically for any mixture without the need of actual signal-based loudness
estimation. This provides a considerable improvement in the computational efficiency of
30 the overall system in which the loudness estimate of various mixtures is needed. For
example, when the end-user changes the rendering settings, the loudness estimate of the
output is immediately available.
In some applications, such as when conforming with the EBU R128 recommendation, the
35 average loudness over the entire program is important. If the loudness estimation in the
receiver, e.g., in a broadcast scenario, is done based on the received signal, the estimate
36
converges to the average loudness only after the entire program has been received.
Because of this, any compensation of the loudness will have errors or exhibit temporal
variations. When estimating the loudness of the component objects as proposed and
transmitting the loudness information, it is possible to estimate the average mixture
loudness in the receiver 5 without a delay.
If it is desired that the average loudness of the output signal remains (approximately)
constant regardless of the changes in the rendering information, the provided concepts
allow determining a compensation factor for this reason. The calculations needed for this
10 in the decoder are from their computational complexity negligible, and the functionality is
thus possible to be added to any decoder.
There are cases in which the absolute loudness level of the output is not important, but
the importance lies in determining the change in the loudness from a reference scene. In
15 such cases the absolute levels of the objects are not important, but their relative levels
are. This allows defining one of the objects as the reference object and representing the
loudness of the other objects in relation to the loudness of this reference object. This has
some benefits considering the transport and/or storage of the loudness information.
20 First of all, it is not necessary to transport the reference loudness level. In the application
case of two meta-objects, this halves the amount of data to be transmitted. The second
benefit relates to the possible quantization and representation of the loudness values.
Since the absolute levels of the objects can be almost anything, the absolute loudness
values can also be almost anything. The relative loudness values, on the other hand, are
25 assumed to have a 0 mean and a rather nicely formed distribution around the mean. The
difference between the representations allows defining the quantization grid of the relative
representation in a way with potentially greater accuracy with the same number of bits
used for the quantized representation.
30 Fig. 12 illustrates another embodiment for conducting loudness compensation. In Fig. 12,
loudness compensation may be conducted, e.g., to compensate the loss in loudness, For
this purpose, e.g., the values DE_loudness_diff_dialogue (= KFGO) and
DE_loudness_diff_background (= KBGO) from DE_control_info may be used. Here,
DE_control_info may specify Advanced Clean Audio “Dialogue Enhancement” (DE)
35 control information
The loudness compensation is achieved by applying a gain value “g” on the SAOC-DE
output signal and the by-passed channels (in case of a multichannel signal).
37
In the embodiment of Fig. 12, this is done as follows:
A limited dialogue modification gain value mG is used to determine the effective gains for
the foreground object (FGO, e.g., dialogue) and for the background 5 object (BGO, e.g.,
ambiance). This is done by the “Gain mapping” block 1220 which produces the gain
values mFGO and mBGO .
The “Output loudness estimator” block 1230 uses the loudness information KFGO and
KBGO , and the effective gain values mFGO and mBGO 10 to estimate this possible change in
the loudness compared to the default downmix case. The change is then mapped into the
“Loudness compensation factor” which is applied on the output channels for producing the
final “Output signals”.
15 The following steps are applied for loudness compensation:
- Receive the limited gain value mG from the SAOC-DE decoder (as defined in
clause 12.8 “Modification range control for SAOC-DE” [DE]), and determine the
applied FGO/BGO gains:
mFGO  mG , and mBGO  1 if mG  1
mFGO  1, and mBGO  mG
1 if mG  1

 
 
20 .
- Obtain the meta-object loudness information KFGO and KBGO .
- Calculate the change in the output loudness compared to the default downmix with
25
2 10 2 10
10
10 10
10 10
10 log
10 10
FGO BGO
FGO BGO
K K
FGO BGO
K K
m m
L

 
 .
- Calculate the loudness compensation gain g  100.05L .
38
- Calculate the scaling factors g 
g1

gN
é
ë
êêêê
ù
û
úúúú
, wherein
if channel belongs to SAOC-DE output
if channel is a by-pass channel i
BGO
g i
g
m g i





,and N is the total
number of output channels. In Fig. 12, the gain adjustment is divided into two
steps: the gain of the possible “by-pass channels” is adjusted with BGO m prior to
combining them with the “SAOC-DE output channels”, and 5 then a common gain
g is then applied on all the combined channels. This is only a possible reordering
of the gain adjustment operations, while g here combines both gain
adjustment steps into one gain adjustment.
- Apply the scaling values g on the audio channels YFULL 10 consisting of the “SAOCDE
output channels” SAOC Y and the possible time-aligned “by-pass channels” BYPASS Y :
YFULL  YSAOC È YBYPASS
Applying the scaling values g on the audio channels YFULL is conducted by the gain
15 adjustment unit 1240.
ΔL as calculated above may be considered as a loudness compensation value. In
general, mFGO indicates a rendering gain for the foreground object FGO (foreground
object group), and mBGO indicates a rendering gain for the background object BGO
20 (background object group).
Although some aspects have been described in the context of an apparatus, it is clear that
these aspects also represent a description of the corresponding method, where a block or
device corresponds to a method step or a feature of a method step. Analogously, aspects
25 described in the context of a method step also represent a description of a corresponding
block or item or feature of a corresponding apparatus.
The inventive decomposed signal can be stored on a digital storage medium or can be
transmitted on a transmission medium such as a wireless transmission medium or a wired
30 transmission medium such as the Internet.
Depending on certain implementation requirements, embodiments of the invention can be
implemented in hardware or in software. The implementation can be performed using a
39
digital storage medium, for example a floppy disk, a DVD, a CD, a ROM, a PROM, an
EPROM, an EEPROM or a FLASH memory, having electronically readable control signals
stored thereon, which cooperate (or are capable of cooperating) with a programmable
computer system such that the respective method is performed.
5
Some embodiments according to the invention comprise a non-transitory data carrier
having electronically readable control signals, which are capable of cooperating with a
programmable computer system, such that one of the methods described herein is
performed.
10
Generally, embodiments of the present invention can be implemented as a computer
program product with a program code, the program code being operative for performing
one of the methods when the computer program product runs on a computer. The
program code may for example be stored on a machine readable carrier.
15
Other embodiments comprise the computer program for performing one of the methods
described herein, stored on a machine readable carrier.
In other words, an embodiment of the inventive method is, therefore, a computer program
20 having a program code for performing one of the methods described herein, when the
computer program runs on a computer.
A further embodiment of the inventive methods is, therefore, a data carrier (or a digital
storage medium, or a computer-readable medium) comprising, recorded thereon, the
25 computer program for performing one of the methods described herein.
A further embodiment of the inventive method is, therefore, a data stream or a sequence
of signals representing the computer program for performing one of the methods
described herein. The data stream or the sequence of signals may for example be
30 configured to be transferred via a data communication connection, for example via the
Internet.
A further embodiment comprises a processing means, for example a computer, or a
programmable logic device, configured to or adapted to perform one of the methods
35 described herein.
A further embodiment comprises a computer having installed thereon the computer
program for performing one of the methods described herein.
40
In some embodiments, a programmable logic device (for example a field programmable
gate array) may be used to perform some or all of the functionalities of the methods
described herein. In some embodiments, a field programmable gate array may cooperate
with a microprocessor in order to perform one of the methods described 5 herein. Generally,
the methods are preferably performed by any hardware apparatus.
An embodiment according to the present invention comprises decoder for generating an
audio output signal comprising one or more audio output channels, where the decoder
10 comprises a receiving interface (110) for receiving an audio input signal comprising a
plurality of audio object signals, for receiving loudness information on the audio object
signals, and for receiving rendering information indicating whether one or more of the
audio object signals shall be amplified or attenuated, and a signal processor (120) for
generating the one or more audio output channels of the audio output signal, the signal
15 processor (120) is configured to determine a loudness compensation value depending on
the loudness information and depending on the rendering information, and the signal
processor (120) is configured to generate the one or more audio output channels of the
audio output signal from the audio input signal depending on the rendering information
and depending on the loudness compensation value.
20
A further embodiment according to the present invention comprises decoder, where the
signal processor (120) is configured to generate the one or more audio output channels of
the audio output signal from the audio input signal depending on the rendering information
and depending on the loudness compensation value, such that a loudness of the audio
25 output signal is equal to a loudness of the audio input signal, or such that the loudness of
the audio output signal is closer to the loudness of the audio input signal than a loudness
of a modified audio signal that would result from modifying the audio input signal by
amplifying or attenuating the audio object signals of the audio input signal according to the
rendering information.
30
A further embodiment according to the present invention comprises a decoder, where the
signal processor (120) is configured to generate the modified audio signal by modifying
the audio input signal by amplifying or attenuating the audio object signals of the audio
input signal according to the rendering information, and the signal processor (120) is
35 configured to generate the audio output signal by applying the loudness compensation
value on the modified audio signal, such that the loudness of the audio output signal is
equal to the loudness of the audio input signal, or such that the loudness of the audio
41
output signal is closer to the loudness of the audio input signal than the loudness of the
modified audio signal.
A further embodiment according to the present invention comprises a decoder, where
each of the audio object signals of the audio input signal is assigned to 5 exactly one group
of two or more groups, wherein each of the two or more groups comprises one or more of
the audio object signals of the audio input signal, the receiving interface (110) is
configured to receive a loudness value for each group of the two or more groups as the
loudness information, the signal processor (120) is configured to determine the loudness
10 compensation value depending on the loudness value of each of the two or more groups,
and the signal processor (120) is configured to generate the one or more audio output
channels of the audio output signal from the audio input signal depending on the loudness
compensation value.
15 A further embodiment according to the present invention comprises a decoder, where at
least one group of the two or more groups comprises two or more of the audio object
signals.
A further embodiment according to the present invention comprises a decoder where each
20 of the audio object signals of the audio input signal is assigned to exactly one group of
more than two groups, each of the more than two groups comprises one or more of the
audio object signals of the audio input signal, the receiving interface (110) is configured to
receive a loudness value for each group of the more than two groups as the loudness
information, the signal processor (120) is configured to determine the loudness
25 compensation value depending on the loudness value of each of the more than two
groups, and the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending on the
loudness compensation value.
30 A further embodiment according to the present invention comprises a decoder where at
least one group of the more than two groups comprises two or more of the audio object
signals.
A further embodiment according to the present invention comprises a decoder, where the
35 signal processor (120) is configured to determine the loudness compensation value
according to the formula
42
L A B 
 
 
/10
/10
2
1
10
2
1
10
, 10 log
10
Li c
Li c
N
i
i
N
i
i
g
L A B
h




 


or according to the formula
L A,BL 
A10Blog
/10
/10
2
1
10
2
1
10
10 log
10
Li
Li
N
i
i
N
i
i
g
h




5
ΔL is the loudness compensation value,
i indicates an i-th audio object signal of the audio object signals,
10
Li is a loudness of the i-th audio object signal,
gi is a first mixing weight for the i-th audio object signal,
15 hi is a second mixing weight for the i-th audio object signal,
c is a constant value, and
N is a number.
20
A further embodiment according to the present invention comprises a decoder, where the
signal processor (120) is configured to determine the loudness compensation value
according to the formula
25
L A B 
/10
/10
2
1
10
2
1
10
, 10log
10
Ki
Ki
N
i
i
N
i
i
g
L A B
h


 


43
ΔL is the loudness compensation value,
i indicates an i-th audio object signal of the audio object signals,
5
gi is a first mixing weight for the i-th audio object signal,
hi is a second mixing weight for the i-th audio object signal,
10 N is a number, and
Ki is defined according to
i i REF K  L  L ,
15
Li is a loudness of the i-th audio object signal, and
REF L is the loudness of a reference object.
20 A further embodiment according to the present invention comprises a decoder, where
each of the audio object signals of the audio input signal is assigned to exactly one group
of exactly two groups as the two or more groups, where each of the audio object signals
of the audio input signal is either assigned to a foreground object group of the exactly two
groups or to a background object group of the exactly two groups, the receiving interface
25 (110) is configured to receive the loudness value of the foreground object group, the
receiving interface (110) is configured to receive the loudness value of the background
object group, the signal processor (120) is configured to determine the loudness
compensation value depending on the loudness value of the foreground object group, and
depending on the loudness value of the background object group, and the signal
30 processor (120) is configured to generate the one or more audio output channels of the
audio output signal from the audio input signal depending on the loudness compensation
value.
A further embodiment according to the present invention comprises a decoder where, the
35 signal processor (120) is configured to determine the loudness compensation value
according to the formula
44
2 10 2 10
10
10 10
10 10
10 log
10 10
FGO BGO
FGO BGO
K K
FGO BGO
K K
m m
L

 

ΔL is the loudness compensation value,
5 KFGO indicates the loudness value of the foreground object group,
KBGO indicates the loudness value of the background object group,
mFGO indicates a rendering gain of the foreground object group, and
10
mBGO indicates a rendering gain of the background object group.
A further embodiment according to the present invention comprises a decoder, where the
15 signal processor (120) is configured to determine the loudness compensation value
according to the formula
L A,B 10log
/10 /10
/10 /10
2 2
10
10 10
10 log
10 10
LFGO LBGO
LFGO LBGO
FGO BGO g  g


20 ΔL is the loudness compensation value,
LFGO indicates the loudness value of the foreground object group,
LBGO indicates the loudness value of the background object group,
25
gFGO indicates a rendering gain of the foreground object group, and
gBGO indicates a rendering gain of the background object group.
30 A further embodiment according to the present invention comprises a decoder, where the
receiving interface (110) is configured to receive a downmix signal comprising one or
more downmix channels as the audio input signal, wherein the one or more downmix
channels comprise the audio object signals, and wherein the number of the one or more
downmix channels is smaller than the number of the audio object signals, where the
45
receiving interface (110) is configured to receive downmix information indicating how the
audio object signals are mixed within the one or more downmix channels, and where the
signal processor (120) is configured to generate the one or more audio output channels of
the audio output signal from the audio input signal depending on the downmix information,
depending on the rendering information and depending on the loudness 5 compensation
value.
A further embodiment according to the present invention comprises a decoder, where the
receiving interface (110) is configured to receive one or more further by-pass audio object
10 signals, wherein the one or more further by-pass audio object signals are not mixed within
the downmix signal, the receiving interface (110) is configured to receive the loudness
information indicating information on the loudness of the audio object signals which are
mixed within the downmix signal and indicating information on the loudness of the one or
more further by-pass audio object signals which are not mixed within the downmix signal,
15 and the signal processor (120) is configured to determine the loudness compensation
value depending on the information on the loudness of the audio object signals which are
mixed within the downmix signal, and depending on the information on the loudness of the
one or more further by-pass audio object signals which are not mixed within the downmix
signal.
20
Another embodiment according to the present invention comprises, an encoder,
comprising: an object-based encoding unit (210; 710) for encoding a plurality of audio
object signals to obtain an encoded audio signal comprising the plurality of audio object
signals, and an object loudness encoding unit (220; 720; 820) for encoding loudness
25 information on the audio object signals, the loudness information comprises one or more
loudness values, each of the one or more loudness values depends on one or more of the
audio object signals.
A further embodiment according to the present invention comprises an encoder, where
30 each of the audio object signals of the encoded audio signal is assigned to exactly one
group of two or more groups, each of the two or more groups comprises one or more of
the audio object signals of the encoded audio signal, where the object loudness encoding
unit (220; 720; 820) is configured to determine the one or more loudness values of the
loudness information by determining a loudness value for each group of the two or more
35 groups, wherein said loudness value of said group indicates an total loudness of the one
or more audio object signals of said group.
46
A further embodiment according to the present invention comprises encoder, where the
object-based encoding unit (210; 710) is configured to receive the audio object signals,
where each of the audio object signals is assigned to exactly one of exactly two groups,
where each of the exactly two groups comprises one or more of the audio object signals,
where the object-based encoding unit (210; 710) is configured to 5 downmix the audio
object signals, being comprised by the exactly two groups, to obtain a downmix signal
comprising one or more downmix audio channels as the encoded audio signal, where the
number of the one or more downmix channels is smaller than the number of the audio
object signals being comprised by the exactly two groups, the object loudness encoding
10 unit (220; 720; 820) is assigned to receive one or more further by-pass audio object
signals, wherein each of the one or more further by-pass audio object signals is assigned
to a third group, where each of the one or more further by-pass audio object signals is not
comprised by the first group and is not comprised by the second group, where the objectbased
encoding unit (210; 710) is configured to not downmix the one or more further by15
pass audio object signals within the downmix signal, and the object loudness encoding
unit (220; 720; 820) is configured to determine a first loudness value, a second loudness
value and a third loudness value of the loudness information, the first loudness value
indicating a total loudness of the one or more audio object signals of the first group, the
second loudness value indicating a total loudness of the one or more audio object signals
20 of the second group, and the third loudness value indicating a total loudness of the one or
more further by-pass audio object signals of the third group, or is configured to determine
a first loudness value and a second loudness value of the loudness information, the first
loudness value indicating a total loudness of the one or more audio object signals of the
first group, and the second loudness value indicating a total loudness of the one or more
25 audio object signals of the second group and of the one or more further by-pass audio
object signals of the third group.
Another embodiment of the present invention a system comprising an encoder (310) for
encoding a plurality of audio object signals to obtain an encoded audio signal comprising
30 the plurality of audio object signals, and a decoder (320), for generating an audio output
signal comprising one or more audio output channels, where the decoder (320) is
configured to receive the encoded audio signal as an audio input signal and to receive the
loudness information, where the decoder (320) is configured to further receive rendering
information, where the decoder (320) is configured to determine a loudness compensation
35 value depending on the loudness information and depending on the rendering information,
and where the decoder (320) is configured to generate the one or more audio output
channels of the audio output signal from the audio input signal depending on the
rendering information and depending on the loudness compensation value.
47
Another embodiment of the present invention comprises a method for generating an audio
output signal comprising one or more audio output channels, where the method
comprises: receiving an audio input signal comprising a plurality of audio object signals,
receiving loudness information on the audio object signals, receiving rendering 5 information
indicating whether one or more of the audio object signals shall be amplified or
attenuated, determining a loudness compensation value depending on the loudness
information and depending on the rendering information, and generating the one or more
audio output channels of the audio output signal from the audio input signal depending on
10 the rendering information and depending on the loudness compensation value.
Another embodiment of the present invention comprises a method for encoding,
comprising encoding an audio input signal comprising a plurality of audio object signals,
and encoding loudness information on the audio object signals, where the loudness
15 information comprises one or more loudness values, where each of the one or more
loudness values depends on one or more of the audio object signals.
Another embodiment of the present invention comprises a computer program for
implementing the method when being executed on a computer or signal processor.
20
The above described embodiments are merely illustrative for the principles of the present
invention. It is understood that modifications and variations of the arrangements and the
details described herein will be apparent to others skilled in the art. It is the intent,
25 therefore, to be limited only by the scope of the impending patent claims and not by the
specific details presented by way of description and explanation of the embodiments
herein.
48
References
[BCC] C. Faller and F. Baumgarte, “Binaural Cue Coding - Part II: Schemes and
applications,” IEEE Trans. on Speech and Audio Proc., vol. 11, no. 6, Nov. 2003.
5
[EBU] EBU Recommendation R 128 “Loudness normalization and permitted maximum
level of audio signals”, Geneva, 2011.
[JSC] C. Faller, “Parametric Joint-Coding of Audio Sources”, 120th AES Convention,
10 Paris, 2006.
[ISS1] M. Parvaix and L. Girin: “Informed Source Separation of underdetermined
instantaneous Stereo Mixtures using Source Index Embedding”, IEEE ICASSP,
2010.
15
[ISS2] M. Parvaix, L. Girin, J.-M. Brossier: “A watermarking-based method for informed
source separation of audio signals with a single sensor”, IEEE Transactions on
Audio, Speech and Language Processing, 2010.
20 [ISS3] A. Liutkus and J. Pinel and R. Badeau and L. Girin and G. Richard: “Informed
source separation through spectrogram coding and data embedding”, Signal
Processing Journal, 2011.
[ISS4] A. Ozerov, A. Liutkus, R. Badeau, G. Richard: “Informed source separation:
25 source coding meets source separation”, IEEE Workshop on Applications of
Signal Processing to Audio and Acoustics, 2011.
[ISS5] S. Zhang and L. Girin: “An Informed Source Separation System for Speech
Signals”, INTERSPEECH, 2011.
30
[ISS6] L. Girin and J. Pinel: “Informed Audio Source Separation from Compressed
Linear Stereo Mixtures”, AES 42nd International Conference: Semantic Audio,
2011.
35 [ITU] International Telecommunication Union: “Recommendation ITU-R BS.1770-3 –
Algorithms to measure audio programme loudness and true-peak audio level”,
Geneva, 2012.
49
[SAOC1] J. Herre, S. Disch, J. Hilpert, O. Hellmuth: “From SAC To SAOC - Recent
Developments in Parametric Coding of Spatial Audio”, 22nd Regional UK AES
Conference, Cambridge, UK, April 2007.
[SAOC2] J. Engdegård, B. Resch, C. Falch, O. Hellmuth, J. Hilpert, 5 A. Hölzer, L.
Terentiev, J. Breebaart, J. Koppens, E. Schuijers and W. Oomen: “Spatial Audio
Object Coding (SAOC) – The Upcoming MPEG Standard on Parametric Object
Based Audio Coding”, 124th AES Convention, Amsterdam 2008.
10 [SAOC] ISO/IEC, “MPEG audio technologies – Part 2: Spatial Audio Object Coding
(SAOC),” ISO/IEC JTC1/SC29/WG11 (MPEG) International Standard 23003-2.
[EP] EP 2146522 A1: S. Schreiner, W. Fiesel, M. Neusinger, O. Hellmuth, R.
Sperschneider, ”Apparatus and method for generating audio output signals
15 using object based metadata“, 2010.
[DE] ISO/IEC, “MPEG audio technologies – Part 2: Spatial Audio Object Coding
(SAOC) – Amendment 3, Dialogue Enhancement,” ISO/IEC 23003-2:2010/DAM
3, Dialogue Enhancement.
20
[BRE] WO 2008/035275 A2.
[SCH] EP 2 146 522 A1.
25 [ENG] WO 2008/046531 A1.
50
We Claim:
1. A decoder for generating an audio output signal comprising one or more audio
output channels, wherein the decoder comprises:
5
a receiving interface (110) for receiving an audio input signal comprising at least
one audio object signal, for receiving loudness information on the at least one
audio object signal, and for receiving rendering information indicating how one or
more of the at least one audio object signal shall be amplified or attenuated, and
10
a signal processor (120) for generating the one or more audio output channels of
the audio output signal,
wherein the signal processor (120) is configured to determine a loudness
15 compensation value depending on the loudness information, and
wherein the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending
on the rendering information and depending on the loudness compensation value,
20
wherein the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending
on the rendering information and depending on the loudness compensation value,
such that a loudness of the audio output signal is equal to a loudness of the audio
25 input signal, or such that the loudness of the audio output signal is closer to the
loudness of the audio input signal than a loudness of a modified audio signal that
would result from modifying the audio input signal by amplifying or attenuating the
at least one audio object signal of the audio input signal according to the rendering
information.
30
2. A decoder as claimed in claim 1,
wherein the signal processor (120) is configured to generate the modified audio
signal by modifying the audio input signal by amplifying or attenuating the at least
35 one audio object signal of the audio input signal according to the rendering
information, and
51
wherein the signal processor (120) is configured to generate the audio output
signal by applying the loudness compensation value on the modified audio signal,
such that the loudness of the audio output signal is equal to the loudness of the
audio input signal, or such that the loudness of the audio output signal is closer to
the loudness of the audio input signal than the loudness of 5 the modified audio
signal.
3. A decoder as claimed in claim 1 or 2,
10 wherein each of the at least one audio object signal of the audio input signal is
assigned to exactly one group of two or more groups, wherein each of the two or
more groups comprises one or more of the at least one audio object signal of the
audio input signal,
15 wherein the receiving interface (110) is configured to receive a loudness value for
each group of the two or more groups as the loudness information,
wherein the signal processor (120) is configured to determine the loudness
compensation value depending on the loudness value of each of the two or more
20 groups, and
wherein the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending
on the loudness compensation value.
25
4. A decoder as claimed in claim 3, wherein at least one group of the two or more
groups comprises two or more of the at least one audio object signal.
5. A decoder as claimed in claim 1 or 2,
30
wherein each of the at least one audio object signal of the audio input signal is
assigned to exactly one group of more than two groups, wherein each of the more
than two groups comprises one or more of the at least one audio object signal of
the audio input signal,
35
wherein the receiving interface (110) is configured to receive a loudness value for
each group of the more than two groups as the loudness information,
52
wherein the signal processor (120) is configured to determine the loudness
compensation value depending on the loudness value of each of the more than
two groups, and
5
wherein the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending
on the loudness compensation value.
10 6. A decoder as claimed in claim 5, wherein at least one group of the more than two
groups comprises two or more of the at least one audio object signal.
7. A decoder as claimed in one of claims 3 to 6,
15 wherein the signal processor (120) is configured to determine the loudness
compensation value according to the formula
L A B 
 
 
/10
/10
2
1
10
2
1
10
, 10 log
10
Li c
Li c
N
i
i
N
i
i
g
L A B
h




 


20
or according to the formula
L A,BL A10Blog
/10
/10
2
1
10
2
1
10
10 log
10
Li
Li
N
i
i
N
i
i
g
h





25 wherein ΔL is the loudness compensation value,
wherein i indicates an i-th audio object signal of the at least one audio object
signal,
30 wherein Li is a loudness of the i-th audio object signal,
53
wherein gi is a first mixing weight for the i-th audio object signal,
wherein hi is a second mixing weight for the i-th audio object signal,
5
wherein c is a constant value, and
wherein N is a number.
10 8. A decoder as claimed in one of claims 3 to 6,
wherein the signal processor (120) is configured to determine the loudness
compensation value according to the formula
L A B 
/10
/10
2
1
10
2
1
10
, 10 log
10
Ki
Ki
N
i
i
N
i
i
g
L A B
h


 

 15
wherein ΔL is the loudness compensation value,
wherein i indicates an i-th audio object signal of the at least one audio object
20 signal,
wherein gi is a first mixing weight for the i-th audio object signal,
wherein hi is a second mixing weight for the i-th audio object signal,
25
wherein N is a number, and
wherein Ki is defined according to
i i REF 30 K  L  L ,
wherein Li is a loudness of the i-th audio object signal, and
wherein REF L is the loudness of a reference object.
54
9. A decoder as claimed in claim 3 or 4,
wherein each of the at least one audio object signal of the audio input signal is
assigned to exactly one group of exactly two groups as the two 5 or more groups,
wherein each of the at least one audio object signal of the audio input signal is
either assigned to a foreground object group of the exactly two groups or to a
background object group of the exactly two groups,
10
wherein the receiving interface (110) is configured to receive the loudness value of
the foreground object group,
wherein the receiving interface (110) is configured to receive the loudness value of
15 the background object group,
wherein the signal processor (120) is configured to determine the loudness
compensation value depending on the loudness value of the foreground object
group, and depending on the loudness value of the background object group, and
20
wherein the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending
on the loudness compensation value.
25 10. A decoder as claimed in claim 9,
wherein the signal processor (120) is configured to determine the loudness
compensation value according to the formula
2 10 2 10
10
10 10
10 10
10 log
10 10
FGO BGO
FGO BGO
K K
FGO BGO
K K
m m
L

 
30 
wherein ΔL is the loudness compensation value,
wherein KFGO indicates the loudness value of the foreground object group,
35
wherein KBGO indicates the loudness value of the background object group,
55
wherein mFGO indicates a rendering gain of the foreground object group, and
wherein mBGO indicates a rendering gain of the background object group.
5
11. A decoder as claimed in claim 9,
wherein the signal processor (120) is configured to determine the loudness
compensation value according to the formula
10
L A, B 10log
/10 /10
/10 /10
2 2
10
10 10
10 log
10 10
LFGO LBGO
LFGO LBGO
FGO BGO g  g


wherein ΔL is the loudness compensation value,
15 wherein LFGO indicates the loudness value of the foreground object group,
wherein LBGO indicates the loudness value of the background object group,
wherein gFGO indicates a rendering gain of the foreground object group, and
20
wherein gBGO indicates a rendering gain of the background object group.
12. A decoder as claimed in one of the preceding claims,
25 wherein the receiving interface (110) is configured to receive a downmix signal
comprising one or more downmix channels as the audio input signal, wherein the
one or more downmix channels comprise the at least one audio object signal, and
wherein the number of the one or more downmix channels is smaller than the
number of the at least one audio object signal,
30
wherein the receiving interface (110) is configured to receive downmix information
indicating how the at least one audio object signal is mixed within the one or more
downmix channels, and
35 wherein the signal processor (120) is configured to generate the one or more audio
output channels of the audio output signal from the audio input signal depending
56
on the downmix information, depending on the rendering information and
depending on the loudness compensation value.
13. A decoder as claimed in claim 12,
wherein the receiving interface (110) is configured to receive one or more further
by-pass audio object signals, wherein the one or more further by-pass audio object
signals are not mixed within the downmix signal,
wherein the receiving interface (110) is configured to receive the loudness
information indicating information on the loudness of the at least one audio object
signal which is mixed within the downmix signal and indicating information on the
loudness of the one or more further by-pass audio object signals which are not
mixed within the downmix signal, and
wherein the signal processor (120) is configured to determine the loudness
compensation value depending on the information on the loudness of the at least
one audio object signal which is mixed within the downmix signal, and depending
on the information on the loudness of the one or more further by-pass audio object
signals which are not mixed within the downmix signal.
14. A method for generating an audio output signal comprising one or more audio
output channels, wherein the method comprises:
receiving an audio input signal comprising at least one audio object signal,
receiving loudness information on the at least one audio object signal,
receiving rendering information indicating how one or more of the at least one
audio object signal shall be amplified or attenuated,
determining a loudness compensation value depending on the loudness
information, and
generating the one or more audio output channels of the audio output signal from
the audio input signal depending on the rendering information and depending on
the loudness compensation value,
wherein generating the one or more audio output channels of the audio output
signal from the audio input signal is conducted depending on the rendering
information and depending on the loudness compensation value, such that a
loudness of the audio output signal is equal to a loudness of the audio 5 input signal,
or such that the loudness of the audio output signal is closer to the loudness of the
audio input signal than a loudness of a modified audio signal that would result from
modifying the audio input signal by amplifying or attenuating the at least one audio
object signal of the audio input signal according to the rendering information.