0001]This technique, the signal processing apparatus and method, and a program, the signal processing apparatus and method capable of reducing the computational load, and a program.
BACKGROUND
[0002]Conventional, object audio technology in movies and games and the like are used, have also been developed coding system that can handle the object audio. Specifically, for example, international standards and is MPEG (Moving Picture Experts Group) -H Part 3: it is such 3D audio standards are known (e.g., see Non-Patent Document 1).
[0003]
　In such a coding system, together with a multi-channel stereo system such as a conventional two-channel stereo system or 5.1 channel is treated as audio objects independent moving sound source or the like, the position information of the object with the signal data of audio objects It can be encoded as metadata.
[0004]
　In this way, in the conventional coding method and volume adjustment of a particular sound source of the sound has been difficult, and add effects with respect to sounds of a particular sound source, it is processed during reproduction of the sound of specific sound source easily it can be made.
[0005]
　Also, in the encoding method described in Non-Patent Document 1, in addition to the above audio objects, also handle data such Ambisonic dealing with spatial audio information around the viewer (HOA (also referred to as High Order Ambisonic)) can.
[0006]
　Meanwhile, audio objects, when rendering the loudspeaker signal or headphones signal, etc., because it is assumed to be a point source, can not represent audio objects having a size.
[0007]
　Therefore, in the coding method that can handle the object audio such as encoding scheme described in Non-Patent Document 1, information called spread representing the size of the object in the metadata of the audio object is stored.
[0008]
　Then, for example, in Non-Patent Document 1 standard, it is rendered output to the reproduction device such as a speaker signal 19 spread audio object for one audio object is newly generated based on the spread at the time of reproduction. Thus, it is possible to represent the audio object with pseudo size.
CITATION
Non-patent literature
[0009]
非特許文献1 : INTERNATIONAL STANDARD ISO/IEC 23008-3 First edition 2015-10-15 Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio
Summary of the Invention
Problems that the Invention is to Solve
[0010]
　However, it newly generates a signal 19 also spreads audio object for one audio object as described above leads to significantly increasing the computational load of the rendering process.
[0011]
　This technology has been made in view of such circumstances, it is desirable to make it possible to reduce the computational load.
Means for Solving the Problems
[0012]
　Signal processing apparatus according to an embodiment of the present technology, based on the spread information of the object, comprising the Ambisonic gain calculator which the object seeks Ambisonic gain when in the predetermined position.
[0013]
　The signal processing unit, and the audio object signal of the object, on the basis on the Ambisonic gain, the Ambisonic signal generator for generating an Ambisonic signal of the object may be further provided.
[0014]
　Wherein the Ambisonic gain calculator, on the basis of the spread information, said object is let obtains a reference position Ambisonic gain when to be in a reference position, based on the object position information indicating the predetermined position , to perform the rotation process on the reference position Ambisonic gain can give prompted the Ambisonic gain.
[0015]
　The said Ambisonic gain calculator can give seeking the reference position Ambisonic gain based on said spread information and gain table.
[0016]
　The gain table can be a spread angle, shall said reference position Ambisonic gain associated.
[0017]
　The said Ambisonic gain calculator, by causing the interpolation processing based on each of the reference position Ambisonic gain associated with each of the plurality of spread angle in the gain table, indicated by the spread information it can give seeking the reference position Ambisonic gain corresponding to the spread angle.
[0018]
　Said reference position Ambisonic gain, respectively of the sum of the values ​​obtained respectively by substituting the spherical harmonic function of the angle indicating each of a plurality of positions in space defined relative to the spread angle shown by spread information can do.
[0019]
　Signal processing method or a program according to an embodiment of the present technology, based on the spread information of an object, comprising the steps of obtaining a Ambisonic gain when the object is in place.
[0020]
　In one aspect of the present technology, based on the spread information of the object, the object is required Ambisonic gain when in the predetermined position.
The invention's effect
[0021]
　According to one aspect of the present technology, it is possible to reduce the computational load.
[0022]
　Here, the advantages described in the present invention is not necessarily limited, it may be any of the effects described in the present disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0023]
FIG. 1 is a diagram illustrating a metadata of the audio objects.
FIG. 2 is a diagram illustrating a position on a 3-dimensional space of the audio objects.
3 is a diagram for explaining the spread audio object.
4 is a diagram illustrating the spread audio object.
5 is a diagram for explaining the spread audio object.
It is a diagram illustrating a configuration example of FIG. 6 signal processor.
7 is a diagram showing the relationship between the spread angle and the front position Ambisonic gain.
It is a flowchart illustrating a FIG. 8 content rendering process.
9 is a diagram describing metadata of the audio objects.
Is a diagram illustrating FIG. 10 audio object spreads.
11 is a diagram illustrating a spread audio object.
12 is a diagram showing the relationship between the spread angle and the front position Ambisonic gain.
13 is a diagram showing the relationship between the spread angle and the front position Ambisonic gain.
14 is a diagram showing a configuration example of the decoder.
15 is a diagram illustrating a configuration example of the decoder.
16 is a diagram showing a configuration example of an encoder.
It is a diagram illustrating a configuration example of FIG. 17 computer.
DESCRIPTION OF THE INVENTION
[0024]
　Hereinafter, with reference to the accompanying drawings, a description will be given of an embodiment according to the present technology.
[0025]

　This technique is calculated by directly determined the Ambisonic gain based on spread information, to obtain the Ambisonic gain obtained, the Ambisonic signal from the audio object signal it is to be able to reduce the load.
[0026]
　First, MPEG-H Part 3: 3D audio standard for audio objects spread (hereinafter, also referred to as spread information) will be described.
[0027]
　Figure 1 is diagram showing an example of the format of the metadata of the audio object containing spread information.
[0028]
　Metadata of the audio objects, the format shown in FIG. 1 for every predetermined time interval is encoded by use.
[0029]
　In Figure 1, num_objects indicates the number of audio objects included in the bit stream. In addition, tcimsbf the Two's complement integer, it is an abbreviation of the most significant bit first, uimsbf is an abbreviation of Unsigned integer, most significant bit first.
[0030]
　In this example, Object_priority each audio object in the meta data, spread, position_azimuth, position_elevation, position_radius, and gain_factor are stored.
[0031]
　object_priority is priority information indicating the priority at the time of rendering the audio objects to the playback device such as a speaker. For example, in the case in computing resources less equipment like for reproducing audio data, it is possible such as to reproduce the signal of object_priority large audio object preferentially.
[0032]
　spread is metadata that represents the magnitude of the audio object (spread information), MPEG-H Part 3: The 3D audio standard is defined as the angle indicating the extent of the spatial position of the audio object. gain_factor is a gain information indicating an individual gain audio objects.
[0033]
　position_azimuth, position_elevation, and position_radius are azimuth representing the spatial positional information of the audio objects are elevation, and radius (distance), these azimuth, elevation, and radius relationships, for example, as shown in FIG. 2 .
[0034]
　That is, through the origin O in FIG. 2, has mutually perpendicular x-axis, y-axis, and z-axis with the axis of the three-dimensional orthogonal coordinate system.
[0035]
　Now, a straight line connecting the position of the audio object OB11 on the origin O and the space is a straight line r, is a straight line obtained by projecting the straight line r on the xy plane and the straight line L.
[0036]
　In this case, the azimuth angle indicating the angle position of the audio object OB11 the x-axis and the straight line L, that is a Position_azimuth, elevation showing the angle position of the audio object OB11 between the straight line r and the xy plane, i.e. the position_elevation It is. The length of the straight line r is a radius indicating the position of the audio object OB11, i.e. are Position_radius.
[0037]
　Referring back to FIG. 1, Object_priority the decoding side are shown in FIG. 1, spread, position_azimuth, position_elevation, position_radius, and gain_factor is read appropriately used.
[0038]
　Then, MPEG-H Part 3: In 3D audio standard, illustrating the audio object with a spread (spread information) about how to render playback device such as a speaker.
[0039]
　For example spread is not, in the case where the angle that is indicated by the spread rendering the normal audio object is 0 degrees, a technique called VBAP (Vector Base Amplitude Panning) is used.
[0040]
　It is to be noted that the VBAP, for example, "INTERNATIONAL STANDARD ISO / IEC 23008-3 First edition 2015-10-15 Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3: 3D audio" because they are described in, etc., and a description thereof will be omitted.
[0041]
　In contrast, when there is a spread of audio objects, the vector p indicates the position of the 19 pieces of spread audio object based on the spread 0 to vector p 18 is obtained.
[0042]
　That is, the vector indicating the position indicated by the metadata of the audio objects first be processed, vector p underlying 0 to. Further, the angle indicated by each of position_azimuth and position_elevation audio objects to be processed and the angle φ and the angle theta. In this case, the basic vector v and basic vector u is determined by the following formula (1) and (2).
[0043]
[Number 1]

[0044]
[Number 2]

[0045]
　Note "×" in equation (2) shows the cross-product.
[0046]
　Then, with these two basic vectors v and base vectors u, the vector p 0 on the basis of the eighteen vector p by the following equation (3) 1 'to the vector p 18 ' is obtained.
[0047]
[Number 3]

[0048]
　Equation (3) 18 vector p obtained by 1 'to the vector p 18 ', and the vector p 0 is plotted on the three-dimensional rectangular coordinate system the position indicated by each, as shown in FIG. In FIG. 3, it represents the position where one circle is indicated by a vector.
[0049]
　Here, the angle indicated by the spread of the audio object and alpha, When those limit this angle alpha below 90 degrees 0.001 degrees alpha ', 19 pieces of vector p deformed by spread m (where, m = 0,1, ..., 18) is as shown in the following equation (4).
[0050]
[Formula 4]

[0051]
　Thus the vector p obtained m a to normalize, spread 19 amino spread audio object corresponding to the (spread information) is generated. Here, one audio object for spread, one vector p m is a virtual object at the position on the space shown by.
[0052]
　The signals of these 19 pieces of spread audio object by rendering the reproducing device such as a speaker, it is possible to output sound of one audio object with a spatial extent corresponding to the spread.
[0053]
　Figure 4 is a diagram showing a plot of the 19 pieces of audio objects spreads when the angle is 30 degrees indicated by the spread in three-dimensional orthogonal coordinate system. Further, FIG. 5 is a diagram showing what angle indicated by the spread was plotted 19 audio objects spreads when it is 90 degrees in the three-dimensional orthogonal coordinate system.
[0054]
　In these FIGS. 4 and 5, represents the position where one circle is indicated by a vector. That is, one circle represents one audio object spreads.
[0055]
　When reproducing the signal of the audio object and to reproduce the audio signal composed of signals of the 19 pieces of spread audio object as a signal of one of the audio objects, an audio object having a size is expressed.
[0056]
　Furthermore, when the angle indicated by the spread is more than 90 degrees, the λ shown in the following equation (5) is a proration ratio, and rendering the results obtained when the angle was 90 degrees as indicated by the spread, the total speaker constant gain an output result when it is a is output after being synthesized by the apportioning ratio lambda.
[0057]
[Formula 5]

[0058]
　As described above, when a signal is reproduced audio objects is generated 19 spreads audio object based on the spread (spread information), audio object with pseudo magnitude is expressed.
[0059]
　However, generating a 19 also audio objects spreads for one audio object leads to significantly increasing the computational load of the rendering process.
[0060]
　Therefore, in this technology, the audio object with spread information, nineteen without audio object generated spreads during rendering, by obtaining the Ambisonic gain based on spread information directly, and to reduce the computational load .
[0061]
　In particular, the present technology and to render decoding the bitstream two methods object audio and Ambisonic is superimposed (decoded), when coding is converted into Ambisonic the sometimes-object audio coding (encoding) it is useful for such.
[0062]

　FIG. 6 is a diagram illustrating a configuration example of an embodiment of a signal processing apparatus according to the present technology.
[0063]
　The signal processing apparatus 11 shown in FIG. 6 includes a Ambisonic gain calculator 21, Ambisonic rotation unit 22, Ambisonic matrix applying unit 23, adding unit 24 and the Ambisonic rendering unit 25,.
[0064]
　The signal processing unit 11, as an audio signal for reproducing the sound of content, and input Ambisonic signal is an audio signal Ambisonic format, an input audio object signal and an audio signal of sound of the audio object supply It is.
[0065]
　For example, the input Ambisonic signal, spherical harmonics S n, m (theta, phi) Ambisonic channel C corresponds to the order n and degree m of n, m are signals. That is, the signal processing device 11, the Ambisonic channel C n, m input Ambisonic signal is supplied.
[0066]
　In contrast, an input audio object signal is a monaural audio signal to reproduce the sound of one audio object, the signal processing device 11 the input audio object signal for each audio object is supplied.
[0067]
　Further, the signal processing unit 11, for each audio object, and the object position information and the spread information supplied as metadata.
[0068]
　Here, the object position information, Position_azimuth described above, the information consisting of Position_elevation, and Position_radius.
[0069]
　position_azimuth shows the azimuth angle indicating the spatial position of the audio object, Position_elevation shows an elevation showing the spatial position of the audio object, Position_radius not exhibit a radius indicating the spatial position of the audio object there.
[0070]
　Moreover, the spread information is spread as described above, the size of the audio object, that is, the angle information indicating the spreading degree of a sound image of the audio objects.
[0071]
　In the following for simplicity of explanation, one audio object to the signal processor 11, the case where the input audio object signal, object position information, and spreads the information supplied.
[0072]
　However, not limited thereto, the plurality of audio objects into the signal processing unit 11, an input audio object signal, object position information, and spread information may of course be supplied.
[0073]
　Ambisonic gain calculator 21, based on the supplied spread information, obtains the Ambisonic gain when the audio object has to be in front position, and supplies the Ambisonic rotating portion 22.
[0074]
　Note that the front position is a position in the front direction when viewed from the user position as a reference in the space, a position position_azimuth and position_elevation respectively become 0 degree as the object position information. In other words, the position_azimuth = 0 and position_elevation = 0 position is a front position.
[0075]
　In the following, in particular audio object in the case where the front position, Ambisonic channel C of audio objects n, m the Ambisonic gain of the front position Ambisonic gain G n, m and also referred to.
[0076]
　For example, each Ambisonic channel C n, m front position Ambisonic gain G of n, m is such as follows.
[0077]
　That is, each Ambisonic channel C n, m front position Ambisonic gain G of n, m by multiplying the input audio object signals, each of these Ambisonic channel C of n, m Ambisonic signal, i.e. the Ambisonic format it is assumed that the signal.
[0078]
　At this time, they each Ambisonic channel C of n, m and play the sound of audio objects based on the signal composed of the Ambisonic signals, the sound image of the sound of the audio object will be localized at the front position.
[0079]
　Moreover, in this case, the sound of the audio objects, a sound with a spread of angles indicated by the spread information. That is, it is possible to express a spread of the same sound and when generating a 19 audio objects spreads with a spread information.
[0080]
　Here, an angle (hereinafter, the spread angle also referred to) indicated by the spread information, each Ambisonic channel C n, m front position of Ambisonic gain G n, m relationship with is as shown in FIG. The vertical axis in FIG. 7 is a front position Ambisonic gain G n, m represents the values of the horizontal axis represents the spread angle.
[0081]
　Curve L11 to the curve L17 in FIG. 7, each Ambisonic channel C for the spread angle n, m front position Ambisonic gain G of n, m shows.
[0082]
　Specifically, the curve L11 is a spherical harmonic function S n, m (theta, phi) when each of the order n and degree m is 1, it ie Ambisonic channel corresponding to the order n = 1 and the order m = 1 C 1,1 front position Ambisonic gain G of 1,1 shows.
[0083]
　Similarly, the curve L12 is Ambisonic channel C corresponds to the order n = 0 and order m = 0 0,0 front position Ambisonic gain G of 0,0 indicates a curve L13 is the order n = 2 and order m = Ambisonic channel C corresponds to 2 2,2 front position Ambisonic gain G of 2,2 shows.
[0084]
　Curve L14 is Ambisonic channel C corresponds to the order n = 3 and the order m = 3 3,3 front position Ambisonic gain G of 3,3 indicates a curve L15 is the order n = 3 and the order m = Ambisonic channel C corresponds to 1 3,1 front position Ambisonic gain G of 3,1 shows.
[0085]
　Further curve L16 is the order n = 2 and the order m = 0 Ambisonic channel C corresponds to 2,0 front position Ambisonic gain G of 2,0 indicates a curve L17 is the order other than the n and degree m ( However, 0 ≦ n ≦ 3, Ambisonic channel corresponding to ≦ m ≦ -3 3) C n, m front position Ambisonic gain G of n, m shows. That is, the curve L17 is Ambisonic channels C 1, -1 , C 1, 0 , C 2,1 , C 2, -1 , C 2, -2 , C 3, 0 , C 3, -1 , C 3, 2 , C 3, -2 , and C 3, -3 shows a front position Ambisonic gain. Here, the front position Ambisonic gain indicated by curve L17 has a 0 regardless of the spread angle.
[0086]
　Incidentally, spherical harmonics S n, m (theta, phi) definition of, for example, "INTERNATIONAL STANDARD ISO / IEC 23008-3 First edition 2015-10-15 Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 3 : because it is F.1.3 chapter in detail description of the 3D audio ", and the description thereof will be omitted.
[0087]
　These spreads angular and the front position Ambisonic gain G n, m relation can be determined in advance.
[0088]
　Specifically, each and θ and φ elevation and azimuth are shown a 3-dimensional spatial position of the spread audio object obtained in response to the spread angle.
[0089]
　In particular, i-th among the 19 pieces of spread audio object (where, 0 ≦ i ≦ 18) the elevation and azimuth of the spread audio objects theta i and phi i and that referred to. Incidentally, elevation theta i and azimuth phi i corresponds to a respective above-mentioned position_elevation and Position_azimuth.
[0090]
　In this case, the elevation angle theta spreads audio object i and the azimuth angle phi i spherical harmonics S n, m (theta, phi) is substituted, the spherical harmonic of the resulting 19 audio objects for each spread was function S n, m (theta i , phi i ) a front position AMBI by adding the sonic gain G n, m can be obtained. That is, the front position Ambisonic gain G by calculating the following equation (6) n, m can be obtained.
[0091]
[Number 6]

[0092]
　Equation (6) in the calculation, the same Ambisonic channel C n, m 19 pieces of spherical harmonics S obtained for n, m (theta i , phi i ) is the sum of its Ambisonic channel C n, m of front position Ambisonic gain G n, m are the.
[0093]
　That is, for the spread angle indicated by the spread information, plurality, here are determined in the spatial position of the 19 pieces of audio objects for each spread, the angle indicating the position of each spread audio object elevation angle θ i and azimuth Fai i become.
[0094]
　Then, elevation angle theta spreads audio object i and the azimuth angle phi i spherical harmonics the assignment and values spherical harmonics obtained by S n, m (theta i , phi i is), 19 pieces of audio spreads spherical harmonic S obtained for objects n, m (theta i , phi i ) the sum of the front position Ambisonic gain G n, m are.
[0095]
　In the example shown in FIG. 7, Ambisonic channel C 0,0 , C 1, 1 , C 2, 0 , C 2, 2 , C 3, 1 , and C 3,3 only substantially front position Ambisonic gain G n, m have, other Ambisonic channel C n, m front position Ambisonic gain G n, m is zero.
[0096]
　For example, in Ambisonic gain calculator 21 performs calculation of equation (6) based on the spread information, each Ambisonic channel C n, m front position Ambisonic gain G of n, m may but be calculated, wherein in the gain table is used by the front position Ambisonic gain G n, m is obtained.
[0097]
　That is, in the Ambisonic gain calculator 21, the spread angle and the front position Ambisonic gain G n, m is gain table and is associated Ambisonic channel C n, m are previously generated for each, it is held.
[0098]
　For example, in the gain table for the value of each spread angle, the front position corresponding to their spread angle Ambisonic gain G n, m the value of may be associated. Further, for example, for a range of the spread angle values, the front position Ambisonic gain G corresponding to the range n, m the value of may be associated.
[0099]
　Note that the resolution of the spread angle in the gain table, the resource size and the apparatus for reproducing the sound of content based on the input audio object signal or the like may be determined according to the playback quality required at the time of content reproduction.
[0100]
　Also, when the spread angle as can be seen from Figure 7 small, front position Ambisonic gain G with respect to the change in the spread angle n, m the amount of change is small. Therefore, in the gain table, the small spread angle, one front position Ambisonic gain G n, m ranges spread angle associating, i.e. to increase the step width of the spread angle, the step width in accordance with the spread angle increases it may be such as to reduce.
[0101]
　Furthermore, the spread angle indicated by the spread information, when there in such as an intermediate value of the two spread angles in the gain table is a front position by performing an interpolation process such as linear interpolation Ambisonic gain G n, m to seek it may be.
[0102]
　In such a case, for example Ambisonic gain calculator 21 is a front position associated with the spread angle in the gain table Ambisonic gain G n, m by performing interpolation processing on the basis of, the spread angle shown by spread information corresponding front position Ambisonic gain G n, m seeking.
[0103]
　Specifically, for example, spread angle indicated by the spread information is assumed to be 65 degrees. Further, in the gain table, spread angle "60 degrees" and the front position Ambisonic gain G n, m "0.2" are associated, spread angle "70 degrees" and the front position Ambisonic gain G n, m "0.3 "is to be associated with.
[0104]
　At this time, Ambisonic gain calculator 21, based on the spread information and gain table, front position Ambisonic gain G corresponding to the spread angle of "65 degrees" n, m and "0.25" is calculated by linear interpolation.
[0105]
　As described above, in the Ambisonic gain calculator 21, the Ambisonic channel C changes according to the spread angle n, m front position Ambisonic gain G of n, m holding the gain table obtained by tabulating advance It is.
[0106]
　Accordingly, without separately generating the 19 audio objects spreads from a spread information, directly from the gain table, the front position Ambisonic gain G n, m can be obtained. With the gain table, direct, front position Ambisonic gain G n, m than the case of calculating the, it is possible to further reduce the computational load.
[0107]
　Here, the audio object will be described an example in which Ambisonic gain is obtained when the front position in the Ambisonic gain calculator 21. However, not limited to the front position, Ambisonic gain may be obtained when in the position in which audio object is another reference in Ambisonic gain calculator 21.
[0108]
　Referring back to FIG. 6, Ambisonic gain calculation unit 21, a spread information supplied, the held gain table and the Ambisonic channel C on the basis of n, m front position Ambisonic gain G n, m When seeking, resulting front position Ambisonic gain G n, m and supplies to the Ambisonic rotating portion 22.
[0109]
　Ambisonic rotating part 22, on the basis of the supplied object position information, supplied from Ambisonic gain calculator 21 a front position Ambisonic gain G n, m performs rotation processing on.

claims

Based on the spread information of the object, comprising the Ambisonic gain calculator which the object seeks Ambisonic gain when in the predetermined position
　signal processing unit.
[Requested item 2]
　And audio object signal of the object, the ambiguous based on the sonic gain, further comprising a Ambisonic signal generator for generating an Ambisonic signal of the object
　signal processing apparatus according to claim 1.
[Requested item 3]
　The Ambisonic gain calculator,
　　on the basis of the spread information, obtains a reference position Ambisonic gain when to be in a position where the object is a reference,
　　based on the object position information indicating the predetermined position, the obtaining the Ambisonic gain performing rotation processing on the reference position Ambisonic gain
　signal processing apparatus according to claim 1.
[Requested item 4]
　The Ambisonic gain calculator obtains the reference position Ambisonic gain based on said spread information and gain table
　signal processing apparatus according to claim 3.
[Requested item 5]
　The gain table, spread angle and the reference position AMBI Sonic gain in which the associated
　signal processing apparatus according to claim 4.
[Requested item 6]
　The Ambisonic gain calculation unit, by performing an interpolation process on the basis of each of the reference position Ambisonic gain associated with each of the plurality of spread angle in the gain table, spread angle shown by the spread information determining the reference position Ambisonic gain corresponding to the
　signal processing apparatus according to claim 5.
[Requested item 7]
　The reference position Ambisonic gain, respectively of the sum of the values obtained respectively by substituting the spherical harmonic function of the angle indicating each of a plurality of positions in space defined relative to the spread angle shown by spread information there
　the signal processing apparatus according to claim 3.
[Requested item 8]
　Based on the spread information of the object, the object is determining the Ambisonic gain when in the predetermined position
　signal processing method comprising the steps.
[Requested item 9]
　Based on the spread information of the object, the object is determining the Ambisonic gain when in the predetermined position
　program for executing the processing including a step in the computer.