Specification
Title of invention: Signal processing device and method, and program
Technical field
[0001]
　The present technology relates to a signal processing device and method, and a program, and particularly relates to a signal processing device and method, and a program capable of more effectively realizing distance perception control.
Background technology
[0002]
　In recent years, object-based audio technology has attracted attention.
[0003]
　In object-based audio, audio data is composed of a waveform signal for an object and metadata indicating localization information of the object represented by a relative position from a viewing point serving as a predetermined reference.
[0004]
　Then, the waveform signal of the object is rendered into a signal having a desired number of channels by, for example, VBAP (Vector Based Amplitude Panning) based on the metadata, and is reproduced (see, for example, Non-Patent Document 1 and Non-Patent Document 2). ..
Prior art documents
Non-patent literature
[0005]
Non-Patent Document 1: ISO/IEC 23008-3 Information technology-High efficiency coding and media delivery in heterogeneous environments-Part 3: 3D audio
Non-Patent Document 2: Ville Pulkki, “Virtual Sound Source Positioning Using Vector Base Amplitude Panning”, Journal of AES, vol.45, no.6, pp.456-466, 1997
Summary of the invention
Problems to be Solved by the Invention
[0006]
　According to the method described above, in object-based audio rendering, it is possible to arrange each object in various directions in a three-dimensional space and localize the sound.
[0007]
　However, it is difficult to effectively control the sense of distance of the audio object. That is, for example, when it is desired to give a sense of distance to the front and back when reproducing the sound of the object, there is no choice but to produce a sense of distance by gain control or frequency characteristic control, and it is not possible to obtain a sufficient effect. Further, although it is possible to use a waveform signal that has been processed to have a sound quality that gives a sense of distance in advance, in such a case, the sense of distance cannot be controlled on the reproducing side.
[0008]
　The present technology has been made in view of such a situation, and makes it possible to more effectively realize the sense of distance control.
Means for solving the problem
[0009]
　A signal processing device according to one aspect of the present technology includes a reverb processing unit that generates a reverb component signal based on object audio data of an audio object and a reverb parameter for the audio object.
[0010]
　A signal processing method or program according to an aspect of the present technology includes a step of generating a signal of a reverb component based on object audio data of an audio object and a reverb parameter for the audio object.
[0011]
　In one aspect of the present technology, a reverb component signal is generated based on object audio data of an audio object and a reverb parameter for the audio object.
Effect of the invention
[0012]
　According to the 1 side of this art, distance sense control can be realized more effectively.
[0013]
　Note that the effects described here are not necessarily limited and may be any effects described in the present disclosure.
Brief description of the drawings
[0014]
FIG. 1 is a diagram showing a configuration example of a signal processing device.
FIG. 2 is a diagram showing an example of reverb parameters.
FIG. 3 is a diagram illustrating Wet component position information and sound image localization of Wet components.
FIG. 4 is a diagram illustrating Wet component position information and sound image localization of Wet components.
FIG. 5 is a flowchart illustrating an audio signal output process.
FIG. 6 is a diagram showing a configuration example of a signal processing device.
FIG. 7 is a diagram showing an example of syntax of meta information.
FIG. 8 is a flowchart illustrating an audio signal output process.
FIG. 9 is a diagram showing a configuration example of a signal processing device.
FIG. 10 is a diagram for explaining components of parametric reverb.
FIG. 11 is a diagram showing a syntax example of meta information.
FIG. 12 is a diagram showing a syntax example of Reverb_Configuration().
FIG. 13 is a diagram showing a syntax example of Reverb_Structure().
FIG. 14 is a diagram showing a syntax example of Branch_Configuration(n).
FIG. 15 is a diagram showing a syntax example of PreDelay_Configuration().
FIG. 16 is a diagram showing a syntax example of MultiTapDelay_Configuration().
FIG. 17 is a diagram showing a syntax example of AllPassFilter_Configuration().
FIG. 18 is a diagram illustrating a syntax example of CombFilter_Configuration().
FIG. 19 is a diagram showing a syntax example of HighCut_Configuration().
FIG. 20 is a diagram showing a syntax example of Reverb_Parameter().
FIG. 21 is a diagram showing a syntax example of Branch_Parameters(n).
FIG. 22 is a diagram showing a syntax example of PreDelay_Parameters().
FIG. 23 is a diagram showing a syntax example of MultiTapDelay_Parameters().
FIG. 24 is a diagram showing a syntax example of HighCut_Parameters().
FIG. 25 is a diagram illustrating a syntax example of AllPassFilter_Parameters().
FIG. 26 is a diagram illustrating a syntax example of CombFilter_Parameters().
FIG. 27 is a diagram showing a syntax example of meta information.
FIG. 28 is a flowchart illustrating an audio signal output process.
FIG. 29 is a diagram showing a configuration example of a signal processing device.
FIG. 30 is a diagram showing an example of syntax of meta information.
FIG. 31 is a diagram showing a configuration example of a computer.
MODE FOR CARRYING OUT THE INVENTION
[0015]
　Hereinafter, embodiments to which the present technology is applied will be described with reference to the drawings.
[0016]

The present technology makes it possible to more effectively realize a sense of distance by adding a reflection component and a reverberation component of sound based on a parameter. It is a thing.
[0017]
　That is, the present technology has the following features.
[0018]
　Features (1) Features
　that control the sense of distance by adding reflection/reverberation components based on the reverb setting parameters for the object
　(2) Features
　that localize the reflection/reverberation components to a position different from the sound image of the object
　(3)
　Reflection /The position information of the reverberation component is specified by the relative position to the localization position of the sound image of the target object.
　(4)
　The position information of the reflection/reverberation component is fixed regardless of the localization position of the sound image of the target object. specified in
　feature (5)
　the impulse response of the reverberation processing for adding to the object and the meta information, the sense of distance control by adding a reflection / reverberation component by the filtering process based on the meta information when rendering
　feature (6 ) 　Feature
　for extracting configuration information and coefficient of reverb processing algorithm to be applied
(7)
　Parameterizing configuration information and coefficient of reverb processing algorithm, and
　feature as meta information (8)
　Based on meta information, reverb processing algorithm on playback side Distance control is performed by reconstructing and adding reverberation components in object-based audio rendering.
[0019]
　For example, when a person perceives sound, he or she hears not only the direct sound from the sound source, but also the reflected sound and reverberant sound from a wall, etc. Feel the distance. Therefore, when rendering an audio object, a reverberation process is used to add reflected or reverberant sound, and the time difference or gain difference between the direct sound and the reflected or reverberant sound is controlled to give the audio object a sense of distance. be able to.
[0020]
　Note that, hereinafter, the audio object will also be simply referred to as an object.
[0021]

　FIG. 1 is a diagram showing a configuration example of an embodiment of a signal processing device to which the present technology is applied.
[0022]
　The signal processing device 11 shown in FIG. 1 includes a demultiplexer 21, a reverb processing unit 22, and a VBAP processing unit 23.
[0023]
　The demultiplexer 21 separates object audio data, reverb parameters, and position information from a bitstream in which various kinds of data are multiplexed.
[0024]
　The demultiplexer 21 supplies the separated object audio data to the reverb processing unit 22, supplies the reverb parameter to the reverb processing unit 22 and the VBAP processing unit 23, and supplies the position information to the VBAP processing unit 23.
[0025]
　Here, the object audio data is audio data for reproducing the sound of the object. The reverb parameter is information for reverb processing that adds a reflected sound component and a reverberation sound component to the object audio data.
[0026]
　Here, the reverb parameter is included in the bitstream as the meta information (metadata) of the object, but the reverb parameter may not be included in the bitstream and may be given as an external parameter.
[0027]
　The position information is information indicating the position of the object in the three-dimensional space. For example, the position information includes a horizontal angle indicating the horizontal position of the object viewed from a predetermined reference position and a position viewed from the predetermined reference position. Contains a vertical angle that indicates the vertical position of the object.
[0028]
　The reverb processing unit 22 performs reverb processing based on the object audio data and the reverb parameter supplied from the demultiplexer 21, and supplies the resulting signal to the VBAP processing unit 23. That is, the reverb processing unit 22 adds a component of reflected sound or reverberant sound, that is, a Wet component (wet component) to the object audio data. The reverb processing unit 22 also controls the gain of the dry component (dry component) that is the direct sound, that is, the object audio data and the Wet component.
[0029]
　In this example, as a result of the reverb processing, a signal of one Dry/Wet component indicated by the letters “Dry/Wet component” and N Wet components indicated by the letters “Wet component 1” to “Wet component N”. And the signal is obtained.
[0030]
　Here, the signal of the Dry/Wet component is a mixed sound of the direct sound, the reflected sound, and the reverberation sound, that is, a signal including the Dry component and the Wet component. Note that the Dry/Wet component signal may include only the Dry component, or may include only the Wet component.
[0031]
　The signal of the Wet component generated by the reverb processing is a signal composed of only the components of reflected sound and reverberation sound. In other words, the Wet component signal is a reverb component signal such as a reflected sound component or a reverberation sound component generated by the reverb processing of the object audio data. Hereinafter, the signals of the Wet components represented by the characters “Wet component 1” to “Wet component N” are also referred to as Wet component 1 to Wet component N.
[0032]
　As will be described in detail later, the signal of the Dry/Wet component is the original object audio data with the components of the reflected sound and reverberation sound added, and is reproduced based on the position information indicating the position of the original object. R. That is, the sound image of the Dry/Wet component is rendered so as to be localized at the position of the object indicated by the position information.
[0033]
　On the other hand, for the signals of Wet component 1 to Wet component N, the rendering process may be performed based on the Wet component position information that is position information different from the position information indicating the original position of the object. it can. Such Wet component position information is included in the reverb parameter, for example.
[0034]
　Further, here, an example in which Dry/Wet component and Wet component are generated by the reverb process will be described, but only Dry/Wet component may be generated by the reverb process, Dry component and Wet component 1 to Wet component It may be N.
[0035]
　The VBAP processing unit 23 is externally supplied with the arrangement of the respective reproduction speakers constituting the reproduction speaker system for reproducing the sound of the object, that is, reproduction speaker arrangement information indicating the speaker structure.
[0036]
　The VBAP processing unit 23, based on the supplied reproduction speaker arrangement information and the reverb parameter and position information supplied from the demultiplexer 21, the Dry/Wet component and the Wet components 1 to Wet supplied from the reverb processing unit 22. It functions as a rendering processing unit that performs VBAP processing and the like on the component N as rendering processing. The VBAP processing unit 23 outputs the audio signal of each channel corresponding to each reproduction speaker, which is obtained by the rendering process, as an output signal to the reproduction speaker in the subsequent stage.
[0037]

　By the way, the reverb parameter supplied to the reverb processing unit 22 and the VBAP processing unit 23 includes information (parameter) necessary for performing the reverb processing.
[0038]
　Specifically, for example, the reverb parameter includes the information shown in FIG.
[0039]
　In the example shown in FIG. 2, the reverb parameter includes Dry gain, Wet gain, reverberation time, pre-delay delay time, pre-delay gain, initial reflection delay time, initial reflection gain, and Wet component position information.
[0040]
　For example, the Dry gain is gain information used for gain control of the Dry component, that is, gain adjustment, and the Wet gain is used for gain control of the Wet component and Wet components 1 to Wet component N included in the Dry/Wet components. It is information.
[0041]
　The reverberation time is time information indicating the reverberation length of the reverberation sound included in the sound of the object. The pre-delay delay time is time information indicating a delay time until the first reflection sound other than the initial reflection sound or the reverberation sound is heard based on the time when the direct sound is heard. The pre-delay gain is gain information indicating a gain difference between a direct sound and a sound component at a time determined by the pre-delay delay time.
[0042]
　The initial reflection delay time is time information indicating the delay time until the initial reflected sound is heard, based on the time when the direct sound is heard, and the initial reflection gain is the gain indicating the gain difference between the direct reflected sound and the direct sound. It is information.
[0043]
　For example, if the pre-delay delay time and the initial reflection delay time are shortened and the pre-delay gain and the initial reflection gain are decreased, the sense of distance between the object and the viewer (user) becomes closer.
[0044]
　On the other hand, if the pre-delay delay time and the initial reflection delay time are lengthened and the pre-delay gain and the initial reflection gain are increased, the sense of distance between the object and the viewer becomes far.
[0045]
　The Wet component position information is information indicating the localization position of each sound image of the Wet component 1 to Wet component N in the three-dimensional space.
[0046]
　When the reverb parameter includes the Wet component position information, by appropriately determining the Wet component position information, the VBAP processing in the VBAP processing unit 23 causes the sound image of the Wet component to be the direct sound of the object, that is, the Dry/Wet component. It can be localized at a position different from the sound image of.
[0047]
　For example, assume that the Wet component position information is composed of a horizontal angle and a vertical angle indicating the relative position of the Wet component with respect to the position indicated by the position information of the object.
[0048]
　In such a case, for example, as shown in FIG. 3, the sound image of each Wet component can be localized around the sound image of the Dry/Wet component of the object.
[0049]
　In the example shown in FIG. 3, there are Wet components 1 to Wet components 4 as Wet components, and the Wet component position information of these Wet components is shown on the upper side in the figure. Here, the Wet component position information is information indicating the position (direction) of each Wet component viewed from the predetermined origin O.
[0050]
　For example, the position of the Wet component 1 in the horizontal direction is a position determined by an angle obtained by adding 30 degrees to the horizontal angle indicating the position of the object, and the position of the Wet component 1 in the vertical direction is The position is determined by the angle obtained by adding 30 degrees to the vertical angle indicating the position of the object.
[0051]
　Further, in the figure, the position of the object and the positions of the Wet components 1 to 4 are shown on the lower side. That is, the position OB11 indicates the position of the object indicated by the position information, and each of the positions W11 to W14 indicates the position of each of Wet component 1 to Wet component 4 indicated by the Wet component position information.
[0052]
　In this example, it can be seen that the Wet components 1 to 4 are arranged so as to surround the periphery of the object. In the VBAP processing unit 23, based on the position information of the object, the Wet component position information, and the reproduction speaker arrangement information, the sound images of Wet components 1 to Wet components 4 are output by VBAP processing so as to be localized at positions W11 to W14. A signal will be generated.
[0053]
　In this way, by appropriately arranging the Wet component at a position different from the position of the object, it is possible to effectively control the sense of distance of the object.
[0054]
　Further, in FIG. 3, the position of each Wet component, that is, the localization position of the sound image of the Wet component is a position relative to the position of the object, but the position is not limited to this, and a predetermined specific position (fixed position Position) or the like.
[0055]
　In such a case, the position of the Wet component indicated by the Wet component position information is an arbitrary absolute position in the three-dimensional space that is unrelated to the position of the object indicated by the position information. Then, for example, as shown in FIG. 4, the sound image of each Wet component can be localized at an arbitrary position in the three-dimensional space.
[0056]
　In the example shown in FIG. 4, there are Wet components 1 to 4 as Wet components, and the Wet component position information of these Wet components is shown on the upper side in the figure. Here, the Wet component position information is information indicating the absolute position of each Wet component viewed from the predetermined origin O.
[0057]
　For example, the horizontal angle indicating the position of the Wet component 1 in the horizontal direction is 45 degrees, and the vertical angle indicating the position of the Wet component 1 in the vertical direction is 0 degrees.
[0058]
　Further, in the figure, the position of the object and the positions of the Wet components 1 to 4 are shown on the lower side. That is, the position OB21 indicates the position of the object indicated by the position information, and each of the positions W21 to W24 indicates the position of each of Wet component 1 to Wet component 4 indicated by the Wet component position information.
[0059]
　In this example, it can be seen that the Wet components 1 to 4 are arranged so as to surround the periphery of the origin O.
[0060]

　Next, the operation of the signal processing device 11 will be described. That is, the audio signal output processing by the signal processing device 11 will be described below with reference to the flowchart of FIG.
[0061]
　In step S11, the demultiplexer 21 receives the bitstream transmitted from the encoding device or the like, and separates the object audio data, the reverb parameter, and the position information from the received bitstream.
[0062]
　The demultiplexer 21 supplies the object audio data and the reverb parameter thus obtained to the reverb processing unit 22, and also supplies the reverb parameter and the position information to the VBAP processing unit 23.
[0063]
　In step S12, the reverb processing unit 22 performs reverb processing on the object audio data supplied from the demultiplexer 21, based on the reverb parameter supplied from the demultiplexer 21.
[0064]
　That is, in the reverb processing, reflected sound and reverberant sound components are added to the object audio data, gain adjustment of direct sound, reflected sound, reverberant sound, that is, gain adjustment of Dry component and Wet component is performed, A signal of Dry/Wet component and signals of Wet component 1 to Wet component N are generated. The reverb processing unit 22 supplies the signal of the Dry/Wet component thus generated and the signals of the Wet component 1 to Wet component N to the VBAP processing unit 23.
[0065]
　In step S13, the VBAP processing unit 23, based on the supplied reproduction speaker arrangement information, the position information from the demultiplexer 21 and the Wet component position information included in the reverb parameter, outputs Dry from the reverb processing unit 22. The /Bet component and the Wet component 1 to Wet component N are subjected to VBAP processing or the like as rendering processing to generate an output signal.
[0066]
　The VBAP processing unit 23 outputs the output signal obtained by the rendering processing to the subsequent stage, and the audio signal output processing ends. For example, the output signal output from the VBAP processing unit 23 is supplied to the reproduction speaker in the subsequent stage, and the reproduction speaker reproduces the sound of the Dry/Wet component or the Wet component 1 to Wet component N based on the supplied output signal ( Output.
[0067]
　As described above, the signal processing device 11 performs the reverb processing on the object audio data based on the reverb parameter to generate the Dry/Wet component and the Wet component.
[0068]
　By doing so, it is possible to more effectively realize the sense of distance control on the reproduction side of the object audio data.
[0069]
　That is, by using the reverb parameter as the meta information of the object, it becomes possible to control the sense of distance in the rendering of the object-based audio.
[0070]
　For example, when a content creator wants to give a sense of distance to an object, an appropriate reverb parameter may be added as meta information instead of processing the object audio data to sound quality that makes sense of distance. Then, in rendering on the reproduction side, reverb processing according to meta information (reverb parameter) can be performed on the audio object to reproduce the sense of distance of the object.
[0071]
　A Wet component is generated separately from the Dry/Wet component, and the sound image of the Wet component is localized at a predetermined position to realize the sense of distance to the object. For example, when VBAP processing is performed as rendering processing, content creation This is particularly effective when the channel configuration of the reproduction speaker is unknown on the side.
[0072]

　By the way, in the method shown in the first embodiment, the reverb processing algorithm handled by the content creator and the reproduction side, that is, the signal processing device 11 It is assumed that the reverb processing algorithms used on the side are the same.
[0073]
　Therefore, when the algorithm on the side of the content creator is different from the algorithm on the side of the signal processing device 11, the sense of distance as intended by the content creator cannot be reproduced.
[0074]
　In addition, content creators generally want to select and apply the optimum reverb processing from various algorithms, so limit the reverb processing algorithm to one or a limited number of reverb processing algorithms. Is not practical.
[0075]
　Therefore, by using the impulse response as the reverb parameter, it is possible to reproduce the sense of distance as intended by the content creator by the reverb processing according to the meta information, that is, the impulse response as the reverb parameter. Good.
[0076]
　In such a case, the signal processing device is configured as shown in FIG. 6, for example. Note that, in FIG. 6, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.
[0077]
　The signal processing device 51 shown in FIG. 6 includes a demultiplexer 21, a reverb processing unit 61, and a VBAP processing unit 23.
[0078]
　The configuration of the signal processing device 51 differs from the configuration of the signal processing device 11 in that a reverb processing unit 61 is provided in place of the reverb processing unit 22 of the signal processing device 11 in FIG. The configuration is similar to that of the device 11.
[0079]
　The reverb processing unit 61 performs reverb processing on the object audio data supplied from the demultiplexer 21 on the basis of the impulse response coefficient included in the reverb parameter supplied from the demultiplexer 21 to obtain a Dry/Wet component and a Wet component. Each signal of components 1 to Wet component N is generated.
[0080]
　In this example, the reverb processing unit 61 is composed of a FIR (Finite Impulse Response) filter. That is, the reverb processing unit 61 includes an amplification unit 71, delay units 72-1-1 to 72-NK, amplification units 73-1-1 to 73-N-(K+1), and an addition unit 74-1. It has an adder 74-N, an amplifier 75-1 to an amplifier 75-N, and an adder 76.
[0081]
　The amplifying unit 71 multiplies the object audio data supplied from the demultiplexer 21 by the gain value included in the reverb parameter to perform gain adjustment, and the object audio data obtained as a result is added to the adding unit 76. Supply. The object audio data obtained by the amplification unit 71 is a Dry component signal, and the gain adjustment process in the amplification unit 71 is the gain control process of the direct sound (Dry component).
[0082]
　The delay unit 72-L-1 (where 1≦L≦N) delays the object audio data supplied from the demultiplexer 21 by a predetermined time, and then the amplification unit 73-L-2 and the delay unit 72-L. -2 supply.
[0083]
　The delay unit 72-LM (where 1≦L≦N, 2≦M≦K-1) delays the object audio data supplied from the delay unit 72-L-(M-1) by a predetermined time. After that, the signal is supplied to the amplification unit 73-L-(M+1) and the delay unit 72-L-(M+1).
[0084]
　The delay unit 72-LK (where 1≦L≦N) delays the object audio data supplied from the delay unit 72-L-(K-1) by a predetermined time, and then the amplification unit 73-L. -(K+1) is supplied.
[0085]
　Note that the delay units 72-M-1 to 72-MK (where 3≦M≦N−1) are not shown here.
[0086]
　Hereinafter, the delay units 72-M-1 to 72-M-K (where 1≦M≦N) are simply referred to as the delay unit 72-M unless it is necessary to distinguish them. Further, hereinafter, the delay units 72-1 to 72-N will be simply referred to as delay units 72 unless it is necessary to distinguish them.
[0087]
　The amplification unit 73-M-1 (where 1≦M≦N) adjusts the gain by multiplying the object audio data supplied from the demultiplexer 21 by the impulse response coefficient included in the reverb parameter. And the object audio data obtained as a result is supplied to the addition unit 74-M.
[0088]
　The amplification unit 73-LM (where 1≦L≦N, 2≦M≦K+1) is included in the reverb parameter for the object audio data supplied from the delay unit 72-L-(M-1). The gain is adjusted by multiplying the coefficient of the impulse response, and the object audio data obtained as a result is supplied to the adder 74-L.
[0089]
　In FIG. 6, the illustration of the amplification units 73-3-1 to 73-(N-1)-(K+1) is omitted.
[0090]
　Further, hereinafter, the amplifiers 73-L-1 to 73-L-(K+1) (where 1≦L≦N) are simply referred to as amplifiers 73-L unless it is necessary to distinguish them. Further, hereinafter, the amplifiers 73-1 to 73-N are simply referred to as the amplifiers 73 unless it is necessary to distinguish them.
[0091]
　The adder 74-M (where 1≦M≦N) adds the object audio data supplied from the amplifiers 73-M-1 to 73-M-(K+1), and the resulting Wet is obtained. The component M (where 1≦M≦N) is supplied to the amplification unit 75-M and the VBAP processing unit 23.
[0092]
　Here, the addition units 74-3 to 74-(N-1) are not shown. Hereinafter, the adders 74-1 to 74-N will be simply referred to as the adders 74 unless it is necessary to distinguish them.
[0093]
　The amplifier 75-M (where 1≦M≦N) is included in the reverb parameter for the signal of the Wet component M (where 1≦M≦N) supplied from the adder 74-M. The gain value is multiplied to adjust the gain, and the signal of the Wet component obtained as a result is supplied to the addition unit 76.
[0094]
　Here, the illustration of the amplification units 75-3 to 75-(N-1) is omitted. Hereinafter, the amplifiers 75-1 to 75-N will be simply referred to as the amplifiers 75 unless it is necessary to distinguish them.
[0095]
　The addition unit 76 adds the object audio data supplied from the amplification unit 71 and the Wet component signals supplied from each of the amplification units 75-1 to 75-N, and the resulting signal is obtained. The signal of the Dry/Wet component is supplied to the VBAP processing unit 23.
[0096]
　When the reverb processing unit 61 has such a configuration, the impulse response of the reverb processing applied at the time of content production is used as the meta information included in the bitstream, that is, the reverb parameter. In such a case, the syntax of the meta information (reverb parameter) is as shown in FIG. 7, for example.
[0097]
　In the example illustrated in FIG. 7, the meta information, that is, the reverb parameter includes a dry gain that is a gain value of the direct sound (Dry component) indicated by the character “dry_gain”. The dry gain dry_gain is supplied to the amplification unit 71 and used for gain adjustment in the amplification unit 71.
[0098]
　Further, in this example, the localization mode information of the Wet component (reflection/reverberation sound) indicated by the character “wet_position_mode” is stored following the dry gain.
[0099]
　For example, when the value of the localization mode information wet_position_mode is “0”, the relative localization mode in which the Wet component position information indicating the position of the Wet component is the information indicating the relative position to the position indicated by the position information of the object Is shown. For example, the example described with reference to FIG. 3 is the relative localization mode.
[0100]
　On the other hand, when the value of the localization mode information wet_position_mode is “1”, the Wet component position information indicating the position of the Wet component indicates an absolute position in the three-dimensional space regardless of the position of the object. It indicates that it is in the absolute localization mode, which is used as information. For example, the example described with reference to FIG. 4 is the absolute localization mode.
[0101]
　Further, after the localization mode information wet_position_mode, the number of signals of the output Wet component (reflection/reverberation sound), that is, the number of outputs, which is indicated by the character “number_of_wet_outputs”, is stored. In the example shown in FIG. 6, N Wet component signals of Wet component 1 to Wet component N are output to the VBAP processing unit 23, and thus the value of the number of outputs number_of_wet_outputs is “N”.
[0102]
　Further, the number of outputs number_of_wet_outputs is followed by the number of gain values ​​of the Wet components stored by the number of outputs number_of_wet_outputs. That is, here, the gain value of the i-th Wet component i indicated by the character “wet_gain[i]” is stored. The gain value wet_gain[i] is supplied to the amplification unit 75 and used for gain adjustment in the amplification unit 75.
[0103]
　When the value of the localization mode information wet_position_mode is “0”, the gain value wet_gain[i] is followed by the horizontal angle indicated by the character “wet_position_azimuth_offset[i]” and the character “wet_position_elevation_offset[i]”. The vertical angle is stored.
[0104]
　The horizontal angle wet_position_azimuth_offset[i] indicates the horizontal position of the i-th Wet component i in the three-dimensional space in the horizontal direction, which is relative to the position of the object. Similarly, the vertical angle wet_position_elevation_offset[i] indicates the vertical angle relative to the position of the object, which indicates the vertical position of the i-th Wet component i in the three-dimensional space.
[0105]
　Therefore, in this case, the position of the i-th Wet component i in the three-dimensional space is obtained from the horizontal angle wet_position_azimuth_offset[i] and the vertical angle wet_position_elevation_offset[i], and the position information of the object.
[0106]
　On the other hand, when the value of the localization mode information wet_position_mode is “1”, the gain value wet_gain[i] is followed by the horizontal angle indicated by the character “wet_position_azimuth[i]” and the character “wet_position_elevation[i]”. The vertical angle indicated by is stored.
[0107]
　The horizontal angle wet_position_azimuth[i] indicates the horizontal angle indicating the absolute horizontal position of the i-th Wet component i in the three-dimensional space. Similarly, the vertical angle wet_position_elevation[i] indicates the vertical angle indicating the absolute position of the i-th Wet component i in the three-dimensional space in the vertical direction.
[0108]
　Further, the reverb parameter stores tap length of the impulse response for the i-th Wet component i, which is indicated by the character “number_of_taps[i]”, that is, tap length information indicating the number of coefficients of the impulse response.
[0109]
　Then, following the tap length information number_of_taps[i], by the number indicated by the tap length information number_of_taps[i], the impulse response for the i-th Wet component i indicated by the characters "coef[i][j]" The coefficient is stored.
[0110]
　The coefficient coef[i][j] is supplied to the amplification unit 73 and used for gain adjustment in the amplification unit 73. For example, in the example shown in FIG. 6, the coefficient coef[0][0] is supplied to the amplification unit 73-1-1, and the coefficient coef[0][1] is supplied to the amplification unit 73-1-2.
[0111]
　In this way, the impulse response is added as meta information (reverb parameter), and the reverb processing according to the meta information is performed on the audio object during rendering on the playback side. You can reproduce the feeling.
[0112]

　Next, the operation of the signal processing device 51 shown in FIG. 6 will be described. That is, the audio signal output processing by the signal processing device 51 will be described below with reference to the flowchart of FIG.
[0113]
　Note that the process of step S41 is similar to the process of step S11 in FIG. 5, so description thereof will be omitted. However, in step S41, the remultiplexer 21 reads the reverberation parameters shown in FIG. 7 from the bitstream and supplies them to the reverberation processing unit 61 and the VBAP processing unit 23.
[0114]
　In step S 42, the amplification unit 71 of the reverb processing unit 61 generates a Dry component signal and supplies it to the addition unit 76.
[0115]
　That is, the reverb processing unit 61 supplies the dry gain dry_gain included in the reverb parameter supplied from the demultiplexer 21 to the amplification unit 71. Further, the amplification unit 71 multiplies the object audio data supplied from the demultiplexer 21 by the dry gain dry_gain to perform gain adjustment, thereby generating a signal of the Dry component.
[0116]
　In step S43, the reverb processing unit 61 generates Wet components 1 to Wet components N.
[0117]
　That is, the reverb processing unit 61 reads the impulse response coefficient coef[i][j] included in the reverb parameter supplied from the demultiplexer 21 and supplies the coefficient coef[i][j] to the amplifying unit 73, and the gain value wet_gain included in the reverb parameter. [i] is supplied to the amplification unit 75.
[0118]
　In addition, each delay unit 72 delays the object audio data supplied from the preceding stage of itself, such as the demultiplexer 21 and the other delay unit 72, for a predetermined time and then supplies the delayed object audio data to the delay unit 72 and the amplifying unit 73 in the subsequent stage. .. The amplification unit 73 multiplies the object audio data supplied from the preceding stage of the demultiplexer 21 and the delay unit 72 by the coefficient coef[i][j] supplied from the reverb processing unit 61, and the addition unit Supply to 74.
[0119]
　The addition unit 74 generates the Wet component by adding the object audio data supplied from the amplification unit 73, and supplies the signal of the obtained Wet component to the amplification unit 75 and the VBAP processing unit 23. Further, the amplification unit 75 multiplies the signal of the Wet component supplied from the addition unit 74 by the gain value wet_gain[i] supplied from the reverb processing unit 61, and supplies the signal to the addition unit 76.
[0120]
　In step S44, the addition unit 76 adds the signal of the Dry component supplied from the amplification unit 71 and the signal of the Wet component supplied from the amplification unit 75 to generate a signal of the Dry/Wet component, It is supplied to the VBAP processing unit 23.
[0121]
　In step S45, the VBAP processing unit 23 performs VBAP processing and the like as rendering processing to generate an output signal.
[0122]
　For example, in step S45, the same process as the process of step S13 of FIG. 5 is performed. In step S45, for example, the horizontal angle wet_position_azimuth_offset[i] and the vertical angle wet_position_elevation_offset[i] included in the reverb parameter in the VBAP process, or the horizontal angle wet_position_azimuth[i] and the vertical angle wet_position_elevation[i] are used as the Wet component position information.
[0123]
　When the output signal is obtained in this way, the VBAP processing unit 23 outputs the output signal to the subsequent stage, and the audio signal output processing ends.
[0124]
　As described above, the signal processing device 51 performs the reverb process on the object audio data based on the reverb parameter including the impulse response to generate the Dry/Wet component and the Wet component. Note that the encoding device generates a bitstream in which the meta information and position information shown in FIG. 7 and the encoded object audio data are stored.
[0125]
　By doing so, it is possible to more effectively realize the sense of distance control on the reproduction side of the object audio data. In particular, by performing the reverb processing using the impulse response, even if the reverb processing algorithm on the signal processing device 51 side and the reverb processing algorithm on the content production side are different, as intended by the content creator. A sense of distance can be reproduced.
[0126]

In
　the second embodiment, the impulse response of the reverb processing that the content creator wants to add is used as the reverb parameter. However, the impulse response of the reverb processing that the content creator wants to add is usually a very long tap length.
[0127]
　Therefore, when transmitting such an impulse response as meta information (reverb parameter), the reverb parameter becomes data with a very large data amount. Further, even if the reverb parameter is slightly changed, the entire impulse response changes, so that it is necessary to retransmit the reverb parameter having a large amount of data each time.
[0128]
　Therefore, the Dry/Wet component or the Wet component may be generated by the parametric reverb. In such a case, the reverb processing unit is configured by parametric reverb obtained by combining a multi-tap delay, a comb filter, an all-pass filter and the like.
[0129]
　Then, by such a reverb processing unit, a reflected sound or a reverberant sound is added to the object audio data based on the reverb parameter, or a direct sound, a reflected sound, or a reverberant sound gain control is performed, and the Dry/ Wet and Wet component signals are generated.
[0130]
　When the reverb processing unit is configured by parametric reverb, for example, the signal processing device is configured as shown in FIG. Note that in FIG. 9, portions corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.
[0131]
　The signal processing device 131 shown in FIG. 9 includes a demultiplexer 21, a reverb processing unit 141, and a VBAP processing unit 23.
[0132]
　The configuration of the signal processing device 131 is different from the configuration of the signal processing device 11 in that a reverb processing unit 141 is provided instead of the reverb processing unit 22 of the signal processing device 11 in FIG. The configuration is similar to that of the device 11.
[0133]
　The reverb processing unit 141 performs reverb processing on the object audio data supplied from the demultiplexer 21 based on the reverb parameter supplied from the demultiplexer 21 to generate a Dry/Wet component signal, and performs the VBAP processing. It is supplied to the part 23.
[0134]
　Note that an example in which only the dry/wet component signal is generated in the reverb processing unit 141 will be described here for the sake of simplicity. However, in the case of the above-described first and second embodiments, Similarly to, the signals of the Wet components 1 to Wet components N as well as the Dry/Wet components may be generated.
[0135]
　In this example, the reverb processing unit 141 has a branch output unit 151, a pre-delay unit 152, a comb filter unit 153, an all-pass filter unit 154, an addition unit 155, and an addition unit 156. That is, the parametric reverb realized by the reverb processing unit 141 is composed of a plurality of constituent elements including a plurality of filters.
[0136]
　In particular, in the reverb processing unit 141, the branch output unit 151, the pre-delay unit 152, the comb filter unit 153, and the all-pass filter unit 154 are constituent elements that configure parametric reverb. Here, the component of parametric reverb refers to each process for realizing reverb processing by parametric reverb, that is, a processing block such as a filter that executes a part of the reverb processing.
[0137]
　Note that the parametric reverb configuration of the reverb processing unit 141 shown in FIG. 9 is merely an example, and any combination of parametric reverb component elements, parameters, and reconstruction method (reconstruction method) may be used. ..
[0138]
　The branch output unit 151 branches the object audio data supplied from the demultiplexer 21 to a branch number determined by the number of components of the generated signal such as the Dry component and the Wet component and the number of processes performed in parallel. , Adjust the gain of the branched signal.
[0139]
　In this example, the branch output unit 151 has an amplifier 171 and an amplifier 172, and the object audio data supplied to the branch output unit 151 is branched into two and supplied to the amplifier 171 and the amplifier 172. To be done.
[0140]
　The amplification unit 171 multiplies the object audio data supplied from the demultiplexer 21 by the gain value included in the reverb parameter to perform gain adjustment, and supplies the object audio data obtained as a result to the addition unit 156. To do. The signal (object audio data) output from the amplification unit 171 is a Dry component signal included in the Dry/Wet component signal.
[0141]
　The amplifying unit 172 multiplies the object audio data supplied from the demultiplexer 21 by the gain value included in the reverb parameter to adjust the gain, and outputs the resulting object audio data to the pre-delay unit 152. Supply. The signal (object audio data) output from the amplification unit 172 is a signal that is a basis of the Wet component included in the signal of the Dry/Wet component.
[0142]
　The pre-delay unit 152 performs a filtering process on the object audio data supplied from the amplification unit 172 to generate a signal of a component of a pseudo reflected sound or a reverberation sound which is a basic component, and the comb filter unit 153 and It is supplied to the adder 155.
[0143]
　The pre-delay unit 152 includes a pre-delay processing unit 181, amplification units 182-1 to 182-3, an addition unit 183, an addition unit 184, an amplification unit 185-1, and an amplification unit 185-2. Note that, hereinafter, the amplifiers 182-1 to 182-3 will be simply referred to as the amplifiers 182 unless it is necessary to distinguish them. Further, hereinafter, the amplifiers 185-1 and 185-2 are simply referred to as the amplifiers 185 unless it is necessary to distinguish them.
[0144]
　The pre-delay processing unit 181 delays the object audio data supplied from the amplification unit 172 for each output destination by the number of delay samples (delay time) included in the reverb parameter, and the amplification unit 182 that is the output destination. And to the amplifier 185.
[0145]
　The amplification unit 182-1 and the amplification unit 182-2 perform gain adjustment by multiplying the object audio data supplied from the pre-delay processing unit 181, by the gain value included in the reverb parameter, and the addition unit Supply to 183. The amplification unit 182-3 performs gain adjustment by multiplying the object audio data supplied from the pre-delay processing unit 181, by the gain value included in the reverb parameter, and supplies the object audio data to the addition unit 184.
[0146]
　The addition unit 183 adds the object audio data supplied from the amplification unit 182-1 and the object audio data supplied from the amplification unit 182-2, and supplies the addition result to the addition unit 184. The addition unit 184 adds the object audio data supplied from the addition unit 183 and the object audio data supplied from the amplification unit 182-3, and supplies the signal of the Wet component obtained as a result to the comb filter unit 153. To do.
[0147]
　In this way, the processing performed by the amplification unit 182, the addition unit 183, and the addition unit 184 is pre-delay filter processing, and the signal of the Wet component generated by this filter processing is, for example, a reflection signal other than the initial reflection sound. A signal of sound or reverberation.
[0148]
　The amplification unit 185-1 performs gain adjustment by multiplying the object audio data supplied from the pre-delay processing unit 181 by the gain value included in the reverb parameter, and the Wet component of the resulting Wet component is obtained. The signal is supplied to the adder 155.
[0149]
　Similarly, the amplification unit 185-2 performs gain adjustment by multiplying the object audio data supplied from the pre-delay processing unit 181 by the gain value included in the reverb parameter, and the result is obtained. The signal of the Wet component is supplied to the addition unit 155.
[0150]
　The processing performed by these amplification units 185 is a filter processing of initial reflection, and the signal of the Wet component generated by this filter processing is, for example, a signal of initial reflected sound.
[0151]
　The comb filter unit 153 is composed of a comb filter, and increases the density of the components of the reflected sound and the reverberation sound by performing a filtering process on the signal of the Wet component supplied from the addition unit 184.
[0152]
　In this example, the comb filter unit 153 is a comb filter with three columns and one stage. That is, the comb filter unit 153 includes the addition units 201-1 to 201-3, the delay units 202-1 to 202-3, the amplification units 203-1 to 203-3, and the amplification units 204-1 to 204-1. It has an amplification section 204-3, an addition section 205, and an addition section 206.
[0153]
　The signal of the Wet component is supplied from the adder 184 of the pre-delay unit 152 to the adders 201-1 to 201-3 of each column.
[0154]
　The adder 201-M (where 1≦M≦3) adds the signal of the Wet component supplied from the amplifier 203-M to the signal of the Wet component supplied from the adder 184, and delays the signal. Supply to 202-M. Note that, hereinafter, the addition units 201-1 to 201-3 are simply referred to as the addition unit 201 unless it is necessary to distinguish them.
[0155]
　The delay unit 202-M (where 1≦M≦3) delays the signal of the Wet component supplied from the addition unit 201-M by the delay sample number (delay time) included in the reverb parameter and amplifies it. It is supplied to the unit 203-M and the amplification unit 204-M. Note that, hereinafter, the delay units 202-1 to 202-3 are simply referred to as the delay unit 202 unless it is necessary to distinguish them.
[0156]
　The amplification unit 203-M (where 1≦M≦3) performs gain adjustment by multiplying the signal of the Wet component supplied from the delay unit 202-M by the gain value included in the reverb parameter. Then, the result is supplied to the addition unit 201-M. Note that, hereinafter, the amplifiers 203-1 to 203-3 are also simply referred to as the amplifiers 203 unless it is necessary to distinguish them.
[0157]
　The amplification unit 204-1 and the amplification unit 204-2 multiply the signal of the Wet component supplied from the delay unit 202-1 and the delay unit 202-2 by the gain value included in the reverb parameter. The gain is adjusted and supplied to the addition unit 205.
[0158]
　Further, the amplification unit 204-3 performs gain adjustment by multiplying the signal of the Wet component supplied from the delay unit 202-3 by the gain value included in the reverb parameter, and supplies the signal to the addition unit 206. To do. Note that, hereinafter, the amplifiers 204-1 to 204-3 are simply referred to as the amplifiers 204 unless it is necessary to distinguish them.
[0159]
　The addition unit 205 adds the signal of the Wet component supplied from the amplification unit 204-1 and the signal of the Wet component supplied from the amplification unit 204-2, and supplies the signal to the addition unit 206.
[0160]
　The addition unit 206 adds the signal of the Wet component supplied from the amplification unit 204-3 and the signal of the Wet component supplied from the addition unit 205, and outputs the signal of the Wet component obtained as a result of the comb filter output. Is supplied to the all-pass filter section 154.
[0161]
　In the comb filter unit 153, the adding unit 201-1 to the amplifying unit 204-1 are the constituent elements of the first column and the first stage of the comb filter, and the adding unit 201-2 to the amplifying unit 204-2 are the two components of the comb filter. It is a constituent element in the first row of the column, and the adding section 201-3 to the amplifying section 204-3 are constituent elements in the first row of the third column of the comb filter.
[0162]
　The all-pass filter unit 154 includes an all-pass filter, and increases the density of the reflected sound and reverberant sound components by performing a filtering process on the signal of the Wet component supplied from the addition unit 206.
[0163]
　In this example, the all-pass filter unit 154 is an all-pass filter with two columns in one row. That is, the all-pass filter unit 154 has an adder 221, a delay unit 222, an amplifier 223, an amplifier 224, an adder 225, a delay unit 226, an amplifier 227, an amplifier 228, and an adder 229.
[0164]
　The addition unit 221 adds the signal of the Wet component supplied from the addition unit 206 and the signal of the Wet component supplied from the amplification unit 223, and supplies the signal to the delay unit 222 and the amplification unit 224.
[0165]
　The delay unit 222 delays the signal of the Wet component supplied from the addition unit 221 by the number of delay samples (delay time) included in the reverb parameter, and supplies the signal to the amplification unit 223 and the addition unit 225.
[0166]
　The amplification unit 223 performs gain adjustment by multiplying the signal of the Wet component supplied from the delay unit 222 by the gain value included in the reverb parameter, and supplies the signal to the addition unit 221. The amplification unit 224 performs gain adjustment by multiplying the signal of the Wet component supplied from the addition unit 221 by the gain value included in the reverb parameter, and supplies the signal to the addition unit 225.
[0167]
　The addition unit 225 adds the signal of the Wet component supplied from the delay unit 222, the signal of the Wet component supplied from the amplification unit 224, and the signal of the Wet component supplied from the amplification unit 227, and the delay unit 226. And to the amplifier 228.
[0168]
　In the all-pass filter unit 154, the addition units 221 to 225 are constituent elements in the first row and first stage of the all-pass filter.
[0169]
　The delay unit 226 delays the signal of the Wet component supplied from the addition unit 225 by the number of delay samples (delay time) included in the reverb parameter, and supplies the signal to the amplification unit 227 and the addition unit 229.
[0170]
　The amplification unit 227 performs gain adjustment by multiplying the signal of the Wet component supplied from the delay unit 226 by the gain value included in the reverb parameter, and supplies the signal to the addition unit 225. The amplification unit 228 performs gain adjustment by multiplying the signal of the Wet component supplied from the addition unit 225 by the gain value included in the reverb parameter, and supplies the signal to the addition unit 229.
[0171]
　The addition unit 229 adds the signal of the Wet component supplied from the delay unit 226 and the signal of the Wet component supplied from the amplification unit 228, and adds the resulting signal of the Wet component as the output of the all-pass filter. Supply to the section 156.
[0172]
　In the all-pass filter unit 154, the addition units 225 to 229 are the constituent elements in the first row and second stage of the all-pass filter.
[0173]
　The addition unit 155 adds the signal of the Wet component supplied from the amplification unit 185-1 of the pre-delay unit 152 and the signal of the Wet component supplied from the amplification unit 185-2, and supplies the signal to the addition unit 156. The adder 156 adds the object audio data supplied from the amplifier 171 of the branch output unit 151, the Wet component signal supplied from the adder 229, and the Wet component signal supplied from the adder 155. The signal obtained as a result is supplied to the VBAP processing unit 23 as a signal of Dry/Wet component.
[0174]
　As described above, the configuration of the reverb processing unit 141 shown in FIG. 9, that is, the parametric reverb is merely an example, and if the configuration is made up of a plurality of constituent elements including one or a plurality of filters, what kind of configuration will be obtained? May be done. For example, the parametric reverb can be configured by combining the respective constituent elements shown in FIG.
[0175]
　In particular, each component provides configuration information indicating the configuration of the component and coefficient information (parameter) indicating the gain value, delay time, etc. used in the processing of the blocks configuring the component, thereby making the object audio It can be reconstructed (reproduced) on the data reproducing side. In other words, if the reproducing side is provided with information indicating what kind of constituent elements the parametric reverb is composed of and the constituent information and coefficient information about each constituent element, the reproducing side reproduces the parametric reverb. Can be built.
[0176]
　In the example illustrated in FIG. 10, the constituent element indicated by the character “Branch” is the constituent element of the branch corresponding to the branch output unit 151 in FIG. 9. This component can be reconstructed by the number of signal branch lines as the configuration information and the gain value in each amplification unit as the coefficient information.
[0177]
　For example, in the example shown in FIG. 9, the number of branch lines of the branch output unit 151 is 2, and the gain value used by each of the amplification unit 171 and the amplification unit 172 is the gain value of the coefficient information.
[0178]
　The component indicated by the character “PreDelay” is the pre-delay corresponding to the pre-delay unit 152 in FIG. This component can be reconstructed by the number of pre-delay taps and the number of initial reflection taps as the configuration information, and the delay time of each signal and the gain value of each amplification unit as the coefficient information.
[0179]
　For example, in the example shown in FIG. 9, the number of pre-delay taps is “3”, which is the number of amplification units 182, and the number of initial reflection taps is “2,” which is the number of amplification units 185. Further, the number of delay samples of the signal output to each amplification unit 182 and amplification unit 185 in the pre-delay processing unit 181 is the delay time of the coefficient information, and the gain value used in the amplification unit 182 and amplification unit 185 is the coefficient information. Is the gain value of.
[0180]
　The component indicated by the letters "Multi Tap Delay" is a duplicate of the basic reflected sound and reverberant sound components generated by the pre-delay section, and more reflected sound and reverberant sound components (Wet component signals). ) Is a multi-tap delay, that is, a filter. This component can be reconstructed by the number of multi-tap as the configuration information, the delay time of each signal as the coefficient information, and the gain value in each amplification unit. Here, the number of multi-tap indicates the number when the signal of the Wet component is duplicated, that is, the number of signals of the Wet component after the duplicate.
[0181]
　The component indicated by the character "All Pass Filters" is an all pass filter corresponding to the all pass filter unit 154 in FIG. This component can be reconstructed by the number of all-pass filter lines (number of columns) and the number of all-pass filter stages as the configuration information, and the delay time of each signal and the gain value in each amplification unit as the coefficient information.
[0182]
　For example, in the example shown in FIG. 9, the number of all-pass filter lines is "1" and the number of all-pass filter stages is "2". Further, the number of delay samples of the signal in the delay unit 222 and the delay unit 226 in the all-pass filter unit 154 is the delay time of the coefficient information, and the gain value used in the amplification unit 223, the amplification unit 224, the amplification unit 227, and the amplification unit 228. Is the gain value of the coefficient information.
[0183]
　The component indicated by the letters "Comb Filters" is a comb filter corresponding to the comb filter unit 153 in FIG. This component can be reconstructed by the number of comb filter lines (number of columns) and the number of comb filter stages as the configuration information, and the delay time of each signal and the gain value in each amplification unit as the coefficient information.
[0184]
　For example, in the example shown in FIG. 9, the number of comb filter lines is “3” and the number of comb filter stages is “1”. Further, the number of delay samples of the signal in the delay unit 202 in the comb filter unit 153 is the delay time of the coefficient information, and the gain value used in the amplification unit 203 and the amplification unit 204 is the gain value of the coefficient information.
[0185]
　The component indicated by the letters "High Cut Filter" is a high cut filter. This component does not require configuration information and can be reconstructed by the gain value in each amplification unit as coefficient information.
[0186]
　As described above, the parametric reverb can be configured by arbitrarily combining the components shown in FIG. 10 with the configuration information and coefficient information about these components. Therefore, the configuration of the reverb processing unit 141 can also be configured by arbitrarily combining these components with the configuration information and the coefficient information.
[0187]

　Next, when the reverb processing unit 141 is configured by parametric reverb, the meta information (reverb parameter) supplied to the reverb processing unit 141 will be described. In such a case, the syntax of the meta information is as shown in FIG. 11, for example.
[0188]
　In the example shown in FIG. 11, the meta information includes Reverb_Configuration() and Reverb_Parameter(). Here, Reverb_Configuration() includes the above-described Wet component position information and configuration information of the constituent elements of parametric reverb, and Reverb_Parameter() includes coefficient information of the constituent elements of parametric reverb.
[0189]
　In other words, Reverb_Configuration() includes information indicating the localization position of the sound image of each Wet component (reverb component) and configuration information indicating the configuration of parametric reverb. Further, Reverb_Parameter() includes parameters used in the processing by the components of parametric reverb as coefficient information.
[0190]
　Hereinafter, Reverb_Configuration() and Reverb_Parameter() will be further described.
[0191]
　The syntax of Reverb_Configuration() is as shown in FIG. 12, for example.
[0192]
　In the example shown in FIG. 12, Reverb_Configuration() includes localization mode information wet_position_mode and the number of outputs number_of_wet_outputs. Since the localization mode information wet_position_mode and the number of outputs number_of_wet_outputs are the same as those shown in FIG. 7, the description thereof will be omitted.
[0193]
　Further, when the value of the localization mode information wet_position_mode is “0”, Reverb_Configuration() includes horizontal angle wet_position_azimuth_offset[i] and vertical angle wet_position_elevation_offset[i] as Wet component position information. On the other hand, when the value of the localization mode information wet_position_mode is “1”, the horizontal angle wet_position_azimuth[i] and the vertical angle wet_position_elevation[i] are included as the Wet component position information.
[0194]
　Since the horizontal angle wet_position_azimuth_offset[i], the vertical angle wet_position_elevation_offset[i], the horizontal angle wet_position_azimuth[i], and the vertical angle wet_position_elevation[i] are the same as those shown in FIG. 7, the description thereof will be omitted. ..
[0195]
　Further, Reverb_Configuration() includes Reverb_Structure() in which the configuration information of each component of parametric reverb is stored.
[0196]
　The syntax of this Reverb_Structure() is as shown in FIG. 13, for example.
[0197]
　In the example shown in FIG. 13, Reverb_Structure() stores information on the constituent element indicated by the element ID (elem_id[]).
[0198]
　For example, the value “0” of elem_id[] indicates a branch component (BRANCH), the value “1” of elem_id[] indicates a pre-delay (PRE_DELAY), and the value “2” of elem_id[] indicates. It indicates an all-pass filter (ALL_PASS_FILTER), and the value "3" of elem_id[] indicates a multi-tap delay (MULTI_TAP_DELAY).
[0199]
　Further, the value “4” of elem_id[] indicates a comb filter (COMB_FILTER), the value “5” of elem_id[] indicates a high-frequency cut filter (HIGH_CUT), and the value “6” of elem_id[]. Indicates the end of the loop (TERM), and the value "7" of elem_id[] indicates the end of the loop (OUTPUT).
[0200]
　Specifically, for example, when the value of elem_id[] is "0", Branch_Configuration(n) which is the configuration information of the branch component is stored, and when the value of elem_id[] is "1". PreDelay_Configuration(), which is predelay configuration information, is stored.
[0201]
　When the value of elem_id[] is “2”, AllPassFilter_Configuration() which is the configuration information of the all-pass filter is stored, and when the value of elem_id[] is “3”, the configuration information of the multi-tap delay. MultiTapDelay_Configuration() is stored.
[0202]
　Further, when the value of elem_id[] is “4”, CombFilter_Configuration() which is the configuration information of the comb filter is stored, and when the value of elem_id[] is “5”, the configuration of the high frequency cut filter is stored. HighCut_Configuration() which is information is stored.
[0203]
　Subsequently, Branch_Configuration(n), PreDelay_Configuration(), AllPassFilter_Configuration(), MultiTapDelay_Configuration(), CombFilter_Configuration(), and HighCut_Configuration() that store configuration information will be further described.
[0204]
　For example, the syntax of Branch_Configuration(n) is as shown in FIG.
[0205]
　In this example, Branch_Configuration(n) stores the number of branch lines indicated by the characters “number_of_lines” as the configuration information of the branch components, and Reverb_Structure() is further stored for each branch line.
[0206]
　The syntax of PreDelay_Configuration() shown in FIG. 13 is as shown in FIG. 15, for example. In this example, PreDelay_Configuration() contains the number of pre-delay taps (pre-delay number) indicated by the letters "number_of_predelays" as the pre-delay configuration information and the number of initial reflection taps (number of initial reflections) indicated by the letter "number_of_early reflections". And are stored.
[0207]
　The syntax of MultiTapDelay_Configuration() shown in FIG. 13 is as shown in FIG. 16, for example. In this example, MultiTapDelay_Configuration() stores the multi-tap number indicated by the character “number_of_taps” as the configuration information of the multi-tap delay.
[0208]
　Furthermore, the syntax of AllPassFilter_Configuration() shown in FIG. 13 is as shown in FIG. 17, for example. In this example, the number of all-pass filter lines indicated by the character “number_of_apf_lines” and the number of all-pass filter steps indicated by the character “number_of_apf_sections” are stored in AllPassFilter_Configuration() as the configuration information of the all-pass filter.
[0209]
　The syntax of CombFilter_Configuration() shown in FIG. 13 is as shown in FIG. 18, for example. In this example, in CombFilter_Configuration(), the number of comb filter lines indicated by the characters “number_of_comb_lines” and the number of comb filter stages indicated by the characters “number_of_comb_sections” are stored as the comb filter configuration information.
[0210]
　The syntax of HighCut_Configuration() shown in FIG. 13 is as shown in FIG. 19, for example. In this example, HighCut_Configuration() does not include any particular configuration information.
[0211]
　Further, the syntax of Reverb_Parameter() shown in FIG. 11 is as shown in FIG. 20, for example.
[0212]
　In the example shown in FIG. 20, Reverb_Parameter() stores the coefficient information of the component indicated by the element ID (elem_id[]). Note that elem_id[] in FIG. 20 is the one indicated by the above Reverb_Configuration().
[0213]
　For example, if the value of elem_id[] is “0”, Branch_Parameters(n), which is the coefficient information of the constituent elements of the branch, is stored, and if the value of elem_id[] is “1”, the coefficient of pre-delay PreDelay_Parameters() which is information is stored.
[0214]
　When the value of elem_id[] is “2”, AllPassFilter_Parameters() which is the coefficient information of the all-pass filter is stored. When the value of elem_id[] is “3”, the coefficient information of the multi-tap delay is stored. MultiTapDelay_Parameters() is stored.
[0215]
　Further, when the value of elem_id[] is "4", CombFilter_Parameters() which is the coefficient information of the comb filter is stored, and when the value of elem_id[] is "5", the coefficient of the high frequency cut filter is stored. HighCut_Parameters() which is information is stored.
[0216]
　Here, Branch_Parameters(n), PreDelay_Parameters(), AllPassFilter_Parameters(), MultiTapDelay_Parameters(), CombFilter_Parameters(), and HighCut_Parameters() in which coefficient information is stored will be further described.
[0217]
　The syntax of Branch_Parameters(n) shown in FIG. 20 is as shown in FIG. 21, for example. In this example, Branch_Parameters(n) stores the gain value gain[i] for the number of branch lines number_of_lines as the coefficient information of the constituent elements of the branch, and Reverb_Parameters(n) is further stored for each branch line. ing.
[0218]
　Here, the gain value gain[i] indicates the gain value used in the amplification unit provided in the i-th branch line. For example, in the example of FIG. 9, the gain value gain[0] is the gain value used in the amplification unit 171 provided on the 0th branch line, that is, the first branch line, and the gain value gain[1] is It is a gain value used in the amplification section 172 provided in the branch line of the second column.
[0219]
　Further, the syntax of PreDelay_Parameters() shown in FIG. 20 is as shown in FIG. 22, for example.
[0220]
　In the example shown in FIG. 22, PreDelay_Parameters() includes predelay tap coefficient number_of_predelays as predelay coefficient information, and predelay_sample[i] of predelay delay samples and predelay gain value predelay_gain[i]. It is stored.
[0221]
　Here, the number of delay samples predelay_sample[i] indicates the number of delay samples for the i-th pre-delay, and the gain value predelay_gain[i] indicates the gain value for the i-th pre-delay. For example, in the example of FIG. 9, the delay sample number predelay_sample[0] is the 0th predelay, that is, the number of delay samples of the signal of the Wet component supplied to the amplification unit 182-1, and the gain value predelay_gain[0] is , A gain value used in the amplification unit 182-1.
[0222]
　In PreDelay_Parameters(), the number of early reflection delay samples earlyref_sample[i] corresponding to the number of initial reflection taps number_of_earlyreflections and the initial reflection gain value earlyref_gain[i] are stored.
[0223]
　Here, the number of delay samples earlyref_sample[i] indicates the number of delay samples for the i-th initial reflection, and the gain value earlyref_gain[i] indicates the gain value for the i-th initial reflection. For example, in the example of FIG. 9, the number of delay samples earlyref_sample[0] is the number of delay samples of the 0th initial reflection, that is, the signal of the Wet component supplied to the amplification section 185-1, and the gain value earlyref_gain[0] is , A gain value used in the amplification section 185-1.
[0224]
　Further, the syntax of MultiTapDelay_Parameters() shown in FIG. 20 is as shown in FIG. 23, for example.
[0225]
　In the example illustrated in FIG. 23, MultiTapDelay_Parameters() has multi-tap delay coefficient information delay_sample[i] and multi-tap delay gain value delay_gain[i] as multi-tap number number_of_taps as coefficient information of the multi-tap delay. And are stored. Here, the delay sample number delay_sample[i] indicates the delay sample number for the i-th delay, and the gain value delay_gain[i] indicates the gain value for the i-th delay.
[0226]
　The syntax of HighCut_Parameters() shown in FIG. 20 is as shown in FIG. 24, for example.
[0227]
　In the example shown in FIG. 24, HighCut_Parameters() stores the gain value gain of the high frequency cut filter as the coefficient information of the high frequency cut filter.
[0228]
　Further, the syntax of AllPassFilter_Parameters() shown in FIG. 20 is as shown in FIG. 25, for example.
[0229]
　In the example shown in FIG. 25, allPassFilter_Parameters() has the allpass filter line number number_of_apf_lines as the coefficient information of the allpass filter, and the delay sample number delay_sample[i][j] of each stage of the allpass filter stage number_of_apf_sections. The gain values ​​gain[i][j] are stored.
[0230]
　Here, the number of delay samples delay_sample[i][j] indicates the number of delay samples in the j-th stage of the i-th column (line) of the all-pass filter, and the gain value gain[i][j] is It is a gain value used in the amplification section of the j-th stage of the i-th column (line) of the all-pass filter.
[0231]
　For example, in the example of FIG. 9, the delay sample number delay_sample[0][0] is the delay sample number in the delay unit 222 at the 0th stage of the 0th column, and the gain value gain[0][0] is 0. It is a gain value used in the amplification section 223 and the amplification section 224 in the 0th stage of the 2nd column. More specifically, the gain value used in the amplification unit 223 and the gain value used in the amplification unit 224 have the same magnitude but different signs.
[0232]
　The syntax of CombFilter_Parameters() shown in FIG. 20 is as shown in FIG. 26, for example.
[0233]
　In the example shown in FIG. 26, as comb filter coefficient information in CombFilter_Parameters(), for each line corresponding to the number of comb filter lines number_of_comb_lines, the delay sample number delay_sample[i][j] of each stage corresponding to the number of comb filter stages number_of_comb_sections The gain value gain_a[i][j] and the gain value gain_b[i][j] are stored.
[0234]
　Here, the number of delay samples delay_sample[i][j] indicates the number of delay samples in the j-th stage of the i-th column (line) of the comb filter, and the gain value gain_a[i][j] and gain The value gain_b[i][j] is a gain value used in the amplification unit of the j-th stage of the i-th column (line) of the comb filter.
[0235]
　For example, in the example of FIG. 9, the delay sample number delay_sample[0][0] is the number of delay samples in the delay unit 202-1 in the 0th stage of the 0th column. Further, the gain value gain_a[0][0] is a gain value used in the amplification unit 203-1 in the 0th stage of the 0th column, and the gain value gain_b[0][0] is the 0th column. It is a gain value used in the amplification section 204-1 in the 0th stage of.
[0236]
　When the parametric reverb of the reverb processing unit 141 is reconstructed (reconstructed) by the above meta information, the meta information becomes as shown in FIG. 27, for example. Here, the coefficient value in Reverb_Parameters() is expressed as an integer X and a floating point number XX, but actually the value set according to the reverb parameter used is entered.
[0237]
　In the example shown in FIG. 27, the value “2” of the branch line number number_of_lines in the branch output unit 151 is stored in the Branch_Configuration() part.
[0238]
　In the PreDelay_Configuration() part, the value “3” of the pre-delay tap number number_of_predelays in the pre-delay unit 152 and the value “2” of the initial reflection tap number number_of_earlyreflections are stored.
[0239]
　In the CombFilter_Configuration() part, the value “3” of the number of comb filter lines number_of_comb_lines in the comb filter unit 153 and the value “1” of the number of comb filter stages number_of_comb_sections are stored.
[0240]
　Further, in the AllPassFilter_Configuration() part, the value “1” of the allpass filter line number number_of_apf_lines in the allpass filter unit 154 and the value “2” of the allpass filter stage number_of_apf_sections are stored.
[0241]
　In addition, in the Branch Parameter(0) portion of Reverb_Parameter(0), the gain value gain[0] used in the amplification unit 171 of the 0th branch line of the branch output unit 151 is stored, and Reverb_Parameter(1) The gain value gain[1] used in the amplification unit 172 of the first branch line of the branch output unit 151 is stored in the portion of.
[0242]
　The PreDelay_Parameters() portion stores the number of predelay delay samples predelay_sample[0], the number of delay samples predelay_sample[1], and the number of delay samples predelay_sample[2] of the predelay processing unit 181 of the predelay unit 152. There is.
[0243]
　Here, the predelay processing unit 181 supplies the delay sample number predelay_sample[0], the delay sample number predelay_sample[1], and the delay sample number predelay_sample[2] to the amplification units 182-1 to 182-3, respectively. This is the delay time of the Wet component signal.
[0244]
　The PreDelay_Parameters() portion also stores the gain value predelay_gain[0], the gain value predelay_gain[1], and the gain value predelay_gain[2] used in each of the amplification units 182-1 to 182-3. ing.
[0245]
　The PreDelay_Parameters() section stores the number of early reflection delay samples earlyref_sample[0] and the number of delay samples earlyref_sample[1] of the pre-delay processing unit 181 of the pre-delay unit 152.
[0246]
　The number of delayed samples earlyref_sample[0] and the number of delayed samples earlyref_sample[1] are the delay times of the Wet component signals supplied to the amplification units 185-1 and 185-2 by the pre-delay processing unit 181 respectively.
[0247]
　Further, the PreDelay_Parameters() portion also stores the gain value earlyref_gain[0] and the gain value earlyref_gain[1] used in the amplification section 185-1 and the amplification section 185-2, respectively.
[0248]
　The CombFilter_Parameters() part includes the number of delay samples delay_sample[0][0] in the delay unit 202-1, the gain value gain_a[0][0] for obtaining the gain value used in the amplification unit 203-1, and The gain value gain_b[0][0] for obtaining the gain value used in the amplification unit 204-1 is stored.
[0249]
　Further, in the CombFilter_Parameters() part, the number of delay samples delay_sample[1][0] in the delay unit 202-2 and the gain value gain_a[1][0] for obtaining the gain value used in the amplification unit 203-2. , And gain value gain_b[1][0] for obtaining the gain value used in the amplification unit 204-2 are stored.
[0250]
　Further, in the CombFilter_Parameters() portion, the delay sample number delay_sample[2][0] in the delay unit 202-3 and the gain value gain_a[2][0] for obtaining the gain value used in the amplification unit 203-3. , And gain value gain_b[2][0] for obtaining the gain value used in the amplification unit 204-3 are stored.
[0251]
　The AllPassFilter_Parameters() part stores the number of delay samples delay_sample[0][0] in the delay unit 222, and a gain value gain[0][0] for obtaining the gain value used in the amplification unit 223 and the amplification unit 224. Has been done.
[0252]
　In the AllPassFilter_Parameters() part, the number of delay samples delay_sample[0][1] in the delay unit 226 and the gain value gain[0][1] for obtaining the gain values ​​used in the amplification units 227 and 228. Is stored.
[0253]
　On the reproduction side (the signal processing device 131 side), the configuration of the reverb processing unit 141 can be reconstructed based on the configuration information and the coefficient information of each of the above components.
[0254]

　Next, the operation of the signal processing device 131 shown in FIG. 9 will be described. That is, the audio signal output processing by the signal processing device 131 will be described below with reference to the flowchart in FIG.
[0255]
　Note that the process of step S71 is similar to the process of step S11 in FIG. 5, so description thereof will be omitted. However, in step S71, the demultiplexer 21 reads the reverb parameter shown in FIG. 27 from the bitstream and supplies it to the reverb processor 141 and the VBAP processor 23.
[0256]
　In step S72, the branch output unit 151 performs branch output processing on the object audio data supplied from the demultiplexer 21.
[0257]
　That is, the amplification unit 171 and the amplification unit 172 perform gain adjustment of the object audio data based on the supplied gain value, and supply the resulting object audio data to the addition unit 156 and the pre-delay processing unit 181.
[0258]
　In step S73, the pre-delay unit 152 performs pre-delay processing on the object audio data supplied from the amplification unit 172.
[0259]
　That is, the pre-delay processing unit 181 delays the object audio data supplied from the amplification unit 172 by the number of delay samples according to the output destination, and then supplies the object audio data to the amplification units 182 and 185.
[0260]
　The amplification unit 182 adjusts the gain of the object audio data supplied from the pre-delay processing unit 181, based on the supplied gain value, and supplies the object audio data to the addition unit 183 or the addition unit 184. Performs addition processing of the supplied object audio data. When the Wet component signal is obtained in this way, the addition unit 184 supplies the obtained Wet component signal to the addition unit 201 of the comb filter unit 153.
[0261]
　Further, the amplification unit 185 performs gain adjustment on the object audio data supplied from the pre-delay processing unit 181, based on the supplied gain value, and supplies the signal of the Wet component obtained as a result to the addition unit 155.
[0262]
　In step S74, the comb filter unit 153 performs comb filter processing.
[0263]
　That is, the addition unit 201 adds the signal of the Wet component supplied from the addition unit 184 and the signal of the Wet component supplied from the amplification unit 203 and supplies the signal to the delay unit 202. The delay unit 202 delays the signal of the Wet component supplied from the addition unit 201 by the number of supplied delay samples, and then supplies the signal to the amplification unit 203 and the amplification unit 204.
[0264]
　The amplification unit 203 adjusts the gain of the signal of the Wet component supplied from the delay unit 202 based on the supplied gain value, and supplies the signal to the addition unit 201. The amplification unit 204 supplies the Wet component supplied from the delay unit 202. The signal is adjusted in gain based on the supplied gain value and supplied to the adder 205 or the adder 206. The addition unit 205 and the addition unit 206 perform addition processing of the supplied Wet component signal, and the addition unit 206 supplies the obtained Wet component signal to the addition unit 221 of the all-pass filter unit 154.
[0265]
　In step S75, the all-pass filter unit 154 performs all-pass filter processing. That is, the addition unit 221 adds the signal of the Wet component supplied from the addition unit 206 and the signal of the Wet component supplied from the amplification unit 223, and supplies the signal to the delay unit 222 and the amplification unit 224.
[0266]
　The delay unit 222 delays the signal of the Wet component supplied from the addition unit 221 by the number of supplied delay samples, and then supplies the signal to the amplification unit 223 and the addition unit 225.
[0267]
　The amplification unit 224 adjusts the gain of the signal of the Wet component supplied from the addition unit 221 based on the supplied gain value, and supplies the signal to the addition unit 225. The amplification unit 223 performs gain adjustment of the signal of the Wet component supplied from the delay unit 222 based on the supplied gain value, and supplies the signal to the addition unit 221.
[0268]
　The addition unit 225 adds the signal of the Wet component supplied from the delay unit 222, the signal of the Wet component supplied from the amplification unit 224, and the signal of the Wet component supplied from the amplification unit 227, and the delay unit 226. And to the amplifier 228.
[0269]
　Further, the delay unit 226 delays the signal of the Wet component supplied from the addition unit 225 by the number of supplied delay samples, and then supplies the signal to the amplification unit 227 and the addition unit 229.
[0270]
　The amplification unit 228 performs gain adjustment on the signal of the Wet component supplied from the addition unit 225 based on the supplied gain value, and supplies the signal to the addition unit 229. The amplification unit 227 performs gain adjustment on the signal of the Wet component supplied from the delay unit 226 based on the supplied gain value, and supplies the signal to the addition unit 225. The addition unit 229 adds the signal of the Wet component supplied from the delay unit 226 and the signal of the Wet component supplied from the amplification unit 228, and supplies the signal to the addition unit 156.
[0271]
　In step S76, the addition unit 156 generates a Dry/Wet component signal.
[0272]
　That is, the addition unit 155 adds the signals of the Wet components supplied from the amplification units 185-1 and 185-2 and supplies the signals to the addition unit 156. The addition unit 156 adds the object audio data supplied from the amplification unit 171, the Wet component signal supplied from the addition unit 229, and the Wet component signal supplied from the addition unit 155, and the result is obtained. The supplied signal is supplied to the VBAP processing unit 23 as a signal of Dry/Wet component.
[0273]
　After the process of step S76 is performed, the process of step S77 is performed and the audio signal output process ends, but the process of step S77 is similar to the process of step S13 of FIG. ..
[0274]
　As described above, the signal processing device 131 performs the reverb process on the object audio data based on the reverb parameter including the configuration information and the coefficient information to generate the Dry/Wet component.
[0275]
　By doing so, it is possible to more effectively realize the sense of distance control on the reproduction side of the object audio data. Particularly, by performing the reverb processing using the reverb parameter including the configuration information and the coefficient information, it is possible to improve the coding efficiency as compared with the case where the impulse response is used as the reverb parameter.
[0276]
　The method described in the third embodiment has been shown to use parametric reverb configuration information and coefficient information as meta information. In other words, it can be said that the parametric reverb can be reconstructed based on the meta information. That is, the parametric reverb used during content production can be reconstructed on the reproducing side based on the meta information.
[0277]
　In particular, according to this method, reverb processing can be applied by an algorithm of any configuration on the content production side. In addition, the sense of distance can be controlled with a relatively small amount of meta information. Then, in rendering on the playback side, reverb processing according to the meta information is performed on the audio object, so that the sense of distance as intended by the content creator can be reproduced. Note that the encoding device generates a bitstream in which the meta information and the position information shown in FIG. 11 and the encoded object audio data are stored.
[0278]

　As described above, the configuration of the parametric reverb can be any configuration. That is, various reverberation algorithms can be configured by combining other arbitrary components.
[0279]
　For example, the parametric reverb can be configured by combining the branch component, the pre-delay, the multi-tap delay, and the all-pass filter.
[0280]
　In such a case, the signal processing device is configured as shown in FIG. 29, for example. 29, parts corresponding to those in FIG. 1 are designated by the same reference numerals, and the description thereof will be omitted as appropriate.
[0281]
　The signal processing device 251 shown in FIG. 29 includes a demultiplexer 21, a reverb processing unit 261, and a VBAP processing unit 23.
[0282]
　The configuration of the signal processing device 251 differs from the configuration of the signal processing device 11 in that a reverb processing unit 261 is provided in place of the reverb processing unit 22 of the signal processing device 11 in FIG. The configuration is similar to that of the device 11.
[0283]
　The reverb processing unit 261 generates a signal of Dry/Wet component by performing reverb processing on the object audio data supplied from the demultiplexer 21 based on the reverb parameter supplied from the demultiplexer 21, and performs the VBAP processing. It is supplied to the part 23.
[0284]
　In this example, the reverb processing unit 261 includes a branch output unit 271, a pre-delay unit 272, a multi-tap delay unit 273, an all-pass filter unit 274, an addition unit 275, and an addition unit 276.
[0285]
　The branch output unit 271 branches the object audio data supplied from the demultiplexer 21 to perform gain adjustment, and supplies the object audio data to the addition unit 276 and the pre-delay unit 272. In this example, the number of branch lines of the branch output unit 271 is 2.
[0286]
　The pre-delay unit 272 performs the same pre-delay processing as that in the pre-delay unit 152 on the object audio data supplied from the branch output unit 271, and adds the obtained signal of the Wet component to the addition unit 275 and the multi-tap. It is supplied to the delay unit 273. In this example, the number of pre-delay taps and the number of initial reflection taps in the pre-delay unit 272 are two.
[0287]
　The multi-tap delay unit 273 delays and branches the signal of the Wet component supplied from the pre-delay unit 272, adjusts the gain, and adds the signals of the Wet component obtained as a result to obtain one signal. After that, it is supplied to the all-pass filter unit 274. Here, the number of multi-tap of the multi-tap delay unit 273 is 5.
[0288]
　The all-pass filter unit 274 performs the same all-pass filter processing on the Wet component signal supplied from the multi-tap delay unit 273 as in the case of the all-pass filter unit 154, and outputs the obtained Wet component signal to the addition unit 276. Supply. Here, the all-pass filter unit 274 is an all-pass filter with two rows and two stages.
[0289]
　The adding unit 275 adds the two Wet component signals supplied from the pre-delay unit 272 and supplies the result to the adding unit 276. The addition unit 276 adds the object audio data supplied from the branch output unit 271, the signal of the Wet component supplied from the all-pass filter unit 274, and the signal of the Wet component supplied from the addition unit 275, and obtains the result. The supplied signal is supplied to the VBAP processing unit 23 as a signal of Dry/Wet component.
[0290]
　When the reverb processing unit 261 has the configuration shown in FIG. 29, the reverb processing unit 261 is supplied with the meta information (reverb parameter) shown in FIG. 30, for example.
[0291]
　In the example shown in FIG. 30, the meta information stores number_of_lines, number_of_predelays, number_of_earlyreflections, number_of_taps, number_of_apf_lines, and number_of_apf_sections as configuration information.
[0292]
　In addition, as meta information, as coefficient information, gain[0] and gain[1] of branch components, predelay_sample[0] of predelay, predelay_gain[0], predelay_sample[1], predelay_gain[1], initial Reflection earlyref_sample[0], earlyref_gain[0], earlyref_sample[1], earlyref_gain[1] are stored.
[0293]
　Further, as coefficient information, delay_sample[0], delay_gain[0], delay_sample[1], delay_gain[1], delay_sample[2], delay_gain[2], delay_sample[3], delay_gain[3], delay_sample of multi-tap delay are used as coefficient information. [4], delay_gain[4], allpass filter delay_sample[0][0], gain[0][0], delay_sample[0][1], gain[0][1], delay_sample[1][0 ], gain[1][0], delay_sample[1][1], gain[1][1] are stored.
[0294]
　As described above, according to the present technology, in object-based audio rendering, more effective sense-of-distance control based on meta information can be realized.
[0295]
　Particularly, according to the first embodiment and the third embodiment, it is possible to realize the sense of distance control with relatively few parameters.
[0296]
　Further, according to the second embodiment and the third embodiment, it is possible to add reverberation as desired and intended by the creator in content creation. That is, the reverb processing can be selected without being restricted by the algorithm.
[0297]
　Furthermore, according to the third embodiment, it is possible to reproduce the reverb effect as desired and intended by the content creator in rendering object-based audio without using a huge impulse response.
[0298]

　By the way, the series of processes described above can be executed by hardware or software. When the series of processes is executed by software, a program forming the software is installed in the computer. Here, the computer includes a computer incorporated in dedicated hardware and, for example, a general-purpose personal computer capable of executing various functions by installing various programs.
[0299]
　FIG. 31 is a block diagram showing a configuration example of hardware of a computer that executes the series of processes described above by a program.
[0300]
　In a computer, a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503 are connected to each other by a bus 504.
[0301]
　An input/output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a recording unit 508, a communication unit 509, and a drive 510 are connected to the input/output interface 505.
[0302]
　The input unit 506 includes a keyboard, a mouse, a microphone, an image sensor, and the like. The output unit 507 includes a display, a speaker and the like. The recording unit 508 includes a hard disk, a non-volatile memory, or the like. The communication unit 509 includes a network interface or the like. The drive 510 drives a removable recording medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory.
[0303]
　In the computer configured as described above, the CPU 501 loads the program recorded in the recording unit 508 into the RAM 503 via the input/output interface 505 and the bus 504 and executes the program, thereby performing the above-described series of operations. Is processed.
[0304]
　The program executed by the computer (CPU 501) can be provided, for example, by recording it on a removable recording medium 511 such as a package medium. Further, the program can be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital satellite broadcasting.
[0305]
　In the computer, the program can be installed in the recording unit 508 via the input/output interface 505 by mounting the removable recording medium 511 in the drive 510. Further, the program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the recording unit 508. In addition, the program can be installed in the ROM 502 or the recording unit 508 in advance.
[0306]
　The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.
[0307]
　Further, the embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present technology.
[0308]
　For example, the present technology may have a configuration of cloud computing in which a plurality of devices share one function via a network and jointly process the functions.
[0309]
　In addition, each step described in the above-described flowcharts can be executed by one device or shared by a plurality of devices.
[0310]
　Further, when one step includes a plurality of processes, the plurality of processes included in the one step can be executed by one device or shared by a plurality of devices.
[0311]
　Furthermore, the present technology may be configured as below.
[0312]
(1) A 　signal processing device including
　a reverb processing unit that generates a signal of a reverb component based on object audio data of an audio object and a reverb parameter for the audio object
.
(2) 　The signal processing device according to (1),
　further including a rendering processing unit that performs a rendering process on the signal of the reverb component based on the reverb parameter
.
(3) The
　reverb parameter includes position information indicating a localization position of a sound image of the reverb component, and the
　rendering processing unit performs the rendering process based on the position information
　. Signal processing device.
(4) 　The signal processing device according to (3),
　wherein the position information is information indicating an absolute localization position of the sound image of the reverb component
.
(5) 　The signal processing device according to (3),
　wherein the position information is information indicating a relative localization position of a sound image of the reverb component with respect to the audio object
.
(6) The
　reverb parameter includes an impulse response,

　The signal processing device according to any one of (1) to (5), 　wherein the reverb processing unit generates a signal of the reverb component based on the impulse response and the object audio data .
(7) The
　reverb parameter includes configuration information indicating the configuration of parametric reverb, and the
　reverb processing unit generates a signal of the reverb component based on the configuration information and the object audio data.
　The signal processing device according to any one of (1) to (5).
(8) 　The signal processing device
　according to
(7), wherein the parametric reverb is composed of a plurality of constituent elements including one or a plurality of filters .
(9) 　The signal processing device according to (8),
　wherein the filter is a low-pass filter, a comb filter, an all-pass filter, or a multi-tap delay
.
(10) 　The signal processing device according to (8) or (9),
　wherein the reverb parameter includes a parameter used in processing by the component
.
(11) The
　signal processing device
　A
　signal processing method for generating a signal of a reverb component based on object audio data of an audio object and a reverb parameter for the audio object .
(12) 　A program for causing a computer to execute processing including
　a
step of generating a signal of a reverb component based on object audio data of an audio object and a reverb parameter for the audio object .
Explanation of symbols
[0313]
　11 signal processing device, 21 demultiplexer, 22 reverb processing unit, 23 VBAP processing unit, 61 reverb processing unit, 141 reverb processing unit, 151 branch output unit, 152 pre-delay unit, 153 comb filter unit, 154 all-pass filter unit, 155 Adder, 156 adder
The scope of the claims
[Claim 1]
　A
　signal processing device comprising a reverb processing unit that generates a signal of a reverb component based on object audio data of an audio object and a reverb parameter for the audio object .
[Claim 2]

　The signal processing device according to claim 1, 　further comprising a rendering processing unit that performs rendering processing on the signal of the reverb component based on the reverb parameter .
[Claim 3]
　The signal processing device according to claim 2, 　wherein the reverb parameter includes position information indicating a localization position of a sound image of the reverb component, and the
　rendering processing unit performs the rendering process based on the position information.
..
[Claim 4]

　The signal processing device according to claim 3, 　wherein the position information is information indicating an absolute localization position of a sound image of the reverb component .
[Claim 5]

　The signal processing device according to claim 3, 　wherein the position information is information indicating a relative localization position of a sound image of the reverb component with respect to the audio object .
[Claim 6]
　The signal processing device according
　to
claim 1, 　wherein the reverb parameter includes an impulse response, and the reverb processing unit generates a signal of the reverb component based on the impulse response and the object audio data. ..
[Claim 7]
　The said reverb parameters, and contains configuration information indicating a configuration of a parametric reverb,
　the reverb processing unit, said configuration information to generate a signal of the reverberation components on the basis of said object audio data
　according to claim 1 The signal processing device according to.
[Claim 8]

　The signal processing device 　according to claim 7, wherein the parametric reverb includes a plurality of components including one or a plurality of filters .
[Claim 9]

　The signal processing device according to claim 8, 　wherein the filter is a low-pass filter, a comb filter, an all-pass filter, or a multi-tap delay .
[Claim 10]

　The signal processing device according to claim 8, 　wherein the reverb parameter includes a parameter used in processing by the component .
[Claim 11]
　Signal processor,
　generates a signal of reverb based and object audio data of the audio objects, in the reverb parameters for the audio objects
　signal processing method.
[Claim 12]

　A program for causing a computer to execute a process including 　a step of generating a signal of a reverb component based on object audio data of an audio object and a reverb parameter for the audio object .