Specification
[DESCRIPTION]
[Title of Invention]
SOUND PICKUP APPARATUS AND SOUND PICKUP METHOD
[Technical Field]
[0001]
The present invention relates to a sound pickup apparatus and a sound pickup method, and particularly to a sound pickup apparatus and a sound pickup method for generating a stereo signal by using multiple microphones.
[Background Art]
[0002]
Conventionally, in an acoustic system for e.g., a video conference system, sound pickup apparatus using multiple microphones have been used to pick up sound clearly from a sound source (for example, a speaker). Such a sound pickup apparatus generates a multichannel signal for reproducing, for example, the position of a sound source of a communication user site at a communication partner site (achieving sound image localization) using multiple microphones.
[3]
In such a sound pickup apparatus, multiple microphones are provided corresponding to respective channels. Also, the multiple microphones are fixedly installed with respective main axis directions of directivity toward the directions according to the corresponding channels. The sound pickup apparatus then can generate each picked-up sound signal as a multichannel signal for achieving sound image localization. The generated multichannel signal is transmitted to multiple loudspeakers at the communication partner site via a communication network. Accordingly, multichannel sound is reproduced at the communication partner site, and the position of a speaker at the user site is reproduced at the communication partner site.
[4]
In order to generate a multichannel signal for achieving sound image localization, multiple microphones need to be fixedly installed with respective main axis directions of directivity toward the directions according to the corresponding channels. Thus, in the above-mentioned sound pickup apparatus, the speaker cannot freely change the arrangement positions of the multiple microphones.
[5]
Now, in order to solve the above-mentioned problem, a sound pickup apparatus 190 as shown in FIGS. 16 and 17 has been proposed (for example, see PTL 1). FIG. 16 is a schematic view of a conventional sound pickup system. FIG. 17 is a block diagram showing the functional configuration of a conventional video conference system. As shown in FIG. 17, the video conference system includes a first sound pickup system 1000 installed at a user site, and a second sound pickup system 2000 installed at a communication partner site. Because the second sound pickup system 2000 has a configuration similar to that of the first sound pickup system 1000, a schematic view of the second sound pickup system 2000 is omitted in FIG. 16.
[6]
In the examples of FIGS. 16 and 17, a right channel (hereinafter referred to as a "R channel" or "Rch") signal, and a left channel (hereinafter referred to as an "L channel" or "Lch") signal are generated as multichannel signals, and stereo reproduction is achieved at the communication partner site.
[7]
A microphone 90a is installed on a table 103 so as to be placed in the front vicinity of a speaker 102a. A microphone 90b is installed on a table 103 so as to be placed in the front vicinity of a speaker 102b. A monitor 104 is a device for displaying an image captured by a camera 205 at the communication partner site, and is installed in front of the speakers 102a, 102b. The image of the communication partner site is inputted to the monitor 104 via a communication network 107,
[8]
A camera 105 is installed on the upper portion of the monitor 104, and captures the speakers 102a, 102b at the user site. The image of the user site is transmitted to a monitor 204 of the communication partner site via the communication network 107.
[9]
The first and second loudspeakers 106a, 106b reproduce an L channel signal or a R channel signal inputted from a sound pickup apparatus 290 of the communication partner site via the communication network 107. The first and second loudspeakers 106a, 106b are each installed on either side of the monitor 104. Similarly, a first loudspeaker 206a of the communication partner site is installed on the front left as viewed from the communication partner, and a second loudspeaker 206b of the communication partner site is installed on the front right as viewed from the communication partner.
[10]
The sound pickup apparatus 190 is installed at the user site, and the sound pickup apparatus 290 is installed at the communication partner site. Because the internal configuration of the sound pickup apparatus 290 is similar to that of the sound pickup apparatus 190, the drawing and description for the sound pickup apparatus 290 are omitted herein.
[0011]
The sound pickup apparatus 190 includes the microphones 90a and 90b, a microphone position determining unit 91, a coefficient calculating unit 92, a microphone detecting unit 93, and a signal calculating unit 94. In the following, each component of the sound pickup apparatus 190 is specifically described.
[0012]
The microphone position measuring unit 91 outputs a measurement signal to the first and second loudspeakers 106a, 106b. Subsequently, the microphone position measuring unit 91, after outputting the measurement signal, calculates a time period as a delay time until the measurement signal is picked up by the microphones 90a, 90b. The microphone position measuring unit 91 measures the current position of the microphones 90a, 90b using the calculated delay time.
[0013]
In the example of FIG. 16, because the microphone 90a is placed on the right side position as viewed from the monitor 104, the right side position is measured as the current position of the microphone 90a. Also because the microphone 90b is placed on the left side position as viewed from the monitor 104, the left side position is measured as the current position of the microphone 90b. The microphone position measuring unit 91 measures those current positions for every movement of the microphones 90a, 90b so that a speaker can freely move the microphones 90a, 90b.
[14]
The coefficient calculating unit 92 calculates the ratio (coefficient ratio) between the level assigned to the R channel signal and the level assigned to the L channel signal based on the measured current positions of the microphones 90a, 90b so that multichannel signals for achieving sound image localization are generated.
In the example of FIG. 16, the measured current position of the microphone 90a is on the right as viewed from the monitor 104. Thus, the coefficient calculating unit 92 determines, for example, (R channel signal : L channel signal) = (1:0) as the coefficient ratio of the microphone 90a. On the other hand, the measured current position of the microphone 90b is on the left as viewed from the monitor 104. Thus, the coefficient calculating unit 92 determines, for example, (R channel signal : L channel signal) = (0:1) as the coefficient ratio of the microphone 90b.
[16]
When either one of the speakers 102a or 102b speaks, the microphone detecting unit 93 detects a microphone nearest to the speaker based on the levels of the picked-up sound signals from the microphones 90a, 90b. For example, when the speaker 102a speaks, the level of the picked-up sound signal from the microphone 90a becomes greater than that of the picked-up sound signal from the microphone 90b. In this case, the microphone detecting unit 93 detects the microphone 90a as the microphone nearest to the speaker. Subsequently, the coefficient calculating unit 92 determines the coefficient ratio for the microphone 90a, (R channel signal : L channel signal) = (1:0) as the coefficient ratio to be outputted to the signal calculating unit 94 based on the microphone 90a detected by the microphone detecting unit 93.
[17]
The signal calculating unit 94 calculates the R channel signal and L channel signal according to the determined coefficient ratio. For example, in the case where the coefficient ratio for microphone 90a is (R channel signal : L channel signal) = (1 : 0), the signal calculating unit 94 calculates the R channel signal by multiplying respective picked-up sound signals of the microphones 90a, 90b by a coefficient 1 and adding the multiplied picked-up sound signals. On the other hand, the signal calculating unit 94 calculates the L channel signal by multiplying respective picked-up sound signals of the microphones 90a, 90b by a coefficient 0 and adding the multiplied picked-up sound signals.
[18]
Accordingly, the R channel signal forms a signal to which all the picked-up sound signals from the microphones 90a, 90b are added, and the L channel signal has no output, thus multichannel signals for achieving sound image localization are generated. The L channel signal (Lch) and R channel signal (Rch) which are calculated in the signal calculating unit 94 are transmitted to the loudspeakers 206a, 206b of the communication partner site via the communication network 107. Accordingly, at the communication partner site, sound is reproduced as if the speaker 102a speaks from the right position as viewed from a speaker of the communication partner site.
[0019]
In this manner, the sound pickup apparatus 190 shown in FIGS. 16 and 17 measures the position (current position) of each microphone after every movement of the microphone, and multichannel signals for achieving sound image localization are generated by using the information on the current position of the measured microphone. Consequently, the speaker can freely change the arrangement positions of the microphones.
[Citation List] [Patent Literature]
[0020]
[PTL 1]
Japanese Unexamined Patent Application Publication No. 09-182044
[Summary of Invention] [Technical Problem]
[0021]
However, the sound pickup apparatus 190 shown in FIGS. 16 and 17 has the following problems.
[0022]
(1) Because the sound pickup apparatus 190 performs processing using the information on the current position of the microphone, the position of the microphone needs to be measured before multichannel signals are generated (for example, before the start of conference).
[23]
(2) Because the sound pickup apparatus 190 performs processing using the information on the current position of the microphone, each time the position of a microphone is changed, for example, when a speaker moves the microphone in the conference, the conference is interrupted and the position of the microphone needs to be re-measured.
[24]
(3) Because each microphone can be moved freely by the speaker, the main axis direction of the directivity of the microphone is not necessarily toward the speaker, thus the microphone detected in the microphone detecting unit 93 actually may not be nearest to the speaker. In that case, the sound pickup apparatus 190 cannot pick up sound clearly from each sound source.
[25]
Now, the present invention has been made to solve the aforementioned problems, and it is an object of the invention to provide a sound pickup apparatus and a sound pickup method that
-6- increase the flexibility in installation of microphones, while enabling sound from a sound source to be picked up in stereo without using the information on the current positions of the microphones.
[Solution to Problem]
[0026]
In order to achieve the above-mentioned object, a sound pickup apparatus according to one aspect of the present invention includes: a microphone including 4 or more sound pickup units placed so as to have different main axis directions of directivity, the microphone being installed in a position where sound outputted from a loudspeaker can be picked up; a level calculating unit configured to calculate a signal level of sound outputted from the loudspeaker for each output signal of the 4 or more sound pickup units; a mixing coefficient calculating unit configured to calculate a mixing coefficient adjusted to main axis directions of directivity of the 4 or more sound pickup units by using the signal level calculated by the level calculating unit; and a signal mixing unit configured to generate a stereo signal by mixing output signals of the 4 or more sound pickup units by using a mixing coefficient calculated by the mixing coefficient calculating unit.
[27]
According to this configuration, a mixing coefficient adjusted to the main axis direction of directivity can be calculated using the signal levels of the sound outputted from the loudspeakers. Therefore, sound from a sound source can be picked up in stereo without measuring the position of the microphone. Also, each microphone includes 4 or more sound pickup units having respective main axis directions of directivity being different from each other. Therefore, sound from a sound source located in the periphery of a microphone can be clearly picked up in stereo regardless of the orientation of the installed microphone, thus flexibility in installation of the microphones can be increased.
[28]
Preferably, the 4 or more sound pickup units are placed so as to have main axis directions of directivity separated by 90 degrees.
By this configuration, sound from a sound source located in the periphery of a microphone can be picked up in stereo more clearly regardless of the orientation of the installed microphone.
[30]
Preferably, the mixing coefficient calculating unit is configured to calculate the mixing coefficient so that the stereo signal is equivalent to a signal picked up in stereo from two sound pickup directions which are different from each other by 180 degrees.
[31]
By this configuration, stereo sound pickup with high channel separation can be achieved, thus sound from each sound source can be clearly picked up.
[32]
Preferably, the mixing coefficient includes a first mixing coefficient for generating one part of the stereo signal, and a second mixing coefficient for generating another part of the stereo signal; in each one of the 4 sound pickup units, the mixing coefficient calculating unit calculates the first mixing coefficient by using a signal level of another sound pickup unit located next to one side of the sound pickup unit, and calculates the second mixing coefficient by using a signal level of another sound pickup unit located next to another side of the sound pickup unit; and the signal mixing unit is configured to generate one part of the stereo signal by multiplying output signals of the 4 sound pickup units by the respective first mixing coefficients and adding the multiplied output signals together, and generates another part of the stereo signal by multiplying the output signals of the 4 sound pickup units by the respective second mixing coefficients and adding the multiplied output signals together.
[33]
According to this configuration, the mixing coefficient for each sound pickup unit can be calculated using the signal level of adjacent sound pickup units. Accordingly, compared with the case where the installation angle of each microphone is precisely calculated, the mixing coefficient can be calculated relatively easily, thus calculation load for calculating the mixing coefficient can be reduced.
Preferably, the sound pickup apparatus further includes a loudspeaker signal detecting unit configured to determine whether a loudspeaker signal for outputting sound from the loudspeaker is present or not, wherein the level calculating unit is configured to calculate the signal level in a case where a loudspeaker signal is determined to be present by the loudspeaker signal detecting unit.
[35]
According to this configuration, a signal level can be calculated from the signal outputted when a loudspeaker signal is determined to be present. Consequently, it is not necessary to output sound from each loudspeaker only for calculating the mixing coefficient, and for example, in a video conference system, the mixing coefficient also can be calculated using e.g., speaker's voice outputted from the loudspeaker.
[36]
Preferably, the sound pickup apparatus further includes a movement determining unit configured to determine whether the microphone is moved or not, wherein the mixing coefficient calculating unit sets a higher frequency of calculation of the mixing coefficient for a case where the microphone is determined to be moved than for a case where the microphone is determined to be still.
[37]
According to this configuration, when a microphone is moved, corresponding mixing coefficient is updated relatively quickly, while when a microphone is not moved, corresponding mixing coefficient is updated relatively slowly. Consequently, update responsiveness of the mixing coefficient to movement of each microphone can be improved, while the stability of the mixing coefficient also can be increased. Thus, sound from a sound source located in the periphery of a microphone can be picked up in stereo more clearly.
[38]
Preferably, the movement determining unit determines whether the microphone is moved or not based on a rate of change of the mixing coefficient calculated by the mixing coefficient calculating unit.
[39]
According to this configuration, a movement sensor or the like does not need to be installed in the microphone, thus whether the microphone is moved or not can be easily determined with a simple configuration.
[40]
Preferably, the sound pickup apparatus further includes a localization determining unit configured to determine a sound image localized position from a loudspeaker signal for outputting sound from the loudspeaker, wherein the mixing coefficient calculating unit is configured to calculate the mixing coefficient for each sound image localized position determined by the localization determining unit; and the signal mixing unit combines output signals of the 4 or more sound pickup units by using an average of mixing coefficient for each sound image localized position calculated by the mixing coefficient calculating unit.
[41]
According to this configuration, variation in the mixing coefficient due to a change of the sound image localized position of the sound outputted from the loudspeakers can be suppressed, thus sound can be picked up in stereo in a stable manner.
[42]
Preferably, the localization determining unit determines one of left localization, right localization, and center localization, as a sound image localized position, each localization indicating a relative positional relationship between the sound image localized position and the loudspeaker; and the level calculating unit includes: Lch level calculating unit configured to calculate the signal level in a case where sound image localized position is determined to be the left localization by the localization determining unit, Rch level calculating unit configured to calculate the signal level in a case where sound image localized position is determined to be the right localization by the localization determining unit, and Cch level calculating unit configured to calculate the signal level in a case where sound image localized
-10- position is determined to be the center localization by the localization determining unit.
[43]
According to this configuration, a mixing coefficient can be calculated for each of three types of sound image localized positions: left, right, and center.
Preferably, the sound pickup apparatus further includes a movement determining unit configured to determine whether the microphone is moved or not, wherein in a case where the microphone is determined to be moved, the mixing ratio calculating unit calculates a mixing coefficient of a sound image localized position whose signal level is not calculated yet after the microphone is determined to be moved by using a signal level of a sound image localized position whose signal level is already calculated after the microphone is determined to be moved.
[45]
According to this configuration, even in the case where there are some sound image localized positions whose signal levels have not been updated after a microphone is moved, the mixing coefficient after the movement of the microphone can be calculated with a higher precision.
[46]
Preferably, the sound pickup apparatus further includes a band extracting unit configured to extract a signal in a predetermined frequency band from each of output signals of the 4 or more sound pickup units, wherein the level calculating unit calculates the signal level using a signal extracted by the band extracting unit.
[47]
According to this configuration, when a mixing coefficient is calculated, influence of sound other than the sound needed to calculate the mixing coefficient (for example, noise) can be suppressed, thus the mixing coefficient can be calculated with a higher precision.
[48]
Preferably, the predetermined frequency band is a voice band.
[49]
According to this configuration, when a mixing coefficient is calculated by using, for example, a speaker's voice outputted from a loudspeaker, the mixing coefficient can be calculated with a higher precision.
[50]
A sound pickup method according to one aspect of the present invention is a sound pickup method for generating a stereo signal by using a microphone which has 4 or more sound pickup units placed so as to have different main axis directions of directivity, the microphone being installed in a position where sound outputted from a loudspeaker can be picked up, the sound pickup method including: calculating a signal level of sound outputted from the loudspeaker for each output signal of the 4 or more sound pickup units; calculating a mixing coefficient adjusted to main axis directions of directivity of the 4 or more sound pickup units by using a signal level calculated in the calculating of a signal level; and generating a stereo signal by mixing the output signals of the 4 or more sound pickup units by using a mixing coefficient calculated in the calculating of a mixing coefficient.
[51]
Thereby, effects similar to those of the above-mentioned sound pickup apparatus can be achieved.
[52]
An integrated circuit according to one aspect of the present invention is an integrated circuit for generating a stereo signal by using a microphone which has 4 or more sound pickup units placed so as to have different main axis directions of directivity, the microphone being installed in a position where sound outputted from a loudspeaker can be picked up, the integrated circuit including: a level calculating unit configured to calculate a signal level of sound outputted from the loudspeaker for each output signal of the 4 or more sound pickup units; a mixing coefficient calculating unit configured to calculate a mixing coefficient adjusted to main axis directions of directivity of the 4 or more sound pickup units by using
a signal level calculated by the level calculating unit; and a signal mixing unit configured to generate a stereo signal by mixing output signals of the 4 or more sound pickup units by using a mixing coefficient calculated by the mixing coefficient calculating unit.
[53]
Thereby, effects similar to those of the above-mentioned sound pickup apparatus can be achieved.
Also, the present invention can be achieved as a program for making a computer to execute each step included in the above-described sound pickup method. And it is needless to state that such a program can be distributed via recording media such as a CD-ROM (Compact Disc Read Only Memory), or via transmission media such as the Internet.
[Advantageous
Effects of Invention]
[55]
According to the present invention, a mixing coefficient adjusted to the main axis direction of directivity can be calculated using the signal levels outputted from the loudspeakers. Therefore, sound from a sound source can be picked up clearly in stereo without measuring the positions of the microphones. Also, each microphone includes 4 or more sound pickup units having respective main axis directions of directivity being different from each other. Therefore, sound from a sound source located in the periphery of a microphone can be clearly picked up in stereo regardless of the orientation of the installed microphone, thus flexibility in installation of the microphones can be increased.
[Brief Description of Drawings]
[56]
[FIG. 1]
FIG. 1 is a schematic view of a sound pickup system according to Embodiment 1 of the present invention.
[FIG. 2]
FIG. 2 is a schematic view of the sound pickup system according to Embodiment 1 of the present invention.
[FIG. 3]
FIG. 3 is a diagram showing the configuration of a video conference system according to Embodiment 1 of the present invention.
[FIG. 4]
FIG. 4 is a diagram showing an Lch sound picl + {Px2(t) + Px3 (t)}).
[0122]
The directional characteristic of an unidirectional microphone is expressed by (l+cose)/2. Therefore, when two unidirectional microphones whose main axis directions of directivity are different from each other by 180 degrees are combined, the directional characteristic is (l+cose)/2+ (1+cos (0+18O°)/2 = 1, which indicates non-directional.
[0123]
That is to say, Px(t) can be considered to be the sum of signal power of two non-directional microphones. Accordingly, when voice is amplified from a loudspeaker and no sound other than the speaker amplified voice is present, Pv(t) = Px(t) / 2 as shown in Expression (5).
[0124]
[125]
From Expression (2) and Expression (5), the mixing ratio calculating unit 113 calculates the Lch mixing coefficient All(t) and the Rch mixing coefficients A21(t) to A24(t) as in Expression (6) by using the signal power Pxl(t) to Px4(t) of the sound pickup units. That is to say, the mixing ratio calculating unit 113 calculates the mixing coefficients adjusted to the main axis direction of directivity of each sound pickup unit by using the signal levels.
[126]
[127]
FIG. 4 shows polar patterns of directivity which are formed when a stereo signal is generated by using the Lch mixing coefficients All(t) to A14(t) and the Rch mixing coefficients A21(t) to A24(t) in this manner.
[128]
The signal power Pxl(t) to Px4(t) of the first to fourth sound pickup units 110a to 110d are normalized with respect to the total value Px(t) in Expression (4), but may be approximated by values which are normalized using the maximum values of the signal power Pxl(t) to Px4(t). That is to say, the mixing ratio calculating unit 113 may calculate the mixing coefficients adjusted to the main axis direction of directivity of each sound pickup unit using the signal levels.
[129]
As described above, according to the sound pickup apparatus 101 in the present embodiment, the mixing coefficients adjusted to the main axis direction of directivity can be calculated by using the signal levels of the sound outputted from the loudspeakers. Therefore, sound from a sound source can be picked up clearly in stereo without measuring the positions of the microphones.
[130]
Also, the microphone 110 includes the first to fourth sound pickup units 110a to 110d which are arranged to have main axis directions of directivity separated by 90 degrees. Accordingly, sound from a sound source located in the periphery of the microphone 110 can be clearly picked up in stereo regardless of the orientation of the installed microphone 110. That Is to say, the speakers can freely install the microphone 110.
[131]
(Embodiment 2)
Next, a sound pickup apparatus according to Embodiment 2 of the present invention is described with reference to FIG. 6.
[132]
FIG. 6 is a block diagram showing the functional configuration of the sound pickup apparatus 101 according to Embodiment 2 of the present invention. The sound pickup apparatus 101 shown in FIG. 6 is installed to replace the sound pickup apparatus 101 according to Embodiment 1 shown in FIG. 3.
Also, the sound pickup apparatus 101 according to the present embodiment differs from the sound pickup apparatus 101 according to Embodiment 1 shown in FIG. 3 only in that a band extracting unit 111 is newly added. In the following, points of difference between the present embodiment and Embodiment 1 are mainly described.
[134]
The band extracting unit 111 extracts only those signals in a predetermined frequency band from the output signals of the first to fourth sound pickup units 110a to 110d, and outputs the signals to the level calculating unit 112.
[135]
The predetermined frequency band is, for example, the voice band of the speakers 102a, 102b (300 Hz to 7 kHz). More preferably, the predetermined frequency band is a band within the voice band, which has a relatively small influence of room noise (for example, 1 kHz to 4 kHz). By this extraction of signal, the sound pickup apparatus 101 can suppress the influence of noise such as the sound other than the sound outputted from the loudspeakers as the mixing coefficients are calculated by using the voice of the speakers outputted from the loudspeakers. That is to say, the sound pickup apparatus 101 can calculate the mixing coefficients with a higher accuracy.
[136]
Also, the predetermined frequency band may be a band which is determined by taking, for example, directional characteristic of each of the first to fourth sound pickup units 110a to llOd (frequency characteristic of directivity ) into consideration. Specifically, the predetermined frequency band may be a band, in which the directivity for all the sound pickup units can be stably obtained.
[137]
By extracting the signals in such predetermined frequency band, the influence of noise other than the voice of the speakers 102a, 102b on the signal levels inputted to the mixing ratio calculating unit 113 can be reduced.
[138]
As described above, according to the present embodiment, the influence of noise other than the voice of the speakers 102a, 102b, included in the sound pickup units' signals inputted to the mixing ratio calculating unit 113 can be reduced. Thereby, accuracy of the mixing coefficients calculated by the mixing ratio calculating unit 113 is increased, and stereo signals for allowing clearer sound to be reproduced can be generated.
(Embodiment 3)
Next, a sound pickup apparatus according to Embodiment 3 of the present invention is described witli reference to FIGS. 7 and 8.
[140]
FIG. 7 is a blocl< diagram showing the functional configuration of the sound pickup apparatus 101 according to Embodiment 3 of the present invention. The sound pickup apparatus 101 shown in FIG. 7 is installed to replace the sound pickup apparatus 101 according to Embodiment 1 or 2 shown in FIG. 3 or 6.
[141]
Also, the sound pickup apparatus 101 according to the present embodiment differs from the sound pickup apparatus 101 according to Embodiment 2 shown in FIG. 6 only in that an updating unit 117 and a movement determining unit 114 are newly added. In the following, points of difference between the present embodiment and Embodiment 1 or 2 are mainly described.
[142]
The microphone 110 may be easily rotated or moved by the speakers 102a, 102b. In the case where the microphone 110 is rotated or moved, generation of a stereo signal using the mixing coefficients calculated before the rotation or movement of the microphone 110 causes a mismatch between an image and its sound image. For this reason, the mixing coefficients need to be calculated again after the movement.
[143]
In the case where the mixing coefficients are calculated correctly, and the microphone 110 is not rotated or moved, it is preferable that already calculated mixing coefficients are not updated or their update frequency is controlled to be lowered so that the sound images are not changed.
Now, in the present embodiment, the movement determining unit 114 determines whether the microphone is moved or not. Specifically, the movement determining unit 114 determines whether the microphone 110 is moved or not based on the rate of change of the mixing coefficients calculated by the mixing ratio calculating unit 113.
[145]
The updating unit 117 then outputs Lch smoothing mixing coefficients B 110d(t) to B14(t) and Rch smoothing mixing B21(t) to B24(t) by averaging the Lch mixing coefficients All(t) to A14(t) and the Rch mixing coefficients A21(t) to A24(t) over time.
[146]
Further, the updating unit 117 adjusts the frequency of calculation of the Lch smoothing mixing coefficients B101d(t) to B14(t) and the Rch smoothing mixing coefficients B21(t) to B24(t) based on the determination result by the movement determining unit 114. That is to say, the updating unit 117 sets a higher frequency of calculation of the smoothing mixing coefficients for the case where the microphone 110 is determined to be moved than for the case where the microphone 110 is determined to be still.
[147]
Consequently, the sound pickup apparatus 101 can improve update responsiveness of the mixing coefficients to the movement of the microphone 110, while increasing the stability of the mixing coefficients.
[148]
Next, the detail of the movement determining unit 114 is described using FIGS. 8.
[149]
FIG. 8 is a block diagram showing the functional configuration in detail of the movement determining unit 114 according to Embodiment 3 of the present invention. As shown in FIG. 8, the movement determining unit 114 includes a variation detecting unit 123, and a variation determining unit 124.
[150]
The variation detecting unit 123 receives, as inputs, the Lch mixing coefficients All(t) to A14(t) and the Rch mixing coefficients A21(t) to A24(t) calculated by the mixing ratio calculating unit 113, and the Lch smoothing mixing coefficients Bll(t) to B14(t) and the
Rch smoothing mixing coefficients B21(t) to B24(t) calculated by the updating unit 117, then detects a variation indicating a rate of change of the mixing coefficients as shown in
Expression (7).
[151]
[152]
Where Bt-1 and At are expressed by Expression (8).
[153]
While the microphone 110 is still, the mixing coefficient At and the smoothing mixing coefficient Bt-1 have almost same value, thus the value of variation mic_var is reduced. On the other hand, when the microphone 110 is moved, the value of the mixing coefficient At is changed first. Accordingly, the difference between the mixing coefficient At and the smoothing mixing coefficient Bt-1 is increased. Therefore, the value of the variation mic_var is increased.
[155]
Now, the variation determining unit 124 monitors the value of the variation mic_var. Then in the case where the variation mic_var exceeds a threshold value, the variation determining unit 124 determines that the microphone 110 is moved, and sets move_flg = 1. On the other hand, in the case where the variation mic_var does not exceed a threshold value, the variation determining unit 124 determines that the microphone 110 is not moved, and sets move_flg = 0.
[156]
According to the value of move_flg set in this manner, the updating unit 117 adjusts the frequency of calculation of the smoothing mixing coefficients.
As described above, according to tlie sound pickup apparatus 101 in the present embodiment, update responsiveness of the sound image localization can be improved when the microphone 110 is moved. Also, while the microphone 110 is still, stability of the sound image localization can be secured. In other words, according to the sound pickup apparatus 101 in the present embodiment, update responsiveness of the mixing coefficients to the movement of the microphone 110 can be improved, while the stability of the mixing coefficients also can be increased. Thus, sound from a sound source located in the periphery of the microphone 110 can be picked up in stereo more clearly.
[158]
Also, the movement determining unit 114 can determine whether the microphone 110 is moved or not based on the rate of change of the mixing coefficients, and a movement sensor or the like does not need to be installed, thus whether the microphone 110 is moved or not can be easily determined with a simple configuration.
[159]
The movement determining unit 114 does not necessarily need to determine whether the microphone 110 is moved or not based on a rate of change of the mixing coefficients. For example, the movement determining unit 114 may determine whether the microphone 110 is moved or not based on sensor signals from a movement sensor such as a gyro sensor attached to the microphone 110. In this case, the sound pickup apparatus has more complex configuration than that of the sound pickup apparatus 101 of the present embodiment, however, update responsiveness of the mixing coefficients to the movement of the microphone 110 can be improved, while the stability of the mixing coefficients can be increased.
[160]
Also, in the present embodiment, the movement determining unit 114 determines whether the microphone 110 is moved or not by using the mixing coefficients and the smoothing mixing coefficients for both Lch and Rch, but may use the mixing coefficients and the smoothing mixing coefficients for either one channel to determine whether the microphone 110 is moved or not.
[0161]
[Expression 9]
Considering that the main axis directions of directivity formed by Lch and Rch are different from each other by 180 degrees, the updating unit 117 may use the smoothing mixing coefficients for either one channel to calculate the smoothing mixing coefficients for the other channel.
[0162]
[163]
Also, in the present embodiment, the sound pickup apparatus 101 includes the updating unit 117, but does not necessarily need to include the updating unit 117. In this case, the mixing ratio calculating unit 113 may set a higher frequency of calculation of the mixing coefficients for the case where the microphone 110 is determined to be moved than for the case where the microphone 110 is determined to be still. Thereby, the sound pickup apparatus 101 can improve update responsiveness of the mixing coefficients to the movement of the microphone 110, while increasing the stability of the mixing coefficients.
[164]
(Embodiment 4)
Next, a sound pickup apparatus according to Embodiment 4 of the present invention is described with reference to FIG. 9A.
[165]
FIG. 9A is a block diagram showing the functional configuration of the sound pickup apparatus 101 according to Embodiment 4 of the present invention. The sound pickup apparatus 101 shown in FIG. 9A is installed to replace the sound pickup apparatus 101 according to Embodiment 1, 2, or 3 shown in FIG. 3, 6, or 7.
Also, the sound pickup apparatus 101 according to the present embodiment differs from the sound pickup apparatus 101 according to Embodiment 2 shown in FIG. 6 only in that a localization determining unit 122 is newly added, and the level calculating unit 112 includes an Lch level calculating unit 112a, a Cch level calculating unit 112b, and a Rch level calculating unit 112c. In the following, points of difference between the present embodiment and Embodiments 1 to 3 are mainly described.
[167]
The voice signals reproduced from the first and second loudspeakers 106a, 106b are stereo signals. Accordingly, a level difference is generated in the voice signals reproduced from the first and second loudspeakers 106a, 106b. Consequently, the sound image localized positions for the following cases are different: the case where the reproduced sound from the first loudspeaker 106a is greater than that from the second loudspeaker 106b (the case where the sound image is localized to the left); the case where the reproduced sound from the first loudspeaker 106a is the same as that from the second loudspeaker 106b (the case where the sound image is localized to the center); and the case where the reproduced sound from the second loudspeaker 106b is greater than that from the first loudspeaker 106a (the case where the sound image is localized to the right).
[168]
Accordingly, even when the first to fourth sound pickup units 110a to 110d are fixed, the first to fourth gains D1 to D4 depending on the respective main axis directions of directivity are changed by the sound image localized positions. That is to say, every time a sound image localized position is changed, the mixing coefficients calculated by the mixing ratio calculating unit 113 are varied and unstable.
[0169]
Now in the present embodiment, the sound pickup apparatus 101 achieves a state in which the mixing coefficients can be calculated in a stable manner even when the sound image localized position of the sound outputted from a loudspeaker is changed. In the following, the components of such sound pickup apparatus 101 are described.
[170]
The localization determining unit 122 determines the sound image localized position from loudspeaker signals for outputting sound from a loudspeaker. In the present embodiment, the localization determining unit 122 determines one of left localization, right localization, and center localization, as a sound image localized position, each localization indicating a relative positional relationship between a sound image localized position and a loudspeaker.
[171]
Specifically, in the case where the sound image localized position of the sound reproduced from the first and second loudspeakers 106a, 106b is on the left side as viewed from the speakers 102a, 102b (on the right side as viewed from the first and second loudspeakers 106a, 106b), the localization determining unit 122 determines the sound image localized position to be Lch localized (left localized), and sets dir_flg = 0. Also, in the case where the sound image localized position is on the center as viewed from the speakers 102a, 102b (on the center as viewed from the first and second loudspeakers 106a, 106b), the localization determining unit 122 determines the sound image localized position to be Cch localized (center localized), and sets dir_flg = 1. Also, in the case where the sound image localized position is on the right side as viewed from the speakers 102a, 102b (on the left side as viewed from the first and second loudspeakers 106a, 106b), the localization determining unit 122 determines the sound image localized position to be Rch localized (right localized), and sets dir_flg = 2.
[172]
More specifically, in the case where the signal level of the Lch loudspeaker signal exceeds that of the Rch loudspeaker signal by a threshold value or more, for example, the localization determining unit 122 determines the sound image localized position to be Lch localized. Also, in the case where the signal level of the Rch loudspeaker signal exceeds that of the Lch loudspeaker signal by a threshold value or more, for example, the localization determining unit 122 determines the sound image localized position to be Rch localized. Also, in the case where the difference between the signal level of the Rch loudspeaker signal and that of the Lch loudspeaker signal is less than a threshold value, for example, the localization determining unit 122 determines the sound image localized position to be Cch localized.
[173]
The level calculating unit 112 calculates a signal level according to the determination result dir_flg of the sound image localized position of the sound. That is to say, according to the sound image localized position determined by the localization determining unit 122, one of the Lch level calculating unit 112a, the Cch level calculating unit 112b and the Rch level calculating unit 112c calculates a signal level.
[174]
Specifically, in the case where the sound image localized position is determined to be Lch localized, only the Lch level calculating unit 112a updates the signal power. In the case where the sound image localized position is determined to be Cch localized, only the Cch level calculating unit 112b updates the signal power. In the case where the sound image localized position is determined to be Rch localized, only the Rch level calculating unit 112c updates the signal power.
[175]
The mixing ratio calculating unit 113 calculates the mixing coefficients for each sound image localized position determined by the localization determining unit 122.
[176]
Specifically, the mixing ratio calculating unit 113 calculates Lch mixing coefficients Alla(t) to A14a(t) and Rch mixing coefficients A21a(t) to A24a(t) in Lch localization by using Expression (6) based on each signal power calculated by the Lch level calculating unit 112a. Similarly, the mixing ratio calculating unit 113 calculates Lch mixing coefficients Allb(t) to A14b(t) and Rch mixing coefficients A21b(t) to A24b(t) in Cch localization by using Expression (6) based on each signal power calculated by the Cch level calculating unit 112b.
Similarly, the mixing ratio calculating unit 113 calculates Rch mixing coefficients Allc(t) to A14c(t) and Lch mixing coefficients A21b(t) to A24b(t) in Rch localization by using Expression (6) based on each signal power calculated by the Rch level calculating unit 112c.
The mixing ratio calculating unit 113 then calculates the Lch mixing coefficients All(t) to A14(t) and the Rch mixing coefficients A21(t) to A24(t) by determining the average of the Lch mixing coefficients, and the average of the Rch mixing coefficients in Lch localization, Cch localization, and Rch localization as shown in Expression (10).
[178]
That is to say, the signal mixing unit 115 combines the output signals of the first to fourth sound pickup units 110a to 110d by using the average of the mixing coefficients for each sound image localized position.
[179]
[Expression 10]
[0180]
Alternatively, the mixing ratio calculating unit 113 may calculate the mixing coefficient for one of Lch and Rch, and uses the calculation result to calculate the mixing coefficient for the other based on the relationship in Expression (2).
[0181]
As described above, according to the sound pickup apparatus 101 in the present embodiment, variation in the mixing coefficient due to a change of the sound image localized position of the reproduced sound of the first and second loudspeakers 106a, 106b can be
suppressed, thus sound can be picked up in stereo in a stable manner.
[0182]
Althougli tlie level calculating unit 112 according to the present embodiment includes the Lch level calculating unit 112a, the Cch level calculating unit 112b, and the Rch level calculating unit 112c, the level calculating unit 112 does not necessarily need to include these calculating units. For example, as shown in FIG. 9B, the sound pickup apparatus 101 may include the level calculating unit 112 as in Embodiment 1. In this case, the mixing ratio calculating unit 113 may calculate the mixing coefficients for each sound image localized position based on the determination result by the localization determining unit 122. Thereby, even with the sound pickup apparatus 101 shown in FIG. 9B, effects similar to those of the sound pickup apparatus 101 shown in FIG. 9A can be achieved.
[183]
(Embodiment 5)
Next, a sound pickup apparatus according to Embodiment 5 of the present invention is described with reference to FIGS. 10 to 12.
[184]
FIG. 10 is a block diagram showing the functional configuration of the sound pickup apparatus 101 according to Embodiment 5 of the present invention. The sound pickup apparatus 101 shown in FIG. 10 is installed to replace the sound pickup apparatus 101 shown in e.g., FIG. 3.
[185]
Also, the sound pickup apparatus 101 shown in FIG. 10 differs from the sound pickup apparatus 101 shown in FIG. 9A only in that an updating unit 117 and a movement determining unit 114, and a correcting unit 121 are newly added. The movement determining unit 114 is similar to the movement determining unit described in Embodiment 3, thus detailed description is omitted. In the following, points of difference between the present embodiment and Embodiments 1 to 4 are mainly described.
[186]
In the Lch level calculating unit 112a, the Cch level calculating unit 112b, and the Rch level calculating unit 112c, whether signal power is calculated or not is determined according to the determination result by the localization determining unit 122. Accordingly, in the case where the microphone 110 is moved, signal power may not be calculated for a while in one or two of the Lch level calculating unit 112a, the Cch level calculating unit 112b, and the Rch level calculating unit 112c. As a result, some coefficients in Expression (10) may coincide, for a while, with the coefficients calculated before the microphone 110 is moved.
[187]
In the case where the mixing ratio calculating unit 113 calculates mixing coefficients according to Expression (10) in such a situation, the directivity before the movement of the microphone 110 and the directivity after the movement are combined, thus correct directivity cannot be formed.
[188]
Now, the correcting unit 121 corrects the signal power based on the determination result by the movement determining unit 114 as well as the determination result by the localization determining unit 122. Specifically, in the case where the microphone 110 is determined to be moved, the correcting unit 121 corrects the signal power which has not been updated after the determination. At this point, the correcting unit 121 does not correct the signal power which has already been updated after the determination.
[189]
FIG. 11 is a block diagram showing the functional configuration in detail of the correcting unit 121 according to Embodiment 5 of the present invention. As shown in FIG. 11, the correcting unit 121 includes a selection controlling unit 125, an Lch correction selecting unit 126a, a Cch correction selecting unit 126b, and a Rch correction selecting unit 126c.
After the microphone 110 is moved, the correcting unit 121 corrects the signal power as shown below for the following cases: (1) the case where only the Lch level calculating unit 112a calculates the signal power; (2) the case where only the Cch level calculating unit 112b calculates the signal power; and (3) the case where only the Rch level calculating unit 112c calculates the signal power.
[191]
(1) The Cch correction selecting unit 126b corrects the signal power Pxlb(t) to Px4b(t) calculated by the Cch level calculating unit 112b before the movement of the microphone 110 to the product between coefficient al and the signal power Pxla(t) to Px4a(t) calculated by the Lch level calculating unit 112a.
[192]
Similarly, the Rch correction selecting unit 126c corrects the signal power Pxlc(t) to Px4c(t) calculated by the Rch level calculating unit 112c before the movement of the microphone 110 to the product between coefficient al and the signal power Pxla(t) to Px4a(t) calculated by the Lch level calculating unit 112a.
[193]
(2) The Lch correction selecting unit 126a corrects the signal power Pxla(t) to Px4a(t) calculated by the Lch level calculating unit 112a before the movement of the microphone 110 to the product between coefficient a2 and the signal power Pxlb(t) to Px4b(t) calculated by the Cch level calculating unit 112b.
[194]
Similarly, the Rch correction selecting unit 126c corrects the signal power Pxlc(t) to Px4c(t) calculated by the Rch level calculating unit 112c before the movement of the microphone 110 to the product between coefficient a2 and the signal power Pxlb(t) to Px4b(t) calculated by the Cch level calculating unit 112b.
[195]
(3) The Lch correction selecting unit 126a corrects the signal power Pxla(t) to Px4a(t) calculated by the Lch level calculating unit 112a before the movement of the microphone 110 to the product between coefficient a3 and the signal power Pxlc(t) to Px4c(t) calculated by the Rch level calculating unit 112c.
[196]
Similarly, the Cch correction selecting unit 126b corrects the signal power Pxlb(t) to Px4b(t) calculated by the Cch level calculating unit 112b before the movement of the microphone 110 to the product between coefficient a3 and the signal power Pxlc(t) to Px4c(t) calculated by the Rch level calculating unit 112c.
Next, the coefficients ol, a2, and a3 are described.
[198]
When move_flg as the determination result by the movement determining unit 114 is changed from 0 to 1 (when a still state is changed to a movement state), the selection controlling unit 125 sets al = l, 02 = 1, and a3 = l.
[199]
However, when the determination result by the localization determining unit 122 is dir_flg = 0 (Lch localization), the selection controlling unit 125 changes to al = 0. Also, when the determination result by the localization determining unit 122 is dir_flg = l (Cch localization), the selection controlling unit 125 changes to a2=0. Also, when the determination result by the localization determining unit 122 is dir_flg = 2 (Rch localization), the selection controlling unit 125 changes to G3 = 0.
[200]
Next, the details of the operation of the selection controlling unit 125 when the coefficient ol, a2, and G3 are set are described by using FIG. 12.
[0201]
FIG. 12 is a flowchart showing the operation of the selection controlling unit 125 according to Embodiment 5 of the present invention.
[0202]
First, the selection controlling unit 125 determines whether a state in which the microphone 110 is determined to be still by the movement determining unit 114 has been changed to a state in which the microphone 110 is determined to be moved by the movement determining unit 114 (S201). In the case where the state of the microphone 110 is determined to be changed (Yes in S201), the selection controlling unit 125 sets the coefficients al, a2, and a3 to "1" (S202).
[203]
On the other hand, in the case where the state of the microphone 110 is determined to be unchanged (No in S201), or after the coefficients al, a2, and a3 are set to "1" in step S202, the selection controlling unit 125 acquires the sound image localized position determined by the localization determining unit 122 (S203).
[204]
Now, in the case where the acquired sound image localization position is Lch localization (dir_flg = 0) (Lch localization in S203), the selection controlling unit 125 sets the coefficient al to "0" (S204). Also, in the case where the acquired sound image localization position is Cch localization (dir_flg = l) (Cch localization in S203), the selection controlling unit 125 sets the coefficient a2 to "0" (S205). Also, in the case where the acquired sound image localization position is Rch localization (dir_flg = 2) (Rch localization in S203), the selection controlling unit 125 sets the coefficient a3 to "0" (S206).
[205]
As described above, according to the sound pickup apparatus 101 in the present embodiment, the signal power of the Lch level calculating unit 112a, the Cch level calculating unit 112b, and the Rch level calculating unit 112c is promptly updated even after the microphone 110 is moved. That is to say, even in the case where some sound image localized positions have not been updated after the microphone 110 is moved, the mixing coefficients after the movement of the microphone 110 can be calculated with a higher accuracy. Consequently, the sound pickup apparatus 101 can achieve more stable stereo sound pickup.
[206]
Alternatively, the signal power which is not updated in the correcting unit 121 may be reset rather than replaced by a signal.
[0207]
The sound pickup apparatus 101 according to the present embodiment includes the correcting unit 121, but does not necessarily need to include the correcting unit 121. Also, the level calculating unit 112 according to the present embodiment includes the Lch level calculating unit 112a, the Cch level calculating unit 112b, and the Rch level calculating unit 112c, but does not necessarily need to include these processing units. For example, the sound pickup apparatus 101 may have a functional configuration as shown in FIG. 13.
[0208]
In this case, when the microphone 110 is determined to be moved, the mixing ratio calculating unit 113 may calculate the mixing coefficients of the sound image localized positions whose signal levels are not calculated yet after the microphone 110 is determined to be moved by using the signal levels of the sound image localized positions whose signal levels are already calculated after the microphone 110 is determined to be moved. Specifically, for example, in the case where only the signal levels in Lch localization are calculated after the microphone 110 is determined to be moved, the mixing ratio calculating unit 113 may calculate the mixing coefficients in Cch localization, and the mixing coefficient in Rch localization by using the signal levels in Lch localization.
[209]
Thereby, even with the sound pickup apparatus 101 shown in FIG. 13, effects similar to those of the sound pickup apparatus 101 shown in FIG. 10 can be achieved.
[210]
So far, the sound pickup apparatus 101 according to one aspect of the present invention is described based on the embodiments, however, the present invention is not limited to these embodiments. As long as not departing from the spirit of the present invention, various modifications made to the present embodiment by those skilled in the art, and embodiment which is constructed by mixing components in different embodiments are also included in the scope of the present embodiment.
[0211]
For example, each sound pickup unit may be placed in a different configuration from that in the above-described embodiments.
For example, as shown in FIG. 14A, the first to fourth sound pickup units 110a to 110d may be arranged to be placed at different positions for every 90 degree on an ellipse rather than a circle. Also, as shown in FIG. 14B, the first to fourth sound pickup units 110a to 110d may be placed so as to have the main axis directions of directivity from the center of each circle to the inner side, and separated by 90 degrees. Also, as shown in FIG. 14C, the first to fourth sound pickup units 110a to 110d may be arranged in a line. Even in this case, the sound pickup apparatus 101 can achieve effects similar to those of the sound pickup apparatus 101 according to the above-described embodiments.
[0212]
Also, as shown in FIG. 14D, the microphone 110 may include the first to fifth sound pickup units 110a to 110e. In FIG. 14D, the microphone 110 may include the first to fifth sound pickup units 110a to 110d are placed so as to have the main axis directions of directivity separated by 72 degrees. With the mixing coefficients calculated similarly to Embodiment 1, a stereo signal generated by the signal mixing unit 115 is equivalent to a signal picked up in stereo from two sound pickup directions which are different from each other by 144 degrees.
Thus, even with the sound pickup apparatus 101 provided with the microphone 110 shown in FIG. 14D, sound from a sound source located in the periphery of the microphone 110 can be picked up in stereo somewhat clearly although not so efficiently as the sound pickup apparatus 101 according to the above-described embodiment. The microphone 110 may include further more sound pickup units.
[214]
Also, in the above-described embodiment, the sound pickup system includes the monitor 104 and the camera 105, but does not necessarily need to include the monitor 104 and the camera 105. In the case where the sound pickup system does not include the monitor 104 and the camera 105, the sound pickup apparatus 101 cannot match an image with its sound image, but can pick up sound in stereo clearly from a sound source located in the periphery of the microphone 110. For example, in a voice conference system such as a teleconference system, the sound pickup apparatus 101 can generate a stereo signal which allows a sound image to be localized at a location according to the position of a sound source, thus clarity of sound can be improved.
[215]
Also, in the above-described embodiment, the mixing ratio calculating unit 113 calculates mixing coefficients, in each of four sound pickup units, by using signal levels of the sound pickup unit located next to the relevant sound pickup unit, but does not necessarily need to calculate the mixing coefficients in this manner. For example, the mixing ratio calculating unit 113 may refer to a table and calculate the mixing coefficients corresponding to signal levels of each sound pickup unit. In this case, the sound pickup apparatus 101 may include, for example, a storage unit which holds a table for storing mixing coefficients associated with the ratio of signal levels of respective sound pickup units. Even in this case, sound from a sound source can be picked up in stereo without measuring the position of the microphone.
[216]
Also, in the above-described embodiments, the sound pickup apparatus 101 calculates the mixing ratio by using reception signals transmitted from the sound pickup system of the communication partner site, but does not necessarily need to use the reception signals. For example, the sound pickup apparatus 101 may generate loudspeaker signals for calculating the mixing coefficients. In this case, the loudspeaker signal is preferably a signal for reproducing sound whose frequency does not interfere with the sound reproduced from the reception signal. Accordingly, the sound pickup apparatus 101 can calculate mixing coefficients without interrupting a conference.
[0217]
Also, in the above-described embodiments, the number of loudspeakers is two, but may be other number.
[0218]
Also, in the above-described embodiments, the sound pickup apparatus 101 includes each component shown in FIG. 3, but does not necessarily need to include all the components shown in FIG. 3. For example, the sound pickup apparatus 101 may include a part of the components shown in FIG. 3 as shown in FIG. 15.
[219]
FIG. 15 is a block diagram showing the function configuration of the sound pickup apparatus 101 according to a modification example of an embodiment of the present invention. In FIG. 15, the sound pickup apparatus 101 includes the microphone 110, the level calculating unit 112, the mixing ratio calculating unit 113, and the signal mixing unit 115.
[220]
Even the sound pickup apparatus 101 has the configuration as shown in FIG. 15,
effects similar to those of the sound pickup apparatus 101 according to Embodiment 1 can be achieved by calculating signal levels of the sound outputted from loudspeakers in the level calculating unit 112.
[0221]
Also, the following cases are included in the present invention.
[0222]
(1) Specifically, each of the above-described apparatus is a computer system including a microprocessor, ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk unit, a display unit, a keyboard, a mouse. The RAM or the hard disk unit stores a computer program. Each apparatus achieves its function by the operation of the microprocessor according to the computer program. The computer program is configured with a combination of multiple operation codes which instruct the computer in order to achieve predetermined functions.
[0223]
(2) A part or all of the components included in the above-mentioned apparatus may include 1 system LSI (Large Scale Integration). For example, as shown in FIG. 15, a system LSI 150 includes the level calculating unit 112, the mixing ratio calculating unit 113, and the signal mixing unit 115. The system LSI is a super-multifunctional LSI manufactured by integrating multiple component units into 1 chip, and is specifically, a computer system including a microprocessor, a ROM, a RAM, and the like. The RAM stores a computer program. The system LSI achieves its function by the operation of the microprocessor according to the computer program.
[224]
(3) A part or all of the components included in the above-mentioned apparatus may include an IC card which is detachably attached to each apparatus, or a single module. The IC card or the module is a computer system which includes a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the above-mentioned super-multifunctional LSI. The IC card or the module achieves its function by the operation of the microprocessor according to the computer program. The IC card or the module may have tamper resistant.
[225]
The present invention may be one of the methods shown above. Alternatively, the present invention may be a computer program which implements these methods by a computer, or digital signals which are formed with the computer program.
[226]
Also, the present invention may be computer readable recording medium, for example, a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a BD (Blu-ray Disc (registered trademark)), and a semiconductor memory, which records the computer program or the digital signal. Also, the present invention may be the digital signal recorded on these recording media.
[0227]
Also, the present invention may be transmission means which transmits the computer program or the digital signal via an electric telecommunication line, a radio or wire communication line, a network represented by the Internet, and a data broadcast.
[0228]
Also, the present invention may be a computer system including a microprocessor and a memory; the memory stores the above-mentioned computer program; and the microprocessor operates according to the computer program.
[229]
Also, the present invention may be implemented by other independent computer system through recording the program or the digital signal on the above-mentioned recording medium to be transferred, or through transferring the program or the digital signal via the network.
[230]
(5) The above-described embodiments and the modification may be combined. [Industrial Applicability]
[231]
The sound pickup apparatus according to one aspect of the present invention can clearly pick up sound in stereo from a sound source located in the periphery of a microphone, and is useful, especially as a video conference system, or a voice conference system.
[Reference Signs List]
[232]
100 First sound pickup system 101, 201 Sound pickup apparatus 102a, 102b Speaker 103 Table
104, 204 Monitor
105, 205 Camera
106a, 106b, 206a, 206b Loudspeaker 107 Communication network
110 Microphone
110a, 110b, 110c, 110d Sound pickup unit
111 Band extracting unit
112 Level Calculating Unit
112a Lch level calculating unit
112b Cch level calculating unit
112c Rch level calculating unit
113 Mixing ratio calculating unit
114 Movement determining unit
115 Signal mixing unit
116 Loudspeaker signal detecting unit
117 Updating unit
121 Correcting unit
122 Localization determining unit
123 Variation detecting unit
124 Variation determining unit
125 Selection controlling unit
126a Lch correction selecting unit
126b Cch correction selecting unit
126c Rch correction selecting unit
150 System LSI
200 Second sound pickup system
[CLAIMS]
[Claim 1] (Amended)
A sound pickup apparatus comprising:
a microphone Including 4 or more sound pickup units placed so as to have different main axis directions of directivity, the microphone being installed in a position where sound outputted from a loudspeaker, which reproduces a voice signal received from a communication partner site, can be picked up;
a level calculating unit configured to calculate a signal level of 0 sound outputted from the loudspeaker for each output signal of the 4 or more sound pickup units;
a mixing coefficient calculating unit configured to calculate a mixing coefficient adjusted to main axis directions of directivity of the 4 or more sound pickup units by using the signal level calculated by said level calculating unit, the mixing coefficient for generating a stereo signal Including R channel signal whose main axis of directivity is inclined to left with respect to the direction of the loudspeaker, and L channel signal whose main axis of directivity is inclined to right with respect to the direction of the loudspeaker; and
a signal mixing unit configured to generate a stereo signal by mixing output signals of the 4 or more sound pickup units by using a mixing coefficient calculated by said mixing coefficient calculating unit.
[Claim 2]
The sound pickup apparatus according to Claim 1, wherein the 4 or more sound pickup units are placed so as to have main axis directions of directivity separated by 90 degrees.
[Claim 3]
The sound pickup apparatus according to Claim 2, wherein said mixing coefficient calculating unit is configured to calculate the mixing coefficient so that the stereo signal is equivalent to a signal picked up in stereo from two sound pickup directions which 35 are different from each other by 180 degrees.
[Claim 4]
The sound pickup apparatus according to Claim 3,
wherein the mixing coefficient includes a first mixing coefficient for generating one part of the stereo signal, and a second mixing coefficient for generating another part of the stereo signal;
in each one of the 4 sound pickup units, said mixing coefficient calculating unit calculates the first mixing coefficient by using a signal level of another sound pickup unit located next to one side of the sound pickup unit, and calculates the second mixing coefficient by using a 0 signal level of another sound pickup unit located next to another side of the sound pickup unit; and
said signal mixing unit is configured to generate one part of the stereo signal by multiplying output signals of the 4 sound pickup units by the respective first mixing coefficients and adding the multiplied output 5 signals together, and generates another part of the stereo signal by multiplying output signals of the 4 sound pickup units by the respective second mixing coefficients and adding the multiplied the output signals together.
[Claim 5]
The sound pickup apparatus according to Claim 1, further comprising
a loudspeaker signal detecting unit configured to determine whether a loudspeaker signal for outputting sound from the loudspeaker is present or not,
wherein said level calculating unit is configured to calculate the signal level in a case where a loudspeaker signal is determined to be present by said loudspeaker signal detecting unit.
[Claim 6]
The sound pickup apparatus according to Claim 1, further comprising
a movement determining unit configured to determine whether said microphone is moved or not,
wherein said mixing coefficient calculating unit sets a higher frequency of calculation of the mixing coefficient for a case where said microphone is determined to be moved than for a case where said microphone is determined to be still.
[Claim 7]
The sound pickup apparatus according to Claim 6, wherein said movement determining unit determines whether said microphone is moved or not based on a rate of change of the mixing coefficient calculated by said mixing coefficient calculating unit.
[Claim 8]
The sound pickup apparatus according to Claim 1, further comprising
a localization determining unit configured to determine a sound image localized position from a loudspeaker signal for outputting sound from the loudspeaker,
wherein said mixing coefficient calculating unit is configured to calculate the mixing coefficient for each sound image localized position determined by said localization determining unit; and
said signal mixing unit combines output signals of the 4 or more sound pickup units by using an average of mixing coefficient for each sound image localized position calculated by said mixing coefficient calculating unit.
[Claim 9]
The sound pickup apparatus according to Claim 8, wherein said localization determining unit determines one of left localization, right localization, and center localization, as a sound image localized position, each localization indicating a relative positional relationship between the sound image localized position and the loudspeaker; and
said level calculating unit includes:
Lch level calculating unit configured to calculate the signal level in a case where sound image localized position is determined to be the left localization by said localization determining unit,
Rch level calculating unit configured to calculate the signal level in a case where sound image localized position is determined to be the right localization by said localization determining unit, and
Cch level calculating unit configured to calculate the signal level in a case where sound image localized position is determined to be the center localization by said localization determining unit.
[Claim 10]
The sound pickup apparatus according to Claim 8, further comprising
a movement determining unit configured to determine whether said microphone is moved or not,
wherein in a case where said microphone is determined to be moved, said mixing ratio calculating unit calculates a mixing coefficient of a sound image localized position whose signal level is not calculated yet after said microphone is determined to be moved by using a signal level of a sound image localized position whose signal level is already calculated after said microphone is determined to be moved.
[Claim 11]
The sound pickup apparatus according to Claim 1, further comprising
a band extracting unit configured to extract a signal in a predetermined frequency band from each of output signals of the 4 or more sound pickup units,
wherein said level calculating unit calculates the signal level using a signal extracted by said band extracting unit.
[Claim 12]
The sound pickup apparatus according to Claim 11, wherein the predetermined frequency band is a voice band.
[Claim 13] (Amended)
A sound pickup method for generating a stereo signal by using a microphone which has 4 or more sound pickup units placed so as to have different main axis directions of directivity, the microphone being installed in a position where sound outputted from a loudspeaker, which reproduces a voice signal received from a communication partner site, can be picked up,
said sound pickup method comprising:
calculating a signal level of sound outputted from the loudspeaker for each output signal of the 4 or more sound pickup units;
calculating a mixing coefficient adjusted to main axis directions of directivity of the 4 or more sound pickup units by using a signal level calculated in said calculating of a signal level, the mixing coefficient for generating a stereo signal including R channel signal whose main axis of directivity is inclined to left with respect to the direction of the loudspeaker, and L channel signal whose main axis of directivity is inclined to right with respect to the direction of the loudspeaker; and generating a stereo signal by mixing the output signals of the 4 or more sound pickup units by using a mixing coefficient calculated in said calculating of a mixing coefficient.
[Claim 14]
A program causing a computer to execute the sound pickup method according to Claim 13.
[Claim 15] (Amended)
An integrated circuit for generating a stereo signal by using a microphone which has 4 or more sound pickup units placed so as to have different main axis directions of directivity, the microphone being installed in a position where sound outputted from a loudspeaker, 30 which reproduces a voice signal received from a communication partner site, can be picked up,
said integrated circuit comprising:
a level calculating unit configured to calculate a signal level of sound outputted from the loudspeaker for each output signal of the 4 or more sound pickup units;
a mixing coefficient calculating unit configured to calculate a mixing coefficient adjusted to main axis directions of directivity of the 4 or more sound pickup units by using a signal level calculated by said level calculating unit, the mixing coefficient for generating a stereo signal including R channel signal whose main axis of directivity is inclined to left with respect to the direction of the loudspeaker, and L channel signal whose main axis of directivity is inclined to right with respect to the direction of the loudspeaker; and
a signal mixing unit configured to generate a stereo signal by mixing output signals of the 4 or more sound pickup units by using a mixing coefficient calculated by said mixing coefficient calculating unit.