JP4298466B2 - Sound collection method, apparatus, program, and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize sound collection with localization feeling of a sound image even when there is a speaker at a location where an opening angle viewed from a microphone is small. <P>SOLUTION: A covariance matrix computing part 104 calculates a covariance matrix from a sound reception signal, and stores it in a covariance matrix storing part 106 for each location of a speaker detected by a speaker location detecting part 105. Then, L(R) channel filter coefficient calculating parts 107<SB>L</SB>and 107<SB>R</SB>calculate L(R) channel filter coefficients from the stored covariance matrix and L(R) channel mixing coefficients under such conditions that their received speaker voice components are mixed by the L(R) channel mixing coefficients corresponding to the respective speaker locations, and set those L(R) channel filter coefficients to L(R) channel filters 102<SB>L1</SB>to 102<SB>LM</SB>and 102<SB>R1</SB>to 102<SB>RM</SB>. The outputs of the L channel filters 102<SB>L1</SB>to 102<SB>LM</SB>are added by an L channel adder 103<SB>L</SB>, and the outputs of the R channel filters 102<SB>R1</SB>to 102<SB>RM</SB>are added by an R channel adder 103<SB>R</SB>to obtain an L channel output signal and an R channel output signal. <P>COPYRIGHT: (C)2005,JPO&amp;NCIPI

Description

  The present invention relates to a sound collection method and apparatus for TV conferences, audio conferences, telephone calls, remote lectures, and the like.

FIG. 10 is a block diagram of a conventional sound collecting device. The sound pickup device of the prior art is composed of directional microphones 901 _L and 901 _R , and the main axis of the directivity is arranged with an opening angle of about 120 °.

  By arranging the two directional microphones in different directions, a difference occurs in the sound level picked up by the L channel and the R channel depending on the position of the speaker. By reproducing these output signals from two speakers, it is possible to reproduce the sound image with a sense of localization.

For example, since the speaker C in FIG. 10 is in the main axis direction of the L channel microphone 901 _L , the voice level of the collected speaker C is higher in the L channel, and a sound image is displayed on the speaker on the L channel side during playback. I'll pan. In addition, since the voice of the speaker A in the middle between the L-channel and R-channel microphones 901 _L and 901 _R is picked up by both microphones at almost the same level, a sound image is present between the L-channel and R-channel speakers. I'll pan.

In this way, the conventional technique can perform stereo sound collection with a sense of localization of the sound image.
Heitaro Nakajima et al., Applied Electroacoustics, Corona Publishing, edited by the Acoustical Society of Japan, pp. 262-268, 1979

  The above-described conventional sound collection method has the following problems.

  Due to the effect of voice attenuation, the voice level of speakers far away from the microphone is small and difficult to hear. Even if the sensitivity of the microphone is increased to an appropriate level for a speaker who is far away, the voice of the speaker close to the microphone becomes an excessive level.

  When the opening angle viewed from the microphone is small as in the case of speaker A and speaker B in FIG. 10, the level difference between the microphones of speaker A and speaker B is almost the same, and the sense of localization of the sound image cannot be obtained.

  Noise and the received signal reproduced from the speaker are collected by the microphone, and the sound is difficult to hear.

  When the conventional technology is applied to a TV conference or an audio conference in which a table is surrounded by a plurality of people, the above-described problem occurs, and it is difficult to perform communication with a sense of localization of sound images with high-quality audio.

  The object of the present invention is to realize a call with a volume that is easy to hear by setting the voice of a speaker far away from the microphone to an appropriate level, even when the speaker is at a position where the opening angle seen from the microphone is small. Sound collection method, apparatus, and program for realizing high-quality transmitted sound with suppressed noise and received signal by realizing sound collection with a sense of localization of sound image by setting a level difference between LR channels as a desired level difference Is to provide.

According to the first aspect of the present invention, the sound collection method comprises:
A speaker position detection stage for detecting a speaker position from a received sound signal received by each of a plurality of sound pickup means;
A covariance matrix calculation stage for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collection means;
Storing a covariance matrix for each speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix and the L channel mixing coefficient under the condition that each speaker voice component received by each of the plurality of sound pickup means is mixed by the L channel mixing coefficient corresponding to each speaker position. Calculating the L channel filter coefficient from the L channel filter coefficient calculating stage;
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix and the R channel mixing coefficient under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed by the R channel mixing coefficient corresponding to each speaker position. Calculating an R channel filter coefficient from the R channel filter coefficient calculating stage;
An L channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the L channel filter coefficient;
An R channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
An R channel addition stage for adding the output signals of the R channel filter stage;

  Speaker position is detected from sound signals received from a plurality of microphones, a covariance matrix is obtained, a filter coefficient that gives a desired level difference between LR channels is obtained for each speaker position, and the microphone sound reception signal is obtained using these filter coefficients. Is filtered for each channel, a stereo output signal having a desired level difference for each speaker position can be obtained.

According to the second aspect of the present invention, the sound collection method comprises:
A speaker position detection stage for detecting a speaker position and a noise section from a received signal received by each of a plurality of sound pickup means;
A covariance matrix calculation stage for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collection means;
Storing a covariance matrix for each noise interval and speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position and the noise component is suppressed. And an L channel filter coefficient calculation step of calculating an L channel filter coefficient from the L channel mixing coefficient,
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, and the noise component is suppressed. And an R channel filter coefficient calculating step of calculating an R channel filter coefficient from the R channel mixing coefficient;
An L channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the L channel filter coefficient;
An R channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
An R channel addition stage for adding the output signals of the R channel filter stage;

  Estimate the speaker position and noise interval, store the covariance matrix for each speaker position and the covariance matrix for noise, and use these to determine the L channel and R channel filter coefficients, with these filter coefficients: Each of the microphone sound reception signals is filtered and added to obtain an L channel output signal and an R channel output signal. This realizes noise suppression and good sound image localization in which the audio signal from each speaker has a level difference between the L channel and the R channel.

According to the third aspect of the present invention, the sound collection method comprises:
A transmission / reception detection stage for detecting a transmission interval, a reception interval, and a noise interval from a reception signal received by each of a plurality of sound collection means, an L channel reception signal and an R channel reception signal from a communication partner;
A speaker position detection stage for detecting a speaker position from a received sound signal received by each of a plurality of sound pickup means;
A covariance matrix calculation stage for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collection means;
Storing a covariance matrix for each reception interval, noise interval, and speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. Calculating an L channel filter coefficient from the stored covariance matrix and the L channel mixing coefficient;
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. An R channel filter coefficient calculation step of calculating an R channel filter coefficient from the stored covariance matrix and the R channel mixing coefficient;
An L channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the L channel filter coefficient;
An R channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
An R channel addition stage for adding the output signals of the R channel filter stage;

  Detecting the reception interval, transmission interval, and noise interval from the reception signal and microphone reception signal, estimating the speaker position if it is the transmission interval, and covariance matrix and noise covariance matrix for each speaker position And the echo covariance matrix are stored, and the L channel and R channel filter coefficients are obtained using them, and the microphone sound reception signal is filtered and added by these filter coefficients, respectively, and the L channel output signal and the R channel are added. Get the output signal. As a result, noise and echo are suppressed, and the voice signal from each speaker has a level difference between the L channel and the R channel, thereby realizing a high quality voice call and good sound image localization.

According to the fourth aspect of the present invention, the sound collection method comprises:
A transmission / reception detection stage for detecting a transmission interval, a reception interval, and a noise interval from a reception signal received by each of a plurality of sound collection means, an L channel reception signal and an R channel reception signal from a communication partner;
A speaker position detection stage for detecting a speaker position from a received sound signal received by each of a plurality of sound pickup means;
A covariance matrix calculating step of calculating a covariance matrix from the received sound signal received by each of the plurality of sound collecting means, the L channel received signal, and the R channel received signal;
Storing a covariance matrix for each reception interval, noise interval, and speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. Calculating an L channel filter coefficient from the stored covariance matrix and the L channel mixing coefficient;
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. An R channel filter coefficient calculation step of calculating an R channel filter coefficient from the stored covariance matrix and the R channel mixing coefficient;
An L channel filter stage for filtering a received sound signal, an L channel received signal, and an R channel received signal received by each of the plurality of sound collecting means, respectively, with the L channel filter coefficient;
An R channel filter step of filtering a received sound signal, an L channel received signal, and an R channel received signal received by each of the plurality of sound collecting means, respectively, with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
An R channel addition stage for adding the output signals of the R channel filter stage;

  Detects the reception interval, noise interval, and transmission interval from the received signals of multiple microphones, detects the speaker position in the transmission interval, obtains a covariance matrix, and obtains desired echo suppression, noise suppression, and speaker position for each speaker position. Filter coefficients that give the level difference between LR channels are obtained, and the microphone sound reception signal is filtered for each channel using those filter coefficients, so that echo and noise are suppressed, and a desired level difference is provided for each speaker position. A stereo output signal can be obtained. By adding filtering of the received signal to the sound collection method of the third aspect, echo suppression higher than that of the sound collection method of the third aspect is realized.

According to the first embodiment of the present invention, the sound collection method comprises:
A speaker voice level estimating step for estimating a voice level of each speaker from the stored covariance matrix of each speaker;
A gain calculating unit for calculating a gain for each speaker for outputting each speaker's voice at an appropriate level from the voice level of each speaker;
In the L channel filter coefficient calculation step, the gain for each speaker is further multiplied, the received signal component is suppressed, and the noise component is suppressed. From the stored covariance matrix and the L channel mixing coefficient, L channel filter coefficient is calculated,
In the R channel filter coefficient calculation step, the gain for each speaker is further multiplied, the received signal component is suppressed, and the noise component is suppressed, from the stored covariance matrix and the R channel mixing coefficient. R channel filter coefficients are calculated.

  Detects the reception interval, noise interval, and transmission interval from the received signals of multiple microphones, detects the speaker position in the transmission interval, obtains a covariance matrix, and obtains desired echo suppression, noise suppression, and speaker position for each speaker position. By providing a level difference between LR channels, obtaining filter coefficients for picking up sound emitted from each speaker at an appropriate level, and filtering the microphone sound reception signal for each channel using those filter coefficients, echo and noise are obtained. , The sound emitted from each speaker is picked up at an appropriate level, and a stereo output signal having a desired level difference for each speaker position can be obtained.

According to the second embodiment of the present invention, the sound collection method comprises:
The stored covariance is a gain for smoothing the frequency characteristic of the diagonal component of the stored covariance matrix having the highest power or the added value of the diagonal components of the stored covariance matrix. The method further comprises a whitening step of inputting a whitened covariance matrix multiplied to the matrix to the L channel filter coefficient calculation step and the R channel filter coefficient calculation step.

  By whitening the covariance matrix, a filter that does not depend on the frequency characteristics of the sound source can be obtained. Thereby, even if the frequency characteristic of the sound source changes, there is no change in the filter coefficient, and the change in timbre due to the processing of the present invention can be prevented.

According to the third embodiment of the present invention, the sound collection method comprises:
An FFT stage for converting a signal received by each of the plurality of sound pickup means and a time domain signal of the received signal into a frequency domain signal;
An IFFT stage for converting the output signal of the L channel addition stage and the R channel addition stage from a frequency domain signal to a time domain signal;
Each step is performed in the frequency domain.

  Thereby, a low calculation amount can be realized as compared with the calculation in the time domain.

According to the fourth embodiment of the present invention, the sound collection method comprises:
The L and R channel filter coefficient calculation stage, the L and R channel filter stage, and the L and R channel addition stage are divided into three or more channels of 1 to J channel filter coefficient calculation stage, 1 to J channel filter stage, and 1 to J. Replaced with the channel addition stage.

  An L channel filter coefficient and an R channel filter coefficient that form directivity that satisfies the following conditions are obtained from signals and received signals collected by a plurality of microphones. (Condition 1) The voice of a speaker who is far away from the microphone is set to an appropriate level (a level that is easy to hear). (Condition 2) Even when a speaker is at a position where the opening angle viewed from the microphone is small, the level difference of the LR channel is set to a desired level difference (a level difference with a sense of localization of a sound image). (Condition 3) Noise and received signal are suppressed. Next, the signal collected by the plurality of microphones and the received signal are filtered with the obtained L channel filter coefficient and R channel filter coefficient, and their outputs are added for each L channel and R channel.

  As a result, the voice levels of the speaker far from the microphone and the speaker close to the microphone are appropriate, and a call with a volume that is easy to hear is realized. Further, even when a speaker is located at a position where the opening angle viewed from the microphone is small, a desired level difference is obtained between the LR channels, and sound collection with a sense of localization of the sound image is realized. Furthermore, high-quality voice communication with suppressed noise and received signal is realized.

[First Embodiment]
FIG. 1 is a block diagram of a sound collecting apparatus according to a first embodiment of the present invention.

The sound collection device of the present embodiment includes microphones 101 _{1 to} 101 _M , L channel filters 102 _{L1 to} 102 _LM , R channel filters 102 _{R1 to} 102 _RM , L channel adder 103 _L , and R channel adder 103. _R , speaker position detection unit 105, covariance matrix calculation unit 104, covariance matrix storage unit 106, L channel filter coefficient calculation unit 107 _L , R channel filter coefficient calculation unit 107 _R , and L channel mixing A coefficient setting unit 203 _L and an R channel mixing coefficient setting unit 203 _R are included.

  In this embodiment, a speaker position is estimated, a covariance matrix for each speaker position is stored, L-channel and R-channel filter coefficients are obtained using these, and microphone reception is performed using these filter coefficients. The signals are filtered and summed to obtain an L channel output signal and an R channel output signal. As a result, the sound signal from each speaker has a desired level difference between the L channel and the R channel, and good sound image localization is realized.

First, the speaker position detection unit 105 detects the position of the speaker from the microphone reception signal received by the microphones 101 _{1 to} 101 _M. As a sound source position estimation method, for example, there is a method based on a cross-correlation method.

Assuming that there are M microphones, the delay time difference between the received sound signals obtained from the signals received by the i-th microphone 100 _i and the j-th microphone 100 _j is

And The delay time difference between the received sound signals can be obtained from the maximum peak position by obtaining the cross-correlation between the signals. Next, the mth sound receiving position is (x _m , y _m , z _m ), and the estimated sound source position is

It expresses. Delay time difference between estimated received sound signals obtained from these positions

Is represented by Formula (1).

Next, the delay time difference between the received signals

Multiplied by the speed of sound c and converted into a distance,

And measured value

And estimated value

Mean square error of

Is obtained, Equation (2) is obtained.

  Mean square error of equation (2)

If the solution for minimizing the difference is obtained, the estimated sound source position at which the error between the measured value and the estimated value of the delay time difference between the received sound signals is minimized can be obtained. However, since Equation (2) is a nonlinear simultaneous equation and difficult to solve analytically, it is obtained by numerical analysis using sequential correction.

  Estimated sound source position that minimizes Equation (2)

Is obtained by calculating the gradient at a certain point, correcting the estimated sound source position in the direction in which the error becomes smaller, and determining the point at which the gradient becomes 0. The correction formula is as shown in equation (3). .

As described above, the estimated sound source position where the error is minimized can be obtained by repeatedly calculating Expression (3).

Next, the covariance matrix calculation unit 104 obtains the covariance of the microphone sound reception signal and makes it a matrix. First, let the frequency domain conversion signal of the microphone received signal be X ₁ (ω) to X _M (ω). The covariance matrix of these signals

Is calculated by equation (9).

  Next, in the covariance matrix storage unit 106, the covariance matrix is based on the detection result of the speaker position detection unit 105.

Is the covariance matrix for each source location

Save as.

The L channel filter coefficient calculation unit 107 _L and the R channel filter coefficient calculation unit 107 _R calculate filter coefficients for collecting sounds emitted from the speakers with a desired level difference. First, the filter coefficients of the L channel filters 102 _{L1 to} 102 _LM and the R channel filters 102 _{R1 to} 102 _RM connected to the respective microphones are converted to the frequency domain, respectively, H _{L, 1} (ω) to H _{L, M} (Ω) and H _{R, 1} (ω) to H _{R, M} (ω). Next, these filter coefficients are converted into a matrix according to equations (10) and (11).

And

Also, let X _{Si, 1} (ω) to X _{Si, M} (ω) be the frequency domain transform signal of the microphone sound reception signal during the period when the i-th sound source is sounding.

  Where the filter coefficient matrix

The required condition is that the microphone sound signal X _{Si, 1} (ω) to X _{Si, M} (ω) of the i-th speaker is a filter coefficient matrix

When the filtered signals are added for each channel, the L-channel and R-channel speaker audio signals correspond to the i-th speaker positions (X _i , Y _i , Z _i ). This is a level difference P _Diff (X _i , Y _i , Z _i ) (a level difference that realizes a good sound image localization and is set in advance for each speaker position). Therefore, the mixing coefficient matrix of M rows so that the signal obtained by filtering and adding the signals of the respective sound sources has a desired level difference P _Diff (X _i , Y _i , Z _i ).

Equations (12) and (13), which are signals obtained by multiplying the microphone sound reception signal, respectively, are ideal conditions.

Mixing coefficient matrix realizing desired level difference P _Diff (X _i , Y _i , Z _i )

It is previously set for each speaker position L channel mixing coefficient setting unit 203 _L and R channel mixing coefficient setting unit 203 _R. For example, it is set as described below. FIG. 9 is a diagram showing an example of directivity realized in the present invention. In the arrangement of the microphone and the speaker shown in FIG. 9, there are a plurality of speakers at positions where the opening angle viewed from the microphone is small. In such a case, the stereo microphone of the prior art can hardly obtain a sense of localization of the sound image. Therefore, the present invention forms directivity that emphasizes the sense of localization of the sound image. For example, for speaker position 3, the L channel level is increased and the R channel level is decreased so that a large level difference is provided. At this time, the mixing coefficient matrix is

Set as follows. In this way, a level difference of 20 dB can be obtained for the utterance from the speaker position 3. If the mixing coefficient matrix is similarly set for other speaker positions, the directivity as shown in FIG. 9 can be obtained and good sound image localization can be realized.

  Next, the conditions of the equations (12) and (13) are changed to the filter coefficient matrix.

Is solved by least squares solution, Equations (14) and (15) are obtained. However, C _Si is a constant of weight for the sensitivity constraint of the sound source position, and the sensitivity constraint becomes stronger as the value increases.

  Thus, the equations (14) and (15) for obtaining the filter coefficient for obtaining the desired level difference are derived.

  Next, the L channel filter coefficient obtained by the equations (14) and (15)

And R channel filter coefficients

Are copied to the L channel filters 102 _{L1 to} 102 _LM and the R channel filters 102 _{R1 to} 102 _RM , respectively, and respectively filter the microphone reception signals. The filtered signals are added by adders 103 _L and 103 _R for each channel, and output as a stereo output signal.

  As described above, in this embodiment, the speaker position is detected from the received signals of a plurality of microphones, a covariance matrix is obtained, and a filter coefficient that gives a desired level difference between LR channels is obtained for each speaker position. By filtering the microphone sound reception signal for each channel using these filter coefficients, a stereo output signal having a desired level difference for each speaker position can be obtained.

[Second Embodiment]
A sound collection device according to a second embodiment of the present invention will be described. The block diagram of this embodiment is FIG. 1 which is the same as that of the first embodiment. In the present embodiment, a noise suppression function is added to the sound collection device of the first embodiment.

The sound collection device of the present embodiment includes microphones 101 _{1 to} 101 _M , L channel filters 102 _{L1 to} 102 _LM , R channel filters 102 _{R1 to} 102 _RM , L channel adder 103 _L , and R channel adder 103. _R , speaker position detection unit 105, covariance matrix calculation unit 104, covariance matrix storage unit 106, L channel filter coefficient calculation unit 107 _L , R channel filter coefficient calculation unit 107 _R , and L channel mixing A coefficient setting unit 203 _L and an R channel mixing coefficient setting unit 203 _R are included.

  In this embodiment, the speaker position and the noise interval are estimated, a covariance matrix for each speaker position and a covariance matrix for noise are stored, and L channel and R channel filter coefficients are obtained using these, and these are calculated. Each of the microphone sound reception signals is filtered and added with the filter coefficient of L, and an L channel output signal and an R channel output signal are obtained. As a result, noise suppression and good sound image localization in which the sound signal from each speaker has a level difference between the L channel and the R channel are realized.

First, the speaker position detection unit 105 detects a noise section and a speech section from the power of a microphone sound reception signal received by the microphones 101 _{1 to} 101 _M. For example, for each microphone reception signal, a short time average power (about 0.1 to 1 s) and a long time average power (about 1 s to 100 s) are obtained, and the ratio of the short time average power and the long time average power is noise. When it is less than the threshold of the section, it is determined as a noise section, and when it is equal to or more than the threshold of utterance, it is determined as an utterance section. If it is determined that the utterance section, the speaker position is detected as in the first embodiment.

  Next, the covariance matrix calculation unit 104 performs the covariance matrix in the same manner as in the first embodiment.

Is calculated.

  In the covariance matrix storage unit 108, the covariance matrix is based on the detection result of the speaker position detection unit 105.

Is the covariance matrix for each source location

And noise covariance matrix

Save as.

The L channel filter coefficient calculation unit 107 _L and the R channel filter coefficient calculation unit 107 _R pick up sounds emitted from the speakers with a desired level difference, and calculate filter coefficients for suppressing noise. Here, L channel filter connected to each microphone

And R channel filter

The following two conditions are required. The first is that the speaker audio signals of the L channel and the R channel shown in the equations (12) and (13) of the first embodiment are the i-th speaker positions (X _i , Y _i , Z _i). ), A desired level difference P _Diff (X _i , Y _i , Z _i ). The second condition is to suppress noise. When the noise components X _{N, 1} (ω) to X _{N, M} (ω) of the microphone reception signal are input to the filter unit, the output of each channel is 0. (16) and (17).

  Next, the conditions of Expression (12) and Expression (16), Expression (13), and Expression (17) are changed to the filter coefficient matrix.

Is solved by the least squares solution, equations (18) and (19) are obtained. However, C _Si is a constant of weight for the sensitivity constraint of the sound source position, and the sensitivity constraint becomes stronger as the value increases. C _N is a constant of the weight for noise suppression, and the noise suppression amount increases as the value increases.

  Thus, equations (18) and (19) for obtaining filter coefficients for suppressing noise and obtaining a desired level difference are derived.

  Next, the L channel filter coefficient obtained by Expression (18) and Expression (19)

And R channel filter coefficients

Are copied to the L channel filters 102 _{L1 to} 102 _LM and the R channel filters 102 _{R1 to} 102 _RM , respectively, and respectively filter the microphone reception signals. The filtered signals are added by adders 103 _L and 103 _R for each channel, and output as a stereo output signal.

  As described above, in this embodiment, speaker positions and noise intervals are detected from the received signals of a plurality of microphones, a covariance matrix is obtained, and noise suppression and the level between desired LR channels for each speaker position are obtained. By obtaining the filter coefficients that give the difference and filtering the microphone sound reception signal for each channel using those filter coefficients, it is possible to suppress noise and obtain a stereo output signal having a desired level difference for each speaker position. it can.

[Third Embodiment]
FIG. 2 is a block diagram of a sound collecting apparatus according to the third embodiment of the present invention.

The sound collection device of the present embodiment includes microphones 101 _{1 to} 101 _M , L channel filters 102 _{L1 to} 102 _LM , R channel filters 102 _{R1 to} 102 _RM , L channel adder 103 _L , and R channel adder 103. _R , transmission / reception detection unit 201, speaker position detection unit 105, covariance matrix calculation unit 104, covariance matrix storage unit 106, L channel filter coefficient calculation unit 107 _L , and R channel filter coefficient calculation unit 107 _R , an L channel mixing coefficient setting unit 203 _L, and an R channel mixing coefficient setting unit 203 _R.

  In the present embodiment, an echo (received signal collected by a microphone) suppression function is added to the sound collection device of any one of the first and second embodiments.

  In the present embodiment, a reception section, a transmission section, and a noise section are detected from the reception signal and the microphone reception signal, and the speaker position is estimated in the case of the transmission section, and the covariance matrix for each speaker position is The covariance matrix of noise and the covariance matrix of echo are stored, L channel and R channel filter coefficients are obtained using these, and the microphone sound reception signal is filtered and added with these filter coefficients, respectively. An output signal and an R channel output signal are obtained. As a result, noise and echo are suppressed, and the voice signal from each speaker has a level difference between the L channel and the R channel, thereby realizing a high quality voice call and good sound image localization.

First, the transmission / reception detecting unit 201 detects a reception section, a speech section, and a noise section from the microphone reception signals received by the microphones 101 _{1 to} 101 _M and the powers of the L channel and R channel reception signals. For example, for a received signal of each channel, a short time average power (about 0.1 to 1 s) and a long time average power (about 1 s to 100 s) are obtained, and a ratio between the short time average power and the long time average power is a reception interval. If it is equal to or greater than the threshold value, it is determined as the reception interval. Further, for each microphone reception signal, a short time average power (about 0.1 to 1 s) and a long time average power (about 1 s to 100 s) are obtained, and the ratio of the short time average power and the long time average power is a noise interval. If it is less than the threshold value, it is determined as a noise interval, and if it is greater than or equal to the utterance threshold value, it is determined as a speech interval.

  The speaker position detection unit 105 detects the speaker position in the same manner as in the first embodiment when the transmission / reception detection unit 201 determines that it is an utterance section.

  Next, the covariance matrix calculation unit 104 performs the covariance matrix in the same manner as in the first embodiment.

Is calculated.

  The covariance matrix storage unit 106 is based on the detection results of the transmission / reception detection unit 201 and the speaker position detection unit 105, and uses a covariance matrix.

Is the covariance matrix for each source location

And noise covariance matrix

And echo covariance matrix

Save as.

The L channel filter coefficient calculation unit 107 _L and the R channel filter coefficient calculation unit 107 _R collect sounds emitted from each speaker with a desired level difference, and calculate filter coefficients for suppressing echo and noise. . Here, L channel filter connected to each microphone

And R channel filter

The following three conditions are required. The first is that the speaker audio signals of the L channel and the R channel shown in the equations (12) and (13) of the first embodiment are the i-th speaker positions (X _i , Y _i , Z _i). ), A desired level difference P _Diff (X _i , Y _i , Z _i ). The second is a condition for suppressing the noise expressed by the equations (16) and (17) of the second embodiment. The third condition is a condition for suppressing the echo component. When the echo components X _{E, 1} (ω) to X _{E, M} (ω) of the microphone reception signal are input to the filter unit, Expressions (20) and (21) in which the output is 0.

  Next, the conditions of Expression (12), Expression (16), Expression (20), Expression (13), Expression (17), and Expression (21) are set as the filter coefficient matrix.

Is solved by the least squares solution, equations (22) and (23) are obtained. However, C _Si is a constant of weight for the sensitivity constraint of the sound source position, and the sensitivity constraint becomes stronger as the value increases. C _N is a constant of the weight for noise suppression, and the noise suppression amount increases as the value increases. _CE is a constant of weight for echo suppression, and the echo suppression amount increases as the value increases.

  As described above, the expressions (22) and (23) for obtaining the filter coefficients for suppressing the noise and the echo and obtaining the desired level difference are derived.

  Next, the L channel filter coefficient obtained by Equation (22) and Equation (23)

And R channel filter coefficients

Are copied to the L channel filters 102 _{L1 to} 102 _LM and the R channel filters 102 _{R1 to} 102 _RM , respectively, and respectively filter the microphone reception signals. The filtered signals are added by adders 103 _L and 103 _M for each channel and output as a stereo output signal.

  As described above, in the present embodiment, the reception interval, the noise interval, and the transmission interval are detected from the reception signals of a plurality of microphones, the speaker position is detected in the transmission interval, a covariance matrix is obtained, and echo suppression is performed. The filter coefficient that gives the desired level difference between the LR channels for each speaker position and the noise suppression is obtained, and the microphone reception signal is filtered for each channel by these filter coefficients to suppress the echo and the noise. A stereo output signal having a desired level difference for each person position can be obtained.

[Fourth Embodiment]
FIG. 3 is a block diagram of a sound collecting apparatus according to the fourth embodiment of the present invention.

The sound collection device of the present embodiment includes microphones 101 _{1 to} 101 _M , L channel filters 102 _{L1 to} 102 _LM , 301 _LL and 301 _LR , R channel filters 102 _{R1 to} 102 _RM , 301 _RL and 301 _RR , L Channel adder 103 _L , R channel adder 103 _R , transmission / reception detector 201, speaker position detector 105, covariance matrix calculator 104, covariance matrix storage 106, L channel filter coefficient The calculation unit 107 _L , the R channel filter coefficient calculation unit 107 _R , the L channel mixing coefficient setting unit 203 _L, and the R channel mixing coefficient setting unit 203 _{R are} configured.

This embodiment is a configuration in which filters 301 _LL , 301 _LR , 301 _RL , and 301 _RR for filtering received signals are added to the L channel filter and the R channel filter of the sound collection device of the third embodiment. Echo suppression higher than that of the embodiment is realized.

  First, the transmission / reception detecting unit 201 detects a reception section, a speech section, and a noise section as in the third embodiment.

  The speaker position detection unit 105 detects the speaker position in the same manner as in the first embodiment when the transmission / reception detection unit 201 determines that it is an utterance section.

  Next, the covariance matrix calculation unit 104 includes a covariance matrix including the received signal.

Is calculated. The frequency domain conversion signals of the microphone reception signal are X ₁ (ω) to X _M (ω), and the frequency domain conversion signals of the L channel and R channel reception signals are Z _L (ω) and Z _R (ω), respectively. To do. The covariance matrix of these signals

Is calculated by equation (24).

  Next, the covariance matrix storage unit 106 is based on the detection results of the transmission / reception detection unit 201 and the speaker position detection unit 105, and the covariance matrix

Is the covariance matrix for each source location

And noise covariance matrix

And echo covariance matrix

Save as.

Next, the L channel filter coefficient calculation unit 107 _L and the R channel filter coefficient calculation unit 107 _R pick up sounds emitted from the speakers with a desired level difference, and filter coefficients for suppressing echo and noise. Calculate First, the filter coefficients of the L channel filters 102 _{L1 to} 102 _LM and the R channel filters 102 _{R1 to} 102 _RM connected to the respective microphones are converted to the frequency domain, respectively, H _{L, 1} (ω) to H _{L, M} (Ω) and H _{R, 1} (ω) to H _{R, M} (ω), and the L channel filters for filtering the L and R channel received signals are F _{L, L} (ω) and F _{L, R} (ω ), And R channel filters for filtering the L and R channel reception signals are F _{R, L} (ω) and F _{R, R} (ω). Next, these filter coefficients are converted into a matrix according to equations (25) and (26).

And

  Where L channel filter

And R channel filter

The following three conditions are required.

First, each speaker voice signal of the output signals of the L channel and the R channel has a desired level difference P _Diff (X _i , Y) corresponding to the i-th speaker position (X _i , Y _i , Z _i ). _i , Z _i ). Similar to the first embodiment, this condition is expressed by Expression (26) and Expression (27).

  However, the mixing coefficient matrix

Where the 1st to Mth elements are the mixing coefficient matrix of the first embodiment.

Is an M + 2 × 1 matrix where the M + 1 and M + 2th elements are 0.

The second is a condition for suppressing noise. When the echo components X _{E, 1} (ω) to X _{E, M} (ω) of the microphone sound reception signal are input to the filter unit, Expressions (28) and (29) in which the output of each channel becomes 0 are as follows: This is the condition.

The third condition is a condition for suppressing the echo component. When the echo components X _{E, 1} (ω) to X _{E, M} (ω) of the microphone received signal and the received signals Z _L (ω) and Z _R (ω) are input to the filter unit, the output of each channel is Expressions (30) and (31) that become 0 are the conditions.

  Next, the conditions of Expression (26), Expression (28), Expression (30), Expression (27), Expression (29), and Expression (31) are set as the filter coefficient matrix.

Is solved by least squares solution, equations (32) and (33) are obtained. However, C _Si is a constant of the weight for the sensitivity constraint of the sound source position, and the sensitivity constraint becomes stronger as the value becomes larger. C _N is a constant of the weight for noise suppression, and the noise suppression amount increases as the value increases. _CE is a constant of weight for echo suppression, and the echo suppression amount increases as the value increases.

  Thus, the equations (32) and (33) for obtaining the filter coefficient for suppressing the noise and the echo and obtaining a desired level difference are derived.

  Next, the L channel filter coefficient obtained by Expression (32) and Expression (33)

And R channel filter coefficients

Are copied to the L channel filters 102 _{L1 to} 102 _LM , 301 _LL and 301 _LR and the R channel filters 102 _{R1 to} 102 _RM , 301 _RL and 301 _RR , respectively, and filter the microphone sound reception signal and the reception signal, respectively. The filtered signals are added by adders 103 _L and 103 _R for each channel, and output as a stereo output signal.

  As described above, in this embodiment, the reception interval, the noise interval, and the transmission interval are detected from the reception signals of a plurality of microphones, the speaker position is detected in the transmission interval, a covariance matrix is obtained, and echo suppression is performed. The filter coefficient that gives the desired level difference between the LR channels for each speaker position and the noise suppression is obtained, and the microphone reception signal is filtered for each channel by these filter coefficients to suppress the echo and the noise. A stereo output signal having a desired level difference for each person position can be obtained. Further, by adding a filter for filtering the received signal to the sound collecting device of the third embodiment, higher echo suppression is realized than in the sound collecting device of the third embodiment.

[Fifth Embodiment]
FIG. 4 is a block diagram of a main part of a sound collecting apparatus according to the fifth embodiment of the present invention.

  The sound collection device of this embodiment has a configuration in which a speaker voice level estimation unit 108 and a gain calculation unit 109 are added to any of the sound collection devices of the first to fourth embodiments. The speaker voice level estimation unit 108 estimates the voice level of each speaker from the covariance matrix of each speaker, and the gain calculation unit 109 determines that the output voice level of each speaker is appropriate from the voice level of each speaker. Calculate the level gain. As a result, the sound level of all speakers is set to an appropriate level, and sound collection at a volume that is easy to hear is realized. In addition, as in any of the sound collection devices of the first to fourth embodiments, echo sound suppression, noise suppression, and good sound localization with a level difference between channels are realized at the same time.

  First, the transmission / reception detection unit 201, the speaker position detection unit 105, the covariance matrix calculation unit 104, and the covariance storage unit 106 are processed in the same manner as the sound collection device of any of the first to fourth embodiments. Become.

The speaker voice level estimation unit 108 uses the voice level P _Si of each speaker as a covariance matrix for each speaker position stored in the covariance matrix storage unit 106.

And a single-channel mixing coefficient matrix for each speaker location

Is obtained using equation (34) or equation (35). Single channel mixing coefficient matrix

For example, a mixing coefficient matrix

Matrix that is the average of or mixing coefficient matrix

A matrix obtained by averaging is used.

Next, the gain calculation unit 109 calculates a gain A _Si for setting each speaker voice level P _Si to an appropriate level P _opt (a level that is easy to hear and set in advance). Gain A _Si can be obtained by equation (36).

The L channel filter coefficient calculation unit 107 _L and the R channel filter coefficient calculation unit 107 _R pick up the sound emitted from each speaker at an appropriate level, and the sound emitted from each speaker is set to a desired level on the LR channel. The filter coefficient for suppressing the echo and the noise is calculated as the difference. The conditions for suppressing echo and noise are the same as those in any of the sound collection devices of the first to fourth embodiments. The conditions for picking up the sound emitted from each speaker at an appropriate level and setting the sound emitted from each speaker to a desired level difference in the LR channel are the expressions (37) and (38) or ( 39) and formula (40).

  Next, conditions for suppressing echo and noise of any of the sound collection devices of the first to fourth embodiments and sound emitted from each speaker were collected at an appropriate level and emitted from each speaker. Expression (37) and Expression (38), or Expression (39) and Expression (40), which are conditions for making the sound have a desired level difference in the LR channel, are expressed as a filter coefficient matrix.

Or

Is solved by the least squares solution, Equation (41) and Equation (42), Equation (43) and Equation (44), Equation (45) and Equation (46), Equation (47) and Equation (48), It becomes.

  However, Equation (41) and Equation (42) were obtained under the condition that the sound emitted from each speaker was picked up at an appropriate level and the sound emitted from each speaker was set to a desired level difference in the LR channel. Filter coefficients, Equation (43) and Equation (44), pick up the sound emitted from each speaker at an appropriate level, and make the sound emitted from each speaker a desired level difference in the LR channel to suppress noise. (45) and (46) are the filter coefficients obtained under the above conditions, the sound emitted from each speaker is picked up at an appropriate level, and the sound emitted from each speaker is obtained at a desired level on the LR channel. The filter coefficients obtained under the condition that the echo and noise are suppressed as the difference, Equations (47) and (48) have a filter for filtering the received signal, and the sound emitted from each speaker is collected at an appropriate level. The sound from each speaker to the desired level on the LR channel And to a filter coefficient calculated by the condition for suppressing the echo and noise.

Next, it was calculated | required by Formula (41) and Formula (42), Formula (43) and Formula (44), Formula (45) and Formula (46), or Formula (47) and Formula (48), L channel filter coefficients and the R-channel filter coefficients, and L the channel filter _{102 L1 ~102 LM, 301 LL,} 301 LR, respectively copied to the R channel filter _{102 R1 ~102 RM, 301 RL,} 301 RR, microphone received sound signal And the received signal are respectively filtered. The filtered signals are added by adders 103 _L and 103 _R for each channel, and output as a stereo output signal.

  As described above, in the present embodiment, the reception interval, the noise interval, and the transmission interval are detected from the reception signals of the plurality of microphones, the speaker position is detected in the transmission interval, the covariance matrix is obtained, and the echo is detected. A level difference between desired LR channels is given for each suppression, noise suppression, and speaker position, and a filter coefficient for collecting sounds emitted from each speaker at an appropriate level is obtained. By filtering each channel, the echo and noise are suppressed, the sound emitted from each speaker is collected at an appropriate level, and a stereo output signal having a desired level difference for each speaker position is obtained. Can do.

[Sixth Embodiment]
FIG. 5 is a block diagram of a main part of a sound collecting apparatus according to the sixth embodiment of the present invention.

The sound collection device of this embodiment has a configuration in which a whitening unit 110 is added to any one of the sound collection devices of the first to fifth embodiments. Whitening unit 110, whitening in the preceding stage of each covariance matrix LR channel filter coefficient calculating section 107 _L, 107 _R. As a result, a filter coefficient that is not related to the frequency characteristic of the signal received by the microphone is obtained, and stable operation is possible.

  Whitening is the covariance matrix

Whitening gain that averages R _kk (ω) with the largest power among the diagonal components of

Or a whitening gain that averages the mean power of the diagonal components of the covariance matrix

This is done by multiplying These are represented by the formula (49) or the formula (50) and the formula (51) or the formula (52), respectively.

  However, β is a coefficient for adjusting the degree of whitening. When it is 1, it becomes complete whitening, and when it becomes 0, whitening is not performed.

  In the sound collection device of the fifth embodiment, a filter independent of the frequency characteristics of the sound source can be obtained by whitening the covariance matrix. Thereby, even if the frequency characteristic of the sound source changes, there is no change in the filter coefficient, and the change in timbre due to the processing of the present invention can be prevented.

  Since other parts are the same as those of any of the sound collecting apparatuses of the first to fifth embodiments, the description thereof is omitted.

[Seventh to ninth embodiments]
6 to 8 are block diagrams of sound collecting apparatuses according to seventh, eighth and ninth embodiments of the present invention, respectively.

The sound collection device of the present embodiment has a configuration in which FFT units 401 _{1 to} 401 _M , 501 _L , and 501 _R and IFFT units 402 _L and 402 _R are added to the sound collection device of any of the _{first to} sixth embodiments. is there. The FFT units 401 _{1 to} 401 _M perform FFT on each microphone sound reception signal and convert the time domain signal into a frequency domain signal. The FFT units 501 _L and 501 _R each perform FFT on the LR channel reception signal and convert the time domain signal into the frequency domain signal. IFFT sections 402 _L and 402 _R perform IFFT on the output signals of LR channel adders 103 _L and 103 _R , respectively, and convert the frequency domain signals into time domain signals. Further, all the processing units of the first to sixth embodiments are calculated in the frequency domain, and realize a low calculation amount as compared with the calculation in the time domain.

  Since other parts are the same as those of any of the sound collecting apparatuses of the first to sixth embodiments, the description thereof is omitted.

[Tenth embodiment]
A tenth embodiment of the present invention will be described.

The sound collection device of the present embodiment is one in which the number of output channels of any of the sound collection devices of the first to ninth embodiments is a J channel of 3 channels or more. L and R channel filter coefficient calculation units 107 _L and 107 _R and L and R channel filters 102 _{L1 to} 102 _LM , 301 _LL , 301 _LR , 102 _{R1 to} 102 _RM , 301 _RL , 301 _RR and L and R channel adders 103 _L, and 103 _R, and more 1~J channel filter coefficient calculation means 3 channels, and 1~J channel filter unit, a configuration obtained by replacing the 1~J channel adding means.

  By increasing the number of output channels, it is possible to collect sound on multiple channels of three or more channels.

  Since other parts are the same as those of any of the sound collecting apparatuses of the first to ninth embodiments, the description thereof is omitted.

  In addition to what is implemented by dedicated hardware, the present invention records a program for realizing the function on a computer-readable recording medium, and the program recorded on the recording medium is stored in a computer system. It may be read and executed. The computer-readable recording medium refers to a recording medium such as a floppy disk, a magneto-optical disk, a CD-ROM, or a storage device such as a hard disk device built in the computer system. Furthermore, a computer-readable recording medium is a server that dynamically holds a program (transmission medium or transmission wave) for a short time, such as when transmitting a program via the Internet, and a server in that case. Some of them hold programs for a certain period of time, such as volatile memory inside computer systems.

It is a block diagram which shows the sound collection apparatus of the 1st and 2nd embodiment of this invention. It is a block diagram which shows the sound collection device of the 3rd Embodiment of this invention. It is a block diagram which shows the sound collection device of the 4th Embodiment of this invention. It is a block diagram which shows the principal part of the sound collection device of the 5th Embodiment of this invention. It is a block diagram which shows the principal part of the sound collection device of the 6th Embodiment of this invention. It is a block diagram which shows the sound collection device of the 7th Embodiment of this invention. It is a block diagram which shows the sound collection device of the 8th Embodiment of this invention. It is a block diagram which shows the sound collection device of the 9th Embodiment of this invention. It is a figure which shows the directivity formed by this invention. It is a figure explaining the conventional sound collection method.

Explanation of symbols

101 _{1 to} 101 _M microphones 102 _{L1 to} 102 _LM , 301 _LL , 301 _LR L channel filters 102 _{R1 to} 102 _RM , 301 _RL , 301 _RR R channel filters 103 _L L channel adders 103 _R R channel adders 104 covariance matrix Calculation unit 105 Speaker position detection unit 106 Covariance matrix storage unit 107 _L L channel filter coefficient calculation unit 107 _R R channel filter coefficient calculation unit 108 Speaker voice level estimation unit 109 Gain calculation unit 110 Whitening unit 201 Transmission / reception detection unit 202 _L L channel speaker 202 _R R channel speaker 203 _L L channel mixing coefficient setting unit 203 _R R channel mixing coefficient setting unit 401 _{1 to} 401 _M , 501 _L , 501 _R FFT
402 _L , 402 _R IFFT
901 _L , 901 _R Conventional directional microphones 902 _L , 902 _R Directional characteristics formed by the present invention 903 Processing of the present invention

Claims (18)

A sound collection method,
A speaker position detection stage for detecting a speaker position from a received sound signal received by each of a plurality of sound pickup means;
A covariance matrix calculation stage for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collection means;
Storing a covariance matrix for each speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix and the L channel mixing coefficient under the condition that each speaker voice component received by each of the plurality of sound pickup means is mixed by the L channel mixing coefficient corresponding to each speaker position. Calculating the L channel filter coefficient from the L channel filter coefficient calculating stage;
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix and the R channel mixing coefficient under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed by the R channel mixing coefficient corresponding to each speaker position. Calculating an R channel filter coefficient from the R channel filter coefficient calculating stage;
An L channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the L channel filter coefficient;
An R channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
And an R channel addition step of adding the output signals of the R channel filter step. A sound collection method,
A speaker position detection stage for detecting a speaker position and a noise section from a received signal received by each of a plurality of sound pickup means;
A covariance matrix calculation stage for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collection means;
Storing a covariance matrix for each noise interval and speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position and the noise component is suppressed. And an L channel filter coefficient calculation step of calculating an L channel filter coefficient from the L channel mixing coefficient,
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, and the noise component is suppressed. And an R channel filter coefficient calculating step of calculating an R channel filter coefficient from the R channel mixing coefficient;
An L channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the L channel filter coefficient;
An R channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
And an R channel addition step of adding the output signals of the R channel filter step. A sound collection method,
A transmission / reception detection stage for detecting a transmission interval, a reception interval, and a noise interval from a reception signal received by each of a plurality of sound collection means, an L channel reception signal and an R channel reception signal from a communication partner;
A speaker position detection stage for detecting a speaker position from a received sound signal received by each of a plurality of sound pickup means;
A covariance matrix calculation stage for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collection means;
Storing a covariance matrix for each reception interval, noise interval, and speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. Calculating an L channel filter coefficient from the stored covariance matrix and the L channel mixing coefficient;
An R channel mixing coefficient setting stage for presetting an R channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. An R channel filter coefficient calculation step of calculating an R channel filter coefficient from the stored covariance matrix and the R channel mixing coefficient;
An L channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the L channel filter coefficient;
An R channel filter stage for filtering each received signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
And an R channel addition step of adding the output signals of the R channel filter step. A sound collection method,
A transmission / reception detection stage for detecting a transmission interval, a reception interval, and a noise interval from a reception signal received by each of a plurality of sound collection means, an L channel reception signal and an R channel reception signal from a communication partner;
A speaker position detection stage for detecting a speaker position from a received sound signal received by each of a plurality of sound pickup means;
A covariance matrix calculating step of calculating a covariance matrix from the received sound signal received by each of the plurality of sound collecting means, the L channel received signal, and the R channel received signal;
Storing a covariance matrix for each reception interval, noise interval, and speaker position;
An L channel mixing coefficient setting step for presetting an L channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. Calculating an L channel filter coefficient from the stored covariance matrix and the L channel mixing coefficient;
An R channel mixing coefficient setting step for setting in advance an R channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. An R channel filter coefficient calculation step of calculating an R channel filter coefficient from the stored covariance matrix and the R channel mixing coefficient;
An L channel filter stage for filtering a received sound signal, an L channel received signal, and an R channel received signal received by each of the plurality of sound collecting means, respectively, with the L channel filter coefficient;
An R channel filter step of filtering a received sound signal, an L channel received signal, and an R channel received signal received by each of the plurality of sound collecting means, respectively, with the R channel filter coefficient;
An L channel addition stage for adding the output signals of the L channel filter stage;
And an R channel addition step of adding the output signals of the R channel filter step. A speaker voice level estimating step for estimating a voice level of each speaker from the stored covariance matrix of each speaker;
A gain calculating unit for calculating a gain for each speaker for outputting each speaker's voice at an appropriate level from the voice level of each speaker;
In the L channel filter coefficient calculation step, the gain for each speaker is further multiplied, the received signal component is suppressed, and the noise component is suppressed. From the stored covariance matrix and the L channel mixing coefficient, L channel filter coefficient is calculated,
In the R channel filter coefficient calculation step, the gain for each speaker is further multiplied, the received signal component is suppressed, and the noise component is suppressed, from the stored covariance matrix and the R channel mixing coefficient. Calculating R channel filter coefficients;
The sound collection method according to claim 1.   The stored covariance is a gain for smoothing the frequency characteristic of the diagonal component of the stored covariance matrix having the highest power or the sum of the diagonal components of the stored covariance matrix. 6. The whitening step according to claim 1, further comprising a whitening step of multiplying a matrix and inputting a whitened covariance matrix to the L channel filter coefficient calculation step and the R channel filter coefficient calculation step. Sound collection method. An FFT stage for converting a signal received by each of the plurality of sound pickup means and the received signal from a time domain signal to a frequency domain signal;
An IFFT stage for converting the output signal of the L channel addition stage and the R channel addition stage from a frequency domain signal to a time domain signal;
Each of the above steps is performed in the frequency domain,
The sound collection method according to claim 1. The L and R channel filter coefficient calculation stage, the L and R channel filter stage, and the L and R channel addition stage are divided into three or more channels of 1 to J channel filter coefficient calculation stage, 1 to J channel filter stage, and 1 to J. Replaced with the channel addition stage,
The sound collection method according to claim 1. A sound collecting device,
Speaker position detecting means for detecting a speaker position from a received sound signal received by each of a plurality of sound collecting means;
A covariance matrix calculating means for calculating a covariance matrix from the received sound signals received by each of the plurality of sound collecting means;
Covariance matrix storage means for storing the covariance matrix for each speaker position;
L channel mixing coefficient setting means for presetting an L channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix and the L channel mixing coefficient under the condition that each speaker voice component received by each of the plurality of sound pickup means is mixed by the L channel mixing coefficient corresponding to each speaker position. L channel filter coefficient calculating means for calculating L channel filter coefficients from:
R channel mixing coefficient setting means for setting in advance an R channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix and the R channel mixing coefficient under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed by the R channel mixing coefficient corresponding to each speaker position. R channel filter coefficient calculating means for calculating R channel filter coefficients from
L channel filter means for filtering received sound signals received by each of the plurality of sound collection means, respectively, with the L channel filter coefficients;
R channel filter means for filtering the received sound signal received by each of the L channel plural sound collecting means with the R channel filter coefficient;
L channel addition means for adding the output signals of the L channel filter means;
A sound collecting device comprising: R channel adding means for adding the output signals of the R channel filter means. A sound collecting device,
Speaker position detecting means for detecting a speaker position and a noise section from a received sound signal received by each of a plurality of sound collecting means;
A covariance matrix calculating means for calculating a covariance matrix from the received sound signals received by each of the plurality of sound collecting means;
Covariance matrix storage means for storing the covariance matrix for each noise interval and speaker position;
L channel mixing coefficient setting means for presetting an L channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position and the noise component is suppressed. And L channel filter coefficient calculation means for calculating an L channel filter coefficient from the L channel mixing coefficient,
R channel mixing coefficient setting means for setting in advance an R channel mixing coefficient corresponding to each speaker position;
The stored covariance matrix under the condition that each speaker speech component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, and the noise component is suppressed. And R channel filter coefficient calculating means for calculating an R channel filter coefficient from the R channel mixing coefficient,
L channel filter means for filtering received sound signals received by each of the plurality of sound collection means, respectively, with the L channel filter coefficients;
R channel filter means for filtering the received sound signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
L channel addition means for adding the output signals of the L channel filter means;
A sound collecting device comprising: R channel adding means for adding the output signals of the R channel filter means. A sound collecting device,
A transmission / reception detecting means for detecting a transmission section, a reception section, and a noise section from a reception signal received by each of a plurality of sound collection means, an L channel reception signal and an R channel reception signal from a communication partner;
Speaker position detecting means for detecting a speaker position from a received sound signal received by each of a plurality of sound collecting means;
A covariance matrix calculating means for calculating a covariance matrix from the received sound signals received by each of the plurality of sound collecting means;
Covariance matrix storage means for storing the covariance matrix for each reception interval, noise interval, and speaker position;
L channel mixing coefficient setting means for presetting an L channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. L channel filter coefficient calculation means for calculating an L channel filter coefficient from the stored covariance matrix and the L channel mixing coefficient;
R channel mixing coefficient setting means for setting in advance an R channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. R channel filter coefficient calculating means for calculating an R channel filter coefficient from the stored covariance matrix and the R channel mixing coefficient;
L channel filter means for filtering received sound signals received by each of the plurality of sound collection means, respectively, with the L channel filter coefficients;
R channel filter means for filtering the received sound signal received by each of the plurality of sound collecting means with the R channel filter coefficient;
L channel addition means for adding the output signals of the L channel filter means;
A sound collecting device comprising: R channel adding means for adding the output signals of the R channel filter means. A sound collecting device,
A transmission / reception detecting means for detecting a transmission interval, a reception interval, and a noise interval from a received sound signal received by each of a plurality of sound collection means, and an L channel reception signal and an R channel reception signal from a communication partner;
Speaker position detecting means for detecting a speaker position from a received sound signal received by each of a plurality of sound collecting means;
Covariance matrix calculating means for calculating a covariance matrix from the received sound signal received by each of the plurality of sound collecting means, the L channel received signal, and the R channel received signal;
Covariance matrix storage means for storing the covariance matrix for each reception interval, noise interval, and speaker position;
L channel mixing coefficient setting means for presetting an L channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the L channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. L channel filter coefficient calculating means for calculating an L channel filter coefficient from the stored covariance matrix and the L channel mixing coefficient;
R channel mixing coefficient setting means for setting in advance an R channel mixing coefficient corresponding to each speaker position;
Each speaker voice component received by each of the plurality of sound pickup means is mixed with the R channel mixing coefficient corresponding to each speaker position, the received signal component is suppressed, and the noise component is suppressed. R channel filter coefficient calculating means for calculating an R channel filter coefficient from the stored covariance matrix and the R channel mixing coefficient;
L channel filter means for filtering the received sound signal, the L channel received signal, and the R channel received signal received by each of the plurality of sound collecting means, respectively, with the L channel filter coefficient;
R channel filter means for filtering the received sound signal, the L channel received signal, and the R channel received signal received by each of the plurality of sound collecting means, respectively, with the R channel filter coefficient;
L channel addition means for adding the output signals of the L channel filter means;
A sound collecting device comprising: R channel adding means for adding the output signals of the R channel filter means. Speaker voice level estimating means for estimating the voice level of each speaker from the stored covariance matrix of each speaker;
A gain calculating unit for calculating a gain for each speaker for outputting each speaker's voice at an appropriate level from the voice level of each speaker;
The L channel filter coefficient calculation means further multiplies the gain for each speaker, suppresses the received signal component, and suppresses the noise component, from the stored covariance matrix and the L channel mixing coefficient. L channel filter coefficient is calculated,
The R channel filter coefficient calculation means further multiplies the gain for each speaker, suppresses the received signal component, and suppresses the noise component, from the stored covariance matrix and the R channel mixing coefficient. Calculating R channel filter coefficients;
The sound collection device according to claim 9. The stored covariance is a gain for smoothing the frequency characteristic of the diagonal component of the stored covariance matrix having the highest power or the added value of the diagonal components of the stored covariance matrix. A whitening unit that multiplies the matrix and inputs the whitened covariance matrix to the L channel filter coefficient calculation unit and the R channel filter coefficient calculation unit;
The sound collecting device according to claim 9. FFT means for converting a signal received by each of the plurality of sound pickup means and the received signal from a time domain signal to a frequency domain signal;
IFFT means for converting the output signal of the L channel addition means and the R channel addition means from a frequency domain signal to a time domain signal;
Each means operates in the frequency domain,
The sound collection device according to claim 9. The L and R channel filter coefficient calculation means, the L and R channel filter means, and the L and R channel addition means are divided into three or more channels, 1 to J channel filter coefficient calculation means, 1 to J channel filter means, and 1 to J. Replaced with channel addition means,
The sound collecting device according to claim 9.   A sound collection program for causing a computer to execute the sound collection method according to claim 1.   A recording medium on which the sound collecting program according to claim 17 is described.

Images