JP3424761B2 - Sound source signal estimation apparatus and method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention is, for example, under an environment noisy around, is use when extracting a desired sound signal, a desired source signal estimation unit estimates the sound source and
Regarding the method .

[0002]

2. Description of the Related Art One of the important problems in conventional speech recognition.
First, there is a problem in that a predetermined voice signal is separated from noise or voice of a person other than the target person. For example, in an office or the like, there are computer fan sounds, air conditioning sounds, and the like in the surroundings. In addition, engine noise and running noise are extremely large in an automobile, and the environment is about -20 dB in terms of S / N ratio. As described above, when the voice recognition device is used in an environment where noise is present in the surroundings, there is a problem that a desired voice signal to be the target of voice recognition is masked by these noises, and the voice signal cannot be accurately detected. Moreover, if a voice signal is taken in together with noise, the recognition rate of voice recognition is fatally lowered.

To solve such a problem, the following method has been conventionally proposed. [1] Method of utilizing directivity of microphone. [2] Filter assuming a stationary signal (Wiener Filter)
How to use. [3] A method of estimating and removing an interfering sound by adaptive signal processing.

An example of the above-mentioned method [1] is shown in FIG.
For example, the target sound source is sufficiently far compared to the size of the microphone array (distance from microphone 0 to microphone (M-1)), and as shown in FIG. It should be possible. At this time, assuming that the distance between the microphones is b, the sound waves of the signal coming from a direction having an angle of ω ₀ with the straight line (the vertical line in the figure) formed by the microphone array are displaced by a time proportional to bsinω _0. Input to the next microphone.

Now, assuming that the response of the microphone 0 to the sound wave from this sound source is a signal at the time t of y _0,0 (t) = s (t) (1), this results in a distance of b × m. The response of the microphone m is y _{m, 0} (t) = s (t + mbsinω ₀ ) ... (2).

It is also assumed that the interfering wave comes from the direction of the angle ω ₁ with respect to the microphones 0 to M-1. This disturbing wave (noise)
If the response of the microphone 0 to y is y _0,1 (t) = n (t) (3), the response of the microphone m is y _{m, 1} (t) = n (t + mbsinω ₁ ).・ It becomes (4).

Therefore, the outputs of the microphones 0 to M-1 are delayed by the delay circuits 0 to M- by the time difference corresponding to the target signal.
After delaying by 1, adder 1 adds
Synchronous addition can be performed on the target sound. The output of an arbitrary microphone m can be expressed as y _m (t) = y _{m, 0} (t) + y _{m, 1} (t) (5). a delay time, by setting the Mbsinomega _0, the signal after the delay z _m (t) _{is, zm (t) = y m} (t-mbsinω 0) = y m, 0 (t-mbsinω 0) + y m, ₁ (t−mbsinω ₀ ) = s (t + mbsinω ₀ −mbsinω ₀ ) + n (t + mbsinω ₁ −mbsinω ₀ ) = s (t) + n (t + mbsinω ₁ −mbsinω ₀ ) ... (6)

As a result, the output u (t) of the adder 1 that adds for all m is as follows.

[0009]

[Equation 1]

Therefore, the amplitude of the target signal s (t) becomes M times, but the interference wave signal n (t) is delayed averaged as shown in the above equation, and the low-pass filter is used. Will pass, and its level will decrease.

On the other hand, the above method [2] is to extract a signal component in a desired frequency band from a signal detected by a microphone by using a Wiener Filter ( or a bandpass filter).

Further, an example of the above method [3] is shown in FIG. For example, when trying to detect the voice s (t) at the time t with the microphone 15, the engine noise n ₁ (t) generated by the engine 11 is also detected with the microphone 15. Therefore, the pickup 12 that detects only engine noise (vibration related to engine noise) (does not detect the sound s (t)) is directly installed in the engine room. The output n ₂ (t) of the pickup 12 is supplied to an adaptive filter 13 having a variable tap Wi and controlled to a desired characteristic to obtain a signal h (t) n ₂ (t). Then, this signal is supplied to the subtractor 14, and the output y (t) (= s of the microphone 15 is output.
Subtracting from (t) + n ₁ (t)), an error signal of the following equation is obtained. y (t) -h (t) n ₂ (t) = s (t) + n ₁ (t) -h (t) n ₂ (t) (8)

This error signal is supplied to the LMS circuit 16,
The coefficient (variable tap Wi) of the applied filter 13 that minimizes the root mean square is calculated using the gradient method.

That is, in the normal LMS algorithm,
The sample sequence of y (t), n ₂ (t), h (t) is converted into y (k), n ₂ (k), h using the time index k.
Expressed as (k), the output h (k) * n of the applied filter 13
₂ (k) is expressed by the following equation.

[0015]

[Equation 2]

Then, the update amount dWi is calculated from the following equation, and dWi = −αn ₂ (ki) (y (k) −h (k) * n ₂ (k) ) (10) This update According to the amount dWi, the variable tap Wi is changed by Wi ← Wi + dWi by changing by the update amount dWi. Incidentally, “←” is a symbol that means replacement, and α is a positive constant.

[0017]

However, the above-mentioned conventional method has the following problems. That is, since the method [1] of adding the outputs of the plurality of microphones after delaying them by a predetermined time is based on the synchronous addition, it is expected that the S / N is improved only by 3 dB per one microphone even in an ideal state. I can't. Therefore, it is difficult to actually apply the voice recognition.

The method [2] for extracting a signal in a predetermined frequency band with a filter is effective only for stationary disturbing sounds, and when the target signal and the noise frequency band overlap. , The target signal component is also lost.

Furthermore, the method [3] shown in FIG. 5 has to pick up only the signals related to the interfering sound, which is often impossible in practice.

The present invention has been made in view of the above circumstances, and by accurately estimating a sound source, it is possible to reliably detect a target signal without being influenced by noise. Is intended to provide.

[0021]

A sound source signal estimation apparatus of the present invention detects a signal generated from a plurality of sound sources and a sound source from a sound source to a detection means corresponding to an output of the detection means. Transfer function estimating means for estimating a transfer function of a signal, sound source estimating means for estimating a sound source that generated a signal and generating an estimated signal based on the transfer function and a signal generated from a sound source, and a transfer function and a sound source estimating means Estimated by
Correction means for correcting the transfer function based on the estimated signal, and the sound source estimation means corrects the estimated signal by the transfer function corrected by the correcting means.

[0022]

Transforming means for Fourier transforming the signal detected by the detecting means may be further provided. The sound source signal designating method of the present invention detects a signal generated from a plurality of sound sources, and a transfer function of a signal from the sound source to a detection position in the detecting step corresponding to an output in the processing of the detecting step. A transfer function estimation step for estimating, a sound source estimation step for estimating a sound source that generated a signal and generating an estimated signal based on the transfer function and the signal, a transfer function and a sound
And a correction step for correcting the transfer function based on the estimated signal estimated by the processing of the source estimation step, and in the processing of the sound source estimation step, the estimated signal is calculated based on the transfer function corrected by the processing of the correction step. It is characterized by correction.

[0024]

[Action] is Te source signal estimation apparatus and method odor <br/> above configuration, estimates the transfer function of the signal generated from a sound source from the sound source to the signal detection position, corrects the estimated signal by the correction transfer function By accurately estimating the sound source, it is possible to reliably detect the target signal without being affected by noise.

[0025]

Embodiments of the present invention will be described below with reference to the drawings.

1 to 3 relate to an embodiment of the present invention. FIG. 1 is a block diagram showing a configuration of an embodiment of a sound source signal estimating apparatus of the present invention, and FIG. 2 is an embodiment of FIG. 3 is an explanatory view for explaining the arrangement of microphones of the signal detection unit 21 in FIG.
FIG. 3 is an explanatory diagram illustrating the positions of a microphone and a sound source of the signal detection unit 21 in the embodiment of FIG.

First, the principle of interference sound separation (suppression) in this embodiment will be described. Suppose now that there are S sound sources (this sound source includes not only a sound source that generates a signal to be extracted but also a sound source that generates noise to be suppressed), and a signal detector that detects a sound signal from this sound source ( Microphone)
Let there be M. The transfer function from the sound source s to the signal detector m is H _{m, s} . At this time, the signal x _s (t) in the sound source s or X _s (ω) which is the Fourier transform,
The relationship with the signal y _m (t) in the signal detector m or Y _m (ω) which is the Fourier transform is expressed by the following equation. Y _m (ω) = H _{m, s} X _s (ω) (11) where m = 0,1, ..., M-1, s = 0,1 ,.
.., S-1.

The above equation can be expressed as a matrix as follows. Y = HX (12)

Here, Y, H, and X are respectively represented by the following equations, and [] ^t in the equations represents the transpose of the matrix. Y = [Y ₀ (ω), Y ₁ (ω), ..., Y _M-1 (ω)] ^t (13) X = [X ₀ (ω), X ₁ (ω), ... .., X _S-1 (ω)] ^t・ ・ ・ (14)

[0030]

[Equation 3]

Therefore, if H is a regular square matrix, X, that is, each sound source can be independently obtained from the above equation.
Even when H is not square or regular, X can be estimated in the least squares sense by using the generalized inverse matrix H _L ^-1 .

By the way, in order to estimate the sound source X as described above, H must be known or accurately estimated. The present embodiment relates to a sound source signal estimation device characterized by performing the H estimation and application control.

The structure of a specific example to which the above principle is applied will be described below with reference to the drawings. Now, it is assumed that there are M sound sources s (s = 0, 1, ..., M−1). The signal detection unit 21 includes M microphones m (m = 0, 1,
..., M-1). A space from an arbitrary sound source s to an arbitrary microphone m has a transfer function H _{m, s} .

In the sound source signal estimating apparatus of this embodiment,
As shown in FIG. 1, the signal detection unit 21 detects an audio signal input by the microphone m and outputs a signal y _m (t) (y _m (k) in the sample sequence). The output y _{m of} this microphone _m
(T) is transformed into the Fourier domain by the Fourier transform unit 22 and becomes Y _m (ω), and the sound source estimation error minimizing unit 2
4 is output.

The output y _m (t) of the microphone m is also output to the transfer function estimating unit 23, and the transfer function estimating unit 2
3 is a transfer function ratio H _{m, s} / H _{m0, s0} using y _m (t).
To estimate. This H _{m, s} / H _{m0, s0} is output to the sound source estimation error minimizing unit 24, and the sound source estimation error minimizing unit 24 outputs H
_The sound source signal X ′ is estimated using _{m, s} / H _{m0, s0} and Y _m (ω). The estimated sound source signal X ′ and the Y _m (ω) are output to the transfer function updating unit 25. The transfer function updating unit 25 obtains the update amount dH of H by using the sound source signals X ′ and Y _m (ω), and updates H, and the sound source estimation error minimizing unit 24 uses the updated H to obtain the sound source signal. It is configured to re-estimate X '.

Next, the operation of the sound source signal estimating apparatus of this embodiment having the above configuration will be described.

The arrangement of the M microphones of the signal detecting section 21 is known, and as shown in FIG.
The position vector Am represents the coordinates of an arbitrary microphone m. The microphone m detects a signal (voice signal or noise) from the sound sources 0 to M-1, and the detected signal y _m (t) is calculated by the Fourier transform unit 2
2 and output to the transfer function estimation unit 23.

The Fourier transform unit 22 outputs the microphone output y
Fourier transform is performed according to the following equation so that _m (t) can be easily processed. Y _m (ω) = however _{∫g (t) y m (t} ) exp (-jωt) ··· (16), g (t) is a window function, such as the Hanning window. The Y _m (ω) thus obtained is output to the sound source estimation error minimizing section 24 and the transfer function updating section 25.

On the other hand, the transfer function estimating unit 23 estimates the transfer function H _{m, s} between the sound source s and the microphone m. In this estimation, the transfer function Hm, s is modeled by the propagation time of sound waves, and further formulated by the difference in propagation time between microphones,
Then, the transmission time difference is regarded as a function depending on the direction of the sound source, and the direction is obtained.

First, the modeling of the transfer function H _{m, s} will be described. In this embodiment, the transfer function H _{m, s} is approximated (modeled) as follows. H _{m, s} = Rh _{m, s} exp (−jωτ _{m, s} ) (1
7) Here, Rh _{m, s} represents the amplitude, and τ _{m, s} is a value (arrival time) obtained by dividing the distance from the sound source s to the microphone m by the sound velocity.

Further, if a predetermined microphone, that is, microphone 0 (m = 0 microphone), is used, the signal Y ₀ (ω) (= H _{0, s} X _s (from H _{0, s} X _s ( ω))
With reference to, the equation (12) is transformed as follows. Y '= H'X ... (18) However, H'is represented by the following formula.

[0042]

[Equation 4]

By using the equation (17), the element H _{m, s} / H _{0, s} in the equation (19) can be expressed by the following equation. H _{m, s} / H _{0, s} = (Rh _{m, s} / Rh _{0, s} ) exp (jωΔτ _{m, s} ) (20) where Δτ _{m, s} is the transmission time difference and Δτ _{m, s s} = τ _{m, s} −τ _{0, s} (21).

Assuming that Rh _{m, s} / Rh _{0, s} ≈1, H ′ (ratio of transfer functions) depends only on the transmission time difference Δτ _{m, s} between the microphone 0 and the microphone m of the sound source s. . That is,
The above-mentioned propagation time difference Δτ _{m, s} should be estimated first. The estimation of Rh _{m, s} / Rh _{0, s} is adaptively performed by the transfer function updating unit 25 described later.

Therefore, the propagation time difference Δτ _{m, s} is estimated. The estimation of this transmission time difference Δτ _{m, s} will be described.

Assuming that the distance from each microphone to the sound source is sufficiently larger than the distance between the microphones, the transmission time difference Δτ _{m, s} depends only on the direction of the sound source s. As shown in FIG. 3, when the unit vector of the direction and the direction of the sound source s from the microphone 0 arranged at the origin is Bs, the following equation holds. That is, the transmission time difference Δτ _{m, s} is
Is represented by the inner product of the position vector Am ^t and B s. Δτ _{m, s} = Amt ^t Bs (22)

There are several methods for specifically determining the transmission time difference. For example, the transmission is performed by searching the peak of the cross-correlation coefficient between the two signals obtained by the reference microphone and each of the other microphones. The time difference can be calculated. That is, for example, the cross-correlation coefficient C _{r0, m} (τ) between the signal y ₀ (t) obtained by the microphone 0 and the signal y _m (t) obtained by the microphone m is expressed by the following equation. Then, S, which is the maximum, are obtained in descending order, and this τ is used as the transmission time difference.

[0048]

[Equation 5]

Further, if y ₀ (t), ..., Y _M-1 (t) are used as they are, τ that maximizes the cross-correlation coefficient C _{r0, m} (τ) is sufficiently obtained. In such a case, in such a case, y _m (t) is band-limited by using an appropriate filter, and the same operation is performed for each band, so that the transmission time difference between the sound sources having different frequency components is obtained. Can be asked.

When the number of microphones is M = 2, τ obtained as described above can be used as the transmission time difference from each sound source to each microphone, but when M ≧ 3, which transmission time difference is obtained? It must be determined consistently that corresponds to one source signal.

Therefore, for example, from the transmission time difference τ of the microphones 0 and 1, as shown in FIG. 3, in the directions (θ1, θ2) in which the angle from the x-axis in the xy plane is θ1 and the angle with respect to the xy plane is θ2. Assuming that there is a sound source s, the other microphones have a τ
Is selected and the outputs of the microphones 0 to M-1 are added together with their transmission time differences, and y (θ1,
θ2) is obtained.

[0052]

[Equation 6]

Here, S (θ1, θ2) is the direction (θ1,
It is a unit vector to θ2) and is expressed by the following equation.   S (θ1, θ2) = [cos θ2 cosθ1, cosθ2 sinθ1, sinθ2]                                                       ... (25)

This y for all τ combinations
The power (intensity) of (θ1, θ2) is detected and S
Take a combination of τ and use it as the direction of the sound source.
Thereby, the estimated value of H'is calculated.

The sound source estimation error minimizing unit 24 estimates the sound source signal X ′ from the transfer function matrix H ′ thus estimated by the transfer function estimating unit 23 and the output Y of the Fourier transform unit 22.

That is, the generalized inverse matrix H _L ^-1 of H'is obtained, and X'is estimated by X = H _L ^-1 Y (26).

Here, instead of finding the generalized inverse matrix,
It is also possible to estimate X by minimizing the evaluation function E1 (X) such that E1 (X) = | Y-HX || ² ... (27) based on the least squares. In addition, ‖‖ indicates the L2 norm.

Further, by applying a constraint on X, it is possible to obtain an appropriate solution for X even if the rank r of H is r ≦ S.

From the transfer function matrix H ′ estimated by the transfer function estimation unit 23 and X ′ estimated by the sound source estimation error minimizing unit 24, the transfer function updating unit 25 updates the estimated value of H ′. . That is, the evaluation function E (H) for H = ‖
The gradient method is applied to Y-HX ² to update H.

Using equation (17) and equation (22), equation (1
Rewriting 9) gives the following equation.

[0061]

[Equation 7]

Next, the amplitude Rh _{m, s of} H and the transmission time difference Δ
The gradient method is applied to each of τ _{m, s} .

First, the amplitude parameter r _s = [1, Rh
_{, S} , ..., Rh _{M-1, s} ], and the evaluation function E (H) is differentiated by the amplitude parameter r _s ,
It becomes the following formula.

[0064]

[Equation 8]

Similarly, Δτ _s = [1, Δτ _{1, s} , ...
, Δτ _{M-1, s} ] (transmission time difference parameter) is applied, and the evaluation function E (H) is subjected to variable differentiation with the transmission time difference parameter Δτ _s to obtain the following equation.

[0066]

[Equation 9]

However, h _s is defined as H = [h ₀ , h ₁ , ..., H _S-1 ] (32).

These ∂E (H) / ∂r _s and ∂E (H)
Update H by setting / ∂Δτ _s to dH. That is, H = H + dH (33) is updated.

The updated H is output again to the sound source estimation error minimizing section 24. Then, the sound source X is re-estimated, and H is updated using the re-estimated X. This process is repeated until both converge, or a certain number of times is made the upper limit, and the result of the repeated calculation is used as the estimated value of the separated sound source signal.

As described above, according to the sound source signal estimating apparatus of this embodiment, the transfer function updating unit 25 applies the gradient method for the amplitude parameter r _s and the transfer time difference parameter Δτ _s to the evaluation function E (H). The transfer function H is updated, the sound source signal X is re-estimated by the updated H, and the reestimated X is further estimated.
Since the process of updating H is repeated by obtaining the estimated value of the sound source signal, an accurate transfer function H can be obtained,
Therefore, each sound source signal can be estimated independently by this transfer function. This sound source includes not only the signal source of the audio signal to be extracted but also a sound source that generates noise. Therefore, by estimating each sound source signal independently, the sound completely separated from the noise can be obtained. You can get a signal.

[0071]

According to the source signal estimation apparatus and method of the present invention as described in the foregoing, as well as estimates the transfer function of the signal generated from the sound source to the detection position from the sound source, complement
Since corrects the estimated signal by a positive transfer function, by accurately estimating the sound source, without being affected by the noise, there is an effect that the signal to the target can be reliably detected.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of a sound source signal estimation device of the present invention.

FIG. 2 is an explanatory diagram illustrating the arrangement of microphones of the signal detection unit 21 in the embodiment of FIG.

FIG. 3 is an explanatory diagram illustrating positions of a microphone and a sound source of the signal detection unit 21 in the embodiment of FIG.

FIG. 4 is a block diagram showing a configuration of an example of a conventional sound source signal estimation device.

FIG. 5 is a block diagram showing the configuration of another example of the conventional sound source signal estimation device.

[Explanation of symbols]

21 signal detector 22 Fourier transform unit 23 Transfer Function Estimator 24 Sound source estimation error minimization unit 25 Transfer Function Update Unit

Continuation of front page (56) References JP-A-4-64075 (JP, A) JP-A-4-66888 (JP, A) JP-A-6-195097 (JP, A) Yasuda Masuzou, Kakusho, General Reverse Application of matrix to acoustic signal processing, Journal of Acoustical Society of Japan, Japan, June 1, 1981, Vol. 37, No. 6, p. 290-294 Hiroshi Kanai, Position estimation of multiple sound sources in one-dimensional space by singular value decomposition using tapered window and pole estimation, The Acoustical Society of Japan, Japan, September 1, 1989, Vol. 45, No. 9, p. 681-688 Liu Xiang, Masato Abe, Kenichi Kido, Effect of moving noise source on sound source localization method by cross spectrum between two microphone outputs, The Acoustical Society of Japan, Japan, July 1, 1990, Vol. 46, No. 7, p. 523-530 Huangji, Noboru Onishi, Noboru Sugie, Multiple Sound Source Localization System Using Time Difference Histogram, The Robotics Society of Japan, Japan, February 15, 1991, Vol. 9, No. 1, p. 29 −38 Hitoshi Nagata, Masato Abe, Kenichi Kido, Estimation of Source Waveform by Multiple Sensors, Journal of Acoustical Society of Japan, Japan, April 1, 1991, Vol. 47, No. 4, p. 268-273 Huangji, Noboru Onishi, Noboru Sugie, Separation of Multiple Sound Sources Using Directional Information of Sound Source, Journal of the Robotics Society of Japan, Japan, August 15, 1991, Vol. 9, No. 4, p. 9-14 (58) Fields investigated (Int.Cl. ⁷ , DB name) G10L 15/28 G10L 21/02 H04R 3/04 JISST file (JOIS)

Claims (3)

(57) [Claims] 1. A detection means for detecting signals generated from a plurality of sound sources, and a transfer function estimation means for estimating a transfer function of the signal from the sound source to the detection means in response to an output of the detection means. A sound source estimation unit that estimates the sound source that generated the signal based on the transfer function and the signal and generates an estimated signal; and the transfer function and the sound source estimation unit that estimate the sound source.
Based on the estimated signal, and a correcting means for correcting the transfer function, the sound source estimation means, and corrects the estimated signal based on the transfer function corrected by said correction means sound source Signal estimator. 2. Transforming means for Fourier transforming the signal
The sound source signal according to claim 1, further comprising:
Estimator. 3. A detection step of detecting signals generated from a plurality of sound sources, and a transfer function of the signal from the sound source to a detection position in the detection step corresponding to an output in the processing of the detection step. A transfer function estimating step of estimating; a sound source estimating step of estimating the sound source that generated the signal and generating an estimated signal based on the transfer function and the signal ; processing of the transfer function and the sound source estimating step; Estimated by
Based on said estimated signal, and a correction step of correcting the transfer function, the processing of the sound source estimation step, correcting the estimation signal based on the transfer function corrected by the processing of the correction step A method for estimating a sound source signal, comprising: