KR20100010356A - Sound source separation method and system for using beamforming - Google Patents

Abstract

In the sound source separation method and system using the beamforming technology according to the present invention, when an integrated voice signal including at least one voice is input through a microphone array, the integrated voice signal is divided into frames by writing a hanning window in a predetermined direction. windowing processing unit; A DFT converter for converting each individual voice signal classified for each frame into a frequency axis; At least one TF estimator for estimating a transfer function of an individual voice signal by obtaining an impulse response for a frame converted into a frequency axis; A noise estimator for estimating a noise signal by removing transfer functions of individual speech signals from the integrated speech signal; At least one voice signal extracting unit for removing remaining individual voice signals from the integrated voice signals except for the individual voice signals to be extracted among the individual voice signals; And at least one voice signal detector for extracting an individual voice signal from which noise is removed by removing a noise portion provided through a noise estimator in a transfer function of an individual voice signal to be detected, wherein at least one sound source is input at the same time. Even though the sound sources are separated from each other, there is an advantage of separately storing and managing the first sound source.

Description

Sound source separation method and system for beamforming technology

The present invention relates to sound source separation required for voice communication / recognition.

Here, sound source separation refers to separating at least two sound sources when input to the input device (microphone array) at the same time.

The conventional noise reduction system using a microphone array array has at least one microphone array, a short section analyzer connected to each of the microphone arrays, an echo canceller, a directional noise canceller, and an adaptation of filter weight update on / off based on the presence or absence of frontal sound. Adaptive beamforming processing unit, front-side sound detection unit using the correlation of the signal between the microphone, the front noise detection unit, post-filtering and overlapping to control the residual noise based on the presence and absence of the residual noise and overlapping And an overlap (add) processing unit.

In beamforming, when the microphone array is used, the gain of the input signal varies depending on the angle due to the difference in the signal input to each microphone. This forms a directivity pattern according to the angle.

1 is a directivity pattern in the case where the front face of the microphone array is 90 degrees.

This is shown in Equation 1 below.

[Equation 1]

Where f = frequency, N = number of microphones, d = spacing between microphones,

는 amplitude weight이고 는 phase weight이다.

Is the amplitude weight and is the phase weight.

과

을 조절함으로써 원하는 각도의 방향으로 바꿔주는 것이다. Therefore, beamforming technology eliminates the directivity pattern generated by using a microphone array.

and

By adjusting, you change the direction of the desired angle.

In this way, only the signal in the desired direction can be received.

Thereafter, a frequency domain blind source separation (FDBSS) technique is performed.

The FDBSS technique is a technique for separating two sound sources when they are mixed and performed on the frequency axis. On the frequency axis, the algorithm implementation is simpler and the computation time is reduced.

The mixed signal of two input sources is short-time Fourier transform (STFT) to change the frequency axis. The sound source is then transformed into separate signals through three stages of independent component analysis (ICA).

First step is linear transformation.

If the number of microphones is larger than the number of sound sources, this step reduces the dimensions of the input signal to the dimensions of the sound source through transformation (V). In general, the dimension reduction part is included in the ICA because the number of microphones is larger than the number of sound sources.

Step 2 calculates the value of the frequency domain of the separated signal by multiplying the signal processed in step 1 by a unitary matrix (B).

In the third step, the separation matrix (V * B) obtained in steps 1 and 2 is gradually improved by using a learning rule.

The above procedure obtains the separated signal and takes localization.

Positioning distinguishes which direction the sound source separated by the ICA is coming from.

The next step is permutation.

In this stage, the direction of the previously separated sound source is kept unchanged.

The next step is scaling & smoothing.

The step is to adjust the size of the sound source separated signal so that it is not distorted. This can be solved by calculating the pseudo inverse of the separation matrix used for sound source separation.

Frequency responses sampled to L points with an interval of fs / L (fs: sampling frequency) in the FDBSS are represented by periodic signals having a period L / fs on the time axis.

This is not realistic as a periodic infinite-length filter.

So, in general, you use a filter in which the signal has one period on the time axis.

Using this filter results in loss of signal and poor separation performance.

This requires a smoothing step to solve.

By multiplying the Hanning window, where both ends get closer to zero, the frequency response is smoother, reducing signal loss and improving separation.

It is FDBSS to separate sound sources through this method.

However, the conventional beamforming technique can receive only a signal in a desired direction by adjusting the directivity pattern of the microphone array, but there is a problem in that performance decreases when there are other sound sources around the direction. The conventional beamforming technique is able to adjust the direction pattern in the desired direction to some extent, but it is difficult to make it point out exactly in the desired direction.

In addition, the FDBSS technique has a performance difference in constraints such as the number of sound sources, reverberation, and user position shift. In addition, there was a problem that the compensation of the missing feature compensation in terms of speech recognition.

When two people speak at the same time, there is a problem that the performance of speech recognition is significantly reduced as the signals are mixed.

Therefore, the present invention is to solve the conventional problems as described above, the object of the present invention is a sound source using a beamforming technology that separates each voice to improve the performance of the voice communication / recognizer in the situation where two voices at the same time To provide a separation method and system.

According to an aspect of the sound source separation system using the beamforming technology according to the present invention for achieving the above object, a windowing processing unit for covering the window on the integrated voice signal input through the beam-formed at least one microphone array; A DFT converter for converting a signal covered by a window into a frequency axis through the windown processor; A TF estimator for estimating a transfer function having feature values of the two or more different voice signals from the windowed signal; A noise estimator for removing noise of individual speech signals from a transfer function having characteristic values of two or more different speech signals estimated by the TF estimator; And a voice signal detector extracting the different individual voice signals from the noise-removed voice signal.

The transfer function estimator estimates a transfer function using an impulse response obtained through a value converted through the DFT converter.

The DFT converter is a Discrete Fourier Transform (DFT).

The TF estimator is equal to the number of different sound sources.

The system may further include at least one voice signal extracting unit which removes the remaining individual voice signals except the individual voice signals to be extracted from the individual voice signals provided through the TF estimator from the integrated voice signal provided through the DFT converter. Include.

The windowing processor, the length of the hanning window is 32ms, the moving section is 16ms.

The TF estimator estimates a transfer function TF by obtaining an impulse response between microphone arrays for a predetermined time with respect to a voice signal in a predetermined direction.

The voice signal detector may further include an IDFT converter configured to convert individual voice signals on a frequency axis into individual voice signals on a time axis.

After detecting the transfer function of the individual voice signal, send the signal in the corresponding direction and compare the magnitude of the value of the signal. When the signal gain is 1, it is determined that the signal of the desired direction is correctly received. The value may further include a TF accuracy check unit for determining that the transfer function of the individual voice signal is incorrect.

According to an aspect of a sound source separation method using a beamforming technology according to the present invention, the windowing processing step of covering the window on the integrated voice signal input through the beam-formed at least one microphone array; A DFT conversion step of converting a window-covered signal to a frequency axis through the windown processing step; A TF estimating step of estimating a transfer function having characteristic values of said two or more different speech signals from the windowed signal; A noise estimation step of removing noise of individual voice signals from a transfer function having characteristic values of two or more different voice signals estimated through the TF estimation step; And a voice signal detecting step of extracting the different individual voice signals from the noise canceled voice signal.

The transfer function estimating step estimates a transfer function using an impulse response obtained through a value converted through the DFT converter.

Here, the DFT transform step is a Discrete Fourier transform (DFT).

Meanwhile, the TF estimating step is performed by different number of sound sources.

The method may further include a voice signal extraction step of removing individual voice signals except for individual voice signals to be extracted from the individual voice signals provided through the TF estimation step from the integrated voice signal provided through the DFT conversion step. It includes more.

In the windowing step, the length of the hanning window is 32ms and the moving section is 16ms.

The TF estimating step estimates a transfer function TF by obtaining an impulse response between microphone arrays for a predetermined time with respect to the voice signal in a predetermined direction.

The voice signal detecting step may further include an IDFT processing step of converting individual voice signals on the frequency axis into individual voice signals on the time axis.

After detecting the transfer function of the individual voice signal, send the signal in the corresponding direction and compare the magnitude of the value of the signal. When the signal gain is 1, it is determined that the signal of the desired direction is correctly received. If the value comes out, it may further include a step of verifying the TF accuracy to determine that the transfer function (Transfer Function) of the individual voice signal is incorrect.

As described above, according to the sound source separation method and system using the beamforming technology according to the present invention, even if at least one or more sound sources are input at the same time, there is an excellent effect that can be separately stored and managed separately or the first sound source can be managed separately have.

Hereinafter, a preferred embodiment of a sound source separation method and system using a beamforming technology according to the present invention will be described in detail with reference to the accompanying drawings. At this time, it will be understood by those of ordinary skill in the art that the system configuration described below is a system cited for the purpose of the present invention and does not limit the present invention to the following system.

2 is a diagram illustrating a directional noise removing system using a conventional microphone array array, wherein at least one microphone array 10, a short-term analysis unit 20 connected to the microphone array 10, an echo canceller 30, Adaptive beamforming processing unit 40 for directional noise removal and filter weight update on / off based on presence / absence of frontal sound, frontal sound detection unit 50, residual sound using frontal signal correlation between microphones It includes a post-filtering unit 60 and the overlap and add processing unit 70 for controlling the residual noise based on the noise removal and the presence / absence of the front sound.

Speech input through each microphone array 10 analyzes a frequency domain through each short-term analysis unit 20.

For example, one frame corresponds to 2.56 ms and the moving section is 128 ms. Therefore, at 16Khz sampling, 256ms are sampled at 4,096, and Hanning window can be used.

Then, real-FFT is used for DFT, and source code uses ETSI standard feature extraction program.

The directional noise is removed by the adaptive beamforming processor 40.

The adaptive beamforming processing unit 40 uses a generalized sidelobe canceller (hereinafter referred to as "GSC").

This is then equivalent to canceling the echo by estimating the path of the far-end signal to the array in the speaker.

3 is a view showing the configuration of a sound source separation system using a beamforming technology according to the present invention, the sound source separation system using a beamforming technology according to the present invention is at least one microphone array 10, microphone array 10 Each of the connected short-term analysis unit 20, echo cancellation unit 30, front sound detection unit 50, post-filtering unit 60 and overlap and add processing unit 70, the range setting (windowing) ( 100, a DFT converter 200, at least one TF estimator 300, a noise estimator 400, at least one voice signal extractor 500 and at least one voice signal detector 600 The voice signal detector 600 includes an IDFT converter 610.

When the integrated voice signal including at least one voice is input through the microphone array 10, the range setting unit 100 divides the integrated voice signal into a frame by writing a hanning window in a predetermined direction. In this case, the range setting unit 100 may be provided with an integrated voice signal through the microphone array 10 through the short-term analysis unit 20 and the echo canceller 30.

In this case, the range setting unit 100 has a length of 32 ms and a moving section is 16 ms.

In addition, the DFT converter 200 converts each individual voice signal divided for each frame into the frequency axis through the window setting unit 100.

In addition, the at least one TF estimator 300 obtains an impulse response for the frame converted into the frequency axis through the DFT converter 200 and estimates a transfer function of the individual voice signal. In this case, the TF estimator 300 estimates a transfer function TF by obtaining an impulse response between the microphone arrays for a predetermined time with respect to the voice signal in a predetermined direction.

The noise estimator 400 detects individual voice signals detected through a transfer function estimated through each TF estimator 300 from the integrated voice signal converted into the frequency axis through the DFT converter 200. Remove to estimate the noise signal.

In addition, the at least one voice signal extractor 500 provides the remaining individual voice signals except the individual voice signals to be extracted from the individual voice signals provided through the TF estimator 300 through the DFT converter 200. From the integrated voice signal.

The at least one voice signal detector 600 extracts an individual voice signal from which noise is removed by removing a noise portion provided through the noise estimator 400 from the individual voice signal to be detected through a transfer function. do. In this case, the voice signal detector 600 further includes an IDFT converter 610 for converting individual voice signals on the frequency axis into individual voice signals on the time axis.

General functions and detailed operations of the above-described elements will be omitted, and the operations will be described based on operations corresponding to the present invention.

First, the microphone array 10 receives an integrated voice signal in which two voice signals are mixed, and provides the received voice to the range setting unit 100. In this case, the signal input through the microphone array 10 is a voice signal slightly different due to the gap between the microphone array 10.

Then, the range setting unit 100 divides the provided integrated voice signal into frames of 32 ms intervals by writing a hanning window in a predetermined direction. In this process, the divided frames are cut while moving by 16ms.

Meanwhile, the range setting unit 100 has a predetermined direction for covering the hanning window, and the number of the hanning windows may vary depending on the number of persons, but is not limited thereto.

Subsequently, the DFT converter 200 converts each individual voice signal divided for each frame into the frequency axis through the window setting unit 100.

Thereafter, the TF estimator 300 obtains an impulse response for the frame converted into the frequency axis through the DFT converter 200 and estimates a transfer function of the individual voice signal. In this case, the TF estimator 300 estimates a transfer function of two individual voice signals or estimates a transfer function of each individual voice signal through two TF estimators 300. Can be. In this case, the TF estimator 300 estimates a transfer function TF by obtaining an impulse response between the microphone arrays for a predetermined time with respect to the voice signal in a predetermined direction.

When the transfer function of the individual voice signal is estimated through the TF estimator 300 or the respective TF estimator 300, the noise estimator 400 moves to the frequency axis through the DFT converter 200. A noise signal is estimated by removing individual speech signals detected through a transfer function estimated by the TF estimator 300 from the converted integrated speech signal.

Subsequently, the voice signal extractor 500 separates the individual voices other than the transfer functions of the individual voice signals to be extracted from among the transfer functions of the individual voice signals provided through the TF estimator 300. The transfer function of the signal is removed from the integrated voice signal provided through the DFT converter 200. Then, the individual voice signal to be extracted can be extracted.

Thereafter, the at least one voice signal detector 600 removes the noise portion provided through the noise estimator 400 from the individual voice signal to be detected through a transfer function, and removes the noise from the individual voice signal. Extract In this case, the voice signal detector 600 converts the individual voice signals on the frequency axis into individual time signals on the frequency axis through the IDFT converter 610.

Next, a sound source separation method using the beamforming technique according to the present invention having the above configuration will be described with reference to FIG. 4.

First, when an integrated voice signal including at least one voice is input through the microphone array 10, the frame is divided into a frame by writing a hanning window in a predetermined direction (S1). Here, in the windowing process step S1, the length of the hanning window is 32ms, and the moving section is 16ms.

Subsequently, each individual voice signal divided for each frame is converted into a frequency axis (S2).

Then, the impulse response of the frame converted to the frequency axis is obtained to estimate a transfer function of the individual voice signals (S3). Meanwhile, the TF estimating step estimates a transfer function TF by obtaining an impulse response between microphone arrays for a predetermined time (5 seconds) with respect to a voice signal in a predetermined direction.

Subsequently, the noise signal is estimated by removing individual voice signals detected through a transfer function from the integrated voice signal converted into the frequency axis (S4).

Thereafter, the individual voice signals except for the individual voice signals to be extracted from the individual voice signals are removed from the integrated voice signal (S5).

Then, the noise part is removed from the individual voice signal to be detected through the transfer function to extract the individual voice signal from which the noise is removed (S6). On the other hand, the voice signal detection step S6 converts the individual voice signals on the frequency axis into individual time signals on the time axis.

Although the present invention has been described in detail only with respect to the specific embodiments described, it will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the present invention, and such modifications and modifications belong to the appended claims. .

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a view showing a directivity pattern when the front of a microphone array is 90 degrees in a user-oriented sound detection system for adaptive beamforming.

2 is a functional block diagram illustrating a user-oriented sound detection system for conventional adaptive beamforming.

Figure 3 is a functional block diagram showing the configuration of a sound source separation system using a beamforming technology according to the present invention.

4 is a flowchart showing a sound source separation method using a beamforming technique according to the present invention.

<Explanation of symbols for the main parts of the drawings>

10: at least one microphone array 20: short-term analysis unit

30: echo canceller 40: adaptive beamforming processor

50: front sound detection unit 60: post-filtering unit

70: Overlap and add processing unit

100: range setting unit 200: DFT conversion unit

300: at least one TF estimator 400: noise estimator

500: at least one voice signal extraction unit

600: at least one voice signal detector

610: IDFT conversion unit

Claims (18)

In a system for separating one or more different sound sources, A windowing processor to cover the integrated voice signal input through the beamformed at least one microphone array; A DFT converter for converting a signal covered by a window into a frequency axis through the windown processor; A TF estimator for estimating a transfer function having feature values of the two or more different voice signals from the windowed signal; A noise estimator for removing noise of individual speech signals from a transfer function having characteristic values of two or more different speech signals estimated by the TF estimator; And And a voice signal detector for extracting the individual voice signals from the noise-removed voice signal. The method of claim 1, The transfer function estimating unit, A sound source separation system using beamforming technology, characterized in that the transfer function is estimated using the impulse response obtained through the value converted by the DFT converter. The method of claim 1, The DFT converter, Discrete Fourier Transform (DFT) A sound source separation system using beamforming technology. The method of claim 1, TF estimator for estimating the transfer function of the speech signal from the value converted by the Fourier transform unit, A sound source separation system using beamforming technology, characterized in that the same number of different sound source. The method of claim 1, The system, Beamforming further comprising at least one voice signal extracting unit for removing the remaining individual voice signal except for the individual voice signal to be extracted from the individual voice signals provided through the TF estimator from the integrated voice signal provided through the DFT converter Sound source separation system using technology. The method of claim 1, The windowing processing unit, A sound source separation system using beamforming technology, characterized in that the length of the hanning window is 32ms and the moving section is 16ms. The method of claim 6, The TF estimator, A sound source separation system using beamforming technology, characterized by estimating a transfer function (TF) by obtaining an impulse response between microphone arrays for a predetermined time with respect to a voice signal in a predetermined direction. The method of claim 7, wherein The voice signal detector, A sound source separation system using a beamforming technology further comprises an IDFT conversion unit for converting the individual voice signal of the frequency axis into a separate audio signal of the time axis. The method of claim 8, After detecting the transfer function of the individual voice signal, send the signal in the corresponding direction and compare the magnitude of the value of the signal. The sound source separation system using the beamforming technology further comprises a TF accuracy check unit for determining that the transfer function of the individual voice signal is incorrect when the value comes out. In the method of separating sound sources using one or more different sound sources, A windowing process of covering the integrated voice signal input through the beamformed at least one microphone array; A DFT conversion step of converting a window-covered signal to a frequency axis through the windown processing step; A TF estimating step of estimating a transfer function having characteristic values of said two or more different speech signals from the windowed signal; A noise estimation step of removing noise of individual voice signals from a transfer function having characteristic values of two or more different voice signals estimated through the TF estimation step; And And a voice signal detection step of extracting the different individual voice signals from the noise canceled voice signal. The method of claim 10, The transfer function estimating step, A sound source separation method using a beamforming technique, characterized in that the transfer function is estimated using the impulse response obtained through the value converted by the DFT converter. The method of claim 10, The DFT conversion step, Discrete Fourier transform (DFT) characterized in that the sound source separation method using a beamforming technology. The method of claim 10, The TF estimation step, A sound source separation system using beamforming technology, characterized in that the number of different sound source to perform. The method of claim 10, The method, And a voice signal extracting step of removing the remaining individual voice signals except the individual voice signals to be extracted from the individual voice signals provided through the TF estimation step from the integrated voice signal provided through the DFT conversion step. Sound source separation method using technology. The method of claim 10, The windowing processing step, The length of the hanning window is 32ms, the moving section is a sound source separation method using the beamforming technology, characterized in that 16ms. The method of claim 15, The TF estimation step, A method of separating sound sources using beamforming technology, comprising: estimating a transfer function (TF) by obtaining an impulse response between microphone arrays for a predetermined time with respect to a voice signal in a predetermined direction. The method of claim 16, The voice signal detection step, And an IDFT processing step of converting the individual voice signals on the frequency axis into the individual voice signals on the time axis. The method of claim 17, After detecting the transfer function of the individual voice signal, send the signal in the corresponding direction and compare the magnitude of the value of the signal. When the value comes out, the sound source separation method using the beamforming technology further comprises the step of verifying the TF accuracy of determining that the transfer function (transfer function) of the individual voice signal is incorrect.

Images