KR102045953B1 - Method for cancellating mimo acoustic echo based on kalman filtering - Google Patents

Abstract

Disclosed are a Kalman filter-based multi-channel input/output acoustic echo cancellation method and system. According to the present invention, a method for cancelling an acoustic echo can comprise the steps of: receiving a voice signal and an audio signal output through a speaker through a multi-channel microphone; estimating a multi-channel echo path based on a Kalman filter on a multi-channel microphone input signal input through the multi-channel microphone; estimating a multi-channel echo signal based on the estimated multi-channel echo path; and generating a multi-channel echo cancellation output signal based on the estimated multi-channel echo signal and the multi-channel microphone input signal.

Description

Kalman Filter-Based Multichannel Input / Output Acoustic Echo Cancellation {METHOD FOR CANCELLATING MIMO ACOUSTIC ECHO BASED ON KALMAN FILTERING}

Embodiments of the present invention relate to a technique for canceling acoustic echo in a multiple input / output environment.

An acoustic echo signal occurs when a speaker and a microphone are present in a space, and a signal output from the speaker is input to a microphone. Thus, the speaker output represents a signal input to the microphone again. These echo signals lower the sensitivity of voice communication and act as a factor of lowering the voice recognition rate.

In order to reduce performance degradation due to the echo signal, an acoustic echo signal cancellation method is used. The acoustic echo signal removal method has a difference depending on the number of microphones. For example, echo cancellation based on a single channel microphone estimates an echo signal using an adaptive filter such as least mean square (LMS) or recursive least square (RLS), and removes the estimated echo signal from the microphone input signal. In addition, by using an adaptive filter such as LMS, RLS to remove the echo signal for each microphone channel, and outputting a single channel using beamforming, or to remove the echo signal for a single channel output through the beamforming There is a way.

However, the conventional echo signal cancellation method is superior to the single channel echo cancellation method by combining the beamforming tube with excellent spatial filtering capability, but still has the following two disadvantages. First, since the output signal is a single channel, it is impossible to obtain spatial information after removing the echo signal, and thus it is impossible to perform meaningful direction estimation and multi-channel microphone-based situation after removing the echo signal. Second, there is a deterioration in echo cancellation performance because the adaptive filter with a fixed step size does not adapt quickly to the echo environment, which is easily changed even by the speaker's movement.

[1] S. Malik and G. Enzner , "Recursive Bayesian control of multichannel acoustic echo cancellation." IEEE Signal Processing Letters, Vol. 18, No. 11, pp. 619-622, Nov. In 2011 , Kalman filter based echo cancellation technology is proposed, but it assumes the environment where single channel microphone and multi-channel speaker exist. It is difficult to improve the echo cancellation performance by increasing.

Accordingly, even if the number of microphones is increased in a multi-channel input and output environment, a technique for effectively removing echo signals is required.

Korean Patent Laid-Open Publication No. 10-2015-0058980 relates to a multi-channel linear prediction echo canceller, an echo canceller, and a signal processing device and a signal processing method using the same. A technique using multi-channel linear predictive multi input equalization for echo cancellation is described.

[1] S. Malik and G. Enzner, "Recursive Bayesian control of multichannel acoustic echo cancellation." IEEE Signal Processing Letters, Vol. 18, No. 11, pp. 619-622, Nov. 2011. [2] S. Y. Lee and N. S. Kim, "A statistical model based residual echo suppression," IEEE Signal Process. Lett., Vol. 14, No. 10, pp. 758-761, Oct. 2007.

One embodiment of the present invention is to remove the echo signal by expressing the input and output signals in a linear relationship to enable multiple input and output, and dynamic modeling the multi-channel input and output linear relationship using a Kalman filter. That is, state-space modeling for echo path estimation considering the correlation between the multi-channel microphones is performed, and the multi-channel echo path modeled as state-space (ie, dynamic modeling) is extended to MIMO. By using the estimated Kalman filter and removing the multichannel echo path from the multichannel microphone input signal based on the estimated multichannel echo path, MIMO type echo cancellation is performed.

In addition, the spatial correlation of the speaker's voice input through the microphone is maintained through MIMO type echo cancellation, so that the location of the speaker's speech can be found even after echo cancellation.

A method of removing acoustic echo, the method comprising: receiving a voice signal and an audio signal output through a speaker through a multichannel microphone, and a multichannel microphone input through the multichannel microphone Estimating a linear multichannel echo path based on a Kalman filter for the signal, estimating a multichannel echo signal based on the estimated multichannel echo path, and estimating the estimated multichannel And generating a multichannel echo cancellation output signal based on the echo signal and the multichannel microphone input signal.

According to one aspect, the multi-channel microphone input signal may be based on the product of the multi-channel echo path and the near-end signal before output through the speaker.

According to another aspect, estimating the multichannel echo path may include dynamically modeling a linear relationship between the multichannel microphone input signal and the output signal based on the Kalman filter.

According to another aspect, estimating the multichannel echo path may include estimating covariance of the process noise to estimate the multichannel acoustic transfer function based on the process noise.

According to another aspect, estimating the multichannel echo path may further include an updating step of correcting the predicted covariance based on the multichannel microphone input signal.

According to another aspect, the covariance updated through the correction may be used to estimate the multichannel acoustic transfer function corresponding to the next frame.

According to another aspect, the generating of the multi-channel echo cancellation output signal may generate the multi-channel echo cancellation output signal by removing the estimated multi-channel echo signal from the multi-channel microphone input signal.

In the system for removing acoustic echo, an input control unit for receiving an audio signal and an audio signal output through a speaker through a multichannel microphone, and a multichannel microphone input through the multichannel microphone An echo signal estimator for estimating a linear multichannel echo path based on a Kalman filter for the input signal and estimating a multichannel echo signal based on the estimated multichannel echo path, and an estimated And an output signal generator configured to generate a multichannel echo cancellation output signal based on the multichannel echo signal and the multichannel microphone input signal.

According to one aspect, the multi-channel microphone input signal may be based on the product of the multi-channel echo path and the near-end signal before output through the speaker.

According to another aspect, the echo signal estimator may dynamically model a linear relationship between the multichannel microphone input signal and the output signal based on the Kalman filter.

According to another aspect, the echo signal estimator may predict the covariance of the process noise to estimate the multi-channel acoustic transfer function based on the process noise.

According to another aspect, the echo signal estimator may update to correct the predicted covariance based on the multi-channel microphone input signal.

According to another aspect, the covariance updated through the correction may be used to estimate the multichannel acoustic transfer function corresponding to the next frame.

According to another aspect, the output signal generator may generate the multichannel echo cancellation output signal by removing the estimated multichannel echo signal from the multichannel microphone input signal.

According to embodiments of the present invention, an echo signal may be removed by expressing an input and an output signal in a linear relationship to enable multiple inputs and outputs, and by dynamically modeling the multichannel input / output linear relationship using a Kalman filter.

In addition, by maintaining the spatial correlation of the speaker's voice input through the microphone through echo cancellation in the form of MIMO, the location of the speaker's utterance can be found even after the echo cancellation.

1 is a block diagram illustrating an operation of a Kalman filter-based multi-channel input and output echo cancellation system according to an embodiment of the present invention.
2 is a flowchart illustrating a method of removing acoustic echo signals according to an embodiment of the present invention.
3 is a block diagram illustrating an internal configuration of an acoustic echo signal removing system according to an exemplary embodiment of the present invention.
4 is a block diagram illustrating a mathematical relationship for performing dynamic modeling using a Kalman filter according to an embodiment of the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for removing multi-channel input and output acoustic echo signals. In particular, the input and output signals are represented in a linear relationship, and the multi-channel input and output linear relationships are dynamically modeled using a Kalman filter to provide an echo signal. Relates to a technique for removing it. That is, dynamic modeling (state-space) for multichannel echo path estimation is performed by considering correlation between each of the plurality of microphone channels, and the dynamic modeled multichannel echo path is estimated using a Kalman filter extended in MIMO form. Afterwards, the present invention relates to a technique for estimating a multichannel acoustic echo signal using an estimated multichannel echo path and removing the estimated multichannel acoustic echo signal from a multichannel microphone input signal.

In the present embodiments, 'dynamic modeling' may refer to state-space modeling.

1 is a block diagram illustrating an operation of a Kalman filter-based multi-channel input and output echo cancellation system according to an embodiment of the present invention.

According to FIG. 1, in an environment in which a multichannel microphone and a speaker exist in one space, a multichannel echo path based on a Kalman filter (ie, a multichannel acoustic transfer function 101) is estimated, and the estimated multichannel echo path is estimated. The multichannel echo signal 102 may be estimated based on the above.

For example, electronic devices such as artificial intelligence speakers, voice recognition TVs, and smartphones not only perform commands based on voice recognition, but also play various functions such as playing video or music or providing a phone call function. Can be done. For example, in a state where music is being played through the AI speaker, when the speaker commands “play singer A music,” the multi-channel microphone (ie, a plurality of microphones) provided in the AI speaker is used for playback. The audio signal corresponding to the music being output and the voice signal of the speaker may be input. Here, the information related to the music output through the speaker is information already known by the system, and the audio signal corresponding to the music immediately before being output through the speaker may correspond to a near-end signal. As such, the speaker's voice signal input through the multi-channel microphone and the audio signal output through the speaker may correspond to the multi-channel microphone input signal.

The acoustic echo signal removal system may then remove the echo signal from the multi-channel microphone input signal to recognize the speech signal spoken by the speaker. In this case, in order to remove the echo signal, the multichannel echo path may be estimated based on the Kalman filter, and the multichannel echo cancellation signal 102 is removed from the multichannel microphone input signal 103, thereby outputting the multichannel echo cancellation output. Signal 104 may be generated. For example, the multi-channel microphone input signal 103 may include an input signal for each channel in a vector form, and by subtracting the multi-channel echo signal 102 for each channel, an output signal from which an echo signal is removed for each channel may be obtained. Can be generated. In this case, since the signal before output through the speaker is already known to the system, a value multiplied based on a path between the speaker and each of the multichannel microphones may be defined as the multichannel microphone input signal. Accordingly, the echo signal is estimated based on a signal before the speaker output (i.e., the near-end signal) which is already known, and the estimated echo signal is removed from the multi-channel microphone input signal 103, that is, removed to correspond to each channel. Thereby, the multichannel echo cancellation output signal 104 can be generated.

2 is a flowchart illustrating an acoustic echo signal removing method according to an embodiment of the present invention, and FIG. 3 is a block diagram illustrating an internal configuration of an acoustic echo signal removing system according to an embodiment of the present invention. to be.

Referring to FIG. 3, the acoustic echo signal removal system 300 may include an input controller 310, an echo signal estimator 320, and an output signal controller 330. In addition, each step (ie, steps 210 to 240) of the acoustic echo signal removing method illustrated in FIG. 2 may include an input control unit 310 and an echo signal which are components of the acoustic echo signal removing system 300 of FIG. 3. It may be performed by the estimator 320 and the output signal controller 330.

In operation 210, the input controller 310 may receive an audio signal and an audio signal output through the speaker through the multi-channel microphone.

For example, an input signal mixed with an audio signal corresponding to a music or video being played through a speaker and a voice signal spoken toward the acoustic echo signal removing system 300 by a user (ie, a speaker) located at a long distance may be provided. Multi-channel microphone can be input as a multi-channel microphone input signal. For example, in an environment in which a multi-channel micro echo signal is input, a multi-channel microphone input signal in a short time fourier transform (STFT) region may be defined as in Equation 1 below.

 [Equation 1]

는 다채널 마이크 입력신호 벡터로서,

는 m 채널 마이크 입력의 STFT 계수를 나타내고, M은 전체 마이크의 개수를 나타낼 수 있다. 그리고,

는 다채널 음성신호 벡터를 나타내고,

처럼 정의될 수 있다.

는 스피커를 통해 출력되기 전의 근단 신호의 STFT 계수를 나타내고,

는 필터차수 L(즉, 칼만 필터)의 다채널 음향학적 전달함수를 나타낼 수 있다.In Equation 1, k may represent an STFT frequency index and n may represent a frame index. And,

Is the multichannel microphone input signal vector,

Denotes an STFT coefficient of the m channel microphone input, and M denotes the total number of microphones. And,

Represents a multi-channel speech signal vector,

Can be defined as

Denotes the STFT coefficient of the near-end signal before being output through the speaker,

Denotes the multi-channel acoustic transfer function of filter order L (ie, Kalman filter).

와 근단 신호

의 곱으로 표현될 수 있으며, 보다 상세하게는 상기 다채널 음향학적 전달 함수와 근단 신호의 곱에 다채널 음성 신호

를 합한 형태로 정의될 수 있다. That is, according to Equation 1, the multi-channel microphone input signal in the STFT (Short Time Fourier Transform) region is a multi-channel acoustic transfer function representing a multi-channel echo path.

And near-end signal

It can be expressed as a product of, more specifically, the multi-channel speech signal to the product of the multi-channel acoustic transfer function and the near-end signal

It can be defined as the sum form.

In operation 220, the echo signal estimator 320 may estimate a linear multichannel echo path based on a Kalman filter, targeting the multichannel microphone input signal input through the multichannel microphone. Here, the linear multichannel echo path may indicate that the path between the multichannel microphone and the speaker has linearity.

와 다채널 마이크 입력신호

는 시스템에서 이미 알고 있는 신호이므로, 다채널 반향 경로를 나타내는 다채널 음향학적 전달함수

를 추정함으로써, 반향제거 신호를 획득할 수 있다. 이때, 반향신호 추정부(320)는 칼만 필터를 이용하여 동적 모델링을 수행함으로써, 다채널 반향경로를 추정할 수 있다. 즉, 반향신호 추정부(320)는 다채널 마이크 입력 신호와 출력 신호 간의 선형적 관계를 칼만 필터에 기초하여 동적 모델링할 수 있다. 여기서, 칼만 필터는 크게 예측(prediction) 및 업데이트(update)를 수행하는 과정으로 구분될 수 있다.For example, in the multi-channel microphone input signal represented by Equation 1, the near-end signal

And multi-channel microphone input signal

Since the signal is already known in the system, the multichannel acoustic transfer function representing the multichannel echo path

By estimating, the echo cancellation signal can be obtained. In this case, the echo signal estimator 320 may estimate the multi-channel echo path by performing dynamic modeling using the Kalman filter. That is, the echo signal estimator 320 may dynamically model the linear relationship between the multi-channel microphone input signal and the output signal based on the Kalman filter. Here, the Kalman filter may be classified into a process of performing prediction and update.

In operation 221, the echo signal estimator 320 may predict an average and covariance related to the multi-channel acoustic transfer function based on the inter-frame variation constant and the process noise.

In operation 222, the echo signal estimator 320 may perform an update for correcting the predicted average and covariance based on the multi-channel microphone input signal. As such, the mean and covariance updated through the correction may be used to estimate the multichannel acoustic transfer function (ie, multichannel echopath) corresponding to the next frame.

In operation 230, the echo signal estimator 320 may estimate the multichannel echo signal based on the estimated multichannel echo path.

In operation 240, the output signal controller 330 may generate a multichannel echo cancellation output signal based on the estimated multichannel echo signal and the multichannel microphone input signal.

For example, the output signal controller 330 may perform echo cancellation in the form of MIMO by removing the estimated multichannel echo signal from the multichannel microphone input signal.

4 is a block diagram illustrating a mathematical relationship for performing dynamic modeling using a Kalman filter according to an embodiment of the present invention.

That is, FIG. 4 is a block diagram of an equation in which state-space modeling and a multi-channel microphone input signal are combined.

를 추정함으로써, 반향제거 신호를 획득할 수 있다. 이때, 반향신호 추정부(320)는 추정된 다채널 음향학적 전달함수

를 기반으로 아래의 수학식 2와 같이 표현되는 반향 제거 신호를 계산할 수 있다.Multi-channel acoustic transfer function from multi-channel microphone input signal expressed as in Equation 1 above

By estimating, the echo cancellation signal can be obtained. In this case, the echo signal estimator 320 estimates the multi-channel acoustic transfer function.

Based on the Echo cancellation signal can be calculated as shown in Equation 2 below.

[Equation 2]

For example, for the Kalman filter-based dynamic modeling, the multi-channel acoustic transfer function may be expressed in the form of a first-order markov model as shown in Equation 3 below.

[Equation 3]

는

의 프레임 간 변이 상수를 나타내고,

는 프로세스 노이즈(process noise)로서 평균이 0인 복소 가우시안으로 정의될 수 있다.In Equation 3,

Is

Represents the inter-frame transition constant of

May be defined as a complex Gaussian with an average of 0 as process noise.

를 효과적으로 추정하기 위해 칼만 필터가 이용될 수 있다. 예컨대, 아래의 수학식 4 및 5에 기초하여 칼만 필터 기반의 예측이 수행되고(즉, 도 2에서 설명한 221 단계에 해당함), 아래의 수학식 7 및 8에 기초하여 칼만 필터 기반의 업데이트가 수행될 수 있다(즉, 도 2에서 설명한 222 단계에 해당함). As such, the multi-channel acoustic transfer function modeled dynamically as

The Kalman filter can be used to effectively estimate For example, a Kalman filter based prediction is performed based on Equations 4 and 5 below (ie, corresponds to step 221 described in FIG. 2), and an Kalman filter based update is performed based on Equations 7 and 8 below. (Ie, correspond to step 222 described with reference to FIG. 2).

[Equation 4]

[Equation 5]

는 칼만 필터의 예측(prediction step) 프로세스를 나타낼 수 있다. 그리고,

는 프로세스 노이즈

의 공분산을 나타낼 수 있다. 즉, 수학식 5에 기초하여 프로세스 노이즈의 공분산이 예측/추정될 수 있다.In Equations 4 and 5 above,

May represent a prediction step process of the Kalman filter. And,

Process noise

It can represent the covariance of. In other words, the covariance of the process noise can be predicted / estimated based on Equation (5).

The gain of the Kalman filter may be calculated based on the average and the covariance obtained through the prediction step performed based on Equations 4 and 5 above. For example, the echo signal estimator 320 may calculate the gain of the Kalman filter based on Equation 6 below.

[Equation 6]

As such, when the gain of the Kalman filter is calculated and the covariance of the process noise is predicted, an update for precisely correcting the noise of the process may be performed based on Equations 7 and 8 below. Here, a plurality of Gaussians with an average of 0 are assumed, but the average of the process noise may also be predicted based on the Kalman filter, and the update may be performed based on the predicted average and the covariance.

[Equation 7]

[Equation 8]

, 프레임간 변이 상수

를 기반으로 추정된 다채널 음향학적 전달함수

가

를 통과함으로써 에코(즉, 다채널 반향신호)가 되고, 다채널 반향신호와 다채널 음성 신호

가 더해져

으로 표현될 수 있다. 그러면, 위의 수학식 7 및 8, 그리고 도 4에서,

및

는 아래의 수학식 9 및 10과 같이 정의될 수 있다.Referring to FIG. 4, the process noise of Equation 3 above

, Interframe variation constant

Estimated Multichannel Acoustic Transfer Function

end

By passing through, an echo (i.e., a multichannel echo signal) becomes a multichannel echo signal and a multichannel audio signal.

Is added

It can be expressed as. Then, in the above Equations 7 and 8, and Figure 4,

And

May be defined as in Equations 9 and 10 below.

[Equation 9]

[Equation 10]

는 크로니컬 곱을 나타내고, I는 단위 행렬(identity matrix)를 나타내고, L은 칼만필터의 필터차수를 나타낼 수 있다. 그리고, H는 허미션(Hermition) 연산자를 나타낼 수 있다.In Equations 9 and 10 above,

Denotes a chromatic product, I denotes an identity matrix, and L denotes a filter order of the Kalman filter. In addition, H may represent a Hermition operator.

를 기반으로 채널 별 반향신호가 추정될 수 있다. 그러면, 출력신호 제어부(330)는 추정된 다채널 음향학적 전달함수와 위의 수학식 2에 기초하여 최종적으로 다채널 반향제거 출력신호

를 생성할 수 있다. 즉, 수학식 2와 같이 표현되는

를 계산할 수 있다. 예컨대, 출력신호 제어부(330)는 다채널 마이크 입력 신호에서 해당 채널 별로 추정된 반향신호를 감산(즉, 제거)함으로써, 채널 별 반향제거 출력신호를 생성할 수 있다.As such, the multi-channel acoustic transfer function is estimated based on Equations 4 to 10, and the estimated transfer function

The echo signal for each channel can be estimated based on the. Then, the output signal controller 330 finally outputs the multichannel echo cancellation output signal based on the estimated multichannel acoustic transfer function and Equation 2 above.

Can be generated. That is, expressed as in Equation 2

Can be calculated. For example, the output signal controller 330 may generate an echo cancellation output signal for each channel by subtracting (ie, removing) the echo signal estimated for each channel from the multi-channel microphone input signal.

Table 1 below shows the echo cancellation performance evaluation according to the number of microphones.

In Table 1, a simulation environment for performance measurement by generating a multichannel echo cancellation output signal by performing dynamic modeling using a Kalman filter and updating the Kalman filter in a multichannel input / output environment may be as follows.

의 공간에서 잔향시간 0.3초, 0.5초에 대한 임펄스 함수(impulse function)를 기반으로 시뮬레이션이 수행될 수 있다. 마이크는 2개, 4개 사용하고, 마이크 사이의 간격은 8cm의 원형구조를 가정할 수 있다. 이때, 사용자(즉, 화자)는 마이크로부터 1m 거리를 두고 떨어져 있음을 가정하고, 반향신호 재생을 위한 스피커는 마이크로부터 0.5m의 이격된 위치에 배치되어 있고, 화자와 스피커는 90도의 각도 차로 배치되어 있음을 가정하였다. 화자신호는 무향실에서 녹음된 남성 사용자 음성에 해당하는 음성 신호를 나타내고, 반향신호는 무향실에서 녹음된 여성 화자의 음성에 해당하는 음성 신호를 포함함을 가정하였다. 1024크기의 STFT를 이용하였으며, 75%의 오버랩이 이용되었다. 그리고, 필터차수 L은 1의 값을 사용하였다. 이처럼, 반향신호는 스피커를 통해 출력되는 신호가 다시 마이크를 통해 입력되는 신호뿐만 아니라, 시스템으로 음성 인식을 위한 명령을 발화하는 사용자 이외에 다른 사용자가 이야기하는 음성 신호가 마이크를 통해 입력되는 신호를 포함할 수도 있다. 이외에, 음성 인식을 위한 명령을 발화하는 사용자 이외에 마이크를 통해 입력되는 자연에서 존재하는 다양한 신호(예컨대, 새소리, 의자 끄는 소리, 음식을 조리하는 소리 등)가 반향 신호에 해당할 수 있다.For example,

Simulation can be performed based on the impulse function for the reverberation time 0.3 seconds, 0.5 seconds in the space of. Two or four microphones may be used, and the space between the microphones may assume a circular structure of 8 cm. In this case, it is assumed that the user (ie, the speaker) is 1 m away from the microphone, and the speaker for reproducing the echo signal is disposed at a position 0.5 m away from the microphone, and the speaker and the speaker are arranged at an angle of 90 degrees. It is assumed. It is assumed that the speaker signal represents a voice signal corresponding to the male user voice recorded in the anechoic chamber, and the echo signal includes a voice signal corresponding to the voice of the female speaker recorded in the anechoic chamber. A 1024 size STFT was used and 75% overlap was used. In addition, the filter order L used the value of 1. As described above, the echo signal includes not only a signal output through a speaker, but also a signal input through a microphone, and a signal in which a voice signal spoken by another user other than the user who commands a voice recognition command to the system is input through the microphone. You may. In addition, various signals existing in nature (for example, a bird sound, a chair drag, a food cooking sound, etc.) input through a microphone may correspond to the echo signal in addition to a user who speaks a command for speech recognition.

In Table 1, above non-patent literature [1] S. Malik and G. Enzner , "Recursive Bayesian control of multichannel acoustic echo cancellation." IEEE Signal Processing Letters, Vol. 18, No. 11, pp. 619-622, Nov. ERLE, which measures the degree of echo cancellation presented in 2011. , and non-patent literature [2] SY Lee and NS Kim, "A statistical model based residual echo suppression," IEEE Signal Process. Lett., Vol. 14, No. 10, pp. 758-761, Oct. PESQ, a measure of speech distortion, presented in 2007. was used to verify the echo cancellation performance.

According to Table 1, it can be seen that as the number of microphones increases in two reverberation environments, ERLE and PESQ are relatively improved. In other words, the echo signal in the multi-channel microphone environment is more accurately estimated than the single channel, and as a result, it is confirmed that the echo cancellation performance is improved while reducing the speech distortion relatively compared to the single channel method based on the Kalman filter. That is, as the number of microphones increases, the acoustic echo signal cancellation performance is improved, so that the speech distortion may be reduced, and thus the overall speech quality may be improved. Accordingly, after removing the echo signal, by applying a multi-channel microphone-based signal processing technology as a post-processing, it may be applied to various fields such as direction estimation of a sound source, beamforming, and the like.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (14)

In a method for removing acoustic echo,
Receiving a voice signal and an audio signal output through the speaker through a multi-channel microphone;
Estimating a linear multichannel echo path based on a Kalman filter on the multichannel microphone input signal input through the multichannel microphone;
Estimating a multichannel echo signal based on the estimated multichannel echo path; And
Generating a multichannel echo cancellation output signal based on the estimated multichannel echo signal and the multichannel microphone input signal;
Including,
The multi-channel microphone input signal,
A speech signal other than the speech signal and the speech signal that is a speech recognition object input through a plurality of microphones, and is based on a product of the multi-channel echo path and a near-end signal before being output through a speaker;
The generating of the multichannel echo cancellation output signal may include:
Generating the multichannel echo cancellation output signal for each channel by removing the multichannel echo signal estimated for each channel from the multichannel microphone input signal;
Acoustic echo signal removal method characterized in that. delete The method of claim 1,
Estimating the multichannel echo path,
Dynamically modeling a linear relationship between the multichannel microphone input signal and the output signal based on the Kalman filter
Acoustic echo signal removal method comprising a. The method of claim 1,
Estimating the multichannel echo path,
Predicting covariance of process noise to estimate multichannel acoustic transfer function based on process noise
Acoustic echo signal removal method comprising a. The method of claim 4, wherein
Estimating the multichannel echo path,
An update step of correcting the predicted covariance based on the multichannel microphone input signal
Acoustic echo signal removal method further comprising. The method of claim 5,
The covariance updated through the correction is used to estimate the multichannel acoustic transfer function corresponding to the next frame.
Acoustic echo signal removal method characterized in that. delete In a system for canceling acoustic echo,
An input controller configured to receive a voice signal and an audio signal output through the speaker through a multi-channel microphone;
Estimating a linear multichannel echo path based on a Kalman filter on a multichannel microphone input signal input through the multichannel microphone, and multichannel echo signal based on the estimated multichannel echo path An echo signal estimator for estimating a? And
An output signal generator for generating a multichannel echo cancellation output signal based on the estimated multichannel echo signal and the multichannel microphone input signal
Including,
The multi-channel microphone input signal,
A speech signal other than the speech signal and the speech signal that is a speech recognition object input through a plurality of microphones, and is based on a product of the multi-channel echo path and a near-end signal before being output through a speaker;
The output signal generator,
Generating the multichannel echo cancellation output signal for each channel by removing the multichannel echo signal estimated for each channel from the multichannel microphone input signal;
Acoustic echo signal cancellation system characterized in that. delete The method of claim 8,
The echo signal estimator,
Dynamically modeling a linear relationship between the multichannel microphone input signal and an output signal based on the Kalman filter
Acoustic echo signal cancellation system characterized in that. The method of claim 8,
The echo signal estimator,
Predicting covariance of process noise to estimate multichannel acoustic transfer function based on process noise
Acoustic echo signal cancellation system characterized in that. The method of claim 11,
The echo signal estimator,
Updating the predicted covariance based on the multichannel microphone input signal
Acoustic echo signal cancellation system characterized in that. The method of claim 12,
The covariance updated through the correction is used to estimate the multichannel acoustic transfer function corresponding to the next frame.
Acoustic echo signal cancellation system characterized in that. delete

Images