JP2015195510A - Echo suppression device, echo suppression program and echo suppression method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To maintain an echo suppression performance by preventing erroneous learning of an echo path property due to a spectral microstructure of frequencies of a far-end signal and a near-end input signal.SOLUTION: An echo suppression device includes spectrum envelope extraction means for amplitude spectra of the far-end signal and the near-end input signal. For an extracted spectrum envelope of the far-end signal, the presence/absence of a frequency component is determined for each frequency bin. On the basis of a frequency component of a frequency bin for the spectrum envelope of the amplitude spectrum of the far-end signal for which the presence of the frequency component in a frame of a single talk is determined, and a frequency component of a corresponding frequency bin for the spectrum envelope of the amplitude spectrum of the near-end input signal, the echo path property of the frequency bin is estimated.

Description

  The present invention relates to an echo suppression device, an echo suppression program, and an echo suppression method, and can be applied to, for example, an echo suppression device, an echo suppression program, and an echo suppression method used in a video conference system, a telephone conference system, and the like. .

  For example, in a loudspeaker system such as a video conference system or a telephone conference system, an acoustic echo that is emitted from a speaker (where sound includes sound, voice, etc.) wraps around a microphone and returns to the transmitting side. A signal is generated. Since the acoustic echo signal significantly hinders a call, much research and development have been conducted on acoustic echo signal suppression methods.

  As one method of suppressing the acoustic echo signal, there is a method of using an echo suppression device (echo suppressor). The echo suppressor is a method for suppressing the acoustic echo signal by multiplying the near-end input signal and the echo suppress gain by obtaining the echo path characteristics, the estimated echo signal, and the echo suppress gain from the far-end signal and the near-end input signal. It is. Non-Patent Document 1 proposes a technique for suppressing an acoustic echo signal using this echo suppressor.

  The echo suppressor disclosed in Non-Patent Document 1 obtains echo path characteristics based on the far-end signal and the near-end input signal of the past frame. Then, a signal obtained by multiplying the obtained echo path characteristic and the far-end signal is used as an estimated echo signal, and the echo suppressor obtains an echo suppress gain based on the near-end input signal and the estimated echo signal, and the echo suppress gain is calculated. Multiply with the input signal to suppress the acoustic echo signal.

  The echo suppression device described in Non-Patent Document 1 will be described with reference to FIG.

  FIG. 3 is a block diagram for explaining a conventional echo suppressor described in Non-Patent Document 1. In FIG.

  The echo cancellation apparatus 300 described in Non-Patent Document 1 includes a far-end signal input terminal 301, a DA converter 302, a speaker 303, a microphone 304, an AD converter 305, a first frequency analysis unit 306, a second frequency analysis unit 307, An acoustic coupling amount calculation unit 308, an estimated echo signal calculation unit 309, a gain calculation unit 310, an integration unit 311, a frequency-time conversion unit 312, and a near-end signal output terminal 313 are included.

  The far-end signal input terminal 301 is connected to, for example, a radio wave such as a network such as an Internet protocol (IP) network or a wireless network such as a mobile phone, and the far-end signal input terminal 301 is connected to the far end side (the other party side) ) Sound signal (digital signal).

  The far-end signal input to the far-end signal input terminal 301 is converted from a digital sound signal to an analog sound signal in the DA converter 302 and is output to the near-end side (self side) through the speaker 303.

  On the other hand, a sound signal such as a voice uttered by a speaker on the near end side, an environmental sound, an acoustic echo signal (a signal in which an analog sound signal output from the speaker 303 circulates through the space on the near end side), etc. The analog sound signal on which is superimposed is received by the microphone 304, converted into a digital sound signal by the AD converter 305, and input to the echo canceller 300 as a near-end input signal.

  The first frequency analysis unit 306 converts the far end signal into a frequency domain signal and outputs the frequency spectrum of the far end signal.

  The second frequency analysis unit 307 converts the near-end input signal into a frequency domain signal and outputs a spectrum of the near-end input signal.

  The acoustic coupling amount calculation unit 308 receives the frequency spectrum of the far-end signal and the frequency spectrum of the near-end input signal as input, calculates an echo path characteristic, and outputs it to the estimated echo signal calculation unit 309.

  The estimated echo signal calculation unit 309 calculates an estimated echo signal based on the spectrum of the far end signal and the estimated value of the echo path characteristic, and outputs the estimated echo signal to the gain calculation unit 310.

  Gain calculation section 310 receives the estimated echo signal and the frequency spectrum of the near-end input signal as input, calculates a suppression gain coefficient, and outputs it to integration section 311.

  The accumulator 311 outputs the frequency spectrum of the signal obtained by suppressing the acoustic echo signal by multiplying the frequency spectrum of the near-end input signal by the suppress gain coefficient to the frequency-time converter 312 as the frequency spectrum of the near-end output signal.

  The frequency-time conversion unit 312 converts the frequency spectrum of the near-end output signal into a time-domain signal and outputs it to the near-end output terminal.

  The near-end signal output terminal 213 is connected to a radio wave of a network such as an Internet protocol (IP) network or a wireless network such as a mobile phone, and communicates via a line to which the near-end output signal is connected. Outputs to the far end that is the other party.

C. Faller and C.M. Turnery, “ESTIMATION THE DELAY AND COLORATION EFFECT OF THE ACOUSTIC ECHO PATH FOR LOW COMPLEITY ECHO SUPPRESSION”, Proc. Intl. Works. on Acoustic. Echo and Noise Control (IWAENC) 2005, pp. 53-56, Oct. 2005

  However, in the echo suppression device described in Non-Patent Document 1, since the far-end signal mainly includes the audio signal, the frequency bin that becomes the valley of the fine structure of the audio signal and the spectral envelope of the audio signal There is a frequency bin with a small frequency component of the far-end input signal due to its shape or the like.

  Therefore, when there is background noise on the far end side or near end side, the valley portion of the fine structure of the audio signal or the frequency bin with a small frequency component due to the shape of the spectrum envelope is buried in the frequency component of the background noise. Even if the echo path characteristic of the frequency bin is obtained from the far-end signal and the near-end input signal, the frequency component of the background noise is superimposed, so a value completely different from the actual echo path characteristic is calculated and the echo path characteristic is updated. Will do.

  Further, the fine structure changes greatly due to the influence of the background noise on the near end side or the shift of the phase relationship between the pitch and the analysis window due to the delay of the echo path characteristic. Therefore, it must be said that it is extremely lacking in stability to measure the echo path characteristics by directly comparing the spectral fine structures of the far-end signal and the near-end input signal.

  Therefore, even if the echo path characteristics are estimated using the fine structure, they are updated using echo path characteristics that are different from the actual echo path characteristics, so the estimated echo path characteristics are mis-learned and the estimated echo path signal is calculated correctly. This may cause a problem that the acoustic echo signal cannot be appropriately suppressed.

  Therefore, the present invention has been made in view of the above problems, and by removing the spectral envelope of the far-end signal and the near-end input signal, the fine structure is removed, and then there is a frequency component of the spectral envelope of the far-end signal. An echo suppressor, an echo suppression program, and an echo suppression method capable of preventing erroneous learning of an echo path characteristic of a frequency bin having a small frequency component and maintaining echo suppression performance by obtaining and updating only the echo path characteristic of the frequency bin It is something to be offered.

  According to a first aspect of the present invention, in an echo suppression apparatus for suppressing an acoustic echo signal based on a near-end input signal and a far-end output signal, (1) the input far-end signal is converted into a frequency domain, A far-end signal amplitude spectrum calculating means for obtaining an amplitude spectrum; (2) a near-end input signal amplitude spectrum calculating means for converting the input near-end input signal into a frequency domain and obtaining an amplitude spectrum of the near-end input signal; (3) a far-end signal spectrum envelope extracting means for extracting the spectrum envelope of the amplitude spectrum of the far-end signal; and (4) a near-end input signal spectrum envelope extracting means for extracting the spectrum envelope of the amplitude spectrum of the near-end input signal; (5) Frequency bin component presence / absence determining means for determining the presence / absence of a frequency component for each frequency bin from the spectrum envelope of the amplitude spectrum of the far-end signal, and (6) Single talk The echo path characteristic of the frequency bin is updated by using the echo path characteristic and the echo path characteristic of the previous frame only for the frequency bin determined to have a frequency component based on the determination result of the frequency bin component presence / absence determination unit in the determined frame. (7) an estimated echo signal calculating means for calculating an estimated echo signal based on the echo path characteristics for each frequency bin estimated by the frequency bin echo path characteristic estimating means and the amplitude spectrum of the far end signal; 8) An echo suppression apparatus comprising echo suppression means for calculating an echo suppression gain from the estimated echo signal and the amplitude spectrum of the near-end input signal and suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal. is there.

  According to a second aspect of the present invention, there is provided a computer mounted on an echo suppressor that suppresses an acoustic echo signal based on a far end signal from a near end input signal; (1) converting the input far end signal into a frequency domain; A far-end signal amplitude spectrum calculating means for obtaining an amplitude spectrum of the far-end signal; and (2) a near-end input signal amplitude spectrum for obtaining an amplitude spectrum of the near-end input signal by converting the input near-end input signal into a frequency domain. A calculation means; (3) a far-end signal spectrum envelope extraction means for extracting the spectrum envelope of the amplitude spectrum of the far-end signal; and (4) a near-end input signal spectrum envelope for extracting the spectrum envelope of the amplitude spectrum of the near-end input signal. (5) frequency bin component presence / absence determination means for determining the presence / absence of a frequency component for each frequency bin from the spectrum envelope of the amplitude spectrum of the far-end signal, (6) Only the frequency bin determined to have a frequency component based on the determination result of the frequency bin component presence / absence determination means in the frame determined to be single talk, the echo path characteristic of the frequency bin using the echo path characteristic and the echo path characteristic one frame before. Frequency bin echo path characteristic updating means for updating the frequency bin, and (7) estimated echo signal calculation means for calculating an estimated echo signal based on the echo path characteristic for each frequency bin estimated by the frequency bin echo path characteristic estimation means and the amplitude spectrum of the far-end signal And (8) an echo that calculates echo suppression gain from the estimated echo signal and the amplitude spectrum of the near-end input signal and functions as an echo suppression means that suppresses the estimated echo signal from the amplitude spectrum of the near-end input signal. It is a suppression program.

  According to a third aspect of the present invention, there is provided an echo suppression method for suppressing an acoustic echo signal based on a far end output signal from a near end input signal, a far end signal amplitude spectrum calculating means, a near end input signal amplitude spectrum calculating means, a far end signal amplitude. Spectral envelope extraction means, near-end input signal amplitude spectral envelope extraction means, frequency bin component presence / absence determination means, frequency bin echo path characteristic update means, estimated echo signal calculation means, echo suppression means, (1) input far-end signal Is converted to the frequency domain, and the far end signal amplitude spectrum for obtaining the amplitude spectrum of the far end signal is obtained. (2) The input near end input signal is transformed to the frequency domain, and the amplitude spectrum of the near end input signal is obtained. Obtain the near-end input signal amplitude spectrum to be obtained, and (3) extract the far-end signal spectrum envelope to extract the spectrum envelope of the far-end signal amplitude spectrum. (4) The near-end input signal spectrum envelope is extracted to extract the spectrum envelope of the amplitude spectrum of the near-end input signal. (5) The presence / absence of a frequency component is determined for each frequency bin from the spectrum envelope of the amplitude spectrum of the far-end signal. (6) The echo path characteristic and the echo path one frame before are determined only for the frequency bin determined to have a frequency component based on the determination result of the frequency bin component presence / absence determination means in the frame determined to be single talk. The frequency bin echo path characteristics are updated using the characteristics, and the frequency bin echo path characteristics are updated. (7) Estimation is performed based on the echo path characteristics for each frequency bin estimated by the frequency bin echo path characteristics estimation means and the amplitude spectrum of the far-end signal. Obtain the estimated echo signal to calculate the echo signal, (8) Estimated echo signal and near-end input Is echo suppression method characterized by in the amplitude spectrum of the signal to echo suppression for suppressing an estimated echo signal an echo suppression gain from the amplitude spectrum of the calculated near-end input signal.

  According to the present invention, the echo path characteristic is calculated and updated only from the frequency envelope having the frequency component in the spectrum envelope of the amplitude spectrum of the far-end signal and the spectrum envelope of the amplitude spectrum of the far-end signal and the amplitude spectrum of the near-end input signal. Thus, it is possible to prevent erroneous learning of the echo path characteristics due to the fine structure of the voice signal of the far-end signal, and to maintain the echo suppression performance.

It is a block diagram which shows the structure of the echo suppression apparatus of 1st Embodiment. It is a block diagram which shows the structure of the echo suppression apparatus of 2nd Embodiment. It is a block diagram for demonstrating the conventional echo suppression apparatus.

(A) First Embodiment Hereinafter, a first embodiment of an echo suppression device, an echo suppression program, and an echo suppression method of the present invention will be described in detail with reference to the drawings.

  1st Embodiment illustrates the case where this invention is applied to the echo suppression apparatus and program of the audio | voice transmission / reception apparatus of loudspeaking call systems, such as a video conference system and a telephone conference system, for example.

(A-1) Configuration of First Embodiment FIG. 1 is a block diagram illustrating a configuration of an echo suppression apparatus 100 according to the first embodiment.

  The echo suppressor 100 according to the first embodiment is implemented with the above-described echo suppressor of the present invention. The far-end signal is transmitted to the near-end side space and is transmitted as an acoustic echo signal to the near-end microphone. Even when the signal is input by wrapping around, echo suppression processing (echo suppression processing) is performed on the near-end input signal to appropriately suppress the acoustic echo signal.

  The echo suppression apparatus 100 according to the first embodiment may be configured as a dedicated board, for example, or may be realized by writing an echo suppression program into a DSP (digital signal processor). Although it may be realized by a CPU and software (echo suppression program) executed by the CPU, it can be functionally represented in FIG.

  In FIG. 1, an echo suppression apparatus 100 according to the first embodiment includes a far-end signal input terminal 101, a DA converter 102, a speaker 103, a microphone 104, an AD converter 105, a far-end signal frequency domain transform unit 106, End signal amplitude spectrum calculation unit 107, echo path characteristic holding unit 108, estimated echo signal calculation unit 109, near end input signal frequency domain conversion unit 110, near end input signal amplitude spectrum calculation unit 111, echo suppression gain calculation unit 112, echo suppression Unit 113, near-end output signal amplitude spectrum calculation unit 114, single talk determination unit 115, far-end signal spectrum envelope extraction unit 116, near-end input signal spectrum envelope extraction unit 117, echo path characteristic calculation unit 118, far-end signal frequency bin component Presence / absence determination unit 119, frequency bin echo path characteristic update unit 120, near-end output signal Time domain transform section 121, having a proximal end output signal output terminal 122.

  The far-end signal input terminal 101 is connected to a radio wave of a network such as an Internet protocol (IP) network or a wireless network such as a mobile phone, and the far-end signal is input via a connected line. The

  The far-end signal input to the far-end signal input terminal 101 is converted from a digital sound signal to an analog sound signal by the DA converter 102 and output to the near-end side through the speaker 103.

  On the other hand, sound signals such as voices uttered by the near-end speaker, environmental sounds, and acoustic echo signals (for example, analog sound signals output from the speaker 103 circulate through the near-end space) ) And the like are received by the microphone 104, converted into a digital sound signal by the AD converter 105, and input to the echo suppression apparatus 100 as a near-end input signal.

  The far-end signal frequency domain transforming unit 106 transforms the far-end signal into a frequency-domain signal by, for example, fast Fourier transform (FFT), and outputs the frequency spectrum of the far-end signal.

  The far-end signal amplitude spectrum calculation unit 107 calculates the amplitude spectrum of the far-end signal based on the frequency spectrum of the far-end signal, and uses the calculated far-end signal amplitude spectrum as the far-end signal spectrum envelope extraction unit 116 and the estimated echo. The signal is output to the signal calculation unit 109.

  The echo path characteristic holding unit 108 holds the estimated echo path characteristic for each frequency bin, and outputs the held echo path characteristic to the estimated echo signal calculation unit 109.

  The estimated echo signal calculation unit 109 calculates the amplitude spectrum of the estimated echo signal by multiplying the amplitude spectrum of the far-end signal and the estimated echo path characteristic, and outputs the calculated estimated echo signal to the echo suppression gain calculation unit 112.

  The near-end input signal frequency domain transform unit 110 transforms the near-end input signal into a frequency domain signal by, for example, fast Fourier transform (FFT), and outputs the frequency spectrum of the near-end input signal.

  The near-end input signal amplitude spectrum calculation unit 111 calculates the amplitude spectrum of the near-end input signal based on the frequency spectrum of the near-end input signal, and the echo suppression gain calculation unit 112 calculates the calculated amplitude spectrum of the near-end input signal. And output to the single talk determination unit 115 and the near-end input signal spectrum envelope extraction unit 117.

  The echo suppression gain calculation unit 112 calculates an echo suppression gain for suppressing the acoustic echo signal superimposed on the near-end input signal, using the amplitude spectrum of the estimated echo signal and the amplitude spectrum of the near-end input signal. The echo suppression gain thus output is output to the echo suppression unit 113.

  The echo suppression unit 113 obtains a frequency spectrum in which the acoustic echo signal superimposed on the near-end input signal is suppressed by multiplying the echo suppress gain and the frequency spectrum of the near-end input signal, and the frequency spectrum of the near-end output signal Are output to the near-end output signal amplitude spectrum calculation unit 114 and the near-end output signal time domain conversion unit 121.

  The near end output signal amplitude spectrum calculation unit 114 calculates the amplitude spectrum of the near end output signal based on the frequency spectrum of the near end output signal, and outputs the calculated amplitude spectrum of the near end output signal to the single talk determination unit 115. To do.

  The single talk determination unit 115 determines a single talk and other than a single talk (double talk or a state where only a near-end audio signal is generated). Here, the single talk is a state in which an acoustic echo signal is input to the near-end input signal and a voice signal of the near-end speaker is not input. More specifically, the far-end signal is output from the speaker 103, is transmitted through the space on the near-end side, wraps around as an acoustic echo signal, and is received by the microphone 104. Double talk is a state in which an acoustic echo signal and a near-end speaker's voice signal are simultaneously input to the near-end input signal. Specifically, the far-end signal is output from the speaker 103, is transmitted through the space on the near-end side, wraps around as an acoustic echo signal, and is received by the microphone 104. It is in a state of being. When the single talk determination unit 115 determines whether the talk is single talk or other than single talk, the result is output to the echo path characteristic calculation unit 118.

  The far-end signal spectrum envelope extraction unit 116 extracts the spectrum envelope from the amplitude spectrum of the far-end signal, and outputs the spectrum envelope to the echo path characteristic calculation unit 118 and the far-end signal frequency bin component presence / absence determination unit 119.

  The near-end input signal spectrum envelope extraction unit 117 extracts a spectrum envelope from the amplitude spectrum of the near-end input signal and outputs the spectrum envelope to the echo path characteristic calculation unit 118.

  The echo path characteristic calculation unit 118 determines the echo path characteristic of the frame determined as single talk by the single talk determination unit 115 based on the spectrum envelope of the amplitude spectrum of the far end signal and the spectrum envelope of the amplitude spectrum of the near end input signal. Calculate for each frequency bin, and output the echo path characteristic for each frequency bin of the frame determined to be single talk.

  The frequency bin component presence / absence determination unit 119 determines the presence / absence of a frequency component for each frequency pin based on the spectrum envelope of the amplitude spectrum of the far-end signal, and sends the frequency presence / absence determination result for each frequency bin to the frequency bin echo path characteristic update unit 120. Output.

  The frequency bin echo path characteristic update unit 120 determines the presence / absence of frequency for each frequency bin of the echo path characteristic held in the echo path characteristic holding unit 108, the echo path characteristic of the frame determined to be single talk, and the spectrum envelope of the amplitude spectrum of the far-end signal. The echo path characteristic is updated using the result, and the updated echo path characteristic is stored in the echo path characteristic holding unit. At this time, the echo path characteristic is updated only for the frequency bin determined to have a frequency component, and the echo path characteristic is not updated for the frequency bin determined to have no frequency component. Thereby, it is possible to update the echo path characteristic of the frequency bin having a component in the amplitude spectrum of the far-end signal by single talk.

  The near-end output signal time domain conversion unit 121 converts the frequency spectrum of the near-end output signal into a time-domain digital sound signal by, for example, inverse fast Fourier transform (Inverse FFT), and converts the converted time-domain near-end output. The signal is output to the near end output signal output terminal 122.

  The near-end output signal output terminal 122 is connected to, for example, a network such as an Internet protocol (IP) network or a radio wave of a wireless network such as a mobile phone, and the near-end output signal is connected via a line to which the near-end output signal is connected. Output to the far end.

(A-2) Operation of the First Embodiment Next, the operation of echo suppression processing in the echo suppression device 100 according to the first embodiment will be described in detail.

  First, after the operation of the echo suppression apparatus 100 is started, for example, a far-end signal is transmitted via a connected line connected to a radio wave of a network such as an Internet protocol (IP) network or a wireless network such as a mobile phone. Input to the far-end signal input terminal 101.

  The far-end signal input to the far-end signal input terminal 101 is converted from a digital sound signal to an analog sound signal by the DA converter 102 and output to the near-end side through the speaker 103.

On the other hand, a sound signal such as a voice uttered by a near-end speaker, an analog sound signal superimposed with an environmental sound, an acoustic echo signal, or the like is received by the microphone 104 and converted into a digital sound signal by the AD converter 105. And input to the echo suppression apparatus 100 as a near-end input signal.

  The far-end signal frequency domain transform unit 106 transforms the far-end signal into a frequency-domain signal by, for example, fast Fourier transform (FFT), and the far-end signal frequency spectrum RIN (i, ω) is converted to the far-end signal. It outputs to the signal amplitude spectrum calculation part 107.

The far-end signal amplitude spectrum calculation unit 107 obtains the far-end signal amplitude spectrum | RIN (i, ω) | according to the equation (1) using the far-end signal frequency spectrum RIN (i, ω).

Here, i is a frame, ω is a frequency bin, and RIN_real (i, ω) and RIN_image (i, ω) are a real part and an imaginary part of the frequency spectrum of the far-end signal of the frequency bin ω in frame i. The frequency spectrum RIN (i, ω) of the far-end signal can be expressed as Equation (2).

  In equation (2), j represents an imaginary number. The far-end signal amplitude spectrum calculation unit 107 outputs the calculated frequency spectrum | RIN (i, ω) | of the far-end signal to the estimated echo signal calculation unit 109 and the far-end signal spectrum envelope extraction unit 116.

In the estimated echo signal calculation unit 109, the echo path characteristic | H (i−1, ω) | estimated one frame before held in the echo path holding unit 108 and the amplitude spectrum | RIN (i, ω) of the far end signal | Is used to calculate the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal according to equation (3).

  The expression (3) indicates that the frequency bin corresponding to the echo path characteristic | H (i−1, ω) | estimated one frame before is held in the amplitude spectrum | RIN (i, ω) | of the far-end signal. Multiplication is performed to calculate the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal of the frequency bin. The estimated echo signal calculation unit 109 outputs the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal to the echo suppression unit 113.

  On the other hand, the near-end input signal frequency domain transform unit 110 transforms the near-end input signal into a frequency-domain signal by, for example, fast Fourier transform (FFT), and the frequency spectrum SIN ( i, ω) is output to the near-end input signal amplitude spectrum calculation unit 111 and the echo suppression unit 113.

The near-end input signal amplitude spectrum calculation unit 111 calculates the near-end input signal amplitude spectrum | SIN (i, ω) | according to the equation (4) using the near-end input signal frequency spectrum SIN (i, ω). calculate.

Here, SIN_real (i, ω) and SIN_image (i, ω) indicate a real part and an imaginary part of the frequency spectrum of the near-end input signal of the frequency bin ω in the frame i, and the frequency spectrum of the near-end input signal. SIN (i, ω) can be expressed as Equation (5).

  In the formula (5), j represents an imaginary number. The near-end input signal amplitude spectrum calculation unit 111 converts the calculated amplitude spectrum | SIN (i, ω) | of the near-end input signal into an echo suppression gain calculation unit 112, a single talk determination unit 115, and a near-end input signal spectrum envelope extraction unit. It outputs to 117.

The echo suppression gain calculation unit 112 uses the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal and the amplitude spectrum | SIN (i, ω) | of the near-end input signal according to the equation (6), An echo suppression gain G (i, ω) is calculated.

  Equation (6) is obtained by subtracting the amplitude spectrum | ECHO (i, ω) | of the estimated echo signal from the amplitude spectrum | SIN (i, ω) | of the near-end input signal for each frequency bin. This is an equation for obtaining the echo suppression gain G (i, ω) by dividing by the amplitude spectrum of the signal | SIN (i, ω) |. The echo suppression gain calculation unit 112 outputs the calculated echo suppression gain G (i, ω) to the echo suppression unit 113.

The echo suppression unit 113 uses the near-end input signal spectrum SIN (i, ω) and the echo suppression gain G (i, ω) in accordance with the equations (7) and (8) to calculate the near-end input signal. The acoustic echo signal superimposed on the spectrum SIN (i, ω) is suppressed.

Here, SOUT_real (i, ω) and SOUT_image (i, ω) indicate the real part and the imaginary part of the frequency spectrum of the near-end output signal of the frequency bin ω at time i, and the frequency spectrum of the near-end output signal SOUT (i, ω) can be expressed as Equation (9).

  (9) j represents an imaginary number. Equations (7) and (8) are obtained by multiplying the real part and imaginary part of the frequency spectrum by the echo suppression gain G (i, ω) for each frequency bin, and the frequency spectrum of the near-end input signal in which the acoustic echo signal is suppressed. Is an expression for obtaining. The echo suppression unit 113 outputs the frequency spectrum of the near-end input signal in which the calculated acoustic echo signal is suppressed as the frequency spectrum SOUT (i, ω) of the near-end output signal to the near-end output signal amplitude spectrum calculation unit 114. .

The near-end output signal amplitude spectrum calculation unit 114 calculates the amplitude spectrum | SOUT (i, ω) | of the near-end output signal according to the equation (10) using the frequency spectrum of the near-end output signal.

  The near-end output signal amplitude spectrum calculation unit 114 outputs the calculated amplitude spectrum | SOUT (i, ω) | of the end output signal to the single talk determination unit 115.

  The single talk determination unit 115 compares the amplitude spectrum | SIN (i, ω) | of the near-end input signal with the amplitude spectrum | SOUT (i, ω) | of the near-end output signal, except for single talk and single talk. Determine.

As a determination means other than single talk and single talk, for example, as shown in equation (), the frame total value of the amplitude spectrum of the near-end output signal is multiplied by the frame total value of the amplitude spectrum of the near-end input signal, and the value is When it is smaller than the threshold value TH1, it is determined that the state is single talk, and otherwise, it is determined that the state is other than single talk. Since the left side of the equation (11) takes a value from 0 to 1, the threshold TH1 is set to a value from 0 to 1 and a small value (for example, TH1 = 0.3).

  In the equation (11), FFT_LEN represents the point of the fast Fourier transform. The single talk determining means of the single talk determining unit 115 can apply various methods widely. For example, a single talk is used for a frame having a strong correlation between the far-end signal and the near-end input signal, and a frame having a weak correlation is used. Other than single talk. Single talk determining section 115 outputs the determination result to echo path characteristic calculating section 118.

  The far-end signal spectrum envelope extraction unit 116 calculates the spectrum envelope | RIN_ENVE (i, ω) | of the amplitude spectrum | RIN (i, ω) | of the far-end signal and calculates the calculated spectrum envelope | RIN_ENVE (i, ω). | Is output to the echo path characteristic calculator 118 and the far-end signal frequency bin component presence / absence determiner 119.

  The near-end input signal spectrum envelope extraction unit 117 calculates the spectrum envelope | SIN_ENVE (i, ω) | of the amplitude spectrum | SIN (i, ω) | of the near-end output signal, and calculates the calculated spectrum envelope | SIN_ENVE (i, ω) | is output to the echo path characteristic calculator 118.

Here, as a means for extracting the spectrum envelope of the amplitude spectrum of the far-end signal and the near-end input signal, for example, by using the moving average value of the frequency bins represented by the equations (12) and (13), the voice Remove the fine structure of the signal and extract the spectral envelope.

[] In Equations (12) and (13) is a Gaussian symbol, and [x] is a maximum integer less than or equal to x with respect to the real number x.

  Note that various methods can be widely applied to the spectral envelope extraction means by the far-end signal spectral envelope extraction unit 116 and the near-end input signal spectral envelope extraction unit 117, such as liftering in linear cepstrum analysis and linear prediction analysis. The spectral envelope may be extracted by a method such as frame averaging.

In the echo path characteristic calculation unit 118, the echo path characteristic | H1 (i, ω) | of the frame is determined as the amplitude spectrum of the far-end signal according to the equation (14) for the frame determined as single talk by the single talk determination unit 115. Is calculated using the spectral envelope | RIN_ENVE (i, ω) | and the spectral envelope | SIN_ENVE (i, ω) | of the amplitude spectrum of the near-end output signal.

  Then, the echo path characteristic calculation unit 118 outputs the calculated echo path characteristic | H1 (i, ω) | to the echo path characteristic update unit 120.

  The frequency bin component presence / absence determination unit 119 determines, for each frequency bin, the presence or absence of the frequency component of the spectrum envelope | RIN_ENVE (i, ω) | of the amplitude spectrum of the far-end signal, for example, threshold TH2 (i, ω). Are compared for each frequency bin, and the presence or absence of a frequency component is determined for each frequency bin. When the frequency component is larger than the threshold, it is determined that there is a frequency component of the frequency bin. When the frequency component is equal to or less than the threshold, it is determined that there is no frequency component of the frequency bin, and the determination result is used as the frequency bin echo path characteristic update unit 120. Output to. The value of the threshold TH2 (i, ω) is, for example, a constant, a frame average value of the far end signal, a background noise included in the far end signal from the far end signal, and a frame average value of the background noise. Set. The threshold value TH2 (i, ω) may be set to a different value for each frequency bin.

  When the echo path characteristic | H1 (i, ω) | is output from the echo path characteristic calculation unit, the frequency bin echo path characteristic update unit 120 estimates the echo path characteristic | H (1) estimated one frame before held in the echo path characteristic holding unit 108. i−1, ω) | is read out, and the echo path characteristic | H (i−1, ω) | estimated one frame before and the echo path characteristic | H1 (i, ω) | of the frame determined to be single talk are used. Update the echo path characteristics. At this time, the frequency bin component presence / absence determining unit 120 updates the echo path characteristic only for the frequency bin determined to have the frequency envelope of the spectrum envelope | RIN_ENVE (i, ω) | of the amplitude spectrum of the far-end signal, and has no frequency component. The frequency bin determined to be does not update the echo path characteristic.

Specifically, the frequency bin echo path characteristic updating unit 120 updates the echo path characteristic | H (i, ω) | according to the equation (15).

  In the equation (15), a is a coefficient of the time constant filter, and a is a value of 1 or less, and a value close to 1 (for example, a = 0.99 or the like) when it is desired to delay the update of the echo path characteristic. It is desirable that a value close to 0 (for example, a = 0.01 or the like) is desirable when it is desired to update the data quickly. Expression (12) is expressed as follows: For a frequency bin having a frequency component of the spectral envelope | RIN_ENVE (i, ω) | of the amplitude spectrum of the far-end signal, the echo path characteristic of the frequency bin held in the echo path characteristic holding unit 108 | H The echo path characteristic | H (i, ω) | of the frequency bin used in the next frame is calculated using (i−1, ω) | and the echo path characteristic | H1 (i, ω) | For frequency bins having no frequency component of the spectral envelope | RIN_ENVE (i, ω) | of the amplitude spectrum of the current frequency bin, the frequency bin echo path characteristic updating unit 120 calculates the stored echo path characteristic | H (i, ω) | It is a formula of using.

  In the near-end output signal time domain conversion unit 121, the spectrum SOUT (i, ω) of the near-end output signal is converted into a time domain signal by, for example, inverse fast Fourier transform (InverseFFT), and the time domain digital sound is converted. The signal is converted into a signal and output to the near-end output signal output terminal 122.

  The near-end output signal output terminal 122 is connected to, for example, a network such as an Internet protocol (IP) network or a radio wave of a wireless network such as a mobile phone, and the near-end output signal is connected via a line to which the near-end output signal is connected. Output to the far end that is the other party.

(A-3) Effect of First Embodiment As described above, according to the first embodiment, erroneous learning of echo path characteristics due to the spectral fine structure of the frequencies of the far-end signal and the near-end input signal is prevented, and echo Suppression performance can be maintained.

(B) Second Embodiment Next, a second embodiment of the echo suppression device, the echo suppression program, and the echo suppression method of the present invention will be described with reference to the drawings.

(B-1) Configuration of Second Embodiment In the second embodiment, the single talk determination process is different from that of the echo suppression apparatus 100 according to the first embodiment, and the other components are the first. This is the same as or corresponds to the constituent elements of the echo suppressor 100 of FIG. 1 according to the first embodiment.

  FIG. 2 is a block diagram showing a configuration of an echo suppression apparatus 200 according to the second embodiment.

  In FIG. 2, an echo suppression apparatus 200 according to the second embodiment includes a far-end signal input terminal 101, a DA converter 102, a speaker 103, a microphone 104, an AD converter 105, a far-end signal frequency domain transform unit 106, End signal amplitude spectrum calculation unit 107, echo path characteristic holding unit 108, estimated echo signal calculation unit 109, near end input signal frequency domain conversion unit 110, near end input signal amplitude spectrum calculation unit 111, echo suppression gain calculation unit 112, echo suppression 113, near-end output signal amplitude spectrum calculation unit 114, spectrum envelope correlation single talk determination unit 202, far-end signal spectrum envelope extraction unit 116, near-end input signal spectrum envelope extraction unit 117, echo path characteristic calculation unit 118, far-end signal Frequency bin component presence / absence determination unit 119, frequency bin echo path characteristic update unit 20, a near-end output signal time domain transform section 121, the near-end output signal output terminal 122, the spectral envelope correlation determination unit 201.

  That is, the echo suppression apparatus 200 according to the second embodiment includes a spectrum envelope correlation determination unit 201 in addition to the components of the echo suppression apparatus 100 according to the first embodiment, and the single talk detection unit of the echo suppression apparatus 100 Instead of 115, a spectrum envelope correlation single talk determination unit 202 is provided.

  In FIG. 2, the same or corresponding components as those of the echo suppression device 100 according to the first embodiment are denoted by the same reference numerals. Further, detailed descriptions of the same or corresponding components as those in the first embodiment will be omitted, and will be omitted here.

  The spectrum envelope correlation determination unit 201 calculates a correlation coefficient between the spectrum envelope of the amplitude spectrum of the far-end signal and the spectrum envelope of the amplitude spectrum of the near-end input signal, determines the presence / absence of correlation, and spectrum envelope correlation single talk The data is output to the determination unit 202.

  The spectrum envelope correlation single talk determination unit 202 determines whether or not it is a single talk or a single talk based on the amplitude spectrum of the near-end input signal, the amplitude spectrum of the near-end output signal, and the presence or absence of the correlation output from the spectrum envelope correlation determination unit 201. Is determined for each frame, and the determination result is output to the echo path characteristic calculator 118.

  Thereby, the frequency bin echo path characteristic updating unit 120 updates the echo path characteristic only in a frame in which the spectrum envelope of the amplitude spectrum of the far-end signal and the spectrum envelope of the amplitude spectrum of the near-end input signal are correlated.

(B-2) Operation | movement of 2nd Embodiment Next, the operation | movement of the echo suppression process in the echo suppression apparatus 200 of 2nd Embodiment is demonstrated in detail. Here, the echo suppression apparatus 200 according to the second embodiment will describe in detail processing operations in the spectrum envelope correlation determination unit 201 and the spectrum envelope correlation single talk determination unit 202.

In the echo suppression apparatus 200, the spectrum envelope correlation determination unit 201, the spectrum envelope | RIN_ENVE (i, ω) | of the amplitude spectrum of the far-end signal extracted by the far-end signal spectrum envelope extraction unit 116, and the near-end input signal spectrum The presence or absence of correlation is determined using the spectral envelope | SIN_ENVE (i, ω) | of the amplitude spectrum of the near-end input signal extracted by the envelope extraction unit 117. For the determination of the presence or absence of correlation, for example, the correlation coefficient corr (i) shown in Expression (16) is obtained.

In the equation (16), FFT_LEN represents the point of fast Fourier transform, aveRIN represents the frame average value of the spectrum envelope of the amplitude spectrum of the far-end signal, and aveSIN represents the frame average value of the spectrum envelope of the amplitude spectrum of the near-end input signal. The frame average value of the spectrum envelope of the amplitude spectrum of the far-end signal and the frame average value of the spectrum envelope of the amplitude spectrum of the near-end input signal are obtained by equations (17) and (18).

  Spectral envelope correlation determination section 201 determines the presence or absence of a correlation when the correlation coefficient corr (i) between the spectral envelope of the amplitude spectrum of the far-end signal and the spectral envelope of the amplitude spectrum of the near-end input signal is obtained. The correlation coefficient corr (i) obtained by the equation (16) changes from “−1” to “1”, and if it is close to “1”, there is a positive correlation. For example, the correlation coefficient corr (i) Is “0.9 or more”, it is determined that there is a correlation, and the determination result is output to the spectrum envelope correlation single talk determination unit 202. Note that various methods can be widely applied as the correlation determination method by the spectrum envelope correlation calculation unit 201. For example, the spectrum envelope correlation calculation unit 201 can detect the spectrum envelope of the amplitude spectrum of the far-end signal and the near-end input. The spectrum envelope of the amplitude spectrum of the signal may be input, a difference may be obtained for each frequency bin, and it may be determined that there is a correlation when the sum of the differences is small.

  Similar to the single talk determination unit 115, the spectrum envelope correlation single talk determination unit 202 compares each frame and determines whether it is single talk or other than single talk. In the frame determined to be single talk, if the spectrum envelope correlation determination unit 201 determines that there is a correlation, single talk is generated, otherwise double talk, or only the near-end audio signal is generated. The state is determined, and the determination result is output to the echo path characteristic calculator 118.

(B-3) Effects of Second Embodiment As described above, according to the second embodiment, in addition to the effects of the first embodiment, the spectral envelope and the near-end input of the amplitude spectrum of the far-end signal Since the echo path characteristic is updated only for the frames having a correlation in the spectrum envelope of the amplitude spectrum of the signal, the single-talk determination accuracy is improved and the echo can be suppressed.

(C) Other Embodiments In the above-described embodiments, various modified embodiments have been mentioned, but the present invention can also be applied to the following modified embodiments.

  The echo suppression device described in each of the above-described embodiments may be mounted on a device including a voice communication device used in a video conference system, a telephone conference system, or the like. Further, the echo suppression device of the present invention may be mounted on a mobile terminal such as a mobile phone, a smart phone, or a tablet terminal.

  In each of the above-described embodiments, the echo path characteristic is updated for each frequency bin in the case of single talk. When the single talk determination unit determines that the single talk is not performed, the echo path characteristic may not be updated for each frequency bin.

  DESCRIPTION OF SYMBOLS 100 and 200 ... Echo suppression apparatus, 101 ... Far end signal input terminal, 102 ... DA converter, 103 ... Speaker, 104 ... Microphone, 105 ... AD converter, 106 ... Far end signal frequency domain conversion part, 107 ... Far end Signal amplitude spectrum calculation unit 108... Echo path characteristic holding unit 109. Estimated echo signal calculation unit 110 110 Near end input signal frequency domain conversion unit 111. Near end input signal amplitude spectrum calculation unit 112 112 Echo suppression gain calculation unit , 113 ... Echo suppression unit, 114 ... Near end output signal amplitude spectrum calculation unit, 115 ... Single talk determination unit, 116 ... Far end signal spectrum envelope extraction unit, 117 ... Near end output signal spectrum envelope extraction unit, 118 ... Echo path characteristic Update unit, 119 ... Far-end signal frequency bin component presence / absence determination unit, 120 ... Frequency bin echo path characteristic update 121: Near end output signal time domain conversion unit 122 ... Near end output signal output terminal 201 ... Spectral envelope correlation determination unit 202: Spectral envelope correlation single talk determination unit 300 ... Echo canceling device 301 ... Far end Input signal input terminal 302... DA converter 303 303 speaker 304 microphone 305 AD converter 306 first frequency analysis unit 307 second frequency analysis unit 308 acoustic coupling amount calculation unit 309 ... Estimated echo calculator 310. Echo suppress gain calculator 311. Accumulator 312 Frequency-time converter 313 Near-end output signal output terminal

Claims (5)

In an echo suppressor that suppresses an acoustic echo signal based on a near-end input signal and a far-end output signal,
Far-end signal amplitude spectrum calculating means for converting the input far-end signal into the frequency domain and obtaining the amplitude spectrum of the far-end signal;
Near-end input signal amplitude spectrum calculating means for converting the input near-end input signal into the frequency domain and obtaining the amplitude spectrum of the near-end input signal;
A far-end signal spectrum envelope extracting means for extracting a spectrum envelope of the amplitude spectrum of the far-end signal;
A near-end input signal spectrum envelope extracting means for extracting a spectrum envelope of the amplitude spectrum of the near-end input signal;
Frequency bin component presence / absence determining means for determining the presence / absence of a frequency component for each frequency bin from the spectrum envelope of the amplitude spectrum of the far-end signal,
Only the frequency bin determined to have a frequency component based on the determination result of the frequency bin component presence / absence determination means in the frame determined to be single talk, using the echo path characteristic and the echo path characteristic of the previous frame, the echo path of the frequency bin. A frequency bin echo path characteristic updating means for updating the characteristic;
Estimated echo signal calculating means for calculating an estimated echo signal based on the echo path characteristics for each frequency bin estimated by the frequency bin echo path characteristic estimating means and the amplitude spectrum of the far-end signal;
Echo suppression means comprising: echo suppression means for calculating an echo suppression gain from the estimated echo signal and the amplitude spectrum of the near-end input signal and suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal. apparatus. Single-talk determination means for determining, for each frame, single-talk or non-single-talk from the single-talk amplitude spectrum of the far-end signal, the near-end input signal, and the near-end output signal;
Echo path characteristic calculation for calculating an estimated echo signal based on the spectrum envelope of the amplitude spectrum of the far-end signal and the spectrum envelope of the amplitude spectrum of the near-end input signal in a frame determined as single talk by the single-talk determining means The echo suppressor according to claim 1, further comprising: means.   The frequency bin echo path characteristic estimation means has a frequency component of the amplitude spectrum of the far-end output signal in a frame determined to have a correlation between the spectrum envelope of the far-end output signal and the spectrum envelope of the near-end input signal. The echo suppression apparatus according to claim 2, wherein the echo path characteristic of the frequency bin is updated. A computer mounted on an echo suppression device that suppresses an acoustic echo signal based on a far-end signal from a near-end input signal,
Far-end signal amplitude spectrum calculating means for converting the input far-end signal into the frequency domain and obtaining the amplitude spectrum of the far-end signal;
Near-end input signal amplitude spectrum calculating means for converting the input near-end input signal into the frequency domain and obtaining the amplitude spectrum of the near-end input signal;
A far-end signal spectrum envelope extracting means for extracting a spectrum envelope of the amplitude spectrum of the far-end signal;
A near-end input signal spectrum envelope extracting means for extracting a spectrum envelope of the amplitude spectrum of the near-end input signal;
Frequency bin component presence / absence determining means for determining the presence / absence of a frequency component for each frequency bin from the spectrum envelope of the amplitude spectrum of the far-end signal,
Only the frequency bin determined to have a frequency component based on the determination result of the frequency bin component presence / absence determination means in the frame determined to be single talk, using the echo path characteristic and the echo path characteristic of the previous frame, the echo path of the frequency bin. A frequency bin echo path characteristic updating means for updating the characteristic;
Estimated echo signal calculating means for calculating an estimated echo signal based on the echo path characteristics for each frequency bin estimated by the frequency bin echo path characteristic estimating means and the amplitude spectrum of the far-end signal;
An echo suppression gain is calculated from the estimated echo signal and the amplitude spectrum of the near-end input signal, and functions as echo suppression means for suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal. Echo suppression program. In an echo suppression method for suppressing an acoustic echo signal based on a far end output signal from a near end input signal,
Far-end signal amplitude spectrum calculating means, near-end input signal amplitude spectrum calculating means, far-end signal amplitude spectrum envelope extracting means, near-end input signal amplitude spectrum envelope extracting means, frequency bin component presence / absence determining means, frequency bin echo path characteristic updating means, estimation Having echo signal calculation means, echo suppression means,
Convert the input far-end signal to the frequency domain, find the far-end signal amplitude spectrum to obtain the amplitude spectrum of the far-end signal,
Transform the input near-end input signal into the frequency domain, find the near-end input signal amplitude spectrum to obtain the amplitude spectrum of the near-end input signal,
Extract the far-end signal spectrum envelope to extract the spectrum envelope of the amplitude spectrum of the far-end signal,
Extract the near-end input signal spectrum envelope to extract the spectrum envelope of the amplitude spectrum of the near-end input signal,
Determine the presence or absence of a frequency bin component that determines the presence or absence of a frequency component for each frequency bin from the spectrum envelope of the amplitude spectrum of the far-end signal,
Only the frequency bin determined to have a frequency component based on the determination result of the frequency bin component presence / absence determination unit in the frame determined to be single talk, using the echo path characteristic and the echo path characteristic of the previous frame, the echo path of the frequency bin. Update frequency bin echo path characteristics to update characteristics,
Obtain an estimated echo signal for calculating an estimated echo signal based on the echo path characteristic for each frequency bin estimated by the frequency bin echo path characteristic estimating means and the amplitude spectrum of the far-end signal,
An echo suppression method comprising: calculating an echo suppression gain from the estimated echo signal and the amplitude spectrum of the near-end input signal and suppressing the estimated echo signal from the amplitude spectrum of the near-end input signal.

Images