JP6970422B2 - Acoustic signal processing device, acoustic signal processing method, and acoustic signal processing program - Google Patents

Description

The present invention relates to devices, methods and programs for removing acoustic echoes.
do.

In a device that receives a user's sound with a microphone and outputs a predetermined system sound from a speaker in a loudspeaker, a phenomenon that the system sound output from the speaker propagates through air or the like and is picked up by the microphone often occurs. .. At this time, the sound derived from the system voice picked up by the microphone is called the echo of the system voice.

For example, in a robot that recognizes a user's voice and provides a service, the system voice emitted by the robot becomes an echo and is picked up by its own microphone, even though the user is not speaking. There is a problem that the robot mistakenly recognizes the echo as the user's voice and starts some kind of response.

Therefore, it is necessary to remove the echo of the system voice mixed in the microphone input so that such an erroneous response does not occur. At this time, since it is known what kind of voice the robot's system voice is output from the speaker, it is possible to erase the echo of the system voice from the microphone input by a function generally called an echo eraser. ..

The echo eraser trains an adaptive filter to learn the transfer function of the system voice propagation path (hereinafter referred to as echo path) using a learning identification method (Normalized Rest Mean Square algorithm), etc., and multiplies this filter coefficient by the system voice. By generating an echo-simulated sound (hereinafter referred to as an echo replica) and subtracting this echo replica from the microphone input, an output sound in which the echo is erased (hereinafter referred to as an error output) is generated. If the learning of such an adaptive filter is successful, the output audio should remain without echo.

However, in the situation where non-echo voice (user's voice, environmental noise, etc.) is input from the microphone together with echo (with non-echo voice), learning of the adaptive filter does not proceed well, and the echo remains and the user May cause audio distortion. In order to avoid this, a detector is provided to detect the state with non-echo voice, and control is added to stop or delay the learning of the adaptive filter when the state with non-echo voice is detected.

Patent Document 1 discloses a call determination device that determines a call state (presence or absence of non-echo voice, etc.) from a loudspeaker output voice, an echo replica, and a level or correlation of a microphone input. In this call determination device, it may be erroneous if each station making a call independently determines the call status. Therefore, it is necessary to compare the determination results made by both stations to determine the call status.

Further, Patent Document 2 discloses a technique for ON / OFF control of learning of an adaptive filter according to the output voice power after echo elimination when learning an adaptive filter by a learning identification method or the like in a signal adaptation processing device and an echo suppression device. ing. In this technique, if the voice level after erasing the echo component from the microphone input exceeds a predetermined threshold value, it is assumed that the microphone input contains voice other than echo, and the state with non-echo voice is detected.

In the case of this technique, since the learning of the adaptive filter is turned ON / OFF depending on the output voice after echo erasing, the correct ON / OFF control is possible when the correct learning of the adaptive filter is advanced to some extent.

However, in the initial stage of learning in which the filter is not sufficiently adapted, an error occurs in this ON / OFF control, and as a result, the incorrect filter is learned, and learning may be delayed. This is because non-echo speech detection and adaptive filter learning are interdependent like chickens and eggs.

Japanese Unexamined Patent Publication No. 2008-060938 Japanese Unexamined Patent Publication No. 08-065214

The present invention has been enthusiastically researched and completed focusing on such problems, and its purpose is to eliminate the interdependence between non-echo speech detection and adaptive filter learning, but to eliminate echo (or echo). Based on the voice (weakened), it is necessary to promptly and correctly determine whether or not the microphone input contains voice other than echo (user's voice, etc.).

In order to solve the above problems, the present invention has a first conversion unit that converts an audio signal before being output from a speaker into first spectral data, and a second spectral data that converts an audio signal input from a microphone into second spectral data. A second conversion unit that converts to, a determination unit that determines the presence or absence of non-echo voice based on the first spectrum data and the second spectrum data, the first spectrum data, and the second spectrum. The echo erasing unit includes an echo erasing unit that inputs data and erases echoes using an adaptive filter, and the echo erasing unit, in the presence of the non-echo voice, obtains a coefficient indicating the learning strength of the adaptive filter. It is an acoustic signal processing device that lowers the value compared to the case where there is no non-echo voice.

According to the present invention, it is possible to quickly and correctly determine whether or not the microphone input contains voice other than echo (user's voice, etc.).

It is a functional block diagram of the acoustic signal processing apparatus which concerns on Example 1 of this invention. It is a functional block diagram of the echo erasing part 6 which concerns on Example 1 of this invention. It is a functional block diagram of the non-echo voice presence / absence determination unit 9 which concerns on Example 1 of this invention. It is a flowchart which shows the processing flow of the acoustic signal processing apparatus which concerns on Example 1 of this invention. It is a hardware block diagram of the acoustic signal processing apparatus which concerns on Example 1 of this invention. It is a figure which shows the result of the processing by the acoustic signal processing apparatus which concerns on Example 1 of this invention. It is a functional block diagram of the acoustic signal processing apparatus which concerns on Example 2 of this invention. It is a flowchart which shows the processing flow of the acoustic signal processing apparatus which concerns on Example 2 of this invention. It is a figure which shows the result of the processing by the acoustic signal processing apparatus which concerns on Example 2 of this invention.

Embodiments of the present invention will be described with reference to the drawings. In addition, the same reference numerals are given to common parts in each figure, and duplicate description is omitted.

FIG. 1 is a functional block diagram of the acoustic signal processing device according to the first embodiment of the present invention. In this embodiment, an acoustic signal processing device applied to a robot that recognizes a user's voice and provides a service will be described. Here, the system voice means the voice emitted by the robot.

(composition)
The speaker 1 outputs a loudspeaker signal x (t) of the system voice. The microphone 2 is a microphone for inputting a user's voice or the like as a voice signal m (t). The echo 3 indicates an echo of the system sound output from the speaker 1 in a loudspeaker.

The echo propagation path H indicates a propagation path (echo path) in which the echo 3 of the system voice reaches the microphone 2. Here, t is an index representing the time in the sampling cycle of the audio signal.

The first frequency decomposition unit 4 converts the voice signal x (t) in the time domain of the system voice into the spectral data x (ω, f) in the frequency domain by FFT (Fast Fourier Transform) processing. That is, it can be said that the frequency decomposition unit 4 is a first conversion unit that converts the audio signal x (t) before being output from the speaker 1 into the first spectral data x (ω, f).

Here, ω is an index representing the frequency bin number of the FFT output. In the FFT process, the time domain signal in the analysis window is converted into frequency domain spectral data while shifting the analysis window of a predetermined number of samples (frame length FL) by a predetermined number of samples (frame shift amount FS). This is the FFT processing unit (frame). f is an index representing the time (frame number) counted in the processing unit of the FFT. The first spectral data x (ω, f) obtained as a complex number at time f is two-dimensional vector data composed of the scalar value of the real part and the scalar value of the imaginary part. The length and direction of the vector represent the amplitude and phase of the frequency component represented by ω.

Similarly, the audio signal m (t) input from the microphone 2 is also converted into the spectrum data m (ω, f) in the frequency domain by the second frequency decomposition unit 5. That is, it can be said that the frequency decomposition unit 5 is a second conversion unit that converts the audio signal m (t) input from the microphone 2 into the second spectral data m (ω, f).

FIG. 2 is a functional block diagram of the echo erasing unit 6 according to the first embodiment of the present invention. The echo erasing unit 6 includes an adaptive filter 11 and a subtractor 12, and includes a second spectrum data m (ω, f) which is a microphone input and a first spectrum data x (ω, f) which is a system sound. Is an input, and the error output e (ω, f) is calculated from the equation (1) to eliminate the echo component from the second spectral data m (ω, f). Here, y (ω, f) is an echo replica, and is calculated by multiplying the first spectral data x (ω, f) by the filter coefficient w (ω, f).

The filter coefficient w (ω, f) in the equation (1) learns the transmission characteristics of the echo path H by the learning identification method (Normalized Last Men Squares algorithm) shown in the equation (2). Here, * (asterisk) represents the complex conjugate, and μ is the step size that controls the learning speed.

Since the learning identification method (NLMS algorithm) is a probability gradient algorithm of the mean square error minimization norm, learning of the filter coefficient w (ω, f) using this is always included in the error output e (ω, f). The process proceeds so as to obtain a filter coefficient w (ω, f) that minimizes the power of the component correlated with the first spectral data x (ω, f). Therefore, if the second spectrum data m (ω, f) contains components other than echo such as user voice, it is included in a part of the user voice (first spectrum data x (ω, f)). The filter is trained so that even the frequency component) is erased.

However, since the adaptive filter 11 learned in this way is by no means a correct value, it causes residual echo and distortion of the user's voice. Therefore, it is necessary to detect a situation in which the second spectral data m (ω, f) contains voice other than echo (state with non-echo voice) and stop or weaken the learning of the adaptive filter 11. Become.

In order to realize this, the echo erasing unit 6 controls the value of the step size μ according to whether the DT (f) described later is 0 (state without non-echo sound) or 1 (state with non-echo sound). That is, μ is reduced in the state with non-echo voice as compared with the state without non-echo voice, and as a result, the learning intensity (also referred to as learning speed) of the adaptive filter 11 is suppressed to a low level during this period. That is, the step size μ is a coefficient indicating the learning strength of the adaptive filter 11.

In the non-echo voice presence / absence determination unit 9, whether the microphone input includes voice other than echo (user voice, etc.) in order to control the echo erasing unit 6 to stop or weaken the learning of the adaptive filter 11. It is a functional block that determines whether or not it is. The details will be described later, but by making this determination more quickly and correctly, there is an effect that the performance of the echo erasing unit 6 is improved.

However, even if such an echo erasing unit 6 is used, residual echo may remain. The residual echo may adversely affect the accuracy, for example, when recognizing the voice after echo elimination. Therefore, in this embodiment, the residual echo suppression unit 7 is provided after the echo erasing unit 6. However, it should be noted that the residual echo suppression unit 7 is not an indispensable configuration for this embodiment because the influence of the residual echo changes depending on the request of the post-stage processing such as the voice recognition processing.

The residual echo suppression unit 7 shown in FIG. 1 receives an error output e (ω, f) and a first spectral data x (ω, f) as inputs, and remains in the error output e (ω, f) according to the equation (3). Generates voice o2 (ω, f) that suppresses the echo component.

As shown in the equation (4), o1 (ω, f) is a voice obtained by multiplying the amplitude of e (ω, f) by G. This G is a coefficient that approximates the ratio of the magnitudes of the residual echoes included in the error output e (ω, f). The numerical value of G is obtained experimentally. Further, DS (ω, f) is an instantaneous value of the suppression coefficient, and gain (ω, f) is a moving average value of DS (ω, f) approximately calculated by the forgetting coefficient. In addition, gs is a coefficient for giving the strength of suppression.

The waveform generation unit 8 shown in FIG. 1 generates a waveform signal O (t) in the time domain by performing inverse FFT processing on the voice o2 (ω, f) by the residual echo suppression unit 7. This O (t) is the final output audio signal in this embodiment.

Next, the non-echo voice presence / absence determination unit 9 shown in FIG. 1 will be described. FIG. 3 is a functional block diagram of the non-echo voice presence / absence determination unit 9 according to the first embodiment of the present invention. The non-echo voice presence / absence determination unit 9 includes an echo suppression unit 21, a waveform generation unit 22, and a determination unit 23.

The echo suppression unit 21 takes the second spectrum data m (ω, f) and the first spectrum data x (ω, f) as inputs, and uses the equation (5) as the second spectrum data m (ω, f). The voice s (ω, f) in which the contained echo component is suppressed is obtained.

Here, gain (ω, f) in the equation (5) is a suppression coefficient calculated by the following equation (6).

The MR (ω, f) in the equation (6) represents the moving average value of the gain from the speaker 1 to the microphone 2, and the EL (ω, f) is the magnitude of the echo estimated from the MR (ω, f). NL (ω, f) is the current non-echo voice magnitude calculated from EL (ω, f), and gs is a coefficient for giving the strength of suppression. FL (ω, f) is a quantity that gives a lower limit value of NL (ω, f), and is determined from the second spectral data m (ω, f). As a result of the above, XX (ω, f) is calculated by flooring NL (ω, f) with FL (ω, f) as the size of the current non-echo voice. Then, gain (ω, f) is calculated as the ratio of the magnitude XX (ω, f) of the non-echo voice to the second spectral data m (ω, f).

The s (ω, f) calculated by the equation (5) is converted into an audio signal s (t) in the time domain by the waveform generation unit 22 in the next stage.

In the subsequent determination unit 23, as shown in the equation (7), the number S of samples whose amplitude absolute value | s (t) | is equal to or greater than the threshold value th2 in s (t) for the latest frameshift amount is calculated. Will be done. When the calculation result S becomes the threshold value th1 or more, it is determined that the frame f is in a state with non-echo sound, and DT (f) = 1 is output. In other cases, DT (f) = 0 is output assuming that there is no non-echo sound.

In this way, by applying the echo suppression process to the second spectral data m (ω, f), the echo contained in the s (ω, f) can be suppressed faster than the echo suppressed by the adaptive filter 11.

Further, by converting s (ω, f) once into a waveform s (t) in the time domain, counting the amplitude values having a threshold value th2 or more in the waveform and performing threshold processing, suppression that does not last long even if the amplitude is large. It can be robust against noise other than early echoes and sudden echoes.

As described above, the echo suppression unit 21 in the non-echo voice presence / absence determination unit 9 can quickly suppress the echo because the gain (ω, f) of the equation (5) is quickly obtained. Since the gain (ω, f) is a real number, the echo suppression unit 21 changes only the amplitude without changing the phase of the second spectral data m (ω, f). This is not accurate enough, but it is sufficient to determine the presence or absence of non-echo voice based on the amplitude. In short, the echo suppression unit 21 suppresses the echo included in the second spectral data m (ω, f) with an emphasis on speed rather than accuracy. On the other hand, the echo erasing unit 6 is a means for erasing the echo included in the second spectral data m (ω, f) with an emphasis on accuracy. Since both y (ω, f) and w (ω, f) in the equation (1) are complex numbers, the echo erasing unit 6 controls the amplitude and phase of the second spectral data m (ω, f) to increase the value. Performs accurate echo elimination. However, it takes time because the calculation of the equation (2) must be repeated many times before the echo disappearing w (ω, f) can be obtained.

FIG. 4 is a flowchart showing a processing flow of the acoustic signal processing apparatus according to the first embodiment of the present invention. When the acoustic signal processing device of this embodiment is started, the initialization processing step S1 is first executed. In this process, the time index t and the frame number f are initialized to 0.

In the subsequent FS sample input processing step S2, each audio signal of m (t) and x (t) is input by the frame shift amount FS sample.

Next, in the FL sample accumulation determination processing step S3, it is determined whether or not the number of samples of the m (t) and x (t) audio signals input so far is equal to or greater than the frame length FL, which is the length of the FFT analysis window. judge. If the number of m (t) and x (t) audio signal samples input so far is less than the frame length FL, the subsequent FFT processing cannot be performed, so branch to the left (No) in the figure. Then, the dummy output generation processing step S9 is executed. On the other hand, if this is not the case, the frequency decomposition process step S4 is executed by branching to the lower part (Yes) in the figure.

In the dummy output generation processing step S9, for example, as the output audio signal O (t) = m (t), the microphone input signal is output as it is, or silence is output.

The frequency decomposition processing step S4 is a processing step corresponding to the first frequency decomposition unit 4 and the second frequency decomposition unit 5, and the input x (t) and m (t) audio signals are converted into the first spectrum. It is converted into data x (ω, f) and second spectral data m (ω, f).

The non-echo voice detection processing step S5 is a processing step corresponding to the non-echo voice presence / absence determination unit 9, and is the second spectrum data m by the calculation of the formula (5), the formula (6), and the formula (7). The value of DT (f) is determined from (ω, f) and the first spectral data x (ω, f).

The echo erasing process step S6 is a processing step corresponding to the echo erasing unit 6, and e (ω, f) is calculated by the calculation of the equations (1) and (2), and the step size based on the DT (f) is calculated. The filter coefficient w (ω, f) is updated by controlling μ.

The residual echo suppression processing step S7 is a processing step corresponding to the residual echo suppression unit 7, and the voice o2 (ω) in which the residual echo is suppressed from e (ω, f) by the calculation of the equations (3) and (4). , F) is calculated.

The output generation processing step S8 is a processing step corresponding to the waveform generation unit 8, and calculates the output audio signal O (t) from o2 (ω, f) by reverse FFT processing. When the dummy output generation processing step S9 and the output generation processing step S8 are executed, the processing returns to the FS sample input processing step S2. At that time, the time index t is increased by FS, and the frame number f is increased by 1.

FIG. 5 is a hardware configuration diagram of the acoustic signal processing device according to the first embodiment of the present invention. The present embodiment is not limited to the acoustic signal processing apparatus shown by the functional blocks of FIGS. 1 to 3 and the acoustic signal processing method shown by the flowchart of FIG. For example, the computer may function as the acoustic signal processing device of FIG. 1 or may be implemented as a program for executing the processing procedure of the acoustic signal processing method of FIG.

Specifically, this embodiment can be carried out using a computer as shown in FIG. The CPU (Central Processing Unit) 103 includes a RAM (Random Access Memory) 104, a ROM (Read Only Memory) 105, an HDD (Hard Disk Drive) 106, a LAN (Local Area Network) 107, and a mouse / keyboard. Be connected. These are the general elements that make up a computer.

The storage 110 includes drives for supplying programs and data to a computer from the outside via a storage medium, specifically, an optical disk drive, a magnetic disk drive, and a CF (Compact Flash) / SD (Secure Digital) card slot. And USB (Universal Serial Bus) memory.

The microphone 101 and the speaker 112 correspond to the microphone 2 and the speaker 1 shown in FIG. Further, the microphone 113 corresponds to the noise input microphone described in the second embodiment described later.

The sound waves are converted into electrical signals by the microphone 101 and the microphone 113, and converted into digital data by the A / D converter 102 and the A / D converter 114. The digital data from the A / D converter 102 and the A / D converter 114 is processed by the CPU 103 in the process of executing the program instruction.

In the computer device shown in FIG. 5, an acoustic signal processing program for executing the processing step shown in FIG. 4 is stored in the HDD 106, read into the RAM 104, and executed by the CPU 103. At that time, the microphone 101 and the A / D converter 102 are used for the input of the audio signal m (t) including the user's voice, and the D / A converter 111 and the speaker 112 are used for the loudspeaker output of the system voice x (t). Further, the microphone 113 and the A / D converter 114 are used for the input of the noise signal n (t) described in the second embodiment described later. Then, the output voice O (t) is generated and output by processing these m (t) and x (t) or m (t), x (t) and n (t) by the CPU 103.

As a result, the computer device shown in FIG. 5 functions as the acoustic signal processing device according to the present embodiment. Further, the computer device can be supplied with an acoustic signal processing program from a recording medium inserted in the storage 110 or another device connected via the LAN 107.

It should be noted that this computer device can also accept user's operation input and present information to the user via the mouse / keyboard 108 or the display 109. Further, when this computer device is applied not only to an acoustic signal processing device but also to a robot that recognizes a user's voice and provides a service, elements unnecessary at the time of service provision such as a mouse / keyboard 108 are computer devices. It is removable from.

FIG. 6 is a diagram showing the result of processing by the acoustic signal processing apparatus according to the first embodiment of the present invention. The microphone input signal m (t) in the figure (a) is converted into the second spectral data m (ω, f) by the second frequency decomposition unit 5.

The user voice and the system voice echo are mixed in the microphone input signal m (t). The system voice x (t) is also converted into the first spectral data x (ω, f) by the first frequency decomposition unit 4. Cooley-Tukey DFT Algorithm is used for FFT and reverse FFT, frame length FL is 512 samples, frame shift amount FS is 160 samples, and FFT and reverse FFT are executed using a Hanning window for window hanging.

In the figure, (b) is an output audio signal O (t). The output o2 (ω, f) obtained through the echo erasing unit 6 and the residual echo suppressing unit 7 is output as a time domain signal O (t) by the waveform generation unit 8. This is the output voice of the acoustic signal processing device of this embodiment.

Only the user voice remains strongly in the output voice O (t). At this time, the graph of the output DT (f) of the non-echo voice presence / absence determination unit 9 shown in the figure (c) also stands up during the period in which the user voice exists.

In the figure, m (t) / O (t) shown in (d) represents an evaluation amount called ERLE (Echo Return Loss Relationship). ERRE is a quantity defined by the following equation (8), and indicates how small the output power is with respect to the input power by a dB value, and the larger the value, the higher the erasing performance. E [*] in the formula indicates that the average value is calculated for each n samples.

If the echo erasing performance is to be determined, the ERLE must be calculated over the period in which only the system voice echo is present. Therefore, three such periods are selected from the graph and designated as R1, R2, and R3 in the figure. All of these periods are periods in which only system audio echoes are present. Therefore, the correct answer is that DT (f) does not stand up (indicating a state without non-echo voice), and it can be seen from (c) in the figure that this is the case. When the average value of ERLE is picked up at each of the early stage (R1), the middle stage (R2), and the final stage (R3) of learning, it can be seen that the values are as high as 67.7 dB, 85.9 dB, and 102.6 dB.

(effect)
According to this embodiment, the non-echo voice presence / absence determination unit 9 detects a situation (state with non-echo voice) in which the second spectral data m (ω, f) contains voice other than echo, and the adaptive filter 11 Since it controls to stop or weaken the learning, it has the effect of being able to quickly and correctly determine whether or not the microphone input contains voice other than echo (user's voice, etc.).

The echo suppression unit 21 in the non-echo voice presence / absence determination unit 9 executes the echo suppression process in which the echo suppression amount can be increased faster than the echo elimination unit 6, so that the adaptive filter 11 is quickly and correctly learned from the initial stage of learning. It has the effect of speeding up and increasing accuracy. This is particularly suitable for mobile robots whose echo state frequently changes as they move.

In addition, when this embodiment is applied to a system that interacts with a user by voice, the system does not mistakenly recognize one's own voice, so that unnecessary voice recognition processing can be reduced, which saves CPU. , When using voice recognition on a cloud server, it has the effect of reducing the amount of communication.

In addition, it has the effect of expanding the system operating conditions by expanding the allowable range of the speaker volume and microphone sensitivity of the system, such as increasing the microphone sensitivity for users with low voice.

Further, even if the user and the system speak at the same time, only the voice of the user can be extracted and recognized, so that the user can talk to the system at any time, which has the effect of improving the usability of the system.

In the second embodiment, an acoustic signal processing device corresponding to environmental noise will be described. FIG. 7 is a functional block diagram of the acoustic signal processing device according to the second embodiment of the present invention. Reference numerals 1 to 9 in the figure are the same functional blocks as in the first embodiment, and thus description thereof will be omitted. The second embodiment has a configuration in which the functional blocks 31, 32, 33, and 34 are added to the first embodiment. Hereinafter, these added functional blocks will be mainly described.

The microphone 31 is a microphone arranged at a position farther from the speaker 1 and the user as compared with the microphone 2. This arrangement is designed so that the microphone 31 receives only a weak sound of the system voice and the user voice. As a result, the surrounding environmental noise is exclusively received by the microphone 31 and input as a noise signal n (t). The noise signal n (t) is converted into frequency domain data n (ω, f) by the third frequency decomposition unit 32. That is, the third frequency decomposition unit 32 can be said to be a third conversion unit that converts the noise signal n (t) into the third spectral data n (ω, f).

On the other hand, since environmental noise arrives from a relatively long distance such as the sound of a train, it is mixed in the input signal m (t) of the microphone 2 at the same level as the microphone 31. Since the environmental noise mixed in this m (t) has no correlation with the system voice x (t), it cannot be eliminated by the processing of the echo erasing unit 6 and the residual echo suppressing unit 7.

33 of FIG. 7 is an environmental noise suppressing unit for suppressing this environmental noise. The environmental noise suppression unit 33 receives the output o2 (ω, f) of the residual echo suppression unit 7 and the third spectrum data n (ω, f) which is noise data as inputs, and the environment noise suppression unit 33 inputs o2 (ω, f) according to the equation (9). ) Suppresses the noise component, and the voice o3 (ω, f) is calculated.

Here, gain (ω, f) in the equation (9) is a suppression coefficient calculated by the following equation (10). This is a calculation without the flooring process from the equation (6).

34 in FIG. 7 is a minute frequency component suppression unit. The minute frequency component suppressing unit 34 takes the output o3 (ω, f) of the environmental noise suppressing unit 33 as an input, and according to the equation (11), the minute frequency component having an amplitude less than a predetermined threshold value included in the o3 (ω, f). The voice o4 (ω, f) that suppresses is calculated.

Here, gain (ω, f) in the equation (11) is a suppression coefficient calculated by the following equation (12). Note that 0.01 in the equation is a non-negative value smaller than 1.0 that gives a suppressing effect, and may be another value such as 0.0 or 0.02.

In the audio signal processing apparatus of this embodiment, the waveform generation unit 8 generates an output signal O (t) in the time domain by performing inverse FFT processing on the output audio o4 (ω, f) of the minute frequency component suppression unit 34. .. This O (t) is the output audio signal of the audio signal processing device of this embodiment.

FIG. 8 is a flowchart showing a processing flow of the acoustic signal processing apparatus according to the second embodiment of the present invention. When the acoustic signal processing device of this embodiment is started, the initialization processing step S21 is first executed. In this process, the time index t and the frame number f are initialized to 0.

In the subsequent FS sample input processing step S22, each audio signal of m (t), x (t), and n (t) is input by the frame shift amount FS sample.

Next, in the FL sample accumulation determination processing step S23, the frame length FL in which the number of samples of each of the m (t), x (t), and n (t) audio signals input so far is the length of the FFT analysis window. It is determined whether or not it is the above. If the number of m (t), X (t), and n (t) audio signal samples input so far is less than the frame length FL, the subsequent FFT processing cannot be performed, so the left in the figure. Branch to (No) and execute the dummy output generation processing step S29. On the other hand, if this is not the case, the frequency decomposition process step S24 is executed by branching to the lower part (Yes) in the figure.

In the dummy output generation processing step S29, for example, as the output audio signal O (t) = m (t), the microphone input signal is output as it is, or silence is output.

The frequency decomposition processing step S24 is a processing step corresponding to the first frequency decomposition unit 4, the second frequency decomposition unit 5, and the third frequency decomposition unit 32, and the input x (t), m (t), Each voice signal of n (t) is converted into the first spectrum data x (ω, f), the second spectrum data m (ω, f), and the third spectrum data n (ω, f).

The non-echo voice detection processing step S25 is a processing step corresponding to the non-echo voice presence / absence determination unit 9, and the second spectral data m (ω) is calculated by the formulas (5), (6), and (7). , F) and the first spectral data x (ω, f) determine the value of DT (f).

The echo erasing processing step S26 is a processing step corresponding to the echo erasing unit 6, and e (ω, f) is calculated by the calculation of the equations (1) and (2), and the step size μ based on the DT (f) is calculated. The filter coefficient w (ω, f) is updated by the control of.

The residual echo suppression processing step S27 is a processing step corresponding to the residual echo suppression unit 7, and the voice o2 (ω, ω, f) in which the residual echo is suppressed from e (ω, f) by the calculation of the equations (3) and (4). f) is calculated.

The environmental noise suppression processing step S30 is a processing step corresponding to the environmental noise suppression unit 33, and the noise component is suppressed from o2 (ω, f) by the calculation of the equations (9) and (10). f) is calculated.

The minute frequency component suppression processing step S31 is a processing step corresponding to the minute frequency component suppression unit 34, and the minute frequency component is suppressed from o3 (ω, f) by the calculation of the equations (11) and (12). The voice o4 (ω, f) is calculated.

The output generation processing step S28 is a processing step corresponding to the waveform generation unit 8, and calculates the output audio signal O (t) from o4 (ω, f) by reverse FFT processing.

When the dummy output generation processing step S29 and the output generation processing step S28 are executed, the processing returns to the FS sample input processing step S22. At that time, the time index t is increased by FS, and the frame number f is increased by 1.

The hardware configuration of the acoustic signal processing device according to the second embodiment is the same as the hardware configuration of the first embodiment described with reference to FIG. 5, and is therefore omitted.

FIG. 9 is a diagram showing the results of processing by the acoustic signal processing apparatus according to the second embodiment of the present invention. The microphone input signal m (t) in the figure (a) is converted into the second spectral data m (ω, f) by the second frequency decomposition unit 5. This voice is a mixture of user voice, system voice echo, and environmental noise. The system voice x (t) is also converted into the first spectral data x (ω, f) by the first frequency decomposition unit 4. Further, the environmental noise n (t) is also converted into the third spectral data n (ω, f) by the third frequency decomposition unit 32. In Example 2, as in Example 1, FFT and reverse FFT are executed using Cooley-Tukey DFT Algorithm, frame length FL is 512 samples, frame shift amount FS is 160 samples, and Hanning window is used for window hanging. doing.

In the figure, (b) is an output audio signal O (t). The output voice o4 (ω, f) obtained through the echo erasing unit 6, the residual echo suppressing unit 7, the environmental noise suppressing unit 33, and the minute frequency component suppressing unit 34 is converted into a time domain signal O (t) by the waveform generation unit 8. Will be converted. This is the output voice of this embodiment. Only the user voice remains strongly in the output voice O (t). At this time, the graph of the output DT (f) of the non-echo voice presence / absence determination unit 9 shown in FIG.

In the figure, m (t) / O (t) shown in (d) shows the transition of the ERLE value. However, ERLE should be calculated in the period when only the system voice echo exists, but this time there is no user voice and only the disturbing sound which is the combination of the system voice echo and the environmental noise exists. It was calculated in the sense of finding the erasing performance. From the graph, when the average values of the three points of the initial stage (R1 in the figure), the middle stage (R2 in the figure), and the final stage (R3 in the figure) are picked up, they are 133.5 dB, 109.4 dB, and 150.0 dB. It can be seen that a high value is recorded. This erasing performance is a numerical value including the effect of suppressing environmental noise and minute frequency components in addition to the effect of erasing and suppressing echoes.

(effect)
According to this embodiment, the effects described in the first embodiment are of course obtained, and further, the system operation conditions can be expanded even in a noisy place (for example, an exhibition hall).

Although the examples (including modified examples) of the present invention have been described above, two or more of these examples may be combined and carried out. Alternatively, one of these examples may be partially implemented. Furthermore, among these, two or more examples may be partially combined and carried out.

The present invention is not limited to the description of the embodiments of the above invention. Various modifications are also included in the present invention to the extent that those skilled in the art can easily conceive without departing from the description of the scope of claims. For example, the voice signal processing device of the present invention can also be applied as front-end processing of a guidance type robot having another voice conversation function.

1 Speaker 2, 31 Microphone 3 Echo 4 First frequency decomposition section (first conversion section)
5 Second frequency decomposition unit (second conversion unit)
6 Echo erasing unit 7 Residual echo suppression unit 8 Waveform generation unit 9 Non-echo voice presence / absence determination unit 11 Adaptive filter 12 Subtractor 21 Echo suppression unit 22 Waveform generation unit 23 Judgment unit 32 Third frequency decomposition unit (third conversion unit) )
33 Environmental noise suppression section 34 Micro frequency component suppression section

Claims (7)

The first conversion unit that converts the audio signal before being output from the speaker into the first spectral data, and
A second conversion unit that converts the audio signal input from the microphone into the second spectral data, and
A non-echo voice presence / absence determination unit that determines the presence / absence of non-echo voice based on the first spectrum data and the second spectrum data,
An echo elimination unit that inputs the first spectrum data and the second spectrum data and calculates an error output using an adaptive filter for eliminating echoes.
Equipped with
The echo erasing unit is
In the presence of the non-echo voice, the coefficient indicating the learning strength of the adaptive filter is lowered as compared with the case without the non-echo voice.
The non-echo voice presence / absence determination unit is
An echo suppression unit that suppresses the echo component of the second spectral data, a generation unit that generates an audio signal in the time domain from the output result of the echo suppression unit, and the presence / absence of the non-echo audio from the audio signal in the time domain. An acoustic signal processing device having a determination unit for determining. The determination unit
The number of data in which the amplitude of the waveform data of the audio signal generated by the generation unit is equal to or greater than a predetermined threshold value is calculated.
Wherein when the number of data is equal to or greater than a predetermined threshold value, the acoustic signal processing apparatus according to claim 1 determines that the non-echo sound is present. The acoustic signal processing device according to claim 1, wherein the coefficient indicating the learning strength of the adaptive filter is a step size. The acoustic signal processing device according to claim 1, further comprising a residual echo suppression unit after the echo erasing unit. A third converter that converts environmental noise signals input from other microphones into third spectral data, and
The acoustic signal processing device according to claim 1, further comprising an environmental noise suppressing unit that suppresses the third spectral data. The first conversion step of converting the audio signal before being output from the speaker into the first spectral data, and
A second conversion step of converting the audio signal input from the microphone into the second spectral data,
A non-echo voice presence / absence determination step for determining the presence / absence of non-echo voice based on the first spectrum data and the second spectrum data,
An echo elimination step in which the first spectral data and the second spectral data are input and the error output is calculated using an adaptive filter for eliminating echoes.
Equipped with
The echo erasing step is
In the presence of the non-echo voice, the coefficient indicating the learning strength of the adaptive filter is lowered as compared with the case without the non-echo voice.
The non-echo voice presence / absence determination step is
An echo suppression step that suppresses the echo component of the second spectral data, a generation step that generates an audio signal in the time domain from the output result of the echo suppression step, and the presence / absence of the non-echo audio from the audio signal in the time domain. A determination step for determining an acoustic signal processing method. The first conversion step of converting the audio signal before being output from the speaker into the first spectral data, and
A second conversion step of converting the audio signal input from the microphone into the second spectral data,
A non-echo voice presence / absence determination step for determining the presence / absence of non-echo voice based on the first spectrum data and the second spectrum data,
An echo elimination step in which the first spectral data and the second spectral data are input and the error output is calculated using an adaptive filter for eliminating echoes.
Is an acoustic signal processing program that can be executed on a computer.
The echo erasing step is
In the presence of the non-echo voice, the coefficient indicating the learning strength of the adaptive filter is lowered as compared with the case without the non-echo voice.
The non-echo voice presence / absence determination step is
An echo suppression step that suppresses the echo component of the second spectral data, a generation step that generates an audio signal in the time domain from the output result of the echo suppression step, and the presence / absence of the non-echo audio from the audio signal in the time domain. A determination step, and an acoustic signal processing program having.

Images