JP2003500936A - Improving near-end audio signals in echo suppression systems - Google Patents

Abstract

(57) Abstract: In a hands-free environment, receiving an audio signal, generating an estimated acoustic echo signal, and generating a processed signal by removing the estimated acoustic echo signal from the audio signal. As a result, an improved near-end audio signal is generated. Then, a near-end improvement spectrum having one or more ranges of continuous frequencies at which the detector spectrum has a maximum is determined. Here, the continuous frequency range is associated with a relatively high echo return loss in the processed signal. The processed signal is filtered according to a near-end enhancement spectrum, thereby producing an enhanced near-end audio signal. The enhanced near-end audio signal is then applied to any number of elements that are intended to process near-end speech. For example, the amount of energy contained in the enhanced near-end audio signal when applied to the audio activity detector is measured. The presence or absence of near-end audio activity is determined based on the measured energy of the improved near-end audio signal. The process is repeated periodically to provide a dynamically adjustable operation.

Description

Detailed Description of the Invention

BACKGROUND OF THE INVENTION The present invention relates to the processing of audio signals in communication systems, and more particularly to improving near-end audio in signals containing near-end audio combined with far-end audio echoes.

In the field of communications, for example with speakerphones and in cellular telephones, it is often desirable for a user to be able to operate a communication device without the need for a user to subsequently occupy one or more of their hands. There is. This is an important factor in the automotive environment. In that environment, if the driver is immersive in holding telephone equipment, he may endanger not only his own safety, but also the safety of others sharing the road. The freedom to use the hand for anything other than holding a microphone would be equally useful in other applications, such as internet communication with a personal computer, voice recognition with a computer, or an audiovisual presentation system. is there.

In order to accommodate these important needs, so-called “hands-free” devices have been developed. In that device, the microphone and loudspeaker are mounted in a hands-free environment, thereby obviating the need to hold them. For example, for automotive applications, the microphone of the cellular phone may be mounted on the sun visor, while the loudspeaker may be a dashboard mounted unit or may be associated with car stereo equipment. The components mounted in this way allow the user of the cellular telephone to have a conversation without having to hold the cellular unit or its handset. Similarly, personal computers often have, for example, a microphone and a loudspeaker mounted within the monitor in close proximity to each other.

A problem with hands-free configuration is that the microphone tends to pick up the sound from the nearby loudspeakers in addition to the voice of the user of the hands-free device (so-called “near end user”). This is also a problem in some devices that are not hands-free, such as the increasingly smaller handheld mobile phones. (Because of their small size, mobile phone microphones cannot be completely shielded from the sound emitted by their loudspeakers.) It is in many applications that the sound produced by that loudspeaker is perceived by the microphone. Cause problems. For example, in a communication device, the delay introduced by the communication system as a whole causes the sound from the loudspeaker to be heard by the individual at the other end of the call (so-called "far end") as an echo of their voice. Such echoes reduce the audio quality, which mitigation is desired.
Similar problems exist, for example, in automated systems that include speech recognition elements that synthesize speech through loudspeakers and recognize and respond to commands spoken by a microphone and other words sensed. In such applications, the presence of echoes of synthesized speech in the microphone signal significantly degrades the performance of the speech recognition element. Solutions to improve such echoes include the use of adaptive echo cancellation filters and echo attenuators.

As a representative example of hands-free equipment in general, a typical “hands-free” mobile telephone with a conventional echo canceller in the form of an adaptive filter arrangement is depicted in FIG. For example, the hands-free communication environment may be the interior of a car with a mobile telephone installed. Such an environment causes acoustic signal propagation there, but the effect is usually unknown. In the future, this type of environment will be referred to as the unknown system H (z) throughout this specification. The microphone 105 is intended to detect the user's voice, but may have the undesired effect of detecting the audio signal emanating from the loudspeaker 109. That is the undesired behavior of introducing an echo signal into the system.

If it cannot be removed, a circuit for attenuating the echo includes an adaptive filter 101 such as an adaptive finite shock response (FIR) filter and a least mean square (L).
MS) Adaptation unit 103 such as cross-correlator and subtractor 107
including. In its operation, the adaptive filter 101 produces an echo estimate signal 102, commonly referred to as the u signal. The echo estimate signal 102 is a convolution of the far-end signal 112 with a series of m filter weighting factors (h _i ) of the filter 101 (see equation 1). Where x (n) is the input signal, m is the number of weighting factors, and n is the number of samples.

When its weighting factors are set correctly, adaptive filter 101 produces a shock response approximately equal to the response produced by loudspeaker 109 in the unknown system H (z). The echo estimate signal 102 generated by the adaptive filter 101 is subtracted from the input digitized microphone signal 126 (represented by u (n) in Equation 2) to produce an error signal e (n) ( (See Equation 2). Ideally, the unknown system H (z
A) is removed from the digitized microphone signal 126 by subtraction of the echo estimate signal 102. Usually the number of weighting factors needed to effectively cancel the echo (hereinafter referred to as "multiple factors")
Depends on its application. For handheld phones, a number of coefficients less than 100 may be suitable. In the case of a hands-free telephone of a car, about 200-
400 coefficients are required. Larger spaces may require filters with over 1000 coefficients to provide adequate echo cancellation.

It can be seen that the effectiveness of the echo canceller is directly related to how well the adaptive filter 101 can replicate the shock response of the unknown system H (z). This in turn is directly related to the set of coefficients h _i maintained by the filter 101.

A system H (z) in which the adaptive filter 101 changes the coefficient h _i dynamically and the adaptive filter 101 is unknown
It would be advantageous to provide a mechanism that would allow it to adapt to changes in. In cars with a hands-free cellular configuration, such changes may occur when opening or closing windows or car doors. The known coefficient application method is the least mean square (
LMS process, which was first introduced by Widrow and Hoff in 1960 and is frequently used because of its efficiency and resistant nature. When applied to the echo cancellation problem, the LMS process is a statistical gradient step method with a rough (noisy) estimate of the gradient g (n) = e (n) x (n). , Increment steps towards minimizing the energy of the echo signal in the microphone signal e (n). Where x (
n) is a vector notation corresponding to the expression x (n) = [x (n) x (n-1) x (n-2) ... x (n-m + 1)]. The update information generated by the LMS process e (n) x (n) is used to determine the value of the coefficient at the next sample. The formula for calculating the next coefficient value h _i (n + 1) is given as: Where x (n) is the digitized input signal 134, (h _j ) is the filter weighting coefficient, i indicates a particular coefficient, m is the number of coefficients, n is the number of samples And μ is the step or update gain parameter.

The LMS method produces information in each incremental portion that may have positive or negative values. The information generated by the LMS process is provided to the filter to update its filter coefficients.

Returning again to FIG. 1, the conventional echo cancellation circuit includes a filter adaptation unit 103 in the form of an LMS cross-correlator that provides coefficient update information 104 to the filter 101. In this arrangement, the filter adaptation unit 103 represents a modified signal e (which represents the digitized microphone signal 126 minus the echo estimate signal 102 produced by the filter 101.
n) is monitored. As described above, the echo evaluation signal 102 is generated using the update information 104 provided to the adaptive filter 101 by the filter adaptation unit 103. The coefficient h _i of the adaptive filter 101 accumulates the update information 104 as shown in Expression 3.

[0012] To reduce the presence of acoustic echoes from the microphone signal, the resulting signal is fed to additional components for further application-specific processing. For example, in addition to acoustic echo cancellation circuitry as described above, transceivers such as those depicted in FIG. 1 typically include near-end voice activity detector 150.
including. It outputs a signal 153 indicating whether the near-end user is talking. The most commonly used way to perform near-end voice activity detection is to use power calculations in the time domain. Usually, the decision as to whether or not there is voice activity is mainly based on a comparison of a threshold energy level (corresponding to background noise) with a measurement of the signal energy filtered by a bandpass filter. The purpose of bandpass filtering is to remove the signal energy associated with background noise.

The signal indicating the presence or absence of near-end speech is useful to any of a large number of users. One is that in cellular communication systems such as the Pan-European Digital Mobile Telephone System (GSM), the digitized speech signal is not transmitted in the raw form over the network, but instead from another location. Are encoded in a manner that reduces the number of bits that need to be transmitted to the location. In GSM,
Speech coders make use of the fact that each participant on average speaks less than 40% of the time in normal conversation. By incorporating a voice activity detector as part of the function of the speech coder, the GSM system makes the discontinuous transmission mode (D
TX). In that mode, the GSM transmitter is not active during periods of silence (ie, when the near-end voice activity detector 150 indicates that the near-end user is not speaking). This approach prolongs subscriber battery life and reduces momentary radio interference. A comfortable noise subsystem at the receiving side introduces background acoustic noise to compensate for the annoying switching mute caused by DTX.

The near-end voice activity detector may also be used to control the attenuation factor of an active acoustic echo canceller based on whether the speech signal contains near-end speech components.

Furthermore, the near-end voice activity detector may also be used to control the adaptation rate of adaptive filter 101.

Voice activity detectors are not the only type of component that processes signals that represent near-end speech. Such a signal may also be provided to the voice recognition module, for example. Voice recognition modules are well known and are useful in applications that allow a user to control a device or computer via voice control, or in applications where the user can create an electronic document by simply dictating the document. .

Furthermore, a signal representative of near-end speech may also be fed back within the system and used to control the echo cancellation filter 101 itself, eg to control the rate of adaptation.

Despite the echo cancellation circuitry as described above, for further processing (eg for transmission to far-end users in a communication system, or for near-end speech recognition, or The signal generated (to control the operation of the echo cancellation filter 101) may still frequently contain echo components. This means that, for example, whenever the adaptive filter has not yet converged to a fully adapted state, or even after such convergence, the unknown system H (z) changes. It may occur because the adaptation process needs to be repeated. Strong echo signal components in the signal can cause these echo signal components to be mistaken for near-end speech, causing signal degradation or even malfunctioning downstream processing elements. Or

Conventional applications that process near-end speech signals, such as conventional voice activity detectors, voice recognition modules, etc., typically assume that the signal being processed is free of echoes, and thus human It lacks the ability to focus on near-end speech to the extent that it removes echo signal components that may be in the frequency range of voice activity.

SUMMARY Therefore, it is an object of the present invention to provide a method and apparatus for producing a signal in which the near-end speech component is enhanced relative to the echo signal component.

The foregoing and other objects are achieved in a method and apparatus for producing an improved near-end audio signal. According to one aspect of the invention, improved near-end audio signal generation includes receiving an audio signal, generating an estimated acoustic echo signal, and removing the estimated acoustic echo signal from the audio signal. Generating the processed signal is thereby included. These steps are useful, for example, in hands-free phones. At that telephone, the loudspeaker signal carrying information from the far-end user is picked up as an acoustic echo by the hands-free telephone's microphone. Next, the near-end improvement spectrum is determined. Here, the near-end improved spectrum has at least one contiguous range of frequencies and has a quantity over that range that is greater than a predetermined threshold, the contiguous range of frequencies being relatively large in the processed signal. It is related to echo return loss. The processed signal is filtered according to the near-end improved spectrum, thereby producing an improved near-end audio signal.

According to another aspect of the invention, the amount of energy contained in the improved near-end audio signal is measured. Based on the measured energy of the improved near-end speech signal, it is detected whether speech is being made at the near-end.

According to yet another aspect of the invention, the improved near-end speech signal may be applied to a near-end speech recognizer, which may result in improved speech recognition performance.

According to another aspect of the invention, the above process is repeated cyclically to allow the determination of whether speech is being made at the near end to be dynamically adjustable and adaptable to changing conditions.

In yet another aspect of the invention, determining the near-end improved spectrum includes determining the near-end improved spectrum as a function of the weighted spectrum, the weighted spectrum. Is
It is defined as follows. Here, Γ is the spectrum of the acoustic echo generated from the far-end signal, E is the echo return loss improvement spectrum representing the echo cancellation performance of the step c), and N is the processed signal. Is a spectrum, S is an echo spread spectrum that represents the spread spectrum characteristic of the path of the echo, Γ _max = max (Γ), E _max = max (E), S _max = max (S), α, β and γ are constants, and α + β + γ> 0.

[0026]   From yet another aspect of the present invention, α + β + γ = 1.

In yet another aspect of the invention, determining the near-end improved spectrum as a function of the weighted spectrum includes determining the spectrum of the detector according to the equation: Here, Speech _{min (i)} is the i-th frequency when N is larger than a predetermined threshold value, Speech _{max (i)} is the i-th frequency when N is less than the predetermined threshold value, Spectrum _totalmax is the maximum frequency of interest in the weighted spectrum W (f).

The objects and advantages of the present invention will be understood by reading the following detailed description in connection with the accompanying drawings.

DETAILED DESCRIPTION Various features of the present invention are described with reference to the drawings. Similar parts in the drawings are identified by the same reference signs.

According to one aspect of the present invention, a signal in which a near-end speech component is emphasized relative to an echo signal component is well-performed by the echo canceller and the signal energy is probably due to near-end voice activity. It is generated using information about the frequency that determines the bandwidth of the wax frequency. By calculating the power at those selected frequencies, mainly where echo cancellation is known to be effective, rather than for a wider frequency range that is commonly associated with voice activity, Greater differences between the echo component and near-end speech are obtained. Increasing this difference improves the performance of downstream components designed to handle near-end speech, such as voice activity detectors, speech recognizers, or feedback paths that control the echo cancellation behavior itself. It

The technique of which frequency to select for improvement depends on what type of echo canceller is used. For example, in the LMS type echo cancellation scheme, the echo return loss enhancement (ERLE) for each frequency depends on the spectral power of the signal. In FIG. 2, the solid line 201 shows the power spectrum of the speech signal (one sentence) before applying the echo cancellation. For comparison, the dashed line 203 illustrates the power spectrum of the same speech signal after echo cancellation has been applied. Substantial loss in performing echo cancellation is less than 250 Hz or 1500 Hz
It is observable at frequencies above. Therefore, a near-end speech processing unit (eg, voice activity detector or speech recognizer) whose analysis is limited to speech signal frequencies in the range of 250 Hz to 1500 Hz will be less likely to erroneous echo components for near-end speech. In general, the particular frequency band in which the near end speech processing unit should operate for improved performance will also depend on the type of echo canceller used with the signal spectral power.

The following are considerations that should be considered when selecting a frequency band to improve or focus when it is desired to process near-end speech to the extent that it removes far-end echo signals. Is. It has to be recognized that the true spectrum of the near-end audio signal is unknown because the microphone mixes the near-end audio signal with the far-end echo signal. Conventional techniques for detecting speech in a noisy environment include removing (eg, by filtering) frequencies where the noise is dominant. However,
In the case of the far-end echo, the frequencies associated with the far-end echo signal are themselves speech-related. That is, it is attempting to detect near-end speech in the presence of other (eg, far-end) speech. Therefore, simply removing the frequencies associated with the echo will probably also remove some of the signal associated with near-end speech, thereby defeating the purpose.

As mentioned above, since it is not possible to make a measurement of the near-end speech spectrum, a clear copy of the near-end speech signal is not available. (Indeed, the problem we are dealing with would not exist if a clear copy of the near-end speech signal was available.) However, a far-end speech signal 112 that is not contaminated by near-end speech is available, This works well. First, generally, the spectral energy contained in the echo signal corresponds to the spectral energy of the near-end speech signal (because both are speech signals). Thus, to some extent, the far-end speech signal (or the signal resulting from this signal) is used as a source for focusing on searching for near-end speech.

A measurement of the frequency at which echo cancellation is most effective can also be made. Since this information is conveniently used to improve near-end speech processing, it is unlikely that the near-end speech signal at these frequencies will be obscured by the presence of the acoustic speech component.

The number of frequency bands used in the improved spectral calculation for near-end speech is left to the designer. The maximum number of frequency bands present in the calculated frequency spectrum is half the number of signal samples for which the spectrum was calculated. However, it is not always necessary to calculate the maximum number of frequency bands. A more meaningful number may be obtained by determining fewer frequency bands from the same number of signal samples. For example, suppose the frequency spectrum is generated from 1600 samples of a signal propagated in a GSM cellular communication system. In GSM, these 1600 samples represent 200 milliseconds of speech. Therefore, the maximum expressible frequency is 4000 Hz (Nyquist frequency). These 1600 samples are divided into 10 groups, each with 160 samples. A 256-point Fast Fourier Transform (FFT) for each of the 10 groups produces 10 spectra, which are combined by a suitable weighted averaging technique. For example,
If an exponential averaging technique is used, this will cause the frequency band associated with the newly generated frequency spectrum to have a much lower weight than the previously determined average (and thus The average is slow to respond to changes in the spectrum over time). The result of this combination of spectra is a single F
As much as 10 times more information gives the spectrum at which each point (frequency band) is generated, as if the FT was performed on the original 1600 samples to generate more frequency bands. By using the technique of weighted combining, a single spectrum generated from a non-representative set of samples will have no substantial effect on overall operation.

In one embodiment of the invention, the designer first calculates one or more frequency bands in which the echo canceller is expected to work well, and then follows to operate only in those frequency bands. Adjustments will be made for near-end audio processing.

In another embodiment, the frequency band in which subsequent near-end speech processing will operate may be dynamically determined. This provides the ability to adjust near-end speech processing to changing conditions in response to dynamically changing conditions such as changes in echo canceller performance and changes in the spectral quality of the far end signal 112. is there. An exemplary embodiment of near-end speech improvement according to this aspect of the invention is described with reference to the block diagram of FIG.

A typical acoustic echo canceling configuration 301 includes an adaptive filter 101, a filter adaptation unit 103, a loudspeaker 109, a microphone 105, a D / A converter 136, an A / D converter 124, and a subtractor 107. Including, they operate the same as those depicted in FIG. Therefore, the description of these components will not be repeated here. Also shown in the exemplary transceiver is the noise suppression unit 303, although this element is optional. With this, the noise suppression unit 303 itself is dynamically adjusted based on the information generated in accordance with the invention (eg the operation of the noise suppression unit 303 depends on the signal e generated at the output of the subtractor 107). (n) is a function of detecting the presence or absence of near-end speech). Far-end signal 112 may be generated by any number of sources, depending on the particular application. For example, in a cellular telephone, the far end signal 112 is provided at the output of a speech decoder (not shown) that produces the far end signal 112 from the received signal. As an output of the acoustic echo canceling arrangement 301, a processed near-end audio signal 313 may be generated and provided to the input of a near-end audio processor (not shown). The near end voice processor functionality is application specific and will not be described in detail here. In the cellular telephone example, the near-end voice processor may be a voice activity detector (not shown), as well as a speech encoder (not shown) that produces an encoded signal for transmission to the far-end user.

According to the invention, the acoustic echo canceling arrangement 301 further comprises a near-end improved spectrum generator 309. Near-end improved spectrum generator 30
The output of 9 is fed to the control input of the near-end voice processor to improve its performance. For example, if the near-end voice processor is a voice activity detector, the voice activity detector will be the near-end improved spectrum generator 3.
A determination of voice activity may be made based on the characteristics of the particular spectral band of the processed near-end voice signal 313, as indicated by 09. That is, the output of the near-end improved spectrum generator 309 determines what type of filtering is applied to the processed near-end voice signal 313 as part of the method of voice activity detection.

Similar control adjustments are made to other types of near-end speech processing equipment, such as speech recognition equipment.

The near-end improved spectrum generator 309 is implemented in numerous ways, and
Each of them is considered within the scope of the present invention. Such forms include random access memory (RAM), magnetic storage media (eg, magnetic disk, diskette, or tape), and optical storage media (eg, compact disc read-only memory (CD-ROM)). Computer program instructions embodied as signals on a computer-readable storage medium. Alternatively, the present invention may be implemented as a programmable program that executes such instructions. The near-end enhanced spectrum generator 309 may alternatively be implemented in numerous configurations of hardwired components or programmed logic arrays.

The following terms are defined to describe the operation of the near-end enhanced spectrum generator 309.

The estimated echo spectrum (Γ) is the spectrum of the estimated echo signal y (n) provided by the adaptive filter 101 (ie subtracted from the digitized microphone signal d (n). Signal). The estimated echo spectrum Γ may be generated from the microphone signal d (n) digitized by FFT, for example, and is therefore a function of frequency f. The estimated echo spectrum Γ should normally represent the locally stationary spectrum of the far-end spectrum echo. In applications such as GSM cellular phones, this should be a 20 millisecond speech spectrum. If we recognize in this case that the spectrum does not modify the contents of the spectrum faster than 20 ms, the number of samples used to calculate the estimated echo spectrum Γ is It is preferably the same as the number of samples used by the vessel. If a combining technique (eg, weighted averaging) is applied to some measurements of the evaluated echo spectrum Γ, the weights are such that the newly calculated evaluated echo spectrum Γ quickly affects the combination. It should be In some preferred embodiments, no averaging is applied on the estimated echo spectrum Γ. Note that the estimated echo spectrum Γ is used to show the frequencies associated with the relatively high echo return loss.

The echo return loss improvement (ERLE) spectrum (E) is a spectrum expressing the echo canceling performance of the echo canceling filter. E
RLE spectrum E is a function of frequency f. Several alternative measurements of ERLE spectrum E may be used. In some embodiments, the ERLE spectrum may be determined according to the formula: here, Represents a Fourier transform, d (n) is a digitized microphone signal containing echo and noise components along with near-end speech, and e '(n) is the processed near-end speech signal 313.

In another embodiment, different ERLE spectra may be determined by first making a time domain measurement according to the following equation: From this, a spectrum in the frequency domain may be generated according to the following equation: Any measurement of the ERLE spectrum E may be used to indicate the frequency associated with the relatively high echo return loss. Also, in any of these embodiments, the ERLE spectrum E may be determined separately for each of the groups of samples and the resulting spectrum combined as described above (eg, by weighted averaging). The speed of averaging (ie, the speed that significantly affects the averaging in the newly calculated spectrum) is
Is approximately the same as the adaptation speed of
The LE spectrum E will accurately reflect the performance of echo cancellation.

Near-end spectrum (N) is the spectrum of the signal received after echo cancellation and optional noise suppression (ie, it is the spectrum of the processed near-end speech signal 313). Near-end spectrum N is a function of frequency f, which is the processed near-end speech signal 31.
It may be calculated as an FFT of 3 (e ′ (n)). It is preferably calculated using the same number of samples used to calculate the estimated echo spectrum Γ.

The echo spread spectrum (S) expresses the spread spectrum characteristic of the echo path. That is, it is a measure of how much different frequencies are transmitted between loudspeaker 109 and microphone 105. The echo spread spectrum S is a function of the frequency f and may be calculated as a Fourier transform of the coefficient h (n) that determines the characteristics of the filtering performed by the adaptive filter 101. That is, it is the following formula. As in the earlier described embodiments, the ERLE spectrum (E) is used to determine the frequency band in which near-end speech processing should operate (hereinafter referred to as the "detector spectrum"). Performance is improved.
According to another aspect of the invention, the advantage resulting from the use of the spectrum E is that the detector spectrum is determined as follows without degrading the performance when the evaluated echo spectrum (Γ) does not correspond to E: Achieved by

In the flowchart of FIG. 4, various spectra Γ, E, S, and N are first determined as described above (step 401).

Next, in step 403, the weighted spectrum W (f) is determined from the evaluated echo spectrum Γ, ERLE spectrum E, and echo spread spectrum S according to the following equation. Here, Γ _max = max (Γ), E _max = max (E), S _max = max (S), and α, β, and γ are constants.

The purpose of dividing each of the spectra Γ, E, and S by their respective maximum value is the normalized spectrum that is combined after scaling by the corresponding one of the weighting factors α, β, and γ. It will soon be obvious that

In a preferred embodiment, α + β + γ is close to one value (eg it may range from a fractional value close to but not equal to zero to a value of about 2).
, But this is not a strict requirement.

Next, in step 405, the compression factor C is determined. It describes the extent to which the weighted spectrum W (f) contains the power in which the near-end spectrum N lies within one or more frequency bands with its maximum energy component. In FIG. 5, as illustrated by the first band between Speech _{min (1)} and Speech _{max (1)} and the _second band between Speech _{min (2)} and Speech _{max (2)} . At least one reference is made to one or more frequency bands, since the near-end spectrum N has several discontinuous frequency bands and may have values over its range above a predetermined threshold level. The compression factor C is given by the following equation. Where Speech _{min (i)} is the i-th frequency where N is greater than a predetermined threshold specific to the application, and is therefore set by the designer, and Speech _{max (i)} is N equal to that predetermined threshold. Is the i-th frequency below the threshold,
Spectrum _{total max} is the maximum frequency we _note in the weighted spectrum W (f). That is, it may be assumed that the value of the function W (f) is equal to zero for all frequencies above Spectrum _totalmax .

Again, the compression factor C is defined as the ratio of two integrals, but in practice it is often simply done by approximating the corresponding spectrum as being substantially flat over various ranges of frequencies. May be calculated. This is further explained in some examples presented below.

Having determined the compression factor C and the weighted spectrum W (f), the detector spectrum is obtained in step 407 by calculating the following equation: It will be appreciated that the resulting near-end improved spectrum is a function of frequency f.

The near-end enhancement spectrum may then be provided to the control input of a near-end voice processor (not shown). For example, the near-end improved spectrum is used to determine the bandpass filtering performed by the near-end voice activity detector in a cellular phone.

For dynamically adjustable operation, these steps are repeated cyclically, starting again at step 401, as shown in FIG. For example, in a system in which a frame of 160 samples is generated once every 20 ms, the new near-end improved spectrum may also be determined once every 20 ms.

Several examples are presented to illustrate the techniques described above. In each case, all the described spectra are normalized except for the near-end spectrum N. (The reason for not normalizing N is the processed near-end speech signal 31
3 to hold information about the actual energy level. Moreover, in the following examples, the spread spectrum is often considered to be evenly distributed. Further, to facilitate understanding of the invention, N is shown to have only one region where the power is above a given threshold level. This avoids summing the separately calculated integrals.

A first example will be described with reference to FIGS. 6A to 6E. FIG. 6A is a graph of near-end speech spectrum N. N = 0.25 between f = 0 and f = 250 Hz
And f = 250 Hz to f = 750 Hz, N = 1.0, and f = 750.
Between Hz and f = 1500 Hz, N = 0.25 (note that the maximum value 1.0 drawn is for illustrative purposes only, and in general, N is not normalized. ).

Continuing with the example, FIG. 6B is a graph of the normalized ERLE spectrum E. E = 1.0 between f = 0 to f = 750 Hz, and f = 750 Hz to f
Between 1500 Hz, E = 0.25.

A graph of the normalized estimated echo spectrum Γ is depicted in FIG. 6C. Γ = 1.0 between f = 0 to f = 750 Hz, and f = 750 Hz to f = 1.
At 500 Hz, Γ = 0.25.

In this example, the weighted spectrum is given by: (Because in this example, the weighting factor γ = 0, so that what appears to be similar in echo spread spectrum S is irrelevant.) Normalized evaluated echo spectrum Γ (see FIG. 6C). (As depicted) and the normalized ERLE spectrum E (as depicted in FIG. 6B),
For this example, the result is a weighted spectrum W (f),
It is depicted in Figure 6D.

Next, the compression factor C is calculated. Given that the predetermined threshold is 0.25, it can be seen from Figure 6A that there is only one frequency band above this threshold. This frequency band is bounded by Speech _min = _250Hz ; Speech _max = _750Hz ; Spectrum _totalmax = _1500Hz .

Therefore, according to equation (7), Since the weighted spectrum W (f) is constant for each of several ranges, its integral and hence C is relatively easy to calculate.

The near-end improved spectrum can now be calculated according to equation (8). The leftmost spectrum in FIG. 6E depicts the resulting near-end improved spectrum for this example. It can be seen that it has a magnitude of 1.0 between f = 0 and f = 750 Hz and a value of 0.600 ... Between f = 750 Hz and f = 1500 Hz.

FIG. 6E further depicts applying this near-end improved spectrum to controlling a near-end voice processor, such as a voice activity detector. Such voice activity detectors have bandpass filtering capabilities tuned to comply with the near-end improved spectrum. As a result, when the processed near-end voice signal 313 is applied to the voice activity detector (see middle spectrum in FIG. 6E), the resulting voice activity detector spectrum is shown on the right side of FIG. 6E. Looks like something like The resulting detector spectrum is f = 0-f = 250
It is equal to 0.25 between Hz and 1.0 between f = 250 Hz and f = 750 Hz.
And is equal to 0.15 between f = 750 Hz and f = 1500 Hz. As a result, those frequency bands (ie, between f = 0 Hz and f = 750 Hz, as shown in FIG.
For a typical weighted spectrum in), there is no change in behavior, where the behavior of echo cancellation is good.
However, at frequencies associated with poor echo canceling performance, there is only a small impact on near-end detector performance. As a result, the performance of the near-end detector is improved.

A second example will be described with reference to FIGS. 7A to 7E. FIG. 7A is a graph of near-end speech spectrum N. N = 0.25 between f = 0 and f = 250 Hz
And f = 250 Hz to f = 750 Hz, N = 1.0, and f = 750.
Between Hz and f = 1500 Hz, N = 0.25 (note that the maximum value 1.0 drawn is for illustrative purposes only, and in general, N is not normalized. ).

Continuing with the example, FIG. 7B is a graph of the normalized ERLE spectrum E. E = 1.0 between f = 0 to f = 750 Hz, and f = 750 Hz to f
Between 1500 Hz, E = 0.25.

So far, the example described above with respect to FIGS. 6A-6E is followed. However,
Here, a graph of different normalized evaluated echo spectra Γ is shown in FIG. 7C.
Is depicted in. Between f = 0 and f = 750 Hz, Γ = 0.25, and f = 7
Γ = 1.0 between 50 Hz and f = 1500 Hz.

In this example, again assume that the weighted spectrum is given by: (Because, in this example, the weighting factor γ = 0, so that what appears to be similar in echo spread spectrum S is irrelevant.) Normalized evaluated echo spectrum Γ (see FIG. 7C). (As depicted) and the normalized ERLE spectrum E (as depicted in FIG. 7B),
For this example, the resulting weighted spectrum W (f) is obtained, which is depicted in Figure 7D. Note that it is a constant (= 0.625) over the entire range from f = 0 to f = 1500.

Next, the compression factor C is calculated. It can be seen from FIG. 7A that Speech _min = 250 Hz; Speech _max = 750 Hz; Spectrum _{total max} = 1500 Hz.

Therefore, according to equation (7), Since the weighted spectrum W (f) is constant over the entire range between f = 0 and f = 1500 Hz, its integral and hence C is relatively easy to calculate again.

The near-end improved spectrum can now be calculated according to equation (8). The leftmost spectrum in FIG. 7E depicts the resulting near-end improved spectrum for this example. It can be seen that it has a magnitude of 0.875 over the entire range between f = 0 and f = 1500 Hz.

FIG. 7E further depicts applying this near-end improved spectrum to controlling a near-end voice processor, such as a voice activity detector. Such voice activity detectors have bandpass filtering capabilities tuned to comply with the near-end improved spectrum. As a result, when the processed near-end voice signal 313 is applied to the voice activity detector (see middle spectrum of FIG. 7E), the resulting voice activity detector spectrum is shown on the right side of FIG. 7E. Looks like something like The resulting detector spectrum is f = 0-f = 250
In the range of Hz, it is equal to 0.21875, in the range of f = 250Hz to f = 750Hz, it is equal to 0.875, and in the range of f = 750Hz to f = 1500Hz, it is 0.
Equal to 21875. It can be seen that the entire detector spectrum is attenuated when the ERLE spectrum E and the evaluated echo spectrum Γ are uncorrelated or small. Nevertheless, the near-end detector still responds best to frequencies where the near-end spectrum N has its largest component.

The invention has been described with reference to particular embodiments. However, it will be readily apparent to those skilled in the art that the present invention may be embodied in specific forms other than the preferred embodiments described above. This is done without departing from the spirit of the invention.

For example, the spectrum shown is idealized to facilitate discussion of the present invention. However, in practice, any or all of these spectra may not fit the representative step function depicted in FIGS. 6A-6E and 7A-7E. Rather, some or all of these spectra may be described by more complex mathematical functions. Despite that difference, the resulting detector spectrum is characterized by a continuous range of frequencies, the detector spectrum has a maximum over that frequency range, and the continuous range of frequencies is the processed signal. Is expected to be associated with the relatively high echo return loss at.

Therefore, the preferred embodiments are merely illustrative and should not be considered limiting in any way. The scope of the invention should be given by the appended claims, rather than by the foregoing description, and all variations and equivalents that fall within the scope of the claims are to be included within the scope of the claims. Is intended.

[Brief description of drawings]

FIG. 1 is a block diagram of a conventional hands-free transceiver including an acoustic echo canceller and a near-end voice activity detector.

FIG. 2 is a power spectrum (1) of a voice signal before and after applying echo cancellation.
It is a comparison diagram of a sentence).

[Figure 3]   FIG. 3 is a block diagram of an exemplary embodiment of the present invention.

[Figure 4]   3 is a flowchart depicting steps performed in accordance with the present invention.

FIG. 5 is a representative near-end spectrum N illustrating the case of several discontinuous frequency bands whose amplitude exceeds a predetermined threshold level.

FIG. 6A   3 is a graph of an exemplary normalized near-end speech spectrum N.

FIG. 6B   6 is a graph of a representative normalized ERLE spectrum E.

FIG. 6C   3 is a graph of an exemplary normalized loudspeaker spectrum Γ.

FIG. 6D   3 is a graph of an exemplary weighted spectrum according to one aspect of the present invention.

FIG. 6E   6 is a graph depicting the determination of an exemplary compression factor C according to one aspect of the present invention.

FIG. 7A is a graph of another exemplary normalized near-end speech spectrum N.

FIG. 7B   6 is another representative normalized ERLE spectrum E graph.

FIG. 7C is a graph of another exemplary normalized loudspeaker spectrum Γ.

FIG. 7D is a graph of another exemplary weighted spectrum in accordance with one aspect of the present invention.

FIG. 7E is a graph depicting another exemplary compression factor C determination according to one aspect of the invention.

[Procedure for Amendment] Submission for translation of Article 34 Amendment of Patent Cooperation Treaty

[Submission date] July 26, 2001 (2001.26)

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be amended] Claims

[Correction method] Change

[Contents of correction]

[Claims]

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0019

[Correction method] Change

[Contents of correction]

Conventional applications that process near-end speech signals, such as conventional voice activity detectors, voice recognition modules, etc., typically assume that the signal being processed is free of echoes, and thus human It lacks the ability to focus on near-end speech to the extent that it removes echo signal components that may be in the frequency range of voice activity. European Patent Application No. 0854626 discloses an echo cancellation technique that involves comparing a received raw near-end signal with an estimated echo signal in the frequency domain. The information from this comparison is then provided to a filter that produces an improved near-end audio signal based on the information from that comparison.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04M 1/60 G10L 3/02 301C 9/08 (81) Designated country EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, IT, LU, MC, NL, PT, SE), OA (BF, BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, GM, K E, LS, MW, SD, SL, SZ, TZ, UG, ZW), EA (AM, AZ, BY, KG, KZ) , MD, RU, TJ, TM), AE, AG, AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, CA, CH, CN, CR, CU, C , DE, DK, DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, JP, KE, KG, KP, KR, KZ, LC, LK, LR, LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK , SL, TJ, TM, TR, TT, TZ, UA, UG, UZ, VN, YU, ZA, ZW F term (reference) 5D015 EE04 KK00 5D020 CC05 5K027 DD07 DD10 5K038 AA07 CC01 FF13 5K046 HH01 HH18 HH24 HH78 H79 [H] Continuation of Summary: Provided dynamically adjustable behavior.

Claims (14)

[Claims] 1. A method for producing an improved near-end speech signal, comprising the steps of: a) receiving an audio signal, b) producing an estimated acoustic echo signal, and c) from the audio signal. Generating a processed signal by removing the evaluated acoustic echo signal, d) having a continuous frequency range, the continuous frequency range having a relatively large echo return in the processed signal. Determining a near-end improved spectrum that is associated with loss and has an amount greater than a predetermined threshold over the range; and e) filtering the processed signal according to the near-end improved spectrum, Thereby producing the improved near-end audio signal. 2. f) measuring how much energy is included in the improved near-end audio signal; and g) based on the measured energy of the improved near-end audio signal.
The method of claim 1, further comprising the step of detecting whether near-end audio is being emitted. 3. The method of claim 1, further comprising: f) recognizing near-end speech included in the improved near-end speech signal. 4. The method according to claim 1, wherein the steps a) to e) are repeated periodically. 5. The step of determining the near-end improved spectrum comprises:
Determining the near-end improved spectrum as a function of a weighted spectrum, the weighted spectrum comprising: , Γ is the spectrum of the acoustic echo generated from the far-end signal, E is the echo return loss improvement spectrum representing the echo canceling performance of the step c), and N is the process S is a spectrum of the generated signal, S is an echo spread spectrum representing a spread spectrum characteristic of the path of the echo, Γ _max = max (Γ), E _max = max (E), S _max = max (S) The method of claim 1, wherein α, β, and γ are constants and α + β + γ> 0. 6. The method according to claim 5, wherein α + β + γ = 1. 7. The step of determining the near-end improved spectrum as a function of the weighted spectrum comprises: Determining the improved spectrum of the near end according to the following: Speech _{min (i)} is the i-th frequency when N is greater than a predetermined threshold, and Speech _{max (i)} is N where N is the predetermined threshold. 6. The method of claim 5, which is the i-th frequency below a threshold and Spectrum _totalmax is the maximum frequency of interest in the weighted spectrum W (f). 8. An improved near-end audio signal generator comprising: a) means for receiving an audio signal; b) means for generating an estimated acoustic echo signal; and c) the estimated signal from the audio signal. Means for producing a processed signal by removing the acoustic echo signal, d) having a continuous frequency range, the continuous frequency range resulting in a relatively large echo return loss in the processed signal. Means for determining a near-end improved spectrum having a quantity greater than a predetermined threshold over the range, and e) filtering the processed signal according to the near-end improved spectrum, thereby Improved near-end audio signal and a filter for generating the improved near-end audio signal. Voice signal generator. 9. f) means for measuring how much energy is included in the improved near-end audio signal; and g) based on the measured energy of the improved near-end audio signal.
9. An improved near-end audio signal generator according to claim 8 further comprising means for detecting whether near-end audio is being emitted. 10. The improved near-end speech signal generator of claim 8, further comprising: f) a speech recognizer coupled to receive the improved near-end speech signal. 11. An improved near-end audio signal generator according to claim 8, wherein the components a) to e) operate cyclically and repeatedly. 12. The means for determining the near-end improved spectrum includes means for determining the near-end improved spectrum as a function of the weighted spectrum, the weighted spectrum comprising: , Γ is the spectrum of the acoustic echo produced from the far end signal, E is the echo return loss improvement spectrum representing the echo canceling performance of the means for producing the processed signal, N Is the spectrum of the processed signal, S is the echo spread spectrum representing the spread spectrum characteristic of the path of the echo, Γ _max = max (Γ), E _max = max (E), S _max = 9. The improved near-end audio signal generator of claim 8 wherein max (S), α, β, and γ are constants and α + β + γ> 0. 13. The improved near-end audio signal generator of claim 12, wherein α + β + γ = 1. 14. Means for determining the near-end improved spectrum as a function of the weighted spectrum, According to the above, said speech end _min is determined by said means: Speech _{min (i)} is the i-th frequency when N is greater than a predetermined threshold, and Speech _{max (i)} is N said predetermined frequency. Improved near-end speech signal generation according to claim 12, characterized in that it is the i-th frequency below a threshold and the Spectrum _totalmax is the maximum frequency of interest in the weighted spectrum W (f). vessel.