EP2014132A2 - Echo and noise cancellation - Google Patents
Echo and noise cancellationInfo
- Publication number
- EP2014132A2 EP2014132A2 EP07759872A EP07759872A EP2014132A2 EP 2014132 A2 EP2014132 A2 EP 2014132A2 EP 07759872 A EP07759872 A EP 07759872A EP 07759872 A EP07759872 A EP 07759872A EP 2014132 A2 EP2014132 A2 EP 2014132A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- echo
- signal
- output
- microphone
- adaptive filter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
- 230000003044 adaptive effect Effects 0.000 claims abstract description 77
- 238000000034 method Methods 0.000 claims abstract description 44
- 230000000295 complement effect Effects 0.000 claims abstract description 15
- 230000001419 dependent effect Effects 0.000 claims abstract description 6
- 238000001914 filtration Methods 0.000 claims description 19
- 238000010219 correlation analysis Methods 0.000 claims description 15
- 238000012545 processing Methods 0.000 claims description 9
- 230000000694 effects Effects 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims description 6
- 238000001514 detection method Methods 0.000 description 12
- 230000008901 benefit Effects 0.000 description 10
- 230000006978 adaptation Effects 0.000 description 9
- 230000006870 function Effects 0.000 description 8
- 238000013459 approach Methods 0.000 description 7
- 238000010586 diagram Methods 0.000 description 7
- 230000007246 mechanism Effects 0.000 description 5
- 230000002093 peripheral effect Effects 0.000 description 4
- 230000010354 integration Effects 0.000 description 3
- 230000002452 interceptive effect Effects 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000001629 suppression Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000003247 decreasing effect Effects 0.000 description 2
- 238000002592 echocardiography Methods 0.000 description 2
- 238000010606 normalization Methods 0.000 description 2
- 238000010183 spectrum analysis Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000001276 controlling effect Effects 0.000 description 1
- 238000012937 correction Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000009290 primary effect Effects 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M9/00—Arrangements for interconnection not involving centralised switching
- H04M9/08—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic
- H04M9/082—Two-way loud-speaking telephone systems with means for conditioning the signal, e.g. for suppressing echoes for one or both directions of traffic using echo cancellers
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
- G10L2021/02163—Only one microphone
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/18—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being spectral information of each sub-band
Definitions
- Embodiments of the invention are related to audio signal processing and more particularly to echo and noise cancellation in audio signal processing.
- Such a device typically includes a microphone that receives a local speech signal s(t) from a user of the device and a speaker that emits a speaker signal x(t) that is audible to the user.
- a microphone that receives a local speech signal s(t) from a user of the device
- a speaker that emits a speaker signal x(t) that is audible to the user.
- To make the video game controller more compact it is often desirable to place the microphone and speaker relatively close to each other, e.g., within about 10 centimeters of each other.
- the user by contrast may be much further from the microphone, e.g., about 3 to 5 meters away.
- the microphone produces a signal d(t) that includes both the local speech signal s(t) and a speaker echo signal xi(t).
- Speaker echo is a commonly observed phenomenon in telecommunications applications and echo suppression and echo cancellation are relatively mature technologies. Echo suppressors work by detecting if there is a voice signal going in one direction on a circuit, and then inserting a great deal of loss in the other direction. Usually the echo suppressor at the far-end of the circuit adds this loss when it detects voice coming from the near-end of the circuit. This added loss prevents the speaker signal x(t) from being retransmitted in the local speech signal d(t).
- echo suppression While effective, echo suppression often leads to several problems. For example it is common for the local speech signal s(t) and the remote speaker signal x(t) to occur at the same time, at least briefly. This situation is sometimes referred to as double-talk. The situation where only the remote speaker signal is present is sometimes referred to as remote single talk. Because each echo suppressor will then detect voice energy coming from the far-end of the circuit, the effect would ordinarily be for loss to be inserted in both directions at once, effectively blocking both parties. To prevent this, echo suppressors can be set to detect voice activity from the near-end speaker and to fail to insert loss (or insert a smaller loss) when both the near-end speaker and far-end speaker are talking. Unfortunately, this temporarily defeats the primary effect of having an echo suppressor at all.
- the echo suppressor is alternately inserting and removing loss, there is frequently a small delay when a new speaker begins talking that causes clipping of the first syllable from that speaker's speech. Furthermore, if the far-end party on a call is in a noisy environment, the near-end speaker will hear that background noise while the far-end speaker is talking, but the echo suppressor will suppress this background noise when the near-end speaker starts talking. The sudden absence of the background noise gives the near-end user the impression that the line has gone dead.
- Echo cancellation may use some form of analog or digital filter to remove unwanted noise or echoes from an input signal and produce a filtered signal e(t).
- complex algorithmic procedures are used to compute speech models. This involves feeding the microphone signal d(t) and some of the remote signal x(t) to an echo cancellation processor, predicting the speaker echo signal xi(t) then subtracting this from microphone signal d(t).
- the format of the echo prediction must be learned by the echo cancellation processor in a process known as adaptation.
- ESR echo suppression ratio
- ITU International Telecommunication Union
- embodiments of the invention are directed to methods and apparatus for echo cancellation in a system having a speaker and a microphone.
- the speaker receives a speaker signal x(t).
- the microphone receives a microphone signal d(t) containing a local signal s(t) and an echo signal X 1 (I;) that is dependent on the speaker signal x(t).
- the microphone signal d(t) is filtered in parallel with first and second adaptive filters having complementary echo cancellation properties relative to each other.
- a minimum echo output e 3 (t) is determined from an output ei(t) of the first adaptive filter and an output e 2 (t) of the second adaptive filter.
- the minimum echo output has a smaller energy and less correlation to the speaker signal x(t).
- a microphone output is then generated using the minimum echo output e 3 (t). Residual echo cancellation and/or noise cancellation may optionally be applied to the minimum echo output.
- FIG. IA is a schematic diagram of an echo cancellation apparatus according to an embodiment of the present invention.
- FIG. IB is a schematic diagram of an adaptive filter with voice activity detection that may be used in the echo cancellation apparatus of FIG. IA.
- FIG. 1C is a schematic diagram of an adaptive filter with cross-correlation analysis that may be used in the echo cancellation apparatus of FIG. IA.
- FIG. 2A is a flow diagram illustrating a method for echo cancellation according to an embodiment of the present invention.
- FIG. 2B is a flow diagram illustrating an alternative method for echo cancellation according to an embodiment of the present invention.
- FIG. 3 is a schematic diagram of an echo cancellation apparatus according to an alternative embodiment of the present invention.
- Embodiments of the invention propose a novel structure of an integrated echo and noise- canceller that has two functionally identical filters with orthogonal controls and representations.
- the two orthogonal filters complement each other in a way that can boost the overall system robustness in a noisy hands free voice communication.
- an integrated echo-noise-canceller uses two separately controlled sub-systems in parallel, each with orthogonal control mechanism.
- the echo-noise-canceller includes a front echo canceller and a backup echo canceller.
- the front echo-canceller uses double-talk detection and takes conservative adaptation approaches to ensure it is robust against local voice, yet provide less echo-suppression and slow adaptation speech to echo-change.
- the backup echo-canceller uses cross-correlation to measure the similarity between the error signal and echo signal, it takes an aggressive strategy to ensure the filter is quickly updated and provides large echo-suppression, yet it is unstable to local voice/noise, due to possible over adaptation.
- An integration of these two echo-canceller outputs is based on cross- correlation analysis that measures which echo canceller has a larger differentiation against the echo signal. The integration also checks the filter stability of both echo cancellers. If one filter is overestimated or underestimated it is then complemented by the other filter. Such a system is designed to ensure there is at one filter working correctly at any given time.
- the system may optionally include an echo-residual-noise-estimator that takes a similar approach using two independent sub estimators in parallel, each with orthogonal controlling.
- a first estimator is based on echo-distance-mismatch that depends on a robust double-talk- detector. The first estimator is relatively accurate yet not stable due to double-talk-detection errors.
- a second estimator is based on cross-spectrum-analysis. Its estimation is biased but stable and consistent without depending on local voice detection. The integration of these two estimations of the residual echo takes a minimum/maximum approach for far-end -talk only and double-talk respectively.
- FIG. IA shows an example of an audio system 99 utilizing an echo cancellation apparatus 100 according to an embodiment of the present invention. Operation of the apparatus 100 may be understood by referring to the flow diagrams of method 200 shown in FIG. 2A and method 220 shown in FIG. 2B.
- the audio system 99 generally includes a speaker 102 that receives a remote signal x(t) and a microphone 104.
- a local source 101 produces local speech signal s(t).
- the microphone 104 receives both the local speech signal s(t) and an echo signal X 1 (I;) that is related to the speaker signal x(t).
- the microphone 104 may also receive noise n(t) originating from the environment in which the microphone 104 is located.
- the echo cancellation apparatus 100 generally includes a first adaptive echo cancellation filter EC(I) and a second adaptive echo cancellation filter EC(2). Each adaptive filter receives the microphone signal d(t) and the speaker signal x(t). As shown in FIGs. 2A-2B, filter EC(I) adaptively filters the microphone signal d(t) as indicated at 202 while the second filter EC(2) adaptively filters the microphone signal d(t) in parallel with the first filter EC(I) as indicated at 204. As used herein, the filters are said to operate in parallel if they receive the substantially the same input d(t). Parallel operation is distinguished from serial operation where an output of one filter serves as an input for the other.
- one filter serves as a main or "front” filter and the other serves as a "backup” filter.
- One filter takes a conservative approach to echo cancellation while the other takes a more aggressive approach.
- the status of the filters EC(I), EC(2) may be understood with respect to the following signal model:
- y(t) is the echo synthesized by the echo-canceller filter.
- x(t) is the echo that is playing in loudspeaker
- h(n) is the adaptive filter function of echo-canceller filter
- d(t) is the noisy signal received by microphone
- s(t) is local voice
- e(t) is echo-cancelled residual signal produced by the echo canceller filter.
- the two filters EC(I), EC(2) have complementary echo cancellation properties.
- two adaptive filters receiving the same inputs are said to have complementary echo cancellation if the filtering properties of the two filters are such that one filter is well adapted to its inputs when the other is not.
- a filter is said to be "well adapted" when its filter function h(n) is stable, converged to the true echo-path- filter and neither overestimated nor underestimated.
- ⁇ E(e(t) * y(t)) / E(y(t) * y(t)
- the numerator stands for cross-correlation of e(t) and y(t)
- the denominator stands for auto-correlation of y(t) which serves as a normalization term.
- ⁇ should be close to "0"(since the residual signal e(t) does not include y(t) ). If h(n) is not converged, ⁇ should be close to "1" (since e(t) contains strong echo of y(t). if h(n) is illness and diverged, ⁇ should be negative (since e(t) contains 180 phase shifted strong echo due to filter divergence).
- the value of the coherence function ⁇ may be used to define four different possible states for the status of the filters EC(I), EC(2):
- a filter is under-estimated or over-estimated, its adaptation step-size may be increased or decreased respectively by small delta amount to accelerate or decelerate the adaptation- speed for faster convergence or better tracking stability.
- faster convergence takes a larger step-size, which sacrifices good tracking to detail and lowers the echo-suppression- ratio ESR.
- Slower convergence by using a smaller step-size is more stable and has better capability to track subtle changes but is less suitable for fast tracking of echo dislocation.
- the other filter's settings may be copied to reinitiate the diverged filter if the other filter is in good status. Otherwise the diverged filter may be recovered using filter settings from a previously saved good status.
- echo canceling adaptive filters EC(I) and EC(2) may be based on frequency domain normalized least mean square adaptive filters.
- Each filter may be implemented in hardware or software or a combination of hardware and software.
- FIGs. IB and 1C depict examples of suitable complementary adaptive filters.
- FIG. IB depicts an adaptive echo cancellation filter 120 with voice activity detection that may be used as the first adaptive filter EC(I).
- the filter 120 includes a variable filter 122 having finite impulse response (FIR) filters characterized by filter coefficients w t .
- the variable filter 122 receives the microphone signal d(t) and filters it according to the values of the filter coefficients w t to produce a filtered signal d'(t).
- the variable filter 124 estimates a desired signal by convoluting the input signal with the impulse response determined by the coefficients w t i.
- Each filter coefficient w t i is updated at regular intervals by an amount ⁇ w t according to an update algorithm 124.
- the filter coefficients w t may be chosen such that the filtered signal d'(t) attempts to estimate the speaker echo signal X 1 (I;) as the desired signal.
- a difference unit 126 subtracts the filtered signal d'(t) from the microphone signal d(t) to provide an estimated signal ei(t), which estimates the local speech signal s(t).
- the filtered signal d'(t) can be subtracted from the remote signal x(t) to produce an error signal e(t) that is used by the update algorithm 124 to adjust the filter coefficients w t .
- the adaptive algorithm 124 generates a correction factor based on the remote signal x(t) and the error signal.
- coefficient update algorithms include least mean squares (LMS) and recursive least squares (RLS).
- the step size ⁇ may be dynamically adjusted depending on the state of the adaptive filter. Specifically, if the filter is under-estimated the step size ⁇ may be increased by small delta amount to accelerate the adaptation-speed for faster convergence. Alternatively, if the filter is overestimated, its adaptation step-size ⁇ may be decreased respectively by small delta amount to decelerate the adaptation-speed for better tracking stability.
- the time-domain expression e(t)x(t) is a multiplication. This computation may be implemented in the frequency domain as follows. First e(t), x(t) and h(n) may be converted from the time domain to the frequency domain, e.g., by fast Fourier transform (fft).
- fft fast Fourier transform
- H(k) H(k) + ( ⁇ * conj(X(k)) .* E(k)) / ( ⁇ + XQa)* conj(X(k)).
- ⁇ is the filter adaptation stepsize, it is dynamic
- conj (a) refers the complex conjugate of a complex number a
- * refers to complex multiplication
- ⁇ is a regulator that prevents the denominator from becoming numerically unstable.
- Voice activated detection VAD adjusts the update algorithm 124 such that variable filter 122 only adaptively filters the microphone signal d(t) when the remote signal x(t) is present, i.e., above a predetermined threshold.
- An adaptive filter of the type shown in FIG. IB that uses voice activated detection (sometimes also referred to as double talk detection), is a relatively slowly adapting filter but it is also very accurate in that it generates very few false positives.
- a complementary adaptive filter to the filter 120 would be one that, e.g., adapts relatively quickly, but tends to generate frequent false positives.
- FIG. 1C depicts an adaptive filter 130 that is complementary to the filter 120 of FIG. IB.
- the adaptive filter 130 includes a variable filter 132 characterized by filter coefficients w t2 and an update algorithm 134 (e.g., a LMS update algorithm as described above).
- the filter 132 attempts to estimate the speaker echo signal xi(t) as the desired signal.
- a difference unit 136 subtracts the filtered signal d'(t) from the microphone signal d(t) to provide an estimated signal e 2 (t), which estimates the local speech signal s(t).
- the filtered signal d'(t) can be subtracted from the remote signal x(t) to produce an error signal e(t) that is used by the update algorithm 134 to adjust the filter coefficients w t2 .
- a cross- correlation analysis CCA adjusts the update algorithm 134 such that variable filter 132 attempts to reduce cross-correlation between the estimated signal e 2 (t) and the speaker echo signal x(t).
- the filtering is said to be underestimated and the update algorithm 134 may be adjusted to increase ⁇ w t2 . If the cross-correlation between e 2 (t) and x(t) is below a threshold the filtering is said to be overestimated and the update algorithm 134 may be adjusted to reduce ⁇ w t2 .
- An adaptive filter of the type shown in FIG. 1C that uses cross-correlation analysis (sometimes also referred to as cross-spectrum analysis), is a relatively rapidly adapting filter but it is also unstable in that generates frequent false positives.
- filters 120 and 130 are examples of complementary filters.
- an integrator 106 is coupled to the first adaptive filter EC(I) and the second adaptive filter EC(2).
- the integrator 106 is configured to determine a minimum echo output e3(t) from the outputs ei(t), e ⁇ t) of the first and second adaptive filters respectively.
- the minimum echo output e3(t) is the one of ei(t) and e 2 (t) having the smaller energy and less correlation to the speaker signal x(t).
- the minimum energy may be determined by determining a minimum of .ZsJe 1 (I;)) and is ⁇ e 2 (t) ⁇ , where E ⁇ refers to the operation of determining the expectation value of the quantity in braces.
- a cross correlation analysis may be performed on ei(t) and e 2 (t) to determine which of ei(t) and e 2 (t) has the smaller cross-correlation with the speaker signal x(t).
- the cross-correlation analysis may include determining a minimum of .ZsJe 1 (I;) * x(t) ⁇ and is ⁇ e 2 (t) * x(t) ⁇ , where the " ⁇ * • " operator refers to the operation of taking the cross-correlation between the quantities on either side of the operator, e.g. as defined above.
- the minimum echo output e3(t) may be used as a filtered output of the microphone 104.
- one of the filters EC(I), EC(2) may excessively filter out the local signal s(t).
- the filter is said to be "diverged".
- EC(2) is cross-correlation filter, e.g., of the type shown in FIG. 1C.
- the integrator 106 may be configured to determine whether the second adaptive echo cancellation filter EC(2) is excessively filtering out the local signal s(t).
- the EC(2) filter coefficients Wt 2 may be set to the values of the EC(I) filter coefficients w t i at 212.
- EC(2) may then be re- initialized at 215 to zero or to a previous known, well-adapted state.
- the filter coefficients may be saved at regular intervals, e.g., about every 10 to 20 seconds so that they may be used to re-initialize EC(2) if it diverges. Normally, a cross-correlation filter is well-adapted if it is not diverged. Since EC(2) and EC(I) have complementary filtering properties, when EC(2) is well-adapted EC(I) is underestimated.
- the filter coefficients w t i for the first adaptive filter EC(I) may be swapped with the filter coefficients Wt 2 for the second adaptive filter EC(2) as indicated at 214
- the coefficients Wti, Wt 2 may be stored at locations in a memory that are identified by pointers.
- the coefficients w t i, w t2 may be swapped, e.g., by switching the pointers to w t i and Wt 2 .
- the minimum echo output e 3 (t) may include some residual echo xe(t) from the speaker signal x(t).
- the apparatus 100 may optionally include echo first and second residual estimators ER(I) and ER(2) coupled to the integrator 106 and a residual echo cancellation module 108 coupled to the echo residual estimators ER(I) and ER(2).
- the first echo residual estimator ER(I) may be configured to generate a first residual echo estimation ERi (t) that includes a cross-correlation analysis between the minimum echo output e 3 (t) and the speaker signal x(t). As indicated at 222 of FIG.
- the first residual echo estimation ERi (t) may be determined from a cross-correlation analysis between the minimum echo output e 3 (t) and the speaker signal x(t), e.g., by determining argmin(is ⁇ e 3 (t) ⁇ x(t) ⁇ ), where argmin(is ⁇ e3(t) *x(t) ⁇ ) is true if e3(t) minimizes the expectation value of the cross- correlation e 3 (t) ⁇ * x(t).
- This minimization problem may essentially be realized adaptively. For example, assume first echo residual estimator ER(I) is initially a unit filter (all "1" values).
- the first residual echo estimation ERi(t) may be increased toward a tangent direction on search surface. This may be implemented using a Newton solver algorithm.
- the second echo residual estimator ER(2) may be configured to determine a second residual echo estimation ER 2 (t) that includes an echo-distance mismatch between the minimum echo output e 3 (t) and the speaker signal x(t). As indicated at 224 of FIG.
- the second residual echo estimation ER 2 (t) may be determined from an echo-distance mismatch between the minimum echo output e 3 (t) and the speaker signal x(t), e.g., by determining argmin(is ⁇ (e 3 (t)) 2 /(x(t)) 2 ⁇ ), where argmin(is ⁇ (e 3 (t)) 2 /(x(t)) 2 ⁇ ) is true if e 3 (t) minimizes the expectation value of the quotient (e 3 (t)) 2 /(x(t)) 2 .
- the minimization may be implemented using a Newton solver algorithm.
- the residual echo cancellation module 108 determines a minimum residual echo estimation ER3(t) of the two residual echo estimations ERi (t) and ER 2 (t) and adjusts the filtered signal e 3 (t) according to the minimum value ER 3 (t).
- the minimum residual echo estimation ER 3 (t) may be the one of ERi (t) and ER 2 (t) having a minimum energy and minimum correlation to x(t). For example, as indicated at 226 in FIG. 2B, is set equal to the minimum of ERi(t) and ER 2 (t) and the resulting value of ER 3 is subtracted from e 3 (t) to produce a residual echo cancelled filtered signal e 3 '(t), as indicated at 228.
- ER 3 is equal to ERi(t) then the residual echo xe(t) is minimally removed when the local speech signal s(t) has a non-zero magnitude. If ER 3 (Y) is equal to ER 2 (t) then the residual echo xe(t) is maximally removed when far end echo x(t) only is present (far-end-talk only period).
- N(I), N(2) may be computed for the two echo residual estimators ER(I) and ER(2) respectively.
- N(2) Il ER(2)
- the echo residual estimator having smaller norm is applied to e 3 (t) to remove the echo residual noise.
- the echo residual estimator having the larger norm is applied to e 3 (t) to remove the echo residual noise.
- the echo cancellation may remove the noise n(t) from the filtered signal e 3 (t) or the residual echo cancelled filtered signal e 3 '(t). Such noise cancellation may be undesirable since a remote recipient of the signal e 3 (t) or e 3 '(t) may interpret the absence of noise an indication that all communication from the microphone 104 has been lost.
- the apparatus 100 may optionally include a noise canceller unit 110.
- the noise cancellation module 110 may be configured to compute an estimated noise signal n'(t) from the microphone signal d(t), e.g., as indicated at 217 of FIGs. 2A-2B.
- the reduced noise signal n"(t) may be incorporated into a microphone output signal s'(t) by adding to e 3 (t) as indicated at 218 of FIG. 2A or by adding it to e 3 '(t) as indicated at 230 of FIG. 2B.
- the apparatus described above with respect to FIGs. IA- 1C and the methods described above with respect to FIG. 2A-2B may be implemented in software on a system having a programmable processor and a memory
- a signal processing method of the type described above with respect to FIG. 1, and FIGs. 2A-2B operating as described above may be implemented as part of a signal processing apparatus 300, as depicted in FIG. 3.
- the system 300 may include a processor 301 and a memory 302 (e.g., RAM, DRAM, ROM, and the like).
- the signal processing apparatus 300 may have multiple processors 301 if parallel processing is to be implemented.
- the memory 302 includes data and code configured as described above.
- the memory 302 may program code 304 and signal data 306.
- the code 304 may implement the echo canceling adaptive filters EC(I), EC(2), integrator 106, echo residual filters ER(I), ER(2) residual echo cancellation module 108 and noise canceller 110 described above.
- the signal data 306 may include digital representations of the microphone signal d(t) and/or the speaker signal x(t) .
- the apparatus 300 may also include well-known support functions 310, such as input/output (I/O) elements 311, power supplies (P/S) 312, a clock (CLK) 313 and cache 314.
- the apparatus 300 may optionally include a mass storage device 315 such as a disk drive, CD- ROM drive, tape drive, or the like to store programs and/or data.
- the controller may also optionally include a display unit 316 and user interface unit 318 to facilitate interaction between the controller 300 and a user.
- the display unit 316 may be in the form of a cathode ray tube (CRT) or flat panel screen that displays text, numerals, graphical symbols or images.
- the user interface 318 may include a keyboard, mouse, joystick, light pen or other device.
- a speaker 322 and a microphone 324 may be coupled to the processor 301 via the I/O elements 311.
- the processor 301, memory 302 and other components of the system 300 may exchange signals (e.g., code instructions and data) with each other via a system bus 320 as shown in FIG. 3.
- I/O generally refers to any program, operation or device that transfers data to or from the system 300 and to or from a peripheral device. Every data transfer may be regarded as an output from one device and an input into another.
- Peripheral devices include input-only devices, such as keyboards and mouses, output-only devices, such as printers as well as devices such as a writable CD-ROM that can act as both an input and an output device.
- peripheral device includes external devices, such as a mouse, keyboard, printer, monitor, microphone, game controller, camera, external Zip drive or scanner as well as internal devices, such as a CD-ROM drive, CD-R drive or internal modem or other peripheral such as a flash memory reader/writer, hard drive.
- the processor 301 may perform digital signal processing on signal data 306 as described above in response to the data 306 and program code instructions of a program 304 stored and retrieved by the memory 302 and executed by the processor module 301.
- Code portions of the program 304 may conform to any one of a number of different programming languages such as Assembly, C++, JAVA or a number of other languages.
- the processor module 301 forms a general-purpose computer that becomes a specific purpose computer when executing programs such as the program code 304.
- the program code 304 is described herein as being implemented in software and executed upon a general purpose computer, those skilled in the art will realize that the method of task management could alternatively be implemented using hardware such as an application specific integrated circuit (ASIC) or other hardware circuitry.
- ASIC application specific integrated circuit
- the program code 304 may include a set of processor readable instructions that implement a method having features in common with the method 200 of FIG. 2A or the method 220 of FIG. 2B.
- the program code 304 may generally include instructions that direct the processor 301 to filter a microphone signal d(t) in parallel with first and second adaptive filters having complementary echo cancellation properties to produce echo cancelled outputs ei(t) and e 2 (t); instructions for determining a minimum echo output e3(t) from ei(t) and e 2 (t); and instructions for generating a microphone output using the minimum echo output e 3 (t).
- Embodiments of the present invention allow for more robust yet accurate echo cancellation that would be possible with cross-correlation analysis alone or with voice activity detection (double talk detection) alone.
- Such improved echo cancellation can extract the local speech signal s(t) from a microphone signal d(t) that is dominated by speaker echo x(t).
- Embodiments of the present invention may be used as presented herein or in combination with other user input mechanisms and notwithstanding mechanisms that track or profile the angular direction or volume of sound and/or mechanisms that track the position of the object actively or passively, mechanisms using machine vision, combinations thereof and where the object tracked may include ancillary controls or buttons that manipulate feedback to the system and where such feedback may include but is not limited light emission from light sources, sound distortion means, or other suitable transmitters and modulators as well as controls, buttons, pressure pad, etc. that may influence the transmission or modulation of the same, encode state, and/or transmit commands from or to a device, including devices that are tracked by the system and whether such devices are part of, interacting with or influencing a system used in connection with embodiments of the present invention.
Landscapes
- Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Circuit For Audible Band Transducer (AREA)
- Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
- Telephone Function (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
Abstract
Description
Claims
Applications Claiming Priority (11)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
PCT/US2006/017483 WO2006121896A2 (en) | 2005-05-05 | 2006-05-04 | Microphone array based selective sound source listening and video game control |
US11/381,724 US8073157B2 (en) | 2003-08-27 | 2006-05-04 | Methods and apparatus for targeted sound detection and characterization |
US11/381,721 US8947347B2 (en) | 2003-08-27 | 2006-05-04 | Controlling actions in a video game unit |
US11/381,728 US7545926B2 (en) | 2006-05-04 | 2006-05-04 | Echo and noise cancellation |
US11/381,729 US7809145B2 (en) | 2006-05-04 | 2006-05-04 | Ultra small microphone array |
US11/381,725 US7783061B2 (en) | 2003-08-27 | 2006-05-04 | Methods and apparatus for the targeted sound detection |
US11/418,988 US8160269B2 (en) | 2003-08-27 | 2006-05-04 | Methods and apparatuses for adjusting a listening area for capturing sounds |
US11/381,727 US7697700B2 (en) | 2006-05-04 | 2006-05-04 | Noise removal for electronic device with far field microphone on console |
US11/429,047 US8233642B2 (en) | 2003-08-27 | 2006-05-04 | Methods and apparatuses for capturing an audio signal based on a location of the signal |
US11/418,989 US8139793B2 (en) | 2003-08-27 | 2006-05-04 | Methods and apparatus for capturing audio signals based on a visual image |
PCT/US2007/065686 WO2007130765A2 (en) | 2006-05-04 | 2007-03-30 | Echo and noise cancellation |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2014132A2 true EP2014132A2 (en) | 2009-01-14 |
EP2014132A4 EP2014132A4 (en) | 2013-01-02 |
Family
ID=56290936
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP07759884A Withdrawn EP2012725A4 (en) | 2006-05-04 | 2007-03-30 | Narrow band noise reduction for speech enhancement |
EP07759872A Withdrawn EP2014132A4 (en) | 2006-05-04 | 2007-03-30 | Echo and noise cancellation |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP07759884A Withdrawn EP2012725A4 (en) | 2006-05-04 | 2007-03-30 | Narrow band noise reduction for speech enhancement |
Country Status (3)
Country | Link |
---|---|
EP (2) | EP2012725A4 (en) |
JP (3) | JP4476355B2 (en) |
WO (2) | WO2007130766A2 (en) |
Families Citing this family (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8738367B2 (en) | 2009-03-18 | 2014-05-27 | Nec Corporation | Speech signal processing device |
JP4964267B2 (en) * | 2009-04-03 | 2012-06-27 | 有限会社ケプストラム | Adaptive filter and echo canceller having the same |
JP2010249939A (en) * | 2009-04-13 | 2010-11-04 | Sony Corp | Noise reducing device and noise determination method |
EP2858068A4 (en) * | 2012-05-31 | 2016-02-24 | Toyota Motor Co Ltd | Audio source detection device, noise model generation device, noise reduction device, audio source direction estimation device, approaching vehicle detection device and noise reduction method |
CN109166589A (en) * | 2018-08-13 | 2019-01-08 | 深圳市腾讯网络信息技术有限公司 | Using sound suppressing method, device, medium and equipment |
CN114868183A (en) | 2019-12-18 | 2022-08-05 | 杜比实验室特许公司 | Filter adaptive step size control for echo cancellation |
CN113689871A (en) * | 2020-05-19 | 2021-11-23 | 阿里巴巴集团控股有限公司 | Echo cancellation method and device |
CN112017679B (en) * | 2020-08-05 | 2024-01-26 | 海尔优家智能科技(北京)有限公司 | Method, device and equipment for updating adaptive filter coefficients |
Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2006040734A1 (en) * | 2004-10-13 | 2006-04-20 | Koninklijke Philips Electronics N.V. | Echo cancellation |
Family Cites Families (20)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US4802227A (en) * | 1987-04-03 | 1989-01-31 | American Telephone And Telegraph Company | Noise reduction processing arrangement for microphone arrays |
JP3135937B2 (en) * | 1991-05-16 | 2001-02-19 | 株式会社リコー | Noise removal device |
JP3110201B2 (en) * | 1993-04-16 | 2000-11-20 | 沖電気工業株式会社 | Noise removal device |
DE69428119T2 (en) * | 1993-07-07 | 2002-03-21 | Picturetel Corp | REDUCING BACKGROUND NOISE FOR LANGUAGE ENHANCEMENT |
US5806025A (en) * | 1996-08-07 | 1998-09-08 | U S West, Inc. | Method and system for adaptive filtering of speech signals using signal-to-noise ratio to choose subband filter bank |
SE9700772D0 (en) * | 1997-03-03 | 1997-03-03 | Ericsson Telefon Ab L M | A high resolution post processing method for a speech decoder |
FR2771542B1 (en) * | 1997-11-21 | 2000-02-11 | Sextant Avionique | FREQUENTIAL FILTERING METHOD APPLIED TO NOISE NOISE OF SOUND SIGNALS USING A WIENER FILTER |
DE19806015C2 (en) * | 1998-02-13 | 1999-12-23 | Siemens Ag | Process for improving acoustic attenuation in hands-free systems |
US6263078B1 (en) * | 1999-01-07 | 2001-07-17 | Signalworks, Inc. | Acoustic echo canceller with fast volume control compensation |
WO2000049602A1 (en) * | 1999-02-18 | 2000-08-24 | Andrea Electronics Corporation | System, method and apparatus for cancelling noise |
US6426979B1 (en) * | 1999-04-29 | 2002-07-30 | Legerity, Inc. | Adaptation control algorithm for echo cancellation using signal-value based analysis |
US6522746B1 (en) * | 1999-11-03 | 2003-02-18 | Tellabs Operations, Inc. | Synchronization of voice boundaries and their use by echo cancellers in a voice processing system |
JP3358731B2 (en) * | 2000-04-24 | 2002-12-24 | 株式会社富建設 | Nursing equipment |
US7139401B2 (en) * | 2002-01-03 | 2006-11-21 | Hitachi Global Storage Technologies B.V. | Hard disk drive with self-contained active acoustic noise reduction |
JP2003284181A (en) * | 2002-03-20 | 2003-10-03 | Matsushita Electric Ind Co Ltd | Sound collection apparatus |
DE10305369B4 (en) * | 2003-02-10 | 2005-05-19 | Siemens Ag | User-adaptive method for noise modeling |
US6947549B2 (en) * | 2003-02-19 | 2005-09-20 | The Hong Kong Polytechnic University | Echo canceller |
US7885420B2 (en) * | 2003-02-21 | 2011-02-08 | Qnx Software Systems Co. | Wind noise suppression system |
JP4227529B2 (en) * | 2004-01-06 | 2009-02-18 | パナソニック株式会社 | Periodic noise suppressor |
US7254535B2 (en) * | 2004-06-30 | 2007-08-07 | Motorola, Inc. | Method and apparatus for equalizing a speech signal generated within a pressurized air delivery system |
-
2007
- 2007-03-30 EP EP07759884A patent/EP2012725A4/en not_active Withdrawn
- 2007-03-30 EP EP07759872A patent/EP2014132A4/en not_active Withdrawn
- 2007-03-30 WO PCT/US2007/065701 patent/WO2007130766A2/en active Application Filing
- 2007-03-30 JP JP2009509908A patent/JP4476355B2/en not_active Expired - Fee Related
- 2007-03-30 WO PCT/US2007/065686 patent/WO2007130765A2/en active Application Filing
- 2007-03-30 JP JP2009509909A patent/JP4866958B2/en not_active Expired - Fee Related
-
2010
- 2010-01-29 JP JP2010019147A patent/JP4833343B2/en not_active Expired - Fee Related
Patent Citations (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2006040734A1 (en) * | 2004-10-13 | 2006-04-20 | Koninklijke Philips Electronics N.V. | Echo cancellation |
Non-Patent Citations (1)
Title |
---|
See also references of WO2007130765A2 * |
Also Published As
Publication number | Publication date |
---|---|
JP4866958B2 (en) | 2012-02-01 |
JP2009535997A (en) | 2009-10-01 |
JP4833343B2 (en) | 2011-12-07 |
WO2007130766A3 (en) | 2008-09-04 |
WO2007130765A2 (en) | 2007-11-15 |
EP2012725A4 (en) | 2011-10-12 |
JP2010171985A (en) | 2010-08-05 |
JP2009535996A (en) | 2009-10-01 |
EP2014132A4 (en) | 2013-01-02 |
JP4476355B2 (en) | 2010-06-09 |
EP2012725A2 (en) | 2009-01-14 |
WO2007130765A3 (en) | 2008-12-18 |
WO2007130766A2 (en) | 2007-11-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7545926B2 (en) | Echo and noise cancellation | |
JP4833343B2 (en) | Echo and noise cancellation | |
EP2783504B1 (en) | Acoustic echo cancellation based on ultrasound motion detection | |
CN108141502B (en) | Method for reducing acoustic feedback in an acoustic system and audio signal processing device | |
AU754246B2 (en) | Methods and apparatus for improved echo suppression in communications systems | |
JP4702372B2 (en) | Echo suppression method and apparatus | |
EP3568854A1 (en) | Post-mixing acoustic echo cancellation systems and methods | |
JP5125690B2 (en) | Acoustic echo canceller | |
CN110176244B (en) | Echo cancellation method, device, storage medium and computer equipment | |
CN104395957A (en) | A universal reconfigurable echo cancellation system | |
US8693678B2 (en) | Device and method for controlling damping of residual echo | |
WO2009126410A2 (en) | Integrated latency detection and echo cancellation | |
EP1129526A1 (en) | Methods and apparatus for providing echo suppression using frequency domain nonlinear processing | |
WO2005125168A1 (en) | Echo canceling apparatus, telephone set using the same, and echo canceling method | |
CN111052767B (en) | Audio processing device, audio processing method, and information processing device | |
WO2019068115A1 (en) | Echo canceller and method therefor | |
US7333605B1 (en) | Acoustic echo cancellation with adaptive step size and stability control | |
JP2000502861A (en) | Convergence measurement device and convergence measurement method | |
US8345860B1 (en) | Method and system for detection of onset of near-end signal in an echo cancellation system | |
WO2014147552A1 (en) | Method and apparatus for audio processing | |
JP2918025B2 (en) | 2-wire audio conference device | |
CN111355855B (en) | Echo processing method, device, equipment and storage medium | |
JPWO2009038182A1 (en) | System identification apparatus and system identification method | |
CN113241084B (en) | Echo cancellation method, device and equipment | |
CN111681666B (en) | Backup method and device for filter coefficient and computer storage medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20081107 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA HR MK RS |
|
R17D | Deferred search report published (corrected) |
Effective date: 20081218 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04M 9/08 20060101AFI20090204BHEP |
|
DAX | Request for extension of the european patent (deleted) | ||
A4 | Supplementary search report drawn up and despatched |
Effective date: 20121130 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 3/033 20130101ALI20121126BHEP Ipc: H04M 9/08 20060101ALI20121126BHEP Ipc: A63F 13/06 20060101AFI20121126BHEP |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: SONY COMPUTER ENTERTAINMENT INCORPORATED |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION HAS BEEN WITHDRAWN |
|
18W | Application withdrawn |
Effective date: 20130626 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: SONY COMPUTER ENTERTAINMENT INC. |