EP1451813B1 - Method for suppressing surrounding noise in a hands-free device, and hands-free device - Google Patents

Method for suppressing surrounding noise in a hands-free device, and hands-free device Download PDF

Info

Publication number
EP1451813B1
EP1451813B1 EP02795098A EP02795098A EP1451813B1 EP 1451813 B1 EP1451813 B1 EP 1451813B1 EP 02795098 A EP02795098 A EP 02795098A EP 02795098 A EP02795098 A EP 02795098A EP 1451813 B1 EP1451813 B1 EP 1451813B1
Authority
EP
European Patent Office
Prior art keywords
power density
fourier transform
spectral
input
signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP02795098A
Other languages
German (de)
French (fr)
Other versions
EP1451813A1 (en
Inventor
Stefan Gierl
Christoph Benz
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harman Becker Automotive Systems GmbH
Original Assignee
Harman Becker Automotive Systems GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harman Becker Automotive Systems GmbH filed Critical Harman Becker Automotive Systems GmbH
Publication of EP1451813A1 publication Critical patent/EP1451813A1/en
Application granted granted Critical
Publication of EP1451813B1 publication Critical patent/EP1451813B1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02165Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208Noise filtering
    • G10L21/0216Noise filtering characterised by the method used for estimating noise
    • G10L2021/02168Noise filtering characterised by the method used for estimating noise the estimation exclusively taking place during speech pauses

Definitions

  • the invention relates to a method for suppressing ambient noise in a speakerphone with two microphones arranged at a predeterminable distance.
  • the invention further relates to a hands-free device having two microphones arranged at a predeterminable distance from one another.
  • ambient noise is a major disruptive factor that can significantly affect speech intelligibility.
  • Car telephones are equipped with hands-free equipment so that the driver can fully concentrate on driving the vehicle and on the road. In a vehicle but especially loud and disturbing ambient noise occur.
  • Glantzige microphone arrays are necessary.
  • the hands-free device is equipped with two microphones, which are arranged at a predeterminable distance from one another.
  • the distance of the speaker to the microphones is chosen smaller than the so-called. Hall radius, so that the direct sound components of the speaker at the location of the microphones dominate over the reflection components occurring in space.
  • the sum and the difference signal are formed, from which the Fourier transform of the sum and the Fourier transform of the difference signal are formed by means of a respective Fourier transformer.
  • the speech pauses z. B. detected by the fact that their average short-term performance are determined.
  • the short-term powers of the sum and difference signals are approximately equal, because with uncorrelated signal components, it does not matter if they are added or subtracted prior to the power calculation, while at the beginning of speech, due to the highly correlated speech component, the short-term power in the sum signal is greater than the short-term power in the difference signal increases significantly. This increase can be easily detected and used for the reliable detection of a speech break. A speech break can therefore be detected with great certainty even in the case of loud ambient noise.
  • the method according to the invention further provides for determining the spectral power density from the Fourier transform of the sum signal and from the Fourier transform of the difference signal, from which the transfer function for an adaptive transformation filter is calculated.
  • This adaptive transformation filter generates the interference power density by multiplying the power density of the Fourier transform of the difference signal by its transfer function.
  • the transfer function of a likewise adaptive Spektralsubt Trackionsfilters is calculated, which filters the Fourier transform of the sum signal and provides at its output a largely free from ambient noise audio signal in the frequency domain, by means of an inverse Fourier transformer is transformed back into the time domain. At the output of this inverse Fourier transformer can therefore a largely of ambient noise free audio or voice signal in the time domain removed and further processed.
  • the output of a first microphone M1 is connected to the first input of an adder AD and the first input of a subtractor SU, while the output of a second microphone M2 is connected to the second input of the adder AD and the second input of the subtractor SU.
  • the output of the adder AD is connected to the input of a first Fourier transformer F1, whose output to the first input of a speech pause detector P, the input of a first arithmetic unit LS for calculating the spectral power density S rr of the Fourier transform R (f) of the sum signal S and Input of an adaptive spectral subtraction filter SF is connected.
  • the output of the subtractor SU is connected to the input of a second Fourier transformer F2 whose output is connected to the second input of the speech pause detector P and to the input of a second arithmetic unit LD for calculating the spectral power density S DD of the Fourier transform D (f) of the difference signal D.
  • the output of the first arithmetic unit LS is connected to a third arithmetic unit for calculating the transfer function of an adaptive transformation filter TF and to the first control input of the adaptive spectral subtraction filter SF whose output is connected to the input of an inverse Fourier transformer IF.
  • the output of the second arithmetic unit LD is connected to the third arithmetic unit R and to the input of the adaptive transformation filter TF whose output is connected to the second control input of the adaptive spectral subtraction filter SF.
  • the output of the speech pause detector P is also connected to the third arithmetic unit R whose output is connected to the control input of the adaptive transformation filter TF.
  • the two microphones M1 and M2 are, as already mentioned, arranged at a distance from the speaker, which is smaller than the so-called Hall radius. For this reason, the direct sound components of the speaker at the location of the microphones dominate over those in a closed space, eg. B. the interior of a vehicle, occurring reflection components.
  • the sum signal S of the microphone signals MS1 and MS2 of the two microphones M1 and M2 is formed in the adder AD, while the difference signal D of the microphone signals MS1 and MS2 is formed in the subtracter SU.
  • the first Fourier transformer F1 forms the Fourier transform R (f) of the sum signal S.
  • the second Fourier transformer F2 forms the Fourier transform D (f) of the difference signal D.
  • the short-time power of the Fourier transform R (f) of the sum signal S and the Fourier transform D (f) of the difference signal D is determined.
  • the two short-term performances hardly differ from each other, because it does not matter for uncorrelated signal components whether they are added or subtracted before the power calculation.
  • the short-time power in the sum signal increases significantly compared with the short-time power in the difference signal. This increase therefore indicates the end of a speech break and the beginning of speech.
  • the first arithmetic unit LS calculates by time averaging the spectral power density S rr of the Fourier transform R (f) of the sum signal S.
  • the second arithmetic unit LD calculates the spectral power density S DD of the Fourier transform D (f) of the difference signal D.
  • the spectral power density S rr (f) is obtained by temporal averaging of the Fourier transform R (f) of the sum signal S, while in the same way the spectral power density S DD (f) calculated by time averaging from the Fourier transform D (f) of the difference signal D. becomes.
  • the transfer function H sub of the spectral subtraction filter SF is determined according to the following rule (5).
  • the parameter a represents the so-called overestimation factor, while b represents the so-called "spectral floor”.
  • the inventive method and the hands-free circuit according to the invention which are particularly suitable for a car phone, are characterized by an excellent voice quality and speech intelligibility, because the estimated value for the noise power density Snn irrespective of the voice activity is permanently updated.
  • the transfer function of the spectral subtraction filter SF is constantly updated, both during voice activity and during the speech pauses.
  • speech pauses are reliably and accurately detected, which is necessary for updating the transformation filter TF.
  • the audio signal at the output of the spectral subtraction filter SF which is largely free from ambient noise, is fed to an inverse Fourier transformer IF, which transforms the audio signal back into the time domain.

Abstract

In order to suppress as much surrounding noise as possible in a hands-free device in a motor vehicle for example, two microphones (M1, M2) are arranged at a distance from each other. The output signals (MS1, MS2) of said microphones are added in an adder (AD) and subtracted in a subtracter (SU). The adder's (AD) cumulative signal (S) is Fourier transformed in a first Fourier transformer (F1), and the subtracter's (SU) differential signal (D) is Fourier transformed in a second Fourier transformer (F2). A speech pause detector (P) detects speech pauses from the two Fourier transformed R(f) and D(f) during which a third arithmetic unit (R) calculates the transfer function HT of an adaptable transformation filter (TF) from the spectral power density Srr of the cumulative signal (S) and the spectral power density SDD of the differential signal (D). The transfer function of a spectral subtraction filter (SF), at the inlet of which the cumulative signal's (S) Fourier transformed R(f) is located, is created from the spectral power density Srr of the cumulative signal (S) and the interference power density Snn generated by the adaptable transformation filter (TF). The outlet of the spectral subtraction filter (SF) is connected to the inlet of an inverse Fourier transformer (IF), at the outlet of which an audio signal (A) can be picked up in the time domain that is virtually free of surrounding noise.

Description

Die Erfindung betrifft ein Verfahren zur Unterdrückung von Umgebungsgeräuschen bei einer Freisprecheinrichtung mit zwei in vorgebbarem Abstand zueinander angeordneten Mikrofonen.The invention relates to a method for suppressing ambient noise in a speakerphone with two microphones arranged at a predeterminable distance.

Die Erfindung betrifft weiter eine Freisprecheinrichtung mit zwei in vorgebbarem Abstand zueinander angeordneten Mikrofonen.The invention further relates to a hands-free device having two microphones arranged at a predeterminable distance from one another.

Beim Einsatz von Freisprecheinrichtungen stellen Umgebungsgeräusche einen starken Störfaktor dar, der die Sprachverständlichkeit erheblich beeinträchtigen kann. Autotelefone sind mit Freisprecheinrichtungen ausgerüstet, damit der Fahrer sich voll auf das Führen des Fahrzeuges und den Straßenverkehr konzentrieren kann. In einem Fahrzeug treten aber besonders laute und störende Umgebungsgeräusche auf.When using hands-free equipment, ambient noise is a major disruptive factor that can significantly affect speech intelligibility. Car telephones are equipped with hands-free equipment so that the driver can fully concentrate on driving the vehicle and on the road. In a vehicle but especially loud and disturbing ambient noise occur.

Die Dissertation Stefan Gierl: "Geräuschreduktion bei Sprachübertragung mit Hilfe von Mikrofonarraysystemen", Universität Karlsruhe, 1990 beschreibt ein Mikrofonarraysystem zur Geräuschreduktion bei der Sprachübertragung, welches in der Lage ist, sowohl kohärenten als auch inkohärenten Störschall zu unterdrücken. Durch die Anwendung eines Spektralsubtraktionsverfahrens werden dabei aus Laufzeit ausgeglichenen Mikrofonsignalen sprachbefreite Störreferenzen berechnet, aus denen wiederum Schätzwerte für die Störleistungsdichtespektren ermittelt und permanent aktualisiert werden. Auf Sprachpausendetektoren kann dadurch verzichtet werden.The dissertation Stefan Gierl: "Noise reduction in speech transmission with the help of microphone array systems", University of Karlsruhe, 1990 describes a microphone array system for noise reduction in speech transmission, which is capable of suppressing both coherent and incoherent noise. By applying a spectral subtraction method, speech-canceled interference references are calculated from balanced microphone signals, from which in turn estimated values for the interference power density spectra are determined and permanently updated. On speech pause detectors can be dispensed with.

Es ist Aufgabe der Erfindung, ein Verfahren zur Unterdrückung von Umgebungsgeräuschen für eine Freisprecheinrichtung sowie eine Freisprecheinrichtung so zu gestalten, daß Umgebungsgeräusche möglichst vollständig unterdrückt werden, ohne dass dazu mehrkanälige Mikrofonarrays notwendig sind.It is an object of the invention to design a method for suppressing ambient noise for a hands-free device and a speakerphone so that ambient noise can be suppressed as completely as possible, without the need mehrkanälige microphone arrays are necessary.

Verfahrensmäßig wird diese Aufgabe mit den im Anspruch 1 angegebenen Merkmalen gelöst.The method, this object is achieved with the features specified in claim 1.

Vorrichtungsmäßig wird-diese Aufgabe mit den im Anspruch 10- angegebenen Merkmalen gelöst.In terms of apparatus, this object is achieved with the features specified in claim 10.

Die erfindungsgemäße Freisprecheinrichtung ist mit zwei Mikrofonen ausgerüstet, die in einem vorgebbaren Abstand zueinander angeordnet sind. Der Abstand des Sprechers zu den Mikrofonen ist kleiner gewählt als der sog. Hallradius, so daß die Direktschallanteile des Sprechers am Ort der Mikrofone über die im Raum auftretenden Reflexionsanteile dominieren. Aus den von den Mikrofonen gelieferten Mikrofonsignalen werden das Summen- und das Differenzsignal gebildet, aus denen mittels je eines Fouriertransformators die Fouriertransformierte des Summen- und die Fouriertransformierte des Differenzsignales gebildet werden.The hands-free device according to the invention is equipped with two microphones, which are arranged at a predeterminable distance from one another. The distance of the speaker to the microphones is chosen smaller than the so-called. Hall radius, so that the direct sound components of the speaker at the location of the microphones dominate over the reflection components occurring in space. From the microphone signals supplied by the microphones, the sum and the difference signal are formed, from which the Fourier transform of the sum and the Fourier transform of the difference signal are formed by means of a respective Fourier transformer.

Aus diesen Fouriertransformierten werden die Sprachpausen z. B. dadurch detektiert, daß deren mittlere Kurzzeitleistungen ermittelt werden. Während Sprachpausen sind die Kurzzeitleistungen des Summen- und des Differenzsignales ungefähr gleich, weil es bei unkorrelierten Signalanteilen keine Rolle spielt, ob sie vor der Leistungsberechnung addiert oder subtrahiert werden, während bei Sprachbeginn aufgrund des stark korrelierte Sprachanteils die Kurzzeitleistung im Summensignal gegenüber der Kurzzeitleistung im Differenzsignal deutlich ansteigt. Dieser Anstieg läßt sich leicht detektieren und zur sicheren Detektion einer Sprachpause nutzen. Eine Sprachpause kann deshalb sogar bei lauten Umgebungsgeräuschen mit großer Sicherheit detektiert werden.From these Fourier transform the speech pauses z. B. detected by the fact that their average short-term performance are determined. During speech pauses, the short-term powers of the sum and difference signals are approximately equal, because with uncorrelated signal components, it does not matter if they are added or subtracted prior to the power calculation, while at the beginning of speech, due to the highly correlated speech component, the short-term power in the sum signal is greater than the short-term power in the difference signal increases significantly. This increase can be easily detected and used for the reliable detection of a speech break. A speech break can therefore be detected with great certainty even in the case of loud ambient noise.

Das erfindungsgemäße Verfahren sieht weiter vor, aus der Fouriertransformierten des Summensignales und aus der Fouriertransformierten des Differenzsignales die spektrale Leistungsdichte zu ermitteln, aus denen die Übertragungsfunktion für ein adaptives Transformationsfilter berechnet wird. Dieses adaptive Transformationsfilter erzeugt durch Multiplikation der Leistungsdichte der Fouriertransformierten des Differenzsignales mit seiner Übertragungsfunktion die Störleistungsdichte. Aus der spektralen Leistungsdichte der Fouriertransformierten des Summensignals und aus der vom adaptiven Transformationsfilter erzeugten Störleistungsdichte wird die Übertragungsfunktion eines ebenfalls adaptiven Spektralsubtraktionsfilters berechnet, das die Fouriertransformierte des Summensignals filtert und an seinem Ausgang ein weitestgehend von Umgebungsgeräuschen freies Audiosignal im Frequenzbereich liefert, das mittels eines inversen Fouriertransformators in den Zeitbereich zurück transformiert wird. Am Ausgang dieses inversen Fouriertransformators kann daher ein weitestgehend von Umgebungsgeräuschen freies Audio- oder Sprachsignal im Zeitbereich abgenommen und weiter verarbeitet werden.The method according to the invention further provides for determining the spectral power density from the Fourier transform of the sum signal and from the Fourier transform of the difference signal, from which the transfer function for an adaptive transformation filter is calculated. This adaptive transformation filter generates the interference power density by multiplying the power density of the Fourier transform of the difference signal by its transfer function. From the spectral power density of the Fourier transform of the sum signal and from the interference power density generated by the adaptive transform filter, the transfer function of a likewise adaptive Spektralsubttraktionsfilters is calculated, which filters the Fourier transform of the sum signal and provides at its output a largely free from ambient noise audio signal in the frequency domain, by means of an inverse Fourier transformer is transformed back into the time domain. At the output of this inverse Fourier transformer can therefore a largely of ambient noise free audio or voice signal in the time domain removed and further processed.

Das erfindungsgemäße Verfahren und die erfindungsgemäße Freisprecheinrichtung werden anhand des in der Figur gezeigten Ausführungsbeispieles näher beschrieben und erläutert.The inventive method and the hands-free device according to the invention are described and explained in more detail with reference to the embodiment shown in the figure.

Der Ausgang eines ersten Mikrofons M1 ist mit dem ersten Eingang eines Addierers AD und dem ersten Eingang eines Subtrahierers SU verbunden, während der Ausgang eines zweiten Mikrofons M2 mit dem zweiten Eingang des Addierers AD und dem zweiten Eingang des Subtrahierers SU verbunden ist. Der Ausgang des Addierers AD ist mit dem Eingang eines ersten Fouriertransformators F1 verbunden, dessen Ausgang mit dem ersten Eingang eines Sprachpausendetektors P, dem Eingang einer ersten Recheneinheit LS zur Berechnung der spektralen Leistungsdichte Srr der Fouriertransformierten R(f) des Summensignales S und mit dem Eingang eines adaptiven Spektralsubtraktionsfilters SF verbunden ist.The output of a first microphone M1 is connected to the first input of an adder AD and the first input of a subtractor SU, while the output of a second microphone M2 is connected to the second input of the adder AD and the second input of the subtractor SU. The output of the adder AD is connected to the input of a first Fourier transformer F1, whose output to the first input of a speech pause detector P, the input of a first arithmetic unit LS for calculating the spectral power density S rr of the Fourier transform R (f) of the sum signal S and Input of an adaptive spectral subtraction filter SF is connected.

Der Ausgang des Subtrahierers SU ist mit dem Eingang eines zweiten Fouriertransformators F2 verbunden, dessen Ausgang mit dem zweiten Eingang des Sprachpausendetektors P und mit dem Eingang einer zweiten Recheneinheit LD zur Berechnung der spektralen Leistungsdichte SDD der Fouriertransformierten D(f) des Differenzsignales D verbunden ist. Der Ausgang der ersten Recheneinheit LS ist mit einer dritten Recheneinheit zur Berechnung der Übertragungsfunktion eines adaptiven Transformationsfilters TF und mit dem ersten Steuereingang des adaptiven Spektralsubtraktionsfilters SF verbunden, dessen Ausgang mit dem Eingang eines inversen Fouriertransformators IF verbunden ist. Der Ausgang der zweiten Recheneinheit LD ist mit der dritten Recheneinheit R und dem Eingang des adaptiven Transformationsfilters TF verbunden, dessen Ausgang mit dem zweiten Steuereingang des adaptiven Spektralsubtraktionsfilters SF verbunden ist. Der Ausgang des Sprachpausendetektors P ist ebenfalls mit der dritten Recheneinheit R verbunden, deren Ausgang mit dem Steuereingang des adaptiven Transformationsfilters TF verbunden ist.The output of the subtractor SU is connected to the input of a second Fourier transformer F2 whose output is connected to the second input of the speech pause detector P and to the input of a second arithmetic unit LD for calculating the spectral power density S DD of the Fourier transform D (f) of the difference signal D. , The output of the first arithmetic unit LS is connected to a third arithmetic unit for calculating the transfer function of an adaptive transformation filter TF and to the first control input of the adaptive spectral subtraction filter SF whose output is connected to the input of an inverse Fourier transformer IF. The output of the second arithmetic unit LD is connected to the third arithmetic unit R and to the input of the adaptive transformation filter TF whose output is connected to the second control input of the adaptive spectral subtraction filter SF. The output of the speech pause detector P is also connected to the third arithmetic unit R whose output is connected to the control input of the adaptive transformation filter TF.

Die beiden Mikrofone M1 und M2 sind, wie bereits erwähnt, in einem Abstand zum Sprecher angeordnet, der kleiner als der sog. Hallradius ist. Aus diesem Grund dominieren die Direktschallanteile des Sprechers am Ort der Mikrofone über die in einem geschlossenen Raum, z. B. dem Innenraum eines Fahrzeuges, auftretenden Reflexionsanteile.The two microphones M1 and M2 are, as already mentioned, arranged at a distance from the speaker, which is smaller than the so-called Hall radius. For this reason, the direct sound components of the speaker at the location of the microphones dominate over those in a closed space, eg. B. the interior of a vehicle, occurring reflection components.

Im Addierer AD wird das Summensignal S der Mikrofonsignale MS1 und MS2 der beiden Mikrofone M1 und M2 gebildet, während im Subtrahierer SU das Differenzsignal D der Mikrofonsignale MS1 und MS2 gebildet wird.The sum signal S of the microphone signals MS1 and MS2 of the two microphones M1 and M2 is formed in the adder AD, while the difference signal D of the microphone signals MS1 and MS2 is formed in the subtracter SU.

Der erste Fouriertransformator F1 bildet die Fouriertransformierte R(f) des Summensignals S. Ebenso bildet der zweite Fouriertransformator F2 die Fouriertransformierte D(f) des Differenzsignals D.The first Fourier transformer F1 forms the Fourier transform R (f) of the sum signal S. Likewise, the second Fourier transformer F2 forms the Fourier transform D (f) of the difference signal D.

Im Sprachpausendetektor P wird die Kurzzeitleistung der Fouriertransformierten R(f) des Summensignals S und der Fouriertransformierten D(f) des Differenzsignals D ermittelt. Während Sprachpausen unterscheiden sich die beiden Kurzzeitleistungen kaum voneinander, weil es für unkorrelierte Signalanteile keine Rolle spielt, ob sie vor der Leistungsberechnung addiert oder subtrahiert werden. Bei Sprachbeginn steigt dagegen aufgrund des stark korrelierten Sprachanteils die Kurzzeitleistung im Summensignal gegenüber der Kurzzeitleistung im Differenzsignal deutlich an. Dieser Anstieg zeigt daher das Ende einer Sprachpause und den Beginn von Sprache an.In the speech pause detector P, the short-time power of the Fourier transform R (f) of the sum signal S and the Fourier transform D (f) of the difference signal D is determined. During speech pauses, the two short-term performances hardly differ from each other, because it does not matter for uncorrelated signal components whether they are added or subtracted before the power calculation. At the beginning of speech, on the other hand, due to the strongly correlated speech component, the short-time power in the sum signal increases significantly compared with the short-time power in the difference signal. This increase therefore indicates the end of a speech break and the beginning of speech.

Die erste Recheneinheit LS berechnet durch zeitliche Mittelung die spektrale Leistungsdichte Srr der Fouriertransformierten R(f) des Summensignals S. Ebenso berechnet die zweite Recheneinheit LD die spektrale Leistungsdichte SDD der Fouriertransformierten D(f) des Differenzsignales D. Die dritte Recheneinheit R berechnet nun aus der Leistungsdichte Srrp (f) und aus der spektralen Leistungsdichte SDDp (f) während der Sprachpausen die Übertragungsfunktion HT (f) des adaptiven Transformationsfilters TF nach folgender Formel (1) H T f = S rrp f / S DDp f

Figure imgb0001
The first arithmetic unit LS calculates by time averaging the spectral power density S rr of the Fourier transform R (f) of the sum signal S. Similarly, the second arithmetic unit LD calculates the spectral power density S DD of the Fourier transform D (f) of the difference signal D. The third arithmetic unit R calculates now from the power density S rrp (f) and from the spectral power density S DDp (f) during the speech pauses the transfer function H T (f) of the adaptive transformation filter TF according to the following formula (1) H T f = S rrp f / S DDp f
Figure imgb0001

Vorzugsweise wird durch eine zusätzliche zeitliche Mittelung - also eine Glättung - der auf diese Weise gewonnenen Koeffizienten der Übertragungsfunktion die Unterdrückung von Umgebungsgeräuschen erheblich verbessert, weil das Auftreten von sog. Artefakten, die häufig auch als "musical tones" bezeichnet werden, verhindert wird.Preferably, by means of an additional temporal averaging-that is to say a smoothing-of the coefficients of the transfer function obtained in this way, the suppression of environmental noises is considerably improved because the occurrence of so-called artifacts, which are frequently also referred to as "musical tones", is prevented.

Die spektrale Leistungsdichte Srr (f) wird durch zeitliche Mittelung aus der Fouriertransformierten R(f) des Summensignals S gewonnen, während auf gleiche Weise die spektrale Leistungsdichte SDD (f) durch zeitliche Mittelung aus der Fouriertransformierten D(f) des Differenzsignales D berechnet wird.The spectral power density S rr (f) is obtained by temporal averaging of the Fourier transform R (f) of the sum signal S, while in the same way the spectral power density S DD (f) calculated by time averaging from the Fourier transform D (f) of the difference signal D. becomes.

Beispielsweise wird die spektrale Leistungsdichte Srr nach folgender Formel (2) berechnet: S rr f k = c * R f 2 + 1 - c * S rr f , k - 1

Figure imgb0002
For example, the spectral power density S rr is calculated according to the following formula (2): S rr f k = c * R f 2 + 1 - c * S rr f . k - 1
Figure imgb0002

Analog hierzu wird z. B. die spektrale Leistungsdichte SDD (f) nach folgender Formel (3) berechnet: S DD f k = c * D f 2 + 1 - c * S DD f , k - 1

Figure imgb0003
Analogously, z. B. the spectral power density S DD (f) calculated according to the following formula (3): S DD f k = c * D f 2 + 1 - c * S DD f . k - 1
Figure imgb0003

c ist eine zwischen 0 und 1 liegende Konstante, welche die Mittelungszeitdauer bestimmt. Für c = 1 findet keine zeitliche Mittelung mehr statt, vielmehr werden direkt die Betragsquadrate der Fouriertransformierten R(f) und D(f) als Schätzwerte für die spektralen Leistungsdichten genommen. Die Berechnung der restlichen spektralen Leistungsdichten, die für die Durchführung des erfindungsgemäßen Verfahrens benötigt werden, erfolgt vorzugsweise in der gleichen Weise.c is a constant lying between 0 and 1, which determines the averaging period. For c = 1 no time averaging takes place, but instead the squares of the Fourier transforms R (f) and D (f) are taken directly as estimates for the spectral power densities. The calculation of the remaining spectral power densities, which are required for carrying out the method according to the invention, preferably takes place in the same way.

Das adaptive Transformationsfilter TF erzeugt mittels seiner Übertragungsfunktion HT (f) aus der spektralen Leistungsdichte SDD (f) der Fouriertransformierten D(f) die Störleistungsdichte Snn nach folgender Formel (4): S nn f = H T * S DD f

Figure imgb0004
The adaptive transformation filter TF generates H T (f) from the spectral power density by means of its transfer function S DD (f) of the Fourier transform D (f) the interference power density S nn according to the following formula (4): S nn f = H T * S DD f
Figure imgb0004

Mit Hilfe der aus der Fouriertransformierten D(f) des Differenzsignales D berechneten Störleistungsdichte Snn und der von der ersten Recheneinheit LS berechneten spektralen Leistungsdichte Srr des Summensignals, also des gestörten Signals wird die Übertragungsfunktion Hsub des Spektralsubtraktionsfilters SF nach folgender Vorschrift (5) berechnet: H sub f = 1 - a * S nn f / S rr f f u ¨ r 1 - a * S nn f / S rr f > b H sub f = b f u ¨ r 1 - a * S nn f / S rr f b

Figure imgb0005
With the aid of the interference power density S nn calculated from the Fourier transform D (f) of the difference signal D and the spectral power density S rr of the sum signal, ie of the disturbed signal, calculated by the first arithmetic unit LS, the transfer function H sub of the spectral subtraction filter SF is determined according to the following rule (5). calculated: H sub f = 1 - a * S nn f / S rr f f u ¨ r 1 - a * S nn f / S rr f > b H sub f = b f u ¨ r 1 - a * S nn f / S rr f b
Figure imgb0005

Der Parameter a stellt hierbei den sog. Überschätzfaktor dar, während b den sog. "spectral floor" repräsentiert.The parameter a represents the so-called overestimation factor, while b represents the so-called "spectral floor".

Die von den Mikrofonen M1 und M2 aufgenommenen Störanteile, die als diffuse Schallwellen auf die Mikrofone M1 und M2 treffen, können für fast das gesamte interessierende Frequenzband als nahezu unkorreliert betrachtet werden. Allerdings besteht in Abhängigkeit vom Abstand der beiden Mikrofone M1 und M2 zueinander bei tiefen Frequenzen noch eine gewisse Korrelation, die dazu führt, daß die im Referenzsignal enthaltenen Störanteile gewissermaßen hochpaßgefiltert erscheinen. Damit eine Fehleinschätzung der tieffrequenten Störanteile bei der Spektralsubtraktion vermieden wird, erfolgt eine spektrale Anhebung der tieffrequenten Anteile des Referenzsignals mit Hilfe des in der Figur gezeigten adaptiven Transformationsfilters TF.The noise components picked up by the microphones M1 and M2, which strike the microphones M1 and M2 as diffuse sound waves, can be considered to be almost uncorrelated for almost the entire frequency band of interest. However, depending on the distance between the two microphones M1 and M2 to each other at low frequencies, there is still a certain correlation, which leads to the fact that the interference components contained in the reference signal appear to be high-pass filtered. In order to avoid a misjudgment of the low-frequency interference components during spectral subtraction, a spectral increase of the low-frequency components of the reference signal takes place with the aid of the adaptive transformation filter TF shown in the FIGURE.

Das erfindungsgemäße Verfahren und die erfindungsgemäße Freisprechschaltung, die insbesondere für ein Autotelefon geeignet sind, zeichnen sich durch eine hervorragende Sprachqualität und Sprachverständlichkeit aus, weil der Schätzwert für die Störleistungsdichte Snn unabhängig von der Sprachaktivität permanent aktualisiert wird. Somit wird auch die Übertragungsfunktion des Spektralsubtraktionsfilters SF ständig, sowohl während Sprachaktivität als auch während der Sprachpausen, aktualisiert. Wie bereits erwähnt, werden Sprachpausen sicher und genau detektiert, was für die Aktualisierung des Transformationsfilters TF erforderlich ist.The inventive method and the hands-free circuit according to the invention, which are particularly suitable for a car phone, are characterized by an excellent voice quality and speech intelligibility, because the estimated value for the noise power density Snn irrespective of the voice activity is permanently updated. Thus, also the transfer function of the spectral subtraction filter SF is constantly updated, both during voice activity and during the speech pauses. As already mentioned, speech pauses are reliably and accurately detected, which is necessary for updating the transformation filter TF.

Das Audiosignal am Ausgang des Spektralsubtraktionsfilters SF, das weitgehend frei von Umgebungsgeräuschen ist, wird einem inversen Fouriertransformator IF zugeführt, der das Audiosignal zurück in den Zeitbereich transformiert.The audio signal at the output of the spectral subtraction filter SF, which is largely free from ambient noise, is fed to an inverse Fourier transformer IF, which transforms the audio signal back into the time domain.

BezugszeichenlisteLIST OF REFERENCE NUMBERS

AA
in den Zeitbereich zurück transformiertes Audiosignalin the time domain transformed back audio signal
ADAD
Addiereradder
DD
Differenzsignaldifference signal
D(f)D (f)
Fouriertransformierte des DifferenzsignalsFourier transform of the difference signal
F1F1
erster Fouriertranformatorfirst Fourier transformer
F2F2
zweiter Fouriertransformatorsecond Fourier transformer
Hsub H sub
Übertragungsfunktion des SpektralsubtraktionsfiltersTransfer function of the spectral subtraction filter
HT H T
Übertragungsfunktion des TransformationsfiltersTransfer function of the transformation filter
IFIF
inverser Fouriertransformatorinverse Fourier transformer
LDLD
zweite Recheneinheit zur Berechnung der spektralen Leistungsdichtesecond arithmetic unit for calculating the spectral power density
LSLS
erste Recheneinheit zur Berechnung der spektralen Leistungsdichtefirst arithmetic unit for calculating the spectral power density
MS1MS1
Mikrofonsignalmicrophone signal
MS2MS2
Mikrofonsignalmicrophone signal
M1M1
Mikrofonmicrophone
M2M2
Mikrofonmicrophone
PP
SprachpausendetektorSpeech pause detector
RR
dritte Recheneinheit zur Berechnung der Übertragungs- funktion des Transformationsfiltersthird arithmetic unit for calculating the transfer function of the transformation filter
R(f)R (f)
Fouriertransformierte des SummensignalsFourier transform of the sum signal
SS
Summensignalsum signal
SFSF
SpektralsubtraktionsfilterSpektralsubtraktionsfilter
SUSU
Subtrahierersubtractor
SDD S DD
spektrale Leistungsdichte des Differenzsignalsspectral power density of the difference signal
Snn S nn
Störleistungsdichteinterference power density
Srr S rr
spektrale Leistungsdichte des Summensignalsspectral power density of the sum signal
TFTF
Transformationsfiltertransform filter

Claims (18)

  1. A method of suppressing ambient noise in a hands-free device with two microphones (M1, M2) at a predeterminable distance from one another, which supply a respective microphone signal (MS1, MS2) with the following method steps:
    the sum signal (S) and the difference signal (D) of the two microphone signals (MS1, MS2) are formed,
    the Fourier transform R(f) of the sum signal (S) and the Fourier transform D(f) of the difference signal (D) are formed,
    gaps in conversation are detected from the Fourier transforms R(f) and D(f),
    the spectral power density Sn is determined from the Fourier transform R(f) of the sum signal (S),
    the spectral power density SDD is determined from the Fourier transform D(f) of the difference signal (D),
    during the gaps in conversation the transmission function HT(f) for an adaptive transformation filter (TF) is calculated from the spectral power density Sn of the Fourier transform R(f) of the sum signal (S) and from the spectral power density SDD of the Fourier transform D(f) of the difference signal (D),
    the adaptive transformation filter (TF) produces the interference power density Snn(f) by multiplication of the power density SDD of the Fourier transform D(f) of the difference signal (B) by its transmission function HT(f),
    the transmission function Hsub(f) of a spectral subtraction filter (SF) is calculated from the interference power density Snn(f) and from the spectral power density Sn of the Fourier transform R(f) of the sum signal (S),
    the spectral subtraction filter (SF) filters the Fourier transform R(f) of the sum signal (S) and
    the output signal of the spectral subtraction filter (SF) is transformed back into the time domain.
  2. A method as claimed in claim 1, characterised in that the transmission function HT(f) of the transformation filter (TF) during gaps in conversation is given by the following formula: H T f = S rrP f / S DDP f
    Figure imgb0014
  3. A method as claimed in claim 2, characterised in that the coefficient of the transmission function HT(f) of the transformation filter (TF) is averaged over time.
  4. A method as claimed in claim 1, 2 or 3, characterised in that the calculation of the spectral power density Srr from the Fourier transform R(f) of the sum signal (S) and the spectral power density SDD from the Fourier transform D(f) of the difference signal (D) is effected by averaging over time.
  5. A method as claimed in claim 4, characterised in that the spectral power density Srr is calculated in accordance with the following formula: S rr f k = c * R f 2 + 1 - c * S rr f , k - 1
    Figure imgb0015

    wherein k represents a time index and c is a constant for determining the averaging duration.
  6. A method as claimed in claim 4 or 5, characterised in that the spectral power density SDD is calculated in accordance with the following formula: S DD f k = c * D f 2 + 1 - c * S DD f , k - 1
    Figure imgb0016

    wherein k represents a time index and c is a constant for determining the averaging duration.
  7. A method as claimed in one of claims 1 to 6, characterised in that the short term power of the Fourier transform R(f) of the sum signal (S) and of the Fourier transform D(f) of the difference of signal (D) is determined for the detection of the gaps in conversation and that a gap in conversation is detected when the two determined short term powers lie within a predeterminable, common tolerance range.
  8. A method as claimed in one of claims 1 to 7, characterised in that the transmission function Hsub(f) of the spectral subtraction filter (SF) is calculated in accordance with the following formula: H sub f = 1 - a * S nn f / S rr f for 1 - a * S nn f / S rr f > b H sub f = b for 1 - a * S nn f / S rr f b
    Figure imgb0017

    wherein a represents the so-called `overestimation factor' and b represents the so-called `spectral floor'.
  9. A method as claimed in one of claims 1 to 8, characterised in that the delay time differences between the two microphone signals (MS1, MS2) are balanced out.
  10. A hands-free device with two microphones (M1, M2) arranged at a predeterminable spacing from one another, characterised in that the output of the first microphone (M1) is connected to the first input of an adder (AD) and the first input of a subtractor (SU),
    that the output of the second microphone (M2) id connected to the second input of the adder (AD) and the second input of the subtractor (SU),
    that the output of the adder (AD) is connected to the input of a first Fourier transformer (F1), the output of which is connected to the first input of a conversational gap detector (P), to the input of a first computing unit (LS) for calculating the spectral power density Srr and to the input of an adaptive spectral subtraction filter (SF),
    that the output of the subtractor (SU) is connected to the input of a second Fourier transformer (F2), whose output is connected to the second input of the conversational gap detector (P) and to the input of a second computing unit (LD) for calculating the spectral power density SDD,
    that the outputs of the conversational gap detector (P), of the first computing unit (LS) and of the second computing unit (LD) are connected to a third computing unit (R) for calculating the transmission function HT(F) of an adaptive transformation filter (TF) during detected gaps in conversation,
    that the output of the first computing unit (LS) is connected to the first control input of the adaptive spectral subtraction filter (SF),
    that the output of the third computing unit (R) is connected to the control input of the adaptive transformation filter (TF), the input of which is connected to the output of the second computing unit (LD) and the output of which is connected to the second control input of the adaptive spectral subtraction filter (SF), and
    that the output of the adaptive spectral subtraction filter (SF) is connected to the input of an inverse Fourier transformer (IF), at whose output an audio signal (A) transformed back into the time domain may be taken off.
  11. A hands-free device as claimed in claim 10, characterised in that the transmission function HT(f) of the transformation filter (TF) during gaps in conversation is formed in accordance with the following formula: H T f = S rrp f / S DDp f
    Figure imgb0018
  12. A hands-free device as claimed in claim 11, characterised in that the coefficients of the transmission function HT(f) of the transformation filter (TF) are averaged over time.
  13. A hands-free device as claimed in claim 10, 11 or 12, characterised in that the spectral power density Srr is formed from the Fourier transform R(f) of the sum signal (S) and that the spectral power density SDD is formed from the Fourier transform D(f) of the difference signal (D) by averaging over time.
  14. A hands-free device as claimed in claim 13, characterised in that the spectral power density Srr is calculated in accordance with the following formula: S rr f k = c * R f 2 + 1 - c * S rr f , k - 1
    Figure imgb0019

    wherein k represents a time index and c is a constant for determining the averaging duration.
  15. A hands-free device as claimed in claim 13 or 14, characterised in that the spectral power density SDD is calculated in accordance with the following formula: S DD f k = c * D f 2 + 1 - c * S DD f , k - 1
    Figure imgb0020

    wherein k represents a time index and c is a constant for determining the averaging duration.
  16. A hands-free device as claimed in one of claims 10 to 15, characterised in that the short term power of the Fourier transform R(f) of the sum signal (S) and of the Fourier transform D(f) of the difference signal (D) is determined in order to detect the gaps in conversation and that a gap in conversation is detected when the two determined short term powers lie within a predeterminable, common tolerance range.
  17. A hands-free device as claimed in one of claims 10 to 16, characterised in that the transmission function Hsub(f) of the spectral function filter (SF) is calculated in accordance with the following formula: H sub f = 1 - a * S nn f / S rr f for 1 - a * S nn f / S rr f > b H sub f = b for 1 - a * S nn f / S rr f b
    Figure imgb0021

    wherein a represents the so-called 'overestimation factor' and b represents the so-called 'spectral floor'.
  18. A hands-free device as claimed in one of claims 10 to 17, characterised in that the delay time differences between the two microphone signals (M1, M2) may be balanced out.
EP02795098A 2001-12-04 2002-12-04 Method for suppressing surrounding noise in a hands-free device, and hands-free device Expired - Lifetime EP1451813B1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10159281 2001-12-04
DE10159281A DE10159281C2 (en) 2001-12-04 2001-12-04 Method for suppressing ambient noise in a hands-free device and hands-free device
PCT/EP2002/013742 WO2003049082A1 (en) 2001-12-04 2002-12-04 Method for suppressing surrounding noise in a hands-free device, and hands-free device

Publications (2)

Publication Number Publication Date
EP1451813A1 EP1451813A1 (en) 2004-09-01
EP1451813B1 true EP1451813B1 (en) 2011-03-16

Family

ID=7707848

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02795098A Expired - Lifetime EP1451813B1 (en) 2001-12-04 2002-12-04 Method for suppressing surrounding noise in a hands-free device, and hands-free device

Country Status (5)

Country Link
EP (1) EP1451813B1 (en)
AT (1) ATE502378T1 (en)
AU (1) AU2002360959A1 (en)
DE (2) DE10159281C2 (en)
WO (1) WO2003049082A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10311587A1 (en) * 2003-03-14 2004-09-23 Volkswagen Ag Hand free speech system for use in a road vehicle has signal processing and filtering to remove background noise
US7162212B2 (en) 2003-09-22 2007-01-09 Agere Systems Inc. System and method for obscuring unwanted ambient noise and handset and central office equipment incorporating the same
CN113257282B (en) * 2021-07-15 2021-10-08 成都时识科技有限公司 Speech emotion recognition method and device, electronic equipment and storage medium

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19818608C2 (en) * 1998-04-20 2000-06-15 Deutsche Telekom Ag Method and device for speech detection and noise parameter estimation

Also Published As

Publication number Publication date
AU2002360959A1 (en) 2003-06-17
DE50214964D1 (en) 2011-04-28
EP1451813A1 (en) 2004-09-01
DE10159281C2 (en) 2003-09-25
ATE502378T1 (en) 2011-04-15
DE10159281A1 (en) 2003-06-18
WO2003049082A1 (en) 2003-06-12

Similar Documents

Publication Publication Date Title
DE69738288T2 (en) DEVICE FOR SUPPRESSING A DISTURBING COMPONENT OF AN INPUT SIGNAL
DE112012000052B4 (en) Method and device for eliminating wind noise
EP0948237B1 (en) Method for noise suppression in a microphone signal
EP0668007B1 (en) Mobile radiotelephone set with handsfree device
DE102008027848A1 (en) Echo cancellers, echo cancellation and computer readable storage media
EP1143416A2 (en) Time domain noise reduction
WO1998003965A1 (en) Method of reducing voice signal interference
EP0698986A2 (en) Method for adaptive echo compensation
EP3375204B1 (en) Audio signal processing in a vehicle
DE69731573T2 (en) Noise reduction arrangement
EP0614304A1 (en) Process for improvement of acoustic feedback suppression in electro-acoustic devices
DE112012005782T5 (en) Reverberation suppression device
EP1189419B1 (en) Method and device for eliminating the loudspeaker interference on microphone signals
EP1155561B1 (en) Method and device for suppressing noise in telephone devices
DE19806015C2 (en) Process for improving acoustic attenuation in hands-free systems
EP1055317A1 (en) Method for improving acoustic noise attenuation in hand-free devices
DE112014007255B4 (en) Echo sounder and echo canceling
DE102013011761A1 (en) Motor vehicle has estimating unit and background noise spectrum unit that are designed to open dynamic filter with low background noise and close with strong background noise
DE102018117557B4 (en) ADAPTIVE FILTERING
EP1451813B1 (en) Method for suppressing surrounding noise in a hands-free device, and hands-free device
DE10137348A1 (en) Noise filtering method in voice communication apparatus, involves controlling overestimation factor and background noise variable in transfer function of wiener filter based on ratio of speech and noise signal
DE19729521B4 (en) Method and device for suppressing noise and echo
DE10025655B4 (en) A method of removing an unwanted component of a signal and system for distinguishing between unwanted and desired signal components
DE102018117558A1 (en) ADAPTIVE AFTER-FILTERING
DE102019105458B4 (en) System and method for time delay estimation

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20040618

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH

17Q First examination report despatched

Effective date: 20060817

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 50214964

Country of ref document: DE

Date of ref document: 20110428

Kind code of ref document: P

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 50214964

Country of ref document: DE

Effective date: 20110428

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20110316

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110627

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110617

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110616

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110718

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20111219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 50214964

Country of ref document: DE

Effective date: 20111219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

BERE Be: lapsed

Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS G.M.B.H.

Effective date: 20111231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 502378

Country of ref document: AT

Kind code of ref document: T

Effective date: 20111204

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111204

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111204

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20110316

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20131217

Year of fee payment: 12

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20150831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20141231

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20181126

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20181127

Year of fee payment: 17

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50214964

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20191204

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20191204

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200701