EP1451813B1 - Method for suppressing surrounding noise in a hands-free device, and hands-free device - Google Patents
Method for suppressing surrounding noise in a hands-free device, and hands-free device Download PDFInfo
- Publication number
- EP1451813B1 EP1451813B1 EP02795098A EP02795098A EP1451813B1 EP 1451813 B1 EP1451813 B1 EP 1451813B1 EP 02795098 A EP02795098 A EP 02795098A EP 02795098 A EP02795098 A EP 02795098A EP 1451813 B1 EP1451813 B1 EP 1451813B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- power density
- fourier transform
- spectral
- input
- signal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 238000000034 method Methods 0.000 title claims description 18
- 230000003595 spectral effect Effects 0.000 claims abstract description 60
- 230000009466 transformation Effects 0.000 claims abstract description 21
- 230000005236 sound signal Effects 0.000 claims abstract description 6
- 230000003044 adaptive effect Effects 0.000 claims description 21
- 230000005540 biological transmission Effects 0.000 claims description 12
- 238000012935 Averaging Methods 0.000 claims description 11
- 238000001514 detection method Methods 0.000 claims description 2
- 230000001186 cumulative effect Effects 0.000 abstract 4
- 230000002596 correlated effect Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000011410 subtraction method Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
- G10L2021/02165—Two microphones, one receiving mainly the noise signal and the other one mainly the speech signal
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02168—Noise filtering characterised by the method used for estimating noise the estimation exclusively taking place during speech pauses
Definitions
- the invention relates to a method for suppressing ambient noise in a speakerphone with two microphones arranged at a predeterminable distance.
- the invention further relates to a hands-free device having two microphones arranged at a predeterminable distance from one another.
- ambient noise is a major disruptive factor that can significantly affect speech intelligibility.
- Car telephones are equipped with hands-free equipment so that the driver can fully concentrate on driving the vehicle and on the road. In a vehicle but especially loud and disturbing ambient noise occur.
- Glantzige microphone arrays are necessary.
- the hands-free device is equipped with two microphones, which are arranged at a predeterminable distance from one another.
- the distance of the speaker to the microphones is chosen smaller than the so-called. Hall radius, so that the direct sound components of the speaker at the location of the microphones dominate over the reflection components occurring in space.
- the sum and the difference signal are formed, from which the Fourier transform of the sum and the Fourier transform of the difference signal are formed by means of a respective Fourier transformer.
- the speech pauses z. B. detected by the fact that their average short-term performance are determined.
- the short-term powers of the sum and difference signals are approximately equal, because with uncorrelated signal components, it does not matter if they are added or subtracted prior to the power calculation, while at the beginning of speech, due to the highly correlated speech component, the short-term power in the sum signal is greater than the short-term power in the difference signal increases significantly. This increase can be easily detected and used for the reliable detection of a speech break. A speech break can therefore be detected with great certainty even in the case of loud ambient noise.
- the method according to the invention further provides for determining the spectral power density from the Fourier transform of the sum signal and from the Fourier transform of the difference signal, from which the transfer function for an adaptive transformation filter is calculated.
- This adaptive transformation filter generates the interference power density by multiplying the power density of the Fourier transform of the difference signal by its transfer function.
- the transfer function of a likewise adaptive Spektralsubt Trackionsfilters is calculated, which filters the Fourier transform of the sum signal and provides at its output a largely free from ambient noise audio signal in the frequency domain, by means of an inverse Fourier transformer is transformed back into the time domain. At the output of this inverse Fourier transformer can therefore a largely of ambient noise free audio or voice signal in the time domain removed and further processed.
- the output of a first microphone M1 is connected to the first input of an adder AD and the first input of a subtractor SU, while the output of a second microphone M2 is connected to the second input of the adder AD and the second input of the subtractor SU.
- the output of the adder AD is connected to the input of a first Fourier transformer F1, whose output to the first input of a speech pause detector P, the input of a first arithmetic unit LS for calculating the spectral power density S rr of the Fourier transform R (f) of the sum signal S and Input of an adaptive spectral subtraction filter SF is connected.
- the output of the subtractor SU is connected to the input of a second Fourier transformer F2 whose output is connected to the second input of the speech pause detector P and to the input of a second arithmetic unit LD for calculating the spectral power density S DD of the Fourier transform D (f) of the difference signal D.
- the output of the first arithmetic unit LS is connected to a third arithmetic unit for calculating the transfer function of an adaptive transformation filter TF and to the first control input of the adaptive spectral subtraction filter SF whose output is connected to the input of an inverse Fourier transformer IF.
- the output of the second arithmetic unit LD is connected to the third arithmetic unit R and to the input of the adaptive transformation filter TF whose output is connected to the second control input of the adaptive spectral subtraction filter SF.
- the output of the speech pause detector P is also connected to the third arithmetic unit R whose output is connected to the control input of the adaptive transformation filter TF.
- the two microphones M1 and M2 are, as already mentioned, arranged at a distance from the speaker, which is smaller than the so-called Hall radius. For this reason, the direct sound components of the speaker at the location of the microphones dominate over those in a closed space, eg. B. the interior of a vehicle, occurring reflection components.
- the sum signal S of the microphone signals MS1 and MS2 of the two microphones M1 and M2 is formed in the adder AD, while the difference signal D of the microphone signals MS1 and MS2 is formed in the subtracter SU.
- the first Fourier transformer F1 forms the Fourier transform R (f) of the sum signal S.
- the second Fourier transformer F2 forms the Fourier transform D (f) of the difference signal D.
- the short-time power of the Fourier transform R (f) of the sum signal S and the Fourier transform D (f) of the difference signal D is determined.
- the two short-term performances hardly differ from each other, because it does not matter for uncorrelated signal components whether they are added or subtracted before the power calculation.
- the short-time power in the sum signal increases significantly compared with the short-time power in the difference signal. This increase therefore indicates the end of a speech break and the beginning of speech.
- the first arithmetic unit LS calculates by time averaging the spectral power density S rr of the Fourier transform R (f) of the sum signal S.
- the second arithmetic unit LD calculates the spectral power density S DD of the Fourier transform D (f) of the difference signal D.
- the spectral power density S rr (f) is obtained by temporal averaging of the Fourier transform R (f) of the sum signal S, while in the same way the spectral power density S DD (f) calculated by time averaging from the Fourier transform D (f) of the difference signal D. becomes.
- the transfer function H sub of the spectral subtraction filter SF is determined according to the following rule (5).
- the parameter a represents the so-called overestimation factor, while b represents the so-called "spectral floor”.
- the inventive method and the hands-free circuit according to the invention which are particularly suitable for a car phone, are characterized by an excellent voice quality and speech intelligibility, because the estimated value for the noise power density Snn irrespective of the voice activity is permanently updated.
- the transfer function of the spectral subtraction filter SF is constantly updated, both during voice activity and during the speech pauses.
- speech pauses are reliably and accurately detected, which is necessary for updating the transformation filter TF.
- the audio signal at the output of the spectral subtraction filter SF which is largely free from ambient noise, is fed to an inverse Fourier transformer IF, which transforms the audio signal back into the time domain.
Landscapes
- Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Telephone Function (AREA)
- Circuit For Audible Band Transducer (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
Description
Die Erfindung betrifft ein Verfahren zur Unterdrückung von Umgebungsgeräuschen bei einer Freisprecheinrichtung mit zwei in vorgebbarem Abstand zueinander angeordneten Mikrofonen.The invention relates to a method for suppressing ambient noise in a speakerphone with two microphones arranged at a predeterminable distance.
Die Erfindung betrifft weiter eine Freisprecheinrichtung mit zwei in vorgebbarem Abstand zueinander angeordneten Mikrofonen.The invention further relates to a hands-free device having two microphones arranged at a predeterminable distance from one another.
Beim Einsatz von Freisprecheinrichtungen stellen Umgebungsgeräusche einen starken Störfaktor dar, der die Sprachverständlichkeit erheblich beeinträchtigen kann. Autotelefone sind mit Freisprecheinrichtungen ausgerüstet, damit der Fahrer sich voll auf das Führen des Fahrzeuges und den Straßenverkehr konzentrieren kann. In einem Fahrzeug treten aber besonders laute und störende Umgebungsgeräusche auf.When using hands-free equipment, ambient noise is a major disruptive factor that can significantly affect speech intelligibility. Car telephones are equipped with hands-free equipment so that the driver can fully concentrate on driving the vehicle and on the road. In a vehicle but especially loud and disturbing ambient noise occur.
Die Dissertation
Es ist Aufgabe der Erfindung, ein Verfahren zur Unterdrückung von Umgebungsgeräuschen für eine Freisprecheinrichtung sowie eine Freisprecheinrichtung so zu gestalten, daß Umgebungsgeräusche möglichst vollständig unterdrückt werden, ohne dass dazu mehrkanälige Mikrofonarrays notwendig sind.It is an object of the invention to design a method for suppressing ambient noise for a hands-free device and a speakerphone so that ambient noise can be suppressed as completely as possible, without the need mehrkanälige microphone arrays are necessary.
Verfahrensmäßig wird diese Aufgabe mit den im Anspruch 1 angegebenen Merkmalen gelöst.The method, this object is achieved with the features specified in claim 1.
Vorrichtungsmäßig wird-diese Aufgabe mit den im Anspruch 10- angegebenen Merkmalen gelöst.In terms of apparatus, this object is achieved with the features specified in claim 10.
Die erfindungsgemäße Freisprecheinrichtung ist mit zwei Mikrofonen ausgerüstet, die in einem vorgebbaren Abstand zueinander angeordnet sind. Der Abstand des Sprechers zu den Mikrofonen ist kleiner gewählt als der sog. Hallradius, so daß die Direktschallanteile des Sprechers am Ort der Mikrofone über die im Raum auftretenden Reflexionsanteile dominieren. Aus den von den Mikrofonen gelieferten Mikrofonsignalen werden das Summen- und das Differenzsignal gebildet, aus denen mittels je eines Fouriertransformators die Fouriertransformierte des Summen- und die Fouriertransformierte des Differenzsignales gebildet werden.The hands-free device according to the invention is equipped with two microphones, which are arranged at a predeterminable distance from one another. The distance of the speaker to the microphones is chosen smaller than the so-called. Hall radius, so that the direct sound components of the speaker at the location of the microphones dominate over the reflection components occurring in space. From the microphone signals supplied by the microphones, the sum and the difference signal are formed, from which the Fourier transform of the sum and the Fourier transform of the difference signal are formed by means of a respective Fourier transformer.
Aus diesen Fouriertransformierten werden die Sprachpausen z. B. dadurch detektiert, daß deren mittlere Kurzzeitleistungen ermittelt werden. Während Sprachpausen sind die Kurzzeitleistungen des Summen- und des Differenzsignales ungefähr gleich, weil es bei unkorrelierten Signalanteilen keine Rolle spielt, ob sie vor der Leistungsberechnung addiert oder subtrahiert werden, während bei Sprachbeginn aufgrund des stark korrelierte Sprachanteils die Kurzzeitleistung im Summensignal gegenüber der Kurzzeitleistung im Differenzsignal deutlich ansteigt. Dieser Anstieg läßt sich leicht detektieren und zur sicheren Detektion einer Sprachpause nutzen. Eine Sprachpause kann deshalb sogar bei lauten Umgebungsgeräuschen mit großer Sicherheit detektiert werden.From these Fourier transform the speech pauses z. B. detected by the fact that their average short-term performance are determined. During speech pauses, the short-term powers of the sum and difference signals are approximately equal, because with uncorrelated signal components, it does not matter if they are added or subtracted prior to the power calculation, while at the beginning of speech, due to the highly correlated speech component, the short-term power in the sum signal is greater than the short-term power in the difference signal increases significantly. This increase can be easily detected and used for the reliable detection of a speech break. A speech break can therefore be detected with great certainty even in the case of loud ambient noise.
Das erfindungsgemäße Verfahren sieht weiter vor, aus der Fouriertransformierten des Summensignales und aus der Fouriertransformierten des Differenzsignales die spektrale Leistungsdichte zu ermitteln, aus denen die Übertragungsfunktion für ein adaptives Transformationsfilter berechnet wird. Dieses adaptive Transformationsfilter erzeugt durch Multiplikation der Leistungsdichte der Fouriertransformierten des Differenzsignales mit seiner Übertragungsfunktion die Störleistungsdichte. Aus der spektralen Leistungsdichte der Fouriertransformierten des Summensignals und aus der vom adaptiven Transformationsfilter erzeugten Störleistungsdichte wird die Übertragungsfunktion eines ebenfalls adaptiven Spektralsubtraktionsfilters berechnet, das die Fouriertransformierte des Summensignals filtert und an seinem Ausgang ein weitestgehend von Umgebungsgeräuschen freies Audiosignal im Frequenzbereich liefert, das mittels eines inversen Fouriertransformators in den Zeitbereich zurück transformiert wird. Am Ausgang dieses inversen Fouriertransformators kann daher ein weitestgehend von Umgebungsgeräuschen freies Audio- oder Sprachsignal im Zeitbereich abgenommen und weiter verarbeitet werden.The method according to the invention further provides for determining the spectral power density from the Fourier transform of the sum signal and from the Fourier transform of the difference signal, from which the transfer function for an adaptive transformation filter is calculated. This adaptive transformation filter generates the interference power density by multiplying the power density of the Fourier transform of the difference signal by its transfer function. From the spectral power density of the Fourier transform of the sum signal and from the interference power density generated by the adaptive transform filter, the transfer function of a likewise adaptive Spektralsubttraktionsfilters is calculated, which filters the Fourier transform of the sum signal and provides at its output a largely free from ambient noise audio signal in the frequency domain, by means of an inverse Fourier transformer is transformed back into the time domain. At the output of this inverse Fourier transformer can therefore a largely of ambient noise free audio or voice signal in the time domain removed and further processed.
Das erfindungsgemäße Verfahren und die erfindungsgemäße Freisprecheinrichtung werden anhand des in der Figur gezeigten Ausführungsbeispieles näher beschrieben und erläutert.The inventive method and the hands-free device according to the invention are described and explained in more detail with reference to the embodiment shown in the figure.
Der Ausgang eines ersten Mikrofons M1 ist mit dem ersten Eingang eines Addierers AD und dem ersten Eingang eines Subtrahierers SU verbunden, während der Ausgang eines zweiten Mikrofons M2 mit dem zweiten Eingang des Addierers AD und dem zweiten Eingang des Subtrahierers SU verbunden ist. Der Ausgang des Addierers AD ist mit dem Eingang eines ersten Fouriertransformators F1 verbunden, dessen Ausgang mit dem ersten Eingang eines Sprachpausendetektors P, dem Eingang einer ersten Recheneinheit LS zur Berechnung der spektralen Leistungsdichte Srr der Fouriertransformierten R(f) des Summensignales S und mit dem Eingang eines adaptiven Spektralsubtraktionsfilters SF verbunden ist.The output of a first microphone M1 is connected to the first input of an adder AD and the first input of a subtractor SU, while the output of a second microphone M2 is connected to the second input of the adder AD and the second input of the subtractor SU. The output of the adder AD is connected to the input of a first Fourier transformer F1, whose output to the first input of a speech pause detector P, the input of a first arithmetic unit LS for calculating the spectral power density S rr of the Fourier transform R (f) of the sum signal S and Input of an adaptive spectral subtraction filter SF is connected.
Der Ausgang des Subtrahierers SU ist mit dem Eingang eines zweiten Fouriertransformators F2 verbunden, dessen Ausgang mit dem zweiten Eingang des Sprachpausendetektors P und mit dem Eingang einer zweiten Recheneinheit LD zur Berechnung der spektralen Leistungsdichte SDD der Fouriertransformierten D(f) des Differenzsignales D verbunden ist. Der Ausgang der ersten Recheneinheit LS ist mit einer dritten Recheneinheit zur Berechnung der Übertragungsfunktion eines adaptiven Transformationsfilters TF und mit dem ersten Steuereingang des adaptiven Spektralsubtraktionsfilters SF verbunden, dessen Ausgang mit dem Eingang eines inversen Fouriertransformators IF verbunden ist. Der Ausgang der zweiten Recheneinheit LD ist mit der dritten Recheneinheit R und dem Eingang des adaptiven Transformationsfilters TF verbunden, dessen Ausgang mit dem zweiten Steuereingang des adaptiven Spektralsubtraktionsfilters SF verbunden ist. Der Ausgang des Sprachpausendetektors P ist ebenfalls mit der dritten Recheneinheit R verbunden, deren Ausgang mit dem Steuereingang des adaptiven Transformationsfilters TF verbunden ist.The output of the subtractor SU is connected to the input of a second Fourier transformer F2 whose output is connected to the second input of the speech pause detector P and to the input of a second arithmetic unit LD for calculating the spectral power density S DD of the Fourier transform D (f) of the difference signal D. , The output of the first arithmetic unit LS is connected to a third arithmetic unit for calculating the transfer function of an adaptive transformation filter TF and to the first control input of the adaptive spectral subtraction filter SF whose output is connected to the input of an inverse Fourier transformer IF. The output of the second arithmetic unit LD is connected to the third arithmetic unit R and to the input of the adaptive transformation filter TF whose output is connected to the second control input of the adaptive spectral subtraction filter SF. The output of the speech pause detector P is also connected to the third arithmetic unit R whose output is connected to the control input of the adaptive transformation filter TF.
Die beiden Mikrofone M1 und M2 sind, wie bereits erwähnt, in einem Abstand zum Sprecher angeordnet, der kleiner als der sog. Hallradius ist. Aus diesem Grund dominieren die Direktschallanteile des Sprechers am Ort der Mikrofone über die in einem geschlossenen Raum, z. B. dem Innenraum eines Fahrzeuges, auftretenden Reflexionsanteile.The two microphones M1 and M2 are, as already mentioned, arranged at a distance from the speaker, which is smaller than the so-called Hall radius. For this reason, the direct sound components of the speaker at the location of the microphones dominate over those in a closed space, eg. B. the interior of a vehicle, occurring reflection components.
Im Addierer AD wird das Summensignal S der Mikrofonsignale MS1 und MS2 der beiden Mikrofone M1 und M2 gebildet, während im Subtrahierer SU das Differenzsignal D der Mikrofonsignale MS1 und MS2 gebildet wird.The sum signal S of the microphone signals MS1 and MS2 of the two microphones M1 and M2 is formed in the adder AD, while the difference signal D of the microphone signals MS1 and MS2 is formed in the subtracter SU.
Der erste Fouriertransformator F1 bildet die Fouriertransformierte R(f) des Summensignals S. Ebenso bildet der zweite Fouriertransformator F2 die Fouriertransformierte D(f) des Differenzsignals D.The first Fourier transformer F1 forms the Fourier transform R (f) of the sum signal S. Likewise, the second Fourier transformer F2 forms the Fourier transform D (f) of the difference signal D.
Im Sprachpausendetektor P wird die Kurzzeitleistung der Fouriertransformierten R(f) des Summensignals S und der Fouriertransformierten D(f) des Differenzsignals D ermittelt. Während Sprachpausen unterscheiden sich die beiden Kurzzeitleistungen kaum voneinander, weil es für unkorrelierte Signalanteile keine Rolle spielt, ob sie vor der Leistungsberechnung addiert oder subtrahiert werden. Bei Sprachbeginn steigt dagegen aufgrund des stark korrelierten Sprachanteils die Kurzzeitleistung im Summensignal gegenüber der Kurzzeitleistung im Differenzsignal deutlich an. Dieser Anstieg zeigt daher das Ende einer Sprachpause und den Beginn von Sprache an.In the speech pause detector P, the short-time power of the Fourier transform R (f) of the sum signal S and the Fourier transform D (f) of the difference signal D is determined. During speech pauses, the two short-term performances hardly differ from each other, because it does not matter for uncorrelated signal components whether they are added or subtracted before the power calculation. At the beginning of speech, on the other hand, due to the strongly correlated speech component, the short-time power in the sum signal increases significantly compared with the short-time power in the difference signal. This increase therefore indicates the end of a speech break and the beginning of speech.
Die erste Recheneinheit LS berechnet durch zeitliche Mittelung die spektrale Leistungsdichte Srr der Fouriertransformierten R(f) des Summensignals S. Ebenso berechnet die zweite Recheneinheit LD die spektrale Leistungsdichte SDD der Fouriertransformierten D(f) des Differenzsignales D. Die dritte Recheneinheit R berechnet nun aus der Leistungsdichte Srrp (f) und aus der spektralen Leistungsdichte SDDp (f) während der Sprachpausen die Übertragungsfunktion HT (f) des adaptiven Transformationsfilters TF nach folgender Formel (1)
Vorzugsweise wird durch eine zusätzliche zeitliche Mittelung - also eine Glättung - der auf diese Weise gewonnenen Koeffizienten der Übertragungsfunktion die Unterdrückung von Umgebungsgeräuschen erheblich verbessert, weil das Auftreten von sog. Artefakten, die häufig auch als "musical tones" bezeichnet werden, verhindert wird.Preferably, by means of an additional temporal averaging-that is to say a smoothing-of the coefficients of the transfer function obtained in this way, the suppression of environmental noises is considerably improved because the occurrence of so-called artifacts, which are frequently also referred to as "musical tones", is prevented.
Die spektrale Leistungsdichte Srr (f) wird durch zeitliche Mittelung aus der Fouriertransformierten R(f) des Summensignals S gewonnen, während auf gleiche Weise die spektrale Leistungsdichte SDD (f) durch zeitliche Mittelung aus der Fouriertransformierten D(f) des Differenzsignales D berechnet wird.The spectral power density S rr (f) is obtained by temporal averaging of the Fourier transform R (f) of the sum signal S, while in the same way the spectral power density S DD (f) calculated by time averaging from the Fourier transform D (f) of the difference signal D. becomes.
Beispielsweise wird die spektrale Leistungsdichte Srr nach folgender Formel (2) berechnet:
Analog hierzu wird z. B. die spektrale Leistungsdichte SDD (f) nach folgender Formel (3) berechnet:
c ist eine zwischen 0 und 1 liegende Konstante, welche die Mittelungszeitdauer bestimmt. Für c = 1 findet keine zeitliche Mittelung mehr statt, vielmehr werden direkt die Betragsquadrate der Fouriertransformierten R(f) und D(f) als Schätzwerte für die spektralen Leistungsdichten genommen. Die Berechnung der restlichen spektralen Leistungsdichten, die für die Durchführung des erfindungsgemäßen Verfahrens benötigt werden, erfolgt vorzugsweise in der gleichen Weise.c is a constant lying between 0 and 1, which determines the averaging period. For c = 1 no time averaging takes place, but instead the squares of the Fourier transforms R (f) and D (f) are taken directly as estimates for the spectral power densities. The calculation of the remaining spectral power densities, which are required for carrying out the method according to the invention, preferably takes place in the same way.
Das adaptive Transformationsfilter TF erzeugt mittels seiner Übertragungsfunktion HT (f) aus der spektralen Leistungsdichte SDD (f) der Fouriertransformierten D(f) die Störleistungsdichte Snn nach folgender Formel (4):
Mit Hilfe der aus der Fouriertransformierten D(f) des Differenzsignales D berechneten Störleistungsdichte Snn und der von der ersten Recheneinheit LS berechneten spektralen Leistungsdichte Srr des Summensignals, also des gestörten Signals wird die Übertragungsfunktion Hsub des Spektralsubtraktionsfilters SF nach folgender Vorschrift (5) berechnet:
Der Parameter a stellt hierbei den sog. Überschätzfaktor dar, während b den sog. "spectral floor" repräsentiert.The parameter a represents the so-called overestimation factor, while b represents the so-called "spectral floor".
Die von den Mikrofonen M1 und M2 aufgenommenen Störanteile, die als diffuse Schallwellen auf die Mikrofone M1 und M2 treffen, können für fast das gesamte interessierende Frequenzband als nahezu unkorreliert betrachtet werden. Allerdings besteht in Abhängigkeit vom Abstand der beiden Mikrofone M1 und M2 zueinander bei tiefen Frequenzen noch eine gewisse Korrelation, die dazu führt, daß die im Referenzsignal enthaltenen Störanteile gewissermaßen hochpaßgefiltert erscheinen. Damit eine Fehleinschätzung der tieffrequenten Störanteile bei der Spektralsubtraktion vermieden wird, erfolgt eine spektrale Anhebung der tieffrequenten Anteile des Referenzsignals mit Hilfe des in der Figur gezeigten adaptiven Transformationsfilters TF.The noise components picked up by the microphones M1 and M2, which strike the microphones M1 and M2 as diffuse sound waves, can be considered to be almost uncorrelated for almost the entire frequency band of interest. However, depending on the distance between the two microphones M1 and M2 to each other at low frequencies, there is still a certain correlation, which leads to the fact that the interference components contained in the reference signal appear to be high-pass filtered. In order to avoid a misjudgment of the low-frequency interference components during spectral subtraction, a spectral increase of the low-frequency components of the reference signal takes place with the aid of the adaptive transformation filter TF shown in the FIGURE.
Das erfindungsgemäße Verfahren und die erfindungsgemäße Freisprechschaltung, die insbesondere für ein Autotelefon geeignet sind, zeichnen sich durch eine hervorragende Sprachqualität und Sprachverständlichkeit aus, weil der Schätzwert für die Störleistungsdichte Snn unabhängig von der Sprachaktivität permanent aktualisiert wird. Somit wird auch die Übertragungsfunktion des Spektralsubtraktionsfilters SF ständig, sowohl während Sprachaktivität als auch während der Sprachpausen, aktualisiert. Wie bereits erwähnt, werden Sprachpausen sicher und genau detektiert, was für die Aktualisierung des Transformationsfilters TF erforderlich ist.The inventive method and the hands-free circuit according to the invention, which are particularly suitable for a car phone, are characterized by an excellent voice quality and speech intelligibility, because the estimated value for the noise power density Snn irrespective of the voice activity is permanently updated. Thus, also the transfer function of the spectral subtraction filter SF is constantly updated, both during voice activity and during the speech pauses. As already mentioned, speech pauses are reliably and accurately detected, which is necessary for updating the transformation filter TF.
Das Audiosignal am Ausgang des Spektralsubtraktionsfilters SF, das weitgehend frei von Umgebungsgeräuschen ist, wird einem inversen Fouriertransformator IF zugeführt, der das Audiosignal zurück in den Zeitbereich transformiert.The audio signal at the output of the spectral subtraction filter SF, which is largely free from ambient noise, is fed to an inverse Fourier transformer IF, which transforms the audio signal back into the time domain.
- AA
- in den Zeitbereich zurück transformiertes Audiosignalin the time domain transformed back audio signal
- ADAD
- Addiereradder
- DD
- Differenzsignaldifference signal
- D(f)D (f)
- Fouriertransformierte des DifferenzsignalsFourier transform of the difference signal
- F1F1
- erster Fouriertranformatorfirst Fourier transformer
- F2F2
- zweiter Fouriertransformatorsecond Fourier transformer
- Hsub H sub
- Übertragungsfunktion des SpektralsubtraktionsfiltersTransfer function of the spectral subtraction filter
- HT H T
- Übertragungsfunktion des TransformationsfiltersTransfer function of the transformation filter
- IFIF
- inverser Fouriertransformatorinverse Fourier transformer
- LDLD
- zweite Recheneinheit zur Berechnung der spektralen Leistungsdichtesecond arithmetic unit for calculating the spectral power density
- LSLS
- erste Recheneinheit zur Berechnung der spektralen Leistungsdichtefirst arithmetic unit for calculating the spectral power density
- MS1MS1
- Mikrofonsignalmicrophone signal
- MS2MS2
- Mikrofonsignalmicrophone signal
- M1M1
- Mikrofonmicrophone
- M2M2
- Mikrofonmicrophone
- PP
- SprachpausendetektorSpeech pause detector
- RR
- dritte Recheneinheit zur Berechnung der Übertragungs- funktion des Transformationsfiltersthird arithmetic unit for calculating the transfer function of the transformation filter
- R(f)R (f)
- Fouriertransformierte des SummensignalsFourier transform of the sum signal
- SS
- Summensignalsum signal
- SFSF
- SpektralsubtraktionsfilterSpektralsubtraktionsfilter
- SUSU
- Subtrahierersubtractor
- SDD S DD
- spektrale Leistungsdichte des Differenzsignalsspectral power density of the difference signal
- Snn S nn
- Störleistungsdichteinterference power density
- Srr S rr
- spektrale Leistungsdichte des Summensignalsspectral power density of the sum signal
- TFTF
- Transformationsfiltertransform filter
Claims (18)
- A method of suppressing ambient noise in a hands-free device with two microphones (M1, M2) at a predeterminable distance from one another, which supply a respective microphone signal (MS1, MS2) with the following method steps:the sum signal (S) and the difference signal (D) of the two microphone signals (MS1, MS2) are formed,the Fourier transform R(f) of the sum signal (S) and the Fourier transform D(f) of the difference signal (D) are formed,gaps in conversation are detected from the Fourier transforms R(f) and D(f),the spectral power density Sn is determined from the Fourier transform R(f) of the sum signal (S),the spectral power density SDD is determined from the Fourier transform D(f) of the difference signal (D),during the gaps in conversation the transmission function HT(f) for an adaptive transformation filter (TF) is calculated from the spectral power density Sn of the Fourier transform R(f) of the sum signal (S) and from the spectral power density SDD of the Fourier transform D(f) of the difference signal (D),the adaptive transformation filter (TF) produces the interference power density Snn(f) by multiplication of the power density SDD of the Fourier transform D(f) of the difference signal (B) by its transmission function HT(f),the transmission function Hsub(f) of a spectral subtraction filter (SF) is calculated from the interference power density Snn(f) and from the spectral power density Sn of the Fourier transform R(f) of the sum signal (S),the spectral subtraction filter (SF) filters the Fourier transform R(f) of the sum signal (S) andthe output signal of the spectral subtraction filter (SF) is transformed back into the time domain.
- A method as claimed in claim 2, characterised in that the coefficient of the transmission function HT(f) of the transformation filter (TF) is averaged over time.
- A method as claimed in claim 1, 2 or 3, characterised in that the calculation of the spectral power density Srr from the Fourier transform R(f) of the sum signal (S) and the spectral power density SDD from the Fourier transform D(f) of the difference signal (D) is effected by averaging over time.
- A method as claimed in one of claims 1 to 6, characterised in that the short term power of the Fourier transform R(f) of the sum signal (S) and of the Fourier transform D(f) of the difference of signal (D) is determined for the detection of the gaps in conversation and that a gap in conversation is detected when the two determined short term powers lie within a predeterminable, common tolerance range.
- A method as claimed in one of claims 1 to 7, characterised in that the transmission function Hsub(f) of the spectral subtraction filter (SF) is calculated in accordance with the following formula:
wherein a represents the so-called `overestimation factor' and b represents the so-called `spectral floor'. - A method as claimed in one of claims 1 to 8, characterised in that the delay time differences between the two microphone signals (MS1, MS2) are balanced out.
- A hands-free device with two microphones (M1, M2) arranged at a predeterminable spacing from one another, characterised in that the output of the first microphone (M1) is connected to the first input of an adder (AD) and the first input of a subtractor (SU),
that the output of the second microphone (M2) id connected to the second input of the adder (AD) and the second input of the subtractor (SU),
that the output of the adder (AD) is connected to the input of a first Fourier transformer (F1), the output of which is connected to the first input of a conversational gap detector (P), to the input of a first computing unit (LS) for calculating the spectral power density Srr and to the input of an adaptive spectral subtraction filter (SF),
that the output of the subtractor (SU) is connected to the input of a second Fourier transformer (F2), whose output is connected to the second input of the conversational gap detector (P) and to the input of a second computing unit (LD) for calculating the spectral power density SDD,
that the outputs of the conversational gap detector (P), of the first computing unit (LS) and of the second computing unit (LD) are connected to a third computing unit (R) for calculating the transmission function HT(F) of an adaptive transformation filter (TF) during detected gaps in conversation,
that the output of the first computing unit (LS) is connected to the first control input of the adaptive spectral subtraction filter (SF),
that the output of the third computing unit (R) is connected to the control input of the adaptive transformation filter (TF), the input of which is connected to the output of the second computing unit (LD) and the output of which is connected to the second control input of the adaptive spectral subtraction filter (SF), and
that the output of the adaptive spectral subtraction filter (SF) is connected to the input of an inverse Fourier transformer (IF), at whose output an audio signal (A) transformed back into the time domain may be taken off. - A hands-free device as claimed in claim 11, characterised in that the coefficients of the transmission function HT(f) of the transformation filter (TF) are averaged over time.
- A hands-free device as claimed in claim 10, 11 or 12, characterised in that the spectral power density Srr is formed from the Fourier transform R(f) of the sum signal (S) and that the spectral power density SDD is formed from the Fourier transform D(f) of the difference signal (D) by averaging over time.
- A hands-free device as claimed in one of claims 10 to 15, characterised in that the short term power of the Fourier transform R(f) of the sum signal (S) and of the Fourier transform D(f) of the difference signal (D) is determined in order to detect the gaps in conversation and that a gap in conversation is detected when the two determined short term powers lie within a predeterminable, common tolerance range.
- A hands-free device as claimed in one of claims 10 to 16, characterised in that the transmission function Hsub(f) of the spectral function filter (SF) is calculated in accordance with the following formula:
wherein a represents the so-called 'overestimation factor' and b represents the so-called 'spectral floor'. - A hands-free device as claimed in one of claims 10 to 17, characterised in that the delay time differences between the two microphone signals (M1, M2) may be balanced out.
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE10159281A DE10159281C2 (en) | 2001-12-04 | 2001-12-04 | Method for suppressing ambient noise in a hands-free device and hands-free device |
DE10159281 | 2001-12-04 | ||
PCT/EP2002/013742 WO2003049082A1 (en) | 2001-12-04 | 2002-12-04 | Method for suppressing surrounding noise in a hands-free device, and hands-free device |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1451813A1 EP1451813A1 (en) | 2004-09-01 |
EP1451813B1 true EP1451813B1 (en) | 2011-03-16 |
Family
ID=7707848
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP02795098A Expired - Lifetime EP1451813B1 (en) | 2001-12-04 | 2002-12-04 | Method for suppressing surrounding noise in a hands-free device, and hands-free device |
Country Status (5)
Country | Link |
---|---|
EP (1) | EP1451813B1 (en) |
AT (1) | ATE502378T1 (en) |
AU (1) | AU2002360959A1 (en) |
DE (2) | DE10159281C2 (en) |
WO (1) | WO2003049082A1 (en) |
Families Citing this family (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10311587A1 (en) * | 2003-03-14 | 2004-09-23 | Volkswagen Ag | Hand free speech system for use in a road vehicle has signal processing and filtering to remove background noise |
US7162212B2 (en) | 2003-09-22 | 2007-01-09 | Agere Systems Inc. | System and method for obscuring unwanted ambient noise and handset and central office equipment incorporating the same |
CN113257282B (en) * | 2021-07-15 | 2021-10-08 | 成都时识科技有限公司 | Speech emotion recognition method and device, electronic equipment and storage medium |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE19818608C2 (en) * | 1998-04-20 | 2000-06-15 | Deutsche Telekom Ag | Method and device for speech detection and noise parameter estimation |
-
2001
- 2001-12-04 DE DE10159281A patent/DE10159281C2/en not_active Expired - Fee Related
-
2002
- 2002-12-04 DE DE50214964T patent/DE50214964D1/en not_active Expired - Lifetime
- 2002-12-04 AT AT02795098T patent/ATE502378T1/en active
- 2002-12-04 AU AU2002360959A patent/AU2002360959A1/en not_active Abandoned
- 2002-12-04 EP EP02795098A patent/EP1451813B1/en not_active Expired - Lifetime
- 2002-12-04 WO PCT/EP2002/013742 patent/WO2003049082A1/en not_active Application Discontinuation
Also Published As
Publication number | Publication date |
---|---|
EP1451813A1 (en) | 2004-09-01 |
DE10159281C2 (en) | 2003-09-25 |
AU2002360959A1 (en) | 2003-06-17 |
DE10159281A1 (en) | 2003-06-18 |
ATE502378T1 (en) | 2011-04-15 |
DE50214964D1 (en) | 2011-04-28 |
WO2003049082A1 (en) | 2003-06-12 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
DE69738288T2 (en) | DEVICE FOR SUPPRESSING A DISTURBING COMPONENT OF AN INPUT SIGNAL | |
DE112012000052B4 (en) | Method and device for eliminating wind noise | |
DE102008027848B4 (en) | Echo cancellers, echo cancellation and computer readable storage media | |
EP1143416B1 (en) | Time domain noise reduction | |
EP0948237B1 (en) | Method for noise suppression in a microphone signal | |
EP0668007B1 (en) | Mobile radiotelephone set with handsfree device | |
DE69731573T2 (en) | Noise reduction arrangement | |
WO1998003965A1 (en) | Method of reducing voice signal interference | |
EP0698986A2 (en) | Method for adaptive echo compensation | |
EP3375204B1 (en) | Audio signal processing in a vehicle | |
EP0614304A1 (en) | Process for improvement of acoustic feedback suppression in electro-acoustic devices | |
DE112012005782T5 (en) | Reverberation suppression device | |
EP1189419B1 (en) | Method and device for eliminating the loudspeaker interference on microphone signals | |
EP1155561B1 (en) | Method and device for suppressing noise in telephone devices | |
EP1055317A1 (en) | Method for improving acoustic noise attenuation in hand-free devices | |
DE112014007255B4 (en) | Echo sounder and echo canceling | |
DE102013011761A1 (en) | Motor vehicle has estimating unit and background noise spectrum unit that are designed to open dynamic filter with low background noise and close with strong background noise | |
DE19806015A1 (en) | Process for improving acoustic attenuation in hands-free systems | |
EP1451813B1 (en) | Method for suppressing surrounding noise in a hands-free device, and hands-free device | |
DE102018117558A1 (en) | ADAPTIVE AFTER-FILTERING | |
DE10137348A1 (en) | Noise filtering method in voice communication apparatus, involves controlling overestimation factor and background noise variable in transfer function of wiener filter based on ratio of speech and noise signal | |
DE19729521B4 (en) | Method and device for suppressing noise and echo | |
DE10025655B4 (en) | A method of removing an unwanted component of a signal and system for distinguishing between unwanted and desired signal components | |
DE102019105458B4 (en) | System and method for time delay estimation | |
DE102018117556B4 (en) | SINGLE CHANNEL NOISE REDUCTION |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20040618 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK RO |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS GMBH |
|
17Q | First examination report despatched |
Effective date: 20060817 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR IE IT LI LU MC NL PT SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D |
|
REF | Corresponds to: |
Ref document number: 50214964 Country of ref document: DE Date of ref document: 20110428 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 50214964 Country of ref document: DE Effective date: 20110428 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20110316 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110627 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110617 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110616 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FD4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 Ref country code: IE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110718 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20111219 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 50214964 Country of ref document: DE Effective date: 20111219 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 |
|
BERE | Be: lapsed |
Owner name: HARMAN BECKER AUTOMOTIVE SYSTEMS G.M.B.H. Effective date: 20111231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111231 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111231 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111231 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111231 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 502378 Country of ref document: AT Kind code of ref document: T Effective date: 20111204 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111204 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20111204 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20110316 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20131217 Year of fee payment: 12 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20150831 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20141231 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20181126 Year of fee payment: 17 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20181127 Year of fee payment: 17 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R119 Ref document number: 50214964 Country of ref document: DE |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20191204 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191204 Ref country code: DE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200701 |