EP1305975A2 - Systeme de reseau adaptatif de microphones avec preservation des signaux biauriculaires - Google Patents

Systeme de reseau adaptatif de microphones avec preservation des signaux biauriculaires

Info

Publication number
EP1305975A2
EP1305975A2 EP01942048A EP01942048A EP1305975A2 EP 1305975 A2 EP1305975 A2 EP 1305975A2 EP 01942048 A EP01942048 A EP 01942048A EP 01942048 A EP01942048 A EP 01942048A EP 1305975 A2 EP1305975 A2 EP 1305975A2
Authority
EP
European Patent Office
Prior art keywords
noise
data
adaptive
microphone
enhanced
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP01942048A
Other languages
German (de)
English (en)
Other versions
EP1305975A4 (fr
EP1305975B1 (fr
Inventor
Fa-Long Luo
Jun Yang
Brent Edwards
Nick Michael
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GN Hearing AS
Original Assignee
GN Hearing Care Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by GN Hearing Care Corp filed Critical GN Hearing Care Corp
Publication of EP1305975A2 publication Critical patent/EP1305975A2/fr
Publication of EP1305975A4 publication Critical patent/EP1305975A4/fr
Application granted granted Critical
Publication of EP1305975B1 publication Critical patent/EP1305975B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/55Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
    • H04R25/552Binaural
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00Circuits for transducers, loudspeakers or microphones
    • H04R3/005Circuits for transducers, loudspeakers or microphones for combining the signals of two or more microphones
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04RLOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R25/00Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
    • H04R25/40Arrangements for obtaining a desired directivity characteristic
    • H04R25/407Circuits for combining signals of a plurality of transducers

Definitions

  • the present invention relates to adaptive microphone array systems.
  • the combination of multiple-microphone-based spacial processing with binaural listening is receiving increasing attention in many application fields such as hearing aids because this combination should be able to provide both the spacial filtering benefits of a microphone array and the natural benefits of binaural listening for sound-location ability and speech intelligibility.
  • a microphone is positioned on each side of a user's head.
  • the system is designed such that the two microphones can be used to improve the signal-to-noise ratio as well as to maintain the binaural cues.
  • X R (n) is the microphone signal received at the right ear
  • X L (n) is the microphone signal received at the left ear.
  • a low-pass filter and a high-pass filter with cut-off frequency fc are used at each channel.
  • the outputs f R (n) and f L (n) of the two high-pass filters are sent to an adaptive processor 22 whose output is Y(n).
  • the output of the two low-pass filters 24 and 26 are delayed in delays 28 and 30, and combined with the output Y(n) of the adaptive processor 22 to provide binaural outputs Z R (n) and Z L (n).
  • the combination of the adaptive array processing with binaural listening is accomplished by dividing the frequency spectrum.
  • the low-pass portion of the frequency spectrum is devoted to binaural processing, and the high-pass part of the spectrum is devoted to adaptive array processing.
  • the binaural cues of signal parts with higher frequencies than the cut-off frequency will be lost in the system.
  • the benefits of the adaptive array processing for lower frequencies than the cut-off frequency also will be lost.
  • two low-pass filters and two high-pass filters are required.
  • the present invention comprises a microphone system using two adaptive filters, each receiving the same reference signal derived from two ear microphones but having different primary and error (filter adjustment) signals.
  • the primary signals are preferably from the right and left microphone output signals, respectively.
  • One embodiment of the present invention comprises an apparatus including a noise-enhanced data producing unit receiving left- and right-ear microphone data and producing noise-enhanced data.
  • a right adaptive unit receives the right microphone data and the noise-enhanced data, and produces a reduced-noise right data output.
  • the right adaptive unit includes a first adaptive filter receiving the noise-enhanced data.
  • a left adaptive unit receives the left microphone data and the noise-enhanced data.
  • the left adaptive unit produces reduced-noise left data output.
  • the left adaptive unit includes a second adaptive filter receiving the noise- enhanced data.
  • Another embodiment of the present invention is a method including the steps of: calculating the noise-enhanced data from the left and right microphone data; adaptively filtering the noise-enhanced data to produce first filtered noise- enhanced data; combining the first filtered noise-enhanced data with right microphone data to produce a reduced-noise right data output; adaptively filtering the noise-enhanced data to produce second filtered noise-enhanced data; and combining the second filtered noise-enhanced data with left microphone data to produce a reduced-noise left data output.
  • FIG. 1 is a diagram of a prior-art microphone system.
  • Fig. 2 is a diagram of a microphone system of one embodiment of the present invention.
  • Fig. 3 is a diagram of one embodiment of the present invention implementing the system of Fig. 2.
  • Fig. 4 is a flow chart illustrating the operation of one embodiment of the present invention.
  • Fig. 2 is a diagram illustrating the functional portions of the microphone system of the present invention.
  • X R ( ) 42 comprises data from the right microphone.
  • X L (n) 44 comprises the data from the left microphone.
  • Data r(n) is derived from these two data signals.
  • a noise-enhanced signal unit 48 produces an output, r(n), with enhanced noise.
  • the noise-enhanced signal unit 48 includes a summing unit 50 which subtracts one of the microphone signals from the other. This produces a noise-enhanced signal since sound coming from the front, including presumably the desired speech signal, will reach the left and right microphones at nearly the same time, thus forming a null in the output response at the front.
  • the noise-enhanced signal is applied to the right adaptive unit 52 and left adaptive unit 54.
  • the right adaptive unit includes a first adaptive filter 56 receiving the noise-enhance signal r(n).
  • the right adaptive unit 52 preferably receives X R (n) as a primary signal to adjust the adaptive filter 56.
  • the unit 52 also includes a summing unit 59, which subtracts the output of the adaptive filter 56 from the right front signal X R (n) to produce the output Z R (n).
  • the left adaptive unit also includes a second adaptive filter 58.
  • the second adaptive filter also receives the reference signal r(n) and produces an output a L (n).
  • the left adaptive unit 54 receives the output X L (n) as the primary signal to modify the adaptive filter coefficients.
  • first and second adaptive filters receive the same reference signal, the coefficients of the adaptive filters are different since the primary and error signals used to modify the coefficients are different for the first and second adaptive filters.
  • the adaptive filters can be of a variety of different types which are known in the art.
  • the adaptive filters use the same algorithm but, as described above, have different primary signals.
  • the received signals at the right ear microphone and the left ear microphone are X R ( ⁇ ) and X L ( ⁇ ) which consist of the target speech parts S R ( ⁇ ), S L ( ⁇ ) and the noise parts N R ( ⁇ ), N L (n), respectively, that is
  • X L (n) S L (n) + N L (n)
  • R(n) [r(n),r(n - 1), ,r(n -N+ l)] ⁇ and N is the length of two adaptive filters. Note that the length of two filters is here selected to be the same for simplicity but could be different.
  • the primary signals at two adaptive filters are X R (n) and X L (n), and two outputs: Z R (n) and Z R (n) for the right ear and the left ear, respectively, are obtained by
  • the learning algorithm can be an adaptive algorithm such as the LS, RLS, TLS and LMS algorithms.
  • the LMS algorithm version to update the coefficients of two adaptive filters is
  • W L (n + 1) W L ( ⁇ ) + XR(n)Z L (n) 2
  • is a step parameter which is a positive constant less than —
  • P is the power of the input r(n) of these two adaptive filters.
  • can be also time varying as done in the normalized LMS algorithm uses, that is,
  • W R (n + ⁇ ) W R (n) ⁇ R(n)Z ⁇ )
  • W L (n + 1) W L ( ⁇ ) ⁇ ⁇ R(n)Z L (n)
  • is a positive constant less than 2.
  • W R kn + 1) W Rl in) + -L R( ⁇ )Z R £ )
  • the advantages of the present invention include the following: First, no binaural cue of the target speech will be lost because two system outputs Z R (n) and Z L (n) will approximate the signal parts S R (n) and S L (n) respectively. In the prior scheme, the binaural cues of signals with higher frequencies than the cut-off frequency have been lost. Second, the array processing benefit for the part with frequencies less than the cut-off frequency will be preserved because the frequency spectrum is not divided. Note that in the prior scheme, this benefit also has been lost.
  • the present invention no low-pass filters and high-pass filters are required, while the prior scheme requires two low-pass filters and two high-pass filters. Consequently, the related equalization processing is not required in the present invention.
  • the only cost involved in implementing the present invention is to include an additional adaptive filter.
  • the two adaptive filters can use the same structure and adaptive algorithm. This property results in a large convenience in hardware implementation because the related assembly code and machine code of two adaptive filters can be shared.
  • Fig. 3 is a diagram that illustrates an embodiment of the system of the present invention.
  • the system 70 includes a right ear microphone 72 and a left ear microphone 74.
  • the converters 76 and 78 convert the analog signals into digital signals which are sent to the processor 80.
  • the processor 80 is a digital signal processor.
  • the processor 80 loads the adaptive microphone array program 82 from memory 84.
  • the adaptive microphone array program can implement the functional blocks as shown in Fig. 2.
  • Other programs 86 such as hearing aid algorithms can also be stored in the memory 84 for loading into the processor 80.
  • the output signals of the processor 80 can be sent to speakers if a hearing-aid system is used.
  • Fig. 4 is a flow chart that illustrates the operation of one embodiment of the present invention.
  • an enhanced noise signal is calculated using left and right microphone samples.
  • This enhanced noise signal can be the reference signal constructed by subtracting one of the microphone samples from the other microphone sample.
  • a noise-enhanced signal and the right ear signal are used to produce a noise-reduced right-ear signal. This is preferably done by adaptively filtering the noise-enhanced signal and combining the filtered signal with the right-ear microphone system to produce an output. The output being used as an error signal for the adaptive filter.
  • the noise-enhanced signal and the left-ear signal are used to produce a noise-reduced left-ear signal.
  • the output of a second adaptive filter is combined with the left-ear signal, producing the noise-reduced left-ear signal.
  • the noise-reduced left-ear signal is then used as an error signal to affect the coefficients of the adaptive filter. Note that the order of steps 92 and 94 is not important.
  • the left and right noise-reduced signals are used to produce coefficients which are used in the adaptive filters used in the producing steps 92 and 94.
  • Any kind of adaptive algorithms such as LMS-based, LS-based, TLS- based, RLS-based and related algorithms, can be used in this scheme.
  • the weights can also be obtained by directly solving the estimated Wienner-Hopf equations.
  • Equations 1 and 2 and adaptive lattice filters can be used in this scheme as well.
  • the lengths of two adaptive filters are adjustable and can take different values.
  • the step parameters in related adaptive algorithms for two adaptive filters can take different values. Trade-offs between performance and cost (complexity, etc.) in practical applications determine which algorithm is used.
  • the two adaptive filters in Fig. 2 can be nonlinear filters and can be implemented by some neural networks such as multi-layer perceptron networks, radial basis function networks, high-order neural networks, etc.
  • the corresponding learning algorithms in neural networks such as the back-propagation algorithm can be used for the adaptive filters.
  • a matching filter could be added in either the left ear channel or the right ear channel before obtaining the difference signal r(n) so as to compensate for the magnitude mismatch of two-ear microphones.
  • the matching filter can be in either a finite impulse response (FIR) filter or an infinite impulse response (IIR) filter.
  • the matching filter could also be either in a fixed model or in an adaptive model.
  • a speech pause detection system is used and the weight update of the two adaptive filters is made during a pause in the speech.
  • Either a directional microphone or an omnidirectional microphone could be used in this invention.
  • Some pre-processing methods could be used to improve the signal-to-noise ratio of two primary signals X R (n) and X L (n). These pre-processing methods include that more than one microphone can be used in each ear and then these microphone signals are combined in some way to produce the signals X R (n) and X n). This pre-processing could be either in a fixed model or in an adaptive model and also either in the spatial domain or in the temporal domain.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Neurosurgery (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic Arrangements (AREA)
EP01942048A 2000-06-13 2001-06-05 Systeme de reseau adaptatif de microphones avec preservation des signaux biauriculaires Expired - Lifetime EP1305975B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US593728 1990-10-05
US59372800A 2000-06-13 2000-06-13
PCT/US2001/018416 WO2002003749A2 (fr) 2000-06-13 2001-06-05 Systeme de reseau adaptatif de microphones avec preservation des signaux biauriculaires

Publications (3)

Publication Number Publication Date
EP1305975A2 true EP1305975A2 (fr) 2003-05-02
EP1305975A4 EP1305975A4 (fr) 2007-09-19
EP1305975B1 EP1305975B1 (fr) 2011-11-23

Family

ID=24375900

Family Applications (1)

Application Number Title Priority Date Filing Date
EP01942048A Expired - Lifetime EP1305975B1 (fr) 2000-06-13 2001-06-05 Systeme de reseau adaptatif de microphones avec preservation des signaux biauriculaires

Country Status (4)

Country Link
EP (1) EP1305975B1 (fr)
AT (1) ATE535103T1 (fr)
DK (1) DK1305975T3 (fr)
WO (1) WO2002003749A2 (fr)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100669820B1 (ko) * 2005-05-16 2007-01-16 이시동 지향성 음파수신장치
WO2007028250A2 (fr) * 2005-09-09 2007-03-15 Mcmaster University Procede et dispositif d'amelioration d'un signal binaural
GB0609248D0 (en) * 2006-05-10 2006-06-21 Leuven K U Res & Dev Binaural noise reduction preserving interaural transfer functions
US8891777B2 (en) 2011-12-30 2014-11-18 Gn Resound A/S Hearing aid with signal enhancement
EP2611215B1 (fr) * 2011-12-30 2016-04-20 GN Resound A/S Aide auditive dotée d'un meilleur signal
WO2015120475A1 (fr) 2014-02-10 2015-08-13 Bose Corporation Systeme d'aide a la conversation
EP3229498B1 (fr) * 2014-12-04 2023-01-04 Gaudi Audio Lab, Inc. Procédé et appareil de traitement de signal audio destiné à un rendu binauriculaire
EP4038901A1 (fr) 2019-09-30 2022-08-10 Widex A/S Procédé pour faire fonctionner un système audio binaural à porter dans ou sur l'oreille et système audio binaural à porter dans ou sur l'oreille

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU3981489A (en) * 1988-07-08 1990-02-05 Adaptive Control Limited Improvements in or relating to sound reproduction systems
US5633935A (en) * 1993-04-13 1997-05-27 Matsushita Electric Industrial Co., Ltd. Stereo ultradirectional microphone apparatus
US5675659A (en) * 1995-12-12 1997-10-07 Motorola Methods and apparatus for blind separation of delayed and filtered sources

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
See also references of WO0203749A2 *
WELKER D P ET AL: "MICROPHONE-ARRAY HEARING AIDS WITH BINAURAL OUTPUT-PART 11: A TWO- MICROPHONE ADAPTIVE SYSTEM" IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING, IEEE SERVICE CENTER, NEW YORK, NY, US, vol. 5, no. 6, November 1997 (1997-11), pages 543-551, XP000785347 ISSN: 1063-6676 *

Also Published As

Publication number Publication date
WO2002003749A2 (fr) 2002-01-10
DK1305975T3 (da) 2012-02-13
EP1305975A4 (fr) 2007-09-19
EP1305975B1 (fr) 2011-11-23
ATE535103T1 (de) 2011-12-15
WO2002003749A3 (fr) 2002-06-20

Similar Documents

Publication Publication Date Title
EP1417756B1 (fr) Traitement adaptatif du signal par sous-bandes dans un banc de filtres surechantillonne
US7933424B2 (en) Hearing aid comprising adaptive feedback suppression system
US9456275B2 (en) Cardioid beam with a desired null based acoustic devices, systems, and methods
US7974428B2 (en) Hearing aid with acoustic feedback suppression
CN1809105B (zh) 适用于小型移动通信设备的双麦克语音增强方法及系统
WO2004045244A1 (fr) Système de microphone suppresseur de bruit adaptatif
WO2001097558A2 (fr) Directionnalite adaptative basee sur un modele polaire fixe
WO2010112073A1 (fr) Annulation adaptative d'échos sur des caractéristiques introduites ou intrinsèques, et récupération correspondante
AU2001283205A1 (en) Method and apparatus for filtering and compressing sound signals
WO2009130513A1 (fr) Système de réduction du bruit à deux microphones
GB2486639A (en) Reducing noise in an environment having a fixed noise source such as a camera
WO2008104446A2 (fr) Procédé de réduction de bruit dans un signal d'entrée d'un dispositif auditif et dispositif auditif
US20230300516A1 (en) Ear-wearable device with active noise cancellation system that uses internal and external microphones
EP1305975B1 (fr) Systeme de reseau adaptatif de microphones avec preservation des signaux biauriculaires
US11153695B2 (en) Hearing devices and related methods
AU778351B2 (en) Circuit and method for the adaptive suppression of noise
CN115643514A (zh) 一种监听系统
CN107431869B (zh) 听力装置
Puder Adaptive signal processing for interference cancellation in hearing aids
CA2397080C (fr) Traitement adaptatif de signaux en sous-bande dans un banc de filtres surechantillonne
Nema et al. Fir filter bank design for audiogram matching
Vashkevich et al. Low-delay hearing aid based on cochlear model with nonuniform subband acoustic feedback cancellation
EP3886463A1 (fr) Procédé au niveau d'un dispositif auditif
US11330376B1 (en) Hearing device with multiple delay paths
JP2973624B2 (ja) 騒音低減ヘッドホン装置

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20030113

AK Designated contracting states

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

RIN1 Information on inventor provided before grant (corrected)

Inventor name: MICHAEL, NICK

Inventor name: LUO, FA-LONG

Inventor name: YANG, JUN

Inventor name: EDWARDS, BRENT

A4 Supplementary search report drawn up and despatched

Effective date: 20070820

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 25/00 20060101ALI20070813BHEP

Ipc: G01S 3/80 20060101ALI20070813BHEP

Ipc: H04R 3/00 20060101AFI20020704BHEP

17Q First examination report despatched

Effective date: 20071115

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: GN RESOUND A/S

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: PETER RUTZ

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 60145704

Country of ref document: DE

Effective date: 20120119

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20111123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120323

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120224

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 535103

Country of ref document: AT

Kind code of ref document: T

Effective date: 20111123

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20120824

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 60145704

Country of ref document: DE

Effective date: 20120824

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120630

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120305

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120605

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20111123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20120605

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 17

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 18

REG Reference to a national code

Ref country code: CH

Ref legal event code: PCAR

Free format text: NEW ADDRESS: ALPENSTRASSE 14 POSTFACH 7627, 6302 ZUG (CH)

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20200623

Year of fee payment: 20

Ref country code: DK

Payment date: 20200616

Year of fee payment: 20

Ref country code: DE

Payment date: 20200618

Year of fee payment: 20

Ref country code: CH

Payment date: 20200618

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20200617

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 60145704

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: EUP

Expiry date: 20210605

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: GB

Ref legal event code: PE20

Expiry date: 20210604

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20210604