EP0957471B1 - Messverfahren zur gehörrichtigen Qualitätsbewertung von Audiosignalen - Google Patents

Messverfahren zur gehörrichtigen Qualitätsbewertung von Audiosignalen Download PDF

Info

Publication number
EP0957471B1
EP0957471B1 EP99106223A EP99106223A EP0957471B1 EP 0957471 B1 EP0957471 B1 EP 0957471B1 EP 99106223 A EP99106223 A EP 99106223A EP 99106223 A EP99106223 A EP 99106223A EP 0957471 B1 EP0957471 B1 EP 0957471B1
Authority
EP
European Patent Office
Prior art keywords
filter
signal
test
signals
smearing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP99106223A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP0957471A2 (de
EP0957471A3 (de
Inventor
Thilo Thiede
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Deutsche Telekom AG
Original Assignee
Deutsche Telekom AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Deutsche Telekom AG filed Critical Deutsche Telekom AG
Publication of EP0957471A2 publication Critical patent/EP0957471A2/de
Publication of EP0957471A3 publication Critical patent/EP0957471A3/de
Application granted granted Critical
Publication of EP0957471B1 publication Critical patent/EP0957471B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
    • G10L25/69—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for evaluating synthetic or decoded voice signals

Definitions

  • the invention relates to a measuring method for aurally quality assessment of audio signals according to the preamble of patent claim 1.
  • the invention has for its object to provide an objective measurement method for aurally quality assessment of audio signals by means of new fast algorithms for calculating linear phase filter, the duration of the audible noise calculated taking into account the temporal change of the envelopes at the individual filter outputs and a ge surgicalangepasste filter bank are used should, whereby an optimal temporal resolution is to be achieved and that while significantly saving of processing time compared to other filter banks.
  • the inventive solution of the problem is characterized in the characterizing part of claim 1.
  • a significant advantage of the method according to the invention is that a more precise hearing model is achieved, since audible disturbances are calculated taking into account the temporal change of the envelopes at the individual filter outputs.
  • a hearing-adapted filter bank is used, whereby an optimal temporal resolution is achieved, and the temporal behavior of the filter (impulse response, etc.) directly corresponds to the level dependence of the transfer functions.
  • the phase information in the filter channels is retained.
  • the folding with smearing function takes place only after the rectification or amount formation.
  • a signal dependence of the filter characteristics is achieved by convoluting the filter outputs in the frequency domain prior to rectification / magnitude formation with a level dependent smear function.
  • the signal components present in the original signal and changed only in their spectral distribution are separated from interferences generated by nonlinearities, the separation taking place by evaluating the orthogonality relationship between the time profiles of the envelopes at corresponding filter outputs of the signal to be evaluated and the original signal.
  • the separation of these noise components corresponds better to the actual hearing impression.
  • the present measuring method evaluates the noise of an audio signal by comparison with an undistorted reference signal.
  • the input signals After filtering with the Transmission functions of the outer and middle ear, the input signals are converted by a gehöangep gratuitte filter bank in a time-tonal representation.
  • the absolute squares of the filter output signals are calculated (rectification) and a convolution of the filter outputs is performed with a smear function.
  • the folding can be done in contrast to the previously known methods before the rectification or even afterwards.
  • Level differences between test and reference signal as well as linear distortions in the test signal are compensated and evaluated separately.
  • a frequency-dependent offset is added to model the self-noise of the ear and there is a temporal smearing of the output signals.
  • the left and right channel test signals 1a, 1b, and the left and right channel reference signals 1c, 1d are respectively applied to pre-filters 2 for pre-filtering.
  • the actual filtering takes place in the filter bank 3.
  • the spectral smearing 4 and the calculation of the squares squares 5.
  • the box marked 6 in the figure symbolically represent the temporal smearing.
  • the level and frequency response equals 7, wherein also output parameters 11 are supplied.
  • the addition of self-noise 8 and then the temporal smearing 9 takes place.
  • the calculation of output parameters 11 takes place in the structure shown in the symbolically represented block 10.
  • the level and Frequenzganganmaschine 7 can also be done between step or operation 9 and 10.
  • the filter bank 3 consists of an arbitrarily selectable number of filter pairs for test and reference signal 1a, b and 1d, c (meaning values between 30 and 200)
  • the filters can be distributed evenly on largely arbitrary pitch scales.
  • a suitable pitch scale is z.
  • z / barque 7 ⁇ arsinh ( ⁇ / Hz 650 )
  • ⁇ 1 2 ⁇ bw and H in the ( t ) cos n ( ⁇ ⁇ bw ⁇ t ) ⁇ sin ( 2 ⁇ ⁇ ⁇ c ⁇ t )
  • the output values of the filter bank 3 are spectrally blurred to account for simultaneous masking at 31 dB / Bark at the lower edge and between -24 and -6 dB / Bark at the upper edge, that is, crosstalk is produced between the filter outputs.
  • the level L is calculated independently for each filter output from the least squares amount 5 of the corresponding output value filtered with a time constant of 10 ms. This blurring is performed independently for the filters representing the real part of the signal (G1.2) and the filters representing the imaginary part (G1.3) of the signal.
  • the level may be calculated without a low-pass filter, and instead the low-pass filtering factor obtained by delogarithmizing the slew rate (G1.4) may be filtered low. Since this convolution operation is quasi linear and therefore preserves the relation between the resulting frequency response and the resulting impulse response, it can be considered as part of the filter bank 3.
  • the temporal smearing of the filter output signals takes place in two stages.
  • the signals are averaged over a cos 2- shaped time window, which primarily models the pre-masking.
  • the after-mask is modeled, which will be described in more detail later.
  • the cos 2- shaped time window has a length of 400 samples at a 48 kHz sampling rate. The distance between the maximum of the time window and its 3 dB point is thus about 100 samples or 2 ms, which corresponds approximately to a time period often assumed for the pre-occlusion.
  • Level differences and linear distortions (frequency responses of the test object) between the test and reference signals 1a, b and 1c, d can be compensated and thus from the evaluation other types of disturbances are separated.
  • the instantaneous absolute squares at the filter outputs are temporally smoothed by first-order low-pass filters.
  • corr total ( ⁇ P test ⁇ P Ref ⁇ P test ) 2
  • the time constants are calculated according to Eq. 6 determined. If ratio f, t is greater than one, the correction factor for the test signal is set to ratio f, t -1 and the correction factor for the reference signal is set to one. In the reverse In the case, the correction factor for the reference signal is set to ratio f, t and the correction factor for the test signal is set to one.
  • correction factors are temporally smoothed over several adjacent filter channels, and with the same time constants, as indicated above.
  • a frequency-dependent offset for modeling the self-noise of the ear is added to the absolute squares at all filter outputs. Another offset to account for background noise can also be added (but normally set to 0).
  • e ( ⁇ c . t ) e ( ⁇ . t ) + 10 0364 ( ⁇ c kHz ) - 0.8
  • the instantaneous absolute squares in each filter channel are time-blurred by a first-order low-pass filter with a time constant of approximately 10 ms.
  • the time constant can also be calculated as a function of the center frequency of the respective filter. In this case it is 50 ms for low frequencies and 8 ms for high frequencies (like G1.6).
  • Figure 11 has here been designed to provide the specific loudness of the disturbance when no masker is present and provides approximately the ratio between the disturbance and the masker when the disturbance is very small relative to the masker.
  • the "throttled noise” corresponds to the mean of this variable over time and filter channels.
  • the resulting output parameter is referred to as the "loudness of missing signal components".
  • linear distortions can also be determined by using the reference signal before signal equalization as the test signal.
  • the modulation difference is averaged over time and filter bands.
  • the modulation used on the input side is obtained by normalizing the time derivative of the instantaneous values to their time-smoothed value.
  • FIG. 2 shows a filter structure for the recursive calculation of a simple finite impulse response (FIR) bandpass filter.
  • FIR finite impulse response
  • the signal is processed separately according to real part (upper path) and imaginary part (lower path). Since the input signal X was originally purely real, the lower path is missing first.
  • the input signal X is delayed by N samples (21) and, after multiplication by a complex-valued factor cos (N, ⁇ ) + j.sin (N, ⁇ ) from the original input signal subtracted (22).
  • the resulting signal V is added to the one-sample delayed output (23).
  • the result multiplied by another complex-valued factor cos ( ⁇ ) + j.sin ( ⁇ ) gives the new output signal Y (24).
  • the swept identifiers for V and Y each mark the imaginary part.
  • the second complex multiplication continues the input signal periodically.
  • the addition of the delayed and weighted by the first complex multiplication input signal aborts the continuation of the input signal after N samples again.
  • f A denotes the sampling frequency
  • the initially low stopband attenuation of these bandpasses can be increased by calculating K + 1 of such bandpass filters with the same impulse response length N but different values of ⁇ in parallel, adapting their phase responses to one another by a further complex multiplication and adding their output signals weighted:
  • 0 ⁇ n ⁇ N for the real part and a K ( n ) sin K ( ⁇ N n ) ⁇ sin ( 2 ⁇ ⁇ ⁇ ⁇ M ⁇ A ⁇ n )

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Measurement Of Mechanical Vibrations Or Ultrasonic Waves (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Filters That Use Time-Delay Elements (AREA)
EP99106223A 1998-05-13 1999-04-12 Messverfahren zur gehörrichtigen Qualitätsbewertung von Audiosignalen Expired - Lifetime EP0957471B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE19821273 1998-05-13
DE19821273A DE19821273B4 (de) 1998-05-13 1998-05-13 Meßverfahren zur gehörrichtigen Qualitätsbewertung von codierten Audiosignalen

Publications (3)

Publication Number Publication Date
EP0957471A2 EP0957471A2 (de) 1999-11-17
EP0957471A3 EP0957471A3 (de) 2004-01-02
EP0957471B1 true EP0957471B1 (de) 2006-02-01

Family

ID=7867531

Family Applications (1)

Application Number Title Priority Date Filing Date
EP99106223A Expired - Lifetime EP0957471B1 (de) 1998-05-13 1999-04-12 Messverfahren zur gehörrichtigen Qualitätsbewertung von Audiosignalen

Country Status (6)

Country Link
US (1) US7194093B1 (da)
EP (1) EP0957471B1 (da)
AT (1) ATE317151T1 (da)
CA (1) CA2271445C (da)
DE (2) DE19821273B4 (da)
DK (1) DK0957471T3 (da)

Families Citing this family (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2001065543A1 (en) * 2000-02-29 2001-09-07 Telefonaktiebolaget Lm Ericsson (Publ) Compensation for linear filtering using frequency weighting factors
US7278289B2 (en) * 2003-04-28 2007-10-09 Sonora Medical Systems, Inc. Apparatus and methods for testing acoustic systems
WO2004107318A1 (en) * 2003-05-27 2004-12-09 Koninklijke Philips Electronics N.V. Audio coding
US20050085316A1 (en) * 2003-10-20 2005-04-21 Exelys Llc Golf ball location system
DE102004029872B4 (de) * 2004-06-16 2011-05-05 Deutsche Telekom Ag Verfahren und Anordnung zur Verbesserung der Qualität bei der Übertragung codierter Audio-/Video-Signale
WO2007098258A1 (en) * 2006-02-24 2007-08-30 Neural Audio Corporation Audio codec conditioning system and method
DE102006025403B3 (de) * 2006-05-31 2007-08-16 Siemens Audiologische Technik Gmbh Verfahren zum Analysieren eines nichtlinearen Signalverarbeitungssystems
KR101600082B1 (ko) * 2009-01-29 2016-03-04 삼성전자주식회사 오디오 신호의 음질 평가 방법 및 장치
CN102422531B (zh) * 2009-06-29 2014-09-03 三菱电机株式会社 音频信号处理装置
US20110015922A1 (en) * 2009-07-20 2011-01-20 Larry Joseph Kirn Speech Intelligibility Improvement Method and Apparatus
US8682621B2 (en) * 2010-07-16 2014-03-25 Micron Technology, Inc. Simulating the transmission of asymmetric signals in a computer system
CN102881289B (zh) * 2012-09-11 2014-04-02 重庆大学 一种基于听觉感知特性的语音质量客观评价方法
CN104361894A (zh) * 2014-11-27 2015-02-18 湖南省计量检测研究院 一种基于输出的客观语音质量评估的方法
CN113077815B (zh) * 2021-03-29 2024-05-14 腾讯音乐娱乐科技（深圳）有限公司 一种音频评估方法及组件

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4450531A (en) * 1982-09-10 1984-05-22 Ensco, Inc. Broadcast signal recognition system and method
US4860360A (en) * 1987-04-06 1989-08-22 Gte Laboratories Incorporated Method of evaluating speech
JPH0398318A (ja) * 1989-09-11 1991-04-23 Fujitsu Ltd 音声符号化方式
US5210820A (en) * 1990-05-02 1993-05-11 Broadcast Data Systems Limited Partnership Signal recognition system and method
DE4431481A1 (de) * 1994-09-03 1996-03-07 Philips Patentverwaltung Schaltungsanordnung mit steuerbarem Übertragungsverhalten
DE4437287C2 (de) * 1994-10-18 1996-10-24 Fraunhofer Ges Forschung Verfahren zur Messung der Erhaltung stereophoner Audiosignale und Verfahren zur Erkennung gemeinsam codierter stereophoner Audiosignale
DE19523327C2 (de) * 1995-06-27 2000-08-24 Siemens Ag Verfahren zur verbesserten Schätzung der Impulsantwort eines Übertragungskanals
DE19647399C1 (de) * 1996-11-15 1998-07-02 Fraunhofer Ges Forschung Gehörangepaßte Qualitätsbeurteilung von Audiotestsignalen

Also Published As

Publication number Publication date
DK0957471T3 (da) 2006-06-06
CA2271445C (en) 2011-02-22
DE19821273B4 (de) 2006-10-05
US7194093B1 (en) 2007-03-20
ATE317151T1 (de) 2006-02-15
DE19821273A1 (de) 1999-12-02
DE59913088D1 (de) 2006-04-13
EP0957471A2 (de) 1999-11-17
EP0957471A3 (de) 2004-01-02
CA2271445A1 (en) 1999-11-13

Similar Documents

Publication Publication Date Title
EP0957471B1 (de) Messverfahren zur gehörrichtigen Qualitätsbewertung von Audiosignalen
DE60317722T2 (de) Verfahren zur Reduzierung von Aliasing-Störungen, die durch die Anpassung der spektralen Hüllkurve in Realwertfilterbanken verursacht werden
DE2919085C2 (de) Vorverarbeitungsverfahren und -vorrichtung für eine Spracherkennungsvorrichtung
DE3506912C2 (da)
EP1741039B1 (de) Informationssignalverarbeitung durch modifikation in der spektral-/modulationsspektralbereichsdarstellung
DE69609850T2 (de) Gleichkanalstörungsreduktion
EP1386307B2 (de) Verfahren und vorrichtung zur bestimmung eines qualitätsmasses eines audiosignals
EP1525576B1 (de) Vorrichtung und verfahren zum erzeugen einer komplexen spektraldarstellung eines zeitdiskreten signals
EP0065210A2 (de) Verfahren zur Aufbereitung elektrischer Signale mit einer digitalen Filteranordnung
DE2749132A1 (de) Rauschfilter-anordnung
WO1998023130A1 (de) Gehörangepasste qualitätsbeurteilung von audiosignalen
DE69317802T2 (de) Verfahren und Vorrichtung für Tonverbesserung unter Verwendung von Hüllung von multibandpassfiltrierten Signalen in Kammfiltern
EP0485390B1 (de) Verfahren zur übertragung eines signals
DE69803202T2 (de) Verfahren und vorrichtung zur sprachdetektion
WO2015049332A1 (de) Ableitung von multikanalsignalen aus zwei oder mehreren grundsignalen
DE19505435C1 (de) Verfahren und Vorrichtung zum Bestimmen der Tonalität eines Audiosignals
EP1239455A2 (de) Verfahren und Anordnung zur Durchführung einer an die Übertragungsfunktion menschilcher Sinnesorgane angepassten Fourier Transformation sowie darauf basierende Vorrichtungen zur Geräuschreduktion und Spracherkennung
DE10157535B4 (de) Verfahren und Vorrichtung zur Reduzierung zufälliger, kontinuierlicher, instationärer Störungen in Audiosignalen
EP1453355B1 (de) Signalverarbeitung in einem Hörgerät
EP1458216B1 (de) Vorrichtung und Verfahren zur Adaption von Hörgerätemikrofonen
DE3230391C2 (da)
DE69802431T2 (de) Verfahren und vorrichtung zur verbesserung eines digitalen sprachsignals
EP0669606A2 (de) Verfahren zur Geräuschreduktion eines gestörten Sprachsignals
EP0535425B1 (de) Verfahren zur Verstärkung von akustischen Signalen für Hörbehinderte, sowie Vorrichtung zur Durchführung des Verfahrens
DE19516063C2 (de) Verfahren zur Prüfung von Sendefrequenzen und zur Vergabe von Frequenzen für Basisstationen in einem zellularen Funknetz

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

RIC1 Information provided on ipc code assigned before grant

Ipc: 7G 10L 19/00 A

17P Request for examination filed

Effective date: 20040702

AKX Designation fees paid

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

17Q First examination report despatched

Effective date: 20050404

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20060201

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060201

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060201

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060412

REF Corresponds to:

Ref document number: 59913088

Country of ref document: DE

Date of ref document: 20060413

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060430

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060430

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060430

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060512

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20060502

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060703

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20061103

BERE Be: lapsed

Owner name: DEUTSCHE TELEKOM A.G.

Effective date: 20060430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060412

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060201

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 18

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 19

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20180423

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20180423

Year of fee payment: 20

Ref country code: DK

Payment date: 20180424

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20180424

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20180424

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20180403

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 59913088

Country of ref document: DE

REG Reference to a national code

Ref country code: DK

Ref legal event code: EUP

Effective date: 20190412

REG Reference to a national code

Ref country code: NL

Ref legal event code: MK

Effective date: 20190411

REG Reference to a national code

Ref country code: GB

Ref legal event code: PE20

Expiry date: 20190411

REG Reference to a national code

Ref country code: SE

Ref legal event code: EUG

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20190411