EP0874352A2 - Détection d'une activité vocale - Google Patents

Détection d'une activité vocale Download PDF

Info

Publication number
EP0874352A2
EP0874352A2 EP98102842A EP98102842A EP0874352A2 EP 0874352 A2 EP0874352 A2 EP 0874352A2 EP 98102842 A EP98102842 A EP 98102842A EP 98102842 A EP98102842 A EP 98102842A EP 0874352 A2 EP0874352 A2 EP 0874352A2
Authority
EP
European Patent Office
Prior art keywords
speech
output
circuit
switch
background noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP98102842A
Other languages
German (de)
English (en)
Other versions
EP0874352B1 (fr
EP0874352A3 (fr
Inventor
Joachim Dipl.-Ing. Stegmann
Gerhard Dipl.-Ing. Schröder
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Deutsche Telekom AG
Original Assignee
Deutsche Telekom AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Deutsche Telekom AG filed Critical Deutsche Telekom AG
Publication of EP0874352A2 publication Critical patent/EP0874352A2/fr
Publication of EP0874352A3 publication Critical patent/EP0874352A3/fr
Application granted granted Critical
Publication of EP0874352B1 publication Critical patent/EP0874352B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/27—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the analysis technique

Definitions

  • the invention relates to a method and Circuit arrangement for automatic Voice activity recognition according to the generic term of Claims 1 and 5 respectively.
  • Known methods for automatic Speech activity detection usually use Decision parameters based on time averages Based windows of constant length.
  • the autocorrelation coefficients Called zero crossing rate or basic speech period, where these parameters have limited flexibility in the Have selection of time / frequency domain resolution, which is usually determined by the frame length of the associated Speech encoder / decoder is fixed.
  • the well-known wavelet transformation calculates a breakdown into the time / frequency domain too low frequency but high time domain resolution at high frequencies and too low time - but high Frequency domain resolution at low frequencies leads.
  • the invention is therefore based on the object Method and a circuit arrangement for Voice activity detection based on the wavelet transformation to create, it should be decided whether language for the time segment to be considered at all or speech sounds.
  • the present procedure for automatic Speech activity detection for speech encoders / decoders for source-controlled reduction of the middle one Transmission rate is characterized in that after the Segmentation of the speech signal for each frame one Wavelet transformation is calculated from the one sentence Parameters are determined from which using fixed Thresholds are calculated using a set of binary decision variables that control a decision logic, the result After smoothing the time, a statement for each frame "Language available / no language available" returns. In that it is determined whether for the contemplating time segment there is language at all a source-controlled reduction of the middle one Transfer rate reached.
  • the procedure decides whether to do so contemplating time segment there is language at all. This allows it to function control or in general as Preliminary stage for a variable-rate speech coder / decoder Bit rate can be used.
  • 1 is a block diagram for voice activity recognition as a preliminary stage for one variable rate speech encoder / decoder shown on its input 1 receives the respective input language.
  • the input language is both on lines 2 and 3 a switch 4 as well as the input of a Voice activity detection circuit or module 5 given.
  • the switch 4 directs the input language depending on the output signal of the voice activity detection circuit 5, which is used to control the switch 4 is connected to it via a feedback line 6, either on line 7 or on line 8. Die Line 7 leads to a speech encoder 9 and the line 8 to a background noise encoder 10.
  • the bit stream of the speech encoder 9 is connected via a line 11 to the given an input of a switch 13 and the bit stream the background noise encoder 10 via a line 12 to the other input of the switch 13.
  • the switch 13 also receives the output signals via a line 14 the voice activity detection circuit 5, whereby the Switch 13 is controlled.
  • the output of the switch 13 is connected to a transmission channel 16 via a line 15 connected, also on the input side to line 14 for the output signals of the voice activity detection module 5 is connected.
  • the output of the transmission channel 16 reaches the entrance once via a line 17 a further switch 19 and via a line 18 the control input of the switch 19 and the control input a switch 26.
  • the switch 19 is over Output lines 20 and 21 with a speech decoder 22 and with a background noise decoder 23 connected, the outputs on lines 24 and 25 on the Get input of the switch 26 already mentioned, the depending on the control signals on line 18 at the output 27 either signals for the decoded speech or the provides decoded background noise.
  • FIG. 2 is a block diagram of an automatic Voice activity recognizer represented, which in turn on its input 1 receives the input language and to one Segmentation circuit 28 passes on.
  • the exit of this Segmentation circuit 28 is connected via a line 29 transmit a wavelet transform circuit 30 which again via a line 31 with the input of a Processor 32 connected to calculate the energy quantities is.
  • the output of processor 32 or one Computing circuit is connected in parallel with a via a line 33 Pause detector 34, with a circuit for calculating a Stationarity measure 35, with a first Background detector 36 and with a second Background detector 37 connected.
  • the outputs of the mentioned circuits 34 to 37 are about corresponding Lines 38 to 41 with decision logic 42 connected, the output via a line 43 with a Smoothing circuit 44 connected for temporal smoothing whose output 45 is also the output of the speech activity recognizer is.
  • the M 2 L-2 past and the M 2 L-2 future samples of the speech frame are also taken into account and the filter impulse responses - as far as possible - centered around the temporal origin. This increases the algorithmic delay of the method in principle by M 2 L-2 samples. If this is to be avoided, the input frame can alternatively be continued periodically or symmetrically.
  • the frame energies E 1 ..., E L of the detail coefficients D 1 , ..., D L and the frame energy E L + 1 of the approximation coefficients A L are calculated by the processor 32.
  • the total energy of the frame E tot can now be determined efficiently by summing all partial energies if the underlying wavelet basis is orthogonal. All energy values are represented in the logarithmic range.
  • a binary decision variable f sil is set according to the following formula:
  • the binary decision variable f stat is now set using the threshold T 2 taking into account the last K frames:
  • the goal is to obtain a decision criterion that is insensitive to the current level of background noise.
  • the properties of the DWT or wavelet transformation circuit 30 are used efficiently for this by considering the detail coefficients D Q1 in the coarse time interval N and the detail coefficients D Q2 in the finer time interval N / P.
  • P denotes the number of subframes, Q1 a level for coarse and Q2 a level for fine time resolution, whereby the relationships Q1, Q2 ⁇ ⁇ 1, L ⁇ and Q1> Q2 must apply.
  • an estimate B i , i ⁇ ⁇ Q1, Q2 ⁇ for the current level of the background noise is carried out for both stages calculated, the time constant ⁇ being limited by 0 ⁇ ⁇ 1.
  • the P subframe energies ⁇ ( k ,1) Q2 , ..., ⁇ ( k , P ) Q2 determined from the detailed coefficients D 2 and, using the fixed thresholds T 3 and T 4, each determine a binary decision variable f Q1 for stage Q1 and f Q2 for stage Q2 according to the following two formulas:
  • the temporal smoothing takes place in the circuit 44.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Time-Division Multiplex Systems (AREA)
  • Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Geophysics And Detection Of Objects (AREA)
  • Measuring Pulse, Heart Rate, Blood Pressure Or Blood Flow (AREA)
EP98102842A 1997-04-22 1998-02-19 Détection d'une activité vocale Expired - Lifetime EP0874352B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE19716862 1997-04-22
DE19716862A DE19716862A1 (de) 1997-04-22 1997-04-22 Sprachaktivitätserkennung

Publications (3)

Publication Number Publication Date
EP0874352A2 true EP0874352A2 (fr) 1998-10-28
EP0874352A3 EP0874352A3 (fr) 1999-06-02
EP0874352B1 EP0874352B1 (fr) 2003-10-15

Family

ID=7827317

Family Applications (1)

Application Number Title Priority Date Filing Date
EP98102842A Expired - Lifetime EP0874352B1 (fr) 1997-04-22 1998-02-19 Détection d'une activité vocale

Country Status (4)

Country Link
US (1) US6374211B2 (fr)
EP (1) EP0874352B1 (fr)
AT (1) ATE252265T1 (fr)
DE (2) DE19716862A1 (fr)

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1279164A1 (fr) 2000-04-28 2003-01-29 Deutsche Telekom AG Procede de calcul d'une decision d'activite vocale (detecteur d'activite vocale)
DE10026872A1 (de) 2000-04-28 2001-10-31 Deutsche Telekom Ag Verfahren zur Berechnung einer Sprachaktivitätsentscheidung (Voice Activity Detector)
US7505594B2 (en) * 2000-12-19 2009-03-17 Qualcomm Incorporated Discontinuous transmission (DTX) controller system and method
US6725191B2 (en) * 2001-07-19 2004-04-20 Vocaltec Communications Limited Method and apparatus for transmitting voice over internet
US8315865B2 (en) * 2004-05-04 2012-11-20 Hewlett-Packard Development Company, L.P. Method and apparatus for adaptive conversation detection employing minimal computation
US7574353B2 (en) * 2004-11-18 2009-08-11 Lsi Logic Corporation Transmit/receive data paths for voice-over-internet (VoIP) communication systems
US7983922B2 (en) * 2005-04-15 2011-07-19 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V. Apparatus and method for generating multi-channel synthesizer control signal and apparatus and method for multi-channel synthesizing
KR100655953B1 (ko) * 2006-02-06 2006-12-11 한양대학교 산학협력단 웨이블릿 패킷 변환을 이용한 음성 처리 시스템 및 그 방법
US7680657B2 (en) * 2006-08-15 2010-03-16 Microsoft Corporation Auto segmentation based partitioning and clustering approach to robust endpointing
KR100789084B1 (ko) 2006-11-21 2007-12-26 한양대학교 산학협력단 웨이블릿 패킷 영역에서 비선형 구조의 과중 이득에 의한음질 개선 방법
US9361883B2 (en) * 2012-05-01 2016-06-07 Microsoft Technology Licensing, Llc Dictation with incremental recognition of speech
CN104019885A (zh) 2013-02-28 2014-09-03 杜比实验室特许公司 声场分析系统
US9979829B2 (en) 2013-03-15 2018-05-22 Dolby Laboratories Licensing Corporation Normalization of soundfield orientations based on auditory scene analysis
US10917611B2 (en) 2015-06-09 2021-02-09 Avaya Inc. Video adaptation in conferencing using power or view indications
WO2020252782A1 (fr) * 2019-06-21 2020-12-24 深圳市汇顶科技股份有限公司 Procédé de détection de voix, dispositif de détection de voix, puce de traitement de voix et appareil électronique

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5459814A (en) * 1993-03-26 1995-10-17 Hughes Aircraft Company Voice activity detector for speech signals in variable background noise
EP0751495A2 (fr) * 1995-06-30 1997-01-02 Deutsche Telekom AG Procédé et dispositif de codage de la parole

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5152007A (en) * 1991-04-23 1992-09-29 Motorola, Inc. Method and apparatus for detecting speech
GB2272554A (en) * 1992-11-13 1994-05-18 Creative Tech Ltd Recognizing speech by using wavelet transform and transient response therefrom
US5388182A (en) * 1993-02-16 1995-02-07 Prometheus, Inc. Nonlinear method and apparatus for coding and decoding acoustic signals with data compression and noise suppression using cochlear filters, wavelet analysis, and irregular sampling reconstruction
JP3090842B2 (ja) * 1994-04-28 2000-09-25 沖電気工業株式会社 ビタビ復号法に適応した送信装置
FR2727236B1 (fr) * 1994-11-22 1996-12-27 Alcatel Mobile Comm France Detection d'activite vocale
US5822726A (en) * 1995-01-31 1998-10-13 Motorola, Inc. Speech presence detector based on sparse time-random signal samples
DE19538852A1 (de) * 1995-06-30 1997-01-02 Deutsche Telekom Ag Verfahren und Anordnung zur Klassifizierung von Sprachsignalen
CA2188369C (fr) * 1995-10-19 2005-01-11 Joachim Stegmann Methode et dispositif de classification de signaux vocaux

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5459814A (en) * 1993-03-26 1995-10-17 Hughes Aircraft Company Voice activity detector for speech signals in variable background noise
EP0751495A2 (fr) * 1995-06-30 1997-01-02 Deutsche Telekom AG Procédé et dispositif de codage de la parole

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
"Digital cellular telecommunications system; Discontinuous Transmission (DTX) for Enhanced Full Rate (EFR) speech traffic channels (GSM 06.81)" EUROPEAN TELECOMMUNICATION STANDARD, FINAL DRAFT PRETS 300 729,November 1996, XP002098616 European Telecommunications Standards Institute (ETSI) *
BENYASSINE A ET AL: "ITU-T RECOMMENDATION G.729 ANNEX B: A SILENCE COMPRESSION SCHEME FOR USE WITH G.729 OPTIMIZED FOR V.70 DIGITAL SIMULTANEOUS VOICE AND DATA APPLICATIONS" IEEE COMMUNICATIONS MAGAZINE, Bd. 35, Nr. 9, September 1997, Seiten 64-73, XP000704425 *
STEGMANN J ET AL: "ROBUST VOICE-ACTIVITY DETECTION BASED ON THE WAVELET TRANSFORM" PROCEEDINGS OF THE IEEE WORKSHOP ON SPEECH CODING FOR TELECOMMUNICATIONS, 7. September 1997, Seiten 99-100, XP002073237 *

Also Published As

Publication number Publication date
US20010014854A1 (en) 2001-08-16
ATE252265T1 (de) 2003-11-15
EP0874352B1 (fr) 2003-10-15
EP0874352A3 (fr) 1999-06-02
DE19716862A1 (de) 1998-10-29
US6374211B2 (en) 2002-04-16
DE59809897D1 (de) 2003-11-20

Similar Documents

Publication Publication Date Title
EP0874352B1 (fr) Détection d'une activité vocale
DE69926851T2 (de) Verfahren und Vorrichtung zur Sprachaktivitätsdetektion
EP0690436B1 (fr) Détection de début et de la fin des mots pour la reconnaissence de mots
EP1386307B2 (fr) Procede et dispositif pour determiner un niveau de qualite d'un signal audio
DE3101851C2 (de) Vorrichtung zum Erkennen von Sprache
EP0076234B1 (fr) Procédé et dispositif pour traitement digital de la parole réduisant la redondance
DE4320990A1 (de) Verfahren zur Redundanzreduktion
DE69918635T2 (de) Vorrichtung und Verfahren zur Sprachverarbeitung
DE102004023824B4 (de) Vorrichtung und Verfahren zur Beurteilung einer Güteklasse eines zu prüfenden Objekts
DE10234130B3 (de) Vorrichtung und Verfahren zum Erzeugen einer komplexen Spektraldarstellung eines zeitdiskreten Signals
DE69920047T2 (de) Detektion von reiner sprache in einem audio signal, mit hilfe einer detektionsgrösse (valley percentage)
EP0815553B1 (fr) Methode de detection d'une pause de signal entre deux modeles presents dans un signal de mesure variable en temps
DE602004004572T2 (de) Verfolgen von Vokaltraktresonanzen unter Verwendung einer zielgeführten Einschränkung
DE69922769T2 (de) Vorrichtung und Verfahren zur Sprachverarbeitung
DE19581667C2 (de) Spracherkennungssystem und Verfahren zur Spracherkennung
DE19500494A1 (de) Merkmalsextraktionsverfahren für ein Sprachsignal
DE60018690T2 (de) Verfahren und Vorrichtung zur Stimmhaft-/Stimmlos-Entscheidung
EP1755110A2 (fr) Procédé et dispositif destinés à la réduction adaptative de signaux de bruit et de fond dans un système de traitement vocal
DE60025333T2 (de) Sprachdetektion mit stochastischer konfidenzmassbewertung des frequenzspektrums
EP0658878B1 (fr) Système pour transmettre un signal de parole
WO2001084536A1 (fr) Procede de calcul d'une decision d'activite vocale (detecteur d'activite vocale)
DE60110541T2 (de) Verfahren zur Spracherkennung mit geräuschabhängiger Normalisierung der Varianz
WO2019007820A1 (fr) Dispositif et procédé de détermination de l'état d'une broche d'une machine-outil
EP1279168B1 (fr) Procede pour ameliorer la qualite sonore de transmission de la parole
DE10244699B4 (de) Verfahren zur Bestimmung der Sprachaktivität

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

17P Request for examination filed

Effective date: 19991202

AKX Designation fees paid

Free format text: AT BE CH DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

GRAH Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOS IGRA

RIC1 Information provided on ipc code assigned before grant

Ipc: 7G 10L 11/02 A

RIC1 Information provided on ipc code assigned before grant

Ipc: 7G 10L 11/02 A

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20031015

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20031015

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20031015

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20031015

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: GERMAN

REF Corresponds to:

Ref document number: 59809897

Country of ref document: DE

Date of ref document: 20031120

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040115

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040115

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20040115

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20040123

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20040219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20040228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20040229

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20040229

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20040716

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20040315

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20160218

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20160222

Year of fee payment: 19

Ref country code: NL

Payment date: 20160222

Year of fee payment: 19

Ref country code: BE

Payment date: 20160222

Year of fee payment: 19

Ref country code: AT

Payment date: 20160218

Year of fee payment: 19

Ref country code: FR

Payment date: 20160222

Year of fee payment: 19

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170228

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 59809897

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20170301

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 252265

Country of ref document: AT

Kind code of ref document: T

Effective date: 20170219

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20170219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170301

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20171031

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170901

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170228

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20170228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170219