EP1388146B1 - Verfahren zur codierung und zur übertragung von sprachsignalen - Google Patents

Verfahren zur codierung und zur übertragung von sprachsignalen Download PDF

Info

Publication number
EP1388146B1
EP1388146B1 EP02740316A EP02740316A EP1388146B1 EP 1388146 B1 EP1388146 B1 EP 1388146B1 EP 02740316 A EP02740316 A EP 02740316A EP 02740316 A EP02740316 A EP 02740316A EP 1388146 B1 EP1388146 B1 EP 1388146B1
Authority
EP
European Patent Office
Prior art keywords
amplification factor
adaptive
signal
codebook
fixed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP02740316A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1388146A2 (de
Inventor
Tim Fingscheidt
Herve Taddei
Imre Varga
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens AG
Original Assignee
Siemens AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens AG filed Critical Siemens AG
Publication of EP1388146A2 publication Critical patent/EP1388146A2/de
Application granted granted Critical
Publication of EP1388146B1 publication Critical patent/EP1388146B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
    • G10L19/083—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being an excitation gain
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes

Definitions

  • the invention relates to a method for coding speech signals, in particular with the inclusion of a plurality of codebooks, via the entries of which the speech signal is approximated, and a method for the transmission of speech signals.
  • voice coding methods are used to lower the bit rate to be transmitted.
  • the speech coding methods usually provide a bit stream of speech coded bits divided into frames each representing, for example, 20 ms of the speech signal.
  • the bits within a frame generally represent a particular set of parameters.
  • a frame is subdivided into subframes so that some parameters are transmitted once per frame, others once per subframe.
  • EFR Enhanced Full Rate
  • the entries of a codebook are generally called codewords or codevectors.
  • the adaptive codebook is called “adaptive” because the codevectors contained in it do not represent constants or are stored, but are adaptively determined for each subframe from the past of the total excitation signal of the LPC synthesis filter.
  • the fixed codebook is "fixed” insofar as its codevectors are either permanently stored (noise excitation) or at least calculated using deterministic calculation rules (algebraic codebook) which are not dependent on the respective subframe are.
  • the respective assigned gain factors are usually also referred to as “adaptive” or "fixed”. It should be noted that all 4 types of parameters, adaptive and fixed excitation signal, as well as adaptive and fixed gain, are of course to be determined in each subframe, and in this sense are all "adaptive in nature".
  • first amplification factor should be used instead of "adaptive amplification factor” and the term second amplification factor should be used instead of "fixed amplification factor”.
  • the excitation signal S ' should after LPC synthesis filtering as closely as possible to the occurring at that time speech section, the speech signal S reflected.
  • the parameters g_1, g_2, S_a, S_f are thus selected so that the speech signal S can be represented as well as possible.
  • the excitation signal S ' g_1 * S_a + g_2 * S_f thus approximates the speech signal after LPC synthesis filtering on the receiver side.
  • Speech signals contain sequences of frames or subframes in which they can be modeled as stationary, ie without temporal development of their statistical properties. These are periodic sections that can represent vowels, for example. This periodicity flows into the entire excitation signal S 'via the contribution g_1 * S_a.
  • the statistical properties of a frame or subframe with an onset can not be estimated from past frames or subframes.
  • no long-term periodicity can be determined, that is to say the value of a basic speech frequency is completely meaningless and useless.
  • the contribution made up of the adaptive gain factor and the adpative codebook entry, which in fact expresses long-term periodicity in the speech signal, is therefore more of a hindrance in onsets than useful for coding the speech signal segment.
  • the contribution of an adaptive excitation signal to the total excitation signal in onsets can really harm: If there is no periodicity at all, ie no suitable adaptive excitation signal in the context of the adaptive codebook search, then the optimal adaptive amplification factor is zero.
  • g_1 and g_2 are quantized as a pair of numbers (g_1, g_2) by means of another codebook for the gain factors.
  • a parallel, interdependent quantization of the parameters is called vector quantization.
  • the number pairs (g_1, g_2) are used as an entry in this codebook, through whose entirety, ie number pairs with index 0-127, all possible combinations of g_1 and g_2 occur in the best possible way. These are then conventionally available to a so-called vector quantization.
  • an adaptive amplification factor g_1 0
  • arbitrary values of the fixed amplification factor g_2 can occur, since with nonperiodic speech segments, as already explained, the adaptive component g_1 * S_a is substantially smaller than the fixed component, thus the excitation signal S 'for the LPC synthesis Filter is determined by the latter and the fixed proportion in this case can not be calculated from past values.
  • GSM-EFR GSM Enhanced Full Rate Coder
  • a further disadvantage here is that no additional bits are available in order to quantize the fixed excitation or the fixed amplification factor more accurately.
  • the bits of the adaptive codebook that is the basic speech frequency, remain unused in the event that the adaptive gain has been set to zero.
  • GSM-HR GSM half-rate coder
  • a variable-rate multimodal speech coder with gain-matched analysis-by-synthesis discloses a speech coding method in which the input signal is classified into one of four coding modes. In "Unvoiced Mode” the adaptive codebook is not used for the excitation; the bit rate drops accordingly.
  • the present invention is therefore based on the object of specifying a method for coding and for transmission, which works memory-saving, efficient and less susceptible to error, in particular complexity and coding-efficient runs and at the same time has a high signal quality after decoding.
  • the value of the first amplification factor which is assigned to an adaptive codebook, is set.
  • the speech signal is decomposed into individual time segments. These sections may represent, for example, frames or subframes.
  • the signal classifier indicates whether there is a stationary or a non-stationary speech section, that is, whether it is a voice onset, for example.
  • a value determined by the signal classifier can be assigned to the first amplification factor. For example, by appropriate indexing, this value of the first gain factor can be set such that this representation of the value requires fewer bits than a conventional representation.
  • this value of the first gain factor can be set such that this representation of the value requires fewer bits than a conventional representation.
  • this method proves to be advantageous if the first gain factor is set to zero.
  • the quality of the speech-decoded signal is increased as, for example, as explained above, fewer quantization error signal components occur in the case of non-stationary speech sections.
  • the second amplification factor is scalar quantized if the first amplification factor is fixed. For example, then the Resolution of the quantization of the second amplification factor can be increased.
  • an extended value range may be permitted for the second gain factor, allowing a more detailed description of such a speech signal portion.
  • the coder operates at a fixed data rate, that is, a fixed amount of data is provided for a section of a speech signal.
  • a fixed data rate that is, a fixed amount of data is provided for a section of a speech signal.
  • the invention relates to a method for transmitting speech signals, which are encoded according to one of claims 1 or 2. It is essential here that the first gain and / or the adaptive codebook entry is not transmitted.
  • this method has advantages when the receiver, for example the decoder, is informed by information that this reduction has been made in the data volume to represent individual parameters.
  • This information may, for example, occupy a portion of the amount of data not covered by the reduction, or in addition to the amount of data of the frame or subframe.
  • FIG. 1 shows the schematic sequence of a speech coding according to the analysis-by-synthesis principle.
  • the original speech signal 10 is compared with a synthesized speech signal 11.
  • the synthesized speech signal 11 should be such that the deviation between the synthesized speech signal 11 and the original speech signal 10 is minimal. This deviation is optionally weighted spectrally. This is done via a weighting filter W (z).
  • the synthesized speech signal is produced by means of an LPC synthesis filter H (z). This synthesis filter is excited via an excitation signal 12. The parameters of this excitation signal 12 (and possibly also the coefficients of the LPC synthesis filter) are ultimately transmitted and should therefore be encoded as efficiently as possible.
  • the invention thus aims at the most efficient representation of the parameters which describe the excitation generator.
  • FIG. 2 shows the excitation generator without a downstream LPC synthesis filter in detail.
  • the excitation signal 12 is composed of an adaptive component, by means of which predominantly periodic speech segments are represented, and a fixed component, which serves to represent non-periodic segments. This has already been explained in detail at the beginning.
  • the entries of the adaptive codebook 1 are defined by the preceding speech sections. This is done via a feedback loop 2.
  • the first gain factor 3 is determined by the adaptation to the original speech signal 10.
  • the fixed codebook 4 contains, as the name implies, entries which are not determined by a preceding period.
  • Each entry in the codebook, the so-called codeword, an algebraic codevector, is a pulse sequence which has values not equal to 0 at only a few defined times. This entry or excitation sequence is selected by means of which the deviation of the synthesized signal 11 from the original speech signal 10 is minimized.
  • the gain codebook 5 associated with the fixed codebook is set accordingly.
  • each of these possible implementations is that a lower number of bits can be transmitted compared to the state-of-the-art.
  • these bits can now be used to enhance the quantization of the fixed gain, and / or the quantization of the fixed stimulus, and / or the quantization of the LPC coefficients.
  • any remaining codec parameter can potentially benefit from improved quantization.
  • no new parameter is provided (ie no second fixed codebook), but instead the improved quantization of already existing parameters. This saves computational complexity, memory requirements, and enables the consideration of specific characteristics of subframes with onsets.
  • memory-efficient coding can also be performed.
  • a clever embedding of the additional freed bits is briefly outlined below.
  • the zeroing of the adaptive excitation is signaled by a reserved word in the adaptive codebook.
  • the fixed gain previously 7-bit common with the adaptive Amplification factor vector quantized, for example, quantized scalar scaled with 5 bits at approximately the same quantization error.
  • the 5-bit quantized values of the fixed gain could result from a 25% subset of the 7-bit vector codebook, a subset that can be addressed with any 5 bits from the 7-bit.
  • Such a realization of the 5-bit scalar quantizer saves additional memory.
  • the freed-up 2 bits can now be used, for example, for more accurate quantization of the fixed excitation.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
EP02740316A 2001-05-18 2002-05-02 Verfahren zur codierung und zur übertragung von sprachsignalen Expired - Lifetime EP1388146B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10124420 2001-05-18
DE10124420A DE10124420C1 (de) 2001-05-18 2001-05-18 Verfahren zur Codierung und zur Übertragung von Sprachsignalen
PCT/DE2002/001598 WO2002095734A2 (de) 2001-05-18 2002-05-02 Verfahren zur steuerung des verstärkungsfaktors eines prädiktiven sprachkodieres

Publications (2)

Publication Number Publication Date
EP1388146A2 EP1388146A2 (de) 2004-02-11
EP1388146B1 true EP1388146B1 (de) 2007-11-28

Family

ID=7685379

Family Applications (1)

Application Number Title Priority Date Filing Date
EP02740316A Expired - Lifetime EP1388146B1 (de) 2001-05-18 2002-05-02 Verfahren zur codierung und zur übertragung von sprachsignalen

Country Status (5)

Country Link
US (1) US20040148162A1 (zh)
EP (1) EP1388146B1 (zh)
CN (1) CN100508027C (zh)
DE (2) DE10124420C1 (zh)
WO (1) WO2002095734A2 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2486610C2 (ru) * 2008-12-31 2013-06-27 Хуавэй Текнолоджиз Ко., Лтд. Способ кодирования сигнала и способ декодирования сигнала

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102005000828A1 (de) 2005-01-05 2006-07-13 Siemens Ag Verfahren zum Codieren eines analogen Signals
US7546237B2 (en) * 2005-12-23 2009-06-09 Qnx Software Systems (Wavemakers), Inc. Bandwidth extension of narrowband speech
MY152845A (en) * 2006-10-24 2014-11-28 Voiceage Corp Method and device for coding transition frames in speech signals
CN101286319B (zh) * 2006-12-26 2013-05-01 华为技术有限公司 改进语音丢包修补质量的语音编码方法
US8688437B2 (en) 2006-12-26 2014-04-01 Huawei Technologies Co., Ltd. Packet loss concealment for speech coding
US8515767B2 (en) * 2007-11-04 2013-08-20 Qualcomm Incorporated Technique for encoding/decoding of codebook indices for quantized MDCT spectrum in scalable speech and audio codecs
EP2951820B1 (en) * 2013-01-29 2016-12-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for selecting one of a first audio encoding algorithm and a second audio encoding algorithm
WO2015055531A1 (en) 2013-10-18 2015-04-23 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Concept for encoding an audio signal and decoding an audio signal using speech related spectral shaping information
KR20160070147A (ko) * 2013-10-18 2016-06-17 프라운호퍼 게젤샤프트 쭈르 푀르데룽 데어 안겐반텐 포르슝 에. 베. 결정론적 및 잡음 유사 정보를 사용하는 오디오 신호의 인코딩 및 오디오 신호의 디코딩을 위한 개념
FR3013496A1 (fr) * 2013-11-15 2015-05-22 Orange Transition d'un codage/decodage par transformee vers un codage/decodage predictif

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2312360A (en) * 1996-04-12 1997-10-22 Olympus Optical Co Voice Signal Coding Apparatus

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5657418A (en) * 1991-09-05 1997-08-12 Motorola, Inc. Provision of speech coder gain information using multiple coding modes
SE504397C2 (sv) * 1995-05-03 1997-01-27 Ericsson Telefon Ab L M Metod för förstärkningskvantisering vid linjärprediktiv talkodning med kodboksexcitering
US6330531B1 (en) * 1998-08-24 2001-12-11 Conexant Systems, Inc. Comb codebook structure
US6104992A (en) * 1998-08-24 2000-08-15 Conexant Systems, Inc. Adaptive gain reduction to produce fixed codebook target signal
US6192335B1 (en) * 1998-09-01 2001-02-20 Telefonaktieboiaget Lm Ericsson (Publ) Adaptive combining of multi-mode coding for voiced speech and noise-like signals
US6493664B1 (en) * 1999-04-05 2002-12-10 Hughes Electronics Corporation Spectral magnitude modeling and quantization in a frequency domain interpolative speech codec system
US6691092B1 (en) * 1999-04-05 2004-02-10 Hughes Electronics Corporation Voicing measure as an estimate of signal periodicity for a frequency domain interpolative speech codec system
US6574593B1 (en) * 1999-09-22 2003-06-03 Conexant Systems, Inc. Codebook tables for encoding and decoding
US6782360B1 (en) * 1999-09-22 2004-08-24 Mindspeed Technologies, Inc. Gain quantization for a CELP speech coder
US6510407B1 (en) * 1999-10-19 2003-01-21 Atmel Corporation Method and apparatus for variable rate coding of speech

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2312360A (en) * 1996-04-12 1997-10-22 Olympus Optical Co Voice Signal Coding Apparatus

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
PAKSOY E.; MCCREE A.; VISWANATHAN V.: "A VARIABLE-RATE MULTIMODAL SPEECH CODER WITH GAIN-MATCHED ANALYSIS-SYNTHESIS", IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING.SPEECH PROCESSING. MUNICH, APR. 21 - 24, 1997, IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING (ICASSP), 21 April 1997 (1997-04-21), pages 751 - 754, XP000822556 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
RU2486610C2 (ru) * 2008-12-31 2013-06-27 Хуавэй Текнолоджиз Ко., Лтд. Способ кодирования сигнала и способ декодирования сигнала
US8712763B2 (en) 2008-12-31 2014-04-29 Huawei Technologies Co., Ltd Method for encoding signal, and method for decoding signal

Also Published As

Publication number Publication date
DE50211294D1 (de) 2008-01-10
US20040148162A1 (en) 2004-07-29
WO2002095734A2 (de) 2002-11-28
EP1388146A2 (de) 2004-02-11
WO2002095734A3 (de) 2003-11-20
CN1533564A (zh) 2004-09-29
CN100508027C (zh) 2009-07-01
DE10124420C1 (de) 2002-11-28

Similar Documents

Publication Publication Date Title
DE69926821T2 (de) Verfahren zur signalgesteuerten Schaltung zwischen verschiedenen Audiokodierungssystemen
DE60201766T2 (de) Verbesserung der Periodizität der CELP-Anregung für die Sprachkodierung und -dekodierung
DE69604729T2 (de) Verfahren zur sprachkodierung mittels linearer prädiktion und anregung durch algebraische kodes
DE69900786T2 (de) Sprachkodierung
DE60121405T2 (de) Transkodierer zur Vermeidung einer Kaskadenkodierung von Sprachsignalen
DE602004007786T2 (de) Verfahren und vorrichtung zur quantisierung des verstärkungsfaktors in einem breitbandsprachkodierer mit variabler bitrate
DE19604273C5 (de) Verfahren und Vorrichtung zum Durchführen einer Suche in einem Kodebuch im Hinblick auf das Kodieren eines Klangsignales, Zellkommunikationssystem, Zellnetzwerkelement und mobile Zell-Sender-/Empfänger-Einheit
DE69609099T2 (de) Verfahren zur Modifikation von LPC-Koeffizienten von akustischen Signalen
DE69023402T2 (de) Verfahren zur Sprachkodierung und -dekodierung.
DE69029232T2 (de) System und Methode zur Sprachkodierung
DE69837822T2 (de) Verfahren und Vorrichtung zur Dekodierung von Sprachsignalen
DE602004006211T2 (de) Verfahren zur Maskierung von Paketverlusten und/oder Rahmenausfall in einem Kommunikationssystem
DE29825254U1 (de) Sprachcodierer und Sprachdecodierer
EP1388146B1 (de) Verfahren zur codierung und zur übertragung von sprachsignalen
DE69033510T2 (de) Numerischer sprachkodierer mit verbesserter langzeitvorhersage durch subabtastauflösung
DE69902480T2 (de) Verfahren zur quantisierung der parameter eines sprachkodierers
DE69820362T2 (de) Nichtlinearer Filter zur Geräuschunterdrückung in linearen Prädiktions-Sprachkodierungs-Vorrichtungen
DE60309651T2 (de) Verfahren zur Sprachkodierung mittels verallgemeinerter Analyse durch Synthese und Sprachkodierer zur Durchführung dieses Verfahrens
DE69329568T2 (de) Verfahren zur Sprachkodierung
DE68914147T2 (de) Sprachcodierer mit niedriger Datenrate und niedriger Verzögerung.
DE69324732T2 (de) Selektive Anwendung von Sprachkodierungstechniken
DE60016305T2 (de) Verfahren zum Betrieb eines Sprachkodierers
DE69321444T2 (de) Verfahren und Vorrichtung für Sprachkodierung auf der Basis von Analyse-durch-Synthesetechniken
DE60109111T2 (de) Sprachdekoder zum hochqualitativen Dekodieren von Signalen mit Hintergrundrauschen
DE69821895T2 (de) Verfahren zur Sprachkodierung und -dekodierung

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20031112

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE TR

AX Request for extension of the european patent

Extension state: AL LT LV MK RO SI

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20071128

REF Corresponds to:

Ref document number: 50211294

Country of ref document: DE

Date of ref document: 20080110

Kind code of ref document: P

ET Fr: translation filed
PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20080829

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20190517

Year of fee payment: 18

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20190719

Year of fee payment: 18

Ref country code: GB

Payment date: 20190509

Year of fee payment: 18

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50211294

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200502

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20201201