EP1579426B1 - Verfahren zur übertragung von audiosignalen nach dem verfahren der priorisierenden pixelübertragung - Google Patents

Verfahren zur übertragung von audiosignalen nach dem verfahren der priorisierenden pixelübertragung Download PDF

Info

Publication number
EP1579426B1
EP1579426B1 EP03762456A EP03762456A EP1579426B1 EP 1579426 B1 EP1579426 B1 EP 1579426B1 EP 03762456 A EP03762456 A EP 03762456A EP 03762456 A EP03762456 A EP 03762456A EP 1579426 B1 EP1579426 B1 EP 1579426B1
Authority
EP
European Patent Office
Prior art keywords
audio signal
values
groups
priority
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP03762456A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1579426A1 (de
Inventor
Gerd Mossakowski
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telekom Deutschland GmbH
Original Assignee
T Mobile Deutschland GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by T Mobile Deutschland GmbH filed Critical T Mobile Deutschland GmbH
Priority to SI200331788T priority Critical patent/SI1579426T1/sl
Publication of EP1579426A1 publication Critical patent/EP1579426A1/de
Application granted granted Critical
Publication of EP1579426B1 publication Critical patent/EP1579426B1/de
Priority to CY20101100315T priority patent/CY1109952T1/el
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring

Definitions

  • the invention relates to a method for transmitting audio signals according to the method of prioritizing pixel transmission according to the preamble of patent claim 1.
  • these pixels, and the pixel values used for the calculation of the prioritization are transferred or stored.
  • a pixel gets a high priority if the differences to its neighboring pixels are very large.
  • the current pixel values are shown on the display.
  • the not yet transferred pixels are calculated from the already transmitted pixels. In principle, these methods can also be used for transmission of audio signals.
  • the object of the invention is to provide a method for transmitting audio signals, which works as lossless as possible even at low transmission bandwidths.
  • the audio signal is first decomposed into a number n of spectral components.
  • the decomposed audio signal is stored in a two-dimensional array with a plurality of fields, with frequency and time as dimensions and the amplitude as the value to be entered in the field.
  • groups are formed from each individual field and at least two fields of the array adjacent to this field, and a priority is assigned to the individual groups, the priority of a group being greater the larger the amplitudes of the group values are and / or the greater Amplitude differences of the values of a group are and / or the closer the group is to the current time.
  • the new method is based essentially on the foundations of Shannon. Accordingly, signals can be transmitted without loss, if they are scanned at twice the frequency. This means that the sound can be split into individual sine waves of different amplitude and frequency. Accordingly, acoustic signals can be reproduced unambiguously by transmitting the individual frequency components, including the amplitudes and phases, without losses. In this case, particular use is also made of the fact that the frequently occurring sound sources, e.g. Musical instruments, human voice, consist of Resonanzkörpem whose resonant frequency does not change or only slowly.
  • the sound is recorded, converted into electrical signals and divided into its frequency components. This can be done either by FFT (Fast-Fourier Transformation) or by n-single frequency-selecting filters. If n-single filters are used, each filter absorbs only a single frequency or a narrow frequency band (similar to the hair in the human ear). Thus, one has at each time the frequency, and the amplitude value at this frequency. In this case, the number n can assume different values in accordance with the terminal characteristics. The larger n is, the better the audio signal can be reproduced. Thus, n is a parameter with which the quality of the audio transmission can be scaled.
  • FFT Fast-Fourier Transformation
  • the amplitude values are buffered in the fields of a 2-dimensonal array.
  • the first dimension of the array corresponds to the time axis and the second dimension corresponds to the frequency.
  • each sample with respective amplitude value and phase is uniquely determined and can be stored in the associated field of the array as an imaginary number.
  • the speech signal is thus represented in three acoustic dimensions (parameters) in the array: the time eg in milliseconds (ms), perceptually perceived as duration, as the first dimension of the array, the frequency in hertz (Hz), perceptually perceived as pitch, as the second dimension of the array and the energy (or intensity) of the signal, perceived perceptually as volume or intensity, which is stored as a numerical value in the corresponding field of the array.
  • the frequency of the image height, the time of the image width and the amplitude of the audio signal (intensity) corresponds to the color value.
  • groups are formed from adjacent values and prioritized.
  • Each field considered individually, forms a group together with at least one but preferably several adjacent fields.
  • the groups consist of the position value defined by time and frequency, the amplitude value at the position value, and the amplitude values of the surrounding values corresponding to a predetermined shape (see FIG. 2 of the applications DE 101 13 880.6 and DE 101 52 612.1 ).
  • those groups receive a very high priority, which are close to the current time, and / or whose amplitude values are very large in comparison to the other groups and / or in which the amplitude values within the group differ greatly from one another.
  • the pixel group values are sorted in descending order and stored or transmitted in that order.
  • the width of the array (time axis) preferably has only a limited extent (eg 5 seconds), ie there are always only signal sections of eg 5 seconds processed. After this time (eg 5 seconds), the array is filled with the values of the subsequent signal section.
  • the values of the individual groups are received in the receiver.
  • the groups are again entered in a corresponding array.
  • DE 101 13 880.6 and DE 101 52 612.1 can then be generated from the transmitting groups again the three-dimensional spectral representation. The more groups received, the more accurate the reconstruction becomes.
  • the not yet transferred array values are calculated by interpolation from the already transmitted array values.
  • a corresponding audio signal is then generated in the receiver, which can then be converted into sound.
  • n frequency generators can be used whose signals are added to an output signal. This parallel construction of n generators ensures good scalability.
  • the clock rate can be drastically reduced by parallel processing, so that the lower the power consumption, the playback time is increased in mobile devices.
  • FPGA's or ASIC's simple design could be used.
  • the described method is not limited to audio signals.
  • the method can be used effectively wherever multiple sensors (sound sensors, light sensors, tactile sensors, etc.) are used, which continuously measure signals that can then be displayed in an array (nth order).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Stereophonic System (AREA)
  • Communication Control (AREA)
  • Television Systems (AREA)
  • Transmitters (AREA)
  • Time-Division Multiplex Systems (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
  • Reduction Or Emphasis Of Bandwidth Of Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
EP03762456A 2002-07-08 2003-07-07 Verfahren zur übertragung von audiosignalen nach dem verfahren der priorisierenden pixelübertragung Expired - Lifetime EP1579426B1 (de)

Priority Applications (2)

Application Number Priority Date Filing Date Title
SI200331788T SI1579426T1 (sl) 2002-07-08 2003-07-07 Postopek prenosa avdiosignalov po postopku prednostnega prenosa pikslov
CY20101100315T CY1109952T1 (el) 2002-07-08 2010-04-06 Μεθοδος για τη μεταδοση ακουστικων σηματων συμφωνα με τη μεθοδο της ιεραρχημενης μεταδοσησεικονοστοιχειων

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10230809A DE10230809B4 (de) 2002-07-08 2002-07-08 Verfahren zur Übertragung von Audiosignalen nach dem Verfahren der priorisierenden Pixelübertragung
DE10230809 2002-07-08
PCT/DE2003/002258 WO2004006224A1 (de) 2002-07-08 2003-07-07 Verfahren zur übertragung von audiosignalen nach dem verfahren der priorisierenden pixelübertragung

Publications (2)

Publication Number Publication Date
EP1579426A1 EP1579426A1 (de) 2005-09-28
EP1579426B1 true EP1579426B1 (de) 2010-01-06

Family

ID=29796219

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03762456A Expired - Lifetime EP1579426B1 (de) 2002-07-08 2003-07-07 Verfahren zur übertragung von audiosignalen nach dem verfahren der priorisierenden pixelübertragung

Country Status (16)

Country Link
US (1) US7603270B2 (pt)
EP (1) EP1579426B1 (pt)
JP (1) JP4637577B2 (pt)
CN (1) CN1323385C (pt)
AT (1) ATE454695T1 (pt)
AU (1) AU2003250775A1 (pt)
CY (1) CY1109952T1 (pt)
DE (2) DE10230809B4 (pt)
DK (1) DK1579426T3 (pt)
ES (1) ES2339237T3 (pt)
HK (1) HK1081714A1 (pt)
PL (1) PL207103B1 (pt)
PT (1) PT1579426E (pt)
RU (1) RU2322706C2 (pt)
SI (1) SI1579426T1 (pt)
WO (1) WO2004006224A1 (pt)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3469567B2 (ja) * 2001-09-03 2003-11-25 三菱電機株式会社 音響符号化装置、音響復号化装置、音響符号化方法及び音響復号化方法
DE102007017254B4 (de) * 2006-11-16 2009-06-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung zum Kodieren und Dekodieren
EP3121814A1 (en) * 2015-07-24 2017-01-25 Sound object techology S.A. in organization A method and a system for decomposition of acoustic signal into sound objects, a sound object and its use

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2914974B2 (ja) * 1987-02-27 1999-07-05 株式会社日立製作所 可変レート音声信号伝送方法および伝送システム
US5253326A (en) * 1991-11-26 1993-10-12 Codex Corporation Prioritization method and device for speech frames coded by a linear predictive coder
JP3153933B2 (ja) * 1992-06-16 2001-04-09 ソニー株式会社 データ符号化装置及び方法並びにデータ復号化装置及び方法
US5517511A (en) * 1992-11-30 1996-05-14 Digital Voice Systems, Inc. Digital transmission of acoustic signals over a noisy communication channel
US5675705A (en) * 1993-09-27 1997-10-07 Singhal; Tara Chand Spectrogram-feature-based speech syllable and word recognition using syllabic language dictionary
JP2797959B2 (ja) * 1994-03-12 1998-09-17 日本ビクター株式会社 多次元画像圧縮伸張方法
JPH1083623A (ja) * 1996-09-10 1998-03-31 Sony Corp 信号記録方法、信号記録装置、記録媒体および信号処理方法
US5886276A (en) * 1997-01-16 1999-03-23 The Board Of Trustees Of The Leland Stanford Junior University System and method for multiresolution scalable audio signal encoding
SE9700772D0 (sv) * 1997-03-03 1997-03-03 Ericsson Telefon Ab L M A high resolution post processing method for a speech decoder
US6144937A (en) * 1997-07-23 2000-11-07 Texas Instruments Incorporated Noise suppression of speech by signal processing including applying a transform to time domain input sequences of digital signals representing audio information
AU3372199A (en) * 1998-03-30 1999-10-18 Voxware, Inc. Low-complexity, low-delay, scalable and embedded speech and audio coding with adaptive frame loss concealment
US6584509B2 (en) * 1998-06-23 2003-06-24 Intel Corporation Recognizing audio and video streams over PPP links in the absence of an announcement protocol
JP3522137B2 (ja) * 1998-12-18 2004-04-26 富士通株式会社 可変レート符号化・復号装置
JP3797836B2 (ja) * 1999-12-09 2006-07-19 株式会社東芝 リモートメンテナンスシステム
DE10008055A1 (de) * 2000-02-22 2001-08-30 Infineon Technologies Ag Verfahren zur Kompression von Daten
JP3576935B2 (ja) * 2000-07-21 2004-10-13 株式会社ケンウッド 周波数間引き装置、周波数間引き方法及び記録媒体
JP3576936B2 (ja) * 2000-07-21 2004-10-13 株式会社ケンウッド 周波数補間装置、周波数補間方法及び記録媒体
JP2002135122A (ja) * 2000-10-19 2002-05-10 Nec Corp オーディオ信号符号化装置
US6952669B2 (en) * 2001-01-12 2005-10-04 Telecompression Technologies, Inc. Variable rate speech data compression
DE10152612B4 (de) * 2001-03-21 2006-02-23 T-Mobile Deutschland Gmbh Verfahren zur Komprimierung und Dekomprimierung von Bilddaten
DE10113880B4 (de) * 2001-03-21 2004-04-29 T-Mobile Deutschland Gmbh Verfahren zur Komprimierung und Dekomprimierung von Videodaten
WO2002078322A1 (de) * 2001-03-21 2002-10-03 T-Mobile Deutschland Gmbh Verfahren zur komprimierung und dekomprimierung von bilddaten
US7136418B2 (en) * 2001-05-03 2006-11-14 University Of Washington Scalable and perceptually ranked signal coding and decoding
US7079658B2 (en) * 2001-06-14 2006-07-18 Ati Technologies, Inc. System and method for localization of sounds in three-dimensional space
JP3463752B2 (ja) * 2001-07-25 2003-11-05 三菱電機株式会社 音響符号化装置、音響復号化装置、音響符号化方法および音響復号化方法
US20030236674A1 (en) * 2002-06-19 2003-12-25 Henry Raymond C. Methods and systems for compression of stored audio
DE10229706A1 (de) * 2002-07-02 2004-12-09 T-Mobile Deutschland Gmbh Verfahren zur Speicherplatzverwaltung in einem Speichrmedium eines digitalen Endgeräts bei einer Datenspeicherung nach dem Verfahren der priorisierten Pixelübertragung
DE10229976B4 (de) * 2002-07-03 2007-06-28 T-Mobile Deutschland Gmbh Verfahren zur Ver- und Entschlüsselung von nach dem Verfahren der priorisierten Pixelübertragung übertragenen oder gespeicherten digitalen Daten
US7359979B2 (en) * 2002-09-30 2008-04-15 Avaya Technology Corp. Packet prioritization and associated bandwidth and buffer management techniques for audio over IP

Also Published As

Publication number Publication date
US20060015346A1 (en) 2006-01-19
CY1109952T1 (el) 2014-09-10
HK1081714A1 (en) 2006-05-19
AU2003250775A1 (en) 2004-01-23
US7603270B2 (en) 2009-10-13
DE50312330D1 (de) 2010-02-25
DK1579426T3 (da) 2010-05-17
EP1579426A1 (de) 2005-09-28
JP4637577B2 (ja) 2011-02-23
JP2005532580A (ja) 2005-10-27
SI1579426T1 (sl) 2010-05-31
PL374146A1 (en) 2005-10-03
PL207103B1 (pl) 2010-11-30
WO2004006224A1 (de) 2004-01-15
RU2005102935A (ru) 2005-10-27
DE10230809A1 (de) 2004-01-29
DE10230809B4 (de) 2008-09-11
CN1666255A (zh) 2005-09-07
CN1323385C (zh) 2007-06-27
ATE454695T1 (de) 2010-01-15
RU2322706C2 (ru) 2008-04-20
PT1579426E (pt) 2010-04-08
ES2339237T3 (es) 2010-05-18

Similar Documents

Publication Publication Date Title
DE60208426T2 (de) Vorrichtung zur signalkodierung, signaldekodierung und system zum verteilen von audiodaten
EP0290581B1 (de) Verfahren zum übertragen digitalisierter tonsignale
DE19604273C2 (de) Verfahren und Vorrichtung zum Durchführen einer Suche in einem Kodebuch im Hinblick auf das Kodieren eines Klangsignales, Zellkommunikationssystem, Zellnetzwerkelement und mobile Zell-Sender-/Empfänger-Einheit
DE2656044C2 (de) Verfahren und Vorrichtung zur Codierung und Übertragung eines Sprachsignals
DE60012198T2 (de) Kodierung der hüllkurve des spektrums mittels variabler zeit/frequenz-auflösung
DE60103086T2 (de) Verbesserung von quellcodierungssystemen durch adaptive transposition
EP0205200A1 (de) Verfahren zur Übertragung von Audio-Information und Zusatzinformation in digitaler Form
WO2007087824A1 (de) Verfahren und anordnungen zur audiosignalkodierung
DE9006717U1 (de) Anrufbeantworter für die digitale Aufzeichnung und Wiedergabe von Sprachsignalen
EP1023777B1 (de) Verfahren und vorrichtung zur erzeugung eines bitratenskalierbaren audio-datenstroms
DE60311891T2 (de) Audiocodierung
EP0527374A2 (de) Codierverfahren für Audiosignale mit 32 kbit/s
DE2609297C3 (de) Übertragungssystem für Gesprächssignale
EP1579426B1 (de) Verfahren zur übertragung von audiosignalen nach dem verfahren der priorisierenden pixelübertragung
DE4020656A1 (de) Verfahren zur uebertragung eines signals
DE2836738B2 (de) Verfahren zur Aufzeichnung von beim Spielen eines tastenbetätigten Musikinstruments entstehenden Tonsignalen und den damit verbundenen Spielausdrücken, insbesondere der Lautstärke, entsprechenden Datensignalfolgen auf einem Aufzeichnungsträger und Vorrichtungen zur Durchführung des Verfahrens
DE69828849T2 (de) Signalverarbeitungsgerät und -verfahren sowie Informationsaufzeichnungsgerät
DE2051589C3 (de) Elektrischer Synthesator
DE2826818A1 (de) Verfahren und vorrichtung zum erzeugen eines kuenstlichen sprechsignals
DE69732870T2 (de) System zur Kodierung und Dekodierung von Audiosignalen
EP1834322B1 (de) Verfahren zum codieren eines analogen signals
AT247018B (de) Anordnung zur Gewinnung einer digitalisierten Darstellung natürlicher Sprache
DE1926362C (de) Einrichtung zur Sprachanalyse und synthe se nach dem Vocoderpnnzip
AT500636A2 (de) Verfahren zur codierung eindimensionaler digitaler signale
AT509439B1 (de) Verfahren und mittel zur skalierbaren verbesserung der qualität eines signalcodierverfahrens

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20041217

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 50312330

Country of ref document: DE

Date of ref document: 20100225

Kind code of ref document: P

REG Reference to a national code

Ref country code: RO

Ref legal event code: EPE

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20100330

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: LUCHS & PARTNER PATENTANWAELTE

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

Ref country code: GR

Ref legal event code: EP

Ref document number: 20100400762

Country of ref document: GR

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2339237

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: SK

Ref legal event code: T3

Ref document number: E 7088

Country of ref document: SK

REG Reference to a national code

Ref country code: HU

Ref legal event code: AG4A

Ref document number: E007767

Country of ref document: HU

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20101007

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110201

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50312330

Country of ref document: DE

Effective date: 20110201

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 7088

Country of ref document: SK

Effective date: 20110707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100106

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: RO

Payment date: 20210625

Year of fee payment: 19

Ref country code: CZ

Payment date: 20210624

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20210721

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CY

Payment date: 20210630

Year of fee payment: 19

Ref country code: EE

Payment date: 20210720

Year of fee payment: 19

Ref country code: FI

Payment date: 20210720

Year of fee payment: 19

Ref country code: AT

Payment date: 20210720

Year of fee payment: 19

Ref country code: BG

Payment date: 20210719

Year of fee payment: 19

Ref country code: FR

Payment date: 20210722

Year of fee payment: 19

Ref country code: IT

Payment date: 20210730

Year of fee payment: 19

Ref country code: IE

Payment date: 20210726

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GR

Payment date: 20210720

Year of fee payment: 19

Ref country code: HU

Payment date: 20210628

Year of fee payment: 19

Ref country code: SK

Payment date: 20210701

Year of fee payment: 19

Ref country code: SI

Payment date: 20210628

Year of fee payment: 19

Ref country code: SE

Payment date: 20210721

Year of fee payment: 19

Ref country code: BE

Payment date: 20210721

Year of fee payment: 19

Ref country code: CH

Payment date: 20210723

Year of fee payment: 19

Ref country code: DK

Payment date: 20210721

Year of fee payment: 19

Ref country code: ES

Payment date: 20210819

Year of fee payment: 19

Ref country code: GB

Payment date: 20210722

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20210701

Year of fee payment: 19

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20220731

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

Ref country code: SE

Ref legal event code: EUG

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20220801

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 454695

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220707

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220707

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: RO

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230109

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

REG Reference to a national code

Ref country code: SI

Ref legal event code: KO00

Effective date: 20230314

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: HU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: GR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230209

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 7088

Country of ref document: SK

Effective date: 20220707

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20230825

REG Reference to a national code

Ref country code: EE

Ref legal event code: MM4A

Ref document number: E004255

Country of ref document: EE

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20230718

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: EE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707