EP1579426A1 - Procede de transmission de signaux audio d'apres le procede de transmission de pixels par ordre de priorite - Google Patents

Procede de transmission de signaux audio d'apres le procede de transmission de pixels par ordre de priorite

Info

Publication number
EP1579426A1
EP1579426A1 EP03762456A EP03762456A EP1579426A1 EP 1579426 A1 EP1579426 A1 EP 1579426A1 EP 03762456 A EP03762456 A EP 03762456A EP 03762456 A EP03762456 A EP 03762456A EP 1579426 A1 EP1579426 A1 EP 1579426A1
Authority
EP
European Patent Office
Prior art keywords
audio signal
values
groups
priority
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP03762456A
Other languages
German (de)
English (en)
Other versions
EP1579426B1 (fr
Inventor
Gerd Mossakowski
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telekom Deutschland GmbH
Original Assignee
T Mobile Deutschland GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by T Mobile Deutschland GmbH filed Critical T Mobile Deutschland GmbH
Priority to SI200331788T priority Critical patent/SI1579426T1/sl
Publication of EP1579426A1 publication Critical patent/EP1579426A1/fr
Application granted granted Critical
Publication of EP1579426B1 publication Critical patent/EP1579426B1/fr
Priority to CY20101100315T priority patent/CY1109952T1/el
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring

Definitions

  • the invention relates to a method for transmitting audio signals according to the method of prioritizing pixel transmission according to the preamble of claim 1.
  • Non-linear transmission of the samples e.g. with ISDN transmission
  • these pixels and the pixel values used for calculating the prioritization are transmitted or stored.
  • a pixel has high priority if the differences to its neighboring pixels are very large.
  • the current pixel values are shown on the display for reconstruction.
  • the pixels that have not yet been transferred are calculated from the pixels that have already been transferred. In principle, these methods can also be used for the transmission of audio signals.
  • the object of the invention is therefore to provide a method for the transmission of audio signals which works as losslessly as possible even with low transmission bandwidths.
  • the audio signal is first broken down into a number n of spectral components.
  • the split audio signal is stored in a two-dimensional array with a large number of fields, with frequency and time as dimensions and the amplitude as the value to be entered in the field.
  • Groups are then formed from each individual field and at least two fields of the array adjacent to this field, and the individual groups are assigned a priority, the priority of a group being chosen the greater the greater the amplitudes of the group values and / or the greater the Differences in the values of a group are and / or the closer the group is to the current time.
  • the groups are transmitted to the recipient in the order of their priority.
  • the new process is essentially based on Shannon's foundations. Accordingly, signals can be transmitted lossless if they are sampled at twice the frequency. This means that the sound can be broken down into individual sine waves of different amplitudes and frequencies. Accordingly, acoustic signals can be clearly restored without loss by transmitting the individual frequency components, including the amplitudes and phases.
  • the frequently occurring sound sources e.g. Musical instruments, human voice, consist of resonance bodies whose resonance frequency does not change or changes only slowly.
  • the sound is recorded, converted into electrical signals and broken down into its frequency components. This can be done either by FFT (Fast Fourier Transformation) or by n-individual frequency-selecting filters. If n-individual filters are used, each filter only records a single frequency or a narrow frequency band (similar to the hairs in the human ear). So you have the frequency and the amplitude value at this frequency at all times.
  • the number n can assume different values depending on the end device properties. The larger n is, the better the audio signal can be reproduced. Thus n is a parameter with which the quality of the audio transmission can be scaled.
  • the amplitude values are buffered in the fields of a 2-dimensional array.
  • the first dimension of the array corresponds to the time axis and the second dimension to the frequency.
  • Each sample value with its respective amplitude value and phase is thus uniquely determined and can be stored as an imaginary number in the assigned field of the array.
  • the speech signal is thus represented in three acoustic dimensions (parameters) in the array: the time e.g. in milliseconds (ms), perceptually perceived as duration, as the first dimension of the array, the frequency in Hertz (Hz), perceptually perceived as pitch, as the second dimension of the array and the energy (or intensity) of the signal, perceptually as Perceived volume or intensity, which is stored as a numerical value in the corresponding field of the array.
  • groups are formed from neighboring values and these are prioritized.
  • Each field considered together forms a group together with at least one, but preferably several neighboring fields.
  • the groups consist of the position value, defined by time and frequency, the amplitude value at the position value, and the amplitude values of the surrounding values according to a predetermined form (see FIG. 2 of the applications DE 101 13 880.6 and DE 101 52 612.1).
  • those groups that have a close proximity to the current time and / or whose amplitude values are very large compared to the other groups and / or in which the amplitude values within the group differ greatly from one another have a very high priority.
  • the pixel group values are sorted in descending order and saved or transmitted in this order.
  • the width of the array (time axis) preferably has only a limited extent (eg 5 seconds), ie only signal sections of eg Processed 5 seconds in length. After this time (eg 5 seconds) the array is filled with the values of the following signal section.
  • the values of the individual groups are received in the receiver in accordance with the prioritization parameters described above (amplitude, timely position and amplitude differences from neighboring values).
  • the groups are again entered in a corresponding array.
  • the three-dimensional spectral representation can then be generated again from the transmitting groups.
  • the array values that have not yet been transferred are calculated by interpolation from the already transferred array values.
  • a corresponding audio signal is then generated from the array generated in this way in the receiver, which can then be converted into sound.
  • n Frequency generators will be used, the signals of which are added to an output signal.
  • This parallel structure of n generators ensures good scalability.
  • the clock rate can be drastically reduced by parallel processing, so that the playback time for mobile devices is increased due to lower energy consumption.
  • FPGA's or ASIC's of simple design can be used.
  • the described method is not limited to audio signals.
  • the method can be used effectively wherever several sensors (sound sensors, light sensors, touch sensors, etc.) are used, which continuously measure signals, which can then be displayed in an array (nth order).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Communication Control (AREA)
  • Stereophonic System (AREA)
  • Television Systems (AREA)
  • Transmitters (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
  • Reduction Or Emphasis Of Bandwidth Of Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Time-Division Multiplex Systems (AREA)
EP03762456A 2002-07-08 2003-07-07 Procede de transmission de signaux audio d'apres le procede de transmission de pixels par ordre de priorite Expired - Lifetime EP1579426B1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
SI200331788T SI1579426T1 (sl) 2002-07-08 2003-07-07 Postopek prenosa avdiosignalov po postopku prednostnega prenosa pikslov
CY20101100315T CY1109952T1 (el) 2002-07-08 2010-04-06 Μεθοδος για τη μεταδοση ακουστικων σηματων συμφωνα με τη μεθοδο της ιεραρχημενης μεταδοσησεικονοστοιχειων

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10230809A DE10230809B4 (de) 2002-07-08 2002-07-08 Verfahren zur Übertragung von Audiosignalen nach dem Verfahren der priorisierenden Pixelübertragung
DE10230809 2002-07-08
PCT/DE2003/002258 WO2004006224A1 (fr) 2002-07-08 2003-07-07 Procede de transmission de signaux audio d'apres le procede de transmission de pixels par ordre de priorite

Publications (2)

Publication Number Publication Date
EP1579426A1 true EP1579426A1 (fr) 2005-09-28
EP1579426B1 EP1579426B1 (fr) 2010-01-06

Family

ID=29796219

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03762456A Expired - Lifetime EP1579426B1 (fr) 2002-07-08 2003-07-07 Procede de transmission de signaux audio d'apres le procede de transmission de pixels par ordre de priorite

Country Status (16)

Country Link
US (1) US7603270B2 (fr)
EP (1) EP1579426B1 (fr)
JP (1) JP4637577B2 (fr)
CN (1) CN1323385C (fr)
AT (1) ATE454695T1 (fr)
AU (1) AU2003250775A1 (fr)
CY (1) CY1109952T1 (fr)
DE (2) DE10230809B4 (fr)
DK (1) DK1579426T3 (fr)
ES (1) ES2339237T3 (fr)
HK (1) HK1081714A1 (fr)
PL (1) PL207103B1 (fr)
PT (1) PT1579426E (fr)
RU (1) RU2322706C2 (fr)
SI (1) SI1579426T1 (fr)
WO (1) WO2004006224A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3469567B2 (ja) * 2001-09-03 2003-11-25 三菱電機株式会社 音響符号化装置、音響復号化装置、音響符号化方法及び音響復号化方法
DE102007017254B4 (de) * 2006-11-16 2009-06-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung zum Kodieren und Dekodieren
EP3121814A1 (fr) * 2015-07-24 2017-01-25 Sound object techology S.A. in organization Procédé et système pour la décomposition d'un signal acoustique en objets sonores, objet sonore et son utilisation

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2914974B2 (ja) * 1987-02-27 1999-07-05 株式会社日立製作所 可変レート音声信号伝送方法および伝送システム
US5253326A (en) * 1991-11-26 1993-10-12 Codex Corporation Prioritization method and device for speech frames coded by a linear predictive coder
JP3153933B2 (ja) * 1992-06-16 2001-04-09 ソニー株式会社 データ符号化装置及び方法並びにデータ復号化装置及び方法
US5517511A (en) * 1992-11-30 1996-05-14 Digital Voice Systems, Inc. Digital transmission of acoustic signals over a noisy communication channel
US5675705A (en) * 1993-09-27 1997-10-07 Singhal; Tara Chand Spectrogram-feature-based speech syllable and word recognition using syllabic language dictionary
JP2797959B2 (ja) * 1994-03-12 1998-09-17 日本ビクター株式会社 多次元画像圧縮伸張方法
JPH1083623A (ja) * 1996-09-10 1998-03-31 Sony Corp 信号記録方法、信号記録装置、記録媒体および信号処理方法
US5886276A (en) * 1997-01-16 1999-03-23 The Board Of Trustees Of The Leland Stanford Junior University System and method for multiresolution scalable audio signal encoding
SE9700772D0 (sv) * 1997-03-03 1997-03-03 Ericsson Telefon Ab L M A high resolution post processing method for a speech decoder
US6144937A (en) * 1997-07-23 2000-11-07 Texas Instruments Incorporated Noise suppression of speech by signal processing including applying a transform to time domain input sequences of digital signals representing audio information
US6351730B2 (en) * 1998-03-30 2002-02-26 Lucent Technologies Inc. Low-complexity, low-delay, scalable and embedded speech and audio coding with adaptive frame loss concealment
US6584509B2 (en) * 1998-06-23 2003-06-24 Intel Corporation Recognizing audio and video streams over PPP links in the absence of an announcement protocol
JP3522137B2 (ja) * 1998-12-18 2004-04-26 富士通株式会社 可変レート符号化・復号装置
JP3797836B2 (ja) * 1999-12-09 2006-07-19 株式会社東芝 リモートメンテナンスシステム
DE10008055A1 (de) * 2000-02-22 2001-08-30 Infineon Technologies Ag Verfahren zur Kompression von Daten
JP3576936B2 (ja) * 2000-07-21 2004-10-13 株式会社ケンウッド 周波数補間装置、周波数補間方法及び記録媒体
JP3576935B2 (ja) * 2000-07-21 2004-10-13 株式会社ケンウッド 周波数間引き装置、周波数間引き方法及び記録媒体
JP2002135122A (ja) * 2000-10-19 2002-05-10 Nec Corp オーディオ信号符号化装置
US6952669B2 (en) * 2001-01-12 2005-10-04 Telecompression Technologies, Inc. Variable rate speech data compression
DE10113880B4 (de) * 2001-03-21 2004-04-29 T-Mobile Deutschland Gmbh Verfahren zur Komprimierung und Dekomprimierung von Videodaten
WO2002078322A1 (fr) * 2001-03-21 2002-10-03 T-Mobile Deutschland Gmbh Procede de compression et de decompression de donnees d'image
DE10152612B4 (de) * 2001-03-21 2006-02-23 T-Mobile Deutschland Gmbh Verfahren zur Komprimierung und Dekomprimierung von Bilddaten
US7136418B2 (en) * 2001-05-03 2006-11-14 University Of Washington Scalable and perceptually ranked signal coding and decoding
US7079658B2 (en) * 2001-06-14 2006-07-18 Ati Technologies, Inc. System and method for localization of sounds in three-dimensional space
JP3463752B2 (ja) * 2001-07-25 2003-11-05 三菱電機株式会社 音響符号化装置、音響復号化装置、音響符号化方法および音響復号化方法
US20030236674A1 (en) * 2002-06-19 2003-12-25 Henry Raymond C. Methods and systems for compression of stored audio
DE10229706A1 (de) * 2002-07-02 2004-12-09 T-Mobile Deutschland Gmbh Verfahren zur Speicherplatzverwaltung in einem Speichrmedium eines digitalen Endgeräts bei einer Datenspeicherung nach dem Verfahren der priorisierten Pixelübertragung
DE10229976B4 (de) * 2002-07-03 2007-06-28 T-Mobile Deutschland Gmbh Verfahren zur Ver- und Entschlüsselung von nach dem Verfahren der priorisierten Pixelübertragung übertragenen oder gespeicherten digitalen Daten
US7359979B2 (en) * 2002-09-30 2008-04-15 Avaya Technology Corp. Packet prioritization and associated bandwidth and buffer management techniques for audio over IP

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2004006224A1 *

Also Published As

Publication number Publication date
WO2004006224A1 (fr) 2004-01-15
CY1109952T1 (el) 2014-09-10
DE50312330D1 (de) 2010-02-25
PT1579426E (pt) 2010-04-08
ES2339237T3 (es) 2010-05-18
JP4637577B2 (ja) 2011-02-23
PL207103B1 (pl) 2010-11-30
RU2005102935A (ru) 2005-10-27
JP2005532580A (ja) 2005-10-27
HK1081714A1 (en) 2006-05-19
CN1666255A (zh) 2005-09-07
DE10230809B4 (de) 2008-09-11
US20060015346A1 (en) 2006-01-19
CN1323385C (zh) 2007-06-27
RU2322706C2 (ru) 2008-04-20
AU2003250775A1 (en) 2004-01-23
US7603270B2 (en) 2009-10-13
SI1579426T1 (sl) 2010-05-31
EP1579426B1 (fr) 2010-01-06
DK1579426T3 (da) 2010-05-17
DE10230809A1 (de) 2004-01-29
ATE454695T1 (de) 2010-01-15
PL374146A1 (en) 2005-10-03

Similar Documents

Publication Publication Date Title
EP0290581B1 (fr) Procede de transmission de signaux audio numerises
DE60208426T2 (de) Vorrichtung zur signalkodierung, signaldekodierung und system zum verteilen von audiodaten
EP1979901B1 (fr) Procede et dispositifs pour le codage de signaux audio
DE60012198T2 (de) Kodierung der hüllkurve des spektrums mittels variabler zeit/frequenz-auflösung
DE69417445T2 (de) Verfahren und system zur detektion und erzeugung von übergangsbedingungen in tonsignalen
DE19604273C2 (de) Verfahren und Vorrichtung zum Durchführen einer Suche in einem Kodebuch im Hinblick auf das Kodieren eines Klangsignales, Zellkommunikationssystem, Zellnetzwerkelement und mobile Zell-Sender-/Empfänger-Einheit
DE60103086T2 (de) Verbesserung von quellcodierungssystemen durch adaptive transposition
DE69613611T2 (de) System zur Speicherung von und zum Zugriff auf Sprachinformation
WO2007087823A1 (fr) Procédé et dispositifs pour coder un signal audio
EP1023777B1 (fr) Procede et dispositif pour limiter un courant de donnees audio dont le debit binaire peut etre mis a l'echelle
DE2609297C3 (de) Übertragungssystem für Gesprächssignale
DE4020656A1 (de) Verfahren zur uebertragung eines signals
EP1579426B1 (fr) Procede de transmission de signaux audio d'apres le procede de transmission de pixels par ordre de priorite
DE69611987T2 (de) Übertragungssystem mit zeitabhängigen filterbänken
DE1762492B2 (de) Verfahren und schaltung zur hoerbarmachung der menschlichen stimme den hochgradig schwerhoerigen
DE102007041868B4 (de) Diskrete Synthese unter Verwendung versetzter Nyquist-Regionen vermeidet schutzband-induzierte Löcher in der Nähe einer Nyquist-Grenze
DE69127339T2 (de) Methode und Einrichtung zur Kodierung und Dekodierung eines abgetasteten Analogsignals mit Wiederholungseigenschaften
DE2051589A1 (de) Anordnung zur Synthese eines Signals
DE2826818A1 (de) Verfahren und vorrichtung zum erzeugen eines kuenstlichen sprechsignals
EP1020737B1 (fr) Procédé et dispositif pour la détection électromagnétique des objets
DE2407072A1 (de) Abtastverfahren und vorrichtung zum durchfuehren des verfahrens zum fortschreitenden abtasten eines quasi stationaeren signals
EP3217392B1 (fr) Audibilité de signaux ultrasonores à large bande
AT247018B (de) Anordnung zur Gewinnung einer digitalisierten Darstellung natürlicher Sprache
DE3621632A1 (de) Digitale signaluebertragungsanordnung mit veraenderbarer uebertragungsfunktion
DE1926362C (de) Einrichtung zur Sprachanalyse und synthe se nach dem Vocoderpnnzip

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20041217

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 50312330

Country of ref document: DE

Date of ref document: 20100225

Kind code of ref document: P

REG Reference to a national code

Ref country code: RO

Ref legal event code: EPE

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20100330

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: LUCHS & PARTNER PATENTANWAELTE

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

Ref country code: GR

Ref legal event code: EP

Ref document number: 20100400762

Country of ref document: GR

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2339237

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: SK

Ref legal event code: T3

Ref document number: E 7088

Country of ref document: SK

REG Reference to a national code

Ref country code: HU

Ref legal event code: AG4A

Ref document number: E007767

Country of ref document: HU

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20101007

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110201

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50312330

Country of ref document: DE

Effective date: 20110201

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 7088

Country of ref document: SK

Effective date: 20110707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100106

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: RO

Payment date: 20210625

Year of fee payment: 19

Ref country code: CZ

Payment date: 20210624

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20210721

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CY

Payment date: 20210630

Year of fee payment: 19

Ref country code: EE

Payment date: 20210720

Year of fee payment: 19

Ref country code: FI

Payment date: 20210720

Year of fee payment: 19

Ref country code: AT

Payment date: 20210720

Year of fee payment: 19

Ref country code: BG

Payment date: 20210719

Year of fee payment: 19

Ref country code: FR

Payment date: 20210722

Year of fee payment: 19

Ref country code: IT

Payment date: 20210730

Year of fee payment: 19

Ref country code: IE

Payment date: 20210726

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GR

Payment date: 20210720

Year of fee payment: 19

Ref country code: HU

Payment date: 20210628

Year of fee payment: 19

Ref country code: SK

Payment date: 20210701

Year of fee payment: 19

Ref country code: SI

Payment date: 20210628

Year of fee payment: 19

Ref country code: SE

Payment date: 20210721

Year of fee payment: 19

Ref country code: BE

Payment date: 20210721

Year of fee payment: 19

Ref country code: CH

Payment date: 20210723

Year of fee payment: 19

Ref country code: DK

Payment date: 20210721

Year of fee payment: 19

Ref country code: ES

Payment date: 20210819

Year of fee payment: 19

Ref country code: GB

Payment date: 20210722

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20210701

Year of fee payment: 19

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20220731

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

Ref country code: SE

Ref legal event code: EUG

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20220801

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 454695

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220707

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220707

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: RO

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230109

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

REG Reference to a national code

Ref country code: SI

Ref legal event code: KO00

Effective date: 20230314

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: HU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: GR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230209

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 7088

Country of ref document: SK

Effective date: 20220707

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20230825

REG Reference to a national code

Ref country code: EE

Ref legal event code: MM4A

Ref document number: E004255

Country of ref document: EE

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20230718

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: EE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707