EP1579426A1 - Method for transmitting audio signals according to the prioritizing pixel transmission method - Google Patents

Method for transmitting audio signals according to the prioritizing pixel transmission method

Info

Publication number
EP1579426A1
EP1579426A1 EP03762456A EP03762456A EP1579426A1 EP 1579426 A1 EP1579426 A1 EP 1579426A1 EP 03762456 A EP03762456 A EP 03762456A EP 03762456 A EP03762456 A EP 03762456A EP 1579426 A1 EP1579426 A1 EP 1579426A1
Authority
EP
European Patent Office
Prior art keywords
audio signal
values
groups
priority
array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP03762456A
Other languages
German (de)
French (fr)
Other versions
EP1579426B1 (en
Inventor
Gerd Mossakowski
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telekom Deutschland GmbH
Original Assignee
T Mobile Deutschland GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by T Mobile Deutschland GmbH filed Critical T Mobile Deutschland GmbH
Priority to SI200331788T priority Critical patent/SI1579426T1/en
Publication of EP1579426A1 publication Critical patent/EP1579426A1/en
Application granted granted Critical
Publication of EP1579426B1 publication Critical patent/EP1579426B1/en
Priority to CY20101100315T priority patent/CY1109952T1/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring

Definitions

  • the invention relates to a method for transmitting audio signals according to the method of prioritizing pixel transmission according to the preamble of claim 1.
  • Non-linear transmission of the samples e.g. with ISDN transmission
  • these pixels and the pixel values used for calculating the prioritization are transmitted or stored.
  • a pixel has high priority if the differences to its neighboring pixels are very large.
  • the current pixel values are shown on the display for reconstruction.
  • the pixels that have not yet been transferred are calculated from the pixels that have already been transferred. In principle, these methods can also be used for the transmission of audio signals.
  • the object of the invention is therefore to provide a method for the transmission of audio signals which works as losslessly as possible even with low transmission bandwidths.
  • the audio signal is first broken down into a number n of spectral components.
  • the split audio signal is stored in a two-dimensional array with a large number of fields, with frequency and time as dimensions and the amplitude as the value to be entered in the field.
  • Groups are then formed from each individual field and at least two fields of the array adjacent to this field, and the individual groups are assigned a priority, the priority of a group being chosen the greater the greater the amplitudes of the group values and / or the greater the Differences in the values of a group are and / or the closer the group is to the current time.
  • the groups are transmitted to the recipient in the order of their priority.
  • the new process is essentially based on Shannon's foundations. Accordingly, signals can be transmitted lossless if they are sampled at twice the frequency. This means that the sound can be broken down into individual sine waves of different amplitudes and frequencies. Accordingly, acoustic signals can be clearly restored without loss by transmitting the individual frequency components, including the amplitudes and phases.
  • the frequently occurring sound sources e.g. Musical instruments, human voice, consist of resonance bodies whose resonance frequency does not change or changes only slowly.
  • the sound is recorded, converted into electrical signals and broken down into its frequency components. This can be done either by FFT (Fast Fourier Transformation) or by n-individual frequency-selecting filters. If n-individual filters are used, each filter only records a single frequency or a narrow frequency band (similar to the hairs in the human ear). So you have the frequency and the amplitude value at this frequency at all times.
  • the number n can assume different values depending on the end device properties. The larger n is, the better the audio signal can be reproduced. Thus n is a parameter with which the quality of the audio transmission can be scaled.
  • the amplitude values are buffered in the fields of a 2-dimensional array.
  • the first dimension of the array corresponds to the time axis and the second dimension to the frequency.
  • Each sample value with its respective amplitude value and phase is thus uniquely determined and can be stored as an imaginary number in the assigned field of the array.
  • the speech signal is thus represented in three acoustic dimensions (parameters) in the array: the time e.g. in milliseconds (ms), perceptually perceived as duration, as the first dimension of the array, the frequency in Hertz (Hz), perceptually perceived as pitch, as the second dimension of the array and the energy (or intensity) of the signal, perceptually as Perceived volume or intensity, which is stored as a numerical value in the corresponding field of the array.
  • groups are formed from neighboring values and these are prioritized.
  • Each field considered together forms a group together with at least one, but preferably several neighboring fields.
  • the groups consist of the position value, defined by time and frequency, the amplitude value at the position value, and the amplitude values of the surrounding values according to a predetermined form (see FIG. 2 of the applications DE 101 13 880.6 and DE 101 52 612.1).
  • those groups that have a close proximity to the current time and / or whose amplitude values are very large compared to the other groups and / or in which the amplitude values within the group differ greatly from one another have a very high priority.
  • the pixel group values are sorted in descending order and saved or transmitted in this order.
  • the width of the array (time axis) preferably has only a limited extent (eg 5 seconds), ie only signal sections of eg Processed 5 seconds in length. After this time (eg 5 seconds) the array is filled with the values of the following signal section.
  • the values of the individual groups are received in the receiver in accordance with the prioritization parameters described above (amplitude, timely position and amplitude differences from neighboring values).
  • the groups are again entered in a corresponding array.
  • the three-dimensional spectral representation can then be generated again from the transmitting groups.
  • the array values that have not yet been transferred are calculated by interpolation from the already transferred array values.
  • a corresponding audio signal is then generated from the array generated in this way in the receiver, which can then be converted into sound.
  • n Frequency generators will be used, the signals of which are added to an output signal.
  • This parallel structure of n generators ensures good scalability.
  • the clock rate can be drastically reduced by parallel processing, so that the playback time for mobile devices is increased due to lower energy consumption.
  • FPGA's or ASIC's of simple design can be used.
  • the described method is not limited to audio signals.
  • the method can be used effectively wherever several sensors (sound sensors, light sensors, touch sensors, etc.) are used, which continuously measure signals, which can then be displayed in an array (nth order).

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Multimedia (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Stereophonic System (AREA)
  • Communication Control (AREA)
  • Television Systems (AREA)
  • Transmitters (AREA)
  • Reduction Or Emphasis Of Bandwidth Of Signals (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
  • Time-Division Multiplex Systems (AREA)

Abstract

A method for the transmission of audio signals between a transmitter and at least one receiver operates according to the prioritizing pixel transmission method. The audio signal is first broken down into a number of spectral fractions. The broken-down audio signal is stored in a two-dimensional array with a plurality of fields. The dimensions to be registered in the field are frequency and time; the value to be registered in the field is amplitude. Groups are then formed from the individual fields and a priority is assigned to the individual groups, in which the priority will be gauged as higher if the amplitudes of the group values are higher, and/or if the amplitude differences of the values of one group are higher, and/or if the groups are closer to actual time. Finally, the groups are transmitted to the receiver according to the order of their established priority.

Description

VERFAHREN ZUR ÜBERTRAGUNG VON AUDIOSIGNALEN NACH DEM VERFAHREN DER PRIORISIERENDEN PIXE ! BERTRAGUNG METHOD FOR TRANSMITTING AUDIO SIGNALS AFTER THE PRIORIZING PIXE PROCESS! TRANSFER
Die Erfindung betrifft ein Verfahren zur Übertragung von Audiosignalen nach dem Verfahren der priorisierenden Pixelübertragung nach dem Oberbegriff des Patentanspruchs 1.The invention relates to a method for transmitting audio signals according to the method of prioritizing pixel transmission according to the preamble of claim 1.
Zur Zeit existiert eine Vielzahl verschiedener Verfahren zur komprimierten Übertragung von Audiosignalen. Im wesentlichen existieren folgende Verfahren.A variety of different methods for the compressed transmission of audio signals currently exist. The following main procedures exist.
Reduzierung der Abtastrate, z.B. 3 kHz anstelle von 44 kHzReduction of the sampling rate, e.g. 3 kHz instead of 44 kHz
Nichtlineare Übertragung der Abtastwerte, z.B. bei ISDN ÜbertragungNon-linear transmission of the samples, e.g. with ISDN transmission
Benutzung von vorher abgespeicherten Akustiksequenzen, z.B. MIDI oderUse of previously saved acoustic sequences, e.g. MIDI or
Stimmnachbildungvocal imitation
Verwendung von Markov Modellen zur Korrektur von ÜbertragungsfehlernUse of Markov models to correct transmission errors
Die Gemeinsamkeiten der bekannten Verfahren liegen darin, dass auch bei niedrigeren Übertragungsraten eine befriedigende Sprachverständlichkeit vorhanden ist. Dieses wird im wesentlichen durch Mittelwertbildungen erreicht. Jedoch ergeben unterschiedliche Stimmen der Quelle ähnlich klingende Stimmen in der Senke, so dass z.B. Stimmungsschwankungen, die in einem normalen Gespräch erkennbar sind, nicht mehr übertragen werden. Dadurch ergibt sich eine deutliche Einschränkung in der Kommunikationsqualität.The common features of the known methods are that there is satisfactory speech intelligibility even at lower transmission rates. This is essentially achieved by averaging. However, different voices from the source produce similar sounding voices in the depression, so that e.g. Mood fluctuations that are recognizable in a normal conversation can no longer be transmitted. This results in a clear limitation in the quality of communication.
Verfahren zur Komprimierung und Dekomprimierung von Bild- oder Videodaten mittels priorisierter Pixelübertragung sind in den deutschen Patentanmeldungen DE 101 13 880.6 (entspricht PCT/DE02/00987) und DE 101 52 612.1 (entspricht PCT/DE02/00995) beschrieben. Bei diesen Verfahren werden z.B. digitale Bild- oder Videodaten bearbeitet, die aus einem Array einzelner Bildpunkte (Pixel) bestehen, wobei jedes Pixel einen sich zeitlich verändernden Pixelwert aufweist, der Färb- oder Helligkeitsinformation des Pixels beschreibt. Erfindungsgemäß wird jedem Pixel bzw. jeder Pixelgruppe eine Priorität zugeordnet und die Pixel entsprechend ihrer Priorisierung in einem Prioritätenarray abgelegt. Dieses Array enthält zu jedem Zeitpunkt, die nach der Priorisierung sortierten Pixelwerte. Entsprechend der Priorisierung werden diese Pixel, und die für die Berechnung der Priorisierung benutzten Pixelwerte, übertragen bzw. abgespeichert. Ein Pixel bekommt eine hohe Priorität, wenn die Unterschiede zu seinen benachbarten Pixel sehr groß sind. Zur Rekonstruktion werden die jeweils aktuellen Pixelwerte auf dem Display dargestellt. Die noch nicht übertragenden Pixel werden aus den schon übertragenden Pixel berechnet. Diese Verfahren lassen sich prinzipiell auch für eine Übertragung von Audiosignalen verwenden.Methods for compressing and decompressing image or video data by means of prioritized pixel transmission are described in German patent applications DE 101 13 880.6 (corresponds to PCT / DE02 / 00987) and DE 101 52 612.1 (corresponds to PCT / DE02 / 00995). In this process, for example, digital image or Processed video data, which consist of an array of individual pixels (pixels), each pixel having a temporally changing pixel value that describes the color or brightness information of the pixel. According to the invention, a priority is assigned to each pixel or each pixel group and the pixels are stored in a priority array according to their prioritization. This array always contains the pixel values sorted according to the prioritization. In accordance with the prioritization, these pixels and the pixel values used for calculating the prioritization are transmitted or stored. A pixel has high priority if the differences to its neighboring pixels are very large. The current pixel values are shown on the display for reconstruction. The pixels that have not yet been transferred are calculated from the pixels that have already been transferred. In principle, these methods can also be used for the transmission of audio signals.
Die Aufgabe der Erfindung besteht deshalb darin, ein Verfahren zur Übertragung von Audiosignalen anzugeben, das auch bei niedrigen Übertragungsbandbreiten möglichst verlustfrei arbeitet.The object of the invention is therefore to provide a method for the transmission of audio signals which works as losslessly as possible even with low transmission bandwidths.
Diese Aufgabe wird erfindungsgemäß durch die Merkmale des Patentanspruchs 1 gelöst.This object is achieved by the features of claim 1.
Gemäß der Erfindung wird das Audiosignal zunächst in eine Anzahl n von spektralen Anteilen zerlegt. Das zerlegte Audiosignals wird in einem zweidimensionalen Array mit einer Vielzahl von Feldern gespeichert, mit Frequenz und Zeit als Dimensionen und der Amplitude als jeweils einzutragenden Wert im Feld. Dann werden aus jedem einzelnen Feld und mindestens zwei zu diesem Feld benachbarten Feldern des Arrays Gruppen gebildet, und den einzelnen Gruppen eine Priorität zugeordnet, wobei die Priorität einer Gruppe umso größer gewählt wird, je größer die Amplituden der Gruppenwerte sind und/oder je größer die Amplitudenunterschiede der Werte einer Gruppe sind und/oder je näher die Gruppe an der aktuellen Zeit liegt. Schließlich werden die Gruppen in der Reihenfolge ihrer Priorität an den Empfänger übertragen.According to the invention, the audio signal is first broken down into a number n of spectral components. The split audio signal is stored in a two-dimensional array with a large number of fields, with frequency and time as dimensions and the amplitude as the value to be entered in the field. Groups are then formed from each individual field and at least two fields of the array adjacent to this field, and the individual groups are assigned a priority, the priority of a group being chosen the greater the greater the amplitudes of the group values and / or the greater the Differences in the values of a group are and / or the closer the group is to the current time. Finally, the groups are transmitted to the recipient in the order of their priority.
Das neue Verfahren beruht Im wesentlichen auf den Grundlagen von Shannon. Demnach lassen sich Signale verlustfrei übertragen, wenn man sie mit der doppelten Frequenz abtastet. Das bedeutet, dass der Schall in einzelne Sinusschwingung unterschiedlicher Amplitude und Frequenz zerlegbar ist. Demnach lassen sich akustische Signale eindeutig durch Übertragung der einzelnen Frequenzanteile, inklusive der Amplituden und Phasen, ohne Verluste wieder herstellen. Hierbei wird auch insbesondere ausgenutzt, dass die häufig vorkommenden Schallquellen, z.B. Musikinstrumente, menschliche Stimme, aus Resonanzkörpern bestehen, deren Resonanzfrequenz sich nicht bzw. nur langsam ändert.The new process is essentially based on Shannon's foundations. Accordingly, signals can be transmitted lossless if they are sampled at twice the frequency. This means that the sound can be broken down into individual sine waves of different amplitudes and frequencies. Accordingly, acoustic signals can be clearly restored without loss by transmitting the individual frequency components, including the amplitudes and phases. Here, particular use is made of the fact that the frequently occurring sound sources, e.g. Musical instruments, human voice, consist of resonance bodies whose resonance frequency does not change or changes only slowly.
Vorteilhafte Ausgestaltungen und Weiterbildungen der Erfindung sind in den abhängigen Patentansprüchen angegeben.Advantageous refinements and developments of the invention are specified in the dependent claims.
Ein Ausführungsbeispiel der Erfindung wird nachfolgen beschrieben. Hierbei sei insbesondere auch auf die Beschreibung und die Zeichnungen der älteren Patentanmeldungen DE 101 13 880.6 und DE 101 52 612.1 Bezug genommen.An embodiment of the invention is described below. In this context, reference should also be made in particular to the description and drawings of the earlier patent applications DE 101 13 880.6 and DE 101 52 612.1.
Zunächst wird der Schall aufgenommen, in elektrische Signale umgewandelt und in seine Frequenzanteile zerlegt. Dieses kann entweder durch FFT (Fast-Fourier Transformation) oder durch n-einzelne frequenzselektierende Filter geschehen. Werden n-einzelne Filter verwendet, so nimmt jeder Filter nur eine einzelne Frequenz, bzw. ein schmales Frequenzband, auf (ähnlich den Härchen im menschlichen Ohr). Somit hat man zu jedem Zeitpunkt die Frequenz, und den Amplitudenwert bei dieser Frequenz. Dabei kann die Zahl n entsprechend der Endgeräteeigenschaften unterschiedliche Werte annehmen Je größer n ist, desto besser kann das Audiosignal reproduziert werden. Somit ist n ein Parameter mit dem die Qualität der Audioübertragung skaliert werden kann. Die Amplitudenwerte werden in den Feldern eines 2-dimensonalen Arrays zwischengespeichert.First, the sound is recorded, converted into electrical signals and broken down into its frequency components. This can be done either by FFT (Fast Fourier Transformation) or by n-individual frequency-selecting filters. If n-individual filters are used, each filter only records a single frequency or a narrow frequency band (similar to the hairs in the human ear). So you have the frequency and the amplitude value at this frequency at all times. The number n can assume different values depending on the end device properties. The larger n is, the better the audio signal can be reproduced. Thus n is a parameter with which the quality of the audio transmission can be scaled. The amplitude values are buffered in the fields of a 2-dimensional array.
Dabei entspricht die erste Dimension des Arrays der Zeitachse und die zweite Dimension der Frequenz. Damit ist jeder Abtastwert mit jeweiliger Amplitudenwert und Phase eindeutig bestimmt und kann im zugeordneten Feld des Arrays als Imaginäre Zahl abgespeichert werden. Das Sprachsignal wird somit in drei akustischen Dimensionen (Parametern) im Array dargestellt: Die Zeit z.B. in Millisekunden (ms), perzeptiv als Dauer wahrgenommen, als die erste Dimension des Arrays, die Frequenz in Hertz (Hz), perzeptiv als Tonhöhe wahrgenommen, als die zweite Dimension des Arrays und die Energie (bzw. Intensität) des Signals, perzeptiv als Lautstärke bzw. Intensität wahrgenommen, welche als Zahlenwert im entsprechenden Feld des Arrays gespeichert wird.The first dimension of the array corresponds to the time axis and the second dimension to the frequency. Each sample value with its respective amplitude value and phase is thus uniquely determined and can be stored as an imaginary number in the assigned field of the array. The speech signal is thus represented in three acoustic dimensions (parameters) in the array: the time e.g. in milliseconds (ms), perceptually perceived as duration, as the first dimension of the array, the frequency in Hertz (Hz), perceptually perceived as pitch, as the second dimension of the array and the energy (or intensity) of the signal, perceptually as Perceived volume or intensity, which is stored as a numerical value in the corresponding field of the array.
Im Vergleich zu den Anmeldungen DE 101 13 880.6 und DE 101 52 612.1 entspricht z.B. die Frequenz der Bildhöhe, die Zeit der Bildbreite und die Amplitude des Audiosignals (Intensität) dem Farbwert.In comparison to the applications DE 101 13 880.6 and DE 101 52 612.1, e.g. the frequency of the image height, the time of the image width and the amplitude of the audio signal (intensity) the color value.
Ähnlich dem Verfahren der Priorisierung von Pixelgruppen bei der Bild/Videokodierung werden aus benachbarten Werten Gruppen gebildet und diese priorisiert. Jedes Feld für sich betrachtet bildet zusammen mit mindestens einem, vorzugsweise jedoch mehreren benachbarten Feldern eine Gruppe. Die Gruppen bestehen aus dem Positionswert, definiert durch Zeit und Frequenz, dem Amplitudenwert am Positionswert, und die Amplitudenwerte der umliegenden Werte entsprechend einer vorher festgelegten Form (siehe Figur 2 der Anmeldungen DE 101 13 880.6 und DE 101 52 612.1). Dabei bekommen insbesondere diejenigen Gruppen eine sehr hohe Priorität, die nahe der aktuellen Zeit liegen, und/oder deren Amplitudenwerte im Vergleich zu den anderen Gruppen sehr groß sind und/oder bei denen sich die Amplitudenwerte innerhalb der Gruppe stark voneinander unterscheiden. Die Pixelgruppenwerte werden absteigend sortiert und in dieser Reihenfolge gespeichert bzw. übertragen.Similar to the method of prioritizing pixel groups in image / video coding, groups are formed from neighboring values and these are prioritized. Each field considered together forms a group together with at least one, but preferably several neighboring fields. The groups consist of the position value, defined by time and frequency, the amplitude value at the position value, and the amplitude values of the surrounding values according to a predetermined form (see FIG. 2 of the applications DE 101 13 880.6 and DE 101 52 612.1). In particular, those groups that have a close proximity to the current time and / or whose amplitude values are very large compared to the other groups and / or in which the amplitude values within the group differ greatly from one another have a very high priority. The pixel group values are sorted in descending order and saved or transmitted in this order.
Die Breite des Arrays (Zeitachse) besitzt vorzugsweise nur eine begrenzte Ausdehnung (z.B. 5 Sekunden), d.h. es werden immer nur Signalabschnitte von z.B. 5 Sekunden Länge verarbeitet. Nach dieser Zeit (z.B. 5 Sekunden) wird das Array mit den Werten des nachfolgenden Signalabschnitts gefüllt.The width of the array (time axis) preferably has only a limited extent (eg 5 seconds), ie only signal sections of eg Processed 5 seconds in length. After this time (eg 5 seconds) the array is filled with the values of the following signal section.
Entsprechend der oben beschriebenen Priorisierungsparameter (Amplitude, zeitnahe Position und Amplitudenunterschiede zu benachbarten Werten) werden die Werte der einzelnen Gruppen im Empfänger empfangen.The values of the individual groups are received in the receiver in accordance with the prioritization parameters described above (amplitude, timely position and amplitude differences from neighboring values).
Beim Empfänger werden die Gruppen wieder in ein entsprechendes Array eingetragen. Entsprechend der Patentanmeldungen DE 101 13 880.6 und DE 101 52 612.1 kann dann aus den übertragenden Gruppen wieder die dreidimensionale Spektraldarstellung erzeugt werden. Je mehr Gruppen empfangen wurden, umso genauer wird die Rekonstruktion. Die noch nicht übertragenen Arraywerte werden mittels Interpolation aus den schon übertragenden Arraywerten berechnet. Aus dem so erzeugten Array wird dann im Empfänger ein entsprechendes Audiosignal generiert, welches dann in Schall umgewandelt werden kann. Zur Synthese des Audiosignals können z.B. n Frequenzgeneratoren verwenden werden, deren Signale zu einem Ausgangssignal addiert werden. Durch diesen parallelen Aufbau von n Generatoren ist eine gute Skalierbarkeit gegeben. Zudem kann die Taktrate durch parallele Verarbeitung drastisch reduziert werden, so das durch ein geringeren Energieverbrauch die Wiedergabezeit bei mobilen Endgeräten erhöht wird. Für den parallelen Einsatz könnten z.B. FPGA's oder ASIC's einfacher Bauart benutzt werden.At the recipient, the groups are again entered in a corresponding array. According to patent applications DE 101 13 880.6 and DE 101 52 612.1, the three-dimensional spectral representation can then be generated again from the transmitting groups. The more groups were received, the more precise the reconstruction becomes. The array values that have not yet been transferred are calculated by interpolation from the already transferred array values. A corresponding audio signal is then generated from the array generated in this way in the receiver, which can then be converted into sound. For the synthesis of the audio signal, e.g. n Frequency generators will be used, the signals of which are added to an output signal. This parallel structure of n generators ensures good scalability. In addition, the clock rate can be drastically reduced by parallel processing, so that the playback time for mobile devices is increased due to lower energy consumption. For parallel use, e.g. FPGA's or ASIC's of simple design can be used.
Das beschriebene Verfahren ist nicht auf Audiosignale beschränkt. Das Verfahren kann insbesondere überall dort effektiv zur Anwendung kommen, wo mehrere Sensoren (Schallsensoren, Lichtsensoren, Tastsensoren, usw.) verwendet werden, die kontinuierlich Signale messen, die dann in einem Array (n-ter Ordnung) dargestellt werden können.The described method is not limited to audio signals. The method can be used effectively wherever several sensors (sound sensors, light sensors, touch sensors, etc.) are used, which continuously measure signals, which can then be displayed in an array (nth order).
Die Vorteile gegenüber bisherigen Systemen liegen in der flexiblen Einsetzbarkeit bei erhöhten Kompressionsraten. Durch Benutzung eines Arrays, welches aus unterschiedlichen Quellen gespeist wird, erhält man automatisch eine Synchronisation der unterschiedlichen Quellen. Eine entsprechende Synchronisation muss bei herkömmlichen Verfahren durch besondere Protokolle, bzw. Maßnahmen gesichert werden. Insbesondere bei Videoübertragung mit großen Laufzeiten, z.B. Satellitenverbindungen, wo Ton und Bild über verschiedene Kanäle übertragen werden, fällt häufig eine fehlende Synchronisation der Lippen zu der Sprache auf. So etwas kann durch das beschriebene Verfahren beseitigt werdenThe advantages over previous systems are the flexible use with increased compression rates. By using an array that is fed from different sources, you automatically get one Synchronization of the different sources. Corresponding synchronization must be ensured with conventional methods using special protocols or measures. Especially in the case of video transmission with long runtimes, for example satellite connections, where sound and image are transmitted via different channels, the lips are often not synchronized with the speech. Something like this can be eliminated by the described method
Da das gleiche Grundprinzip der priorisierenden Pixelgruppenübertragung sowohl bei Sprache, Bild und Videoübertragung genutzt werden kann, ist ein starker Synergieeffekt bei der Implementierung nutzbar. Außerdem kann auf diese Weise eine einfache Synchronisation zwischen Sprache und Bildern erfolgen. Außerdem könnte beliebig zwischen Bild- und Audioauflösung skaliert werden.Since the same basic principle of prioritizing pixel group transmission can be used for both voice, image and video transmission, a strong synergy effect can be used in the implementation. In addition, a simple synchronization between speech and images can take place in this way. It could also be scaled between image and audio resolution.
Betrachtet man eine einzelne Audioübertragung nach dem neuen Verfahren, so ergibt sich bei Sprache eine natürlichere Wiedergabe, da die für jeden Menschen typischen Frequenzanteile (-gruppen) mit höchster Priorität und damit verlustfrei übertragen werden. If you consider a single audio transmission according to the new method, speech is more natural since the frequency components (groups) typical for every person are transmitted with the highest priority and thus without loss.

Claims

Patentansprüche claims
1. Verfahren zur Übertragung von Audiosignalen zwischen einem Sender und mindestens einem Empfänger nach dem Verfahren der priorisierenden Pixelübertragung, gekennzeichnet durch die Schritte: a) Zerlegen des Audiosignals in eine Anzahl n von spektralen Anteilen, b) Speichern des zerlegten Audiosignals in einem zweidimensionalen Array mit einer Vielzahl von Feldern, mit Frequenz und Zeit als Dimensionen und der Amplitude als jeweils einzutragenden Wert im Feld, c) Bilden von Gruppen aus jedem einzelnen Feld und mindestens zwei zu diesem Feld benachbarten Feldern des Arrays, d) Zuordnen einer Priorität zu den einzelnen Gruppen, wobei die Priorität einer Gruppe umso größer wird, je größer die Amplituden der Gruppenwerte sind und/oder je größer die Amplitudenunterschiede der Werte einer Gruppe sind und/oder je näher die Gruppe an der aktuellen Zeit liegt, und e) Übertragen der Gruppen in der Reihenfolge ihrer Priorität an den Empfänger.1. Method for the transmission of audio signals between a transmitter and at least one receiver according to the method of prioritizing pixel transmission, characterized by the steps: a) decomposing the audio signal into a number n of spectral components, b) storing the decomposed audio signal in a two-dimensional array a plurality of fields, with frequency and time as dimensions and the amplitude as the value to be entered in the field in each case, c) forming groups from each individual field and at least two fields of the array adjacent to this field, d) assigning a priority to the individual groups , with the priority of a group increasing, the greater the amplitudes of the group values and / or the greater the amplitude differences of the values of a group and / or the closer the group is to the current time, and e) transmitting the groups in the Order of priority to the recipient.
2. Verfahren nach Anspruch 1 , dadurch gekennzeichnet, dass das gesamte Audiosignal als Audiodatei vorliegt und als Ganzes bearbeitet und übertragen wird.2. The method according to claim 1, characterized in that the entire audio signal is present as an audio file and is processed and transmitted as a whole.
3. Verfahren nach Anspruch 1 , dadurch gekennzeichnet, dass nur jeweils ein Teil des Audiosignals bearbeitet und übertragen wird.3. The method according to claim 1, characterized in that only a part of the audio signal is processed and transmitted.
4. Verfahren nach einem der Ansprüche 1 bis 3, dadurch gekennzeichnet, dass das Audiosignal mittels FFT in seine spektralen Anteile zerlegt wird. 4. The method according to any one of claims 1 to 3, characterized in that the audio signal is broken down into its spectral components by means of FFT.
5. Verfahren nach einem der Ansprüche 1 bis 3, dadurch gekennzeichnet, dass das Audiosignal durch eine Anzahl n von frequenzselektierenden Filtern in seine spektralen Anteile zerlegt wird.5. The method according to any one of claims 1 to 3, characterized in that the audio signal is broken down into its spectral components by a number n of frequency-selecting filters.
6. Verfahren nach einem der Ansprüche 1 bis 5, dadurch gekennzeichnet, dass im Empfänger die nach ihrer Priorität übertragenen Gruppen einem entsprechenden Array zugeordnet, wobei die noch nicht übertragenen Werte des Arrays aus den bereits vorhandenen Werten durch Interpolation berechnet werden.6. The method according to any one of claims 1 to 5, characterized in that the groups transmitted according to their priority are assigned to a corresponding array in the receiver, the as yet untransferred values of the array being calculated from the already existing values by interpolation.
7. Verfahren nach einem der Ansprüche 1 bis 6, dadurch gekennzeichnet, dass aus den im Empfänger vorliegenden und berechneten Werten ein elektrisches Signal generiert und in ein Audiosignal umgewandelt wird. 7. The method according to any one of claims 1 to 6, characterized in that an electrical signal is generated from the values present and calculated in the receiver and converted into an audio signal.
EP03762456A 2002-07-08 2003-07-07 Method for transmitting audio signals according to the prioritizing pixel transmission method Expired - Lifetime EP1579426B1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
SI200331788T SI1579426T1 (en) 2002-07-08 2003-07-07 Method for transmitting audio signals according to the prioritizing pixel transmission method
CY20101100315T CY1109952T1 (en) 2002-07-08 2010-04-06 METHOD FOR THE TRANSFER OF AUDIO SIGNALS IN ACCORDANCE WITH THE METHOD OF HISTORICAL TRANSMISSION OF HOUSES

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10230809A DE10230809B4 (en) 2002-07-08 2002-07-08 Method for transmitting audio signals according to the method of prioritizing pixel transmission
DE10230809 2002-07-08
PCT/DE2003/002258 WO2004006224A1 (en) 2002-07-08 2003-07-07 Method for transmitting audio signals according to the prioritizing pixel transmission method

Publications (2)

Publication Number Publication Date
EP1579426A1 true EP1579426A1 (en) 2005-09-28
EP1579426B1 EP1579426B1 (en) 2010-01-06

Family

ID=29796219

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03762456A Expired - Lifetime EP1579426B1 (en) 2002-07-08 2003-07-07 Method for transmitting audio signals according to the prioritizing pixel transmission method

Country Status (16)

Country Link
US (1) US7603270B2 (en)
EP (1) EP1579426B1 (en)
JP (1) JP4637577B2 (en)
CN (1) CN1323385C (en)
AT (1) ATE454695T1 (en)
AU (1) AU2003250775A1 (en)
CY (1) CY1109952T1 (en)
DE (2) DE10230809B4 (en)
DK (1) DK1579426T3 (en)
ES (1) ES2339237T3 (en)
HK (1) HK1081714A1 (en)
PL (1) PL207103B1 (en)
PT (1) PT1579426E (en)
RU (1) RU2322706C2 (en)
SI (1) SI1579426T1 (en)
WO (1) WO2004006224A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3469567B2 (en) * 2001-09-03 2003-11-25 三菱電機株式会社 Acoustic encoding device, acoustic decoding device, acoustic encoding method, and acoustic decoding method
DE102007017254B4 (en) * 2006-11-16 2009-06-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Device for coding and decoding
EP3121814A1 (en) * 2015-07-24 2017-01-25 Sound object techology S.A. in organization A method and a system for decomposition of acoustic signal into sound objects, a sound object and its use

Family Cites Families (29)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2914974B2 (en) * 1987-02-27 1999-07-05 株式会社日立製作所 Variable rate audio signal transmission method and transmission system
US5253326A (en) * 1991-11-26 1993-10-12 Codex Corporation Prioritization method and device for speech frames coded by a linear predictive coder
JP3153933B2 (en) * 1992-06-16 2001-04-09 ソニー株式会社 Data encoding device and method and data decoding device and method
US5517511A (en) * 1992-11-30 1996-05-14 Digital Voice Systems, Inc. Digital transmission of acoustic signals over a noisy communication channel
US5675705A (en) * 1993-09-27 1997-10-07 Singhal; Tara Chand Spectrogram-feature-based speech syllable and word recognition using syllabic language dictionary
JP2797959B2 (en) * 1994-03-12 1998-09-17 日本ビクター株式会社 Multidimensional image compression / expansion method
JPH1083623A (en) * 1996-09-10 1998-03-31 Sony Corp Signal recording method, signal recorder, recording medium and signal processing method
US5886276A (en) * 1997-01-16 1999-03-23 The Board Of Trustees Of The Leland Stanford Junior University System and method for multiresolution scalable audio signal encoding
SE9700772D0 (en) * 1997-03-03 1997-03-03 Ericsson Telefon Ab L M A high resolution post processing method for a speech decoder
US6144937A (en) * 1997-07-23 2000-11-07 Texas Instruments Incorporated Noise suppression of speech by signal processing including applying a transform to time domain input sequences of digital signals representing audio information
WO1999050828A1 (en) * 1998-03-30 1999-10-07 Voxware, Inc. Low-complexity, low-delay, scalable and embedded speech and audio coding with adaptive frame loss concealment
US6584509B2 (en) * 1998-06-23 2003-06-24 Intel Corporation Recognizing audio and video streams over PPP links in the absence of an announcement protocol
JP3522137B2 (en) * 1998-12-18 2004-04-26 富士通株式会社 Variable rate encoding / decoding device
JP3797836B2 (en) * 1999-12-09 2006-07-19 株式会社東芝 Remote maintenance system
DE10008055A1 (en) * 2000-02-22 2001-08-30 Infineon Technologies Ag Data compression method
JP3576936B2 (en) * 2000-07-21 2004-10-13 株式会社ケンウッド Frequency interpolation device, frequency interpolation method, and recording medium
JP3576935B2 (en) * 2000-07-21 2004-10-13 株式会社ケンウッド Frequency thinning device, frequency thinning method and recording medium
JP2002135122A (en) * 2000-10-19 2002-05-10 Nec Corp Audio signal coding apparatus
US6952669B2 (en) * 2001-01-12 2005-10-04 Telecompression Technologies, Inc. Variable rate speech data compression
DE10152612B4 (en) * 2001-03-21 2006-02-23 T-Mobile Deutschland Gmbh Method for compressing and decompressing image data
US7359560B2 (en) * 2001-03-21 2008-04-15 T-Mobile Deutschland Gmbh Method for compression and decompression of image data with use of priority values
DE10113880B4 (en) * 2001-03-21 2004-04-29 T-Mobile Deutschland Gmbh Method for compressing and decompressing video data
US7136418B2 (en) * 2001-05-03 2006-11-14 University Of Washington Scalable and perceptually ranked signal coding and decoding
US7079658B2 (en) * 2001-06-14 2006-07-18 Ati Technologies, Inc. System and method for localization of sounds in three-dimensional space
JP3463752B2 (en) * 2001-07-25 2003-11-05 三菱電機株式会社 Acoustic encoding device, acoustic decoding device, acoustic encoding method, and acoustic decoding method
US20030236674A1 (en) * 2002-06-19 2003-12-25 Henry Raymond C. Methods and systems for compression of stored audio
DE10229706A1 (en) * 2002-07-02 2004-12-09 T-Mobile Deutschland Gmbh Method for managing memory space in a storage medium of a digital terminal device for data storage according to the method of prioritized pixel transmission
DE10229976B4 (en) * 2002-07-03 2007-06-28 T-Mobile Deutschland Gmbh Method for encrypting and decrypting digital data transmitted or stored according to the method of prioritized pixel transmission
US7359979B2 (en) * 2002-09-30 2008-04-15 Avaya Technology Corp. Packet prioritization and associated bandwidth and buffer management techniques for audio over IP

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2004006224A1 *

Also Published As

Publication number Publication date
CN1323385C (en) 2007-06-27
EP1579426B1 (en) 2010-01-06
DE10230809B4 (en) 2008-09-11
WO2004006224A1 (en) 2004-01-15
PT1579426E (en) 2010-04-08
RU2005102935A (en) 2005-10-27
US20060015346A1 (en) 2006-01-19
US7603270B2 (en) 2009-10-13
JP4637577B2 (en) 2011-02-23
HK1081714A1 (en) 2006-05-19
PL374146A1 (en) 2005-10-03
DK1579426T3 (en) 2010-05-17
CY1109952T1 (en) 2014-09-10
RU2322706C2 (en) 2008-04-20
PL207103B1 (en) 2010-11-30
DE10230809A1 (en) 2004-01-29
CN1666255A (en) 2005-09-07
DE50312330D1 (en) 2010-02-25
AU2003250775A1 (en) 2004-01-23
JP2005532580A (en) 2005-10-27
ES2339237T3 (en) 2010-05-18
SI1579426T1 (en) 2010-05-31
ATE454695T1 (en) 2010-01-15

Similar Documents

Publication Publication Date Title
EP0290581B1 (en) Process for transmitting digital audio-signals
DE60208426T2 (en) DEVICE FOR SIGNAL CODING, SIGNAL DECODING AND SYSTEM FOR DISTRIBUTING AUDIO DATA
EP1979901B1 (en) Method and arrangements for audio signal encoding
DE60012198T2 (en) ENCODING THE CORD OF THE SPECTRUM BY VARIABLE TIME / FREQUENCY RESOLUTION
DE19604273C2 (en) Method and device for performing a search in a code book with regard to the coding of a sound signal, cell communication system, cell network element and mobile cell transmitter / receiver unit
DE60103086T2 (en) IMPROVEMENT OF SOURCE DELIVERY SYSTEMS BY ADAPTIVE TRANSPOSITION
EP0251028B1 (en) Audio signal transmission method
EP1979899A1 (en) Method and arrangements for encoding audio signals
EP1023777B1 (en) Method and device for limiting a stream of audio data with a scaleable bit rate
DE2609297C3 (en) Transmission system for voice signals
DE4020656A1 (en) METHOD FOR TRANSMITTING A SIGNAL
EP1579426B1 (en) Method for transmitting audio signals according to the prioritizing pixel transmission method
DE102007041868B4 (en) Discrete synthesis using staggered Nyquist regions avoids guard-band-induced holes near a Nyquist boundary
DE2051589A1 (en) Arrangement for the synthesis of a signal
DE2826818A1 (en) METHOD AND DEVICE FOR GENERATING AN ARTIFICIAL VOICE SIGNAL
EP1020737B1 (en) Method and apparatus for electromagnetic detection of objects
DE2407072A1 (en) SAMPLING METHOD AND DEVICE FOR CARRYING OUT THE PROCESS FOR PROGRESSIVE SCANNING OF A QUASI STATIONARY SIGNAL
EP1834322B1 (en) Analog signal encoding method
EP3217392B1 (en) Rendering wideband ultrasonic signals audible
AT247018B (en) Arrangement for obtaining a digitized representation of natural language
DE1926362C (en) Facility for speech analysis and synthesis according to the vocoderpnnzip
DE19904747A1 (en) Information transmission method e.g. underwater using digital states of reference frequency channel and information frequency channels to provide bit pattern
DE2203921C3 (en) Electrical method and device for speech synthesis
EP2380169B1 (en) Method and means for the scalable improvement of the quality of a signal encoding method
AT500636A2 (en) METHOD FOR CODING ONE-DIMENSIONAL DIGITAL SIGNALS

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20041217

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REF Corresponds to:

Ref document number: 50312330

Country of ref document: DE

Date of ref document: 20100225

Kind code of ref document: P

REG Reference to a national code

Ref country code: RO

Ref legal event code: EPE

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20100330

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: LUCHS & PARTNER PATENTANWAELTE

REG Reference to a national code

Ref country code: NL

Ref legal event code: T3

Ref country code: GR

Ref legal event code: EP

Ref document number: 20100400762

Country of ref document: GR

REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2339237

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: SK

Ref legal event code: T3

Ref document number: E 7088

Country of ref document: SK

REG Reference to a national code

Ref country code: HU

Ref legal event code: AG4A

Ref document number: E007767

Country of ref document: HU

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20101007

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110201

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 50312330

Country of ref document: DE

Effective date: 20110201

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 7088

Country of ref document: SK

Effective date: 20110707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20110707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20100707

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100106

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: RO

Payment date: 20210625

Year of fee payment: 19

Ref country code: CZ

Payment date: 20210624

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20210721

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CY

Payment date: 20210630

Year of fee payment: 19

Ref country code: EE

Payment date: 20210720

Year of fee payment: 19

Ref country code: FI

Payment date: 20210720

Year of fee payment: 19

Ref country code: AT

Payment date: 20210720

Year of fee payment: 19

Ref country code: BG

Payment date: 20210719

Year of fee payment: 19

Ref country code: FR

Payment date: 20210722

Year of fee payment: 19

Ref country code: IT

Payment date: 20210730

Year of fee payment: 19

Ref country code: IE

Payment date: 20210726

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GR

Payment date: 20210720

Year of fee payment: 19

Ref country code: HU

Payment date: 20210628

Year of fee payment: 19

Ref country code: SK

Payment date: 20210701

Year of fee payment: 19

Ref country code: SI

Payment date: 20210628

Year of fee payment: 19

Ref country code: SE

Payment date: 20210721

Year of fee payment: 19

Ref country code: BE

Payment date: 20210721

Year of fee payment: 19

Ref country code: CH

Payment date: 20210723

Year of fee payment: 19

Ref country code: DK

Payment date: 20210721

Year of fee payment: 19

Ref country code: ES

Payment date: 20210819

Year of fee payment: 19

Ref country code: GB

Payment date: 20210722

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: PT

Payment date: 20210701

Year of fee payment: 19

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

REG Reference to a national code

Ref country code: DK

Ref legal event code: EBP

Effective date: 20220731

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

Ref country code: SE

Ref legal event code: EUG

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20220801

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 454695

Country of ref document: AT

Kind code of ref document: T

Effective date: 20220707

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20220707

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: RO

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: PT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230109

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: FI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

REG Reference to a national code

Ref country code: SI

Ref legal event code: KO00

Effective date: 20230314

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: HU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: GR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230209

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707

Ref country code: DK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

REG Reference to a national code

Ref country code: SK

Ref legal event code: MM4A

Ref document number: E 7088

Country of ref document: SK

Effective date: 20220707

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20230825

REG Reference to a national code

Ref country code: EE

Ref legal event code: MM4A

Ref document number: E004255

Country of ref document: EE

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20230718

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220708

Ref country code: EE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20220707