EP2126904A1 - Audio encoding method and device - Google Patents

Audio encoding method and device

Info

Publication number
EP2126904A1
EP2126904A1 EP07866270A EP07866270A EP2126904A1 EP 2126904 A1 EP2126904 A1 EP 2126904A1 EP 07866270 A EP07866270 A EP 07866270A EP 07866270 A EP07866270 A EP 07866270A EP 2126904 A1 EP2126904 A1 EP 2126904A1
Authority
EP
European Patent Office
Prior art keywords
signal
filter
frequency
limited
original
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP07866270A
Other languages
German (de)
French (fr)
Other versions
EP2126904B1 (en
Inventor
Alexandre Delattre
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nintendo European Research and Development SAS
Original Assignee
Actimagine
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Actimagine filed Critical Actimagine
Publication of EP2126904A1 publication Critical patent/EP2126904A1/en
Application granted granted Critical
Publication of EP2126904B1 publication Critical patent/EP2126904B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding

Definitions

  • the present invention relates to a method and an audio coding device. It applies, in particular, coding enriched all or part of the audio spectrum, especially for transmission over a computer network, for example Internet, or storage on a digital information carrier.
  • This method and device can be integrated into any system for compressing and decompressing an audio signal on all hardware platforms.
  • audio compression the bit rate is often reduced by limiting the bandwidth of the audio signal. Generally only low frequencies are retained because the human ear has a better resolution and spectral sensitivity in low frequency than in high frequency. Typically, only the low frequencies of the signal are kept, so that the data rate to be transferred is even lower.
  • the invention relates to a method for encoding a signal comprising at least the following steps:
  • the filter is obtained by dividing member to member of a function of the coefficients of a Fourier transform applied on the one hand to the portion of the signal original and secondly to the corresponding portion of the signal obtained by broadening the spectrum of the limited signal.
  • Fourier transforms of different sizes are used to obtain a plurality of filters corresponding to each size used.
  • the generated filter corresponding to a choice among the plurality of filters obtained by comparing the original signal, and the signal obtained by applying the filter to the signal obtained by broadening the spectrum of the limited signal.
  • the choice is extended to a collection of predetermined time filters.
  • the frequency-limited signal being encoded for transmission, the generation of the filter is made from the signal obtained by decoding and broadening the spectrum of the encoded limited signal and the original signal.
  • the invention also relates to a method for decoding a signal comprising at least the following steps:
  • a step of obtaining a reconstructed signal by convolution of the extended signal with the received temporal filter a step of obtaining a reconstructed signal by convolution of the extended signal with the received temporal filter.
  • a filter reduced in size from the generated filter is used in place of this filter generated in the step of obtaining a reconstructed signal.
  • the choice to use a reduced size filter in place of the generated filter is according to the capabilities of the decoder.
  • the invention also relates to a device for encoding a signal comprising at least:
  • the invention also relates to a device for decoding a signal comprising at least the following means:
  • means for receiving a transmitted signal characterized in that it further comprises:
  • means for obtaining an extended signal by broadening the spectrum of the decoded signal means for obtaining a reconstructed signal by convolution of the extended signal with the received temporal filter.
  • the invention also relates to a signal comprising a frequency-limited audio signal representing a frequency limited version of an original audio signal, characterized in that it furthermore comprises generation data of a temporal filter allowing the reconstruction of a signal close to the original signal when applied to an extended frequency version of the frequency-limited audio signal contained in the signal.
  • Fig. 1 represents the general architecture of the encoding method of an exemplary embodiment of the invention.
  • Fig. 2 represents the general architecture of the decoding method of the exemplary embodiment of the invention.
  • Fig. 3 represents the architecture of an embodiment of the encoder.
  • Fig. 4 represents the architecture of an embodiment of the decoder.
  • Fig. 1 represents the encoding method generally.
  • the signal 101 is the source signal to be encoded, this signal is then the original signal not limited in frequency.
  • Step 102 represents a frequency-limiting step of the signal 101.
  • This frequency limitation can, for example, be achieved by subsampling of the signal 101 previously filtered by a low-pass filter. Subsampling consists of keeping only one sample on a set of samples and to remove the other samples from the signal. A sub-sampling of a factor "n" where a sample is kept on n makes it possible to obtain a signal whose width of the spectrum will be divided by n. n is here a natural integer.
  • a rational ratio q / p it is preferable to start with oversampling in order to not lose spectral content.
  • a frequency change of a non-rational ratio one can search for the nearest rational fraction and proceed as above.
  • Other methods of limiting the spectrum of the input signal 101 can also be used as filter-based methods.
  • the resulting signal which we will call the frequency-limited signal, is then encoded in step 106.
  • Any audio encoding or compression means can be used here as, for example, an encoding according to PCM, ADPCM or other.
  • This frequency-limited signal will be supplied to the multiplexer 108 for transmission to the decoder.
  • the signal limited in frequency and encoded at the output of the compression module 106 is also provided at the input of a decoding module 107.
  • This module performs the inverse operation of the encoding module 106 and makes it possible to construct a limited version of the signal. frequency identical to the version to which the decoder will have access when it also performs this operation of decoding the limited signal and encoded it will receive.
  • the limited signal thus decoded is then restored to the original spectral extent by a frequency-widening module 103.
  • This frequency widening may, for example, consist of a simple oversampling of the input signal by the insertion of zero-valued samples between the samples of the input signal.
  • This extended frequency signal from the frequency-widening module 103 is then supplied to a filter generation module 104.
  • This filter generation module 104 also receives the original signal 101 and calculates a time filter allowing, when it is applied to the extended signal from the frequency-widening module 103, to shape it to approach the original signal.
  • the filter thus calculated is then supplied to the multiplexer 108 after an optional compression step 105.
  • Fig. 2 generally represents the corresponding decoding method.
  • the decoder therefore receives the signal from the multiplexer 108 of the encoder. It demultiplexes it to obtain on the one hand the encoded frequency limited signal, called SIb, and the coefficients of the filter F, contained in the transmitted signal.
  • SIb is then decoded by a decoding or decompression module 202 which is functionally equivalent to the module 107 of FIG. 1.
  • the signal is frequency-expanded by module 203 operably equivalent to module 103 of FIG. 1. A decoded and extended version of the signal is thus obtained.
  • the coefficients of the filter F are decoded if they had been encoded or compressed by a decompression module 201, and the filter obtained is applied to the extended time signal in a signal conditioning module 204. then an output signal close to the original signal.
  • This treatment is simple to implement because of the temporal nature of the filter to be applied to the signal for fitness.
  • the transmitted filter and thus applied during the reconstruction of the signal, is transmitted periodically and changes over time.
  • This filter is adapted to a portion of the signal to which it applies. It is thus possible to calculate for each signal portion a time filter particularly adapted according to the dynamic spectral characteristics of this portion of signal. In particular, it is possible to have several types of time filter generators and to select for each portion of signal the filter giving the best result for this portion. This is possible because the filter generation module has on the one hand the original signal and on the other hand the extended signal as it will be reconstructed by the decoder, so it is able, in the case where it is generated by several different filters, to compare the signal obtained by application of each filter to the extended signal portion and the original signal which is to approach closer. This method of filter generation is therefore not limited to choosing a type of filter determined for the whole signal but allows to change the type of filter according to the characteristics of each portion of signal.
  • This signal is then encoded, for example by a method of PCM ("Puise Code Modulation") type, by the module 311 which will then be compressed, for example by an ADPCM the module 312. This gives the subsampled signal containing the low frequencies of the original signal 301. This signal is sent to the multiplexer 314 to be transmitted to the decoder.
  • PCM Peise Code Modulation
  • this signal is transmitted to a decoding module 313.
  • This signal which will be used for the generation of the filter F will thus allow to take into account the artifacts resulting from these phases of coding and decoding, compression and decompression.
  • This signal is then extended in frequency by inserting n-1 zeros between each sample of the time signal in the module 303. In this way, a signal of the same spectral extent as the original signal is reconstructed. According to the Nyquist theorem, we obtain a n-order spectrum folding.
  • the signal is downsampled from an order 2 to the encoding and oversampled from an order 2 to the decoding.
  • the spectrum is duplicated "mirrored" by axial symmetry in the frequency domain.
  • a Fourier transform is performed on the frequency-extended time signal from the module 303.
  • a sliding fast Fourier transform is performed on working windows of given and variable sizes. These sizes are typically 128, 256, 512 samples but can be of any size, even if powers of two will be used to simplify the calculations.
  • a same Fourier transform calculation is performed on the original signal in the module 306.
  • a member-to-member division 305 is then performed between the modules of the Fourier transform coefficients obtained by the steps 304 and 306 to generate by inverse Fourier transforms temporal filters of sizes proportional to those of the windows used, ie 128, 256 or 512.
  • This step therefore generates several filters of different sizes among which we will have to choose the filter finally used. It will be seen that this selection step is performed by the module 309. Since the coefficients of the ratio between the spectra are real symmetrical in the frequency space, the equivalent filter F is then, in the time domain, real and symmetrical.
  • This property of symmetry can be used to transmit only half of the coefficients, the other being deduced by symmetry.
  • Obtaining a symmetrical real filter also makes it possible to reduce the number of operations necessary during the convolution of the received signal extended by the filter in the decoder.
  • Other embodiments make it possible to obtain real unsymmetrical filters. For example, if the temporal signal in a working window is frequency-limited, it is advantageous to iteratively determine the parameters of an infinite impulse response filter, Chebychev low-pass from the spectra from steps 304 and 306. and the cutoff frequency of the window.
  • a module 308 will offer other types of filters.
  • it can offer linear, cubic or other filters. Indeed, these filters are known to allow oversampling.
  • the module 308 therefore contains an arbitrary number of such filters that can be used.
  • the choice module 309 will therefore have as input a collection of filters.
  • the module 309 can compare the application of the different filters to the reconstructed signal coming from the module 303 with the original signal to choose the filter giving, on the signal portion considered, the best output signal, that is to say the spectrally closest to the original signal. For example, it is possible to relate the spectrum obtained by applying the filter to the signal from the module 303 and the spectrum of the same portion of the original signal. We then choose the filter generating the minimum of a function of the distortion.
  • This portion of the signal will have to be larger than the largest window used for calculating the filters. It will be possible to use typically a working window size of 512 samples. The size of this working window may also vary depending on the signal. Indeed, a large working window size can be used for the encoding of a substantially stationary signal portion while a shorter window will be more suitable for a more dynamic signal portion to better take into account the rapid variations. . It is this part that makes it possible to select, for each portion of the signal, the most relevant filter allowing the best reconstruction by the decoder of the signal and to get closer to the original signal.
  • the module 310 will quantify the spectral coefficients of the filter that will be encoded, for example, using a Huffman table to optimize the data to be transmitted.
  • Multiplexer 314 will thus multiplex with each portion of the signal, the most relevant filter for the decoding of this portion of signal.
  • This filter being chosen either from the collection of filters of different sizes generated by analysis of this portion of the signal, or from the collection also comprises a series of determined, typically linear, filters enabling reconstruction, which may be chosen if they prove more interesting for the reconstruction by the decoder of the signal portion.
  • the generated filter is one of the determined filters, it is possible to transmit only an identifier identifying this filter from the collection of determined filters provided by the module 308, as well as possible parameters of the filter.
  • the coefficients of these determined filters are not calculated according to the portion of signal to which we want to apply them, it is unnecessary to carry these coefficients that can be known to the decoder.
  • the bandwidth for transporting information relating to the filter is reduced in this case to a simple identifier of the filter.
  • Fig. 4 represents the corresponding decoding in the particular embodiment described.
  • the signal is received by the decoder which demultiplexes the signal.
  • the audio signal SIb is then decoded by the module 404 and then oversampled by a factor n by the insertion of n-1 zero samples between the samples received by the module 405.
  • the spectral coefficients of the filter F are dequantized and decoded according to the Huffman tables by the module 401.
  • the size of the filter can be adapted by the module 402 of the decoder to its computing or memory capacity or any possible hardware limitation.
  • a decoder with few resources can use a subsampled filter which will allow it to reduce the operations during the application of the filter.
  • the subsampled filter may also be generated by the encoder according to the resources of the transmission channel or the resources of the decoder, provided of course that the latter information is held by the encoder.
  • the spectrum of the filter can be reduced to decoding to perform a smaller oversampling (n-1, n-2 etc. ..) depending on the hardware sound output capabilities of the decoder such as power or sound output capabilities.
  • the module 403 then performs an inverse Fourier transform on the spectral coefficients of the filter to obtain the real filter in the time domain.
  • the filter is moreover symmetrical which makes it possible to reduce the data transported for the transmission of the filter.
  • the module 406 operates the convolution of the oversampled signal from the module 405 with the filter thus reconstituted to obtain the resulting signal.
  • This convolution is particularly greedy in calculation because the oversampling is done by inserting null values.
  • the fact that the filter is real, and even symmetrical in the preferred embodiment also reduces the number of operations required for this convolution.
  • the invention offers the advantage of performing a reshaping, not only of the high part of the spectrum reconstituted from the transmitted lower part but of the whole of the signal thus reconstituted. In this way, it allows to model the part of the spectrum not transmitted but also to correct artifacts due to the various operations of compression, decompression, encoding and decoding of the transmitted low frequency part.
  • a secondary advantage of the invention is the ability to dynamically adapt the filters used according to the nature of each portion of the signal thanks to the module allowing the choice of the best filter, in terms of sound quality and "machine time” used, among several for each portion of the signal.

Abstract

The invention relates to an audio encoding device and method comprising the transmission of data representing a frequency-limited signal and, in addition, information relating to a temporal filter that can be applied to all of the expanded signal, both to the transmitted low frequency part and to the reconstructed high frequency part. The application of said filter enables the reconstructed high frequency part to be reshaped and compression artifacts present in the transmitted low frequency part to be corrected. In this way, the temporal filter can be applied easily and at low cost to all or part of the reconstructed signal in order to obtain a signal with high perceived quality.

Description

Procédé et dispositif de codage audio Audio coding method and device
La présente invention concerne un procédé et un dispositif de codage audio. Elle s'applique, en particulier, au codage avec enrichissement de tout ou partie du spectre audio, notamment en vue de leur transmission sur un réseau informatique, par exemple Internet, ou de leur stockage sur un support d'informations numériques. Ce procédé et ce dispositif peuvent être intégrés à tout système permettant de compresser puis décompresser un signal audio sur toutes plates-formes matérielles. En compressions audio, le débit est souvent réduit en limitant la bande passante du signal audio. Généralement on ne conserve que les basses fréquences car l'oreille humaine a une meilleure résolution et sensibilité spectrales en basse fréquence qu'en haute fréquence. Typiquement on ne conserve que les basses fréquences du signal, ainsi le débit des données à transférer est d'autant plus faible. Comme les harmoniques contenues dans les basses fréquences sont aussi présentes dans les hautes fréquences, certaines méthodes de l'état de l'art tentent, à partir du signal limité aux basses fréquences, d'extraire des harmoniques qui permettent de recréer les hautes fréquences artificiellement. Ces méthodes reposent généralement sur un enrichissement spectral consistant à recréer un spectre haute fréquence par transposition du spectre basse fréquence, ce spectre haute fréquence étant remis en forme spectralement. Le signal résultant se compose donc, pour la partie basse fréquence, du signal basse fréquence reçu et pour la partie haute fréquence de l'enrichissement remis en forme.The present invention relates to a method and an audio coding device. It applies, in particular, coding enriched all or part of the audio spectrum, especially for transmission over a computer network, for example Internet, or storage on a digital information carrier. This method and device can be integrated into any system for compressing and decompressing an audio signal on all hardware platforms. In audio compression, the bit rate is often reduced by limiting the bandwidth of the audio signal. Generally only low frequencies are retained because the human ear has a better resolution and spectral sensitivity in low frequency than in high frequency. Typically, only the low frequencies of the signal are kept, so that the data rate to be transferred is even lower. As the harmonics contained in the low frequencies are also present in the high frequencies, some methods of the state of the art attempt, starting from the signal limited to the low frequencies, to extract harmonics which make it possible to recreate the high frequencies artificially . These methods are generally based on a spectral enrichment consisting in recreating a high frequency spectrum by transposition of the low frequency spectrum, this high frequency spectrum being spectrally re-formed. The resulting signal therefore consists, for the low frequency part, of the received low frequency signal and for the high frequency portion of the reformed enrichment.
Il s'avère que la compression et la méthode utilisée pour compresser et limiter la bande de fréquence du signal initial génèrent des artefacts nuisant à la qualité du signal. D'autre part, la reconstitution d'un signal de qualité en réception doit permettre d'obtenir la meilleure qualité perçue possible tout en ne nécessitant qu'une faible bande passante de données transmises et un traitement simple et rapide à la réception.It turns out that the compression and the method used to compress and limit the frequency band of the initial signal generate artifacts affecting the quality of the signal. On the other hand, the reconstitution of a reception quality signal must make it possible to obtain the best perceived quality possible while requiring only a small bandwidth of transmitted data and a simple and fast processing at the reception.
Ce problème est avantageusement résolu par la transmission, en sus des données représentant le signal limité en fréquences, d'informations relatives à un filtre temporel devant être appliqué à l'intégralité du signal élargi, tant dans sa partie basse fréquence transmise que dans sa partie haute fréquence reconstituée. L'application de ce filtre permettant la remise en forme de la partie haute fréquence reconstituée et la correction d'artefacts de compression présents dans la partie basse fréquence transmise. De cette façon, l'application du filtre temporel, simple et peu coûteuse, à l'intégralité du signal reconstitué, permet d'obtenir un signal de bonne qualité perçue.This problem is advantageously solved by the transmission, in addition to the data representing the frequency-limited signal, of information relating to a temporal filter to be applied to the entirety of the expanded signal, both in its transmitted low frequency part and in its part. high frequency reconstituted. The application of this filter allows the reshaping of the reconstituted high frequency part and the correction of compression artifacts present in the transmitted low frequency part. In this way, the application of the time filter, simple and inexpensive, to the entire reconstituted signal, provides a signal of good quality perceived.
L'invention concerne un procédé d'encodage d'un signal comportant au moins les étapes suivantes :The invention relates to a method for encoding a signal comprising at least the following steps:
- une étape d'obtention d'un signal limité en fréquence, la réduction du spectre du signal original étant obtenue par suppression des hautes fréquences,a step of obtaining a signal limited in frequency, the reduction of the spectrum of the original signal being obtained by suppressing the high frequencies,
- une étape de génération d'un filtre temporel permettant de retrouver un signal proche spectralement du signal original lorsqu'il est appliqué au signal obtenu par élargissement du spectre du signal limité. Selon un mode particulier de réalisation de l'invention, pour une portion du signal original donnée, le filtre est obtenu par division membre à membre d'une fonction des coefficients d'une transformée de Fourier appliquée d'une part à la portion du signal original et d'autre part à la portion correspondante du signal obtenu par élargissement du spectre du signal limité.a step of generating a temporal filter making it possible to recover a signal that is close to the spectral signal of the original signal when it is applied to the signal obtained by broadening the spectrum of the limited signal. According to a particular embodiment of the invention, for a portion of the given original signal, the filter is obtained by dividing member to member of a function of the coefficients of a Fourier transform applied on the one hand to the portion of the signal original and secondly to the corresponding portion of the signal obtained by broadening the spectrum of the limited signal.
Selon un mode particulier de réalisation de l'invention, des transformées de Fourier de tailles différentes sont utilisées pour l'obtention d'une pluralité de filtres correspondant à chaque taille utilisée. Le filtre généré correspondant à un choix parmi la pluralité de filtres obtenus par comparaison du signal original, et du signal obtenu par application du filtre au signal obtenu par élargissement du spectre du signal limité.According to a particular embodiment of the invention, Fourier transforms of different sizes are used to obtain a plurality of filters corresponding to each size used. The generated filter corresponding to a choice among the plurality of filters obtained by comparing the original signal, and the signal obtained by applying the filter to the signal obtained by broadening the spectrum of the limited signal.
Selon un mode particulier de réalisation de l'invention, le choix est étendu à une collection de filtres temporels prédéterminés. Selon un mode particulier de réalisation de l'invention, le signal limité en fréquence étant encodé en vue de sa transmission, la génération du filtre se fait à partir du signal obtenu par décodage et élargissement du spectre du signal limité encodé et du signal original.According to a particular embodiment of the invention, the choice is extended to a collection of predetermined time filters. According to a particular embodiment of the invention, the frequency-limited signal being encoded for transmission, the generation of the filter is made from the signal obtained by decoding and broadening the spectrum of the encoded limited signal and the original signal.
L'invention concerne également un procédé de décodage d'un signal comportant au moins les étapes suivantes :The invention also relates to a method for decoding a signal comprising at least the following steps:
- une étape de réception d'un signal transmis,a step of receiving a transmitted signal,
- une étape de réception d'un filtre temporel relatif au signal reçu,a step of receiving a temporal filter relating to the received signal,
- une étape d'obtention d'un signal décodé par décodage du signal reçu,a step of obtaining a decoded signal by decoding the received signal,
- une étape d'obtention d'un signal étendu par élargissement du spectre du signal décodé,a step of obtaining an extended signal by broadening the spectrum of the decoded signal,
- une étape d'obtention d'un signal reconstruit par convolution du signal étendu avec le filtre temporel reçu.a step of obtaining a reconstructed signal by convolution of the extended signal with the received temporal filter.
Selon un mode particulier de réalisation de l'invention, un filtre réduit en taille à partir du filtre généré est utilisé à la place de ce filtre généré dans l'étape d'obtention d'un signal reconstruit.According to a particular embodiment of the invention, a filter reduced in size from the generated filter is used in place of this filter generated in the step of obtaining a reconstructed signal.
Selon un mode particulier de réalisation de l'invention, le choix d'utiliser un filtre de taille réduite à la place du filtre généré se fait en fonction des capacités du décodeur.According to a particular embodiment of the invention, the choice to use a reduced size filter in place of the generated filter is according to the capabilities of the decoder.
L'invention concerne également un dispositif d'encodage d'un signal comportant au moins :The invention also relates to a device for encoding a signal comprising at least:
- des moyens d'obtention d'un signal limité en fréquence, la réduction du spectre du signal original étant obtenue par suppression des hautes fréquences,means for obtaining a signal limited in frequency, the reduction of the spectrum of the original signal being obtained by suppressing the high frequencies,
- des moyens d'obtention d'un signal limité en fréquence encodé par encodage du signal limité en fréquence,means for obtaining a signal limited in frequency encoded by encoding the signal limited in frequency,
- des moyens de génération d'un filtre temporel permettant de retrouver un signal proche du signal original lorsqu'il est appliqué au signal obtenu par décodage et élargissement du spectre du signal limité. L'invention concerne également un dispositif de décodage d'un signal comportant au moins les moyens suivants :means for generating a temporal filter making it possible to recover a signal close to the original signal when it is applied to the signal obtained by decoding and broadening the spectrum of the limited signal. The invention also relates to a device for decoding a signal comprising at least the following means:
- des moyens de réception d'un signal transmis, caractérisé en ce qu'il comprend en outre :means for receiving a transmitted signal, characterized in that it further comprises:
- des moyens de réception d'un filtre temporel relatif au signal reçu,means for receiving a temporal filter relating to the signal received,
- des moyens d'obtention d'un signal décodé par décodage du signal reçu,means for obtaining a decoded signal by decoding the received signal,
- des moyens d'obtention d'un signal étendu par élargissement du spectre du signal décodé, - des moyens d'obtention d'un signal reconstruit par convolution du signal étendu avec le filtre temporel reçu.means for obtaining an extended signal by broadening the spectrum of the decoded signal; means for obtaining a reconstructed signal by convolution of the extended signal with the received temporal filter.
L'invention concerne également un signal comportant un signal audio limité en fréquence représentant une version limitée en fréquence d'un signal audio original caractérisé en ce qu'il comporte en outre des données de génération d'un filtre temporel permettant la reconstruction d'un signal proche du signal original lorsqu'il est appliqué à une version étendue en fréquence du signal audio limité en fréquence contenu dans le signal.The invention also relates to a signal comprising a frequency-limited audio signal representing a frequency limited version of an original audio signal, characterized in that it furthermore comprises generation data of a temporal filter allowing the reconstruction of a signal close to the original signal when applied to an extended frequency version of the frequency-limited audio signal contained in the signal.
Les caractéristiques de l'invention mentionnées ci-dessus, ainsi que d'autres, apparaîtront plus clairement à la lecture de la description suivante d'un exemple de réalisation, ladite description étant faite en relation avec les dessins joints, parmi lesquels :The characteristics of the invention mentioned above, as well as others, will appear more clearly on reading the following description of an exemplary embodiment, said description being given in relation to the attached drawings, among which:
La Fig. 1 représente l'architecture générale de la méthode d'encodage d'un exemple de réalisation de l'invention.Fig. 1 represents the general architecture of the encoding method of an exemplary embodiment of the invention.
La Fig. 2 représente l'architecture générale de la méthode de décodage de l'exemple de réalisation de l'invention.Fig. 2 represents the general architecture of the decoding method of the exemplary embodiment of the invention.
La Fig. 3 représente l'architecture d'un mode de réalisation de l'encodeur. La Fig. 4 représente l'architecture d'un mode de réalisation du décodeur.Fig. 3 represents the architecture of an embodiment of the encoder. Fig. 4 represents the architecture of an embodiment of the decoder.
La Fig. 1 représente le procédé d'encodage de manière générale. Le signal 101 est le signal source devant être encodé, ce signal est alors le signal original non limité en fréquence. L'étape 102 représente une étape de limitation en fréquence du signal 101. Cette limitation en fréquence peut, par exemple, être réalisée par un sous- échantillonnage du signal 101 préalablement filtré par un filtre passe-bas. Un sous- échantillonnage consiste à ne garder qu'un échantillon sur un ensemble d'échantillons et à supprimer du signal les autres échantillons. Un sous-échantillonnage d'un facteur « n » où l'on garde un échantillon sur n permet d'obtenir un signal dont la largeur du spectre sera divisée par n. n est ici un entier naturel. Il est aussi possible d' effectuer un sous-échantillonnage d'un rapport rationnel q/p, on sur-échantillonne d'un facteur p puis on sous-échantillonne d'un facteur q, il est préférable de commencer par le suréchantillonnage pour ne pas perdre de contenu spectral. Pour un changement de fréquence d'un rapport non rationnel, on peut chercher la fraction rationnelle la plus proche et procéder comme ci-dessus. D'autres méthodes de limitation du spectre du signal d'entrée 101 peuvent également être utilisées comme des méthodes à base de filtrage. Le signal résultant, que nous appellerons le signal limité en fréquence, est alors encodé lors de l'étape 106. Tout moyen d'encodage ou de compression audio peut être ici employé comme, par exemple, un encodage selon les normes PCM, ADPCM ou autres. Ce signal limité en fréquence sera fourni au multiplexeur 108 en vue de sa transmission au décodeur. Le signal limité en fréquence et encodé en sortie du module de compression 106 est également fourni en entrée d'un module de décodage 107. Ce module effectue l'opération inverse du module d'encodage 106 et permet de construire une version du signal limité en fréquence identique à la version à laquelle le décodeur aura accès lorsqu'il effectuera également cette opération de décodage du signal limité et encodé qu'il recevra. Le signal limité ainsi décodé est alors restauré dans l'étendue spectrale d'origine par un module d'élargissement en fréquence 103. Cet élargissement en fréquence peut, par exemple, consister en un simple sur-échantillonnage du signal d'entrée par l'insertion d'échantillons de valeur nulle entre les échantillons du signal d'entrée. Toute autre méthode d'élargissement du spectre du signal peut également être utilisée. Ce signal à fréquence étendue issu du module d'élargissement en fréquence 103 est alors fourni à un module de génération de filtre 104. Ce module de génération de filtre 104 reçoit également le signal original 101 et calcule un filtre temporel permettant, lorsqu'il est appliqué au signal étendu issu du module d'élargissement en fréquence 103, de mettre en forme celui-ci pour se rapprocher du signal original. Le filtre ainsi calculé est alors fourni au multiplexeur 108 après une étape optionnelle de compression 105.Fig. 1 represents the encoding method generally. The signal 101 is the source signal to be encoded, this signal is then the original signal not limited in frequency. Step 102 represents a frequency-limiting step of the signal 101. This frequency limitation can, for example, be achieved by subsampling of the signal 101 previously filtered by a low-pass filter. Subsampling consists of keeping only one sample on a set of samples and to remove the other samples from the signal. A sub-sampling of a factor "n" where a sample is kept on n makes it possible to obtain a signal whose width of the spectrum will be divided by n. n is here a natural integer. It is also possible to sub-sample a rational ratio q / p, to oversample by a factor p and then to sub-sample by a factor q, it is preferable to start with oversampling in order to not lose spectral content. For a frequency change of a non-rational ratio, one can search for the nearest rational fraction and proceed as above. Other methods of limiting the spectrum of the input signal 101 can also be used as filter-based methods. The resulting signal, which we will call the frequency-limited signal, is then encoded in step 106. Any audio encoding or compression means can be used here as, for example, an encoding according to PCM, ADPCM or other. This frequency-limited signal will be supplied to the multiplexer 108 for transmission to the decoder. The signal limited in frequency and encoded at the output of the compression module 106 is also provided at the input of a decoding module 107. This module performs the inverse operation of the encoding module 106 and makes it possible to construct a limited version of the signal. frequency identical to the version to which the decoder will have access when it also performs this operation of decoding the limited signal and encoded it will receive. The limited signal thus decoded is then restored to the original spectral extent by a frequency-widening module 103. This frequency widening may, for example, consist of a simple oversampling of the input signal by the insertion of zero-valued samples between the samples of the input signal. Any other method of broadening the signal spectrum can also be used. This extended frequency signal from the frequency-widening module 103 is then supplied to a filter generation module 104. This filter generation module 104 also receives the original signal 101 and calculates a time filter allowing, when it is applied to the extended signal from the frequency-widening module 103, to shape it to approach the original signal. The filter thus calculated is then supplied to the multiplexer 108 after an optional compression step 105.
De cette manière, il est possible de transporter une version limitée en fréquence et compressée du signal à transmettre et les coefficients d'un filtre temporel. Ce filtre temporel permettant, une fois appliqué au signal décompressé et étendu en fréquence, de remettre celui-ci en forme pour retrouver un signal étendu proche du signal original. Le calcul du filtre se faisant sur le signal original et sur le signal tel qu'il sera obtenu par le décodeur suite à la décompression et à l'élargissement en fréquence permet de corriger les défauts introduits par ces deux phases de traitement. D'une part, le filtre étant appliqué au signal reconstruit dans toute sa plage de fréquence permet de corriger certains artefacts de compression sur la partie basse fréquence transmise. D'autre part, il remet également en forme la partie haute fréquence, non transmise, reconstruite par élargissement en fréquence.In this way, it is possible to carry a limited frequency and compressed version of the signal to be transmitted and the coefficients of a temporal filter. This temporal filter, once applied to the signal decompressed and extended in frequency, to put it in shape to find an extended signal close to the original signal. The calculation of the filter being done on the original signal and on the signal as it will be obtained by the decoder following the decompression and the widening in frequency makes it possible to correct the defects introduced by these two phases of treatment. On the one hand, the filter being applied to the reconstructed signal throughout its frequency range makes it possible to correct certain compression artifacts on the transmitted low frequency part. On the other hand, it also reshapes the high frequency part, not transmitted, reconstructed by frequency widening.
La Fig. 2 représente de manière générale le procédé de décodage correspondant. Le décodeur reçoit donc le signal issu du multiplexeur 108 du codeur. Il le démultiplexe pour obtenir d'une part le signal limité en fréquence encodé, appelé SIb, et les coefficients du filtre F, contenus dans le signal transmis. Le signal SIb est alors décodé par un module de décodage ou de décompression 202 équivalent fonctionnellement au module 107 de la Fig. 1. Une fois décodé, le signal est étendu en fréquence par le module 203 équivalent fonctionnellement au module 103 de la Fig. 1. On obtient donc une version décodée et étendue en fréquence du signal. D'autre part, les coefficients du filtre F sont décodés s'ils avaient été encodés ou compressés par un module de décompression 201, et le filtre obtenu est appliqué au signal temporel étendu dans un module de mise en forme du signal 204. On obtient alors un signal en sortie proche du signal original. Ce traitement est simple à mettre en œuvre du fait de la nature temporelle du filtre à appliquer au signal pour la remise en forme.Fig. 2 generally represents the corresponding decoding method. The decoder therefore receives the signal from the multiplexer 108 of the encoder. It demultiplexes it to obtain on the one hand the encoded frequency limited signal, called SIb, and the coefficients of the filter F, contained in the transmitted signal. The signal SIb is then decoded by a decoding or decompression module 202 which is functionally equivalent to the module 107 of FIG. 1. Once decoded, the signal is frequency-expanded by module 203 operably equivalent to module 103 of FIG. 1. A decoded and extended version of the signal is thus obtained. On the other hand, the coefficients of the filter F are decoded if they had been encoded or compressed by a decompression module 201, and the filter obtained is applied to the extended time signal in a signal conditioning module 204. then an output signal close to the original signal. This treatment is simple to implement because of the temporal nature of the filter to be applied to the signal for fitness.
Le filtre transmis, et donc appliqué lors de la reconstruction du signal, est transmis périodiquement et change dans le temps. Ce filtre est donc adapté à une portion du signal à laquelle il s'applique. Il est ainsi possible de calculer pour chaque portion de signal un filtre temporel particulièrement adapté en fonction des caractéristiques spectrales dynamiques de cette portion de signal. En particulier, il est possible d'avoir plusieurs types de générateurs de filtres temporels et de sélectionner pour chaque portion de signal le filtre donnant le meilleur résultat pour cette portion. Ceci est possible car le module de génération de filtre possède d'une part le signal original et d'autre part le signal étendu tel qu'il sera reconstruit par le décodeur, il est donc en mesure, dans le cas où il est généré par plusieurs filtres différents, de comparer le signal obtenu par application de chaque filtre à la portion de signal étendue et le signal original dont on cherche à s'approcher au plus près. Cette méthode de génération de filtre ne se limite donc pas à choisir un type de filtre déterminé pour l'ensemble du signal mais permet de changer de type de filtre en fonction des caractéristiques de chaque portion de signal.The transmitted filter, and thus applied during the reconstruction of the signal, is transmitted periodically and changes over time. This filter is adapted to a portion of the signal to which it applies. It is thus possible to calculate for each signal portion a time filter particularly adapted according to the dynamic spectral characteristics of this portion of signal. In particular, it is possible to have several types of time filter generators and to select for each portion of signal the filter giving the best result for this portion. This is possible because the filter generation module has on the one hand the original signal and on the other hand the extended signal as it will be reconstructed by the decoder, so it is able, in the case where it is generated by several different filters, to compare the signal obtained by application of each filter to the extended signal portion and the original signal which is to approach closer. This method of filter generation is therefore not limited to choosing a type of filter determined for the whole signal but allows to change the type of filter according to the characteristics of each portion of signal.
Un mode particulier de réalisation de l'invention va maintenant être décrit en détail à l'aide des Fig. 3 et 4. Dans ce mode de réalisation, on cherche à partir d'un signal échantillonné à une fréquence donnée 301, par exemple 32 kHz, à obtenir le signal limité à ses basses fréquences nommé SIb. On cherche également à déterminer un filtre F permettant de mettre en forme le signal obtenu en étendant en fréquence le signal SIb. Le signal original 301 est filtré par un filtre passe-bas et sous-échantillonné d'un facteur n par le module de sous-échantillonnage 302. On ne conserve du signal original qu'un échantillon sur n, où n est un entier naturel. Dans la pratique, n n'excède généralement pas 4. Le signal perd alors en étendue spectrale et, par exemple, pour n = 2, on obtient un signal échantillonné à 16 kHz. Ce signal est ensuite encodé, par exemple par une méthode de type PCM (« Puise Code Modulation »), par le module 311 qui sera ensuite compressé, par exemple par un ADPCM le module 312. On obtient ainsi le signal sous-échantillonné contenant les basses fréquences du signal original 301. Ce signal est envoyé au multiplexeur 314 pour être émis vers le décodeur.A particular embodiment of the invention will now be described in detail with the aid of FIGS. 3 and 4. In this embodiment, it is sought from a signal sampled at a given frequency 301, for example 32 kHz, to obtain the signal limited to its low frequencies named SIb. We also seek to determine a filter F for shaping the signal obtained by extending the signal SIb frequency. The original signal 301 is filtered by a low-pass filter and downsampled by a factor n by the subsampling module 302. Only the original signal is retained on n, where n is a natural integer. In practice, n generally does not exceed 4. The signal then loses in spectral range and, for example, for n = 2, a signal sampled at 16 kHz is obtained. This signal is then encoded, for example by a method of PCM ("Puise Code Modulation") type, by the module 311 which will then be compressed, for example by an ADPCM the module 312. This gives the subsampled signal containing the low frequencies of the original signal 301. This signal is sent to the multiplexer 314 to be transmitted to the decoder.
En parallèle, ce signal est transmis à un module de décodage 313. On simule de cette façon, dans l'encodeur, le signal que le décodeur obtiendra à partir du signal qui lui sera envoyé. Ce signal qui va être utilisé pour la génération du filtre F va donc permettre de tenir compte des artefacts résultant de ces phases de codage et de décodage, de compression et de décompression. Ce signal est ensuite étendu en fréquence par insertion de n-1 zéros entre chaque échantillon du signal temporel dans le module 303. De cette manière, on reconstruit un signal de la même étendue spectrale que le signal d'origine. D'après le théorème de Nyquist, on obtient un repliement de spectre d'ordre n. Par exemple, pour n=2, le signal est sous- échantillonné d'un ordre 2 à l'encodage et sur-échantillonné d'un ordre 2 au décodage. Le spectre est dupliqué « en miroir » par une symétrie axiale dans le domaine des fréquences. Dans le module 304, une transformée de Fourier est effectuée sur le signal temporel étendu en fréquence issu du module 303. En fait, une transformée de Fourier rapide glissante est effectuée sur des fenêtres de travail de tailles données et variables. Ces tailles sont typiquement de 128, 256, 512 échantillons mais peuvent être de tailles quelconques même si on utilisera préférentiellement des puissances de deux pour simplifier les calculs. On calcule ensuite les modules de ces transformées appliquées à ces fenêtres. Un même calcul de transformée de Fourier est effectué sur le signal original dans le module 306.In parallel, this signal is transmitted to a decoding module 313. This way, in the encoder, the signal that the decoder will obtain from the signal sent to it is simulated. This signal which will be used for the generation of the filter F will thus allow to take into account the artifacts resulting from these phases of coding and decoding, compression and decompression. This signal is then extended in frequency by inserting n-1 zeros between each sample of the time signal in the module 303. In this way, a signal of the same spectral extent as the original signal is reconstructed. According to the Nyquist theorem, we obtain a n-order spectrum folding. For example, for n = 2, the signal is downsampled from an order 2 to the encoding and oversampled from an order 2 to the decoding. The spectrum is duplicated "mirrored" by axial symmetry in the frequency domain. In the module 304, a Fourier transform is performed on the frequency-extended time signal from the module 303. In fact, a sliding fast Fourier transform is performed on working windows of given and variable sizes. These sizes are typically 128, 256, 512 samples but can be of any size, even if powers of two will be used to simplify the calculations. The modules of these transformed to these windows. A same Fourier transform calculation is performed on the original signal in the module 306.
Une division membre à membre 305 est alors effectuée entre les modules des coefficients des transformée de Fourier obtenues par les étapes 304 et 306 pour générer par transformées de Fourier inverses des filtres temporels de tailles proportionnelles à celles des fenêtres utilisées, donc 128, 256 ou 512. Plus la taille de la fenêtre choisie sera grande plus le filtre comportera de coefficients et sera plus précis mais plus son application sera coûteuse en calcul au décodage. Cette étape génère donc plusieurs filtres de différentes tailles parmi lesquelles il va falloir choisir le filtre finalement utilisé. On verra que cette étape de choix est effectuée par le module 309. Comme les coefficients du rapport entre les spectres sont réels symétriques dans l'espace des fréquences, le filtre F équivalent est alors, dans le domaine temporel, réel et symétrique. Cette propriété de symétrie peut être utilisée pour ne transmettre que la moitié des coefficients, l'autre se déduisant par symétrie. L'obtention d'un filtre réel symétrique permet aussi de réduire le nombre d'opérations nécessaire lors de la convolution du signal reçu étendu par le filtre dans le décodeur. D'autres modes de réalisation permettent d'obtenir des filtres réels non symétriques. Par exemple, si le signal temporel dans une fenêtre de travail est limité en fréquence, on peut avantageusement déterminer de manière itérative les paramètres d'un filtre à réponse impulsionnelle infinie, passe-bas de Tchebychev à partir des spectres issus des étapes 304 et 306 et de la fréquence de coupure de la fenêtre.A member-to-member division 305 is then performed between the modules of the Fourier transform coefficients obtained by the steps 304 and 306 to generate by inverse Fourier transforms temporal filters of sizes proportional to those of the windows used, ie 128, 256 or 512. The more the size of the chosen window, the more the filter will have coefficients and will be more accurate, but the more expensive it will be to decode. This step therefore generates several filters of different sizes among which we will have to choose the filter finally used. It will be seen that this selection step is performed by the module 309. Since the coefficients of the ratio between the spectra are real symmetrical in the frequency space, the equivalent filter F is then, in the time domain, real and symmetrical. This property of symmetry can be used to transmit only half of the coefficients, the other being deduced by symmetry. Obtaining a symmetrical real filter also makes it possible to reduce the number of operations necessary during the convolution of the received signal extended by the filter in the decoder. Other embodiments make it possible to obtain real unsymmetrical filters. For example, if the temporal signal in a working window is frequency-limited, it is advantageous to iteratively determine the parameters of an infinite impulse response filter, Chebychev low-pass from the spectra from steps 304 and 306. and the cutoff frequency of the window.
On obtient ainsi le filtre, dans l'espace temporel, fourni en entrée du module de choix 309.This gives the filter, in the time space, provided at the input of the choice module 309.
De manière optionnelle, un module 308 va offrir d'autres types de filtres. Par exemple, il peut offrir des filtres linéaires, cubiques ou autres. En effet, ces filtres sont connus pour permettre le sur-échantillonnage. Pour calculer les valeurs des échantillons rajoutés avec une valeur initiale à zéro entre les échantillons du signal limité en fréquence, il est possible de dupliquer la valeur de l'échantillon connu, de faire la moyenne entre les échantillons ce qui revient, à faire une interpolation linéaire entre les valeurs connues des échantillons. Tous ces types de filtres sont indépendants de la valeur du signal et permettent de remettre en forme le signal sur- échantillonné. Le module 308 contient donc un nombre arbitraire de tels filtres pouvant être utilisés. Le module de choix 309 va donc avoir en entrée une collection de filtres. D'une part, il aura les filtres générés par le module 305 et correspondant aux filtres générés pour différentes tailles de fenêtres par division des modules des transformées de Fourier appliquées au signal original et au signal reconstruit. D'autre part, il aura également en entrée le signal original 301 et le signal reconstruit issu du module 303. De cette façon le module 309 peut comparer l'application des différents filtres au signal reconstruit issu du module 303 avec le signal original pour choisir le filtre donnant, sur la portion de signal considérée, le meilleur signal de sortie, c'est-à-dire le plus proche spectralement du signal original. Par exemple, on peut faire le rapport entre le spectre obtenu par application du filtre au signal issu du module 303 et le spectre de la même portion du signal original. On choisit alors le filtre engendrant le minimum d'une fonction de la distorsion. Cette portion de signal, appelée fenêtre de travail, devra être plus grande que la plus grande fenêtre ayant servi au calcul des filtres, on pourra utiliser typiquement une taille de fenêtre de travail de 512 échantillons. La taille de cette fenêtre de travail peut également varier en fonction du signal. En effet, une grande taille de fenêtre de travail peut être utilisée pour l'encodage d'une partie de signal sensiblement stationnaire tandis qu'une fenêtre plus courte sera plus adaptée pour une portion de signal plus dynamique pour mieux prendre en compte les variations rapides. C'est cette partie qui permet de sélectionner, pour chaque portion du signal, le filtre le plus pertinent permettant la meilleure reconstruction par le décodeur du signal et de se rapprocher du signal original.Optionally, a module 308 will offer other types of filters. For example, it can offer linear, cubic or other filters. Indeed, these filters are known to allow oversampling. To calculate the values of the samples added with an initial value to zero between the samples of the signal limited in frequency, it is possible to duplicate the value of the known sample, to average between the samples which amounts to interpolation. linear between the known values of the samples. All of these types of filters are independent of the signal value and allow you to reshape the oversampled signal. The module 308 therefore contains an arbitrary number of such filters that can be used. The choice module 309 will therefore have as input a collection of filters. On the one hand, it will have the filters generated by the module 305 and corresponding to the filters generated for different window sizes by division of the Fourier transform modules applied to the original signal and the reconstructed signal. On the other hand, it will also have in input the original signal 301 and the reconstructed signal coming from the module 303. In this way the module 309 can compare the application of the different filters to the reconstructed signal coming from the module 303 with the original signal to choose the filter giving, on the signal portion considered, the best output signal, that is to say the spectrally closest to the original signal. For example, it is possible to relate the spectrum obtained by applying the filter to the signal from the module 303 and the spectrum of the same portion of the original signal. We then choose the filter generating the minimum of a function of the distortion. This portion of the signal, called the working window, will have to be larger than the largest window used for calculating the filters. It will be possible to use typically a working window size of 512 samples. The size of this working window may also vary depending on the signal. Indeed, a large working window size can be used for the encoding of a substantially stationary signal portion while a shorter window will be more suitable for a more dynamic signal portion to better take into account the rapid variations. . It is this part that makes it possible to select, for each portion of the signal, the most relevant filter allowing the best reconstruction by the decoder of the signal and to get closer to the original signal.
Une fois ce filtre choisi, le module 310 va quantifier les coefficients spectraux du filtre qui seront encodés, par exemple, en utilisant une table de Huffman pour optimiser les données à transmettre. Le multiplexeur 314 va donc multiplexer avec chaque portion du signal, le filtre le plus pertinent pour le décodage de cette portion de signal. Ce filtre étant choisi soit dans la collection de filtres de tailles différentes générés par analyse de cette portion de signal, soit dans la collection comprend également une série de filtres déterminés, typiquement linéaires, permettant la reconstruction, qui pourront être choisis s'ils se révèlent plus intéressants pour la reconstruction par le décodeur de la portion de signal. Quand le filtre généré est un des filtres déterminés, il est possible de ne transmettre qu'un identificateur identifiant ce filtre parmi la collection des filtres déterminés fournie par le module 308, ainsi que des paramètres éventuels du filtre. En effet, les coefficients de ces filtres déterminés n'étant pas calculés en fonction de la portion de signal à laquelle on veut les appliquer, il est inutile de transporter ces coefficients qui peuvent être connus du décodeur. Ainsi, la bande passante pour le transport de l'information relative au filtre se réduit dans ce cas à un simple identificateur du filtre.Once this filter is chosen, the module 310 will quantify the spectral coefficients of the filter that will be encoded, for example, using a Huffman table to optimize the data to be transmitted. Multiplexer 314 will thus multiplex with each portion of the signal, the most relevant filter for the decoding of this portion of signal. This filter being chosen either from the collection of filters of different sizes generated by analysis of this portion of the signal, or from the collection also comprises a series of determined, typically linear, filters enabling reconstruction, which may be chosen if they prove more interesting for the reconstruction by the decoder of the signal portion. When the generated filter is one of the determined filters, it is possible to transmit only an identifier identifying this filter from the collection of determined filters provided by the module 308, as well as possible parameters of the filter. Indeed, the coefficients of these determined filters are not calculated according to the portion of signal to which we want to apply them, it is unnecessary to carry these coefficients that can be known to the decoder. Thus, the bandwidth for transporting information relating to the filter is reduced in this case to a simple identifier of the filter.
La Fig. 4 représente le décodage correspondant dans le mode particulier de réalisation décrit. Le signal est reçu par le décodeur qui démultiplexe le signal. Le signal audio SIb est alors décodé par le module 404 puis sur-échantillonné d'un facteur n par l'insertion de n-1 échantillons à zéros entre les échantillons reçus par le module 405. Parallèlement, les coefficients spectraux du filtre F sont déquantifiés et décodés en suivant les tables de Huffman par le module 401. Avantageusement, la taille du filtre peut être adaptée par le module 402 du décodeur à ses capacités de calcul ou de mémoire ou encore de toute limitation matérielle éventuelle. Un décodeur possédant peu de ressources pourra utiliser un filtre sous-échantillonné ce qui lui permettra de diminuer les opérations lors de l'application du filtre. Le filtre sous- échantillonné peut aussi être généré par l'encodeur suivant les ressources du canal de transmission ou les ressources du décodeur, à condition bien sur que cette dernière information soit détenue par l'encodeur. De plus le spectre du filtre peut être réduit au décodage pour réaliser un suréchantillonnage moins important (n-1, n-2 etc..) en fonction des capacités matérielles de rendu sonore du décodeur telles que la puissance ou les capacités de sortie son. Le module 403 effectue alors une transformée de Fourier inverse sur les coefficients spectraux du filtre pour obtenir le filtre réel dans le domaine temporel. Dans l'exemple de réalisation, le filtre est de plus symétrique ce qui permet de réduire les données transportées pour la transmission du filtre. Le module 406 opère la convolution du signal sur-échantillonné issu du module 405 avec le filtre ainsi reconstitué pour obtenir le signal résultant. Cette convolution est particulièrement peu gourmande en calcul du fait que le sur-échantillonnage s'effectue par insertion de valeurs nulles. D'autre part, le fait que le filtre soit réel, et voire même symétrique dans le mode de réalisation préféré, permet également de réduire le nombre d'opérations nécessaires à cette convolution.Fig. 4 represents the corresponding decoding in the particular embodiment described. The signal is received by the decoder which demultiplexes the signal. The audio signal SIb is then decoded by the module 404 and then oversampled by a factor n by the insertion of n-1 zero samples between the samples received by the module 405. In parallel, the spectral coefficients of the filter F are dequantized and decoded according to the Huffman tables by the module 401. Advantageously, the size of the filter can be adapted by the module 402 of the decoder to its computing or memory capacity or any possible hardware limitation. A decoder with few resources can use a subsampled filter which will allow it to reduce the operations during the application of the filter. The subsampled filter may also be generated by the encoder according to the resources of the transmission channel or the resources of the decoder, provided of course that the latter information is held by the encoder. In addition the spectrum of the filter can be reduced to decoding to perform a smaller oversampling (n-1, n-2 etc. ..) depending on the hardware sound output capabilities of the decoder such as power or sound output capabilities. The module 403 then performs an inverse Fourier transform on the spectral coefficients of the filter to obtain the real filter in the time domain. In the exemplary embodiment, the filter is moreover symmetrical which makes it possible to reduce the data transported for the transmission of the filter. The module 406 operates the convolution of the oversampled signal from the module 405 with the filter thus reconstituted to obtain the resulting signal. This convolution is particularly greedy in calculation because the oversampling is done by inserting null values. On the other hand, the fact that the filter is real, and even symmetrical in the preferred embodiment, also reduces the number of operations required for this convolution.
Le filtre étant appliqué à l'intégralité du signal étendu en fréquence, l'invention offre l'avantage d'effectuer une remise en forme, non seulement de la partie haute du spectre reconstituée à partir de la partie basse transmise mais de l'ensemble du signal ainsi reconstitué. De cette manière, elle permet de modeler la partie du spectre non transmise mais également de corriger des artefacts dus aux différentes opérations de compression, décompression, d'encodage et décodage de la partie basse fréquence transmise.As the filter is applied to the entire extended frequency signal, the invention offers the advantage of performing a reshaping, not only of the high part of the spectrum reconstituted from the transmitted lower part but of the whole of the signal thus reconstituted. In this way, it allows to model the part of the spectrum not transmitted but also to correct artifacts due to the various operations of compression, decompression, encoding and decoding of the transmitted low frequency part.
Un avantage secondaire de l'invention est la possibilité d'adapter dynamiquement les filtres utilisés en fonction de la nature de chaque portion de signal grâce au module permettant le choix du meilleur filtre, en termes de qualité de rendu sonore et de « temps machine » utilisé, parmi plusieurs pour chaque portion du signal. A secondary advantage of the invention is the ability to dynamically adapt the filters used according to the nature of each portion of the signal thanks to the module allowing the choice of the best filter, in terms of sound quality and "machine time" used, among several for each portion of the signal.

Claims

REVENDICATIONS
1/ Procédé d'encodage de tout ou partie d'un signal comportant au moins les étapes suivantes :1 / A method of encoding all or part of a signal comprising at least the following steps:
- une étape d'obtention d'un signal limité en fréquence, la réduction de la fréquence du signal original étant obtenue par suppression des hautes fréquences, caractérisé en ce qu'il comporte en outre - une étape de génération d'un filtre temporel permettant de retrouver un signal proche spectralement du signal original lorsqu'il est appliqué à l'ensemble du signal obtenu par élargissement du spectre du signal limité.a step of obtaining a signal limited in frequency, the reduction of the frequency of the original signal being obtained by suppressing the high frequencies, characterized in that it further comprises a step of generating a temporal filter allowing to find a signal spectrally close to the original signal when it is applied to the entire signal obtained by broadening the spectrum of the limited signal.
2/ Procédé selon la revendication 1 où, pour une portion du signal original donnée, le filtre est obtenu par division membre à membre d'une fonction des coefficients d'une transformée de Fourier appliquée d'une part à la portion du signal original et d'autre part à la portion correspondante du signal obtenu par élargissement du spectre du signal limité.2 / A method according to claim 1 wherein, for a portion of the given original signal, the filter is obtained by dividing member to member of a function of the coefficients of a Fourier transform applied firstly to the portion of the original signal and on the other hand, to the corresponding portion of the signal obtained by broadening the spectrum of the limited signal.
3/ Procédé selon la revendication 2 où des transformées de Fourier de tailles différentes sont utilisées pour l'obtention d'une pluralité de filtres correspondant à chaque taille utilisée, le filtre généré correspondant à un choix parmi la pluralité de filtres obtenus par comparaison du signal original, et du signal obtenu par application du filtre au signal obtenu par élargissement du spectre du signal limité.3 / A method according to claim 2, wherein Fourier transforms of different sizes are used to obtain a plurality of filters corresponding to each size used, the generated filter corresponding to one of the plurality of filters obtained by comparison of the signal. original, and the signal obtained by applying the filter to the signal obtained by broadening the spectrum of the limited signal.
4/ Procédé selon l'une des revendications 1 à 3 où le choix est étendu à une collection de filtres temporels prédéterminés.4 / A method according to one of claims 1 to 3 wherein the choice is extended to a collection of predetermined time filters.
5/ Procédé selon l'une des revendications 1 à 4 où, le signal limité en fréquence étant encodé en vue de sa transmission, la génération du filtre se fait à partir du signal obtenu par décodage et élargissement du spectre du signal limité encodé et du signal original. 6/ Procédé de décodage de tout ou partie d'un signal comportant au moins les étapes suivantes :5 / A method according to one of claims 1 to 4 wherein, the frequency-limited signal being encoded for transmission, the generation of the filter is from the signal obtained by decoding and broadening the spectrum of the encoded limited signal and the original signal. 6 / A method for decoding all or part of a signal comprising at least the following steps:
- une étape de réception d'un signal transmis, caractérisé en ce qu'il comprend en outre : - une étape de réception d'un filtre temporel relatif au signal reçu,a step of receiving a transmitted signal, characterized in that it further comprises: a step of receiving a temporal filter relating to the signal received,
- une étape d'obtention d'un signal décodé par décodage du signal reçu,a step of obtaining a decoded signal by decoding the received signal,
- une étape d'obtention d'un signal étendu par élargissement du spectre du signal décodé,a step of obtaining an extended signal by broadening the spectrum of the decoded signal,
- une étape d'obtention d'un signal reconstruit par convolution de l'intégralité du signal étendu avec le filtre temporel reçu.a step of obtaining a signal reconstructed by convolution of the entirety of the extended signal with the received temporal filter.
Il Procédé selon la revendication 6 où un filtre réduit en taille à partir du filtre généré est utilisé à la place de ce filtre généré dans l'étape d'obtention d'un signal reconstruit.The method of claim 6 wherein a reduced size filter from the generated filter is used in place of that generated filter in the step of obtaining a reconstructed signal.
8/ Procédé selon la revendication 7 où le choix d'utiliser un filtre de taille réduite à la place du filtre généré se fait en fonction des capacités du décodeur.8 / The method of claim 7 wherein the choice to use a reduced size filter in place of the generated filter is based on the capabilities of the decoder.
9/ Dispositif d'encodage d'un signal comportant au moins : — des moyens d'obtention d'un signal limité en fréquence, la réduction du spectre du signal original étant obtenue par suppression des hautes fréquences,9 / Device for encoding a signal comprising at least: means for obtaining a signal limited in frequency, the reduction of the spectrum of the original signal being obtained by suppressing the high frequencies,
- des moyens d'obtention d'un signal limité encodé par encodage du signal limité en fréquence, caractérisé en ce qu'il comporte en outremeans for obtaining a limited signal encoded by encoding the frequency-limited signal, characterized in that it also comprises
- des moyens de génération d'un filtre temporel permettant de retrouver un signal proche du signal original lorsqu'il est appliqué à l'ensemble du signal obtenu par décodage et élargissement du spectre du signal limité.means for generating a temporal filter making it possible to recover a signal close to the original signal when it is applied to the entire signal obtained by decoding and broadening the spectrum of the limited signal.
10/ Dispositif de décodage d'un signal comportant au moins les moyens suivants :10 / Device for decoding a signal comprising at least the following means:
- des moyens de réception d'un signal transmis, caractérisé en ce qu'il comprend en outre : - des moyens de réception d'un filtre temporel relatif au signal reçu,means for receiving a transmitted signal, characterized in that it further comprises: means for receiving a temporal filter relating to the signal received,
- des moyens d'obtention d'un signal décodé par décodage du signal reçu,means for obtaining a decoded signal by decoding the received signal,
- des moyens d'obtention d'un signal étendu par élargissement du spectre du signal décodé, - des moyens d'obtention d'un signal reconstruit par convolution de l'intégralité du signal étendu avec le filtre temporel reçu.means for obtaining an extended signal by broadening the spectrum of the decoded signal, means for obtaining a signal reconstructed by convolution of the entirety of the extended signal with the received temporal filter.
11/ Signal comportant un signal audio limité en fréquence représentant une version limitée en fréquence d'un signal audio original caractérisé en ce qu'il comporte en outre des données de génération d'un filtre temporel permettant la reconstruction d'un signal proche du signal original lorsqu'il est appliqué à l'intégralité d'une version étendue en fréquence du signal audio limité en fréquence contenu dans le signal. 11 / signal comprising a frequency-limited audio signal representing a frequency-limited version of an original audio signal, characterized in that it furthermore comprises data for generating a temporal filter enabling the reconstruction of a signal close to the signal original when applied to an entire extended frequency version of the frequency-limited audio signal contained in the signal.
EP07866270A 2006-12-28 2007-12-27 Audio encoding method and device Active EP2126904B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0611481A FR2911031B1 (en) 2006-12-28 2006-12-28 AUDIO CODING METHOD AND DEVICE
PCT/EP2007/011433 WO2008080605A1 (en) 2006-12-28 2007-12-27 Audio encoding method and device

Publications (2)

Publication Number Publication Date
EP2126904A1 true EP2126904A1 (en) 2009-12-02
EP2126904B1 EP2126904B1 (en) 2010-06-09

Family

ID=38055366

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07866270A Active EP2126904B1 (en) 2006-12-28 2007-12-27 Audio encoding method and device

Country Status (7)

Country Link
US (1) US8595017B2 (en)
EP (1) EP2126904B1 (en)
JP (1) JP5491193B2 (en)
AT (1) ATE470933T1 (en)
DE (1) DE602007007125D1 (en)
FR (1) FR2911031B1 (en)
WO (1) WO2008080605A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954581A (en) * 2021-02-04 2021-06-11 广州橙行智动汽车科技有限公司 Audio playing method, system and device

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2013125346A (en) * 2011-12-13 2013-06-24 Olympus Imaging Corp Server device and processing method

Family Cites Families (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS62234435A (en) * 1986-04-04 1987-10-14 Kokusai Denshin Denwa Co Ltd <Kdd> Voice coding system
DE68927483T2 (en) * 1988-02-29 1997-04-03 Sony Corp Method and device for digital signal processing
US5956674A (en) 1995-12-01 1999-09-21 Digital Theater Systems, Inc. Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels
DE69731355T2 (en) * 1996-05-08 2006-02-09 Koninklijke Philips Electronics N.V. TRANSMITTING A DIGITAL INFORMATION SIGNAL WITH A FIRST SPECIFIC SCAN RATE
US6226616B1 (en) * 1999-06-21 2001-05-01 Digital Theater Systems, Inc. Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility
US6674862B1 (en) 1999-12-03 2004-01-06 Gilbert Magilen Method and apparatus for testing hearing and fitting hearing aids
KR100743534B1 (en) * 2000-01-07 2007-07-27 코닌클리케 필립스 일렉트로닉스 엔.브이. Transmission device and method for transmitting a digital information
US7742927B2 (en) * 2000-04-18 2010-06-22 France Telecom Spectral enhancing method and device
SE0004163D0 (en) * 2000-11-14 2000-11-14 Coding Technologies Sweden Ab Enhancing perceptual performance or high frequency reconstruction coding methods by adaptive filtering
US7711123B2 (en) * 2001-04-13 2010-05-04 Dolby Laboratories Licensing Corporation Segmenting audio signals into auditory events
JP3957589B2 (en) * 2001-08-23 2007-08-15 松下電器産業株式会社 Audio processing device
JP4805540B2 (en) * 2002-04-10 2011-11-02 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Stereo signal encoding
WO2004008437A2 (en) * 2002-07-16 2004-01-22 Koninklijke Philips Electronics N.V. Audio coding
US20070038439A1 (en) 2003-04-17 2007-02-15 Koninklijke Philips Electronics N.V. Groenewoudseweg 1 Audio signal generation
US7725324B2 (en) 2003-12-19 2010-05-25 Telefonaktiebolaget Lm Ericsson (Publ) Constrained filter encoding of polyphonic signals
CA2457988A1 (en) 2004-02-18 2005-08-18 Voiceage Corporation Methods and devices for audio compression based on acelp/tcx coding and multi-rate lattice vector quantization
FI119533B (en) 2004-04-15 2008-12-15 Nokia Corp Coding of audio signals
CN101010985A (en) 2004-08-31 2007-08-01 松下电器产业株式会社 Stereo signal generating apparatus and stereo signal generating method
RU2387024C2 (en) * 2004-11-05 2010-04-20 Панасоник Корпорэйшн Coder, decoder, coding method and decoding method
CN101048649A (en) * 2004-11-05 2007-10-03 松下电器产业株式会社 Scalable decoding apparatus and scalable encoding apparatus
ATE545131T1 (en) 2004-12-27 2012-02-15 Panasonic Corp SOUND CODING APPARATUS AND SOUND CODING METHOD
DE102005000830A1 (en) * 2005-01-05 2006-07-13 Siemens Ag Bandwidth extension method
US7573912B2 (en) 2005-02-22 2009-08-11 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschunng E.V. Near-transparent or transparent multi-channel encoder/decoder scheme
PL1866915T3 (en) * 2005-04-01 2011-05-31 Qualcomm Inc Method and apparatus for anti-sparseness filtering of a bandwidth extended speech prediction excitation signal
KR100818268B1 (en) 2005-04-14 2008-04-02 삼성전자주식회사 Apparatus and method for audio encoding/decoding with scalability
US7761289B2 (en) 2005-10-24 2010-07-20 Lg Electronics Inc. Removing time delays in signal paths
RU2393646C1 (en) 2006-03-28 2010-06-27 Фраунхофер-Гезелльшафт Цур Фердерунг Дер Ангевандтен Форшунг Е.Ф. Improved method for signal generation in restoration of multichannel audio
FR2911020B1 (en) 2006-12-28 2009-05-01 Actimagine Soc Par Actions Sim AUDIO CODING METHOD AND DEVICE

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2008080605A1 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112954581A (en) * 2021-02-04 2021-06-11 广州橙行智动汽车科技有限公司 Audio playing method, system and device

Also Published As

Publication number Publication date
DE602007007125D1 (en) 2010-07-22
JP5491193B2 (en) 2014-05-14
FR2911031A1 (en) 2008-07-04
WO2008080605A1 (en) 2008-07-10
FR2911031B1 (en) 2009-04-10
ATE470933T1 (en) 2010-06-15
US20100094640A1 (en) 2010-04-15
JP2010515090A (en) 2010-05-06
EP2126904B1 (en) 2010-06-09
US8595017B2 (en) 2013-11-26

Similar Documents

Publication Publication Date Title
EP2374123B1 (en) Improved encoding of multichannel digital audio signals
EP2104936B1 (en) Low-delay transform coding using weighting windows
EP1794748B1 (en) Data processing method by passage between different sub-band domains
EP2489039B1 (en) Optimized low-throughput parametric coding/decoding
WO2007007001A2 (en) Hierarchical encoding/decoding device
EP2732448B1 (en) Adaptations of analysis or synthesis weighting windows for transform coding or decoding
WO2007104883A1 (en) Device and method for graduated encoding of a multichannel audio signal based on a principal component analysis
FR2891098A1 (en) Digital audio stream mixing method for use in e.g. multimedia filed, involves mixing sound samples into mixed sound sample, and compressing mixed sound sample by utilizing compression parameters calculated using stored parameters
WO2010004155A1 (en) Spatial synthesis of multichannel audio signals
EP1581930A1 (en) Method for encoding and decoding audio at a variable rate
WO2009010674A1 (en) Hierarchical coding of digital audio signals
CN113470667A (en) Voice signal coding and decoding method and device, electronic equipment and storage medium
EP1275109B1 (en) Spectral enhancing method and device
CA2300647A1 (en) Audio coding, decoding and transcoding procedures
EP2126904B1 (en) Audio encoding method and device
EP2126905B1 (en) Methods and devices for audio signals encoding and decoding, encoded audio signal
EP2347411A1 (en) Pre-echo attenuation in a digital audio signal
WO2023165946A1 (en) Optimised encoding and decoding of an audio signal using a neural network-based autoencoder
CN117198301A (en) Audio encoding method, audio decoding method, apparatus, and readable storage medium
FR2914467A1 (en) Multidimensional digital picture signal coding method for digital processing of multidimensional digital signal, involves locating edges of picture at lower resolution, and attributing data of orientation values as function of result
FR2773653A1 (en) Input sound digital word decoding/coding device, especially for analyzing and compressing inputs for recording
FR2791166A1 (en) METHODS OF ENCODING, DECODING AND TRANSCODING
FR2778039A1 (en) Method of encoding digital images

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20090609

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: MOBICLIP

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

GRAF Information related to payment of grant fee modified

Free format text: ORIGINAL CODE: EPIDOSCIGR3

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REF Corresponds to:

Ref document number: 602007007125

Country of ref document: DE

Date of ref document: 20100722

Kind code of ref document: P

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20100609

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

LTIE Lt: invalidation of european patent or patent extension

Effective date: 20100609

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100616

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101011

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101009

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100910

BERE Be: lapsed

Owner name: MOBICLIP

Effective date: 20101231

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007007125

Country of ref document: DE

Effective date: 20110309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20101231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20101231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20101210

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20101227

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100609

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20111231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100909

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20100920

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602007007125

Country of ref document: DE

Representative=s name: PATENTANWAELTE WEICKMANN & WEICKMANN, DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 602007007125

Country of ref document: DE

Representative=s name: WEICKMANN & WEICKMANN PATENTANWAELTE - RECHTSA, DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 602007007125

Country of ref document: DE

Representative=s name: WEICKMANN & WEICKMANN PATENT- UND RECHTSANWAEL, DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231113

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231227

Year of fee payment: 17

Ref country code: DE

Payment date: 20231122

Year of fee payment: 17