EP2126905B1 - Methods and devices for audio signals encoding and decoding, encoded audio signal - Google Patents

Methods and devices for audio signals encoding and decoding, encoded audio signal Download PDF

Info

Publication number
EP2126905B1
EP2126905B1 EP07866272A EP07866272A EP2126905B1 EP 2126905 B1 EP2126905 B1 EP 2126905B1 EP 07866272 A EP07866272 A EP 07866272A EP 07866272 A EP07866272 A EP 07866272A EP 2126905 B1 EP2126905 B1 EP 2126905B1
Authority
EP
European Patent Office
Prior art keywords
signal
channel
filter
audio stream
channels
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP07866272A
Other languages
German (de)
French (fr)
Other versions
EP2126905A1 (en
Inventor
Alexandre Delattre
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nintendo European Research and Development SAS
Original Assignee
Mobiclip SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from FR0611481A external-priority patent/FR2911031B1/en
Application filed by Mobiclip SAS filed Critical Mobiclip SAS
Publication of EP2126905A1 publication Critical patent/EP2126905A1/en
Application granted granted Critical
Publication of EP2126905B1 publication Critical patent/EP2126905B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/038—Speech enhancement, e.g. noise reduction or echo cancellation using band spreading techniques
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding

Definitions

  • the present invention relates to a method and an audio coding device. It applies, in particular, coding enriched all or part of the audio spectrum, especially for transmission over a computer network, for example Internet, or storage on a digital information carrier.
  • This method and device can be integrated into any system for compressing and decompressing an audio signal on all hardware platforms.
  • the bit rate is often reduced by limiting the bandwidth of the audio signal.
  • low frequencies are retained because the human ear has a better resolution and spectral sensitivity in low frequency than in high frequency.
  • the low frequencies of the signal are kept, so that the data rate to be transferred is even lower.
  • some methods of the state of the art attempt, starting from the signal limited to the low frequencies, to extract harmonics which make it possible to recreate the high frequencies artificially . These methods are usually based on a spectral enrichment of recreating a high frequency spectrum by transposition of the low frequency spectrum, this high frequency spectrum being spectrally re-formed.
  • the resulting signal therefore consists, for the low frequency part, of the received low frequency signal and for the high frequency portion of the reformed enrichment.
  • the patent application US2006 / 0235678 describes the encoding of an audio signal with for each channel the provision of parameters of a spectral band replication ("SBR") spectral band extension method.
  • SBR spectral band replication
  • the invention relates to a method for encoding all or part of a multi-channel audio stream comprising a step of obtaining a composite signal obtained by composing the signals corresponding to each channel of the multi-channel audio stream; a step of obtaining a frequency-limited compound signal, the reduction of the frequency of the original composite signal being obtained by suppressing the high frequencies and a step of generating a temporal filter per channel making it possible to recover a signal that is spectrally close to the original signal of the corresponding channel when applied to the signal obtained by broadening the spectrum of the limited compound signal.
  • the filter corresponding to this channel is obtained by dividing member to member of a function of the coefficients of a Fourier transform applied on the one hand to the portion of the original signal and on the other hand to the corresponding portion of the signal obtained by broadening the spectrum of the limited signal.
  • Fourier transforms of different sizes are used to obtain a plurality of filters corresponding to each size used, the generated filter corresponding to one of the plurality of filters obtained by comparing the original signal and the signal obtained by applying the filter to the signal obtained by broadening the spectrum of the limited signal.
  • the choice of time filter can be made in a collection of predetermined time filters.
  • the filter is generated from the signal obtained by decoding and broadening the spectrum of the encoded limited compound signal and the signal. original.
  • the method further comprises a step of defining one of the channels of the multichannel audio stream as a reference channel; a time correlation step of each of the other channels on said reference channel defining for each channel an offset value and the step of composing the signals of each channel is performed with the signal of the reference channel and the signals correlated temporally for the other channels.
  • the offset value defined by the time correlation of the channel is associated with the generated filter.
  • the method further comprises a step of defining one of the channels of the multichannel audio stream as a reference channel; a step of equalizing each of the other channels on said reference channel defining for each channel an amplification value and the step of composing the signals of each channel is performed with the signal of the reference channel and the signals equalized for the other channels.
  • the amplification value defined by the temporal correlation of the channel is associated with the generated filter.
  • the invention also relates to a method for decoding all or part of a multi-channel audio stream comprising at least one step of receiving a transmitted signal; a step of receiving a time filter relating to the received signal for each channel of the multichannel audio stream; a step of obtaining a decoded signal by decoding the received signal; a step of obtaining an extended signal by enlargement the spectrum of the decoded signal and a step of obtaining a reconstructed signal by convolution of the extended signal with the received time filter for each channel of the multichannel audio stream.
  • a filter reduced in size from the generated filter is used in place of this filter generated in the step of obtaining a reconstructed signal for each channel.
  • the choice to use a reduced-size filter in place of the filter generated for each channel is according to the capabilities of the decoder.
  • one of the channels of the multichannel stream being defined as a reference channel, an offset value being associated with each received filter for the channels other than the reference channel
  • the method comprises in addition, a step of shifting the signal corresponding to each channel other than the reference channel making it possible to generate a temporal phase shift similar to the temporal phase shift between each channel and the reference channel in the original multi-channel audio stream.
  • the method further comprises a step of smoothing the offset values at the boundaries between the working windows so as to avoid a sudden change in the offset value for each channel other than the channel. reference.
  • one of the channels of the multichannel stream being defined as a reference channel, an amplification value being associated with each filter received for the channels other than the reference channel
  • the method further comprises a step of amplifying the signal corresponding to each channel other than the reference channel making it possible to generate a gain difference similar to the difference in gain between each channel and the reference channel in the original multi-channel audio stream .
  • the invention also relates to a device for encoding a multi-channel audio stream comprising at least means for obtaining a composite signal obtained by composing the signals corresponding to each channel of the multi-channel audio stream; means for obtaining a frequency-limited composite signal, the reduction of the spectrum of the original composite signal being obtained by suppressing high frequencies and means for generating a temporal filter per channel making it possible to recover a signal spectrally close to the original signal of the corresponding channel when it is applied to the signal obtained by broadening the spectrum of the limited signal.
  • the invention also relates to a device for decoding a multi-channel audio stream comprising at least the following means means for receiving a transmitted signal; means for receiving a time filter relating to the received signal for each channel of the multichannel audio stream; means for obtaining a decoded signal by decoding the received signal; means for obtaining an extended signal by broadening the spectrum of the decoded signal and means for obtaining a reconstructed signal by convolution of the extended signal with the time filter received for each channel of the multi-channel audio stream.
  • the invention also relates to a signal comprising a frequency-limited audio signal representing a frequency-limited version of an original audio signal resulting from the composition of the different channels of a multi-channel audio stream, characterized in that it also comprises channel time filter generation data for reconstructing a signal near the original signal of each channel when it is applied to an extended frequency version of the frequency limited audio signal contained in the signal.
  • the Fig. 1 represents the encoding method generally.
  • the signal 101 is the source signal to be encoded, this signal is then the original signal not limited in frequency.
  • Step 102 represents a frequency limiting step of the signal 101.
  • This frequency limitation can, for example, be achieved by subsampling of the signal 101 previously filtered by a low-pass filter. Subsampling consists of keeping only one sample on a set of samples and removing the other samples from the signal. A sub-sampling of a factor "n" where a sample is kept on n makes it possible to obtain a signal whose width of the spectrum will be divided by n. n is here a natural integer.
  • the signal limited in frequency and encoded at the output of the compression module 106 is also provided at the input of a decoding module 107.
  • This module performs the inverse operation of the encoding module 106 and makes it possible to construct a limited version of the signal. frequency identical to the version to which the decoder will have access when it also performs this decoded operation of the limited signal and encoded it will receive.
  • the limited signal thus decoded is then restored to the original spectral range by a frequency enrichment module 103.
  • This frequency enrichment may, for example, consist of a simple over-sampling of the input signal by the insertion of zero-valued samples between the samples of the input signal. Any other method of enriching the signal spectrum can also be used.
  • This extended frequency signal derived from the frequency enrichment module 103, is then supplied to a filter generation module 104.
  • This filter generation module 104 also receives the original signal 101 and calculates a time filter allowing, when it is applied to the extended signal from the frequency enrichment module 103, to shape it to get closer to the original signal.
  • the filter thus calculated is then supplied to the multiplexer 108 after an optional compression step 105.
  • the Fig. 2 represents generally the corresponding decoding method.
  • the decoder therefore receives the signal from the multiplexer 108 of the encoder. It demultiplexes it to obtain on the one hand the encoded frequency limited signal, called S1b, and the filter coefficients F, contained in the transmitted signal.
  • the signal S1b is then decoded by a decoding or decompression module 202 that is functionally equivalent to the module 107 of the Fig. 1 .
  • the signal is extended in frequency by the module 203 functionally equivalent to the module 103 of the Fig. 1 . A decoded and extended version of the signal is thus obtained.
  • the coefficients of the filter F are decoded if they had been encoded or compressed by a decompression module 201, and the filter obtained is applied to the extended time signal in a signal conditioning module 204. then an output signal close to the original signal.
  • This treatment is simple to implement because of the temporal nature of the filter to be applied to the signal for shaping.
  • the transmitted filter and thus applied during the reconstruction of the signal, is transmitted periodically and changes over time.
  • This filter is adapted to a portion of the signal to which it applies. It is thus possible to calculate for each signal portion a time filter particularly adapted according to the dynamic spectral characteristics of this portion of signal. In particular, it is possible to have several types of time filter generators and to select for each portion of signal the filter giving the best result for this portion. This is possible because the filter generation module has on the one hand the original signal and on the other hand the extended signal as it will be reconstructed by the decoder, so it is able, in the case where it is generated by several different filters, compare the signal obtained by applying each filter to the extended signal portion and the original signal that is to be approached as closely as possible. This method of filter generation is therefore not limited to choosing a type of filter determined for the entire signal but allows to change the type of filter according to the characteristics of each portion of the signal.
  • a signal sampled at a given frequency 301 for example 32 kHz
  • S1b the signal limited to its low frequencies
  • a filter F for shaping the signal obtained by extending the signal S1b frequency.
  • This signal is then encoded, for example by a method of the PCM ("Pulse Code Modulation") type, by the module 311 which will then be compressed, for example by an ADPCM module 312. This gives the subsampled signal containing the low frequencies of the original signal 301. This signal is sent to the multiplexer 314 to be transmitted to the decoder.
  • PCM Pese Code Modulation
  • this signal is transmitted to a decoding module 313.
  • This signal that the decoder will obtain from the signal sent to it is simulated.
  • This signal that will be used for the generation of the filter F will therefore allow to take into account the artifacts resulting from these coding and decoding phases, compression and decompression.
  • This signal is then frequency-expanded by inserting n-1 zero between each sample of the time signal in the module 303. In this way, a signal of the same spectral range as the original signal is reconstructed. According to the Nyquist theorem, we obtain a n-order spectrum folding.
  • the signal is downsampled from an order 2 to the encoding and oversampled from an order 2 to the decoding.
  • the spectrum is duplicated "mirrored" by axial symmetry in the frequency domain.
  • a Fourier transform is performed on the frequency-extended time signal from the module 303.
  • a sliding fast Fourier transform is performed on work windows of given size and variable. These sizes are typically 128, 256, 512 samples but can be of any size even if one will preferentially use powers of two to simplify calculations.
  • the modules of these transforms applied to these windows are then calculated. A same Fourier transform calculation is performed on the original signal in the module 306.
  • a member-to-member division 305 is then performed between the modules of the Fourier transform coefficients obtained by steps 304 and 306 to generate by inverse Fourier transforms temporal filters of sizes proportional to those of the windows used, ie 128, 256 or 512.
  • This step therefore generates several filters of different sizes among which we will have to choose the filter finally used. It will be seen that this selection step is performed by the module 309. Since the coefficients of the ratio between the windows are real, symmetrical in the frequency space, the equivalent filter F is then, in the time domain, real and symmetrical.
  • This property of symmetry can be used to transmit only half of the coefficients, the other being deduced by symmetry.
  • Obtaining a symmetrical real filter also makes it possible to reduce the number of operations required during the convolution of the received signal extended by the filter in the decoder.
  • Other embodiments make it possible to obtain real unsymmetrical filters. For example, if the temporal signal in a working window is frequency-limited, it is advantageous to iteratively determine the parameters of an infinite impulse response filter, Chebychev low-pass from the spectra from steps 304 and 306. and the cutoff frequency of the window.
  • a module 308 will offer other types of filters.
  • it can offer linear, cubic or other filters.
  • these filters are known to allow oversampling.
  • the module 308 therefore contains an arbitrary number of such filters that can be used.
  • the choice module 309 will therefore have as input a collection of filters. On the one hand, it will have the filters generated by the module 307 and corresponding to the filters generated for different window sizes by dividing the modules of the Fourier transforms applied to the original signal and the reconstructed signal. On the other hand, it will also have in input the original signal 301 and the reconstructed signal coming from the module 303. In this way the module 309 can compare the application of the different filters to the reconstructed signal coming from the module 303 with the original signal to choose the filter giving, on the signal portion considered, the best output signal, that is to say the spectrally closest to the original signal.
  • the filter generating the minimum of a function of the distortion.
  • This portion of the signal called the working window
  • the working window will have to be larger than the largest window used for calculating the filters.
  • the size of this working window may also vary depending on the signal. Indeed, a large working window size can be used for the encoding of a substantially stationary signal portion while a shorter window will be more suitable for a more dynamic signal portion to better take into account the rapid variations. . It is this part that makes it possible to select, for each portion of the signal, the most relevant filter allowing the best reconstruction by the decoder of the signal and to get closer to the original signal.
  • the module 310 will quantify the spectral coefficients of the filter that will be encoded, for example, using a Huffman table to optimize the data to be transmitted.
  • Multiplexer 314 will thus multiplex with each portion of the signal, the most relevant filter for the decoding of this portion of signal.
  • This filter being chosen either in the collection of filters of different sizes generated by analysis of this portion of the signal, or in the collection, also comprises a series of determined, typically linear filters, allowing the reconstruction, which can be chosen if they occur. reveal more interesting for the reconstruction by the decoder of the signal portion.
  • the generated filter is one of the determined filters, it is possible to transmit only an identifier identifying this filter from the collection of determined, typically linear filters, allowing the reconstruction, which can be chosen if they prove more interesting for the reconstruction by the decoder of the portion of signal.
  • the generated filter is one of the determined filters, it is possible to transmit only an identifier identifying this filter from the collection of determined filters provided by the module 308, as well as possible parameters of the filter. Indeed, since the coefficients of these determined filters are not calculated as a function of the portion of the signal to which they are to be applied, it is unnecessary to transport these coefficients which may be known to the decoder. Thus, the bandwidth for transporting information relating to the filter is reduced in this case to a simple identifier of the filter.
  • the Fig. 4 represents the corresponding decoding in the particular embodiment described.
  • the signal is received by the decoder which demultiplexes the signal.
  • the audio signal S1b is then decoded by the module 404 and then oversampled by a factor n by the insertion of n-1 samples at zero between the samples received by the module 405.
  • the spectral coefficients of the filter F are dequantized and decoded according to the Huffman tables by the module 401.
  • the size of the filter can be adapted by the module 402 of the decoder to its computing or memory capacity or any possible hardware limitation.
  • a decoder with few resources can use a subsampled filter which will allow it to reduce the operations during the application of the filter.
  • the subsampled filter can also be generated by the encoder according to the resources of the transmission channel or the resources of the decoder, provided of course that the latter information is held by the encoder.
  • the spectrum of the filter can be reduced to decoding to perform a smaller oversampling (n-1, n-2 etc. ..) depending on the hardware sound output capabilities of the decoder such as the power or the output capabilities its .
  • the module 403 then performs an inverse Fourier transform on the spectral coefficients of the filter to obtain the real filter in the time domain.
  • the filter is moreover symmetrical which makes it possible to reduce the data transported for the transmission of the filter.
  • the module 406 operates the convolution of the oversampled signal from the module 405 with the filter thus reconstituted to obtain the resulting signal.
  • This convolution is particularly greedy in calculation because the oversampling is done by inserting null values.
  • the fact that the filter is real, and even symmetrical in the preferred embodiment also reduces the number of operations required for this convolution.
  • the invention offers the advantage of performing a reshaping, not only of the high part of the spectrum reconstituted from the transmitted lower part but of the whole of the signal thus reconstituted. In this way, it makes it possible to model the part of the non-transmitted spectrum but also to correct artifacts due to the various operations of compression, decompression, encoding and decoding of the transmitted low frequency part.
  • a secondary advantage of the invention is the ability to dynamically adapt the filters used according to the nature of each portion of the signal thanks to the module allowing the choice of the best filter, in terms of sound quality and "machine time” used, among several for each portion of the signal.
  • the encoding method thus described for a single-channel signal can be adapted for a multi-channel signal.
  • a first obvious adaptation is the application of the single-channel solution to each audio channel independently. This solution is nevertheless expensive in that it does not take advantage of the strong correlation between the different channels of a multi-channel audio stream.
  • the proposed solution is to compose a single channel from the different channels of the stream. A treatment similar to that described previously in the case of a single-channel signal is then performed on this compound stream.
  • a filter is determined for each channel so as to reproduce the considered channel when it is applied to the composite stream.
  • the process will now be described more precisely using the Figures 5 and 6 in the case of stereophony.
  • the stereophonic embodiment naturally extends to a stream composed of more than two channels such as a stream 5.1 for home theater for example.
  • the Fig. 5 represents the architecture of a stereophonic encoder according to an embodiment of the invention.
  • the audio stream to be encoded consists of a left channel "L" referenced 501 and a right channel “R” referenced 502.
  • a composition module 503 composes these two signals to generate a composite signal.
  • This composition may, for example, be an average of the two channels, the composite signal is then equal to L + R / 2.
  • This compound signal then undergoes the same processing as the single-channel signal described above. It is sub-sampled by a factor n by the subsampling module 504.
  • the subsampled signal is then coded by an encoder 505 to be encoded by an encoder 506.
  • These modules are the same as the modules already described 311 and 312 of the Fig. 3 .
  • the subsampled and encoded composite signal is transmitted to the recipient of the stream. It is also decoded by a decoding module 507 corresponding to the module 313 of the Fig. 3 . Then, it is oversampled by the sampling module 508 corresponding to the module 303. The signal is then processed by two filter generation modules 509 and 510. Each of these modules corresponds to the modules 304, 305, 306, 308, 309. and 310 of the Fig. 3 . The first, 509, generates a filter F R which, when applied to the compound stream coming from the module 508, generates a signal close to the right channel R. This module takes as input the composite signal from the module 508 and the signal of the original R right channel 502.
  • the second, 510 generates a filter F L which, when applied to the compound stream coming from the module 508, generates a signal close to the left channel L.
  • This module takes as input the composed signal from the module 508 and the original left channel signal L 501. These filters, or an identifier of these filters, are then multiplexed with the subsampled stream and encoded from the encoding module 506 to be sent to the receiver .
  • the different channels of a multi-channel signal have a strong correlation but exhibit a temporal phase shift.
  • a slight time shift occurs between the signals of the different channels.
  • this offset tends to generate noise.
  • This registration is performed by temporal correlation between the channel to be recalibrated and the reference channel.
  • This correlation defines an offset value on the working window chosen for the correlation.
  • This working window is advantageously chosen equal to the working window used for generating the filter.
  • the value of the offset can then be associated with the generated filter to be transmitted in addition to the filters so as to allow reconstruction of the original inter-channel phase shift during the reproduction of the audio stream.
  • a step of equalizing the gains of the signals of the different channels can be used to homogenize the powers of the signals corresponding to the different channels.
  • This equalization defines an amplification value to be applied to the signal on the working window.
  • This amplification value can be introduced into the calculated filter allowing the reconstruction of the signal at decoding.
  • the calculation of this Amplification value is for each channel except one chosen as the reference channel. The introduction of the amplification value makes it possible to reconstruct, at decoding, the differences in gains between the channels in the original signal.
  • the generation calculation of a filter as well as that of the phase shift is done on a portion of signal called work window ( frame in English).
  • work window frame in English
  • the passage from one window to another will therefore cause a change in phase shift between the channels. This change may cause noise during playback.
  • the Fig. 6 represents the architecture of a stereophonic embodiment of the decoder. This figure is the stereophonic counterpart of Fig. 4 .
  • the received audio stream is demultiplexed to obtain the encoded low frequency composite stream called S 1b and the filters F R and F L.
  • the compound stream is then decoded by the decoding module 601 corresponding to the module 404 of the Fig. 4 . Its spectrum is then widened in frequency by the sampling module 602 corresponding to the module 405 of the Fig. 4 .
  • the signal thus obtained is then convoluted by the filters F R and F L decompressed by the modules 603 and 605 to restore the right and left channels S R and S L.
  • phase shift information is introduced into the stream, the channel that does not serve as a reference channel for the phase shift is recalibrated using this information to generate the phase shift of the original channels.
  • This phase shift information may, for example, take the form of an offset value associated with each of the filters for the channels other than the channel defined as reference channel.
  • this offset is smoothed, for example linearly, between the different work windows.

Abstract

The invention relates to an audio encoding device and method comprising the transmission of data representing a frequency-limited signal obtained by composition (503) of signals corresponding to each channel of an audio stream with multiple channels (501, 502) and the suppression of high frequencies (504) and, in addition, of information relating to a plurality of temporal filters (509, 510) to be applied for each channel to all of the enriched signal, both to the transmitted low frequency part and to the reconstructed high frequency part. The application of said filters enables the reconstructed high frequency part to be reshaped for each channel and compression artifacts present in the transmitted low frequency part to be corrected.

Description

La présente invention concerne un procédé et un dispositif de codage audio. Elle s'applique, en particulier, au codage avec enrichissement de tout ou partie du spectre audio, notamment en vue de leur transmission sur un réseau informatique, par exemple Internet, ou de leur stockage sur un support d'informations numériques. Ce procédé et ce dispositif peuvent être intégrés à tout système permettant de compresser puis décompresser un signal audio sur toutes plates-formes matérielles.The present invention relates to a method and an audio coding device. It applies, in particular, coding enriched all or part of the audio spectrum, especially for transmission over a computer network, for example Internet, or storage on a digital information carrier. This method and device can be integrated into any system for compressing and decompressing an audio signal on all hardware platforms.

En compressions audio, le débit est souvent réduit en limitant la bande passante du signal audio. Généralement on ne conserve que les basses fréquences car l'oreille humaine a une meilleure résolution et sensibilité spectrales en basse fréquence qu'en haute fréquence. Typiquement on ne conserve que les basses fréquences du signal, ainsi le débit des données à transférer est d'autant plus faible. Comme les harmoniques contenues dans les basses fréquences sont aussi présentes dans les hautes fréquences, certaines méthodes de l'état de l'art tentent, à partir du signal limité aux basses fréquences, d'extraire des harmoniques qui permettent de recréer les hautes fréquences artificiellement. Ces méthodes reposent généralement sur un enrichissement spectral consistant à recréer un spectre haute fréquence par transposition du spectre basse fréquence, ce spectre haute fréquence étant remis en forme spectralement. Le signal résultant se compose donc, pour la partie basse fréquence, du signal basse fréquence reçu et pour la partie haute fréquence de l'enrichissement remis en forme.In audio compression, the bit rate is often reduced by limiting the bandwidth of the audio signal. Generally only low frequencies are retained because the human ear has a better resolution and spectral sensitivity in low frequency than in high frequency. Typically, only the low frequencies of the signal are kept, so that the data rate to be transferred is even lower. As the harmonics contained in the low frequencies are also present in the high frequencies, some methods of the state of the art attempt, starting from the signal limited to the low frequencies, to extract harmonics which make it possible to recreate the high frequencies artificially . These methods are usually based on a spectral enrichment of recreating a high frequency spectrum by transposition of the low frequency spectrum, this high frequency spectrum being spectrally re-formed. The resulting signal therefore consists, for the low frequency part, of the received low frequency signal and for the high frequency portion of the reformed enrichment.

La demande de brevet US4,757,517 décrit l'encodage de paramètres LPC et la transmission de ces paramètres avec une version d'un signal vocal encodé à bande passante limitée. Cette demande décrit aussi l'utilisation au décodage d'une étape de reconstruction de la bande passante supérieure et de sa mise en forme par les paramètres LPC reçus.The patent application US4,757,517 describes the encoding of LPC parameters and the transmission of these parameters with a version of a coded speech signal with limited bandwidth. This application also describes the decoding of a step of reconstructing the upper bandwidth and its formatting by the received LPC parameters.

La demande de brevet US2006/0235678 décrit l'encodage d'un signal audio avec pour chaque canal la mise à disposition de paramètres d'un procédé d'extension de bande spectrale par réplication de fréquences (spectral band replication "SBR").The patent application US2006 / 0235678 describes the encoding of an audio signal with for each channel the provision of parameters of a spectral band replication ("SBR") spectral band extension method.

La demande de brevet WO2004/093494 décrit la mise en forme spectrale d'un signal étendu en fréquence dans le domaine des sous-bandes fréquentielles, avant la reconstruction des différents canaux.The patent application WO2004 / 093494 describes the spectral shaping of a signal extended in frequency in the domain of the frequency sub-bands, before the reconstruction of the different channels.

Il s'avère que la compression et la méthode utilisée pour compresser et limiter la bande de fréquence du signal initial génèrent des artéfacts nuisant à la qualité du signal. D'autre part, la reconstitution d'un signal de qualité en réception doit permettre d'obtenir la meilleure qualité perçue possible tout en ne nécessitant qu'une faible bande passante de données transmises et un traitement simple et rapide à la réception.It turns out that the compression and the method used to compress and limit the frequency band of the initial signal generate artifacts affecting the quality of the signal. On the other hand, the reconstitution of a reception quality signal must make it possible to obtain the best perceived quality possible while requiring only a small bandwidth of transmitted data and a simple and fast processing at the reception.

Ce problème est avantageusement résolu par la transmission, en sus des données représentant le signal limité en fréquences, d'informations relatives à un filtre temporel devant être appliqué à l'intégralité du signal enrichi, tant dans sa partie basse fréquence transmise que dans sa partie haute fréquence reconstituée, l'application de ce filtre permettant la remise en forme de la partie haute fréquence reconstituée et la correction d'artéfacts de compression présents dans la partie basse fréquence transmise. De cette façon, l'application du filtre temporel, simple et peu coûteuse, à l'intégralité du signal reconstitué, permet d'obtenir un signal de bonne qualité perçue.This problem is advantageously solved by the transmission, in addition to the data representing the signal limited in frequency, of information relating to a temporal filter to be applied to the entire enriched signal, both in its transmitted low frequency part and in its part. reconstituted high frequency, the application of this filter allowing the reshaping of the reconstituted high frequency part and the correction of compression artifacts present in the transmitted low frequency part. In this way, the application of the time filter, simple and inexpensive, to the entire reconstituted signal, provides a signal of good quality perceived.

L'invention concerne un procédé d'encodage de tout ou partie d'un flux audio multi canal comportant une étape d'obtention d'un signal composé obtenu par composition des signaux correspondant à chaque canal du flux audio multi canal ; une étape d'obtention d'un signal composé limité en fréquence, la réduction de la fréquence du signal composé original étant obtenue par suppression des hautes fréquences et une étape de génération d'un filtre temporel par canal permettant de retrouver un signal proche spectralement du signal original du canal correspondant lorsqu'il est appliqué au signal obtenu par élargissement du spectre du signal composé limité.The invention relates to a method for encoding all or part of a multi-channel audio stream comprising a step of obtaining a composite signal obtained by composing the signals corresponding to each channel of the multi-channel audio stream; a step of obtaining a frequency-limited compound signal, the reduction of the frequency of the original composite signal being obtained by suppressing the high frequencies and a step of generating a temporal filter per channel making it possible to recover a signal that is spectrally close to the original signal of the corresponding channel when applied to the signal obtained by broadening the spectrum of the limited compound signal.

Selon un mode particulier de réalisation de l'invention, pour une portion du signal original donnée, pour un canal donné, le filtre correspondant à ce canal est obtenu par division membre à membre d'une fonction des coefficients d'une transformée de Fourier appliquée d'une part à la portion du signal original et d'autre part à la portion correspondante du signal obtenu par élargissement du spectre du signal limité.According to a particular embodiment of the invention, for a portion of the given original signal, for a given channel, the filter corresponding to this channel is obtained by dividing member to member of a function of the coefficients of a Fourier transform applied on the one hand to the portion of the original signal and on the other hand to the corresponding portion of the signal obtained by broadening the spectrum of the limited signal.

Selon un mode particulier de réalisation de l'invention, des transformées de Fourier de tailles différentes sont utilisées pour l'obtention d'une pluralité de filtres correspondant à chaque taille utilisée, le filtre généré correspondant à un choix parmi la pluralité de filtres obtenus par comparaison du signal original, et du signal obtenu par application du filtre au signal obtenu par élargissement du spectre du signal limité.According to a particular embodiment of the invention, Fourier transforms of different sizes are used to obtain a plurality of filters corresponding to each size used, the generated filter corresponding to one of the plurality of filters obtained by comparing the original signal and the signal obtained by applying the filter to the signal obtained by broadening the spectrum of the limited signal.

Selon un mode particulier de réalisation de l'invention, le choix du filtre temporel peut s'effectuer dans une collection de filtres temporels prédéterminés.According to a particular embodiment of the invention, the choice of time filter can be made in a collection of predetermined time filters.

Selon un mode particulier de réalisation de l'invention, le signal composé limité en fréquence étant encodé en vue de sa transmission, la génération du filtre se fait à partir du signal obtenu par décodage et élargissement du spectre du signal composé limité encodé et du signal original.According to a particular embodiment of the invention, the frequency-limited compound signal being encoded for transmission, the filter is generated from the signal obtained by decoding and broadening the spectrum of the encoded limited compound signal and the signal. original.

Selon un mode particulier de réalisation de l'invention, le procédé comporte en outre une étape de définition d'un des canaux du flux audio multi canal comme canal de référence ; une étape de corrélation temporelle de chacun des autres canaux sur ledit canal de référence définissant pour chaque canal une valeur de décalage et l'étape de composition des signaux de chaque canal est effectuée avec le signal du canal de référence et les signaux corrélés temporellement pour les autres canaux.According to a particular embodiment of the invention, the method further comprises a step of defining one of the channels of the multichannel audio stream as a reference channel; a time correlation step of each of the other channels on said reference channel defining for each channel an offset value and the step of composing the signals of each channel is performed with the signal of the reference channel and the signals correlated temporally for the other channels.

Selon un mode particulier de réalisation de l'invention, pour chaque canal autre que le canal de référence, la valeur de décalage définie par la corrélation temporelle du canal est associée au filtre généré.According to a particular embodiment of the invention, for each channel other than the reference channel, the offset value defined by the time correlation of the channel is associated with the generated filter.

Selon un mode particulier de réalisation de l'invention, le procédé comporte en outre une étape de définition d'un des canaux du flux audio multi canal comme canal de référence ; une étape d'égalisation de chacun des autres canaux sur ledit canal de référence définissant pour chaque canal une valeur d'amplification et l'étape de composition des signaux de chaque canal est effectuée avec le signal du canal de référence et les signaux égalisés pour les autres canaux.According to a particular embodiment of the invention, the method further comprises a step of defining one of the channels of the multichannel audio stream as a reference channel; a step of equalizing each of the other channels on said reference channel defining for each channel an amplification value and the step of composing the signals of each channel is performed with the signal of the reference channel and the signals equalized for the other channels.

Selon un mode particulier de réalisation de l'invention, pour chaque canal autre que le canal de référence, la valeur d'amplification définie par la corrélation temporelle du canal est associée au filtre généré.According to a particular embodiment of the invention, for each channel other than the reference channel, the amplification value defined by the temporal correlation of the channel is associated with the generated filter.

L'invention concerne également un procédé de décodage de tout ou partie d'un flux audio multi canal comportant au moins une étape de réception d'un signal transmis ; une étape de réception d'un filtre temporel relatif au signal reçu pour chaque canal du flux audio multi canal ; une étape d'obtention d'un signal décodé par décodage du signal reçu ; une étape d'obtention d'un signal étendu par élargissement du spectre du signal décodé et une étape d'obtention d'un signal reconstruit par convolution du signal étendu avec le filtre temporel reçu pour chaque canal du flux audio multi canal.The invention also relates to a method for decoding all or part of a multi-channel audio stream comprising at least one step of receiving a transmitted signal; a step of receiving a time filter relating to the received signal for each channel of the multichannel audio stream; a step of obtaining a decoded signal by decoding the received signal; a step of obtaining an extended signal by enlargement the spectrum of the decoded signal and a step of obtaining a reconstructed signal by convolution of the extended signal with the received time filter for each channel of the multichannel audio stream.

Selon un mode particulier de réalisation de l'invention, un filtre réduit en taille à partir du filtre généré est utilisé à la place de ce filtre généré dans l'étape d'obtention d'un signal reconstruit pour chaque canal.According to a particular embodiment of the invention, a filter reduced in size from the generated filter is used in place of this filter generated in the step of obtaining a reconstructed signal for each channel.

Selon un mode particulier de réalisation de l'invention, le choix d'utiliser un filtre de taille réduite à la place du filtre généré pour chaque canal se fait en fonction des capacités du décodeur.According to a particular embodiment of the invention, the choice to use a reduced-size filter in place of the filter generated for each channel is according to the capabilities of the decoder.

Selon un mode particulier de réalisation de l'invention, l'un des canaux du flux multi canal étant défini comme canal de référence, une valeur de décalage étant associée à chaque filtre reçu pour les canaux autres que le canal de référence, le procédé comporte en outre une étape de décalage du signal correspondant à chaque canal autre que le canal de référence permettant de générer un déphasage temporel similaire au déphasage temporel entre chaque canal et le canal de référence dans le flux audio multi canal d'origine.According to a particular embodiment of the invention, one of the channels of the multichannel stream being defined as a reference channel, an offset value being associated with each received filter for the channels other than the reference channel, the method comprises in addition, a step of shifting the signal corresponding to each channel other than the reference channel making it possible to generate a temporal phase shift similar to the temporal phase shift between each channel and the reference channel in the original multi-channel audio stream.

Selon un mode particulier de réalisation de l'invention, le procédé comporte en outre une étape de lissage des valeurs de décalage aux frontières entre les fenêtres de travail de manière à éviter un changement brusque de la valeur de décalage pour chaque canal autre que le canal de référence.According to a particular embodiment of the invention, the method further comprises a step of smoothing the offset values at the boundaries between the working windows so as to avoid a sudden change in the offset value for each channel other than the channel. reference.

Selon un mode particulier de réalisation de l'invention, l'un des canaux du flux multi canal étant défini comme canal de référence, une valeur d'amplification étant associée à chaque filtre reçu pour les canaux autres que le canal de référence, le procédé comporte en outre une étape d'amplification du signal correspondant à chaque canal autre que le canal de référence permettant de générer une différence de gain similaire à la différence de gain entre chaque canal et le canal de référence dans le flux audio multi canal d'origine.According to a particular embodiment of the invention, one of the channels of the multichannel stream being defined as a reference channel, an amplification value being associated with each filter received for the channels other than the reference channel, the method further comprises a step of amplifying the signal corresponding to each channel other than the reference channel making it possible to generate a gain difference similar to the difference in gain between each channel and the reference channel in the original multi-channel audio stream .

L'invention concerne également un dispositif d'encodage d'un flux audio multi canal comportant au moins des moyens d'obtention d'un signal composé obtenu par composition des signaux correspondant à chaque canal du flux audio multi canal ; des moyens d'obtention d'un signal composé limité en fréquence, la réduction du spectre du signal composé original étant obtenue par suppression des hautes fréquences et des moyens de génération d'un filtre temporel par canal permettant de retrouver un signal proche spectralement du signal original du canal correspondant lorsqu'il est appliqué au signal obtenu par élargissement du spectre du signal limité.The invention also relates to a device for encoding a multi-channel audio stream comprising at least means for obtaining a composite signal obtained by composing the signals corresponding to each channel of the multi-channel audio stream; means for obtaining a frequency-limited composite signal, the reduction of the spectrum of the original composite signal being obtained by suppressing high frequencies and means for generating a temporal filter per channel making it possible to recover a signal spectrally close to the original signal of the corresponding channel when it is applied to the signal obtained by broadening the spectrum of the limited signal.

L'invention concerne également un dispositif de décodage d'un flux audio multi canal comportant au moins les moyens suivants des moyens de réception d'un signal transmis ; des moyens de réception d'un filtre temporel relatif au signal reçu pour chaque canal du flux audio multi canal ; des moyens d'obtention d'un signal décodé par décodage du signal reçu ; des moyens d'obtention d'un signal étendu par élargissement du spectre du signal décodé et des moyens d'obtention d'un signal reconstruit par convolution du signal étendu avec le filtre temporel reçu pour chaque canal du flux audio multi canal.The invention also relates to a device for decoding a multi-channel audio stream comprising at least the following means means for receiving a transmitted signal; means for receiving a time filter relating to the received signal for each channel of the multichannel audio stream; means for obtaining a decoded signal by decoding the received signal; means for obtaining an extended signal by broadening the spectrum of the decoded signal and means for obtaining a reconstructed signal by convolution of the extended signal with the time filter received for each channel of the multi-channel audio stream.

L'invention concerne également un signal comportant un signal audio limité en fréquence représentant une version limitée en fréquence d'un signal audio original résultant de la composition des différents canaux d'un flux audio multi canal caractérisé en ce qu'il comporte en outre des données de génération d'un filtre temporel par canal permettant la reconstruction d'un signal proche du signal original de chaque canal lorsqu'il est appliqué à une version étendue en fréquence du signal audio limité en fréquence contenu dans le signal.The invention also relates to a signal comprising a frequency-limited audio signal representing a frequency-limited version of an original audio signal resulting from the composition of the different channels of a multi-channel audio stream, characterized in that it also comprises channel time filter generation data for reconstructing a signal near the original signal of each channel when it is applied to an extended frequency version of the frequency limited audio signal contained in the signal.

Les caractéristiques de l'invention mentionnées ci-dessus, ainsi que d'autres, apparaîtront plus clairement à la lecture de la description suivante d'un exemple de réalisation, ladite description étant faite en relation avec les dessins joints, parmi lesquels :

  • La Fig. 1 représente l'architecture générale de la méthode d'encodage d'un exemple de réalisation de l'invention.
  • La Fig. 2 représente l'architecture générale de la méthode de décodage de l'exemple de réalisation de l'invention.
  • La Fig. 3 représente l'architecture d'un mode de réalisation de l'encodeur.
  • La Fig. 4 représente l'architecture d'un mode de réalisation du décodeur.
  • La Fig. 5 représente l'architecture d'un mode de réalisation stéréophonique de l'encodeur.
  • La Fig. 6 représente l'architecture d'un mode de réalisation stéréophonique du décodeur.
The characteristics of the invention mentioned above, as well as others, will appear more clearly on reading the following description of an exemplary embodiment, said description being given in relation to the attached drawings, among which:
  • The Fig. 1 represents the general architecture of the encoding method of an exemplary embodiment of the invention.
  • The Fig. 2 represents the general architecture of the decoding method of the exemplary embodiment of the invention.
  • The Fig. 3 represents the architecture of an embodiment of the encoder.
  • The Fig. 4 represents the architecture of an embodiment of the decoder.
  • The Fig. 5 represents the architecture of a stereophonic embodiment of the encoder.
  • The Fig. 6 represents the architecture of a stereophonic embodiment of the decoder.

La Fig. 1 représente le procédé d'encodage de manière générale. Le signal 101 est le signal source devant être encodé, ce signal est alors le signal original non limité en fréquence. L'étape 102 représente une étape de limitation en fréquence du signal 101. Cette limitation en fréquence peut, par exemple, être réalisée par un sous-échantillonnage du signal 101 préalablement filtré par un filtre passe-bas. Un sous-échantillonnage consiste à ne garder qu'un échantillon sur un ensemble d'échantillons et à supprimer du signal les autres échantillons. Un sous-échantillonnage d'un facteur « n » où l'on garde un échantillon sur n permet d'obtenir un signal dont la largeur du spectre sera divisée par n. n est ici un entier naturel. Il est aussi possible d'effectuer un sous-échantillonnage d'un rapport rationnel q/p, on sur-échantillonne d'un facteur p puis on sous-échantillonne d'un facteur q, il est préférable de commencer par le sur-échantillonnage pour ne pas perdre de contenu spectral. Pour un changement de fréquence d'un rapport non rationnel, on peut chercher la fraction rationnelle la plus proche et procéder comme ci-dessus. D'autres méthodes de limitation de la bande du signal d'entrée 101 peuvent également être utilisées comme des méthodes à base de filtrage. Le signal résultant, que nous appellerons le signal limité en fréquence, est alors encodé lors de l'étape 106. Tout moyen d'encodage ou de compression audio peut être ici employé comme, par exemple, un encodage selon les normes PCM, ADPCM ou autres. Ce signal limité en fréquence sera fourni au multiplexeur 108 en vue de sa transmission au décodeur.The Fig. 1 represents the encoding method generally. The signal 101 is the source signal to be encoded, this signal is then the original signal not limited in frequency. Step 102 represents a frequency limiting step of the signal 101. This frequency limitation can, for example, be achieved by subsampling of the signal 101 previously filtered by a low-pass filter. Subsampling consists of keeping only one sample on a set of samples and removing the other samples from the signal. A sub-sampling of a factor "n" where a sample is kept on n makes it possible to obtain a signal whose width of the spectrum will be divided by n. n is here a natural integer. It is also possible to sub-sample a rational ratio q / p, oversample by a factor p and then sub-sample by a factor q, it is better to start with the oversampling to not lose spectral content. For a frequency change of a non-rational ratio, one can search for the nearest rational fraction and proceed as above. Other methods of limiting the band of the input signal 101 may also be used as filter-based methods. The resulting signal, which we will call the frequency-limited signal, is then encoded in step 106. Any audio encoding or compression means can be used here as, for example, an encoding according to PCM, ADPCM or other. This frequency-limited signal will be supplied to the multiplexer 108 for transmission to the decoder.

Le signal limité en fréquence et encodé en sortie du module de compression 106 est également fourni en entrée d'un module de décodage 107. Ce module effectue l'opération inverse du module d'encodage 106 et permet de construire une version du signal limité en fréquence identique à la version à laquelle le décodeur aura accès lorsqu'il effectuera également cette opération de décodé du signal limité et encodé qu'il recevra. Le signal limité ainsi décodé est alors restauré dans l'étendue spectrale d'origine par un module d'enrichissement en fréquence 103. Cet enrichissement en fréquence peut, par exemple, consister en un simple sur-échantillonnage du signal d'entrée par l'insertion d'échantillons de valeur nulle entre les échantillons du signal d'entrée. Toute autre méthode d'enrichissement du spectre du signal peut également être utilisée. Ce signal à fréquence étendue, issu du module d'enrichissement en fréquence 103, est alors fourni à un module de génération de filtre 104. Ce module de génération de filtre 104 reçoit également le signal original 101 et calcule un filtre temporel permettant, lorsqu'il est appliqué au signal étendu issu du module d'enrichissement en fréquence 103, de mettre en forme celui-ci pour se rapprocher du signal original. Le filtre ainsi calculé est alors fourni au multiplexeur 108 après une étape optionnelle de compression 105.The signal limited in frequency and encoded at the output of the compression module 106 is also provided at the input of a decoding module 107. This module performs the inverse operation of the encoding module 106 and makes it possible to construct a limited version of the signal. frequency identical to the version to which the decoder will have access when it also performs this decoded operation of the limited signal and encoded it will receive. The limited signal thus decoded is then restored to the original spectral range by a frequency enrichment module 103. This frequency enrichment may, for example, consist of a simple over-sampling of the input signal by the insertion of zero-valued samples between the samples of the input signal. Any other method of enriching the signal spectrum can also be used. This extended frequency signal, derived from the frequency enrichment module 103, is then supplied to a filter generation module 104. This filter generation module 104 also receives the original signal 101 and calculates a time filter allowing, when it is applied to the extended signal from the frequency enrichment module 103, to shape it to get closer to the original signal. The filter thus calculated is then supplied to the multiplexer 108 after an optional compression step 105.

De cette manière, il est possible de transporter une version limitée en fréquence et compressée du signal à transmettre et les coefficients d'un filtre temporel. Ce filtre temporel permettant, une fois appliqué au signal décompressé et étendu en fréquence, de remettre celui-ci en forme pour retrouver un signal étendu proche du signal original. Le calcul du filtre se faisant sur le signal original et sur le signal tel qu'il sera obtenu par le décodeur suite à la décompression et à l'enrichissement en fréquence permet de corriger les défauts introduits par ces deux phases de traitement. D'une part, le filtre étant appliqué au signal reconstruit dans toute sa plage de fréquence permet de corriger certains artéfacts de compression sur la partie basse fréquence transmise. D'autre part, il remet également en forme la partie haute fréquence, non transmise, reconstruite par enrichissement en fréquence.In this way, it is possible to carry a limited frequency and compressed version of the signal to be transmitted and the coefficients of a temporal filter. This temporal filter makes it possible, once applied to the decompressed and extended frequency signal, to reset the latter to find an extended signal close to the original signal. The calculation of the filter being done on the original signal and on the signal as it will be obtained by the decoder following the decompression and the frequency enrichment makes it possible to correct the defects introduced by these two phases of treatment. On the one hand, the filter being applied to the reconstructed signal throughout its frequency range makes it possible to correct certain compression artifacts on the transmitted low frequency part. On the other hand, it also reshapes the high frequency part, not transmitted, reconstructed by frequency enrichment.

La Fig. 2 représente de manière générale le procédé de décodage correspondant. Le décodeur reçoit donc le signal issu du multiplexeur 108 du codeur. Il le démultiplexe pour obtenir d'une part le signal limité en fréquence encodé, appelé S1b, et les coefficients du filtre F, contenus dans le signal transmis. Le signal S1b est alors décodé par un module de décodage ou de décompression 202 équivalent fonctionnellement au module 107 de la Fig. 1. Une fois décodé, le signal est étendu en fréquence par le module 203 équivalent fonctionnellement au module 103 de la Fig. 1. On obtient donc une version décodée et étendue en fréquence du signal. D'autre part, les coefficients du filtre F sont décodés s'ils avaient été encodés ou compressés par un module de décompression 201, et le filtre obtenu est appliqué au signal temporel étendu dans un module de mise en forme du signal 204. On obtient alors un signal en sortie proche du signal original. Ce traitement est simple à mettre en oeuvre du fait de la nature temporelle du filtre à appliquer au signal pour la remise en forme.The Fig. 2 represents generally the corresponding decoding method. The decoder therefore receives the signal from the multiplexer 108 of the encoder. It demultiplexes it to obtain on the one hand the encoded frequency limited signal, called S1b, and the filter coefficients F, contained in the transmitted signal. The signal S1b is then decoded by a decoding or decompression module 202 that is functionally equivalent to the module 107 of the Fig. 1 . Once decoded, the signal is extended in frequency by the module 203 functionally equivalent to the module 103 of the Fig. 1 . A decoded and extended version of the signal is thus obtained. On the other hand, the coefficients of the filter F are decoded if they had been encoded or compressed by a decompression module 201, and the filter obtained is applied to the extended time signal in a signal conditioning module 204. then an output signal close to the original signal. This treatment is simple to implement because of the temporal nature of the filter to be applied to the signal for shaping.

Le filtre transmis, et donc appliqué lors de la reconstruction du signal, est transmis périodiquement et change dans le temps. Ce filtre est donc adapté à une portion du signal à laquelle il s'applique. Il est ainsi possible de calculer pour chaque portion de signal un filtre temporel particulièrement adapté en fonction des caractéristiques spectrales dynamiques de cette portion de signal. En particulier, il est possible d'avoir plusieurs types de générateurs de filtres temporels et de sélectionner pour chaque portion de signal le filtre donnant le meilleur résultat pour cette portion. Ceci est possible car le module de génération de filtre possède d'une part le signal original et d'autre part le signal étendu tel qu'il sera reconstruit par le décodeur, il est donc en mesure, dans le cas où il est généré par plusieurs filtres différents, de comparer le signal obtenu par application de chaque filtre à la portion de signal étendue et le signal original dont on cherche à s'approcher au plus près. Cette méthode de génération de filtre ne se limite donc pas à choisir un type de filtre déterminé pour l'ensemble du signal mais permet de changer de type de filtre en fonction des caractéristiques de chaque portion de signal.The transmitted filter, and thus applied during the reconstruction of the signal, is transmitted periodically and changes over time. This filter is adapted to a portion of the signal to which it applies. It is thus possible to calculate for each signal portion a time filter particularly adapted according to the dynamic spectral characteristics of this portion of signal. In particular, it is possible to have several types of time filter generators and to select for each portion of signal the filter giving the best result for this portion. This is possible because the filter generation module has on the one hand the original signal and on the other hand the extended signal as it will be reconstructed by the decoder, so it is able, in the case where it is generated by several different filters, compare the signal obtained by applying each filter to the extended signal portion and the original signal that is to be approached as closely as possible. This method of filter generation is therefore not limited to choosing a type of filter determined for the entire signal but allows to change the type of filter according to the characteristics of each portion of the signal.

Un mode particulier de réalisation de l'invention va maintenant être décrit en détail à l'aide des Fig. 3 et 4. Dans ce mode de réalisation, on cherche à partir d'un signal échantillonné à une fréquence donnée 301, par exemple 32 kHz, à obtenir le signal limité à ses basses fréquences nommé S1b. On cherche également à déterminer un filtre F permettant de mettre en forme le signal obtenu en étendant en fréquence le signal S1b. Le signal original 301 est filtré par un filtre passe-bas et sous-échantillonné d'un facteur n par le module de sous-échantillonnage 302. On ne conserve du signal original qu'un échantillon sur n, où n est un entier naturel. Dans la pratique, n n'excède généralement pas 4. Le signal perd alors en résolution spectrale et, par exemple, pour n = 2, on obtient un signal échantillonné à 16 kHz. Ce signal est ensuite encodé, par exemple par une méthode de type PCM (« Pulse Code Modulation »), par le module 311 qui sera ensuite compressé, par exemple par un ADPCM le module 312. On obtient ainsi le signal sous-échantillonné contenant les basses fréquences du signal original 301. Ce signal est envoyé au multiplexeur 314 pour être émis vers le décodeur.A particular embodiment of the invention will now be described in detail using the Fig. 3 and 4 . In this embodiment, it is sought from a signal sampled at a given frequency 301, for example 32 kHz, to obtain the signal limited to its low frequencies named S1b. It is also sought to determine a filter F for shaping the signal obtained by extending the signal S1b frequency. The original signal 301 is filtered by a low-pass filter and downsampled by a factor n by the subsampling module 302. Only the original signal is retained on n, where n is a natural integer. In practice, n generally does not exceed 4. The signal then loses in spectral resolution and, for example, for n = 2, a signal sampled at 16 kHz is obtained. This signal is then encoded, for example by a method of the PCM ("Pulse Code Modulation") type, by the module 311 which will then be compressed, for example by an ADPCM module 312. This gives the subsampled signal containing the low frequencies of the original signal 301. This signal is sent to the multiplexer 314 to be transmitted to the decoder.

En parallèle, ce signal est transmis à un module de décodage 313. On simule de cette façon, dans l'encodeur, le signal que le décodeur obtiendra à partir du signal qui lui sera envoyé. Ce signal qui va être utilisé pour la génération du filtre F va donc permettre de tenir compte des artéfacts résultant de ces phases de codage et de décodage, de compression et de décompression. Ce signal est ensuite étendu en fréquence par insertion de n-1 zéro entre chaque échantillon du signal temporel dans le module 303. De cette manière, on reconstruit un signal de la même étendue spectrale que le signal d'origine. D'après le théorème de Nyquist, on obtient un repliement de spectre d'ordre n. Par exemple, pour n=2, le signal est sous-échantillonné d'un ordre 2 à l'encodage et sur-échantillonné d'un ordre 2 au décodage. Le spectre est dupliqué « en miroir » par une symétrie axiale dans le domaine des fréquences. Dans le module 304, une transformée de Fourier est effectuée sur le signal temporel étendu en fréquence issu du module 303. En fait, une transformée de Fourier rapide glissante est effectuée sur des fenêtres de travail de taille donnée et variable. Ces tailles sont typiquement de 128, 256, 512 échantillons mais peuvent être de taille quelconque même si on utilisera préférentiellement des puissances de deux pour simplifier les calculs. On calcule ensuite les modules de ces transformées appliquées à ces fenêtres. Un même calcul de transformée de Fourier est effectué sur le signal original dans le module 306.In parallel, this signal is transmitted to a decoding module 313. This way, in the encoder, the signal that the decoder will obtain from the signal sent to it is simulated. This signal that will be used for the generation of the filter F will therefore allow to take into account the artifacts resulting from these coding and decoding phases, compression and decompression. This signal is then frequency-expanded by inserting n-1 zero between each sample of the time signal in the module 303. In this way, a signal of the same spectral range as the original signal is reconstructed. According to the Nyquist theorem, we obtain a n-order spectrum folding. For example, for n = 2, the signal is downsampled from an order 2 to the encoding and oversampled from an order 2 to the decoding. The spectrum is duplicated "mirrored" by axial symmetry in the frequency domain. In the module 304, a Fourier transform is performed on the frequency-extended time signal from the module 303. In fact, a sliding fast Fourier transform is performed on work windows of given size and variable. These sizes are typically 128, 256, 512 samples but can be of any size even if one will preferentially use powers of two to simplify calculations. The modules of these transforms applied to these windows are then calculated. A same Fourier transform calculation is performed on the original signal in the module 306.

Une division membre à membre 305 est alors effectuée entre les modules des coefficients des transformées de Fourier obtenues par les étapes 304 et 306 pour générer par transformées de Fourrier inverses des filtres temporels de tailles proportionnelles à celles des fenêtres utilisées, donc 128, 256 ou 512. Plus la taille de la fenêtre choisie sera grande, plus le filtre comportera de coefficients et sera plus précis mais plus son application sera coûteuse en calcul au décodage. Cette étape génère donc plusieurs filtres de différentes tailles parmi lesquelles il va falloir choisir le filtre finalement utilisé. On verra que cette étape de choix est effectuée par le module 309. Comme les coefficients du rapport entre les fenêtres sont réels, symétriques dans l'espace des fréquences, le filtre F équivalent est alors, dans le domaine temporel, réel et symétrique. Cette propriété de symétrie peut être utilisée pour ne transmettre que la moitié des coefficients, l'autre se déduisant par symétrie. L'obtention d'un filtre réel symétrique permet aussi de réduire le nombre d'opérations nécessaires lors de la convolution du signal reçu étendu par le filtre dans le décodeur. D'autres modes de réalisation permettent d'obtenir des filtres réels non symétriques. Par exemple, si le signal temporel dans une fenêtre de travail est limité en fréquence, on peut avantageusement déterminer de manière itérative les paramètres d'un filtre à réponse impulsionnelle infinie, passe-bas de Tchebychev à partir des spectres issus des étapes 304 et 306 et de la fréquence de coupure de la fenêtre.A member-to-member division 305 is then performed between the modules of the Fourier transform coefficients obtained by steps 304 and 306 to generate by inverse Fourier transforms temporal filters of sizes proportional to those of the windows used, ie 128, 256 or 512. The larger the size of the chosen window, the more coefficients and more precise the filter will be but the more expensive it will be to decode. This step therefore generates several filters of different sizes among which we will have to choose the filter finally used. It will be seen that this selection step is performed by the module 309. Since the coefficients of the ratio between the windows are real, symmetrical in the frequency space, the equivalent filter F is then, in the time domain, real and symmetrical. This property of symmetry can be used to transmit only half of the coefficients, the other being deduced by symmetry. Obtaining a symmetrical real filter also makes it possible to reduce the number of operations required during the convolution of the received signal extended by the filter in the decoder. Other embodiments make it possible to obtain real unsymmetrical filters. For example, if the temporal signal in a working window is frequency-limited, it is advantageous to iteratively determine the parameters of an infinite impulse response filter, Chebychev low-pass from the spectra from steps 304 and 306. and the cutoff frequency of the window.

On obtient ainsi le filtre, dans l'espace temporel, fourni en entrée du module de choix 309.This gives the filter, in the time space, provided at the input of the choice module 309.

De manière optionnelle, un module 308 va offrir d'autres types de filtres. Par exemple, il peut offrir des filtres linéaires, cubiques ou autres. En effet, ces filtres sont connus pour permettre le sur-échantillonnage. Pour calculer les valeurs des échantillons rajoutés avec une valeur initiale à zéro entre les échantillons du signal limité en fréquence, il est possible de dupliquer la valeur de l'échantillon connu, de faire la moyenne entre les échantillons, ce qui revient à faire une interpolation linéaire entre les valeurs connues des échantillons. Tous ces types de filtres sont indépendants de la valeur du signal et permettent de remettre en forme le signal sur-échantillonné. Le module 308 contient donc un nombre arbitraire de tels filtres pouvant être utilisés.Optionally, a module 308 will offer other types of filters. For example, it can offer linear, cubic or other filters. Indeed, these filters are known to allow oversampling. To calculate the values of the samples added with an initial value to zero between the samples of the signal limited in frequency, it is possible to duplicate the value of the known sample, to average between the samples, which amounts to interpolating linear between the known values of the samples. All these types of filters are independent the signal value and reshape the oversampled signal. The module 308 therefore contains an arbitrary number of such filters that can be used.

Le module de choix 309 va donc avoir en entrée une collection de filtres. D'une part, il aura les filtres générés par le module 307 et correspondant aux filtres générés pour différentes tailles de fenêtres par division des modules des transformées de Fourier appliquées au signal original et au signal reconstruit. D'autre part, il aura également en entrée le signal original 301 et le signal reconstruit issu du module 303. De cette façon le module 309 peut comparer l'application des différents filtres au signal reconstruit issu du module 303 avec le signal original pour choisir le filtre donnant, sur la portion de signal considérée, le meilleur signal de sortie, c'est-à-dire le plus proche spectralement du signal original. Par exemple, on peut faire le rapport entre le spectre obtenu par application du filtre au signal issu du module 303 et le spectre de la même portion du signal original. On choisit alors le filtre engendrant le minimum d'une fonction de la distorsion. Cette portion de signal, appelée fenêtre de travail, devra être plus grande que la plus grande fenêtre ayant servi au calcul des filtres, on pourra utiliser typiquement une taille de fenêtre de travail de 512 échantillons. La taille de cette fenêtre de travail peut également varier en fonction du signal. En effet, une grande taille de fenêtre de travail peut être utilisée pour l'encodage d'une partie de signal sensiblement stationnaire tandis qu'une fenêtre plus courte sera plus adaptée pour une portion de signal plus dynamique pour mieux prendre en compte les variations rapides. C'est cette partie qui permet de sélectionner, pour chaque portion du signal, le filtre le plus pertinent permettant la meilleure reconstruction par le décodeur du signal et de se rapprocher du signal original.The choice module 309 will therefore have as input a collection of filters. On the one hand, it will have the filters generated by the module 307 and corresponding to the filters generated for different window sizes by dividing the modules of the Fourier transforms applied to the original signal and the reconstructed signal. On the other hand, it will also have in input the original signal 301 and the reconstructed signal coming from the module 303. In this way the module 309 can compare the application of the different filters to the reconstructed signal coming from the module 303 with the original signal to choose the filter giving, on the signal portion considered, the best output signal, that is to say the spectrally closest to the original signal. For example, it is possible to relate the spectrum obtained by applying the filter to the signal from the module 303 and the spectrum of the same portion of the original signal. We then choose the filter generating the minimum of a function of the distortion. This portion of the signal, called the working window, will have to be larger than the largest window used for calculating the filters. It will be possible to use typically a working window size of 512 samples. The size of this working window may also vary depending on the signal. Indeed, a large working window size can be used for the encoding of a substantially stationary signal portion while a shorter window will be more suitable for a more dynamic signal portion to better take into account the rapid variations. . It is this part that makes it possible to select, for each portion of the signal, the most relevant filter allowing the best reconstruction by the decoder of the signal and to get closer to the original signal.

Une fois ce filtre choisi, le module 310 va quantifier les coefficients spectraux du filtre qui seront encodés, par exemple, en utilisant une table de Huffman pour optimiser les données à transmettre. Le multiplexeur 314 va donc multiplexer avec chaque portion du signal, le filtre le plus pertinent pour le décodage de cette portion de signal. Ce filtre étant choisi soit dans la collection de filtres de tailles différentes générés par analyse de cette portion de signal, soit dans la collection, comprend également une série de filtres déterminés, typiquement linéaires, permettant la reconstruction, qui pourront être choisis s'ils se révèlent plus intéressants pour la reconstruction par le décodeur de la portion de signal. Quand le filtre généré est un des filtres déterminés, il est possible de ne transmettre qu'un identificateur identifiant ce filtre parmi la collection des filtres déterminés, typiquement linéaires, permettant la reconstruction, qui pourront être choisis s'ils se révèlent plus intéressants pour la reconstruction par le décodeur de la portion de signal. Quand le filtre généré est un des filtres déterminés, il est possible de ne transmettre qu'un identificateur identifiant ce filtre parmi la collection des filtres déterminés fournie par le module 308, ainsi que des paramètres éventuels du filtre. En effet, les coefficients de ces filtres déterminés n'étant pas calculés en fonction de la portion de signal à laquelle on veut les appliquer, il est inutile de transporter ces coefficients qui peuvent être connus du décodeur. Ainsi, la bande passante pour le transport de l'information relative au filtre se réduit dans ce cas à un simple identificateur du filtre.Once this filter is chosen, the module 310 will quantify the spectral coefficients of the filter that will be encoded, for example, using a Huffman table to optimize the data to be transmitted. Multiplexer 314 will thus multiplex with each portion of the signal, the most relevant filter for the decoding of this portion of signal. This filter being chosen either in the collection of filters of different sizes generated by analysis of this portion of the signal, or in the collection, also comprises a series of determined, typically linear filters, allowing the reconstruction, which can be chosen if they occur. reveal more interesting for the reconstruction by the decoder of the signal portion. When the generated filter is one of the determined filters, it is possible to transmit only an identifier identifying this filter from the collection of determined, typically linear filters, allowing the reconstruction, which can be chosen if they prove more interesting for the reconstruction by the decoder of the portion of signal. When the generated filter is one of the determined filters, it is possible to transmit only an identifier identifying this filter from the collection of determined filters provided by the module 308, as well as possible parameters of the filter. Indeed, since the coefficients of these determined filters are not calculated as a function of the portion of the signal to which they are to be applied, it is unnecessary to transport these coefficients which may be known to the decoder. Thus, the bandwidth for transporting information relating to the filter is reduced in this case to a simple identifier of the filter.

La Fig. 4 représente le décodage correspondant dans le mode particulier de réalisation décrit. Le signal est reçu par le décodeur qui démultiplexe le signal. Le signal audio S1b est alors décodé par le module 404 puis sur-échantillonné d'un facteur n par l'insertion de n-1 échantillons à zéro entre les échantillons reçus par le module 405. Parallèlement, les coefficients spectraux du filtre F sont déquantifiés et décodés en suivant les tables de Huffman par le module 401. Avantageusement, la taille du filtre peut être adaptée par le module 402 du décodeur à ses capacités de calcul ou de mémoire ou encore de toute limitation matérielle éventuelle. Un décodeur possédant peu de ressources pourra utiliser un filtre sous-échantillonné ce qui lui permettra de diminuer les opérations lors de l'application du filtre. Le filtre sous-échantillonné peut aussi être généré par l'encodeur suivant les ressources du canal de transmission ou les ressources du décodeur, à condition bien sûr que cette dernière information soit détenue par l'encodeur. De plus, le spectre du filtre peut être réduit au décodage pour réaliser un suréchantillonnage moins important (n-1, n-2 etc..) en fonction des capacités matérielles de rendu sonore du décodeur telles que la puissance ou les capacités de sortie son. Le module 403 effectue alors une transformée de Fourier inverse sur les coefficients spectraux du filtre pour obtenir le filtre réel dans le domaine temporel. Dans l'exemple de réalisation, le filtre est de plus symétrique ce qui permet de réduire les données transportées pour la transmission du filtre. Le module 406 opère la convolution du signal sur-échantillonné issu du module 405 avec le filtre ainsi reconstitué pour obtenir le signal résultant. Cette convolution est particulièrement peu gourmande en calcul du fait que le sur-échantillonnage s'effectue par insertion de valeurs nulles. D'autre part, le fait que le filtre soit réel, et voire même symétrique dans le mode de réalisation préféré, permet également de réduire le nombre d'opérations nécessaires à cette convolution.The Fig. 4 represents the corresponding decoding in the particular embodiment described. The signal is received by the decoder which demultiplexes the signal. The audio signal S1b is then decoded by the module 404 and then oversampled by a factor n by the insertion of n-1 samples at zero between the samples received by the module 405. In parallel, the spectral coefficients of the filter F are dequantized and decoded according to the Huffman tables by the module 401. Advantageously, the size of the filter can be adapted by the module 402 of the decoder to its computing or memory capacity or any possible hardware limitation. A decoder with few resources can use a subsampled filter which will allow it to reduce the operations during the application of the filter. The subsampled filter can also be generated by the encoder according to the resources of the transmission channel or the resources of the decoder, provided of course that the latter information is held by the encoder. In addition, the spectrum of the filter can be reduced to decoding to perform a smaller oversampling (n-1, n-2 etc. ..) depending on the hardware sound output capabilities of the decoder such as the power or the output capabilities its . The module 403 then performs an inverse Fourier transform on the spectral coefficients of the filter to obtain the real filter in the time domain. In the exemplary embodiment, the filter is moreover symmetrical which makes it possible to reduce the data transported for the transmission of the filter. The module 406 operates the convolution of the oversampled signal from the module 405 with the filter thus reconstituted to obtain the resulting signal. This convolution is particularly greedy in calculation because the oversampling is done by inserting null values. On the other hand, the fact that the filter is real, and even symmetrical in the preferred embodiment, also reduces the number of operations required for this convolution.

Le filtre étant appliqué à l'intégralité du signal étendu en fréquence, l'invention offre l'avantage d'effectuer une remise en forme, non seulement de la partie haute du spectre reconstituée à partir de la partie basse transmise mais de l'ensemble du signal ainsi reconstitué. De cette manière, elle permet de modeler la partie du spectre non transmise mais également de corriger des artéfacts dus aux différentes opérations de compression, décompression, d'encodage et décodage de la partie basse fréquence transmise.As the filter is applied to the entire extended frequency signal, the invention offers the advantage of performing a reshaping, not only of the high part of the spectrum reconstituted from the transmitted lower part but of the whole of the signal thus reconstituted. In this way, it makes it possible to model the part of the non-transmitted spectrum but also to correct artifacts due to the various operations of compression, decompression, encoding and decoding of the transmitted low frequency part.

Un avantage secondaire de l'invention est la possibilité d'adapter dynamiquement les filtres utilisés en fonction de la nature de chaque portion de signal grâce au module permettant le choix du meilleur filtre, en termes de qualité de rendu sonore et de « temps machine » utilisé, parmi plusieurs pour chaque portion du signal.A secondary advantage of the invention is the ability to dynamically adapt the filters used according to the nature of each portion of the signal thanks to the module allowing the choice of the best filter, in terms of sound quality and "machine time" used, among several for each portion of the signal.

Le procédé d'encodage ainsi décrit pour un signal mono canal peut être adapté pour un signal multi canal. Une première adaptation évidente consiste en l'application de la solution mono canal à chaque canal audio indépendamment. Cette solution se révèle néanmoins coûteuse en ce qu'elle ne tire pas partie de la corrélation forte entre les différents canaux d'un flux audio multi canal. La solution proposée consiste à composer un canal unique à partir des différents canaux du flux. Un traitement similaire à celui décrit précédemment dans le cas d'un signal mono canal est alors effectué sur ce flux composé. A la différence du procédé mono canal, dans le cas du multi canal, un filtre est déterminé pour chaque canal de manière à reproduire le canal considéré lorsqu'il est appliqué au flux composé. On transmet ainsi un flux audio multi canal en ne transmettant qu'un flux composé et autant de filtres qu'il y a de canaux à transmettre. Le procédé va maintenant être décrit plus précisément à l'aide des figures 5 et 6 dans le cas de la stéréophonie. Le mode de réalisation stéréophonique s'étend de manière naturelle à un flux composé de plus de deux canaux comme un flux 5.1 pour le cinéma à domicile par exemple.The encoding method thus described for a single-channel signal can be adapted for a multi-channel signal. A first obvious adaptation is the application of the single-channel solution to each audio channel independently. This solution is nevertheless expensive in that it does not take advantage of the strong correlation between the different channels of a multi-channel audio stream. The proposed solution is to compose a single channel from the different channels of the stream. A treatment similar to that described previously in the case of a single-channel signal is then performed on this compound stream. Unlike the single-channel method, in the case of multi-channel, a filter is determined for each channel so as to reproduce the considered channel when it is applied to the composite stream. This transmits a multi-channel audio stream by transmitting only a compound stream and as many filters as there are channels to transmit. The process will now be described more precisely using the Figures 5 and 6 in the case of stereophony. The stereophonic embodiment naturally extends to a stream composed of more than two channels such as a stream 5.1 for home theater for example.

La Fig. 5 représente l'architecture d'un encodeur stéréophonique selon un mode de réalisation de l'invention. Le flux audio à encoder est composé d'un canal gauche « L » référencé 501 et d'un canal droit « R » référencé 502. Un module de composition 503 compose ces deux signaux pour générer un signal composé. Cette composition peut, par exemple, être une moyenne des deux canaux, le signal composé est alors égal à L+R / 2. Ce signal composé subit alors le même traitement que le signal mono canal décrit précédemment. Il subit un sous échantillonnage d'un facteur n par le module de sous échantillonnage 504. Le signal sous échantillonné est alors codé par un codeur 505 pour être encodé par un encodeur 506. Ces modules sont les mêmes que les modules déjà décrits 311 et 312 de la Fig. 3. Le signal composé sous échantillonné et encodé est transmis au destinataire du flux. Il est également décodé par un module de décodage 507 correspondant au module 313 de la Fig. 3. Ensuite, il est sur échantillonné par le module de sur échantillonnage 508 correspondant au module 303. Le signal est alors traité par deux modules de génération de filtres 509 et 510. Chacun de ces modules correspond aux modules 304, 305, 306, 308, 309 et 310 de la Fig. 3. Le premier, 509, génère un filtre FR qui permet, lorsqu'il est appliqué au flux composé issu du module 508, de générer un signal proche du canal droit R. Ce module prend en entrée le signal composé issu du module 508 et le signal du canal droit R original 502. Le second, 510, génère un filtre FL qui permet, lorsqu'il est appliqué au flux composé issu du module 508, de générer un signal proche du canal gauche L. Ce module prend en entrée le signal composé issu du module 508 et le signal du canal gauche L original 501. Ces filtres, ou un identificateur de ces filtres, sont alors multiplexés avec le flux sous échantillonné et encodé issu du module d'encodage 506 pour être émis à destination du récepteur.The Fig. 5 represents the architecture of a stereophonic encoder according to an embodiment of the invention. The audio stream to be encoded consists of a left channel "L" referenced 501 and a right channel "R" referenced 502. A composition module 503 composes these two signals to generate a composite signal. This composition may, for example, be an average of the two channels, the composite signal is then equal to L + R / 2. This compound signal then undergoes the same processing as the single-channel signal described above. It is sub-sampled by a factor n by the subsampling module 504. The subsampled signal is then coded by an encoder 505 to be encoded by an encoder 506. These modules are the same as the modules already described 311 and 312 of the Fig. 3 . The subsampled and encoded composite signal is transmitted to the recipient of the stream. It is also decoded by a decoding module 507 corresponding to the module 313 of the Fig. 3 . Then, it is oversampled by the sampling module 508 corresponding to the module 303. The signal is then processed by two filter generation modules 509 and 510. Each of these modules corresponds to the modules 304, 305, 306, 308, 309. and 310 of the Fig. 3 . The first, 509, generates a filter F R which, when applied to the compound stream coming from the module 508, generates a signal close to the right channel R. This module takes as input the composite signal from the module 508 and the signal of the original R right channel 502. The second, 510, generates a filter F L which, when applied to the compound stream coming from the module 508, generates a signal close to the left channel L. This module takes as input the composed signal from the module 508 and the original left channel signal L 501. These filters, or an identifier of these filters, are then multiplexed with the subsampled stream and encoded from the encoding module 506 to be sent to the receiver .

Généralement les différents canaux d'un signal multi canal possèdent une forte corrélation mais exhibent un déphasage temporel. Un léger décalage temporel intervient entre les signaux des différents canaux. De ce fait, lorsque l'on moyenne les deux, ou plus, canaux pour générer le signal composé, ce décalage tend à générer du bruit. Avantageusement on choisit donc un des canaux pour servir de référence, par exemple le canal gauche « L », et les autres canaux sont recalés sur ce canal de référence préalablement à la composition du signal composé. Ce recalage est effectué par corrélation temporelle entre le canal à recaler et le canal de référence. Cette corrélation définit une valeur de décalage sur la fenêtre de travail choisie pour la corrélation. Cette fenêtre de travail est avantageusement choisie égale à la fenêtre de travail utilisée pour la génération du filtre. La valeur du décalage peut alors être associée au filtre généré pour être transmise en sus des filtres de façon à permettre de reconstituer le déphasage original inter canal lors de la restitution du flux audio.Generally, the different channels of a multi-channel signal have a strong correlation but exhibit a temporal phase shift. A slight time shift occurs between the signals of the different channels. As a result, when the two or more channels are averaged to generate the composite signal, this offset tends to generate noise. Advantageously one therefore chooses one of the channels to serve as a reference, for example the left channel "L", and the other channels are reset on this reference channel prior to the composition of the composite signal. This registration is performed by temporal correlation between the channel to be recalibrated and the reference channel. This correlation defines an offset value on the working window chosen for the correlation. This working window is advantageously chosen equal to the working window used for generating the filter. The value of the offset can then be associated with the generated filter to be transmitted in addition to the filters so as to allow reconstruction of the original inter-channel phase shift during the reproduction of the audio stream.

Une étape d'égalisation des gains des signaux des différents canaux peut intervenir pour homogénéiser les puissances des signaux correspondant aux différents canaux. Cette égalisation définit une valeur d'amplification devant être appliquée au signal sur la fenêtre de travail. Cette valeur d'amplification peut être introduite dans le filtre calculé permettant la reconstitution du signal au décodage. Le calcul de cette valeur d'amplification se fait pour chaque canal sauf un choisi comme canal de référence. L'introduction de la valeur d'amplification permet de reconstituer au décodage les différences de gains entre les canaux dans le signal d'origine.A step of equalizing the gains of the signals of the different channels can be used to homogenize the powers of the signals corresponding to the different channels. This equalization defines an amplification value to be applied to the signal on the working window. This amplification value can be introduced into the calculated filter allowing the reconstruction of the signal at decoding. The calculation of this Amplification value is for each channel except one chosen as the reference channel. The introduction of the amplification value makes it possible to reconstruct, at decoding, the differences in gains between the channels in the original signal.

Par ailleurs, le calcul de génération d'un filtre ainsi que celui du déphasage se fait sur une portion de signal appelé fenêtre de travail (frame en anglais). Lors de la restitution du flux audio, le passage d'une fenêtre à une autre va donc entraîner un changement de déphasage entre les canaux. Ce changement peut entraîner un bruit lors de la restitution. Pour éviter ce bruit, il est possible de lisser le déphasage aux frontières de fenêtres de travail. Ainsi, le changement de fenêtre n'entraîne plus de changement de valeur de déphasage brusque.Furthermore, the generation calculation of a filter as well as that of the phase shift is done on a portion of signal called work window ( frame in English). When restoring the audio stream, the passage from one window to another will therefore cause a change in phase shift between the channels. This change may cause noise during playback. To avoid this noise, it is possible to smooth the phase shift at the boundaries of working windows. Thus, the window change no longer leads to a sudden phase shift value change.

La Fig. 6 représente l'architecture d'un mode de réalisation stéréophonique du décodeur. Cette figure est le pendant stéréophonique de la Fig. 4. Le flux audio reçu est démultiplexé pour obtenir le flux composé basse fréquence encodé appelé S1b et les filtres FR et FL. Le flux composé est alors décodé par le module de décodage 601 correspondant au module 404 de la Fig. 4. Son spectre est alors élargi en fréquence par le module de sur échantillonnage 602 correspondant au module 405 de la Fig. 4. Le signal ainsi obtenu est alors convolué par les filtres FR et FL décompressés par les modules 603 et 605 pour redonner les canaux droit et gauche SR et SL.The Fig. 6 represents the architecture of a stereophonic embodiment of the decoder. This figure is the stereophonic counterpart of Fig. 4 . The received audio stream is demultiplexed to obtain the encoded low frequency composite stream called S 1b and the filters F R and F L. The compound stream is then decoded by the decoding module 601 corresponding to the module 404 of the Fig. 4 . Its spectrum is then widened in frequency by the sampling module 602 corresponding to the module 405 of the Fig. 4 . The signal thus obtained is then convoluted by the filters F R and F L decompressed by the modules 603 and 605 to restore the right and left channels S R and S L.

Si une information de déphasage est introduite dans le flux, le canal qui ne sert pas de canal de référence pour le déphasage est recalé en utilisant cette information pour générer le déphasage des canaux d'origine. Cette information de déphasage peut, par exemple, prendre la forme d'une valeur de décalage associée à chacun des filtres pour les canaux autres que le canal défini comme canal de référence. Avantageusement ce décalage est lissé, par exemple de manière linéaire, entre les différentes fenêtres de travail.If phase shift information is introduced into the stream, the channel that does not serve as a reference channel for the phase shift is recalibrated using this information to generate the phase shift of the original channels. This phase shift information may, for example, take the form of an offset value associated with each of the filters for the channels other than the channel defined as reference channel. Advantageously, this offset is smoothed, for example linearly, between the different work windows.

Claims (18)

  1. Method for encoding all or part of a multi-channel audio stream, comprising at least the following steps:
    - a step of obtaining a compound signal obtained by compounding signals corresponding to each channel of the multi-channel audio stream;
    - a step of obtaining a limited-frequency compound signal, the reduction in the frequency of the original compound signal being obtained by suppressing the high frequencies;
    characterised in that it further comprises
    - a step of generating a time filter for each channel, making it possible to find a signal that is close in spectrum to the original signal of the corresponding channel when it is applied to the signal obtained by broadening the spectrum of the limited compound signal.
  2. Method according to claim 1 wherein, for a given portion of the original signal, for a given channel, the filter corresponding to this channel is obtained by member-to-member division of a function of the coefficients of a Fourier transform applied on the one hand to the portion of the original signal and on the other hand to the corresponding portion of the signal obtained by broadening the spectrum of the limited signal.
  3. Method according to claim 2, wherein Fourier transforms of different sizes are used to obtain a plurality of filters corresponding to each size used, the filter generated corresponding to a choice between the plurality of filters obtained by comparison of the original signal, and the signal obtained by application of the filter to the signal obtained by broadening the spectrum of the limited signal.
  4. Method according to one of claims 1 to 3, wherein the choice of time filter may be made from a collection of predetermined time filters.
  5. Method according to one of claims 1 to 4, wherein, with the limited-frequency compound signal encoded for transmission, the filter is generated from the signal obtained by decoding and broadening the spectrum of the encoded limited compound signal and the original signal.
  6. Method according to one of claims 1 to 5, characterised in that it further comprises:
    - a step of defining one of.the channels of the multi-channel audio stream as the reference channel;
    - a step of time correlation of each of the other channels with said reference channel, defining an offset value for each channel;
    - the step of compounding the signals of each channel is carried out with the signal of the reference channel and the signals that are time-correlated for the other channels.
  7. Method according to claim 6, characterised in that for each channel other than the reference channel, the offset value defined by the time correlation of the channel is associated with the filter generated.
  8. Method according to one of claims 1 to 5, characterised in that it further comprises:
    - a step of defining one of the channels of the multi-channel audio stream as the reference channel;
    - a step of equalising each of the other channels on said reference channel, defining an amplification value for each channel;
    the step of compounding the signals of each channel is carried out with the signal of the reference channel and the signals that are equalised for the other channels.
  9. Method according to claim 8, characterised in that for each channel other than the reference channel, the amplification value defined by the time correlation of the channel is associated with the filter generated.
  10. Method of decoding all or part of a multi-channel audio stream, comprising at least the following steps:
    - a step of receiving a transmitted signal;
    characterised in that it further comprises:
    - a step of receiving a time filter relating to the signal received for each channel of the multi-channel audio stream;
    - a step of obtaining a decoded signal by decoding the signal received;
    - a step of obtaining an extended signal by broadening the spectrum of the decoded signal;
    - a step of obtaining a reconstructed signal by convoluting the extended signal with the time filter received for each channel of the multi-channel audio stream.
  11. Method according to claim 10 in which a filter that is reduced in size starting from the generated filter is used instead of this generated filter in the step of obtaining a reconstructed signal for each channel.
  12. Method according to claim 11, wherein the choice of using a filter of reduced size instead of the filter generated for each channel is made as a function of the capabilities of the decoder.
  13. Method according to one of claims 10 to 12, characterised in that, with one of the channels of the multi-channel stream defined as the reference channel, an offset value being associated with each filter received for the channels other than the reference channel, the method further comprises:
    - a step of offsetting the signal corresponding to each channel other than the reference channel allowing a time phase shift to be generated which is similar to the time phrase shift between each channel and the reference channel in the original multi-channel audio stream.
  14. Method according to claim 13, characterised in that it further comprises:
    - a step of smoothing the offset values at the frontiers between the working windows, so as to avoid a sudden change in the offset value for each channel other than the reference channel.
  15. Method according to one of claims 10 to 12, characterised in that, with one of the channels of the multi-channel stream being defined as a reference channel, an amplification value being associated with each filter received for the channels other than the reference channel, the method further comprises:
    - a step of amplification of the signal corresponding to each channel other than the reference channel allowing a difference in gain to be generated which is similar to the difference in gain between each channel and the reference channel in the original multi-channel audio stream.
  16. Device for encoding a multi-channel audio stream, comprising at least:
    - means for obtaining a compound signal obtained by compounding signals corresponding to each channel of the multi-channel audio stream;
    - means for obtaining a limited-frequency compound signal, the reduction in the spectrum of the original compound signal being obtained by suppressing the high frequencies;
    characterised in that it further comprises
    - means for generating a time filter for each channel, making it possible to find a signal that is close in spectrum to the original signal of the corresponding channel when it is applied to the signal obtained by broadening the spectrum of the limited signal.
  17. Device for decoding a multi-channel audio stream, comprising at least the following means:
    - means for receiving a transmitted signal;
    characterised in that it further comprises:
    - means for receiving a time filter relating to the signal received for each channel of the multi-channel audio stream;
    - means for obtaining a decoded signal by decoding the signal received;
    - means for obtaining an extended signal by broadening the spectrum of the decoded signal;
    - means for obtaining a reconstructed signal by convoluting the extended signal with the time filter received for each channel of the multi-channel audio stream.
  18. Signal comprising a limited-frequency audio signal representing a limited-frequency version of an original audio signal resulting from the composition of the different channels of a multi-channel audio stream, characterised in that it further comprises data for the generation of a time filter for each channel, enabling the reconstruction of a signal close to the original signal of each channel when it is applied to an extended-frequency version of the limited-frequency audio signal contained in the signal.
EP07866272A 2006-12-28 2007-12-28 Methods and devices for audio signals encoding and decoding, encoded audio signal Active EP2126905B1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
FR0611481A FR2911031B1 (en) 2006-12-28 2006-12-28 AUDIO CODING METHOD AND DEVICE
FR0708067A FR2911020B1 (en) 2006-12-28 2007-11-16 AUDIO CODING METHOD AND DEVICE
PCT/EP2007/011442 WO2008080609A1 (en) 2006-12-28 2007-12-28 Audio encoding method and device

Publications (2)

Publication Number Publication Date
EP2126905A1 EP2126905A1 (en) 2009-12-02
EP2126905B1 true EP2126905B1 (en) 2012-05-30

Family

ID=39083245

Family Applications (1)

Application Number Title Priority Date Filing Date
EP07866272A Active EP2126905B1 (en) 2006-12-28 2007-12-28 Methods and devices for audio signals encoding and decoding, encoded audio signal

Country Status (5)

Country Link
US (1) US8340305B2 (en)
EP (1) EP2126905B1 (en)
JP (1) JP5491194B2 (en)
FR (1) FR2911020B1 (en)
WO (1) WO2008080609A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2911031B1 (en) * 2006-12-28 2009-04-10 Actimagine Soc Par Actions Sim AUDIO CODING METHOD AND DEVICE
US8666752B2 (en) * 2009-03-18 2014-03-04 Samsung Electronics Co., Ltd. Apparatus and method for encoding and decoding multi-channel signal
CN112954581B (en) * 2021-02-04 2022-07-01 广州橙行智动汽车科技有限公司 Audio playing method, system and device

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS62234435A (en) * 1986-04-04 1987-10-14 Kokusai Denshin Denwa Co Ltd <Kdd> Voice coding system
US5956674A (en) * 1995-12-01 1999-09-21 Digital Theater Systems, Inc. Multi-channel predictive subband audio coder using psychoacoustic adaptive bit allocation in frequency, time and over the multiple channels
US6226616B1 (en) * 1999-06-21 2001-05-01 Digital Theater Systems, Inc. Sound quality of established low bit-rate audio coding systems without loss of decoder compatibility
US6674862B1 (en) * 1999-12-03 2004-01-06 Gilbert Magilen Method and apparatus for testing hearing and fitting hearing aids
US7742927B2 (en) * 2000-04-18 2010-06-22 France Telecom Spectral enhancing method and device
SE0004163D0 (en) * 2000-11-14 2000-11-14 Coding Technologies Sweden Ab Enhancing perceptual performance or high frequency reconstruction coding methods by adaptive filtering
JP3957589B2 (en) * 2001-08-23 2007-08-15 松下電器産業株式会社 Audio processing device
PL1621047T3 (en) * 2003-04-17 2007-09-28 Koninl Philips Electronics Nv Audio signal generation
US7725324B2 (en) * 2003-12-19 2010-05-25 Telefonaktiebolaget Lm Ericsson (Publ) Constrained filter encoding of polyphonic signals
CA2457988A1 (en) * 2004-02-18 2005-08-18 Voiceage Corporation Methods and devices for audio compression based on acelp/tcx coding and multi-rate lattice vector quantization
FI119533B (en) * 2004-04-15 2008-12-15 Nokia Corp Coding of audio signals
JP4832305B2 (en) * 2004-08-31 2011-12-07 パナソニック株式会社 Stereo signal generating apparatus and stereo signal generating method
EP1798724B1 (en) * 2004-11-05 2014-06-18 Panasonic Corporation Encoder, decoder, encoding method, and decoding method
US7983904B2 (en) * 2004-11-05 2011-07-19 Panasonic Corporation Scalable decoding apparatus and scalable encoding apparatus
EP1818911B1 (en) * 2004-12-27 2012-02-08 Panasonic Corporation Sound coding device and sound coding method
US7573912B2 (en) * 2005-02-22 2009-08-11 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschunng E.V. Near-transparent or transparent multi-channel encoder/decoder scheme
NZ562190A (en) * 2005-04-01 2010-06-25 Qualcomm Inc Systems, methods, and apparatus for highband burst suppression
KR100818268B1 (en) * 2005-04-14 2008-04-02 삼성전자주식회사 Apparatus and method for audio encoding/decoding with scalability
US7653533B2 (en) * 2005-10-24 2010-01-26 Lg Electronics Inc. Removing time delays in signal paths
PL1999997T3 (en) * 2006-03-28 2011-09-30 Fraunhofer Ges Forschung Enhanced method for signal shaping in multi-channel audio reconstruction

Also Published As

Publication number Publication date
JP5491194B2 (en) 2014-05-14
EP2126905A1 (en) 2009-12-02
FR2911020B1 (en) 2009-05-01
JP2010522346A (en) 2010-07-01
WO2008080609A1 (en) 2008-07-10
FR2911020A1 (en) 2008-07-04
US20100046760A1 (en) 2010-02-25
US8340305B2 (en) 2012-12-25

Similar Documents

Publication Publication Date Title
EP2374123B1 (en) Improved encoding of multichannel digital audio signals
EP1905010B1 (en) Hierarchical audio encoding/decoding
EP1600042B1 (en) Method for the treatment of compressed sound data for spatialization
EP2104936B1 (en) Low-delay transform coding using weighting windows
EP2002424B1 (en) Device and method for scalable encoding of a multichannel audio signal based on a principal component analysis
EP1794748B1 (en) Data processing method by passage between different sub-band domains
EP1599868B1 (en) Method and device for the spectral reconstruction of an audio signal
EP2489039B1 (en) Optimized low-throughput parametric coding/decoding
EP2304721B1 (en) Spatial synthesis of multichannel audio signals
EP2042001B1 (en) Binaural spatialization of compression-encoded sound data
EP2005420A1 (en) Device and method for encoding by principal component analysis a multichannel audio signal
EP3391370A1 (en) Adaptive channel-reduction processing for encoding a multi-channel audio signal
EP2517199A2 (en) Improved method for encoding/decoding a stereo digital stream and associated encoding/decoding device
EP1037196B1 (en) Method for coding, decoding and transcoding an audio signal
EP2126905B1 (en) Methods and devices for audio signals encoding and decoding, encoded audio signal
EP1275109B1 (en) Spectral enhancing method and device
EP2126904B1 (en) Audio encoding method and device
EP1362344A1 (en) Multichannel signal spectral reconstruction method and device
WO2023232823A1 (en) Title: spatialized audio encoding with configuration of a decorrelation processing operation
WO2009081002A1 (en) Processing of a 3d audio stream as a function of a level of presence of spatial components

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20090609

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: MOBICLIP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RTI1 Title (correction)

Free format text: METHODS AND DEVICES FOR AUDIO SIGNALS ENCODING AND DECODING, ENCODED AUDIO SIGNAL

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC MT NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 560385

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120615

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602007023078

Country of ref document: DE

Effective date: 20120802

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20120530

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

Effective date: 20120530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120930

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 560385

Country of ref document: AT

Kind code of ref document: T

Effective date: 20120530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120831

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20121001

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120910

26N No opposition filed

Effective date: 20130301

BERE Be: lapsed

Owner name: MOBICLIP

Effective date: 20121231

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602007023078

Country of ref document: DE

Effective date: 20130301

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121231

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120830

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121228

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121231

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121231

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20120530

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20121228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071228

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 9

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602007023078

Country of ref document: DE

Representative=s name: WEICKMANN & WEICKMANN PATENTANWAELTE - RECHTSA, DE

Ref country code: DE

Ref legal event code: R082

Ref document number: 602007023078

Country of ref document: DE

Representative=s name: WEICKMANN & WEICKMANN PATENT- UND RECHTSANWAEL, DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 10

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20231113

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20231227

Year of fee payment: 17

Ref country code: DE

Payment date: 20231122

Year of fee payment: 17