EP2168121B1 - Quantification after linear conversion combining audio signals of a sound scene, and related encoder - Google Patents

Quantification after linear conversion combining audio signals of a sound scene, and related encoder Download PDF

Info

Publication number
EP2168121B1
EP2168121B1 EP08806144.5A EP08806144A EP2168121B1 EP 2168121 B1 EP2168121 B1 EP 2168121B1 EP 08806144 A EP08806144 A EP 08806144A EP 2168121 B1 EP2168121 B1 EP 2168121B1
Authority
EP
European Patent Office
Prior art keywords
quantization
function
components
module
audio signals
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP08806144.5A
Other languages
German (de)
French (fr)
Other versions
EP2168121A1 (en
Inventor
Adil Mouhssine
Abdellatif Benjelloun Touimi
Pierre Duhamel
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
Orange SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Orange SA filed Critical Orange SA
Publication of EP2168121A1 publication Critical patent/EP2168121A1/en
Application granted granted Critical
Publication of EP2168121B1 publication Critical patent/EP2168121B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/032Quantisation or dequantisation of spectral components
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S3/00Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S3/00Systems employing more than two channels, e.g. quadraphonic
    • H04S3/02Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S2420/00Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]

Definitions

  • the present invention relates to audio signal coding devices, intended in particular to take place in applications for transmission or storage of digitized and compressed audio signals.
  • the invention relates more specifically to the quantization modules included in these audio coding devices.
  • a 3D sound scene also called spatialized sound, comprises a plurality of audio channels each corresponding to monophonic signals.
  • a signal coding technique for a sound stage used in the "MPEG Audio Surround” encoder includes the extraction and coding of spatial parameters from the set of monophonic audio signals on the different channels. These signals are then mixed to obtain a monophonic or stereophonic signal, which is then compressed by a conventional mono or stereo encoder (for example of the MPEG-4 AAC, HE-AAC type, etc.). At the level of the decoder, the synthesis of the rendered 3D sound scene is made from the spatial parameters and the decoded mono or stereo signal.
  • the coding of the multichannel signals in certain cases requires the introduction of a transformation (KLT, Ambiophonic, DCT, etc.) making it possible to better take into account the interactions that may exist between the different signals of the sound scene to be encoded.
  • KLT KLT, Ambiophonic, DCT, etc.
  • the object of the invention is to find an improvement for the quantization in a multichannel system. This object is solved by the independent claims.
  • the invention proposes a method for quantifying components, at least some of these components being each determined according to a plurality of audio signals of a scene. sound and calculable by applying a linear transformation on said audio signals.
  • a quantization function is determined to be applied to said components in a given frequency band by testing a condition relating to at least one audio signal and depending at least on a comparison made between a psychoacoustic masking threshold relative to the audio signal. in the given frequency band, and a value determined according to the inverse linear transformation and quantization errors of the components by said function on the given frequency band.
  • Such a method therefore makes it possible to determine a quantization function which makes it possible to mask, in the playback listening field, the noise introduced with respect to the audio signal of the initial sound scene.
  • the sound scene restored after the coding and decoding operations thus presents a better audio quality.
  • the introduction of a multichannel transform transforms the real signals into a new domain different from the listening domain.
  • the quantification of the components resulting from this transform according to the methods of the state of the art, based on a perceptual criterion (ie respecting the masking threshold on the latter), does not guarantee a minimal distortion on the real signals restored in the listening domain.
  • the calculation of the quantization function according to the invention makes it possible to guarantee that the quantization noises induced on the real signals by the quantization of the transformed components are minimal in the sense of a perceptual criterion. The condition of a maximum improvement of the perceptual quality of the signals in the listening domain is then verified.
  • the condition is relative to several audio signals and depends on several comparisons, each comparison being made between a psychoacoustic masking threshold relative to a respective audio signal in the given frequency band, and a value determined according to the inverse linear transformation and quantization errors of the components by said function.
  • This arrangement further enhances the audio quality of the restored sound stage.
  • the determination of the quantization function is repeated when updating the values of the components to be quantized. This arrangement also makes it possible to increase the audio quality of the restored sound scene, by adapting the quantization over time according to the characteristics of the signals.
  • the condition relating to an audio signal is tested at least by comparing the psychoacoustic masking threshold relative to the audio signal and an element representing the value.
  • a quantization function is determined to apply components in the given frequency band using an iterative process generating at each iteration a parameter of the candidate quantization function satisfying the condition and associated with a corresponding flow rate, the iteration being stopped when the flow rate is below a given threshold.
  • Such an arrangement thus makes it possible to simply determine a quantization function based on the determined parameters, allowing the noise to be masked in the playback listening domain while reducing the coding bit rate below a given threshold.
  • the linear transformation is an ambiophonic transformation.
  • the linear transformation is an ambiophonic transformation (called “ambisonic").
  • ambisonic ambiophonic transformation
  • This arrangement makes it possible on the one hand to reduce the number of data to be transmitted since, in general, the N signals can be very satisfactorily described by a reduced number of ambiophonic components (for example, a number equal to 3 or 5). , which is smaller than N.
  • This arrangement also allows coding adaptability to any type of sound rendering system, since it is sufficient at the decoder level to apply an inverse surround transform of size Q'x (2p '+ 1). , (where Q 'is equal to the number of loudspeakers of the sound rendering system used at the output of the decoder and 2p' + 1 the number of received surround components), to determine the signals to be supplied to the sound rendering system.
  • the invention can be implemented with any linear transformation, for example the DCT or the KLT (in English "Karhunen Loeve Transform") transform which corresponds to a decomposition on principal components in a space representing the statistics of the signals and allows to distinguish the most energetic components from the least energy components.
  • any linear transformation for example the DCT or the KLT (in English "Karhunen Loeve Transform") transform which corresponds to a decomposition on principal components in a space representing the statistics of the signals and allows to distinguish the most energetic components from the least energy components.
  • the invention proposes a quantization module adapted to quantify components, at least some of these components being each determined according to a plurality of audio signals of a sound scene and calculable by application of a transformation. linearly on said audio signals, said quantization module being adapted to implement the steps of a method according to the first aspect of the invention.
  • the invention proposes a computer program to be installed in a quantization module, said program comprising instructions for implementing the steps of a method according to the first aspect of the invention during an execution. of the program by means of processing said module.
  • the invention proposes coding data, determined following the implementation of a quantization method according to the first aspect of the invention.
  • the figure 1 represents an audio coder 1 in one embodiment of the invention. It relies on the technology of perceptual audio coders, for example MPEG-4 AAC type.
  • the encoder 1 comprises a time / frequency transformation module 2, a linear transformation module 3, a quantization module 4, a Huffman entropy coding module 5 and a masking curve calculation module 6, for transmission.
  • a bit stream ⁇ representing the signals supplied at the input of the encoder 1.
  • a 3D sound scene comprises N channels on each a respective audio signal S 1 , ..., S N is delivered.
  • the figure 2 represents an audio decoder 100 in one embodiment of the invention.
  • the decoder 100 comprises a bit sequence reading module 101, an inverse quantization module 102, an inverse linear transformation module 103, a frequency / time transformation module 104.
  • the decoder 100 is adapted to receive as input the bitstream ⁇ transmitted by the encoder 1 and to output Q 'signals S' 1 , ..., S ' Q , for supplying the Q' speakers H1, H2 ..., HQ 'of a sound rendering system 105.
  • the time / frequency conversion module 2 of the encoder 1 receives as input the N signals S 1 ,... S N of the 3D sound scene to be encoded, in the form of successive blocks.
  • Each block m received has N time frames each indicating different values taken over time by a respective signal.
  • the time / frequency transformation module 2 On each time frame of each of the signals, the time / frequency transformation module 2 performs a time / frequency transformation, in this case a modified discrete cosine transform (MDCT).
  • MDCT modified discrete cosine transform
  • the coding of multichannel signals comprises in the case considered a linear transformation, making it possible to take into account the interactions between the different audio signals to be coded, before the monophonic coding, by the quantization module 4, of the components resulting from the linear transformation.
  • the linear transformation module 3 is adapted to perform a linear transformation of the coefficients of the spectral representations ( X i ) 1 i i N N provided. In one embodiment, it is adapted to perform spatial transformation. It then determines the spatial components of the signals ( X i ) 1 i i ⁇ N , in the frequency domain, resulting from the projection on a spatial reference system depending on the order of the transformation. The order of a spatial transformation is related to the angular frequency according to which it "scans" the sound field.
  • Each of the ambiophonic components is therefore determined according to several signals ( S i ) 1 i i ⁇ N.
  • the masking curve calculation module 6 is adapted to determine the spectral masking curve of each frame of a signal Si considered individually in the block m, using its spectral representation Xi and a psychoacoustic model.
  • the masking curve calculation module 6 thus calculates a masking threshold M T m s , i , relating to the frame of each signal ( S i ) 1 i i ⁇ N in the block m, for each frequency band s considered during the quantization.
  • Each frequency band s is part of a set of frequency bands including for example the bands as normalized for the MPEG-4 AAC encoder.
  • Masking thresholds M T m s , i for each signal S i and each frequency band s are delivered to the quantization module 4.
  • the quantization module 4 is adapted to quantize the components ( Y j ) 1 j j ⁇ r that are input to it, so as to reduce the bit rate required for transmission. Respective quantization functions are determined by the quantization module 4 on each frequency band s.
  • the quantization module 4 quantizes each spectral coefficient Y j , t 1 ⁇ i ⁇ r 0 ⁇ t ⁇ M - 1 such that the frequency F t is an element of the frequency band s. It thus determines a quantization index i (k) for each spectral coefficient Y j , t 1 ⁇ i ⁇ r 0 ⁇ t ⁇ M - 1 such that the frequency F t is an element of the frequency band s.
  • k takes the values of the set ⁇ k min, s , k min + 1 , s , ... k max , s ⁇ , and ( k max , s - k min + 1 , s + 1) is equal to the number of spectral coefficients to be quantified in the band s for all the surround components.
  • Arr is a rounding function that delivers an integer value.
  • Arr (x) is for example the function providing the integer closest to the variable x, or the function "integer part" of the variable x, etc.
  • the quantization module 4 is adapted to determine a quantization function to be applied on a frequency band that verifies that the masking threshold M T m s , i of each signal S i in the listening domain, with 1 ⁇ i ⁇ N, is greater than the power of the error made, on an audio signal restored in the listening domain corresponding to the channel i (and not in the linear transformation domain), by the quantization errors made to the ambiophonic components.
  • the quantization module 4 is therefore adapted to determine, during the processing of a block m of signals, the quantization function defined using the scale parameters.
  • B j m s 1 ⁇ j ⁇ r relating to each band s, such that, for all i, 1 ⁇ i ⁇ N, the error introduced on the signal S i in the band s by the quantification of the surround components is less than the masking threshold M T m s , i of the signal S i on the band s.
  • a problem to be solved by the quantization module 4 is therefore to determine, on each band s, the set of scaling coefficients.
  • B j m s 1 ⁇ j ⁇ r verifying the following formula (1): B j m / P e m s , i ⁇ M T m s , i , 1 ⁇ i ⁇ NOT 1 ⁇ j ⁇ r or P e m s , i is the error power introduced on the signal S i following the quantization errors introduced by the quantization, defined by the scaling coefficients B j m s 1 ⁇ j ⁇ r , ambiophonic components.
  • B j (s) is a parameter characterizing the quantization function s in the band on the j-th component.
  • the choice of B j ( s ) determines in a bijective manner the quantization function used.
  • This arrangement has the effect that the noise brought into the listening domain by the quantization on the components resulting from the linear transformation remains masked by the signal in the listening domain, which contributes to a better quality of the signals restored in the listening domain.
  • the problem indicated above by the formula (1) is translated as the following formula (2): B j m / Probability P e m s , i ⁇ M T m s , i ⁇ ⁇ , 1 ⁇ i ⁇ NOT 1 ⁇ j ⁇ r , where ⁇ is a fixed rate of compliance with the masking threshold.
  • the probability is calculated for the frame relating to the signal S i of the block m considered and on all the frequency bands s.
  • Arr (x) is the function providing the integer closest to the variable x
  • e R is equal to 0.5
  • Arr (x) is the function "integer part" of the variable x, e R is equal to 1.
  • This last equation represents a sufficient condition for the noise corresponding to the channel i to be masked at the output in the listening domain.
  • the quantization module 4 is adapted to determine using the latter equation, for a block m of current frames, scaling coefficients. B j m s 1 ⁇ j ⁇ r ensuring that noise in the listening domain is hidden.
  • the quantization module 4 is adapted to determine, for a block m of current frames, scaling coefficients.
  • B j m s 1 ⁇ j ⁇ r guaranteeing that the noise in the listening domain is masked and furthermore making it possible to respect a flow constraint.
  • D j m ⁇ s D j m s or D j m s is the bit rate assigned to the surround component Y j in the band s.
  • the resolution of this constrained optimization problem is for example carried out using the Lagrangian method.
  • the Lagrangian function is written in the following form:
  • the iterative relative gradient method (see in particular the Derrien document) is used to solve this system.
  • the vector m is chosen equal to: 1 M T m s , 1 ⁇ 1 M T m s , NOT
  • the quantization module 4 is adapted to implement the steps of the method described below with reference to FIG. figure 3 on each quantization band s when quantizing a block m of signals ( S i ) 1 i i ⁇ N.
  • the method is based on an iterative algorithm comprising instructions for implementing the steps described below during the execution of the algorithm on the calculation means of the quantization module 4.
  • the steps of the iterative loop for a (k + 1) th iteration, with k integer greater than or equal to 0, are as follows.
  • the value of the function F is calculated on the band s, representing the corresponding bit rate for the band s:
  • a step e / the calculated value F ( s ) is compared with the given threshold D.
  • the value of the Lagrange vector ⁇ for the (k + 1) th iteration is calculated in a step f / using the equation (4 ) indicated above and the Lagrange vector calculated during the k th iteration.
  • a step g / the index k is incremented by one unit and the steps b /, c /, d / and e / are repeated.
  • the quantization function thus determined for the respective s-bands and respective surround components is then applied to the spectral coefficients of the surround components.
  • the quantization indices as well as definition elements of the quantization function are provided to the Huffman coding module.
  • the coding data delivered by the Huffman coding module 5 is then transmitted as a bit stream ⁇ to the decoder 100.
  • the bit sequence reading module 101 is adapted to extract coding data present in the stream ⁇ received by the decoder and to deduce, in each band s, quantization indices i (k) and scale coefficients.
  • the inverse quantization module 102 is adapted to determine the spectral coefficients, relative to the band s, of the corresponding ambiophonic components as a function of the quantization indices i (k) and the scale coefficients.
  • Ambiophonic decoding is then applied to the decoded surround components, so as to determine the signals S ' 1 , S' 2 ,..., S ' Q , for the speakers H1, H2 ..., HQ. .
  • the quantization noise at the output of the decoder 100 is a constant which depends only on the transform R used and the quantization module 4 because the psychoacoustic data used during the coding do not take into account the processing performed during the rendering by the decoder. Indeed, the psychoacoustic model does not take into account the acoustic interactions between the different signals, but calculates the masking curve of a signal as if it were the only one listened to. The error calculated on this signal therefore remains constant and masked for any surround decoding matrix used. This surround decoding matrix will simply change the distribution of the error on the different speakers output.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Human Computer Interaction (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Algebra (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Description

La présente invention concerne les dispositifs de codage de signaux audio, destinés notamment à prendre place dans des applications de transmission ou de stockage de signaux audio numérisés et compressés.The present invention relates to audio signal coding devices, intended in particular to take place in applications for transmission or storage of digitized and compressed audio signals.

L'invention est relative plus précisément aux modules de quantification compris dans ces dispositifs de codage audio.The invention relates more specifically to the quantization modules included in these audio coding devices.

L'invention concerne plus particulièrement le codage de scène sonore 3D. Une scène sonore 3D, encore appelée son spatialisé, comprend une pluralité de canaux audio correspondant chacun à des signaux monophoniques.The invention more particularly relates to 3D sound stage coding. A 3D sound scene, also called spatialized sound, comprises a plurality of audio channels each corresponding to monophonic signals.

Une technique de codage de signaux d'une scène sonore utilisée dans le codeur « MPEG Audio Surround » (cf. « Text of ISO/IEC FDIS 23003-1, MPEG Surround », ISO/IEC JTC1 / SC29 / WG11 N8324, July 2006, Klagenfurt, Austria), comprend l'extraction et le codage de paramètres spatiaux à partir de l'ensemble des signaux audio monophoniques sur les différents canaux. Ces signaux sont ensuite mélangés pour obtenir un signal monophonique ou stéréophonique, qui est alors comprimé par un codeur mono ou stéréo classique (par exemple de type MPEG-4 AAC, HE-AAC, etc). Au niveau du décodeur, la synthèse de la scène sonore 3D restituée se fait à partir des paramètres spatiaux et du signal mono ou stéréo décodé.A signal coding technique for a sound stage used in the "MPEG Audio Surround" encoder (see "Text of ISO / IEC FDIS 23003-1, MPEG Surround", ISO / IEC JTC1 / SC29 / WG11 N8324, July 2006 , Klagenfurt, Austria), includes the extraction and coding of spatial parameters from the set of monophonic audio signals on the different channels. These signals are then mixed to obtain a monophonic or stereophonic signal, which is then compressed by a conventional mono or stereo encoder (for example of the MPEG-4 AAC, HE-AAC type, etc.). At the level of the decoder, the synthesis of the rendered 3D sound scene is made from the spatial parameters and the decoded mono or stereo signal.

Le codage des signaux multicanaux nécessite dans certains cas l'introduction d'une transformation (KLT, Ambiophonique, DCT...) permettant de mieux prendre en compte les interactions qui peuvent exister entre les différents signaux de la scène sonore à coder.The coding of the multichannel signals in certain cases requires the introduction of a transformation (KLT, Ambiophonic, DCT, etc.) making it possible to better take into account the interactions that may exist between the different signals of the sound scene to be encoded.

Il est toujours besoin d'accroitre la qualité audio des scènes sonores restituées après une opération de codage et décodage.It is always necessary to increase the audio quality of the sound scenes restored after a coding and decoding operation.

DERRIEN O & DUHAMEL P: "Une approche statistique pour l'optimisation du MPEG-2/4 AAC (Advanced Audio Coder) en mode stéréophonique matricé (MS stéréo)",ACTES DE COLLOQUES DU GROUPE D'ETUDES DU TRAITEMENT DU SIGNAL ET DES IMAGES (GRETSI), 2003, pages 1-4 , divulgue un procédé de quantification de composantes dans un système MS stéréo. DERRIEN O & DUHAMEL P: "A statistical approach for the optimization of MPEG-2/4 AAC (Advanced Audio Coder) in stereo stereophonic mode (stereo MS)", ACTS OF SYMPOSIUM OF THE GROUP OF STUDIES OF SIGNAL PROCESSING AND IMAGES (GRETSI), 2003, pages 1-4 discloses a method of quantizing components in a stereo MS system.

L'objet de l'invention est de trouver une amélioration pour la quantification dans un système multicanaux. Cet objet est résolu par les revendication indépendantes. Suivant un premier aspect, l'invention propose un procédé de quantification de composantes, certaines au moins de ces composantes étant déterminées chacune en fonction d'une pluralité de signaux audio d'une scène sonore et calculables par application d'une transformation linéaire sur lesdits signaux audio.The object of the invention is to find an improvement for the quantization in a multichannel system. This object is solved by the independent claims. According to a first aspect, the invention proposes a method for quantifying components, at least some of these components being each determined according to a plurality of audio signals of a scene. sound and calculable by applying a linear transformation on said audio signals.

Selon le procédé, on détermine une fonction de quantification à appliquer audites composantes dans une bande de fréquence donnée en testant une condition relative à au moins un signal audio et dépendant au moins d'une comparaison effectuée entre un seuil de masquage psychoacoustique relatif au signal audio dans la bande de fréquence donnée, et une valeur déterminée en fonction de la transformation linéaire inverse et d'erreurs de quantification des composantes par ladite fonction sur la bande de fréquence donnée.According to the method, a quantization function is determined to be applied to said components in a given frequency band by testing a condition relating to at least one audio signal and depending at least on a comparison made between a psychoacoustic masking threshold relative to the audio signal. in the given frequency band, and a value determined according to the inverse linear transformation and quantization errors of the components by said function on the given frequency band.

Un tel procédé permet donc de déterminer une fonction de quantification qui permette de masquer, dans le domaine d'écoute de restitution, le bruit introduit par rapport au signal audio de la scène sonore initiale. La scène sonore restituée après les opérations de codage et décodage présente donc une meilleure qualité audio.Such a method therefore makes it possible to determine a quantization function which makes it possible to mask, in the playback listening field, the noise introduced with respect to the audio signal of the initial sound scene. The sound scene restored after the coding and decoding operations thus presents a better audio quality.

En effet, l'introduction d'une transformée multicanal (par exemple de type ambiophonique) transforme les signaux réels dans un nouveau domaine différent du domaine d'écoute. La quantification des composantes résultant de cette transformée selon les méthodes de l'état de l'art, basées sur un critère perceptuel (i.e. respectant le seuil de masquage sur ces derniers), ne garantit pas une distorsion minimale sur les signaux réels restitués dans le domaine d'écoute. En effet, le calcul de la fonction de quantification selon l'invention permet de garantir que les bruits de quantification induits sur les signaux réels par la quantification des composantes transformées sont minimaux au sens d'un critère perceptuel. La condition d'une amélioration maximale de la qualité perceptuelle des signaux dans le domaine d'écoute est alors vérifiée.Indeed, the introduction of a multichannel transform (for example of the ambiophonic type) transforms the real signals into a new domain different from the listening domain. The quantification of the components resulting from this transform according to the methods of the state of the art, based on a perceptual criterion (ie respecting the masking threshold on the latter), does not guarantee a minimal distortion on the real signals restored in the listening domain. Indeed, the calculation of the quantization function according to the invention makes it possible to guarantee that the quantization noises induced on the real signals by the quantization of the transformed components are minimal in the sense of a perceptual criterion. The condition of a maximum improvement of the perceptual quality of the signals in the listening domain is then verified.

Dans un mode de réalisation la condition est relative à plusieurs signaux audio et dépend de plusieurs comparaisons, chaque comparaison étant effectuée entre un seuil de masquage psychoacoustique relatif à un signal audio respectif dans la bande de fréquence donnée, et une valeur déterminée en fonction de la transformation linéaire inverse et d'erreurs de quantification des composantes par ladite fonction.In one embodiment, the condition is relative to several audio signals and depends on several comparisons, each comparison being made between a psychoacoustic masking threshold relative to a respective audio signal in the given frequency band, and a value determined according to the inverse linear transformation and quantization errors of the components by said function.

Cette disposition accroît encore la qualité audio de la scène sonore restituée.This arrangement further enhances the audio quality of the restored sound stage.

Dans un mode de réalisation, la détermination de la fonction de quantification est réitérée lors de l'actualisation des valeurs des composantes à quantifier. Cette disposition permet également d'accroître la qualité audio de la scène sonore restituée, en adaptant la quantification dans le temps en fonction des caractéristiques des signaux.In one embodiment, the determination of the quantization function is repeated when updating the values of the components to be quantized. This arrangement also makes it possible to increase the audio quality of the restored sound scene, by adapting the quantization over time according to the characteristics of the signals.

Dans un mode de réalisation, on teste la condition relative à un signal audio au moins en comparant le seuil de masquage psychoacoustique relatif au signal audio et un élément représentant la valeur j = 1 r h i , j 2 B j s 3 2 μ 1 2 , j s ,

Figure imgb0001
où s est la bande de fréquence donnée, r est le nombre de composantes, hi,j est le coefficient de la transformée linéaire inverse relatif au signal audio et à la jème composante avec j=1 à r, Bj (s) représente un paramètre de la fonction de quantification dans la bande s relative à la jème composante et μ 1 2 , j s
Figure imgb0002
est l'espérance mathématique dans la bande s de la racine carrée de la jème composante.In one embodiment, the condition relating to an audio signal is tested at least by comparing the psychoacoustic masking threshold relative to the audio signal and an element representing the value. Σ j = 1 r h i , j 2 B j s 3 2 μ 1 2 , j s ,
Figure imgb0001
where s is the given frequency band, r is the number of components, h i, j is the coefficient of the inverse linear transform relative to the audio signal and the j th component with j = 1 to r, B j ( s ) represents a parameter of the quantization function s in the band on the j th component and μ 1 2 , j s
Figure imgb0002
is the expected value in the strip s of the square root of the jth component.

Dans un mode de réalisation, on détermine une fonction de quantification à appliquer audites composantes dans la bande de fréquence donnée à l'aide d'un processus itératif générant à chaque itération un paramètre de la fonction de quantification candidat vérifiant la condition et associé à un débit correspondant, l'itération étant stoppée lorsque le débit est inférieur à un seuil donné.In one embodiment, a quantization function is determined to apply components in the given frequency band using an iterative process generating at each iteration a parameter of the candidate quantization function satisfying the condition and associated with a corresponding flow rate, the iteration being stopped when the flow rate is below a given threshold.

Une telle disposition permet ainsi de déterminer simplement une fonction de quantification à partir des paramètres déterminés, permettant le masquage du bruit dans le domaine d'écoute de restitution tout en réduisant le débit de codage en dessous d'un seuil donné.Such an arrangement thus makes it possible to simply determine a quantization function based on the determined parameters, allowing the noise to be masked in the playback listening domain while reducing the coding bit rate below a given threshold.

Dans un mode de réalisation, la transformation linéaire est une transformation ambiophonique.In one embodiment, the linear transformation is an ambiophonic transformation.

Dans un mode de réalisation particulier, la transformation linéaire est une transformation ambiophonique (appelée en anglais « ambisonic »). Cette disposition permet d'une part de réduire le nombre de données à transmettre puisque, en général, les N signaux peuvent être décrits d'une manière très satisfaisante par un nombre de composantes ambiophoniques réduit (par exemple, un nombre égal à 3 ou 5), inférieur à N. Cette disposition permet en outre une adaptabilité du codage à tout type de système de rendu sonore, puisqu'il suffit au niveau du décodeur, d'appliquer une transformée ambiophonique inverse de taille Q'x(2p'+1), (où Q' est égal au nombre de haut-parleurs du système de rendu sonore utilisé en sortie du décodeur et 2p'+1 le nombre de composantes ambiophoniques reçues), pour déterminer les signaux à fournir au système de rendu sonore.In a particular embodiment, the linear transformation is an ambiophonic transformation (called "ambisonic"). This This arrangement makes it possible on the one hand to reduce the number of data to be transmitted since, in general, the N signals can be very satisfactorily described by a reduced number of ambiophonic components (for example, a number equal to 3 or 5). , which is smaller than N. This arrangement also allows coding adaptability to any type of sound rendering system, since it is sufficient at the decoder level to apply an inverse surround transform of size Q'x (2p '+ 1). , (where Q 'is equal to the number of loudspeakers of the sound rendering system used at the output of the decoder and 2p' + 1 the number of received surround components), to determine the signals to be supplied to the sound rendering system.

L'invention peut être mise en oeuvre avec toute transformation linéaire, par exemple la DCT ou encore la transformée KLT (en anglais « Karhunen Loeve Transform ») qui correspond à une décomposition sur des composantes principales dans un espace représentant les statistiques des signaux et permet de distinguer les composantes les plus énergétiques des composantes les moins énergétiques.The invention can be implemented with any linear transformation, for example the DCT or the KLT (in English "Karhunen Loeve Transform") transform which corresponds to a decomposition on principal components in a space representing the statistics of the signals and allows to distinguish the most energetic components from the least energy components.

Suivant un deuxième aspect, l'invention propose un module de quantification adapté pour quantifier des composantes, certaines au moins de ces composantes étant déterminées chacune en fonction d'une pluralité de signaux audio d'une scène sonore et calculables par application d'une transformation linéaire sur lesdits signaux audio, ledit module de quantification étant adapté pour mettre en oeuvre les étapes d'un procédé suivant le premier aspect de l'invention.According to a second aspect, the invention proposes a quantization module adapted to quantify components, at least some of these components being each determined according to a plurality of audio signals of a sound scene and calculable by application of a transformation. linearly on said audio signals, said quantization module being adapted to implement the steps of a method according to the first aspect of the invention.

Suivant un troisième aspect, l'invention propose un codeur audio adapté pour coder une scène audio comprenant plusieurs signaux respectifs en un flux binaire de sortie, comprenant :

  • un module de transformation adapté pour calculer par application d'une transformation linéaire sur lesdits signaux audio, des composantes dont certaines au moins sont déterminées chacune en fonction d'une pluralité des signaux audio d'une scène sonore ; et
  • un module de quantification suivant le deuxième aspect de l'invention adapté pour déterminer au moins une fonction de quantification sur au moins une bande de fréquence donnée et pour quantifier les composantes sur la bande de fréquence donnée en fonction d'au moins la fonction de quantification déterminée ;
  • le codeur audio étant adapté pour constituer un flux binaire en fonction au moins de données de quantification délivrées par le module de quantification.
According to a third aspect, the invention provides an audio coder adapted to encode an audio scene comprising a plurality of respective signals into an output bit stream, comprising:
  • a transformation module adapted to calculate by applying a linear transformation on said audio signals, components at least some of which are determined each according to a plurality of audio signals of a sound scene; and
  • a quantization module according to the second aspect of the invention adapted to determine at least one quantization function over at least a given frequency band and for quantizing the components on the given frequency band as a function of at least the determined quantization function;
  • the audio coder being adapted to constitute a bit stream according to at least quantization data delivered by the quantization module.

Suivant un quatrième aspect, l'invention propose un programme d'ordinateur à installer dans un module de quantification, ledit programme comprenant des instructions pour mettre en oeuvre les étapes d'un procédé suivant le premier aspect de l'invention lors d'une exécution du programme par des moyens de traitement dudit module.According to a fourth aspect, the invention proposes a computer program to be installed in a quantization module, said program comprising instructions for implementing the steps of a method according to the first aspect of the invention during an execution. of the program by means of processing said module.

Suivant un cinquième aspect, l'invention propose des données de codage, déterminées suite à la mise en oeuvre d'un procédé de quantification suivant le premier aspect de l'invention.According to a fifth aspect, the invention proposes coding data, determined following the implementation of a quantization method according to the first aspect of the invention.

D'autres caractéristiques et avantages de l'invention apparaîtront encore à la lecture de la description qui va suivre. Celle-ci est purement illustrative et doit être lue en regard des dessins annexés sur lesquels :

  • la figure 1 représente un codeur dans un mode de réalisation de l'invention ;
  • la figure 2 représente un décodeur dans un mode de réalisation de l'invention ;
  • la figure 3 est un organigramme représentant des étapes d'un procédé dans un mode de réalisation de l'invention.
Other features and advantages of the invention will become apparent on reading the description which follows. This is purely illustrative and should be read in conjunction with the attached drawings in which:
  • the figure 1 represents an encoder in one embodiment of the invention;
  • the figure 2 represents a decoder in one embodiment of the invention;
  • the figure 3 is a flowchart showing steps of a method in one embodiment of the invention.

La figure 1 représente un codeur audio 1 dans un mode de réalisation de l'invention. Il s'appuie sur la technologie des codeurs audio perceptuels, par exemple de type MPEG-4 AAC.The figure 1 represents an audio coder 1 in one embodiment of the invention. It relies on the technology of perceptual audio coders, for example MPEG-4 AAC type.

Le codeur 1 comprend un module 2 de transformation temps/fréquence, un module 3 de transformation linéaire, un module 4 de quantification, un module 5 de codage entropique de Huffman et un module 6 de calcul de courbe de masquage, en vue de la transmission d'un flux binaire Φ représentant les signaux fournis en entrée du codeur 1.The encoder 1 comprises a time / frequency transformation module 2, a linear transformation module 3, a quantization module 4, a Huffman entropy coding module 5 and a masking curve calculation module 6, for transmission. a bit stream Φ representing the signals supplied at the input of the encoder 1.

Une scène sonore 3D comprend N canaux sur chacun un signal audio respectif S 1, ..., SN est délivré.A 3D sound scene comprises N channels on each a respective audio signal S 1 , ..., S N is delivered.

La figure 2 représente un décodeur audio 100 dans un mode de réalisation de l'invention.The figure 2 represents an audio decoder 100 in one embodiment of the invention.

Le décodeur 100 comprend un module 101 de lecture de séquence binaire, un module 102 de quantification inverse, un module 103 de transformation linéaire inverse, un module 104 de transformation fréquence/temps.The decoder 100 comprises a bit sequence reading module 101, an inverse quantization module 102, an inverse linear transformation module 103, a frequency / time transformation module 104.

Le décodeur 100 est adapté pour recevoir en entrée le flux binaire Φ transmis par le codeur 1 et pour délivrer en sortie Q' signaux S'1, ..., S'Q , destinés à alimenter les Q' haut-parleurs H1, H2 ..., HQ' respectifs d'un système de rendu sonore 105.The decoder 100 is adapted to receive as input the bitstream Φ transmitted by the encoder 1 and to output Q 'signals S' 1 , ..., S ' Q , for supplying the Q' speakers H1, H2 ..., HQ 'of a sound rendering system 105.

Opérations réalisées au niveau du codeur :Operations performed at the encoder level:

Le module 2 de transformation temps/fréquence du codeur 1 reçoit en entrée les N signaux S 1, ..., SN de la scène sonore 3D à coder, sous forme de blocs successifs.The time / frequency conversion module 2 of the encoder 1 receives as input the N signals S 1 ,... S N of the 3D sound scene to be encoded, in the form of successive blocks.

Chaque bloc m reçu comporte N trames temporelles indiquant chacune différentes valeurs prises au cours du temps par un signal respectif.Each block m received has N time frames each indicating different values taken over time by a respective signal.

Sur chaque trame temporelle de chacun des signaux, le module 2 de transformation temps/fréquence effectue une transformation temps/fréquence, dans le cas présent, une transformée en cosinus discrète modifiée (MDCT).On each time frame of each of the signals, the time / frequency transformation module 2 performs a time / frequency transformation, in this case a modified discrete cosine transform (MDCT).

Ainsi, suite à la réception d'un nouveau bloc comportant une nouvelle trame pour chacun des signaux Si , il détermine, pour chacun des signaux Si , i=1 à N, sa représentation spectrale Xi, caractérisée par M coefficients MDCT Xi,t, avec t = 0 à M-1. Un coefficient MDCT Xi,t représente ainsi le spectre du signal Si pour une fréquence Fi. Thus, following the reception of a new block comprising a new frame for each of the signals S i , it determines, for each of the signals S i , i = 1 to N, its spectral representation X i , characterized by M coefficients MDCT X i, t , with t = 0 to M-1. An MDCT coefficient X i, t thus represents the spectrum of the signal Si for a frequency F i .

Les représentations spectrales Xi des signaux Si , i= 1 à N, sont fournies en entrée du module 3 de transformation linéaire.The spectral representations X i of the signals S i , i = 1 to N, are provided at the input of the linear transformation module 3.

Les représentations spectrales Xi des signaux Si , i= 1 à N, sont en outre fournies en entrée du module 6 de calcul des courbes de masquage.The spectral representations X i of the signals S i , i = 1 to N, are further provided at the input of the module 6 for calculating the masking curves.

Le codage de signaux multicanaux comporte dans le cas considéré une transformation linéaire, permettant de prendre en compte les interactions entre les différents signaux audio à coder, avant le codage monophonique, par le module 4 de quantification, des composantes résultant de la transformation linéaire.The coding of multichannel signals comprises in the case considered a linear transformation, making it possible to take into account the interactions between the different audio signals to be coded, before the monophonic coding, by the quantization module 4, of the components resulting from the linear transformation.

Le module 3 de transformation linéaire est adapté pour effectuer une transformation linéaire des coefficients des représentations spectrales (Xi ) 1≤i≤N fournis. Dans un mode de réalisation, il est adapté pour effectuer une transformation spatiale. Il détermine alors les composantes spatiales des signaux (Xi )1≤i≤N , dans le domaine fréquentiel, résultant de la projection sur un référentiel spatial dépendant de l'ordre de la transformation. L'ordre d'une transformation spatiale se rattache à la fréquence angulaire selon laquelle elle « scrute » le champ sonore.The linear transformation module 3 is adapted to perform a linear transformation of the coefficients of the spectral representations ( X i ) 1 i i N N provided. In one embodiment, it is adapted to perform spatial transformation. It then determines the spatial components of the signals ( X i ) 1 i i ≤ N , in the frequency domain, resulting from the projection on a spatial reference system depending on the order of the transformation. The order of a spatial transformation is related to the angular frequency according to which it "scans" the sound field.

Dans le mode de réalisation considéré, le module 3 de transformation linéaire effectue une transformation ambiophonique d'ordre p (par exemple p=1), qui donne une représentation spatiale compacte d'une scène sonore 3D, en réalisant des projections du champ sonore sur les fonctions harmoniques sphériques ou cylindriques associées.In the embodiment considered, the linear transformation module 3 performs an ambiophonic transformation of order p (for example p = 1), which gives a compact spatial representation of a 3D sound scene, by making projections of the sound field onto the spherical or cylindrical harmonic functions associated.

Pour plus d'information sur les transformations ambiophoniques, on pourra se référer aux documents suivants : « Représentation de champs acoustiques, application à la transmission et à la reproduction de scènes sonores complexes dans un contexte multimédia », Thèse de doctorat de l'université Paris 6, Jérôme DANIEL, 31 juillet 2001 , « A highly scalable spherical microphone array based on an orthonormal décomposition of the sound field », Jens Meyer - Gary Elko, Vol. Il - pp. 1781-1784 in Proc. ICASSP 2002 .For more information on surround transformations, refer to the following documents: «Representation of acoustic fields, application to the transmission and reproduction of complex sound scenes in a multimedia context», Doctoral thesis of Paris 6 University, Jérôme DANIEL, July 31, 2001 , "A highly scalable spherical array based microphone on an orthonormal decomposition of the sound field," Jens Meyer - Gary Elko, Vol. He - pp. 1781-1784 in Proc. ICASSP 2002 .

Le module 3 de transformation spatiale délivre ainsi r (r= 2p+1) composantes ambiophoniques (Yj )1≤j≤r. Chaque composante ambiophonique Yj considérées dans le domaine fréquentiel, comporte M paramètres spectraux Yj,t pour t = 0 à M-1. Le paramètre spectral Yj,t se rapporte à la fréquence Ft pour t = 0 à M-1.The spatial transformation module 3 thus delivers r (r = 2p + 1) ambiophonic components ( Y j ) 1 j ≤r . Each ambiophonic component Y j considered in the frequency domain, has M spectral parameters Y j, t for t = 0 to M-1. The spectral parameter Y j, t relates to the frequency F t for t = 0 to M-1.

Les composantes ambiophoniques sont déterminés de la façon suivante : Y 1,0 Y 1, M 1 Y r ,0 Y r , M 1 = R X 1,0 X 1, M 1 X N ,0 X N , M 1

Figure imgb0003
R = R i , j 1 i r 1 j N
Figure imgb0004
est la matrice de transformation ambiophonique d'ordre p pour la scène sonore spatiale, avec R i , j = 1 R i , j = 2 cos 1 2 θ j
Figure imgb0005
si i pair et R i , j = 2 sin i 1 2 θ j
Figure imgb0006
si i impair supérieur ou égale à 3, et θj est l'angle de propagation du signal Sj dans l'espace de la scène 3D.The surround components are determined as follows: Y 1.0 Y 1 M - 1 Y r , 0 Y r , M - 1 = R X 1.0 X 1 M - 1 X NOT , 0 X NOT , M - 1
Figure imgb0003
or R = R i , j 1 i r 1 j NOT
Figure imgb0004
is the p-order ambiophonic transformation matrix for the spatial sound scene, with R i , j = 1 R i , j = 2 cos 1 2 θ j
Figure imgb0005
if i pair and R i , j = 2 sin i - 1 2 θ j
Figure imgb0006
if i odd greater than or equal to 3, and θj is the propagation angle of the signal S j in the space of the 3D scene.

Chacune des composantes ambiophoniques est donc déterminée en fonction de plusieurs signaux (Si )1≤i≤N.Each of the ambiophonic components is therefore determined according to several signals ( S i ) 1 i i ≤ N.

Le module 6 de calcul de courbe de masquage est adapté pour déterminer la courbe de masquage spectral de chaque trame d'un signal Si considéré individuellement dans le bloc m, à l'aide de sa représentation spectrale Xi et d'un modèle psychoacoustique.The masking curve calculation module 6 is adapted to determine the spectral masking curve of each frame of a signal Si considered individually in the block m, using its spectral representation Xi and a psychoacoustic model.

Le module 6 de calcul de courbe de masquage calcule ainsi un seuil de masquage M T m s , i ,

Figure imgb0007
relatif à la trame de chaque signal (Si )1≤i≤N dans le bloc m, pour chaque bande de fréquence s considérée lors de la quantification. Chaque bande de fréquence s est élément d'un ensemble de bandes de fréquence comprenant par exemple les bandes telles que normalisées pour le codeur MPEG-4 AAC.The masking curve calculation module 6 thus calculates a masking threshold M T m s , i ,
Figure imgb0007
relating to the frame of each signal ( S i ) 1 i i ≤ N in the block m, for each frequency band s considered during the quantization. Each frequency band s is part of a set of frequency bands including for example the bands as normalized for the MPEG-4 AAC encoder.

Les seuils de masquage M T m s , i

Figure imgb0008
pour chaque signal Si et chaque bande de fréquences s sont délivrés au module 4 de quantification.Masking thresholds M T m s , i
Figure imgb0008
for each signal S i and each frequency band s are delivered to the quantization module 4.

Le module 4 de quantification est adapté pour quantifier les composantes (Yj )1≤j≤r qui lui sont fournies en entrée, de manière à réduire le débit nécessaire à la transmission. Des fonctions de quantification respectives sont déterminées par le module 4 de quantification sur chaque bande de fréquence s.The quantization module 4 is adapted to quantize the components ( Y j ) 1 j j ≤ r that are input to it, so as to reduce the bit rate required for transmission. Respective quantization functions are determined by the quantization module 4 on each frequency band s.

Dans une bande s quelconque, le module 4 de quantification quantifie chaque coefficient spectral Y j , t 1 i r 0 t M 1

Figure imgb0009
tel que la fréquence Ft est élément de la bande de fréquence s. Il détermine ainsi un indice de quantification i(k) pour chaque coefficient spectral Y j , t 1 i r 0 t M 1
Figure imgb0010
tel que la fréquence Ft est élément de la bande de fréquence s.In any band, the quantization module 4 quantizes each spectral coefficient Y j , t 1 i r 0 t M - 1
Figure imgb0009
such that the frequency F t is an element of the frequency band s. It thus determines a quantization index i (k) for each spectral coefficient Y j , t 1 i r 0 t M - 1
Figure imgb0010
such that the frequency F t is an element of the frequency band s.

Pour une bande s considérée, k prend les valeurs de l'ensemble {k min, s,k min+1 ,s,...k max,s }, et (k max ,s - k min+1,s +1) est égal au nombre de coefficients spectraux à quantifier dans la bande s pour l'ensemble des composantes ambiophoniques.For a considered band s, k takes the values of the set { k min, s , k min + 1 , s , ... k max , s }, and ( k max , s - k min + 1 , s + 1) is equal to the number of spectral coefficients to be quantified in the band s for all the surround components.

La fonction de quantification Qm appliquée par le module 4 de quantification pour les coefficients Y j , t 1 j r 0 t M 1

Figure imgb0011
calculés pour un bloc m de signaux prend la forme suivante, conformément à la norme MPEG-4 AAC :
Q m Y j , t = Arr Y j , t B j m s 3 4
Figure imgb0012
avec la fréquence Ft élément de la bande de fréquence s, et il existe k élément de {k min,s ,k min+1,s ,... kmax,s } tel que Qm(Yj,t ) = i(k). B j m s ,
Figure imgb0013
coefficient d'échelle relatif à la composante ambiophonique Yj, prend des valeurs discrètes. Il dépend du paramètre d'échelle entier relatif ϕ j m s : B j m s = 2 1 4 ϕ j m s .
Figure imgb0014
The quantization function Q m applied by the quantization module 4 for the coefficients Y j , t 1 j r 0 t M - 1
Figure imgb0011
calculated for a m-block of signals takes the following form, in accordance with MPEG-4 AAC:
Q m Y j , t = Arr Y j , t B j m s 3 4
Figure imgb0012
with the frequency F t element of the frequency band s, and there exists k element of { k min , s , k min + 1 , s , ... k max, s } such that Q m ( Y j, t ) = i (k). B j m s ,
Figure imgb0013
scale coefficient relative to the surround component Y j , takes discrete values. It depends on the relative whole scale parameter φ j m s : B j m s = 2 1 4 φ j m s .
Figure imgb0014

Arr est une fonction d'arrondi délivrant une valeur entière. Arr(x) est par exemple la fonction fournissant l'entier le plus proche de la variable x, ou encore la fonction « partie entière » de la variable x, etc.Arr is a rounding function that delivers an integer value. Arr (x) is for example the function providing the integer closest to the variable x, or the function "integer part" of the variable x, etc.

Le module 4 de quantification est adapté pour déterminer une fonction de quantification à appliquer sur une bande de fréquence s vérifiant que le seuil de masquage M T m s , i

Figure imgb0015
de chaque signal Si dans le domaine d'écoute, avec 1 ≤ i ≤ N, est supérieur à la puissance de l'erreur apportée, sur un signal audio restitué dans le domaine d'écoute correspondant au canal i (et non pas dans le domaine de transformation linéaire), par les erreurs de quantification apportée aux composantes ambiophoniques.The quantization module 4 is adapted to determine a quantization function to be applied on a frequency band that verifies that the masking threshold M T m s , i
Figure imgb0015
of each signal S i in the listening domain, with 1 ≤ i ≤ N, is greater than the power of the error made, on an audio signal restored in the listening domain corresponding to the channel i (and not in the linear transformation domain), by the quantization errors made to the ambiophonic components.

Le module 4 de quantification est donc adapté pour déterminer, lors du traitement d'un bloc m de signaux, la fonction de quantification définie à l'aide des paramètres d'échelle B j m s 1 j r

Figure imgb0016
relatifs à chaque bande s, telle que, pour tout i, 1 ≤ i ≤ N, l'erreur introduite sur le signal Si dans la bande s par la quantification des composantes ambiophoniques est inférieure au seuil de masquage M T m s , i
Figure imgb0017
du signal Si sur la bande s.The quantization module 4 is therefore adapted to determine, during the processing of a block m of signals, the quantization function defined using the scale parameters. B j m s 1 j r
Figure imgb0016
relating to each band s, such that, for all i, 1 ≤ i ≤ N, the error introduced on the signal S i in the band s by the quantification of the surround components is less than the masking threshold M T m s , i
Figure imgb0017
of the signal S i on the band s.

Un problème à résoudre par le module 4 de quantification est donc de déterminer, sur chaque bande s, l'ensemble des coefficients d'échelle B j m s 1 j r

Figure imgb0018
vérifiant la formule (1) suivante : B j m / P e m s , i M T m s , i ,1 i N 1 j r
Figure imgb0019
P e m s , i
Figure imgb0020
est la puissance d'erreur introduite sur le signal Si suite aux erreurs de quantification introduites par la quantification, définie par les coefficients d'échelle B j m s 1 j r ,
Figure imgb0021
des composantes ambiophoniques.A problem to be solved by the quantization module 4 is therefore to determine, on each band s, the set of scaling coefficients. B j m s 1 j r
Figure imgb0018
verifying the following formula (1): B j m / P e m s , i M T m s , i , 1 i NOT 1 j r
Figure imgb0019
or P e m s , i
Figure imgb0020
is the error power introduced on the signal S i following the quantization errors introduced by the quantization, defined by the scaling coefficients B j m s 1 j r ,
Figure imgb0021
ambiophonic components.

Ainsi, Bj (s) représente un paramètre caractérisant la fonction de quantification dans la bande s relative à la jème composante. Le choix de Bj (s) détermine de manière bijective la fonction de quantification utilisée.Thus, B j (s) is a parameter characterizing the quantization function s in the band on the j-th component. The choice of B j ( s ) determines in a bijective manner the quantization function used.

Cette disposition a pour effet que le bruit apporté dans le domaine d'écoute par la quantification sur les composantes issues de la transformation linéaire reste masqué par le signal dans le domaine d'écoute, ce qui contribue à une meilleure qualité des signaux restitués dans le domaine d'écoute.This arrangement has the effect that the noise brought into the listening domain by the quantization on the components resulting from the linear transformation remains masked by the signal in the listening domain, which contributes to a better quality of the signals restored in the listening domain.

Dans un mode de réalisation, le problème indiqué ci-dessus par la formule (1) est traduit sous la forme de la formule (2) suivante : B j m / Probabilité P e m s , i M T m s , i α ,1 i N 1 j r ,

Figure imgb0022
α est un taux fixé de respect du seuil de masquage.In one embodiment, the problem indicated above by the formula (1) is translated as the following formula (2): B j m / Probability P e m s , i M T m s , i α , 1 i NOT 1 j r ,
Figure imgb0022
where α is a fixed rate of compliance with the masking threshold.

La probabilité est calculée pour la trame relative au signal Si du bloc m considéré et sur l'ensemble des bandes de fréquence s.The probability is calculated for the frame relating to the signal S i of the block m considered and on all the frequency bands s.

La justification de cette traduction est réalisée dans le document « Optimisation de la quantification par modèles statistiques dans le codeur MPEG Advanced Audio coder (AAC) - Application à la spatialisation d'un signal comprimé en environnement MPEG-4 », Thèse de doctorat de Olivier Derrien - ENST Paris, 22 novembre 2002 , nommé ci-après « document Derrien ». Selon ce document, on cherche à modifier la quantification de manière à diminuer la distorsion perçue par l'oreille d'un signal résultant d'un filtrage de spatialisation HRTF (en anglais « Head Related Transfer Function » encore appelé filtre de tête modélisant l'effet de chemin de propagation entre la position de la source sonore et l'oreille humaine et prenant en compte l'effet dû à la tête et au torse d'un auditeur, appliqué après le décodage.The justification for this translation is realized in the document "Optimization of the quantization by statistical models in the MPEG encoder Advanced Audio coder (AAC) - Application to the spatialization of a compressed signal in MPEG-4 environment", PhD thesis of Olivier Derrien - ENST Paris, 22 November 2002 , hereinafter referred to as "Derrien Document". According to this document, it is sought to modify the quantization so as to reduce the distortion perceived by the ear of a signal resulting from a spatialization filtering HRTF (in English "Head Related Transfer Function" also called head filter modeling the propagation path effect between the position of the sound source and the human ear and taking into account the effect due to the head and the torso of a listener, applied after the decoding.

Par ailleurs, P e m s , i = k = k min k = k max e i m k 2 ,

Figure imgb0023
e i m k k min k k max
Figure imgb0024
sont les erreurs introduites sur les Ks = (k max,s - k min+1,s +1) coefficients spectraux du signal Si correspondant à des fréquences dans la bande s.Otherwise, P e m s , i = Σ k = k min k = k max e i m k 2 ,
Figure imgb0023
or e i m k k min k k max
Figure imgb0024
are the errors introduced on the K s = ( k max, s - k min + 1, s +1) spectral coefficients of the signal S i corresponding to frequencies in the band s.

Soit H = h i , j 1 i N 1 j r

Figure imgb0025
la matrice inverse de la matrice de transformation ambiophonique R, alors e i m k = j = 1 j = r h i , j v j m k ,
Figure imgb0026
v j m k k min , s k k max , s
Figure imgb0027
sont les erreurs de quantification introduites sur les (k max ,s - k min+1,s + 1) coefficients spectraux de composantes ambiophoniques correspondant à des fréquences dans la bande s.Is H = h i , j 1 i NOT 1 j r
Figure imgb0025
the inverse matrix of the ambiophonic transformation matrix R, then e i m k = Σ j = 1 j = r h i , j v j m k ,
Figure imgb0026
or v j m k k min , s k k max , s
Figure imgb0027
are the quantization errors introduced on the ( k max , s - k min + 1 , s + 1) spectral coefficients of ambiophonic components corresponding to frequencies in the band s.

Ainsi P e m s , i = k = k min, s k = k max , s j = 1 j = r h i , j v j m k 2 .

Figure imgb0028
So P e m s , i = Σ k = k min, s k = k max , s Σ j = 1 j = r h i , j v j m k 2 .
Figure imgb0028

On effectue les hypothèses suivantes :

  • les erreurs de quantification e i m k
    Figure imgb0029
    sont des variables aléatoires indépendantes équi-distribuées selon l'indice k ;
  • les erreurs de quantification e i m k
    Figure imgb0030
    sont des variables aléatoires selon l'indice i ;
  • le nombre d'échantillons dans une bande s est suffisamment grand ;
  • le codeur 1 travaille à haute résolution.
The following assumptions are made:
  • quantification errors e i m k
    Figure imgb0029
    are independent random variables equi-distributed according to the index k;
  • quantification errors e i m k
    Figure imgb0030
    are random variables according to the index i;
  • the number of samples in a band s is large enough;
  • the coder 1 works at high resolution.

Sous ces hypothèses et par application du théorème de la limite centrale, la puissance P e m s , i

Figure imgb0031
de l'erreur de quantification, dans une sous-bande s et pour un signal Si , tend, lorsque le nombre de coefficients dans une bande s augmente, vers une gaussienne dont la moyenne m P e m s , S i
Figure imgb0032
et la variance σ P e m s , S i
Figure imgb0033
sont données par les formules suivantes : { m P e m s , i = k = k min , s k max , s E e i m k 2 σ P e m s , i 2 = k = k min , s k max , s E e i m k 4 E e i m k 2 2
Figure imgb0034
où la fonction E[x] délivre la moyenne de la variable x.Under these assumptions and by applying the central limit theorem, the power P e m s , i
Figure imgb0031
the quantization error, in a sub-band s and for a signal S i , tends, when the number of coefficients in a band s increases, to a Gaussian whose average m P e m s , S i
Figure imgb0032
and the variance σ P e m s , S i
Figure imgb0033
are given by the following formulas: { m P e m s , i = Σ k = k min , s k max , s E e i m k 2 σ P e m s , i 2 = Σ k = k min , s k max , s E e i m k 4 - E e i m k 2 2
Figure imgb0034
where the function E [x] delivers the average of the variable x.

La contrainte « Probabilité P e m s , i M T m s , i α

Figure imgb0035
» indiquée dans la formule 2 ci-dessus s'écrit alors à l'aide de la formule (3) suivante : m P e m s , i + β α σ P e m s , i M T m s , i
Figure imgb0036
Avec : β α = 2 Erf 1 2 α 1
Figure imgb0037
et la fonction Erf -1(x) est l'inverse de la fonction d'erreur d'Euler.The "Probability" constraint P e m s , i M T m s , i α
Figure imgb0035
Indicated in formula 2 above is then written using the following formula (3): m P e m s , i + β α σ P e m s , i M T m s , i
Figure imgb0036
With: β α = 2 Erf - 1 2 α - 1
Figure imgb0037
and the function Erf -1 ( x ) is the inverse of the error function of Euler.

Les variables e i m k

Figure imgb0038
étant indépendantes selon l'indice i, on en déduit : E e i m k 2 = j = 1 r h i , j 2 E v i m k 2
Figure imgb0039
The variables e i m k
Figure imgb0038
being independent according to the index i, we deduce: E e i m k 2 = Σ j = 1 r h i , j 2 E v i m k 2
Figure imgb0039

Par conséquent, on obtient : m P e m s , i = k = k min , s k max , s j = 1 r h i , j 2 E v i m k 2 = j = 1 r h i , j 2 k = k min , s k max , s E v j m k 2

Figure imgb0040
Therefore, we get: m P e m s , i = Σ k = k min , s k max , s Σ j = 1 r h i , j 2 E v i m k 2 = Σ j = 1 r h i , j 2 Σ k = k min , s k max , s E v j m k 2
Figure imgb0040

Les variables aléatoires e i m k

Figure imgb0041
étant indépendantes et équi-distribuées selon l'indice k, les variables aléatoires v i m k
Figure imgb0042
sont également indépendantes et équi-distribuées selon l'indice k. Par conséquent : m P e m s , i = K s j = 1 r h i , j 2 E v i m s 2
Figure imgb0043
avec : K s = k max , s k min , s + 1
Figure imgb0044
Random variables e i m k
Figure imgb0041
being independent and equi-distributed according to the index k, the random variables v i m k
Figure imgb0042
are also independent and equi-distributed according to the index k. Therefore : m P e m s , i = K s Σ j = 1 r h i , j 2 E v i m s 2
Figure imgb0043
with: K s = k max , s - k min , s + 1
Figure imgb0044

On suppose que les puissances P e m s , i

Figure imgb0045
d'erreur de quantification tendent vers des gaussiennes, alors : E e i m k 4 = 3 E e i m k 2 2
Figure imgb0046
It is assumed that the powers P e m s , i
Figure imgb0045
of quantification error tend to Gaussians, then: E e i m k 4 = 3 E e i m k 2 2
Figure imgb0046

D'où : σ 2 P e m s , i = 2 k = k min , s k max , s E e i m k 2 2

Figure imgb0047
From where : σ 2 P e m s , i = 2 Σ k = k min , s k max , s E e i m k 2 2
Figure imgb0047

Ainsi on peut écrire : σ 2 P e m s , i = 2 k = k min , s k max , s h i , j 2 j = 1 r E v j m k 2 2

Figure imgb0048
So we can write: σ 2 P e m s , i = 2 Σ k = k min , s k max , s h i , j 2 Σ j = 1 r E v j m k 2 2
Figure imgb0048

A partir de cette dernière équation, et en appliquant l'inégalité de Cauchy-Schwartz : σ P e m s , i = 2 k = k min , s k max , s h i , j 2 j = 1 r E v j m k 2 2 2 k = k min , s k max , s h i , j 2 j = 1 r E v j m k 2

Figure imgb0049
From this last equation, and applying the Cauchy-Schwartz inequality: σ P e m s , i = 2 Σ k = k min , s k max , s h i , j 2 Σ j = 1 r E v j m k 2 2 2 Σ k = k min , s k max , s h i , j 2 Σ j = 1 r E v j m k 2
Figure imgb0049

Ce qui implique que : σ P e m s , i 2 m P e m s , i

Figure imgb0050
Which implies : σ P e m s , i 2 m P e m s , i
Figure imgb0050

Par ailleurs, en haute résolution : E v j 2 16 9 E e R 2 B j m s 3 2 μ 1 2 , j s

Figure imgb0051
avec μ 1 2 , j
Figure imgb0052
représentant l'espérance mathématique de | Y j m | 1 2
Figure imgb0053
dans la sous bande s traitée et eR l'erreur d'arrondi propre à la fonction d'arrondi Arr.In addition, in high resolution: E v j 2 16 9 E e R 2 B j m s 3 2 μ 1 2 , j s
Figure imgb0051
with μ 1 2 , j
Figure imgb0052
representing the mathematical expectation of | Y j m | 1 2
Figure imgb0053
in the sub-band treated and e R the rounding error specific to the rounding function Arr.

Si Arr(x) est par exemple la fonction fournissant l'entier le plus proche de la variable x, eR est égale à 0,5. Si Arr(x) est la fonction « partie entière » de la variable x, eR est égale à 1.For example, if Arr (x) is the function providing the integer closest to the variable x, e R is equal to 0.5. If Arr (x) is the function "integer part" of the variable x, e R is equal to 1.

Ainsi la contrainte donnée par la formule (3) relative au signal Si , i= 1 à N, sur une bande s, s'écrit sous la forme suivante : K s 16 9 E e R 2 1 + 2 β α j = 1 r h i , j 2 B j m s 3 2 μ 1 2 , j s M T m s , i

Figure imgb0054
Thus the constraint given by the formula (3) relating to the signal S i , i = 1 to N, on a band s, is written in the following form: K s 16 9 E e R 2 1 + 2 β α Σ j = 1 r h i , j 2 B j m s 3 2 μ 1 2 , j s M T m s , i
Figure imgb0054

Il est ainsi possible, à partir de cette dernière équation, de déterminer si des coefficients d'échelle B j m s 1 j r

Figure imgb0055
calculés par le module 4 de quantification pour coder les composantes de la transformée, permettent ou non de respecter le seuil de masquage tel que considéré dans le domaine du signal.It is thus possible, from this last equation, to determine whether scaling coefficients B j m s 1 j r
Figure imgb0055
calculated by the quantization module 4 to code the components of the transform, allow or not to respect the masking threshold as considered in the signal domain.

Cette dernière équation représente une condition suffisante pour que le bruit correspondant au canal i soit masqué en sortie dans le domaine d'écoute.This last equation represents a sufficient condition for the noise corresponding to the channel i to be masked at the output in the listening domain.

Dans un mode de réalisation de l'invention, le module 4 de quantification est adapté pour déterminer à l'aide de cette dernière équation, pour un bloc m de trames courant, des coefficients d'échelle B j m s 1 j r

Figure imgb0056
garantissant que le bruit dans le domaine d'écoute est masqué.In one embodiment of the invention, the quantization module 4 is adapted to determine using the latter equation, for a block m of current frames, scaling coefficients. B j m s 1 j r
Figure imgb0056
ensuring that noise in the listening domain is hidden.

Dans un mode de réalisation particulier de l'invention, le module 4 de quantification est adapté pour déterminer, pour un bloc m de trames courant, des coefficients d'échelle B j m s 1 j r

Figure imgb0057
garantissant que le bruit dans le domaine d'écoute est masqué et en outre permettant de respecter une contrainte de débit.In a particular embodiment of the invention, the quantization module 4 is adapted to determine, for a block m of current frames, scaling coefficients. B j m s 1 j r
Figure imgb0057
guaranteeing that the noise in the listening domain is masked and furthermore making it possible to respect a flow constraint.

Dans un mode de réalisation, les conditions à respecter sont les suivantes :

  • Minimiser le débit global D m = j = 1 r D j m
    Figure imgb0058
  • Sous la contrainte : K s 16 9 E e R 2 1 + 2 β α j = 1 r h i , j 2 B j m s 3 2 μ 1 2 , j s M T m s , i
    Figure imgb0059
    pour toute bande s, avec D j m
    Figure imgb0060
    le débit global attribué à la composante ambiophonique Yj .
In one embodiment, the conditions to be respected are the following:
  • Minimize overall throughput D m = Σ j = 1 r D j m
    Figure imgb0058
  • Under duress : K s 16 9 E e R 2 1 + 2 β α Σ j = 1 r h i , j 2 B j m s 3 2 μ 1 2 , j s M T m s , i
    Figure imgb0059
    for any band s, with D j m
    Figure imgb0060
    the overall bit rate assigned to the surround component Y j .

On peut écrire que : D j m = s D j m s

Figure imgb0061
D j m s
Figure imgb0062
est le débit attribué à la composante ambiophonique Yj dans la bande s.We can write that: D j m = Σ s D j m s
Figure imgb0061
or D j m s
Figure imgb0062
is the bit rate assigned to the surround component Y j in the band s.

Minimiser le débit global Dm revient donc à minimiser le débit D m s = j = 1 r D j m s

Figure imgb0063
dans chaque bande s. Dans une première approximation, on peut écrire que le débit attribué à une composante ambiophonique dans une bande s est une fonction logarithmique du coefficient d'échelle, soit : D j m s = D j ,0 m γ ln B j m s
Figure imgb0064
Minimize the overall flow D m is therefore to minimize the flow D m s = Σ j = 1 r D j m s
Figure imgb0063
in each band s. In a first approximation, it can be written that the bit rate attributed to an ambiophonic component in a band s is a logarithmic function of the scale coefficient, namely: D j m s = D j , 0 m - γ ln B j m s
Figure imgb0064

La nouvelle fonction à minimiser s'écrit donc sous la forme suivante : F s = j = 1 r ln B j m s

Figure imgb0065
The new function to be minimized is written in the following form: F s = - Σ j = 1 r ln B j m s
Figure imgb0065

Pour résoudre le problème de quantification par bande en minimisant le débit global sous la contrainte (3), il faut donc minimiser la fonction F sous la contrainte (3).To solve the band quantization problem by minimizing the overall rate under the constraint (3), we must therefore minimize the function F under the constraint (3).

La résolution de ce problème d'optimisation sous contrainte est par exemple effectuée à l'aide de la méthode des Lagrangiens.
La fonction Lagrangienne s'écrit sous la forme suivante : L B , λ = j = 1 r ln B j m s + i = 1 N λ i K s 16 9 E e R 2 1 + 2 β α j = 1 r h i , j 2 B j m s 3 2 μ 1 2 , j s M T m s , i

Figure imgb0066
L B , λ = j = 1 r ln B j m s + Δ j m λ B j m s 3 2 i = 1 N λ i M T m s , i
Figure imgb0067
The resolution of this constrained optimization problem is for example carried out using the Lagrangian method.
The Lagrangian function is written in the following form: The B , λ = - Σ j = 1 r ln B j m s + Σ i = 1 NOT λ i K s 16 9 E e R 2 1 + 2 β α Σ j = 1 r h i , j 2 B j m s 3 2 μ 1 2 , j s - M T m s , i
Figure imgb0066
The B , λ = - Σ j = 1 r ln B j m s + Δ j m λ B j m s 3 2 - Σ i = 1 NOT λ i M T m s , i
Figure imgb0067

Avec : Δ j m λ = μ 1 2 , j s K s 16 9 E e R 2 1 + 2 β α i = 1 N h i , j 2 λ i

Figure imgb0068
et les valeurs λj , 1 ≤ jN, sont les coordonnées du vecteur de Lagrange λ.With: Δ j m λ = μ 1 2 , j s K s 16 9 E e R 2 1 + 2 β α Σ i = 1 NOT h i , j 2 λ i
Figure imgb0068
and the values λ j , 1 ≤ jN , are the coordinates of the Lagrange vector λ.

La mise en oeuvre de la méthode des Lagrangiens permet d'écrire tout d'abord que, pour 1 ≤ j ≤ r : B j m s = 3 2 1 Δ j m λ

Figure imgb0069
The implementation of the Lagrangian method allows us to write first that for 1 ≤ j ≤ r: B j m s = 3 2 1 Δ j m λ
Figure imgb0069

On remplace par ces termes les coefficients d'échelle dans l'équation de Lagrange. Et on cherche alors à déterminer la valeur du vecteur de Lagrange λ qui maximise la fonction ω λ = L B 1 m s , , B r m s , λ ,

Figure imgb0070
par exemple à l'aide de la méthode du gradient de la fonction ω.These terms are used to replace the scale coefficients in the Lagrange equation. And we then try to determine the value of the Lagrange vector λ which maximizes the function ω λ = The B 1 m s , ... , B r m s , λ ,
Figure imgb0070
for example using the gradient method of the function ω .

D'après la méthode du gradient d'Uzawa ∇w(λ), où ω λ = ω λ 1 λ ω λ N λ

Figure imgb0071
les dérivées partielles ne sont autres que les contraintes calculées pour les B j m s = 3 2 1 Δ j m λ .
Figure imgb0072
According to the Uzawa gradient method ∇ w ( λ ), where ω λ = ω λ 1 λ ω λ NOT λ
Figure imgb0071
partial derivatives are just the constraints calculated for the B j m s = 3 2 1 Δ j m λ .
Figure imgb0072

On utilise la méthode itérative de gradient relatif (cf. notamment le document Derrien) pour résoudre ce système.The iterative relative gradient method (see in particular the Derrien document) is used to solve this system.

L'équation générale (formule (4)) de mise à jour du vecteur de Lagrange lors d'une (k+1)ième itération de la méthode s'écrit alors sous la forme suivante : λ k + 1 = λ k 1 + ρ m ω λ k

Figure imgb0073
avec le vecteur de Lagrange λ avec un exposant (k+1) indiquant le vecteur actualisé et le vecteur de Lagrange λ avec un exposant k indiquant le vecteur calculé précédemment lors de la kième itération, ⊗ désignant le produit terme à terme entre deux vecteurs de même taille, ρ désignant le pas de l'algorithme itératif et m étant un vecteur de pondération.The general equation (formula (4)) for updating the Lagrange vector during a (k + 1) th iteration of the method is then written in the following form: λ k + 1 = λ k 1 + ρ m ω λ k
Figure imgb0073
with the vector of Lagrange λ with an exponent (k + 1) indicating the update vector and the vector of Lagrange λ with an exponent k indicating the previously calculated vector at the k th iteration, ⊗ denotes the product term term between two vectors of the same size, ρ designating the pitch of the iterative algorithm and m being a weighting vector.

Dans un mode de réalisation, de manière à assurer la convergence de la méthode itérative, on choisit le vecteur m égal à : 1 M T m s ,1 1 M T m s , N

Figure imgb0074
In one embodiment, in order to ensure the convergence of the iterative method, the vector m is chosen equal to: 1 M T m s , 1 1 M T m s , NOT
Figure imgb0074

Dans le mode de réalisation considéré, le module 4 de quantification est adapté pour mettre en oeuvre les étapes du procédé décrit ci-dessous en référence à la figure 3 sur chaque bande de quantification s lors de la quantification d'un bloc m de signaux (Si )1≤iN .In the embodiment considered, the quantization module 4 is adapted to implement the steps of the method described below with reference to FIG. figure 3 on each quantization band s when quantizing a block m of signals ( S i ) 1 i iN.

Le procédé est basé sur un algorithme itératif comprenant des instructions pour mettre en oeuvre les étapes décrites ci-dessous lors de l'exécution de l'algorithme sur des moyens de calcul du module 4 de quantification.The method is based on an iterative algorithm comprising instructions for implementing the steps described below during the execution of the algorithm on the calculation means of the quantization module 4.

Dans une étape a/ d'initialisation (k=0) : on définit la valeur du pas d'itération ρ, une valeur D représentant un seuil de débit et la valeur des coordonnées (λ 1, ...λN ) du vecteur de Lagrange initial avec λj = λ 0, 1 ≤ j ≤ N. In a step a / of initialization (k = 0): one defines the value of the iteration step ρ , a value D representing a threshold of flow and the value of the coordinates ( λ 1 , ... λ N ) of the vector initial Lagrange with λ j = λ 0 , 1 ≤ j ≤ N.

Les étapes de la boucle itérative pour une (k+1)ème itération, avec k entier supérieur ou égal à 0, sont les suivantes.The steps of the iterative loop for a (k + 1) th iteration, with k integer greater than or equal to 0, are as follows.

Dans une étape b/, les valeurs des coordonnées λj , 1 ≤ jN du vecteur de Lagrange considérées étant celles calculées précédemment lors de la kième itération, on calcule pour 1 ≤ jN : Δ j m λ = μ 1 2 , j s K s 16 9 E e R 2 1 + 2 β α i = 1 N h i , j 2 λ i

Figure imgb0075
In a step b /, the coordinate values λ j, 1 ≤ jN Lagrange vector considered being those calculated previously at the kth iteration, is calculated for 1 ≤ jN: Δ j m λ = μ 1 2 , j s K s 16 9 E e R 2 1 + 2 β α Σ i = 1 NOT h i , j 2 λ i
Figure imgb0075

Puis dans une étape c/, on calcule les coefficients d'échelle, pour 1 ≤ jr : B j m s = 3 2 1 Δ j m λ

Figure imgb0076
Then in a step c /, the scaling coefficients are calculated for 1 ≤ jr: B j m s = 3 2 1 Δ j m λ
Figure imgb0076

Dans une étape d/, on calcule la valeur de la fonction F sur la bande s, représentant le débit correspondant pour la bande s : F s = j = 1 r ln B j m s

Figure imgb0077
In a step d /, the value of the function F is calculated on the band s, representing the corresponding bit rate for the band s: F s = - Σ j = 1 r ln B j m s
Figure imgb0077

Dans une étape e/, on compare la valeur F(s) calculée avec le seuil donné D.In a step e /, the calculated value F ( s ) is compared with the given threshold D.

Si la valeur F(s) calculée est supérieure au seuil donné D, on calcule, dans une étape f/, la valeur du vecteur de Lagrange λ pour la (k+1)ème itération à l'aide de l'équation (4) indiquée ci-dessus et du vecteur de Lagrange calculé lors de la kème itération.If the calculated value F ( s ) is greater than the given threshold D, the value of the Lagrange vector λ for the (k + 1) th iteration is calculated in a step f / using the equation (4 ) indicated above and the Lagrange vector calculated during the k th iteration.

Puis, dans une étape g/, on incrémente l'indice k d'une unité et on réitère les étapes b/, c/, d/ et e/.Then, in a step g /, the index k is incremented by one unit and the steps b /, c /, d / and e / are repeated.

Si la valeur F(s) calculée à l'étape e/, est inférieure au seuil donné D, on stoppe les itérations. On a alors déterminé des coefficients d'échelle B j m s 1 j r

Figure imgb0078
pour la bande de quantification s permettant de masquer, dans le domaine d'écoute, le bruit dû à la quantification dans la bande s, des composantes ambiophoniques (Yj )1≤jr , tout en garantissant que le débit nécessaire pour cette quantification dans la bande s est inférieur à une valeur déterminée, fonction de D.If the value F ( s ) calculated in step e / is less than the given threshold D, the iterations are stopped. Scale coefficients were then determined B j m s 1 j r
Figure imgb0078
for the quantization band s making it possible to mask, in the listening domain, the noise due to the quantization in the band s, of the ambiophonic components ( Y j ) 1 j jr , while ensuring that the bit rate required for this quantization in the band s is less than a determined value, function of D.

On applique ensuite la fonction de quantification ainsi déterminée pour les bandes s respectives et composantes ambiophoniques respectives aux coefficients spectraux des composantes ambiophoniques. Les indices de quantification ainsi que des éléments de définition de la fonction de quantification sont fournis au module 5 de codage de Huffman.The quantization function thus determined for the respective s-bands and respective surround components is then applied to the spectral coefficients of the surround components. The quantization indices as well as definition elements of the quantization function are provided to the Huffman coding module.

Les données de codage délivrées par le module 5 de codage de Huffman sont ensuite transmises sous forme de flux binaire Φ au décodeur 100.The coding data delivered by the Huffman coding module 5 is then transmitted as a bit stream Φ to the decoder 100.

Opérations réalisées au niveau du décodeur :Operations performed at the decoder:

Le module 101 de lecture de séquence binaire est adapté pour extraire des données de codage présentes dans le flux Φ reçu par le décodeur et en déduire, dans chaque bande s, des indices de quantification i(k) et des coefficients d'échelle B j m s 1 j r .

Figure imgb0079
The bit sequence reading module 101 is adapted to extract coding data present in the stream Φ received by the decoder and to deduce, in each band s, quantization indices i (k) and scale coefficients. B j m s 1 j r .
Figure imgb0079

Le module de quantification inverse 102 est adapté pour déterminer les coefficients spectraux, relatifs à la bande s, des composantes ambiophoniques correspondants en fonction des indices de quantification i(k) et des coefficients d'échelles B j m s 1 j r

Figure imgb0080
dans chaque bande s.The inverse quantization module 102 is adapted to determine the spectral coefficients, relative to the band s, of the corresponding ambiophonic components as a function of the quantization indices i (k) and the scale coefficients. B j m s 1 j r
Figure imgb0080
in each band s.

Ainsi un coefficient spectral Yj,t relatif à la fréquence Ft élément de la bande s de la composante ambiophonique Yj et représenté par l'indice de quantification i(k) est restitué par le module 102 de quantification inverse à l'aide de la formule suivante : Y j , t = A j m s i k 4 3

Figure imgb0081
Thus, a spectral coefficient Y j, t relative to the frequency F t of the band s of the ambiophonic component Y j and represented by the quantization index i (k) is returned by the inverse quantization module 102 using of the following formula: Y j , t = AT j m s i k 4 3
Figure imgb0081

Un décodage ambiophonique est ensuite appliqué aux r composantes ambiophoniques décodées, de manière à déterminer Q' signaux S'1, S'2, ..., S'Q, destinés aux Q' haut-parleurs H1, H2 ..., HQ'.Ambiophonic decoding is then applied to the decoded surround components, so as to determine the signals S ' 1 , S' 2 ,..., S ' Q , for the speakers H1, H2 ..., HQ. .

Le bruit de quantification à la sortie du décodeur 100 est une constante qui ne dépend que de la transformée R utilisée et du module 4 de quantification car les données psychoacoustiques utilisées lors du codage ne prennent pas en considération les traitements effectués lors de la restitution par le décodeur. En effet, le modèle psychoacoustique ne prend pas en compte les interactions acoustiques entre les différents signaux, mais calcule la courbe de masquage d'un signal comme s'il était le seul écouté. L'erreur calculée sur ce signal reste donc constante et masquée pour toute matrice de décodage ambiophonique utilisée. Cette matrice de décodage ambiophonique va simplement modifier la distribution de l'erreur sur les différents haut-parleurs en sortie.The quantization noise at the output of the decoder 100 is a constant which depends only on the transform R used and the quantization module 4 because the psychoacoustic data used during the coding do not take into account the processing performed during the rendering by the decoder. Indeed, the psychoacoustic model does not take into account the acoustic interactions between the different signals, but calculates the masking curve of a signal as if it were the only one listened to. The error calculated on this signal therefore remains constant and masked for any surround decoding matrix used. This surround decoding matrix will simply change the distribution of the error on the different speakers output.

Claims (9)

  1. Method for quantizing components, some at least of said components ((Yj )1≤j≤r) each being determined as a function of a plurality of audio signals ((Sj )1≤j≤N) of a sound scene and computed by applying an ambiophonic multichannel linear transformation with more than two channels to said audio signals,
    according to which a quantization function (Qm) to be applied to said components in a given frequency band (s) is determined by testing a condition relating to at least one audio signal (Si ) and depending at least on a comparison performed between:
    - a psychoacoustic masking threshold M m T s , i
    Figure imgb0088
    relating to the audio signal in the given frequency band, and
    - a value determined as a function of the inverse multichannel linear transformation and of errors of quantization of the components by said function on the given frequency band.
  2. Method according to Claim 1, according to which the condition relates to several audio signals and depends on several comparisons, each comparison being performed between a psychoacoustic masking threshold relating to a respective audio signal in the given frequency band, and a value determined as a function of the inverse multichannel linear transformation and of errors of quantization of the components by said function.
  3. Method according to Claim 1 or Claim 2, according to which the determination of the quantization function (Qm) is repeated during the updating of the values of the components to be quantized.
  4. Method according to any one of the preceding claims, according to which the condition relating to an audio signal at least is tested by comparing the psychoacoustic masking threshold relating to the audio signal and an element representing the mathematical value j = 1 r h i , j 2 B j s 3 2 μ 2 1 , j s ,
    Figure imgb0089
    where s is the given band of frequencies, r is the number of components, hi,j is that coefficient of the inverse multichannel linear transform relating to the audio signal (Si) and to the jth component with j=1 to r, Bj(s) is a scale coefficient relating to the ambiophonic component Yj taking discrete values and characterizing the quantization function (Qm) in the band s relating to the jth component and µ12,j (s) is the mathematical expectation in the band s of the square root of the jth component.
  5. Method according to any one of the preceding claims, according to which a quantization function to be applied to said components in the given frequency band is determined with the aid of an iterative process generating at each iteration a parameter of the candidate quantization function satisfying the condition and associated with a corresponding bit rate, the iteration being halted when the bit rate is below a given threshold.
  6. Quantization module (4) adapted for quantizing at least components ((Yj )1≤j≤r) each determined as a function of a plurality of audio signals ((Sj )1≤j≤N) of a sound scene and computed by applying a multichannel linear transformation to said audio signals, said quantization module being adapted for implementing the steps of a method according to any one of Claims 1 to 5.
  7. Audio coder (1) adapted for coding an audio scene comprising several respective audio signals ((Sj )1≤j≤N) as a binary output stream (Φ), comprising:
    - a transformation module (3) adapted for computing by applying a multichannel linear transformation to said audio signals, components ((Yj )1≤j≤r) at least some of which are each determined as a function of a plurality of the audio signals; and
    - a quantization module (4) according to Claim 6 adapted for determining at least one quantization function (Qm) on at least one given frequency band (s) and for quantizing the components on the given frequency band as a function of at least the determined quantization function;
    said coder being adapted for constructing a binary stream as a function at least of quantization data delivered by the quantization module.
  8. Computer program to be installed in a quantization module (4), said program comprising instructions for implementing the steps of a method according to any one of Claims 1 to 5 during execution of the program by processing means of said module.
  9. Coding data (Φ), determined following the implementation of a quantization method according to any one of Claims 1 to 5.
EP08806144.5A 2007-07-03 2008-07-01 Quantification after linear conversion combining audio signals of a sound scene, and related encoder Active EP2168121B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR0704794 2007-07-03
PCT/FR2008/051220 WO2009007639A1 (en) 2007-07-03 2008-07-01 Quantification after linear conversion combining audio signals of a sound scene, and related encoder

Publications (2)

Publication Number Publication Date
EP2168121A1 EP2168121A1 (en) 2010-03-31
EP2168121B1 true EP2168121B1 (en) 2018-06-06

Family

ID=38799400

Family Applications (1)

Application Number Title Priority Date Filing Date
EP08806144.5A Active EP2168121B1 (en) 2007-07-03 2008-07-01 Quantification after linear conversion combining audio signals of a sound scene, and related encoder

Country Status (3)

Country Link
US (1) US8612220B2 (en)
EP (1) EP2168121B1 (en)
WO (1) WO2009007639A1 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2469741A1 (en) * 2010-12-21 2012-06-27 Thomson Licensing Method and apparatus for encoding and decoding successive frames of an ambisonics representation of a 2- or 3-dimensional sound field
JP6267860B2 (en) * 2011-11-28 2018-01-24 三星電子株式会社Samsung Electronics Co.,Ltd. Audio signal transmitting apparatus, audio signal receiving apparatus and method thereof
US9466305B2 (en) 2013-05-29 2016-10-11 Qualcomm Incorporated Performing positional analysis to code spherical harmonic coefficients
US20140355769A1 (en) 2013-05-29 2014-12-04 Qualcomm Incorporated Energy preservation for decomposed representations of a sound field
US9922656B2 (en) 2014-01-30 2018-03-20 Qualcomm Incorporated Transitioning of ambient higher-order ambisonic coefficients
US9489955B2 (en) 2014-01-30 2016-11-08 Qualcomm Incorporated Indicating frame parameter reusability for coding vectors
US10770087B2 (en) 2014-05-16 2020-09-08 Qualcomm Incorporated Selecting codebooks for coding vectors decomposed from higher-order ambisonic audio signals
US9620137B2 (en) 2014-05-16 2017-04-11 Qualcomm Incorporated Determining between scalar and vector quantization in higher order ambisonic coefficients
US9852737B2 (en) 2014-05-16 2017-12-26 Qualcomm Incorporated Coding vectors decomposed from higher-order ambisonics audio signals
US9747910B2 (en) 2014-09-26 2017-08-29 Qualcomm Incorporated Switching between predictive and non-predictive quantization techniques in a higher order ambisonics (HOA) framework
EP3067886A1 (en) * 2015-03-09 2016-09-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoder for encoding a multichannel signal and audio decoder for decoding an encoded audio signal

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1992012607A1 (en) * 1991-01-08 1992-07-23 Dolby Laboratories Licensing Corporation Encoder/decoder for multidimensional sound fields
US7548853B2 (en) * 2005-06-17 2009-06-16 Shmunk Dmitry V Scalable compressed audio bit stream and codec using a hierarchical filterbank and multichannel joint coding

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
WO2009007639A1 (en) 2009-01-15
US20100198585A1 (en) 2010-08-05
EP2168121A1 (en) 2010-03-31
US8612220B2 (en) 2013-12-17

Similar Documents

Publication Publication Date Title
EP2168121B1 (en) Quantification after linear conversion combining audio signals of a sound scene, and related encoder
US11962990B2 (en) Reordering of foreground audio objects in the ambisonics domain
EP2374123B1 (en) Improved encoding of multichannel digital audio signals
EP2374124B1 (en) Advanced encoding of multi-channel digital audio signals
EP2002424B1 (en) Device and method for scalable encoding of a multichannel audio signal based on a principal component analysis
CN105917408B (en) Indicating frame parameter reusability for coding vectors
JP5922684B2 (en) Multi-channel decoding device
RU2439718C1 (en) Method and device for sound signal processing
EP2143102B1 (en) Audio encoding and decoding method and associated audio encoder, audio decoder and computer programs
US20030233236A1 (en) Audio coding system using characteristics of a decoded signal to adapt synthesized spectral components
US20110002393A1 (en) Audio encoding device, audio encoding method, and video transmission device
JP2009524108A (en) Complex transform channel coding with extended-band frequency coding
JP2013506164A (en) Audio signal decoder, audio signal encoder, upmix signal representation generation method, downmix signal representation generation method, computer program, and bitstream using common object correlation parameter values
CN102968996A (en) Temporal envelope shaping for spatial audio coding using frequency domain wiener filtering
WO2010004155A1 (en) Spatial synthesis of multichannel audio signals
EP2145167B1 (en) Audio encoding method, audio encoder, coded signal and associated computer program
WO2020177981A1 (en) Spatialized audio coding with interpolation and quantification of rotations
EP2476114B1 (en) Audio signal encoding employing interchannel and temporal redundancy reduction
FR3049084A1 (en)
WO2010016270A1 (en) Quantizing device, encoding device, quantizing method, and encoding method
JP2022536676A (en) Packet loss concealment for DirAC-based spatial audio coding
EP2198425A1 (en) Method, module and computer software with quantification based on gerzon vectors
FR3112015A1 (en) Optimized coding of information representative of a spatial image of a multichannel audio signal
Cantzos et al. Quality Enhancement of Compressed Audio Based on Statistical Conversion

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20091224

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA MK RS

17Q First examination report despatched

Effective date: 20100512

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ORANGE

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20180118

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAL Information related to payment of fee for publishing/printing deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR3

GRAR Information related to intention to grant a patent recorded

Free format text: ORIGINAL CODE: EPIDOSNIGR71

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

INTG Intention to grant announced

Effective date: 20180118

INTG Intention to grant announced

Effective date: 20180426

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1006935

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180615

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602008055549

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 11

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20180606

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180906

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180906

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180907

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1006935

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181006

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602008055549

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180701

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20180731

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180701

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180731

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180731

26N No opposition filed

Effective date: 20190307

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180731

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180606

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20080701

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230621

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230620

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230620

Year of fee payment: 16