EP1735779B1 - Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé - Google Patents

Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé Download PDF

Info

Publication number
EP1735779B1
EP1735779B1 EP05718592.8A EP05718592A EP1735779B1 EP 1735779 B1 EP1735779 B1 EP 1735779B1 EP 05718592 A EP05718592 A EP 05718592A EP 1735779 B1 EP1735779 B1 EP 1735779B1
Authority
EP
European Patent Office
Prior art keywords
processing
right signals
signal
spatial parameters
channel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP05718592.8A
Other languages
German (de)
English (en)
Other versions
EP1735779A1 (fr
Inventor
Machiel W. Van Loon
Gerard H. Hotho
Dirk J. Breebaart
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Koninklijke Philips NV
Original Assignee
Koninklijke Philips Electronics NV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics NV filed Critical Koninklijke Philips Electronics NV
Priority to EP05718592.8A priority Critical patent/EP1735779B1/fr
Priority to PL05718592T priority patent/PL1735779T3/pl
Publication of EP1735779A1 publication Critical patent/EP1735779A1/fr
Application granted granted Critical
Publication of EP1735779B1 publication Critical patent/EP1735779B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S5/00—Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/03—Application of parametric coding in stereophonic audio systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/02—Systems employing more than two channels, e.g. quadraphonic of the matrix type, i.e. in which input signals are combined algebraically, e.g. after having been phase shifted with respect to each other

Definitions

  • the present invention relates to a method and device for processing a stereo signal obtained from an encoder, which encoder encodes an N-channel audio signal into left and right signals and spatial parameters.
  • the invention also relates to an encoder apparatus comprising such an encoder and such a device.
  • the present invention also relates to a method and device for processing a stereo signal obtained by such a method and such a device for processing a stereo signal obtained from an encoder.
  • the invention also relates to a decoder apparatus comprising such a device for processing a stereo signal.
  • the present invention also relates to an audio system comprising such an encoder apparatus and such a decoder apparatus.
  • the input channels may be basically encoded individually (possibly after matrixing), thus requiring a high bit rate due to the large number of channels.
  • a multi-channel audio encoder may generate a 2-channel down-mix which is compatible with 2-channel reproduction systems, while still enabling high-quality multi-channel reconstruction at the decoder side.
  • the high-quality reconstruction is controlled by transmitted parameters P which control the stereo-to-multi-channel upmix process.
  • These parameters contain information describing, amongst others, the ratio of front versus surround signal which is present in the 2-channel down mix.
  • a decoder can control the amount of front versus surround signal in the upmix process.
  • the parameters describe important properties of the spatial sound field which was present in the original multi-channel signal, but which is lost in the stereo mix due to the down-mix process.
  • the current invention relates to the possibility to use this parameterized spatial information to apply parameter-dependent, preferably invertible, post-processing on a 2-channel down-mix to enhance the downmix, such as the perceptual quality or spatial properties thereof.
  • An object of the present invention is to make post-processing of the down-mix possible after encoding, based upon the parameters as determined in the multi-channel encoder and still maintain the possibility of multi-channel decoding without influences of the post-processing.
  • This object is achieved by a method and a device for processing a stereo signal obtained from an encoder, which encoder encodes an N-channel (N>2) signal into left and a right signals and spatial parameters.
  • the method comprises processing of said left and right channel signals in order to provide a processed stereo signal.
  • the processing is controlled in dependence of said spatial parameters.
  • the general idea is to use the spatial parameters obtained from an N-channel-to-stereo coder to control a certain post-processing algorithm. In this way, the stereo signal obtained from the encoder may be processed, for example for enhancing the spatial impression.
  • the processing is controlled by a first parameter for each input channel, i.e. for each of the left and right signals, which first parameter is dependent on the spatial parameters.
  • the first parameter may be a function of time and/or frequency.
  • the system may have a variable amount of post-processing of which the actual amount of post-processing depends on the spatial parameters.
  • the post-processing may be performed individually in different frequency bands.
  • the encoder delivers independent spatial parameters describing the spatial image for a set of frequency bands. In that case, the first parameter may be frequency-dependent.
  • the post-processing comprises adding a first, second and third signal in order to obtain said processed channel signals.
  • the first signal includes the first input signal, i.e. the left or right signal, modified by a first transfer function
  • the second signal includes the first input signal modified by a second transfer function
  • the third signal includes the second input signal, i.e. the right or left signal, modified by a third transfer function.
  • the second transfer function may comprise said first parameter and a first filter function.
  • the first transfer function may comprise a second parameter, whereby the sum of said first parameter and said second parameter can be unity.
  • the third transfer function may comprise said first parameter of the second input signal and a second filter function.
  • the filter functions may be time-invariant.
  • the filtering effect of the filter functions H 1 , H 2 , H 3 and H 4 is variable by varying the parameters w l and w r . If both parameters have values equal to zero, the post-processed signals L 0w , R 0w are essentially equal to the stereo input signal pair L 0 , R 0 . On the other hand, if the parameters are +1, the post-processed stereo pair L 0w , R 0w is fully processed by the filter functions H 1 , H 2 , H 3 and H 4 .
  • This invention makes possible to control the actual amount of filtering, i.e., the value of the parameters w 1 and w r by the spatial parameters P.
  • the filter functions and parameters are selected so that the transfer function matrix is invertible. This makes reconstruction of the original stereo signal possible.
  • it comprises a device for processing a stereo signal in accordance with the above mentioned methods, and an encoder apparatus comprising such a device.
  • an audio system comprising such an encoder apparatus and such a decoder apparatus.
  • Fig. 1 is a block diagram of an encoder/decoder system in which the present invention is intended to be used.
  • an N-channel audio signal is supplied to an encoder 2, with N being an integer which is larger than 2.
  • the encoder 2 transforms the N-channel audio signals to signals L 0 and R 0 and parametric decoder information P, by means of which a decoder can decode the information and estimate the original N-channel signals to be output from the decoder.
  • the spatial parameter set P is preferably time and/or frequency dependent.
  • the N-channel signals may be signals for a 5.1 system, comprising a center channel, two front channels, two surround channels and an LFE channel.
  • the encoded stereo signal pair L 0 and R 0 and decoder spatial information P are transmitted to the user in a suitable way, such as by CD, DVD, VHS H-i-Fi, broadcast, laser disc, DBS, digital cable, Internet or any other transmission or distribution system, indicated by the circle line 4 in Fig. 1 . Since the left and right signals are transmitted, the system is compatible with the vast number of receiving equipment that can only reproduce stereo signals. If the receiving equipment includes a decoder, the decoder may decode the N-channel signals and provide an estimate thereof, based on the information in the stereo signal pair L 0 and R 0 as well as the decoder spatial information signals or spatial parameters P.
  • a post-processor 5 which processes the stereo signal prior to the transmission/distribution to the receiver.
  • the post-processing may be position-dependent "addition" of bass or reverberation, or removal of vocals (karaoke with vocals in center channel).
  • stereo-base-widening may be performed by making use of the knowledge of the composition of the original surround mix, such as front/back, since the contribution of individual input signals is known from the decoder information signals P.
  • stereo widening can be applied already in the encoder, but this is generally not invertible, since only two signals are available in the decoder, instead ofN, inversion is generally impossible.
  • stereo widening also other post-processing techniques on the individual multi-channel contributions are possible.
  • the post-processed signals are transmitted to a receiver as indicated by the circle 6 in Figure 1 .
  • the inventive device for processing a stereo signal obtained from an encoder comprises the post-processor 5.
  • the encoder apparatus according to the present invention comprises the encoder 2 and the post-processor 5.
  • the signal received may be used directly, for example if the receiver does not include a multi-channel decoder. This may be the case in a computer receiving the signal 6 over the Internet, or in a receiver having only two loudspeakers. Such received signal is perceived as a high quality signal, since it has improved spatial impression or other characteristics as determined in the processing thereof by the encoder and the post-processor.
  • the signal should be used for decoding in a conventional N-channel decoder 3, it must first be inverse post-processed by an inverse post-processor 7, in order to reconstruct the original stereo signal pair L 0 and R 0 which together with the decoder information or spatial parameters P, produces an estimated N-channel signal.
  • an inverse post-processor 7 in order to reconstruct the original stereo signal pair L 0 and R 0 which together with the decoder information or spatial parameters P, produces an estimated N-channel signal.
  • Such reconstruction is possible of the multi-channel mix, which reconstruction is hardly affected by the post-processing.
  • post-processing in the decoder is possible for stereo playback as a user-selectable feature, without the necessity to determine the multi-channel signal first.
  • the inventive device for processing a stereo signal comprising left and right signals comprises the inverse post-processor 7.
  • the decoder apparatus according to the present invention comprises the decoder 3 and the inverse post-processor 7.
  • the down-mix is comparable with a standard ITU down-mix.
  • the inventive method may improve the down-mix significantly.
  • the inventive method is able to determine the contribution in the down-mix of the original channels in the multi-channel mix with the help of the determined spatial parameters P in the encoder.
  • post-processing can be applied to specific channels of the multi-channel mix, for example stereo-base-widening of the rear channels, whilst the other channels are not affected.
  • the post-processing does not affect the final multi-channel reconstruction if the post-processing is invertible. It can also be applied for an improved stereo playback without the necessity to reconstruct the multi-channel mix first.
  • This method differs from existing post-processing techniques in that it uses the knowledge of the original multi-channel mix, i.e. the determined spatial parameters P.
  • the encoder 2 operates in the following way:
  • ⁇ i and ⁇ i are chosen such that the stereo signal consisting of L 0 [k] and R 0 [k] has a good stereo image.
  • L k L f k + L s k / 2
  • R k R f k + R s k / 2
  • spatial parameters P are extracted to enable perceptual reconstruction of the signals L f , R f , C, L s and R s from L 0 and R 0 .
  • the parameter set P includes inter-channel intensity differences (IIDs) and possibly inter-channel cross-correlation (ICCs) values between the signal pairs (L f , L s ) and (R f , R s ).
  • IIDs inter-channel intensity differences
  • ICCs inter-channel cross-correlation
  • (*) denotes the complex conjugation.
  • the parameter IID l describes the relative amount of energy between the left-front and left-surround channels and the parameter ICC l describes the amount of mutual correlation between the left-front and left-surround channels.
  • M c 1 c 2 - 1 c 1 - 1 c 2 1 - c 1 1 - c 2
  • the parameter set P includes ⁇ c 1 , c 2 , IID l , ICC l , IID r , ICC r ⁇ for each time/frequency tile.
  • post-processing can be applied in a way that it mainly affects the contribution of Z i [k], for example L s and R s in the stereo mix.
  • Z i [k] for example L s and R s in the stereo mix.
  • Fig. 1 the position of this block in the codec is shown.
  • Fig. 2 is a detailed view of the post-processor 5 in Fig. 1 according to an embodiment of the invention.
  • the post-processed left signal L 0w is the sum of three signals, namely the left signal L 0 modified by a transfer function H A , the left signal L 0 modified by a transfer function H B and the right signal R 0 modified by a transfer function H D .
  • the post-processed right signal R 0w is the sum of three signals, namely the right signal R 0 modified by a transfer function H F , the right signal R 0 modified by a transfer function HE and the left signal L 0 modified by a transfer function H C .
  • the transfer functions H A - H F may be implemented as FIR or IIR-type filters, or can simply be (complex) scale factors which may be frequency dependent. Furthermore, the transfer function H A may be a multiplication with a second parameter (1-w l ) and transfer function H B may include a first parameter w l whereby this parameter w l determines the amount of post-processing of the stereo signal.
  • the parameter w l determines the amount of post-processing of L 0 [k] and w r of R 0 [k].
  • L 0 [k] is unaffected, and when w l is equal to 1, L 0 [k] is maximally affected.
  • w r with respect to R 0 [k].
  • the blocks H 1 , H 2 , H 3 and H 4 in Fig. 3 are filter functions, which can be various types of filters, for example stereo widening filters, as shown below.
  • the transfer function matrix H can be inverted.
  • the filter functions H 1 , H 2 , H 3 and H 4 and parameters w l and w r should be known at the decoder. This is possible since w l and w r can be calculated from the transmitted parameters. Thus, the original stereo signal L 0 , R 0 will be available again which is necessary for decoding of the multi-channel mix.
  • Another possibility is to transmit the original stereo signal and apply the post-processing in the decoder to make improved stereo playback possible without the necessity to determine the multi-channel mix first.
  • w l f 1 c 1 ⁇ f 2 IID l
  • This invention can be integrated in a multi-channel audio encoder apparatus that creates a stereo-compatible down-mix.
  • the general scheme of such a multi-channel parametric audio encoder which is enhanced by the post-processing scheme as described above can be outlined as follows:
  • a corresponding multi-channel decoder apparatus i.e., a decoder with integrated post-processing inversion
  • a decoder with integrated post-processing inversion can be outlined as follows:
  • the filter functions H 1 to H 4 are preferably converted or approximated in the frequency domain by simple (real-valued or complex) scale factors, which may be frequency dependent.
  • Another application of the invention is to apply the post-processing on the stereo signal at the decoder-side only (i.e., without post-processing at the encoder side).
  • the decoder can generate an enhanced stereo signal from a non-enhanced stereo signal.
  • Extra information can be provided in the bit-stream which signals whether or not the post-processing has been done and the parameter functions f 1 , f 2 and which filter functions H 1 , H 2 , H 3 , and H 4 have been used, which enables inverse post-processing.
  • a filter function may be described as a multiplication in the frequency domain. Since parameters are present for individual frequency bands, the invention may be implemented as simple, complex gains instead of filters, which are applied individually in different frequency bands.
  • frequency bands of L 0w , R 0w are obtained by a simple (2x2) matrix multiplication from corresponding frequency bands from (L 0 ,R 0 ).
  • the actual matrix entries are determined by the parameters and frequency domain representations of the filter functions H thus consisting of the time-invariant gains H and a time/frequency-variant parameter-controlled gains w l and w r . Because the filters are scalars for each band, inversion is possible.
  • the matrix H contains of all scalars.
  • the use of scalars makes post-processing and the inverse post-processing relatively easy.
  • the parameters w l and w r are scalars and functions of the parameter set P. These 2 parameters determine the amount of post-processing of the input channels.
  • the parameters H 1 ??H 4 are complex filter functions.
  • the matrix H -1 contains only scalars.
  • the elements of H -1 , k 1 ?? k 4 are also functions of the parameter set P.
  • the post-processing can be inverted.
  • FIG. 4 A block diagram of an inverse post-processor 3 which performs such inverse post-processing is illustrated in Figure 4 .
  • det(H) When suitable functions h 11 ?? h 22 are chosen, det(H) will be unequal zero, so the process is invertable.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Mathematical Physics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Stereophonic System (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Claims (20)

  1. Procédé de traitement d'un signal stéréo qui est obtenu à partir d'un codeur, lequel codeur code un signal audio du type à canal N en des signaux gauche et droit (L0 ; R0) et des paramètres spatiaux (P), le procédé étant caractérisé en ce qu'il comprend :
    - le traitement desdits signaux gauche et droit afin de fournir un signal stéréo traité (L0w ; R0w), dans lequel ledit traitement est commandé en fonction desdits paramètres spatiaux (P).
  2. Procédé selon la revendication 1, dans lequel ledit traitement est commandé par un premier paramètre (wl ; wr) pour chacun desdits signaux gauche et droit, ledit premier paramètre étant dépendant des paramètres spatiaux (P).
  3. Procédé selon la revendication 2, dans lequel ledit premier paramètre (wl ; wr) est une fonction du temps et/ou de la fréquence.
  4. Procédé selon la revendication 1, 2 ou 3, dans lequel ledit traitement comprend le filtrage d'au moins un desdits signaux gauche et droit avec une fonction de transfert qui dépend des paramètres spatiaux (P).
  5. Procédé selon la revendication 1, 2, 3 ou 4, dans lequel ledit traitement comprend l'étape suivante consistant à :
    - ajouter un premier, un deuxième et un troisième signal afin d'obtenir lesdits signaux de canal traités (L0w ; R0w), dans lequel le premier signal comprend le signal stéréo d'un canal qui est modifié par une première fonction de transfert (L0*HA ; R0*HF), le deuxième signal comprend le signal stéréo du même canal qui est modifié par une deuxième fonction de transfert (L0*HB ; R0*HE) et le troisième signal comprend le signal stéréo de l'autre canal qui est modifié par une troisième fonction de transfert (R0*HD ; L0*HC).
  6. Procédé selon la revendication 5, dans lequel ladite deuxième fonction de transfert (HB ; HE) comprend une multiplication par ledit premier paramètre (W1 ; Wr) qui est suivie d'une multiplication par une première fonction de filtre (H1 ; H4).
  7. Procédé selon la revendication 5, dans lequel ladite première fonction de transfert (HA ; HF) comprend une multiplication par un deuxième paramètre.
  8. Procédé selon la revendication 5, dans lequel ladite première fonction de transfert (HA ; HF) comprend une multiplication par un deuxième paramètre où ledit premier paramètre est une fonction dudit deuxième paramètre.
  9. Procédé selon la revendication 5, 6, 7 ou 8, dans lequel ladite troisième fonction de transfert (HC ; HD) comprend une multiplication du signal gauche ou droit (L0 ; R0) par ledit premier paramètre (W1 ; Wr) qui est suivie d'une deuxième fonction de filtre (H2 ; H3).
  10. Procédé selon la revendication 6, 7, 8 ou 9, dans lequel lesdites fonctions de filtre (H1, H2, H3, H4) sont invariantes dans le temps.
  11. Procédé selon l'une quelconque des revendications précédentes 1 à 10, dans lequel lesdits signaux sont décrits par l'équation : L 0 ⁢ w R 0 ⁢ w = H L 0 R 0
    Figure imgb0031
    dans laquelle la matrice de fonction de transfert (H) est une fonction des paramètres spatiaux (P).
  12. Procédé selon la revendication 11, dans lequel ladite matrice de fonction de transfert (H) est décrite par l'équation : H = 1 - w l a + w l a ⁢ H 1 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ w r a ⁢ H 3 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ w l a ⁢ H 2 ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ 1 - w r a + w r a ⁢ H 4
    Figure imgb0032
    avec a étant une constante.
  13. Procédé selon la revendication 12, dans lequel lesdites fonctions de filtre (H1, H2, H3, H4) et lesdits paramètres (w1 ; wr) sont sélectionnés de telle façon que la matrice de fonction de transfert (H) soit inversible.
  14. Procédé selon l'une quelconque des revendications précédentes 1 à 13, dans lequel lesdits paramètres spatiaux (P) contiennent de l'information qui décrit des niveaux de signal du signal du type à canal N.
  15. Dispositif étant destiné à traiter un signal stéréo qui est obtenu à partir d'un codeur, lequel codeur code un signal audio du type à canal N en des signaux gauche et droit (L0 ; R0) et des paramètres spatiaux (P), le dispositif étant caractérisé en ce qu'il comprend :
    - un post-processeur (5) pour le traitement ultérieur desdits signaux gauche et droit afin de fournir un signal stéréo traité (L0w; R0w) où ledit traitement ultérieur est commandé en fonction desdits paramètres spatiaux (P).
  16. Appareil de codage comprenant :
    - un codeur (2) pour coder un signal audio du type à canal N en des signaux gauche et droit (L0 ; R0) et des paramètres spatiaux (P) ; et
    - un dispositif (5) selon la revendication 15 pour traiter lesdits signaux gauche et droit (L0 ; R0) en fonction desdits paramètres spatiaux (P).
  17. Appareil de décodage comprenant :
    - un dispositif (7) qui est destiné à recevoir des signaux gauche et droit traités (L0w; R0w) et des paramètres spatiaux, les signaux gauche et droit traités (L0w; R0w) étant des signaux gauche et droit (L0 ; R0) qui sont traités en fonction des paramètres spatiaux, les signaux gauche et droit (L0 ; R0) et les paramètres spatiaux représentant un codage d'un signal audio du type à canal N ;
    - des moyens pour traiter les signaux gauche et droit traités (L0w; R0w) en réponse aux paramètres spatiaux de manière à générer des signaux gauche et droit de décodage (L0 ; R0) ; et
    - un décodeur qui est destiné à décoder les signaux gauche et droit de décodage (L0 ; R0) en un signal audio du type à canal N.
  18. Appareil de décodage selon la revendication 17, dans lequel les moyens pour le traitement sont agencés de manière à inverser le traitement des signaux gauche et droit (L0 ; R0) afin de générer les signaux gauche et droit traités (L0w; R0w).
  19. Procédé de décodage comprenant les étapes suivantes consistant à :
    - recevoir des signaux gauche et droit traités (L0w; R0w) et des paramètres spatiaux, les signaux gauche et droit traités (L0w; R0w) étant des signaux gauche et droit (L0 ; R0) qui sont traités en fonction des paramètres spatiaux, les signaux gauche et droit (L0 ; R0) et les paramètres spatiaux représentant un codage d'un signal audio du type à canal N ;
    - traiter les signaux gauche et droit traités (L0w; R0w) en réponse aux paramètres spatiaux de manière à générer des signaux gauche et droit de décodage (L0 ; R0) ; et
    - décoder les signaux gauche et droit de décodage (L0 ; R0) en un signal audio du type à canal N.
  20. Système audio (1) comprenant un appareil de codage selon la revendication 16 et un appareil de décodage selon la revendication 17.
EP05718592.8A 2004-04-05 2005-03-30 Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé Active EP1735779B1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP05718592.8A EP1735779B1 (fr) 2004-04-05 2005-03-30 Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé
PL05718592T PL1735779T3 (pl) 2004-04-05 2005-03-30 Urządzenie kodujące, dekodujące, sposoby z nimi powiązane oraz powiązany system audio

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP04101405 2004-04-05
EP04103367 2004-07-14
EP05718592.8A EP1735779B1 (fr) 2004-04-05 2005-03-30 Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé
PCT/IB2005/051065 WO2005098826A1 (fr) 2004-04-05 2005-03-30 Procede, dispositif, appareil de codage, appareil de decodage et systeme audio

Publications (2)

Publication Number Publication Date
EP1735779A1 EP1735779A1 (fr) 2006-12-27
EP1735779B1 true EP1735779B1 (fr) 2013-06-19

Family

ID=34962191

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05718592.8A Active EP1735779B1 (fr) 2004-04-05 2005-03-30 Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé

Country Status (12)

Country Link
US (1) US9992599B2 (fr)
EP (1) EP1735779B1 (fr)
JP (1) JP5284638B2 (fr)
KR (1) KR101183862B1 (fr)
CN (1) CN1947172B (fr)
BR (1) BRPI0509110B1 (fr)
ES (1) ES2426917T3 (fr)
MX (1) MXPA06011397A (fr)
PL (1) PL1735779T3 (fr)
RU (1) RU2396608C2 (fr)
TW (1) TWI455614B (fr)
WO (1) WO2005098826A1 (fr)

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ES2373728T3 (es) 2004-07-14 2012-02-08 Koninklijke Philips Electronics N.V. Método, dispositivo, aparato codificador, aparato decodificador y sistema de audio.
JP4988716B2 (ja) 2005-05-26 2012-08-01 エルジー エレクトロニクス インコーポレイティド オーディオ信号のデコーディング方法及び装置
EP1905002B1 (fr) 2005-05-26 2013-05-22 LG Electronics Inc. Procede et appareil de decodage d'un signal audio
WO2007007263A2 (fr) * 2005-07-14 2007-01-18 Koninklijke Philips Electronics N.V. Codage et decodage audio
US8626503B2 (en) 2005-07-14 2014-01-07 Erik Gosuinus Petrus Schuijers Audio encoding and decoding
JP5587551B2 (ja) * 2005-09-13 2014-09-10 コーニンクレッカ フィリップス エヌ ヴェ オーディオ符号化
KR100803212B1 (ko) * 2006-01-11 2008-02-14 삼성전자주식회사 스케일러블 채널 복호화 방법 및 장치
EP1974347B1 (fr) * 2006-01-19 2014-08-06 LG Electronics Inc. Procede et appareil pour traiter un signal multimedia
WO2007091850A1 (fr) 2006-02-07 2007-08-16 Lg Electronics Inc. Appareil et procédé de codage/décodage de signal
CN101390443B (zh) * 2006-02-21 2010-12-01 皇家飞利浦电子股份有限公司 音频编码和解码
WO2008046530A2 (fr) 2006-10-16 2008-04-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Appareil et procédé de transformation de paramètres de canaux multiples
EP2054875B1 (fr) 2006-10-16 2011-03-23 Dolby Sweden AB Codage amélioré et représentation de paramètres d'un codage d'objet à mélange abaisseur multi-canal
WO2008063035A1 (fr) * 2006-11-24 2008-05-29 Lg Electronics Inc. Procédé permettant de coder et de décoder des signaux audio basés sur des objets et appareil associé
US8855795B2 (en) 2007-01-09 2014-10-07 Mediatek Inc. Multiple output audio system
US8615316B2 (en) 2008-01-23 2013-12-24 Lg Electronics Inc. Method and an apparatus for processing an audio signal
EP2083584B1 (fr) 2008-01-23 2010-09-15 LG Electronics Inc. Procédé et appareil de traitement de signal audio
KR100998913B1 (ko) * 2008-01-23 2010-12-08 엘지전자 주식회사 오디오 신호의 처리 방법 및 이의 장치
EP2175670A1 (fr) * 2008-10-07 2010-04-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Rendu binaural de signal audio multicanaux
JP5511848B2 (ja) 2009-12-28 2014-06-04 パナソニック株式会社 音声符号化装置および音声符号化方法
CN102280107B (zh) * 2010-06-10 2013-01-23 华为技术有限公司 边带残差信号生成方法及装置
WO2012040898A1 (fr) * 2010-09-28 2012-04-05 Huawei Technologies Co., Ltd. Dispositif et procédé pour post-traiter un signal audio multicanal ou un signal stéréo décodé
ES2643163T3 (es) 2010-12-03 2017-11-21 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Aparato y procedimiento para codificación de audio espacial basada en geometría
CN103329565B (zh) * 2011-01-05 2016-09-28 皇家飞利浦电子股份有限公司 音频系统及其操作方法
EP2804176A1 (fr) 2013-05-13 2014-11-19 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Séparation d'un objet audio d'un signal de mélange utilisant des résolutions de temps/fréquence spécifiques à l'objet
EP2830046A1 (fr) 2013-07-22 2015-01-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Appareil et procédé permettant de décoder un signal audio codé pour obtenir des signaux de sortie modifiés
US9820073B1 (en) 2017-05-10 2017-11-14 Tls Corp. Extracting a common signal from multiple audio signals

Family Cites Families (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4095049A (en) * 1976-03-15 1978-06-13 National Research Development Corporation Non-rotationally-symmetric surround-sound encoding system
US4236039A (en) * 1976-07-19 1980-11-25 National Research Development Corporation Signal matrixing for directional reproduction of sound
DE4209544A1 (de) * 1992-03-24 1993-09-30 Inst Rundfunktechnik Gmbh Verfahren zum Übertragen oder Speichern digitalisierter, mehrkanaliger Tonsignale
JP2693893B2 (ja) * 1992-03-30 1997-12-24 松下電器産業株式会社 ステレオ音声符号化方法
JPH06165079A (ja) * 1992-11-25 1994-06-10 Matsushita Electric Ind Co Ltd マルチチャンネルステレオ用ダウンミキシング装置
DE4409368A1 (de) * 1994-03-18 1995-09-21 Fraunhofer Ges Forschung Verfahren zum Codieren mehrerer Audiosignale
US5727119A (en) * 1995-03-27 1998-03-10 Dolby Laboratories Licensing Corporation Method and apparatus for efficient implementation of single-sideband filter banks providing accurate measures of spectral magnitude and phase
US5642423A (en) 1995-11-22 1997-06-24 Sony Corporation Digital surround sound processor
US6697491B1 (en) 1996-07-19 2004-02-24 Harman International Industries, Incorporated 5-2-5 matrix encoder and decoder system
SG54379A1 (en) 1996-10-24 1998-11-16 Sgs Thomson Microelectronics A Audio decoder with an adaptive frequency domain downmixer
EP0990368B1 (fr) 1997-05-08 2002-04-24 STMicroelectronics Asia Pacific Pte Ltd. Procede et appareil d'abaissement du domaine frequentiel a forcage de commutation de blocs pour fonctions de decodage audio
US6173061B1 (en) * 1997-06-23 2001-01-09 Harman International Industries, Inc. Steering of monaural sources of sound using head related transfer functions
US6067361A (en) * 1997-07-16 2000-05-23 Sony Corporation Method and apparatus for two channels of sound having directional cues
US7292901B2 (en) * 2002-06-24 2007-11-06 Agere Systems Inc. Hybrid multi-channel/cue coding/decoding of audio signals
SE0202159D0 (sv) * 2001-07-10 2002-07-09 Coding Technologies Sweden Ab Efficientand scalable parametric stereo coding for low bitrate applications
US7039204B2 (en) * 2002-06-24 2006-05-02 Agere Systems Inc. Equalization for audio mixing
JP4322207B2 (ja) 2002-07-12 2009-08-26 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ オーディオ符号化方法
US7447317B2 (en) * 2003-10-02 2008-11-04 Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V Compatible multi-channel coding/decoding by weighting the downmix channel
US7394903B2 (en) * 2004-01-20 2008-07-01 Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. Apparatus and method for constructing a multi-channel output signal or for generating a downmix signal
JPWO2005081229A1 (ja) * 2004-02-25 2007-10-25 松下電器産業株式会社 オーディオエンコーダ及びオーディオデコーダ
US7805313B2 (en) * 2004-03-04 2010-09-28 Agere Systems Inc. Frequency-based coding of channels in parametric multi-channel coding systems
US20050247756A1 (en) 2004-03-31 2005-11-10 Frazer James T Connection mechanism and method
CN102122509B (zh) 2004-04-05 2016-03-23 皇家飞利浦电子股份有限公司 多信道解码器和多信道解码方法
ES2373728T3 (es) * 2004-07-14 2012-02-08 Koninklijke Philips Electronics N.V. Método, dispositivo, aparato codificador, aparato decodificador y sistema de audio.

Also Published As

Publication number Publication date
RU2006139068A (ru) 2008-05-20
BRPI0509110A8 (pt) 2016-02-10
MXPA06011397A (es) 2006-12-20
TW200611588A (en) 2006-04-01
JP2007531916A (ja) 2007-11-08
PL1735779T3 (pl) 2014-01-31
ES2426917T3 (es) 2013-10-25
US9992599B2 (en) 2018-06-05
CN1947172A (zh) 2007-04-11
KR20070001205A (ko) 2007-01-03
BRPI0509110A (pt) 2007-08-28
EP1735779A1 (fr) 2006-12-27
CN1947172B (zh) 2011-08-03
BRPI0509110B1 (pt) 2019-07-09
TWI455614B (zh) 2014-10-01
JP5284638B2 (ja) 2013-09-11
US20070183601A1 (en) 2007-08-09
WO2005098826A1 (fr) 2005-10-20
RU2396608C2 (ru) 2010-08-10
KR101183862B1 (ko) 2012-09-20

Similar Documents

Publication Publication Date Title
EP1735779B1 (fr) Appareil de codage, appareil de decodage, procédés correspondants et systeme audio associé
EP1769655B1 (fr) Methode, dispositif, appareil de codage, appareil de decodage et systeme audio
EP1999747B1 (fr) Decodage audio
EP1999999B1 (fr) Procédé de production de mixages réducteurs spatiaux à partir de représentations paramétriques de signaux multicanal
EP2320414B1 (fr) Codage paramétrique combiné de sources audio
EP1866913B1 (fr) Codage et decodage audio
KR101236259B1 (ko) 오디오 채널들을 인코딩하는 방법 및 장치
EP2495722A1 (fr) Procédé, support et système de synthèse d'un signal stéréo
KR20050021484A (ko) 오디오 코딩
CN104246873A (zh) 用于编码多声道音频信号的参数编码器
MX2008011994A (es) Generacion de mezclas descendentes espaciales a partir de representaciones parametricas de señales de multicanal.

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20061106

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

17Q First examination report despatched

Effective date: 20070404

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 618046

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130715

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602005040044

Country of ref document: DE

Effective date: 20130808

REG Reference to a national code

Ref country code: CH

Ref legal event code: PFA

Owner name: KONINKLIJKE PHILIPS N.V., NL

Free format text: FORMER OWNER: KONINKLIJKE PHILIPS ELECTRONICS N.V., NL

RAP2 Party data changed (patent owner data changed or rights of a patent transferred)

Owner name: KONINKLIJKE PHILIPS N.V.

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2426917

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20131025

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130920

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 618046

Country of ref document: AT

Kind code of ref document: T

Effective date: 20130619

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130919

REG Reference to a national code

Ref country code: NL

Ref legal event code: VDEP

Effective date: 20130619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131019

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20131021

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130717

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

REG Reference to a national code

Ref country code: PL

Ref legal event code: T3

REG Reference to a national code

Ref country code: ES

Ref legal event code: PC2A

Owner name: KONINKLIJKE PHILIPS N.V.

Effective date: 20140220

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005040044

Country of ref document: DE

Representative=s name: MEISSNER, BOLTE & PARTNER GBR, DE

REG Reference to a national code

Ref country code: ES

Ref legal event code: GC2A

Effective date: 20140403

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

REG Reference to a national code

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005040044

Country of ref document: DE

Representative=s name: MEISSNER BOLTE PATENTANWAELTE RECHTSANWAELTE P, DE

Effective date: 20140402

Ref country code: DE

Ref legal event code: R082

Ref document number: 602005040044

Country of ref document: DE

Representative=s name: MEISSNER, BOLTE & PARTNER GBR, DE

Effective date: 20140402

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005040044

Country of ref document: DE

Owner name: KONINKLIJKE PHILIPS N.V., NL

Free format text: FORMER OWNER: KONINKLIJKE PHILIPS ELECTRONICS N.V., EINDHOVEN, NL

Effective date: 20130620

Ref country code: DE

Ref legal event code: R081

Ref document number: 602005040044

Country of ref document: DE

Owner name: KONINKLIJKE PHILIPS N.V., NL

Free format text: FORMER OWNER: KONINKLIJKE PHILIPS ELECTRONICS N.V., EINDHOVEN, NL

Effective date: 20140402

26N No opposition filed

Effective date: 20140320

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602005040044

Country of ref document: DE

Effective date: 20140320

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20140330

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140331

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140330

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20130619

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20050330

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160330

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160330

PGRI Patent reinstated in contracting state [announced from national office to epo]

Ref country code: IT

Effective date: 20170710

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: DE

Ref legal event code: R084

Ref document number: 602005040044

Country of ref document: DE

REG Reference to a national code

Ref country code: GB

Ref legal event code: 746

Effective date: 20180903

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230602

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20230424

Year of fee payment: 19

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240328

Year of fee payment: 20

Ref country code: GB

Payment date: 20240319

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: TR

Payment date: 20240319

Year of fee payment: 20

Ref country code: PL

Payment date: 20240320

Year of fee payment: 20

Ref country code: IT

Payment date: 20240321

Year of fee payment: 20

Ref country code: FR

Payment date: 20240326

Year of fee payment: 20