EP2691952B1 - Zuweisung von bits anhand von subbändern zur quantifizierung von rauminformationsparametern für parametrische codierung - Google Patents

Zuweisung von bits anhand von subbändern zur quantifizierung von rauminformationsparametern für parametrische codierung Download PDF

Info

Publication number
EP2691952B1
EP2691952B1 EP12717796.2A EP12717796A EP2691952B1 EP 2691952 B1 EP2691952 B1 EP 2691952B1 EP 12717796 A EP12717796 A EP 12717796A EP 2691952 B1 EP2691952 B1 EP 2691952B1
Authority
EP
European Patent Office
Prior art keywords
sub
band
bits
bands
allocated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12717796.2A
Other languages
English (en)
French (fr)
Other versions
EP2691952A1 (de
Inventor
Adrien Daniel
Rozenn Nicol
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Orange SA
Original Assignee
Orange SA
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Orange SA filed Critical Orange SA
Publication of EP2691952A1 publication Critical patent/EP2691952A1/de
Application granted granted Critical
Publication of EP2691952B1 publication Critical patent/EP2691952B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/002—Dynamic bit allocation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition

Definitions

  • the present invention relates to the coding of multichannel audio streams representing spatialized sound scenes for the purpose of storage or transmission.
  • It relates more particularly to the parametric coding / decoding of multichannel audio streams.
  • This type of coding is based on the coding of a signal resulting from a channel reduction processing ("downmix" in English) of the multichannel audio stream and the associated coding of parameters of spatial information of the sound sources.
  • the spatial information parameters are used to find the spatialization of the sound sources from the “downmix” signal which will be called hereinafter, sum signal.
  • the invention relates more particularly to the coding and decoding of these spatial information parameters.
  • the bit budget available according to the coders is not always sufficient. In the case of coding by frequency sub-band, this budget is divided by sub-bands.
  • Another technique is to perform intra or inter-frame differential coding.
  • a quantification based on psycho-acoustic criteria is proposed by Breebaart in the document of Breebaart, J; Van de Par, S; Kohlrausch, A & Schuijers, E, "Parametric Coding of stereo Audio” in EURASIP Journal on Applied Signal Processing, 2005,9, pp 1305-1322 .
  • the method described in this document is based on the perception that a listener can have on certain frequency bands for particular parameters of the inter-channel difference type, or on the sensitivity to a variation of these parameters as a function of the range of values. concerned. It is for example described that certain parameters are coded only on the frequency bands lower than 1 kHz. Beyond this frequency, the parameters are no longer useful for the hearing system to locate a source.
  • the psycho-acoustic criterion used here relates to a sensitivity to the coded parameters and not to a sensitivity of spatial displacements of the sound sources.
  • the auditory perception or the sensitivity with respect to a spatial resolution in the sub-bands can vary at each instant from one sub-band to another, independently of the parameter to be coded.
  • the method according to the invention uses a psycho-acoustic criterion to optimize the allocation strategy for the quantization bits of the spatial information parameters as a function of the sub-band, so as to favor the sub-bands at all times. which are most useful for the hearing system, regardless of the spatial information parameters to be coded or decoded.
  • the spatial resolution properties of the auditory system are thus exploited.
  • the spatial resolution in a sub-band can be defined as the smallest angle between two sources, which the hearing system is able to discriminate.
  • the spatial resolution associated with a sub-band is inversely proportional to the energy in this sub-band.
  • the energy in a sub-band is strong, this already gives an indication of the little influence that the other sub-bands can have with respect to it and thus gives a first dynamic allocation approach ( taking into account the other sub-bands).
  • the energy properties can correspond to the energy measured in the sub-band or more precisely to a measurement of the energy distance of this sub-band at its masking / audibility threshold.
  • the spectral properties of a sub-band are both energy properties in the sub-band and the central frequency of the sub-band.
  • the spatial resolution of a sub-band is further estimated from the energy properties of the other sub-bands of a set of sub-bands defining the sound sources.
  • the other sub-bands can be considered as competing distractive sources which are liable to degrade the spatial sensitivity associated with this sub-band.
  • the spectral properties of the other frequency sub-bands makes it possible to estimate this degradation and to predict the spatial resolution associated with the sub-band.
  • This taking into account makes it possible to dynamically define with what precision the spatialization information associated with each sub-band must be coded, on the basis of a decrease or an increase in the spatial resolution.
  • the resulting quantization error is adapted as a function of the spatial sensitivity in order to minimize the error when the sensitivity is maximum, and conversely to maximize it when the sensitivity is minimum.
  • the quantification error is thus, from a perceptual point of view, homogeneously minimized.
  • the spectral properties of a sub-band are obtained from a decoded sum signal originating from a reduction processing of the channels of the multi-channel audio stream.
  • the estimation of the spatial resolution by sub-band does not require information of the position type of the sound sources but only information on the spectral properties of the sub-bands. This information can therefore be obtained from the sum signal decoded either locally in an encoder at the coding stage or decoded by the decoder itself at the decoding stage. It is therefore not necessary to send additional information to the decoder to find the allocation strategy for quantization bits. This greatly reduces the amount of information to be transmitted between the coder and the decoder.
  • the energy properties in a sub-band include the primary energy and ambient energy properties in the sub-band.
  • the share of correlated energy (primary energy) between the different channels of the multichannel signal is differentiated from that which is not correlated (ambient) in the psycho-acoustic model allowing the spatial resolution to be estimated.
  • the estimation of the spatial resolution is more precise and closer to reality.
  • the number of bits to be allocated for a sub-band is part of a predetermined number of bits to be distributed between the sub-bands, adding to a number of bits already allocated by sub-bands .
  • the allocation defined here applies to a number of bits remaining to be allocated in a quantization bit budget, a part of the quantification bits of the overall budget having already been distributed between the sub-bands.
  • the decoder it is possible to approximately decode the spatial information parameters from the quantization bits already allocated, the additional bit budget making it possible to refine the decoding and to adapt it to auditory perception.
  • the determination of the number of bits to be allocated for a sub-band is adjusted as a function of the difference between the resolution in this sub-band and a predetermined reference resolution, to which a bit allocation corresponds. of predetermined reference.
  • the method is implemented for a set of unmasked sub-bands determined by an energy masking analysis step between sub-bands.
  • the allocation method is implemented only for the audible, that is to say non-masked, sub-bands, which makes it possible to concentrate the budget of bits to be allocated on these sub-bands.
  • This provides a calculation gain since the method is not implemented in all the sub-bands and a transmission gain since the spatial information parameters associated with the masked sub-bands will not be transmitted (0 bits allocated).
  • This device has the same advantages as the method described above, which it implements.
  • the invention relates to an encoder or a decoder comprising such an allocation device.
  • It relates to a computer program comprising code instructions for the implementation of the steps of the allocation method as described, when these instructions are executed by a processor.
  • the invention relates to a storage medium, readable by a processor, integrated or not into the allocation device, possibly removable, memorizing a computer program implementing an allocation method as described above.
  • the figure 1 thus describes a parametric coding / decoding system for a multichannel audio stream.
  • This figure illustrates the encoder 100, the decoder 110 as well as the allocation device 120 according to an embodiment of the invention.
  • the channels x 1 ( n ) , x 2 ( n ) , ..., x n ( n ) of the multichannel audio stream are first transformed by a time / frequency transformation module 106, before being applied as an input to both a channel reduction processing module 101 or also a “Downmix” module and a module for extracting spatial information parameters 102.
  • the transformation operated by the module 106 can be of different types. It can for example use a filter bank technique, or a Short-Term Fourier Transform (TFCT) technique using an algorithm of FFT type (“Fast Fourier Transform” in English).
  • TFCT Short-Term Fourier Transform
  • the filters can be defined so that the resulting frequency sub-bands describe perceptual frequency scales, for example by choosing constant bandwidths in the ERB scales (for "Equivalent Rectangular Bandwidth" in English).
  • the same process can be applied in the case of a technique by TFCT by grouping the frequency bins of each time frame according to the ERB scales.
  • a “downmix” signal or sum signal, originating from the channel reduction processing module 101 (mono or stereo signal) is obtained by summation, optionally weighted, of the different channels in each sub-band.
  • This sum signal is then coded by a core coding module 103 which may be of different type, for example of MPEG-4 AAC standardized audio coding type.
  • This coded signal is then transmitted over the network to be subsequently decoded by the corresponding core decoder 113.
  • the module 102 extracts the spatial information parameters of the audio channels. These parameters are those which describe the spatial position of the channels. These parameters can for example be the pair of parameters ILD (for “Interaural Level Difference” in English) and IPD (for “Interaural Phase Difference” in English) as defined for the stereo parametric coding method described in the document of Breebaart, J; Van de Par, S; Kohlrausch, A & Schuijers, E, "Parametric Coding of stereo Audio” in EURASIP Journal on Applied Signal Processing, 2005,9, pp 1305-1322 .
  • ILD for “Interaural Level Difference” in English
  • IPD for “Interaural Phase Difference” in English
  • These parameters can, in another example, be of the type of primary and ambient position vectors as for the representation described in the document "Spatial audio scene coding” by Goodwin, M. & Jot, J., 125th AES Convention, 2008 October 2-5, San Francisco, USA, 2008 .
  • the spatial information parameters thus extracted are then quantified by the quantization module 104 according to an allocation of quantization bits defined by the allocation device 120.
  • the allocation device 120 implements an allocation method which will be described with reference to the figure 2 .
  • This allocation device 120 receives as input the sum signal decoded S sd by a local decoder 105 of the coder or in the case of the decoder, decoded by the decoding module 113.
  • a module 121 for estimating a spatial resolution by frequency sub-band determines the spectral properties of the frequency sub-bands.
  • the spectral properties determined are energy properties in the sub-band.
  • the spectral properties are both the energy properties and the center frequency in the subband.
  • This spatial resolution corresponds to the smallest angle between two sources that the human hearing system can discriminate.
  • This spatial resolution can also be called MAA (for “Minimum Audible Angle” in English) as defined by the document of Mills AW "On the Minimum Audible Angle” in The Journal of the Acoustical Society of America, 83 (S1): S122, May 1988 .
  • the spatial resolution by frequency sub-band thus determined makes it possible to determine a number of bits to be allocated to the sub-band for the quantification of the spatial information parameters.
  • This step is implemented by the module 122 for determining the number of bits. This step will be explained in more detail with reference to the figure 2 .
  • This allocation of the number of bits per frequency sub-band is then based on psycho-acoustic and not purely mathematical considerations as was done previously in the state of the art. Thus, this allocation takes into account the perception of the hearing system in the frequency bands.
  • the quantification errors of the spatial parameters result in changes in the position of the sound sources at the time of decoding. These changes in position induce a spatial distortion of the sound scene which, evolving over time, results in spatial instability. Spatial resolution can be interpreted as a sensitivity to this spatial distortion. This sensitivity can be expressed for each sub-band by the module 121. The allocation device 120 will then model the quantization error as a function of this sensitivity in order to minimize the error when the sensitivity is maximum, and vice versa of the maximize when sensitivity is minimal.
  • the allocation thus determined makes it possible to quantify (Q) at the coder, the spatial information parameters by the quantization module 104 or to perform an inverse quantization (Q -1 ) at the decoder by the inverse quantization module 114 to obtain these settings.
  • the synthesis module 112 will be able, from the spatial information thus de-quantified and from the decoded sum signal S sd , to obtain the multichannel audio stream in the frequency domain then after time / inverse frequency transformation of the module 116, the audio stream in the time domain x ⁇ 1 ( n ) , x ⁇ 2 ( n ) , ..., x ⁇ n ( n ).
  • the figure 2 now illustrates the steps of the bit allocation method in an embodiment of the invention.
  • an analysis step E201 of energy masking between the frequency sub-bands can optionally be carried out.
  • This step allows you to select a set of frequency sub-bands audible by the hearing system.
  • a sub-band having a high energy level can potentially mask ( ie . Make inaudible) the neighboring sub-bands having a too low energy level.
  • a comparative analysis of the energies of the different sub-bands can be carried out in order to determine whether certain sub-bands are not masked by other sub-bands. It is then unnecessary to keep the spatial information of the masked sub-bands, which frees quantization bits for the other sub-bands for the quantization bit allocation process given by the following steps of the method.
  • a set of sub-bands ⁇ b k ⁇ is thus defined to implement the steps of the allocation method.
  • each sub-band is considered as a target source, the other sub-bands can be considered as distracting sources.
  • step E202 spectral properties of the sub-bands of the set ⁇ b k ⁇ are extracted.
  • these spectral properties are either only its energy properties (I), or both the central frequency f c of the current sub-band and its energy properties.
  • each sub-band does not quite reflect reality in terms of perception at the time of the restoration, and this because only part of this energy will be restored in a correlated way between the different channels. The rest will be decorrelated. It is therefore interesting to estimate and specify to the psycho-acoustic model what will be the share of correlated energy (primary energy) and uncorrelated (ambient energy).
  • the energy properties can then be discriminated in primary energy (I p ) which represents the energy correlated between the sub-bands and the ambient energy (I a ) representing the energy decorrelated in the current sub-band.
  • step E203 makes an estimation of the spatial resolution in the current sub-band.
  • Each sub-band being considered in turn as a target.
  • a psycho-acoustic model ⁇ is determined and makes it possible to obtain the spatial resolution or even the MAA, associated with each sub-band.
  • the spatial resolution of the hearing system can be defined as the smallest angle between two sound sources that it can discriminate.
  • the benchmark study by Mills mentioned above was supported by more recent studies described for example in the document of Perrott DR and Saberi K., "Minimum audible angle thresholds for sources varying in both elevation and azimuth" in The journal of the acoustical Society of America, 87 (4): 1728-1731, April 1990 .
  • the MAA defines the minimum precision with which the position of a sound source must be described in order not to introduce audible artifacts. A position error lower than the MAA will not be perceived by the hearing system. Thus the MAA represents the “spatial blur” of perception of a sound source.
  • Another simplified psycho-acoustic model only takes into account the energy properties of the current sub-band.
  • the energy properties correspond to the energy measured in the sub-band.
  • the associated MAA is considered to be inversely proportional to the energy in this sub-band.
  • the energy properties correspond to a measurement of the energy distance of this sub-band at its masking / audibility threshold.
  • the MAA associated with this sub-band is also inversely proportional to the audible energy in this sub-band. In other words, the more audible energy a subband contains, the smaller its MAA will be assumed.
  • the psycho-acoustic model takes into account not only the characteristics of the current sub-band but also those of the other sub-bands which are then considered as distractive sub-bands.
  • the effect of the position of the distractive sources on “blurring” is negligible, in the sense that the MAA can be estimated without the position information of the distractive sources.
  • the MAA associated with a source depends on the position of this source in relation to the head of the listener. The best performance (lowest MAA) is observed when the auditor faces the source considered.
  • the psycho-acoustic model according to the invention it is assumed that the listener is free to orient his head within the listening device. Consequently, it is assumed, when estimating the MAA associated with a given source, that the listener always faces the source considered. Consequently, to estimate the MAA associated with a given source, the position information of this source is not necessary. From these results, a psychoacoustic model that describes the MAA associated with a given source can be constructed based on the presence and properties (energy, frequency content) of other sources.
  • the MAAs associated with the different sub-bands can be calculated from the “downmix” or sum signal component as described with reference to the figure 1 .
  • the consequence is that, for decoding, it is not necessary to transmit the quantization strategy, but that it can be deduced from the sum signal according to the same procedure as for encoding.
  • the psycho-acoustic model is described by a function ⁇ (c, d 1 , d 2 , ..., d N ), where c represents the target source, and the d i are the distractive sources.
  • each sub-band constitutes a source characterized by its central frequency and its energy (primary and ambient).
  • the function ⁇ produces the MAA associated with it in the presence of the other sources considered to be distracting, i.e. the maximum non-perceptible position error applicable to this source in the presence of others.
  • each source is characterized in step E202 by three parameters ⁇ f c , I p , I a ⁇ , where f c is the central frequency of the sub-band considered, and I p and I a are respectively the primary and ambient energy in this sub-band.
  • the psycho-acoustic model ⁇ (c, d 1 , d 2 , ..., d N ) produces a couple MAA values ⁇ p , ⁇ a ⁇ , corresponding respectively to the primary and ambient energy components, associated with step E203 with each sub-band considered in turn as target.
  • the MAA value considered will be ⁇ p or ⁇ a respectively , and therefore this distinction will no longer be made in the rest of the document. If the distribution I p / I a is unknown (parameter not transmitted), the decoder will assume that all the energy is correlated (primary energy), as well as the psycho-acoustic model, so as to obtain a correspondence during the restitution .
  • the function ⁇ (b k , b 1 , ..., b k - 1 , b k + 1 , ..., b K ) is called to estimate the spatial “blurring” exerted on this sub-band by the other sub-bands, which are therefore considered to be distracting, and ⁇ produces the MAA associated with this sub-band.
  • the spatial resolution is then estimated dynamically since the influence of the other sub-bands is taken into account.
  • the different spatial resolutions thus estimated in the frequency sub-bands make it possible to determine the number of bits to be allocated for the quantification of the spatial information parameters in each of the sub-bands.
  • step E204 a determination of the number of bits to be allocated to the current sub-band as a function of the estimated spatial resolution, is carried out.
  • the strategy for allocating the quantization bits of the spatialization parameters will then consist in maximizing the number of bits for the sub-bands having the minimum MAA, to the detriment of the sub-bands for which the MAA is maximum.
  • the number of bits to be allocated for a sub-band is inversely proportional to the spatial resolution estimated for this sub-band.
  • the allocation method can therefore adapt the allocation of bits from one sub-band to another according to the sensitivity of the auditory system to spatial distortion. This sensitivity is given by the psycho-acoustic model.
  • This method can be implemented both in the context of transmission at a constrained rate and in the context of transmission at an unconstrained rate.
  • bit budget is left available for variable allocation from one sub-band to another according to the MAA associated with it.
  • a certain budget of “floating” bits is therefore to be distributed between the same parameter of each of the sub-bands so as to perceptively minimize the spatial distortion resulting from the quantization process, in a homogeneous manner in each of the sub-bands.
  • the rest of the bit budget is distributed evenly across all sub-bands.
  • the quality of spatial coding is therefore defined by the average number, over all the sub-bands, of bits allocated to the same parameter, or, in an equivalent manner, by the total number of bits allocated to the same parameter for all the sub- bands.
  • a target spatial coding quality is chosen and imposed by the user.
  • This target quality is defined by the average number, over all time frames and over all sub-bands, of bits allocated to the same setting.
  • the average MAA then considered as a reference resolution value, is assumed to be estimable or predictable, all sub-bands combined, over all or part of the time frames.
  • the sub-bands whose estimated MAA is worth the average MAA will be allocated the average number of bits per parameter defined by the user.
  • the allocation of bits for the other sub-bands is made, as in the context of constrained bit rate, so as to perceptively minimize the spatial distortion resulting from the quantization process, homogeneously in each of the sub-bands, but given the number bits to be allocated to the medium MAA sub-bands.
  • the determination of the number of bits to be allocated for a sub-band is carried out if the resolution in the sub-band is different from a predetermined reference value, here the average MAA.
  • a minimum number of bits is already allocated per sub-band to code each parameter, which on the one hand ensures a minimum quality of spatial reproduction for all the audible sub-bands, and on the other hand provides an approximate value of the parameter concerned which is accessible for decoding.
  • the sub-band coded on the most bits (bm) must be the sub-band having the smallest MAA ( ⁇ m ), and the coding accuracy ratio between the current sub-band bk and bm must be inversely proportional to the ratio of the MAA of these two sub-bands:
  • the sum of the floating bits of each sub-band must not exceed the total number of floating bits available N flo : ⁇ NOT k ⁇ NOT flo .
  • Formulas (2) and (3) respectively give a first approximation of the number of bits to be allocated to the parameter of the sub-bands N k and N m . If there are still bits to allocate, or if too many bits have been allocated, the following heuristic (so-called "gluttonous" algorithm) makes it possible to finalize the process of allocating floating bits.
  • the index of the sub-band to which the next bit is to be allocated or to be resumed will be determined respectively by argmax k ( ⁇ k ) or argmin k ( ⁇ k ).
  • ⁇ k is recalculated after each operation (allocation or withdrawal) on one bit.
  • the allocation is finalized when the total number of floating bits allocated is exactly N float .
  • the coding accuracy ratio between the current subband b k and the reference subband b ⁇ must be inversely proportional to the ratio of the MAA of these two sub-bands:
  • the formula (5) gives the number of bits to allocate in total to the coding of the parameter of the sub-band b k .
  • each parameter is then quantized (Q) to the coder to form the binary train or de-quantized (Q -1 ) to the decoder according to the number of bits allocated to it.
  • the primary and ambient energy distribution parameters which are coded on a fixed number of bits, must be transmitted first, as they will then be necessary for decoding the parameters coded on a variable number of bits. .
  • the inverse quantization of the bit stream of spatial parameters requires knowing the number of bits allocated to each parameter.
  • the invention makes it possible to avoid transmission of additional information on the bit allocation strategy.
  • the primary and ambient energy distribution parameters which are coded on a fixed number of bits, have been transmitted beforehand. They are therefore decoded before decoding the other parameters.
  • n fixed is non-zero, it is possible to recover a first approximate value of each of the parameters without having to know the number of bits allocated to each of the parameters. Indeed, it suffices to organize the bit stream so as to send first n fixed most significant bits for each of the parameters, followed by the N k bits remaining for each parameter. This can be useful if other experimental studies show that certain position information is actually necessary to more accurately estimate the MAA. In this case, the sum signal or “downmix” would no longer suffice, and these approximate values of the parameters could be used to estimate the MAA at encoding (respectively at decoding) in order to know the number of bits to be allocated (respectively allocated) to each setting. Thus, the higher n is fixed , the more we have a good approximation of the parameters available for the estimation of MAA.
  • Encoders and decoders as described with reference to figure 1 as well as the allocation device that is the subject of the invention can be integrated into multimedia equipment of the living room decoder, "set top box” or audio or video content player type. They can also be integrated into mobile phone type communication equipment.
  • the figure 3 shows an exemplary embodiment of such equipment in which the allocation device according to the invention is integrated.
  • This device comprises a processor PROC cooperating with a memory block BM comprising a storage and / or working memory MEM.
  • the memory block can advantageously include a computer program comprising code instructions for implementing the steps of the allocation method within the meaning of the invention, when these instructions are executed by the processor PROC, and in particular the estimation steps.
  • the description of the figure 2 takes the steps of an algorithm of such a computer program.
  • the computer program can also be stored on a memory medium readable by a reader of the device or downloadable in the memory space of the latter.
  • Such equipment includes an input module capable of receiving a decoded sum signal either from an encoder via a local decoder, or from a decoder.
  • the device comprises an output module capable of transmitting the number of bits to be allocated per frequency sub-band to the quantization modules of an encoder or to the inverse quantization module of a decoder.
  • the device thus described may also include the coding and / or decoding functions in addition to the allocation functions according to the invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)

Claims (12)

  1. Verfahren zur Zuweisung von Bits zur Quantifizierung von Rauminformationsparametern je Frequenzteilband für eine parametrische Codierung/Decodierung eines mehrkanaligen Audiostroms, der eine Schallszene darstellt, die aus einer Mehrzahl von Schallquellen besteht und einen Schritt der Quantifizierung/Inversquantifizierung je Frequenzteilband von Rauminformationsparametern umfassend, welche die räumliche Position der Schallquellen der Schallszene beschreiben, dadurch gekennzeichnet, dass es die folgenden Schritte umfasst:
    - Schätzen (E203) einer räumlichen Auflösung des aktuellen Teilbandes anhand von Energieeigenschaften des Teilbandes;
    - Bestimmen (E204) einer Anzahl von Bits, die dem aktuellen Teilband zuzuweisen sind, wobei die zuzuweisende Anzahl von Bits umgekehrt proportional zu der geschätzten räumlichen Auflösung ist.
  2. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass das Schätzen einer räumlichen Auflösung des aktuellen Teilbandes ferner anhand der Mittenfrequenz des Teilbandes erfolgt.
  3. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass die räumliche Auflösung eines Teilbandes ferner anhand der Energieeigenschaften der anderen Subbänder einer Menge von Subbändern geschätzt wird, welche die Schallquellen definieren.
  4. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass die spektralen Eigenschaften eines Teilbandes anhand eines decodierten Summensignals erhalten werden, das aus einer Verarbeitung zur Reduktion der Kanäle des mehrkanaligen Audiostroms hervorgeht.
  5. Verfahren nach einem der Ansprüche 1 bis 4, dadurch gekennzeichnet, dass die Energieeigenschaften in einem Teilband die Primärenergie- und Umgebungsenergieeigenschaften in dem Teilband umfassen.
  6. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass die für ein Teilband zuzuweisende Anzahl von Bits zu einer vorbestimmten Anzahl von Bits gehört, die zu einer bereits je Teilband zugewiesenen Anzahl von Bits addiert wird.
  7. Verfahren nach Anspruch 6, dadurch gekennzeichnet, dass die Bestimmung der für ein Teilband zuzuweisenden Anzahl von Bits in Abhängigkeit von der Differenz zwischen der Auflösung in diesem Teilband und einer vorbestimmten Referenzauflösung angepasst wird, der eine vorbestimmte Zuweisung von Referenzbits entspricht.
  8. Verfahren nach Anspruch 1, dadurch gekennzeichnet, dass es bei einer Menge von unmaskierten Teilbändern eingesetzt wird, die durch einen Schritt der Analyse der energetischen Maskierung zwischen Teilbändern bestimmt wird.
  9. Vorrichtung zur Zuweisung von Bits zur Quantifizierung von Rauminformationsparametern je Frequenzteilband für einen parametrischen Encoder/Decoder eines mehrkanaligen Audiostroms, der eine Schallszene darstellt, die aus einer Mehrzahl von Schallquellen besteht und ein Modul zur Quantifizierung/Inversquantifizierung je Frequenzteilband von Rauminformationsparametern umfassend, welche die räumliche Position der Schallquellen der Schallszene beschreiben, dadurch gekennzeichnet, dass sie umfasst:
    - ein Schätzmodul (121) zum Schätzen einer räumlichen Auflösung des aktuellen Teilbandes anhand von Energieeigenschaften des Teilbandes;
    - ein Bestimmungsmodul (122) zum Bestimmen einer Anzahl von Bits, die dem aktuellen Teilband zuzuweisen sind, wobei die zuzuweisende Anzahl von Bits umgekehrt proportional zu der geschätzten räumlichen Auflösung ist.
  10. Parametrischer Encoder eines mehrkanaligen Audiostroms, dadurch gekennzeichnet, dass er eine Vorrichtung zur Zuweisung von Quantifizierungsbits nach Anspruch 9 umfasst.
  11. Parametrischer Decoder eines mehrkanaligen Audiostroms, dadurch gekennzeichnet, dass er eine Vorrichtung zur Zuweisung von Quantifizierungsbits nach Anspruch 9 umfasst.
  12. Computerprogramm, das Codeanweisungen beinhaltet, die bei der Ausführung dieser Anweisungen durch einen Prozessor die Schritte des Zuweisungsverfahrens nach einem der Ansprüche 1 bis 8 ausführen.
EP12717796.2A 2011-03-29 2012-03-28 Zuweisung von bits anhand von subbändern zur quantifizierung von rauminformationsparametern für parametrische codierung Active EP2691952B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
FR1152602A FR2973551A1 (fr) 2011-03-29 2011-03-29 Allocation par sous-bandes de bits de quantification de parametres d'information spatiale pour un codage parametrique
PCT/FR2012/050649 WO2012131253A1 (fr) 2011-03-29 2012-03-28 Allocation par sous-bandes de bits de quantification de paramètres d'information spatiale pour un codage paramétrique

Publications (2)

Publication Number Publication Date
EP2691952A1 EP2691952A1 (de) 2014-02-05
EP2691952B1 true EP2691952B1 (de) 2020-04-29

Family

ID=46022482

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12717796.2A Active EP2691952B1 (de) 2011-03-29 2012-03-28 Zuweisung von bits anhand von subbändern zur quantifizierung von rauminformationsparametern für parametrische codierung

Country Status (4)

Country Link
US (1) US9263050B2 (de)
EP (1) EP2691952B1 (de)
FR (1) FR2973551A1 (de)
WO (1) WO2012131253A1 (de)

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
FR2973551A1 (fr) * 2011-03-29 2012-10-05 France Telecom Allocation par sous-bandes de bits de quantification de parametres d'information spatiale pour un codage parametrique
CN103778918B (zh) * 2012-10-26 2016-09-07 华为技术有限公司 音频信号的比特分配的方法和装置
CN105976824B (zh) 2012-12-06 2021-06-08 华为技术有限公司 信号解码的方法和设备
TWI546799B (zh) 2013-04-05 2016-08-21 杜比國際公司 音頻編碼器及解碼器
CN104934034B (zh) * 2014-03-19 2016-11-16 华为技术有限公司 用于信号处理的方法和装置
FR3048808A1 (fr) * 2016-03-10 2017-09-15 Orange Codage et decodage optimise d'informations de spatialisation pour le codage et le decodage parametrique d'un signal audio multicanal
CN108959107B (zh) * 2017-05-18 2020-06-16 深圳市中兴微电子技术有限公司 一种共享方法及装置
US10586546B2 (en) 2018-04-26 2020-03-10 Qualcomm Incorporated Inversely enumerated pyramid vector quantizers for efficient rate adaptation in audio coding
US10573331B2 (en) 2018-05-01 2020-02-25 Qualcomm Incorporated Cooperative pyramid vector quantizers for scalable audio coding
US10580424B2 (en) * 2018-06-01 2020-03-03 Qualcomm Incorporated Perceptual audio coding as sequential decision-making problems
US10734006B2 (en) 2018-06-01 2020-08-04 Qualcomm Incorporated Audio coding based on audio pattern recognition
US11133891B2 (en) 2018-06-29 2021-09-28 Khalifa University of Science and Technology Systems and methods for self-synchronized communications
GB2575305A (en) * 2018-07-05 2020-01-08 Nokia Technologies Oy Determination of spatial audio parameter encoding and associated decoding
US10951596B2 (en) * 2018-07-27 2021-03-16 Khalifa University of Science and Technology Method for secure device-to-device communication using multilayered cyphers
US11990141B2 (en) * 2018-12-20 2024-05-21 Telefonaktiebolaget Lm Ericsson (Publ) Method and apparatus for controlling multichannel audio frame loss concealment
GB2595883A (en) * 2020-06-09 2021-12-15 Nokia Technologies Oy Spatial audio parameter encoding and associated decoding
KR20230135665A (ko) * 2021-01-29 2023-09-25 노키아 테크놀로지스 오와이 공간 오디오 파라미터 인코딩 및 관련 디코딩 결정
WO2024111300A1 (ja) * 2022-11-22 2024-05-30 富士フイルム株式会社 音データ作成方法及び音データ作成装置

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
IL76283A0 (en) * 1985-09-03 1986-01-31 Ibm Process and system for coding signals
US4899384A (en) * 1986-08-25 1990-02-06 Ibm Corporation Table controlled dynamic bit allocation in a variable rate sub-band speech coder
US4956871A (en) * 1988-09-30 1990-09-11 At&T Bell Laboratories Improving sub-band coding of speech at low bit rates by adding residual speech energy signals to sub-bands
US5054075A (en) * 1989-09-05 1991-10-01 Motorola, Inc. Subband decoding method and apparatus
JPH05335967A (ja) * 1992-05-29 1993-12-17 Takeo Miyazawa 音情報圧縮方法及び音情報再生装置
US5623577A (en) * 1993-07-16 1997-04-22 Dolby Laboratories Licensing Corporation Computationally efficient adaptive bit allocation for encoding method and apparatus with allowance for decoder spectral distortions
US5632003A (en) * 1993-07-16 1997-05-20 Dolby Laboratories Licensing Corporation Computationally efficient adaptive bit allocation for coding method and apparatus
KR0154387B1 (ko) * 1995-04-01 1998-11-16 김주용 음성다중 시스템을 적용한 디지탈 오디오 부호화기
EP0966109B1 (de) * 1998-06-15 2005-04-27 Matsushita Electric Industrial Co., Ltd. Audiokodierungsmethode und Audiokodierungsvorrichtung
JP2000059227A (ja) * 1998-08-07 2000-02-25 Matsushita Electric Ind Co Ltd 符号化／復号化装置、及び符号化／復号化方法
JP4287545B2 (ja) * 1999-07-26 2009-07-01 パナソニック株式会社 サブバンド符号化方式
US8379868B2 (en) * 2006-05-17 2013-02-19 Creative Technology Ltd Spatial audio coding based on universal spatial cues
US20090198500A1 (en) * 2007-08-24 2009-08-06 Qualcomm Incorporated Temporal masking in audio coding based on spectral dynamics in frequency sub-bands
FR2973551A1 (fr) * 2011-03-29 2012-10-05 France Telecom Allocation par sous-bandes de bits de quantification de parametres d'information spatiale pour un codage parametrique

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
US9263050B2 (en) 2016-02-16
FR2973551A1 (fr) 2012-10-05
US20140219459A1 (en) 2014-08-07
WO2012131253A1 (fr) 2012-10-04
EP2691952A1 (de) 2014-02-05

Similar Documents

Publication Publication Date Title
EP2691952B1 (de) Zuweisung von bits anhand von subbändern zur quantifizierung von rauminformationsparametern für parametrische codierung
EP2374123B1 (de) Verbesserte codierung von mehrkanaligen digitalen audiosignalen
EP2374124B1 (de) Verwaltete codierung von mehrkanaligen digitalen audiosignalen
EP2002424B1 (de) Vorrichtung und verfahren zur skalierbaren kodierung eines mehrkanaligen audiosignals auf der basis einer hauptkomponentenanalyse
JP5485909B2 (ja) オーディオ信号処理方法及び装置
EP3427260B1 (de) Optimierte codierung und decodierung von verräumlichungsinformationen zur parametrischen codierung und decodierung eines mehrkanaligen audiosignals
EP2005420B1 (de) Einrichtung und verfahren zur codierung durch hauptkomponentenanalyse eines mehrkanaligen audiosignals
EP2489039B1 (de) Optimierte parametrische codierung/decodierung mit niedrigem durchsatz
US20110002393A1 (en) Audio encoding device, audio encoding method, and video transmission device
EP2656342A1 (de) Verbesserte stereoparametrische kodierung/dekodierung für gegenphasige kanäle
US10553223B2 (en) Adaptive channel-reduction processing for encoding a multi-channel audio signal
EP2168121B1 (de) Quantifizierung nach linearer umwandlung durch kombination von audiosignalen einer klangszene und kodiergerät dafür
EP2345026A1 (de) Vorrichtung zur kunstkopf-audiokodierung
US20110206209A1 (en) Apparatus
JP2007333785A (ja) オーディオ信号符号化装置およびオーディオ信号符号化方法
FR3049084A1 (de)
EP2347411B1 (de) Vor-echo-dämpfung in einem digitalaudiosignal
WO2011073600A1 (fr) Codage/decodage parametrique stereo avec optimisation du traitement de reduction des canaux
WO2011045549A1 (fr) Decodage parametrique stereo optimise
EP4042418B1 (de) Bestimmung von korrekturen zur anwendung auf ein mehrkanalaudiosignal, zugehörige codierung und decodierung
FR2990551A1 (fr) Codage/decodage parametrique d'un signal audio multi-canal, en presence de sons transitoires

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20131025

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20190417

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602012069666

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0019000000

Ipc: G10L0019002000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 19/02 20130101ALI20191118BHEP

Ipc: G10L 19/008 20130101ALI20191118BHEP

Ipc: G10L 19/002 20130101AFI20191118BHEP

INTG Intention to grant announced

Effective date: 20191211

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1264499

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200515

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012069666

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

RAP2 Party data changed (patent owner data changed or rights of a patent transferred)

Owner name: ORANGE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200429

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200829

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200831

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200730

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1264499

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200429

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200729

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012069666

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20210201

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210331

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210331

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210328

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210328

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210331

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120328

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200429

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240220

Year of fee payment: 13

Ref country code: GB

Payment date: 20240221

Year of fee payment: 13

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240220

Year of fee payment: 13