EP2452336B1 - Codage/décodage perfectionne de signaux audionumériques - Google Patents
Codage/décodage perfectionne de signaux audionumériques Download PDFInfo
- Publication number
- EP2452336B1 EP2452336B1 EP10745327.6A EP10745327A EP2452336B1 EP 2452336 B1 EP2452336 B1 EP 2452336B1 EP 10745327 A EP10745327 A EP 10745327A EP 2452336 B1 EP2452336 B1 EP 2452336B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- coding
- band
- improvement
- decoding
- core
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005236 sound signal Effects 0.000 title description 4
- 230000000873 masking effect Effects 0.000 claims description 65
- 230000006872 improvement Effects 0.000 claims description 43
- 238000000034 method Methods 0.000 claims description 27
- 230000003595 spectral effect Effects 0.000 claims description 24
- 238000010606 normalization Methods 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 10
- 238000004364 calculation method Methods 0.000 claims description 9
- 230000007480 spreading Effects 0.000 claims description 9
- 238000003892 spreading Methods 0.000 claims description 9
- OVOUKWFJRHALDD-UHFFFAOYSA-N 2-[2-(2-acetyloxyethoxy)ethoxy]ethyl acetate Chemical compound CC(=O)OCCOCCOCCOC(C)=O OVOUKWFJRHALDD-UHFFFAOYSA-N 0.000 description 49
- 239000010410 layer Substances 0.000 description 38
- 230000006870 function Effects 0.000 description 22
- 238000001228 spectrum Methods 0.000 description 13
- 238000001914 filtration Methods 0.000 description 8
- 238000013139 quantization Methods 0.000 description 7
- 230000008901 benefit Effects 0.000 description 6
- 230000015572 biosynthetic process Effects 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000003786 synthesis reaction Methods 0.000 description 4
- 230000005540 biological transmission Effects 0.000 description 3
- 238000004422 calculation algorithm Methods 0.000 description 3
- 239000012792 core layer Substances 0.000 description 3
- 238000003860 storage Methods 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000005055 memory storage Effects 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 238000007493 shaping process Methods 0.000 description 2
- 230000006978 adaptation Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000004927 fusion Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
- 238000011002 quantification Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/002—Dynamic bit allocation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/032—Quantisation or dequantisation of spectral components
- G10L19/038—Vector quantisation, e.g. TwinVQ audio
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/16—Vocoder architecture
- G10L19/18—Vocoders using multiple modes
- G10L19/24—Variable rate codecs, e.g. for generating different qualities using a scalable representation such as hierarchical encoding or layered encoding
Definitions
- the present invention relates to a sound data processing.
- This processing is adapted in particular to the transmission and / or storage of digital signals such as audio-frequency signals (speech, music, or other).
- the invention applies more particularly to hierarchical coding (or " scalable " coding) which generates a so-called “hierarchical” bitstream because it comprises a core rate and one or more improvement layer (s).
- the G.722 standard at 48, 56 and 64 kbit / s is an example of scalable scalable codec
- the ITU-T G.729.1 and MPEG-4 CELP codecs are examples of scalable codecs in both bitrate and bitrate. in bandwidth.
- Hierarchical coding having the capacity to provide varied bit rates, is described below by distributing the information relating to an audio signal to be coded in hierarchical subsets, so that this information can be used in order of importance. in terms of audio rendering quality.
- the criterion taken into account for determining the order is a criterion for optimizing (or rather reducing) the quality of the coded audio signal.
- Hierarchical coding is particularly suited to transmission over heterogeneous networks or having variable available rates over time, or to transmission to terminals with varying capacities.
- Hierarchical audio coding (or " scalable ) can be described as follows.
- the bit stream includes a base layer and one or more enhancement layers.
- the base layer is generated by a fixed rate codec, termed a " core codec ", guaranteeing the minimum quality of the coding.
- This layer must be received by the decoder to maintain an acceptable level of quality. Improvement layers are used to improve quality. However, they may not all be received by the decoder.
- the main advantage of hierarchical coding is that it allows an adaptation of the bit rate simply by " truncation of the bit stream ".
- the number of layers i.e., the number of possible truncations of the bitstream
- the number of layers defines the granularity of the coding.
- scalable scalability and bandwidth encoding techniques with a CELP heart-coder, a telephone band, and one or more broadband enhancement layer (s).
- An example of such systems is given in the ITU-T G.729.1 8-32 kbit / s fine grain standard.
- the G.729.1 coding / decoding algorithm is summarized below.
- the G.729.1 encoder is an extension of the ITU-T G.729 coder. It is a modified G.729 heart-coded core encoder producing a bandwidth ranging from narrowband (50-4000 Hz) to wideband (50-7000 Hz) at a rate of 8 to 32 kbit / s for conversational services. This codec is compatible with existing VoIP devices that use the G.729 codec.
- the G.729.1 coder is schematized on the figure 1 .
- the broadband input signal s wb sampled at 16 kHz, is first decomposed into two subbands by QMF (for "Quadrature Mirror Filter") filtering.
- the low band (0-4000 Hz) is obtained by LP low-pass filtering (block 100) and decimation (block 101), and the band high (4000-8000 Hz) by high-pass filtering HP (block 102) and decimation (block 103).
- the LP and HP filters are of length 64.
- the low band is pretreated with a high-pass filter eliminating the components below 50 Hz (block 104), to obtain the signal s LB , before CELP coding in narrow band (block 105) at 8 and 12 kbit / s.
- This high-pass filtering takes into account the fact that the wanted band is defined as covering the interval 50-7000 Hz.
- the narrow-band CELP coding is a cascaded CELP coding comprising as a first stage a modified G.729 coding without a filter. preprocessing and as a second stage an additional fixed CELP dictionary.
- the high band is first pretreated (block 106) to compensate for the folding due to the high-pass filter (block 102) combined with the decimation (block 103).
- the high band is then filtered by a low pass filter (block 107) eliminating the components between 3000 and 4000 Hz from the high band (i.e. the components between 7000 and 8000 Hz in the original signal) to obtain the signal s HB .
- a parametric band extension (block 108) is then performed.
- the LB of error signal of the low band is calculated (block 109) from the output of the CELP coder (block 105) and a predictive transform coding (of TDAC type for "Time Domain Aliasing Cancellation" in the standard G .729.1) is carried out at block 110.
- a predictive transform coding of TDAC type for "Time Domain Aliasing Cancellation" in the standard G .729.1
- Additional parameters can be transmitted by the block 111 to a homologous decoder, this block 111 performing a so-called “FEC” treatment for "Frame Erasure Concealment", in order to reconstitute possible erased frames.
- the different bitstreams generated by the coding blocks 105, 108, 110 and 111 are finally multiplexed and structured into a hierarchical bit stream in the multiplexing block 112.
- the coding is performed by 20 ms sample blocks (or frames). 320 samples per frame.
- the G.729.1 decoder is illustrated on the figure 2 .
- the bits describing each frame of 20 ms are demultiplexed in block 200.
- the bit stream of the 8 and 12 kbit / s layers is used by the CELP decoder (block 201) to generate the narrow-band synthesis (0-4000 Hz).
- the portion of the bit stream associated with the 14 kbit / s layer is decoded by the tape extension module (block 202).
- the portion of the bit stream associated with data rates greater than 14 kbit / s is decoded by the TDAC module (block 203).
- Pre-echo and post-echo processing is performed by blocks 204 and 207 as well as enrichment (block 205) and aftertreatment of the low band (block 206).
- the broadband output signal ⁇ wb sampled at 16 kHz, is obtained via the QMF synthesis filter bank (blocks 209, 210, 211, 212 and 213) incorporating the inverse folding (block 208).
- the TDAC type transform coding in the G.729.1 encoder is illustrated on the figure 3 .
- MDCT transforms (blocks 301 and 302) apply to 20 ms of sampled signal at 8 kHz (160 coefficients).
- This spectrum is divided into eighteen sub-bands, a sub-band j being assigned a number of coefficients noted nb_ coef ( j ).
- the subband splitting is specified in Table 1 below.
- a subband j comprises the coefficients Y ( k ) with sb_bound ( j ) ⁇ k ⁇ sb_bound ( j + 1).
- the coefficients 280-319 corresponding to the frequency band 7000 Hz - 8000 Hz are not coded; they are set to zero at the decoder because the bandwidth of the codec is 50-7000 Hz.
- Table 1 ⁇ / b> TDAC Code Subband Bounds and Size J sb_bound ( j ) nb_coef ( j ) 0 0 16 1 16 16 2 32 16 3 48 16 4 64 16 5 80 16 6 96 16 7 112 16 8 128 16 9 144 16 10 160 16 11 176 16 12 192 16 13 208 16 14 224 16 15 240 16 16 256 16 17 272 8 18 280 -
- the spectral envelope is coded at variable rate in block 305.
- This quantized value rms _ index ( j ) is transmitted to the bit allocation block 306.
- a bit (0 or 1) is transmitted to the decoder to indicate the encoding mode that has been chosen.
- the number of bits allocated to each subband for its quantization is determined at block 306 from the quantized spectral envelope from block 305.
- the bit allocation performed minimizes the squared error while respecting the constraint of a whole number of bits allocated per subband and a number of bits. maximum not to be exceeded.
- the spectral content of the subbands is then encoded by spherical vector quantization (block 307).
- the different bit streams generated by the blocks 305 and 307 are then multiplexed and structured into a hierarchical bit stream at the multiplexing block 308.
- the TDAC type transform decoding step in the G.729.1 decoder is illustrated on the figure 4 .
- the decoded spectral envelope (block 401) makes it possible to find the allocation of the bits (block 402).
- each of the subbands is found by inverse spherical vector quantization (block 403).
- the sub-bands not transmitted, due to a lack of "budget" of bits, are extrapolated (block 404) from the MDCT transform of the signal at the output of the band extension block (block 202 of FIG. figure 2 ).
- IMDCT inverse MDCT transform
- W LB inverse perceptual weighting
- the subbands bit allocation (block 306 of FIG. figure 3 or block 402 of the figure 4 ).
- nbits_ VQ 351 - nbits_rms
- nbits_rms the number of bits used by the coding of the spectral envelope.
- nbit ( j ) arg min r ⁇ R nb_coef j ⁇ nb_coef j ⁇ ip j - ⁇ Opt - r
- the TDAC coding uses the perceptual weighting W LB ( z ) filter in the low band (block 300), as indicated above.
- perceptual weighting filtering allows you to format the coding noise.
- the principle of this filtering is to exploit the fact that it is possible to inject more noise in the frequency zones where the original signal has a high energy.
- the most common perceptual weighting filters used in narrow-band CELP coding are of the form ⁇ (z / ⁇ 1) / ⁇ (z / ⁇ 2) where 0 ⁇ ⁇ 2 ⁇ ⁇ 1 ⁇ 1 and ⁇ (z) represents a prediction spectrum linear (LPC).
- the synthesis analysis in CELP coding thus amounts to minimizing the quadratic error in a signal domain perceptually weighted by this type of filter.
- the fac factor makes it possible to ensure at the junction of the low and high bands (4 kHz) a gain of the filter at 1 to 4 kHz. It is important to note that in the G.729.1 TDAC coding, the coding is based on an energetic criterion only.
- the energy criterion of TDAC coding of G.729.1, used in the high band (4000-7000 Hz), is not optimal from a perceptual point of view, in particular to code musical signals.
- the perceptual weighting filter is particularly suited to speech signals. It is widely used in speech coding standards based on the CELP coding format. However, for the musical signals, it appears that this perceptual weighting based on a shaping of the quantization noise according to the formants of the input signal is insufficient. Most audio coders rely on transform coding using frequency masking or simultaneous masking models; they are more generic (in the sense that they do not use a speech production model like the CELP) and are therefore more suitable for encoding musical signals.
- the present invention improves the situation.
- the coding according to the invention takes advantage of an enhancement coding layer to improve the coding quality from a perceptual point of view.
- the enhancement layer will thus benefit from frequency masking which does not exist in the core coding stage, in order to best allocate the bits in the frequency bands of the enhancement coding.
- This operation does not modify the core coding, which thus remains compatible with the existing standard coding, thus ensuring interoperability with equipment already on the market using existing standard coding.
- the first perceptual importance that will be used for the enhancement layer does not take into account the core coding but only the signal to mask ratio to define a perceptual importance. This perceptual importance is determined on the input signal of the transform coder.
- the encoding of the core is done simply by subtracting the average number of bits per sample already allocated.
- the use of the perceptual importance based on the signal-to-mask ratio would make it possible to obtain an optimal allocation in the perceptual sense. However, this allocation would be useful if the input signal of the transform coding layer was coded directly.
- a first transform coding layer based on an energy allocation has allocated a certain number of bits per subband.
- the perceptual importance is furthermore determined as a function of bits allocated for a previous core coding enhancement coding having a binary allocation according to an energy criterion.
- the sub-bands not transmitted, due to a lack of sufficient bit budget, are extrapolated (block 404) from the MDCT transform of the signal at the output of the band extension block (block 202 of FIG. figure 2 ). Even at the higher bit rate of G.729.1 (32 kbit / s) some frequency bands remain extrapolated.
- a first encoding for improving the core coding to compensate for the lack of bit rate of the core coding for these non-transmitted subbands.
- This first enhancement coding uses the original signal and operates according to energy criteria for bit allocation. According to one embodiment of the invention, this first improvement coding modifies the number of bits nbit ( j ) allocated to the subbands and the decoded subband Yq ( k ) (defined later on in FIG. figure 5 ).
- the improvement coding according to the invention therefore also takes into account the bits allocated during this first improvement coding, in addition to the bits allocated in the core coding.
- the method comprises a step of obtaining an information according to which the signal to be encoded is tonal or non-tonal and the steps of calculating the masking threshold and of determining a perceptual importance according to this masking threshold, are conducted only if the signal is non-tonal.
- the coding is adapted to the signal whether it is tonal or not and allows optimal allocation of the bits.
- the enhancement coding is coding for improving the coding of the MDCT type coefficients in an extended coder whose core coding is of G.729.1 standard encoder type.
- the quality of the G.729.1 codec in the enlarged band (50-7000 Hz) is improved.
- Such an improvement is important to extend the G.729.1 encoder band of the enlarged band (50-7000Hz) to the super-wide band (50-14000Hz).
- the invention relates to a computer program comprising code instructions for implementing the steps of an encoding method according to the invention, when they are executed by a processor and to a computer program comprising instructions. code for implementing the steps of a decoding method according to the invention, when executed by a processor.
- One of the objects of the invention is the improvement of the quality of G.729.1 in wideband (50-7000 Hz), in particular for musical signals. It will be recalled here that the G.729.1 coding has a useful band of 50 to 7000 Hz. Moreover, the quality of G.729.1 for some signals such as music signals is not transparent at its highest bit rate (32 kbit / s) - this limitation is due to the hierarchical structure CELP + TDBWE + TDAC and the bit rate limited to 32 kbit / s .
- This invention is motivated by the ongoing ITU-T standardization of a scalable extension of G.729.1, in particular to extend the G.729.1 coded band to the super-wide band (50-14000 Hz).
- the band extension eg 7000-14000 Hz
- a limited band signal eg 50-7000 Hz
- the band extension highlights the existing defects in this signal.
- the G.729.1 quality improvement can be achieved with one or more additional rate enhancement layers (in addition to 32 kbit / s).
- these additional flow enhancement layers can serve both the band extension (7000-14000 Hz) and the quality improvement in the enlarged band (50-7000 Hz).
- part of the additional bit rate of the enhancement layers can be devoted to improving the broadband signal decoded by a G.729.1 decoder.
- G.729.1 has a narrow-band CELP core coder, while the super-expanded band extension (50-14000Hz) of G.729.1 has for heart G.729.1.
- core coding and heart rate we mean a G.729.1 type coding and the associated bit rate of 32 kbit / s.
- it is more particularly a TDAC encoder and decoder as described above, in which an enhancement layer is integrated.
- the figure 5 describes such an improved TDAC coder.
- a scalable extension of G.729.1 is considered in several enhancement layers.
- the core coding is a G.729.1 coding, which uses TDAC coding in the band [50-7000 Hz] from the bitrate of 14 kbit / s and up to 32 kbit / s. It is assumed that between 32 and 48 kbit / s two enhancement layers of 8 kbit / s are produced in order to extend the band from 7000 to 14000 Hz and to replace the non-transmitted subbands of the TDAC coding of G.729.1 . These enhancement layers of 8 kbit / s ranging from 32 to 48 kbit / s are not described here.
- the present invention provides two additional 8 kbit / s enhancement layers of the TDAC coding in the 50 to 7000 Hz band that increase the bit rate from 48 kbit / s to 56 and 64 kbit / s.
- the encoder applying the present invention has enhancement layers that add G.729.1 core bit rate (32 kbit). These enhancement layers serve both to improve the quality in the enlarged band (50-7000 Hz) and to extend the upper band from 7000 to 14000 Hz. In the following we ignore the extension of 7000 to 14000 Hz, because this feature does not influence the implementation of the present invention. For reasons of simplicity, the modules corresponding to the 7000 to 14000 Hz band extension are not illustrated on the figures 5 and 6 .
- the TDAC encoder according to one embodiment of the invention here comprises an enhancement layer (blocks 509 to 513) which improves the core layer (blocks 504 to 507).
- the block 507 here corresponds to the spherical vector quantization (SVQ) quantification of G.729.1, which may comprise a modification as mentioned above.
- SVQ spherical vector quantization
- This modification uses the original signal Y ( k ) and operates according to energy criteria for bit allocation. The number of bits nbit (j) allocated to the subbands and the decoded subband Yq ( k ) are then modified.
- Block 506 performs a binary allocation based on energy criteria as described with reference to FIG. figure 3 .
- the core layer is thus coded and sent to the multiplexing module 508.
- the heart signal is also decoded locally in the encoder by the block 510 which performs spherical dequantization and scaling; this heart signal is subtracted from the original signal at 509, in the transformed domain, to obtain a residual signal err (k). This residual signal is then coded from a bit rate of 48 kbit / s in block 513.
- An advantageous spreading function is that presented to the figure 7 .
- This is a triangular function whose first slope is + 27dB / Bark and - 10dB / Bark for the second one.
- ⁇ 1 ( j ) and ⁇ 2 ( j ) can be pre-calculated and stored.
- the application of the masking threshold is in this embodiment, limited to the high band.
- the masking threshold is normalized for example by its value on the last sub-band. band of the low band.
- a first perceptual importance calculation step is then performed taking into account the signal-to-mask ratio given by: 1 2 ⁇ log 2 ⁇ ⁇ 2 j M j
- FIG 8 An illustration of the standardization of the masking threshold is given in figure 8 , showing the connection of the high band on which the masking (4-7 kHz) is applied to the low band (0-4 kHz).
- the standardization of the masking threshold can be rather carried out from the value of the band.
- a first perceptual importance ip ( j ) is sent to the binary allocation block 512 for improvement coding.
- This block 512 also receives bit allocation information nbit ( j ) from the core layer of the TDAC coding, G.729.1.
- Block 512 thus defines a new perceptual importance that takes into account these two pieces of information.
- the new perceptual importance is calculated by subtracting at the first perceptual importance, a ratio between the number of bits allocated for the core coding and the number of possible coefficients in the subband.
- the block 512 performs a bit allocation on the residual signal to code the enhancement layer.
- nbits _ VQ _ err corresponding to the additional number of bits in the enhancement layer (320 bits for the 2 8 kbit / s layers).
- the residual signal err ( k ) is then coded by the module 513 by spherical vector quantization, using the number of allocated bits nbit- err ( j ) as previously calculated.
- This coded residual signal is then multiplexed with the signal resulting from the core coding and the coded envelope by the multiplexing module 508.
- This enhancement coding not only extends the allocated bit rate but improves from a perceptual point of view, the coding of the signal.
- the TDAC coding enhancement layer as described can be applied after modifying the TDAC coding of G.729.1.
- a first enhancement (not described here) of the TDAC coding of G.729.1 is performed.
- This enhancement allocates bits to subbands between 4 and 7 kHz at which no bit rate was allocated by the G.729.1 TDAC core encoding even at its higher 32 kbit / s bit rate.
- This first improvement of the TDAC coding of G.729.1 therefore uses the original signal between 4 and 7 kHz and does not implement the steps of calculating a masking threshold or determining the perceptual importance of the coding method of the 'invention.
- Block 507 is considered to correspond to this modified TDAC coding integrating this improvement.
- the determination of the perceptual importance takes into account not only the bits allocated for the core or base coding but also the bits allocated for the previous enhancement coding, in this case the 40 kbit / s rate enhancement coding.
- the figure 6 illustrates the TDAC decoder with an enhancement decoding stage as well as the steps of a decoding method according to an embodiment of the invention.
- the decoder comprises the modules (601, 602, 603, 606, 607, 608, 609 and 610) identical to those described for the TDAC decoding of the G.729.1 coder with reference to FIG. figure 4 (401, 402, 403, 406, 407, 408, 409 and 410).
- the block 606 for processing in the MDCT domain (aimed at shaping the coding noise) is here optional because the invention improves the quality of the decoded MDCT spectrum from block 603.
- the module 605 of the decoder corresponds to the encoder module 511 and operates in the same way from the quantized values of the spectral envelope.
- the allocation module 604 determines a second perceptual importance in taking into account the allocation of bits received from the core coding, in the same way as in the coding module 512.
- This bit allocation for the enhancement coding allows the module 611 to decode the signal received from the demultiplexing module 600 by spherical vector dequantization.
- the decoded signal from the module 611 is an error signal err (k) which is then combined at 612 with the decoded heart signal at 603.
- This signal is then processed as for the G.729.1 encoding described with reference to the figure 4 to give a difference signal d LB in low band and a signal S HB in high band.
- the calculation of a frequency masking performed by the module 511 or 605 and as described above may or may not be carried out according to the signal to be encoded (in particular if it is tonal or not).
- the application of the spreading function B (v) results in a masking threshold very close to a tone a little more spread out in frequencies.
- the criterion of minimization of the masked coding noise ratio then gives a bit allocation that is not necessarily optimal.
- the calculation of the masking threshold and the determination of the perceptual importance as a function of this masking threshold according to the invention is applied only if the signal to be encoded is not tonal.
- the bit relating to the mode of the coding of the spectral envelope indicates a "differential Huffman" mode or a "natural direct binary” mode.
- This mode bit can be interpreted as a tone detection, since, in general, a tonal signal leads to envelope coding by the "natural direct binary” mode, while most non-tonal signals, having a spectral dynamic more limited, lead to envelope coding by the "Differential Huffman" mode.
- the super-expanded band extension of the G.729.1 encoder as shown consists of an extension of the frequencies coded by the module 915, the frequency band used from [50Hz-7KHz] to [50Hz-14kHz] and an improvement of the G.729.1 base layer by the TDAC coding module (block 910) and as described with reference to FIG. figure 5 .
- the encoder as represented in figure 9 has the same modules as the G.729.1 core coding represented in figure 1 and an additional band extension module 915 which provides an extension signal to the multiplexing module 912.
- This frequency band extension is calculated on the original full-band signal S SWB while the input signal of the core encoder is obtained by decimation (block 913) and low-pass filtering (block 914). At the output of these blocks, the broadband input signal S WB is obtained.
- the TDAC coding module 910 is different from that illustrated in FIG. figure 1 .
- This module is for example that described with reference to the figure 5 and provides the multiplexing module with both the coded core signal and the coded enhancement signal according to the invention.
- a G.729.1 extended-band decoder is described with reference to the figure 10 . It has the same modules as the G.729.1 decoder described with reference to the figure 2 .
- the band extension module 1014 which receives from the demultiplexing module 1000, the band extension signal.
- the TDAC decoding module 1003 is also different from the TDAC decoding module illustrated with reference to FIG. figure 2 .
- This module is for example that described and illustrated with reference to the figure 6 . It thus receives from the demultiplexing module, both the heart signal and the improvement signal.
- the invention is used to improve the quality of TDAC coding in the G.729.1 codec.
- the invention applies to other types of transform coding with a binary allocation and scalable extension of other core codecs than G.729.1.
- the figure 11a illustrates an encoder or terminal comprising an encoder as described in figure 5 . It comprises a PROC processor cooperating with a memory block BM having a memory storage and / or working MEM.
- This terminal comprises an input module able to receive a low band signal d LB and a high band signal S HB or any type of digital signal to be coded. These signals may come from another coding stage or a communication network, a digital content storage memory.
- the description of the figure 5 takes the steps of an algorithm of such a computer program.
- the computer program can also be stored on a memory medium readable by a reader of the terminal or encoder or downloadable in the memory space thereof.
- the terminal comprises an output module capable of transmitting a multiplexed stream derived from the coding of the input signals.
- figure 11b illustrates an example of a decoder or terminal including a decoder as described with reference to the figure 6 .
- This terminal comprises a PROC processor cooperating with a memory block BM having a memory storage and / or working MEM.
- the terminal comprises an input module adapted to receive a multiplexed stream coming for example from a communication network, a storage module.
- the description of the figure 6 takes the steps of an algorithm of such a computer program.
- the computer program can also be stored on a memory medium readable by a reader of the terminal or downloadable in the memory space thereof.
- the terminal comprises an output module capable of transmitting decoded signals ( d LB , S HB ) for another coding stage or for content reproduction.
- such a terminal may comprise both the encoder and the decoder according to the invention.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Quality & Reliability (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR0954682A FR2947944A1 (fr) | 2009-07-07 | 2009-07-07 | Codage/decodage perfectionne de signaux audionumeriques |
PCT/FR2010/051307 WO2011004097A1 (fr) | 2009-07-07 | 2010-06-25 | Codage/décodage perfectionne de signaux audionumériques |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2452336A1 EP2452336A1 (fr) | 2012-05-16 |
EP2452336B1 true EP2452336B1 (fr) | 2013-11-27 |
Family
ID=41531514
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP10745327.6A Active EP2452336B1 (fr) | 2009-07-07 | 2010-06-25 | Codage/décodage perfectionne de signaux audionumériques |
Country Status (7)
Country | Link |
---|---|
US (1) | US8812327B2 (ko) |
EP (1) | EP2452336B1 (ko) |
KR (1) | KR101698371B1 (ko) |
CN (1) | CN102576536B (ko) |
CA (1) | CA2766864C (ko) |
FR (1) | FR2947944A1 (ko) |
WO (1) | WO2011004097A1 (ko) |
Families Citing this family (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2525355B1 (en) * | 2010-01-14 | 2017-11-01 | Panasonic Intellectual Property Corporation of America | Audio encoding apparatus and audio encoding method |
FR3003683A1 (fr) * | 2013-03-25 | 2014-09-26 | France Telecom | Mixage optimise de flux audio codes selon un codage par sous-bandes |
FR3003682A1 (fr) * | 2013-03-25 | 2014-09-26 | France Telecom | Mixage partiel optimise de flux audio codes selon un codage par sous-bandes |
CN104282312B (zh) | 2013-07-01 | 2018-02-23 | 华为技术有限公司 | 信号编码和解码方法以及设备 |
US10424305B2 (en) * | 2014-12-09 | 2019-09-24 | Dolby International Ab | MDCT-domain error concealment |
JP6611042B2 (ja) * | 2015-12-02 | 2019-11-27 | パナソニックIpマネジメント株式会社 | 音声信号復号装置及び音声信号復号方法 |
MX2020002972A (es) * | 2017-09-20 | 2020-07-22 | Voiceage Corp | Metodo y dispositivo para asignar un presupuesto de bits entre subtramas en un codec celp. |
CN114708874A (zh) * | 2018-05-31 | 2022-07-05 | 华为技术有限公司 | 立体声信号的编码方法和装置 |
EP3751567B1 (en) * | 2019-06-10 | 2022-01-26 | Axis AB | A method, a computer program, an encoder and a monitoring device |
CN111246469B (zh) * | 2020-03-05 | 2020-10-16 | 北京花兰德科技咨询服务有限公司 | 人工智能保密通信系统及通信方法 |
CN111294367B (zh) * | 2020-05-14 | 2020-09-01 | 腾讯科技(深圳)有限公司 | 音频信号后处理方法和装置、存储介质及电子设备 |
Family Cites Families (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5495552A (en) * | 1992-04-20 | 1996-02-27 | Mitsubishi Denki Kabushiki Kaisha | Methods of efficiently recording an audio signal in semiconductor memory |
JPH07160297A (ja) * | 1993-12-10 | 1995-06-23 | Nec Corp | 音声パラメータ符号化方式 |
DE19743662A1 (de) * | 1997-10-02 | 1999-04-08 | Bosch Gmbh Robert | Verfahren und Vorrichtung zur Erzeugung eines bitratenskalierbaren Audio-Datenstroms |
FI109393B (fi) * | 2000-07-14 | 2002-07-15 | Nokia Corp | Menetelmä mediavirran enkoodaamiseksi skaalautuvasti, skaalautuva enkooderi ja päätelaite |
DE60214599T2 (de) * | 2002-03-12 | 2007-09-13 | Nokia Corp. | Skalierbare audiokodierung |
DE10236694A1 (de) * | 2002-08-09 | 2004-02-26 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Vorrichtung und Verfahren zum skalierbaren Codieren und Vorrichtung und Verfahren zum skalierbaren Decodieren |
FR2849727B1 (fr) * | 2003-01-08 | 2005-03-18 | France Telecom | Procede de codage et de decodage audio a debit variable |
DE602004004950T2 (de) * | 2003-07-09 | 2007-10-31 | Samsung Electronics Co., Ltd., Suwon | Vorrichtung und Verfahren zum bitraten-skalierbaren Sprachkodieren und -dekodieren |
KR100561869B1 (ko) * | 2004-03-10 | 2006-03-17 | 삼성전자주식회사 | 무손실 오디오 부호화/복호화 방법 및 장치 |
US7272567B2 (en) * | 2004-03-25 | 2007-09-18 | Zoran Fejzo | Scalable lossless audio codec and authoring tool |
FR2888699A1 (fr) * | 2005-07-13 | 2007-01-19 | France Telecom | Dispositif de codage/decodage hierachique |
CN101263554B (zh) * | 2005-07-22 | 2011-12-28 | 法国电信公司 | 在比特率分级和带宽分级的音频解码中的比特率切换方法 |
KR100827458B1 (ko) * | 2006-07-21 | 2008-05-06 | 엘지전자 주식회사 | 오디오 부호화 방법 |
FR2912249A1 (fr) * | 2007-02-02 | 2008-08-08 | France Telecom | Codage/decodage perfectionnes de signaux audionumeriques. |
US8032359B2 (en) * | 2007-02-14 | 2011-10-04 | Mindspeed Technologies, Inc. | Embedded silence and background noise compression |
JP4871894B2 (ja) * | 2007-03-02 | 2012-02-08 | パナソニック株式会社 | 符号化装置、復号装置、符号化方法および復号方法 |
JP4708446B2 (ja) * | 2007-03-02 | 2011-06-22 | パナソニック株式会社 | 符号化装置、復号装置およびそれらの方法 |
US20100292986A1 (en) * | 2007-03-16 | 2010-11-18 | Nokia Corporation | encoder |
US8209190B2 (en) * | 2007-10-25 | 2012-06-26 | Motorola Mobility, Inc. | Method and apparatus for generating an enhancement layer within an audio coding system |
JP5383676B2 (ja) * | 2008-05-30 | 2014-01-08 | パナソニック株式会社 | 符号化装置、復号装置およびこれらの方法 |
US8219408B2 (en) * | 2008-12-29 | 2012-07-10 | Motorola Mobility, Inc. | Audio signal decoder and method for producing a scaled reconstructed audio signal |
US8200496B2 (en) * | 2008-12-29 | 2012-06-12 | Motorola Mobility, Inc. | Audio signal decoder and method for producing a scaled reconstructed audio signal |
-
2009
- 2009-07-07 FR FR0954682A patent/FR2947944A1/fr not_active Withdrawn
-
2010
- 2010-06-25 CA CA2766864A patent/CA2766864C/fr active Active
- 2010-06-25 CN CN2010800396757A patent/CN102576536B/zh active Active
- 2010-06-25 EP EP10745327.6A patent/EP2452336B1/fr active Active
- 2010-06-25 KR KR1020127003321A patent/KR101698371B1/ko active IP Right Grant
- 2010-06-25 WO PCT/FR2010/051307 patent/WO2011004097A1/fr active Application Filing
- 2010-06-25 US US13/382,786 patent/US8812327B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN102576536B (zh) | 2013-09-04 |
US20120185255A1 (en) | 2012-07-19 |
CA2766864A1 (fr) | 2011-01-13 |
US8812327B2 (en) | 2014-08-19 |
CA2766864C (fr) | 2015-10-27 |
KR101698371B1 (ko) | 2017-01-26 |
EP2452336A1 (fr) | 2012-05-16 |
CN102576536A (zh) | 2012-07-11 |
WO2011004097A1 (fr) | 2011-01-13 |
KR20120032025A (ko) | 2012-04-04 |
FR2947944A1 (fr) | 2011-01-14 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2115741B1 (fr) | Codage/decodage perfectionnes de signaux audionumeriques | |
EP2452336B1 (fr) | Codage/décodage perfectionne de signaux audionumériques | |
EP2452337B1 (fr) | Allocation de bits dans un codage/décodage d'amélioration d'un codage/décodage hiérarchique de signaux audionumériques | |
EP1989706B1 (fr) | Dispositif de ponderation perceptuelle en codage/decodage audio | |
EP1905010B1 (fr) | Codage/décodage audio hiérarchique | |
EP2366177B1 (fr) | Codage de signal audionumerique avec mise en forme du bruit dans un codeur hierarchique | |
JP5129117B2 (ja) | 音声信号の高帯域部分を符号化及び復号する方法及び装置 | |
EP2251861B1 (en) | Encoding device and method thereof | |
EP2239731B1 (en) | Encoding device, decoding device, and method thereof | |
CA2512179C (fr) | Procede de codage et de decodage audio a debit variable | |
US20090157413A1 (en) | Speech encoding apparatus and speech encoding method | |
WO2007107670A2 (fr) | Procede de post-traitement d'un signal dans un decodeur audio | |
Marie | Docteur en Sciences | |
FR2737360A1 (fr) | Procedes de codage et de decodage de signaux audiofrequence, codeur et decodeur pour la mise en oeuvre de tels procedes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20120203 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: KOVESI, BALAZS Inventor name: RAGOT, STEPHANE Inventor name: BERTHET, PIERRE Inventor name: VIRETTE, DAVID |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: BERTHET, PIERRE Inventor name: RAGOT, STEPHANE Inventor name: KOVESI, BALAZS Inventor name: VIRETTE, DAVID |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R079 Ref document number: 602010012084 Country of ref document: DE Free format text: PREVIOUS MAIN CLASS: G10L0019020000 Ipc: G10L0019240000 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G10L 19/24 20130101AFI20130515BHEP Ipc: G10L 19/002 20130101ALI20130515BHEP Ipc: G10L 19/02 20130101ALN20130515BHEP |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20130624 |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: ORANGE |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 642997 Country of ref document: AT Kind code of ref document: T Effective date: 20131215 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: FRENCH |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 602010012084 Country of ref document: DE Effective date: 20140123 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: VDEP Effective date: 20131127 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MK05 Ref document number: 642997 Country of ref document: AT Kind code of ref document: T Effective date: 20131127 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140227 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140327 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140327 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602010012084 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20140828 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 602010012084 Country of ref document: DE Effective date: 20140828 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: LU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140625 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140625 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140630 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20140630 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 7 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140228 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20100625 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 Ref country code: BE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20140630 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20131127 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230523 Year of fee payment: 14 Ref country code: DE Payment date: 20230523 Year of fee payment: 14 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230524 Year of fee payment: 14 |