EP1192618A1 - Codage audio avec liftrage adaptif - Google Patents
Codage audio avec liftrage adaptifInfo
- Publication number
- EP1192618A1 EP1192618A1 EP00949620A EP00949620A EP1192618A1 EP 1192618 A1 EP1192618 A1 EP 1192618A1 EP 00949620 A EP00949620 A EP 00949620A EP 00949620 A EP00949620 A EP 00949620A EP 1192618 A1 EP1192618 A1 EP 1192618A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- cepstral
- module
- audio signal
- spectrum
- frequency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/26—Pre-filtering or post-filtering
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
- G10L19/08—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters
- G10L19/10—Determination or coding of the excitation function; Determination or coding of the long-term prediction parameters the excitation function being a multipulse excitation
Definitions
- the present invention relates to the field of coding of audio signals. It applies in particular, but not exclusively, to coding of speech in narrow band or in wide band, in various ranges of coding bit rates.
- the design of an audio codec mainly aims to provide a good compromise between the bit rate of the stream transmitted by the coder and the quality of the audio signal which the decoder is capable of reconstructing from this stream.
- the coder estimates a fundamental frequency of the signal, representing its pitch, and spectral analysis consists in determining parameters representing the harmonic structure of the signal at frequencies which are integer multiples of this fundamental frequency Modeling of the non-harmonic or non-voiced component can also be carried out in the spectral domain
- the parameters transmitted to the decoder typically represent the module of the spectrum of the voiced and unvoiced components
- coder families include MBE type coders
- Multi-Band Excitation Multi-Band Excitation
- STC Seusoidal Transform Coder
- An object of the present invention is to allow, in a coding scheme with analysis in the spectral domain, to improve the signal spectrum module modeling
- the invention thus proposes a method of coding an audio signal, in which a fundamental frequency of the audio signal is estimated, a spectrum of the audio signal is determined by a transform in the field frequency of a frame of the audio signal, a compressed upper envelope of the spectrum of the audio signal is transformed in the cepstral domain to obtain cepstral coefficients, and quantification data of said cepstral coefficients are included in a digital output stream.
- the cepstral coefficients are transformed by liftrage in the cepstral domain before being quantified, a value of the modulus of the spectrum of the audio signal is recalculated at at least one frequency multiple of the fundamental frequency on the basis of the cepstral coefficients transformed , and adapting said lifter so as to minimize a modulus difference between the spectrum of the audio signal and at least one recalculated module value.
- the cepstral coefficients quantified by retrofitting by liftrage and smoothing in the cepstral domain are retransformed.
- the minimum phases of the audio signal are calculated at frequencies multiple of the fundamental frequency on the basis of the retransformed cepstral coefficients, and the adjustment of the lifter carried out before quantization is adjusted so as to minimize a difference between the spectrum of the audio signal and at least one value complex whose modulus has a recalculated value for a frequency multiple of the fundamental frequency and whose phase is given by the minimum phase calculated for said multiple frequency.
- the invention also provides an audio coder comprising means for implementing a method as defined above.
- FIG. 1 is a block diagram of an audio encoder according to the invention.
- Figures 2 and 3 are diagrams illustrating the formation of audio signal frames in the encoder of Figure 1;
- FIGS. 4 and 5 are graphs showing an example of the audio signal spectrum and illustrating the extraction of the upper and lower envelopes of this spectrum
- FIG. 6 is a block diagram of an example of quantization means usable in the encoder of Figure 1;
- FIG. 7 is a block diagram of means used to extract parameters relating to the phase of the non-harmonic component in a variant of the encoder of Figure 1;
- FIG. 8 is a block diagram of an audio decoder corresponding to the coder of FIG. 1,
- FIG. 9 is a flow diagram of an example of a procedure for smoothing spectral coefficients and extracting minimum phases implemented in the decoder of FIG. 8,
- FIG. 10 is a block diagram of analysis and spectral mixing modules of harmonic and non-harmonic components of the audio signal
- FIGS 14 and 15 are diagrams illustrating a way of proceeding to the temporal synthesis of the signal frames in the decoder of Figure 8
- - Figures 16 and 17 are graphs showing windowing functions usable in the synthesis of the frames according to Figures 14 and 15,
- FIGS. 18 and 19 are block diagrams of interpolation means which can be used in an alternative embodiment of the coder and the decoder,
- FIG. 20 is a block diagram of interpolation means which can be used in another variant embodiment of the coder.
- FIGS. 21 and 22 are diagrams illustrating another way of proceeding with the temporal synthesis of the signal frames in the decoder of FIG. 8, using an interpolation of parameters,
- FIG. 25 are block diagrams of variants of post-processing means of cepstral coefficients representing I upper envelope of the signal spectrum in the encoder of Figure 1, and - Figure 26 is a partial block diagram of a decoder associated with an encoder according to FIG. 25
- the coder and the decoder described below are digital circuits which can, as is usual in the field of audio signal processing, be produced by programming a digital signal processor (DSP) or an integrated circuit of specific application (ASIC)
- DSP digital signal processor
- ASIC integrated circuit of specific application
- the audio encoder shown in Figure 1 processes an audio signal input x which, in the nonlimiting example considered below, is a speech signal
- the signal x is available in digital form, for example at a sampling frequency F e of 8 kHz II is for example delivered by an analog-digital converter processing the amplified output signal of a microphone
- the input signal x can also be formed from another version, analog or digital, coded or not, of the speech signal
- the encoder comprises a module 1 which forms successive audio signal frames for the different processing operations carried out, and an output multiplexer 6 which delivers an output stream ⁇ containing for each frame sets of quantization parameters from which a decoder will be capable. synthesize a decoded version of the audio signal
- Each frame 2 is composed of a number N of consecutive samples of the audio signal x
- N 256
- the module 1 multiplies the samples of each frame 2 by a window function f A , preferably chosen for its good spectral properties
- a window function f A preferably chosen for its good spectral properties
- ⁇ is a coefficient for example equal to 6 and l 0 () denotes the function of Bessel with index 0.
- the coder in FIG. 1 analyzes the audio signal in the spectral domain. It includes a module 3 which calculates the fast Fourier transform (TFR) of each signal frame.
- TFR fast Fourier transform
- the TFR 3 module obtains the signal spectrum for each frame, the module and phase of which are respectively denoted
- a fundamental frequency detector 4 estimates for each signal frame a value of the fundamental frequency F 0 .
- the detector 4 can apply any known method of analysis of the speech signal of the frame to estimate the fundamental frequency F 0 , for example a method based on the autocorrelation function or the AMDF function, possibly preceded by a whitening module. by linear prediction.
- the estimation can also be performed in the spectral domain or in the cepstral domain.
- Another possibility is to evaluate the time intervals between the consecutive breaks in the speech signal attributable to closures of the glottis of the intervening speaker during the duration of the frame.
- Well-known methods which can be used to detect such micro-ruptures are described in the following articles: M.
- the estimated fundamental frequency F 0 is subject to quantification, for example scalar, by a module 5, which supplies the output multiplexer 6 with an index iF for quantizing the fundamental frequency for each frame of the signal.
- the encoder uses cepstral parametric models to represent an upper envelope and a lower envelope of the spectrum of the audio signal.
- the first step of the cepstral transformation consists in applying to the signal spectrum module a spectral compression function, which can be a logarithmic or root function.
- the coder module 8 thus operates, for each value X (i) of the signal spectrum (0 ⁇ i ⁇ N), the following transformation:
- the compressed spectrum LX of the audio signal is processed by a module 9 which extracts spectral amplitudes associated with the harmonics of the signal corresponding to the multiples of the estimated fundamental frequency F0. These amplitudes are then interpolated by a module 10 in order to obtain a compressed upper envelope denoted LX_sup.
- the spectral compression could be carried out in an equivalent manner after the determination of the amplitudes associated with the harmonics. It could also be done after interpolation, which would only change the form of the interpolation functions.
- the maxima extraction module 9 takes account of the possible variation of the fundamental frequency on the analysis frame, of the errors that the detector 4 can make, as well as of the inaccuracies linked to the discrete nature of the frequency sampling. For this, the search for the amplitudes of the spectral peaks does not simply consist in taking the values LX (i) corresponding to the indices i such that iF e / 2N is the frequency closest to a harmonic of frequency kF 0 (k> 1 ).
- Spectral amplitude retained for a harmonic of order k is a local maximum of the spectrum module in the vicinity of the frequency k F 0 (this amplitude is obtained directly in compressed form when the spectral compression 8 is carried out before the extraction of the maxima 9)
- the figures 4 and 5 show an example of the shape of the compressed spectrum
- the interpolation is carried out between points whose abscissa is the frequency corresponding to the maximum of the amplitude of a spectral peak, and whose ordinate is this maximum, before or after compression
- the interpolation performed to calculate the upper envelope LX_sup is a simple linear interpolation
- Another form of interpolation could be used (for example polynomial or spline)
- the interpolation is carried out between points whose abscissa is a frequency k F 0 multiple of the fundamental frequency (in fact the closest frequency in the discrete spectrum) and whose ordinate is the maximum amplitude, before or after compression, of the spectrum in the vicinity of this multiple frequency
- the maximum amplitude search interval associated with a harmonic of rank k is centered on the index i of the frequency of the highest TFR
- the width of this search interval depends on the sampling frequency F e , the size 2N of the TFR and the range of possible variation of the fundamental frequency. This width is typically of the order of ten frequencies with the examples of values previously considered. It can be made adjustable as a function of the value F 0 of the fundamental frequency and of the number k of the harmonic.
- a non-linear distortion of the frequency scale is operated on the upper envelope compressed by a module 12 before the module 13 performs the inverse fast Fourier transform (TFRI) providing the cepstral coefficients cx_sup.
- TFRI inverse fast Fourier transform
- the non-linear distortion makes it possible to minimize the modeling error more effectively. It is for example carried out according to a Mel or Bark type frequency scale. This distortion may possibly depend on the estimated fundamental frequency F Q.
- Figure 1 illustrates the case of the Mel scale. The relationship between the frequencies F of the linear spectrum, expressed in hertz, and the frequencies F 'of the Mel scale is as follows: c 100 °, ( ⁇ F / e ⁇
- NCS can be equal to 16.
- Post-filtering in the cepstral domain is applied by a module 15 to the compressed upper envelope LX_sup
- This post-liftring corresponds to a manipulation of the cepstral coefficients cx_sup delivered by the module of TRFI 13, which corresponds approximately to a post-filtering of the harmonic part of the signal by a function of transfer in the classic form
- a (z) is the transfer function of a linear prediction filter of the audio signal
- ⁇ 1 and ⁇ 2 are coefficients between 0 and 1
- ⁇ is a possibly zero pre-emphasis coefficient
- a normalization module 16 further modifies the cepstral coefficients by imposing the exact modeling constraint of a point on the initial spectrum, which is preferably the most energetic point among the spectral maxima extracted by the module 9 In practice, this normalization only modifies the value of the coefficient c p (0)
- the normalization module 16 operates as follows: it recalculates a value of the spectrum synthesized at the frequency of the maximum indicated by the module 9, by Fourier transform of the truncated and post-liftral cepstral coefficients, taking into account the non-linear distortion of the frequency axis, it determines a normalization gain g N by the logarithmic difference between the maximum value provided by the module 9 and this recalculated value, and it adds the gain g N to the post-liftrated cepstral coefficient cp (0 ) This standardization can be seen as part of the post-liftering
- the post-liftrated and normalized cepstral coefficients are subject to quantification by a module 18 which transmits corresponding quantization indexes icxs to the output multiplexer 6 of the coder.
- the module 18 can operate by vector quantization from cepstral vectors formed from post-liftred and normalized coefficients, denoted here cx [n] for the signal frame of rank n.
- cx [n] 16 cepstral coefficients cx [n, 0], cx [n, 1], ..., cx [n, NCS-1] is distributed in four sub - cepstral vectors each containing four coefficients of consecutive orders.
- the cepstral vector cx [n] can be processed by the means shown in FIG. 6, which are part of the quantization module 18.
- rcx_q [n-1] designates the quantized residual vector for the frame of rank n-1, whose components are respectively noted rcx_q [n, 0], rcx_q [n, 1], ... , rcx_q [n, NCS-1].
- the numerator of the relation (10) is obtained by a subtractor 20, the components of the output vector of which are divided by the quantities 2- ⁇ (i) at 21.
- the residual vector rcxfn] is subdivided into four sub-vectors, corresponding to the subdivision into four cepstral sub-vectors.
- the unit 22 proceeds to the vector quantization of each sub-vector of the residual vector rcx [n]. This quantification can consist, for each sub-vector srcxfn], in selecting in the dictionary the quantized sub-vector srcx_q [n] which minimizes the quadratic error
- the unit 22 also delivers the values of the quantized residual sub-vectors, which form the vector rcx_q [n] This is delayed by a frame at 23, and its components are multiplied by the coefficients ⁇ ( ⁇ ) at 24 for supply the vector to the negative input of the subtractor 20 This latter vector is also supplied to an adder 25, the other input of which receives a vector formed by the components of the quantized residue rcx_q [n] respectively multiplied by the quantities 1 - ⁇ ( ⁇ ) at 26
- the adder 25 thus delivers the quantized cepstral vector cx_q [n] that will recover the decoder
- the prediction coefficient ⁇ ( ⁇ ) can be optimized separately for each of the cepstral coefficients
- the quantization dictionaries can also be optimized separately for each of four cepstral sub-vectors
- it is possible, in a manner known per se, to normalize cepstral vectors before applying the prediction / quantification scheme, from the variance of cepstrums II it should be noted that the above scheme for quantifying cepstral coefficients may be applied only for some of the frames For example, it is possible to provide a second quantization mode as well as a selection process of that of the two modes which minimizes a criterion of least squares with the cepstral coefficients to be quantified, and to transmit with the quantization indexes of the frame a bit indicating which of the two modes has been selected
- the adaptation module 29 controls the post-lifter 15 so as to minimize a module gap between the spectrum of the audio signal and the corresponding module values calculated at 28
- This module gap can be expressed by a sum of absolute difference values amplitudes, comp ⁇ mated or not, corresponding to one or more of the harmonic frequencies This sum can be weighted according to the spectral amplitudes associated with these frequencies
- the modulus difference taken into account in the adaptation of the post-liftring would take into account all the harmonics of the spectrum
- the module 28 can resynthesize the spectral amplitudes only for one or more frequencies multiple of the fundamental frequency F 0 , selected on the basis of the size of the spectrum module in absolute value
- the adaptation module 29 can for example consider the three most intense spectral peaks in the calculation of the module deviation to be minimized
- the adaptation module 29 estimates a spectral masking curve of the audio signal by means of a psychoacoustic model, and the frequencies taken into account in the calculation of the module deviation to be minimized are selected on the basis the importance of the spectrum modulus relative to the masking curve (we can for example take the three frequencies for which the spectrum modulus exceeds the masking curve the most)
- Different conventional methods can be used to calculate the masking curve at using the audio signal We can for example use the one developed by JD Johnston ("Transform Coding of Audio Signais Usmg Perceptual Noise Cnteria", IEEE Journal on Selected Area in Communications, Vol 6, No 2, February 1988)
- the module 29 can use a filter identification model.
- a simpler method consists in predefining a set of sets of post-liftering parameters, that is to say a set of couples ⁇ , ⁇ 2 in the case of a post-liftring according to relations (8), to carry out the operations incumbent on modules 15, 16, 18 and 28 for each of these sets of parameters, and to retain that of the sets of parameters which leads to the minimum module deviation between the signal spectrum and the recalculated values
- the quantization indexes provided by the module 18 are then those which relate to the best set of parameters
- the coder determines coefficients cx_ ⁇ nf representing a compressed lower envelope LX nf
- a module 30 extracted from the compressed spectrum LX spectral amplitudes associated with frequencies located in regions of the intermediate spectrum with respect to the multiple frequencies of the estimated fundamental frequency F Q
- each amplitude associated with a frequency situated in an intermediate zone between two successive harmonics k F 0 and (k + 1) F 0 simply corresponds to the modulus of the spectrum for the frequency (k + 1/2) F 0 located in the middle of the interval separating the two harmonics
- this amplitude could be an average of the spectrum module over a small range surrounding this frequency (k + 1/2) F 0
- a module 31 proceeds to an interpolation, for example linear, of the spectral amplitudes associated with the frequencies located in the intermediate zones to obtain the compressed lower envelope LX_ ⁇ nf
- the cepstral transformation applied to this compressed lower envelope LX_ ⁇ nf is carried out according to a frequency scale resulting from a non-linear distortion applied by a module 32
- the non-linear transformation of the frequency scale for the cepstral transformation of the lower envelope can be carried out towards a finer scale at high frequencies than at low frequencies, which advantageously makes it possible to model well the non-voiced components of the signal at high frequencies
- the cepstral coefficients cx_ ⁇ nf representing the compressed lower envelope are quantified by a module 34, which can operate in the same way as the module 18 for quantifying the cepstral coefficients representing the compressed upper envelope
- a module 34 which can operate in the same way as the module 18 for quantifying the cepstral coefficients representing the compressed upper envelope
- the coder represented in FIG. 1 does not include any particular device for coding the phases of the spectrum with the harmonics of the audio signal. On the other hand, it includes means 36-40 for coding time information linked to the phase of the non-harmonic component represented by the lower envelope
- a spectral decompression module 36 and a TFRI module 37 form a temporal estimate of the frame of the non-harmonic component.
- the module 36 applies a reciprocal decompression function of the compression function applied by the module 8 (i.e. - say an exponential or a power function 1 / ⁇ ) to the compressed lower envelope LXjnf produced by the interpolation module 31 This provides the module of the estimated frame of the non-harmonic component, whose phase is taken equal to that ⁇ ⁇ of the spectrum of signal X on the frame
- the inverse Fourier transform performed by module 37 provides the estimated frame of the non-harmonic component
- the module 38 subdivides this estimated frame of the non-harmonic component into several time segments
- the module 38 calculates the energy equal to the sum of the squares of the samples, and forms a vector E1 formed of eight positive real components equal to the eight calculated energies
- the largest of these eight energies, denoted EM is also determined to be supplied, with the vector E1, to a normalization module 39
- This can perform vector quantization with a dictionary determined during a prior learning
- FIG. 7 shows an alternative embodiment of the means employed by the coder of FIG. 1 to determine the vector Emix of energy weighting of the frame of the non-harmonic component.
- the modules 36, 37 of spectral decompression and of TFRI operate like those which bear the same references in FIG.
- a selection module 42 is added to determine the value of the module of the spectrum subjected to the inverse Fourier transform 37 On the basis of the estimated fundamental frequency F 0 , the module 42 identifies harmonic regions and non-harmonic regions of the audio signal spectrum For example, a frequency will be considered to belong to a harmonic region if it is in a frequency range centered on a harmonic k F 0 and of width corresponding to a synthesized spectral line width , and to a non-harmonic region otherwise In non-harmonic regions, the complex signal submitted to the TFRI 37 is equal to the value of the spectrum, i.e.
- this complex signal has the same phase ⁇ ⁇ that the spectrum and a modulus given by the lower envelope after spectral decompression 36 This way of proceeding according to FIG. 7 provides a more precise modeling of the non-harmonic regions
- the decoder represented in FIG. 8 comprises an input demultiplexer 45 which extracts from the bit stream ⁇ , coming from a coder according to FIG. 1, the indexes iF, icxs, ICXI, lEm for quantizing the fundamental frequency F 0 , cepstral coefficients representing the compressed upper envelope, coefficients representing the compressed lower envelope, and the weighting vector Emix, and distribute them respectively to modules 46, 47, 48 and 49
- These modules 46-49 include similar quantization dictionaries to those of modules 5, 18, 34 and 40 of FIG.
- modules 47 and 48 have dictionaries to form the quantized prediction residues rcx_q [n], and they deduce the vectors cepstral quantified cx_q [n] with elements identical to elements 23-26 of figure 6
- cepstral vectors quantized cx_q [n] provide the cepstral coefficients cx_sup_q and cx _ ⁇ nf_q processed by the decoder
- a module 51 calculates the fast Fourier transform of the cepstral coefficients cx_sup for each signal frame
- the frequency spectrum of the resulting compressed spectrum is modified non-linearly by a module 52 applying the reciprocal non-linear transformation of that of module 12 of figure 1, and which provides the estimate LX_sup of the compressed upper envelope
- a spectral decompression of LX_sup operated by a module 53, provides the upper envelope X_sup comprising the estimated values of the spectrum module at frequencies multiple of the frequency fundamental F 0
- the module 54 synthesizes the spectral estimate X v of the harmonic component of the audio signal, by a sum of spectral lines centered on the frequencies multiple of the fundamental frequency F 0 and whose amplitudes (in module) are those given by the envelope superior X_sup
- the digital input stream ⁇ does not contain specific information on the phase of the spectrum of the signal at the harmonics of the fundamental frequency
- the decoder of figure 8 is capable of extracting information on this phase from the cepstral coefficients cx_sup_q representing the compressed upper envelope
- This phase information is used to assign a phase ⁇ (k) to each of the spectral lines determined by the module 54 in the estimation of the harmonic component of the signal
- the speech signal can be considered to be at minimum phase
- the minimum phase information can easily deduced from cepstral modeling This minimum phase information is therefore calculated for each harmonic frequency
- the minimum phase hypothesis means that the energy of the synthesized signal is localized at the start of each period of the
- module 56 deduces post-liftrated cepstral coefficients and smoothed the phase minimum assigned to each spectral line representing a harmonic peak of the spectrum
- the operations carried out by the modules 56, 57 for smoothing and extracting the minimum phase are illustrated by the flow diagram of FIG. 9
- the module 56 examines the variations of the cepstral coefficients in order to apply a lesser smoothing in the presence of sudden variations than 'in the presence of slow variations For this, it performs the smoothing of the cepstral coefficients by means of a forgetting factor ⁇ c chosen as a function of a comparison between a threshold d th and a distance d between two successive sets of cepstral coefficients post-liftrés
- the threshold d th is itself adapted according to variations in cepstral coefficients
- the first step 60 consists in calculating the distance d between the two successive vectors relating to the frames n-1 and n
- These vectors, denoted here cxp [n-1] and cxpfn] correspond for each frame to the set of NCS cepstral coefficients post-liftrés representing the compressed upper envelope
- the distance used can in particular be the Euclidean distance between the two vectors or a quadratic distance
- Two smoothings are first carried out, respectively by means of forgetting factors ⁇ m ⁇ n and ⁇ max , to determine a minimum distance d m ⁇ n and a maximum distance d max
- the forgetting factors ⁇ m ⁇ n and ⁇ max are themselves selected from two distinct values, respectively ⁇ m ⁇ n1 , ⁇ m ⁇ n2 and ⁇ max1 , ⁇ ma ⁇ 2 between 0 and 1, the indices ⁇ mm1 , ⁇ max1 each being substantially closer to 0 than the indices ⁇ m ⁇ n2 , ⁇ max2 If d> d m ⁇ n (test 61), the forget factor ⁇ m ⁇ n is equal to ⁇ m ⁇ n1 (step 62), otherwise it is taken equal to ⁇ mm2 (step 63) In step 64, the minimum distance d mm is taken equal to ⁇ m ⁇ n d mn + O ⁇ nm) d If d> d ma ⁇ (test 65), the forget factor ⁇ max is equal to ⁇ ma ⁇ 1 (step 66), otherwise it is taken equal to ⁇ max2 (step 67) In step 68, the minimum distance d ma ⁇ is taken equal to ⁇ max
- step 72 a value ⁇ c1 relatively close to 0 is adopted for the forgetting factor ⁇ c (step 72)
- the corresponding signal is of non-stationary type, so that there is no need to keep a large memory of the previous cepstral coefficients
- the module 57 calculates the minimum phases ⁇ (k) associated with the harmonics k F 0 In known manner, the minimum phase for a harmonic of
- the harmonic index k is initialized to 1
- the phase ⁇ (k) and the cepstral index m are respectively initialized to 0 and 1 in step 76
- module 57 adds to phase ⁇ (k) the quantity -2 cxl [n, m] s ⁇ n (2 ⁇ mk F 0 / F e )
- the module 54 takes into account a constant phase over the width of each spectral line, equal to the minimum phase ⁇ (k) supplied for the corresponding harmonic k by the module 57
- the estimate X v of the harmonic component is synthesized by summing spectral lines positioned at the harmonic frequencies of the fundamental frequency F 0 During this synthesis, the spectral lines can be positioned on the frequency axis with a resolution greater than the resolution of the Fourier transform For this, we precalculate once for all a reference spectral line according to the higher resolution This calculation can consist of a Fourier transform of the analysis window f ⁇ with a transform size of 16384 points, providing a resolution of 0.5 Hz per point.
- the synthesis of each harmonic line is then performed by the module 54 by positioning the high resolution reference line on the frequency axis, and by sub-sampling this reference spectral line to reduce to the resolution of 16.625 Hz of the Fourier transform on 512 points This allows to precisely position the spectral line
- the TFR module 85 of the decoder of FIG. 8 receives the NCI quantified cepstral coefficients cx_ ⁇ f_q of orders 0 to NCI - 1, and it advantageously supplements them by the NCS - NCI cepstral coefficients cx_sup_q d NCI to NCS order - 1 representing the upper envelope Indeed, we can estimate as a first approximation that the rapid variations of the compressed lower envelope are well reproduced by those of the compressed upper envelope In another embodiment, the module of TFR 85 could only consider the NCI cepstraux parameters cx_ ⁇ nf_q
- the module 86 converts the frequency scale reciprocally from the conversion operated by the module 32 of the coder, in order to restore the estimate LXjnf of the compressed lower envelope, subjected to the spectral decompression module 87 At the output of the module 87, the decoder has a lower envelope X_ ⁇ nf comprising the values of the spectrum module in the valleys located between the harmonic peaks
- This envelope X_ ⁇ nf will modulate the spectrum of a noise frame whose phase is processed as a function of the quantized weighting vector Emix extracted by the module 49
- a generator 88 delivers a normalized noise frame whose 4 ms segments are weighted in a module 89 in accordance with the normalized components of the Emix vector provided by module 49 for the current frame
- This noise is a high-pass filtered white noise to take account of the low level which in principle the non-voiced component has at low frequencies
- the Fourier transform of the resulting frame is calculated by the TFR 91 module
- the spectral estimate X uv of the non-harmonic component is determined by the spectral synthesis module 92 which performs frequency-by-frequency weighting. This weighting consists in multiplying each complex spectral value supplied by the TFR module 91 by the value of the lower envelope Xjnf obtained for the same frequency by the spectral decompression module 87
- the analysis module 96 comprises a unit 97 for estimating a degree of voicing W dependent on the frequency, from which four gains dependent on the frequency, namely two gains g v , g uv controlling the relative importance of the harmonic and non-harmonic components in the synthesized signal, and two gains g v g uv used to noise the phase of the harmonic component
- the degree of voicing W ( ⁇ ) is a continuously variable value between 0 and 1 determined for each frequency index i (0 ⁇ i ⁇ N) as a function of the upper envelope X_sup ( ⁇ ) and the lower envelope X_ ⁇ nf ( ⁇ ) obtained for this frequency i by the decompression modules 53, 87
- the degree of voicing W ( ⁇ ) is estimated by the unit 97 for each frequency index i corresponding to a harmonic of the fundamental frequency F 0 ,
- the threshold Vth (F 0 ) corresponds to the average dynamics calculated on a synthetic spectrum purely voiced at the fundamental frequency II is advantageously chosen depending on the fundamental frequency F 0
- the degree of voicing W ( ⁇ ) for a frequency other than the harmonic frequencies is obtained simply as being equal to that estimated for the nearest harmonic
- the gain g v ( ⁇ ), which depends on the frequency, is obtained by applying a non-linear function to the degree of voicing W ( ⁇ ) (block 98)
- phase ⁇ v of the component mixed harmonic is the result of a linear combination of the phases ⁇ v , ⁇ uv of the harmonic and non-harmonic components X v , X uv synthesized by the modules 54, 92
- the gains g v g uv respectively applied to these phases are calculated at starting from the degree of voicing W and also weighted as a function of the frequency index i, since the sound effects of the phase are only really useful beyond a certain frequency
- a first gain g v1 is calculated by applying a non-linear function to the degree of voicing W ( ⁇ ), as shown diagrammatically by block 100 in FIG. 10
- This non-linear function can have the form shown in FIG. 12
- g v1 _ ⁇ (.) G1 if 0 ⁇ W ( ⁇ ) ⁇ W3
- the thresholds W3 and W4 being such that 0 ⁇ W3 ⁇ W4 ⁇ 1, and the minimum gain G1 being between 0 and 1
- a multiplexer 101 multiplies for each index frequency i the gain g v1 by another gain g v2 depending only on the frequency index i, to form the gain g v (i)
- the gain g v2 (i) depends not -linearly of the frequency index i, for example as indicated in FIG. 13
- ⁇ Xp ⁇ ⁇ ' v)] + QuvO x uv (( 17 ) with ⁇ ⁇ i) g v _ ⁇ ( ⁇ ) ⁇ v (+ g uv _ ⁇ ( ⁇ uv 0) ( 1 8 ) where ⁇ v ( ⁇ ) denotes the argument of the complex number X v ( ⁇ ) supplied by the module 54 for the frequency of index i (block 104 of FIG.
- the frames successively obtained in this way are finally processed by the time synthesis module 116 which forms the decoded audio signal x
- the time synthesis module 116 performs a sum at overlapping of frames modified with respect to those successively evaluated at the output of module 115.
- the modification can be seen in two stages illustrated respectively in FIGS. 14 and 15.
- the first step (FIG. 14) consists in multiplying each frame 2 ′ delivered by the TFRI module 115 by a window 1 / f A opposite to the analysis window f A used by the module 1 of the coder. Frame samples
- each sample of the decoded audio signal x thus obtained is assigned a uniform overall weight, equal to A.
- This overall weight comes from the contribution of a single frame if the sample has in this frame a rank i such that L ⁇ i ⁇ N - L, and includes the summed contributions of two successive frames if 0 ⁇ i ⁇ L where N - L ⁇ i ⁇ NOT.
- 2 'of N 256 samples delivered by module 115 through the compound window f c before performing the overlapping summation.
- FIG. 16 shows the appearance of the compound window f c in the case where the analysis window f A is a Hamming window and the synthesis window f s has the form given by the relations (19) to (21) .
- Other forms of the summary window f s verifying the relations (19) and (20) can be used.
- the coder in FIG. 1 can increase the rate of formation and analysis of the frames, in order to transmit more quantization parameters to the decoder.
- a frame of N 256 samples (32 ms) is formed every 20 ms.
- the notations cx_q [n-1] and cX-qf ⁇ ] denote quantized cepstral vectors determined, for two successive frames of whole rank, by the quantization module 18 and / or by the quantization 34. These vectors comprise for example four consecutive cepstral coefficients each. They could also include more cepstral coefficients.
- a module 120 performs an interpolation of these two cepstral vectors cx_q [n-1] and cx_q [n], in order to estimate an intermediate value cx_i [n-1/2].
- the interpolation performed by the module 120 can be a simple arithmetic mean of the vectors cx_q [n-1] and cx_q [n]
- the module 120 could apply a more sophisticated interpolation formula, for example polynomial, also relying on the cepstral vectors obtained for frames prior to the frame n-1
- the interpolation takes into account the relative position of each interpolated frame
- the coder uses the means described above to calculate the cepstral coefficients cx [n-1/2] relating to the half-integer row frame.
- these cepstral coefficients are those supplied by the TFR1 module 13 after post-liftering 15 (for example with the same post-liftering coefficients as for the previous frame n-1) and normalization 16
- the cepstral coefficients cx [n-1 / 2] are those delivered by the TFRI 33 module
- a subtractor 121 forms the difference ecx [n-1/2] between the cepstral coefficients cx [n-1/2] calculated for the half-integer row frame and the coefficients cx_ ⁇ [n-1/2] estimated by interpolation
- This difference is provided to a quantization module 122 which addresses quantization indices ⁇ cx [n-1/2] to the output multiplexer 6 of the encoder.
- the module 122 operates for example by vector quantization of ecx interpolation errors [n-1 / 2] successively determined for half-integer rows
- the decoder functions essentially like that described with reference to FIG. 8 to determine the signal frames of whole rank.
- An interpolation module 124 identical to the module 120 of the encoder estimates the intermediate coefficients cx_ ⁇ [n-1/2] from the coefficients quantified cx_q [n-1] and cx_q [n] supplied by the module 47 and / or the module 48 from the indexes icxs, icxi extracted from the stream ⁇
- a module for extracting parameters 125 receives the quantization index ⁇ cx [ n-1/2] from the input demultiplexer 45 of the decoder, and deduces the quantized interpolation error ecx_q [n-1/2] from the same quantization dictionary as that used by the module 122 of the coder
- An adder 126 sums the cepstral vectors cx_ ⁇ [n-1/2] and ecx_q [ ⁇ -1/2] in order to provide the cepstral coefficients cx [n-1/2] which will be
- the decoder can also interpolate the other parameters F 0 , Emix used to synthesize the signal frames.
- the fundamental frequency F 0 can be interpolated linearly, either in the time domain, or (preferably) directly in the frequency domain.
- the interpolation should be carried out after denormalization and of course taking account of the time offsets between frames.
- the coder uses the cepstral vectors cx_q [n], cx_q [n-1], ..., cx_q [nr] and cx_q [n-1/2] calculated for the last frames passed (r> 1) to identify an optimal interpolator filter which, when subject to the quantized cepstral vectors cx_q [nr], ..., cx_q [n] relating to frames of whole rank, delivers an interpolated cepstral vector cx_i [n -1/2] which has a minimum distance with the vector cx [n-1/2] calculated for the last frame of rank half-integer.
- this interpolator filter 128 is present in the coder, and a subtractor 129 subtracts its output cx_i [n-1/2] from the calculated cepstral vector cx [n-1/2].
- a minimization module 130 determines the set of parameters ⁇ P ⁇ of the interpolator filter 128, for which the interpolation error ecx [n-1/2] delivered by the subtractor 129 has a minimum standard. This set of parameters ⁇ P ⁇ is addressed to a quantization module 131 which provides a corresponding quantization index iP to the output multiplexer 6 of the coder.
- the decoder From the quantization indexes iP of the parameters ⁇ P ⁇ obtained in the bit stream ⁇ , the decoder reconstructs the interpolator filter 128 (except for quantization errors), and processes the spectral vectors cx_q [nr], ..., cx_q [ n] in order to estimate the cepstral coefficients cx [n-1/2] used to synthesize the half-integer rank frames.
- the decoder can use a simple interpolation method (without transmission of parameters from the coder for half-integer rank frames), an interpolation method with an interpolation error taken into account. quantized (according to Figures 17 and 18), or an interpolation method with an optimal interpolator filter (according to Figure 19) to evaluate the half-integer rank frames in addition to the whole rank frames evaluated directly as explained with reference to FIGS. 8 to 13.
- the time synthesis module 1 16 can then combine all of these evaluated frames to form the synthesized signal x in the manner explained below with reference to FIGS. 14, 21 and 22.
- the module 1 16 performs an overlap sum of modified frames with respect to those successively evaluated at the output of the module 1 15, and this modification can be seen in two stages, the first of which is identical to that previously described with reference to FIG. 14 (divide the samples of the frame 2 ′ by the analysis window f /).
- fs (i) + f s (i + M / p) A for N / 2 - M / p ⁇ i ⁇ N / 2 (25)
- the summary window f s (i) gradually increases for i ranging from N / 2 - M / p to N / 2. It is for example a sinusoid raised on the interval
- the synthesis window f s can be, over this interval, a Hamming window (as shown in FIG. 21) or a Hanning window.
- FIG. 21 shows the successive frames 2 "repositioned in time by the module 116.
- the hatching indicates the portions eliminated from the frames (summary window at 0). It can be seen that by performing the overlapping sum of the samples of the successive frames, the property (25) ensures a homogeneous weighting of the samples of the synthesized signal.
- the interpolated frames can be the subject of a reduced transmission of coding parameters, as described above, but this is not compulsory.
- This embodiment makes it possible to maintain a relatively large interval M between two analysis frames, and therefore to limit the required transmission rate, while limiting the discontinuities likely to appear due to the size of this interval relative to the scales. of time typical of variations in audio signal parameters, including cepstral coefficients and fundamental frequency.
- Figures 23 to 25 show other embodiments of the means used to process the cepstral coefficients cx_sup delivered by the TFRI module 13 of Figure 1, representing the upper envelope.
- post-lifter 15, normalization 16, quantization 18 and spectral amplitude 28 calculation modules are essentially identical to those previously described with reference to FIG. 1.
- post-lifter modules 140, smoothing. 141 and minimal phase extraction 142 are provided to process the post-raised and quantified cepstral coefficients cx_sup_q delivered by the quantization module 18.
- These modules 140-142 operate essentially like the corresponding modules 55-57 of the decoder of FIG. 8 .
- the adaptation module 144 performs a function similar to that of the module 29 in FIG. 1. But the adaptation is not carried out solely on the basis of the spectrum module.
- the module 144 determines the best set of coefficients for the post-lifter 15 by minimizing the difference between the spectrum of the audio signal, in module
- the modules of these latter complex values are given by the calculation module 28, and their phases correspond to the minimum phases ⁇ (k) supplied by the extraction module 142.
- the module 144 can take into account any appropriate distance in the complex plane, for example the Euclidean distance.
- the adaptation of the post-lifter 15 by the module 144 takes into account in a combined manner frequency aspects of the signal, reflected by the spectrum module, and temporal aspects, reflected by the phase of the spectrum.
- the post-lifter 140 can also be adaptive, the adaptation carried out by the module 144 bearing jointly on the two post-lifters 15, 140.
- the post-lifter 55 of the decoder ( Figure 8) is adapted, like the post-lifter 140, according to parameters i ⁇ f that the adaptation module 144 provides to the multiplexer 6 so that it includes them in the digital stream ⁇ .
- a few sets of coefficients ⁇ . ,, ⁇ 2 are provided for the post-lifter 140 and 55, and the module 144 performs an exhaustive test of these different sets of coefficients to retain the one that minimizes the deviation in the complex plane.
- FIG. 24 shows a module 145 for estimating a masking curve allowing the module 29 to select, for the minimization of the difference in module, the harmonic frequency or frequencies which exceed most of the masking curve calculated on the basis of the spectrum in module
- the post-lifter 140 in FIG. 24 is adapted separately by a module 146 which minimizes the differences between the phase ⁇ ⁇ of the signal spectrum and the minimum phase ⁇ (k) calculated by the module 142 for one or more of the harmonics.
- the harmonics selected for the calculation of the minimized phase difference can be selected as a function of the masking curve estimated by the module 145.
- the module 146 supplies the output multiplexer 6 of the coder with the iLif parameters which represent the optimal post-lifter 140, so that they are used in the post-filter 55 of the decoder.
- the post-lifter 140 used in the calculation of the minimum phases is not adaptive.
- the minimum phases ⁇ (k) calculated by the module 142 for the harmonics of the fundamental frequency are compared with the phases ⁇ ⁇ of the spectrum of the audio signal, and the phase difference is the subject of a quantification by a module 148.
- the corresponding quantization indices i ⁇ are supplied by the module 148 to the output multiplexer 6 of the encoder.
- a module 149 uses these quantization indices i ⁇ supplied by the demultiplexer 45 to obtain the values of the quantized phase differences, which an adder 150 adds to the minimum phases ⁇ (k) calculated by module 57 (the post-lifter 140 and 55 being identical). The phases provided by the adder 150 are then used by the module 54 which synthesizes the spectral lines of the harmonic component X v .
- the phase difference quantified by the module 148, and which the modules 149 and 150 of the decoder use to correct the minimum phases ⁇ (k), can be of two types: - it can represent, for each frequency of index i corresponding to a harmonic of order k of the fundamental frequency F 0 the difference between the phase ⁇ ⁇ (i) of the spectrum of the signal at frequency i and the minimum phase ⁇ ( k) calculated by the module 142 for the harmonic k; - alternatively or cumulatively, this phase difference can represent the variation of the phase ⁇ x of the spectrum over the width of one or more spectral peaks corresponding to signal harmonics, this variation being relative to the minimum phase ⁇ (k) affected to the peaks in question.
- the peak (s) for which the phase difference is quantified can be chosen as a function of the spectral energy represented by the upper envelope, which is available to the coder and to the decoder, which allows the decoder to determine to which spectral line the deviations must be applied.
- the phase differences can be the subject of a scalar quantification, or vector if they are grouped for several peaks.
- the variation of the phase ⁇ ⁇ around the minimum phase ⁇ (k) over the width of a harmonic peak can be represented simply by the slope of a linear segment selected as being that which presents a minimum quadratic distance with the phase variation curve of the spectrum over the width of the line, and possibly by an offset at the origin.
- slopes can be scaled, or vectorial if they are grouped for several peaks.
- the quantification of phase variations on harmonic peaks can relate to all of the harmonic frequencies. Another possibility is to quantify several slopes each obtained by averaging the harmonic slopes on one or more sub-bands of the spectrum. This averaging can be weighted to take into account the energies relating to the different harmonic frequencies, represented by the upper envelope.
- the module 148 can also model the phase variation over the width of a peak by a curve more complex than a linear segment, by example a spline, whose parameters are quantified to be transmitted to the decoder.
- phase models with harmonics representative of the phase variations over the width of the peaks, observed on a corpus of reference signals.
- These models are stored in a dictionary stored by the modules 148 and 149.
- the coder module 148 determines the i ⁇ index indexes corresponding to the addresses of the models closest to the phase variations in the vicinity of the harmonic peaks considered, and the module 149 of the decoder recovers these models for the synthesis of the phase of the harmonic component.
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR9908637A FR2796193B1 (fr) | 1999-07-05 | 1999-07-05 | Procede et dispositif de codage audio |
FR9908637 | 1999-07-05 | ||
PCT/FR2000/001905 WO2001003117A1 (fr) | 1999-07-05 | 2000-07-04 | Codage audio avec liftrage adaptif |
Publications (2)
Publication Number | Publication Date |
---|---|
EP1192618A1 true EP1192618A1 (fr) | 2002-04-03 |
EP1192618B1 EP1192618B1 (fr) | 2004-09-22 |
Family
ID=9547706
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP00949620A Expired - Lifetime EP1192618B1 (fr) | 1999-07-05 | 2000-07-04 | Codage audio avec liftrage adaptif |
Country Status (6)
Country | Link |
---|---|
EP (1) | EP1192618B1 (fr) |
AT (1) | ATE277402T1 (fr) |
AU (1) | AU6291900A (fr) |
DE (1) | DE60014084D1 (fr) |
FR (1) | FR2796193B1 (fr) |
WO (1) | WO2001003117A1 (fr) |
Family Cites Families (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH03136100A (ja) * | 1989-10-20 | 1991-06-10 | Canon Inc | 音声処理方法及び装置 |
-
1999
- 1999-07-05 FR FR9908637A patent/FR2796193B1/fr not_active Expired - Fee Related
-
2000
- 2000-07-04 DE DE60014084T patent/DE60014084D1/de not_active Expired - Lifetime
- 2000-07-04 WO PCT/FR2000/001905 patent/WO2001003117A1/fr active IP Right Grant
- 2000-07-04 EP EP00949620A patent/EP1192618B1/fr not_active Expired - Lifetime
- 2000-07-04 AT AT00949620T patent/ATE277402T1/de not_active IP Right Cessation
- 2000-07-04 AU AU62919/00A patent/AU6291900A/en not_active Abandoned
Non-Patent Citations (1)
Title |
---|
See references of WO0103117A1 * |
Also Published As
Publication number | Publication date |
---|---|
ATE277402T1 (de) | 2004-10-15 |
WO2001003117A1 (fr) | 2001-01-11 |
FR2796193B1 (fr) | 2001-10-05 |
AU6291900A (en) | 2001-01-22 |
DE60014084D1 (de) | 2004-10-28 |
FR2796193A1 (fr) | 2001-01-12 |
EP1192618B1 (fr) | 2004-09-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP0782128B1 (fr) | Procédé d'analyse par prédiction linéaire d'un signal audiofréquence, et procédés de codage et de décodage d'un signal audiofréquence en comportant application | |
EP1593116B1 (fr) | Procédé pour le traitement numérique différencié de la voix et de la musique, le filtrage de bruit, la création d'effets spéciaux et dispositif pour la mise en oeuvre dudit procédé | |
EP1692689B1 (fr) | Procede de codage multiple optimise | |
EP0801790B1 (fr) | Procede de codage de parole a analyse par synthese | |
WO2007093726A2 (fr) | Dispositif de ponderation perceptuelle en codage/decodage audio | |
EP0428445B1 (fr) | Procédé et dispositif de codage de filtres prédicteurs de vocodeurs très bas débit | |
FR2784218A1 (fr) | Procede de codage de la parole a bas debit | |
WO2004088633A1 (fr) | Procede d'analyse d'informations de frequence fondamentale et procede et systeme de conversion de voix mettant en oeuvre un tel procede d'analyse | |
EP1192619B1 (fr) | Codage et decodage audio par interpolation | |
EP0616315A1 (fr) | Dispositif de codage et de décodage numérique de la parole, procédé d'exploration d'un dictionnaire pseudo-logarithmique de délais LTP, et procédé d'analyse LTP | |
EP1192618B1 (fr) | Codage audio avec liftrage adaptif | |
EP1192621B1 (fr) | Codage audio avec composants harmoniques | |
EP1190414A1 (fr) | Codage et decodage audio avec composantes harmoniques et phase minimale | |
EP1194923B1 (fr) | Procedes et dispositifs d'analyse et de synthese audio | |
WO2001003119A1 (fr) | Codage et decodage audio incluant des composantes non harmoniques du signal | |
WO2023165946A1 (fr) | Codage et décodage optimisé d'un signal audio utilisant un auto-encodeur à base de réseau de neurones | |
WO2013135997A1 (fr) | Modification des caractéristiques spectrales d'un filtre de prédiction linéaire d'un signal audionumérique représenté par ses coefficients lsf ou isf | |
FR2773653A1 (fr) | Dispositifs de codage/decodage de donnees, et supports d'enregistrement memorisant un programme de codage/decodage de donnees au moyen d'un filtre de ponderation frequentielle | |
EP0454552A2 (fr) | ProcédÀ© et dispositif de codage bas débit de la parole | |
FR2980620A1 (fr) | Traitement d'amelioration de la qualite des signaux audiofrequences decodes | |
FR2739482A1 (fr) | Procede et dispositif pour l'evaluation du voisement du signal de parole par sous bandes dans des vocodeurs | |
WO2002029786A1 (fr) | Procede et dispositif de codage segmental d'un signal audio | |
FR2737360A1 (fr) | Procedes de codage et de decodage de signaux audiofrequence, codeur et decodeur pour la mise en oeuvre de tels procedes |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20020103 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL |
|
AX | Request for extension of the european patent |
Free format text: AL;LT;LV;MK;RO;SI |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: NORTEL NETWORKS FRANCE |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040922 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED. Effective date: 20040922 Ref country code: IE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040922 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040922 Ref country code: AT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20040922 |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: FRENCH |
|
REF | Corresponds to: |
Ref document number: 60014084 Country of ref document: DE Date of ref document: 20041028 Kind code of ref document: P |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041222 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041222 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041222 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20041223 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20050102 |
|
GBT | Gb: translation of ep patent filed (gb section 77(6)(a)/1977) |
Effective date: 20041230 |
|
LTIE | Lt: invalidation of european patent or patent extension |
Effective date: 20040922 |
|
NLV1 | Nl: lapsed or annulled due to failure to fulfill the requirements of art. 29p and 29m of the patents act | ||
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FD4D |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20050614 Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050704 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20050704 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20050706 Year of fee payment: 6 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050731 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050731 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050731 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050731 |
|
26N | No opposition filed |
Effective date: 20050623 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20060704 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20060704 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: ST Effective date: 20070330 |
|
BERE | Be: lapsed |
Owner name: *NORTEL NETWORKS FRANCE Effective date: 20050731 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20050222 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20060731 |