EP2951813B1 - Improved correction of frame loss when decoding a signal - Google Patents

Description

The present invention relates to a signal correction, in particular in a decoder, in the event of loss of frame upon reception of the signal by this decoder.

The signal is in the form of a succession of samples, divided into successive frames and "frame" is then understood to mean a signal segment composed of one or more samples (an embodiment where a frame comprises a single sample being possible if the signal is in the form of a series of samples, for example in codecs according to ITU-T Recommendation G.711).

The invention lies in the field of digital signal processing, in particular but not exclusively in the field of coding / decoding of an audio signal. Frame loss occurs when communication (either real-time transmission or storage for later transmission) using an encoder and a decoder is disturbed by channel conditions (because of radio problems, access network congestion, etc.).

In this case, the decoder uses frame loss correction mechanisms (or "masking") to try to substitute the missing signal with a reconstituted signal, using the information available within the decoder (for example the already decoded signal or parameters received in previous frames). This technique maintains a good quality of service despite degraded channel performance.

Frame loss correction techniques are most often very dependent on the type of coding used.

In the case of coding a speech signal based on CELP (Code Excited Linear Prediction) technologies, the frame loss correction exploits in particular the CELP model. For example, in ITU-T G.722.2 coding, the solution to replace a lost frame (or a "packet") is to prolong the use of a long-term prediction gain by attenuating it, and to extend the use of each ISF parameter (for "Imittance Spectral Frequency") by making them tend towards their respective averages. The pitch of the speech signal (or "pitch", parameter designated "LTP lag") is also repeated. In addition, the decoder of the random values of parameters characterizing "innovation" (excitation in CELP coding).

It should be noted already that the application of this type of method, for transform coding or "PCM" or "ADPCM" type waveform coding, requires a CELP parametric analysis of the signal passed to the decoder level, which introduces additional complexity.

In ITU-T Recommendation G.711 corresponding to a waveform encoder, an informative example of frame loss correction processing (given in the Appendix I part of the text of this recommendation) is to find a pitch period in the already decoded speech signal and to repeat the last overlap-add pitch period between the already decoded signal and the repeated signal (reconstructed by masking). This processing makes it possible to "erase" the audio artifacts but requires an additional delay to the decoder (delay corresponding to the duration of the recovery).

The most used technique for correcting frame loss in the case of transform coding is to repeat the decoded spectrum in the last received frame. For example, in the case of ITU-T G.722.1 coding, the modulated lapped transform (MLT) is equivalent to a modified discrete cosine transform (MDCT) with a 50% overlap and sinusoidal analysis / synthesis windows, ensures a transition (between the last lost frame and the repeated frame) which is slow enough to erase the artifacts related to the simple repetition of the spectrum; typically, if more than one frame is lost, the repeated spectrum is set to zero.

Advantageously, this masking method does not require additional delay since it exploits the overlap-addition between the reconstituted signal and the passed signal to achieve a sort of "crossfade" (with time folding due to the MLT transform). This is a very inexpensive technique in terms of resources.

However, it has a defect related to the time inconsistency between the signal just before the frame loss and the repeated signal. This results in discontinuity (or incoherence) of phase, which can produce significant audio artifacts if the overlap time between the signals associated with two frames is reduced (as is particularly the case when MDCT windows called "low" delay 'are used). This situation of short duration of recovery on the Figure 1B in the case of a low delay MLT transform, compared to the usual situation of the Figure 1A in which long sinus windows are used according to the recommendation G.722.1 (thus offering a long duration of recovery ZRA, with a very progressive modulation). It appears that a modulation by a low delay window produces a phase shift that is audible due to a short overlap zone ZRB, as shown in FIG. Figure 1B .

In this case, even if a solution combining a pitch search (case of the decoding according to the recommendation G.711 App.I) and a recovery-addition produces the window of an MDCT transform would be implemented, it would not be possible. not enough to remove the audio artifacts related in particular to the phase difference between different frequency components.

Another example of solution to the problem of correcting the loss of frames when decoding an audio signal is proposed in " Frame Erasure Concealment Using Sinusoidal Analysis-Synthesis and Its Application to MDCT-Based Codecs "by Parikh et al., In Acoustics, Speech, and Signal Processing, 2000, ICASSP '00 .

The present invention improves the situation.

1. a) recherche, dans un signal valide disponible au décodage, d'un segment de signal, de durée correspondant à une période déterminée en fonction dudit signal valide,
2. b) analyse spectrale du segment, pour une détermination de composantes spectrales du segment,
3. c) synthèse d'au moins une trame de remplacement de la trame perdue, par construction d'un signal de synthèse à partir d'une partie au moins des composantes spectrales.

To this end, it proposes a method of processing a signal comprising a succession of samples distributed in successive frames, the method being implemented during a decoding of said signal to replace at least one lost signal frame at decoding. In particular, the method comprises the steps:

1. a) search, in a valid signal available for decoding, of a signal segment, of duration corresponding to a period determined according to said valid signal,
2. b) spectral analysis of the segment, for a determination of spectral components of the segment,
3. c) synthesis of at least one replacement frame of the lost frame, by constructing a synthesis signal from at least a part of the spectral components.

Here, "frame" is understood to mean a block of at least one sample. In most codecs, these frames consist of several samples. However, in particular codecs of the PCM (for "Pulse Code Modulation") type, for example according to the G.711 recommendation, the signal consists simply of a succession of samples (a "frame" within the meaning of the invention with only one sample). The invention can then also be applied to this type of codecs.

For example, the valid signal may consist of the last valid frames received before the frame loss. Optionally, one can also use one or more following valid frames, received after the lost frame (although such an implementation leads to a delay in decoding). The samples of the valid signal that are used can be directly those of the frames, and possibly those which correspond to the memory of the transform and which typically contain a folding (or "aliasing") in the case of a transforming decoding with type MLT or MDCT overlay.

The invention then provides an advantageous solution to the loss of frame correction (s), especially in the case where an additional delay to the decoder is prohibited, for example when using a decoder transform with windows not allowing have a sufficiently large overlap between the substitution signal and the signal resulting from the temporal unfolding (typical case of low delay windows for an MDCT or an MLT, as shown in FIG. Figure 1B ). The invention offers a particular advantage for an overlap, because of the use of the spectral components on the last valid frames received to construct a synthesis signal comprising the spectral coloring of these last valid frames. Nevertheless, the invention applies of course to any type of coding / decoding (by transform, CELP, PCM, or other).

In one embodiment, the method comprises searching, by correlation in the valid signal, of a repetition period, the duration of the aforementioned segment then comprising at least one repetition period.

Such a "repetition period" corresponds for example to a pitch period in the case of a voiced speech signal (inverse of the fundamental frequency of the signal). Nevertheless, the signal may also be derived from a music signal, for example, having a global tone associated with a fundamental frequency, as well as a fundamental period that could correspond to the aforementioned repetition period.

For example, it may be necessary to search for a repetition period related to the tone of the signal. For example, we can build a first buffer (or "buffer" in French) of the last few validly received samples and search by correlation in a second buffer of larger size, the few samples of the second buffer that best fit in their succession to those of the first buffer. The time difference between these identified samples of the second buffer and those of the first buffer may constitute a repetition period or a multiple of this period (depending on the fineness of the correlation search). It may be noted that the fact of taking a multiple of the repetition period does not degrade the implementation of the invention, because, in this case, the spectral analysis is simply done over a length covering several periods instead of only one, which helps to increase the fineness of the analysis.

• une durée correspondant à une période de répétition (si une tonalité du signal est bien identifiable),
• une durée correspondant à plusieurs périodes de répétition (cycles de pitch par exemple), si la corrélation donne un premier résultat de corrélation supérieur à un seuil prédéterminé, comme expliqué dans un mode de réalisation optionnel ci-après,
• une durée arbitraire de signal (par exemple quelques dizaines d'échantillons), si une telle tonalité n'est pas identifiable (signal comportant essentiellement du bruit).

Thus, it is possible to determine the signal duration on which the spectral analysis is performed as being:

• a duration corresponding to a repetition period (if a signal tone is clearly identifiable),
• a duration corresponding to several repetition periods (pitch cycles for example), if the correlation gives a first correlation result greater than a predetermined threshold, as explained in an optional embodiment below,
• an arbitrary duration of signal (for example a few tens of samples), if such a tone is not identifiable (signal comprising essentially noise).

In a particular embodiment, the aforementioned repetition period corresponds to a duration for which the correlation exceeds a predetermined threshold value. Thus, in this embodiment, the duration of the signal is identified as soon as the correlation exceeds a predetermined threshold value for this duration. The duration thus identified corresponds to one or more periods associated with a frequency of the above-mentioned overall tone. Such an embodiment advantageously makes it possible to limit the complexity of the search by correlation (for example, by fixing a correlation threshold at 60 or 70%), even if in fact one detects not only one, but several pitch periods (for example between two and five pitch periods). On the one hand, the complexity of the search by correlation is then lower. On the other hand, the spectral analysis over several periods is finer and the spectral components obtained are more finely analyzed.

As regards obtaining the spectral components by segment analysis (for example by fast Fourier transform, or "FFT"), the method further comprises a determination of the respective phases associated with these spectral components and the construction of the synthesis signal then comprises the phases of the spectral components. The construction of the signal then integrates these phases, as will be seen later, for an optimization of the connection of the synthesis signal to the last valid frames and, in most natural cases, to the following valid frames.

In a particular embodiment also, the method further comprises a determination of respective amplitudes associated with the spectral components, and the construction of the synthesis signal comprises these amplitudes of the spectral components (for their inclusion in the construction of the synthesis signal).

In a particular embodiment, it is possible to select components resulting from the analysis for the construction of the synthesis signal. For example, in an embodiment where the process comprises a determination of respective amplitudes associated with the spectral components, the spectral components of the highest amplitudes may be those selected for the construction of the synthesis signal. It is also possible to select, in addition or alternatively, those whose amplitude forms a peak in the frequency spectrum.

In the case where only a part of the spectral components is selected, in a particular embodiment, noise is added to the synthesis signal to compensate for a loss of energy relative to spectral components not selected for the construction of the synthesis signal.

In one embodiment, the aforementioned noise is obtained by a weighted (temporally) residual between the segment signal and the synthesis signal. For example, it may be weighted by overlapping windows, as in the case of overlap transformation encoding / decoding.

The spectral analysis of the segment comprises a Fast Fourier Transform (FFT) sinusoidal analysis, preferably of length 2 k, where k is greater than or equal to log ₂ (P), where P is the number of samples in the signal. Such an embodiment reduces the complexity of the treatment, as detailed below. It may be noted that other transforms are possible, for example a Modulated Complex Lapped Transform (MCLT) transform as a possible alternative to the FFT transform.

• une interpolation des échantillons du segment pour obtenir un deuxième segment comportant un nombre d'échantillons 2^ceil(log₂(P)), où ceil(x) est l'entier supérieur ou égal à x,
• un calcul de la transformée de Fourier du deuxième segment ; et
• après détermination des composantes spectrales, identification de fréquences associées aux composantes, et construction du signal de synthèse par ré-échantillonnage avec modification desdites fréquences en fonction du ré-échantillonnage.

In particular, it can be provided in the spectral analysis step:

• an interpolation of the samples of the segment to obtain a second segment comprising a number of samples 2 ^ ceil (log ₂ (P)), where ceil (x) is the integer greater than or equal to x,
• a calculation of the Fourier transform of the second segment; and
• after determination of the spectral components, identification of frequencies associated with the components, and construction of the synthesis signal by resampling with modification of said frequencies as a function of resampling.

The present invention finds an advantageous but in no way limiting application to the context of decoding by transform with overlap. In such a context, it may be advantageous for the synthesis signal to be constructed (repeated) over a period of at least two frames, so as to cover also the parts having an aliasing beyond one frame.

In a particular embodiment, the synthesis signal can be constructed over two frame times and still an additional duration corresponding to a delay introduced by a resampling filter (in particular in the embodiment described above and where a resampling is planned).

It may be advantageous to manage a jitter buffer in some embodiments. In the case where the frame loss correction is carried out jointly with the management of a jitter buffer, the invention can then be applied under these conditions by adapting the duration of the synthesis signal.

In one embodiment, the method further comprises a separation in a high frequency band and a low frequency band, of the signal coming from the valid frame (s), and the spectral components are selected in the band of low frequencies. Such an embodiment makes it possible to limit the complexity of the processing essentially to the low frequency band, the high frequencies providing little spectral richness to the synthesis signal and which can be repeated in a simpler way.

• d'un premier signal construit à partir de composantes spectrales sélectionnées dans la bande de fréquences basses, et
• d'un deuxième signal issu d'un filtrage dans la bande de fréquences hautes,

le deuxième signal étant obtenu par duplication successive d'au moins une demi-trame valide et sa version retournée temporellement.In this embodiment, the replacement frame can be synthesized by addition:

• a first signal constructed from selected spectral components in the low frequency band, and
• a second signal resulting from a filtering in the high frequency band,

the second signal being obtained by successive duplication of at least one valid half-frame and its version returned temporally.

The present invention is also directed to a computer program comprising instructions for implementing the method (of which, for example, a general flow chart may be the general diagram of the figure 2 , and possibly specific flow charts of figures 5 and / or 8 in some embodiments).

1. a) des moyens de recherche, dans un signal valide disponible au décodage, d'un segment de signal, de durée correspondant à une période déterminée en fonction dudit signal valide,
2. b) des moyens d'analyse spectrale du segment, pour une détermination de composantes spectrales du segment,
3. c) des moyens de synthèse d'au moins une trame de remplacement de la trame perdue, par construction d'un signal de synthèse à partir d'une partie au moins des composantes spectrales.

The present invention also relates to a device for decoding a signal comprising a succession of samples distributed in successive frames, the device comprising means for replacing at least one lost signal frame, comprising:

1. a) search means, in a valid signal available at decoding, of a signal segment, of duration corresponding to a determined period as a function of said valid signal,
2. b) spectral analysis means of the segment, for a determination of spectral components of the segment,
3. c) means for synthesizing at least one replacement frame of the lost frame, by constructing a synthesis signal from at least a part of the spectral components.

Such a device can take the physical form of, for example, a processor and possibly a working memory, typically in a communication terminal.

• la figure 1A illustre un recouvrement avec des fenêtres classiques dans le cadre d'une transformée MLT,
• la figure 1B illustre un recouvrement avec des fenêtres à faible retard, en comparaison de la représentation de la figure 1A,
• la figure 2 représente un exemple de traitement général au sens de l'invention,
• la figure 3 illustre la détermination d'un segment de signal correspondant à une période fondamentale,
• la figure 4 illustre la détermination d'un segment de signal correspondant à une période fondamentale, avec, dans cet exemple de réalisation, un décalage la recherche de corrélation,
• la figure 5 représente un mode de réalisation d'une analyse spectrale du segment de signal,
• la figure 6 illustre un exemple de réalisation pour recopier, dans les hautes fréquences, une trame valide en remplacement de plusieurs trames perdues,
• la figure 7 illustre la reconstruction du signal des trames perdues, avec la pondération par les fenêtres de synthèse,
• la figure 8 illustre un exemple d'application du procédé au sens de la présente invention, au décodage d'un signal,
• la figure 9 représente schématiquement un dispositif comportant des moyens de mise en oeuvre du procédé au sens de l'invention.

Other advantages and characteristics of the invention will appear on reading the following detailed description of embodiments of the invention and on examining the drawings in which:

• the Figure 1A illustrates an overlap with conventional windows as part of an MLT transform,
• the Figure 1B illustrates a recovery with low delay windows, compared to the representation of the Figure 1A ,
• the figure 2 represents an example of general treatment within the meaning of the invention,
• the figure 3 illustrates the determination of a signal segment corresponding to a fundamental period,
• the figure 4 illustrates the determination of a signal segment corresponding to a fundamental period, with, in this embodiment, an offset correlation search,
• the figure 5 represents an embodiment of a spectral analysis of the signal segment,
• the figure 6 illustrates an exemplary embodiment for copying, at high frequencies, a valid frame replacing several lost frames,
• the figure 7 illustrates the reconstruction of the signal of the lost frames, with the weighting by the windows of synthesis,
• the figure 8 illustrates an example of application of the method within the meaning of the present invention, to the decoding of a signal,
• the figure 9 schematically represents a device comprising means for implementing the method within the meaning of the invention.

A treatment according to the invention is illustrated on the figure 2 . It is implemented with a decoder. The decoder can be of any type, the processing being generally independent of the nature of the coding / decoding. In the example described, the processing applies to a received audio signal. However, it can be applied more generally to any type of signal analyzed by time windowing and transformation, with harmonization to ensure with one or more replacement frames during a recovery-addition synthesis.

During a first step S1 of the treatment of the figure 2 N audio samples are stored successively in a buffer or "buffer" (eg FIFO type). The audio buffer b (n) can thus be constituted for example by 47 ms of signal, for example of 2.35 = 47/20 audio frames of 20 ms each, at a given sampling frequency Fe, for example Fe = 32 kHz. These samples correspond to samples already decoded and therefore accessible at the time of the loss of frame correction processing (s). If the first sample to be synthesized is the time index sample N (of one or more consecutive frames lost), the audio buffer b (n) then corresponds to the N preceding samples of time indices 0 to N-1. In the case of a transform coder, the audio buffer corresponds to the samples already decoded in the past frame (and are therefore non-modifiable). If the addition of an additional delay to the decoder is possible (for example of D samples), the buffer may contain only a part of the samples available at the decoder, leaving, for example, the last D samples for the recovery-addition ( from step S10 of the figure 2 ).

At the filtering step S2, the audio buffer b (n) is then separated into two frequency bands, a low frequency band BB and a high frequency band BH with a separation frequency denoted Fc hereinafter, with example Fc = 4kHz. This filtering is preferably a filtering without delay. The size of the audio buffer defined above then preferably corresponds, with this frequency Fc, now to N '= N Fc / Fe.

• un segment cible du buffer (référence CIB de la figure 3), ce segment étant de taille Ns comprise entre N'-Ns et N'-1 (d'une durée par exemple de 6ms), et
• un segment glissant de taille Ns qui commence à un échantillon occupant une position entre l'échantillon 0 et l'échantillon Nc (avec Nc>N'-Ns ; Nc correspondant pas exemple à une durée de 35 ms),

avec : $$\mathit{Corr}\left(n\right)=\frac{\sum _{k=0}^{k=\mathit{Ns}}b\left(n+k\right)b\left(\mathrm{Nʹ}-\mathit{Ns}+k\right)}{\sqrt{\sum _{k=0}^{k=\mathit{Ns}}b{\left(n+\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">k</figure-callout>}\right)}^2}\sqrt{\sum _{k=0}^{k=\mathit{Ns}}b{\left(\mathrm{Nʹ}-\mathit{Ns}+k\right)}^2}}n\in \left[0,\mathit{Nc}\right]$$

Step S3, applied to the low frequency band, consists in then searching for a loop point and a segment P corresponding to the fundamental period (or "pitch" period) within the buffer b (n) resampled with the frequency Fc. For this purpose, a standard correlation corr (n) between:

• a target segment of the buffer (reference CIB of the figure 3 ), this segment being of size Ns between N'-Ns and N'-1 (of a duration for example of 6 ms), and
• a sliding segment of size Ns which starts at a sample occupying a position between the sample 0 and the sample Nc (with Nc>N'-Ns; Nc corresponding, for example, to a duration of 35 ms),

with: $$\mathit{Corr}\left(\mathrm{not}\right)=\frac{\underset{k=0}{\overset{k=\mathit{ns}}{\Sigma }}b\left(\mathrm{not}+k\right)b\left(\mathrm{NOT}-\mathit{ns}+k\right)}{\sqrt{\underset{k=0}{\overset{k=\mathit{ns}}{\Sigma }}b{\left(\mathrm{not}+\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">k</figure-callout>}\right)}^2}\sqrt{\underset{k=0}{\overset{k=\mathit{ns}}{\Sigma }}b{\left(\mathrm{NOT}-\mathit{ns}+\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">k</figure-callout>}\right)}^2}}\mathrm{not}\in \left[0,\mathit{Nc}\right]$$

With reference to the figure 3 if the maximum correlation is found for the time index sample n = mc, the loop point with a pitch period, index n = pb, corresponds to the sample mc + Ns and the segment noted p (n) following on the figure 3 corresponds to a pitch period of size P = N'-Ns-mc defined between the samples n = pb and n = N'-1.

The sliding segment, of search, is prior to the target segment, as represented on the figure 3 . In particular, the first sample of the target segment corresponds to the last sample of the search segment. If the maximum correlation with the target segment CIB is previously found in the search segment at a point of index mc, then at least one pitch period (with the same sinusoidal intensity for example) flows between the point of temporal index mc and the temporal index sample mc + P. In the same way, at least one pitch period is passed between the sample of index mc + Ns (loopback point, of index pb) and the last sample of buffer N '.

It should be noted that a variant of this embodiment consists of an autocorrelation on the buffer, returning to find an average period P identified in the buffer. In this case, the segment serving for the synthesis comprises the last P samples of the buffer. However, a self-correlation calculation on a large segment can be complex and require more computing resource than a simple correlation of the type described above.

Moreover, another variant of this embodiment consists in not necessarily seeking the maximum correlation over the entire search segment, but simply looking for a segment where the correlation with the target segment is greater than a chosen threshold (for example 70 %). Such an embodiment does not give precisely a single pitch period P (but possibly several successive periods), but nevertheless the complexity related to the processing of a long synthetic segment (of several pitch periods) requires as much or less resource, as the search for maximum correlation across the entire search segment.

In what follows, it is assumed that a single pitch period P is used for the synthesis of the signal, but it should be noted, however, that the principle of the treatment applies equally well for a segment extending over several fundamental periods. The results are even better with several pitch periods, in terms of precision on the FFT transform and richness on the spectral components obtained.

In the case where transients are present in the audio signal contained in the buffer (very short intensity peaks temporally, in the audio signal), it is possible to adapt the correlation search zone, for example by shifting the search correlation (by starting it typically 20 ms after the start of the audio buffer as shown by way of example on the figure 4 , or performing the correlation search in a time zone beginning after the end of a transient).

The next step S4 consists of breaking down the segment p (n) into a sum of sines. A conventional way of breaking down a signal into a sum of sines is to calculate the discrete Fourier transform (or DFT in English) of the signal over a duration corresponding to the length of the signal. This gives the frequency, the phase and the amplitude of each of the sinusoidal components that make up the signal. In a particular embodiment of the invention, for complexity reduction reasons, this analysis is performed by a fast Fourier transform FFT, size 2 ^ k (with k greater than or equal to log ₂ (P)).

• l'opération S41 où les échantillons du segment p(n) sont interpolés de manière à obtenir un segment p'(n) composé de P' échantillons avec P' = 2 ^{ceil(log2(P))} > P, où ceil(x) est l'entier supérieur ou égal à x (on peut par exemple et de manière non restrictive utiliser une interpolation linéaire ou encore de type « spline cubique ») ;
• l'opération S42 avec le calcul de la transformée FFT de p'(n) : Π(k) = FFT(p'(n)); et
• l'opération S43 dans laquelle, à partir de la transformée FFT, on obtient directement les phases ϕ(k)et amplitudes A(k) des composantes sinusoïdales, les fréquences normalisées entre 0 et 1 étant données par : $$f\left(k\right)=\frac{2\mathit{kPt}}{p^2}k\subset \left[0;\frac{\mathit{pʹ}}{2}-1\right]$$
  

In this particular mode, step S4 is decomposed into three operations, with, with reference to figure 5 :

• the operation S41 where the samples of the segment p (n) are interpolated so as to obtain a segment p '(n) composed of P' samples with P '= 2 ^{ceil (log 2 ( P ))} > P , where ceil (x) is the integer greater than or equal to x (it is possible, for example and without limitation, to use linear interpolation or else of "cubic spline"type);
• operation S42 with the calculation of the FFT transform of p (n): Π (k) = FFT (p '(n)); and
• the operation S43 in which, from the FFT transform, the phases φ ( k ) and amplitudes A ( k ) of the sinusoidal components are obtained directly, the normalized frequencies between 0 and 1 being given by: $$f\left(k\right)=\frac{2\mathit{KPT}}{p^2}k\subset \left[0;\frac{\mathit{p\text{'}}}{2}-1\right]$$
  

• sélectionner tout d'abord les amplitudes A(k) pour lesquelles A(k)>A(k-1) et A(k)>A(k+1) avec $$k\in \left[0;\frac{\mathit{pʹ}}{2}-1\right],$$
  
• ensuite, parmi les amplitudes de cette première sélection, on sélectionne les composantes, par exemple par ordre décroissant d'amplitude, de manière à ce que l'amplitude cumulée des pics sélectionnés soit d'au moins x% (par exemple x=70%) de l'amplitude cumulée du demi-spectre.

At step S5 of the figure 2 , the sinusoidal components are selected so as to keep only the most important components. In a particular embodiment, the selection of the components amounts to:

• first select the amplitudes A (k) for which A (k)> A (k-1) and A (k)> A (k + 1) with $$k\in \left[0;\frac{\mathit{p\text{'}}}{2}-1\right],$$
  
• then, among the amplitudes of this first selection, the components are selected, for example in descending order of amplitude, so that the cumulative amplitude of the selected peaks is at least x% (for example x = 70% ) of the cumulative amplitude of the half-spectrum.

It is also possible in addition to limit the number of components (for example to 20) so as to make the synthesis less complex. Alternatively, a search for a predetermined number of the most important peaks can be used.

Of course, the spectral component selection method is not limited to the examples presented above. It is susceptible of variants. It can in particular be based on any criterion making it possible to identify spectral components useful for the synthesis of the signal (for example subjective criteria related to masking, criteria related to the harmonicity of the signal, or others).

The next step S6 is a sinusoidal synthesis. In an exemplary embodiment, it consists in generating a segment s (n) of length at least equal to the size of a lost frame (T). In a particular embodiment, a length equal to 2 frames (for example 40 msec) is generated so as to be able to perform a "cross-fade" sound mix (as a transition) between the synthesized signal (by loss correction). a frame) and the decoded signal to the next valid frame when such a frame is received again correctly.

où k est l'indice des K composantes sélectionnées de l'étape S5. Plusieurs méthodes classiques pour réaliser cette synthèse sinusoïdale sont possibles.To anticipate resampling of the frame (length of samples noted LF), the number of samples to be synthesized can be increased by half the size of the resampling filter (LF). The synthesis signal s (n) is calculated as a sum of the selected sinusoidal components: $$s\left(\mathrm{not}\right)={\displaystyle \underset{k=0}{\overset{k=K}{\Sigma }}}\left[\mathrm{AT}\left(\mathrm{k}\right)\sin (\mathit{\pi f}\left(k\right)\right]\mathrm{not}+\phi \left(k\right))\mathrm{not}\in \left[0;2T+\frac{\mathit{<figure-callout\; id="2"\; label="LF"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">LF</figure-callout>}}{2}\right]$$

where k is the index of K selected components of step S5. Several conventional methods for performing this sinusoidal synthesis are possible.

Step S7 of the figure 2 consists in injecting noise so as to compensate for the energy loss associated with the omission of certain frequency components in the low frequency band. A particular embodiment consists in calculating the residue r (n) = p (n) -s (n) between the segment corresponding to the pitch p (n) and the synthesized signal s (n), with: n ∈ [0; P - 1].

Le signal s(n) est ensuite mixé (additionné avec éventuellement une pondération) au signal r(n).This residue of size P is repeated so that it reaches a size $2T+\frac{\mathit{<figure-callout\; id="2"\; label="LF"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">LF</figure-callout>}}{2}\mathrm{.}$

The signal s (n) is then mixed (added with possibly weighting) to the signal r (n).

Of course, the noise generation method (to obtain a natural background noise) is not limited to the example above and admits variants. For example, it is also possible to calculate the residual in the frequency domain (by removing the selected spectral components of the original spectrum) and to obtain a background noise by inverse transform.

At the same time, step S8 consists of processing the high frequency band simply by repeating the signal. For example, it may be to repeat a frame length T. In a more sophisticated embodiment, the synthesis of the BH band is obtained by taking the last T 'samples before the frame loss (with for example T' = N / 2), and turning them over temporally, then repeating them without turning them over, and so on, as illustrated on the figure 6 . Such an embodiment advantageously makes it possible to avoid audible artifacts by putting the intensities at the beginning and end of the frames at the same level.

In a particular embodiment, the frame of size T 'may be weighted so as to avoid certain artifacts when the contents are particularly energetic in the high frequency band. The weighting (denoted W on the figure 6 ) can for example take the form of a sinusoidal half-window of 1 ms at the beginning and at the end of the frame of size T / 2. The successive frames can also be overlapped.

In a step S9, the signal is synthesized by resampling the low frequency band at its original frequency Fc, and adding it to the signal from the repetition of step S8 in the high frequency band.

avec par exemple, et de manière non limitative, des fonctions de recouvrement définies par : $$W2\left(n\right)=\sin {\left(\frac{\pi \left(n+0.5\right)}{2L}\right)}^2\mathrm{et}W3\left(n\right)=1-W2\left(n\right)n\in \left[0;L-1\right]$$

In step S10, a recovery-addition is carried out which ensures a continuity between the signal before the loss of frame and the synthesized signal. For example, in the case of a low-delay transform coding, for the implementation of this step S10, the L samples located between the beginning of the "aliased" portion (the remaining folded portion) of the MDCT transform are used. and three quarters of the size of the window (with for example a temporal folding axis of windows as usually in the context of an MDCT transform). With reference to the figure 7 these samples are already covered by the synthesis window W1 of the MDCT transform. In order to be able to apply a cover window W2, the samples are divided by the window W1 (which is already known to the decoder), then multiplied by the window W2. The signal S (n) synthesized by the implementation of the steps S1 to S9 described above is expressed as follows: $$S\left(\mathrm{not}\right)=\mathrm{The}\left(\mathrm{not}\right)\frac{\mathrm{<figure-callout\; id="3"\; label="W"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">W</figure-callout>}3\left(\mathrm{not}\right)}{\mathrm{<figure-callout\; id="1"\; label="W"\; filenames="imgf0002.png,imgf0003.png"\; state="\{\{state\}\}">W</figure-callout>}1\left(\mathrm{not}\right)}+S\left(\mathrm{not}\right)\mathrm{<figure-callout\; id="2"\; label="W"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">W</figure-callout>}2\left(\mathrm{not}\right)\mathrm{not}\in \left[0,\mathrm{The}-1\right]$$

with, for example, and without limitation, overlay functions defined by: $$\mathrm{<figure-callout\; id="2"\; label="W"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">W</figure-callout>}2\left(\mathrm{not}\right)=\sin {\left(\frac{\pi \left(\mathrm{not}+0.5\right)}{2\mathrm{The}}\right)}^2\mathrm{and}\mathrm{<figure-callout\; id="3"\; label="W"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">W</figure-callout>}3\left(\mathrm{not}\right)=1-\mathrm{<figure-callout\; id="2"\; label="W"\; filenames="imgf0001.png,imgf0002.png"\; state="\{\{state\}\}">W</figure-callout>}2\left(\mathrm{not}\right)\mathrm{not}\in \left[0;\mathrm{The}-1\right]$$

As previously described, if a delay is allowed at the decoder, this delay time can be used to overlap with the synthesized portion, using any appropriate weighting for the overlay.

Of course, the present invention is not limited to the embodiment described above; it extends to other variants.

For example, separation in high and low frequency bands in step S2 is optional. In an alternative embodiment, the signal from the buffer (step S1) is not separated into two subbands and the steps S3 to S10 remain identical to those described above. Nevertheless, the processing of the spectral components in the low frequencies only advantageously makes it possible to limit their complexity.

The invention can be implemented in a conversational decoder, in the case of a loss of frame. Materially, it can be implemented in a circuit for decoding, typically in a telephony terminal. For this purpose, such a circuit CIR may comprise or be connected to a processor PROC, as illustrated on the figure 9 , and may include a MEM working memory, programmed with computer program instructions according to the invention to perform the above method.

For example, the invention can be implemented in a real-time transform decoder. With reference to the figure 8 , the decoder sends requests to obtain an audio frame in a frame buffer (step S81). If the frame is available (OK output of the test), the decoder decodes the frame (S82) to obtain a signal in the transformed domain, operates an IMDCT inverse transform (S83) which then makes it possible to obtain "aliased" time samples, and proceeds to a final step S84 of windowing (through a synthesis window) and overlapping to obtain temporal samples free of aliasing that will then be sent to a digital-to-analog converter for playback.

When a frame is missing (KO output of the test), the decoder then uses the already decoded signal as well as the "aliased" part of the previous frame (step S85), in the frame loss correction method according to the invention. 'invention.

Claims (16)

1. Method for processing a signal comprising a succession of samples distributed into successive frames, the method being implemented during a decoding of said signal so as to replace at least one lost signal frame on decoding,
   the method comprising the steps:

   a) searching (S3), in a valid signal available on decoding, for a signal segment of duration corresponding to a period determined as a function of said valid signal,

   b) spectral analysis of the segment (S4), for a determination of spectral components of the segment,

   c) synthesis (S6) of at least one replacement frame to replace the lost frame, by construction of a synthesis signal on the basis of part at least of the spectral components.
2. Method according to Claim 1, comprising a search, by correlation in said valid signal, for a repetition period, the duration of the segment comprising at least one repetition.
3. Method according to Claim 2, in which the repetition period corresponds to a duration for which the correlation exceeds a predetermined threshold value.
4. Method according to one of the preceding claims, furthermore comprising a determination of respective phases associated with the spectral components, and in which the construction of the synthesis signal comprises said phases of the spectral components.
5. Method according to one of the preceding claims, furthermore comprising a determination of respective amplitudes associated with the spectral components, and in which the construction of the synthesis signal comprises said amplitudes of the spectral components.
6. Method according to one of the preceding claims, furthermore comprising a determination of respective amplitudes associated with the spectral components, and in which the spectral components of highest amplitudes are selected (S5) for the construction of the synthesis signal.
7. Method according to one of the preceding claims, in which noise (S7) is added to the synthesis signal to compensate a loss of energy relative to spectral components not selected for the construction of the synthesis signal.
8. Method according to Claim 7, in which the noise is obtained by a residual weighted between the signal of the segment and the synthesis signal.
9. Method according to one of the preceding claims, in which the spectral analysis of the segment comprises a sinusoidal analysis by fast Fourier transform of length 2^k, where k is greater than or equal to log₂(P), P being the number of samples in the signal segment.
10. Method according to Claim 9, in which the spectral analysis comprises:

    - an interpolation (S41) of the samples of the segment so as to obtain a second segment comprising a number of samples 2^ceil(log₂(P)), where ceil(x) is the integer greater than or equal to x,

    - a calculation (S42) of the Fourier transform of the second segment; and

    - after determination of the spectral components, identification of frequencies associated with the components, and construction of the synthesis signal by resampling with modification of said frequencies as a function of the resampling.
11. Method according to one of the preceding claims, applied in a context of transform decoding with overlap, in which the synthesis signal is constructed over at least two frame durations.
12. Method according to Claims 10 and 11, in which the synthesis signal is constructed over two frame durations and an additional duration corresponding to a lag introduced by a resampling filter.
13. Method according to one of the preceding claims, furthermore comprising a separation (S2) into a high-frequency band and a low-frequency band, of a signal arising from said valid frame or frames, and in which the spectral components are selected from the low-frequency band.
14. Method according to Claim 13, in which the replacement frame is synthesized by addition:

    - of a first signal constructed on the basis of spectral components selected from the low-frequency band,

    - of a second signal arising from a filtering in the high-frequency band,
    the second signal being obtained by successive duplication (S8) of at least one valid half-frame and its time-reversed version.
15. Computer program comprising instructions which, when executed by a processor, allow the implementation of the method according to one of Claims 1 to 14.
16. Device for decoding a signal comprising a succession of samples distributed into successive frames, the device comprising means (MEM, PROC) for replacing at least one lost signal frame, comprising:

    a) means of searching, in a valid signal available on decoding, for a signal segment of duration corresponding to a period determined as a function of said valid signal,

    b) means of spectral analysis of the segment, for a determination of spectral components of the segment,

    c) means of synthesis of at least one replacement frame to replace the lost frame, by construction of a synthesis signal on the basis of part at least of the spectral components.

Images