Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
  • G10L19/022—Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
  • G10L19/16—Vocoder architecture
  • G10L19/18—Vocoders using multiple modes
  • G10L19/20—Vocoders using multiple modes using sound class specific coding, hybrid encoders or object based coding

Definitions

• the present invention relates to the field of coding / decoding of digital signals in particular for the loss of frame correction.
• the invention is advantageously applied to the coding / decoding of sounds that can contain speech and music mixed or alternately.
• CELP Code Excited Linear Prediction
• transform coding techniques are preferred.
• CELP coders are predictive coders. They aim to model the production of speech from various elements: a short-term linear prediction to model the vocal tract, a long-term prediction to model the vibration of vocal cords in voiced period, and an excitation derived from a fixed dictionary (white noise, algebraic excitation) to represent ⁇ "innovation" which could not be modeled.
• Transformers such as MPEG AAC, AAC-LD, AAC-ELD or ITU-T
• G.722.1 Annex C use critical-sampling transforms to compact the signal in the transformed domain.
• a "critical-sampling transform” is a transform for which the number of coefficients in the transformed domain is equal to the number of time samples in each frame analyzed.
• One solution for efficiently coding a mixed speech / music content signal consists in selecting, over time, the best technique between at least two coding modes, one of the CELP type, the other of the transformed type.
• RMO Model 0 Reference
• LPD Linear Predictive Domain
• a mode TCX for "Transform Coded eXcitation” in English
• wLPT for "weighted Linear Predictive Transform”
• MDCT (unlike the AMR-WB + codec) that uses an FFT transform.
• FD mode for "Frequency Domain” in English
• MDCT for "Modified Discrete Cosine Transform” in English
• MPEG AAC type for "Advanced Audio Coding" out of 1024 samples.
• the MDCT transformation is typically divided into three steps, the signal being cut into frames of M samples before MDCT coding:
• MDCT window of length 2M
• the MDCT window is divided into 4 adjacent portions of equal lengths M / 2, here called "quarters".
• the signal is multiplied by the analysis window and folds are made: the first quarter (windowed) is folded (ie inverted in time and overlapped) on the second quarter and the fourth quarter is folded on the third.
• the temporal folding from one quarter to another is done in the following way: the first sample of the first quarter is added (or subtracted) to the last sample of the second quarter, the second sample of the first quarter is added (or subtracted ) second-last sample of the second quarter, and so on until the last sample of the first quarter that is added (or subtracted) to the first sample of the second quarter.
• the folded two quarters are then coded together after DCT (Type IV) transformation.
• DCT Type IV transformation
• the third and fourth quarters of the previous frame then become the first and second quarters of the current frame.
• the decoded version of these folded signals is thus obtained.
• Two consecutive frames contain the result of 2 different folds of the same quarters, that is to say for each pair of samples one has the result of 2 linear combinations with different but known weights: an equation system is thus solved to obtain the decoded version of the input signal, the time folding can be thus removed using 2 consecutive decoded frames.
• implementation variants of the MDCT transformation exist, in particular on the definition of the DCT transform, on how to fold the block to be transformed temporarily (for example, the signs applied to the folded quarters can be reversed). left and right, or fold the second and third quarter over the first and fourth quarters respectively), etc. These variants do not change the principle of the MDCT analysis-synthesis with the reduction of the sample block by windowing, temporal folding then transformation and finally windowing, folding and addition-recovery.
• the transition frame is defined as the transform-encoded current frame that succeeds a previous frame encoded by predictive coding.
• a part of the transition frame for example a subframe of 5 ms, in the case of a CELP core coding at 12.8 kHz, and two additional CELP frames of 4 ms each, in the case of a CELP core coding at 16 kHz, are coded by a predictive coding restricted with respect to the predictive coding of the previous frame.
• Restricted predictive coding consists in using the stable parameters of the preceding frame coded by a predictive coding, such as the coefficients of the linear prediction filter and coding only a few minimum parameters for the additional subframe in the transition frame.
• the aforementioned patent application WO2012 / 085451 also proposes modifying the first half of the MDCT window so as not to have time folding in the first quarter normally folded. It is also proposed to integrate an addition-recovery part (also called “fade-in” or “overlap-add” in English) between the decoded CELP frame and the decoded MDCT frame by modifying the coefficients of the analysis window. /synthesis.
• the mixed lines lines alternating points and lines
• the bold lines separate the frames of new samples at the input of the encoder.
• the coding of a new MDCT frame can be started when a so-defined frame of new input samples is fully available. It is important to note that these lines in bold at the coder do not correspond to the current frame but to the block of new samples arriving for each frame: the current frame is in fact delayed by 5 ms which correspond to an anticipation, called "lookahead" in English.
• the bold lines separate the decoded frames at the output of the decoder.
• the transition window is zero to the point of folding.
• the portion between the folding point and the end of the CELP transition subframe (TR) corresponds to a (half) sinusoidal window.
• the same window is applied to the signal.
• the coefficients of the window correspond to a window of form sin 2 .
• coded audio signal frames may be lost on the channel between the encoder and the decoder.
• the frame loss correction is often related to the speech model.
• ITU-T Standard G.722.2 proposes to replace a lost packet by extending the long-term prediction gain by attenuating it, and by extending the spectral line frequencies (ISF in English).
• ISF spectral line frequencies
• "Immitance Spectral Frequencies” representing the coefficients A (z) of the LPC filter, making them tend towards their respective averages.
• the fundamental period (or "pitch") is also repeated.
• the contribution of the fixed dictionary is filled with random values. Applying such methods for transform or PCM decoders would require CELP analysis at the decoder, which would introduce significant additional complexity. It should also be noted that more advanced methods of frame loss correction during CELP decoding are described in ITU-T G.718 for both 8 and 12 kbit / s rates as well as interoperable decoding rates with AMR-WB.
• a common technique for correcting a frame loss is to repeat the last frame received.
• Such a technique is implemented in several standardized coders / decoders (G.719, G.722.1 and G.722.1C in particular).
• G.719, G.722.1 and G.722.1C standardized coders / decoders
• an MLT transform for "Modulated Lapped Transform”
• MDCT transform equivalent to an MDCT transform
• Such a technique is inexpensive but its main defect is the inconsistency between the signal just before the loss of frame and the repeated signal. This results in a phase discontinuity that can introduce important audio artifacts if the overlap time between the two frames is low, as is the case when the windows used for the MLT are so-called low delay windows.
• a replacement frame is generated by using a suitable packet loss masking algorithm (for "Packet Loss Concealment").
• a packet may contain several frames, so the term PLC may be ambiguous, and it is here taken to indicate the correction of the current frame lost.
• a PLC-based replacement frame adapted to CELP encoding is used, exploiting the CELP encoder memories.
• a replacement frame based on a PLC adapted to the MDCT encoding is generated.
• the transition frame is composed of a CELP subframe (which is at the same sampling rate as the previous CELP frame) and a MDCT frame with a modified MDCT window canceling the folding "left", there are situations for which existing techniques provide no solution.
• a previous CELP frame has been correctly received and decoded, a current transition frame is lost, and the next frame is an MDCT frame.
• the PLC algorithm after receiving the CELP frame, does not know that the lost frame is a transition frame and therefore generates a replacement CELP frame.
• the first folded portion of the next MDCT frame can not be compensated and the delay between the two types of encoder can not be bridged with the CELP subframe contained in the transition frame (which is lost with the transition frame). There is no known solution to deal with this situation.
• a previous CELP frame at 12.8 kHz is correctly received and decoded, a current CELP frame at 16 kHz is lost, and the next frame is a transition frame.
• the PLC algorithm then generates a CELP frame at the frequency of the last correctly received frame, ie 12.8 kHz, and the CELP transition subframe (partially coded from CELP parameters of the 16 kHz CELP frame lost ) can not be decoded.
• the present invention improves this situation.
• a first aspect of the invention relates to a method for decoding a coded digital signal according to a predictive coding and according to a transform coding, comprising the following steps:
• an additional segment of digital signal is available each time a replacement CELP frame is generated.
• the predictive decoding of the previous frame includes the predictive decoding of a correctly received CELP frame or the generation of a replacement CELP frame by a CELP-adapted PLC algorithm.
• this additional segment makes possible a transition between a CELP coding and a transform coding, even in the case of a frame loss.
• the transition with the next MDCT frame can be provided by the additional segment.
• the additional segment may be added to the next MDCT frame to compensate for the first folded portion of this MDCT frame by cross-fading the area containing the non-canceled time folding.
• the decoding of the transition frame is made possible by the use of the additional segment. Indeed, if it is not possible to decode the transition CELP subframe (unavailability of CELP parameters of the previous frame coded at 16 kHz), it is possible to replace it with the additional segment as described above. after.
• the calculations relating to the management of the frame loss and the transition are distributed over time. Indeed, the additional segment is generated and stored for each generated replacement CELP frame. The transition segment is therefore generated as soon as a frame loss is detected, without waiting for a transition to be subsequently detected. The transition is therefore anticipated at each frame loss, which avoids having to manage a "complexity peak" at the moment when a new correct frame is received and decoded.
• the method further comprises the following steps:
• decoding the next frame comprising an overlap adding sub-step between the additional digital signal segment and the transform coded segment.
• the overlapping addition sub-step makes it possible to cross-fade the output signal.
• Such a dissolve limits the appearance of sound artifacts (for example of the "metallic noise” type) and ensures an energy coherence of the signal.
• next frame is fully coded according to transform coding and the lost current frame is a transition frame between the predecessor coded predictive coding frame and the transform coded next frame.
• the preceding frame is coded according to a predictive coding by a predictive coder core operating at a first frequency.
• the following frame is a transition frame comprising at least one sub-frame coded according to a predictive coding by a predictive coding core operating at a second frequency distinct from the first frequency.
• the following transition frame may comprise a bit indicating the frequency of the predictive coding core used.
• the CELP encoding type (12.8 or 16 kHz) used in the transition CELP subframe can be indicated in the bitstream of the transition frame.
• the invention thus provides to add a systematic indication (one bit) in a transition frame, in order to allow the detection of a CELP coding / decoding frequency difference between the transition CELP subframe and the previous CELP frame.
• the overlap addition is given by applying the following formula implementing a linear weighting:
• r is a coefficient representative of the length of the additional segment generated
• T (i) the amplitude of the additional digital signal segment, for sample i.
• the overlap addition can therefore be made from linear combinations and operations that are simple to implement. The time required for the decoding is thus reduced while requiring less the processor (s) used for these calculations.
• other forms of cross-fade can be implemented without changing the principle of the invention.
• the step of generating by prediction of the replacement frame further comprising an update of internal memories of the decoder, the step of generating by prediction of an additional segment of digital signal may comprise the sub-frames. -following steps :
• the internal memories of the decoder are not updated for the generation of the additional segment. Therefore, the generation of the additional signal segment does not impact the decoding of the next frame, in the event that the next frame is a CELP frame.
• the internal memories of the decoder must correspond to the states of the decoder at the end of the replacement frame.
• the step of generating by prediction of an additional segment of digital signal comprises the following substeps:
• the efficiency of the method is further improved because the temporary calculation data used for the generation of the replacement CELP frame is directly available for the generation of the additional CELP frame.
• the registers and caches, on which the temporary calculation data are stored may not be updated in order to reuse these data directly for the generation of the additional CELP frame.
• a second aspect of the invention is directed to a computer program comprising instructions for implementing the method according to the first aspect of the invention, when these instructions are executed by a processor.
• a third aspect of the invention provides a decoder of a coded digital signal according to a predictive coding and a transform coding, comprising:
• a predictive decoder comprising a processor arranged to perform the following operations: Predictive decoding of a previous frame of the digital signal coded by a set of predictive coding parameters;
• the decoder according to the third aspect of the invention further comprises a transform decoder comprising a processor arranged to perform the following operations:
• decoding the next frame comprising an overlap adding sub-step between the additional digital signal segment and the transform coded segment.
• the invention may include inserting in the transition frame an information bit on the CELP core used for encoding the transition subframe.
• FIG. 1 illustrates an audio decoder according to an embodiment of the invention
• FIG. 2 illustrates a CELP decoder of an audio decoder, such as the decoder audio of Figure 1, according to one embodiment of the invention.
• FIG. 3 is a diagram illustrating the steps of a decoding method, implemented by the audio decoder of FIG. 1, according to one embodiment of the invention
• FIG. 4 illustrates a computing device according to one embodiment of the invention.
• Figure 1 illustrates an audio decoder 100 according to one embodiment of the invention.
• the coded digital audio signal received by the decoder according to the invention may be derived from an encoder able to encode an audio signal in the form of CELP frames, MDCT frames and CELP / MDCT transition frames, such as the encoder described in the application WO2012 / 085451.
• a transition-coded transition frame may furthermore comprise a segment (a sub-frame for example) coded by a predictive coding.
• the encoder may further add a bit in the transition frame to identify the frequency of the CELP core used.
• the CELP coding example is given for illustrative purposes to describe any type of predictive coding.
• the MDCT coding example is given for illustrative purposes to describe any type of transform coding.
• the decoder 100 comprises a reception unit 101 of a coded digital audio signal.
• the digital signal is encoded as CELP frames, MDCT frames and CELP / MDCT transition frames.
• CELP coding may be replaced by another type of predictive coding
• MDCT coding may be replaced by another type of transform coding.
• the decoder 100 further comprises a classification unit 102 capable of determining - generally by simple reading of the bitstream and interpretation of the indications received from the coder - whether a current frame is a CELP frame, an MDCT frame, or a transition frame. Depending on the classification of the current frame, the latter may be transmitted to a CELP decoder 103 or to an MDCT decoder 104 (or both, in the case of a transition frame, the transition CELP subframe being transmitted to a decoding unit 105 described below).
• the classification unit 102 may determine the CELP encoding type used in the additional CELP subframe - this type of encoding is indicated as an output bit rate of the encoder.
• a reception unit 201 which may include a demultiplexing function, is able to receive CELP coding parameters of the current frame. These parameters can include excitation parameters (gain vectors, fixed dictionary vector, adaptive dictionary vector for example) transmitted to a decoding unit 202 capable of generating an excitation.
• the CELP coding parameters may comprise LPC coefficients represented in the form of LSF or ISF, for example. The LPC coefficients are decoded by a decoding unit 203 capable of supplying the LPC coefficients to a synthetic LPC filter 205.
• the CELP decoder 103 may comprise a low-frequency post-processing 207 (or "bass-post filter" in English) similar to that described in the ITU-T G.718 standard.
• the CELP decoder 103 further comprises a resampling 208 of the signal synthesized at the output frequency (the output frequency of the MDCT decoder 104), and an output interface 209.
• post-processing additional CELP synthesis can be implemented before or after resampling.
• the CELP decoder 103 may comprise a high frequency decoding unit 204, the low frequency signal being decoded by the units 202 to 208 described.
• CELP synthesis may involve updating internal CELP coder states (or updating internal memories), such as:
• the decoder further comprises a frame loss management unit 108 and a temporary memory 107.
• the decoder 100 further comprises a decoding unit 105 able to receive the transition CELP subframe and the decoded transition frame by transforming the output of the MDCT decoder 104, in order to decode the frame of the decoder. addition-based transition with recovery of the received signals.
• the decoder 100 may further comprise an output interface 106.
• FIG. 3 is a diagram showing the steps of a method according to one embodiment of the invention.
• a coded digital audio signal current frame may or may not be received by the reception unit 101 from an encoder. It is considered that the previous audio signal frame is a correctly received and decoded frame or a frame of replacement.
• the classification unit 102 determines whether the coded current frame is a CELP frame.
• the method comprises a step 304 of decoding and resampling the coded CELP frame, by the decoder CELP 103.
• the aforementioned internal memories of the CELP decoder 103 can then be set in step 305.
• the decoded and resampled signal is outputted from the decoder 100.
• the excitation parameters of the current frame, as well as the LPC coefficients, can be stored in the memory 107. .
• the current frame comprises at least one segment coded according to a transform coding (MDCT frame or transition frame). It is then checked at a step 307 if the coded current frame is an MDCT frame. If this is the case, the current frame is decoded at a step 308 by the MDCT decoder 104 and the decoded signal is transmitted at the output of the decoder 100 in step 306.
• MDCT frame or transition frame transform coding
• the current frame is not an MDCT frame
• it is a transition frame which is decoded at a step 309 by decoding both the CELP transition subframe and the MDCT transformed current frame and by effecting addition with recovery of the signals from the decoder CELP and the decoder MDCT to obtain a digital signal transmitted at the output of the decoder 100 in step 306.
• a PLC algorithm adapted to the MDCT implemented in the lossy frame management unit 108 generates a replacement frame MDCT decoded by the decoder MDCT 104 in order to obtain a signal digital output, at a step 311.
• a PLC algorithm adapted to the CELP is implemented by the frame loss management unit 108 and the CELP decoder 103 to generate a replacement CELP frame at a step 312.
• the PLC algorithm can include the following steps:
• deemphasizing the synthesized signal using the de-emphasis unit 206 and updating, in step 313, the memory of the deemphasis unit 206;
• post-processing 207 of the synthesized signal by updating, in step 313, the memory of the post-processing - it may be noted that the post-processing can be deactivated during the frame loss correction because the information that they use are not reliable because simply extrapolated, in this case the memories of the postprocessing must nevertheless be updated to allow a normal operation to the next received frame;
• the memories thus updated can be copied into the temporary memory 107.
• the decoded replacement CELP frame is transmitted at the output of the decoder at a step 315.
• Step 316 the method according to the invention provides the generation by prediction of an additional segment of digital signal, by implementing a PLC algorithm adapted to CELP.
• Step 316 may include the following substeps:
• the interpolation estimation can be implemented according to the same method as that used for the interpolation estimation for the replacement frame described above (without updating the memories of the LSF quantizers);
• the excitation can be determined by the same method as that used to determine the excitation for the replacement frame (without the update of the adaptive gain and fixed gain values);
• postprocessing of the signal synthesized by using the post-processing memory 207 optionally, postprocessing of the signal synthesized by using the post-processing memory 207;
• the invention provides for storing in temporary variables the states of the CELP decoding that are modified at each step, before performing these steps, so that the predetermined states can be restored. their stored values after generating the temporary segment.
• the additional signal segment generated is stored in the memory 107 at a step 317.
• a next digital signal frame is received by the reception unit 101. It is verified in a step 319 that the next frame is an MDCT frame or a transition frame.
• next frame is a CELP frame and is decoded by the CELP decoder 103 at a step 320.
• the additional segment synthesized in step 316 is not used and can be deleted. the memory 107.
• next frame is an MDCT frame or a transition frame
• it is decoded by the MDCT decoder 104 in a step 322.
• the additional digital signal segment stored in the memory 107 is retrieved at a step 323 by the management unit 108 and transmitted to the decoding unit 105.
• the additional signal segment obtained makes it possible to carry out an overlay by the unit 103 in order to correctly decode the first part of the next MDCT frame at a step 324.
• the additional segment is a sub-frame half
• a linear gain between 0 and 1 can be applied when overlapping over the first half of the MDCT frame and a gain linear between 1 and 0 is applied to the additional signal segment.
• MDCT decoding can give rise to discontinuities due to quantization errors.
• transition frame In the case where the following frame is a transition frame, two cases are to be distinguished as considered below. It is recalled that the decoding of the transition frame is based not only on the classification of the current frame as a "transition frame” but also an indication of the CELP coding type (12.8 or 16 kHz) when several coding frequencies CELP are possible. So :
• the transition subframe can not be decoded, and the additional signal segment then allows the decoding unit 105 to provide overlap with the signal from the MDCT decoding of step 322.
• the additional segment is a sub-frame half, a linear gain between 0 and 1 may be applied during overlap over the first half of the MDCT frame and a linear gain between 1 and 0 is applied to the additional signal.
• the transition CELP subframe can be decoded and used by the decoding unit 105 for addition-overlap with the digital signal from the decoder MDCT 104 having decoded the transition frame.
• r a coefficient representative of the length of the additional segment generated, the length being equal to L / r.
• r a coefficient representative of the length of the additional segment generated, the length being equal to L / r.
• L the length of the next frame (for example 20 ms);
• the invention provides, on loss of a current frame following a previous CELP frame, the generation of an additional segment in addition to a replacement frame. In some cases, and especially if the next frame is a CELP frame, such an additional segment is not used. However, its calculation does not induce any additional complexity insofar as the coding parameters of the previous frame are reused.
• next frame is an MDCT frame or a transition frame with a CELP subframe at a core frequency different from the core frequency used for encoding the previous CELP frame
• the additional signal segment generated and stored allows the decoding of the next frame, which was not allowed by the solutions of the prior art.
• FIG. 4 represents an exemplary computing device 400 that can be integrated in the CELP encoder 103 and in the MDCT encoder 104.
• the device 400 comprises a RAM 404 and a processor 403 for storing instructions for implementing steps of the method described above (implemented by the CELP encoder 103 or by the MDCT encoder 104).
• the device also comprises a mass memory 405 for storing data intended to be stored after the application of the method.
• the device 400 further comprises an input interface 401 and an output interface 406 respectively for receiving the frames of the digital signal and transmitting the decoded signal frames.
• the device 400 may further include a digital signal processor (DSP) 402.
• DSP digital signal processor
• This DSP 402 receives the digital signal frames for shaping, demodulating and amplifying, in a manner known per se, these frames.
• a set-top box can be embedded in any type of larger device such as a mobile phone, a computer, etc.

Applications Claiming Priority (2)

Publications (2)

Family

ID=51894139

Family Applications (1)

Country Status (8)

Families Citing this family (6)

Family Cites Families (69)

• 2014
  • 2014-07-29 FR FR1457356A patent/FR3024582A1/fr active Pending
• 2015
  • 2015-07-27 EP EP15757533.3A patent/EP3175444B1/de active Active
  • 2015-07-27 ES ES15757533.3T patent/ES2676834T3/es active Active
  • 2015-07-27 JP JP2017504685A patent/JP6687599B2/ja active Active
  • 2015-07-27 CN CN201580041610.9A patent/CN106575505B/zh active Active
  • 2015-07-27 WO PCT/FR2015/052075 patent/WO2016016567A1/fr active Application Filing
  • 2015-07-27 CN CN202110612907.3A patent/CN113571070B/zh active Active
  • 2015-07-27 KR KR1020177005826A patent/KR102386644B1/ko active IP Right Grant
  • 2015-07-27 US US15/329,428 patent/US10600424B2/en active Active
• 2020
  • 2020-02-05 US US16/782,539 patent/US11475901B2/en active Active
  • 2020-02-13 JP JP2020022302A patent/JP7026711B2/ja active Active

Also Published As

Similar Documents

Legal Events