Classifications

• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/005—Correction of errors induced by the transmission channel, if related to the coding algorithm

Definitions

• the present invention relates to the coding or decoding of audio signals and in particular to the concealment of errors (“error concealment”) in digitally coded audio signals.
• muting error concealment
• Another known method that avoids the sudden drop and rise of signal energy is the data retry method. If, for example, a block or several blocks of audio data fails, some of the last data sent is repeated in a loop until error-free, i.e. H. sound data is intact. However, this method leads to annoying artifacts. If only short parts of the audio signal are repeated, the repeated signal sounds machine-like with a basic frequency at the repetition frequency, regardless of the original signal. If longer parts are repeated, certain echo effects arise, which are also perceived as annoying.
• spectral values in a block are defective, these spectral values can be predicted, ie predicted or estimated, based on the spectral values of a preceding block or a plurality of previous blocks.
• the predicted spectral values correspond to the incorrect spectral values within certain limits if the audio signal is relatively stationary, ie if the audio signal is not subjected to such rapid changes in the signal envelope.
• a normal block of coded audio data has 1024 spectral values.
• 1024 predictors working in parallel are therefore required in the decoder in order to, for example, B. in the event of a complete block failure ("frame loss") to be able to predict all spectral values.
• a disadvantage of this method is the relatively high computing effort, which currently makes real-time decoding of a received multimedia or audio data signal impossible.
• MDCT modified discrete cosine transformation
• DE 40 34 017 AI relates to a method for detecting errors in the transmission of frequency-coded digital signals.
• an error function is formed from frequency coefficients of past and possibly future blocks, on the basis of which the occurrence of an error is determined. An incorrect frequency coefficient is no longer used for the evaluation of subsequent blocks.
• DE 197 35 675 AI discloses a method for concealing errors in an audio data stream. For this, the spectral energy of a subset of intact audio data is calculated. After forming a template for replacement data based on the spectral energy calculated for the subgroup of the intact audio data, replacement data for faulty or non-existent audio data corresponding to the subgroup are generated on the basis of the template.
• the object of the present invention is to provide a precise and flexible error concealment for audio signals, which can be implemented with limited computing effort.
• This object is achieved by a method for concealing an error according to claim 1 and a device for Disguising an error according to claim 12 solved.
• Another object of the present invention is to provide error-proof and flexible decoding of audio signals.
• This object is achieved by a method for decoding a coded audio signal according to claim 10 and by a device for decoding a coded audio signal according to claim 13.
• the present invention is based on the knowledge that the disadvantages of the spectral-value-based prediction, which consist in the dependency on the transformation algorithm used and in the dependency on window shape and block length, can be avoided by using a prediction for error concealment which, in the "quasi "Time range works.
• a set of spectral values which preferably corresponds to a long block or a number of short blocks, is divided into subbands.
• a subband of the current set of spectral coefficients can then be back-transformed to obtain a time signal that corresponds to the spectral coefficients of the subband.
• a prediction is carried out on the basis of the time signal of this subband.
• this prediction takes place in the quasi-time domain, since the temporal signal on the basis of which the prediction is carried out is only the time signal of a subband of the coded audio signal and not the time signal of the entire spectrum of the audio signal.
• the time signal generated by prediction is subjected to a forward transformation in order to obtain estimated, ie predicted spectral coefficients for the subband of the following set of spectral coefficients. Now it is found that in the following set of spectral coefficients If one or more incorrect spectral coefficients are present, the incorrect spectral coefficients can be replaced by the estimated, ie predicted, spectral coefficients.
• the method according to the invention for concealing errors requires less computation effort, since, because of the grouping of spectral coefficients, predictions only have to be carried out for each subband and no longer for each spectral coefficient.
• the method according to the invention provides a high degree of flexibility, since the properties of the signals to be processed can be taken into account.
• noise substitution according to the present invention works particularly well for tonal signals.
• tonal signal components tend to occur in the lower frequency range of the spectrum of an audio signal, while the higher frequency signal components tend to be non-stationary, i. H. are intoxicated.
• "Noisy signal components" in the sense of the present description are signal components that are not very stationary. However, these noisy signal components do not necessarily have to represent noise in the classic sense, but only rapidly changing useful signals.
• the present invention therefore makes it possible to subject only lower-frequency signal components to a prediction, while higher-frequency signal components are not processed at all.
• This feature of the present invention compares to a complete transformation of the whole Audio signal in the time domain and a prediction of the entire temporal audio signal from block to block using a so-called "long-term" predictor represent a significant advantage, since according to the invention the advantages of prediction in the time domain are combined with the advantages of spectral decomposition. Only the spectral decomposition enables properties of the audio signal, which are dependent on the frequency, to be taken into account. The number of subbands that is generated when dividing the set of spectral coefficients can be selected as desired. If only two subbands are selected, there is already the advantage of considering the tonality in the lower frequency range of the audio signal.
• the predictor in the quasi-time range will have a relatively short length such that its delay does not become too great. Since the individual subbands are preferably processed in parallel, many parallel predictor circuits would be necessary in an embodiment of the present invention using a hard-wired integrated circuit.
• the present invention is used in connection with a transformation encoder which uses different block lengths, there is the advantage that the predictor itself is independent of block length ("frame length”) and window shape ("window shape").
• the back-transformation eliminates the dependency on the transformation algorithm itself, which was carried out above with regard to the MDCT.
• the concept according to the invention for error concealment provides estimated spectral coefficients which are in phase due to the backward transformation, the prediction in the time domain and the forward transformation, ie there are no phase jumps in the time signal due to a predicted spectral coefficient compared to a time signal of a preceding intact set of spectral coefficients. Tonal signals can thus be substituted so well for incorrect or missing signal components that a common listener in the vast majority of cases will not even notice that an error has occurred.
• the method according to the invention is particularly suitable for a combination with an error concealment technique, which is described in DE 197 35 675 AI, which is suitable for the substitution of noisy signal components. If tonal signal components of a missing block are obscured by the method according to the invention, and if noisy signal components are combined by the aforementioned known method, which is based on an energy similarity between substituted data and intact data, completely failed blocks can be obscured almost inaudibly by a normal listener .
• FIG. 1 shows a decoder which has an error concealment device according to the invention
• FIG. 2 is a more detailed block diagram of the error concealment device of FIG. 1;
• FIG. 3 shows a more detailed block diagram of the error concealment device from FIG. 1, which also has noise substitution and operates based on the prediction gain;
• FIG. 5 is a detailed block diagram of a preferred embodiment of the error concealer for an MPEG-2 AAC decoder
• Fig. 6 is a detailed block diagram of the predictor of Fig. 5;
• Fig. 7 is a schematic representation of the block structure according to the AAC standard.
• Fig. 1 shows a block diagram of a decoder according to a preferred embodiment of the present invention.
• the decoder block diagram shown in FIG. 1 basically corresponds to the MPEG-2 AAC decoder as defined in the MPEG-2 AAC 13818-7 standard.
• the encoded audio signal first goes into a bitstream demultiplexer 100 to separate spectral data and side information.
• the Huffman-coded spectral coefficients are then fed into a Huffman decoder 200 in order to obtain quantized spectral values from the Huffman code words.
• the quantized spectral values are then fed into an inverse quantizer 300 and then multiplied by scale factor band by corresponding scale factors.
• the encoder according to the invention can have several additional functionalities following the inverse quantizer 300, such as, for. B. a middle / side level, a predictor level, a TNS level etc. as defined in the standard.
• the decoder immediately before a synthesis filter bank 400 comprises an error concealer 500 which operates according to the invention and ensures that the effects of transmission errors in the encoded audio signal which is fed into the bitstream demultiplexer 100 are alleviated or can be made completely inaudible.
• the error concealer 500 causes transmission errors to be concealed, i. H. that they are not or only slightly audible in a temporal audio signal at the output of the synthesis filter bank.
• FIG. 2 shows a general block diagram of the error concealer 500.
• the latter includes a backward transformation device 502, a device 504 for generating estimated values and a device 506 for forward transformation. Both the reverse transformation device 502 and the forward transformation device 506 can be controlled via a block type line 508, depending on the block type that is currently present.
• the error encryption device 500 further comprises a parallel branch in order to direct the input spectral coefficients directly bypassing the backward transformation device 502, the device for generating estimated values 504 and the forward transformation device 506 from the input to the output.
• This parallel branch comprises a time delay stage 510 in order to ensure that the estimated spectral coefficients for a subsequent block behind the forward transformation device 506 are present at an error selection device 512 at the same time as "real", possibly incorrect spectral coefficients for the following block, in order to possibly incorrect spectral coefficients to be able to replace the real spectral coefficients for the following block with estimated spectral coefficients for the following block.
• This spectral value-based replacement is represented by a switch symbol 512 in FIG. 2.
• the error replacement device 512 can operate either spectrally or block-wise or block-wise. Depending on the requirement, it can also work sub-band.
• the following set of spectral coefficients is then available at the output of the error replacement device 512, in which spectral coefficients which were originally faulty have been replaced by estimated spectral coefficients, ie in which errors are masked.
• the block diagram shown in FIG. 2 represents only part of the error concealment device 500. However, this representation was chosen for reasons of clarity. As explained in more detail in FIG. 5 using a preferred exemplary embodiment of the present invention will be, the circuit shown in Fig. 2 is preceded by a device for subdivision into subbands. Analogously, the error replacement device 512 is followed by a device for undoing the division into subbands in such a way that the filter bank 400 (FIG. 1) obtains a "normal" set of spectral coefficients without noticing anything of the previous error concealment. Error concealer 500 (FIG. 1) thus comprises a plurality of circuits described with reference to FIG. 2, one circuit for each subband. The parallel circuits are connected on the input side by the device for dividing and on the output side by the device for undoing the division, as will be explained in detail later.
• FIG. 7 comprises a time axis 700, with respect to which the extension of a long block 702 is shown
• a long block contains 2048 samples, resulting in 1024 spectral coefficients when 50% window overlap is used, as is known. Background information on the modified discrete cosine transform (MDCT) used and the window overlap can be found in the standard already cited. 7 also shows eight short blocks 704, each of which has 256 samples, in order to again give 128 spectral coefficients due to the 50% overlap. For reasons of clarity, the overlap of the short blocks and the overlap of the long block with a preceding long block or with a preceding or a subsequent start or stop window were not shown in FIG. 7. In any event, it can be seen from Fig. 7 that the number of spectral coefficients of a long block is eight times the number of spectral coefficients of a short block. In other words, a long block comprises the same duration of the audio signal as eight short blocks.
• MDCT modified discrete cosine transform
• the backward transformation device 502 is controlled via the block type line 508 in such a way that it carries out eight successively backward transformations of the spectral coefficients in corresponding subbands of short blocks and simply strings the quasi-time signals obtained in series around the device 504 for generating estimated values with a time signal of a certain length.
• the forward transformation device 506 will again carry out eight successive forward transformations, one after the other with the values which are serially output by the device 504 for generating estimated values.
• this "duty cycle" requires that the same number of spectral coefficients be output in the case of short blocks as in the case of long blocks.
• the spectral coefficients that are output by the error concealment device 500 in a “work cycle” are referred to in the sense of the present invention as Denoted set of estimated spectral coefficients.
• the number of spectral coefficients in a set corresponds to the number of spectral coefficients in a long block and the number of spectral coefficients from eight short blocks.
• any other ratio between long and short block can be used, for example 2, 4 or 16.
• the situation will be such that the number of spectral coefficients of a long block is divisible by the number of spectral coefficients of a short block.
• the number of a set of spectral coefficients would correspond to the smallest common multiple of long and short blocks, such that independence of the block type is achieved at the predictor level, ie in the device 504 for generating estimated values becomes.
• the error concealment device is expanded by a noise substitution device 514 which, depending on a prediction gain signal 516, can be connected to the error replacement device via a noise substitution switch 518 instead of the forward transformation device 506.
• the noise reduction device 514 works according to the method described in DE 197 35 675 AI in order to approximate noisy signal components in the audio signal. Since the spectral components are noisy, the phase of the spectral coefficients is no longer taken into account, but only the energy of several spectral coefficients in a subgroup.
• the noise substitution device 514 Based on the energy in a subset of the last available intact audio data, the noise substitution device 514 generates a corresponding subset of spectral coefficients, the energy in the subset of the generated spectral coefficients being the energy of the corresponding subset of the preceding spectral coefficients corresponds to or is derived from the same. However, the phases of the spectral coefficients generated during noise substitution are randomly determined.
• the noise substitution switch 518 is controlled by a prediction gain signal 516.
• the prediction gain relates to the ratio of the output signal of the device 504 for generating estimated values to the input signal. If it is found that the output signal differs relatively little from the input signal in a subband, it can be assumed that the audio signal in this subband is relatively stationary, i. H. tonal, is. If, on the other hand, the output signal of the predictor differs greatly from the input signal, it can be assumed that the signal is unsteady; H. atonal or intoxicating. In this case, noise replacement will give better results than prediction because noisy signals per se cannot be reliably predicted.
• the noise reduction switch 518 could be controlled to connect the forward transformer 506 to the error replacement 512 if the prediction gain exceeds a certain threshold, or to connect the noise replacement 514 to the error replacement 512 when the prediction gain falls below this threshold in order to optimally combine both substitution methods.
• FIG. 4 assumes that the current set of spectral coefficients only has intact spectral coefficients or has already been subjected to an error concealment method according to FIG. 2 or 3.
• the current set of spectral coefficients is processed by the filter bank 400 (FIG. 1) and output to a loudspeaker, for example (12).
• the current set of spectral coefficients is used to predict, ie to estimate or predict, a subsequent set of spectral coefficients.
• the current set of spectral coefficients is subdivided into subbands (14).
• the division into subbands takes place in such a way that only one subband with a corresponding frequency range is generated per block.
• the current set of spectral coefficients will comprise a plurality of complete spectra in time.
• Corresponding subbands are then generated in step 14 for each complete spectrum, ie a plurality of subbands per set of spectral coefficients.
• a reverse transformation is carried out for each subband (16).
• a reverse transformation is carried out for each subband (16).
• a single inverse transformation per subband is carried out before proceeding to prediction 18.
• several back-transformations are carried out in accordance with the subbands of each “short” spectrum before a prediction 18 is then carried out for all subbands together.
• the prediction 18 takes place in the quasi-time domain, ie for each subband "time" signal, in order to obtain an estimated subband time signal for the following sentence.
• This estimated quasi-time signal is then again subjected to a forward transformation 20, the forward transformation being carried out once only for a long block or N times for short blocks, where N is the ratio between the number of spectral coefficients of a long block to the number of Spectral coefficient of a short block.
• step 20 estimated spectral coefficients are available for each subband.
• the subdivision introduced in step 14 is canceled, such that after step 22 there is a following set of spectral coefficients.
• the decoder receives the following set of spectral coefficients.
• This set is subjected to an error detection 26 to determine whether one spectral coefficient, several spectral coefficients or even all spectral coefficients of the following set are incorrect.
• the flow chart of FIG. 4 effectively represents a snapshot of processing from one set of spectral coefficients to a next set of spectral coefficients.
• a single filter bank 400 FIG. 1
• steps 12 and 30 the flow chart of FIG. 4
• only a single device for receiving the current set of spectral coefficients or for receiving the following set of spectral coefficients will be required in order to implement steps 10 and 24.
• the temporal synchronicity for steps 10 and 24 is determined in a device which implements the method according to the invention is ensured by the time delay stage 510 in the parallel branch (FIG. 2).
• FIG. 5 shows a more detailed illustration of the general block diagram of FIG. 2 using the example of an MPEG-2 AAC transformation encoder which has the error concealment device 500 according to the invention.
• the error concealment device 500 (FIG. 1) comprises a device 520 for dividing the blocks of spectral coefficients into preferably 32 subbands. In the case of long blocks, each subband has 32 spectral coefficients. Since the subbands of the short blocks cover the same frequency ranges, each subband has 4 spectral coefficients in the case of short blocks. A division of an entire spectrum into subbands of the same size is preferred for reasons of simplicity, but a subdivision into unequal subbands would also be possible, for example based on the psychoacoustic frequency groups.
• Each subband is then subjected to an inverse modified discrete cosine transformation.
• the IMDCT runs once and receives 32 input values.
• eight consecutive IMDCTs are carried out, each with 4 of the spectral coefficients, such that 32 quasi-time samples again result at the output. These are then fed to the predictor 504, which in turn generates 32 estimated quasi-time samples that are transformed using the MDCT 506.
• the predictor 504 which in turn generates 32 estimated quasi-time samples that are transformed using the MDCT 506.
• a single MDCT with 32 temporal values is carried out, while in the case of short blocks eight temporally successive MDCTs with 4 samples each are carried out. Although only one branch for the zeroth subband is shown in FIG.
• the subbands are all of the same length, there is an identical branch for each subband. If the subbands have different lengths, the orders of the IMDCT and MDCT are adapted to them. For one Practical implementation offers parallel processing. However, serial processing of the subbands in succession is of course also possible if appropriate storage capacities are provided.
• the output values of the MDCT 506 for each subband are fed into a device 522 for undoing the division, ie into an inverse division device, in order to output an estimated set of spectral values in the preferred embodiment at the AAC-MDCT level.
• a time delay stage 504b is connected upstream of the LMSL predictor 504a.
• the predictor 504 further comprises a parallel-serial converter 504c on the input side and a serial-parallel converter 504d on the output side.
• Predictor 504 in its implementation as an LMSL predictor, also includes two switches 504f and 504g, which have two switch positions. Switch position "1" relates to the case that spectral coefficients of the following block are error-free, while switch position "2" relates to the case that spectral coefficients of the following block are incorrect.
• FIG. 6 shows the case in which the spectral coefficients are faulty.
• a reference signal with a value of 0 is fed into the predictor at switch 504g instead of the input signal.
• the output values of the parallel-serial converter are fed into the LMSL predictor from below.
• error concealment method is used in connection with an AAC encoder, it is preferred to use the corresponding transformation algorithms (MDCT or IMDCT) for all forward and backward transformations. For error concealment, however, it is not necessary that the same transformation method that was used when coding the audio signal is used for the backward or forward transformation in order to form the spectral coefficients.
• MDCT transformation algorithms
• IMDCT transformation algorithms
• frequency-time domain transformations with a lower order than the frequency resolution are used for each subband.
• Special predictive values for tonal signal components in the intermediate level are thus generated by means of the predictor.
• Time-frequency domain transformations of a lower order than the original frequency resolution are used as the forward transformation / synthesis, the same order being chosen as for the frequency-time domain transformation used.
• the error concealment according to the invention thus provides flexibility, on the one hand, by utilizing prior knowledge of spectral properties of audio signals and, on the other hand, independence from the transformation method used in the encoder by generating the estimated values in the quasi-time signal, ie not at the spectral coefficient level.

Landscapes

• Engineering & Computer Science (AREA)
• Computational Linguistics (AREA)
• Signal Processing (AREA)
• Health & Medical Sciences (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Human Computer Interaction (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Multimedia (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)
• Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)

Applications Claiming Priority (3)

Publications (2)

Family

ID=7907325

Family Applications (1)

Country Status (6)

Cited By (2)

Families Citing this family (39)

Family Cites Families (9)

• 1999
  • 1999-05-07 DE DE19921122A patent/DE19921122C1/de not_active Expired - Lifetime
• 2000
  • 2000-04-12 AT AT00926896T patent/ATE221244T1/de active
  • 2000-04-12 WO PCT/EP2000/003294 patent/WO2000068934A1/fr active IP Right Grant
  • 2000-04-12 JP JP2000617440A patent/JP3623449B2/ja not_active Expired - Lifetime
  • 2000-04-12 DE DE50000306T patent/DE50000306D1/de not_active Expired - Lifetime
  • 2000-04-12 US US09/980,534 patent/US7003448B1/en not_active Expired - Lifetime
  • 2000-04-12 EP EP00926896A patent/EP1145227B1/fr not_active Expired - Lifetime

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events