DE19735675C2 - Method for concealing errors in an audio data stream - Google Patents

Description

The present invention relates to audio coding servers drive and especially on error concealment techniques when encoding audio signals.

The so-called "Internet radio broadcasting", i. H. the Broadcast a radio program live over the Internet, he calls for thousands of real-time connections between one Transmitter and receiver programs, for example, in a personal computer of an Internet subscriber, build.

The currently most widely used Internet protocol http ensures error-free transmission, the same however, has no real-time properties. This protocol requires a proportion to the number of stations received network load at the transmitter. So the number of is the same early listener depending on the data rate of the coded signal typi limited to a few hundred, which is what "Rundfunkan ", i.e. applications where radio or long distance visual signals are to be transmitted, certainly not sufficient.

New protocols for transferring data over the Internet, such as B. rtp (rtp = realtime transmission protocol = real time transfer protocol), bypass this bottleneck, in which, in contrast to the http error in the protocol Allow transfer.

This poses the problem for the recipient like him to deal with faulty transmission data. As is for example the lack of parts of a multimedia or Bridged audio data stream?  

In the prior art, some ver drive already known. When muting, which is the easiest way to conceal errors, playback is switched off as long as data is missing or are faulty. The missing data is indicated by a zero signal replaced. Due to psychoacoustic effects This sudden drop and rise in signal energy than felt very uncomfortable. A mute procedure is at for example in Detlef Wiese: "Error Concealment Strategies for Digital Audio Broadcasting ", 92nd AES Convention, Vienna 1992, Preprint 3264, and in Detlef Wiese: "Application of Error concealment techniques for digital radio broadcasting (DAB) ", 9th ITG conference" Hörrundfunk ", Mannheim 2/92, described.

Another known method is the Da method tenwiederholung. If a data stream fails, part of the last sent data repeated in a loop. at block-oriented procedures, which the majority of known Audio coding methods are, for example, would be the or repeats the blocks last received. Such a The procedure is from the references already cited known.

However, this procedure leads to annoying artifacts: who which only repeats short parts of the audio signal, that is what it sounds like the repeated signal - regardless of the original signal - ma shine-like with a basic frequency at the repetition frequency frequency. If longer parts are repeated, "Echo occurs effects ", which are also perceived as disturbing.

Another method, the method of data interpolation tion, builds on the likelihood of being transmitted a data stream with a relatively high delay already like the valid audio data is available if there is a defect in the Audio data signal must be obscured. It can be an in terpolation can be used to gene data in the gap like Laurent Filliat, Mario Rossi, Joseph Maisano,  "Error Correction by Interpolation in Digital Audio", 92nd AES Convention, Vienna 1992, preprint 3281 is.

A disadvantage of this method is, for example, that in many cases the delay is unacceptable, especially when thinking about telephony applications.

With block-oriented transform encoders / decoders can the signal energy per spectral line from one block the next are predicted or "predicted". This Known procedures (Jürgen Herre: "Error concealment with spectrally coded audio signals ", dissertation, university Erlangen-Nürnberg, Erlangen 1995) leads to a good one Error concealment, however it is a relatively high re This requires real-time decoding a received multimedia or audio data signal impossible Lich makes.

It is also known in the art to provide redundancy in the Increase data flow. It is possible in an audio data current already at the encoder or transmitter a second data to insert current, the same signal (with possibly low lower data rate and correspondingly lower quality) wearing. If you offset this second data stream in time compared to the former, it is likely that to everyone Time valid Da from at least one of the two streams be received. Thus, the failure of a signal be bridged by the other signal. This method provides a true to information in the event of a faulty main signal Playback, however, is usually of poorer quality will have acted.

Like the previous procedure, this procedure increases the delay between sender and receiver, which is why the same disadvantages as with data interpolation are. Furthermore, the data rate is increased, since there is a second one a certain identical signal is transmitted, which in particular  with low data rates or available transfers bandwidths may not be acceptable.

When transmitting a stereo signal, there is a wide one res procedure to disguise transmission errors. In the process of left / right replacement, the ge disturbed or failed channel through the other Ka nal be replaced, as in Detlef Wiese: "Error Conceal Strategies for Digital Audio Broadcasting ", 92nd AES Convention, Vienna 1992, Preprint 3264, Detlef Wiese: "Anwen Development of error concealment techniques for digital hearing Rundfunk (DAB) ", 9th ITG conference" Hörrundfunk ", Mannheim 2/92, and Jürgen Herre: "Error concealment at spectral coded audio signals ", dissertation, University of Erlan gen-Nürnberg, Erlangen 1995. In general however, the data for both channels are over simultaneously wear and therefore disturbed at the same time. Otherwise han it is a special case for the procedure of the Er Increased redundancy in the data stream. This procedure can furthermore only be used if the left and the right channels can be decoded independently of each other. It is however, for example with the so-called "joint stereo" - Modes defined by the standard MPEG Layer-3 not possible.

The object of the present invention is a Method for concealing errors in an audio data to create electricity that is effective and if possible not Audible error concealment reached.

This object is achieved by a method according to claim 1 solves.

The method for concealing errors in an audio data stream according to the present invention comprises the following steps:
Detecting the occurrence of an error in an audio data stream, wherein audio data is intact audio data before the error occurs;
Computing a spectral energy in a subset of the intact audio data;
Forming a template for replacement data based on the spectral energy calculated for the subset of the intact audio data; and
Generate the replacement data for faulty or nonexistent audio data corresponding to the sub-group based on the template.

A subgroup in the sense of this application comprises several successive frequency lines. The frequency area is thus divided into successive subgroups divides, each subgroup being consecutive frequency has lines. The subgroups can preferably be those from the Psychoacoustics correspond to known frequency groups. It can but also a division that differs from the frequency groups be selected for the subgroups as it continues is described.

As it will be apparent from the later detailed description becomes apparent, this method solves in the prior art existing disadvantages as there is neither an increased delay yet an increase in the data rate, i. H. a change in Source coding still requires the uncomfortable time and Frequency structure of a simply repeated signal section having. In contrast, for example, to the process of spectral prediction that has already been outlined the method according to the invention a hearing-adapted noise substitution. A frequency representation is assumed of a temporal audio signal, which affects many for professionals te known ways can be obtained. Like it by Zwicker and Fastl in "Psychoacoustics, Facts and Mo dels ", 1990, the spectrum of a signal  be grouped into so-called frequency groups. This leads to a scaling of the frequency axis in Bark, which Hearing-relevant properties are taken into account, namely in Contrary to a usual scaling of the frequency axis in Hertz. A bark is the frequency difference between neighbors bartended frequency groups. A frequency group is like it from Zwicker has defined the maximum bandwidth within which is the intensity of individual tones by human Ear are summed to form hearing thresholds.

The present invention is based on the finding that that it is sufficient for an error concealment, a "good" generate spectrally shaped random signal. "Good" means in this context that the spectral energies in Fre Sequence groups of the random signal with the spectral energies of the intact audio data before the error occurred agree or are derived from the same.

Before going into various embodiments of the present Invention is entered, other terms used for the understanding of the invention are essential, briefly tert.

If in this application of a "spectral energy" egg ner frequency line, the Be supporting square of this, for example, by a Fourier trans formation calculated spectral component understood. If on the other hand from the "spectral energy" of a subgroup is spoken, the sum of the squared is underneath Spectral component amounts in this subgroup ver stood, which for example by a Fourier Trans formation have been calculated. However, it is important to start notice that the term "spectral energy" is not entirely is used strictly. For example, if the process of present invention carried out in an encoder which works according to MPEG Layer-3, so the procedure runs on the Basis of MDCT lines (MDCT = modified discrete cosine Transformation). The modified discrete used there  Cosine transformation, which is an overlap and fen includes, but is not strictly strict gieerhaltend. That means that there is no easy way gives the energy per frequency line in the MDCT spectra For example, to find a Fourier transform. Conversely, knowing this energy is not without white teres possible to generate an MDCT spectrum that the ge required energy contains. The expression "spectral energy" therefore relates both to the energy-conserving Fourier Transformation as well as on the "non" energy-conserving modified discrete cosine transformation or similar che non-energy-conserving transformation processes, which are known. The usefulness of the procedure according to the above However, lying invention does not depend on one exact energy conservation, since the procedure for wear errors in an audio data stream are not just one exact restoration of the original data, which in all is not possible in general, but just an estimate of substitute data that causes an occurred error is as inaudible as possible and thus obscured.

Furthermore, audio data is often used in this application Occurrence of the error or intact audio data chen. It should be noted that these terms do not refer to the actual reception process, for example one from In ternet reading programs. With some internet Protocols can send data packets sequentially arrive in a different order. So is preceded by audio data If an error occurs, it is not spoken of referenced in real time, but to those by the Au Diodata stream given order. Furthermore, it should be noted notices that audio data self-ver of course, also at an earlier point in time imprisoned or could not have been present. But when an error is detected in an audio data stream, means this is that in any case before the error occurs There was audio data. So this doesn't mean that the method of the present invention only the  can handle very first errors in an audio stream.

As already mentioned, the un expected absence of audio data as well the presence of faulty, i.e. H. partially destroyed audio data called errors. So if faulty Audio data are replaced by replacement data, so means this means that either partially destroyed audio data by Er record data are replaced, or that at the point where the un As expected, no audio data is available, replacement data be used.

When it is also said in this application that calculate spectral energies based on intact audio data net, so it should not mean that the whole intact audio data between the current and the last one Errors are used. Instead, the amount of in clock audio data that the audio data stream before the occurrence of the Represent errors, depending on the individual application gig. In embodiments of the present invention, which work block-based, the amount of intact Audio data, for example, only audio data from the last block or more last blocks. Furthermore, do not have to absolutely all intact audio data, for example one Blocks are used, but only for example Audio data, the information about energies per subgroup exhibit. A strength of the method of the present Invention is precisely that not all Frequency lines are saved and predicted in the event of an error have to. However, the more audio data to form a pre location for substitute data can be used to be more precise Substitute data are generated from the spectral template to to hide a mistake better.

The method according to the invention is described below of an audio coding method, the spectrally coded audio data generated and transmitted, shown as an example. The most modern audio coding methods, such as. B. MPEG layer  1-3, MPEG AAC, Dolby AC-3, PAC, provide spectrally coded Audio data, which emp from a corresponding decoder catch, decode and put back into the time domain become.

As is well known, an ana log signal with a suitable sampling frequency, such as. B. 44.1 kHz, sampled to obtain a discrete time signal Such a discrete time signal can now be created using a appropriate window function to be windowed around blocks with a certain number of individual time-discrete signals to get values, in particular to the edges of the Blocks are weighted through the window function like it is well known in communications engineering. A block with time-discrete weighted by means of a window function Signals are now transformed using a suitable transformation procedure from the time domain to the frequency domain transfor mized. Suitable transformation methods can be one fast Fourier transformation (FFT), a filter bank process or a modified discrete cosine transform (MDCT). A block of discrete-time signals is behind the transformation as a block of frequency lines. The Block of frequency lines or the frequency spectrum of the Blocks of discrete-time signal values can now be quantized become. Taking into account psychoacoustic facial points it is possible under certain conditions to get a quantization disturbance introduced by the quantization to be designed so that it is not audible, d. H. that she is below the hearing threshold of human hearing.

With modern audio coding methods, they are now quanti spectral lines of a redundancy re duction, for example by means of a Huffman coding, un trained and together with the necessary for decoding information is packed into an audio data stream, which is now is ready for transmission to a recipient.

At this point it should be noted that the expression "audio data stream"  in this application not only as previously described ben recorded audio data streams, but any form of audio data, be it encoded or uncoded. That's the way it is not required for the present invention that Audio data stream in blocks of equal length or at all Blocks is divided. However, the audio data stream lies spectrally coded in block form, so the fiction appropriate procedures can be implemented with the least effort. If the audio data stream is in a different form, then the audio data using known steps and procedures be processed to carry out the method according to the invention respectively.

The method according to the present invention thus has four steps up. The first step is the up detection of an error in an audio data stream the second step and the third step are a spectral energy of a subset in the audio data to calculate the audio stream before the occurrence of the Represent error, d. H. in the intact audio data, and then a template for replacement data based on that for the Subset of the intact audio data calculated spectral To form energies. The final step is Replacement data for incorrect or non-existent audio data based on the template for the subset for which the template is formed to generate.

The first step is to record the occurrence of a mistake. If the input data stream drops or points he errors, which by means of suitable facilities for example to check a checksum or similar in an encoded audio signal can be detected at for example a certain proportion of the data before occurrence this error z. B. stored in a ring buffer.

The method according to the present invention now calculates the spectral energy for each subgroup. While the well-known variant for line-by-line prediction typically  20 to 30 times more memory and computing operations in the Comparison to the method of the present invention (when the Subgroups correspond to the frequency groups) the spectral energy for each sub-group according to the present ing invention can be calculated efficiently and economically. It however, it should be noted that the aforementioned Memory and arithmetic operation effort only with regard the error concealment applies. The overall complexity (Spei or computing time) for example of a complete MPEG Layer 3 decoders generally do not increase significantly. If, on the other hand, a line-by-line prediction is used, then the total computing effort of the encoder / decoder very well increase significantly.

The spectral energy per subgroup is calculated as Sum of the squares of the amounts contained in the subgroup Spectral components. The spectral energy per subgroup corresponds to the sum of the squared lines in this Subgroup. This applies both when using an MDCT as well as when using an FFT for the time fre quenz transform to discrete-time audio signals in the Transform frequency range, as is the case with modern Co which is common. At this point, however, be on the front comments made that the MDCT in Ge The Fourier transformation principle is not exactly energy-saving is tend.

The spectral energy of a subgroup is therefore the reason was able to create a template for replacement data. The he Record data is supposed to be a "spectral average" spectrum in the sense of short-term statistics. "Spectral by average "means that part of such a signal, if he according to the method of the present invention is not particularly noticeable, d. H. that the Replacement is veiled.

For reasons of simplicity, the following illustration is shown the present invention based on a block-oriented  Procedure, as is the case with most modern audio code is the case. However, it should be noted that the present invention not to a block oriented Ver driving is limited. The prerequisite for the procedure is that information about the spectral composition of the Si gnales can be obtained to make the spectra out of the same to calculate le energy per subgroup. (This is natural especially with block-oriented transformation codecs simple.)

To determine the template, d. H. short-term statistics, for Substitute data is received in the past (intact) Data used. In principle, it would also be possible after inserting a delay in the signal path "future ge "to use data after an error and an interpola tion. However, the disadvantage is that delay already mentioned. The procedure of the present However, the invention does not use interpolation but one Prediction. One can speak of prediction, since the The result of this operation is always later is the spectrum that is derived from earlier data becomes. So there is always audio data before the occurrence of a Error, d. H. intact audio data, used to in case of an occurring error artificially generated replacement data instead of incorrect data or instead of nonexistent data to hide the mistake. Still there pointed out that the expression "prediction" in the Li usually tighter. Making an original for substitute data based on that for the subset of the intact The spectral energy calculated audio data can be more or less complex and therefore more or less precise be designed.

For the individual procedures described below, the following conventions are first defined. X _{i means} the short-term spectrum, ie the vector of short-term spectral components, of a block i. E _i (f) represents the spectral energy of a subgroup of the center frequency f in block i. It should be noted that i = 0 represents the block to be predicted, ie the block of replacement data, while negative i represent the stored blocks of the intact audio data, and N corresponds to the number of blocks stored. E _V (f) may stand for the energy required by the submission in the center frequency f subgroup.

Sorted in ascending order according to the complexity of the calculation, the following methods can be used to create a template for replacement data:

• 1. The spectral energies per subgroup of the last block received are used directly as a template for the missing block. This is expressed in the following equation:
  E _V = E _-1
• 2. The template is created as a linear combination of potencies of the spectral energies of corresponding subgroups of past blocks:
  a _i are constant coefficients for linear combination, while b _{i are} exponents for weighting the individual spectral energies. The simplest case is the averaging, which can be represented by the following coefficient values:
  a _i = 1 / N and b _i = 1.
  
• 3. The template for substitute data can also be obtained by median education over past energies. This is represented by the following equation:
  E _V (f) = median {E _i (f), i = -1. , , N}
  Alternatively, the smallest, the second smallest,. , , ., the second largest or the largest element from the E _i (f) can be used.
• 4. However, the template for the replacement data based on the spectral energy calculated for the subgroup of the intact audio data can also be calculated using a linear adaptive prediction. This corresponds to the equation set out in the second case, which, however, must now have time-dependent coefficients a _i (t). In this case, future energies can also be calculated. A linear adaptive prediction for forming the template for replacement data is particularly advantageous when more than one block of data is missing or has to be replaced in the data stream.

At this point it should be noted that the above list no claim to calculate a template for replacement data completeness. It is only important that the submission for the replacement data of a subgroup from spectral energy gien correspond to a certain number of previous the subgroups of intact audio data is obtained.

Now that we have a template for replacement data based on the for the subset of the intact audio data calculated spec energy has been formed, finally there is The last step in the process is to add the replacement data generate that instead of faulty or nonexistent NEN audio data inserted into the audio data stream or sub should be stituted.

Substitute data should be generated that have the same spec central energy per subgroup like the template for the Er should have record data. The basic procedure So it consists of faulty or nonexistent audio data which will ultimately be spectral components,  to be replaced by artificially generated spectral components. The boundary condition is therefore that the artificially generated Spectral components in a subset together the same have spectral energy given by the template is. There are various ways of doing this. It can for example all spectral values in a sub-group are equated with the size of the spectral values or Frequency lines are given by the spectral energy the same of the corresponding spectral energy of the pre location corresponds.

Another option could be from the front to set a certain weight in it, which in it could exist spectral values at the edge of a subgroup more likely to attenuate and spectral values in the middle of a sub highlight group. The corresponding spectral values too together, however, the same spectral energy as one have the corresponding subgroup of the template.

The preferred method of generating replacement data be However, it says all spectral components within one Subgroup on evenly distributed random values X (f) between To set -1 and +1, and subsequently a scaling or To perform standardization at which the signal just generated the level is raised / weakened so that it is within each subgroup the spectral required by the template Owns energy.

For this purpose, after generating the random signal X, each line X (f), which by definition will have a value between -1 and +1, must be multiplied within the corresponding subgroup by the following quotient, which is also referred to as the normalization factor α:

E _V (f _k ) here represents the spectral energy in the template within a subgroup k of the center frequency f _k . E _V (f _k ) thus represents the template for the subgroup k with the center frequency f _k .

To simplify the previous regulation, in order to be able to do without the summation in the denominator, a random signal can also be generated, which contains values equally distributed between + P and -P, where P must satisfy the following value:

The energy of this signal is then very likely to be close to the required energy E _V (f _k ). This certainly applies to subgroups that contain a larger (approximately 10) number of frequency lines. This is because the square mean of equally distributed values between -1 and +1 is 1/3.

The replacement data or spectral components for the subset are processed by a decoder as well works like error-free data or spectral values. by virtue of the fact that the spectral energies of the subgroup the spectral energies in the artificially generated data correspond to the subgroups in the last intact block or from the spectral energies of subgroups of several before outgoing blocks are derived, the block with artificially generated substitute data or the blocks with art Lich generated replacement data with somewhat evenly distributed Signal statistics of the audio signal to be inaudible. A mistake will be veiled.

In the inventive method for concealing However, errors in an audio data stream can be transient Signals can be problematic because then the basis of the substitu tion may be a block with atypical for the signal Statistics is used. To prevent a block  with atypical statistics, on which an erroneous The block follows, forming a template for replacement data dominated for the faulty block, causing audible errors could be introduced, it may not only be cheap the last intact block, but several last before save existing intact blocks and then for one to use the prediction mentioned to provide a template for Er to obtain sentence data which is the greater tendency of the signal history or signal statistics and not just the status stik or the course of a block.

Contains the undisturbed or intact signal or the unsound disrupted or intact audio data stream a prominent or outstanding spectral component that is close to the edge of a Subgroup, so one is by the just described Process generated similarly within the subgroup divided ("white") noise with a different pitch, namely the center frequency of the subgroup. Around To avoid this effect, the subgroups in Frequency ranges in which the ear is at such pitches fluctuations is sensitive, narrow, even only one frequency line wide. In the latter case united the procedure fuses instead of with the square of the amount to calculate the spectral component of the template - directly the Spectral component of the template with a random from (+1, -1) lig selected number can be multiplied.

Claims (9)

1. A method for concealing errors in an audio data stream, the audio data stream having successive blocks of spectral values, each block of spectral values forming the spectrum of a part of the temporal audio signal on which the audio data stream is based, the spectral values of a block being grouped into subgroups of spectral values are, with a subgroup having a plurality of spectral values, and with the grouping into subgroups for the successive blocks each comprising the same frequency ranges, with the following steps:
Detecting the occurrence of an error in a block of spectral values that has a faulty subset, wherein spectral values in a block that precedes the faulty block are intact spectral values;
Computing a spectral energy of the subset of the block of intact spectral values corresponding to the defective subset;
Forming a template for substitute spectral values for the defective subgroup, the template being equal to or derived from the calculated spectral energy of the subgroup of the intact spectral values; and
Generating replacement spectral values for the defective subgroup, such that the spectral energy of the generated replacement spectral values in the defective subgroup essentially corresponds to the spectral energy specified by the template for the defective subgroup. 2. The method according to claim 1,
in which, when a defective block occurs, the spectral energies are calculated for each subgroup of the last intact block; and
in which the substitute spectral values for each subgroup of the defective block have the spectral energies of the corresponding subgroups of the last intact block. 3. The method according to claim 1,
in which, when a defective block occurs, the spectral energies are calculated for corresponding subgroups of at least two preceding intact blocks; and
in which the template for the replacement spectral values for each subgroup of the defective block is determined by means of prediction from the corresponding subgroups of the at least two preceding intact blocks. 4. The method according to claim 2 or 3, wherein the template for a subgroup by a weighted linear com combination of potencies of the spectral energies corresponding subgroups of the previous intact blocks is formed. 5. The method according to claim 2 or 3, wherein the template for a subgroup by averaging the spectral energies of corresponding subgroups of the preceding intact blocks is formed. 6. The method according to claim 2 or 3, wherein the template for a subgroup by median formation over the spec central energies of corresponding subgroups outgoing intact blocks is formed.   7. The method according to claim 2 or 3, wherein the template for a subset by a linear adaptive pre diction about the spectral energies of corresponding Un Subgroups of the previous intact blocks are formed becomes. 8. The method according to any one of the preceding claims, wherein the step of generating replacement spectral values comprises the following steps:
Setting spectral values within the faulty subgroup to equally distributed random values between -1 and +1; and
Scaling the set spectral values such that the spectral energy of the set spectral values of the defective subgroup is equal to the template for the substitute spectral values in order to obtain the substitute spectral values. 9. The method according to claim 1, wherein the following steps are carried out before the detection of the occurrence of an error in an audio data stream:
Receiving the audio data stream; and
Saving at least the data of the audio data stream received last in time in the sequence that is predetermined by the audio data stream.