EP1697930B1 - Device and method for processing a multi-channel signal - Google Patents

Device and method for processing a multi-channel signal Download PDF

Info

Publication number
EP1697930B1
EP1697930B1 EP05715611A EP05715611A EP1697930B1 EP 1697930 B1 EP1697930 B1 EP 1697930B1 EP 05715611 A EP05715611 A EP 05715611A EP 05715611 A EP05715611 A EP 05715611A EP 1697930 B1 EP1697930 B1 EP 1697930B1
Authority
EP
European Patent Office
Prior art keywords
channel
prediction
block
similarity
spectral
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP05715611A
Other languages
German (de)
French (fr)
Other versions
EP1697930A1 (en
Inventor
Jürgen HERRE
Michael Schug
Alexander Groeschl
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP1697930A1 publication Critical patent/EP1697930A1/en
Application granted granted Critical
Publication of EP1697930B1 publication Critical patent/EP1697930B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
    • G10L19/03—Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders

Definitions

  • the present invention relates to audio encoders, and more particularly to audio encoders that are transformation-based, that is, where a temporal representation is converted to a spectral representation at the beginning of the encoder pipeline.
  • FIG. 3 A known transform-based audio encoder is shown in FIG.
  • the encoder shown in Fig. 3 is in International Standard ISO / IEC 14496-3: 2001 (E), Subpart 4, page 4 , and also known in the art as an AAC encoder.
  • an audio signal to be coded is fed. This is first supplied to a scaling stage 1002 in which a so-called AAC gain control is performed to set the level of the audio signal. Scaling page information is provided to a bitstream formatter 1004, as indicated by the arrow between block 1002 and block 1004. The scaled audio signal is then applied to an MDCT filter bank 1006.
  • the filter bank implements a modified discrete cosine transform with 50% overlapping windows, the window length being determined by a block 1008.
  • block 1008 is for windowing transient signals with shorter windows, and for windowing stationary signals with longer windows. This serves to achieve a higher time resolution (at the expense of frequency resolution) due to the shorter transient signal windows while For more stationary signals, a higher frequency resolution (at the expense of time resolution) is achieved by longer windows, with longer windows tending to be preferred because they promise greater coding gain.
  • temporally successive blocks of spectral values are present, which, depending on the embodiment of the filter bank, may be MDCT coefficients, Fourier coefficients or even subband signals, each subband signal having a certain limited bandwidth passing through the corresponding subband channel in the filter bank 1006, and wherein each subband signal has a certain number of subband samples.
  • the filter bank outputs temporally successive blocks of MDCT spectral coefficients, which generally represent successive short-term spectra of the audio signal to be encoded at input 1000.
  • a block of MDCT spectral values is then fed into a TNS processing block 1010 in which temporal noise shaping (TNS) takes place.
  • TNS temporal noise shaping
  • the TNS technique is used to shape the temporal shape of the quantization noise within each window of the transform. This is achieved by applying a filtering process to parts of the spectral data of each channel.
  • the coding is performed on a window basis.
  • the following steps are performed to apply the TNS tool to a window of spectral data, that is, to a block of spectral values.
  • a frequency range is selected for the TNS tool.
  • a suitable choice is to cover a frequency range of 1.5 kHz up to the highest possible scale factor band with a filter. It should be noted that this frequency range depends on the sampling rate as specified in the AAC standard (ISO / IEC 14496-3: 2001 (E)).
  • LPC linear predictive coding
  • the expected prediction gain PG is obtained. Further, the reflection coefficients or Parcor coefficients are obtained.
  • the TNS tool is not applied. In this case, control information is written in the bit stream for a decoder to know that no TNS processing has been performed.
  • TNS processing is applied.
  • the reflection coefficients are quantized.
  • the order of the noise shaping filter used is determined by removing all the reflection coefficients having an absolute value less than a threshold from the "tail" of the reflection coefficient array. The number of remaining reflection coefficients is on the order of the noise shaping filter.
  • a suitable threshold is 0.1.
  • the remaining reflection coefficients are typically converted to linear prediction coefficients, which technique is also known as a "step-up" procedure.
  • the calculated LPC coefficients are then used as coder noise shaping filter coefficients, ie as prediction filter coefficients.
  • This FIR filter is routed over the specified target frequency range.
  • the decoding uses an autoregressive filter, while the coding uses a so-called moving average filter.
  • the page information for the TNS tool is also supplied to the bit stream formatter as shown by the arrow shown between the block TNS processing 1010 and the bitstream formatter 1004 in FIG.
  • the center / side encoder 1012 is active when the audio signal to be encoded is a multi-channel signal, that is, a stereo signal having a left channel and a right channel. So far, that is, in the processing direction before the block 1012 in Fig. 3, the left and right stereo channels have been separately processed, that is, scaled, transformed by the filter bank, subjected to TNS processing or not, etc.
  • middle / side encoder In the middle / side encoder is then first checked whether a middle / side encoding makes sense, that brings a coding gain at all. A middle / side encoding will then bring a coding gain if the left and the right channel are more similar, because then the center channel, that is the sum of the left and the right channel is almost equal to the left or the right channel, apart from the scaling by the factor 1/2, while the page channel has only very small values, since it is equal to the difference between the left and the right channel.
  • the quantizer 1014 is given a allowed perturbation per scale factor band by a psycho-acoustic model 1020.
  • the quantizer operates iteratively, d. H.
  • An external iteration loop is called first, which then calls an inner iteration loop.
  • a quantization of a block of values is made at the input of the quantizer 1014.
  • the inner loop quantizes the MDCT coefficients, consuming a certain number of bits.
  • the outer loop calculates the distortion and modified energy of the coefficients using the scale factor to again invoke an inner loop. This process is iterated until a certain conditional set is met.
  • the signal is reconstructed to compute the perturbation introduced by the quantization and to compare it with the allowable perturbation provided by the psycho-acoustic model 1020. Furthermore, the scale factors are increased from iteration to iteration by one step, for each iteration of the outer iteration loop.
  • the iteration ie the analysis-by-synthesis procedure is terminated, and the resulting scale factors are encoded as set forth in block 1014 and supplied in encoded form to bitstream formatter 1004, as indicated by the arrow drawn between the block 1014 and the block 1004.
  • the quantized values are then fed to entropy coder 1016, which typically performs entropy coding using several Huffman code tables for different scale factor bands to transmit the quantized values into a binary format.
  • entropy coding in the form of Huffman coding relies on code tables that are created on the basis of expected signal statistics and in which frequently occurring values get shorter code words than more rarely occurring values.
  • the entropy-coded values are then also supplied as actual main information to the bit stream formatter 1004, which then outputs the coded audio signal on the output side in accordance with a specific bit stream syntax.
  • predictive filtering is used in TNS processing block 1010 to time-shaping the quantization noise within an encoding frame.
  • the temporal shaping of the quantization noise is performed by filtering the spectral coefficients over the frequency in the encoder before the quantization and subsequent inverse filtering in the decoder.
  • TNS processing causes the quantization noise envelope to be timed below the envelope of the signal to avoid pre-echo artifacts.
  • the application of the TNS results from an estimation of the prediction gain of the filtering as stated above.
  • the filter coefficients for each encoding frame are determined via a correlation measure. The calculation of the filter coefficients is done separately for each channel. They are also transmitted separately in the coded bit stream.
  • a disadvantage of the activation / deactivation of the TNS concept is the fact that for each stereo channel, if Once TNS processing has been activated due to the good expected coding gain, the TNS filtering for each channel takes place separately. So this is still unproblematic with relatively different channels.
  • the left and right channels are relatively similar, then the left and right channels in an extreme example have exactly the same payload, such as a speaker, and differ only in terms of the noise inevitably contained in the channels, so when standing Nevertheless, the technology calculates and uses a separate TNS filter for each channel.
  • the TNS filter depends directly on the left or right channel, and in particular reacts relatively sensitively to the spectral data of the left and the right channel, the signal is also very similar in the case of a signal in which the left and the right channel are very similar
  • a TNS processing with a separate prediction filter is carried out for each channel. This leads to a different temporal noise shaping taking place in the two stereo channels due to the different filter coefficients.
  • the known procedure has a far possibly even more serious disadvantage.
  • the TNS output values that is, the spectral residuals
  • the spectral residuals are subjected to center / side coding in the center / side encoder 1002 of FIG. While the two channels were still relatively the same before TNS processing, this can not be said after TNS processing.
  • the described stereo effect introduced by the separate TNS processing, makes the spectral residuals of the two channels more dissimilar than they would actually be. This leads to a immediate drop in coding gain due to the mid / side coding, which is particularly disadvantageous for applications where a low bit rate is required.
  • the known TNS activation is thus problematic for stereo signals that use similar but not exactly identical signal information in both channels, such as mono-like speech signals. If different filter coefficients are determined for both channels in the case of TNS detection, this leads to a temporally different shaping of the quantization noise in the channels. This can lead to audible artifacts because z. B. the original mono-like sound image gets an unwanted stereo character through these temporal differences. Furthermore, as has been stated, the TNS-modified spectrum is subjected to center / side encoding in a subsequent step. Different filters in both channels additionally reduce the similarity of the spectral coefficients and thus the center / side gain.
  • the DE 19829284C2 discloses a method and apparatus for processing a temporal stereo signal and a method and apparatus for decoding an audio bitstream encoded using prediction over frequency.
  • the left, the right and the mono channel can be subjected to their own prediction over the frequency, ie a TNS processing.
  • a separate complete prediction can be performed for each channel.
  • calculation of the prediction coefficients for the left channel may be performed, which are then used to filter the right channel and the mono channel.
  • the object of the present invention is to provide a concept for processing a multi-channel signal, the lower artifacts and still allows a good compression of the information.
  • the present invention is based on the finding that if the left and the right channel are similar, that is to say exceed a similarity measure, the same TNS filtering is to be used for both channels. This ensures that the TNS processing no pseudo-stereo artifacts are introduced into the multi-channel signal, as is achieved by using the same prediction filter for both channels that the temporal shaping of the quantization noise for both channels takes place identically, so that no pseudo Stereo artifacts are heard.
  • the similarity of the signals after the TNS filtering ie the similarity of the residual spectral values corresponds to the similarity of the input signals in the filter and not, as in the prior art, the similarity of the input signals, which is still reduced by different filters.
  • Fig. 1 shows an apparatus for processing a multi-channel signal, wherein the multi-channel signal is represented by one block of spectral values for at least two channels, as shown by L and R.
  • the blocks of spectral values are represented by z.
  • the blocks of spectral values for each channel are then fed in a preferred embodiment of the present invention to a means 12 for determining a similarity between the two channels.
  • the means for determining the similarity between the two channels may also be as shown in FIG Using time domain samples 1 (t) or r (t) for each channel.
  • the means 12 for determining the similarity between the first and the second channel is operative to generate, based on a similarity measure or alternatively a measure of dissimilarity, a control signal on a control line 14 having at least two states, one of which expresses that Blocks of spectral values of the two channels are similar, or that in its other state states that the blocks of spectral values are dissimilar for each channel.
  • the decision as to whether similarity or dissimilarity prevails can be made using a preferably numerical similarity measure.
  • Both the block of spectral values for the left channel and the block of spectral values for the right channel are fed to a means 16 for performing a prediction filtering.
  • predictive filtering is performed over the frequency, the means being adapted to perform, to perform the prediction versus frequency, a common prediction filter 16a for the block of spectral values of the first channel and for the block of spectral values of the first channel second channel if the similarity is greater than a threshold similarity.
  • the means 16 for performing the prediction filtering is notified by the similarity determining means 12 that the two blocks of spectral values are dissimilar for each channel, that is, similar to less than a threshold similarity, then the means 16 is to perform the prediction filtering apply different filters 16b to the left and right channels.
  • the output signals of device 16 are thus left-channel spectral residuals at output 18a as well as right-channel spectral residuals at output 18b, where, depending on the similarity of the left and right channels, the spectral residuals of the two channels using the same prediction filter (Case 16a) or using different prediction filters (Case 16b).
  • the spectral residuals of the left and right channels may be either directly or after multiple processing such as described in US Pat. B.
  • AAC standard are supplied to a center / side stereo encoder, which outputs at an output 21 a, the center signal as half of the sum of left and right channel, while the side signal as half of the difference of left and right right channel is output.
  • the page signal is now smaller due to the synchronization of the TNS processing of the two channels than in the case where different TNS filters are used for similar channels.
  • this promises a higher coding gain.
  • Fig. 2 there is shown a preferred embodiment of the present invention in which the first stage of the TNS calculation is already performed in the similarity determining means 12, namely the calculation of the parc reflection coefficients and the prediction gain for both the left channel and the right channel, as represented by blocks 12a, 12b.
  • This TNS processing thus provides both the filter coefficients for the final prediction filter to be used and the prediction gain, and this prediction gain is also needed to decide whether or not TNS processing should be performed at all.
  • the prediction gain for the first, left channel, denoted by PG1 in FIG. 2, as well as the prediction gain for the right channel, denoted PG2 in FIG. 2, are fed to a similarity measure determiner, shown in FIG 12c is designated.
  • This similarity determining means is operable to calculate the absolute amount of the difference or the relative difference of the two prediction gains and to see if it is below a predetermined deviation threshold S. If the absolute amount of the difference of the prediction gains is below the threshold S, then it is assumed that the two signals are similar, and the question in the block 12c is answered with Yes. If, on the other hand, it is determined that the difference is greater than the similarity threshold S, the question is answered with no.
  • a common filter is used for both channels L and R, while in the case of answering the question in block 12c with No separate filters are used, ie a TNS processing, as in the state the technique can be performed.
  • the device 16 is supplied with a set of filter coefficients FKL for the left channel and a set of filter coefficients FKR for the right channel from the devices 12a and 12b, respectively.
  • a particular selection is made in a block 16c.
  • block 16c it is decided which channel has the greater energy. If it is determined that the left channel has the greater energy, the filter coefficients FKL calculated by the left channel device 12a are used for the common filtering. On the other hand, if it is determined in block 16c that the right channel has the greater energy, then for common filtering, the set of filter coefficients FKR calculated for the right channel in the device 12b is used.
  • both the time signal and the spectral signal can be used for energy determination. Due to the fact that transformation artifacts that may have already taken place in the spectral signal are preferred, it is preferable to use the spectral signals of the left and right channels for the "energy decision" in block 16c.
  • a TNS synchronization that is, the use of the same filter coefficients for both channels is used when the prediction gains for the left and right channels differ by less than three percent. If both channels differ by more than three percent, the question is answered in block 12c of FIG. 2 with "no".
  • the similarity determination may also be achieved using other details of the signal, so that when a similarity has been determined, only the TNS filter coefficient set needs to be calculated for the channel that will be used for the prediction filtering of both stereo channels. This has the advantage that, if Fig. 2 is considered, and if the signals are similar, only either block 12a or block 12b will be active.
  • the concept according to the invention can also be used to further reduce the bit rate of the coded signal. While different TNS page information is transmitted for both channels when using two different reflection coefficients, when filtering the two channels with the same prediction filter, TNS information must be transmitted only once for both channels. Therefore, the concept of the invention can also achieve a reduction of the bit rate such that a set of TNS page information is "saved" if the left and right channels are similar.
  • the inventive concept is not basically limited to stereo signals, but could be applied in a multi-channel environment between different channel pairs or even groups of more than 2 channels.
  • a determination of the left-right channel cross-correlation measure k or a determination of the TNS prediction gain and the TNS filter coefficients may be made separately for each channel.
  • the synchronization decision is made if k exceeds a threshold (e.g., 0.6) and MS stereo coding is enabled.
  • a threshold e.g., 0.6
  • MS stereo coding is enabled.
  • the MS criterion can also be omitted.
  • TNS prediction gain and TNS filter coefficients are made separately for each channel. Then a decision is made. If the prediction gain of both channels differs by no more than a degree, e.g. B. 3%, the synchronization takes place.
  • the reference channel can also be chosen arbitrarily, if one can assume a similarity of the channels. Again, there is a copying of the TNS filter coefficients from the reference channel to the other channel, whereupon an application of the synchronized or unsynchronized TNS filters to the spectrum takes place.
  • TNS in a channel is always activated depends on the prediction gain in this channel. If this exceeds a certain threshold, TNS is activated for this channel. Alternatively, a TNS synchronization is made for 2 channels if TNS was activated in only one of the two channels. Condition is then that e.g. the prediction gain is similar, ie one channel just above the activation limit, and one channel just below the activation limit. From this comparison, the activation of TNS for both channels with equal coefficients is derived, or possibly the deactivation for both channels.
  • the inventive method for processing a multi-channel signal can be implemented in hardware or in software.
  • the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which may interact with a programmable computer system such that the method is performed.
  • the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer.
  • the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Signal Processing (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Mathematical Physics (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Time-Division Multiplex Systems (AREA)
  • Stereo-Broadcasting Methods (AREA)
  • Stereophonic System (AREA)
  • Transmission Systems Not Characterized By The Medium Used For Transmission (AREA)
  • Radio Relay Systems (AREA)
  • Color Image Communication Systems (AREA)
  • Electrical Discharge Machining, Electrochemical Machining, And Combined Machining (AREA)
  • Detergent Compositions (AREA)

Abstract

An apparatus for processing a multi-channel signal includes a means for determining a similarity between a first one of two channels and a second one of the two channels. Furthermore, a means for performing a prediction filtering of the spectral coefficients is provided, which is formed to perform a prediction filtering with only a single prediction filter for both channels in case of high similarity between the first and the second channel, and to perform a prediction filtering with two separate prediction filters in case of a dissimilarity between the first and the second channel. With this, an introduction of stereo artifacts and a deterioration of the coding gain in stereo coding techniques are avoided.

Description

Die vorliegende Erfindung bezieht sich auf Audiocodierer und insbesondere auf Audiocodierer, die Transformationsbasiert sind, d.h., bei denen zu Beginn der Codierer-Pipeline eine Umsetzung einer zeitlichen Darstellung in eine spektrale Darstellung erfolgt.The present invention relates to audio encoders, and more particularly to audio encoders that are transformation-based, that is, where a temporal representation is converted to a spectral representation at the beginning of the encoder pipeline.

Ein bekannter Transformations-basierter Audiocodierer ist in Fig. 3 gezeigt. Der in Fig. 3 gezeigte Codierer ist im internationalen Standard ISO/IEC 14496-3: 2001 (E), Subpart 4, Seite 4 , dargestellt und in der Technik auch als AAC-Codierer bekannt.A known transform-based audio encoder is shown in FIG. The encoder shown in Fig. 3 is in International Standard ISO / IEC 14496-3: 2001 (E), Subpart 4, page 4 , and also known in the art as an AAC encoder.

Nachfolgend wird der bekannte Codierer dargestellt. An einem Eingang 1000 wird ein zu codierendes Audiosignal eingespeist. Dieses wird zunächst einer Skalierungsstufe 1002 zugeführt, in der eine sogenannte AAC-Verstärkungssteuerung durchgeführt wird, um den Pegel des Audiosignals festzulegen. Seiteninformationen aus der Skalierung werden einem Bitstromformatierer 1004 zugeführt, wie es durch den Pfeil zwischen dem Block 1002 und dem Block 1004 dargestellt ist. Das skalierte Audiosignal wird hierauf einer MDCT-Filterbank 1006 zugeführt. Beim AAC-Codierer implementiert die Filterbank eine modifizierte diskrete Cosinustransformation mit 50 % überlappenden Fenstern, wobei die Fensterlänge durch einen Block 1008 bestimmt wird.The known coder is shown below. At an input 1000, an audio signal to be coded is fed. This is first supplied to a scaling stage 1002 in which a so-called AAC gain control is performed to set the level of the audio signal. Scaling page information is provided to a bitstream formatter 1004, as indicated by the arrow between block 1002 and block 1004. The scaled audio signal is then applied to an MDCT filter bank 1006. In the AAC encoder, the filter bank implements a modified discrete cosine transform with 50% overlapping windows, the window length being determined by a block 1008.

Allgemein gesagt ist der Block 1008 dazu vorhanden, dass transiente Signale mit kürzeren Fenstern gefenstert werden, und dass eher stationäre Signale mit längeren Fenstern gefenstert werden. Dies dient dazu, dass aufgrund der kürzeren Fenster für transiente Signale eine höhere Zeitauflösung (auf Kosten der Frequenzauflösung) erreicht wird, während für eher stationäre Signale eine höhere Frequenzauflösung (auf Kosten der Zeitauflösung) durch längere Fenster erreicht wird, wobei tendenziell längere Fenster bevorzugt werden, da sie einen größeren Codiergewinn versprechen. Am Ausgang der Filterbank 1006 liegen zeitlich betrachtet aufeinanderfolgende Blöcke von Spektralwerten vor, die je nach Ausführungsform der Filterbank MDCT-Koeffizienten, Fourier-Koeffizienten oder auch Subbandsignale sein können, wobei jedes Subbandsignal eine bestimmte begrenzte Bandbreite hat, die durch den entsprechenden Subbandkanal in der Filterbank 1006 festgelegt wird, und wobei jedes Subbandsignal eine bestimmte Anzahl von Subband-Abtastwerten aufweist.Generally speaking, block 1008 is for windowing transient signals with shorter windows, and for windowing stationary signals with longer windows. This serves to achieve a higher time resolution (at the expense of frequency resolution) due to the shorter transient signal windows while For more stationary signals, a higher frequency resolution (at the expense of time resolution) is achieved by longer windows, with longer windows tending to be preferred because they promise greater coding gain. At the output of the filter bank 1006, temporally successive blocks of spectral values are present, which, depending on the embodiment of the filter bank, may be MDCT coefficients, Fourier coefficients or even subband signals, each subband signal having a certain limited bandwidth passing through the corresponding subband channel in the filter bank 1006, and wherein each subband signal has a certain number of subband samples.

Nachfolgend wird beispielhaft der Fall dargestellt, bei dem die Filterbank zeitlich betrachtet aufeinanderfolgende Blöcke von MDCT-Spektralkoeffizienten ausgibt, die allgemein gesagt, aufeinanderfolgende Kurzzeitspektren des zu codierenden Audiosignals am Eingang 1000 darstellen. Ein Block von MDCT-Spektralwerten wird dann in einen TNS-Verarbeitungsblock 1010 eingespeist, in dem eine zeitliche Rauschformung stattfindet (TNS = temporary noise shaping). Die TNS-Technik wird dazu verwendet, um die zeitliche Form des Quantisierungsrauschens innerhalb jedes Fensters der Transformation zu formen. Dies wird dadurch erreicht, dass ein Filterprozess auf Teile der Spektraldaten jedes Kanals angewendet wird. Die Codierung wird auf einer Fensterbasis durchgeführt. Insbesondere werden die folgenden Schritte ausgeführt, um das TNS-Tool auf ein Fenster spektraler Daten, also auf einen Block von Spektralwerten anzuwenden.The following is an example of the case in which the filter bank outputs temporally successive blocks of MDCT spectral coefficients, which generally represent successive short-term spectra of the audio signal to be encoded at input 1000. A block of MDCT spectral values is then fed into a TNS processing block 1010 in which temporal noise shaping (TNS) takes place. The TNS technique is used to shape the temporal shape of the quantization noise within each window of the transform. This is achieved by applying a filtering process to parts of the spectral data of each channel. The coding is performed on a window basis. In particular, the following steps are performed to apply the TNS tool to a window of spectral data, that is, to a block of spectral values.

Zunächst wird ein Frequenzbereich für das TNS-Tool ausgewählt. Eine geeignete Auswahl besteht darin, einen Frequenzbereich von 1,5 kHz bis zum höchsten möglichen Skalenfaktorband mit einem Filter abzudecken. Es sei darauf hingewiesen, dass dieser Frequenzbereich von der Abtastrate abhängt, wie es im AAC-Standard (ISO/IEC 14496-3: 2001 (E)) spezifiziert ist.First, a frequency range is selected for the TNS tool. A suitable choice is to cover a frequency range of 1.5 kHz up to the highest possible scale factor band with a filter. It should be noted that this frequency range depends on the sampling rate as specified in the AAC standard (ISO / IEC 14496-3: 2001 (E)).

Anschließend wird eine LPC-Berechnung (LPC = linear predictive coding = lineare prädiktive Codierung) ausgeführt, und zwar mit den spektralen MDCT-Koeffizienten, die in dem ausgewählten Zielfrequenzbereich liegen. Für eine erhöhte Stabilität werden Koeffizienten, die Frequenzen unter 2,5 kHz entsprechen, aus diesem Prozess ausgeschlossen. Übliche LPC-Prozeduren, wie sie aus der Sprachverarbeitung bekannt sind, können für die LPC-Berechnung verwendet werden, beispielsweise der bekannte Levinson-Durbin-Algorithmus. Die Berechnung wird für die maximal zulässige Ordnung des Rauschformungsfilters ausgeführt.Subsequently, LPC (LPC = linear predictive coding) calculation is performed with the spectral MDCT coefficients lying within the selected target frequency range. For increased stability, coefficients corresponding to frequencies below 2.5 kHz are excluded from this process. Conventional LPC procedures, as known from speech processing, can be used for the LPC calculation, for example the known Levinson-Durbin algorithm. The calculation is performed for the maximum allowable order of the noise shaping filter.

Als Ergebnis der LPC-Berechnung wird der erwartete Prädiktionsgewinn PG erhalten. Ferner werden die Reflexionskoeffizienten oder Parcor-Koeffizienten erhalten.As a result of the LPC calculation, the expected prediction gain PG is obtained. Further, the reflection coefficients or Parcor coefficients are obtained.

Wenn der Prädiktionsgewinn eine bestimmte Schwelle nicht überschreitet, wird das TNS-Tool nicht angewendet. In diesem Fall wird eine Steuerinformation in den Bitstrom geschrieben, damit ein Decodierer weiß, dass keine TNS-Verarbeitung ausgeführt worden ist.If the prediction gain does not exceed a certain threshold, the TNS tool is not applied. In this case, control information is written in the bit stream for a decoder to know that no TNS processing has been performed.

Wenn der Prädiktionsgewinn jedoch eine Schwelle überschreitet, wird die TNS-Verarbeitung angewendet.However, if the prediction gain exceeds a threshold, TNS processing is applied.

In einem nächsten Schritt werden die Reflexionskoeffizienten quantisiert. Die Ordnung des verwendeten Rauschformungsfilters wird durch Entfernen aller Reflexionskoeffizienten mit einem Absolutwert kleiner als eine Schwelle von dem "Schwanz" des Reflexionskoeffizienten-Arrays bestimmt. Die Anzahl der verbleibenden Reflexionskoeffizienten liegt in der Größenordnung des Rauschformungsfilters. Eine geeignete Schwelle liegt bei 0,1.In a next step, the reflection coefficients are quantized. The order of the noise shaping filter used is determined by removing all the reflection coefficients having an absolute value less than a threshold from the "tail" of the reflection coefficient array. The number of remaining reflection coefficients is on the order of the noise shaping filter. A suitable threshold is 0.1.

Die verbleibenden Reflexionskoeffizienten werden typischerweise in lineare Prädiktionskoeffizienten umgewandelt, wobei diese Technik auch als "Step-Up"-Prozedur bekannt ist.The remaining reflection coefficients are typically converted to linear prediction coefficients, which technique is also known as a "step-up" procedure.

Die berechneten LPC-Koeffizienten werden dann_als Codierer-Rauschformungsfilterkoeffizienten, also als Prädiktionsfilterkoeffizienten verwendet. Dieses FIR-Filter wird über den spezifizierten Zielfrequenzbereich geführt. Bei der Decodierung wird ein autoregressives Filter verwendet, während bei der Codierung ein sogenanntes Moving-Average-Filter verwendet wird. Schließlich werden noch die Seiteninformationen für das TNS-Tool dem Bitstromformatierer zugeführt, wie es durch den Pfeil dargestellt ist, der zwischen dem Block TNS-Verarbeitung 1010 und dem Bitstromformatierer 1004 in Fig. 3 gezeigt ist.The calculated LPC coefficients are then used as coder noise shaping filter coefficients, ie as prediction filter coefficients. This FIR filter is routed over the specified target frequency range. The decoding uses an autoregressive filter, while the coding uses a so-called moving average filter. Finally, the page information for the TNS tool is also supplied to the bit stream formatter as shown by the arrow shown between the block TNS processing 1010 and the bitstream formatter 1004 in FIG.

Hierauf werden mehrere in Fig. 3 nicht gezeigte optionale Tools durchlaufen, wie beispielsweise ein Langzeitprädiktions-Tool, ein Intensity/Kopplungs-Tool, ein Prädiktions-Tool, ein Rauschsubstitutions-Tool, bis schließlich zu einem Mitte/Seite-Codierer 1012 gelangt wird. Der Mitte/Seite-Codierer 1012 ist dann aktiv, wenn das zu codierende Audiosignal ein Multikanalsignal ist, also ein Stereosignal mit einem linken Kanal und einem rechten Kanal. Bisher, also in der Verarbeitungsrichtung vor dem Block 1012 in Fig. 3 wurden der linke und der rechte Stereokanal getrennt voneinander verarbeitet, also skaliert, durch die Filterbank transformiert, der TNS-Verarbeitung unterzogen oder nicht etc.This is followed by a number of optional tools, not shown in FIG. 3, such as a long-term prediction tool, an intensity / coupling tool, a prediction tool, a noise substitution tool, and finally to a mid / side encoder 1012. The center / side encoder 1012 is active when the audio signal to be encoded is a multi-channel signal, that is, a stereo signal having a left channel and a right channel. So far, that is, in the processing direction before the block 1012 in Fig. 3, the left and right stereo channels have been separately processed, that is, scaled, transformed by the filter bank, subjected to TNS processing or not, etc.

Im Mitte/Seite-Codierer wird dann zunächst überprüft, ob eine Mitte/Seite-Codierung sinnvoll ist, also überhaupt einen Codiergewinn bringt. Eine Mitte/Seite-Codierung wird dann einen Codiergewinn bringen, wenn der linke und der rechte Kanal eher ähnlich sind, da dann der Mitte-Kanal, also die Summe aus dem linken und dem rechten Kanal nahezu gleich dem linken oder dem rechten Kanal ist, abgesehen von der Skalierung durch den Faktor 1/2, während der Seite-Kanal nur sehr kleine Werte hat, da er gleich der Differenz zwischen dem linken und dem rechten Kanal ist. Damit ist zu sehen, dass dann, wenn der linke und der rechte Kanal annähernd gleich sind, die Differenz annähernd Null ist bzw. nur ganz kleine Werte umfasst, die - so ist die Hoffnung - in einem nachfolgenden Quantisierer 1014 zu Null quantisiert werden und somit sehr effizient übertragen werden können, da dem Quantisierer 1014 ein Entropie-Codierer 1016 nachgeschaltet ist.In the middle / side encoder is then first checked whether a middle / side encoding makes sense, that brings a coding gain at all. A middle / side encoding will then bring a coding gain if the left and the right channel are more similar, because then the center channel, that is the sum of the left and the right channel is almost equal to the left or the right channel, apart from the scaling by the factor 1/2, while the page channel has only very small values, since it is equal to the difference between the left and the right channel. Thus, it can be seen that when the left and right channels are approximately equal, the difference is approximately zero or only very small values, which is the hope, are quantized to zero in a subsequent quantizer 1014 and thus can be transmitted very efficiently since the quantizer 1014 is followed by an entropy coder 1016.

Dem Quantisierer 1014 wird von einem psycho-akustischen Modell 1020 eine erlaubte Störung pro Skalenfaktorband zugeführt. Der Quantisierer arbeitet iterativ, d. h. es wird zunächst eine äußere Iterationsschleife aufgerufen, die dann eine innere Iterationsschleife aufruft. Allgemein gesagt wird zunächst, ausgehend von Quantisiererschrittweiten-Startwerten, eine Quantisierung eines Blocks von Werten am Eingang des Quantisierers 1014 vorgenommen. Insbesondere quantisiert die innere Schleife die MDCT-Koeffizienten, wobei eine bestimmte Anzahl von Bits verbraucht wird. Die äußere Schleife berechnet die Verzerrung und modifizierte Energie der Koeffizienten unter Verwendung des Skalenfaktors, um wieder eine innere Schleife aufzurufen. Dieser Prozess wird iteriert, bis ein bestimmter Bedingungssatz erfüllt ist. Für jede Iteration in der äußeren Iterationsschleife wird dabei das Signal rekonstruiert, um die durch die Quantisierung eingeführte Störung zu berechnen und mit der von dem psycho-akustischen Modell 1020 gelieferten erlaubten Störung zu vergleichen. Ferner werden die Skalenfaktoren von Iteration zu Iteration um eine Stufe vergrößert, und zwar für jede Iteration der äußeren Iterationsschleife.The quantizer 1014 is given a allowed perturbation per scale factor band by a psycho-acoustic model 1020. The quantizer operates iteratively, d. H. An external iteration loop is called first, which then calls an inner iteration loop. Generally speaking, first, based on quantizer step size start values, a quantization of a block of values is made at the input of the quantizer 1014. In particular, the inner loop quantizes the MDCT coefficients, consuming a certain number of bits. The outer loop calculates the distortion and modified energy of the coefficients using the scale factor to again invoke an inner loop. This process is iterated until a certain conditional set is met. For each iteration in the outer iteration loop, the signal is reconstructed to compute the perturbation introduced by the quantization and to compare it with the allowable perturbation provided by the psycho-acoustic model 1020. Furthermore, the scale factors are increased from iteration to iteration by one step, for each iteration of the outer iteration loop.

Dann, wenn eine Situation erreicht ist, bei der die durch die Quantisierung eingeführte Quantisierungsstörung unterhalb der durch das psycho-akustische Modell bestimmten erlaubten Störung ist, und wenn gleichzeitig Bitanforderungen erfüllt sind, nämlich, dass eine Maximalbitrate nicht überschritten wird, wird die Iteration, also das Analyse-Durch-Synthese-Verfahren beendet, und es werden die erhaltenen Skalenfaktoren codiert, wie es in dem Block 1014 ausgeführt ist und in codierter Form dem Bitstromformatierer 1004 zugeführt, wie es durch den Pfeil gekennzeichnet ist, der zwischen dem Block 1014 und dem Block 1004 gezeichnet ist. Die quantisierten Werte werden dann dem Entropie-Codierer 1016 zugeführt, der typischerweise unter Verwendung mehrerer Huffman-Code-Tabellen für verschiedene Skalenfaktorbänder eine Entropie-Codierung durchführt, um die quantisierten Werte in ein binäres Format zu übertragen. Wie es bekannt ist, wird bei der Entropie-Codierung in Form der Huffman-Codierung auf Code-Tabellen zurückgegriffen, die aufgrund einer erwarteten Signalstatistik erstellt werden, und bei denen häufig auftretende Werte kürzere Code-Wörter bekommen als seltener auftretende Werte. Die entropiecodierten Werte werden dann ebenfalls als eigentliche Hauptinformationen dem Bitstromformatierer 1004 zugeführt, der dann gemäß einer bestimmten Bitstromsyntax ausgangsseitig das codierte Audiosignal ausgibt.Then, when a situation is reached where the quantization disturbance introduced by the quantization is below the allowed disturbance determined by the psycho-acoustic model, and at the same time bit requirements are met, namely that a maximum bitrate is not exceeded, the iteration, ie the analysis-by-synthesis procedure is terminated, and the resulting scale factors are encoded as set forth in block 1014 and supplied in encoded form to bitstream formatter 1004, as indicated by the arrow drawn between the block 1014 and the block 1004. The quantized values are then fed to entropy coder 1016, which typically performs entropy coding using several Huffman code tables for different scale factor bands to transmit the quantized values into a binary format. As is well known, entropy coding in the form of Huffman coding relies on code tables that are created on the basis of expected signal statistics and in which frequently occurring values get shorter code words than more rarely occurring values. The entropy-coded values are then also supplied as actual main information to the bit stream formatter 1004, which then outputs the coded audio signal on the output side in accordance with a specific bit stream syntax.

Wie es bereits ausgeführt worden ist, wird im TNS-Verarbeitungsblock 1010 zur zeitlichen Formung des Quantisierungsgeräusches innerhalb eines Codierframes eine Prädiktionsfilterung verwendet.As already stated, predictive filtering is used in TNS processing block 1010 to time-shaping the quantization noise within an encoding frame.

Insbesondere erfolgt die zeitliche Formung des Quantisierungsrauschens durch Filterung der Spektralkoeffizienten über der Frequenz im Codierer vor der Quantisierung und anschließenden inversen Filterung im Decodierer. Die TNS-Verarbeitung bewirkt, dass die Hüllkurve des Quantisierungsrauschens zeitlich unter die Hüllkurve des Signals geschoben wird, um Vorecho-Artefakte zu vermeiden. Die Anwendung des TNS ergibt sich aus einer Schätzung des Prädiktionsgewinns der Filterung, wie es vorstehend ausgeführt worden. Die Filterkoeffizienten für jeden Codierframe werden über ein Korrelationsmaß bestimmt. Die Berechnung der Filterkoeffizienten erfolgt separat für jeden Kanal. Sie werden ebenfalls separat im codierten Bitstrom übertragen.In particular, the temporal shaping of the quantization noise is performed by filtering the spectral coefficients over the frequency in the encoder before the quantization and subsequent inverse filtering in the decoder. TNS processing causes the quantization noise envelope to be timed below the envelope of the signal to avoid pre-echo artifacts. The application of the TNS results from an estimation of the prediction gain of the filtering as stated above. The filter coefficients for each encoding frame are determined via a correlation measure. The calculation of the filter coefficients is done separately for each channel. They are also transmitted separately in the coded bit stream.

Nachteilig an der Aktivierung/Deaktivierung des TNS-Konzepts ist die Tatsache, dass für jeden Stereokanal, wenn einmal eine TNS-Verarbeitung aufgrund des guten erwarteten Codiergewinns aktiviert worden ist, die TNS-Filterung für jeden Kanal getrennt stattfindet. So ist dies bei relativ unterschiedlichen Kanälen noch unproblematisch. Sind jedoch der linke und der rechte Kanal relativ ähnlich, haben also der linke und der rechte Kanal in einem Extrembeispiel genau dieselbe Nutzinformation, wie beispielsweise ein Sprecher, und unterscheiden sich lediglich im Hinblick auf das in den Kanälen unweigerlich enthaltene Rauschen, so wird beim Stand der Technik dennoch für jeden Kanal ein eigenes TNS-Filter berechnet und verwendet. Nachdem das TNS-Filter direkt vom linken bzw. rechten Kanal abhängt, und insbesondere auf die Spektraldaten des linken und des rechten Kanals relativ empfindlich reagiert, wird auch im Falle eines Signals, bei dem der linke und der rechte Kanal sehr ähnlich sind, also im Falle eines sogenannten "Quasi-Mono-Signals", für jeden Kanal eine TNS-Verarbeitung mit einem eigenen Prädiktionsfilter durchgeführt. Dies führt dazu, dass aufgrund der unterschiedlichen Filterkoeffizienten auch eine unterschiedliche zeitliche Rauschformung in den beiden Stereokanälen stattfindet.A disadvantage of the activation / deactivation of the TNS concept is the fact that for each stereo channel, if Once TNS processing has been activated due to the good expected coding gain, the TNS filtering for each channel takes place separately. So this is still unproblematic with relatively different channels. However, if the left and right channels are relatively similar, then the left and right channels in an extreme example have exactly the same payload, such as a speaker, and differ only in terms of the noise inevitably contained in the channels, so when standing Nevertheless, the technology calculates and uses a separate TNS filter for each channel. Since the TNS filter depends directly on the left or right channel, and in particular reacts relatively sensitively to the spectral data of the left and the right channel, the signal is also very similar in the case of a signal in which the left and the right channel are very similar In the case of a so-called "quasi-mono signal", a TNS processing with a separate prediction filter is carried out for each channel. This leads to a different temporal noise shaping taking place in the two stereo channels due to the different filter coefficients.

Nachteilig an diesem Effekt ist, dass er zu hörbaren Artefakten führen kann, da z. B. das ursprüngliche mono-artige Klangbild durch diese zeitlichen Differenzen einen unerwünschten Stereocharakter bekommt.The disadvantage of this effect is that it can lead to audible artifacts, since z. B. the original mono-like sound image gets an unwanted stereo character through these temporal differences.

Das bekannte Prozedere hat jedoch einen weiten möglicherweise noch schwerwiegenderen Nachteil. Durch die TNS-Verarbeitung werden die TNS-Ausgangswerte, also die spektralen Restwerte einer Mitte/Seite-Codierung im Mitte/Seite-Codierer 1002 von Fig. 3 unterzogen. Während vor der TNS-Verarbeitung die beiden Kanäle noch relativ gleich waren, kann dies nach der TNS-Verarbeitung nicht mehr gesagt werden. Durch den beschriebenen Stereo-Effekt, der durch die getrennte TNS-Verarbeitung eingeführt worden ist, werden die spektralen Restwerte der beiden Kanäle unähnlicher gemacht, als sie eigentlich sein würden. Dies führt zu einem unmittelbaren Abfall an Codiergewinn aufgrund der Mitte/Seite-Codierung, was insbesondere für Anwendungen, bei denen eine niedrige Bitrate erforderlich ist, besonders nachteilhaft ist.However, the known procedure has a far possibly even more serious disadvantage. By the TNS processing, the TNS output values, that is, the spectral residuals, are subjected to center / side coding in the center / side encoder 1002 of FIG. While the two channels were still relatively the same before TNS processing, this can not be said after TNS processing. The described stereo effect, introduced by the separate TNS processing, makes the spectral residuals of the two channels more dissimilar than they would actually be. This leads to a immediate drop in coding gain due to the mid / side coding, which is particularly disadvantageous for applications where a low bit rate is required.

Zusammenfassend ist die bekannte TNS-Aktivierung somit für Stereosignale, die in beiden Kanäle ähnliche aber nicht exakt identische Signalinformationen benutzen, wie beispielsweise mono-ähnliche Sprachsignale, problematisch. Sofern bei der TNS-Detektion für beide Kanäle unterschiedliche Filterkoeffizienten ermittelt werden, führt das zu einer zeitlich unterschiedlichen Formung des Quantisierungsgeräusches in den Kanälen. Dies kann zu hörbaren Artefakten führen, da z. B. das ursprüngliche mono-artige Klangbild durch diese zeitlichen Differenzen einen unerwünschten Stereocharakter bekommt. Weiterhin wird, wie es ausgeführt worden ist, das TNS-modifizierte Spektrum in einem nachfolgenden Schritt einer Mitte/Seite-Codierung unterzogen. Unterschiedliche Filter in beiden Kanälen verringern zusätzlich die Ähnlichkeit der Spektralkoeffizienten und damit den Mitte/Seite-Gewinn.In summary, the known TNS activation is thus problematic for stereo signals that use similar but not exactly identical signal information in both channels, such as mono-like speech signals. If different filter coefficients are determined for both channels in the case of TNS detection, this leads to a temporally different shaping of the quantization noise in the channels. This can lead to audible artifacts because z. B. the original mono-like sound image gets an unwanted stereo character through these temporal differences. Furthermore, as has been stated, the TNS-modified spectrum is subjected to center / side encoding in a subsequent step. Different filters in both channels additionally reduce the similarity of the spectral coefficients and thus the center / side gain.

Die DE 19829284C2 offenbart ein Verfahren und eine Vorrichtung zum Verarbeiten eines zeitlichen Stereosignals und ein Verfahren und eine Vorrichtung zum Decodieren eines unter Verwendung einer Prädiktion über der Frequenz codierten Audiobitstroms. Je nach Implementierung können der linke, der rechte und der Monokanal einer eigenen Prädiktion über der Frequenz, d. h. einer TNS-Verarbeitung unterzogen werden. So kann für jeden Kanal eine eigene vollständige Prädiktion durchgeführt werden. Alternativ kann bei einer unvollständigen Prädiktion eine Berechnung der Prädiktionskoeffizienten für den linken Kanal erfolgen, die dann zur Filterung des rechten Kanals und des Monokanals eingesetzt werden.The DE 19829284C2 discloses a method and apparatus for processing a temporal stereo signal and a method and apparatus for decoding an audio bitstream encoded using prediction over frequency. Depending on the implementation, the left, the right and the mono channel can be subjected to their own prediction over the frequency, ie a TNS processing. Thus, for each channel a separate complete prediction can be performed. Alternatively, in case of incomplete prediction, calculation of the prediction coefficients for the left channel may be performed, which are then used to filter the right channel and the mono channel.

Die Aufgabe der vorliegenden Erfindung besteht darin, ein Konzept zum Verarbeiten eines Multikanalsignals zu schaffen, das geringere Artefakte und dennoch eine gute Komprimierung der Informationen ermöglicht.The object of the present invention is to provide a concept for processing a multi-channel signal, the lower artifacts and still allows a good compression of the information.

Diese Aufgabe wird durch eine Vorrichtung zum Verarbeiten eines Multikanalsignals nach Patentanspruch 1, ein Verfahren zum Verarbeiten eines Multikanalsignals nach Patentanspruch 11 oder ein Computerprogramm nach Patentanspruch 12 gelöst.This object is achieved by a device for processing a multi-channel signal according to claim 1, a method for processing a multi-channel signal according to claim 11 or a computer program according to claim 12.

Der vorliegenden Erfindung liegt die Erkenntnis zugrunde, dass dann, wenn der linke und der rechte Kanal ähnlich sind, also ein Ähnlichkeitsmaß überschreiten, für beide Kanäle die gleiche TNS-Filterung anzuwenden ist. Damit wird sichergestellt, dass durch die TNS-Verarbeitung keine Pseudo-Stereo-Artefakte in das Multikanalsignal eingeführt werden, da durch Verwendung desselben Prädiktionsfilters für beide Kanäle erreicht wird, dass auch die zeitliche Formung des Quantisierungsrauschens für beide Kanäle identisch stattfindet, also dass keine Pseudo-Stereoartefakte zu hören sind.The present invention is based on the finding that if the left and the right channel are similar, that is to say exceed a similarity measure, the same TNS filtering is to be used for both channels. This ensures that the TNS processing no pseudo-stereo artifacts are introduced into the multi-channel signal, as is achieved by using the same prediction filter for both channels that the temporal shaping of the quantization noise for both channels takes place identically, so that no pseudo Stereo artifacts are heard.

Darüber hinaus wird sichergestellt, dass die Signale nicht unähnlicher werden als sie eigentlich sein müssten. Die Ähnlichkeit der Signale nach der TNS-Filterung, also die Ähnlichkeit der spektralen Restwerte entspricht dabei der Ähnlichkeit der Eingangssignale in die Filter und nicht, wie im Stand der Technik, der Ähnlichkeit der Eingangssignale, die noch durch unterschiedliche Filter reduziert wird.It also ensures that the signals do not become more dissimilar than they should be. The similarity of the signals after the TNS filtering, ie the similarity of the residual spectral values corresponds to the similarity of the input signals in the filter and not, as in the prior art, the similarity of the input signals, which is still reduced by different filters.

Damit wird eine nachfolgende Mitte/Seite-Codierung keine Bitratenverluste haben, da die Signale nicht unähnlicher gemacht worden sind, als sie eigentlich sind.Thus, a subsequent center / page encoding will not have bit rate losses because the signals have not been made more dissimilar than they actually are.

Natürlich wird durch Verwenden des selben Prädiktionsfilters für beide Signale ein kleiner Verlust an Prädiktionsgewinn auftreten. Dieser Verlust wird jedoch nicht so groß sein, da die Synchronisierung der TNS-Filterung für beide Kanäle ohnehin nur dann eingesetzt wird, wenn die beiden Kanäle ähnlich zueinander sind. Dieser kleine Verlust an Prädiktionsgewinn wird jedoch, wie sich herausgestellt hat, ohne weiteres durch den Mitte/Seite-Gewinn ausgeglichen, da durch die TNS-Verarbeitung keine zusätzliche Unähnlichkeit zwischen linkem und rechtem Kanal eingeführt wird, welche zu einer Reduktion des Mitte/Seite-Codiergewinns führen würde.Of course, using the same prediction filter will result in a small loss of prediction gain for both signals. However, this loss will not be as great as synchronizing the TNS filtering for both Channels anyway only used when the two channels are similar to each other. This small loss of prediction gain, however, has been found to be easily counterbalanced by the mid / side gain, since TNS processing introduces no additional dissimilarity between the left and right channels leading to a reduction in the center / side gain. Codiergewinns would result.

Bevorzugte Ausführungsbeispiele der vorliegenden Erfindung werden nachfolgend Bezug nehmend auf die beiliegenden Zeichnungen detailliert erläutert. Es zeigen:

Fig. 1
ein Blockschaltbild einer erfindungsgemäßen Vorrichtung zum Verarbeiten eines Multikanalsignals;
Fig. 2
eine bevorzugte Ausführungsform der Einrichtung zum Ermitteln einer Ähnlichkeit und der Einrichtung zum Durchführen der Prädiktionsfilterung; und
Fig. 3
ein Blockschaltbild eines bekannten Audiocodierers gemäß dem AAC-Standard.
Preferred embodiments of the present invention will be explained below in detail with reference to the accompanying drawings. Show it:
Fig. 1
a block diagram of an apparatus according to the invention for processing a multi-channel signal;
Fig. 2
a preferred embodiment of the means for determining a similarity and the means for performing the prediction filtering; and
Fig. 3
a block diagram of a known audio encoder according to the AAC standard.

Fig. 1 zeigt eine Vorrichtung zum Verarbeiten eines Multikanalsignals, wobei das Multikanalsignal durch je einen Block von Spektralwerten für wenigstens zwei Kanäle dargestellt ist, wie es durch L und R gezeigt ist. Die Blöcke von Spektralwerten werden durch z. B. MDCT-Filterung mittels einer MDCT-Filterbank 10 aus Zeitbereichs-Abtastwerten 1(t) bzw. r(t) für jeden Kanal ermittelt.Fig. 1 shows an apparatus for processing a multi-channel signal, wherein the multi-channel signal is represented by one block of spectral values for at least two channels, as shown by L and R. The blocks of spectral values are represented by z. B. MDCT filtering by means of an MDCT filter bank 10 of time domain samples 1 (t) and r (t) determined for each channel.

Die Blöcke von Spektralwerten für jeden Kanal werden dann bei einem bevorzugten Ausführungsbeispiel der vorliegenden Erfindung einer Einrichtung 12 zum Ermitteln einer Ähnlichkeit zwischen den beiden Kanälen zugeführt. Alternativ kann die Einrichtung zum Ermitteln der Ähnlichkeit zwischen den beiden Kanälen auch, wie es in Fig. 1 gezeigt ist, unter Verwendung von Zeitbereichs-Abtastwerten 1(t) oder r(t) für jeden Kanal durchgeführt werden. Es wird jedoch bevorzugt, die aus der Filterbank 10 erhaltenen Blöcke von Spektralwerten zur Ähnlichkeitsbestimmung zu verwenden, da diese gleichermaßen durch mögliche Effekte der Filterung in der Filterbank 10 beeinflusst sind.The blocks of spectral values for each channel are then fed in a preferred embodiment of the present invention to a means 12 for determining a similarity between the two channels. Alternatively, the means for determining the similarity between the two channels may also be as shown in FIG Using time domain samples 1 (t) or r (t) for each channel. However, it is preferred to use the blocks of spectral values obtained from the filter bank 10 for the similarity determination, since these are likewise influenced by possible effects of the filtering in the filter bank 10.

Die Einrichtung 12 zum Ermitteln der Ähnlichkeit zwischen dem ersten und dem zweiten Kanal ist wirksam, um basierend auf einem Ähnlichkeitsmaß oder alternativ einem Unähnlichkeitsmaß ein Steuersignal an einer Steuerleitung 14 zu erzeugen, das wenigstens zwei Zustände hat, von denen einer zum Ausdruck bringt, dass die Blöcke von Spektralwerten der beiden Kanäle ähnlich sind, oder das in seinem anderen Zustand aussagt, dass die Blöcke von Spektralwerten für jeden Kanal unähnlich sind. Die Entscheidung darüber, ob Ähnlichkeit oder Unähnlichkeit vorherrscht, kann unter Verwendung eines vorzugsweise numerischen Ähnlichkeitsmaßes getroffen werden.The means 12 for determining the similarity between the first and the second channel is operative to generate, based on a similarity measure or alternatively a measure of dissimilarity, a control signal on a control line 14 having at least two states, one of which expresses that Blocks of spectral values of the two channels are similar, or that in its other state states that the blocks of spectral values are dissimilar for each channel. The decision as to whether similarity or dissimilarity prevails can be made using a preferably numerical similarity measure.

So existieren verschiedene Möglichkeiten zur Bestimmung der Ähnlichkeit zwischen den zwei Blöcken von Spektralwerten für jeden Kanal, von denen eine Möglichkeit eine Kreuzkorrelationsberechnung ist, die einen Wert ergibt, der dann mit einer vorbestimmten Ähnlichkeitsschwelle verglichen werden kann. Alternative Ähnlichkeitsmessverfahren sind bekannt, wobei eine bevorzugte Form nachfolgend beschrieben wird.Thus, there are various ways of determining the similarity between the two blocks of spectral values for each channel, one possibility being a cross-correlation calculation that yields a value that can then be compared to a predetermined similarity threshold. Alternative similarity measuring methods are known, a preferred form being described below.

Sowohl der Block von Spektralwerten für den linken Kanal als auch der Block von Spektralwerten für den rechten Kanal wird einer Einrichtung 16 zum Durchführen einer Prädiktionsfilterung zugeführt. Insbesondere wird eine Prädiktionsfilterung über der Frequenz durchgeführt, wobei die Einrichtung zum Durchführen ausgebildet ist, um zum Durchführen der Prädiktion über der Frequenz ein gemeinsames Prädiktionsfilter 16a für den Block von Spektralwerten des ersten Kanals und für den Block von Spektralwerten des zweiten Kanals zu verwenden, wenn die Ähnlichkeit größer als eine Schwellenähnlichkeit ist. Wird der Einrichtung 16 zum Durchführen der Prädiktionsfilterung dagegen von der Einrichtung 12 zum Ermitteln einer Ähnlichkeit mitgeteilt, dass die beiden Blöcke von Spektralwerten für jeden Kanal unähnlich sind, also eine Ähnlichkeit haben, die kleiner als eine Schwellenähnlichkeit ist, so wird die Einrichtung 16 zum Durchführen der Prädiktionsfilterung unterschiedliche Filter 16b auf den linken und den rechten Kanal anwenden.Both the block of spectral values for the left channel and the block of spectral values for the right channel are fed to a means 16 for performing a prediction filtering. In particular, predictive filtering is performed over the frequency, the means being adapted to perform, to perform the prediction versus frequency, a common prediction filter 16a for the block of spectral values of the first channel and for the block of spectral values of the first channel second channel if the similarity is greater than a threshold similarity. On the other hand, if the means 16 for performing the prediction filtering is notified by the similarity determining means 12 that the two blocks of spectral values are dissimilar for each channel, that is, similar to less than a threshold similarity, then the means 16 is to perform the prediction filtering apply different filters 16b to the left and right channels.

Die Ausgangssignale der Einrichtung 16 sind somit spektrale Restwerte des linken Kanals an einem Ausgang 18a als auch spektrale Restwerte des rechten Kanals an einem Ausgang 18b, wobei, je nach Ähnlichkeit des linken und des rechten Kanals die spektralen Restwerte der beiden Kanäle unter Verwendung des selben Prädiktionsfilters (Fall 16a) oder unter Verwendung unterschiedlicher Prädiktionsfilter (Fall 16b) erzeugt worden sind.The output signals of device 16 are thus left-channel spectral residuals at output 18a as well as right-channel spectral residuals at output 18b, where, depending on the similarity of the left and right channels, the spectral residuals of the two channels using the same prediction filter (Case 16a) or using different prediction filters (Case 16b).

Je nach tatsächlicher Codiererimplementierung können die spektralen Restwerte des linken und des rechten Kanals entweder direkt oder nach mehreren Verarbeitungen, wie sie z. B. im AAC-Standard vorgesehen sind, einem Mitte/Seite-Stereocodierer zugeführt werden, welcher an einem Ausgang 21a das Mitte-Signal als Hälfte der Summe von linkem und rechtem Kanal ausgibt, während das Seite-Signal als Hälfte der Differenz von linkem und rechtem Kanal ausgegeben wird.Depending on the actual encoder implementation, the spectral residuals of the left and right channels may be either directly or after multiple processing such as described in US Pat. B. AAC standard are supplied to a center / side stereo encoder, which outputs at an output 21 a, the center signal as half of the sum of left and right channel, while the side signal as half of the difference of left and right right channel is output.

Wie es ausgeführt worden ist, ist das Seite-Signal nunmehr, falls vorher eine hohe Ähnlichkeit zwischen den Kanälen bestand, aufgrund der Synchronisierung der TNS-Verarbeitung der beiden Kanäle, kleiner als in dem Fall, bei dem für ähnliche Kanäle unterschiedliche TNS-Filter verwendet werden, was somit, aufgrund der Tatsache, dass das Seite-Signal kleiner ist, einen höheren Codiergewinn in Aussicht stellt.As has been stated, if there was previously a high similarity between the channels, the page signal is now smaller due to the synchronization of the TNS processing of the two channels than in the case where different TNS filters are used for similar channels Thus, due to the fact that the page signal is smaller, this promises a higher coding gain.

Nachfolgend wird Bezug nehmend auf Fig. 2 ein bevorzugtes Ausführungsbeispiel der vorliegenden Erfindung dargestellt, bei dem in der Einrichtung 12 zum Ermitteln einer Ähnlichkeit bereits die erste Stufe der TNS-Berechnung durchgeführt wird, nämlich die Berechnung der Parcor- bzw. Reflexionskoeffizienten und des Prädiktionsgewinns für sowohl den linken Kanal als auch den rechten Kanal, wie es durch die Blöcke 12a, 12b dargestellt ist.Referring now to Fig. 2, there is shown a preferred embodiment of the present invention in which the first stage of the TNS calculation is already performed in the similarity determining means 12, namely the calculation of the parc reflection coefficients and the prediction gain for both the left channel and the right channel, as represented by blocks 12a, 12b.

Diese TNS-Verarbeitung liefert somit sowohl die Filterkoeffizienten für das letztendlich zu verwendende Prädiktionsfilter als auch den Prädiktionsgewinn, wobei dieser Prädiktionsgewinn auch dazu benötigt wird, um zu entscheiden, ob überhaupt eine TNS-Verarbeitung durchgeführt werden soll oder nicht.This TNS processing thus provides both the filter coefficients for the final prediction filter to be used and the prediction gain, and this prediction gain is also needed to decide whether or not TNS processing should be performed at all.

Der Prädiktionsgewinn für den ersten, linken Kanal, der in Fig. 2 mit PG1 bezeichnet ist, wird ebenso wie der Prädiktionsgewinn für den rechten Kanal, der in Fig. 2 mit PG2 bezeichnet ist, in eine Ähnlichkeitsmaßbestimmungseinrichtung eingespeist, die in Fig. 2 mit 12c bezeichnet ist. Diese Ähnlichkeitsbestimmungseinrichtung ist wirksam, um den absoluten Betrag der Differenz oder die relative Differenz der beiden Prädiktionsgewinne zu berechnen und zu schauen, ob dieser unter einer vorbestimmten Abweichungsschwelle S ist. Liegt der absolute Betrag der Differenz der Prädiktionsgewinne unterhalb der Schwelle S, so wird davon ausgegangen, dass die beiden Signale ähnlich sind, und die Frage im Block 12c wird mit Ja beantwortet. Wird dagegen festgestellt, dass die Differenz größer als die Ähnlichkeitsschwelle S ist, so wird die Frage mit Nein beantwortet. Im Falle der Beantwortung dieser Frage mit Ja wird in der Einrichtung 16 ein gemeinsames Filter für beide Kanäle L und R verwendet, während im Falle der Beantwortung der Frage im Block 12c mit Nein getrennte Filter verwendet werden, also eine TNS-Verarbeitung, wie im Stand der Technik durchgeführt werden kann.The prediction gain for the first, left channel, denoted by PG1 in FIG. 2, as well as the prediction gain for the right channel, denoted PG2 in FIG. 2, are fed to a similarity measure determiner, shown in FIG 12c is designated. This similarity determining means is operable to calculate the absolute amount of the difference or the relative difference of the two prediction gains and to see if it is below a predetermined deviation threshold S. If the absolute amount of the difference of the prediction gains is below the threshold S, then it is assumed that the two signals are similar, and the question in the block 12c is answered with Yes. If, on the other hand, it is determined that the difference is greater than the similarity threshold S, the question is answered with no. In the case of answering this question with Yes, in the device 16 a common filter is used for both channels L and R, while in the case of answering the question in block 12c with No separate filters are used, ie a TNS processing, as in the state the technique can be performed.

Hierzu wird der Einrichtung 16 ein Satz von Filterkoeffizienten FKL für den linken Kanal und ein Satz von Filterkoeffizienten FKR für den rechten Kanal von den Einrichtungen 12a bzw. 12b zugeführt.To this end, the device 16 is supplied with a set of filter coefficients FKL for the left channel and a set of filter coefficients FKR for the right channel from the devices 12a and 12b, respectively.

Bei einem bevorzugten Ausführungsbeispiel der vorliegenden Erfindung wird zur Filterung mittels eines gemeinsamen Filters eine besondere Auswahl in einem Block 16c getroffen. Im Block 16c wird entschieden, welcher Kanal die größere Energie hat. Wird festgestellt, dass der linke Kanal die größere Energie hat, so werden die von der Einrichtung 12a für den linken Kanal berechneten Filterkoeffizienten FKL für die gemeinsame Filterung verwendet. Wird dagegen im Block 16c festgestellt, dass der rechte Kanal die größere Energie hat, so wird zur gemeinsamen Filterung der Satz von Filterkoeffizienten FKR, der für den rechten Kanal in der Einrichtung 12b berechnet worden ist, verwendet.In a preferred embodiment of the present invention, for filtering by means of a common filter, a particular selection is made in a block 16c. In block 16c it is decided which channel has the greater energy. If it is determined that the left channel has the greater energy, the filter coefficients FKL calculated by the left channel device 12a are used for the common filtering. On the other hand, if it is determined in block 16c that the right channel has the greater energy, then for common filtering, the set of filter coefficients FKR calculated for the right channel in the device 12b is used.

Wie es aus Fig. 2 hervorgeht, kann zur Energiebestimmung sowohl das Zeitsignal als auch das Spektralsignal verwendet werden. Aufgrund der Tatsache, dass im Spektralsignal bereits möglicherweise stattgefundene Transformations-Artefakte enthalten sind, wird es bevorzugt, zur "Energieentscheidung" im Block 16c die Spektralsignale des linken und rechten Kanals zu verwenden.As is apparent from Fig. 2, both the time signal and the spectral signal can be used for energy determination. Due to the fact that transformation artifacts that may have already taken place in the spectral signal are preferred, it is preferable to use the spectral signals of the left and right channels for the "energy decision" in block 16c.

In einem bevorzugten Ausführungsbeispiel der vorliegenden Erfindung wird eine TNS-Synchronisierung, also die Verwendung der selben Filterkoeffizienten für beide Kanäle eingesetzt, wenn sich die Prädiktionsgewinne für den linken und rechten Kanal um weniger als drei Prozent unterscheiden. Unterscheiden sich beide Kanäle um mehr als drei Prozent, wird die Frage im Block 12c von Fig. 2 mit "Nein" beantwortet.In a preferred embodiment of the present invention, a TNS synchronization, that is, the use of the same filter coefficients for both channels is used when the prediction gains for the left and right channels differ by less than three percent. If both channels differ by more than three percent, the question is answered in block 12c of FIG. 2 with "no".

Wie es bereits ausgeführt worden ist, werden - im Sinne einer einfachen und wenig rechenintensiven Detektion der Ähnlichkeit - die Prädiktionsgewinne der beiden Kanäle bei der Filterung verglichen. Unterschreitet eine Differenz der Prädiktionsgewinne eine bestimmte Schwelle, werden beide Kanäle mit der gleichen TNS-Filterung beaufschlagt, um die beschriebenen Probleme zu vermeiden.As has already been stated, in the sense of a simple and less computation-intensive detection of the similarity, the prediction gains of the two channels in the case of Filtering compared. If a difference of the prediction gains falls below a certain threshold, both channels are subjected to the same TNS filtering in order to avoid the problems described.

Alternativ kann auch ein Vergleich der Reflexionskoeffizienten der beiden separat berechneten TNS-Filter erfolgen.Alternatively, it is also possible to compare the reflection coefficients of the two separately calculated TNS filters.

Wieder alternativ kann die Ähnlichkeitsbestimmung auch unter Verwendung anderer Details des Signals erreicht werden, so dass dann, wenn eine Ähnlichkeit bestimmt worden ist, nur der TNS-Filterkoeffizientensatz für den Kanal berechnet werden muss, der für die Prädiktionsfilterung beider Stereokanäle eingesetzt werden wird. Dies hat zum Vorteil, dass, wenn Fig. 2 betrachtet wird, und wenn die Signale ähnlich sind, nur entweder der Block 12a oder der Block 12b aktiv sein wird.Again alternatively, the similarity determination may also be achieved using other details of the signal, so that when a similarity has been determined, only the TNS filter coefficient set needs to be calculated for the channel that will be used for the prediction filtering of both stereo channels. This has the advantage that, if Fig. 2 is considered, and if the signals are similar, only either block 12a or block 12b will be active.

Darüber hinaus kann das erfindungsgemäße Konzept ferner dahin gehend eingesetzt werden, um die Bitrate des codierten Signals weiter zu reduzieren. Während bei der Verwendung zweier unterschiedlicher Reflexionskoeffizienten unterschiedliche TNS-Seiteninformationen für beide Kanäle übertragen werden, müssen bei der Filterung der beiden Kanäle mit dem selben Prädiktionsfilter nur einmal TNS-Informationen für beide Kanäle übertragen werden. Daher kann durch das erfindungsgemäße Konzept auch eine Reduktion der Bitrate dahin gehend erreicht werden, dass ein Satz von TNS-Seiteninformationen "eingespart" wird, wenn linker und rechter Kanal ähnlich sind.Moreover, the concept according to the invention can also be used to further reduce the bit rate of the coded signal. While different TNS page information is transmitted for both channels when using two different reflection coefficients, when filtering the two channels with the same prediction filter, TNS information must be transmitted only once for both channels. Therefore, the concept of the invention can also achieve a reduction of the bit rate such that a set of TNS page information is "saved" if the left and right channels are similar.

Das erfindungsgemäße Konzept ist nicht grundsätzlich auf Stereo-Signale beschränkt, sondern könnte in einer Mehrkanalumgebung zwischen verschiedenen Kanalpaaren oder auch Gruppen von mehr als 2 Kanälen angewandt werden.The inventive concept is not basically limited to stereo signals, but could be applied in a multi-channel environment between different channel pairs or even groups of more than 2 channels.

Zur Ähnlichkeitsbestimmung kann, wie es ausgeführt worden ist, eine Bestimmung des Kreuzkorrelationsmaßes k zwischen linkem und rechtem Kanal oder eine Bestimmung des TNS-Prädiktionsgewinns und der TNS-Filterkoeffizienten für jeden Kanal separat erfolgen.For similarity determination, as stated, a determination of the left-right channel cross-correlation measure k or a determination of the TNS prediction gain and the TNS filter coefficients may be made separately for each channel.

Die Synchronisationsentscheidung erfolgt, falls k eine Schwelle überschreitet (z.B. 0.6) und MS-Stereo-Coding aktiviert ist. Das MS-Kriterium kann auch weggelassen werden.The synchronization decision is made if k exceeds a threshold (e.g., 0.6) and MS stereo coding is enabled. The MS criterion can also be omitted.

Bei der Synchronisation erfolgt eine Bestimmung des Referenzkanals, dessen TNS-Filter für den anderen Kanal übernommen werden soll. Z. B. wird der Kanal mit der größeren Energie als Referenzkanal verwendet. Insbesondere erfolgt dann ein Kopieren der TNS-Filterkoeffizienten vom Referenzkanal auf den anderen Kanal.During synchronization, a determination is made of the reference channel whose TNS filter is to be adopted for the other channel. For example, the larger energy channel is used as the reference channel. In particular, then copying the TNS filter coefficients from the reference channel to the other channel.

Schließlich erfolgt eine Anwendung der synchronisierten oder nicht-synchronisierten TNS-Filter auf das Spektrum.Finally, the synchronized or unsynchronized TNS filters are applied to the spectrum.

Alternativ erfolgt eine Bestimmung des TNS-Prädiktionsgewinns und der TNS-Filterkoeffizienten für jeden Kanal separat. Dann erfolgt eine Entscheidung. Falls sich der Prädiktionsgewinn beider Kanäle um nicht mehr als ein gewisses Maß unterscheidet, z. B. 3%, erfolgt die Synchronisation. Hier kann der Referenzkanal auch willkürlich gewählt werden, wenn man von einer Ähnlichkeit der Kanäle ausgehen kann. Auch hier gibt es ein Kopieren der TNS-Filterkoeffizienten vom Referenzkanal auf den anderen Kanal, woraufhin eine Anwendung der synchronisierten oder nicht-synchronisierten TNS-Filter auf das Spektrum stattfindet.Alternatively, a determination of the TNS prediction gain and TNS filter coefficients is made separately for each channel. Then a decision is made. If the prediction gain of both channels differs by no more than a degree, e.g. B. 3%, the synchronization takes place. Here, the reference channel can also be chosen arbitrarily, if one can assume a similarity of the channels. Again, there is a copying of the TNS filter coefficients from the reference channel to the other channel, whereupon an application of the synchronized or unsynchronized TNS filters to the spectrum takes place.

Alternative Möglichkeiten sind folgende: Ob TNS in einem Kanal grundsätzlich aktiviert wird, hängt vom Prädiktionsgewinn in diesem Kanal ab. Überschreitet dieser eine gewisse Schwelle, wird TNS für diesen Kanal aktiviert. Alternativ wird auch eine TNS-Synchronisation für 2 Kanäle gemacht, wenn nur in einem der beiden Kanäle TNS aktiviert war. Bedingung ist dann, dass z.B. der Prädiktionsgewinn ähnlich ist, also ein Kanal knapp über der Aktivierungsgrenze, und ein Kanal knapp unter der Aktivierungsgrenze liegt. Aus diesem Vergleich wird dann die Aktivierung von TNS für beide Kanäle mit gleichen Koeffizienten abgeleitet, oder unter Umständen auch die Deaktivierung für beide Kanäle.Alternative options are as follows: Whether TNS in a channel is always activated depends on the prediction gain in this channel. If this exceeds a certain threshold, TNS is activated for this channel. Alternatively, a TNS synchronization is made for 2 channels if TNS was activated in only one of the two channels. Condition is then that e.g. the prediction gain is similar, ie one channel just above the activation limit, and one channel just below the activation limit. From this comparison, the activation of TNS for both channels with equal coefficients is derived, or possibly the deactivation for both channels.

Abhängig von der Gegebenheit, kann das erfindungsgemäße Verfahren zum Verarbeiten eines Mehrkanalsignals in Hardware oder in Software implementiert werden. Die Implementierung kann auf einem digitalen Speichermedium, insbesondere einer Diskette oder CD mit elektronisch auslesbaren Steuersignalen erfolgen, die so mit einem programmierbaren Computersystem zusammenwirken können, dass das Verfahren ausgeführt wird. Allgemein besteht die Erfindung somit auch in einem Computer-Programm-Produkt mit einem auf einem maschinenlesbaren Träger gespeicherten Programmcode zur Durchführung des erfindungsgemäßen Verfahrens, wenn das Computer-Programm-Produkt auf einem Rechner abläuft. In anderen Worten ausgedrückt, kann die Erfindung somit als ein Computer-Programm mit einem Programmcode zur Durchführung des Verfahrens realisiert werden, wenn das Computer-Programm auf einem Computer abläuft.Depending on the circumstances, the inventive method for processing a multi-channel signal can be implemented in hardware or in software. The implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which may interact with a programmable computer system such that the method is performed. In general, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer. In other words, the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Claims (12)

  1. Apparatus for encoding a multi-channel audio signal, wherein the multi-channel signal is represented by a block of spectral values each for at least two channels, comprising:
    a means (12) for determining a similarity between a first one of the two channels and a second one of the two channels, wherein the means (12) for determining is formed to calculate a first prediction gain from a prediction of the block of the first channel and a second prediction gain from a prediction of the block of the second channel, or first reflection coefficients for a first prediction filter for the first channel and second reflection coefficients for a second prediction filter of the second channel, and to obtain (12c) the similarity using the first prediction gain and the second prediction gain or using the first reflection coefficients and the second reflection coefficients;
    a means (16) for performing a prediction filtering, wherein the means for performing is formed to
    use a common prediction filter for the block of spectral values of the first channel and the block of spectral values of the second channel for performing the prediction filtering if a similarity is greater than a threshold similarity, or
    use two different prediction filters for performing the prediction filtering if the similarity is smaller than a threshold similarity.
  2. Apparatus of claim 1, wherein the means (16) for performing is formed to output spectral residual values as a result of the prediction, and
    wherein the apparatus further comprises:
    a means (20) for jointly coding spectral residual values or values of the first channel derived from the spectral residual values, and spectral residual values or values of the second channel derived from the spectral residual values, if the similarity is greater than a threshold similarity.
  3. Apparatus of claim 2, wherein the joint coding is a mid/side coding.
  4. Apparatus of claim 3, wherein the means (20) for jointly coding is formed to calculate a mid signal on the basis of a sum of the first and the second channel, and to calculate a side signal on the basis of a difference of the first and the second channel.
  5. Apparatus of one of the preceding claims, wherein the block of spectral values for a channel represents a short-time spectrum of this channel, or wherein the block of spectral values includes a plurality of bandpass signals for a plurality of subbands.
  6. Apparatus of one of the preceding claims, wherein the means (16) for performing is formed to perform a TNS processing.
  7. Apparatus of one of the preceding claims, wherein the means (12) for determining is formed to calculate a cross correlation of the first and the second channel.
  8. Apparatus of claim 7, wherein the means (16) for performing is formed to use a single prediction filter if the first prediction gain and the second prediction gain differ by less than or equal to three percent.
  9. Apparatus of one of the preceding claims, wherein the means (16) for performing is formed to use, as the common prediction filter, a prediction filter the coefficients of which are derived from the block of spectral values containing more energy than the other block of spectral values.
  10. Apparatus of one of the preceding claims, wherein the means (16) for performing is formed to perform an autocorrelation calculation and an LPC calculation using the Levinson-Durbin algorithm on the block of spectral values for the prediction over the frequency, in order to obtain Parcor coefficients or reflection coefficients as well as a prediction gain, and to filter the block of spectral values with the Parcor coefficients to obtain spectral residual values.
  11. Method of encoding a multi-channel audio signal, wherein the multi-channel signal is represented by a block of spectral values each for at least two channels, comprising the steps of:
    determining (12) a similarity between a first one of the two channels and a second one of the two channels
    by calculating a first prediction gain from a prediction of the block of the first channel and a second prediction gain from a prediction of the block of the second channel, in order to obtain (12c) the similarity from the first prediction gain and the second prediction gain, or
    by calculating first reflection coefficients for a first prediction filter for the first channel and second reflection coefficients for a second prediction filter of the second channel, in order to obtain the similarity using the first reflection coefficients and the second reflection coefficients;
    performing a prediction filtering with a common prediction filter for the block of spectral values of the first channel and the block of spectral values of the second channel if a similarity is greater than a threshold similarity, or
    performing the prediction filtering with two different prediction filters for the block of spectral values of the first channel and the block of spectral values of the second channel if the similarity is smaller than a threshold similarity.
  12. Computer program with program code for performing all steps of the method of encoding a multi-channel audio signal of claim 11, when the program is executed on a computer.
EP05715611A 2004-03-01 2005-02-28 Device and method for processing a multi-channel signal Active EP1697930B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102004009954A DE102004009954B4 (en) 2004-03-01 2004-03-01 Apparatus and method for processing a multi-channel signal
PCT/EP2005/002110 WO2005083678A1 (en) 2004-03-01 2005-02-28 Device and method for processing a multi-channel signal

Publications (2)

Publication Number Publication Date
EP1697930A1 EP1697930A1 (en) 2006-09-06
EP1697930B1 true EP1697930B1 (en) 2007-06-13

Family

ID=34894904

Family Applications (1)

Application Number Title Priority Date Filing Date
EP05715611A Active EP1697930B1 (en) 2004-03-01 2005-02-28 Device and method for processing a multi-channel signal

Country Status (18)

Country Link
US (1) US7340391B2 (en)
EP (1) EP1697930B1 (en)
JP (1) JP4413257B2 (en)
KR (1) KR100823097B1 (en)
CN (1) CN1926608B (en)
AT (1) ATE364882T1 (en)
AU (1) AU2005217517B2 (en)
BR (1) BRPI0507207B1 (en)
CA (1) CA2558161C (en)
DE (2) DE102004009954B4 (en)
DK (1) DK1697930T3 (en)
ES (1) ES2286798T3 (en)
HK (1) HK1095194A1 (en)
IL (1) IL177213A (en)
NO (1) NO339114B1 (en)
PT (1) PT1697930E (en)
RU (1) RU2332727C2 (en)
WO (1) WO2005083678A1 (en)

Families Citing this family (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7725324B2 (en) * 2003-12-19 2010-05-25 Telefonaktiebolaget Lm Ericsson (Publ) Constrained filter encoding of polyphonic signals
US7809579B2 (en) * 2003-12-19 2010-10-05 Telefonaktiebolaget Lm Ericsson (Publ) Fidelity-optimized variable frame length encoding
US9626973B2 (en) * 2005-02-23 2017-04-18 Telefonaktiebolaget L M Ericsson (Publ) Adaptive bit allocation for multi-channel audio encoding
KR100718416B1 (en) 2006-06-28 2007-05-14 주식회사 대우일렉트로닉스 Method for coding stereo audio signal between channels using prediction filter
JP4940888B2 (en) * 2006-10-23 2012-05-30 ソニー株式会社 Audio signal expansion and compression apparatus and method
KR20080053739A (en) * 2006-12-11 2008-06-16 삼성전자주식회사 Apparatus and method for encoding and decoding by applying to adaptive window size
JPWO2008090970A1 (en) * 2007-01-26 2010-05-20 パナソニック株式会社 Stereo encoding apparatus, stereo decoding apparatus, and methods thereof
US7991622B2 (en) * 2007-03-20 2011-08-02 Microsoft Corporation Audio compression and decompression using integer-reversible modulated lapped transforms
US8086465B2 (en) 2007-03-20 2011-12-27 Microsoft Corporation Transform domain transcoding and decoding of audio data using integer-reversible modulated lapped transforms
JP5355387B2 (en) * 2007-03-30 2013-11-27 パナソニック株式会社 Encoding apparatus and encoding method
CN101067931B (en) * 2007-05-10 2011-04-20 芯晟(北京)科技有限公司 Efficient configurable frequency domain parameter stereo-sound and multi-sound channel coding and decoding method and system
CN101981616A (en) * 2008-04-04 2011-02-23 松下电器产业株式会社 Stereo signal converter, stereo signal reverse converter, and methods for both
CN101770776B (en) 2008-12-29 2011-06-08 华为技术有限公司 Coding method and device, decoding method and device for instantaneous signal and processing system
ES2400661T3 (en) * 2009-06-29 2013-04-11 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Encoding and decoding bandwidth extension
EP3779975B1 (en) * 2010-04-13 2023-07-12 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio decoder and related methods for processing multi-channel audio signals using a variable prediction direction
EP2707873B1 (en) * 2011-05-09 2015-04-08 Dolby International AB Method and encoder for processing a digital stereo audio signal
CN104269173B (en) * 2014-09-30 2018-03-13 武汉大学深圳研究院 The audio bandwidth expansion apparatus and method of switch mode
ES2904275T3 (en) * 2015-09-25 2022-04-04 Voiceage Corp Method and system for decoding the left and right channels of a stereo sound signal
CN107659888A (en) * 2017-08-21 2018-02-02 广州酷狗计算机科技有限公司 Identify the method, apparatus and storage medium of pseudostereo audio
EP3483883A1 (en) 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio coding and decoding with selective postfiltering
EP3483886A1 (en) 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Selecting pitch lag
EP3483880A1 (en) * 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Temporal noise shaping
WO2019091573A1 (en) 2017-11-10 2019-05-16 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Apparatus and method for encoding and decoding an audio signal using downsampling or interpolation of scale parameters
EP3483879A1 (en) 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Analysis/synthesis windowing function for modulated lapped transformation
WO2019091576A1 (en) 2017-11-10 2019-05-16 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio encoders, audio decoders, methods and computer programs adapting an encoding and decoding of least significant bits
EP3483878A1 (en) 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audio decoder supporting a set of different loss concealment tools
EP3483884A1 (en) 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Signal filtering
EP3483882A1 (en) 2017-11-10 2019-05-15 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Controlling bandwidth in encoders and/or decoders
CN108962268B (en) * 2018-07-26 2020-11-03 广州酷狗计算机科技有限公司 Method and apparatus for determining monophonic audio
CN112151045B (en) 2019-06-29 2024-06-04 华为技术有限公司 Stereo encoding method, stereo decoding method and device
CN111654745B (en) * 2020-06-08 2022-10-14 海信视像科技股份有限公司 Multi-channel signal processing method and display device
CN112053669B (en) * 2020-08-27 2023-10-27 海信视像科技股份有限公司 Method, device, equipment and medium for eliminating human voice

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5488665A (en) * 1993-11-23 1996-01-30 At&T Corp. Multi-channel perceptual audio compression system with encoding mode switching among matrixed channels
US5812971A (en) * 1996-03-22 1998-09-22 Lucent Technologies Inc. Enhanced joint stereo coding method using temporal envelope shaping
US5913187A (en) * 1997-08-29 1999-06-15 Nortel Networks Corporation Nonlinear filter for noise suppression in linear prediction speech processing devices
DE19747132C2 (en) * 1997-10-24 2002-11-28 Fraunhofer Ges Forschung Methods and devices for encoding audio signals and methods and devices for decoding a bit stream
DE19829284C2 (en) * 1998-05-15 2000-03-16 Fraunhofer Ges Forschung Method and apparatus for processing a temporal stereo signal and method and apparatus for decoding an audio bit stream encoded using prediction over frequency
US6771723B1 (en) * 2000-07-14 2004-08-03 Dennis W. Davis Normalized parametric adaptive matched filter receiver
US6622117B2 (en) * 2001-05-14 2003-09-16 International Business Machines Corporation EM algorithm for convolutive independent component analysis (CICA)
KR100443405B1 (en) * 2001-07-05 2004-08-09 주식회사 이머시스 The equipment redistribution change of multi channel headphone audio signal for multi channel speaker audio signal
GB0124352D0 (en) * 2001-10-11 2001-11-28 1 Ltd Signal processing device for acoustic transducer array
CN1311426C (en) * 2002-04-10 2007-04-18 皇家飞利浦电子股份有限公司 Coding of stereo signals
JP2007009804A (en) * 2005-06-30 2007-01-18 Tohoku Electric Power Co Inc Schedule system for output-power control of wind power-plant
JP2007095002A (en) * 2005-09-30 2007-04-12 Noritsu Koki Co Ltd Photograph processor

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
AU2005217517B2 (en) 2008-06-26
BRPI0507207A8 (en) 2018-06-12
US20070033056A1 (en) 2007-02-08
PT1697930E (en) 2007-09-25
NO20064431L (en) 2006-09-29
DK1697930T3 (en) 2007-10-08
KR100823097B1 (en) 2008-04-18
BRPI0507207A (en) 2007-06-12
ES2286798T3 (en) 2007-12-01
CN1926608A (en) 2007-03-07
DE102004009954B4 (en) 2005-12-15
ATE364882T1 (en) 2007-07-15
BRPI0507207B1 (en) 2018-12-26
NO339114B1 (en) 2016-11-14
DE102004009954A1 (en) 2005-09-29
CN1926608B (en) 2010-05-05
RU2332727C2 (en) 2008-08-27
AU2005217517A1 (en) 2005-09-09
JP2007525718A (en) 2007-09-06
RU2006134641A (en) 2008-04-10
IL177213A (en) 2011-10-31
KR20060121982A (en) 2006-11-29
US7340391B2 (en) 2008-03-04
EP1697930A1 (en) 2006-09-06
HK1095194A1 (en) 2007-04-27
DE502005000864D1 (en) 2007-07-26
CA2558161C (en) 2010-05-11
WO2005083678A1 (en) 2005-09-09
IL177213A0 (en) 2006-12-10
CA2558161A1 (en) 2005-09-09
JP4413257B2 (en) 2010-02-10

Similar Documents

Publication Publication Date Title
EP1697930B1 (en) Device and method for processing a multi-channel signal
EP1687810B1 (en) Device and method for determining a quantiser step size
EP3544003B1 (en) Device and method of determining an estimated value
DE60000412T2 (en) DATA FRAME STRUCTURE FOR ADAPTIVE BLOCK LENGTH CODING
EP1145227B1 (en) Method and device for error concealment in an encoded audio-signal and method and device for decoding an encoded audio signal
DE60313332T2 (en) AUDIOCODING WITH LOW BITRATE
DE69737489T2 (en) Forming the recognizable noise signal in the time domain by means of LPC prediction in the frequency domain
DE60014363T2 (en) REDUCING DATA QUANTIZATION DATA BLOCK DISCOUNTS IN AN AUDIO ENCODER
EP1495464B1 (en) Device and method for encoding a time-discrete audio signal and device and method for decoding coded audio data
EP1502255B1 (en) Device and method for scalable coding and device and method for scalable decoding
DE602004005020T2 (en) AUDIO SIGNAL SYNTHESIS
DE69432012T2 (en) Perceptual coding of audio signals
DE4320990B4 (en) Redundancy reduction procedure
DE69901273T2 (en) Methods for coding and quantizing audio signals
EP1953739B1 (en) Method and device for reducing noise in a decoded signal
EP1397799B1 (en) Method and device for processing time-discrete audio sampled values
WO1999004505A1 (en) Method for signalling a noise substitution during audio signal coding
EP1825461A1 (en) Method and apparatus for artificially expanding the bandwidth of voice signals
DE69932861T2 (en) METHOD FOR CODING AN AUDIO SIGNAL WITH A QUALITY VALUE FOR BIT ASSIGNMENT
WO2001043503A2 (en) Method and device for processing a stereo audio signal
DE10331803A1 (en) Apparatus and method for converting to a transformed representation or for inverse transformation of the transformed representation
EP1277346B1 (en) Device and method for analysing a spectral representation of a decoded time-variable signal
EP0905918A2 (en) Method and apparatus for encoding audio signals
DE10065363B4 (en) Apparatus and method for decoding a coded data signal

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20060721

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWAN

RIN1 Information on inventor provided before grant (corrected)

Inventor name: SCHUG, MICHAEL

Inventor name: GROESCHL, ALEXANDER

Inventor name: HERRE, JUERGEN

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1095194

Country of ref document: HK

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

DAX Request for extension of the european patent (deleted)
AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REF Corresponds to:

Ref document number: 502005000864

Country of ref document: DE

Date of ref document: 20070726

Kind code of ref document: P

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20070705

REG Reference to a national code

Ref country code: SE

Ref legal event code: TRGR

REG Reference to a national code

Ref country code: PT

Ref legal event code: SC4A

Free format text: AVAILABILITY OF NATIONAL TRANSLATION

Effective date: 20070912

ET Fr: translation filed
REG Reference to a national code

Ref country code: DK

Ref legal event code: T3

REG Reference to a national code

Ref country code: HK

Ref legal event code: GR

Ref document number: 1095194

Country of ref document: HK

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2286798

Country of ref document: ES

Kind code of ref document: T3

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070913

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071013

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070914

26N No opposition filed

Effective date: 20080314

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20071214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070613

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 12

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230512

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: LU

Payment date: 20240220

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20240319

Year of fee payment: 20

Ref country code: NL

Payment date: 20240220

Year of fee payment: 20

Ref country code: IE

Payment date: 20240216

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20240216

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: MC

Payment date: 20240219

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FI

Payment date: 20240219

Year of fee payment: 20

Ref country code: DE

Payment date: 20240216

Year of fee payment: 20

Ref country code: CH

Payment date: 20240301

Year of fee payment: 20

Ref country code: PT

Payment date: 20240222

Year of fee payment: 20

Ref country code: GB

Payment date: 20240222

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: SE

Payment date: 20240221

Year of fee payment: 20

Ref country code: IT

Payment date: 20240229

Year of fee payment: 20

Ref country code: FR

Payment date: 20240222

Year of fee payment: 20

Ref country code: DK

Payment date: 20240221

Year of fee payment: 20

Ref country code: BE

Payment date: 20240219

Year of fee payment: 20