TR201900414T4 - Decoding a stereo sound signal using complex prediction. - Google Patents

Abstract

An audio encoder and an audio decoder are disclosed based on combining two audio channels 201, 202 to obtain a residual signal 205 that can be derived using a first combination signal 204 as a middle signal and a predicted side signal derived from the middle signal. . The first combination signal and the prediction residual signal are encoded 206 and written to a data stream 213 with predictive information 206 derived by an optimizer 207 based on an optimization target 208. A decoder uses the prediction residual signal, the first combination signal, and predictive information to derive the decoded first channel signal and the decoded second channel signal. In an encoder sample or a decoder example, a real-virtual transform can be applied to estimate the imaginary part of the spectrum of the first combination signal. While the real valued first combination signal is multiplied by a real part of the complex predictive information to calculate the prediction signal used in the derivation of the prediction residual signal; The predicted imaginary part of the first combination signal is multiplied by an imaginary part of the complex predictive information.

Description

DESCRIPTION DECODING A STEREO AUDIO SIGNAL USING COMPLEX PREDICTION The present invention relates to audio processing, particularly to multi-channel audio processing of a multi-channel signal having two or more channel signals. In the field of multi-channel or stereo processing, the application of a method known as mid/side (M/S) stereo coding is known. In this concept, a combination of the left or first audio channel signal and the right or second audio channel signal is created to obtain a middle or mono signal (M). In addition, a difference is created between the 501 or first channel signal and the right or second channel signal to obtain the side signal (S). Since the side signal will be quite small when the left and right signals are quite similar, this method of center/side coding results in a significant coding gain. In general, the coding gain at the quantizer/entropy coder stage will increase as the range of values to be quantized/entropy-coded becomes smaller. Thus, the coding gain for a PCM (Pulse Code Modulation) or Huffman-based or arithmetic entropy coder will increase as the side signal becomes smaller. However, there are some cases where center/side coding does not result in coding gain. This may happen when the signals in both channels are phase-shifted to each other by, for example, 90°. Then, the middle signal and the side signal may be in a very similar range, so coding the middle signal and the side signal using the entropy-encoder will not result in coding gain, but will instead result in an increased bit rate. Therefore, a frequency-selective middle/side coding can be applied to disable the middle/side coding in the bands, for example, where the side signal is not reduced to some extent relative to the original left signal.

While the side signal is zero when the left and right signals are identical, resulting in maximum coding gain due to the elimination of the side signal, the situation is different when the middle signal and the side signal are identical in waveform, with the only difference between them being their overall amplitudes. In this case, assuming no phase-shift from the side signal to the middle signal, the side signal increases significantly, while the middle signal does not drop significantly in terms of range. When this occurs in a certain frequency band, middle/side coding can be disabled due to the lack of coding gain. Middle/side coding can be applied either frequency-selectively or alternatively in the time domain.

There are alternative multi-channel coding techniques that do not rely on a waveform approach such as center/side coding, but rather on parametric processing based on specific stereophonic cues. These techniques are known as "surround coding". Here, certain cues are calculated for multiple frequency bands. These cues include inter-channel level differences, inter-channel coherence measurements, inter-channel time differences, and/or inter-channel phase differences. These approaches assume that a multi-channel effect perceived by the listener is not necessarily based on the detailed waveforms of the channels, but rather on accurate frequency-selective cues or inter-channel information. This means that care must be taken to render multi-channel signals that accurately reflect the cues in a rendering machine, but that the waveforms are not of decisive importance.

This approach can be particularly complicated when the decoder has to apply a decorrelation process to artificially obtain decorrelated stereo signals from each other, even though all these channels are derived from one or the same downmix channel. Depending on the application, decorrelators for this purpose are complicated and can cause artifacts, especially in the transient signal portions. Furthermore, the parametric coding approach is a lossy coding approach, which, unlike waveform coding, inevitably results in information loss, both from typical quantization and from looking at stereophonic cues instead of specific waveforms. This approach allows very low bit rates, but may require quality compromises.

Recently, improvements have been made in Unified Speech and Audio Coding (USAC), as shown in Figure 7. A core decoder 700 performs decoding of the encoded stereo signal, which may be center/side encoded on input 701. The core decoder outputs a center signal on line 702 and a side or residual signal on line 703. Both signals are converted to a QMF domain with QMF (Quadrature Mirror Filter) filter banks 704 and 705. An MPEG surround decoder 706 is then applied to produce a left channel signal 707 and a right channel signal 708. The low-band signals are then passed to a spectral band replication (SBR) decoder (709), which produces wide-band left and right signals on lines 710 and 711, which are then converted to a time domain by QMF synthesis filter banks (712, 713) to obtain the wide-band left and right signals (L, R).

Figure 7 shows the case where the MPEG surround decoder 706 performs a center/side decoding. Alternatively, the MPEG surround decoder block 706 performs a stereophonic cue-based parametric decoding to produce stereo signals from a single mono core decoder signal.

The MPEG surround decoder 706 may also generate multiple low-band output signals to be input to the SBR decoder block 709 using parametric information such as inter-channel level differences, inter-channel coherence measurements, or other such inter-channel information parameters.

When the MPEG surround decoder block 706 performs the mid/side decoding shown in Figure 7b, a real-gain coefficient (g) can be applied; DMX/RES and L/R are the downmix/residual and left/right signals shown in the complex composite QMF region, respectively.

Using a combination of block 706 and block 709 results in only a very small increase in numerical complexity compared to using a stereo decoder as the underlying, since the complex QMF representation of the signal is already available as part of the SBR decoder. In a non-SBR architecture, QMF-based stereo coding as described in the USAC context can result in a significant increase in numerical complexity due to the required QMF blocks, in this example requiring 64-band analysis blocks and 64-band synthesis blocks. These filter blocks should be added for stereo coding purposes only.

In the MPEG USAC system currently being developed, there are generally coding modes at high bit rates where SBR is not used.

The following two MPEG USAC documents provide examples of multi-channel audio coding/decoding schemes where a signal other than a downmix/sum/mono signal is estimated by means of a complex-valued prediction coefficient: 0 HEIKO PURNHAGEN ET AL.: "Technical description of proposed Unified Stereo Coding in USAC", 90th MPEG MEETING; 26- -23); 0 MAX NEUENDORF (EDITOR): "WD5 of USAC", 90th MPEG MEETING; 12-08), pages 1-146; describes methods and devices for stereo coding and decoding using complex prediction. In one embodiment, a decoding method for obtaining an output stereo signal from an input stereo signal encoded by complex prediction coding and comprising first frequency domain representations of two input channels includes the following steps of upmixing: (i) calculating a second frequency domain representation of a first input channel; and (ii) calculating an output signal based on the first and second frequency domain representations of the first input channel, the first frequency domain representation of the second input channel, and a complex prediction coefficient. The upmixing process may be suspended in response to control data. A parametric stereo upmix equipment that produces a left signal and a right signal from a mono downmix signal based on the first and second frequency domain representations of the first input channel, the first frequency domain representation of the second input channel, and a complex prediction coefficient. The said parametric stereo upmix is characterized by comprising a device for estimating a difference signal containing a difference between the left signal and the right signal based on the mono downmix signal scaled by an estimation coefficient. The said coefficient is derived from spatial parameters. The said stereo upmix equipment also comprises a sum and an arithmetic means for deriving the left signal and the right signal based on a difference of the mono downmix signal and the said difference signal.

One object of the present invention is to provide an improved audio processing concept that provides high coding gain while resulting in good audio quality and/or reduced computational complexity.

This objective is achieved by an MPEG combined speech and audio decoder as in claim 1, an MPEG combined speech and audio decoding method as in claim 1Z, or a computer program as in claim 13.

The present invention is based on the finding that a coding gain in a high quality waveform coding approach can be significantly increased by using a first combination signal for a second combination signal, where both signals are derived from the original channel signals using a combination rule such as the mid/side combination rule. Since the inventive prediction is a waveform-based coding approach rather than a parameter-based dual or multi-channel coding approach, it has been found that this prediction information is calculated by a predictor in the audio encoder in such a way that an optimization goal is achieved, incurring only a small overhead, but resulting in a significant reduction in the bit rate required for the side signal, without any loss in audio quality. In order to reduce the computational complexity, it is preferable to implement a frequency-domain coding in which the predictive information is derived from the frequency-domain input in a band-selective manner. The transformation algorithm for transforming the time-domain representation into a spectral representation is preferably a critically sampled operation such as the modified discrete cosine transform (MDCT) or the modified discrete sine transform (MDST), which differ from the complex transform in that only real values or imaginary values are calculated, while in the complex transform the real and complex values of a spectrum are calculated with 2 times oversampling, resulting in oversampling.

Preferably, a transformation based on overlapping representation and cancellation is used. MDCT in particular is such a transformation and allows cross-fading between subsequent blocks without any additional overhead due to the well-known Time Domain Aliasing Cancellation (TDAC) property achieved by the overlap-addition-processing at the decoder side.

Preferably, the prediction information calculated in the encoder, transmitted to the decoder and used in the decoder advantageously includes an imaginary part reflecting the phase differences between two audio channels chosen at random from quantities between 0° and 360°. The numerical complexity is significantly reduced when a pure real-valued transformation, or in general a transformation that provides only a real spectrum or only an imaginary spectrum, is applied. To utilize this virtual prediction information, which indicates the phase shift between a certain band of the left signal and a corresponding band of the right signal, a real-to-virtual converter or a virtual-to-real converter, depending on the conversion application, is provided in the decoder to calculate a prediction residual signal from the first combination signal that is phase-rotated with respect to the original combination signal. This phase-rotated prediction residual signal can then be combined with the prediction residual signal transmitted in the bitstream to reproduce a side signal, which can then be combined with a middle signal to obtain the decoded left channel in a certain band and the decoded right channel in that band.

To improve the audio quality, the same real-to-virtual or virtual-to-real converter applied on the decoder side is also applied on the encoder side when the predictive signal is calculated in the encoder.

The present invention is advantageous in that it provides improved sound quality and a reduced bit rate when compared to systems having the same bit rate and the same sound quality.

There are also advantages in terms of the computational efficiency of combined stereo coding, which is generally useful in the MPEG USAC system at high bit rates, where SBR is not used. In these approaches, instead of processing the signal in the complex composite QMF domain, residual-based predictive stereo coding is applied in the local MDCT domain of the underlying stereo transform encoder. Examples include a device or method for generating a stereo signal by complex prediction in the MDCT domain, where the complex prediction is performed in the MDCT domain using a real-to-complex transformation, where the stereo signal in question may be an encoded stereo signal at the encoder side, or alternatively a decoded/transmitted stereo signal when the stereo signal generating device or method is implemented at the decoder side.

Below, preferred embodiments of the present invention will be described with reference to the accompanying figures, of which; Figure 1 is a schematic showing an example of an audio decoder; Figure 2 is a block diagram showing an example of an audio encoder; Figure 3a shows an implementation of the encoder calculator of Figure 2; Figure 3b shows an alternative implementation of the encoder calculator of Figure 2; Figure 3c shows a middle/side combination rule to be applied on the encoder side; Figure 4a shows an implementation of the decoder calculator of Figure 1; Figure 4b shows an alternative implementation of the decoder calculator in the form of a matrix calculator; Figure 4c shows a mid/side reverse combination rule corresponding to the combination rule shown in Figure 3c; Figure 5a shows an example of an audio coder operating in the frequency domain, preferably a real-valued frequency domain; Figure 5b shows an implementation of an audio decoder operating in the frequency domain; Figure 6a shows an alternative implementation of an audio coder operating in the MDCT domain and using a real-to-virtual transformation; Figure 6b shows an audio decoder operating in the MDCT domain and using a real-to-virtual transformation according to a preferred embodiment of the invention; Figure 7a shows an audio postprocessor using a stereo decoder and a SBR decoder connected later; Figure 7b shows a mid/side upmix matrix; Figure 8a shows a detailed view of the MDCT block in Figure 6a; Figure 8b shows a detailed view of the MDCT_1 block in Figure 6b; Figure 9a shows an implementation of an optimizer operating at reduced resolution with respect to the MDCT output; Figure 9b shows a representation of the corresponding low resolution bands where an MDCT spectrum and predictive information are calculated; Figure 10a shows an implementation of the real-to-virtual converter in Figure 6a or 6b, and Figure 10b shows a possible implementation of the virtual spectrum calculator in Figure 10a.

Figure 1 shows an audio decoder for decoding an encoded multi-channel audio signal obtained on an input line 100. The encoded multi-channel audio signal includes a coded first combination signal, an coded predictive residual signal, and a predictive information generated using a combination rule for combining a first channel signal representing the multi-channel audio signal and a second channel signal. The coded multi-channel signal may be a data stream, such as a bit stream having three components in multiplexed form. Additional side information may be included in the encoded multi-channel signal on line 100. The signal is input to the input interface 102.

The input interface 102 may be implemented as a data stream demultiplexer that extracts the encoded first combination signal on line 104, the encoded residual signal on line 106, and the prediction information on line 108. The prediction information is preferably a factor having a real part that is not equal to zero and/or an imaginary part that is different from zero. The encoded combination signal and the encoded residual signal are input to a signal decoder 110 to decode the first combination signal to obtain a decoded first combination signal on line 112. Additionally, the signal decoder 110 is configured to decode the encoded residual signal to obtain a decoded residual signal on line 114. Depending on the encoding process on the audio coder side, the signal decoder may include an entropy decoder such as a Huffman decoder, an arithmetic decoder or any other entropy decoder and a subsequent dequantization (recovery of lost detail in a quantized image) step to perform a dequantization operation coupled to a quantizer operation in an associated audio coder. The signals on lines 112 and 114 are input to a decoder calculator 115 which outputs the first channel signal on line 117 and the second channel signal on line 118, where the two signals are two channels or stereo signals of a multi-channel audio signal. For example, if a multichannel audio signal contains five channels, then the two signals in question are the two channels from the multichannel signal. To fully encode such a multichannel signal with five channels, two decoders can be applied as shown in Figure 1, where the first decoder processes the left channel and the right channel, the second decoder processes the left surround channel and the right surround channel, and a third mono decoder can be used to apply mono-encoding to the center channel.

However, other groupings and combinations of waveform coders and parametric coders can also be applied. An alternative way to generalize the prediction scheme to more than two channels is to process three (or more) signals simultaneously, i.e. to predict a 3rd combination signal from signals 1 and 2 using two prediction coefficients, very similar to the "two-to-three" approach in MPEG Environment.

The decoder calculator 116 is configured to calculate a decoded multi-channel signal having a decoded first channel signal 117 and a decoded second channel signal 118 using the decoded residual signal 114, the prediction information 108 and the decoded first combination signal 112. The decoder calculator 116 is specifically configured to process the decoded first channel signal and the decoded second channel signal such that at least an approximation of a first channel signal and a second channel signal of the multi-channel signal is input to a corresponding encoder, combined by the combination rule when the first combination signal and the prediction residual signal are generated. Specifically, the forecast information in line 108 includes a non-zero real-valued part and/or a non-zero imaginary part.

The decoder calculator 116 can be implemented in different ways.

A first embodiment is shown in Figure 4a. The embodiment includes a predictor 1160, a combination signal calculator 1161, and a combiner 1162. The predictor receives the decoded first combination signal 112 and the prediction information 108 and outputs a prediction signal 1163. Again, specifically, the predictor 1160 is configured to apply the prediction information 108 to the decoded first combination signal 112 or to a signal derived from the decoded first combination signal. The derivation rule for deriving the signal to which the prediction information 108 is applied may be a real-to-virtual transformation or, equivalently, a virtual-to-real transformation or a weighting operation, or, depending on the implementation, a phase shift operation or a combined weighting/phase shift operation. The prediction signal 1163 is input into the combination signal calculator 1161 together with the decoded residual signal to calculate the second decoded combination signal 1165. Both signals 112 and 1165 are input to combiner 1162, which combines the decoded first combination signal and the second combination signal to obtain a decoded multi-channel audio signal having the decoded first channel signal and the decoded second channel signal on lines 1166 and 1167, respectively.

Alternatively, the decoder calculator is implemented as a matrix calculator 1168 that takes as input the decoded first combination signal or M signal, the decoded residual signal or D signal, and the prediction information 108. The matrix calculator 1168 applies a transformation matrix 1169 to the M, D signals to obtain the output signals L, R, where L is the decoded first channel signal and R is the decoded second channel signal. The plot in Figure 4b is similar to a stereo plot with one left channel (L) and one right channel (R). This notation is applied for ease of understanding, but it will be understood by those skilled in the art that the L, R signals can be any combination of two channel signals in a multi-channel signal with more than two channel signals. Matrix operations combine into a type of "single-shot" matrix calculation, where the inputs to the circuit in Figure 4a and the outputs from the circuit in Figure 4a are identical to the inputs to the matrix calculator (1168) and the outputs from the matrix calculator (1168).

Figure 4c shows an example of a reverse combination rule applied by the combiner 1162 in Figure 4a. In particular, this combination rule is similar to the well-known decoder-side combination rule in mid/side coding, where L = M + 8 and R = M - S. It should be noted that the combination signal 5 used by the reverse combination rule in Figure 4c is a combination of the signal calculated by the calculator, i.e., the prediction signal on line 1163, and the decoded residual signal on line 114. Again, it should be noted that in this specification, signals on lines are named in some cases using the reference numbers used for the lines, and in some cases using the reference numbers themselves used with respect to the lines. Therefore, the writing is that a line with a particular signal represents the signal itself. A line may be a physical line in a hardware-integrated application. In a computer-hardware application, there is no physical line, instead the signal represented by the line is transmitted from one computing module to another computing module.

Figure 2 shows an audio encoder for encoding a multi-channel audio signal 200 having two or more channel signals, where the first channel signal is shown at 201 and the second channel is shown at 202. When the prediction residual signal 205 is combined with a prediction signal derived from the first combination signal 204 and the prediction information 206 results in a first combination signal 204, the first combination signal and the second combination signal can be derived from the first channel signal 201 and the second channel signal 202 using a combination rule, both signals are input to an encoder calculator 203 for calculating a first combination signal 204 and a prediction residual signal 205 using the first channel signal 201 and the second channel signal 202 and the prediction information 206. The prediction information is generated by an optimizer (207) to calculate the prediction information (206) such that the prediction residual signal satisfies an optimization objective (208).

The first combination signal 204 and the residual signal 205 are input to a signal encoder 209 for encoding the first combination signal 210 and encoding the residual signal 205 to obtain a coded residual signal 211. Both of the coded signals 210, 211 are input to an output interface for combining the coded first combination signal 201 with the coded prediction residual signal 211 and the prediction information 206 to obtain a coded multi-channel signal 213 similar to the coded multi-channel signal 100 input to the input interface 102 of the audio decoder shown in Fig. 1.

Depending on the implementation, the optimizer 207 either receives the first channel signal 201 and the second channel signal 202, or receives the first combination signal 214 and the second combination signal 215 derived from the combiner 2031 (described later) in Figure 3a, indicated by lines 214 and 215.

Figure 2 shows a preferred optimization target, where the coding gain is maximized, i.e. the bit rate is minimized as much as possible. In this optimization target, the residual signal D is minimized with respect to a. In other words, the predictive information a is selected to be minimized to "5 - dMHZIminimum". This results in a solution for d shown in Figure 2. The signals S, M are given blockwise and are preferably spectral domain signals; where H..." is the 2-norm of the variable and <...> represents the normal inner product.

When the first channel signal 201 and the second channel signal 202 are input to the optimizer 207, the optimizer may apply the combination rule (an example combination rule is shown in Figure 3c). However, when the first combination signal 214 and the second combination signal 215 are input to the optimizer 207, the optimizer 207 does not need to apply the combination rule by itself.

Other optimization goals may be related to perception quality. The optimization goal may be to achieve maximum perception quality. In this case, the optimizer may require additional information from a perceptual model. Other applications of the optimization goal may be related to achieving a minimum or constant bit rate. The optimizer 207 may then be applied to perform the quantization/entropy-coding operation to determine the required bit rate for certain values of a, such that a is adjusted to satisfy requirements such as a minimum bit rate or, alternatively, a constant bit rate.

Other applications of the optimization goal may concern the minimum use of encoder or decoder resources. If such an optimization goal is implemented, information regarding the resources required for a particular optimization may be available to the optimizer 207. Additionally, a combination of these or other optimization goals may be implemented to control the optimizer 207 that calculates the predictive information 206.

While the encoder calculator in Figure 2 can be implemented in a variety of ways, a first example implementation where an explicit combination rule is implemented in the combiner 2031 is shown in Figure 3a. An alternative example implementation where a matrix calculator 2039 is used is shown in Figure 3b. The combiner 2031 in Figure 3a can be implemented to implement the combination rule shown in Figure 3c, which is a well-known example of a middle/side coding rule where a weighting factor of 0.5 is applied to all branches. However, depending on the implementation, other weighting factors can be applied, or no weighting factor at all. It should also be noted that other combination rules, such as other linear combination rules or nonlinear combination rules, can also be applied, as long as there is a corresponding inverse combination rule that can be applied in the decoder combiner 1162 shown in Figure 4a, which implements a combination rule that is opposite to the combination rule implemented by the encoder. Any reversible prediction can be used, since the effect on the waveform is "balanced" by the prediction, i.e., any error present in the transmitted residual signal remains, since the prediction process performed by the optimizer 207 in combination with the encoder calculator 203 is a waveform-protecting process.

The combiner 2031 outputs a second combination signal 2032 from the first combination signal 204. The first combination signal is input to a predictor 2033 while the second combination signal 2032 is input to the residual calculator 2034. The predictor 2033 calculates a predictor signal 2035 which is then combined with the second combination signal 2032 to obtain a residual signal 205. The combiner 2031 is specifically configured to combine two channel signals 201 and 202 of the multi-channel audio signal in different ways to obtain the first combination signal 204 and the second combination signal 2032, the two different ways being illustrated in the illustrative example in Figure 3c. The predictor 2033 is configured to apply the prediction information to the first combination signal 204 or a signal derived from the first combination signal to obtain the prediction signal 2035.

The signal derived from the combination signal can be derived by any linear or nonlinear operation, where the real-to-virtual transformation is preferred. This can be implemented by using a linear filter such as a FIR (Finite Impulse Response) filter, which performs weighted addition (/addition) of certain values. A subtraction operation can also be applied to subtract from the combination signal. However, it is now possible to perform other operations in the calculator. In parallel, the combination signal calculator 1161 in Figure 4a may perform a summing operation where the decoded residual signal 114 and the prediction signal 1163 are added to obtain the second combination signal 1165.

Figure 5a shows a preferred embodiment of a voice coder. Compared to the voice coder shown in Figure 3a, the first channel signal 201 is a spectral representation of a time domain first channel signal 55a. Accordingly, the second channel signal 202 is a spectral representation of a time domain channel signal 55b. The conversion from the time domain to the spectral representation is performed by a time/frequency converter 50 for the first channel signal and another time/frequency signal 51 for the second channel signal.

Spectral converters (50, 51) are preferably implemented as real-valued converters, but this is not mandatory.

The transformation algorithm can be a discrete cosine transform, a FFT (Fast Fourier Transform) transform using only one real part, an MDCT transform, or any other transform that provides real-valued spectral values. Alternatively, both transforms can be implemented as a virtual transform, such as DST (Discrete Sine Transform), MDST, FFT, etc., where only one imaginary part is used and the real part is discarded. Any other transform that provides only imaginary values can also be implemented. One purpose of using a pure real-valued transform or a pure imaginary transform is numerical complexity, since for each spectral value a single value, such as magnitude, or real part, must be processed, or alternatively the phase or imaginary part must be processed.

Unlike a fully complex transform such as the FFT, two values, one real part and one imaginary part for each spectral line, may need to be processed, with a numerical complexity increase of at least a factor of 2. Another reason to use a real-valued transform here is that such a transform is critically sampled, thus providing a convenient (and commonly used) region for signal quantization and entropy coding (the standard "perceptual audio coding" paradigm implemented in "MP3", AAC or similar audio coding systems).

Figure 5a also shows a residual calculator (2034) used as a collector that receives the side signal on its "plus" input and the predictor (2033) outputs the predictive signal on its "minus" input. In addition, Figure 5a shows the case where the predictive control information is routed to the multiplexer (212) that outputs a multiplexed bit stream representing the encoded multi-channel audio signal from the optimizer. Specifically, the prediction process is performed in such a way that the side signal is predicted from the center signal, as shown by the Equations on the right of Figure 5a.

Preferably, the predictive control information 206 is a factor as shown on the right of Figure 3b. In an example where the predictive control information includes a real part such as a magnitude of the complex value a or a real part of the complex value a, and this part corresponds to a nonzero factor, a significant coding gain can be achieved when the waveform structures of the center signal and the side signal are similar to each other but have different amplitudes.

In contrast, in the case where the predictive control information only contains a second part, which can be the imaginary part of a complex-valued factor or the phase information of a complex-valued factor, and where the imaginary part or the phase information is non-zero, the present invention provides a significant coding gain for signals that are phase-shifted to each other by a value other than 0° or 180° and have similar waveform characteristics and similar amplitude relationships, as well as phase shift.

A predictive control information is preferably complex-valued. In this way, a significant coding gain can be achieved for signals that differ in amplitude and are subject to phase shift. In a case where time/frequency transforms yield complex spectra, the process 2034 will be a complex process where the real part of the predictive control information is applied to the real part of the complex M spectrum, and the imaginary part of the complex predictive information is applied to the imaginary part of the complex spectrum. Then, in the adder 2034, the result of this predictive operation will be a predicted real spectrum and a predicted imaginary spectrum; to obtain a complex residual D spectrum, the predicted real spectrum is subtracted (bandwise) from the real spectrum of the side S signal, and the predicted virtual spectrum is subtracted from the imaginary part of the S spectrum.

Time domain signals (L and R) are real-valued signals, but frequency domain signals can be real-valued or complex-valued. If frequency domain signals are real-valued, their transformation is also a real-valued transformation. If frequency domain signals are complex, the transformation is also a complex-valued transformation. This shows that the input from time-to-frequency and the output from frequency-to-time transformations are real-valued, but frequency domain signals can be, for example, complex-valued QMF domain signals.

Figure 5b shows an audio decoder, which corresponds to the audio coder shown in Figure 5a. Similar elements have similar reference numbers, as for the audio decoder in Figure 1.

The bitstream output from the bitstream multiplexer 212 in Figure 5b is input to a bitstream demultiplexer 102 in Figure 5b. The bitstream demultiplexer 102 multiplexes the bitstream into a downmix signal M and a residual signal D. The downmix signal M is input to a dequantizer llOa. The residual signal D is input to another dequantizer llOb. In addition, the bitstream demultiplexer 102 multiplexes the predictive control information 108 from the bitstream and inputs it to the predictor 1160. The predictor 1160 outputs a predicted side signal (d - M) and the combiner 1161 combines the predicted side signal and the residual signal output via the dequantizer llOb to obtain the reconstructed side signal S in the final stage.

The signal is then input to the combiner 1162, which performs a sum/difference operation, for example, as shown in Figure 4c, with respect to the mid/side coding. Block 1162 specifically performs a (reverse) mid/side decoding to obtain a frequency domain representation of the left channel and a frequency domain representation of the right channel. The frequency domain representation is then converted to a time domain representation by means of the corresponding frequency/time converters 52 and 53.

Depending on the system implementation, the frequency/time converters (52, 53) are real-valued frequency/time converters if the frequency domain representation is a real-valued representation, or complex-valued frequency/time converters if the frequency domain representation is a complex-valued representation, in accordance with one embodiment of the invention.

However, to increase the efficiency, it is preferable to perform a real-valued transformation, as in another implementation shown in Figure 6a for the encoder and Figure 6b for the decoder. The real-valued transformations (50 and 51) are implemented with an MDCT. Also, the prediction information is calculated as a complex value having a real part and an imaginary part. Since both the M, 8 spectra are real-valued spectra and therefore there is no imaginary part of the spectrum, a real-to-virtual converter 2070 is available that calculates an imaginary spectrum estimated from the real-valued spectrum of the M signal. The virtual spectrum 600 estimated by this real-to-virtual converter is entered into the optimizer stage (2071) together with the real spectrum 14 to calculate the prediction information 206, which now has a real-valued factor specified in 2073 and a virtual factor specified in 2074. In this stage, in accordance with the example in question, the real-valued spectrum of the first M combination signal is multiplied by the real part (dR, 2073) to obtain the prediction signal, which is then subtracted from the real-valued side spectrum.

Also, to obtain the additional prediction signal, the virtual spectrum (600) is multiplied by the virtual part (di) shown at 2074; then this prediction signal is subtracted from the real-valued side spectrum shown at 2034b. Then, the prediction residual signal D is quantized in the quantizer 209b, and the real-valued spectrum M is quantized in the quantized/coded block 209a. Also, it is preferred to quantize and encode the prediction information a in the quantizer/entropy coder 2072 to obtain the coded complex value a, which is routed to the bitstream multiplexer 212 in Fig. Sa and inputted into the bitstream as the prediction information in the final stage. Regarding the location of the quantization/coding (Q/C) module for a, it should be noted that the multipliers 2073 and 2074 preferably use the same (quantized) a that will be used in the decoder.

Therefore, it is possible to proceed directly to the 2071 output, or it can be assumed that the quantization of a is already taken into account in the optimization process in 2071.

Although a complex spectrum can be calculated at the encoder, since all the information is available, it is preferable to perform the real-to-complex transformation at block 2070 in the encoder to produce the conditions for a preferred decoder, as shown in Figure 6b. The decoder receives a real-valued encoded spectrum of the first combination signal and a real-valued spectral representation of the encoded residual signal. In addition, a complex prediction information encoded at 108 is obtained; an entropy-decoding and dequantization are performed to obtain the real part (dR), shown at 1160b, and the imaginary part (di), shown at 1160c.

The intermediate signals extracted by the weighting elements 1160b and 1160c are added to the decoded and dequantized prediction signal. Specifically, the spectral values input to the weighting element 1160c, using the imaginary part of the complex prediction factor as the weighting factor, are derived from the real-valued M spectrum by means of the real-to-virtual converter 1160a, which is preferably implemented as in block 2070 on the encoder side of Figure 6a. On the decoder side, unlike on the encoder side, there is no complex-valued representation of the intermediate signal and the side signal. This is because, due to bit rates and complexity reasons, only encoded real-valued spectra are transmitted from the encoder to the decoder.

The real-to-virtual converter (II60a) in Figure 6a or as published in corresponding patents or US Patent No. 6,980,933 may be implemented. Alternatively, any other implementation known from the relevant technical field may be implemented; a preferred implementation in this regard is described with reference to Figures 10a, 10b.

Specifically, as shown in Figure 10a, the virtual-to-real converter 1160a includes a spectral frame selector 1000 coupled to a virtual spectrum calculator 1001.

The spectral frame selector 1000 receives an indication of the current frame i at input 1002 and, depending on the implementation, control information at a control input 1003. For example, if the indication at line 1002 indicates that a virtual spectrum is to be calculated for the current frame i and the control information 1003 indicates that only the current frame is to be used for that calculation, the spectral frame selector 1000 selects only the current frame i and sends this information to the virtual spectrum calculator. The virtual spectrum calculator then uses only the spectral lines of the current frame i to perform a weighted combination of the lines positioned in the current frame (block 1008) that are close in frequency to or around the current spectral line k for which a virtual line is to be calculated, as shown at 1004 in Figure 10b. However, when the spectral frame selector 1000 receives a control information indicating that the previous frame (i-1) and the next frame (i+1) will also be used to calculate the virtual spectrum, the virtual spectrum calculator additionally takes the values from frames i-1 and i+1 and performs a weighted combination on the corresponding frames for frame i-. The results of the weighting operations are combined with a weighted combination in block 1007 to obtain a virtual k line for frame fi in the final stage; which is then multiplied by the virtual part of the prediction information in item 1160c to obtain the prediction signal for that line; which is then added to the corresponding line of the middle signal in the adder 1161 for the decoder. The same operation is performed in the encoder, but a subtraction operation is performed at element 2034b.

It should be noted that control information 1003 may also specify using more than the two surrounding frames, or, for example, using only the current frame and exactly one or more of the previous frames, without using "ahead" frames in order to reduce systematic delay.

Again, it should be noted that in another order, a stepwise weighted combination can also be performed, as shown in Figure 10b, where the lines from a frame are combined in a first operation and then the results of these frame combination operations are automatically combined.

What is meant here by the other order is that in a first step, the valid k frequency lines from a number of adjacent frames specified by the control information (103) are combined with a weighted combination. Depending on the number of adjacent lines to be used to estimate the virtual line, this weighted combination is applied for k, k-l, k-2, k+l, k+2 etc. Then, the results obtained from these "temporal" combinations are subjected to a weighted combination in the "frequency direction" to obtain the virtual k line for frame fi in the final stage. The weights are preferably set to values between -1 and 1 and a direct FIR or IIR (Infinite Impulse Response) filter can be applied in combination, which performs a linear combination of spectral lines or spectral signals from different frequencies and different frames.

As indicated in Figures 6a and 6b, the preferred transform algorithm is the MDCT transform algorithm, which is applied in the forward direction in items 50 and 51 in Figure 6a, and in the backward direction after the (1162) combination operation in the combiner operating in the spectral domain in items 52, 53.

Figure 8a shows a more detailed implementation of block 50 or 51. In particular, a sequence of time domain audio samples is fed into an analysis windower 500, which performs a windowing operation using an analysis window, and in particular, performs this operation frame by frame within a frame, but using a 50% overlap or overlap. The analysis windower results, i.e., the sequences of frames of the windowed samples, are fed into an MDCT transform block 501, which extracts sequences of real-valued MDCT frames, and where the frames are subjected to the overlapping effect. For example, the analysis windower applies analysis windows of 2048 samples in length. Then, the MDCT transform block (50l) extracts the MDCT spectra with 1024 real spectral lines or MDCT values. Preferably, the analysis can be controlled by means of the windower (a window length or transform length control (502)), which reduces the window length/transform length for transition portions in the signal, for example, to obtain better coding results.

Figure 8 shows the inverse MDCT operation performed in blocks 52 and 53. For example, block 52 contains block 250 for performing a frame-by-frame inverse MDCT transformation. For example, if a frame of MDCT values has 1024 values, the output of this MDCT inverse transform would be 2048 overlapped time samples. Such a frame is sent to a synthesis windower 521, which applies a synthesis window to this frame of 2048 samples. The windowed frame is then routed to an aliasing/summing processor 522 which performs sample by sample summing between the following frames, resulting in 1024 new non-aliasing output signal samples. Again, as noted at 523, it is preferred to implement window/transform length control using information transmitted in the encoded multi-channel signal side information. Prediction values can be computed for each individual spectral line in an MDCT spectrum. However, it has been found that this is not necessary and a significant amount of side information can be recovered by calculating the prediction information bandwise. In other words, a spectral converter 50, which is an MDCT processor as described in the context of Figure 8a, provides a high-frequency resolution spectrum having certain spectral lines as shown in Figure 9b. This high-frequency resolution spectrum is used by a spectral line selector 90, which provides a low-frequency resolution spectrum containing certain bands, B1, B2, B3, ... , BN. This low-frequency resolution spectrum is routed to the optimizer 207 to calculate the predictive information, such that the predictive information is calculated only for each band, not for each spectral line.

For this purpose, the optimizer (207) takes the spectral lines per band and calculates the optimization process based on the assumption that the same d value is used for each spectral line in the band.

Preferably, the bands are psychoacoustically shaped so that the bandwidth of the bands increases from low to high frequencies, as shown in Figure 9b.

Alternatively, although not as desirable as bandwidth expansion, it is also possible to use equally sized frequency bands, where each frequency band has at least two or typically many more, for example at least 30 frequency lines.

In general, for a spectrum with 1024 spectral lines, less than 30 complex a values are calculated, preferably more than 5 a values. For spectra with less than 1024 spectral lines (e.g. 128 lines), fewer frequency bands are preferably used for a (e.g. 6). MDCT spectrum is not mandatory for calculating a values.

Alternatively, a filter bank with a frequency resolution similar to the resolution required to calculate the a values can be used. This filter bank should have a variable bandwidth when increasing frequency bands are to be applied.

However, where a constant bandwidth from low to high frequencies is sufficient, a conventional filter bank with subbands of equal width can be used.

Depending on the implementation, the signature of the value a specified in Figure 3b or 4b can be inverted. However, for consistency reasons, this inversion must be used on both the encoder and decoder sides. Figure Sa shows a general view of the encoder, compared to Figure 6a, where item 2033 is a predictor controlled by the predictor control information 206) specified in item 207 and included in the bit stream as side information. In Figure 5a, a generalized time/frequency transform is used in place of the MDCT used in blocks 50, 51 of Figure 6. As previously mentioned, Figure 6a is an encoder process corresponding to the Decoder process in Figure 6b, where L represents the left channel signal; R represents the right channel signal; M represents the middle signal or downmix signal; S represents the side signal; and D represents the residual signal. Alternatively, L can be referred to as the first channel signal (201); R as the second channel signal (202); M as the first combination signal (204); and 5 as the second combination signal (2032).

Preferably, the modules 2070 in the encoder and ll60a in the decoder should be exactly matched to each other to ensure proper waveform encoding. This is also preferably the case where these modules use some sort of approximation such as cut filters or, in the case where there are only one or two MDCT frames (i.e., the current MDCT frame on line 60) instead of three MDCT frames, the previous MDCT frame on line 61 and the next MDCT frame on line 62.

In addition, while the real-to-virtual (RZI) module ll60a in the decoder has only the quantized MDCT spectrum as input, it is preferable for the module 2070 in the encoder in Figure 6a to use the unquantized MDCT spectrum as input. Alternatively, it is possible to use an implementation in which the encoder uses the quantized MDCT coefficients as an input to module 2070. However, from a perceptual standpoint, using the unquantized MDCT spectrum as an input to module 2070 is a preferable approach.

Some aspects will be explained in more detail later.

Standard parametric stereo coding relies on the ability of the oversampled complex QMF domain to allow perceptually oriented signal processing that varies in time and frequency without introducing aliasing artifacts. However, in the case of downmix/residual coding (as used at the high bit rates considered here), the resulting combined stereo coder functions as a waveform coder. This allows processing in a critically oversampled region, such as the MDCT region, because the waveform coding paradigm ensures that the aliasing removal property of the MDCT-IMDCT (Inverse Modified Discrete Cosine Transform) processing chain is sufficiently well preserved.

However, in order to benefit from the coding efficiency that can be achieved with stereo signals with inter-channel time or phase differences through a complex-valued prediction coefficient a, a complex-valued frequency domain representation of the downmix signal (DMX) is required as an input to the complex-valued upmix matrix. This can be made possible by using an MDST transform in addition to the MDCT transform for the DMX signal. The MDST spectrum can be calculated (exactly or approximately) from the MDCT spectrum.

Also, the parameterization of the upmix matrix can be simplified by passing the complex prediction coefficient a instead of the MPS (MPEG Surround) parameters. Thus, instead of three parameters (ICC (Inter-Channel Coherence), CLD (Channel Level Difference) and IPD (Inter- channel Phase Difference)), only two parameters (real and imaginary part of a) are passed.

This is possible due to the redundancy in the MPS parameterization in the case of downmix/residual coding. The MP8 parameterization includes information on the relative amount of decorrelation to be added to the decoder (i.e. the energy ratio between the RES (Residual) and DMX (Downmix) signals), and this information is more than necessary when the actual DMX and RES signals are transmitted.

For the same reason, the gain coefficient (g) shown in the upmix net above is no longer used in the case of downmix/residual coding. Therefore, the upmix matrix for complex predictive downmix/residual coding is now as follows: R _1+a -1 RES' Compared to Equation ll69 in Figure 4b, in this equation the alpha sign is reversed; DMX=M and RES=DT. Therefore, this implementation is an alternative implementation/writing with respect to Figure 4b.

There are two options for calculating the prediction residual signal in the encoder. One is to use the quantized MDCT spectral values of the downmix. Since the encoder and decoder use the same values to generate the prediction, this option results in the same quantization error distribution as in M/S coding. The other option is to use the unquantized MDCT spectral values. This means that the encoder and decoder do not use the same data to generate the prediction, which allows the coding error to be spatially redistributed according to the instantaneous masking properties of the signal, at the cost of a somewhat reduced coding gain.

It is preferable to directly calculate the MDST spectrum in the frequency domain by means of a two-dimensional FIR filter of the three adjacent MDCT frames described. The latter case can be considered as a "real-to-virtual" (RZI) transformation. The complexity of the frequency domain calculation of MDST can be reduced in several ways, i.e., only an approximation of the MDST spectrum can be calculated: o Limiting the number of FIR filter taps.

- Estimating MDST from the current MDCT frame only. o Estimating MDST from the current and previous MDCT frames.

As long as the same approximation is used in the encoder and decoder, the waveform coding characteristics are not affected. However, these approximations in the MDST spectrum may cause a decrease in the coding gain obtained with complex prediction.

If the underlying MDCT encoder supports window-shaped transitions, the coefficients of the two-dimensional FIR filter used to calculate the MDST spectrum must be adapted to the actual window shapes. The filter coefficients applied to the MDCT spectrum of the current frame are window-dependent, i.e., a set of coefficients is required for each window type and each window transition. The filter coefficients applied to the MDCT spectrum of the previous/next frame depend only on the window that overlaps half of the current frame, i.e. only one set of coefficients is required for each window type (additional coefficients for transitions). If the underlying MDCT encoder uses the transform length transition, a more complicated situation occurs in the transitions between different transform lengths, including the previous and/or next MDCT frame in the approximation. Two-dimensional filtering is more complicated in this case due to the different number of MDCT coefficients in the current and the previous/next frame. In order to eliminate the increased numerical and structural complexity, the previous/next frame is filtered for the transform length transitions, at the cost of decreasing the approximation accuracy for special frames. may not be included in the process.

Also, special attention should be paid to the lowest and highest parts of the MDST spectrum (near DC and fs/2), where the surrounding MDCT coefficient is less than that required for FIR filtering.

Here, the filtering process must be adapted to correctly calculate the MDST spectrum. This can be done by symmetrically stretching/expanding the MDCT spectrum for omitted coefficients (according to the periodicity of the spectra of time-discontinuous signals) or by appropriately adapting the filter coefficients. These special cases can be handled in a simplified way, at the cost of decreasing accuracy near the boundaries of the MDST spectrum.

Computing the full MDST spectrum from the transmitted MDCT spectra at the decoder reduces the decoder delay by one frame (assuming 1024 samples here).

Additional delay can be avoided by using an approximation of the MDST spectrum that does not require the MDCT spectrum of the next frame as input.

The advantages of MDCT-based combined stereo coding over QMF-based combined stereo coding are listed below: . Only a small increase in computational complexity (when SBR is not used). o Scales up to perfect reconstruction when MDCT spectra are not quantized. This is not the case with QMF-based combined stereo coding.

Natural expansion of M/S coding and loudness stereo coding.

A cleaner structure that simplifies encoder matching, as stereo signal processing and quantization/coding can be tightly coupled. It should be noted here that in QMF-based coupled stereo coding, the MPEG Surround frames and MDCT frames are not aligned and the scale factor bands do not match the parameter bands.

Efficient coding of stereo parameters, as only two parameters (complex a) need to be transmitted, instead of three as in MPEG Environment (ICC, CLD, IPD).

If the MDST spectrum is calculated as an approximation (without using the next frame), there is no additional decoder delay.

The important features of an implementation can be summarized as follows: .a) MDST spectra are calculated from the current, previous and next MDCT spectra by means of a two-dimensional FIR filter. Different complexity/quality trade-offs are possible for the MDST calculation (approximation) by reducing the FIR filter taps and/or the number of MDCT frames used. If an adjacent frame is not available due to a frame loss during transmission or transform length transition, that frame is not included in the MDST estimation (exclusion). In transform length transition, an exclusion signal is given in the bit stream. .b) Instead of ICC, CLD and IPD, only two parameters (i.e. the real and imaginary parts of the complex prediction coefficient d) are transmitted. The real and imaginary parts of a are processed independently, limited by the interval. If a particular parameter (real or imaginary part of a) is not used in a given frame, its signal is given in the bitstream and the irrelevant parameter is not transmitted. The parameters are encoded time- or frequency-differentially, and finally Huffman coding is applied using the scale factor code table. The prediction coefficients are updated in the scale factor band every second, resulting in a frequency resolution similar to that of MPEG Environment. This quantization and coding scheme results in an average bit rate of about 2 kb/s for stereo side information in a typical configuration with a target bit rate of 96 kb/s.

Additional or alternative implementation details that are preferred include: c) For each of the two parameters of d, non-differential (PCM) or differential (DPCM (Differential Pulse Code Modulation)) coding can be selected, signaled by a corresponding bit in the bit stream, operating on a per-frame or per-stream principle. For DPCM coding, time or frequency differential coding is possible. This can also be signaled using a one-bit signal. d) Instead of reusing a predefined codebook such as the AAC (Advanced Audio Coding) scale coefficient book, a special invariant or signal-adaptive codebook can be used to encode the d parameter values, or fixed-length (e.g. 4-bit) unsigned or two's complement codewords can be used. e) The range of a parameter values, as well as the parameter quantization step size, can be randomly selected and optimized for the signal characteristics at hand. f) The number and spectral and/or temporal width of the active d parameter bands can be randomly selected and optimized for the specific signal characteristics. In particular, band structuring accords to the per-frame or per-stream principle g) in addition to or as an alternative to the mechanisms described in a) above, or as an alternative, bits per frame in the bit stream can be explicitly signaled that only the MDCT spectrum of the current frame is used to calculate the MDST spectrum approximation, adjacent MDCT frames are not taken into account.

The embodiments relate to the inventive system for combined stereo coding in the MDCT domain. This allows the advantages of combined stereo coding in the MPEG USAC system to be exploited even at higher bit rates (SBR is not used here) without the significant increase in computational complexity introduced by the QMF-based approach.

The following two lists describe the preferred configuration aspects described earlier, which can be used as an alternative to each other or in addition to the other aspects: la) general concept: complex prediction of side MDCT from middle MDCT and MDST; lb) calculate/approximate MDST from MDCT in the frequency domain using 1 or more frames (leading to a 3-frame delay); lc) filter truncation to reduce numerical complexity (l-frame down to 2-tap, i.e. [-1 0 l]); ld) proper processing of DC and fs/2; le) proper processing of window shape transition; lf) use previous/next frame if there is a different transform size; lg) prediction based on unquantized or quantized MDCT coefficients in the encoder; 2a) directly quantize and encode the real and imaginary parts of the complex prediction coefficient (i.e. no MPEG Environment parameterization); 2b) use a uniform quantizer for this (e.g. step 2c) use appropriate frequency resolution for the prediction coefficients (e.g. 1 coefficient for 2 Scale Coefficient Bands); 2d) cheap signaling if all prediction coefficients are real; Ze) clear bit per frame to start the l-frame RZI process.

In one example, the encoder further includes a spectral converter (50, 51) for converting the time domain representation of the two channel signals into a spectral representation of the two channel signals having subband signals for the two channels, wherein the combiner (2031), predictor (2033) and residual signal calculator (2034) are configured to process each subband signal separately to obtain the first combined signal and the residual signal for multiple subbands; wherein the output interface (212) is configured to combine the first combined signal encoded for multiple subbands and the encoded residual signal.

Although some aspects are described in the context of a device, it should be understood that these aspects also cover the description of a related method where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also cover the description of a related block or element or a feature of the related device.

In one embodiment of the present invention, a smooth processing of the window shape transition is implemented. Referring to Figure 10a, a window shape information 109 can be input into the virtual spectrum calculator 1001. Specifically, the virtual spectrum calculator, which performs the real-to-virtual transformation of a real-valued spectrum such as the MDCT spectrum (such as item 2070 in Figure 6a or item 1160a in Figure 6b), can be implemented as a FIR or IIR filter.

The FIR or IIR coefficients in this real-to-virtual module (1001) depend on the window shape for the left and right halves of the current frame. This window shape can be different for a sine window or a Kaiser-Bessel Derived (KAIS) window and can be a long window, a start window, a stop window, a stop-start window or a short window, depending on the configuration of the respective window array. The real-to-virtual module can include a two-dimensional FIR filter, where the first dimension is a time dimension where two consecutive MDCT frames are input into the FIR filter, and the second dimension is a frequency dimension where the frequency coefficients of a frame are input.

The following table gives the different MDST filter coefficients for a valid window array for different window shapes and different implementations of the left half and right half of the window.

Table A - MDST Filter Parameters for Current Window Current Window Sequence ONLY_LONG_SEQUENCE, EIGHT_SHORT_SEQUENCE LONG_START_SEQUENCE LONG_STOP_SEQUENCE STOPISTARTISEQUENCE Left Half: Right Half: Left Half: KBD Right Half: KBD .581427, .581427, .091497] .047969, .608574, .608574, - .O47969, - .150512] 0.608574, 0.608574, 0.150512] 0.635722, 0.635722, Left Half: Sine Left Half: KBD Current Window Sequence Left Half: Sine Left Half: KBD Current Window Sequence ONLY LONG SEQUENCE, EIGHT SHORT SEQUENCE Left Half: Sine Left Half: KBD Current Window Sequence In addition, when the previous window is used to calculate the MDST spectrum from the MDCT spectrum, the window shape information (109) provides the window shape information for the previous window. Corresponding MDST filter coefficients for the previous window Table B - MDST Filter Parameters for the Previous Window Current Valid Current Window Sequence Half: Half Sine: KBD LONG STOP SEQUENCE, STOP START SEQUENCE Finally, depending on the window shape information (109), the virtual spectrum calculator (lOOl) in Fig. lOa can be adapted by applying different groups of filter coefficients.

The window shape information used on the decoder side is calculated on the encoder side and transmitted as side information along with the encoder output signal. On the decoder side, the window shape information (109) is extracted from the bit stream by means of the bit stream demultiplexer (e.g., 102 in Figure 5b) and provided to the virtual spectrum calculator (101) as shown in Figure 10a.

If the window shape information (109) signals that the previous frame has a different transform size, it is preferable not to use the previous frame to calculate the virtual spectrum from the real-valued spectrum. The same applies when the window shape information (109) is interpreted to find that the next frame has a different transform size. In this case, the next frame is not used to calculate the virtual spectrum from the real-valued spectrum. In such a case, for example, when the previous frame has a different transform size than the current frame or when the next frame has a different transform size than the current frame, only the spectral values of the current frame, i.e. the current window, are used to estimate the virtual spectrum.

The prediction in the encoder is based on unquantized or quantized frequency coefficients such as the MDCT coefficients. For example, when the prediction is based on unquantized data, as shown in item 2033 in Figure 3a, the residual calculator 2034 preferentially also operates on the unquantized data, and the residual calculator output signal, the residual signal 205, is quantized before being entropy coded and passed to a decoder. In an alternative example, it is preferable for the prediction to be based on quantized MDCT coefficients. In this case, quantization can occur before the combiner 2031 in Figure 3a, so that a first quantized channel and a second quantized channel are the basis for calculating the residual signal. Alternatively, the quantization process can occur after the combiner 2031, so that the first combination signal and the second combination signal are calculated in a non-quantized form and quantized before the residual signal is calculated. Alternatively, the predictor 2033 can operate in the non-quantized region, and the predictor signal 2035 is quantized before being input to the residual calculator. In this case, it is useful to quantize the second combination signal 2032, another signal input to the residual calculator 2034, before calculating the residual signal 1070 in Figure 6a, which can be implemented in the predictor 2033 in Figure 3a and processed with the same quantized data available at the decoder. In this way, it can be guaranteed that the MDST spectrum estimated in the encoder for the purpose of performing the calculation of the residual signal is exactly the same as the MDST spectrum at the decoder side used to perform the reverse Prediction, i.e., to calculate the side signal from the residual signal. For this purpose, the first combination signal, such as the M signal on line 204 in Figure 6a, is quantized before being input to block 2070. Then, the MDST spectrum calculated using the quantized MDCT spectrum of the current frame, and the quantized MDCT spectrum of the previous or next frame depending on the control information, is input to the multiplier 2074; in this case, the output of the multiplier 2074 in Figure a will also be a non-quantized spectrum. This unquantized spectrum will be output from the spectrum input to the collector (2034b) and finally quantized in the quantizer (209b).

In one example, the real part and the imaginary part of the complex prediction coefficient per prediction band are quantized and coded directly, i.e. without parameterization of the MPEG environment.

The quantization process can be performed using a uniform quantizer with a step size of 0.1, for example. This means that no logarithmic quantization step size and so on are applied, but any linear step size is applied. In one implementation, the range of values of the real part and the imaginary part of the complex prediction coefficient varies from -3 to 3, which means that 60 or 61 quantization steps are used for the real part and the imaginary part of the complex prediction coefficient, depending on the implementation details.

Preferably, the real part applied in the multiplier 2073 in Figure 6a and the imaginary part 2074 applied in Figure 6a are also quantized before being applied, so that the same prediction value is used at the encoder side as at the decoder side. This ensures that the prediction residual signal covers the quantization error introduced, as well as all errors that may occur when a non-quantized prediction coefficient is applied at the encoder side while a quantized prediction coefficient is applied at the decoder side. Preferably, the quantization process is applied in such a way that, as much as possible, the same situation and the same signals are present at both the encoder and the decoder. Therefore, it is preferred to quantize the input to the real-to-virtual calculator 2070 using the same quantization as that applied in quantizer 209a. It is also preferred to quantize the real part and the imaginary part of the prediction coefficient to perform the multiplications in item 2073 and item 2074. This quantization operation is the same as that applied in quantizer 2072. In addition, the side signal extracted by block 2031 in Figure 6a can also be quantized before the adders 2034a and 2034b.

However, it is not problematic to perform the quantization performed by the quantizer (209b) after the addition operation, where the addition performed by the said adders is performed with a non-quantized side signal.

In another example of the description, cheap signaling is applied when all the prediction coefficients are real. There may be a situation where all the prediction coefficients are calculated as real for a certain frame, i.e. for the same time segment of the audio signal. Such a situation may occur when there is no or very little phase-shift from the exact middle signal or the exact side signal to the other. In order to record the bits, this is indicated by a single real indicator. In this case, the imaginary part of the prediction coefficient does not need to be signaled in the bit stream with a code word representing a zero value. On the decoder side, the bitstream decoder interface, i.e. the bitstream demultiplexer, will interpret this new indicator and will not search for code words for the imaginary part, instead it will assume that all bits are in the corresponding part of the bitstream for the real-valued prediction coefficients. In addition, when the predictor receives an indicator indicating that all imaginary parts of the prediction coefficients in the (2033) frame are zero, it will not need to calculate an MDST spectrum from the real-valued MDCT spectrum, or in general a virtual spectrum. Therefore, element 1160a in the decoder in Figure 6b will be deactivated and the reverse prediction will be performed using only the real-valued prediction coefficient applied in the multiplier (1160b) in Figure 6b. The same applies to the encoder side, where element 2070 will be deactivated and prediction will be performed using only the multiplier 2073. This side information is preferably used as an additional bit per frame, in which case the decoder will read this bit frame by frame to decide whether the virtual-to-real converter ll60a will be active for a frame. Providing this information therefore results in a reduction of the bitstream size due to more efficient signaling of all virtual parts of the prediction coefficient being zero for a frame, and in addition, it provides less complexity in the decoder for such a frame, which quickly leads to a reduction in battery consumption of such a processor, for example, when implemented in a battery-powered mobile device. Complex stereo prediction in accordance with preferred embodiments of the present invention is a means for efficiently encoding channel pairs with level and/or phase differences between the channels. Using the complex-valued parameter a, the left and right channels are reconstructed with the following matrix. Here, dmxnm denotes the MDST corresponding to the MDCT of the downmix channels (dmxw). r l-aRE -aIm 1 1 = 1 1 drmcim The above equation is another representation of an equation for a Prediction/combination process, separated and combined in terms of the real part and the imaginary part of a, where the calculation of the predicted signal S is not necessary.

The following data elements are preferably used for this tool: cplx_pred all cplx_pred_used[g][sfb] complex_coef use_prev_frame delta_code_time heod-alpha_g_re hcod-alpha_g;im 0: Some bands use L/R coding, this is signaled by cplx_pred_used[] uses prediction (after matching from prediction bands) a one-bit flag per window group (g) and scale factor band (sfb) indicates O: complex prediction not used, L/R coding used 1: complex prediction used 0: for all prediction bands CXImZO 1: for all prediction bands dMItransmitted O: Use only the current frame for MDST estimation and use the previous frame O: Frequency differential coding of prediction coefficients dm Huffman code dmiHuffman code These data elements are computed in an encoder and embedded in the side information of a multi-channel audio signal. The elements are extracted from the side information by a side information extractor at the decoder side and used to control the decoder calculator to perform a corresponding operation.

Complex stereo prediction requires the downmix MDCT spectrum of the current channel pair; in the case of complex_coef = 1, the downmix MDST spectrum of the current channel pair, i.e., the virtual counterpart of the MDCT spectrum. The downmix MDST estimation is calculated from the MDCT downmix of the current frame; in the case of use_prev_frame == 1, it is calculated from the MDCT downmix of the previous frame. The MDCT downmix of the previous frame belonging to the window group (g) and the group window (b) is obtained from the reconstructed left and right spectra of the frame in question.

In calculating the downmix MDST estimate, the length of the double-valued MDCT transform is used, which depends on the window_sequence as well as filter_coefs and filter_coefs_prev, which are sequences containing the filter kernels and derived according to the previous tables.

For all prediction coefficients (in time or frequency) the previous value is encoded using a Huffman codebook. In the case of cplx_pred_used = 0, prediction coefficients are not transmitted for prediction bands.

The inverse quantized prediction coefficients alpha_re and alpha_im are given as follows; alpha_re = alpha_q_re*0.1 alpha_im = alpha_q_im*0.1 Depending on certain implementation requirements, embodiments of the invention may be implemented in hardware or software. The implementation may be implemented in a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or FLASH memory, which stores electronically readable control signals that operate (or have the ability to operate) with a programmable computer system, to enable the implementation of the method in question.

Some embodiments of the invention include a carrier carrying permanent or tangible data with electronically readable control signals that are capable of interoperating with a programmable computer system to perform one of the methods described herein.

In general, embodiments of the present invention may be implemented as a computer program having a program code that has the ability to perform one of the methods when the computer program product is running on a computer. The program code may be stored on a carrier that is readable by, for example, Inakine.

Other embodiments include a computer program for performing one of the methods described herein, stored on a machine-readable carrier.

In other words, an embodiment of the method of the invention is a computer program having a program code for performing one of the methods described herein when the computer program is running on a computer.

In other words, an embodiment of the method of the invention is a computer program having a program code for performing one of the methods described herein when the computer program is running on a computer.

Thus, another embodiment of the method of the invention is a data stream or set of signals representing a computer program for performing one of the methods described herein. The data stream or set of signals may be configured to be transmitted over a data communication link, such as the Internet.

Another embodiment includes a processing means configured or adapted to perform one of the methods described herein, such as a computer or a programmable logic device.

Another embodiment includes a computer into which a computer program is installed to perform one of the methods described herein.

In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to implement some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate array may be operated in conjunction with a microprocessor to implement one of the methods described herein. The methods may generally be implemented with any hardware device, preferably any hardware device.

The embodiments described above are for the purpose of illustrating the principles of the present invention only. It will be obvious to those skilled in the art that various modifications and variations can be made to the embodiments and details described herein.

Therefore, the scope of the present invention is determined by the appended patent claims, not by the specific details set forth in the description and explanations of the regulations herein.

Claims (1)

CLAIMS 1. An MPEG combined speech and audio decoder of an encrypted stereo audio signal (100), the encrypted stereo audio signal comprising an encrypted downmix signal generated based on a combination rule for combining a first channel audio signal and a second channel audio signal of a stereo audio signal, having: a dequantizer (110; 110a, 110b) for dequantizing the encrypted downmix signal (104) to obtain a decoded downmix signal (112, M) and dequantizing the encrypted residual signal (106) to obtain a decoded residual signal (114, D); and a decoding calculator (116) for calculating a decoded stereo signal including a decoded first channel signal (117, L) and a decoded second channel signal (118, R) using the decoded residual signal (114, D), the prediction information (108) and the decoded downmix signal (112, M), such that the decoded first channel signal (117) and the decoded second channel signal (118) are at least approximations of the stereo signal, the first channel signal, and the second channel signal, wherein the prediction information (108) comprises a real part (dR) and an imaginary part (dI) of a complex prediction coefficient, wherein the decoding calculator (116) includes an estimator (1160) configured to estimate (1160a) an imaginary part (601) of the decoded downmix signal (112) of a current frame, the decoded downmix signal for the current frame being a fraction of the current frame. is the MDCT spectrum, where the estimator 1160 includes a truly imaginary transformer to calculate an MDST spectrum of the current frame as the imaginary part, where the truly imaginary transformer is applied in a frequency domain and converts an MDCT spectrum of the decoded downmix signal of a preceding frame into the decoded downmix signal of the current frame. The estimator (1160) is configured to apply two-dimensional finite impulse response filtering to the MDCT spectrum and the MDCT spectrum of the decoded downmix signal of the next frame, or to calculate an approximation of an MDST spectrum using the MDCT spectrum of the current frame and the MDCT spectrum of the previous frame, or to calculate an approximation of an MDST spectrum using only the MDCT spectrum of the current frame, wherein the estimator (1160) is configured to multiply the imaginary part of the decoded downmix signal (601) by the negative imaginary part of the estimation coefficient (dI) to obtain a second part of the estimation signal, and the real part of the decoded downmix signal by the negative real part of the estimation coefficient (dR) to obtain a first part of the estimation signal; wherein the solver calculator (116) further comprises a combination signal calculator (1161a, 1161b) configured to add a first part of the prediction signal and the decoded residual signal and a second part of the prediction signal to obtain a side signal (1165); wherein the solver calculator (116) further comprises a combiner (1162) for combining the side signal (1165) and the decoded downmix signal, a middle signal, to obtain a first channel signal (117, L), which is a left channel signal, and a decoded second channel signal (118, R), which is a right channel signal; wherein the solver further comprises an inverse modified discrete cosine transform element (52) for generating a time-domain first channel signal from the left channel signal and an inverse modified discrete cosine transform element (53) for generating a time-domain second channel signal from the right channel signal. 2. The MPEG combined speech and audio decoder according to claim 1, wherein the estimator is configured to specifically remove a neighboring frame from the calculation of the MDCT spectrum of the current frame, particularly when the neighboring frame is not available due to a frame during communication or transform-length change, wherein the removal for the transform-length change case is signaled in a bitstream. The MPEG combined speech and audio decoder according to claim 1, wherein when either the real part (dR) or the imaginary part (dI) is not used in a given frame, this case is signaled in a bitstream and the unused part is not carried. The MPEG combined speech and audio decoder of claim 1, wherein no differential coding or no differential coding is used on a per frame basis of each stream for each of the real part (dR) and the imaginary part (dI), which are signaled in a bitstream, and for the differential coding, either time or frequency differential coding is used and is signaled by a bit flag in the bitstream. The MPEG combined speech and audio decoder of claim 1, wherein the prediction information is encoded using a predefined codebook, or a separate immutable codebook, or a signal adaptive codebook, or a fixed length unsigned coding, or using two's complement code words. The MPEG combined speech and audio decoder of claim 1, wherein the real part (dR) and the imaginary part (dI) are updated in both scale factor bands. The MPEG combined speech and audio decoder of claim 1, wherein a bitstream includes one bit for each frame where only the MDCT spectrum of the current frame is used to calculate the MDST spectrum of the current frame, and the MDCT spectra of neighboring frames are not taken into account. The MPEG combined speech and audio decoder of claim 1, wherein the filter applied by the virtual phantom converter is a 1-frame 2-end cut filter. The MPEG combined speech and audio decoder of claim 1, wherein the virtual phantom converter includes two-dimensional FIR filtering of the MDCT spectra of the decoded downmix signal. The MPEG combined speech and audio decoder according to claim 9, wherein if an underlying MDCT encoder uses window shape variation, the filter coefficients of the FIR filter are adapted to the actual window shapes implemented by the MDCT encoder. The MPEG combined speech and audio decoder according to claim 9 or 10, wherein if the underlying MDCT encoder uses turn length variation resulting in different numbers of MDCT coefficients present in the current frame and the previous or next frame, the previous or next frame is omitted from the computation of the MDST spectrum for the current frame. The method of MPEG combined speech and audio decoding of an encrypted stereo audio signal (100), the encrypted stereo audio signal comprising an encrypted downmix signal generated based on a combination rule for combining a first channel audio signal and a second channel audio signal of a stereo audio signal, having: dequantizing the encrypted downmix signal (104) to obtain a decoded downmix signal (112), and dequantizing the encrypted residual signal (106) to obtain a decoded residual signal (114); and calculating (116) a decoded stereo signal comprising a decoded first channel signal (117) and a decoded second channel signal (118) using the decoded residual signal (114), the estimation information (108) and the decoded downmix signal (112), such that the decoded first channel signal (117) and the decoded second channel signal (118) are at least approximations of the stereo signal, the first channel signal and the second channel signal, wherein the estimation information (108) comprises a real part (dR) and an imaginary part (gl) of a complex estimation coefficient, wherein an imaginary part of the decoded downmix signal (112) is estimated (1160a) by an estimator (1160) using the decoded downmix signal (112), the decoded downmix signal for the current frame being an MDCT spectrum of the current frame, wherein the estimator (1160) estimates an MDST spectrum of the current frame as the imaginary part. to calculate the truly imaginary transform, wherein the truly imaginary transform is applied in a frequency domain and performs a two-dimensional finite impulse response filtering of an MDCT spectrum of the decoded downmix signal of a preceding frame, the MDCT spectrum of the decoded downmix signal of the current frame and the MDCT spectrum of the decoded downmix signal of the next frame, or calculating an approximation of an MDST spectrum using the MDCT spectrum of the current frame and the MDCT spectrum of the previous frame, or calculating an approximation of an MDST spectrum using only the MDCT spectrum of the current frame, wherein the imaginary part (601) of the decoded downmix signal is multiplied by the negative imaginary part of the prediction information (108) to obtain a second part of the prediction signal, and wherein the real part of the decoded downmix signal is multiplied by the negative real part (dR) of the prediction information (108) to obtain a first part of the prediction signal; wherein the first part of the prediction signal and the decoded residual signal are added and the second part of the prediction signal is additionally added to obtain the side signal (1165); wherein the side signal (1165) and the decoded downmix signal are combined to obtain the decoded first channel signal (117, L), which is a left channel signal, and the decoded second channel signal (118, R), which is a right channel signal, wherein the method further comprises applying an inverse modified discrete cosine transform to the left channel signal to produce a time domain first channel signal and applying another inverse modified discrete cosine transform to the right channel signal to produce a time domain second channel signal. 13. A computer program adapted to carry out the method of claim 10-12 when executed on a computer or processor.