RU2799737C2 - Audio upmixing device with the possibility of operating in the mode with/without prediction - Google Patents

Abstract

FIELD: acoustics.

SUBSTANCE: devices for decoding audio signals. An apparatus for outputting a stereo audio signal having a left channel and a right channel, the apparatus comprising: a demultiplexer capable of receiving an audio bitstream and extracting a prediction coefficient therefrom; a decoder configured to generate a downmix signal and a residual signal from the encoded audio bitstream; and an upmixer configured to operate in either a predictive mode or a non-predictive mode based on a parameter encoded in the audio bitstream and output a left channel and a right channel as a stereo audio signal, wherein when the upmixer operates in predictive mode, the residual signal represents the difference between the side signal and the predicted version of the side signal, and the upmixer generates a left channel and a right channel from the combination of the downmix signal, the residual signal, and the prediction coefficients for each of the plurality of frequency bands, and when the upmixer operates in non-prediction mode, the residual signal is a side signal, the upmixer generates a left channel based on the sum of the downmix signal and the residual signal passed through the decoder, and the upmixer generates a right channel based on the difference between the downmix signal and the residual signal passed through the decoder.

EFFECT: increased computational efficiency, achievement of a range of high data bit rates.

16 cl, 16 dwg

Description

Scope of technical application

The invention disclosed herein generally relates to stereo coding, and more specifically to stereo coding techniques using complex frequency domain prediction.

Background of the invention

Joint coding of the left (L) and right (R) channels of a stereo signal enables more efficient coding compared to independent L and R coding. A common approach for joint stereo coding is middle/side (M/S) coding. Here, the average (M) signal is formed by adding the L and R signals, for example, the M signal can be of the form

M = (L+R)/2.

Also, by subtracting the two channels L and R, a side signal (S) is generated, for example, the S signal may be of the form

S = (L–R)/2.

In the case of M/S coding, instead of the L and R signals, the M and S signals are encoded.

The upmixer is configured to add the residual signal to the side signal when operating in the predictive mode.

In MPEG (Moving Picture Expert Group) AAC (Advanced Audio Coding) (see ISO/IEC standard document 13818-7), L/R stereo coding and M/S stereo coding may be selected in a time-varying or frequency-varying manner. Thus, a stereo encoder may apply L/R coding for some frequency bands of a stereo signal, while M/S coding (variation depending on frequency) is used to encode other frequency bands of a stereo signal. In addition, the encoder may switch between L/R and M/S encoding over time (change over time). In the MPEG AAC standard, stereo coding is performed in the frequency domain, more specifically in the MDCT (Modified Discrete Cosine Transform) domain. This allows either L/R or M/S coding to be adaptively selected in a frequency-varying as well as time-varying manner.

Parametric stereo coding is a way to efficiently encode a stereo audio signal as a mono signal plus a small amount of side information for the stereo parameters. It forms part of the MPEG-4 Audio standard (see ISO/IEC 14496-3 document). A mono signal can be encoded using any audio encoder. Stereo parameters can be embedded in the ancillary portion of a mono bitstream, and thus full forward and backward compatibility is achieved. In the decoder, the mono signal is first decoded, after which the stereo signal is reconstructed using the stereo parameters. A decorrelated version of the decoded mono signal that has zero cross-correlation with the mono signal is generated by a decorrelator, such as an appropriate phase filter, which may include one or more delay lines. Essentially, a decorrelated signal has the same spectral and temporal energy distribution as a monophonic signal. The mono signal together with the decorrelated signal are input to an up-mix process which is controlled by the stereo parameters and which reconstructs the stereo signal. For more information, see "Low Complexity Parametric Stereo Coding in MPEG-4", H. Purnhagen, Proc. of the ^7th Int. Conference on Digital Audio Effects (DAFx'04), Naples, Italy, October 5-8, 2004, pages 163-168.

MPEG Surround (MPS; see ISO/IEC 23003-1 and "MPEG Surround - The ISO/MPEG Standard for Efficient and Compatible Multi-Channel Audio Coding", J. Herre et al., Audio Engineering Convention Paper 7084, 122 ^nd Convention, May 5-8, 2007) allows the principles of parametric stereo coding to be combined with residual coding that replaces the decorrelated signal with a transmitted residual com and thus improving the perceived quality. Residual coding may be performed by downmixing the multi-channel signal and optionally by extracting spatial marks. During the downmix process, the residual signals representing the error signal are calculated and then encoded and transmitted. In the decoder, they can replace decorrelated signals. In a hybrid approach, they can replace decorrelated signals in some frequency bands, preferably in relatively low frequency bands.

In accordance with the current MPEG Unified Speech and Audio Coding (USAC) system, two examples of which are shown in FIG. 1, the decoder includes a complex-valued quadrature mirror filter (QMF) downstream from the base decoder. The QMF output of the filterbank is complex—and thus doubled—and can be organized as a downmix signal (or, equivalently, an average signal) M and a residual signal D, to which a complex upmix matrix is applied. The L and R signals (in the QMF region) are obtained as:

where g is a real-valued gain and α is a complex-valued prediction coefficient. Preferably, α is chosen to minimize the residual signal energy D. The gain may be determined by normalization, i.e. so as to ensure that the power of the total signal will be equal to the sum of the powers of the left and right signals. The real and imaginary parts of each of the L and R signals are mutually redundant—in principle, each can be computed from the other—but they are useful to enable subsequent use of a Spectral Band Replication (SBR) decoder without incurring audible artifacts due to aliasing. For similar reasons, the use of an oversampled signal representation may also be chosen to prevent artifacts associated with other time or frequency adaptive signal processing (not shown), such as, for example, upmixing a mono signal to a stereo signal. The last processing step in the decoder is inverse QMF filtering. It is noted that the QMF representation of the bandlimited signal allows for bandlimited difference and "leftover" methods that can be integrated into decoders of this type.

The above coding structure is well suited for low bit rates, typically below 80 kbps, but is not optimal for higher bit rates in terms of computational complexity. More specifically, at higher bit rates, the SBR tool is generally not used (because it will not improve coding efficiency). Therefore, in a decoder without an SBR stage, only the presence of a complex-valued upmix matrix justifies the presence of a QMF filter bank, which is computationally intensive and introduces a delay (with a frame length of 1024 samples, the QMF analysis/synthesis filter bank introduces a delay of 961 samples). This clearly indicates the need for a more efficient coding structure.

Brief description of the invention

The aim of the present invention is to provide methods and apparatus for stereo coding that are computationally efficient also in the high bit rate range.

The invention fulfills this purpose by providing an encoder and a decoder, encoding and decoding methods and, accordingly, computer software products for encoding and decoding as defined by the independent claims. The dependent claims define embodiments of the invention.

In a first aspect, the invention provides a decoder system for generating a stereo signal by complex prediction stereo coding, and the decoder system includes:
- an upmix adapted to generate a stereo signal based on the first representations of the downmix signal (M) of the downmix and the residual signal (D) in the frequency domain, where each of the first representations in the frequency domain includes the first spectral components representing the spectral composition of the corresponding signal expressed in the first subspace of the multidimensional space, and the stage of the upmix includes:
- - a module for calculating a second representation of the downmix signal in the frequency domain based on its first representation in the frequency domain, where the second representation in the frequency domain includes the second spectral components representing the spectral composition of the signal expressed in the second subspace of the multidimensional space, which includes a part of the multidimensional space not included in the first subspace;
a weighted adder for calculating a side signal (S) based on the first and second representations of the downmix signal in the frequency domain, the first representation of the residual signal in the frequency domain, and the complex prediction coefficient (α) encoded in the bitstream signal; And
a sum-difference stage for calculating the stereo signal based on the first representation of the downmix signal in the frequency domain and the side signal,
where the upmix stage can also operate in a relay mode in which said downmix signal and the residual signal are applied directly to the sum-difference stage.

In a second aspect, the invention provides an encoder system for encoding a stereo signal with a bitstream signal by complex prediction stereo coding, which includes:
- an estimator designed to estimate the coefficient of complex prediction;
- coding stage valid for:
(a) converting the stereo signal into frequency domain representations of the downmix signal and the residual signal in a relationship determined by a complex prediction coefficient value;
And
a multiplexer for receiving the output data of the coding stage and the estimator and for encoding them by said bitstream signal.

In the third and fourth aspects of the invention, methods are provided for encoding a stereo signal into a bitstream and decoding the bitstream into at least one stereo signal. The technical features of each of the methods are similar to those for the encoder system and the decoder system, respectively. In the fifth and sixth aspects, the invention also provides a computer program product containing instructions for executing each of the methods on a computer.

The invention takes advantage of the benefits of unified stereo coding in the MPEG USAC system. These advantages are maintained at higher bit rates, when SBR is not typically used, without the significant increase in computational complexity that would accompany the QMF approach. This is possible because the critically sampled MDCT, which is the basis of the transform coding in the MPEG USAC system, according to the invention can be applied to stereo complex prediction coding, at least in cases where the bandwidths of the encoded audio signals of the downmix channel and the residual channel are the same, and the upmixing process does not include decorrelation. This means that an additional QMF transformation is no longer required. An exemplary implementation of complex prediction stereo coding in the QMF domain would actually significantly increase the number of operations per unit time compared to traditional L/R or M/S stereo. Therefore, the encoder according to the invention is competitive at these bit rates, providing high sound quality at moderate computational cost.

As those skilled in the art will appreciate, the fact that the upmix stage can also operate in the relay mode allows the decoder to adaptively perform decoding in accordance with traditional direct or joint coding and complex prediction coding as determined at the encoder side. Therefore, in cases where a decoder cannot positively improve the quality level above that of conventional direct L/R stereo coding, or joint M/S stereo coding, it can at least guarantee that the same level will be maintained. Thus, a decoder according to this feature of the invention can, from a functional point of view, be considered extended with respect to the premises.

As an advantage over a stereo signal encoded with QMF prediction, a perfect reconstruction of the signal is possible (apart from quantization errors, which can be made as small as desired).

Thus, the invention provides an encoder for stereo encoding a signal based on complex prediction transformation. Preferably, the device according to the invention is not limited to complex prediction stereo coding, but can also operate in direct L/R stereo coding or joint M/S stereo coding according to prerequisites, so that the most suitable coding method for a particular application or during a particular period of time can be selected.

As a basis for complex prediction according to the invention, an oversampled (e.g., complex) representation of a signal is used, including said first and second spectral components, and therefore, modules for calculating said oversampled representation are located in the encoder system and in the decoder system according to the invention. The spectral components refer to the first and second subspaces of a multidimensional space, which may be a set of time-dependent functions over an interval of a given length (eg, a predetermined time frame length) sampled at a finite sampling rate. It is well known that functions in such a special multidimensional space can be approximated by a finite weighted sum of basis functions.

As will be understood by those skilled in the art, an encoder adapted to work with a decoder is equipped with equivalent modules to generate a resampled representation on which predictive coding is based so as to enable faithful reproduction of the encoded signal. Said equivalent modules may be identical or similar modules having identical or similar transmission characteristics. In particular, the encoder and decoder modules, respectively, may be similar or dissimilar processing units executing respective computer programs that perform equivalent sets of mathematical operations.

In some embodiments of a decoder system or an encoder system, the first spectral components have real values expressed in a first subspace and the second spectral components have imaginary values expressed in a second subspace. The first and second spectral components together form a complex spectral representation of the signal. The first subspace may be the linear span of the first set of basis functions, while the second subspace may be the span of the set of second basis functions, some of which are linearly independent of the first set of basis functions.

In one of the embodiments of the invention, the module for calculating the complex representation is a real-to-imaginary transformation, i.e. a module for calculating the imaginary parts of the spectrum of a discrete signal based on the actual spectral representation of this signal. The transformation can be based on exact or approximate mathematical dependencies, such as formulas from harmonic analysis or heuristic dependencies.

In some embodiments of the decoder system or encoder system, the first spectral components may be obtained by transforming a discrete time domain signal from the time domain to the frequency domain, preferably via a Fourier transform such as a discrete cosine transform (DCT), modified discrete cosine transform (MDCT), discrete sine transform (DST), modified discrete sine transform (MDST), fast Fourier transform (FFT), prime factor Fourier algorithm, or the like. In the first four cases, the second spectral components can then be obtained by DST, MDST, DCT, and MDCT, respectively. As is well known, the linear span of cosines that are periodic on the unit interval forms a subspace that does not completely fit in the linear span of sines that are periodic on the same interval. Preferably, the first spectral components can be obtained by MDCT, and the second spectral components can be obtained by MDST.

In one of the embodiments of the invention, the decoder system includes at least one temporal noise limiting module (TNS module, or TNS filter), which is located in the upstream direction relative to the upmix stage. Generally speaking, the use of TNS improves the perceived audio quality for transient type signals, and this also applies to embodiments of the decoder system of the invention having TNS as a feature. In conventional L/R or M/S stereo coding, the TNS filter may be applied as the last step in the frequency domain processing just before inverse transform. In the case of complex prediction stereo coding, however, it is often more advantageous to apply a TNS filter on the downmix signal and the residual signal, i. before the upmix matrix. In other words, TNS is applied to linear combinations of the left and right channels, which has some advantages. First, it may turn out that in this situation TNS is only useful for, say, a downmix signal. Then, for the residual signal, the TNS filtering may be suppressed or skipped, and the TNS filter coefficients need only be transmitted for the downmix signal, which may mean more economical use of the available bandwidth. Secondly, computing an oversampled representation of the downmix signal (e.g., MDST data derived from the MDCT data so that a complex frequency domain representation can be generated) that is needed for complex prediction coding may require that the time domain representation of the downmix signal can be calculated. In turn, this means that the downmix signal should preferably be available as a time series of MDCT spectra obtained in a uniform manner. If the TNS filter was applied in the decoder after the upmix matrix that converts the downmix/residual representation to left/right representation, only the sequence of downmix signal MDCT spectra remaining after TNS will be available. This can make it very difficult to efficiently calculate the corresponding MDST spectra, especially if the left and right channels used TNS filters with different characteristics.

It should be emphasized that the availability of the time sequence of the MDCT spectra is not an absolute criterion for obtaining an MDST representation suitable to serve as the basis for complex prediction coding. In addition to experimental evidence, this fact can be explained by the fact that TNS is generally applicable only to higher frequencies, such as frequencies above a few kilohertz, so the residual signal filtered by TNS approximately corresponds to the unfiltered residual signal for lower frequencies. Thus, the invention can be implemented as a decoder for complex prediction stereo coding, in which, as indicated below, the TNS filters have other placement than upstream of the upmix stage.

In one of the embodiments of the invention, the decoder system includes at least one additional TNS module located in the downstream direction relative to the upmix stage. The position of the selector selects either the TNS module(s) in the upstream direction of the upmix stage, or the TNS module(s) in the downstream direction of the upmix stage. In some circumstances, the calculation of the complex representation in the frequency domain does not require that the representation of the downmix signal in the time domain can be calculated. In addition, as discussed below, the decoder may selectively operate in the direct or joint coding mode without applying complex prediction coding, and then it may be more appropriate to use the traditional location of the TNS units, i.e. as one of the last steps in frequency domain processing.

In one embodiment of the invention, the decoder system is adapted to save processing resources and possibly energy by disabling the module for calculating the second representation of the downmix signal in the frequency domain when the latter is not needed. It is assumed that the downmix signal is divided into consecutive time blocks, each of which is associated with a value of the complex prediction coefficient. This value may be determined by a decision made for each time block by the encoder in cooperation with the decoder. In addition, in this embodiment, the module for calculating the second representation of the downmix signal in the frequency domain is adapted to disable itself if, for a given time block, the absolute value of the imaginary part of the complex prediction coefficient is equal to zero or does not exceed a predetermined tolerance. Disabling the module may imply that for a given time block, a second representation of the downmix signal in the frequency domain is not calculated. If no mute occurs, the second frequency domain representation (eg, the set of MDST coefficients) may be multiplied by zero, or by a number having substantially the same order of magnitude as the machine epsilon (rounding unit) of the decoder, or some other suitable threshold value.

In a further development of the previous embodiment of the invention, savings in processing resources are achieved at the time block sublayer into which the downmix signal is divided. For example, such a sublayer within a time block may be a frequency band, where the decoder determines the value of the complex prediction coefficient for each frequency band within the time block. Similarly, a module for generating a second frequency domain representation is adapted to suppress its effect for a frequency band within a time block where the complex prediction coefficient is zero or has an absolute value less than a tolerance.

In one embodiment, the first spectral components are transform coefficients located in one or more transform coefficient time blocks, where each block is generated by applying a transform to a time slice of the signal in the time domain. In addition, a module for calculating the second representation of the downmix signal in the frequency domain is adapted to

• obtaining one or more first intermediate components from at least some of the first spectral components;

• forming a combination of the specified one or more first spectral components in accordance with at least part of one or more impulse responses to obtain one or more second intermediate components; And

• obtaining said one or more second spectral components from said one or more second intermediate components.
This procedure performs the calculation of the second frequency domain representation directly from the first frequency domain representation, as described in more detail in US Pat.

For an example implementation of complex prediction stereo coding according to the invention, it has been estimated that the computational complexity increases only slightly (significantly less than the increase caused by complex prediction stereo coding in the QMF domain) compared to conventional L/R or M/S stereo. One embodiment of this type of invention, involving precise calculation of the second spectral components, introduces a delay that is typically only a few percent longer than the delay introduced by a QMF-based implementation (assuming a time block length of 1024 samples and comparing it to a hybrid QMF analysis/synthesis filter bank delay of 961 samples).

Accordingly, in at least some of the previous embodiments of the invention, the impulse responses are adapted to the transform by which the first representation in the frequency domain can be obtained, and more precisely adapted according to its frequency responses.

In some embodiments of the invention, the first representation of the downmix signal in the frequency domain is obtained by a transform that is applied in connection with one or more analysis window functions (or cutoff functions, e.g., rectangular window, sine window, Kaiser-Bessel window, etc.), one of the goals of which is temporal segmentation without introducing a detrimental amount of noise or undesirable spectrum modification. It is possible that these window functions are partially overlapping. In such a case, preferably, the frequency characteristics of the transform depend on the characteristics of said one or more analysis window functions.

By continuing to address embodiments of the invention including, as a feature, computing a second frequency domain representation within the frequency domain, the computational burden can be reduced by using an approximate second frequency domain representation. This approximation can be performed by waiving the requirement for complete information on which the calculation is based. According to the ideas of US patent No. 6980933 B2, for example, to accurately calculate the second representation of the downmix signal in the frequency domain in one block, first of all, data in the frequency domain is required from three time blocks, namely: a block simultaneous with an output block, a previous block, and a subsequent block. For the purposes of complex prediction coding according to the present invention, useful approximations can be obtained by skipping—or nulling out—data originating from a succeeding block (whereby the action of the module can become causal, i.e., it will not introduce a delay) and/or from a preceding block, such that the calculation of the second frequency domain representation is based on data from only one or two time blocks. It should be noted that even though omission of input data may imply a rescaling of the second representation in the frequency domain - in the sense that, for example, it will no longer represent equal power - it can still be used as the basis for complex prediction coding since, as mentioned above, it is computed at the encoder and decoder ends in an equivalent manner. In reality, a possible change in scale of this type will be compensated by a corresponding change in the value of the prediction coefficient.

Yet another approximate method for calculating the portion of the second representation of the downmix signal in the frequency domain, forming the spectral components, may include combining at least two components from the first representation in the frequency domain. The last components may be contiguous in time and/or frequency. Alternatively, they can be combined through Finite Impulse Response (FIR) filtering with a relatively small number of links. For example, in a system using a time block size of 1024, these filters may include 2, 3, 4, and so on. links. Descriptions of methods of approximate calculation of this kind can be found, for example, in US patent application No. 2005/0197831 A1. If a window function is used, such as, for example, a non-rectangular function that gives relatively small weights to the surroundings of each of the time block boundaries, it may be appropriate to base the second spectral components in the time block only on combinations of the first spectral components in the same time block, implying that not the same amount of information is available for the edge components. The approximation error possibly introduced by such a practical implementation is to some extent suppressed, or hidden, by the form of the window function.

In one embodiment of a decoder that is designed to output a stereo signal in the time domain, it includes the ability to switch between direct, or joint, stereo coding and complex prediction coding. This is achieved by supplying:

• a switch that can selectively act either as a relay stage (not modifying signals) or as a sum-difference conversion;

• an inverse conversion stage designed to perform the frequency-to-time conversion; And

• a selector circuit designed to supply the inverse transform stage with either a directly (or jointly) encoded signal or a signal encoded with complex prediction.

As will be understood by those skilled in the art, this flexibility of the decoder part allows the encoder the freedom to choose between traditional direct, or joint, coding and complex prediction coding. Therefore, in cases where the quality level of conventional L/R stereo coding, or joint M/S stereo coding, cannot be increased, this embodiment can at least guarantee that the same level will be maintained. Thus, the decoder according to the invention can be considered as an extension of the current state of the art.

Another group of embodiments of the invention of the decoder system performs the calculation of the second spectral components in the second representation in the frequency domain through the time domain. More precisely, a transformation is applied that is the reverse of that by which the first spectral components are (or can be) obtained, followed by another transformation that produces the second spectral components at the output. In particular, a reverse MDCT may be followed by an MDST. In order to reduce the number of transforms and inverse transforms, the output of the inverse MDCT in such an embodiment of the invention may be applied to both the MDST and the data output terminals of the decoding system (which may be preceded by additional processing steps).

For an example implementation of complex prediction coding according to the invention, it has been estimated that the computational complexity increases only slightly compared to conventional L/R or M/S stereo coding (even less than the increase caused by complex prediction coding in the QMF domain).

As a further development of the embodiment of the invention relating to the previous paragraph, the upmix stage may include an additional inverse conversion stage for side signal processing. In this case, the sum-difference stage is provided with a time domain side signal representation which is generated by said additional inverse transform stage and a time domain downmix signal representation which is generated by the already mentioned inverse transform stage. It should be reiterated that it is advantageous from the point of view of computational complexity to apply the last signal to both the sum-difference stage and to the other conversion stage mentioned above.

In one embodiment of the invention, a decoder designed to output a stereo signal in the time domain includes the ability to switch between L/R stereo coding, or joint M/S stereo coding, and complex prediction coding. This is achieved by equipping:

• a switch capable of acting either as a relay stage or as a sum-difference stage;

• an additional inverse transform stage designed to calculate the representation of the side signal in the time domain;

• a selector circuit for associating the inverse transform stages or with the sum-difference stage associated with a point upstream of the upmix stage and downstream of the switch (preferably when the switch is actuated to perform a band pass filter function, as may be the case for decoding a stereo signal generated by complex prediction coding), or for combining the downmix signal from the switch to side signal from the weighted adder (preferably when the switch is operated to perform the function of the sum-difference stage, as may be the case when decoding a directly encoded stereo signal).

As will be appreciated by those skilled in the art, this gives the encoder the freedom to choose between traditional direct or joint coding and complex prediction coding, which implies the possibility of guaranteeing that the quality level will be at least equivalent to that of direct or joint stereo coding.

In one embodiment of the invention, the encoder system according to the second aspect of the invention may include an estimator for estimating the complex prediction coefficient in order to reduce or minimize the signal power, or average signal power, for the residual signal. The minimization may take place over a period of time, preferably a time slice or time block or time frame of the signal being encoded. The square of the amplitude can be chosen as a measure of the instantaneous signal power, and the integral of the square of the amplitude (waveform) over a time interval can be chosen as a measure of the average signal power over that interval. Accordingly, the complex prediction coefficient is determined based on the time block and the frequency band, i. e. its value is set such that it reduces the average power (ie, total energy) of the residual signal in that time block and frequency band. In particular, the output signal, on which, in accordance with mathematical relationships known to those skilled in the art, the complex prediction coefficient can be calculated, such modules for estimating parameters of parametric stereo coding as IID, ICC and IPD or similar can be created.

In one embodiment of the invention, the coding stage in the encoder system may also operate to function as a relay stage so as to enable direct stereo coding. By choosing direct stereo encoding in situations where higher quality is expected, the encoder system can ensure that the encoded stereo signal is at least as good as direct encoding. Similarly, in situations where the greater computational complexity introduced by complex prediction coding is not motivated by a significant improvement in quality, the encoder system thus has a readily available opportunity to save computational resources. The decision to choose between joint coding, direct coding, real prediction coding and complex prediction coding in the encoder is mainly based on speed/distortion optimization considerations.

In one embodiment of the invention, the encoder system may include a module for calculating a second frequency domain representation directly (i.e., without applying inverse time domain transformation and without using time domain signal data) based on the first spectral components. With respect to the corresponding embodiments of the decoder system described above, this module may have a similar design, namely: include similar processing operations, but in a different order, so that the encoder is adapted to output data suitable as input on the decoder side. For the purpose of illustrating this embodiment of the invention, it is assumed that the stereo signal being encoded includes, or has been converted to, the middle and side channels, and the encoding stage is adapted to receive a first representation in the frequency domain. The coding stage includes a module for computing a second frequency domain representation of the average signal. (The first and second frequency domain representations referred to herein are the same as those defined above; in particular, the first frequency domain representations may be MDCT representations, and the second frequency domain representation may be an MDST representation.) The encoding stage also includes a weighted adder for computing the residual signal as a linear combination formed from the side signal and the two frequency domain average signal representations, weighted respectively by the real and imaginary parts of the coefficient of the complex predictions. The average signal, or a suitable first frequency domain representation thereof, may be used directly as a downmix signal. In addition, in this embodiment, the evaluator determines the value of the complex prediction coefficient in order to minimize the power, or average power, of the residual signal. The last operation (optimization) can be performed either by feedback control, where the evaluator can receive a residual signal obtained through the current values of the prediction coefficient and intended for further adjustment if necessary, or, in a direct connection method, by calculations performed directly on the left/right channels of the original stereo signal or on the middle/side channels. A feed-forward method is preferred in which the complex prediction coefficient is determined directly (in particular, not iteratively and without feedback) based on the first and second representations of the average signal in the frequency domain and the first representation of the side signal in the frequency domain. It should be noted that the determination of the complex prediction coefficient may be followed by a decision on which coding to apply: direct, joint, real or complex prediction coding, which takes into account the resulting quality (preferably the perceived quality, taking into account, for example, signal/mask) for each of the available possibilities; therefore, the above statements should not be interpreted as statements that there is no feedback mechanism in the encoder.

In one embodiment of the invention, the encoder system includes modules for computing a second representation of an average (or downmix) signal in the frequency domain across the time domain. It should be understood that the implementation details relating to this embodiment of the invention, at least insofar as the calculation of the second frequency domain representation is considered, are similar or can be handled similarly to the respective decoder embodiments. In this embodiment, the encoding stage includes:

• sum-difference stage designed to convert a stereo signal into a form that includes middle and side channels;

• a conversion stage for generating a frequency domain representation of the side channel and a complex-valued (and therefore resampled) frequency domain representation of the middle channel; And

• a weighted adder designed to calculate the residual signal, where the complex prediction coefficient is used as a weighting factor.

Here, the evaluator may receive the residual signal and determine, possibly in a feedback control manner, the complex prediction coefficient so as to reduce or minimize the power, or average power, of the residual signal. Preferably, however, the evaluator receives the stereo signal to be encoded and determines a prediction coefficient based on it. From the point of view of computational savings, it is advantageous to use a critically sampled frequency domain side channel representation, since the latter in this embodiment of the invention will not be subjected to multiplication by a complex number. Accordingly, the transform stage may include an MDCT stage and a parallel MDST stage, where both stages receive as input a representation of the middle channel in the time domain. Thus, an oversampled frequency domain representation of the middle channel and a critically sampled frequency domain representation of the side channel are generated.

It should be noted that the methods and apparatus disclosed in this section, after appropriate modifications within the capabilities of specialists, including typical experiments, can be used to encode signals containing more than two channels. Modifications intended to provide suitability for said multi-channel operation may follow, for example, along the path of the directions described in sections 4 and 5 of the article by J. Herre et al. cited above.

Features from two or more embodiments of the invention described above may be combined in further embodiments of the invention unless they are explicitly additional. The fact that two features are set forth in different claims does not prevent them from being advantageously combined. Likewise, further embodiments of the invention may also be provided omitting certain features that are not necessary or essential for the intended purpose. For example, the decoding system according to the invention can be implemented without a dequantizing stage in cases where the encoded signal to be processed is not quantized or is already available in a form suitable for processing by the upmixing stage.

Brief description of graphic materials

The invention will be further illustrated below by means of the embodiments of the invention described in the following section, with reference to the accompanying drawings, in which:

figure 1 consists of two generalized block diagrams showing decoders based on QMF according to the current state of the art;

Figure 2 is a generalized block diagram of a complex prediction MDCT stereo decoder system according to one embodiment of the present invention, where the complex channel representation of the signal to be decoded is computed in the frequency domain;

Figure 3 is a generalized block diagram of a complex prediction stereo MDCT based decoder system according to one embodiment of the present invention, where a complex channel representation of a signal to be decoded is computed in the time domain;

figure 4 shows an alternative implementation of the decoder system of figure 2, where the position of the active stage of the TNS can be selected;

Figure 5 includes generalized block diagrams showing complex prediction MDCT-based stereo encoder systems in accordance with embodiments of another feature of the present invention;

Figure 6 is a generalized block diagram of a complex prediction MDCT stereo encoder system according to one embodiment of the present invention, where a complex channel representation of a signal to be encoded is computed based on its representation in the time domain;

figure 7 shows an alternative embodiment of the encoder system of figure 6, which can also operate in direct L/R encoding mode;

Figure 8 is a generalized block diagram of a complex prediction MDCT-based stereo encoder system according to one embodiment of the present invention, where the complex channel representation of the signal to be decoded is computed based on its first representation in the frequency domain, the decoder system of which can also operate in the direct L/R encoding mode;

Figure 9 shows an alternative embodiment of the encoder system of Figure 7 which also includes a TNS stage located downstream of the encoding stage;

Figure 10 shows alternative embodiments of the part labeled A in Figures 2-8;

figure 11 shows an alternative embodiment of the encoder system of figure 8, which also includes two frequency domain modification devices located respectively in the downstream and upstream directions of the coding stage;

Figure 12 is a graphical representation of 96 kbps listening test results for six objects, showing various possibilities for trade-offs between complexity and quality in computing, or approximating, the MDST spectrum, where data points marked with "+" refer to the hidden reference, "x" refers to a 3.5 kHz bandwidth anchor, "*" refers to conventional USAC stereo coding (M/S or L/R), "□" refers to un unified stereo coding in the MDCT domain by complex prediction with the imaginary coefficient blocked (i.e., real prediction not requiring MDST), "■" refers to unified stereo coding in the MDCT domain by complex prediction using the current MDCT frame to calculate the MDST approximation, "_°"refers to unified stereo coding in the MDCT domain by complex prediction using the current and previous MDCT frames to calculate the MDST approximation, and "●" refers to unified stereo coding in the MDCT domain by complex prediction using the current, previous and next MDCT frames to calculate the MDST;

Figure 13 represents the data of Figure 12, but as a differential estimate with respect to unified stereo coding in the MDCT region by complex prediction using the current MDCT frame to calculate the MDST approximation;

Figure 14 includes generalized block diagrams showing three embodiments of a decoder system in accordance with embodiments of the invention;

Figure 15 is a flowchart showing a decoding method according to one embodiment of the invention; And

Figure 16 is a flowchart showing an encoding method according to one embodiment of the invention.

Detailed description of embodiments of the invention

I. Decoder systems

Figure 2, in the form of a generalized block diagram, shows a decoding system for decoding a bit stream including at least one complex prediction coefficient value and an MDCT representation of a stereo signal containing the downmix channel M and the residual channel D. Real and imaginary parts the prediction coefficients may be quantized and/or co-coded. It is preferred, however, that the real and imaginary parts be quantized independently and uniformly, typically with a step size of 0.1 (a dimensionless number). According to the MPEG standard, the resolution of the spectral band used for the complex prediction coefficient is not necessarily equal to the resolution for the scalefactor bands (sfb; i.e., the MDCT line group that uses the same quantization step size and quantization range). In particular, the bandwidth resolution for the prediction coefficient may be psychoacoustically sound, such as the Bark scale. The demultiplexer 201 is adapted to extract the indicated MDCT representations and the prediction coefficient (the part of the Control Information shown in the figure) from the bit stream that is fed into it. In fact, more control information than only complex prediction coefficient information, such as instructions on whether to decode the bitstream in predictive or non-predictive mode, TNS information, and so on, may be encoded in the bitstream. The TNS information may include TNS parameter values to be applied by the (synthesis) TNS filters of the decoder system. If multiple TNS filters are expected to use the same TNS parameter sets, it is more economical to receive this information in the form of a bit indicating such parameter set identity rather than receiving the two parameter sets independently. Information on whether to apply TNS before or after the upmix stage may also be included, as appropriate based on, for example, a psychoacoustic evaluation of the two possibilities. In addition, the control information may then indicate the individually limited bandwidths for the downmix signal and the residual signal. For each channel, the frequency bands above the passband boundary will not be decoded, but will be set to zero. In some cases, the highest frequency bands have so little energy that they are already quantized to zero. In standard practice (cf. the max_sfb parameter in the MPEG standard), it is common to use the same bandwidth limit for both the downmix signal and the residual signal. However, the residual signal, more than the downmix signal, has its energy stored localized in the lower frequency bands. Therefore, by placing a dedicated upper bandwidth limit on the residual signal, it is possible to lower the data bit rate without significantly reducing quality. For example, this can be controlled by two independent bitstream encoded max_sfb parameters, one for the downmix signal and one for the residual signal.

In this embodiment, the MDCT representation of a stereo signal is segmented into consecutive time frames (time blocks) including a fixed number of data points (for example, 1024 points), one of several fixed numbers of data points (for example, 128 or 1024 points), or a variable number of points. As is known to those skilled in the art, MDCT is critically sampled. The output signal of the decoding system, indicated on the right side of the illustration, is a stereo signal in the time domain, containing the left L and right R channels. Dequantizers 202 are adapted to manipulate the bitstream included in the decoding system or, where necessary, the two bitstreams obtained after demultiplexing the original bitstream and corresponding to each of the downmix and residual channels. The dequantized channel signals are applied to the switching node 203, operating either in the relay mode or in the sum-difference mode, which are related to the respective transformation matrices.

And

As will be further explained in the following paragraph, the decoder system includes a second switching node 205 . Both switching nodes 203, 205, like most other switches and switching nodes in this embodiment of the invention and in the embodiments of the invention to be described below, operate in a frequency selective manner. This makes it possible to decode a wide variety of decoding modes, such as frequency-dependent L/R or M/S decoding, as is known in the art. Therefore, the decoder according to the invention can be considered as an extension of the current state of the art.

Assuming so far that the switch node 203 is in relay mode, in this embodiment, the dequantized signals are passed through the appropriate TNS filters 204. The TNS filters 204 are not essential to the operation of the decoding system and can be replaced by relay elements. Thereafter, the signal is applied to the second switch node 205, which performs the same function as the switch node 203 located in the upstream direction. Provided that the input signals are as described above, and the second switching node 205 is set to the relay mode, the output signal of the first is a downmix channel signal and a residual channel signal. The downmix signal, still represented by its time-consecutive MDCT spectrum, is fed into a real-to-imaginary transform 206 adapted to compute the MDST spectrum of the downmix signal from it. In this embodiment, one MDST frame is based on three MDCT frames: one previous frame, one current (or simultaneous) frame, and one subsequent frame. Symbolically indicated - , - that the input side of the real-to-imaginary conversion 206 includes delay elements.

The MDST representation of the downmix signal obtained from the real-to-imaginary transform 206 is weighted by the imaginary part prediction coefficient and is added to the MDCT representation of the downmix signal, weighted by the real part a prediction coefficient, and an MDCT representation of the residual signal. The two additions and multiplications are performed by multipliers and adders 210, 211 collectively forming a (functionally) weighted adder, which are provided with a complex prediction coefficient value α encoded in the bitstream initially received by the decoder system. The complex prediction coefficient may be determined once for each time frame. It may also be determined more frequently, such as once for each frequency band within a frame, where the frequency bands represent a psychoacoustically motivated division. It may also be less frequently defined, as will be described below in connection with the coding systems of the invention. The real-to-imaginary transform 206 is synchronized with the weighted adder such that the current MDST frame of the downmix channel signal is combined with the simultaneous MDCT frames for each of the signals, the downmix channel signal, and the residual channel signal. The sum of these three signals is a side signal In this expression, M includes both the MDCT and MDST representations of the downmix signal, namely: , Where is valid. Thus, a stereo signal is obtained containing a downmix channel and a side channel, from which the sum-difference transform 207 reconstructs the left and right channels as follows:

These signals are represented in the MDCT region. The last step of the decoding system is to apply the inverse MDCT 209 to each of the channels, whereby a left/right stereo time domain representation is obtained.

A possible implementation of real-to-imaginary conversion 206 is further described in Applicant's US Pat. No. 6,980,933 B2, as noted above. According to formula 41 in this patent, the transformation can be expressed as a FIR filter, for example, for even points:

Where — -th MDST data point, is the MDCT data for each of the frames, and N is the length of the frame. Besides, are impulse responses that depend on the window function used and are therefore determined for each choice of window function, such as rectangular, sinusoidal, or derivative of the Kaiser-Bessel window, and for each frame length. The complexity of this calculation can be reduced by skipping those impulse responses that have relatively less energy and contribute relatively less to the MDST data. As an alternative or extension to this implementation, the impulse responses themselves may be shortened, for example, from a full frame length N to fewer points. For example, the length of the impulse response can be reduced from 1024 points (links) to 10 points. The most extreme truncation that can still be considered significant is

Other direct approximations can be found in US Patent Application No. 2005/0197831 A1.

You can also reduce the amount of input on which the calculation is based. To illustrate, the real-to-imaginary transform 206 and its upstream connections, which are indicated by the section labeled "A" in the illustration, can be replaced by simplified options, two of which, A' and A", are shown in Figure 10. Option A' creates an approximate imaginary representation of the signal. In this case, the calculation of MDST takes into account only the current and previous frames. With reference to the formula above in this paragraph, this can be done by specifying at p=0, ..., N–1 (index III denotes the next time frame). Since option A' does not require the MDCT spectrum for the next frame as input, the MDST calculation is not subject to any time delay. Obviously, this approximation slightly reduces the accuracy of the received MDST signal, but may also imply that the energy of this signal will become reduced; the latter fact can be fully compensated by increasing as a result of the predictive coding entity.

Also shown in Figure 10 is Option A" which uses only the MDCT data for the current frame as input. It is possible that Option A" generates a less accurate representation of the MDST than Option A'. On the other hand, it, like option A', operates with zero delay and has less high computational complexity. As already mentioned, the waveform encoding properties are not affected since the encoder system and the decoder system use the same approximation.

It should be noted that, regardless of whether options A, A' or A", or any improvement thereof, are used, it is necessary to calculate only those parts of the MDST spectrum for which the imaginary part of the complex prediction coefficient is non-zero, . In terms of practical application, this means that the absolute value the imaginary part must be greater than a predetermined threshold value that can be referred to the rounding unit of the hardware used. In the case where the imaginary part of the coefficient is zero for all frequency bands within a time frame, there is no need to calculate any MDST data for that frame. Thus, accordingly, the real-to-imaginary conversion 206 adapts to the response to the occurrence of too small values by not generating the MDST output, whereby computing resources can be saved. However, in embodiments where more frames than the current frame are used to generate a single frame of MDST data, any upstream elements of the transform 206 must continue to operate accordingly even when no MDST spectrum is required—in particular, the second switching node 205 must continue to forward the MDCT spectra—so that sufficient input data is already available for the real-to-imaginary transform 206 when the next time frame associated with a non-zero prediction coefficient arrives; of course, this frame can be the next frame.

Returning to Figure 2, the operation of the decoding system is described on the assumption that both switching nodes 203, 205 are set to their respective relay modes. As will be described below, the decoder system can also decode signals that are not predictively coded. For such an application, the second switching node 205 is set to the sum-difference mode and, accordingly, as indicated in the illustration, the selector circuit 208 will be set to the lower position, thus providing signals directly to the inverse transform 209 from the source location between the TNS filters 204 and the second switching node 205. To ensure correct decoding, the signal, respectively, has an L/R shape at the source location. Therefore, in order to ensure that the correct average signal (i.e., the downmix signal) is fed into the real-to-imaginary conversion at all times (and not, say, periodically - the left signal), the second switching node 205 during decoding of a stereo signal that has not been subjected to prediction encoding is preferably set to sum-difference mode. As mentioned above, predictive coding can be replaced for some frames with conventional direct or joint coding based on, for example, audio rate/quality decision data. The outcome of this decision may be reported from the encoder to the decoder in various ways, for example, by the value of a dedicated indicator bit in each frame, or by the absence or presence of a prediction coefficient value. Once these facts are established, the role of the first switching node 203 becomes easily understood. Indeed, in the non-prediction coding mode, the decoder system can process signals both in accordance with direct (L/R) stereo coding and in accordance with joint (M/S) stereo coding, and by operating the first switching node 203 in either relay mode or sum-difference mode, it can be ensured that the source location will always be provided with a directly encoded signal. Obviously, the switching node 203, when functioning as a sum-difference stage, will convert the input signal in M/S form to the input signal (supplied to the optional TNS filters 204) in L/R form.

The decoder system receives a signal as to whether the decoder system should decode a specific temporal frame in a predictive coding mode or in a non-prediction coding mode. The non-prediction mode can be signaled by the value of a dedicated indicator bit in each frame, or by the absence (or zero value) of the prediction coefficient. The prediction mode may be reported similarly. A particularly benevolent implementation, which makes it possible to fall back without any overhead, uses the reserved fourth value of the two-bit ms_mask_present field (see MPEG-2 AAC, ISO/IEC 13818-7 document), which is transmitted in every frame and is defined as follows:

Table 1: Definition of ms_mask_present in USAC Meaning Meaning 00 L/R coding for all frequency bands 01 One signal bit per frequency band is used to indicate L/R or M/S 10 M/S coding for all frequency bands eleven Reserved

By redefining the value 11 as meaning "complex prediction coding", the decoder can operate in all compatibility modes, in particular M/S and L/R coding modes, without any bit rate loss, and still be available to receive a signal indicating the complex prediction coding mode for the respective frames.

Figure 4 shows a decoder system of the same general design as shown in figure 2, but including, however, at least two other designs. First, the system of Figure 4 includes switches 404, 411 that enable any processing stage involving modification in the frequency domain, upstream and/or downstream of the upmix stage. This is done on the one hand by a first set of frequency domain modifiers 403 (depicted in this figure as TNS synthesis filters) provided in conjunction with the first switch 404 downstream of the dequantizers 401 and the first switching node 402, but downstream of the second switching node 405 located in the upstream direction immediately before the boost stage 406, 407, 408, 409 kshirovaniya. On the other hand, the decoder system includes a second set of modifiers 410 in the frequency domain, which are provided in conjunction with the second switch 411 in a downward direction relative to the upmix stage 406, 407, 408, 409, but in an upward direction relative to the inverse transform stage 412. Preferably, as shown in the illustration, each frequency domain modifier is located in parallel with a relay line that is connected upstream of the input side of the frequency domain modifier and downstream of its associated switch. By virtue of this design, the frequency domain modifier is supplied with signal data at all points in time, which makes it possible to process in the frequency domain based on more frames than just the current frame. The decision whether to apply the first 403 or second 410 set of modifiers in the frequency domain may be made by the encoder (and transmitted in the bitstream), or may be based on whether predictive coding is applied, or may be based on some other criteria found appropriate in the practical application. For example, if the modifiers in the frequency domain are TNS filters, then for some types of signals it is advantageous to use the first set 403, while the use of the second set 410 may be advantageous for other types of signals. If the result of this selection is encoded in the bitstream, then the decoder system will drive the appropriate set of TNS filters accordingly.

To facilitate understanding of the decoder system shown in Figure 4, it should be unambiguously noted that the decoding of the directly encoded (L/R) signal occurs when α=0 (this implies that the pseudo-L/R and L/R are identical, and that the side and residual signals do not differ), the first switching node 402 is in the relay mode, the second switching node is in the sum-difference mode, thus causing the signal between the second node 40 5 switching and the sum-difference stage 409 of the upmix stage has an M/S shape. Then, since the upmix stage will effectively be a relay stage, it does not matter whether the first or second set of modifiers will be driven (using the respective switches 404, 411).

Figure 3 illustrates a decoder system according to one of the embodiments of the invention, which, in relation to the options in figures 2 and 4, represents a different approach to creating the MDST data needed for upmixing. As with the decoder systems already described, the system of Figure 3 includes dequantizers 301, a first switching node 302 operating in either relay mode or sum-difference mode, and TNS (synthesizing) filters 303 that are sequentially arranged starting from the input end of the decoder system. Modules downstream from this point are selectively utilized by the two second switches 305, 310, which preferably act together so that they are both in either the up or down positions as indicated in the figure. At the output end of the decoder system is a sum-difference stage 312 and, immediately before it in the upstream direction, two inverse MDCT modules 306, 311 for converting the MDCT domain representation of each of the channels to a time domain representation.

In complex prediction decoding, where the decoder system is provided with a bit stream encoded with a downmix signal/residual stereo signal and complex prediction coefficient values, the first switching node 302 is set to the relay mode, and the second switches 305, 310 are set to the up position. Downstream of the TNS filters, the two channels (dequantized, TNS filtered, MDCT) of the audio signal are processed differently. The downmix signal is applied, on the one hand, to a multiplier and adder 308 which adds the MDCT representation of the downmix channel, weighted by the real part α _R of the prediction coefficient, to the MDCT representation of the residual channel, and, on the other hand, to one of the inverse MDCT transform modules 306. The time domain representation of the downmix channel M, which is output from the inverse MDCT transform module 306, is applied to both the final sum-difference stage 312 and the MDST transform module 307 . This dual use of the time domain representation of the downmix channel is advantageous in terms of computational complexity. The thus obtained MDST representation of the downmix channel is fed to the next multiplier and adder 309 which, after being weighted by the imaginary part α _I of the prediction coefficient, adds this signal to the linear combination at the output of the adder 308; thus, the output signal of the adder 309 is a side channel signal, Similarly to the decoder system shown in Figure 2, the multipliers and adders 308, 309 can easily be combined to form a weighted multi-signal adder that outputs the MDCT and MDST representations of the downmix signal, the MDCT representation of the residual signal, and the value of the complex prediction coefficient. Downstream from this point, in the present embodiment, all that remains is passing through the inverse MDCT transform module 311 before the side channel signal is applied to the final sum-difference stage 312.

The necessary synchronicity in the decoder system can be achieved by applying the same transform lengths and window shapes for both inverse MDCT transform modules 306, 311, as is already practiced in frequency selective M/S and L/R coding. The combination of some embodiments of the MDCT inverse transform module 306 and some embodiments of the MDST module 307 introduces a delay of one frame. Therefore, five optional delay blocks 313 (or software commands - to perform this action in a computer implementation) are provided so that the part of the system located to the right of the dashed line can, if necessary, be delayed one frame relative to the left part. Obviously, all intersections between the dotted line and the links are provided with delay blocks, except for the link between the inverse MDCT module 306 and the MDST transform module 307, which is where the resulting delay needs to be compensated.

Computing MDST data for one time frame requires one frame data from the time domain representation. However, the inverse MDCT is based on one (current), two (preferably previous and current), or three (preferably previous, current and next) consecutive frames. Due to the well-known low frequency spurious cancellation (TDAC) associated with MDCT, the three-frame capability performs full overlap of input frames and thus provides the best (and probably perfect) accuracy, at least for frames containing low frequency spurious components in the time domain. It is clear that the three-frame inverse MDCT operates with a delay of one frame. By allowing the approximate time domain representation to be used as input to the MDST transform, this delay and thus the need to compensate for delays between different parts of the decoder system can be avoided. In a two-frame opportunity, aliasing/addition occurs in the earlier half of the frame, enabling TDAC, and low frequency spurious components may only be present in the later half of the frame. In the single frame capability, the absence of TDAC implies that low frequency spurious components may appear in the frame; however, the MDST representation thus achieved and used as an intermediate signal in complex prediction coding can still provide satisfactory quality.

The decoding system illustrated in Figure 3 can also operate in two non-prediction decoding modes. To decode the L/R stereo signal encoded directly, the second switches 305, 310 are set to the down position, and the first switch part 302 is set to the relay mode. Thus, the signal is L/R-shaped in the upstream direction with respect to the sum-difference stage 304, which converts it to M/S-form, on which the inverse MDCT transform and sum-difference operation takes place. To decode a stereo signal represented in jointly encoded M/S-form, the first switch node 302 is instead set to sum-difference mode so that the signal between the first switch node 302 and the sum-difference stage 304 has an L/R waveform, which is often more suitable in terms of TNS filtering than an M/S waveform could be.

The downstream processing with respect to the sum-difference stage 304 is identical to the processing in the case of direct L/R decoding.

Figure 14 consists of three generalized block diagrams of decoders according to embodiments of the invention. Unlike some of the other block diagrams accompanying this application, the link in Figure 14 may represent a multi-channel signal. In particular, said link may be arranged to transmit a stereo signal including left/right, middle/side, downmix/residual, pseudo-left/pseudo-right channels, and other combinations.

Figure 14A shows a decoder system for decoding a frequency domain representation of an input signal (referred to for the purposes of this figure as an MDCT representation). The decoder system is adapted to output as an output a time domain representation of a stereo signal that is generated based on the input signal. In order to be able to decode an input signal encoded by complex prediction stereo coding, the decoder system is equipped with an upmix stage 1410. However, it is also capable of handling an input signal encoded in other formats, and possibly a signal that alternates in time between multiple coding formats, for example, a sequence of time frames encoded with complex prediction coding may be followed by a time slice encoded with direct left/right coding. The ability of the decoder system to handle different coding formats is achieved by creating a communication link (relay) located in parallel with said upmix stage 1410. Switch 1411 can select whether the output of the upmix stage 1410 (bottom switch position in the figure) or the raw signal available above the link (upper switch position in the figure) will be fed to the further downstream decoder modules. In this embodiment, the inverse MDCT module 1412, which converts the MDCT representation of a signal to a time domain representation, is located downstream of the switch. For example, the signal applied to the upmix stage 1410 may be a stereo signal in downmix/residual form. The upmix stage 1410 is adapted to obtain a side signal and to perform a sum-difference operation so that the output signal is a left/right stereo signal (in the MDCT region).

Figure 14B shows a decoder system similar to that of Figure 14A. The present system is adapted to receive a bitstream as input. First, the bitstream is processed by a combined demultiplexer and dequantizer module 1420, which creates as a first output an MDCT representation of a multi-channel stereo signal for further processing, which is determined by the position of a switch 1422 having the same function as the switch 1411 of Figure 14A. More precisely, the switch 1422 determines whether the first output signal from the demultiplexer and dequantization should be processed by the upmix stage 1421 and the reverse MDCT module 1423 (low position) or only by the reverse MDCT module 1423 (high position). The combined demultiplexer and dequantizer module 1420 also outputs control information. In the present case, the control information associated with the stereo signal may include data indicating which switch 1422 position, low or high, is suitable for decoding the signal, or, more abstractly, according to which coding format the stereo signal is to be decoded. The control information may also include parameters for adjusting properties of the upmix stage 1421, such as the value of the complex prediction coefficient α used in complex prediction coding, as already described above.

Figure 14C shows a decoder system that, in addition to objects similar to those in figure 14B, includes first and second frequency domain modification devices 1431, 1435, respectively, located in the up and down directions relative to the upmix stage 1433. For the purposes of this figure, each frequency domain modification device is illustrated with a TNS filter. However, the term "frequency domain modification device" can also be understood to mean other processes than TNS filtering, which can be used either before or after the upmix stage. Examples of modifications in the frequency domain include prediction, interference accumulation, bandwidth stretching, and non-linear processing. Psychoacoustic considerations and similar reasons, which may include the properties of the signal being processed and/or the configuration or settings of the frequency domain modification device, sometimes indicate that it is advantageous to apply said modification in the frequency domain in an upstream rather than downstream direction relative to the upmix stage 1433. In other cases, by similar considerations, it may be found that the location of the modification in the frequency domain in the downstream direction is more preferable than the location in the upstream direction. Through the switches 1432, 1436, the frequency domain modification devices 1431, 1435 can be selectively actuated so that, in response to the control information, the decoder system can select the desired configuration. For example, Figure 14C shows a configuration in which the stereo signal from the combined demultiplexer and dequantizer module 1430 is first processed by the first frequency domain modifier 1431, then fed to the upmix stage 1433, and finally routed directly to the inverse MDCT module 1437 without passing through the second frequency domain modifier 1435. As explained in the Brief Description of the Invention, in complex prediction coding, this configuration is preferred over being able to perform TNS after upmixing.

II. encoder systems

An encoder system according to the invention will be described below with reference to Figure 5, which is a generalized block diagram of an encoder system for encoding a left/right (L/R) stereo signal as an output bitstream by complex prediction coding. The encoder system takes a time or frequency domain representation of the signal and feeds it to the downmix stage and the prediction coefficient estimator. The real and imaginary parts of the prediction coefficients are passed to the downmix stage to control the conversion of the left and right channels into the downmix channel and the residual channel, which are then fed to the final MUX. If the signal is not fed to the encoder as a frequency domain representation, then it is converted to such a representation in the downmix stage or in the multiplexer.

One of the principles of predictive coding is to transform the left/right signal into a middle/side form, i.e.:

and then using the residual correlation between these channels, i.e. exercise

where α is the complex prediction coefficient to be determined and D is the residual signal. One can choose α so that the energy of the residual signal was minimized. Energy minimization can be performed with respect to instantaneous power, short-term or long-term energy (average power), which in the case of a discrete signal is equivalent to optimization in the rms value.

The real and imaginary parts α _R , α _I of the prediction coefficient may be quantized and/or co-coded. Preferably, however, the real and imaginary parts are quantized independently and uniformly, typically with a step size of 0.1 (a dimensionless number). According to the MPEG standard, the bandwidth resolution used for the complex prediction coefficient is not necessarily the same as the resolution for scalefactor bands (sfb; i.e., groups of MDCT lines that use the same MDCT quantization step size and quantization range). In particular, the bandwidth resolution for the prediction coefficient can be justified psychoacoustically, such as the Bark scale. It should be noted that the bandwidth resolution may change if the transform length changes.

As already noted, the encoder system according to the invention may have the freedom to apply stereo predictive coding or not, where the latter case implies a reverse transition to a lower level - to L/R or M/S coding. This decision may be made on the basis of a time frame or, more precisely, on the basis of a bandwidth within a time frame. As noted above, a negative decision may be signaled to the decoding entity in various ways, for example, by the value of a dedicated indicator bit in each frame, or by the absence (or zero value) of a prediction coefficient value. A positive decision can be transmitted similarly. A particularly advantageous implementation that allows back downgrading without any overhead uses a reserved fourth value of the two-bit ms_mask_present field (see MPEG-2 AAC document ISO/IEC 131818-7) which is transmitted in every frame and is defined as follows:

Table 1: Definition of ms_mask_present in USAC Meaning Meaning 00 L/R coding for all frequency bands 01 One signal bit per lane is used to indicate L/R- or M/S 10 M/S coding for all frequency bands eleven reserved

By redefining the value 11 to mean "complex prediction coding", the encoder can operate in all compatibility modes, in particular M/S and L/R coding modes, without any bit rate loss and still be able to signal complex prediction coding for those frames where it is advantageous.

The appropriate decision may be based on considerations regarding the relationship between data rate and audio quality. As a measure of quality, data obtained using the psychoacoustic model contained in the encoder can be used (as is often the case with available MDCT-based audio encoders). In particular, some embodiments of the encoder provide for the selection of a prediction coefficient optimized based on the trade-off between rate and distortion. Accordingly, in such embodiments of the invention, the imaginary part, and possibly also the real part, of the prediction coefficient is set to zero in case the prediction gain does not save enough bits to encode the residual signal so as to justify the bits required to encode the prediction coefficient.

Embodiments of the encoder may encode TNS related information in the bitstream. This information may include TNS parameter values that are applied by the (synthesis) TNS filters on the decoder side. If identical TNS parameter sets are used for both channels, it is more economical to include a signal bit indicating this parameter set identity rather than to transmit the two parameter sets independently. Information may also be included as to whether TNS should be applied before or after the upmix stage, as appropriate based on, for example, a psychoacoustic evaluation of both available options.

As another optional feature that is potentially beneficial in terms of complexity and data bit rate, the encoder can be adapted to use a separately limited bandwidth when encoding the residual signal. Bands above this limit will not be sent to the decoder, but will be set to zero. In some cases, the highest frequency bands have such a low energy content that they are quantized to zero. In standard practice (cf. the max_sfb parameter in the MPEG standard), it is common to use the same bandwidth limit for both the downmix signal and the residual signal. So far, the inventors have experimentally found that the residual signal, to a greater extent than the downmix signal, has an energy content localized in lower frequency bands. Therefore, by placing a dedicated upper bandwidth limit on the residual signal, it is possible to reduce the bit rate without significant loss of quality. For example, this can be done by passing two independent max_sfb parameters, one for the downmix signal and one for the residual signal.

It should be noted that, although the issues of optimal determination of the prediction coefficient, its quantization and coding, reverse downgrading to M/S or L/R mode, TNS filtering, upper bandwidth limitation, etc. discussed in relation to the decoder system shown in figure 5, the same facts apply equally to the embodiments of the invention, which will be disclosed below with reference to the following figures.

Figure 6 shows another encoder system according to the invention adapted to perform complex prediction stereo coding. The system takes as input a representation of a stereo signal in the time domain, segmented into successive, possibly overlapping, time frames and including left and right channels. The sum-difference stage 601 converts the signal into the middle and side channels. The middle channel is provided to both the MDCT module 602 and the MDST module 603, while the side channel is provided only to the MDCT module 604. The prediction coefficient estimator 605 estimates for each time frame—and possibly for individual frequency bands within a frame—the value of the complex prediction coefficient α, as explained above. The coefficient value α is fed as a weighting factor to the weighted adders 606, 607, which form the residual signal D as a linear combination of the MDCT and MDST representations of the average signal and the MDCT representation of the side signal. Preferably, the complex prediction coefficient is fed into the weighted adders 606, 607 represented in the same quantization scheme that will be used when it is encoded into the bitstream; this obviously provides a more reliable reconstruction since both the encoder and the decoder use the same value of the prediction coefficient. The residual signal, the middle signal (which is more appropriately referred to as the downmix signal when it occurs in combination with the residual signal), and the prediction coefficient are provided to a combined quantization and multiplexer stage 608 which encodes them and possible side information into the output bitstream.

Figure 7 shows a variation of the encoder system of Figure 6. As can be seen from the similarity of the symbols in the figure, it has a similar construction, but also contains an additional function of the direct L/R encoding down-level operation. The encoder system is operated between the complex prediction coding mode and the reverse down-pass mode by means of a switch 710 provided in the upstream direction just before the combined quantization and multiplexer stage 709 . In the up position, as shown in the figure, switch 710 will cause the encoder to operate in a back-down mode. From points downstream just beyond the MDCT modules 702, 704, the middle/side signal is fed to a sum-difference stage 705 which, after conversion to left/right form, passes it to a switch 710 which couples it to the combined quantization and multiplexer stage 709.

Figure 8 shows the encoder system according to the present invention. Unlike the encoder systems of Figures 6 and 7, this embodiment obtains the MDST data needed for complex prediction encoding directly from the MDCT data, i. by converting real to imaginary in the frequency domain. The real-to-imaginary transformation employs any of the approaches described in connection with the decoder systems of Figures 2 and 4. It is important that the calculation method in the decoder matches the calculation method in the encoder so that reliable decoding can be performed; preferably, the encoder side and the decoder side use the same real-to-imaginary conversion methods. As with the decoder embodiments, the portion A surrounded by a dashed line and including the real-to-imaginary transform 804 can be replaced by approximations using fewer input time frames as input. Likewise, encoding may be simplified using any of the methods described above, or some other approximate method.

At a higher level, system 8 has a design that differs from that which would follow the direct action of replacing the MDST module in FIG. 7 with a (properly connected) real-to-imaginary conversion module. The present architecture is understandable and performs the function of switching between predictive coding and direct L/R coding in a robust and computationally efficient manner. The input stereo signal is applied to MDCT transform modules 801, which output a frequency domain representation of each of the channels. It is fed both to the final switch 808 to drive the encoder system between predictive and direct encoding modes, and to the sum-difference stage 802. 807 quantizer and a multiplexer located at the output end of the system where the bitstream is fed. In predictive coding, each of the channels undergoes further processing between the sum-difference stage 802 and the switch 808. From the MDCT average signal representation, the real-to-imaginary transform 804 obtains the MDST data and sends it to both the prediction coefficient estimator 803 and the weighted adder 806. As in the encoder systems of Figures 6 and 7, the following weighted adder 805 is used to combine the side signal with the weighted MDCT and MDST representations of the middle signal to form a residual channel signal that is co-coded with the middle channel signal (i.e., the downmix channel) and with the prediction coefficient using the combined quantizer and multiplexer 807.

With reference to Figure 9, it will be illustrated that each of the embodiments of the encoder system can be combined with one or more (analyzing) TNS filters. In accordance with previous discussions, it is often more advantageous to apply TNS filtering to a signal in its downmix form. Therefore, as shown in FIG. 9, the adaptation of the encoder system of FIG. 7 to include TNS is done by adding upstream TNS filters 911 just before the combined quantizer and multiplexer 909.

Instead of a downstream right/residual TNS filter 911b, two separate TNS filters (not shown) may be provided directly downstream of the switch portion 910 adapted for right or residual channel manipulation. Thus, each of the two TNS filters will be supplied with signal data of the corresponding channel at all times, which allows TNS filtering based on one or more time frames and not just the current frame. As already noted, TNS filters are just an example of frequency domain modification devices, in particular devices that base processing on more frames than the current frame alone, which in this arrangement can reap the same or more benefit than a TNS filter.

As another possible alternative to the embodiment of the invention shown in Figure 9, TNS filters for selective actuation may be located at more than one point for each channel. This design is similar to that of the decoder system shown in Figure 4, where different TNS filter banks can be linked via switches. This allows the selection of the most appropriate stage in TNS filtering for each time frame. In particular, it may be advantageous to switch between different TNS positions in connection with switching between complex prediction stereo coding and other coding modes.

Figure 11 shows a variation based on the encoder system of Figure 8 in which, by real-to-imaginary conversion 1105, a second frequency domain representation of the downmix signal is obtained. Similar to the decoder system shown in Figure 4, this encoder system also includes selectively actuated frequency domain modifier modules: one 1102 provided upstream of the downmix stage and one 1109 provided downstream of that stage. Modules 1102, 1109 in the frequency domain, which in this figure is represented by the example of TNS filters, can be associated with each of the signal paths using four switches 1103a, 1103b, 1109a and 1109b.

III. Non-hardware embodiments of the invention

Embodiments of the third and fourth features of the invention are shown in Figures 15 and 16. Figure 15 shows a method for decoding a bitstream into a stereo signal, which includes the following steps:
1. The input signal is a bitstream.
2. The bitstream is dequantized, whereby a representation of the downmix channel and the residual channel in the frequency domain is obtained.
3. A second frequency domain representation of the downmix channel is calculated.
4. Based on the three frequency domain channel representations, the side channel signal is calculated.
5. Based on the side channel and the downmix channel, a stereo signal is calculated, preferably in left/right form.
6. The output signal is the stereo signal thus obtained.
Steps 3-5 can be viewed as an upmix process. Each of steps 1-6 is analogous to the corresponding function in any of the decoder systems disclosed in the previous parts of this text, and other details relating to their implementation can be found in the same parts.

Figure 16 shows a method for encoding a stereo signal as a bitstream signal, which includes the following steps:
1. The input signal is a stereo signal.
2. The stereo signal is converted to the first representation in the frequency domain.
3. The coefficient of complex prediction is determined.
4. The frequency domain representation is downmixed.
5. The downmix signal and the residual signal are encoded as a bitstream together with a complex prediction coefficient.
6. The output is a bit stream.
Each of steps 1-5 is analogous to the corresponding function in any of the encoder systems covered in the previous parts of this text, and other details related to their implementation can be found in the same parts.

Both methods can be expressed as computer readable instructions in the form of programs implemented in software and can be executed by a computer. The scope of protection of the present invention extends to the specified software and computer program products intended for distribution of the specified software.

IV. Experimental evaluation

Some of the embodiments of the invention disclosed herein have been quantified experimentally. In this subsection, the most important parts of the experimental material obtained in this process will be summarized.

The embodiment of the invention used for the experiments had the following characteristics:
(i) each MDST spectrum (per time frame) was computed by FIR filtering from the current, previous, and next MDCT spectra.
(ii) The psychoacoustic model from the USAC stereo encoder was used.
(iii) Instead of the PS, ICC, CLD and IPD parameters, the real and imaginary parts of the complex prediction coefficient were transmitted. The real and imaginary parts were processed independently, limited to the interval [–3.0, 3.0], and quantized using a step size of 0.1. They were then time-differentiated encoded and finally Huffman encoded using the USAC scaling factor code book. The prediction coefficients were updated in every second scalefactor band, resulting in a frequency resolution that is close to that of MPEG Surround (see, for example, ISO/IEC 230031). This quantization and coding scheme results in an average bit rate of approximately 2 kbps for the specified stereo ancillary information in a typical configuration with a target bit rate of 96 kbps.
(iv) The bitstream format is modified without breaking the current USAC bitstreams, since the 2-bit element of the ms_mask_present bitstream has only three possible values at the current time. The use of a fourth value to indicate complex prediction allows for a back transition to a lower level of middle/side coding without loss of bits (see the previous subsection of this disclosure for further details on this topic).

Test listening was performed according to the MUSHRA method, which consists of playing individually through headphones and using 8 test elements with a sampling rate of 48 kHz. Three, five or six test objects took part in this test. The impact of various MDST approximations has been evaluated to illustrate the practical trade-off between complexity and quality that exists between these capabilities. The results are in Figures 12 and 13 where the former shows the absolute scores obtained and the latter shows the differential scores relative to 96s USAC cp1f, ie. with respect to unified stereo coding in the complex prediction MDCT domain using the current MDCT frame to calculate the MDST approximation. As can be seen, the quality gain achieved by MDCT-based unified stereo coding increases when computationally more complex approaches to MDST spectrum calculation are applied. With respect to the average overall test, the USAC cp1f system 96s on a single frame basis provides a significant increase in coding efficiency over traditional stereo coding. In turn, even better results are obtained for 96s USAC cp3f, namely: unified stereo coding in the MDCT domain with complex prediction using the current, previous and next MDCT frames to calculate the MDST.

V. Embodiments of the Invention

Also, the invention may be implemented as a decoder system for decoding a bitstream signal into a stereo signal by complex prediction stereo coding, where the decoder system includes:
- a dequantizing stage (202; 401) for generating first frequency domain representations of the downmix signal (M) and residual signal (D) based on the bitstream signal, where each of the first frequency domain representations includes first spectral components that represent the spectral content of the corresponding signal expressed in a first subspace of multidimensional space, where the first spectral components are transform coefficients located in one or more transform coefficient time frames, where each block is generated by applying a transform to the time slice of the signal in the time domain; And
- an upmix stage (206, 207, 210, 211; 406, 407, 408, 409) located downstream of the dequantization stage, adapted to generate a stereo signal based on the downmix signal and the residual signal, and comprising:
- - a module (206; 408) for calculating a second representation of the downmix signal in the frequency domain based on its first representation in the frequency domain, where the second representation in the frequency domain includes the second spectral components representing the spectral composition of the signal expressed in the second subspace of the multidimensional space, which includes a part of the multidimensional space not included in the first subspace, where the specified module is adapted for:
- - - obtaining one or more first intermediate components from at least some of the first spectral components;
- - - the formation of a combination of the specified one or more spectral components in accordance with at least part of one or more impulse responses in order to obtain one or more second intermediate components; And
- - - obtaining said one or more second spectral components from said one or more second intermediate components;
- a weighted adder (210, 211; 406, 407) for calculating a side signal (S) based on the first and second representations of the downmix signal in the frequency domain, the first representation of the residual signal in the frequency domain, and the complex prediction coefficient (α) encoded in the bitstream signal; And
- a sum-difference stage (207; 409) for calculating the stereo signal based on the first representation in the frequency domain of the downmix signal and the side signal.

In addition, the invention may be implemented as a decoder system for decoding a bitstream signal into a stereo signal by complex prediction stereo coding, where the decoder system includes:
a dequantizing stage (301) for generating first frequency domain representations of the downmix signal (M) and residual signal (D) based on the bitstream signal, where each of the first frequency domain representations includes first spectral components representing the spectral composition of the corresponding signal expressed in the first subspace of the multidimensional space; And
- an upmix stage (306, 307, 308, 309, 312) located downstream of the dequantization stage, adapted to generate a stereo signal based on the downmix signal and the residual signal, and comprising:
- - a module (306, 307) for calculating a second frequency domain representation of the downmix signal based on its first frequency domain representation, where the second frequency domain representation includes second spectral components representing the spectral composition of the signal expressed in the second subspace of the multidimensional space, which includes a part of the multidimensional space not included in the first subspace, where the module includes:
- - - an inverse transform stage (306) for calculating a representation of the downmix signal in the time domain based on the first representation of the downmix signal in the frequency domain in the first subspace of the multidimensional space; And
- - - conversion stage (307) for calculating a second representation of the downmix signal in the frequency domain based on the representation of this signal in the time domain;
- a weighted adder (308, 309) for calculating a side signal (S) based on the first and second representations of the downmix signal in the frequency domain, the first representation of the residual signal in the frequency domain, and the complex prediction coefficient (α) encoded in the bitstream signal; And
- a sum-difference stage (312) for calculating a stereo signal based on the first representation of the downmix signal in the frequency domain and the side signal.

In addition, the invention may be implemented as a decoder system with the features set forth in the independent claims relating to a decoder system, wherein a module for computing a second representation of a frequency domain downmix signal includes:
an inverse transform stage (306) for calculating a time domain representation of the downmix signal and/or side signal based on a first frequency domain representation of the corresponding signal in a first subspace of the multidimensional space; And
- a conversion stage (307) for calculating a second representation of the corresponding signal in the frequency domain based on the representation of this signal in the time domain,
where, preferably, the inverse transform stage (306) performs an inverse modified discrete cosine transform, and the transform stage performs a modified discrete sine transform.

In the decoder system described above, the stereo signal may be represented in the time domain, and the decoder system may include:
- a switching unit (302) located between said dequantizing stage and said upmixing stage, capable of operating to perform the following functions:
(a) relay stage, for use in joint stereo coding; or
(b) sum-difference stage for use in direct stereo coding;
- additional stage (311) inverse transformation, located on stage upmixing, designed to calculate the representation of the side signal in the time domain;
- a switching unit (305, 310) located in the upstream direction relative to the inverse conversion stages (306, 301), adapted for their selective association with:
(a) an additional sum-difference stage (304), which, in turn, is associated with a point in the downstream direction relative to the switching node (302) and with a point in the upward direction relative to the upmix stage; or
(b) a downmix signal obtained from the switching node (302) and a side signal obtained from the weighted adder (308, 309).

VI. Final remarks

Further embodiments of the present invention will become clear to those skilled in the art upon reading the above description. Although the present description and drawings disclose embodiments of the invention and examples, the invention is not limited to these specific examples. Numerous modifications and changes may be made without departing from the scope of the present invention, which is defined by the accompanying claims.

It should be noted that the methods and apparatus disclosed in this application, after appropriate modifications, can be used within the capabilities of a person skilled in the art, including typical experiments, for encoding signals containing more than two channels. It should be especially emphasized that any signals, parameters and matrices mentioned in connection with the described embodiments of the invention may be frequency dependent or frequency independent and/or time dependent or time independent. The calculation steps described may be carried out by frequency or for all frequency bands at a time, and all objects may be carried out so that they have a frequency selective effect. For the purposes of the application, any quantization schemes may be adapted according to psychoacoustic models. In addition, it should be noted that all different sum-difference transformations, i.e. downmix/residual to pseudo-L/R conversions, as well as L/R to M/S conversion and M/S to L/R conversion, are of the form

,

where only the gain can change. Thus, by adjusting the gains individually, a certain encoding gain can be compensated by appropriately selecting the decoding gain. In addition, as will be appreciated by those skilled in the art, an even number of stacked sum-difference transforms produces the effect of a relay stage, possibly with a non-unity gain.

The systems and methods described above in this specification may be implemented as software, firmware, hardware, or a combination thereof. Some or all of the components may be implemented as software executable by a digital signal processor or microprocessor, or implemented as hardware or as an application-specific integrated circuit. Said software may be distributed on computer-readable media, which may include computer storage media or communications media. As is well known to those skilled in the art, computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules, or other data in memory. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, DVD-format CD or other optical disc media, magnetic tape cassettes, magnetic tape, magnetic disk media or other magnetic storage devices, or any other media that can be used to store desired information in memory and that can be accessed by a computer. In addition, those skilled in the art will recognize that communication media typically embeds computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and includes any information delivery media.

Claims (27)

1. A device for outputting a stereo audio signal having a left channel and a right channel, the device comprising: a demultiplexer configured to receive the audio bitstream and extract a prediction coefficient therefrom, wherein the audio bitstream is segmented into frames and the value of the prediction coefficient may change for each of the frames; a decoder configured to generate a downmix signal and a residual signal from the encoded audio bitstream; And an upmixer capable of operating either in predictive or non-predictive mode based on a parameter encoded in the audio bitstream and output the left channel and right channel as a stereo audio signal, wherein when the upmixer operates in predictive mode, the residual signal represents the difference between the side signal and the predicted version of the side signal, and the upmixer generates a left channel and a right channel from the combination of the downmix signal, the residual signal, and the prediction coefficient, and wherein, when the upmixer operates in a non-prediction mode, the residual signal represents a side signal, the upmixer generates a left channel based on the sum of the downmix signal and the residual signal, and the upmixer generates a right channel based on the difference between the downmix signal and the residual signal. 2. The device according to claim 1, characterized in that the parameter is a prediction coefficient. 3. The device according to claim 2, characterized in that the upmixer operates in a non-prediction mode during a time frame when the value of the prediction coefficient is zero or less than a predetermined tolerance, and the upmixer operates in a predictive mode during a time frame for all other values of the prediction coefficient. 4. The apparatus of claim. 1, characterized in that the coefficient of prediction reduces or minimizes the energy of the residual signal. 5. The apparatus of claim 1, further comprising a noise limiter configured to limit noise associated with the downmix signal, the noise limiter being located in an upstream direction relative to the upmixer. 6. The device according to claim 5, characterized in that the noise limiting device is a temporal noise limiting device configured to limit noise over time. 7. Apparatus according to claim 1, wherein when the upmixer operates in predictive mode, the upmixer generates a left channel and a right channel with a filter having three stages. 8. The apparatus of claim. 1, characterized in that the downmix signal contains an average signal formed by a linear combination of the original left channel and the original right channel. 9. The device according to claim 1, characterized in that the prediction coefficient is a real-valued coefficient. 10. The device according to claim. 1, characterized in that the coefficient of prediction is a complex coefficient. 11. The apparatus of claim 1, wherein the upmixer combines the side signal with the downmix signal by adding the downmix signal version to the side signal version to generate a left channel, and by subtracting the side signal version from the downmix signal version to generate a right channel. 12. The device according to claim 1, characterized in that the prediction coefficient is encoded in the audio bitstream and the demultiplexer is additionally configured to decode the prediction coefficient. 13. The apparatus of claim 1, wherein the upmixer is further configured to add the residual signal to the side signal when operating in predictive mode. 14. The device according to claim 1, characterized in that the downmix signal is divided into frequency bands, the prediction coefficient contains a prediction coefficient for each of the frequency bands, the demultiplexer is configured to receive a bit audio stream and extract each of the prediction coefficients from it, and the upmixer, when operating in predictive mode, generates a left channel and a right channel from a combination of the downmix signal, the residual signal and each of the prediction coefficients. 15. A method for outputting a stereo audio signal having a left channel and a right channel, the method comprising: obtaining an audio bitstream and extracting a prediction coefficient therefrom, wherein the audio bitstream is segmented into frames and the value of the prediction coefficient may change for each of the frames; generating at the decoder a downmix signal and a residual signal from the audio bitstream; upmixing either in predictive or non-predictive mode based on a parameter encoded in the audio bitstream; And output the left channel and right channel as a stereo audio signal, wherein when the upmix is operating in a predictive mode, the residual signal represents the difference between the side signal and the predicted version of the side signal, and the upmix generates a left channel and a right channel from the combination of the downmix signal, the residual signal, and the prediction coefficient, and moreover, when the upmix is performed in a non-prediction mode, the residual signal represents a side signal, and the upmix generates a left channel based on the sum of the downmix signal and the residual signal passed from the decoder, and generates a right channel based on the difference between the downmix signal and the residual signal. 16. A persistent computer-readable medium containing instructions that, when executed by the processor, perform the method of claim 14.