ES2935962T3 - Stereo encoding using a prediction mode or a non-prediction mode - Google Patents

Abstract

The invention relates to a method and system for encoding a stereo audio signal having a left (L) channel and a right (R) channel, and a corresponding decoding method and system. The coding method comprises estimating a prediction coefficient (α) of the left channel and the right channel, mixing the left and right channel to produce a mean signal (M) and a residual signal (D) using the prediction coefficient, and multiplexing the mean signal, the residual signal and the prediction coefficient to generate an audio bitstream. The encoding operates in a predictive mode or a non-predictive mode for a period of time, and the method comprises issuing an indication of whether the encoding operated in the predictive mode or the non-predictive mode, wherein said indication is provides as a value. of the prediction coefficient. (Automatic translation with Google Translate, without legal value)

Description

DESCRIPTION

Stereo encoding using a prediction mode or a non-prediction mode

Cross reference to related applications

This application is a European divisional application of Euro-PCT patent application EP 11713754.7 (reference: D10010AEP01), filed on April 6, 2011.

technical field

The invention described herein relates generally to stereo audio coding. In particular, the invention described herein relates to providing a stereo signal by complex prediction stereo coding.

Background of the invention

Cocoding the left (L) and right (R) channels of a stereo signal allows for more efficient coding compared to independent coding of L and R. A common approach for joint stereo coding is center/side (M) coding. /S). Here, a central signal (M) is formed by the sum of the signals L and R, for example, the signal M can have the form

M = (L R)/2

Also, a side signal (S) is formed by subtracting the two channels L and R, for example, the S signal can have the form

In the case of M/S encoding, the M and S signals are encoded instead of the L and R signals.

In the standard (see standard document ISO/IEC 13818-7) AAC (Advanced Audio Coding) of MPEG (Moving Pictures Experts Group), L/R stereo coding and M/S stereo coding can be chosen in a time-varying and frequency-varying manner. Therefore, the stereo encoder can apply L/R encoding to some frequency bands of the stereo signal, while M/S encoding is used to encode other frequency bands of the stereo signal (frequency variant). In addition, the encoder can time-switch between L/R and M/S (time-variant) encoding. In MPEG AAC, stereo coding is performed in the frequency domain, more specifically in the MDCT (Modified Discrete Cosine Transform) domain. This allows to adaptively choose either the L/R or the M/S encoding in a frequency and also time variable manner.

Parametric stereo encoding is a technique for efficiently encoding a stereo audio signal as a monaural signal plus a small amount of side information for the stereo parameters. It is part of the MPEG-4 Audio standard (see standard document ISO/IEC 14496-3). The monaural signal can be encoded using any audio encoder. Stereo parameters can be embedded in the auxiliary part of the mono bitstream, thus achieving full forward and backward compatibility. In the decoder, it is the monaural signal that is decoded first, after which the stereo signal is reconstructed with the help of stereo parameters. A decorrelated version of the decoded mono signal, having a zero-crossing correlation with the mono signal, is generated by means of a decorrelator, eg, an appropriate all-pass filter which may include one or more delay lines. Essentially the decorrelated signal has the same spectral and temporal energy distribution as the mono signal. The monaural signal together with the de-mapped signal are fed into the upmix process which is controlled by the stereo parameters and which reconstructs the stereo signal. For more information, see the paper "Low Complexity Parametric Stereo Coding in MPEG-4", H. Purnhagen, Proc. from the 7th Int. Conference on Digital Audio Effects ( ^d A ^fx '04), Naples, Italy, October 5-8, 2004, pp. 163-168.

MPEG Surround (MPS; see ISO/IEC 23003-1 and the document “MPEG Surround - the ISO/MPEG Standard for Efficient and Compatible Multi-Channel Audio Coding”, J. Herre et al., Convention Document 7084 of Audio Engineering, 122nd Convention, May 5-8, 2007) allows combining the principles of parametric stereo coding with residual coding, replacing the decorrelated signal with a transmitted residual and therefore improving perceptual quality. Residual coding can be achieved by downmixing a multi-channel signal and optionally extracting the spatial signals. During the downmix process, the residual signals representing the error signal are calculated and then encoded and transmitted. These take the place of the decorrelated signals in the decoder. In a hybrid approach, they can replace decorrelated signals in certain frequency bands, preferably in the relatively low bands.

Based on the current MPEG Unified Audio and Speech Coding (USAC) system, of which two are shown Examples in Figure 1, the decoder comprises a complex valued quadrature mirror (QMF) filterbank located downstream of the core decoder. The QMF representation obtained as the output of the filter bank is complex - hence oversampled by a factor of 2 - and can be arranged as a downmix M signal (or, equivalently, a center signal) and a D signal. residual, to which an upmix matrix with complex inputs is applied. The L and R signals (in the QMF domain) are obtained as:

where g is a real-valued gain factor and a is a complex-valued prediction coefficient. Preferably, a is chosen such that the energy of the residual signal D is minimized. The gain factor can be determined by normalization, that is, to ensure that the energy of the sum signal is equal to the sum of the energies of the left and right signals. The real and imaginary parts of each of the left and right signals are mutually redundant - in principle each can be calculated based on the other - but are beneficial in allowing the later application of a band-replication decoder (SBR) without audible aliasing artifacts. The use of an upsampled signal representation may also, for similar reasons, be chosen in order to avoid artifacts connected with other time- or frequency-adaptive signal processing (not shown), such as mono-to-mono upmixing. stereo. Inverse QMF filtering is the last processing step in the decoder. It is noted that the bandlimited QMF representation of the signal allows bandlimited residual techniques and "residual padding" techniques, which can be integrated into decoders of this type.

The above coding structure is well suited for low bit rates, typically below 80 kb/s, but is not optimal with respect to computational complexity for higher bit rates. More precisely, at higher bit rates, the SBR tool is normally not used (since it would not improve encoding efficiency). So, in a decoder without an SBR stage, only the complex-valued upmix matrix motivates the presence of the QMF filterbank, which is computationally demanding and introduces delay (with a frame length of 1024 samples, the filterbank analysis/synthesis QMF introduces a delay of 961 samples). This clearly indicates a need for a more efficient coding structure.

A technical description of a proposal for unified stereo coding at USAC is provided in H. Purnhagen et al. “Technical description of proposed Unified Stereo Coding in USAC”, 90. MPEG Meeting, 26-10 2009-30-10-220; Xian, China; (Moving Image Expert Group or ISO/IEC JTC1/SC29/WG11), number M16921, October 23, 2009 (2009-10-23).

Compendium of the invention

It is an object of the present invention to provide methods and apparatus for stereo coding that are computationally efficient also in high bit rate range.

The invention fulfills this objective by providing a decoding system, a decoding method and a computer program product defined by the independent claims. The dependent claims define embodiments of the invention.

In a first aspect, the invention provides a decoder system for providing a stereo signal by complex prediction stereo coding, the decoder system comprising:

an upmix adapted to generate the stereo signal based on first frequency domain representations of a downmix signal ( M) and a residual signal ( D) , each of the first frequency domain representations comprising first spectral components that represent spectral content of the corresponding signal expressed in a first subspace of a multidimensional space, comprising the upmixing stage:

a module for calculating a second frequency domain representation of the downmix signal based on the first frequency domain representation thereof, the second frequency domain representation comprising second spectral components representing spectral content of the signal expressed in a second subspace of the multidimensional space that includes a portion of the multidimensional space not included in the first subspace, in which the module is adapted to determine the second spectral components of the downmix signal by applying a filter of finite impulse response (FIR) for combining at least two of the first two spectral components of the downmix signals;

a weighted adder for calculating a side signal (S) based on the first and second frequency-domain representations of the downmix signal, the first frequency-domain representation of the residual signal, and a coefficient ( a) of complex prediction encoded in the signal of bit stream; and

a sum and difference stage for calculating the stereo signal on the basis of the first frequency domain representation of the downmix signal and the side signal,

wherein the upmix stage is further operable in a pass mode, wherein said downmix signal and residual signal are supplied to the sum and difference directly.

In a second aspect of the invention, where a decoder method is provided for providing a stereo signal by complex prediction coding, as defined in independent claim 7. The technical characteristics of the method are analogous to those of the decoder system. In a third aspect, the invention further provides a computer program product containing instructions for executing the decoding method on a computer, as defined in independent claim 8.

The invention benefits from the advantages of unified stereo coding in the USAC MPEG system. These advantages are also maintained at higher bit rates, where SBR is not typically used, without the significant increase in computational complexity that would accompany a QMF-based approach. This is possible since the critically sampled MDCT transform, which is the basis of the USAC MPEG transform coding system, can be used for the complex prediction stereo coding that is provided by the invention, at least in cases where that the encoded audio bandwidths of the downmix channels are the same and the upmix process does not include decorrelation. This means that an additional QMF transform is no longer required. A representative implementation of complex prediction stereo coding in the QMF domain would actually increase the number of operations per unit time significantly compared to traditional L/R or M/S stereo. Therefore, the encoding apparatus according to the invention appears to be competitive at such bit rates, providing high audio quality with moderate computational expense.

As those skilled in the art will appreciate, the fact that the upmix stage is more operable in a pass mode allows the decoder to adaptively decode according to conventional straight or joint coding, and complex prediction coding, as determined on the decoder side. Therefore, in these cases where the decoder cannot positively increase the quality level beyond that of conventional straight L/R stereo coding or joint M/S coding, it can at least guarantee that the same is maintained. level. Therefore, a decoder according to this aspect of the invention can, from a functional point of view, be considered as a super set in relation to the prior art.

As an advantage over the QMF-based prediction-encoded stereo signal, a perfect reconstruction of the signal is possible (apart from quantization errors, which can be made arbitrarily small). Thus, the invention provides a coding apparatus for transform-based stereo coding by complex prediction. Preferably, the apparatus according to the invention is not limited to complex prediction stereo coding, but is also operable in a direct L/R stereo coding or joint M/S stereo coding regime according to the prior art, so that it is possible to select the most suitable encoding method for a specific application or during a specific time interval. An oversampled (eg complex) representation of the signal, including both said first and said second spectral components, is used as the basis for complex prediction according to the invention, and therefore the modules for calculating said oversampled representation. are arranged in the decoder system according to the invention. Spectral components refer to the first and second subspaces of a multidimensional space, which can be the set of time-dependent functions in an interval of a given length (for example, a predefined time frame length) sampled at a frequency of finite sampling. It is well known that functions in this concrete multidimensional space can be approximated by a finite weighted sum of basis functions.

As a person skilled in the art will appreciate, an encoder adapted to cooperate with a decoder with equivalent modules is equipped to provide the oversampled representation on which the prediction coding is based, to thereby allow faithful reproduction of the encoded signal. . Said equivalent modules may be identical or similar modules or modules having identical or similar transfer characteristics. In particular, the modules in the encoder and decoder, respectively, may be similar or different processing units running respective computer programs that perform equivalent sets of mathematical operations.

In some embodiments of the decoder system, the downmix signal and the residual signal are segmented into time frames, and the upmix stage is further adapted to receive, for each time frame, a two-bit data field associated with that frame and to operate, in response to the value of the data field, in its active mode or in its walkthrough mode.

In some embodiments of the decoder system, the downmix signal and the residual signal are segmented into time frames, and the upmix stage is further adapted to receive in an MPEG bit stream, for each time frame, a ms_mask_present field associated with that frame and to operate, in response to the value of the ms_mask_present field, on its active mode or in its passing mode.

In some embodiments of the decoder system, the first spectral components have real values expressed in the first subspace, and the second spectral components have imaginary values expressed in the second subspace. The first and second components together form a complex spectral representation of the signal. The first subspace may be the linear space of a first set of basis functions, while the second subspace may be the linear space of a set of second basis functions, some of which are linearly independent of the first set of basis functions.

In one embodiment, the module for computing the complex representation is a real-to-imaginary transform, that is, a module for computing the imaginary parts of the spectrum of a discrete-time signal based on a real spectral representation of the signal. The transform can be based on exact or approximate mathematical relationships, such as harmonic analysis formulas or heuristic relationships.

In some embodiments of the decoder system, the first spectral components are obtained by a time-to-frequency domain transform, preferably a Fourier transform, of a discrete time-domain signal, such as by a discrete cosine transform. (DCT), a modified discrete cosine transform (MDCT), a discrete sine transform (DST), a modified discrete sine transform (MDST), a fast Fourier transform (FFT), a factor-based Fourier algorithm cousin or similar. In the first four cases, the second spectral components can then be obtained by DST, MDST, DCT, and MDCT, respectively. As is well known, the linear space of cosines that are periodic in the unit interval forms a subspace that is not entirely contained in the linear space of periodic sines in the same interval. Preferably, the first spectral components are obtainable by MDCT and the second spectral components are obtainable by MDCt.

In one embodiment, the decoder system includes at least one temporal noise shaping module (TNS module, or TNS filter), which is fitted above the upmix stage. Generally speaking, the use of TNS increases the perceived audio quality for signals with transient components, and this also applies to embodiments of the inventive decoder system featuring TNS. In conventional L/R and M/S stereo coding, the TNS filter can be applied as a last processing step in the frequency domain, directly before the inverse transform. In the case of complex prediction stereo coding, however, it is often advantageous to apply the TNS filter on the downmix and residual signals, ie before the upmix matrix. In other words, the TNS is applied to the linear combinations of the left and right channels, which has several advantages. Firstly, it may result in a given situation that the TNS is beneficial only for the downmix signal. Then, the TNS filtering can be suppressed or omitted for the residual signal and, which may imply a more economical use of the available bandwidth, the TNS filter coefficients only need to be transmitted for the downmix signal. Second, the computation of the oversampled representation of the downlink signal (for example, the MDST data that is derived from the MDCT data to form a complex frequency domain representation), which is necessary in complex prediction coding, it may require that the time-domain representation of the downmix signal be computable. This in turn implies that the downmix signal is preferably available as a time sequence of the MDCT spectrum obtained in a uniform manner. If the TNS filter were applied in the decoder after the upmix matrix, which converts a downmix/residual representation to a left/right representation, only a sequence of the TNS residual MDCT spectrum of the downmix signal would be available. This would make the efficient calculation of the corresponding MDST spectrum very challenging, especially if the left and right channels are using TNS filters with different characteristics.

It is emphasized that the availability of a time sequence of the MDCT spectrum is not an absolute criterion for obtaining a fitted MDST representation to serve as a basis for complex prediction coding. Apart from the experimental evidence, this fact can be explained by the TNS generally applying only to the higher frequencies, such as above a few kilohertz, so the residual signal filtered by the TNS corresponds approximately to the signal residual unfiltered for lower frequencies. Thus, the invention can be implemented as a decoder for complex prediction stereo coding, in which the TNS filters have a different placement than above the upmix stage, as indicated below.

In one embodiment, the decoder system includes at least one additional TNS module located below the upmix stage. By means of a selector arrangement, either the TNS module(s) above the upmix stage or the TNS module(s) below the upmix stage. Under certain circumstances, computation of the complex frequency domain representation does not require a time domain representation of the downmix signal to be computable. Furthermore, as discussed above, the decoder may be selectively operable in a direct or joint coding mode, not applying complex prediction coding, and then it may be more appropriate to apply location conventional of TNS modules, that is, as one of the last processing steps in the frequency domain.

In one embodiment, the decoder system is adapted to economize processing resources, and possibly power, by disabling the module to calculate a second frequency-domain representation of the downmix signal when the latter is not needed. The downmix signal is assumed to be split into successive time blocks, each of which is associated with a complex prediction coefficient value. This value can be determined by a decision made for each time block by an encoder cooperating with the decoder. Furthermore, in this embodiment, the module for calculating a second frequency-domain representation of the downmix signal is adapted to turn itself off if, during a given time block, the absolute value of the imaginary part of the coefficient complex prediction tolerance is zero or is less than a predetermined tolerance. Deactivation of the module may mean that a second frequency domain representation of the downmix signal is not calculated for this time block. If deactivation does not occur, the second frequency-domain representation (for example, a set of MDST coefficients) would be multiplied by zero or by a number of substantially the same order of magnitude as the machine epsilon (rounding unit) of the decoder or some other suitable threshold value.

In a further development of the above embodiment, saving of processing resources is achieved at a sub-level of the time block within which the downmix signal is split. For example, said sub-level within a time block may be a frequency band, where the encoder determines a complex prediction coefficient value for each frequency band within a time block. Similarly, the module for producing a second frequency-domain representation is adapted to eliminate its operation for a band of frequencies in a time block for which the complex prediction coefficient is zero or has a magnitude less than one. tolerance.

In one embodiment, the first spectral components are transform coefficients arranged in one or more time blocks of the transform coefficients, each block generated by applying a transform to a time slice of a time domain signal. Furthermore, the module for calculating a second frequency-domain representation of the downmix signal is adapted to:

• derive one or more first intermediate components from at least some of the spectral components;

• forming a combination of said one or more first spectral components according to at least a part of one or more impulse responses to obtain one or more second intermediate components; and

• deriving said one or more second spectral components from said one or more second intermediate components.

This procedure achieves the computation of the second frequency domain representation directly from the first frequency domain representation, as described in more detail in US 6,980,933 B2. , notably in columns 8-28 and specifically in equation 41 thereof. As skilled persons will realize, the computation is not performed through the time domain, unlike, for example, the inverse transform followed by a different transform.

For an exemplary transformation of complex prediction stereo coding according to the invention, it has been estimated that the computational complexity increases only slightly (significantly less than the increase caused by complex prediction stereo coding in the QMF domain) compared to stereo Traditional L/R or M/S. Such an implementation that includes the exact calculation of the second spectral components introduces a delay that is typically a few percent longer than that introduced by an implementation based on q Mf (assuming the length of the time block is 1024 samples). and comparing to the delay of the hybrid QMF analysis/synthesis filter bank, which is 961 samples).

Suitably, in at least some of the above embodiments, the impulse responses are adapted to the transform by which the first representation in the frequency domain is obtainable, and more specifically according to the characteristic frequency response of the same.

In some embodiments, the first frequency-domain representation of the downmix signal is obtained by a transform that is applied in connection with one or more analysis window functions (or slice functions, for example, the rectangular window , the sinusoidal window, the Kaiser-Bessel-derived window, etc), one goal of which is to achieve time segmentation without introducing a detrimental amount of noise or changing the spectrum in an undesirable way. Possibly, said window functions partially overlap. Then, preferably, the frequency response characteristics of the transform are dependent on the characteristics of said one or more analysis window functions.

Still referring to the embodiments that perform the calculation of the second representation in the frequency domain Within the frequency domain, it is possible to decrease the computational load involved by using a second approximate representation in the frequency domain. Such an approximation can be achieved by not requiring the complete information on which to base the calculation. By the teachings of US 6,980,933 B2, for example, the first frequency-domain data of three time blocks is required for the exact calculation of the second frequency-domain representation of the downmix signal in a block, ie a block contemporaneous with the output block, a pre-block and a post-block. For the purpose of complex prediction coding according to the present invention, suitable approximations can be obtained by omitting - or replacing by zero - the data that is obtained from the subsequent block (whereby the modulo operation may be causal, i.e. , does not contribute to the delay) and/or from the previous block, for the calculation of the second frequency domain representation is based on the data of one or more time blocks only. It is observed that even if the omission of the input data implies a rescaling of the second representation in the frequency domain - in the sense that, for example, it no longer represents the same energy - it can already be used as a basis for complex prediction coding provided it is computed in an equivalent manner at both the encoder and decoder ends, as noted above. Indeed, a possible rescaling of this type will be compensated by the corresponding change in the value of the prediction coefficient.

Still another approximate method for calculating the spectral component that is part of the second frequency domain representation of the downmix signal may include combining at least two components of the first frequency domain representation. The latter components may be adjacent with respect to time and/or frequency. Alternatively, they can be combined using finite impulse response (FIR) filtering, with relatively few pulses. For example, in a system that applies a time block size of 1024, such FIR filters may include 2, 3, 4 etc. ticks. Descriptions of approximate calculation methods of this nature can be found, for example, in US 2005/0197831 A1. If a window function is used that gives relatively less weights to the surroundings of each time block boundary, for example a non-rectangular function, it may be convenient to base the second spectral components in a time block only on the combinations of the first spectral components in the same time block, which implies that the same amount of information is not available for the outermost components. The approximation error possibly introduced by such practice is to some extent suppressed or canceled by the shape of the window function.

In one embodiment of a decoder, which is designed to output a stereo signal in the time domain, a switchability between direct or joint coding and complex prediction coding is included. This is achieved by providing:

• a switch that can be selectively operated either as a step stage (which does not modify the signals) or as a sum and difference transform;

• an inverse transform stage to perform a frequency-to-time transform; and

• a selector arrangement for feeding the inverse transform stage with either a direct (or joint) coded signal or with a complex prediction coded signal.

As the skilled person will realize, such flexibility on the decoder side gives the encoder freedom to choose between direct or joint conventional coding and complex prediction coding. Therefore, in cases where the quality level of conventional direct L/R stereo coding or joint M/S stereo coding cannot be exceeded, this embodiment can at least guarantee that the same level is maintained. Therefore, the decoder according to this embodiment can be considered as a super set with respect to the related art.

Another group of embodiments of the decoder system performs the calculation of the second spectral components in the second representation in the frequency domain through the time domain. More precisely, the inverse of the transform by which the first spectral components were obtained (or are obtainable) is applied and is followed by a different transform outputting the second spectral components. Specifically, the reverse MDCT can be followed by an MDST. To reduce the number of transforms and inverse transforms, the output of the inverse MDCt may, in such an embodiment, be provided to both the MDST and the output terminals (possibly preceded by additional processing steps) of the decoding system.

For an exemplary implementation of complex prediction stereo coding according to the invention, it has been estimated that the computational complexity increases only slightly (still significantly less than the increase caused by complex prediction stereo coding in the QMF domain) compared to L stereo. /R or conventional M/S.

As a further development of the embodiment referred to in the previous paragraph, the upmix stage may comprise a further inverse transform stage for processing the side signal. The sum and difference stage is then supplied with a time domain representation of the side signal, generated by said sum and difference stage. additional inverse transform stage, and a time-domain representation of the downmix signal, generated by the already referred to inverse transform stage. It is recalled that, advantageously from the point of view of computational complexity, the last signal is supplied to both the sum and difference stage and to said different transform stage referred to the previous one.

In one embodiment, a decoder designed to output a time-domain stereo signal includes the ability to switch between straight L/R stereo encoding, joint M/S stereo encoding, and complex prediction stereo encoding. This is achieved by providing:

• a switch operable either as a step stage or as a sum and difference stage;

• an additional inverse transform stage to compute a time domain representation of the side signal;

• a selector arrangement for connecting the inverse transform stages to either a further sum and difference stage connected to a point above the upmix stage and below the switch (preferably where the switch has been actuated to function as a pass filter, as may be the case when decoding a stereo signal generated by complex prediction coding) or a combination of a downmix signal from the switch and a side signal from the weighted adder (preferably when the switch has been turned on). driven to function as a sum and difference stage, as may be the case when decoding as a directly encoded stereo signal).

As skilled persons will realize, this gives the encoder the freedom to choose between conventional straight or joint coding and complex prediction coding which means that a level of quality at least equivalent to that of straight stereo coding can be guaranteed. or joint,

An encoder system (not covered by the claims) may comprise an estimator for estimating the complex prediction coefficient with the aim of reducing or minimizing the signal power or mean signal power of the residual signal. The minimization can take place over a time interval, preferably a time slot or a time block or a time frame of the signal to be encoded. The square of the amplitude can be taken as a measure of the momentary energy of the signal, and an integral over the time interval of the squared amplitude (waveform) can be taken as a measure of the energy of the signal. average signal in that interval. Suitably, the complex prediction coefficient is determined on a frequency band and time block basis, that is, its value is set such that it reduces the average energy (i.e., total energy) of the residual signal. in that time block and frequency band. Specifically, modules for estimating parametric stereo coding parameters such as IID, ICC and IPD or the like can provide the output on which the complex prediction coefficient can be calculated according to mathematical relationships known to the skilled person. .

The encoding stage of the encoding system may further be operable to function as a pass-through stage to allow direct stereo encoding. By selecting straight stereo encoding in situations where it is expected to provide higher quality, the encoding system can ensure that the encoded stereo signal has at least the same quality as straight encoding. Similarly, in situations where the increased computational effort incurred by complex prediction coding is not motivated by a significant increase in quality, there is therefore an option available to save computational resources easily for the coding system. The decision between joint, direct, real-prediction, and complex-prediction coding at the encoder is generally based on rate/distortion optimization.

The encoder system may comprise a module for directly calculating a second frequency-domain representation (that is, without applying an inverse intra-time-domain transform and without using the time-domain data of the signal) based on the first spectral components. In relation to the corresponding embodiments of the decoder system described above, this module may have an analogous structure, i.e. comprise analogous processing operations but in a different order, whereby the encoder is adapted to output the appropriate data as input to the encoder. decoder side. For illustrative purposes, it is assumed that the stereo signal to be encoded comprises the center and side channels, or has been transformed into this structure, and the encoding stage is adapted to receive a first frequency domain representation. The coding stage comprises a module for calculating a second frequency domain representation of the central channel. (The first and second frequency domain representations referred to herein are as defined above; in particular, the first frequency domain representations may be MDCT representations and the second frequency domain representation may be an MDST representation. ). The encoding stage further comprises a weighted adder for calculating the residual signal as a linear combination formed from the side signal and the two frequency-domain representations of the center signal weighted by the real and imaginary parts, respectively, of the signal. complex prediction coefficient. The central signal, or suitably the first frequency domain representation thereof, can be used directly as a downmix signal. In this embodiment, furthermore, the estimator determines the value of the complex prediction coefficient with the aim of minimizing the energy or average energy of the residual signal. The final operation (optimization) can be performed either by feedback control, where the estimator can receive the residual signal obtained by the current prediction coefficient values to be further adjusted if necessary, or, in a manner of fed forward, by calculations made directly or into the left/right channels of an original stereo signal or the center/side channels. The feed-forward method is preferred, whereby the complex prediction coefficient is determined directly (ie, in a non-iterative or non-feedback manner) based on the first and second frequency-domain representations of the center signal and the first frequency domain representation of the side signal. It is noted that the determination of the complex prediction coefficient can be followed by a decision to either apply direct, joint, real prediction, or complex prediction coding, where the resulting quality (preferably perceptual quality, taking into account , for example, the signal-to-mask effects) of each available option; therefore the previous statements are not interpreted with the effect that there is any feedback mechanism in the encoder.

The encoder system may comprise the modules for calculating a second frequency domain representation of the center (or downmix) signal through the time domain. It is understood that the implementation details in relation to said encoder system, at least as far as the computation of the second frequency-domain representation is concerned, are similar or analogously resolvable to the corresponding decoder implementations. . The coding stage comprises:

• a sum and difference stage for converting the stereo signal into a form comprising center and side channels;

• a transform stage to provide a frequency domain representation of the side channel and a frequency domain representation of the complex value (and hence oversampled) of the center channel; and

• a weighted adder to compute a residual signal, where the complex prediction coefficient is used as a weight.

Here, the estimator can receive the residual signal and determine, possibly in a feedback control mode, the complex prediction coefficient to reduce or minimize the energy or mean of the residual signal. Preferably, however, the estimator receives the stereo signal to be coded and determines the prediction coefficient based on this. It is advantageous from a computational economics point of view to use a critically sampled frequency domain representation of the side channel, since the latter will not be subject to multiplication by a complex number in this embodiment. Suitably, the transform stage may comprise an MDCT stage and a parallel MDST stage, both having the center channel time-domain representation as input. Thus, an oversampled frequency domain representation of the center channel and a critically sampled frequency domain representation of the side channel are produced.

It is noted that the decoding methods and decoding apparatus described in this section can be applied, after appropriate modifications within the capabilities of the skilled person including routine experimentation, for the encoding of signals having more than two channels. Modifications in such multi-channel operability can proceed, for example, along the lines of sections 4 and 5 in the J. Herre et al. above.

The features of the two or more embodiments of a decoder system indicated above may be combined, unless they are clearly complementary, in additional embodiments. The fact that two features are recited in different claims does not prevent them from being combined as an advantage. Also, additional embodiments may provide for the omission of certain features that are not necessary or essential for the intended purpose. As an example, the decoding system according to the invention can be realized without a dequantization stage in cases where the encoded signal to be processed is not quantized or is already available in a form suitable for processing by the mixing stage. upward.

Brief description of the drawings

The invention will now be further illustrated by the embodiments of various decoding systems and decoding methods described in the following section, reference being made to the accompanying drawings. The drawings also illustrate various encoding systems and an encoding method, which appear for illustrative purposes only and are not covered by the claims.

In the drawings:

Figure 1 consists of two generalized block diagrams showing decoders based on the QMF according to the prior art;

Figure 2 is a generalized block diagram of an MDCT-based stereo decoder system with complex prediction, according to an embodiment of the present invention, in which the complex representation of a channel of the signal to be decoded is computed in the domain of the frequency;

Figure 3 is a generalized block diagram of an MDCT-based stereo decoder system with complex prediction, according to an embodiment of the present invention, in which the complex representation of a channel of the signal to be decoded is computed in the domain weather;

Figure 4 shows an alternative embodiment of the decoder system of Figure 2, in which the location of the active TNS stage is selectable;

Figure 5 comprises generalized block diagrams showing MDCT-based stereo coder systems with complex prediction;

Figure 6 is a generalized block diagram of a complex prediction MDCT-based stereo encoder, in which a channel representation of the signal to be encoded is computed based on the time domain representation thereof;

Figure 7 shows an alternative to the encoding system of Figure 6, which is also operable in a direct L/R encoding mode;

Figure 8 is a generalized block diagram of an MDCT-based stereo encoder system with complex prediction, in which a one-channel representation of the signal to be encoded is computed based on a first frequency-domain representation. thereof, which decoder system is also operable in a direct L/R coding mode;

Figure 9 shows an alternative to the encoder system of Figure 7, further including a TNS stage arranged below the encoding stage;

Figure 10 shows alternatives to the part labeled A in Figures 2 and 8;

Figure 11 shows an alternative to the encoder system of Figure 8, further including two frequency domain modification devices arranged respectively below and above the encoding stage;

Figure 12 is a graphical representation of 96 kb/s listening test results from six subjects showing different complexity-quality trade-off options for the MDST spectrum calculation or approximation, where the points Data labels labeled “+” refer to hidden reference, “x” refers to 3.5 kHz band limited clamp, “*” refers to USAC traditional stereo (M/S or L/R), “□ ” refers to unified stereo coding in the MDCT domain using complex prediction with the imaginary part of the prediction coefficient disabled (that is, real value prediction, which does not require MDST), “■” refers to stereo coding unified in the MDCT domain by complex prediction using a current MDCT frame to calculate an approximation of the MDST, “o” refers to unified stereo coding in the MDCT domain by complex prediction using the current and previous MDCT frames to calculate an approximation of MDST and “• ” refers to unified stereo coding in the MDCT domain by complex prediction using the current, previous, and next MDCT frames to compute the MDST; Figure 13 presents the data of Figure 12, however as differential scores relative to unified stereo coding in the MDCT domain using complex prediction using the current MDCT frame to compute an approximation of the MDST;

Fig. 14 comprises generalized block diagrams showing three embodiments of a decoder system according to embodiments of the invention;

Fig. 15 is a flowchart showing a decoding method according to an embodiment of the invention; and

Fig. 16 is a flowchart showing a coding method.

Detailed description of embodiments

I. Decoding systems

Figure 2 shows, in the form of a generalized block diagram, a decoding system for decoding a bit stream comprising at least one value of a complex prediction coefficient a = aR iai and an MDCT representation of a stereo signal. which has a downmix M and residual D channels. The real and imaginary parts aR, ai of the prediction coefficient can be quantized and/or coded accordingly. joint way. Preferably however, the real and imaginary parts are quantized independently and uniformly, typically with a step size of 0.1 (dimensionless number). The resolution of the frequency band used for the complex prediction coefficient is not necessarily the same as the resolution for the scale factor bands (sfb; that is, a group of MDCT lines using the same size of quantization step and quantization range of the MDCT) according to the MPEG standard. In particular, the frequency band resolution for the prediction coefficient may be one that is psychoacoustically justified, such as the Bark scale. The demultiplexer 201 is adapted to extract these MDCT representations and the prediction coefficient (part of the Control Information as indicated in the figure) from the bit stream supplied to it. In effect, more control information than just the complex prediction coefficient can be encoded in the bit stream, for example instructions whether the bit stream is to be decoded in prediction or non-prediction mode, information TNS etc The TNS information may include TNS parameter values to be applied by the TNS (synthesis) filters of the decoder system. If identical TNS parameter sets are to be used for several TNS filters, such as for both channels, it is economical to receive this information in the form of a bit indicating said identity of the parameter sets instead of receiving both parameter sets. independently. Information may also be included on whether to apply the TNS before or after the upmix step, as appropriate based on, for example, a psychoacoustic evaluation of the two available options. In addition, the control information can then indicate limited bandwidths individually for the downmix and for the residual signals. For each channel, frequency bands above a bandwidth limit will not be decoded but will be set to zero. In certain cases, the higher frequency bands have such a low energy content that they are quantized as zero. The normal practice (cf, the max_sfb parameter in the MPEG standard) has been to use the same bandwidth limitation for both the downmix and residual signals. However, the residual signal, to a greater extent than the downmix signal, has its energy content located in the lower frequency bands. Therefore, by placing a dedicated upper bandwidth limit on the residual signal, bit rate reduction is possible with no significant quality loss. For example, this can be determined by two independent max_sfb parameters encoded in the bitstream, one for the downmix signal and one for the residual signal.

In this embodiment, the MDCT representation of the stereo signal is segmented into successive time frames (or time blocks) comprising a fixed number of data points (for example, 1024 points), one of several fixed numbers of data points. data points (for example, 128 or 1024 points) or a variable number of points. As is known to those skilled in the art, the MDCT is critically sampled. The output of the decoding system, indicated on the right side of the drawing, is a time-domain stereo signal having left L and right R channels. Dequantization modules 202 are adapted to handle the bitstream input to the system. or, where appropriate, two bitstreams obtained after demultiplexing of an original bitstream and corresponding to each of the downmix and residual channels. The dequantized channel signals are provided to a switching assembly 203 operable in either a step mode or a sum and difference mode corresponding to the respective transformation matrices.

and

As will be further discussed in the next paragraph, the decoder system includes a second switch assembly 205. Both switch assemblies 203 and 205, like many other switches and switch assemblies, are operable in a frequency selective manner. This allows decoding of a wide variety of decoding modes, eg, frequency dependent L/R or M/S decoding, as is known in the related art. Therefore, the decoder according to the invention can be considered a super set in relation to the related art.

Assuming for now that the switching assembly 203 is in pass-through mode, the dequantized channel signals are, in this embodiment, passed through the respective TNS filters 204. The TNS filters 204 are not essential in the operation of the decoding system and may be replaced by pass elements. After this, the signal is supplied to the second switch assembly 205 which has the same functionality as the switch assembly 203 located above. With the input signals as described above and with the second switch assembly 205 set to its pass-through mode, the output of the former is the downmix channel and residual channel signal. The downmix signal, still represented by its successive MDCT spectra in time, is supplied to a real-to-imaginary transform 206 adapted to calculate, based on this, the MDST spectra of the downmix signal. In this embodiment, an MDST frame is based on three MDCT frames, a previous frame, a current (or contemporaneous) frame, and a subsequent frame. It is symbolically indicated (Z-1,Z) that the input side of the real-to-imaginary transform 206 comprises the delay components.

The MDST representation of the downmix signal obtained from the real-to-imaginary transform 206 is weighted by the imaginary part aI of the prediction coefficient and added to the MDCT representation of the downmix signal weighted by the real part aR of the prediction coefficient and the MDCT representation of the residual signal. The two additions and multiplications are performed by the multipliers and adders 210, 211, which together (functionally) form a weighted adder, which is supplied with the value of the complex prediction coefficient a encoded in the bit stream initially received by the decoder system. The complex prediction coefficient can be determined once per time frame. It can also be determined more often, such as once per time frame. It can also be determined more often, such as once per frequency band within a frame, the frequency bands being psychoacoustically motivated partitioning. It can also be determined less frequently, as discussed above in connection with coding systems provided for illustrative purposes only and not covered by the claims. The real-to-imaginary transform 206 is synchronized with the weighted adder such that a current MDST frame of the downmix channel signal is combined with contemporaneous MDCT frames of each downmix channel signal and signal of the residual channel. The sum of these three signals is a side signal S = Re{aM} D. In this expression, M includes both the MDCT and MDST representations of the downmix signal, that is, M = M ^mdct - ÍM ^mdst , while D = D ^mdct is real valued. Thus, a stereo signal is obtained having a downmix channel and a side channel, from which a sum and difference transform 207 restores the left and right channels as follows:

( L = MS

he R = M - S

These signals are represented in the MDCT domain. The last step of the decoding system is to apply an inverse MDCT 209 to each of the channels, whereby a time domain representation of the left/right stereo signal is obtained.

A possible implementation of the real-to-imaginary transform 206 is further described in Applicant's US Patent 6,980,933 B2, as noted above. Using formula 41, the transform can be expressed as a finite impulse response filter, for example, for even points,

n-l

S(2v) = ^ [ ( - D P+1^ (P ) + Xl ll ( p')]hlj„ ( 2 v - V)

p = 0

^N -1

+ i í X i=0 * " C 2 í i ) M 2 v - 2 i - i ) '

where S(2v) is the 2vth MDST data point, Xi, Xii, Xiii are the MDCT data of each of the frames and N is the length of the frame. Furthermore, hi, ⁱⁱⁱ , hii are impulse responses that depend on the applied window function, and are therefore determined for each choice of window function, such as rectangular, sinusoidal, and Kaiser-Bessel-derivative, and for each length of plot. The complexity of this calculation can be reduced by omitting those impulse responses that have relatively lower energy content and contribute relatively less to the MDST data. As an alternative or extension to this simplification, the impulse responses themselves can be shortened, for example, from the total frame length N to a smaller number of points. As an example, the impulse response length can be decreased from 1024 points (beats) to 10 points. The most extreme truncation that can be considered significant is

Other direct approaches can be found in US 2005/0197831 A1.

It is also possible to reduce the amount of input data on which the calculation is based. To illustrate, the real-to-imaginary transform 206 and its connections above, which are indicated as the part denoted by "A" in the drawing, can be replaced by a simplified variable, two of which A' and A'' are shown in Figure 10. The variable A' provides an approximate imaginary representation of the signal. Here, the MDST calculation only takes the current frame and the next one into account. Referring to the formula above this paragraph, this can be done by setting X ⁱⁱⁱ ( ^p ) = 0 for p = 0, ..., N-1 (index III denotes the next time frame). Since the variable A' does not require the MDCT spectrum of the next frame as input, the calculation of the MDST does not incur any time delay. Clearly, this approximation somewhat reduces the precision of the obtained MDST signal, but it may also imply that the energy of this signal is reduced; the latter fact can be compensated for by an increase in ai as a result of the nature of the prediction coding.

Also shown in Figure 10 is the variable A'', which uses as input only the MDCT data for the current time frame. The variable A'' probably produces a less accurate representation of the MDST than the variable A'. On the other hand, it works with zero delay, like the variable A', and has less computational complexity.

As already mentioned, the properties of the waveform coding are not affected as long as the same approximation is used in the encoder system and in the encoder system.

Note, regardless of whether the variable A, A' or A'' or any further development thereof is used, that only those parts of the MDST spectrum for which the imaginary part of the complex prediction coefficient is not zero ai t 0 need to be computed. In practical circumstances this will mean that the absolute value |ai| of the imaginary part of the coefficient is greater than a predetermined threshold value, which may be related to unit rounding of the used hardware. In case the imaginary part of the coefficient is zero for all frequency bands within a time frame, there is no need to compute any MDST data for that frame. Thus, appropriately, the real-to-imaginary transform 206 adapts to respond to occurrences of very small values of |ai| not generating the output of the MDST, whereby the calculation resources can be saved. In implementations where more frames than the current one are used to produce an MDST data frame, however, any units above transform 206 should continue to function properly even if no MDST spectrum is needed - specifically, the second switching assembly 205 should continue to send MDCT spectra - so that sufficient input data is available for the real-to-imaginary transform 206 already when the next time frame associated with a non-zero prediction coefficient occurs; this can of course be the next block of time.

Returning to Figure 2, the function of the decoding system has been described under the assumption that both switch assemblies 203, 205 are set to their respective pitch modes. As will now be discussed, the decoding system may also decode signals that are not prediction encoded. For this use, the second switch assembly 205 will be set in its sum and difference mode and suitably, as indicated in the drawing, a selector arrangement 208 will be set in its lower position, thus ensuring that the signals are provided. directly to the inverse transform 209 from an origin point between the TNS filters 204 and the second switching assembly 205. To ensure correct decoding, the proper signal has an L/R shape at the point of origin. Therefore, to ensure that the real-to-imaginary transform is supplied with the correct center (i.e., downmix) signal at all times (instead of, intermittently, by a left signal), the second assembly 205 of switching is preferably set to its sum and difference mode during decoding of a prediction uncoded stereo signal. As noted above, prediction coding can be replaced by conventional forward or joint coding for certain frames based on, for example, a decision between data rate or audio quality. The result of such a decision can be communicated from the encoder to the decoder in various ways, for example, by the value of a dedicated flag bit in each frame, or by the absence or presence of a prediction coefficient value. Having established these facts, the role of the first switch assembly 203 can be easily achieved. In fact, in the non-prediction coding mode, the decoder system can process both signals according to direct stereo (L/R) coding or joint (M/S) coding, and operating the first switching assembly 203 either in step or sum and difference mode, it is possible to ensure that the point of origin is always provided with a directly encoded signal. Clearly, the switching assembly 203 when functioning as a sum and difference stage converts an input signal in M/S form to an output signal (supplied to optional TNS filters 204) in L/R form.

The decoder system receives a signal whether a particular time frame is to be decoded by the decoder system in the prediction coding mode or the non-prediction coding mode. The non-prediction mode can be signaled by the value of a dedicated indicator bit in each frame or by the absence (or zero value) of the prediction coefficient. The prediction mode can be reported in an analogous manner. A particularly advantageous implementation, which allows backtracking without any overhead, makes use of a fourth reserved value of the two-bit ms_mask_present field (see MPEG-2 AAC, document ISO/IEC 13818-7), which is transmitted per time frame and is defined as follows:

By redefining the value 11 to mean "complex prediction encoding", the decoder can operate in all traditional modes, namely M/S and L/R encoding, without any penalty and is still capable of receiving a signal indicating the mode of prediction. complex prediction coding for the relevant frames.

Figure 4 shows a decoder system of the same general structure as that shown in Figure 2 but including, however, at least two different structures. First, the system of Figure 4 includes switches 404, 411 that allow the application of some processing step involving frequency domain modification above and below the upmix stage. This is achieved, on the one hand, by means of a first set of modifiers 403 in the frequency domain (drawn with TNS synthesis filters in this figure) provided together with the first switch 404 below the dequantization modules 401 and a first switching assembly 402 but above a second switching assembly 405 disposed immediately above the upmix stage 406, 407, 408, 409. On the other hand, the decoder system includes a second set of frequency domain modifiers 410 provided in conjunction with a second switch 411 below the upmix stage 406, 407, 408, 409 but above an upmix stage 412. inverse transform. Advantageously, as indicated in the drawing, each frequency domain modifier is arranged in parallel with a feed line which is connected above the input side of the frequency domain modifier and connected below. of the associated switch. By virtue of this structure, the frequency domain modifier is supplied with the signal data at all times, allowing frequency domain processing based on more time frames than just the current one. The decision whether to apply the first 403 or the second set of modifiers 410 in the time domain may be made by the encoder (and carried in the bitstream), or may be based on whether prediction coding is applied, or it may be based on some other criterion found suitable in the practical circumstances. As an example, if the frequency domain modifier is TNS filters, then it may be advantageous to use the first set 403 for some types of signals, while the second set 410 may be advantageous for other types of signals. If the result of this selection is encoded in the bitstream, then the decoder system will therefore activate the respective set of TNS filters.

To facilitate understanding of the decoder system shown in Figure 4, it is explicitly noted that decoding of a direct encoded signal (L/R) takes place when a = 0 (implying that the pseudo-L/R and L/R are identical and that the side and residual channels do not differ), the first switch assembly 402 is in the pass mode, the second switch assembly is in the sum and difference mode, thereby causing the signal is in the form M/S between the second switching assembly 405 and the sum and difference stage 409 of the upmix stage. Since the upmix stage will effectively be a pass stage, it is immaterial whether (using the respective switches 404, 411) the first or second set of frequency domain modifiers is activated.

Figure 3 illustrates a decoder system according to one embodiment of the invention which, in relation to Figures 2 and 4, represents a different approach to the provision of the MDST data required for upmixing. Like the decoder systems already described, the system of Figure 3 comprises dequantization modules 301, a first switching assembly 302 operable in either a step or sum and difference mode, and TNS (synthesis) filters 303, which they are arranged in series from the input end of the decoder system. Modules below this point are used selectively by means of two second switches 305, 310, which are preferably co-operated so that both are either in their upper or lower positions, as indicated. in the figure. At the output end of the decoder system, there is a sum and difference stage 312 and, immediately above this, two inverse MDCT modules 306, 311 for transforming an MDCT-domain representation of each channel into an inverse-domain representation of each channel. weather.

In complex prediction decoding, where the decoder system is supplied with a bitstream encoding a stereo downmix/residual signal and complex prediction coefficient values, the first switching assembly 302 is set to its pass mode. and the second switches 305, 310 are set in their upper position. Below the TNS filters, the two channels of the stereo signal (dequantized, TNS filtered, MDCT) are processed in different ways. The downmix channel is provided, on the other hand, to a multiplier and adder 308, which adds the MDCT representation of the downmix channel weighted by the real part aR of the prediction coefficient to the MDCT representation of the residual channel, and, on the other hand, to one 306 of the MDCT transform modules. The time domain representation of the downmix channel M, which is the output of the inverse MDCT transform module 306, is supplied to both the final sum and difference stage 312 and the MDST transform module 307. This dual use of the time-domain representation of the downmix channel is advantageous from the standpoint of computational complexity. The thus obtained MDST representation of the downmix channel is supplied to a further multiplier and adder 309, which after weighting by the imaginary part ai of the prediction coefficient adds this signal to the linear combination output from the adder 308; therefore, the output of adder 309 is a side channel signal, S = Re{aM} D. Similar to the decoder system shown in Figure 2, multipliers and adders 308, 309 can easily be combined to form an adder. multi-signal weighted with inputs the MDCT and MDST representations of the downmix signal, the MDCT representation of the residual signal, and the value of the complex prediction coefficient. Below this point in the present embodiment, only one pass through the MDCT transform module 311 is maintained before the side channel signal is supplied to the sum and difference final stage 312.

The necessary synchronization in the decoder system can be achieved by applying the same transform lengths and window shapes to both inverse MDCT transform modules 306, 311, as is already the practice in frequency selective M/S and L/R coding. A one frame delay is introduced by combining certain embodiments of the inverse MDCT module 306 and embodiments of the MDST module 307. Therefore, five optional delay blocks 313 (or software instructions to this effect in a computer implementation) are provided, whereby the part of the system located to the right of the dashed line can be delayed by one frame relative to the left side when necessary. Apparently all intersections between the dashed line and the connecting lines are provided with delay blocks, with the exception of the connecting line between the inverse MDCT module 306 and the MDST transform module 307, which is where the delay arises. requires compensation.

The computation of the MDST data for a time frame requires the data of a time domain representation frame. However, the inverse MDCT transform is based on one (current), two (preferably previous and current) or three (preferably previous, current and subsequent) consecutive frames. By virtue of the well-known time-domain aliasing cancellation (TDAC) associated with MDCT, the three-frame option achieves complete overlap of the input frames and thus provides the best (and possibly perfect) accuracy. , at least in the frames that contain the aliasing effect in the time domain. Clearly, the three-frame inverse MDCT works with one frame of delay. By agreeing to use a rough time domain approximation as input to the MDST transform, this delay and thus the need to compensate for delays between different parts of the decoder system can be avoided. In the two-frame option, the overlap/addition that TDAC allows occurs in the first half of the frame, and the overlap effect may only be present in the last half. In the one frame option, the absence of the TDAC implies that the aliasing effect can occur in the whole frame; however, an MDST representation achieved in this way, and used as an intermediate signal in complex prediction coding, can still provide satisfactory quality.

The decoding system illustrated in Figure 3 can also be operated in two non-predictive decoding modes. To decode an L/R encoded stereo signal directly, the second switches 305, 310 are set to the down position and the first switch assembly 302 is set to pass-through mode. Thus, the signal has the L/R form above the sum and difference stage 304, which converts it to the M/S form, after which an inverse MDCT transform and a final sum and difference operation take place. To decode a stereo signal provided in the jointly M/S encoded form, the first switching assembly 302 is instead set to its sum and difference mode, whereby the signal has the L/R form between the first assembly switching 302 and sum and difference stage 304, which is often more suitable from a TNS filtering point of view than an M/S form would be. The processing below the sum and difference step 304 is identical to that in the case of direct L/R decoding.

Figure 14 is composed of three generalized block diagrams of decoders according to embodiments of the invention. Contrary to the various other block diagrams accompanying this application, a connecting line in Figure 14 can symbolize a multi-channel signal. Specifically, said connecting line may be arranged to transmit a stereo signal comprising left/right, center/side, downmix/residual, pseudo-left/pseudo-right channels and other combinations.

Figure 14A shows a decoder system for decoding a frequency domain representation (indicated, for the purpose of this figure, as an MDCT representation) of an input signal. The decoder system is adapted to provide as its output a time domain representation of a stereo signal, which is generated based on the input signal. To be able to decode an input signal encoded by complex prediction stereo coding, the decoder system is provided with an upmix stage 1410. However, it is also capable of handling an input signal encoded in other formats and possibly switching between various coding formats in time, for example a sequence of time frames encoded by complex prediction coding may be followed by a part of time encoded by direct left/right encoding. The ability of the decoder system to handle the different coding formats is achieved by providing a connecting line (pass) arranged in parallel with said upmix stage 1410. By means of a switch 1411 it is possible to select whether the output of the upmix stage 1410 (lower switch position in the figure) or the raw signal available on the feeder (upper switch position in the figure) has been to supply the decoder modules arranged below. In this embodiment, an inverse MDCT module 1412 is arranged below the switch, which transforms the MDCT representation of a signal into a time domain representation. As an example, the signal supplied to the upmix stage 1410 may be a stereo signal in the form of a downmix/residual. The upmix stage 1410 is adapted to derive a side signal and to perform a sum and difference operation so that a left/right stereo signal is output (in the MDCT domain).

Figure 14B shows a decoder system similar to that of Figure 14A. The present system is adapted to receive a bit stream as its input signal. The bit stream is initially processed by a combined demultiplexer and dequantization module 1420, which provides, as a first output signal, an MDCT representation of a multi-channel stereo signal for further processing, as determined by the position of a switch. 1422 having analogous functionality to switch 1411 of Figure 14A. More precisely, switch 1422 determines if the first output from the demultiplexer and dequantizer is to be processed by an upmix stage 1421 and reverse MDCT module 1423 (lower position) or by reverse MDC ^t module 1423 only (upper position). The combined demultiplexer and dequantization module 1420 outputs the control information as well. In the current case, the control information associated with the stereo signal may include data indicating whether the upper or lower position of switch 1422 is appropriate for decoding the signal or, more abstractly, depending on which encoding format it has been converted to. to decode the stereo signal. The control information may include parameters for adjusting the properties of the upmix stage 1421, eg, a complex prediction coefficient value a used in complex prediction coding as already described above.

Figure 14C shows a decoder system which, in addition to entities analogous to those of Figure 14B, comprises first and second frequency domain modification devices 1431, 1435 arranged respectively above and below stage 1433. upmix. For the purpose of this figure, each frequency domain modification device is illustrated by a TNS filter. However, by the term frequency domain modifying device one could also understand processes other than TNS filtering that are likely to be applied either before or after the upmixing step. Examples of frequency domain modifications include prediction, noise addition, bandwidth extension, and nonlinear processing. Psychoacoustic considerations and similar reasons, possibly including the properties of the signal to be processed and/or the configuration or settings of such a frequency-domain modifying device, sometimes indicate that it is advantageous to apply such modification in the frequency domain above the upmix stage 1433 instead of below it. In other cases, it can be established by similar considerations that the position below the change in the frequency domain is preferable to that above it. By means of switches 1432, 1436, frequency domain modification devices 1431, 1435 can be selectively activated so that, in response to control information, the decoder system can select the desired configuration. As an example, Fig. 14C shows a configuration in which the stereo signal from the combined demultiplexer and dequantization module 1430 is initially processed by a first frequency domain shifter 1431, then supplied to the upmix stage 1433. and is finally sent directly to a reverse MDCT module 1437, without passing through the second frequency domain modification device 1435. As explained in the Summary section, this configuration is preferred over the option to perform the TNS after upmixing in complex prediction coding.

II. encoding systems

This section describes various encoding systems. The description of the various encoding systems is for illustrative purposes only, and the various encoding systems are not covered by the claims.

An encoder system will now be described with reference to Figure 5, which is a generalized block diagram of an encoder system for encoding a left/right (L/R) stereo signal as an output bit stream using complex prediction coding. . The encoder system receives a time-domain or frequency-domain representation of the signal and supplies it to both a downmix stage and a prediction coefficient estimator. The real and imaginary parts of the prediction coefficients are provided to the downmix stage to control the conversion of the left and right channels into downmix and residual channels, which are then supplied to a final MUX multiplexer. If the signal was not supplied as a frequency domain representation to the encoder, it is transformed into such a representation in the downmix stage or multiplexer.

One of the principles of prediction coding is to convert the left/right signal into the center/side form, that is,

and then make use of the remaining correlation between these channels, i.e. setting

S = Re{trM} D,

where a is the complex prediction coefficient to be determined and D is the residual signal. It is possible to choose a so that the energy of the residual signal D = S - Re{aM} is minimized. Energy minimization can be performed with respect to momentary energy, short-term energy, or long-term energy (mean energy), which in the case of a discrete signal is equivalent to optimization in the mean square sense.

The real and imaginary parts aR, ai of the prediction coefficient can be jointly quantized and/or coded. Preferably however, the real and imaginary parts are quantized independently and uniform, typically with a step size of 0.1 (number without dimension). The resolution of the frequency band used for the complex prediction coefficient is not necessarily the same as the resolution for the scale factors of the bands (sfb; that is, a group of MDCT lines using the same quantization step size. and quantization range) according to the MPEG standard. In particular, the frequency band resolution for the prediction coefficient may be one that is psychoacoustically justified, such as the Bark scale. It is noted that the resolution of the frequency band can vary in cases where the length of the transform varies.

As already noted, the encoding system may be free to apply prediction stereo encoding or not, the latter case involving falling back to L/R or M/S encoding. Said decision can be made based on the time frame or more precisely, based on the frequency band within a time frame. As noted above, a negative decision result can be communicated to the decoding entity in various ways, for example, by the value of a dedicated flag bit in each frame, or by the absence (or zero value) of a value of the prediction coefficient. The positive decision can be communicated in an analogous way. A particularly advantageous implementation, which allows fallback without any overhead, makes use of a fourth reserved value of the ms_mask_present two-bit field (see MPEG-2 AAC, document ISO/IEC 131818-7) that is transmitted per time frame and is defined as follows:

By redefining the value 11 to mean "complex prediction coding", the encoder can be operated in all traditional modes, namely M/S and L/R coding, without any penalty and is still capable of complex prediction coding for those plots in which it is advantageous.

The substantive decision can be based on the relationship between data rate to audio quality. As a measure of quality, data obtained using a psychoacoustic model embedded in the encoder (as is often the case with available MDCT-based audio encoders) can be used. In particular, the encoder may provide an optimized selection of the distortion rate of the prediction coefficient. Consequently, the imaginary part - and possibly the real part as well - of the prediction coefficient can be set to zero if the increase in prediction gain does not save enough bits for coding the residual signal to justify the expense of the required bits. to encode the prediction coefficient.

The encoder may encode the TNS related information in the bit stream. Said information may include the values of the TNS parameters to be applied by the TNS (synthesis) filters on the decoder side. If identical TNS parameter sets are to be used for both channels, it is economical to include a signaling bit indicating this identity of the parameter sets instead of transmitting the two parameter sets independently. Information may also be included on whether to apply the TNS before or after the upmix stage, as appropriately based on, for example, the psychoacoustic evaluation of the two available options.

As yet another optional feature, which is potentially beneficial from complexity and bit rate standpoints, the encoder can be adapted to use individually limited bandwidth for encoding the residual signal. Frequency bands above this limit will not be transmitted to the decoder but will be set to zero. In certain cases, the higher frequency bands have such a low energy content that they are already quantized to zero. Normal practice (cf. the max_sfb parameter in the MPEG standard) has meant using the same bandwidth limitation for both the downmix and residual signals. Therefore, by placing a capped upper dedicated band on the residual signal, bit rate reduction is possible with insignificant quality loss. For example, this can be achieved by passing two separate max_sfb parameters, one for the downmix signal and one for the residual signal.

It is pointed out that although the problems of optimal determination of the prediction coefficient, its quantization and coding, fallback to M/S or L/R mode, TNS filtering and upper bandwidth limitation etc. were discussed with reference to the decoder system shown in Figure 5, the same facts are equally applicable to the encoder systems which will be described below with reference to the subsequent figures.

Figure 6 shows another coding system adapted to perform complex prediction stereo coding. The system receives as input a time domain representation of a stereo signal segmented into successive, possibly overlapping, time frames and comprising left and right channels. A sum and difference stage 601 converts the signal into the center and side channels. The center channel is supplied to both the 602 MDCT module and the 603 MDST module, while the side channel is supplied to a 604 MDCT module only. The prediction coefficient estimator 605 estimates for each time frame - and possibly for the individual frequency bands within a frame - a value of the complex prediction coefficient a as explained above. The value of the coefficient a is supplied as a weight to the weighted adders 606, 607, which form a residual signal D as a linear combination of the MDCT and MDST representations of the center signal and the MDCT representation of the side signal. Preferably, the complex prediction coefficient is supplied as a weight to the weighted adders 606, 607 represented by the same quantization scheme that will be used when encoding in the bit stream; this obviously provides a more faithful reconstruction, since both the encoder and the decoder apply the same value of the prediction coefficient. The residual signal, the center signal (which may more appropriately be called a downmix signal when it occurs in combination with a residual signal), and the prediction coefficient are supplied to a quantization and multiplexer stage 608, which encodes this and possible additional information as an output bitstream.

Figure 7 shows a variation to the encoder system of Figure 6. As is clear from the similarity of the symbols in the figure, it has a similar structure but also the added functionality of operating in a reverse L/R encoding mode. direct. The coding system is operated between the complex prediction coding mode and the backward mode by means of a switch 710 provided immediately above the combined quantization and multiplexing stage 709. In its upper position, as shown in the figure, switch 710 will cause the encoder to operate in the reverse mode. From the points immediately below the MDCT modules 702, 704, the center/side signal is supplied to a sum and difference stage 705, which after converting it to left/right form passes it to switch 710, which connects to the combined quantization and multiplexer stage 709.

Figure 8 shows an encoder system. Contrary to the coding systems of Figures 6 and 7, this embodiment derives the MDST data required for complex prediction coding directly from the MDCT data, that is, by means of a real-to-imaginary transform in the frequency domain. The real-to-imaginary transform applies any of the approaches discussed in connection with the decoding systems of Figures 2 and 4. It is important to match the computational method of the decoder to that of the encoder, so that faithful decoding can be achieved; preferably, identical real-to-imaginary transform methods are used on the encoder side and the decoder side. As for the decoder implementations, the part A enclosed by the dashed line and comprising the real-to-imaginary transform 804 can be replaced by approximate variants or by using fewer input time frames as input. Similarly, coding can be simplified using any of the other approximation approaches described above.

At a higher level, the encoder system of Figure 8 has a structure that differs from what would likely be followed by a straightforward action of replacing the MDST module in Figure 7 with a (suitably connected) real-to-imaginary module. The present architecture is clean and achieves the switching functionality between prediction coding and direct L/R coding in a robust and computationally inexpensive manner. The input stereo signal is supplied to MDCT transform modules 801, which output a frequency domain representation of each channel. This is provided to both a final switch 808 to drive the encoding system between its predictive and forward encoding modes and a sum and difference stage 802. In direct L/R coding or joint M/S coding - which is carried out in the time frame for which the prediction coefficient is set to zero - the input signal can be subjected to only transformation, quantization and multiplexing, MDCT, the last two steps being performed by a combined multiplexing and quantizing stage 807 arranged at the output end of the system, where the bit stream is supplied. In prediction coding, each of the channels undergoes additional processing between the sum and difference stage 802 and the switch 808. From the MDCT representation of the center signal, the real-to-imaginary transform 804 derives the MDSt data. and sends them to both the prediction coefficient estimator 803 and the heavy adder 806. As in the encoder systems shown in Figures 6 and 7, an additional heavy adder 805 is used to combine the side signal with the weighted MDCT and MDST representations of the center signal to form a residual channel signal, which is encoded along with the encoder. center channel (ie downmix) and prediction coefficient by combined quantization and multiplexer module 807.

Turning now to Figure 9, it will be illustrated that the various encoding systems can be combined with one or more TNS (parsing) filters. In accordance with the above discussions, it is often advantageous to apply TNS filtering to the signal in its downmix form. Thus, as shown in Figure 9, adaptation of the encoder system of Figure 7 to include TNS is effected by adding the TNS filters 911 immediately above the combined quantization and multiplexer module 909.

Instead of the right/residual TNS filter 911 b, two separate TNS filters (not shown) of way immediately above the portion of the 910 switch adapted to handle the right or residual channel. Thus, each of the two TNS filters will be supplied with the respective channel signal data at all times, allowing TNS filtering based on more time frames than just the current one. As already noted, TNS filters are just one example of frequency domain modification devices, devices that remarkably base their processing on more frames than the current one, which can benefit from such placement as much or more than does the TNS filter.

As another possible alternative to the scrambler system shown in Figure 9, the TNS filters for selective activation can be arranged at more than one point for each channel. This is similar to the structure of the decoder system shown in Figure 4, where the different sets of TNS filters can be connected by means of switches. This allows selection of the most suitable available stage for TNS filtering for each time frame. In particular, it may be advantageous for switching between different locations in connection with switching between complex prediction stereo coding and other coding modes.

Figure 11 shows a variation based on the encoder system of Figure 8, in which the second frequency domain representation of the downmix signal is derived by means of a real-to-imaginary transform 1105 . Similar to the decoder system shown in Figure 4, this decoder system also includes selectively triggerable frequency-domain modifier modules, one 1102 provided above the downmix stage and another 1109 provided below it. . The frequency domain modules 1102, 1109, which have been exemplified in this figure by TNS filters, can be connected in each of the signal paths using the four switches 1103a, 1103b, 1109a and 1109b.

III. Realizations of non-apparatus

This section describes methods for decoding and encoding. The description of the method for encoding in this section is for illustrative purposes only, and the method of encoding in this section is not covered by the claims.

Figure 15 shows a method for decoding a bitstream into a stereo signal, comprising the following steps:

1. A bit stream as input.

2. The bit stream is dequantized, whereby a first frequency domain representation of the downmix and residual channels of a stereo signal is obtained.

3. A second frequency domain representation of the downmix is computed.

4. A side channel signal is calculated based on the three frequency domain representations of the channels.

5. A stereo signal is calculated, preferably in left/right form, based on the side and downmix channels.

6. The stereo signal thus obtained is emitted.

Steps 3 through 5 can be considered as an upmix process. Each of the steps 1 to 6 is analogous to the corresponding functionality in any of the decoder systems described in the previous parts of this text, and additional details regarding their implementation can be retrieved in the same parts.

Figure 16 shows a method (not covered by the claims) for encoding a stereo signal as a bitstream signal, comprising the following steps:

1. A stereo signal is input.

2. The stereo signal is transformed into a first representation in the frequency domain.

3. A complex prediction coefficient is determined.

4. The frequency domain representation is downmixed.

5. The downmix and residual channels are coded as a bit stream along with the complex prediction coefficient.

6. The bit stream is output.

Each of the steps 1 to 5 is analogous to the corresponding functionality in any of the coding systems in the earlier parts of this text, and additional details regarding their implementation will be found. can recover in the same parts.

Both methods can be expressed as computer-readable instructions in the form of software programs and can be executed by a computer. The scope of protection of this invention extends to such software and computer program products for distributing such software when it is related to decryption.

IV. Empirical evaluation

Several of the embodiments described herein have been empirically evaluated. The most important parts of the experimental material obtained in this process will be summarized in this subsection.

The embodiment used for the experiments had the following characteristics.

(i) Each MDST spectrum (for one time frame) was computed by a two-dimensional finite impulse response filtering of the current, previous and next spectra.

(ii) A psychoacoustic model of the USAC stereo encoder was used.

(iii) The real and imaginary parts of the complex prediction coefficient a were transmitted instead of the PS ICC, CLD and IPD parameters. The real and imaginary parts were handled independently, limited to the range [-3.0, 3.0], and quantized using a step size of 0.1. They were then differentially coded in time and finally Huffman coded using the USAC scale factor codebook. The prediction coefficients were updated every second in the scale factor band, resulting in a frequency resolution similar to that of the MPEG Envelope (see, for example, ISO/iEc 23003-1). This quantization and coding scheme resulted in an average bit rate of approximately 2 kb/s for this stereo side information in a typical configuration with a target bit rate of 96 kb/s.

(iv) The bitstream format was changed without breaking current USAC bitstreams, since the 2-bit bitstream element ms_mask_present currently has only three possible values. Using a fourth value to indicate complex prediction allows a fallback mode of basic mid/side coding without any wasted bits (for more details on this topic, see the previous subsection of this description).

Listening tests were performed according to the MUSHRA methodology, specifically involving playback over headphones and the use of 8 test items with a 48 kHz sampling rate. Three, five, or six test subjects participated in each test.

The impact of the different MDST approaches was evaluated to illustrate the practical trade-off between complexity and quality that exists between these options. The results were found in figures 12 and 13, the first one showing the absolute scores obtained and the last one showing the differential scores in relation to 96s USAC cp1f, that is, the unified stereo coding in the MDCT domain by complex prediction using an MDCT frame. current to calculate an approximation of the MDST. It can be seen that the quality gain achieved by MDCT-based unified stereo coding increases when more computationally complex approaches are applied to calculate the MDST spectrum. Considered the average over all tests, the single frame based 96s USAC cp1f system provides a significant increase in coding efficiency over conventional stereo coding. In contrast, even significantly better results are obtained for the 96s USAC cp3f, ie unified stereo coding in the MDCT domain by complex prediction using the current, previous and next MDCT frames to calculate the MDST.

SAW. concluding remarks

Additional embodiments of the present invention will become apparent to a person skilled in the art after reading the above description. Even though the present description and drawings describe the embodiments and examples, the invention is not restricted to these specific examples. Numerous modifications and variations can be made without departing from the scope of the present invention, which is defined by the appended claims.

It is noted that the methods and apparatus described in this application can be applied, after appropriate modifications within the capabilities of a skilled person including routine experimentation, for encoding signals having more than two channels. It is particularly emphasized that any signals, parameters and matrices mentioned in connections to the described embodiments may be frequency-variant or frequency-invariant and/or time-variant or time-invariant. The described computation steps can be carried out for one frequency or for all frequency bands at each moment, and all entities can be carried out to have a frequency selective action. For the purposes of the application, any quantization scheme can be adapted based on psychoacoustic models. It is further noted that the various conversions of sum and difference, that is, the conversion of the form of downmix/residual to pseudo L/R form as well as L/R to M/S conversion and M/S to L/R conversion are all of the form

where, mainly, the gain factor g can vary. Thus, by adjusting the gain factors individually, it is possible to compensate for a certain coding gain by appropriately choosing the decoding gain. Furthermore, as those skilled in the art will realize, an even number of sum and difference transforms arranged in series have the effect of a step stage, possibly with non-unity gain.

The systems and methods described herein above may be implemented as software, firmware, hardware, or a combination thereof. Certain components or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or implemented as hardware or as an application specific integrated circuit. Said software may be distributed on computer-readable media, which may comprise computer storage media and communication media. As is known by a person skilled in the art, computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for information storage such as instructions, data structures, program modules or other data readable by a computer. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile discs (DVD) or other optical disc storage, cassettes magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. In addition, it is known to a skilled person that communication media typically carry instructions, data structures, program modules or other computer-readable data into a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery medium.

Claims (8)

1. A decoder system for providing a stereo signal by complex prediction stereo coding, the decoder system comprising: a stage (206, 207, 210, 211; 306, 307, 308, 309, 312; 406, 407, 408, 409; 1410; 1421; 1433) adapted to generate the stereo signal based on the first representations in the frequency domain of a downmix signal (M) and a residual signal (D), each of the frequency domain representations comprising first spectral components representing the spectral content of the corresponding signal expressed in a first subspace of a multidimensional space, comprising the upmix stage: a module (206; 306, 307; 408) for calculating a second frequency-domain representation of the downmix signal based on the first frequency-domain representation thereof, the second representation comprising the frequency domain the second spectral components that represent the spectral content of the signal expressed in a second subspace of the multidimensional space that includes a part of the multidimensional space not included in the first subspace, in which the module is adapted to determine the second components spectral measurements of the downmix signal by applying a finite impulse response, FIR, filter to combine at least two of the first two spectral components of the downmix signal; a weighted adder (210, 211; 308, 309; 406, 407) for calculating a side signal (S) based on the first and second frequency-domain representations of the downmix signal, the first representation in the frequency domain of the residual signal and a complex prediction coefficient (a) encoded in a bit stream signal; and a sum and difference stage (207; 312; 409) for calculating the stereo signal based on the first frequency-domain representation of the downmix signal and the side signal, wherein the upmix stage is operable also in a pass-through mode, in which the downmix and residual signals are supplied to the sum and difference stage directly. The decoder system of claim 1, wherein said downmix signal and residual signal are segmented into time frames, said upmix stage being further adapted to receive, for each time frame, a two-bit data field associated with that frame and to operate, in response to the value of the data field, in its active mode or its passing mode. . The decoder system of claim 1, wherein said residual and downmix signal are segmented into time frames, said upmixing stage being further adapted to receive in an MPEG bit stream, for each time frame, a ms_mask_present field associated with that frame and to operate, in response to the value of the ms_mask_present field, in its active mode or its passed. 4. The decoder system of the preceding claims, wherein: the first spectral components have real values expressed in the first subspace; and the second spectral components have imaginary values expressed in the second subspace; Optionally, the first few spectral components can be obtained by one of the following: a discrete cosine transform, DCT, or a modified discrete cosine transform, MDCT, and, optionally, the second spectral components can be obtained by one of the following: a discrete sine transform, DST, or a modified discrete sine transform, MDST. The decoder system of any of the preceding claims, wherein: the first spectral components are transform coefficients arranged in one or more transform coefficient time blocks, each block generated by applying a transform to a time slice of a time domain signal; and The module for calculating a second frequency-domain representation of the downmix signal is adapted to: deriving one or more first intermediate components from at least some first spectral components; forming a combination of said one or more first spectral components according to at least a part of one or more impulse responses to obtain one or more intermediate second components; and deriving said one or more second spectral components from said one or more second intermediate components. The decoder system of any of the preceding claims, said stereo signal being represented in the time domain and the decoder system further comprising: a switching assembly (203) arranged between a dequantization stage and said upmix stage, operable to function as either: (a) a step step, or (b) a sum and difference stage, thus allowing switching between the co- and direct-encoded stereo input signals; an inverse transform stage (209) adapted to compute a time domain representation of the stereo signal; and a selector arrangement (208) disposed above the inverse transform stage, adapted to selectively connect the latter to either: (a) a point below the upmix stage, through which the stereo signal obtained by complex prediction is supplied to the inverse transform stage; either (b) a point below the switching assembly (203) and above the upmix stage, whereby a stereo signal obtained by direct stereo encoding is supplied to the inverse transform stage. 7. A decoding method for providing a stereo signal by complex prediction stereo coding, the method comprising the steps of: reception of the first frequency domain representations of a downmix signal (M) and a residual signal (D), each of the first frequency domain representations comprising the first spectral components representing the spectral content of the corresponding signal expressed in a first subspace of a multidimensional space; reception of a control signal; and upmixing said downmix and residual signals, using an upmix stage, to obtain said stereo signal by, in response to the value of said control signal: (a) performing the sub-steps of: calculation of a second frequency-domain representation of the downmix signal based on the first frequency-domain representation thereof, the second frequency-domain representation comprising the second spectral components representing the spectral content of the signal expressed in a second subspace of the multidimensional space including a part of the multidimensional space not included in the first subspace, wherein calculating a second frequency-domain representation of the downmix signal includes determining the second spectral components of the downmix signal by applying a finite impulse response, FIR, filter to combine at least two of the first two spectral components of the downmix signal; calculation of a side signal (S) based on the first and second frequency-domain representations of the downmix signal, encoded the first frequency-domain representation of the residual signal, and a coefficient (a) of complex prediction on a bitstream signal; and calculating said stereo signal by applying a sum and difference transform to the first frequency domain representation of the downmix signal and the side signal, either (b) calculating said stereo signal by applying a sum and difference transform directly to the first frequency domain representations of the downmix signal and the residual signal. A computer program product comprising a computer readable medium that stores instructions which when executed by a general purpose computer performs the method set forth in claim 7.