KR20130095851A - Advanced stereo coding based on a combination of adaptively selectable left/right or mid/side stereo coding and of parametric stereo coding - Google Patents

Abstract

The present invention relates to audio encoder and decoder systems. An embodiment of an encoder system includes a downmix stage for generating a downmix signal and a residual signal based on a stereo signal. The encoder system also includes a parameter determination stage for determining parametric stereo parameters, such as inter-channel intensity differences and inter-channel cross-correlation. Preferably, the parametric stereo parameters are time-variable and frequency-variable. The encoder system also includes a conversion stage. The conversion stage generates a pseudo left / right stereo signal by performing a conversion based on the downmix signal and the residual signal. The pseudo-stereo signal is processed by the perceptual stereo encoder. For stereo encoding, left / right encoding or mid / side encoding is selectable. Preferably, the choice between left / right stereo encoding and mid / side stereo encoding is time-variable and frequency-variable.

Description

ADVANCED STEREO CODING BASED ON A COMBINATION OF ADAPTIVELY SELECTABLE LEFT / RIGHT OR MID / SIDE STEREO CODING AND OF PARAMETRIC BASED ON A COMBINATION OF ADAPTIVE SELECTABLE LEFT / RIGHT OR MID / SIDE STEREO CODING AND PARAMETRIC STEREO CODING STEREO CODING}

The present invention relates to audio coding, and more particularly to stereo audio coding combining parametric and waveform based coding techniques.

Joint coding of the left (L) channel and the right (R) channel of the stereo signal allows for more efficient coding compared to independent coding of L and R. A common technique for joint stereo coding is mid / side (M / S) coding. Here, the mid (M) signal is formed by adding an L signal and an R signal. For example, the M signal can take the following form.

Further, the side (S) signal is formed by subtracting two channels L and R from each other. For example, the S signal can take the following form.

In the case of M / S coding, the M signal and the S signal are coded in place of the L signal and the R signal.

In the Moving Picture Experts Group (MPEG) Advanced Audio Coding (MPEG) standard (see standard document ISO / IEC 13818-7), L / R stereo coding and M / S stereo coding are selected in a time-variable and frequency- . Thus, the stereo encoder can apply L / R coding for some frequency bands of the stereo signal, while M / S coding is used to encode other frequency bands of the stereo signal (frequency variable). In addition, the encoder can switch between L / R and M / S coding over time (time-variable). In MPEG AAC, stereo encoding is performed in the frequency domain, especially in the MDCT (Modified Discrete Cosine Transform) domain. This allows adaptive selection of L / R or M / S coding with frequency and time variability. The decision between L / R and M / S stereo encoding can be based on evaluating the side signal. When the energy of the side signal is low, M / S stereo encoding should be more efficient and used. Alternatively, in determining between the two stereo coding techniques, the two coding techniques may be tried and the selection may be based on the quantization result, i.e., the observed cognitive entropy, as indicated.

An alternative approach to joint stereo coding is parametric stereo (PS) coding. Here, the stereo signal is transmitted as a mono down-mix signal after encoding the down-mix signal with a conventional audio encoder such as an AAC encoder. The downmix signal is an overlap of the L channel and the R channel. The mono downmix signal is delivered in combination with additional time-varying and frequency-varying PS parameters such as inter-channel (i.e., L and R) intensity difference (IID) and interchannel cross-correlation (ICC). At the decoder, a stereo signal approximating the perceived stereo image of the original stereo signal is reconstructed, based on the decoded downmix signal and the parametric stereo parameters. To reconstruct, a decorrelated version of the downmix signal is generated by the decorrelator. This decorrelator can be realized by a suitable global-pass filter. PS encoding and decoding is described in the paper "Low Complexity Parameter Stereo Coding in MPEG-4", H. Purnhagen, Proc. Of the 7th Int. Conference on Digital Audio Effects (DAFX'04), Naples, Italy, October 5-8, 2004, pages 163-168. Incorporated herein by reference in its entirety.

The MPEG Surround standard (document ISO / IEC 23003-1) uses the concept of PS coding. In an MPEG surround decoder, a plurality of output channels are generated based on several input channels and control parameters. MPEG Surround decoders and encoders are constructed by connecting parametric stereo modules in series. In MPEG Surround, they are called line OTT modules (one-to-two modules) for decoders and R-OTT modules Modules). The OTT module determines the two output channels by a single input channel (downmix signal) with PS parameters. The OTT module corresponds to the PS decoder and the R-OTT module corresponds to the PS encoder. Parametric stereo can be realized by using MPEG surround with a single OTT module on the decoder side and a single R-OTT module on the encoder side, which is also referred to as the "MPEG surround 2-1-2" mode. The bitstream syntax may be different, but the basic theory and signal processing are the same. Therefore, all references to PS in the following include "MPEG Surround 2-1-2" or MPEG Surround based parametric stereo.

In the PS encoder (for example, in the MPEG surround PS encoder), the residual signal RES in addition to the downmix signal can be determined and transmitted. This residual signal represents the error associated with representing the original channels with their downmix and PS parameters. In the decoder, the residual signal may be used in place of the decorrelated version of the downmix signal. This allows for better reconstruction of the waveforms of the original channel L and channel R. [ The use of additional residual signals is described, for example, in the MPEG Surround standard (see document ISO / IEC 23003-1) and in the article "MPEG Surround - The ISO / MPEG Standard for Efficient and Compatible Multi-Channel Audio Coding, al., Audio Engineering Convention Paper 7084, are described in the ^{122 nd Convention, may 5-8, 2007} . causes the contents of both documents, particularly to those included here by reference in the description of the residual signal.

PS coding using residuals is a more common technique for joint stereo coding than M / S coding, and M / S coding performs signal rotation when converting L / R signals to M / S signals. In addition, the residual use PS coding performs signal rotation when converting the L / R signals into a downmix signal and a residual signal. In the latter case, however, the signal rotation is variable and dependent on the PS parameters.

Due to the more general technique of residual use PS coding, residual use PS coding makes coding of some types of signals, such as paned mono signals, more efficient than M / S coding. Thus, the proposed coder enables efficient combination of parametric stereo coding techniques with waveform based stereo coding techniques.

Often, perceptual stereo encoders such as MPEG AAC or stereo encoders can determine either L / R stereo encoding and M / S stereo encoding, in the latter case the mid / side signal is generated based on the stereo signal. This selection may be frequency-variable, i.e., L / R stereo encoding may be used for some frequency bands, and M / S stereo encoding may be used for other frequency bands.

In the situation where the L and R channels are essentially independent signals, this perceptual stereo encoder typically does not provide M / S stereo encoding in this situation since this encoding approach does not provide any coding gain over L / R stereo encoding I will not. The encoder will revert to a simple L / R stereo encoding, which basically handles L and R independently.

In the same situation, the PS encoder system will generate a downmix signal containing both L and R channels, which prevents independent processing of the L and R channels. For PS coding with residual signal, this may mean less efficient coding compared to stereo encoding, and L / R stereo encoding or M / S stereo encoding may be adaptively selected.

Thus, there are situations in which the PS coder is lower than the perceived stereo coder by adaptively selecting one of L / R stereo encoding and M / S stereo encoding, and in other situations the latter coder is better than the PS coder.

It is an object of the present invention to provide stereo audio coding that combines parameter and waveform based coding techniques.

The present invention describes an imaging-based audio encoder system and encoding method that combines PS coding using residuals into adaptive L / R or M / S or stereo coding (e.g., AAC or joint stereo coding in the MDCT domain) . This can combine the advantages of adaptive L / R or M / S stereo coding (e.g. used in MPEG AAC) and the benefits of PS coding using residual signals (e.g. used in MPEG surround) Let's do it. The present invention also describes a corresponding audio decoder system and decoding method.

A first aspect of the invention relates to an encoder system for encoding a stereo signal into a bitstream signal. According to an embodiment of the encoder system, the encoder system comprises a downmix stage for generating a downmix signal and a residual signal based on the stereo signal. The residual signal may include only a portion or all of the audio frequency range being used. The encoder system also includes a parameter determination stage for determining PS parameters, such as inter-channel intensity differences and inter-channel cross-correlation. Preferably, the PS parameters are frequency-variable. These downmix stages and parameter determination stages are typically part of a PS encoder.

The encoder system also includes a perceptual encoding means on the downstream side of the downmix stage for encoding based on the sum of the downmix signal and the residual signal and based on the difference between the downmix signal and the residual signal, And encoding based on the residual signal are selectable.

Note that when the encoding is based on a downmix signal and a residual signal, the downmix signal and the residual signal may be encoded or the signals proportional thereto may be encoded. If the encodings are based on sum and difference, the sum and difference may be encoded or signals that are proportional thereto may be encoded.

The selection may be frequency-variable (and time-variant), that is, for the first frequency band, encoding may be selected based on the sum signal and the difference signal, and for the second frequency band, And the residual signal can be selected.

Such an encoder system is advantageous (preferably frequency-variable) by allowing the use of residuals to switch between L / R stereo coding and PS coding, wherein the cognitive encoding means selects an encoding based on the downmix signal and the residual signal (For a particular band or for the entire used frequency range), the encoding system operates like a system using residual use standard PS coding. However, if the recognition encoding means selects an encoding based on the sum signal of the downmix signal and the residual signal and based on the difference signal of the downmix signal and the residual signal (for a particular band or for the entire used frequency range) Under some circumstances, sum and difference operations essentially compensate for previous downmix operations (possibly other gain factors) so that the entire system can actually perform L / R encoding for the entire stereo signal or its frequency band . For example, these situations occur when the L channel and R channel of the stereo signal are independent and have the same level as described below.

Preferably, the adaptation of the encoding technique is time and frequency dependent. Thus, preferably some frequency bands of the stereo signal are encoded by the L / R encoding technique and other frequency bands of the stereo signal are encoded by a PS coding technique that utilizes residuals.

As discussed above, if the encoding is based on a downmix signal and a residual signal, the actual signal input to the core encoder is divided into two series of operations (presumably for the other gain factors, except for the downmix signal and the opposite inverse signal) And < / RTI > For example, the downmix signal and the residual signal are supplied to the M / S to L / R conversion stage and then the output of the conversion stage is supplied to the L / R to M / S conversion stage. The resulting signal (which is then used for encoding) corresponds to the downmix signal and the residual signal (except perhaps for other gain factors).

The following example uses this implant. According to an embodiment of the encoder system, the encoder system comprises a downmix stage and a parameter decision stage as discussed above. The encoder system also includes a conversion stage (e.g., as part of the encoding means discussed above). The conversion stage generates a pseudo L / R stereo signal by performing conversion of the downmix signal and the residual signal. The conversion stage preferably performs a sum-of-product conversion, wherein the downmix signal and the residual signals are summed to produce one channel of the pseudo-stereo signal (possibly summed with a factor) and the other channel of the pseudo-stereo signal They are subtracted from each other to produce (perhaps the difference is multiplied by the argument). Preferably, the first channel (e.g., pseudo left channel) of the pseudo stereo signal is proportional to the sum of the downmix signal and the residual signal, and the second channel (e.g., pseudo right channel) It is proportional to the difference in signal. Thus, the downmix signal (DMX) and a residual signal (RES) from the PS encoder may be converted to a pseudo-stereo signal (L _p, R _p) in accordance with the following equations.

L _p = g (DMX + RES)

R _p = g (DMX - RES)

In the above equations, the gain normalization rate g has, for example, the following values.

The pseudo-stereo signal is preferably processed by a perceptual stereo encoder (e.g., as part of the encoding means). For encoding, L / R stereo encoding or M / S stereo encoding may be selected. The adaptive L / R or M / S stereo encoder may be an AAC based encoder. Preferably, the choice between L / R stereo encoding and M / S stereo encoding is frequency-variable, so the selection may be different for different frequency bands as discussed above. In addition, the choice between L / R encoding and M / S encoding is preferably time-variant. The decision between L / R encoding and M / S encoding is preferably done by a perceptual stereo encoder.

These cognitive encoders that are selectable for M / S encoding can compute (pseudo) M and S signals (in the time domain or in selected frequency bands) internally based on the pseudo-stereo L / R signal. These pseudo M and S signals correspond to the downmix signal and the residual signal (except perhaps for other gain factors). Thus, if the perceptual stereo encoder chooses the M / S encoding, it actually encodes the downmix signal residual signal (corresponding to the pseudo M signal S signal) that has been made in the system using residual standard PS coding.

Further, under special circumstances, the conversion stage may also perform a previous downmix operation (if the L / R encoding is selected in the cognitive encoder) so that the entire encoder system can actually perform the L / R encoding for the entire stereo signal or its frequency band Perhaps except for other benefit rates). This is the case, for example, when the L channel and the R channel of the stereo signal are independent and have the same level as described in detail later. Thus, for a given frequency band, the pseudo-stereo signal essentially corresponds to or is proportional to the stereo signal if the left and right channels of the stereo signal are fundamentally independent and essentially at the same level for the frequency band.

Thus, the encoder system is able to actually switch between L / R stereo coding and residual use PS coding in order to be able to match the characteristics of a given stereo input signal. Preferably, the adaptation of the encoding technique is time and frequency dependent. Thus, preferably, some frequency bands of the stereo signal are encoded by the L / R encoding technique, and the other frequency bands of the stereo signal are encoded by the residual use PS coding technique. Note that M / S coding is basically a special case of residual use PS coding (since L / R versus M / S conversion is a special case of PS downmix operation) and therefore the encoder system can also perform full M / do.

The embodiment with the conversion stage on the downstream side of the PS encoder and on the upstream side of the L / R or M / S or stereo encoder has the advantage that a conventional PS encoder and a conventional cognitive encoder can be used. Nonetheless, the PS encoder or cognitive encoder can be adaptive due to the particular use here.

The new concept improves the adaptation of stereo coding by allowing efficient combination of PS coding and joint stereo coding.

According to an alternative embodiment, the encoding means as discussed above may be implemented as a combination of two or more frequency bands (for example, And a conversion stage for executing the conversion. The transformation can be performed in the frequency domain or the time domain. The conversion stage generates pseudo left / right stereo signals for one or more frequency bands. One channel of the pseudo-stereo signal corresponds to the sum, and the other channel corresponds to the difference.

Thus, if the encoding is based on summing signals, then the output of the transforming stage can be used for encoding, and if the encoding is based on a downmix signal and a residual signal, the signals upstream of the encoding stage Lt; / RTI > Thus, this embodiment does not use two series of sum-of-product transforms for the downmix signal and the residual signal, resulting in a downmix signal and a residual signal (perhaps for other gain factors).

When encoding is selected based on the downmix signal and the residual signal, the parametric stereo encoding of the stereo signal is selected. When selecting an encoding based on the sum (i. E. Encoding based on a pseudo stereo signal), the L / R encoding of the stereo signal is selected.

The conversion stage is a part of a cognitive encoder that adaptively selects between L / R and M / S stereo encoding (presumably the gain ratio is different compared to a conventional L / R to M / S conversion stage) / S conversion stage. Note that the decision between L / R and M / S stereo encoding must be reversed. Thus, the encoding based on the downmix signal and the residual signal is selected when the decision means has decided to decode the M / S (i.e., the encoded signal has not gone through the conversion stage), and the pseudo stereo signal generated by the conversion stage Based encoding is selected when the decision means determines L / R or decoding (i. E., The encoded signal has passed the conversion stage).

The encoder system according to any one of the embodiments discussed above may include an additional SBR (spectral band replica) encoder. SBR is a form of HFR (High Frequency Reconstruction). The SBR encoder determines the side information for reconstruction of the high frequency range of the audio signal at the decoder. Only the low frequency range is encoded by the cognitive encoder, thereby reducing the bit rate. Preferably, the SBR encoder is connected upstream of the PS encoder. Thus, the SBR encoder may be in the stereo region and generate SBR parameters for the stereo signal. This will be discussed in detail with reference to the drawings.

Preferably, the PS encoder (i.e., downmix stage and parameter decision stage) operates in the oversampled frequency domain (as discussed below, the PS decoder preferably operates in the oversampled frequency domain). In converting time to frequency, for example, a complex valued hybrid filter bank with quadrature mirror filters (QMFs) and Nyquist filters may be used for the PS encoders as described in the MPEG Surround standard (see document ISO / IEC 23003-1) As shown in Fig. This allows time- and frequency-adaptive signal processing without audible aliasing artifacts. On the other hand, the adaptive L / R or M / S encoding is preferably performed in a critically sampled MDCT region (e.g., as described in AAC) in order to be able to be an efficient quantized signal representation.

The conversion between the downmix signal and the residual signal and the pseudo L / R stereo signal can be performed in the time domain since the PS encoder and the perceptual stereo encoder are typically connected in any time domain. Thus, the conversion stage for generating the pseudo L / R signal can operate in the time domain.

In other embodiments discussed with respect to the figures, the conversion stage operates in the oversampled frequency domain or in the critical sampling MDCT domain.

A second aspect of the present invention is directed to a decoder system for decoding a bitstream signal generated by an encoder system as discussed above.

According to an embodiment of the decoder system, the decoder system comprises perceptual decoding means for decoding based on the bitstream signal. The decoding means is configured to generate the (internal) first signal and the (internal) second signal by decoding and output the downmix signal and the residual signal. The downmix signal and the residual signal are optionally based on a sum of the first signal and the second signal and based on a difference between the first signal and the second signal, or based on the first signal and based on the second signal.

As discussed above with respect to the encoder system, the choice here can also be frequency-variable or frequency-invariant.

The system also includes an upmix stage for generating a stereo signal based on the downmix signal and the residual signal, and the upmix operation of the upmix stage is dependent on one or more parametric stereo parameters.

Similar to the encoder system, the decoder system preferably allows time and frequency variability to actually switch between L / R decoding and residual use PS decoding.

According to yet another embodiment, the decoder system includes a perceptual stereo decoder (e.g., as part of decoding means) that decodes the bitstream signal and generates a pseudo-stereo signal. The perceptual decoder may be an AAC based decoder. For a perceptual stereo decoder, L / R or M / S decoding may be chosen to be frequency-variable or frequency-invariant (the actual selection is preferably determined by an encoder which is transmitted as side- Lt; / RTI > The decoder selects the decoding technique based on the encoding technique used for encoding. The encoding scheme used may be indicated to the decoder by the embedded information in the received bitstream.

The conversion stage is also provided for generating a downmix signal and a residual signal by performing conversion of the pseudo-stereo signal. That is, the pseudo-stereo signal obtained from the perceptual decoder is converted back to the downmix and residual signals. This transform is a sum-of-product transform and the resulting downmix signal is proportional to the sum of the left and right channels of the pseudo-stereo signal. The resulting residual signal is proportional to the difference between the left channel and the right channel of the pseudo-stereo signal. Therefore, the sub L / R to M / S conversion was performed. A pseudo-stereo signal having two channels L _p , R _p can be converted into downmix and residual signals according to the following equations.

의 값을 가질 수 있다. 디코더에서 이용되는 잔차 신호(RES)는 전체 이용되는 오디오 주파수 범위 또는 이용된 오디오 주파수 범위의 부분만을 포함할 수 있다.In the above equations, the gain normalization rate g is, for example, g =

It can have a value of. The residual signal RES used in the decoder may only include the entire used audio frequency range or the portion of the audio frequency range used.

The downmix signal and the residual signal are then processed by the upmix stage of the PS decoder to obtain the final stereo output signal. Upmixing the downmix signal and the residual signal into a stereo signal is dependent on the received PS parameters.

According to an alternative embodiment, the perceptual decoding means may comprise a sum-of-transformation stage for performing based on the first signal and the second signal for one or more frequency bands (e.g., for the entire used frequency range) . Thus, the conversion stage generates a downmix signal and a residual signal for the case where the downmix signal and the residual signal are based on the sum of the first and second signals and based on the difference between the first signal and the second signal. The conversion stage may operate in the time domain or the frequency domain.

As discussed similarly with regard to the encoder system, the conversion stage is selected by applying between L / R and M / S stereo decoding (presumably the gain ratio is different compared to a conventional M / S to L / R conversion stage) / RTI > L / R conversion stage as part of the perceptual decoder that performs the L / R conversion. L / R and M / S stereo decoding.

A decoder system according to any one of the embodiments described above may include an additional SBR decoder that decodes the side information from the SBR encoder and generates high frequency components of the audio signal. Preferably, the SBR decoder is located on the downstream side of the PS decoder. This will be discussed in detail with reference to the drawings.

Preferably, the upmix stage operates in the oversampled frequency domain, for example, the hybrid filter bank discussed above may be used upstream of the PS decoder.

The L / R to M / S conversion can be performed in the time domain since the perceptual decoder and the PS decoder (including the upmix stage) are typically connected in the time domain.

In other embodiments discussed with respect to the figures, the L / R to M / S transform is performed in the oversampled frequency domain (e.g., QMF) or in the critical sampling frequency domain (e.g., MDCT) .

A third aspect of the invention relates to a method for encoding a stereo signal into a bitstream signal. The method operates similarly to the encoder system discussed above. Therefore, the above-mentioned matters relating to the encoder system can basically be applied to the encoding method.

A fourth aspect of the invention relates to a method of decoding a bitstream signal comprising PS parameters for generating a stereo signal. The method operates in the same manner as the decoder system discussed above. Therefore, the above-mentioned matters related to the decoder system can basically be applied to the decoding method.

BRIEF DESCRIPTION OF THE DRAWINGS The invention will be described below by examples with reference to the accompanying drawings.

Figure 1 also illustrates an embodiment of an encoder system in which PS parameters optionally assist acoustic psychological control in a perceptual stereo encoder.
Figure 2 also shows an embodiment of a PS encoder.
Figure 3 also shows an embodiment of a decoder system.
Figure 4 illustrates another embodiment of a PS encoder including a detector for deactivating PS encoding if L / R encoding is advantageous.
Figure 5 also shows an embodiment of a conventional PS encoder system with an additional SBR encoder for downmixing.
6 shows an embodiment of an encoder system with an additional SBR encoder for the downmix signal.
Figure 7 also shows an embodiment of an encoder system with an additional SBR encoder in the stereo region.
Figures 8a-8d illustrate multiple time-frequency representations of one of two output channels at decoder output.
Figure 9a also illustrates an embodiment of a core encoder.
FIG. 9B also illustrates an embodiment of an encoder that enables switching between coding in a linear prediction region (typically for mono signals only) and coding in a transform domain (typically for both mono and stereo signals).
Figure 10 also illustrates an embodiment of an encoder system.
11A shows a portion of an embodiment of an encoder system;
FIG. 11B shows an embodiment of the embodiment in FIG. 11A. FIG.
FIG. 11C shows an alternative example to the embodiment in FIG. 11A. FIG.
Figure 12 also illustrates an embodiment of an encoder system.
Figure 13 illustrates an embodiment of a stereo coder as part of the encoder system of Figure 12;
Figure 14 also illustrates an embodiment of a decoder system for decoding a bitstream signal generated by the encoder system of Figure 6;
Figure 15 also illustrates an embodiment of a decoder system for decoding a bitstream signal generated by the encoder system of Figure 7;
16A illustrates a portion of an embodiment of a decoder system;
Figure 16b illustrates an embodiment of the embodiment in Figure 16a.
FIG. 16C is a view showing an alternative example of the embodiment in FIG. 16A. FIG.
Figure 17 also illustrates an embodiment of an encoder system.
Figure 18 also shows an embodiment of a decoder system.

Figure 1 illustrates an embodiment of an encoder system that combines PS encoding using residuals with adaptive L / R or M / S or stereo encoding. This embodiment is only intended to illustrate the principles of the present invention. Modifications and variations of the embodiments will be apparent to those skilled in the art. The encoder system includes a PS encoder 1 for receiving stereo signals (L, R). The PS encoder 1 has a downmix stage for generating downmix (DMX) and residual (RES) signals based on the stereo signals (L, R). This operation can be described by a 2 占 2 downmix matrix H ^-1 that converts the L signal and R signal into a downmix signal DMX and a residual signal RES.

Typically, the matrix H ^-1 is frequency-variable and time-variant, that is, the elements of the matrix H ^-1 vary with frequency and change from time slot to time slot. The matrix H ^-1 can be updated every frame (e.g., every 21 or 42 ms) and can be updated in multiple bands (e.g., 28, 20, Or 10 bands (referred to as "parameter bands").

The elements of the matrix H ^-1 depend on time-varying and frequency-varying PS parameters (IID (interchannel intensity difference; also called CLD-channel level difference) and ICC (cross-channel cross-correlation)). In order to determine the PS parameters 5, for example IID and ICC, the PS encoder 1 comprises a parameter determination stage. An example of computing the matrix elements of the inverse matrix H is given in the MPEG Surround specification document ISO / IEC 23003-1, subclause 6.5.3.2, which is incorporated herein by reference.

, 및

이고,

, And

ego,

, 및

이고,here,

, And

ego,

이다.

to be.

The encoder system also includes a conversion stage 2 for converting the downmix signal DMX and the residual signal RES from the PS encoder 1 into a pseudo-stereo signal L _p , R _p , for example, ).

L _p = g (DMX + RES)

R _p = g (DMX - RES)

값을 갖는다. g =

에 대해서, 의사 스테레오 신호(L_p, R_p)에 대한 2개의 식들은 다음처럼 다시 쓸 수 있다.The gain normalization rate g in the above equations is, for example, g =

Has a value. g =

, The two equations for the pseudo-stereo signal (L _p , R _p ) can be rewritten as:

The pseudo stereo signals Lp and Rp are then supplied to the perceptual stereo encoder 3 which adaptively selects the L / R or M / S stereo encoding. The M / S encoding is in the form of a joint stereo coding. The L / R encoding may also be based on joint encoding features, for example, the bits may be jointly allocated for the L channel and the R channel from a common bit reservoir.

The choice between L / R or M / S stereo encoding is preferably frequency-variable, i.e. some frequency bands may be L / R encoded and other frequency bands may be M / S encoded. An embodiment for implementing selection between L / R or M / S stereo encoding is described in the document "Sum-Difference Stereo Transform Coding" Johnston et al., IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) 1992, pages 569-572. Include here a discussion of the choice between L / R or M / S stereo encoding in this document, especially paragraphs 5.1 and 5.2.

Based on the pseudo-stereo signal (L _p , R _p ), the perceptual encoder 3 can compute (pseudo) mid / side signals M _p , S _p internally. These signals basically correspond to the downmix signal DMX and the residual signal RES (presumably for the other gain factors). Therefore, if the cognitive encoder 3 chooses the M / S encoding for the frequency band, the cognitive encoder 3 will be able to perform this frequency band (presumably because the cognitive encoder 3 will be done in a conventional cognitive encoder system, (Except for the other gain factors), the downmix signal DMX and the residual signal RES are basically encoded. The PS parameters 5 and the output bit stream 4 of the cognitive encoder 3 are multiplexed into a single bit stream by the multiplexer 7.

In addition to the PS encoding of a stereo signal, in Figure 1 the encoder system as to be able to code L / R stereo signals as will be described in the following, discussed above, the downmix matrix H ^-l elements of encoders (in the decoder The upmix matrix H elements used also depend on time-varying and frequency-varying PS parameters (IID (interchannel intensity difference; also called CLD-channel level difference) and ICC (cross-channel cross-correlation)). An example of calculating the matrix elements of the upmix matrix H has been described above. In the case of using residual coding, the right column of the 2 < 2 > upmix matrix H is given as follows.

However, preferably, the right column of the 2 < 2 > upmix matrix H should be modified next instead.

The left column is preferably calculated as given in the MPEG Surround specification.

By modifying the right-hand column of the upmix matrix H, IID = 0 dB and ICC = 0 (i.e., if the stereo channel L and channel R are independent and have the same level for each band) The upmix matrix H can be obtained.

Note that the upmix matrix H and the downmix matrix H ^-1 are also typically time-variable and frequency-variable. Thus, the values of the matrices are different for each time / frequency tile (the tile corresponds to the intersection of a particular frequency band and a particular time period). In the above case, the downmix matrix H ^-1 is equal to the upmix matrix H. Therefore, the pseudo stereo signal (L _p , R _p ) for the band can be calculated by the following equation.

Therefore, in this case, the PS encoding using the residual matrix using the downmix matrix H- ¹ and the subsequent generation of the pseudo L / R signals in the conversion stage 2 correspond to the unit matrix and the stereo signals for the respective frequency bands Do not change at all. In other words,

L _p = L

R _p = R

That is, the conversion stage 2 compensates the downmix matrix H ^-1 for the pseudo-stereo signals L _p and R _p to correspond to the input stereo signals L and R.

This enables the original input stereo signal (L, R) to be encoded by the perceptual encoder 3 for a particular band. When the L / R encoding is selected by the perceptual encoder 3 to encode a particular band, the encoder system behaves like an encoder, L / R, to encode the band of the stereo input signals (L, R).

In Fig. 1, the encoder system enables adaptive switching between PS coding using L / R coding and residual coding in a frequency-variable and time-variable manner. The encoder system eliminates discontinuities in the waveform when switching the coding scheme. This prevents artifacts. For the sake of convenience, linear interpolation can be applied to the elements of the matrix H ^{-1 at} the encoder and to the elements of the matrix H at the decoder for the samples between the two stereo parameter updates.

Fig. 2 shows an embodiment of the PS encoder 1. Fig. The PS encoder 1 includes a downmix stage 8 for generating a downmix signal DMX and a residual signal RES based on the stereo signals L and R. [ The PS encoder 1 also includes a parameter estimation stage 9 for estimating the PS parameters 5 based on the stereo signals L, R.

FIG. 3 shows an embodiment of a corresponding decoder system configured to decode the bitstream 6 generated by the encoder system of FIG. This embodiment is merely illustrative of the principles of the invention. Modifications and variations of the embodiments will be apparent to those skilled in the art. The decoder system includes a demultiplexer 10 for separating the PS parameters 5 and the audio bitstream 4 generated by the perceptual encoder 3. The audio bitstream 4 is supplied to the perceptual stereo decoder 11, which can selectively decode the L / R encoded bitstream or the M / S encoded audio bitstream. The operation of the decoder 11 is opposite to that of the encoder 3. Similar to the cognitive encoder 3, the perceptual decoder 11 preferably enables frequency-variable and time-variant decoding techniques. Some frequency bands are L / R encoded by the encoder 3 and L / R decoded by the decoder 11, while the other frequency bands are M / S encoded by the encoder 3 and decoded by the decoder 11 M / S < / RTI > The decoder 11 outputs the pseudo stereo signal L _p , R _{p that} was previously input to the perceptual encoder 3. The pseudo stereo signals L _p and R _p obtained from the perceptual decoder 11 are converted into the downmix signal DMX and the residual signal RES again by the L / R to M / S conversion stage 12. The operation of the L / R to M / S conversion stage 12 on the decoder side is opposite to that of the conversion stage 2 on the encoder side. Preferably, the conversion stage 12 determines the downmix signal DMX and the residual signal RES according to the following equations.

의 값을 갖는다.In the above equations, the gain normalization rate (g) is equal to the gain normalization rate (g) at the encoder side and g =

Lt; / RTI >

The downmix signal DMX and the residual signal RES are then processed by the PS decoder 13 to obtain the final L and R output signals. In the decoding process for residual use PS coding, the upmixing step may be described by a 2 < 2 > upmix matrix H for converting the downmix signal DMX and the residual signal RES back to the L channel and the R channel.

The calculation of the elements of the upmix matrix H has already been discussed above.

The PS encoding and PS decoding processes in the PS encoder 1 and the PS decoder 13 are preferably executed in the oversampling frequency domain. Value hybrid filter with quadrature mirror (QMF) and Nyquist filters, such as the filter bank described in the MPEG Surround standard (see document ISO / IEC 23003-1) A bank may be used on the upstream side of the PS encoder. The complex QMF representation of the signal is a complex number and is not a real number, so it is oversampled twice. This allows time- and frequency-adaptive signal processing without audible aliasing artifacts. Such a hybrid filter bank typically provides high frequency resolution (narrowband) at low frequencies, while at high frequencies some QMF bands are grouped into a broader band. &Quot; Low Complexity Parameter Stereo Coding in MPEG-4 ", H. Purnhagen, Proc. of the 7th Int. Conference on Digital Audio Effects (DAFX'04), Naples, Italy, October 5-8, 2004, pages 163-168, describes an embodiment of a hybrid filter bank (see paragraph 3.2 and figure 4). The disclosure of which is incorporated herein by reference. In this document, a 48 kHz sampling rate is taken, and the (nominal) bandwidth of one band from a 64-band QMF bank is 375 Hz. However, the perceptual Bark frequency scale requires a bandwidth of approximately 100 Hz for frequencies below 500 Hz. Therefore, the first three QMF bands can be divided into narrower sub-bands by the Nyquist filter bank. The first QMF band may be divided into four bands (two more are added for negative frequencies), and the second and third QMF bands may be divided into two bands each.

On the other hand, preferably, the adaptive L / R or M / S encoding is performed in the critical sampling MDCT region (e.g., as described in AAC) to be an efficient quantized signal representation. It converts the conversion stage (2) a downmix signal (DMX) and a residual signal (RES) in a pseudo-stereo signal (L _p, R _p) is PS encoder (1) and that the encoder (3) can be connected in the time domain Therefore, it can be executed in the time domain. Further, in the decoding system, the perceptual stereo decoder 11 and the PS decoder 13 are preferably connected in the time domain. Accordingly, the conversion of the pseudo stereo signal L _p , R _p into the downmix signal DMX and the residual signal RES in the conversion stage 12 can also be performed in the time domain.

An adaptive L / R or M / S stereo coder as shown in Fig. 1 as encoder 3 is typically a cognitive audio coder that uses a psychoacoustic model to enable high coding efficiency at low bit rates. An example of such an encoder is the AAC encoder, which employs a psychoacoustic model to control time-varying and frequency-variable quantization and employs transcoding in the critical sampling MDCT domain. In addition, time-variable and frequency-variable decisions between L / R and M / S coding are typically controlled with the help of cognitive entropy measurements computed using acoustic psychological models.

The perceptual stereo encoder (such as encoder 3 in Figure 1) operates on pseudo L / R stereo signals (see L _p , R _p in Figure 1). To optimize the coding efficiency of the stereo encoder (in particular to make a correct decision between L / R encoding and M / S encoding), the signal modification (pseudo L / R < / RTI > and RES transformations followed by PS decoding), a control mechanism that determines between the L / R and M / S stereo encodings in the stereo encoder, and the time-variable and frequency- Lt; RTI ID = 0.0 > a < / RTI > control mechanism). These signal modifications can affect the phenomenon of binaural masking that is utilized in acoustic psychological control mechanisms. Therefore, these acoustic psychological control mechanisms should preferably be adapted accordingly. To this end, not only the acoustic psychological control mechanisms can access the pseudo L / R signal (see L _p , R _p in FIG. 1) but also PS parameters (see 5 in FIG. 1) and / or the original stereo signal L , R). ≪ / RTI > The PS parameters and the access of the acoustic psychological control mechanisms to the stereo signals (L, R) are indicated by dashed lines in FIG. Based on this information, for example, the masking threshold value (s) may be adapted.

An alternative approach to optimizing acoustical psychological control is to add to the encoder system a detector that, when appropriate, forms a deactivation stage that can effectively deactivate the PS encoding, preferably in a time-variable and frequency-variable manner. Deactivating the PS encoding is suitable, for example, when L / R stereo coding is expected to be advantageous, or when acoustic psychological control seems to be a problem in efficiently encoding the pseudo L / R signal. PS encoding the downmix matrix H ^-1, and, following this conversion (see stage 2 in Fig. 1) (i.e., the identity operation) or 1 times the identity matrix corresponding to the downmix matrix H ^-1 to the identity matrix Can be effectively deactivated. For example, the PS encoding may be effectively deactivated by forcing the PS parameters IID and / or ICC to be IID = 0 dB and ICC = 0. In this case, the pseudo stereo signal (L _p , R _p ) corresponds to the stereo signal (L, R) as discussed above.

This detector, which controls PS parameter modification, is shown in FIG. Here, the detector 20 receives the PS parameters 5 determined by the parameter estimation stage 9. When the detector does not deactivate the PS encoding, the detector 20 sends the PS parameters via the downmix stage 8 to the multiplexer 7, in which case the PS parameters 5 are sent to the downmix stage 8 Corresponds to the supplied PS parameters 5 '. If the detector detects (for one or more frequency bands) that the PS encoding is not beneficial and the PS encoding should be deactivated, the detector may modify the affected PS parameters 5 (e.g., IID and / or ICC to IID = 0 dB and ICC = 0) to the downmix stage 8. The detector may also consider left and right signals L, R to determine for PS parameter modification (see dotted line in FIG. 4).

In the following, the term QMF (quadrature mirror filter or filter bank) includes a QMF sub-band filter bank in combination with a Nyquist filter bank, i.e., a hybrid filter bank structure. Also, all values in the following description may be frequency dependent, e.g., different downmix and upmix matrices may be extracted for different frequency ranges. In addition, the residual coding may only include portions of the audio frequency range used (i.e., the residual signal is coded only for the portion of the audio frequency range used). The features of the downmix outlined below will be done in the QMF domain (e.g., according to the prior art) for some frequency ranges and only the phase planes, for example, for the other frequency ranges, will be processed in the complex QMF domain The amplitude transformation is processed in the real-valued MDCT domain.

A typical PS encoder system is shown in Fig. Each of the stereo channels L, R is first analyzed by a QMF with a complex QMF 30 with M sub-bands, e.g., M = 64 sub-bands. The sub-band signals are used in the PS encoder 31 to estimate the PS parameters 5 and the downmix signal DMX. The downmix signal DMX is used to estimate the SBR (Spectrum Bandwidth Replication) parameters 33 in the SBR encoder 32. The SBR encoder 32 extracts the SBR parameters 33 representing the spectral envelope of the original high-band signal, possibly in combination with the noise and tonality measurements. Contrary to the PS encoder 31, the SBR encoder 32 does not affect the signal transmitted to the core coder 34. [ The downmix signal DMX of the PS encoder 31 is synthesized using the inverse QMF 35 with N sub-bands. For example, a complex QMF with N = 32 may be used, where only the 32 lowest sub-bands out of the (64) sub-bands used by PS encoder 31 and SBR encoder 32 . Thus, by using half the number of sub-bands for the same frame size, a time-domain signal of half the bandwidth compared to the input is obtained and sent to the core coder 34. [ Due to the reduced bandwidth, the sampling rate may be reduced by half (not shown). The core encoder 34 performs perceptual encoding of the mono input signal to produce a bit stream 36. The PS parameters 5 are inserted into the bit stream 36 by a multiplexer (not shown).

Figure 6 illustrates another embodiment of an encoder system that combines PS coding using residuals with a stereo core coder 48 capable of stereo coding with adaptive L / R or M / S. This embodiment is only intended to illustrate the principles of the present invention. Modifications and variations of the embodiments will be apparent to those skilled in the art. The input channels L, R representing the left and right source channels are analyzed by the complex QMF 30 in a manner similar to that discussed with respect to Fig. In contrast to the PS encoder 31 in Fig. 5, the PS encoder 41 in Fig. 6 outputs not only the downmix signal DMX but also the residual signal RES. The DMX / RES conversion (i.e., M / S to L / R conversion) fixed with the pseudo L / R is applied to the downmix (DMX) and residual (RES) signals in the conversion stage 2. The conversion stage 2 in Fig. 6 corresponds to the conversion stage 2 in Fig. The conversion stage 2 generates the "pseudo" left and right channel signals L _p and R _p to which the core encoder 48 will operate. In this embodiment, prior to sub-band synthesis by filter banks 35, an inverse conversion from L / R to M / S is applied in the QMF domain. Preferably, the number N of sub-bands for synthesis (e.g., N = 32) corresponds to half the number of sub-bands M used for analysis (e.g., M = (64) The core coder 48 operates at half the sampling rate. There is no restriction to use sub-band channels 64 for QMF analysis in the encoder and 32 sub-bands for synthesis, and depending on what sampling rate is desired for the signal received by the core coder 48, Note that values are also possible. The core stereo encoder 48 performs perceptual encoding of the signals in the filter banks 35 to generate a bitstream signal 46. The PS parameters 5 are inserted into the bitstream signal 46 by a multiplexer (not shown). Alternatively, the PS parameters and / or the original L / R input signal may be utilized by the core encoder 48. [ This information informs the core encoder 48 how the PS encoder 41 has rotated the stereo space. The information can guide the core encoder 48 how to control the quantization to be cognitively optimal. This is indicated by a dotted line in Fig.

Figure 7 illustrates another embodiment of an encoder system similar to the embodiment of Figure 6. Compared with the embodiment in Fig. 5, in Fig. 7, the SBR encoder 42 is connected to the upstream side of the PS encoder 41. Fig. 7, the SBR encoder 42 is moved to the front side of the PS encoder 41, so that the SBR encoder 42 is not operated on the downmix signal DMX as shown in FIG. 6, but on the left and right channels (here, in the QMF region) Lt; / RTI >

Due to the rearrangement of the SBR encoder 42, the PS encoder 41 may be configured to operate in the frequency range below the SBR crossover frequency as well as the full bandwidth of the input signal. In Fig. 7, the SBR parameters 43 are in stereo for the SBR range, and the output from the corresponding PS decoder will generate the stereo source frequency range over which the SBR decoder will operate, as discussed later in connection with Fig. This modification, i.e., connecting the SBR encoder module 42 upstream of the PS encoder module 41 in the encoder system and correspondingly placing the SBR decoder module after the PS decoder module in the decoder system (see FIG. 15) The advantage is that the use of the decorrelated signal to produce an output can be reduced. Note that if there is no residual signal at all or in a particular frequency band, then an undocoded version of the downmix signal DMX is used in the PS decoder instead. However, reconstruction based on the decorrelated signal reduces audio quality. Therefore, not using the decorrelated signal increases the audio quality.

This advantage of the embodiment in Fig. 7 compared to the embodiment in Fig. 6 will now be explained in more detail with reference to Figs. 8a-8d.

In Figure 8A, one of the two output channels (L, R) (on the decoder side) is visualized in time frequency. In the case of FIG. 8A, an encoder in which a PS encoding module is placed before the SBR encoding module, such as an encoder in FIG. 5 or 6, is used (see FIG. 14 in a decoder in which the PS decoder is placed after the SBR decoder). In addition, the residual is coded only in the low bandwidth frequency range 50, which is less than the frequency range 51 of the core coder. As is apparent from the spectrogram visualization in Fig. 8A, the frequency range 52, in which the decorrelated signal is used by the PS decoder, is the sum of all frequencies (50) except for the lower frequency range 50 Range. The SBR also includes a frequency range 53 that starts at a significantly higher range than the decorrelated signal. Thus, the entire frequency range is divided into the following frequency ranges: in the low frequency range (see range 50 in FIG. 8A), waveform coding is used; In the intermediate frequency range (see the intersecting range of frequency range 51 and frequency range 52), waveform coding is used in combination with the decorrelated signal; In the high frequency range (see frequency range 53), the SBR reproduction signal reproduced from the low frequencies is used in combination with the decorrelated signal generated by the PS decoder.

In Figure 8b, two output channels (L, R) (on the decoder side) for the SBR encoder connected upstream of the PS encoder in the encoder system (and the SBR decoder is located after the PS decoder in the decoder system) Lt; RTI ID = 0.0 > 1 < / RTI > In FIG. 8B, a low bit rate scenario is shown in which the residual signal bandwidth 60 (residual coding is performed on it) is lower than the bandwidth of the core coder 61. Since the SBR decoding process operates on the decoder after the PS decoder (see FIG. 15), the residual signal used for the lower frequencies is at least part of the higher frequencies (see frequency range 64) in SBR range 63 It is also used for reconstruction.

The advantage becomes even more apparent when the residual signal bandwidth is close to or equal to the core coder bandwidth and operates at the same intermediate bit rates. In this case, the time frequency representation of FIG. 8A (where the order of PS encoding and SBR encoding as shown in FIG. 6 is used) becomes the time frequency representation shown in FIG. 8C. In Figure 8c, the residual signal essentially comprises the full low band range 51 of the core coder, where the decorrelated signal is used by the PS decoder in the SBR frequency range 53. [ In FIG. 8D, a time-frequency representation is visualized in the case of a preferred order of encoding / decoding modules (i.e., SBR encoding operating on a stereo signal prior to PS encoding, as shown in FIG. 7). Here, the PS decoding module operates before the SBR decoding module in the decoder, as shown in Fig. Thus, the residual signal is part of the low band used for high frequency reconstruction. When the residual signal bandwidth is equal to the mono downmix signal bandwidth, no decoded signal information will be required to decode the output signal (see the full frequency range hatched in FIG. 8D).

In FIG. 9A, an embodiment of a stereo core encoder 48 for L / R or M / S stereo encoding that is adaptively selectable in the MDCT transform domain is shown. Such a stereo encoder 48 can be used in Figs. 6 and 7. Fig. The mono core encoder 34 as shown in Fig. 5 can be regarded as a special case of the stereo core encoder 48 in Fig. 9A, in which case only a single mono input channel is processed (i.e., There is no second input channel indicated by < / RTI >

In Figure 9b, an embodiment of a more generalized encoder is shown. For mono signals, the encoding can switch between coding in the linear prediction region (see block 71) and coding in the transform domain (see block 48). This type of core coder introduces several coding methods that can be used adaptively according to the characteristics of the input signal. Here, the coder uses the AAC style conversion coder 48 (which can be used for mono and stereo signals, where L / R or M / S coding can be adaptively selected for stereo signals) or AMR- And may choose to code the signal using a WB + (adaptive multirate-broadband plus) style core coder 71 (which may only be used for mono signals). The AMR-WB + core coder 71 evaluates the residual of the linear predictor 72 and then selects between the linear predictive residual coding or the classical speech coder ACELP (Algebraic Code Excited Linear Prediction) technique for coding the linear prediction residual do. A mode determination stage 73 is used to determine between the two coder 48 and the coder 71 based on the input signal to determine between the AAC style conversion coder 48 and the AMR-WB + style core coder 71. [

The encoder 48 is a stereo AAC style MDCT based coder. When the mode determination 73 adjusts the input signal to use MDCT-based coding, the mono input signal or the stereo input signals are coded by the AAC-based MDCT coder 48. The MDCT coder 48 performs MDCT analysis of one or two signals in the MDCT stages 74. In the case of a stereo signal, the M / S or L / R decision based on the frequency band is performed in the stage 75 prior to quantization and coding. L / R stereo encoding or M / S stereo encoding can be selected in a frequency-variable manner. The stage 75 performs L / R to M / S conversion. If M / S encoding is determined for a particular frequency band, the stage 75 outputs an M / S signal for this frequency band. Otherwise, the stage 75 outputs the L / R signal for this frequency band.

Thus, when a transform coding mode is used, the overall efficiency of the stereo coding function of the basic core coder can be used for stereo.

When the mode determination 73 adjusts the mono signal to the linear prediction area coder 71, the mono signal is then analyzed at block 72 by linear prediction analysis. Subsequently, a determination is made whether the LP residual is to be coded by the time-domain ACELP style coder 76 or by the TCX style coder 77 (Transform Coded eXcitation) operating in the MDCT domain. The linear prediction area coder 71 has no inherent stereo coding capability. Therefore, in order to be able to code the stereo signal with the linear prediction area coder 71, an encoder configuration similar to that shown in Fig. 5 can be used. In this configuration, the PS encoder generates PS parameters 5 and a mono downmix signal DMX, which is encoded by a linear predictive region coder.

Figure 10 illustrates another embodiment of an encoder system in which the portions of Figures 7 and 9 are combined in a novel manner. 7, the DMX / RES versus pseudo L / R block 2 is placed in the AAC style downmix coder 70 before the stereo MDCT analysis 74. As shown in FIG. This embodiment has the advantage that the DMX / RES to pseudo L / R conversion 2 is applied only when a stereo MDCT core coder is used. Thus, when a transform coding mode is used, the overall efficiency of the stereo coding function of the basic core coder can be used for stereo coding of the frequency range contained by the residual signal.

In Figure 9B, when mode determination 73 operates on a mono input signal or an input stereo signal, mode determination 73 'in Figure 10 operates on the downmix signal DMX and the residual signal RES. In the case of a mono input signal, the mono signal can be used directly as a DMX signal, the RES signal is set to zero, and the PS parameters can be defaulted to IID = 0 dB and ICC = 1.

When the mode determination 73 'adjusts the downmix signal DMX to the linear prediction area coder 71, the downmix signal DMX is then analyzed at block 72 by a linear prediction analysis. It is then determined whether the LP residual is to be coded by the time-domain ACELP style coder 76 or by the TCX style coder 77 (Transform Coded eXcitation) operating in the MDCT domain. The linear prediction region coder 71 has no inherent stereo coding capability that can be used to code the residual signal in addition to the downmix signal DMX. Therefore, a dedicated residual coder 78 is employed to encode the residual signal RES when the downmix signal DMX is encoded by the predictive area coder 71. [ For example, the coder 78 may be a mono AAC coder.

It should be noted that the coder 71,78 can be omitted in Fig. 10 (in which case the mode determination stage 73'is no longer needed).

11A shows details of yet another alternative embodiment of an encoder system that achieves the same benefits as shown in FIG. 11A, the DMX / RES to pseudo L / R transform 2 is placed after the MDCT analysis 74 of the core coder 70, i.e., the transform operates in the MDCT domain . In block 2, the transform is linear and time-invariant and, therefore, can be placed after the MDCT analysis 74. The remaining blocks of FIG. 10, which are not shown in FIG. 11, may optionally be added in the same manner in FIG. 11A. Alternatively, MDCT analysis blocks 74 may be placed after conversion block 2.

FIG. 11B illustrates an implementation of the embodiment in FIG. 11A. In FIG. 11B, an implementation of stage 75 for selecting between M / S or L / R encoding is shown. Stage 75 includes sum and difference transform stages 98 (more precisely, L / R transform stages in M / S) that receive pseudo stereo signals L _p and R _p . The conversion stage 98 generates the pseudo mid / side signals M _p and S _p by performing the L / R conversion to the M / S. Except for the possible gain rates, the following applies: M _p = DMX and S _p = RES.

Stage 75 determines one of L / R or M / S encoding. Based on the judgment, the pseudo stereo signal (L _p , R _p ) or the pseudo mid / side signal (M _p , S _p ) is selected and encoded in the AAC block (97). A first AAC block 97 is assigned to the pseudo stereo signal L _p and R _p and a second AAC block 97 is assigned to the pseudo mid / side signals M _p and S _p , 97) may be used (not shown in Figure 11B). In this case, the L / R or M / S selection is performed by selecting the output of the first AAC block 97 or the output of the second AAC block 97.

Figure 11C shows an alternative to the embodiment of Figure 11A. Here, no explicit conversion stage 2 is used. Rather, the conversion stage 2 and the stage 75 are combined in a single stage 75 '. The downmix signal DMX and the residual signal RES are supplied to the sum and difference conversion stage 99 (more precisely, the conversion stage of the DMX / RES in the pseudo L / R) as part of the stage 75 '. The conversion stage 99 generates a pseudo stereo signal L _p , R _p . 11C, the DMX / RES vs. pseudo L / R conversion stage 99 is similar to the L / R to M / S conversion stage 98 (except for other gain factors) in FIG. Nevertheless, the choice between M / S and L / R decoding in Fig. 11c needs to be reversed compared to Fig. 11b. And the note shown in both Fig. 11b and Fig. 11c, a L _p / R _p where the L / R or bottom position In Figure 11c is a switch located in the upper position In Figure 11b for the M / S selection. This is a visual representation of the reversed L / R or M / S selection.

In Figs. 11B and 11C, it is noted that the switch preferably exists separately for each frequency band in the MDCT region so that the choice between L / R and M / S is time-varying and frequency-variable. That is, the position of the switch is preferably frequency-variable. Conversion stages 98 and 99 may convert the entire frequency range used or only a single frequency band.

It is also noted that all blocks 2, 98 and 99 can be referred to as " summed transform blocks "since all blocks implement the following type of transform matrix.

However, the gain factor c may be different in blocks 2, 98, 99.

Another embodiment of the encoder system is outlined in Fig. This can be summed characterized by a phase relationship between the two channels (L) and the channel (R) (refer to the phase difference, or less φ _ipd between channels) IID addition ICC (described above) is 2 as the parameter of the additional IPD and the stereo signal utilizes the PS parameters of a set extension containing OPD (see full phase, more than φ _opd) in. Examples of these phase parameters are given in ISO / IEC 14496-3 subclause 8.6.4.6.3, which is incorporated herein by reference. When phase parameters are used, the resulting upmix matrix (H _COMPLEX ) (and its inverse H ^-1 _COMPLEX ) becomes a complex value according to the following equation:

H _COMPLEX = H _? H

From here,

Lt; / RTI >

to be.

The stage 80 of the PS encoder operating in the complex QMF domain handles only the phase dependency between the channels L and R. [ The downmix rotation (i.e., the conversion from the L / R region to the DMX / RES region described above by matrix H ^-1 ) is processed in the MDCT region as part of the stereo core coder 81. Therefore, the phase dependence between the two channels is extracted in the complex QMF domain, and the real-valued waveform dependence is extracted in the critical sampling MDCT domain, which is a real value as part of the stereo coding mechanism of the used core coder. This allows the extraction of linear dependencies between channels to be reliably integrated into the stereo coding of the core coder (however, in order to avoid aliasing in the critical sampling MDCT domain, a "guard "quot; guard band ") is advantageous.

In Fig. 12, the PS encoder phase adjustment stage 80 extracts PS parameters related to the phase, e.g., parameters IPD (interchannel phase difference) and OPD (full phase difference). Therefore, the phase control matrix H ^-1 that it generates can be:

As discussed above, the downmix rotation portion of the PS module is processed in the stereo coding module 81 of the core coder in Fig. The stereo coding module 81 operates in the MDCT domain and is shown in FIG. The stereo coding module 81 receives the phase adjusted stereo signals L [ _phi] , R [ _phi ] in the MDCT domain. This signal is downmixed in the downmix stage 82 by the downmix rotation matrix H ^-1 , which is the real part of the complex number downmix matrix H ^-1 _COMPLEX , as discussed above, DMX) and a residual signal (RES). The downmix operation is followed by an inverse L / R to M / S conversion according to the present invention (see conversion stage 2), thereby generating a pseudo stereo signal L _p , R _p . The pseudo-stereo signal (L _p , R _p ) may be represented by a stereo coding algorithm (see adaptive M / S or L / R stereo encoder 83), in this particular embodiment an L / RTI ID = 0.0 > / S < / RTI > This determination is preferably time-variable and frequency-variable.

An embodiment of a decoder system suitable for decoding the bit stream 46 generated by the encoder system shown in FIG. 14 to FIG. 6 is shown. This embodiment is only intended to illustrate the principles of the present invention. Modifications and variations of the embodiments will be apparent to those skilled in the art. The core decoder 90 decodes the bit stream 46 into pseudo left channel and right channel, which are transformed in the QMF domain by the filter banks 91. The fixed pseudo L / R to DMX / RES conversions of the resulting pseudo stereo signals L _p and R _p are then performed in the conversion stage 12 and thus the downmix signal DMX and the residual signal RES, . When using SBR coding, these signals are low-band signals, for example, the downmix signal DMX and the residual signal RES may only contain audio information for a low frequency band of up to approximately 8 kHz. The downmix signal DMX is used by the SBR decoder 93 to reconstruct the high frequency band based on the received SBR parameters (not shown). Both the output signal (including the low and reconstructed high frequency bands of the downmix signal DMX) and the residual signal RES from the SBR decoder 93 operate in the QMF domain (especially in the hybrid QMF + Nyquist filter domain) And is input to a PS decoder 94 which performs a decoding process. The downmix signal DMX at the input of the PS decoder 94 also contains audio information in the high frequency band (e.g., up to 20 kHz), while the residual signal RES at the input of the PS decoder 94, Signal (e.g., limited to 8 kHz). Thus, for a high frequency band (e.g., for the 8 kHz to 20 kHz band), the PS decoder 94 may use the band-limited residual signal RES instead of using the down-mix signal DMX Version. Therefore, the decoded signals at the output of the PS decoder 94 are based on residual signals only up to 8 kHz. After PS decoding, the two output channels of the PS decoder 94 are transformed in the time domain by filter banks 95, thereby generating an output stereo signal L, R.

An embodiment of a decoder system suitable for decoding the bit stream 46 generated by the encoder system shown in Fig. 7 is shown in Fig. This embodiment is only intended to illustrate the principles of the present invention. Modifications and variations of the embodiments will be apparent to those skilled in the art. The main operation of the embodiment in Fig. 15 is similar to the main operation of the decoder system outlined in Fig. Contrary to FIG. 14, the SBR decoder 96 is located at the output of the PS decoder 94 in FIG. In addition, the SBR decoder uses SBR parameters (not shown) that form the stereo envelope data as opposed to the mono SBR parameters in FIG. The downmix signal and the residual signal at the input of the PS decoder 94 are typically low band signals such that the downmix signal DMX and the residual signal RES are at a low frequency Only the audio information for the band can be contained. Based on the low-band downmix signal DMX and the residual signal RES, the PS encoder 94 determines a low-band stereo signal, for example, up to approximately 8 kHz. Based on the low-band stereo signal and the stereo SBR parameters, the SBR decoder 96 reconstructs the high frequency portion of the stereo signal. Compared with the embodiment in Fig. 14, the embodiment of Fig. 15 provides the advantage that no decorrelated signal is required (see Fig. 8d) and thus an enhanced audio quality is achieved, but in Fig. 14, A correlated signal is required (see FIG. 8C), and audio quality is reduced.

FIG. 16A illustrates an embodiment of a decoding system that is opposite to the encoding system shown in FIG. 11A. An input bitstream signal is supplied to a decoder block 100 and a decoder block 100 generates a first decoded signal 102 and a second decoded signal 103. In the encoder, M / S coding or L / R coding was selected. This is specified in the received bit stream. Based on this information, M / S or L / R is selected in the selection stage 101. When M / S is selected in the encoder, the first 102 and the second 103 signals are converted into a (pseudo) L / R signal. When L / R is selected in the encoder, the first 102 and the second 103 signals can pass through the stage 101 without conversion. At the output of the stage 101, the pseudo L / R signals L _p and R _p are converted into DMX / RES signals by the conversion stage 12 (this stage performs apparent L / R versus M / S conversion) do. Preferably, the stages 100, 101, 12 in FIG. 16A operate in the MDCT domain. In order to convert the downmix signal DMX and the residual signals RES into time domain, the transform blocks 104 may be used. The resulting signal is then supplied to a PS decoder (not shown), and optionally to an SBR decoder, as shown in FIGS. 14 and 15. FIG. Alternatively, blocks 104 may be placed before block 12.

FIG. 16B shows an embodiment of the embodiment in FIG. 16A. In Fig. 16B, an example of the stage 101 for selecting between M / S or L / R decoding is shown. Stage 101 includes a sum-of-transformation stage 105 (M / S to L / R conversion) for receiving first 102 and second 103 signals.

Based on the encoding information given to the bitstream, the stage 101 selects L / R or M / S decoding. When L / R decoding is selected, the output signal of the decoding block 100 is supplied to the conversion stage 12.

Figure 16c shows an alternative to the embodiment of Figure 16a. No bright conversion stage 12 is used here. Rather, the conversion stage 12 and the stage 101 are combined into a single stage 101 '. The first 102 and the second 103 signals are supplied to the Sum conversion stage 105 '(more precisely, the pseudo L / R versus DMX / RES conversion stage) as part of stage 101'. The translation stage 105 'generates a DMX / RES signal. In FIG. 16C, the conversion stage 105 'is similar or equivalent to the conversion stage 105 in FIG. 16B (except perhaps for the other gain factors). 16C, the selection between M / S and L / R decoding needs to be reversed in comparison with FIG. 16B. In Fig. 16C, the switch is in the lower position, and in Fig. 16B, the switch is in the upper position. This is a reverse visualization of the L / R or M / S selection (the selection signal can simply be inverted by the inverter).

16B and 16C, it is noted that the switch preferably exists separately for each frequency band in the MDCT domain so that the choice between L / R and M / S can be time-varying and frequency-variable. The conversion stages 105 and 105 'may convert the entire used frequency range or may convert only a single frequency band.

17 shows another embodiment of an encoding system for coding a stereo signal (L, R) into a bitstream signal. The encoding system includes a downmix stage 8 for generating a downmix signal DMX and a residual signal RES based on a stereo signal. The encoding system also includes a parameter determination stage 9 for determining one or more parametric stereo parameters 5.

The encoding can be selected as follows:

- encoding based on the sum signal of the downmix signal DMX and the residual signal RES and based on the difference signal of the downmix signal DMX and the residual signal RES,

- An encoding based on the downmix signal (DMX) and the residual signal (RES).

Preferably, the selection is time-variable and frequency-variable.

The encoding means 110 includes a sum-of-transformation stage 111 for generating sum signal (s). The encoding means 110 also includes a selection block 112 for selecting the encoding based on the summed signals or based on the downmix signal DMX and the residual signal RES. An encoding block 113 is also provided. Alternatively, two encoding blocks 113 may be used, wherein the first encoding block 113 encodes the DMX and RES signals and the second encoding block 113 encodes the sum signals. In this case, the selection 112 is on the downstream side of the two encoding blocks 113.

In block 111, the Sum transformation is of the following form.

The conversion block 111 may correspond to the conversion block 99 in Fig. 11C.

The output of the cognitive encoder 110 is combined with the parametric stereo parameters 5 in the multiplexer 7 to form the resulting bitstream 6.

17, the encoding based on the downmix signal DMX and the residual signal RES converts the downmix signal DMX and the residual signal RES into two series of sum-of-products conversion as shown in Fig. (See two conversion blocks 2 and 98). [0031] In the case of the first and second conversion blocks 2 and 98, The resultant signal after the two summation transforms corresponds to the downmix signal DMX and the residual signal RES (except for possibly other gain factors).

Figure 18 illustrates an embodiment of a decoder system that is opposite to the encoder system in Figure 17; The decoder system includes means (120) for perceptual decoding based on the bitstream signal. Before decoding, the PS parameters are separated from the bitstream signal 6 in the demultiplexer 10. The decoding means 120 includes a core decoder 121 for generating a first signal 122 and a second signal 123 (by decoding). The decoding means outputs the downmix signal DMX and the residual signal RES.

The downmix signal DMX and the residual signal RES are alternately,

Based on the difference between the sum of the first signal 122 and the second signal 123 and the first signal 122 and the second signal 123,

- based on the first signal 122 and the second signal 123.

Preferably, the selection is time-variable and frequency-variable. Selection is performed in the selection stage 125.

The decoding means 120 includes a sum-of-transformation stage 124 for generating summed signals.

In block 124, the sum-of-products transformation is of the following form.

Conversion block 124 may correspond to conversion block 105 'in Figure 16C.

After selection, the DMX and RES signals are supplied to the upmix stage 126 for generating the stereo signals L and R based on the downmix signal DMX and the residual signal RES. The upmix operation is dependent on the PS parameters 5.

Preferably, the selection in Figures 17 and 18 is frequency-variable. In Figure 17, for example, a time-to-frequency conversion (e.g., by an MDCT or an analysis filter bank) may be performed as a first step in the perceptual encoding means 110. In FIG. 18, for example, a frequency-to-time conversion (e.g., by an inverse MDCT or synthesis filter bank) may be performed as a last step in the perceptual decoding means 120.

It is noted that in the embodiments described above, signals, parameters and matrices may be frequency-variable or frequency-invariant and / or time-variable or time-invariant. The described computation steps may be performed on a frequency-by-frequency basis or over an entire audio band.

In addition, it is also possible to use various sums of transformations (DMX) / RES to pseudo L / R conversion, pseudo L / R to DMX / RES conversion, L / R to M / S conversion, and M / S to L / Note that the following form is used.

However, the gain factor c may be different from each other. Therefore, in principle, each of these transforms can be exchanged for another transform of these transforms. If the gain is not correct during the encoding process, this can be compensated in the decoding process. Further, when two identical or two different summed transformations are performed in a series, the resulting transform corresponds to an identity matrix (possibly multiplied by the gain factor).

In an encoder system that includes both PS encoder and SBR encoder, different PS / SBR configurations are possible. In the first configuration shown in Fig. 6, the SBR encoder 32 is connected to the downstream side of the PS encoder 41. Fig. In the second configuration shown in Fig. 7, the SBR encoder 42 is connected to the upstream side of the PS encoder 41. In Fig. Depending on, for example, the desired target bit rate, the characteristics of the core encoder, and / or one or more various other factors, one of the configurations may be preferable to other configurations to provide the best performance. Typically, for low bit rates, a first configuration may be preferred and a second configuration for high bit rates may be desirable. Thus, for example, it is desirable if the encoder system supports all of the different configurations in order to be able to select the desired configuration according to the desired target bit rate and / or one or more other criteria.

Also, in a decoder system that includes both a PS decoder and an SBR decoder, different PS / SBR configurations are possible. In the first configuration shown in FIG. 14, the SBR decoder 93 is connected to the upstream side of the PS decoder 94. In the second configuration shown in FIG. 15, the SBR decoder 96 is connected to the downstream side of the PS decoder 94. In order to achieve correct operation, the configuration of the decoder system must match the configuration of the encoder system. If the encoder is constructed according to Fig. 6, the decoder is correspondingly constructed according to Fig. If the encoder is constructed according to FIG. 7, the decoder is correspondingly configured according to FIG. To ensure correct operation, the encoder preferably informs the decoder which PS / SBR configuration was selected for encoding (and therefore which PS / SBR configuration is to be selected for decoding). Based on this information, the decoder selects a suitable decoder configuration.

As discussed above, in order to ensure correct decoder operation, here is a mechanism to inform the decoder in the encoder which configuration is preferably to be used in the decoder. This can be done explicitly (for example, by dedicated bits or fields in the configuration header of the bitstream as discussed below) or implicitly (e.g., if the PS data is present, whether the SBR data is mono or stereo) Check).

As discussed above, a dedicated element may be used in the bitstream header of the bitstream delivered from the encoder to the decoder, signaling the selected PS / SBR configuration. This bitstream header conveys necessary configuration information so that the decoder can correctly decode the data in the bitstream. A dedicated element in the bitstream header may be, for example, a one-bit flag, a field, or it may be an index pointing to a particular entry in the table that specifies different decoder configurations.

Instead of including in the bitstream header an additional dedicated element for informing the PS / SBR configuration, the information already in the bitstream can be evaluated in the decoding system to select the correct PS / SBR configuration. For example, the selected PS / SBR configuration may be derived from the bitstream header configuration information for the PS decoder and the SBR decoder. This configuration information typically indicates whether the SBR decoder should be configured for mono operation or stereo operation. For example, if the PS decoder is activated and the SBR decoder is configured for mono operation (as indicated in the configuration information), the PS / SBR configuration according to FIG. 14 can be selected. If the PS decoder is enabled and the SBR decoder is configured for stereo operation, the PS / SBR configuration according to FIG. 15 may be selected.

The embodiments described above merely illustrate the principles of the present invention. It will be appreciated that modifications and variations of the configurations and details described herein will be apparent to those skilled in the art. Therefore, the scope of the present invention is not limited to the specific details presented by the description of the embodiments herein.

The systems and methods disclosed herein may be implemented as software, firmware, hardware, or a combination thereof. Some or all of the components may be implemented as software executing in a digital signal processor or microprocessor, or as hardware and / or as application specific integrated circuits.

Exclusive devices that utilize the disclosed systems and methods are portable audio players, mobile communication devices, set-top boxes, TV, AVR (audio-video receiver), personal computers.

1, 31: PS encoder 3: Acknowledgment stereo encoder
7: Multiplexer
11: Acknowledgment stereo decoder 13: PS decoder
20: Detector 32, 42: SBR encoder
34: core coder 41: PS encoder module
48: Stereo Core Coder 71: Linear Prediction Area Coder
78: dedicated residual coder

Claims (20)

1. An encoder system configured for encoding a stereo signal into a bitstream signal (6) comprising:
Downmixing means (8) configured to generate a downmix signal and a residual signal based on the stereo signal;
Parameter determining means (9) configured to determine one or more parametric stereo parameters; And
(2, 3) on the downstream side of said downmixing means (8), said perceptual encoding means (2, 3) being configured to encode said downmix signal and said residual signal, The cognitive encoding means (2, 3) are configured to select left / right encoding or mid / side encoding. The method of claim 1,
Wherein the perceptual encoding means (2, 3) comprise:
- conversion means (2) configured to perform a conversion based on the downmix signal and the residual signal and thereby generate a pseudo left / right stereo signal; And
- a cognitive encoder (3, 48) configured for encoding said pseudo left and right stereo signals, said cognitive encoder (3, 48)
- Left / right encoding, or
- mid / side encode encoding. 3. The method of claim 2,
The cognitive encoder 3 is frequency-variable or frequency-invariant based on the pseudo stereo signal,
- left / right encoding, or
- < / RTI > mid / side encoding. The method according to claim 2 or 3,
The cognitive encoder (3, 48) is configured to perform a left / right to mid / side transform (98) based on the pseudo stereo signal. The method according to any one of claims 1 to 4,
The parametric stereo parameters (5)
A frequency-variable or frequency-invariant parameter representing the interchannel intensity difference; And
- a frequency-variable or frequency-invariant parameter indicative of inter-channel cross-correlation. 6. The method according to any one of claims 2 to 5,
The first channel of the pseudo stereo signal is proportional to the sum of the downmix and residual signals;
The second channel of the pseudo stereo signal is proportional to the difference between the downmix and the residual signals. 7. The method according to any one of claims 1 to 6,
Wherein the perceptual encoding means (3) comprises an AAC based stereo encoder (48). The method according to any one of claims 1 to 7,
Characterized in that the perceptual encoding means (3) comprises a psychoacoustic control mechanism,
- at least one of said parametric stereo parameters, and / or
- an encoder system capable of accessing said stereo signal. The method according to any one of claims 1 to 8,
Wherein the encoder system further comprises an SBR encoder (32). The method of claim 9,
Wherein the SBR encoder (32) is connected to the upstream side of the downmixing means (8). A decoder system configured to decode a bitstream signal comprising one or more parametric stereo parameters (5) into a stereo signal, the system comprising:
Perceptual decoding means (11, 12) for decoding on the basis of the bitstream signal (6), wherein said decoding means (11, 12) are configured to generate a downmix signal and a residual signal, said decoding means (11) 12) is
- decoding left / right or
Said decoding means (11, 12), configured to selectively perform mid / side aware decoding; And
- an upmixing means (13) configured to perform an upmix operation to generate the stereo signal based on the downmix signal and the residual signal, wherein the upmixing operation of the upmixing means comprises applying the one or more parametric stereo parameters (13), which is dependent on the first and second signals (5). The method of claim 11,
Wherein the perceptual decoding means (11, 12) comprise:
- a perceptual stereo decoder (11) configured to decode based on the bitstream signal (6) and to generate a pseudo-stereo signal,
- decode left or right, or
- the perceptual stereo decoder configured to selectively perform mid / side perceptual decoding; And
- conversion means (12) configured to perform a conversion based on the pseudo-stereo signal and thereby generate the downmix signal and the residual signal. 13. The method of claim 12,
Wherein the perceptual stereo decoder (11) is configured to perform a mid / side to left / right transform (105) based on the decoded pseudo mid / side signal. 14. The method according to any one of claims 11 to 13,
Wherein the parametric stereo parameters comprise:
A frequency-variable or frequency-invariant parameter representing the interchannel intensity difference, and
A frequency-variable or frequency-invariant parameter indicative of inter-channel cross-correlation. 13. The method of claim 12,
The downmix signal being proportional to the sum of the two channels of the pseudo-stereo signal,
The residual signal being proportional to the difference between the two channels of the pseudo-stereo signal. 16. The method according to any one of claims 11 to 15,
Wherein the perceptual decoding means comprises an AAC-based decoder. 17. The method according to any one of claims 11 to 16,
When the left channel of the stereo signal and the right channel of the stereo signal are independent and have the same level for the frequency band, the upmix operation is

, ≪ / RTI >
Here, L represents a frequency band component of the left channel of the stereo signal, R represents a frequency band component of the right channel of the stereo signal, DMX represents a frequency band component of the downmix signal, C denotes the frequency band component of the residual signal, and c is a scaling factor. 18. The method according to any one of claims 11 to 17,
Wherein said decoder system further comprises a SBR decoder, said SBR decoder being located downstream of said upmixing means (13). A method for encoding a stereo signal into a bitstream signal (6) comprising:
Generating a downmix signal and a residual signal based on the stereo signal;
- determining one or more parametric stereo parameters (5);
- cognition encoding after said downmix signal and said residual signal generation,
- Left / right encoding, or
A method for encoding a stereo signal into a bitstream signal, wherein mid / side encoding is selectable. A method for decoding a bitstream signal (6) comprising parametric stereo parameters (5) into a stereo signal, comprising:
- decoding perceptually based on said bitstream signal (6)
- decode left or right, or
- generating a downmix signal and a residual signal by selectively performing mid / side perceptual decoding; And
- generating the stereo signal based on the downmix signal and the residual signal by an upmix operation, wherein the upmixing operation comprises generating the stereo signal, which is dependent on the parametric stereo parameters (5) The method comprising the steps of: generating a stereo signal by decoding a bitstream signal including parametric stereo parameters into a stereo signal.

Images