RU2380767C2 - Method and device for audio signal decoding - Google Patents

Abstract

FIELD: information technologies.

SUBSTANCE: method includes receiving audio signal and spatial information, creation of modified spatial information using spatial information and audio signal decoding using modified spatial information. In this process modified spatial information includes at least one thing from partial spatial information, combined spatial information and extended spatial information. At the same time, audio signal is decoded into configuration that differs from configuration defined by decoder. Even when number of loudspeakers is less or greater than multiple channels number before execution of lowering mixing, it is possible to form output channels from lowering mixing audio signal in the quantity equal to the number of loudspeakers.

EFFECT: providing audio signal generation with configuration that differs from predefined tree-shaped configuration by shaping variously configured output signals; and significant decrease in load of operations required for audio signal decoding.

18 cl, 17 dwg

Description

FIELD OF THE INVENTION

The present invention relates to the processing of audio signals, and more particularly, to an audio decoding apparatus and method thereof. Although the present invention is applicable to a wide range of applications, it is particularly applicable to decoding audio signals.

State of the art

In general, when an encoder encodes an audio signal, in case the audio signal to be encoded is a multi-channel audio signal, the multi-channel audio signal is downmixed to two channels or one channel to generate a down-mix audio signal, and spatial information is extracted from the multi-channel audio signal. Spatial information is information used to up-mix a multi-channel audio signal from a down-mix audio signal. Meanwhile, the encoder down-mixes the multi-channel audio signal according to a predetermined tree configuration. In this case, the predetermined tree configuration may be the structure (s) agreed between the audio decoder and the audio encoder. In particular, if identification information indicating the type of one of the predetermined tree configurations is present, the decoder may know the structure of the audio signal for which upmixing is performed, for example, the number of channels, the position of each channel, etc.

Thus, if the encoder down-mixes the multi-channel signal according to a predetermined tree configuration, the spatial information extracted in this process also depends on the structure. Thus, in the event that the decoder up-mixes the down-mix audio signal using spatial information dependent on the structure, a multi-channel audio signal is generated according to the structure. That is, if the decoder uses the spatial information generated by the encoder, as it is, up-mix is performed according to the structure agreed only between the encoder and the decoder. Thus, it is not possible to generate an audio signal of an output channel unless a consistent structure is followed. For example, it is not possible to up-mix a signal into an audio signal having a channel number different (less or more) from the number of channels determined according to the agreed structure.

SUMMARY OF THE INVENTION

Therefore, the present invention is directed to an apparatus for decoding an audio signal and a method for this, which largely eliminates one or more problems due to limitations and disadvantages of the prior art.

An object of the present invention is to provide an audio signal decoding apparatus and method for doing this, by means of which an audio signal can be decoded so as to have a structure different from that defined by the encoder.

Another object of the present invention is to provide an audio signal decoding apparatus and a method for doing this, by means of which an audio signal can be decoded using spatial information generated from a modification of previous spatial information generated from encoding.

Additional features and advantages of the invention will be set forth in the description that follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention should be realized and achieved through the structure, in particular, disclosed in the written description and claims, as well as in the accompanying drawings.

In order to achieve these and other advantages in accordance with the purpose of the present invention, implemented and described in detail herein, an audio signal decoding method according to the present invention includes receiving an audio signal and spatial information, identifying a type of modified spatial information, generating modified spatial information using spatial information and decoding an audio signal using modified spatial information, the type of modified spatial information includes at least one of the partial spatial information, the combined spatial information and the extended spatial information.

To further achieve these and other advantages in accordance with the purpose of the present invention, a method for decoding an audio signal includes receiving spatial information, generating combined spatial information using spatial information, and decoding an audio signal using combined spatial information, wherein the combined spatial information is generated by combining spatial parameters included in the space actual information.

To further achieve these and other advantages in accordance with the purpose of the present invention, a method for decoding an audio signal includes receiving spatial information including at least one spatial information and spatial filtering information including at least one filtering parameter, the formation of combined spatial information having surround sound effect by combining the spatial parameter and the filtering parameter and converting the audio signal and the virtual surround signal using the combined spatial information.

To further achieve these and other advantages in accordance with the purpose of the present invention, an audio decoding method includes receiving an audio signal, receiving spatial information including tree configuration information and spatial parameters, generating modified spatial information by adding extended spatial information to the spatial information, and upmixing audio using modified space information, which comprises including converting the audio signal to a primary up-mixed audio signal based on spatial information and converting the primary up-mixed audio signal to a secondary up-mixed audio signal based on the extended spatial information.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention according to the claims.

Brief Description of the Drawings

The accompanying drawings, which are included to provide an additional understanding of the invention and are included and form part of this detailed description, illustrate embodiments of the invention and together with the description serve to explain the principles of the invention.

In the drawings:

FIG. 1 is a block diagram of an audio signal encoding apparatus and an audio signal decoding apparatus according to the present invention; FIG.

figure 2 - schematic representation of an example of the application of partial spatial information;

figure 3 - schematic representation of another example of the application of partial spatial information;

4 is a schematic representation of a further example of the application of partial spatial information;

5 is a schematic representation of an example of the application of combined spatial information;

6 is a schematic representation of another example of the application of combined spatial information;

7 is a diagram of the paths of sound from the speakers to the listener, which shows the position of the speakers;

Fig. 8 is a diagram for explaining a signal output from each speaker position for a surround effect;

Fig.9 is a conceptual diagram for explaining a method for generating a 3-channel signal using a 5-channel signal;

10 is a diagram of an example configuration of extended channels based on configuration information of extended channels;

11 is a diagram for explaining the configuration of the extended channels shown in figure 10, and the relationship with the expanded spatial parameter;

12 is a position diagram of a multi-channel audio signal for 5.1 channels and an output channel audio signal for 6.1 channels;

13 is a diagram for explaining the relationship between the position of the virtual sound source and the level difference between the two channels;

14 is a diagram for explaining levels of two rear channels and a level of a rear center channel;

Fig. 15 is a diagram for explaining a position of a multi-channel audio signal for 5.1 channels and a position of an audio signal of an output channel for 7.1 channels;

16 is a diagram for explaining levels of two left channels and a level of a left front side channel (Lfs); and

17 is a diagram for explaining levels of three front channels and a level of a left front side channel (Lfs).

Optimum Mode for Carrying Out the Invention

The following is a detailed description of preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

General terms that are currently globally used are selected as terms in the present invention. In addition, terms are provided that are arbitrarily selected by the applicant for special cases, the detailed meaning of which is explained in detail in the description of the preferred embodiments of the present invention. Therefore, the present invention should not be understood with the names of the terms, but with the meanings of the terms.

First, the present invention generates modified spatial information using spatial information, and then decodes the audio signal using the generated modified spatial information. In this case, spatial information is spatial information extracted during the downmix according to a predetermined tree configuration, and modified spatial information is spatial information newly generated using spatial information.

The present invention is explained in detail with reference to figure 1 as follows.

FIG. 1 is a block diagram of an audio signal encoding apparatus and an audio signal decoding apparatus according to an embodiment of the present invention.

Referring to FIG. 1, an audio signal encoding apparatus 100 (hereinafter abbreviated as encoding apparatus) includes a downmix unit 110 and a spatial information extraction unit 120. In addition, the audio signal decoding apparatus 200 (hereinafter abbreviated decoding apparatus) includes an output signal generating unit 210 and a modified spatial information generating unit 220.

The down-mix unit 110 of the encoding apparatus 100 generates the down-mix audio signal d by down-mix the multi-channel audio signal IN_M. The down-mix audio signal d may be a signal generated from the down-mix of the multi-channel audio signal IN_M by the down-mix unit 110, or an arbitrary down-mix audio signal arbitrarily generated by the user from the down-mix of the multi-channel audio signal IN_M.

The spatial information extraction unit 120 of the encoding apparatus 100 extracts the spatial information s from the multi-channel audio signal IN_M. In this case, the spatial information is the information required to perform the up or down mixing of the audio signal d into the multi-channel audio signal IN_M.

The spatial information may be information extracted during down-mixing of the multi-channel audio signal IN_M according to a predetermined tree configuration. In this case, the tree configuration may correspond to the tree configuration (s) agreed upon between the audio decoding and encoding devices, which is not limited to the present invention.

In addition, spatial information may include tree configuration information, an indicator, spatial parameters, and the like. Tree configuration information is information such as a tree configuration. Thus, the number of channels in a multi-channel configuration, the down-mix sequence per channel, etc. vary according to the type of tree configuration. An indicator is information indicating whether extended spatial information is present or not, etc. In addition, the spatial parameters may include the difference of the channel levels (hereinafter abbreviated CLD) during down-mixing of at least two channels to at most two channels, inter-channel correlation or coherence (hereinafter abbreviated ICC), channel prediction coefficients (hereinafter abbreviated) CPC), etc.

The spatial information extraction unit 120 may further retrieve the extended spatial information as well as the spatial information. In this case, the expanded spatial information is the information required to further expand the downmix audio signal d subjected to upmixing using the spatial parameter. In addition, the extended spatial information may include extended channel configuration information and extended spatial parameters. The extended spatial information, which is explained below, is not limited to information retrieved by the spatial information extraction unit 120.

In addition, the encoding device 100 may further include a coding unit using a basic codec (not shown in the drawing) generating an audio bitstream obtained through downmixing by decoding the downmix audio signal d, a spatial information encoding unit (not shown in the drawing) ), forming a spatial information bitstream by encoding spatial information s, and a multiplexing unit (not shown in the drawing), forming ay bitstream iosignala by multiplexing a bitstream of audio obtained by downmixing, and spatial information bitstream, on which the present invention imposes no restrictions.

In addition, the decoding apparatus 200 may further include a demultiplexing unit (not shown in the drawing) dividing the audio signal bitstream into an audio bitstream obtained through downmixing and a spatial information bitstream, a decoding unit using a base codec (not shown in figure), a decoding audio bitstream obtained through down-mixing, and a spatial information decoding unit (not shown in the drawing), decoding a spatial information bitstream and to which the present invention is not limited.

The modified spatial information generating unit 220 of the decoding apparatus 200 identifies the type of modified spatial information using the spatial information and then generates the modified spatial information s' of the type that is identified based on the spatial information. In this case, the spatial information may be spatial information s carried from the encoding device 100. In addition, modified spatial information is information that is newly generated using spatial information.

Various types of modified spatial information may exist. In addition, various types of modified spatial information may include at least one of: a) partial spatial information, b) combined spatial information, and c) extended spatial information, to which the present invention is not limited.

Partial spatial information includes partial spatial parameters, combined spatial information is generated from a combination of spatial parameters, and extended spatial information is generated using spatial information and extended spatial information.

The modified spatial information generating unit 220 generates the modified spatial information in a manner that can vary according to the type of the modified spatial information. In addition, the method for generating modified spatial information for each type of modified spatial information is explained in more detail below.

The reference data for determining the type of modified spatial information may correspond to tree-like configuration information in spatial information, an indicator in spatial information, output channel information, and the like. Tree configuration information and an indicator may be included in the spatial information s from the encoding device. The output channel information is information for speakers interacting with the decoding device 200, and it may include the number of output channels, position information for each output channel, and the like. The output channel information can be entered in advance by the manufacturer or entered by the user.

The method for determining the type of modified spatial information using these types of information is explained in detail below.

The output channel generating unit 210 of the decoding apparatus 200 generates an output channel audio signal OUT_N from the down-mix audio signal d using the modified spatial information s ′.

The spatial filter information 230 is information for sound paths, and it is provided to the modified spatial information generating unit 220. If the modified spatial information generating unit 220 generates combined spatial information having a surround effect, spatial filtering information may be used.

Next, a method for decoding an audio signal by generating modified spatial information by the type of modified spatial information in the order of (1) partial spatial information, (2) combined spatial information and (3) more detailed spatial information is explained.

(1) Partial spatial information

Since the spatial parameters are calculated during down-mixing of the multi-channel audio signal according to a predetermined tree configuration, the original multi-channel audio signal before down-mixing can be restored if the down-mixing audio signal is decoded using intact spatial parameters. In an attempt to make the number N of channels of the audio signal of the output channel less than the number M of channels of the multi-channel audio signal, the down-mix audio signal can be decoded by partially applying spatial parameters.

This method may vary according to the order and method of down-mixing the multi-channel audio signal in the encoding device, i.e. type of tree configuration. Also, the type of tree configuration can be requested using the tree configuration information from spatial information. In addition, this method may vary according to the number of output channels. Moreover, it is possible to query the number of output channels using the output channel information.

Then, if the number of channels of the audio signal of the output channel is less than the number of channels of the multi-channel audio signal, a method for decoding an audio signal by applying partial spatial information including partial spatial parameters is explained by considering various tree configurations as examples in the following description.

(1) -1. The first tree configuration example (tree configuration 5-2-5)

Figure 2 is a schematic representation of an example of the application of partial spatial information.

The left side of FIG. 2 shows a down-mix sequence of a multi-channel audio signal having 6 channels (left front channel L, left surround channel L _s , center channel C, low-frequency channel LFE, right front channel R, right surround channel R _s ) into the stereo channels L _o and R _o obtained through down-mixing, and the relationship between the multi-channel audio signal and spatial parameters.

First, down-mixing is performed between the left channel L and the left surround channel L _s , down-mixing between the center channel C and the low frequency channel LFE, and down-mixing between the right channel R and the right surround channel R _s . In this primary downmix process, a left aggregate channel L _t , a central aggregate channel C _t, and a right aggregate channel R _{t are formed} . In addition, the spatial parameters computed in this primary downmix process include CLD ₂ (including ICC ₂ ), CLD ₁ (including ICC ₁ ), CLD ₀ (including ICC ₀ ), etc.

In the secondary process following the primary downmix process, joint downmixing of the left aggregate channel L _t , the central aggregate channel C _t and the right aggregate channel R _t is performed to form the left channel L _o and the right channel R _o . In addition, the spatial parameters computed in this secondary downmix process may include CLD _TTT , CPC _TTT , ICC _TTT , etc.

In other words, the down-mix of a multi-channel signal from a total of six channels is performed in the sequential manner described above to form stereo channels L _o and R _o obtained through down-mix.

If the spatial parameters (CLD ₂ , CLD ₁ , CLD ₀ , CLD _TTT , etc.) calculated as described above are used as they are, their up-mix is performed in the reverse order of the down-mix order to form a multi-channel audio signal having the number of channels 6 (left front channel L, left surround channel L _s , center channel C, low-frequency channel LFE, right front channel R, right surround channel R _s ).

According to the right side of FIG. 2, in case partial spatial information corresponds to CLD _TTT from spatial parameters (CLD ₂ , CLD ₁ , CLD ₀ , CLD _TTT , etc.), upmixing to the left aggregate channel L _t , central aggregate is performed channel C _t and right aggregate channel R _t . If the left aggregate channel L _t and the right aggregate channel R _t are selected as the audio signal of the output channel, it is possible to generate an audio signal of the output channel from the two channels L _t and R _t . If the left aggregate channel L _t , the central aggregate channel C _t and the right aggregate channel R _t are selected as the audio signal of the output channel, it is possible to generate an audio signal of the output channel from the three channels L _t , C _t and R _t . After the upmix is done with CLD _1, among other things, if the left aggregate channel L _t , the right aggregate channel R _t , the center channel C and the low frequency channel LFE are selected, the audio signal of the output channel of the four channels (L _t , R _t , C and LFE).

(1) -2. The second example of a tree configuration (tree configuration 5-1-5)

Figure 3 is a schematic representation of another example of the application of partial spatial information.

The left side of FIG. 3 shows the down-mix sequence of a multi-channel audio signal having 6 channels (left front channel L, left surround channel L _s , center channel C, low frequency channel LFE, right front channel R, right surround channel R _s ) , into the mono audio signal M obtained through the down-mix, and the relationship between the multi-channel audio signal and the spatial parameters.

First, as in the first example, down-mixing is performed between the left channel L and the left surround channel L _s , down-mixing between the center channel C and the low-frequency channel LFE, and down-mixing between the right channel R and the right surround channel R _s . In this primary downmix process, a left aggregate channel L _t , a central aggregate channel C _t, and a right aggregate channel R _{t are formed} . In addition, the spatial parameters computed in this primary downmix process include CLD ₃ (including ICC ₃ ), CLD ₄ (including ICC ₄ ), CLD ₅ (including ICC ₅ ), etc. (in this case, CLD _x and ICC _x are different from the previous CLD _x in the first example).

In the secondary process following the primary downmix process, a joint downmix of the left aggregate channel L _t and the right aggregate channel R _t is performed to form the left center channel LC, and a joint downmix of the central aggregate channel C _t and the right aggregate channel R _t is performed to form the right center RC channel. In addition, the spatial parameters calculated in this secondary downmix process may include CLD ₂ (including ICC ₂ ), CLD ₁ (including ICC ₁ ), etc.

Thereafter, in the tertiary downmix process, downmixing of the left center channel LC and the right center channel RC is performed to form a mono signal M obtained through downmix. Also, the spatial parameters calculated in the tertiary downmix process include CLD ₀ (including ICC ₀ ), etc.

According to the right side of FIG. 3, in case partial spatial information corresponds to CLD ₀ from the spatial parameters (CLD ₃ , CLD ₄ , CLD ₅ , CLD ₁ , CLD ₂ , CLD ₀ , etc.), the left center channel LC and right center channel RC. If the left center LC channel and the right center RC channel are selected as the output channel audio signal, the output channel audio signal from the two LC and RC channels can be generated.

If the partial spatial information corresponds to CLD ₀ , CLD ₁ and CLD ₂ from the spatial parameters (CLD ₃ , CLD ₄ , CLD ₅ , CLD ₁ , CLD ₂ , CLD ₀ , etc.), the left aggregate channel L _t , the central aggregate channel C _t and right aggregate channel R _t .

If the left aggregate channel L _t and the right aggregate channel R _t are selected as the audio signal of the output channel, it is possible to generate an audio signal of the output channel from the two channels L _t and R _t . If the left aggregate channel L _t , the central aggregate channel C _t and the right aggregate channel R _t are selected as the audio signal of the output channel, it is possible to generate an audio signal of the output channel from the three channels L _t , C _t and R _t .

If the partial spatial information includes CLD _4, among other things, after up-mixing to the center channel and the low frequency channel LFE is performed, if the left aggregate channel L _t , the right aggregate channel R _t , the center channel C and the low frequency channel LFE are selected in As the audio signal of the output channel, it is possible to generate the audio signal of the output channel of four channels (L _t , R _t , C and LFE).

(1) -3. The third tree configuration example (tree configuration 5-1-5)

4 is a schematic representation of a further example of the application of partial spatial information.

The left side of FIG. 4 shows a down-mix sequence of a multi-channel audio signal having 6 channels (left front channel L, left surround channel L _s , center channel C, low frequency channel LFE, right front channel R, right surround channel R _s ) , into the mono audio signal M obtained through the down-mix, and the relationship between the multi-channel audio signal and the spatial parameters.

First, as in the first and second examples, down-mix is performed between the left channel L and the left surround channel L _s , down-mixes between the center channel C and the low-frequency channel LFE, and down-mix between the right channel R and the right surround channel R _s . In this primary downmix process, a left aggregate channel L _t , a central aggregate channel C _t, and a right aggregate channel R _t are formed. In addition, the spatial parameters calculated in this primary downmix process include CLD ₁ (including ICC ₁ ), CLD ₂ (including ICC ₂ ), CLD ₃ (including ICC ₃ ), etc. (in this case, CLD _x and ICC _x are different from the previous CLD _x and ICC _x in the first or second example).

In the secondary process following the primary downmix process, joint downmixing of the left aggregate channel L _t , the central aggregate channel C _t and the right aggregate channel R _t is performed to form the left center channel LC and the right center channel RC. In addition, the spatial parameter CLD _TTT (including ICC _TTT ) is calculated.

Thereafter, in the tertiary downmix process, downmixing of the left center channel LC and the right channel R is performed to form a mono signal M obtained through downmix. The spatial parameter CLD ₀ (including ICC ₀ ) is also calculated.

According to the right side of FIG. 4, in case partial spatial information corresponds to CLD ₀ and CLD _TTT from spatial parameters (CLD ₁ , CLD ₂ , CLD ₃ , CLD _TTT , CLD ₀ , etc.), the left aggregate channel L _t , the central aggregate channel C _t and the right aggregate channel R _t .

If the left aggregate channel L _t and the right aggregate channel R _t are selected as the audio signal of the output channel, it is possible to generate an audio signal of the output channel from the two channels L _t and R _t .

If the left aggregate channel L _t , the central aggregate channel C _t and the right aggregate channel R _t are selected as the audio signal of the output channel, an audio signal of the output channel from the three channels L _t , C _t and R _t can be generated.

In the case where the partial spatial information includes CLD _2, among other things, after the upmix to the center channel C and the low frequency channel LFE is performed, if the left aggregate channel L _t , the right aggregate channel R _t , the center channel C and the low frequency channel LFE are selected as the audio signal of the output channel, it is possible to generate an audio signal of the output channel of four channels (L _t , R _t , C and LFE).

In the above description, the process of generating an output channel audio signal by only partially applying spatial parameters is explained by using three types of tree configurations as examples. In addition, it is also possible to additionally apply combined spatial information or extended spatial information, as well as partial spatial information. Thus, it is possible to process the process of applying the modified spatial information to the audio signal hierarchically or jointly and synthetically.

(2) Combined spatial information

Since spatial information is computed during down-mixing of a multi-channel audio signal according to a predetermined tree configuration, the original multi-channel audio signal before down-mixing can be restored if the down-mixing audio signal is decoded using the spatial spatial information parameters as they are. If the number of channels M of the multi-channel audio signal is different from the number of channels N of the audio signal of the output channel, new combined spatial information is generated by combining the spatial information, and then you can perform up-mixing of the audio signal, down-mixing using the generated information. In particular, by applying spatial parameters to the transformation formula, combined spatial parameters can be generated.

This method may vary according to the sequence and method of down-mixing the multi-channel audio signal in the encoding device. You can also request the sequence and method of down-mixing using tree-like configuration information in spatial information. In addition, this method may vary according to the number of output channels. Moreover, you can request the number of output channels, etc. using the information of the output channels.

The following explains detailed embodiments of a method for modifying spatial information and embodiments for providing a virtual three-dimensional effect.

(2) -1. General combined spatial information

A method for generating combined spatial parameters by combining spatial parameters of spatial information is provided for upmixing according to a tree configuration different from that in the downmix process. Thus, this method is applicable to all kinds of downmix signals, regardless of which tree configuration is available according to the tree configuration information.

If the multi-channel audio signal is 5.1-channel and the down-mix audio signal is 1-channel (mono channel), the method of generating an audio signal of an output channel from two channels is explained with reference to two types of examples as follows.

(2) -1-1. The fourth embodiment of the tree configuration (tree configuration 5-1-5 ₁ )

5 is a schematic representation of an example of application of combined spatial information.

Referring to the left side of FIG. 5, CLD ₀ -CLD ₄ and ICC ₀ -ICC ₄ (not shown in the drawing) can be called spatial parameters that can be calculated during down-mixing of a multi-channel audio signal for 5.1 channels. For example, in spatial parameters, the inter-channel level difference between the left channel signal L and the right channel signal R is CLD ₃ , and the inter-channel correlation between L and R is ICC ₃ . In addition, the inter-channel level difference between the left surround channel L _s and the right surround channel R _s is CLD ₂ , and the inter-channel correlation between L _s and R _s is ICC ₂ .

On the other hand, referring to the right-hand side of FIG. 5, if the left channel signal L _t and the right channel signal R _t are generated by applying the combined spatial parameters CLD _α and ICC _α to the down-mix mono audio signal m, the audio signal L _t and R _t can be directly generated stereo output channel from mono channel audio signal m. In this case, the combined spatial parameters CLD _α and ICC _α can be calculated by combining the spatial parameters CLD ₀ -CLD ₄ and ICC ₀ -ICC ₄ .

Next, the process of computing CLD _α from the combined spatial parameters by combining CLD ₀ -CLD _{4 is} first explained, and then the process of computing ICC _α from the combined spatial parameters by combining CLD ₀ -CLD ₄ and ICC ₀ -ICC _{4 is} explained as follows.

(2) -1-1-a. Conclusion CLD _α

Firstly, since CLD _α is the level difference between the left output signal L _t and the right output signal R _t , the result of inputting the left output signal L _t and the right output signal R _t into the CLD determination formula is shown as follows:

[Formula 1]

CLD _α = 10 * log ₁₀ (P _Lt / P _Rt ),

where P _Lt is the degree of L _t , and P _Rt is the degree of R _t .

[Formula 2]

CLD _α = 10 * log ₁₀ (P _Lt + a / P _Rt + a),

where P _Lt is the degree of L _t , P _Rt is the degree of R _t , a is a very small constant.

Therefore, CLD _α is defined as formula 1 or formula 2.

To represent P _Lt and P _Rt using the spatial parameters CLD ₀ -CLD ₄ , a relationship formula is required between the left output signal L _{t of the} audio signal of the output channel, the right output signal R _{t of the} audio signal of the output channel and the multi-channel signal L, L _s , R, R _s , C and LFE. Also, the corresponding relationship formula can be defined as follows:

[Formula 3]

L _t = L + L _s + C / √2 + LFE / √2,

R _t = R + R _s + C / √2 + LFE / √2.

Since a relationship formula similar to formula 3 can vary according to how the audio signal of the output channel is determined, it can be defined by a formula different from formula 3. For example, 1 / √2 in C / √2 or LFE / √2 can be 0 or one.

Formula 3 allows you to derive formula 4 as follows:

[Formula 4]

P _Lt = P _L + P _LS + P _C / 2 + P _LFE / 2,

P _Rt = P _R + P _RS + P _C / 2 + P _LFE / 2.

You can represent CLD _α according to formula 1 or formula 2 using P _Lt and P _Rt . Also, P _Lt and P _Rt can be represented according to formula 4 with P _L , P _Ls , P _c , P _LFE , P _R and P _Rs . Thus, it is necessary to find the relationship formula that makes it possible to represent P _L , P _Ls , P _c , P _LFE , P _R, and P _Rs using the parameters CLD ₀ -CLD ₄ .

In the case of the tree configuration shown in FIG. 5, the relationship between the multi-channel audio signal (L, R, C, LFE, L _s , R _s ) and the mono channel signal m obtained through downmix is shown below.

[Formula 5]

Formula 5 also leads to Formula 6 as follows:

[Formula 6]

In particular, by entering formula 6 into formula 4 and by entering formula 4 into formula 1 or formula 2, it is possible to represent the combined spatial parameter CLD _{α by} combining the spatial parameters CLD ₀ -CLD ₄ .

The extension resulting from entering formula 6 in P _c / 2 + P _LFE / 2 in formula 4 is shown in formula 7.

[Formula 7]

P _C / 2 + P _LFE / 2 = [(C _{1, OTT4)} ² + (C _{2, OTT4)} ^2] * (C _{2, OTT1} * C _{1, OTT0)} * m ^{2 2/2.}

In this case, according to the definitions of C ₁ and C ₂ (compare formula 5), since (C _{1, x} ) ² + (C _{2, x} ) ² = 1, this leads to (C _{1, OTT4} ) ² + (C _{2, OTT4} ) ² = 1.

Thus, formula 7 can be summarized as follows:

[Formula 8]

P _C / 2 + P _LFE / 2 = (C _{2, OTT1} * C _{1, OTT0)} * m ^{2 2/2.}

Therefore, by entering formula 8 and formula 6 into formula 4 and by entering formula 4 into formula 1, one can represent the combined spatial parameter CLD _{α by} combining the spatial parameters CLD ₀ -CLD ₄ .

(2) -1-1-b. Conclusion ICC _α

First, since ICC _α is the correlation between the left output signal L _t and the right output signal R _t , the result of inputting the left output signal L _t and the right output signal R _t into the corresponding determination formula is shown as follows:

[Formula 9]

In formula 9, P _Lt and P _Rt can be represented by CLD ₀ -CLD ₄ in formula 4, formula 6 and formula 8. In addition, P _Lt P _Rt can be expanded by the method of formula 10.

[Formula 10]

P _LtRt = P _LR + P _LsRs + P _C / 2 + P _LFE / 2.

In formula 10, P _C / 2 + P _LFE / 2 can be represented as CLD ₀ -CLD ₄ according to formula 6. Also, P _LR and P _LsRs can be deployed according to the ICC definition as follows:

[Formula 11]

ICC ₃ = P _LR / √ (P _L P _R ),

ICC ₂ = P _LsRs / √ (P _Ls P _Rs ).

In formula 11, if √ (P _L P _R ) or √ (P _LsRs ) is transposed, then formula 12 is obtained.

[Formula 12]

P _LR = ICC ₃ * √ (P _L P _R ),

P _LsRs = ICC ₂ * √ (P _LsRs ).

In formula 12, P _L , P _R , P _Ls and P _Rs can be represented as CLD ₀ -CLD ₄ according to formula 6. The formula resulting from the input of formula 6 into formula 12 corresponds to formula 13.

[Formula 13]

P _LR = ICC ₃ * c _{l, OTT3} * c _{2, OTT3} * (c _{l, OTT1} * c _{l, OTT0} ) ² * m ² ,

P _LrRs = ICC ₂ * c _{l, OTT2} * c _{2, OTT2} * (c _{2, OTT0} ) ² * m ² .

So, by entering formula 6 and formula 13 into formula 10 and by entering formula 10 and formula 4 into formula 9, we can represent the combined spatial parameter ICC _α as the spatial parameters CLD ₀ -CLD ₃ , ICC ₂ and ICC ₃ .

(2) -1-2. Fifth Embodiment of a Tree Configuration (Tree Configuration 5-1-5 ₂ )

6 is a schematic representation of another example of the application of combined spatial information.

Referring to the left side of FIG. 6, CLD ₀ -CLD ₄ and ICC ₀ -ICC ₄ (not shown in the drawing) can be called spatial parameters, which can be calculated in the process of down-mixing the multi-channel audio signal for 5.1 channels.

In spatial parameters, the inter-channel level difference between the left channel signal L and the surround channel left signal L _s is CLD ₃ , and the inter-channel correlation between L and L _s is ICC ₃ . In addition, the inter-channel level difference between the right channel R and the right channel R _{s of the} surround sound is CLD ₄ , and the inter-channel correlation between R and R _s is ICC ₄ .

On the other hand, referring to the right-hand side of FIG. 6, if the left channel signal L _t and the right channel signal R _t are generated by applying the combined spatial parameters CLD _β and ICC _β to the down-mix mono audio signal m, the audio signal L _t and R _t can be directly generated stereo output channel from mono channel audio signal m. In this case, the combined spatial parameters CLD _β and ICC _β can be calculated by combining the spatial parameters CLD ₀ -CLD ₄ and ICC ₀ -ICC ₄ .

Next, the process of computing CLD _β from combined spatial parameters by combining CLD ₀ -CLD _{4 is} first explained, and then the process of computing ICC _β from combined spatial parameters by combining CLD ₀ -CLD ₄ and ICC ₀ -ICC _{4 is} explained as follows:

(2) -1-2-a. Conclusion CLD _β

First, since CLD _β is the level difference between the left output signal L _t and the right output signal R _t , the result of inputting the left output signal L _t and the right output signal R _t in the CLD determination formula is shown as follows:

[Formula 14]

CLD _β = 10 * log ₁₀ (P _Lt / P _Rt ),

where P _Lt is the degree of L _t , P _Rt is the degree of R _t .

[Formula 15]

CLD _β = 10 * log ₁₀ (P _Lt + a / P _Rt + a),

where P _Lt is the degree of L _t , P _Rt is the degree of R _t , a is a very small number.

Therefore, CLD _β is defined as formula 14 or formula 15.

Meanwhile, in order to represent P _Lt and P _Rt using the spatial parameters CLD ₀ -CLD ₄ , a relationship formula is required between the left output signal L _{t of the} output channel audio signal, the right output signal R _{t of the} output channel audio signal and the multi-channel signal L, L _s , R , R _s , C and LFE. Also, the corresponding relationship formula can be defined as follows:

[Formula 16]

L _t = L + L _s + C / √2 + LFE / √2,

R _t = R + R _s + C / √2 + LFE / √2.

Since a relationship formula similar to formula 16 may vary according to how the audio signal of the output channel is determined, it can be defined by a formula other than formula 16. For example, 1 / √2 in C / √2 or LFE / √2 can be 0 or one.

Formula 16 allows you to derive formula 17 as follows:

[Formula 17]

P _Lt = P _L + P _LS + P _C / 2 + P _LFE / 2,

P _Rt = P _R + P _RS + P _C / 2 + P _LFE / 2.

You can represent CLD _β according to formula 14 or formula 15 using P _Lt and P _Rt . Also, P _Lt and P _Rt can be represented according to formula 15 with P _L , P _Ls , P _c , P _LFE , P _R and P _Rs . Thus, it is necessary to find the relationship formula that makes it possible to represent P _L , P _Ls , P _c , P _LFE , P _R, and P _Rs using the parameters CLD ₀ -CLD ₄ .

In the case of the tree configuration shown in FIG. 6, the relationship between the multi-channel audio signal (L, R, C, LFE, L _s , R _s ) and the monochannel signal m obtained through the downmix is shown below.

[Formula 18]

Formula 18 also leads to Formula 19 as follows:

[Formula 19]

In particular, by entering formula 19 into formula 17 and by entering formula 17 into formula 14 or formula 15, the combined spatial parameter CLD _β can be represented by combining the spatial parameters CLD ₀ -CLD ₄ .

The expansion formula obtained by entering formula 19 into P _L + P _Ls in formula 17 is shown in formula 20.

[Formula 20]

P _L + P _Ls = [(c _{l, OTT3} ) ² + (c _{2, OTT3} ) ² ] (c _{l, OTT1} * c _{l, OTT0} ) ² * m ² .

In this case, according to the definitions c ₁ and c ₂ (compare formula 5), since (c _{1, x} ) ² + (c _{2, x} ) ² = 1, this leads to (c _{1, OTT3} ) ² + (c _{2, OTT3} ) ² = 1.

Thus, formula 20 can be summarized as follows:

[Formula 21]

P _{L_} = P _L + P _Ls = (c _{l, OTT1} * c _{l, OTT0} ) ² * m ² .

The expanded record obtained by entering formula 19 for P _R + P _Rs into formula 17 is represented as formula 22.

[Formula 22]

P _R + P _Rs = [(c _{l, OTT4} ) ² + (c _{2, OTT4} ) ² ] (c _{l, OTT1} * c _{l, OTT0} ) ² * m ² .

In this case, according to the definitions c ₁ and c ₂ (compare formula 5), since (c _{1, x} ) ² + (c _{2, x} ) ² = 1, this leads to (c _{1, OTT4} ) ² + (c _{2, OTT4} ) ² = 1.

Thus, formula 22 can be summarized as follows:

[Formula 23]

P _{R_} = P _R + P _Rs = (c _{2, OTT1} * c _{l, OTT0} ) ² * m ² .

On the other hand, the extended entry obtained by entering formula 19 for P _c / 2 + P _LFE / 2 into formula 17 is represented as formula 24.

[Formula 24]

P _C / 2 + P _LFE / 2 _{= [(c l, OTT2)} 2+ (c _{2, OTT2)} ^2] (c _{2, OTT0)} * m ^{2 2/2.}

In this case, according to the definitions c ₁ and c ₂ (compare formula 5), since (c _{1, x} ) ² + (c _{2, x} ) ² = 1, this leads to (c _{1, OTT2} ) ² + (c _{2, OTT2} ) ² = 1.

Thus, formula 24 can be summarized as follows:

[Formula 25]

P _C / 2 + P _LFE / 2 = (c _{2, OTT0)} * m ^{2 2/2.}

Therefore, by entering formula 21, formula 23 and formula 25 into formula 17, and by entering formula 17 into formula 14 or formula 15, the combined spatial parameter CLD _β can be represented by combining the spatial parameters CLD ₀ -CLD ₄ .

(2) -1-2-b. Conclusion ICC _β

First, since ICC _β is the correlation between the left output signal L _t and the right output signal R _t , the result of inputting the left output signal L _t and the right output signal R _t into the corresponding determination formula is shown as follows:

[Formula 26]

In formula 26, P _Lt and P _Rt can be represented according to formula 19 using CLD ₀ -CLD ₄ . In addition, P _Lt P _Rt can be deployed according to formula 27.

[Formula 27]

P _LtRt = P _{L_R_} + P _C / 2 + P _LFE / 2.

In formula 27, P _C / 2 + P _LFE / 2 can be represented as CLD ₀ -CLD ₄ according to formula 19. Also, P _{L_R_} can be expanded according to the ICC definition as follows:

[Formula 28]

ICC ₁ = P _{L_R_} / √ (P _{L_} P _{R_} ).

If √ (P _{L_} P _{R_} ) is transposed, formula 29 can be obtained.

[Formula 29]

P _{L_R_} = ICC ₁ * √ (P _{L_} P _{R_} ).

In formula 29, P _{L_} and P _{R_} can be represented as CLD ₀ -CLD ₄ according to formula 21 and formula 23. The formula resulting from the entry of formula 21 and formula 23 into formula 29 corresponds to formula 30.

[Formula 30]

P _{L_R_} = ICC ₁ * c _{1, OTT1} * c _{1, OTT1} * c _{2, OTT1} * c _{1, OTT0} * m ² .

So, by entering formula 30 into formula 27 and by entering formula 27 and formula 17 into formula 26, we can represent the combined spatial parameter ICC _β as spatial parameters CLD ₀ -CLD ₄ and ICC ₁ .

The above methods for modifying spatial parameters are only one embodiment. Also, when finding P _x or P _xy, it is obvious that the above formulas can vary in various forms by considering correlations (for example, ICC ₀ , etc.) between the respective channels, as well as in addition to the signal energy.

(2) -2. Combined spatial information with surround effect

First, in the case of considering sound paths, in order to generate combined spatial information by combining spatial information, a virtual surround effect can be caused.

A virtual surround effect or a virtual three-dimensional effect can cause an effect that is predominantly present in the surround channel speaker without the surround channel speaker. For example, 5.1-channel audio is output from two stereo speakers.

The sound path may correspond to spatial filtering information. The spatial filtering information may use a function called HRTF (Sound Perception Modeling Function), which is not limited by the present invention. The spatial filtering information may include a filtering parameter. By entering the filter parameter and spatial parameters into the transformation formula, a combined spatial parameter can be generated. Also, the combined spatial parameter generated may include filtering coefficients.

Further, suppose that a multi-channel audio signal is 5 channels, and that an audio signal of an output channel of three channels is generated, and a method for considering sound paths to generate combined spatial information having a surround effect is explained as follows.

Fig. 7 is a diagram of sound paths from speakers to a listener, showing speaker positions.

According to Fig. 7, the positions of the three speakers SPK1, SPK2 and SPK3 are the left front L, the center C and the right R, respectively. Also the positions of the virtual surround channels are the left surround sound Ls and the right surround sound Rs, respectively.

The sound paths to the r and l positions of the right and left ears of the listener from the positions L, C and R of the three speakers and the positions Ls and Rs of the virtual surround channels, respectively, are shown. The G _{x_y indicator} indicates the sound path from position x to position y. For example, the indicator G _{L_r} indicates the sound path from the position of the left front L to the position of the right ear r of the listener.

If there are speakers in five positions (i.e. there are speakers in the left surround sound Ls and the surround right Rs also), and if the listener is in the position shown in FIG. 7, the signal L ₀ entering the listener's left ear, and the signal R ₀ entering the listener's right ear is represented by formula 31.

[Formula 31]

L ₀ = L * G _{L_l} + C * G _{C_l} + R * G _{R_l} + Ls * G _{Ls_l} + Rs * G _{Rs_l} ,

R ₀ = L * G _{L_r} + C * G _{C_r} + R * G _{R_r} + Ls * G _{Ls_r} + Rs * G _{Rs_r} ,

where L, C, R, Ls and Rs are channels in positions, respectively, G _{x_y} indicates the sound path from position x to position y, and '*' indicates convolution.

As mentioned in the above description, if the speakers are only in three positions L, C and R, the L _{0_real} signal entering the listener's left ear and the R _{0_real} signal coming into the listener's right ear are presented as follows:

[Formula 32]

L _{0_real} = L * G _{L_l} + C * G _{C_l} + R * G _{R_l} ,

R _{0_real} = L * G _{L_r} + C * G _{C_r} + R * G _{R_r} .

Since the Ls and Rs signals of the surround channels are not taken into account in the signals shown in Formula 32, it is not possible to produce a virtual surround effect. To cause a virtual surround effect, the signal Ls entering the listening position (l, r) from the speaker position Ls is set equal to the signal Ls entering the listening position (l, r) from the speaker in each of the three positions L, C and R different from the starting position Ls. In addition, this is equally applicable to the case of the right surround channel signal Rs.

Consider the left surround channel signal Ls if the left surround channel signal Ls is output from the speaker in the left surround position Ls as the starting position, then the signals coming to the listener's left and right ears l and r are presented as follows:

[Formula 33]

Ls * G _{Ls_l} , Ls * G _{Ls_r} .

And if the signal of the right surround sound channel Rs is output from the speaker in the right surround sound position Rs as the starting position, the signals coming to the listener's left and right ears l and r are presented as follows:

[Formula 34]

Rs * G _{Rs_l} , Rs * G _{Rs_r} .

If the signals coming into the left and right ear l and r of the listener are equal to the components of formula 33 and formula 34, even if they are output through speakers of any position (for example, through the speaker SPK1 in the left front position), the listener can perceive them like this as if the speakers were in the left and right surround positions of Ls and Rs respectively.

If the components shown in Formula 33 are output from the speaker in the left surround position Ls, they are signals coming into the listener's left and right ears l and r, respectively. Thus, if the components shown in formula 33 are output unchanged from the speaker SPK1 in the left front position, the signals coming into the left and right ear l and r of the listener can be represented as follows:

[Formula 35]

Ls * G _{Ls_l} * G _{L_l} , Ls * G _{Ls_r} * G _{L_r} .

In formula 35, the component G _{L_l} (or G _{L_r} ) is _added , corresponding to the sound path from the left front position L to the left ear l (or right ear r) of the listener.

Also, the signals coming into the left and right ears l and r of the listener should be the components shown in formula 33 instead of formula 35. If the sound output from the speaker and the left front position L comes to the listener, the component G _{L_l} (or G _{L_r} ). Thus, if the components shown in formula 33 are output from the speaker SPK1 in the left front position, the inverse function G _{L_l} ^-1 (or G _{L_r} ^-1 ) of G _{L_l} (or G _{L_r} ) should be considered for the sound path. In other words, if the components corresponding to formula 33 are output from the speaker SPK1 in the left front position L, they must be modified according to the following formula:

[Formula 36]

Ls * G _{Ls_l} * G _{L_l} ^-1 , 'Ls * G _{Ls_r} * G _{L_r} ^-1 .

Also, if the components corresponding to formula 34 are output from the speaker SPK1 in the left front position L, they must be modified according to the following formula:

[Formula 37]

Rs * G _{Rs_l} * G _{L_l} ^-1 , Rs * G _{Rs_r} * G _{L_l} ^-1 .

Thus, the signal output from the speaker SPK1 in the left front position L 'is summarized as follows:

[Formula 38]

L '= L + Ls * G _{Ls_l} * G _{L_l} ^-1 + Rs * G _{Rs_l} * G _{L_l} ^-1

(Components Ls * G _{Ls_r} * G _{L_r} ^-1 and Rs * G _{Rs_r} * G _{L_l} ^-1 are omitted).

If the signal that is shown in formula 38, which is to be output from the speaker SPK1 in the left front position L, arrives at the location of the left ear L of the listener, the sound path coefficient G _{L_1 is} added. Thus, the terms G _{L_l} in the formula are mutually annihilated, as a result of which the coefficients shown in formula 33 and formula 34 remain.

8 is a diagram for explaining a signal output from each speaker position for a virtual surround effect.

8, if the signals Ls and Rs outputted from the surround sound positions Ls and Rs are to be included in the signal L 'output from each position of the speaker SPK1 by considering sound paths, they correspond to formula 38.

In the formula 38 G _{Ls_l} * G _{L_l} ^{-1 is} briefly written as H _{Ls_L} as follows:

[Formula 39]

L '= L + Ls * H _{Ls_L} + Rs * H _{Rs_L} .

For example, the signal C 'output from the speaker SPK2 in the center position C is summarized as follows:

[Formula 40]

C '= C + Ls * H _{Ls_C} + Rs * H _{Rs_C} .

As another example, the signal R 'output from the speaker SPK3 in the front right position R is summarized as follows:

[Formula 41]

R '= R + Ls * H _{Ls_R} + Rs * H _{Rs_R} .

Fig.9 is a conceptual diagram for explaining a method for generating a 3-channel signal using a 5-channel signal according to formula 38, formula 39 and formula 40.

If a 2-channel signal R 'and L' is generated using a 5-channel signal, or if the surround channel signals Ls or Rs are not included in the center channel signal C 'H, _{Ls_C} or H _{Rs_C} becomes equal to 0.

For ease of implementation, H _{x_y} can be modified in different ways so that H _{x_y} is replaced by G _{x_y} , or H _{x_y is} used by considering crosstalk.

The above detailed description relates to one example of combined spatial information having a surround effect. It is also obvious that it can vary in various forms according to the method of applying spatial filtering information. As mentioned in the above description, the signals output by the speakers (in the above example, the left front channel L ', the right front channel R' and the center channel C ') according to the above process can be generated from the downmix audio signal using combined spatial information and, more specifically, using combined spatial parameters.

(3) More detailed spatial information

Firstly, by adding extended spatial information to spatial information, more detailed spatial information can be generated. You can also perform up-mixing of the audio signal using extended spatial information. In the corresponding upmixing process, the audio signal is converted to the primary upmix audio signal based on spatial information, and the primary upmix audio signal is then converted to the secondary upmix audio signal based on the extended spatial information.

In this case, the extended spatial information may include extended channel configuration information, extended channel matching information, and extended spatial parameters.

Extended channel configuration information is information for a configurable channel, as well as a channel that can be configured by means of tree configuration information from spatial information. The extended channel configuration information may include at least one of a section identifier and a non-partition identifier, which are described in more detail below. Extended channel matching information is the position information of each channel that configures the extended channel. Extended spatial parameters can also be used to up-mix one channel into at least two channels. Enhanced spatial parameters may include inter-channel level differences.

The above-described extended spatial information may be included in the spatial information after being generated by the encoding device (i) or generated by the decoding device (ii) itself. If the extended spatial information is generated by the encoding device, the presence or absence of the extended spatial information can be determined based on the spatial information indicator. In the event that the expanded spatial information is generated by the decoding device itself, the expanded spatial parameters of the extended spatial information can flow from the calculation using the spatial parameters of the spatial information.

The process of up-mixing an audio signal using more detailed spatial information generated on the basis of spatial information and extended spatial information can be performed sequentially and hierarchically, or collectively and synthetically. If more detailed spatial information can be calculated as a single matrix based on spatial information and extended spatial information, it is not possible to perform up-mix or down-mix of an audio signal into a multi-channel audio signal collectively and directly using the matrix. In this case, the coefficients configuring the matrix can be set according to spatial parameters and extended spatial parameters.

Further, after completing the explanation of the case in which the extended spatial information generated by the encoding device is used, the case of generating the extended spatial information in the decoding device itself is explained.

(3) -1. The case of using the extended spatial information generated by the encoding device: an arbitrary tree configuration

Firstly, more detailed spatial information is generated by the encoding device during generation by adding extended spatial information to the spatial information. Also explained is the case where the encoding device receives extended spatial information. In addition, the extended spatial information may be information extracted in the process when the encoding device down-mixes the multi-channel audio signal.

As mentioned in the above description, the extended spatial information includes extended channel configuration information, extended channel matching information, and extended spatial parameters. In this case, the extended channel configuration information may include at least one of a section identifier and an identifier without a section. The following describes in detail the process of configuring an extended channel based on a matrix of partition identifiers and non-partition identifiers as follows.

10 is a diagram of an example configuration of extended channels based on configuration information of extended channels.

According to the bottom of FIG. 10, zeros and ones are alternately arranged one after another. In this case, 0 means an identifier without a section, and 1 means an identifier for a section. An identifier without a partition 0 is in the first order (1), the channel corresponding to the identifier without a partition 0 of the first order is the left channel located at the very top edge. Thus, the left channel L corresponding to the identifier without section 0 is selected as the output channel instead of splitting. In the second order (2), there is a section identifier 1. The channel corresponding to the section identifier is the left surround channel Ls after the left channel L. Thus, the left surround channel Ls corresponding to the identifier of section 1 is divided into two channels.

Since there are identifiers without section 0 in the third order (3) and fourth order (4), two channels separated from the left surround channel Ls are selected unchanged as output channels without separation. After the above order is repeated to the last order (10), all extended channels can be configured.

The channel splitting process is repeated as many times as the identifiers of section 1, and the channel selection process as the output channel is repeated as many times as the identifiers without section 0. Thus, the number of channel division blocks AT0 and AT1 is equal to the number (2) of section 1 identifiers, and the number of extended channels (L, Lfs, Ls, R, Rfs, Rs, C, and LFE) is equal to the number (8) of identifiers without section 0.

After the extended channel is configured, it is possible to negotiate the position of each output channel using the extended channel matching information. In the case of FIG. 10, matching is performed in the sequence of the left front channel L, left front side channel Lfs, left surround channel Ls, right front channel R, right front side channel Rfs, right surround channel Rs, center channel C and low frequency channel LFE .

As mentioned in the above description, the extended channel can be configured based on the configuration information of the extended channels. This requires a channel separation unit dividing one channel into at least two channels. When dividing one channel into at least two channels, the channel splitting unit may use advanced spatial parameters. Since the number of extended spatial parameters is equal to the number of channel separation blocks, it is also equal to the number of separation identifiers. Thus, advanced parameters can be retrieved as many times as section identifiers.

11 is a diagram for explaining the configuration of the extended channels shown in FIG. 10, and the relationship with the advanced spatial parameters.

11, there are two channel separation units AT ₀ and AT ₁ and the extended spatial parameters ATD ₀ and ATD ₁ applied to them, respectively.

If the expanded spatial parameter is an inter-channel level difference, the channel separation unit can determine the levels of two separated channels using the extended spatial parameter.

Thus, when performing upmixing by adding extended spatial information, the expanded spatial parameters may not be fully applied, but partially.

(3) -2. The case of the formation of extended spatial information: interpolation / extrapolation

First, more detailed spatial information can be generated by adding extended spatial information to spatial information. The case of generating extended spatial information using spatial information is explained in the description below. In particular, it is possible to form extended spatial information using spatial parameters of spatial information. In this case, interpolation, extrapolation, etc. can be used.

(3) -2-1. Extend to 6.1 channels

If the multi-channel audio signal is 5.1 channels, the case of generating the output channel audio signal for 6.1 channels is explained with reference to examples as follows.

Fig is a diagram of the position of the multi-channel audio signal for 5.1 channels and the position of the audio signal of the output channel for 6.1 channels.

Referring to (a) of FIG. 12, it can be seen that the channel positions of the 5.1-channel multi-channel audio signal are the left front channel L, the right front channel R, the center channel C, the low-frequency channel (not shown in the drawing) LFE, the left channel Ls surround sound and the right channel Rs surround sound respectively.

In the case where the 5.1-channel multi-channel audio signal is a down-mix signal, if spatial parameters are applied to the down-mix audio signal, up-mixing of the down-mix audio signal to the 5.1-channel multi-channel audio signal is again performed.

In addition, the channel signal for the rear center RC, as shown in (b) of FIG. 12, needs to be further formed in order to up-mix the down-mix audio signal into a multi-channel audio signal for 6.1 channels.

The signal channel for the rear center RC can be formed using spatial parameters associated with the two rear channels (left surround channel Ls surround and right surround channel Rs). In particular, the inter-channel level difference (CLD) of the spatial parameters indicates the level difference between the two channels. Thus, by adjusting the level difference between the two channels, it is possible to form a virtual sound source existing between the two channels.

The principle in which the position of the virtual sound source varies according to the level difference between the two channels is explained as follows.

13 is a diagram for explaining the relationship between the position of the virtual sound source and the level difference between the two channels, in which the levels of the left and right surround channels Ls and Rs are a and b, respectively.

Referring to (a) of FIG. 13, if the level a of the surround left channel Ls is greater than the level b of the surround right channel Rs, it can be seen that the position of the virtual sound source VS is closer to the position of the surround left channel Ls than to the position right channel rs surround sound.

If the audio signal is output from two channels, the listener feels that a virtual sound source practically exists between the two channels. In this case, the position of the virtual sound source is closer to the position of the channel having a level above the level of another channel.

In case (b) of FIG. 13, since the level a of the left surround channel Ls is almost equal to the level b of the right surround channel Rs, the listener feels that the position of the virtual sound source is in the center between the left surround channel Ls and the right surround channel Rs sounding.

Therefore, it is possible to determine the level of the rear center using the above principle.

14 is a diagram for explaining levels of two rear channels and a level of a rear center channel.

According to FIG. 14, it is possible to calculate the level c of the rear center channel RC by interpolating the difference between the level a of the left surround channel Ls and the level b of the right surround channel Rs. In this case, non-linear interpolation can be used in the same way as linear interpolation for calculation.

The level c of the new channel (for example, the rear center channel RC) existing between the two channels (for example, Ls and Rs) can be calculated according to linear interpolation using the following formula:

[Formula 40]

c = a * k + b * (1-k),

where a and b are the levels of two channels, respectively, k is the relative position of the beta channel of level a, channel of level b and channel of level c.

If the channel (for example, the rear center channel RC) at level c is located in the center between the channel (for example, Ls) at level a and the channel Rs at level b, k is 0.5. If k is 0.5, then formula 41 follows from formula 40.

[Formula 41]

c = (a + b) / 2.

According to formula 41, if the channel (for example, the rear center channel RC) at level c is located in the center between the channel (for example, Ls) at level a and channel Rs at level b, the level c of the new channel corresponds to the average value of levels a and b previous channels. In addition, formula 40 and formula 41 are for example purposes only. Thus, it is also possible to adjust the definition of level c and the values of level a and level b.

(3) -2-2. Extend to 7.1 channels

When the multi-channel audio signal is 5.1 channels, a case of attempting to generate an output channel audio signal for 7.1 channels is explained as follows.

FIG. 15 is a diagram for explaining a position of a multi-channel audio signal of 5.1 channels and a position of an audio signal of an output channel of 7.1 channels.

Referring to (a) of FIG. 15, similarly to (a) of FIG. 12, it can be seen that the channel positions of the 5.1 channel multi-channel audio signal are left front channel L, right front channel R, center channel C, low frequency channel (not shown in the drawing) LFE, left surround channel Ls and right surround channel Rs, respectively.

In the case where the 5.1-channel multi-channel audio signal is a down-mix signal, if spatial parameters are applied to the down-mix audio signal, up-mixing of the down-mix audio signal to the 5.1-channel multi-channel audio signal is again performed.

In addition, the left front side channel Lfs and the right front side channel Rfs, as shown in (b) of FIG. 15, need to be further formed to up-mix the down-mix audio signal to the 7.1-channel multi-channel audio signal.

Since the left front side channel Lfs is located between the left front channel L and the left surround channel Ls, it is possible to determine the level of the left front side channel Lfs by interpolation using the level of the left front channel L and the level of the left surround channel Ls.

16 is a diagram for explaining levels of two left channels and a level of a left front side channel (Lfs).

Referring to FIG. 16, it can be seen that the level c of the left front side channel Lfs is a linearly interpolated value based on the level a of the left front channel L and the level b of the left surround channel Ls.

Although the left front side channel Lfs is between the left front channel L and the left surround channel Ls, it can be placed outside the left front channel L, the center channel C and the right front channel R. Thus, the level of the left front side channel Lfs can be determined by extrapolation using the levels of the left front channel L, the center channel C and the right front channel R.

17 is a diagram for explaining levels of three front channels and a level of a left front side channel (Lfs).

Referring to FIG. 17, it can be seen that the level c of the left front side channel Lfs is a linearly interpolated value based on the level a of the left front channel L, the level c of the central channel C and the level b of the right front channel.

In the above description, the process of generating an audio signal of an output channel by adding extended spatial information to spatial information is explained with reference to two examples. As mentioned in the above description, in the process of upmixing with the addition of extended spatial information, the expanded spatial parameters may not be fully applied, but partially. Thus, the process of applying spatial parameters to an audio signal can be performed sequentially and hierarchically or collectively and synthetically.

Industrial applicability

Therefore, the present invention provides the following results.

Firstly, the present invention makes it possible to generate an audio signal having a configuration different from a predetermined tree configuration, thereby generating differently configured output signals.

Secondly, since it is possible to generate an audio signal having a configuration different from a predetermined tree configuration, even if the number of the plurality of channels before performing downmix is less than or more than the number of speakers, it is possible to generate output channels in an amount equal to the number of speakers from the downmix audio signal.

Thirdly, in the case of generating output channels in an amount smaller than the plurality of channels, since the multi-channel audio signal is directly generated from the down-mix audio signal instead of the down-mix of the output channel audio signal from the multi-channel audio signal generated during the up-mix of the down-mix audio signal, the load of operations required to decode the audio signal.

Fourth, since sound paths are taken into account when generating combined spatial information, the present invention provides a pseudo-surround sound effect in case the surround channel output is not available.

Although the present invention is shown and described in this embodiment with reference to its preferred embodiments, it will be apparent to those skilled in the art that various modifications and changes in form and content can be made without departing from the spirit and scope of the invention . Thus, it is intended that the present invention cover the modifications and variations of this invention that come within the scope of the claims and their equivalents.

Claims (18)

1. A method of decoding an audio signal, comprising:
receiving audio and spatial information;
the formation of modified spatial information using spatial information and
decoding an audio signal using modified spatial information,
wherein the modified spatial information includes at least one of the partial spatial information, the combined spatial information and the extended spatial information. 2. The method according to claim 1, in which the formation of the modified spatial information is performed based on an indicator included in the spatial information. 3. The method according to claim 1, in which the formation of the modified spatial information is performed based on tree configuration information included in the spatial information. 4. The method according to claim 1, in which the formation of the modified spatial information is performed based on the information of the output channel. 5. The method according to claim 1, in which spatial information includes spatial parameters, and partial spatial information includes part of the spatial parameters. 6. The method according to claim 5, in which the spatial parameters are hierarchical, and partial spatial information includes spatial parameters of the upper level. 7. The method according to claim 6, in which the partial spatial information further includes partially spatial parameters of the lower level. 8. The method according to claim 1, in which the spatial information includes spatial parameters, and the combined spatial information is formed from a combination of spatial parameters. 9. The method according to claim 1, in which the extended spatial information is generated using spatial information and extended spatial information. 10. An apparatus for decoding an audio signal, comprising:
a modified spatial information generating unit, wherein said modified spatial information generating unit generates modified spatial information using spatial information; and
an output channel generating unit decoding an audio signal using modified spatial information,
wherein the modified spatial information includes at least one of the partial spatial information, the combined spatial information and the extended spatial information. 11. The device according to claim 10, in which the modified spatial information is generated based on the indicator included in the spatial information. 12. The device according to claim 10, in which the modified spatial information is generated on the basis of tree configuration information included in the spatial information. 13. The device according to claim 10, in which the modified spatial information is generated based on the information of the output channel. 14. The device according to claim 10, in which the spatial information includes spatial parameters, and partial spatial information includes part of the spatial parameters. 15. The device according to 14, in which the spatial parameters are hierarchical, and partial spatial information includes spatial parameters of the upper level. 16. The device according to clause 15, in which the partial spatial information further includes partially spatial parameters of the lower level. 17. The device according to claim 10, in which the spatial information includes spatial parameters, and the combined spatial information is formed from a combination of spatial parameters. 18. The device according to claim 10, in which the extended spatial information is generated using spatial information and extended spatial information.

Images