RU2406166C2 - Coding and decoding methods and devices based on objects of oriented audio signals - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: audio decoding method involves receiving a step-down mixing signal based on additional information objects and control information, where the step-down mixing signal contains at least two step-down mixing channel signals and is generated through step-down mixing of multiple object signals, and control information controls the position or level or the object signal contained in the step-down mixing signal; extraction of the ratio of distribution of the object signals between channels from the object-based additional information; generation of pre-processing information for modifying step-down mixing channel signals based on information on the ratio of distribution between channels and control information; generation of a mutichannel audio signal based on the object-based additional information and control information and modified step-down mixing channel signals.

EFFECT: efficient processing of object-based audio signals.

11 cl, 33 dwg

Description

FIELD OF THE INVENTION

The present invention relates to an audio encoding method and apparatus, and an audio decoding method and apparatus in which object-based audio signals can be efficiently processed by performing encoding and decoding operations.

State of the art

In general, in multi-channel audio encoding and decoding techniques, for a certain number of channel signals of a multi-channel signal, down-mixing is performed to obtain fewer channel signals, additional information regarding the original channel signals is transmitted, and the multi-channel signal having the same number of channels as the original multi-channel signal is restored.

The encoding and decoding techniques of object-based audio are essentially the same as the multichannel audio encoding and decoding techniques for down-mixing multiple audio sources into fewer audio source signals and transmitting additional information regarding the original audio sources. However, in the encoding and decoding techniques of object-based audio, object signals, which are the basic signals (for example, a musical instrument or human voice) of a channel signal, are interpreted in the same way as channel signals in encoding and decoding techniques of multi-channel audio, and thereby most can be encoded.

In other words, in object-based audio encoding and decoding techniques, object signals are considered objects to be encoded. In this sense, object-based audio encoding and decoding techniques are different from multi-channel audio encoding and decoding techniques, in which the multi-channel audio encoding operation is simply based on inter-channel information regardless of the number of channel signal elements to be encoded.

SUMMARY OF THE INVENTION

Technical challenge

The present invention provides an audio encoding method and apparatus and an audio decoding method and apparatus in which audio signals can be encoded or decoded so that the audio signals can be applied to various embodiments.

Technical solution

According to an aspect of the present invention, there is provided a method for decoding audio, including: receiving a downmix signal and object-based additional information, wherein the downmix signal includes at least two channel downmix signals; extracting gain information from the object-based additional information and generating modification information for modifying channel down-mix signals on a per-channel basis based on gain information; and modifying the downmix channel signals by applying the modification information to the downmix channel signals.

According to another aspect of the present invention, there is provided an audio encoding method including: generating a downmix signal by downmixing an object signal, wherein the downmix signal includes at least two channel downmix signals; extracting object-related information related to the object signal and generating object-based additional information based on the object-related information; and inserting gain information for modifying the channel down-mix signals on a per-channel basis into object-based additional information.

According to another aspect of the present invention, there is provided an audio decoding apparatus including: a demultiplexer configured to extract a down-mix signal and object-based additional information from an input audio signal, wherein the down-mix signal includes at least two down-channel signals mixing; and a transcoder configured to generate modification information for modifying the channel down-mix signals on a per-channel basis based on gain information extracted from the object-based additional information, and modify the down-mix channel signals by applying the modification information to the down-mix channel signals.

According to another aspect of the present invention, there is provided a computer-readable recording medium on which a computer program for executing an audio decoding method is recorded, wherein the audio decoding method includes: receiving a downmix signal and object-based additional information, wherein the downmix signal includes, at least two channel down-mix signals; extracting gain information from the object-based additional information and generating modification information for modifying channel down-mix signals on a per-channel basis based on gain information; and modifying the downmix channel signals by applying the modification information to the downmix channel signals.

According to another aspect of the present invention, there is provided a computer-readable recording medium on which a computer program for executing an audio encoding method is recorded, wherein the audio encoding method includes: generating a downmix signal by downmixing an object signal, wherein the downmix signal includes at least two channel down-mix signals; extracting object-related information related to the object signal and generating object-based additional information based on the object-related information; and inserting gain information for modifying the channel down-mix signals on a per-channel basis into object-based additional information.

Brief Description of the Drawings

Figure 1 illustrates a block diagram of a typical encoding / decoding system of object-based audio;

2 illustrates a block diagram of an audio decoding apparatus according to a first embodiment of the present invention;

Figure 3 illustrates a block diagram of an audio decoding apparatus according to a second embodiment of the present invention;

4 illustrates a block diagram of an audio decoding apparatus according to a third embodiment of the present invention;

FIG. 5 illustrates a block diagram of an arbitrary down-mix gain (ADG) module that can be used in the audio decoding module illustrated in FIG. 4;

6 illustrates a block diagram of an audio decoding apparatus according to a fourth embodiment of the present invention;

7 illustrates a block diagram of an audio decoding apparatus according to a fifth embodiment of the present invention;

Fig. 8 illustrates a block diagram of an audio decoding apparatus according to a sixth embodiment of the present invention;

9 illustrates a block diagram of an audio decoding apparatus according to a seventh embodiment of the present invention;

10 illustrates a block diagram of an audio decoding apparatus according to an eighth embodiment of the present invention;

11 and 12 illustrate diagrams for explaining the operation of the transcoder;

13-16 illustrate diagrams for explaining the configuration of object-based additional information;

17-22 illustrate diagrams for explaining combining a plurality of fragments of object-based additional information into one fragment of additional information;

23-27 illustrate diagrams for explaining a preprocessing operation; and

28-33 are diagrams illustrating a case of combining a plurality of bit streams decoded using object-based signals into a single bit stream.

Optimum Mode for Carrying Out the Invention

The invention will now be described in more detail with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown.

The audio encoding method and apparatus and the audio decoding method and apparatus according to the present invention can be applied to object-based audio processing operations, but the present invention is not limited thereto. In other words, the audio encoding method and apparatus and the audio decoding method and apparatus can be applied to various signal processing operations other than processing operations based on audio objects.

Figure 1 illustrates a block diagram of a typical encoding / decoding system for object-based audio. In general, the audio signals input to the object-based audio encoding device do not correspond to the channels of the multi-channel signal, but are independent object signals. In this sense, an object-based audio encoding device is different from a multi-channel audio encoding device into which channel signals of a multi-channel signal are input.

For example, channel signals, such as a front left channel signal and a front right channel signal for a 5.1 channel signal, can be input into a multi-channel audio signal, while object signals, such as a human voice or the sound of a musical instrument (for example, the sound of a violin or piano ), which are smaller objects than channel signals, can be input into an object-based audio encoding device.

Referring to FIG. 1, an object-based audio encoding / decoding system includes an object-based audio encoding device and an object-based audio decoding device. An object-based audio encoding apparatus includes an object encoder 100, and an object-based audio decoding apparatus includes an object decoder 111 and a mixer / renderer 113.

The object encoder 100 receives N object signals and generates an object-based downmix signal with one or more channels and additional information including a series of pieces of information extracted from N object signals, such as energy difference information, phase difference information and correlation information. The additional information and the object-based down-mix signal are combined into a single bit stream, and the bit stream is transmitted to the object-based decoding device.

Additional information may include a flag indicating whether encoding of channel-based audio or encoding of object-based audio should be performed, and thereby it can be determined whether encoding of channel-based audio or encoding of object-based audio should be performed based on an additional flag. information. Additional information may also include energy information, grouping information, silent period information, downmix gain information, and delay information regarding object signals.

The additional information and the object-based downmix signal may be combined into a single bitstream, and the bitstream may be transmitted to the object-based audio decoding device.

The object decoder 111 receives the object-based downmix signal and additional information from the object-based encoding device of the audio, and restores object signals having properties similar to the properties of the N object signals based on the object-based downmix signal and additional information. The object signals generated by the object decoder 111 have not yet been assigned any position in the multi-channel space. Thus, the mixer / renderer 113 assigns each of the object signals generated by the object decoder 111 to a predetermined position in the multi-channel space and determines the levels of the object signals so that the object signals can be reproduced from the corresponding respective positions indicated by the mixer / renderer 113, with the appropriate appropriate levels determined by the mixer / renderer 113. Control information regarding each of the object signals generated osredstvom object decoder 111 may vary over time, and thus the spatial positions and the levels of object signals generated by the object decoder 111 may vary according to the control information.

FIG. 2 illustrates a block diagram of an audio decoding apparatus 120 according to a first embodiment of the present invention. Referring to FIG. 2, an audio decoding apparatus 120 may be able to perform adaptive decoding by analyzing control information.

Referring to FIG. 2, an audio decoding apparatus 120 includes an object decoder 121, a mixer / renderer 123, and a parameter converter 125. Audio decoding apparatus 120 may also include a demultiplexer (not shown) that extracts a downmix signal and additional information from a bit stream input thereto, and it is applied to all audio decoding apparatuses according to other embodiments of the present invention.

The object decoder 121 generates a series of object signals based on the down-mix signal and modified additional information provided by the parameter converter 125. The mixer / renderer 123 assigns each of the object signals generated by the object decoder 121 to a predetermined position in the multi-channel space and determines the levels of object signals generated by the object decoder 121 according to the control information. A parameter converter 125 generates modified additional information by combining additional information and control information. Then, the parameter converter 125 transmits the modified additional information to the object decoder 121.

The object decoder 121 may be able to perform adaptive decoding by analyzing control information in the modified additional information.

For example, if the control information indicates that the first object signal and the second object signal are assigned to the same position in the multi-channel space and have the same level, a typical audio decoding device can decode the first and second object signals separately and then compose them in the multi-channel space through a mixing operation / rendering.

On the other hand, the object decoder 121 of the audio decoding apparatus 120 recognizes from the control information in the modified supplementary information that the first and second object signals are assigned to the same position in the multi-channel space and have the same level as if they were the same sound source. Accordingly, the object decoder 121 decodes the first and second object signals by interpreting them as a single sound source without decoding them separately. As a result, decoding complexity is reduced. In addition, due to the reduction in the number of sound sources to be processed, the complexity of mixing / rendering is also reduced.

Audio decoding apparatus 120 can be effectively used when the number of object signals is greater than the number of output channels, since many object signals are most likely to be assigned to one spatial position.

Alternatively, audio decoding apparatus 120 may be used when the first object signal and the second object signal are assigned to the same position in the multi-channel space, but at different levels. In this case, the audio decoding apparatus 120 decodes the first and second object signals by interpreting the first and second object signals as a single signal, instead of decoding the first and second object signals separately and transmitting the decoded first and second object signals to the mixer / renderer 123. More specifically, the object decoder 121 may receive information regarding the difference between the levels of the first and second object signals from the control information in the modified additional information, and decode the first and second object signals based on the received information. As a result, even if the first and second object signals have different levels, the first and second object signals can be decoded as if they were a single sound source.

Still alternatively, object decoder 121 may adjust the levels of object signals generated by object decoder 121 according to control information. Further, the object decoder 121 can decode object signals whose levels are adjusted. Accordingly, the mixer / renderer 123 does not have to adjust the levels of the decoded object signals provided by the object decoder 121, but simply composes the decoded object signals provided by the object decoder 121 in a multi-channel space. Briefly, since the object decoder 121 adjusts the levels of the object signals generated by the object decoder 121, according to the control information, the mixer / renderer 123 can easily compose the object signals generated by the object decoder 121 in multi-channel space without the need to further adjust the levels of the object signals generated by the object decoder 121. Consequently, mixing / rendering complexity can be reduced.

According to the embodiment of FIG. 2, the object decoder of the audio decoding apparatus 120 can adaptively perform a decoding operation by analyzing control information, thereby reducing the decoding complexity and the complexity of mixing / rendering. A combination of the above methods performed by the audio decoding apparatus 120 may be used.

FIG. 3 illustrates a block diagram of an audio decoding apparatus 130 according to a second embodiment of the present invention. Referring to FIG. 3, the audio decoding apparatus 130 includes an object decoder 131 and a mixer / renderer 133. The audio decoding device 130 is different by providing additional information not only to the object decoder 131, but also to the mixer / renderer 133.

An audio decoding apparatus 130 can efficiently perform a decoding operation even when there is an object signal corresponding to a period of silence. For example, the second to fourth object signals may correspond to a music playback period during which a musical instrument is played, and the first object signal may correspond to a mute period during which only background music is played, and the first object signal may correspond to a silence period during which accompaniment is played. In this case, information indicating which of the plurality of object signals corresponds to the silence period can be included in the additional information, and additional information can be provided in the mixer / renderer 133, as well as in the object decoder 131.

The object decoder 131 can minimize the decoding speed not only by decoding the object signal corresponding to the silence period. The object decoder 131 sets the object signal corresponding to a value of 0, and transmits the level of the object signal to the mixer / renderer 133. In general, object signals having a value of 0 are interpreted in the same way as object signals having a value other than 0, and thereby undergo mixing / rendering operations.

On the other hand, the audio decoding apparatus 130 transmits additional information including information indicating which of the plurality of object signals corresponds to the silence period to the mixer / renderer 133, and thereby does not allow processing of the object signal corresponding to the silence period through an operation mixing / rendering performed by the mixer / renderer 133. Consequently, the audio decoding apparatus 130 avoids unnecessarily increasing mixing / rendering complexity.

FIG. 4 illustrates a block diagram of an audio decoding apparatus 140 according to a third embodiment of the present invention. Referring to FIG. 4, an audio decoding apparatus 140 uses a multi-channel decoder 141 instead of an object decoder and a mixer / renderer and decodes a series of object signals after the object signals are properly arranged in the multi-channel space.

More specifically, the audio decoding apparatus 140 includes a multi-channel decoder 141 and a parameter converter 145. The multi-channel 141 decoder generates a multi-channel signal, the object signals of which are already arranged in the multi-channel space, on the basis of the down-mix signal and the spatial parameter information, which is information based on the parameter channels provided by the parameter converter 145. The parameter converter 145 analyzes additional information and control information transmitted by an audio encoding device (not shown), and generates spatial parameter information based on the analysis result. More specifically, the parameter transformer 145 generates spatial parameter information by combining additional information and control information, which includes reproduction setting information and mixing information. Those. a parameter converter 145 converts a combination of additional information and control information into spatial data corresponding to a one-to-two (OTT) module or a two-to-three module (TTT).

The audio decoding apparatus 140 may perform a multi-channel decoding operation in which an object-based decoding operation and a mixing / rendering operation are combined, and thereby can skip decoding of each object signal. Therefore, it is possible to reduce the complexity of decoding and / or mixing / rendering.

For example, when there are 10 object signals, and a multi-channel signal derived from 10 object signals must be reproduced by a 5.1 channel speaker system, a typical object-based audio decoding device generates decoded signals appropriately corresponding to 10 object signals based on the signal downmix and additional information, and then generates a 5.1-channel signal by appropriately linking 10 object signals into a multi-channel space GUSTs, so that the object signals can become suitable for 5.1-channel acoustic environments. However, it is not enough to generate 10 object signals during the generation of the 5.1-channel signal, and this problem becomes more serious as the difference between the number of object signals and the number of channels of the multichannel signal to be generated increases.

On the other hand, in the embodiment of FIG. 4, the audio decoding apparatus 140 generates spatial parameter information suitable for a 5.1 channel signal based on additional information and control information and provides spatial parameter information and a downmix signal to a multi-channel decoder 141. Then, multi-channel decoder 141 generates a 5.1 channel signal based on spatial parameter information and a downmix signal. In other words, when the number of channels to be output is 5.1 channels, the audio decoding apparatus 140 can simply generate a 5.1 channel signal based on the downmix signal without the need to generate 10 object signals and is thus more efficient than a conventional device audio decoding regarding complexity.

The audio decoding apparatus 140 is considered effective when the amount of computation required to calculate the spatial parameter information corresponding to each of the OTT module and the TTT module by analyzing additional information and control information transmitted by the audio encoding apparatus is less than the computational amount required in order to perform the mixing / rendering operation after decoding each object signal.

An audio decoding apparatus 140 can be obtained by adding a module for generating spatial parameter information by analyzing additional information and control information into a typical multi-channel audio decoding device and therefore can maintain compatibility with a typical multi-channel audio decoding device. Also, decoding apparatus 140 can improve sound quality using existing means of a typical multichannel audio decoding apparatus, such as an envelope shaper, subband temporal processing (STP) means, and a decorrelator. Given all this, it should be concluded that all the advantages of a typical method for decoding multi-channel audio can be easily applied to a method for decoding object audio.

The spatial parameter information transmitted to the multi-channel decoder 141 by the parameter converter 145 may be compressed so as to be suitable for transmission. Alternatively, the spatial parameter information may have the same format as the data format transmitted by a typical multi-channel encoding device. Those. the spatial parameter information may be subjected to a Huffman decoding operation or a control decoding operation, and thereby may be transmitted to each module as uncompressed spatial mark data. The first is suitable for transmitting spatial parameter information to a multi-channel audio decoding device in a remote location, and the second is convenient since there is no need for the multi-channel audio decoding device to convert compressed spatial label data to uncompressed spatial label data, which can be easily used in the decoding operation.

The configuration of spatial delay information based on the analysis of additional information and control information may cause a delay. To compensate for this delay, an additional buffer for the downmix signal may be provided so that the delay between the downmix signal and the bitstream can be compensated. Alternatively, an additional buffer may be provided for spatial parameter information obtained from the control information so that the delay between the spatial parameter information and the bitstream can be compensated. These methods, however, are inconvenient due to the need to provide an additional buffer. Alternatively, additional information may be transmitted ahead of the downmix signal, taking into account the possibility of a delay between the downmix signal and the spatial parameter information. In this case, the spatial parameter information obtained by combining additional information and control information does not have to be adjusted, but can easily be used.

If the plurality of object signals from the downmix signal have different levels, an arbitrary downmix gain module (ADG) that can directly compensate for the downmix signal can determine the relative levels of the object signals, and each of the object signals can be assigned to a predetermined position in the multi-channel space using spatial label data, such as channel level difference (CLD) information, inter-channel correlation information (ICC) and channel prediction coefficient (CPC) information.

For example, if the control information indicates that a predefined object signal should be assigned to a predefined position in multi-channel space and is higher than other object signals, a typical multi-channel decoder can calculate the difference between the channel energies in the downmix signal and divide the downmix signal mixing on the number of output channels based on the calculation results. However, a typical multi-channel decoder cannot increase or decrease the volume of a particular sound in a downmix signal. In other words, a typical multi-channel decoder simply distributes the downmix signal over the number of output channels and thus cannot increase or decrease the sound volume in the downmix signal.

It is relatively simple to assign each of a number of object signals in a downmix signal generated by the object decoder to a predetermined position in the multi-channel space according to the control information. However, special techniques are required in order to increase or decrease the amplitude of a predefined object signal. In other words, if the downmix signal generated by the object decoder is used as is, it is difficult to reduce the amplitude of each object signal in the downmix signal.

Therefore, according to an embodiment of the present invention, the relative amplitudes of the object signals can be varied according to the control information by using the ADG module 147 illustrated in FIG. The ADG module 147 may be installed in the multi-channel decoder 141 or may be separate from the multi-channel decoder 141.

If the relative amplitudes of the object signals in the downmix signal are properly adjusted using the ADG module 147, you can perform object decoding using a typical multi-channel decoder. If the down-mix signal generated by the object decoder is a mono or stereo signal or a multi-channel signal with three or more channels, the down-mix signal can be processed by the ADG module 147. If the down-mix signal generated by the object decoder has two or more channels, and a predefined object signal to be adjusted by the ADG module 147 exists in only one of the channels of the downmix signal In contrast, the ADG module 147 can only be applied to a channel that includes a predefined object signal, instead of applying a down-mix signal to all channels. The downmix signal processed by the ADG module 147 in the manner described above can be easily processed using a typical multi-channel encoder without the need to modify the structure of the multi-channel decoder.

Even when the final output signal is not a multi-channel signal that can be reproduced by a multi-channel speaker system, but is a stereo (binaceal) signal, the ADG module 147 can be used to adjust the relative amplitudes of the object signals of the final output signal.

Alternative to using the ADG module 147, gain information specifying a gain value to be applied to each object signal may be included in the control information during the generation of a number of object signals. For this, the structure of a typical multi-channel decoder can be modified. Despite the need to modify the structure of an existing multi-channel decoder, this method is convenient in terms of decoding complexity by applying a gain value to each object signal during the decoding operation without the need to calculate ADG and compensate for each object signal.

The ADG module 147 can be used not only to control the object signals, but also to modify the spectrum information of a particular object signal. More specifically, the ADG module 147 can be used not only to raise or lower the level of a specific object signal, but also to modify the spectrum information of a specific object signal, such as amplification of the high-tone or low-tone part of a specific object signal. It is not possible to modify spectrum information without using the ADG module 147.

6 illustrates a block diagram of an audio decoding apparatus 150 according to a fourth embodiment of the present invention. Referring to FIG. 6, an audio decoding apparatus 150 includes a multi-channel stereo decoder 151, a first parameter converter 157 and a second parameter converter 159.

The second parameter converter 159 analyzes additional information and control information that is provided by the audio encoding device, and configures the spatial parameter information based on the analysis result. The first parameter converter 157 configures virtual three-dimensional (3D) parameter information that can be used by the multi-channel stereo decoder 151 by adding three-dimensional (3D) information, such as a sound perception modeling (HRTF) function, to spatial parameter information. A multi-channel stereo decoder 151 generates a stereo signal by applying stereo information to a downmix signal.

The first parameter converter 157 and the second parameter converter 159 can be replaced by one module, i.e. a parameter conversion module 155, which receives additional information, control information, and 3D information and configures stereo parameter information based on the additional information, control information, and HRTF parameters.

Traditionally, in order to generate a stereo signal for reproducing a downmix signal including 10 object signals using headphones, the object signal must generate 10 decoded signals appropriately corresponding to 10 object signals based on the downmix signal and additional information. The mixer / renderer then assigns each of 10 object signals of a predetermined position in the multi-channel space with reference to control information so as to satisfy the requirements of the 5-channel acoustic environment. After that, the mixer / renderer generates a 5-channel signal, which can be reproduced by means of a 5-channel speaker system. Next, the mixer / renderer applies 3D information to the 5-channel signal, thereby forming a 2-channel signal. Briefly, the aforementioned conventional audio decoding method includes reproducing 10 object signals, converting 10 object signals to a 5-channel signal, and generating a 2-channel signal based on the 5-channel signal, and this is thus ineffective.

On the other hand, the audio decoding apparatus 150 can easily generate a stereo signal, which can be reproduced using headphones, based on object signals. In addition, the audio decoding apparatus 150 configures spatial parameter information by analyzing additional information and control information, and thereby can generate a stereo signal using a typical multi-channel stereo decoder. Moreover, the audio decoding apparatus 150 may use a typical multi-channel stereo decoder even when equipped with an integrated parameter converter that receives additional information, control information and HRTF parameters and configures stereo parameter information based on the additional information, control information and HRTF parameters.

7 illustrates a block diagram of an audio decoding apparatus 160 according to a fifth embodiment of the present invention. Referring to FIG. 7, an audio decoding apparatus 160 includes a preprocessor 161, a multi-channel 163 decoder, and a parameter converter 165.

The parameter converter 165 generates spatial parameter information that can be used by the multi-channel decoder 163, and parameter information that can be used by the preprocessor 161. The preprocessor 161 performs the preprocessing operation with the downmix signal and transmits the downmix signal resulting from the preliminary operation processing to the multi-channel decoder 163. The multi-channel decoder 163 performs the decoding operation of the signal ala downmix signal transmitted by the preprocessor 161, thereby outputting a stereo signal, stereo and multi-channel stereo signal. Examples of the preprocessing operation performed by preprocessor 161 include modifying or converting a downmix signal in a time domain or a frequency domain by filtering.

If the down-mix signal input to the audio decoding apparatus 160 is a stereo signal, the down-mix signal may need to be subjected to down-mix processing performed by the preprocessor 161 before being input to the multi-channel decoder 163, since the multi-channel decoder 163 cannot convert the object signal, corresponding to the left channel of the stereo down-mix signal, into the right channel of the multi-channel signal by decoding. Therefore, in order to shift the object signal belonging to the left channel of the stereo down-mix signal to the right channel, the stereo down-mix signal may need to be pre-processed by preprocessor 161, and the pre-processed down-mix signal can be input to multi-channel decoder 163.

The preprocessing of the stereo down-mix signal can be performed based on the preprocessing information obtained from the additional information and from the control information.

FIG. 8 illustrates a block diagram of an audio decoding apparatus 170 according to a sixth embodiment of the present invention. Referring to FIG. 8, an audio decoding apparatus 170 includes a multi-channel decoder 171, a post-processor 173, and a parameter converter 175.

The parameter converter 175 generates spatial parameter information that can be used by the multi-channel decoder 163, and parameter information that can be used by the post-processor 173. The post-processor 173 performs a post-processing operation on a signal output by the multi-channel decoder 173. Examples of the signal output by the multi-channel decoder 173 include a stereo signal, a stereo stereo signal, and a multi-channel signal.

Examples of the post-processing operation performed by the post-processor 173 include modifying and converting each channel or all channels of the output signal. For example, if the additional information includes natural frequency information regarding a predetermined object signal, post processor 173 may remove harmonic components from the predetermined object signal with reference to natural frequency information. A multi-channel audio decoding method may not be effective enough to be used in a karaoke system. However, if the natural frequency information regarding the vocal object signals is included in the additional information, and the harmonic components of the vocal object signals are removed during the post-processing operation, a high-performance karaoke system can be implemented by using the embodiment of FIG. 8. The embodiment of FIG. 8 can also be applied to object signals other than vocal object signals. For example, you can delete the sound of a predefined musical instrument by using the embodiment of FIG. Also, it is possible to amplify predetermined harmonic components with natural frequency information regarding the object signals by using the embodiment of FIG. In short, post-processing parameters may provide the ability to apply various effects, such as inserting a reverb effect, adding noise, and amplifying the low part that cannot be applied by multi-channel decoder 171.

Postprocessor 173 can directly apply an additional effect to the downmix signal or add a downmix signal, to which the effect has already been applied, to the output of multi-channel decoder 171. Postprocessor 173 can always change the object spectrum or modify the downmix signal if necessary. If it is not suitable to directly perform an effect processing operation, such as reverb, for the downmix signal and transmit the signal obtained by the effect processing operation to the multi-channel decoder 171, the preprocessor 173 can simply add the signal obtained by the effects processing operation to the output of the multi-channel decoder 171 instead of directly executing the effects processing with the downmix signal and transmitting the result of the effects processing to the multi-channel decoder 171.

FIG. 9 illustrates a block diagram of an audio decoding apparatus 180 according to a seventh embodiment of the present invention. Referring to FIG. 9, an audio decoding apparatus 180 includes a preprocessor 181, a multi-channel decoder 183, a post-processor 185, and a parameter converter 187.

The description of preprocessor 161 is directly applicable to preprocessor 181. Postprocessor 185 can be used to summarize the output of preprocessor 181 and the output of multi-channel decoder 185 and thereby provide the final signal. In this case, the post-processor 185 acts merely as an adder for summing the signals. An effect parameter may be provided to that of preprocessor 181 and postprocessor 185 that implements the effect. In addition, adding the signal obtained by applying the effect of the down-mix signal to the output of the multi-channel decoder 183 and applying the effect to the output of the multi-channel decoder 185 can be performed simultaneously.

The preprocessors 161 and 181 of FIGS. 7 and 9 may render the downmix signal according to control information provided by the user. In addition, the preprocessors 161 and 181 of FIGS. 7 and 9 can raise or lower levels of object signals and change the spectrum of object signals. In this case, the preprocessors 161 and 181 of FIGS. 7 and 9 can perform the functions of an ADG module.

Rendering of the object signal according to the direction information of the object signal, adjusting the level of the object signal and changing the spectrum of the object signal can be performed simultaneously. In addition, part of the rendering of the object signal according to the information of the direction of the object signal, adjusting the level of the object signal and changing the spectrum of the object signal can be performed using preprocessor 161 or 181, and even from rendering the object signal according to the information of the direction of the object signal, adjusting the level of the object signal and changing the spectrum of the object a signal that is not executed by the preprocessor 161 or 181 can be performed by the ADG module. For example, it is inefficient to change the spectrum of an object signal by using an ADG module that uses a quantization level interval and a parametric band interval. In this case, preprocessor 161 or 181 can be used to constantly change the spectrum of the object signal on a frequency basis, and the ADG module can be used to adjust the level of the object signal.

10 illustrates a block diagram of an audio decoding apparatus according to an eighth embodiment of the present invention. Referring to FIG. 10, an audio decoding apparatus 200 includes a rendering matrix generator 201, a transcoder 203, a multi-channel decoder 205, a preprocessor 207, an effects processor 208, and an adder 209.

The rendering matrix generator 201 generates a rendering matrix that represents object position information regarding the positions of the object signals and the playback configuration information regarding the object signal levels and provides the rendering matrix to the transcoder 203. The rendering matrix generator 201 generates 3D information, such as an HRTF coefficient based on the position information of the objects. HRTF is a transfer function that describes the transmission of sound waves between a sound source in an arbitrary position and the eardrum, and returns a value that varies according to the direction and height of the sound source. If a signal with no directivity is filtered using HRTF, the signal can be heard as if it were being played back from a specific direction.

Object position information and playback configuration information that is received by the rendering matrix generator 201 may vary over time and may be provided by the end user.

Transcoder 203 generates channel-based additional information based on object-based additional information, a rendering matrix, and 3D information, and provides, to multi-channel decoder 209, channel-based additional information and 3D information required for multi-channel decoder 209. That is, transcoder 203 transmits channel-based additional information regarding M channels, which is obtained from object-based parameter information regarding N object signals and 3D information for each of N object signals to multi-channel decoder 205.

Multi-channel decoder 205 generates a multi-channel audio signal based on the down-mix signal and channel-based additional information provided by transcoder 203, and performs 3D rendering of the multi-channel audio signal according to 3D information, thereby generating a 3D multi-channel signal. The rendering matrix generator 201 may include a database of 3D information (not shown).

If there is a need to pre-process the down-mix signal before the down-mix signal is input to the multi-channel decoder 205, the transcoder 203 transmits information regarding the preprocessing to the preprocessor 207. The object-based additional information includes information regarding all the object signals, and the rendering matrix includes information about the position of the objects and the configuration information of the playback. The transcoder 203 may generate channel-based additional information based on the object-based additional information and the rendering matrix, and then generates the channel-based additional information required for mixing and reproducing the object signals according to the channel information. Then, transcoder 203 transmits the channel-based supplemental information to multi-channel decoder 205.

Channel-based supplemental information and 3D information provided by transcoder 205 may include frame indices. Thus, the multi-channel decoder 205 can synchronize the channel-based additional information and 3D information by using frame indices, and thereby may be able to apply 3D information only to certain frames of the bitstream. In addition, even if the 3D information is updated, it is easy to synchronize the channel-based additional information and the updated 3D information by using frame indices. Those. frame indices can be included in the channel-based additional information and 3D information, respectively, so that the multi-channel decoder 205 synchronizes the channel-based additional information and 3D information.

The preprocessor 207 can pre-process the input down-mix signal, if necessary, before the input down-mix signal is input to the multi-channel decoder 205. As described above, if the input down-mix signal is a stereo signal, and there is no need to reproduce the object signal belonging to the left channel, from the right channel, the down-mix signal may need to be pre-processed by preprocessor 207, per ed by input to multi-channel decoder 205, since multi-channel decoder 205 cannot shift an object signal from one channel to another. The information required for preprocessing the input down-mix signal can be provided to preprocessor 207 by transcoder 205. The down-mix signal obtained by preprocessing performed by preprocessor 207 can be transmitted to multi-channel decoder 205.

The effects processor 208 and the adder 209 can directly apply an additional effect to the downmix signal or add a downmix signal to which the effect has already been applied to the output of the multi-channel decoder 205. The effects processor 208 can always change the spectrum of an object or modify the downmix signal if necessary. If it is not suitable to directly perform an effect processing operation, such as reverb, for the downmix signal and transmit the signal obtained by the effect processing operation to the multi-channel decoder 205, the effects processor 208 can simply add the signal obtained by the effects processing operation to the output of the multi-channel decoder 205 instead of directly executing the effects processing with the downmix signal and transmitting the result of the effects processing to the multi-channel decoder 205.

The following describes in detail the rendering matrix generated by the renderer 201 of the rendering matrix.

A rendering matrix is a matrix that represents the position and reproduction configuration of object signals. Those. if there are N object signals and M channels, the rendering matrix may indicate how N object signals are converted to M channels in various ways.

More specifically, when N object signals are mapped onto M channels, an N × M rendering matrix can be set. In this case, the rendering matrix includes N rows, which respectively represent N object signals, and M columns, which respectively represent M channels. Each of the M coefficients in each of the N lines may be a real number or an integer indicating the ratio of the part of the object signal assigned to the corresponding channel to the entire object signal.

More specifically, the M coefficients in each of the N rows of the N × M rendering matrix may be real numbers. In this case, if the sum of the M coefficients in the row of the N × M rendering matrix is equal to a predetermined reference value, for example 1, it can be determined that the level of the object signal did not vary. If the sum of the M coefficients is less than 1, it is determined that the level of the object signal has decreased. If the sum of the M coefficients is greater than 1, it is determined that the level of the object signal has increased. The predefined reference value may be a numerical value other than 1. The value by which the level of the object signal is varied may be limited to a range of 12 dB. For example, if the predetermined reference value is 1 and the sum of the M coefficients is 1.5, it can be determined that the level of the object signal has increased by 12 dB. If the predefined reference value is 1 and the sum of the M coefficients is 0.5, it is determined that the level of the object signal has decreased by 12 dB. If the predetermined reference value is 1, and the sum of the M coefficients is 0.5-1.5, it is determined that the object signal varies by a predetermined value between -12 dB and +12 dB, and the predetermined value can be linearly determined according to sum of M coefficients.

The M coefficients in each of the N rows of the N × M rendering matrix can be integers. In this case, if the sum of the M coefficients in the row of the N × M rendering matrix is equal to a predetermined reference value, for example, 10, 20, 30, or 100, it can be determined that the level of the object signal did not vary. If the sum of the M coefficients is less than a predetermined reference value, it can be determined that the level of the object signal has not decreased. If the sum of the M coefficients is greater than a predetermined reference value, it can be determined that the level of the object signal has not increased. The value by which the level of the object signal is varied can be limited, for example, to a range of 12 dB. The amount by which the sum of the M coefficients differs from the predefined reference value may represent the amount (in dB) by which the level of the object signal was varied. For example, if the sum of the M coefficients is greater than a predetermined reference value, it can be determined that the level of the object signal has increased by 2 dB. Therefore, if the predetermined reference value is 20, and the sum of the M coefficients is 23, it can be determined that the level of the object signal has increased by 6 dB. If the predetermined reference value is 20, and the sum of the M coefficients is 15, it can be determined that the level of the object signal has decreased by 10 dB.

For example, if there are six object signals and five channels (i.e., front left (FL), front right (FR), center (C), rear left (RL) and rear right (RR) channels), a 6 × rendering matrix 5, having six rows appropriately corresponding to six object signals, and five columns appropriately corresponding to five channels can be set. The coefficients of a 6 × 5 rendering matrix can be integers indicating the ratio with which each of the six object signals is distributed across five channels. The 6 × 5 rendering matrix may have a reference value of 10. Thus, if the sum of the five coefficients in any one of the six rows of the 6 × 5 rendering matrix is 10, it can be determined that the level of the corresponding signal varied. The value by which the sum of the five coefficients in any one of the six rows of the 6 × 5 rendering matrix differs from the reference value represents the amount by which the level of the corresponding object signal was varied. For example, if the sum of the five coefficients in any one of the six rows of the 6x5 rendering matrix differs from the reference value by 1, it can be determined that the level of the corresponding object signal varied by 2 dB. A 6x5 rendering matrix can be represented using equation (1):

Equation 1

Referring to the 6 × 5 rendering matrix according to equation (1), the first row corresponds to the first object signal and represents the ratio with which the first object signal is distributed between the FL, FR, C, RL and RR channels. Since the first coefficient of the first line has the largest integer value of 3, and the sum of the coefficients of the first line is 10, it is determined that the first object signal is mainly distributed in the FL channel and that the level of the first object signal did not vary. Since the second coefficient of the second line, which corresponds to the second object signal, has the largest integer value of 4 and the sum of the coefficients of the second line is 12, it is determined that the second object signal is mainly distributed in the FR channel and that the level of the second object signal increased by 4 dB. Since the third coefficient of the third line, which corresponds to the third object signal, has the largest integer value of 12 and the sum of the coefficients of the third line is 12, it is determined that the third object signal is distributed only in the C channel and that the level of the third object signal increased by 4 dB. Since all the coefficients of the fifth row, which corresponds to the fifth object signal, have the same integer value of 2 and the sum of the coefficients of the fifth row is 10, it is determined that the fifth object signal is equally distributed between the FL, FR, C, RL and RR channels and that the level of the fifth the object signal did not vary.

Alternatively, when N object signals are mapped onto M channels, an N × (M + 1) rendering matrix can be set. The N × (M + 1) rendering matrix is very similar to the N × M rendering matrix. More specifically, in the N × (M + 1) rendering matrix, as in the N × M rendering matrix, the first through Mth coefficients in each of the N lines represent the ratio with which the corresponding object signal is distributed between FL, FR, C, RL and RR channels. However, the N × (M + 1) rendering matrix, unlike the N × M rendering matrix, has an additional column (i.e., the (M + 1) th column) to represent the levels of object signals.

The N × (M + 1) rendering matrix, unlike the N × M rendering matrix, indicates how the object signal is distributed between the M channels and whether the level of the object signal was varied individually. Thus, by using the N × (M + 1) rendering matrix, it is easy to obtain information regarding the variation, if any, of the level of the object signal without the need for additional calculations. Since the N × (M + 1) rendering matrix is almost the same as the N × M rendering matrix, the N × (M + 1) rendering matrix can be easily converted to the N × M rendering matrix, and vice versa, without the need for additional information .

Still alternatively, when N object signals are mapped onto M channels, an N × 2 rendering matrix can be set. The N × 2 rendering matrix has a first column indicating the angular positions of the object signals and a second column indicating the variation, if any, of the level of each of the object signals. An N × 2 rendering matrix can represent the angular positions of object signals at regular intervals of 1 or 3 degrees within a range of 0-360 degrees. An object signal that is evenly distributed between all directions can be represented by a predetermined value, rather than an angle.

An N × 2 rendering matrix can be converted to an N × 3 rendering matrix, which can indicate not only 2D directions of object signals, but also 3D directions of object signals. More specifically, the second column of the N × 3 rendering matrix can be used to indicate 3D directions of object signals. The third column of the N × 3 rendering matrix indicates the variation, if any, of the level of each object signal using the same method as used by the N × M rendering matrix. If the final playback mode of the object decoder is stereo stereo, the rendering matrix generator 201 may transmit 3D information indicating the position of each object signal, or an index corresponding to 3D information. In the second case, the transcoder 203 may need to have 3D information corresponding to the index transmitted by the rendering matrix generator 201. In addition, if 3D information indicating the position of each object signal is received from the rendering matrix generator 201, the transcoder 203 may be able to calculate 3D information that can be used by the multi-channel encoder 205 based on the received 3D information, the rendering matrix, and the object-based rendering additional information.

The rendering matrix and 3D information can be adaptively varied in real time according to a modification made to the position information of the objects and the playback configuration information by the end user. Therefore, information regarding whether the rendering matrix and 3D information is updated, and updates, if any, in the rendering matrix and 3D information can be transmitted to transcoder 203 at regular time intervals, for example, at 0.5 s intervals. Further, if updates are found in the rendering matrix and 3D information, transcoder 203 can linearly convert the received updates and the existing rendering matrix and existing 3D information, provided that the rendering matrix and 3D information vary linearly in time.

If the object position information and the playback configuration information has not been modified by the end user since the rendering matrix and 3D information were transmitted to the transcoder 203, information indicating that the rendering matrix and 3D information did not vary can be transmitted to the transcoder 203. On the other hand, if information about the positions of objects and information about the configuration of the playback was modified by the end user from the moment the transfer of the rendering matrix and 3D information to the transcoder 203, the information indicated which means that the rendering matrix and 3D information varied, and updates in the rendering matrix and 3D information can be transmitted to transcoder 203. More specifically, updates in the rendering matrix and 3D information updates can be separately transmitted to transcoder 203. Alternatively, updates in the rendering matrix and / or updates to 3D information may be jointly represented by a predetermined representative value. Further, a predefined representative value may be transmitted to transcoder 203 along with information indicating that the predefined representative value corresponds to updates in the rendering matrix or updates to 3D information. Thus, it is possible to easily notify the transcoder 203 about whether or not the rendering matrix and 3D information were updated.

The N × M rendering matrix, similar to that indicated by equation (1), may also include an additional column for representing 3D information of the direction of the object signals. In this case, an additional column can represent 3D information of the direction of the object signals as angles in the range from -90 to +90 degrees. An additional column can be provided not only in the N × M matrix, but also in the N × (M + 1) rendering matrix and in the N × 2 matrix. The 3D direction information of the object signals may not be necessary for use in the normal decoding mode of a multi-channel decoder. Instead, 3D direction information of object signals may be required for use in stereo mode of a multi-channel decoder. 3D information of the direction of the object signals can be transmitted along with the rendering matrix. Alternatively, 3D information of the direction of the object signals may be transmitted along with 3D information. The 3D direction information of the object signals does not affect the additional information based on the channels, but affects the 3D information in the decoding operation in stereo.

Information regarding spatial positions and levels of object signals may be provided as a rendering matrix. Alternatively, information regarding spatial positions and levels of object signals may be represented as modifications to the spectrum of the object signal, such as intensification of low-tone parts or high-tone parts of object signals. In this case, information regarding modifications to the spectrum of object signals can be transmitted as level variations in each parametric band that is used in a multi-channel codec. If the end user controls modifications to the spectrum of object signals, information regarding modifications to the spectrum of object signals can be transmitted as a spectrum matrix separately from the rendering matrix. The spectrum matrix can have as many rows as there are object signals, and as many columns as parameters. Each coefficient in the spectrum matrix indicates information regarding the level control of each parametric band.

The operation of transcoder 203 is described in detail below. Transcoder 203 generates channel-based additional information for multi-channel decoder 205 based on object-based additional information, rendering matrix information and 3D information, and transmits channel-based additional information to multi-channel decoder 205. In addition, transcoder 203 generates 3D information for multi-channel decoder 205 and transmits 3D information to multi-channel decoder 205. If the input down-mix signal to Should be pre-processed before entering the multi-channel decoder 205, the transcoder 203 may transmit information regarding the downmix input.

Transcoder 203 may receive object-based additional information indicating how many object signals are included in the downmix input. The object-based additional information may indicate how a plurality of object signals are included in the downmix input by using the OTT module and the TTT module and by using the CLD, ICC and CPC information. The object-based additional information may provide descriptions of various methods that may be performed by the object encoder to indicate information regarding each of the plurality of object signals, and thereby may be able to indicate how the object signals are included in the additional information.

In the case of the TTT module of a multi-channel encoder, the L-, C- and R-signals can be downmixed or up-mixed into the L- and R-signals. In this case, the C signal may share the low-order bit of the L and R signals. However, this rarely happens in the case of downmixing or upmixing of object signals. Therefore, the OTT module is widely used to perform upmixing or downmixing for object coding. Even if the C signal includes an independent signal component, and not parts of the L and R signals, the TTT module can be used to perform upmixing or downmixing for object coding.

For example, if there are six object signals, six object signals can be converted into a downmix signal using the OTT module, and information regarding each of the object signals can be obtained using the OTT module, as illustrated in FIG. 11.

Referring to FIG. 11, six object signals can be represented by a single downmix signal and information (such as CLD and ICC information) provided by a total of five OTT modules 211, 213, 215, 217 and 219. The structure illustrated in FIG. .11 may be modified in various ways. Those. referring to FIG. 11, the first OTT module 211 may receive two of six object signals. In addition, the manner in which the OTT modules 211, 213, 215, 217 and 219 are hierarchically connected can be freely varied. Therefore, additional information may include hierarchical structure information indicating how the OTT modules 211, 213, 215, 217 and 219 are hierarchically connected, and input position information indicating which OTT module each object signal is input into. If the OTT modules 211, 213, 215, 217 and 219 form an arbitrary tree structure, the method used in the multi-channel codec to represent an arbitrary tree structure can be used to indicate this hierarchical structure information. In addition, such input position information may be indicated in various ways.

Additional information may also include information regarding the muting period of each object signal. In this case, the tree structure of the OTT modules 211, 213, 215, 217, and 219 can adaptively vary in time. For example, referring to FIG. 11, when the first object signal OBJECT1 is muted, information regarding the first OTT module 211 is redundant, and only the second object signal OBJECT2 can be input to the fourth OTT module 217. Then, the tree structure of the OTT modules 211 , 213, 215, 217 and 219 may vary accordingly. Thus, information regarding variation, if any, in the tree structure of OTT modules 211, 213, 215, 217 and 219 can be included in additional information.

If the predefined object signal is muted, information indicating that the OTT module corresponding to the predefined object signal is not used, and information indicating that there are no labels available from the OTT module can be provided. Thus, it is possible to reduce the size of additional information by not including information regarding OTT modules or TTT modules that are not used in additional information. Even if the tree structure of the plurality of OTT or TTT modules is modified, it is easy to determine which of the OTT or TTT modules are on or off based on information indicating that the object signals are muted. Therefore, it is not necessary to often transmit information regarding modifications, if any, in the tree structure of OTT or TTT modules. Instead, information is transmitted indicating that the object signal is muted. The decoder can then easily determine which part of the tree structure of the OTT or TTT modules should be modified. Therefore, it is possible to minimize the size of information to be transmitted to the decoder. In addition, labels relating to object signals can be easily transmitted to a decoder.

12 is a diagram for explaining how a plurality of object signals are included in a downmix signal. In the embodiment of FIG. 11, the structure of the multi-channel coding OTT module is adapted as is. However, in the embodiment of FIG. 12, an embodiment of the structure of an OTT multi-channel coding unit is used. That is, referring to FIG. 12, a plurality of object signals are input to each module, and only one downmix signal is generated at the end. Referring to FIG. 12, information regarding each of a plurality of object signals can be represented by the ratio of the energy level of each of the object signals to the total energy level of the object signals. However, as the number of object signals increases, the ratio of the energy level of each of the object signals to the total energy level of the object signals decreases. To resolve this, one of the many object signals (hereinafter referred to as the object signal with the highest energy), having the highest energy level in a predefined parametric band, and the ratio of the energy levels of other object signals (hereinafter referred to as object signals with the lowest energy) to the energy level of the object signal with the highest energy can be provided as information regarding each of the object signals. In this case, after the information indicating the object signal with the highest energy and the absolute value of the energy level of the object signal with the highest energy are set, the energy levels of other object signals with not the highest energy can be easily determined.

The energy level of the highest energy object signal is required to combine multiple bit streams into a single bit stream, performed at a multilateral communications control unit (MCU). However, in most cases the energy level of the object signal with the highest energy is not necessary, since the absolute value of the energy level of the object signal with the highest energy can be easily determined from the ratios of the energy levels of other object signals with the highest energy to the energy level of the object signal with the highest energy.

For example, suppose that there are four object signals A, B, C and D that belong to a predefined parametric strip, and that object signal A is the object signal with the highest energy. Further, the energy E of a predetermined parametric band and the absolute value E of the energy level of the object signal A satisfy equation (2):

Equation 2

In this case, a, b and c respectively indicate the ratio of the energy level of the object signals B, C and D to the energy level of the object signal. Referring to equation (2), it is possible to calculate the absolute value E _{A of} the energy level of the object signal A based on the ratios a, b and c and the energy E _{P of} a predetermined parametric band. Therefore, unless it is necessary to combine multiple bit streams into one bit stream using the MCU, the absolute value E _{A of} the energy level of the object signal A may not need to be included in the bit stream. Information indicating whether the absolute value E _{A of} the energy level of the object signal A is included in the bit stream can be included in the header of the bit stream, thereby reducing the size of the bit stream.

On the other hand, if there is a need to combine multiple bit streams into one bit stream using the MCU, the energy level of the object signal with the highest energy is mandatory. In this case, the sum of the energy levels calculated on the basis of the ratio of the energy levels of the object signals with the highest energy to the energy level of the object signal with the highest energy may not be the same as the energy level of the downmix signal obtained by downmixing all the object signals. For example, when the energy level of the down-mix signal is 100, the sum of the calculated energy levels may be 98 or 103 due to, for example, errors that occur during quantization and dequantization operations. To resolve this, the difference between the energy level of the downmix signal and the sum of the calculated energy levels can be properly compensated by multiplying each of the calculated energy levels by a predetermined coefficient. If the energy level of the down-mix signal is X, and the sum of the calculated energy levels is Y, each of the calculated energy levels can be multiplied by X / Y. If the difference between the energy level of the down-mix signal and the sum of the calculated energy levels are not compensated, these quantization errors can be included in the parametric bands and frames, thereby causing signal distortion.

Therefore, information indicating which of the plurality of object signals has the largest absolute energy value in a predetermined parametric band is mandatory. This information may be represented by the number of bits. The number of bits required to indicate which of the plurality of object signals has the largest absolute energy value in a predetermined parametric band varies according to the number of object signals. As the number of object signals increases, the number of bits required to indicate which of the plurality of object signals has the largest absolute energy value in a predetermined parametric band increases. On the other hand, as the number of object signals decreases, the number of bits required to indicate which of the plurality of object signals has the largest absolute energy value in a predetermined parametric band decreases. A predetermined number of bits can be assigned in advance to indicate which of the plurality of object signals has the largest absolute energy value in a predetermined parametric band. Alternatively, the number of bits to indicate which of the plurality of object signals has the largest absolute energy value in a predetermined parametric band can be determined based on certain information.

The size of the information indicating which of the plurality of object signals has the largest absolute energy value in each parametric band can be reduced by using the same method as used to reduce the size of the CLD, ICC and CPC information for use in OTT- and / or TTT modules of a multi-channel codec, for example, by using a time differential method, a frequency differential method, or a control coding method.

To indicate which of the plurality of object signals has the largest absolute energy value in each parametric band, an optimized Huffman table can be used. In this case, information indicating in which order the energy levels of the object signals are compared with the energy level of any object signal having the highest absolute energy may be required. For example, if there are five object signals (i.e., first to fifth object signals) and the third object signal is the highest energy object signal, information regarding the third object signal may be provided. Further, the ratio of the energy levels of the first, second, fourth and fifth object signals to the energy level of the third object signal can be provided in various ways, and this is described in more detail below.

The ratios of the energy levels of the first, second, fourth and fifth object signals to the energy level of the third object signal can be provided sequentially. Alternatively, the ratios of the energy levels of the fourth, fifth, first and second object signals to the energy level of the third object signal can be provided sequentially in a circular manner. Further, information indicating the order in which the ratio of the energy levels of the first, second, fourth and fifth object signals to the energy level of the third object signal is provided can be included in the file header or can be transmitted at intervals of the number of frames. A multi-channel codec can determine CLD and ICC information based on the sequence numbers of OTT modules. Likewise, information is needed indicating how each object maps to the bitstream.

In the case of a multi-channel codec, information regarding the signals corresponding to each channel can be identified by the sequence numbers of the OTT or TTT modules. According to an encoding method for object-based audio, if there are N object signals, these N object signals may need to be numbered accordingly. However, sometimes the user needs to control N object signals using an object decoder. In this case, the user may need not only serial numbers of N object signals, but also descriptions of N object signals, such as descriptions indicating that the first object signal corresponds to the woman’s voice and that the second object signal corresponds to the sound of the piano. Descriptions of N object signals may be included in the bitstream as metadata and then transmitted along with the bitstream. More specifically, descriptions of N object signals may be provided as text or may be provided using a code table or code words.

Sometimes correlation information is needed regarding correlations between object signals. For this, correlations between the object signal with the highest energy and other object signals with not the highest energy can be calculated. In this case, one correlation value can be indicated for all object signals, which is comparable to using one ICC value in all OTT modules.

If the object signals are stereo signals, information is needed on the ratios of the energy of the left channel to the energy of the right channel for object signals and ICC. The ratios of the energy of the left channel to the energy of the right channel for object signals can be calculated using the same method as used to calculate the energy levels of a plurality of object signals based on the absolute value of the energy level of that of the object signals, which is the object signal with the highest energy, and the ratio of the energy levels of other object signals not with the highest energy to the energy level of the object signal with the highest energy. For example, if the absolute values of the energy levels of the left and right channels of the object signal with the highest energy are A and B, respectively, and the ratio of the energy level of the left channel of the object signal is not with the highest energy to A and the ratio of the energy level of the right channel of the object signal with not the highest energy to B is x and y, respectively, the energy levels of the left and right channels of the object signal with not the highest energy can be calculated as A * x and B * y. Thus, the ratio of the energy of the left channel to the energy of the right channel of the object stereo signal can be calculated.

The absolute value of the energy level of an object signal with the highest energy and the ratio of the energy levels of other object signals not of the highest energy to the energy level of the object signal with the highest energy can also be used when the object signals are mono signals, the down-mix signal obtained by object mono signals is a stereo signal , and object mono signals are included in both channels of the stereo down-mix signal. In this case, the ratio of the energy of the part of each object mono signal included in the left channel of the stereo down-mix signal and the energy of part of each object mono signal included in the right channel of the stereo down-mix signal and correlation information are necessary, and this is directly applied to object stereo signals. If the object mono signal is included in the L- and R-channels of the stereo down-mix signal, the components of the L- and R-channels of the object mono signal can have only the channel difference, and the object mono signal can have a correlation value of 1 over all parametric bands. In this case, in order to reduce the amount of data, information indicating that the object mono signal has a correlation value of 1 over all parametric bands can be further provided. Further, there is no need to indicate a correlation value of 1 for each of the parametric bands. Instead, a correlation value of 1 can be specified for all parametric bands.

During the formation of the downmix signal by summing a plurality of object signals, clipping may occur. To resolve this, the downmix signal can be multiplied by a predetermined gain so that the maximum level of the downmix signal can exceed the cutoff threshold. The preset gain may vary over time. Therefore, information regarding a predefined gain is needed. If the downmix signal is a stereo signal, different gain values can be provided for the L- and R-channels of the downmix signal to prevent clipping. To reduce the amount of data transmission, various gain values may not be transmitted separately. Instead, a sum of different gain values and a ratio of different gain values can be transmitted. Further, it is possible to reduce the dynamic range and reduce the amount of data transfer in comparison with the case of transmitting various gain values separately.

To further reduce the amount of data transmission, a bit may be provided indicating whether clipping occurred during separation of the downmix signal during the summation of the plurality of object signals. Then, only if it is determined that clipping has occurred, can gain values be transmitted. This clipping information may be necessary to prevent clipping during the summation of a plurality of downmix signals in order to combine a plurality of bit streams. To prevent clipping, the sum of the plurality of downmix signals can be multiplied by the reciprocal of the predefined gain value to prevent clipping.

13-16 illustrate diagrams for explaining various methods of configuring object-based additional information. The embodiments of FIGS. 13-16 can be applied not only to object mono and stereo signals, but also to multi-channel object signals.

Referring to FIG. 13, a multi-channel object signal (from OBJECT A (CH1) to OBJECT A (CHn)) is input to the object encoder 221. Then, the object encoder 221 generates a downmix signal and additional information based on the multi-channel object signal (from OBJECT A ( CH1) to OBJECT A (CHn)). The object encoder 223 receives a plurality of object signals from OBJECT1 to OBJECTn and a downmix signal generated by the object encoder 221, and generates another downmix signal and additional information based on the object signals from OBJ.1 to OBJ.N and the received downmix signal mixing. Multiplexer 225 combines the additional information generated by the object encoder 221 and the additional information generated by the object encoder 223.

Referring to FIG. 14, an object encoder 233 generates a first bit stream based on a multi-channel object signal (from OBJECT A (CH1) to OBJECT A (CHn)). Next, the object encoder 231 generates a second bit stream based on the set of multi-channel object signals from OBJECT1 to OBJECTn. Then, the object encoder 235 combines the first and second bitstreams into a single bitstream by using practically the same method as used to combine multiple bit streams into one bit stream using the MCU.

Referring to FIG. 15, multi-channel encoder 241 generates a down-mix signal and channel-based additional information based on a multi-channel object signal (from OBJECT A (CH1) to OBJECT A (CHn)). The object encoder 243 receives the down-mix signal generated by the multi-channel encoder 241 and a plurality of multi-channel object signals from OBJECT1 to OBJECTn and generates the object bit stream and additional information based on the received down-mix signal and the object signals from OBJECT1 to OBJECTn. A multiplexer 245 combines the channel-based additional information generated by the multi-channel encoder 241 and the additional information generated by the object encoder 243 and outputs the combination result.

Referring to FIG. 16, multi-channel encoder 253 generates a down-mix signal and channel-based additional information based on a multi-channel object signal (from OBJECT A (CH1) to OBJECT A (CHn)). The object encoder 251 generates a down-mix signal and additional information based on a plurality of multi-channel object signals from OBJECT1 to OBJECTn. The object encoder 255 receives the down-mix signal generated by the multi-channel encoder 253 and the down-mix signal generated by the object encoder 251 and combines the received down-mix signals. A multiplexer 257 combines the additional information generated by the object encoder 251 and the channel-based additional information generated by the multi-channel encoder 243 and outputs the result of the combination.

In the case of encoding object-based audio in newsgroups, it is sometimes necessary to combine multiple object bitstreams into a single bitstream. The following describes in detail the combination of multiple object bitstreams into a single bitstream.

17 illustrates a diagram for explaining combining two object bitstreams. Referring to FIG. 17, when two object bitstreams are combined into one object bitstream, additional information, such as CLD and ICC information presented in the two object bitstreams, may be modified, respectively. Two object bitstreams can be combined into one object bitstream simply by using an additional OTT module, i.e. of the eleventh OTT module, and the use of additional information such as CLD and ICC information provided by the eleventh OTT module.

The tree configuration information of each of the two object bitstreams must be combined into an integrated tree configuration information to combine the two object bitstreams into one object bitstream. For this, additional configuration information, if available, generated by combining two object bitstreams can be modified, the indices of the number of OTT modules used to form two object bitstreams can be modified, and only a few additional processes, such as a calculation process performed by the eleventh OTT module and down-mixing of two down-mixing signals in two object bit streams can be performed. Thus, two object bitstreams can easily be combined into one object bitstream without the need to modify information regarding each of the plurality of object signals from which two object signals are emitted.

Referring to FIG. 17, an eleventh OTT module may be optional. In this case, two down-mix signals of two object bitstreams can be used, as is, as a two-channel down-mix signal. Thus, two object bitstreams can be combined into a single bitstream without the need for additional computations.

FIG. 18 illustrates a diagram for explaining combining two or more independent object bitstreams into a single object bitstream having a stereo downmix signal. Referring to FIG. 18, if two or more independent bitstreams have a different number of parametric bands, the conversion of parametric bands can be performed for object bitstreams, so that the number of parametric bands of one of the object bitstreams having fewer parametric bands can be increased so that be the same as the number of parametric bands of another object bitstream.

More specifically, parametric band conversion can be performed using a predetermined conversion table. In this case, the transformation of the parametric strip can be performed using a simple linear formula.

If there are overlapping parametric bands, the parameter values may be appropriately mixed taking into account the amount by which the overlapping parametric bands overlap. In situations where low complexity is a priority, the conversion of parametric bands can be performed for two object bitstreams, so that the number of parametric bands of one of two object bitstreams having more parametric bands can be reduced to be the same as the number parametric bands of another object bitstream.

In the embodiments of FIGS. 17 and 18, two or more independent object bitstreams can be combined into an integrated object bitstream without having to calculate existing parameters of independent object bitstreams. However, in the case of combining a plurality of downmix signals, parameters regarding downmix signals may need to be calculated again by QMF / hybrid analysis.

However, this calculation requires a significant amount of computation, thereby jeopardizing the advantages of the embodiments of FIGS. 17 and 18. Therefore, it is necessary to propose methods for extracting parameters without the need for QMF / hybrid analysis or synthesis, even when downmix signals are downmixed. To this end, energy information regarding the energies of each parametric band of each downmix signal can be included in the object bit stream. Then, when the downmix signals are downmixed, information such as CLD information can be easily calculated based on such energy information without the need for QMF / hybrid analysis or synthesis. This energy information may represent the highest energy level for each parametric band or the absolute value of the energy level of the object signal with the highest energy for each parametric band. The amount of computation can be further reduced by using ICC values obtained from the time domain for each parametric band.

During downmixing of a plurality of downmix signals, clipping may occur. To resolve this, the downmix signal levels can be reduced. If the levels of the downmix signals are reduced, level information regarding the reduced levels of the downmix signals may need to be included in the object bitstream. Level information to prevent clipping can be applied to each frame of the object bitstream, or can only be applied to some frames in which clipping occurs. The levels of the original down-mix signals can be calculated by reverse applying the level information to prevent clipping during the decoding operation. Level information to prevent clipping can be computed in the time domain and thus should not be subjected to QMF / hybrid synthesis or analysis. Combining a plurality of object signals into a single object bit stream may be performed using the structure illustrated in FIG. 12, and this will be described in more detail below with reference to FIG.

19 is a diagram for explaining combining two independent object bitstreams into a single object bitstream. Referring to FIG. 19, the first module 261 generates a first object bit stream, and the second module 263 generates a second object bit stream. Further, the third module 265 generates a third bit stream by combining the first and second bit stream. In this case, if the first and second object bitstreams include information on the absolute value of the energy level of the object signal with the highest energy for each parametric strip and the ratios of the energy levels of other object signals not with the highest energy level to the energy level of the object signal with the highest energy level and gain information regarding gain values that are multiplied by downmix signals by the first and second modules 261 and 263, the third module 265 may generate rd object bitstream simply by combining the first and second bitstreams without further calculation or extraction parameters.

The third module 265 receives a plurality of downmix signals DOWNMIX_A and DOWNMIX_B. The third module 265 converts the downmix signals DOWNMIX_A and DOWNMIX_B into PCM signals and sums the PCM signals, thereby generating one downmix signal. However, clipping may occur during this process. To enable this, the downmix signals DOWNMIX_A and DOWNMIX_B can be multiplied by a predefined gain value. Information regarding a predetermined gain value may be included in the third object bit stream and transmitted along with the third object bit stream.

The following describes in detail the combination of multiple object bitstreams into a single bitstream. Referring to FIG. 19, paramA may include information regarding which of a plurality of object signals from OBJECT1 to OBJECTn is an object signal with the highest energy, and ratios of energy levels of other object signals not with the highest energy to the energy level of the object signal with the highest energy. Similarly, SIDE_INFO may include SIDE_INFO_A, which may include information regarding which of a plurality of object signals from OBJECT1 to OBJECTn is an object signal with the highest energy, and ratios of energy levels of other object signals not with the highest energy to the energy level of the object signal with the highest energy.

SIDE_INFO_A and SIDE_INFO_B may be included in parallel in one bit stream, as illustrated in FIG. In this case, a bit may further be provided indicating that there is more than one bit stream in parallel.

Referring to FIG. 20, to indicate whether a predetermined bit stream is an integrated bit stream including more than one bit stream, information indicating whether a predetermined bit stream is an integrated bit stream, information regarding the number of bit streams, if any, included in a predetermined bit stream, and information regarding the initial positions of the bit streams, if any, included in a predetermined bit stream, may be provided in the header a predefined bitstream followed by more than one bitstream, if any, into a predefined bitstream. In this case, the decoder can determine whether the predetermined bit stream is an integrated bit stream including more than one bit stream by analyzing the information in the header of the predefined bit stream. This type of combining bit streams does not require additional processes, in addition to adding multiple identifiers to the bit stream. However, such identifiers should be provided at intervals of a number of frames. In addition, this type of method for combining the bitstream requires the decoder to determine whether each bitstream that the decoder receives is an integrated bitstream.

As an alternative to the above method of combining bit streams, a plurality of bit streams could be combined into a single bit stream so that the decoder cannot recognize whether or not one bit stream is an integrated bit stream. This is described in detail below with reference to FIG.

Referring to FIG. 21, the energy level of the highest energy object signal represented by SIDE_INFO_A and the energy level of the highest energy object signal represented by SIDE_INFO_B are compared. Then, one of the two object signals that has the highest energy level is defined as the object signal with the highest energy of the integrated bit stream. For example, if the energy level of the highest energy object signal represented by SIDE_INFO_A is higher than the energy level of the highest energy object represented by SIDE_INFO_B, the highest energy object signal represented by SIDE_INFO_A may become the highest energy signal of the integrated bit stream. Then, the energy ratio information SIDE_INFO_A can be used in the integrated bitstream as is, while the energy ratio information SIDE_INFO_B can be multiplied by the ratio of energy levels.

Further, the energy ratio information of that of SIDE_INFO_A and SIDE_INFO_B, which includes information relating to the object signal with the highest energy of the integrated bit stream, can be used as the integrated bit stream, and information of the energy ratio of the object signal with the highest energy represented by Param A , and the highest energy object signal represented by SIDE_INFO_B. This method entails the conversion of the energy ratio information SIDE_INFO_B. However, recalculation of the energy ratio information SIDE_INFO_B is relatively simple. In this method, the decoder may not be able to determine whether the bit stream it receives is an integrated bit stream including more than one bit stream, and thus a typical decoding method can be used.

Two object bit streams including stereo downmix signals can easily be combined into one object bit stream without having to recalculate the information regarding the object signals using practically the same method as used to combine bit streams including downmix mono signals. In the object bitstream, information regarding the tree structure that reduces the object signal follows the information of the object signals obtained from each branch (i.e., each module) of the tree structure.

The object bitstreams are described above in view of the fact that a certain object is allocated not only to the left channel or the right channel of the stereo down-mix signal. However, object signals are generally distributed between both channels of the stereo downmix signal. Therefore, the following describes in detail how to generate an object bit stream based on the bit streams that are distributed between the two channels of the stereo down-mix signal.

FIG. 22 illustrates a diagram for explaining a method of generating a stereo down-mix signal by mixing a plurality of object signals, and more specifically, a method of down-mixing four object signals from OBJECT1 to OBJECT4 into L- and R-stereo signals. Referring to FIG. 22, some of the four object signals from OBJECT1 to OBJECT4 belong to both the L- and R-channels of the downmix signal. For example, the first object signal OBJECT1 is distributed between the L- and R-channels with the ratio a: b, as indicated by equation (3):

Equation 3

If the object signal is distributed between the L- and R-channels of the stereo down-mix signal, channel distribution ratio information regarding the ratio (a: b) with which the object signal is distributed between the L- and R-channels may be additionally required. Further, information regarding the object signal, such as CLD and ICC information, can be calculated by performing downmixing using OTT modules for the L- and R-channels of the downmix stereo signal, and this will be described in more detail below with reference to FIG.

Referring to FIG. 23, after the CLD and ICC information obtained from the plurality of OTT modules during the downmix operation and the distribution ratio information between the channels of each of the plurality of object signals are provided, a multi-channel bit stream that adaptively varies according to any a modification made to the position information of the objects and the playback configuration information by the end user. In addition, if the stereo downmix signal needs to be processed by preprocessing the downmix, you can get information regarding how the stereo downmix signal is processed by the preprocessing downmix and transmit the information to the preprocessor. Those. if the information of the distribution ratio over the channels of each of the plurality of object signals is not provided, there is no way to calculate the multi-channel bitstream and obtain the information required for the preprocessor to work. Information of the distribution ratio between the channels of the object signal can be represented as the ratio of two integers or a scalar value (in dB).

As described above, if the object signal is distributed between two channels of the stereo down-mix signal, distribution ratio information between the channels of the object signal may be required. The distribution ratio information between the channels may have a fixed value indicating the ratio with which the object signal is distributed between the two channels of the stereo downmix signal. Alternatively, the distribution ratio information between the channels of the object signal may vary from one frequency band to another frequency band of the object signal, especially when the distribution ratio information between the channels is used as ICC information. If the stereo down-mix signal is obtained by a complicated down-mix operation, i.e. if the object signal belongs to two channels of the downmix stereo signal and is downmixed by varying the ICC information from one frequency band to another frequency band of the object signal, a detailed description of the downmix of the object signal may be additionally required to decode the object signal finally prepared by rendering. This embodiment can be applied to all possible object structures that have already been described.

Next, preprocessing is described in detail with reference to FIGS. 24-27. If the downmix signal input to the object decoder is a stereo signal, the downmix input may need to be pre-processed before being input to the multichannel decoder of the object decoder because the multichannel decoder cannot convert the signal belonging to the left channel of the downmix input signal to the right channel . Therefore, in order for the end user to shift the position of the object signal belonging to the left channel of the downmix input signal to the right channel, the downmix input signal may need to be pre-processed, and the pre-processed down-mix signal can be input into a multi-channel decoder.

The preprocessing of the downmix stereo signal can be performed by obtaining the preprocessing information from the object bitstream and from the rendering matrix and the proper processing of the downmix stereo signal according to the processing information, and this is described in further detail below.

24 illustrates a diagram for explaining how to configure a stereo down-mix signal based on four object signals from OBJECT1 to OBJECT4. Referring to Fig. 24, the first object signal OBJECT1 is distributed between L- and R-channels with the ratio a: b, the second object signal OBJECT2 is distributed between L- and R-channels with the ratio c: d, the third object signal OBJECT3 is distributed only L- channel, and the fourth object signal OBJECT4 is allocated only to the R-channel. Information, such as CLD and ICC, can be generated by transmitting each of the first to fourth object signals from OBJECT1 to OBJECT4 through an OTT series, and a downmix signal can be generated based on the generated information.

Suppose that the end user receives a rendering matrix by appropriately setting positions and levels one through four of the object signals from OBJECT1 to OBJECT4 and that there are five channels. The rendering matrix can be represented by equation (4):

Equation 4

Referring to equation (4), when the sum of five coefficients in each of the four rows is equal to a predetermined reference value, i.e. 100, it is determined that the level of the corresponding object signal has not been varied. The value by which the sum of the five coefficients in each of the four lines differs from the predefined reference value may be the amount (in dB) by which the level of the corresponding object signal was varied. The first, second, third, fourth and fifth columns in the rendering matrix of equation (4) represent the FL, FR, C, RL and RR channels, respectively.

The first row of the rendering matrix according to equation (4) corresponds to the first object signal OBJECT1 and has only five coefficients, i.e. 30, 10, 20, 30, and 10. Since the sum of the five coefficients of the first row is 100, it is determined that the level of the first object signal OBJECT1 has not varied, and that only the spatial position of the first object signal OBJECT1 has changed. Even if the five coefficients of the first row represent different channel directions, they can be broadly classified into two channels: L and R channels. In this case, the ratio with which the first object signal OBJECT1 is distributed between the L- and R-channels can be calculated as 70% (= (30 + 30 + 20) * 0.5): 30% (= (10 + 10 + 20) * 0.5). Therefore, the rendering matrix according to equation (4) indicates that the level of the first object signal OBJECT1 did not vary, and that the first object signal OBJECT1 is distributed between L and R channels with a ratio of 70%: 30%. If the sum of the five coefficients of any of the rows of the rendering matrix according to equation (4) is less than or not more than 100, it can be determined that the level of the corresponding object signal has changed, and then the corresponding object signal can be processed by preprocessing or can be converted and transmitted as ADG .

In order to pre-process the downmix signals, the ratio with which the downmix signals are distributed between the parametric bands from which the parameters are extracted from the signals obtained by performing QMF / hybrid conversion of the downmix signals can be calculated, and the downmix signals can be redistributed between the parametric stripes according to the setting of the rendering matrix. Various methods for redistributing down-mix signals between parametric bands are described in detail below.

In the first redistribution method, the down-mix signals of the L- and R-channel are decoded separately using their respective additional information (such as CLD and ICC information) and using almost the same method as used by the multi-channel codec. Next, the object signals distributed between the down-mix signals of the L- and R-channels are restored. To reduce the amount of computation, the down-mix signals of the L- and R-channels can only be decoded using CLD information. The relationship with which each of the reconstructed object signals is distributed between the down-mix signals of the L- and R-channels can be determined based on additional information.

Each of the reconstructed object signals can be redistributed between the down-mix signals of the L- and R-channels according to the rendering matrix. Next, the redistributed object signals are mixed downward on a per-channel basis using OTT modules, thereby completing pre-processing. In short, the first redistribution method accommodates the same method used by the multi-channel codec. However, the first redistribution method requires as many decoding processes as there are object signals for each channel, and requires a redistribution process and a channel-based downmix process.

In the second redistribution method, unlike the first redistribution method, object signals are not reconstructed from the L- and R-downmix signals. Instead, each of the L- and R-downmix signals is divided into two parts: one part L_L or R_R, which should be left in the corresponding channel, and the other part L_R or R_L, which should be redistributed, as illustrated in FIG. . Referring to FIG. 25, L_L indicates the portion of the L-channel down-mix signal that should be left in the L-channel, and L_R indicates the portion of the L-channel down-mix signal that should be added to the R-channel. Similarly, R_R indicates the portion of the R-channel down-mix signal that should be left in the R-channel, and R_L indicates the portion of the R-channel down-mix signal that should be added to the L-channel. Each of the L- and R-channel down-mix signals can be divided into two parts (L_L and L_R or R_R and R_L) according to the ratio with which each object signal is distributed between the L- and R-down-mix signals, as defined by the equation ( 2), and the ratio with which each object signal must be distributed between the pre-processed L and R in the L- and R-channels, as defined by equation (3). Therefore, it can be determined that the downmix signals of the L and R channels should be redistributed between the pre-processed L and R in the L and R channels by comparing the ratio with which each object signal is distributed between the L and R signals of the downmix mixing, and the relationship with which each object signal must be distributed between the pre-processed L and R in the L- and R-channels.

The separation of the L-channel signal into L_L and L_R signals according to a predetermined energy ratio is described above. After the L-channel signal is divided into L_L and L_R signals, perhaps an ICC between the L_L and L_R signals should be determined. The ICC between the L_L and L_R signals can be easily determined based on the ICC information regarding the object signals. Those. The ICC between the L_L and L_R signals can be determined based on the relationship with which each object signal is distributed between the L_L and L_R signals.

The second method for redistributing the downmix is described in detail below. Assume that the L and R down-mix signals L and R are obtained by the method shown in FIG. 24, and that the first, second, third and fourth object signals OBJECT1, OBJECT2, OBJECT3 and OBJECT4 are distributed between the down-mix signals L and R mixing L- and R-channels with ratios of 1: 2, 2: 3, 1: 0 and 0: 1, respectively. A plurality of object signals can be downmixed by a series of OTT modules, and information such CLD and ICC information can be obtained from downmixing the object signals.

An example of a rendering matrix set for the first to fourth object signals from OBJECT1 to OBJECT4, such as represented by equation (4). The rendering matrix includes location information from first to fourth object signals from OBJECT1 to OBJECT4. Thus, the L and R down-mix preliminary signals L and R channels can be obtained by performing preprocessing using a rendering matrix. How to establish and interpret the rendering matrix has already been described above with reference to equation (3).

The ratio with which each of the first to fourth object signals from OBJECT1 to OBJECT4 is distributed between the pre-processed L and R down-mix signals L and R can be calculated as indicated by equation (5):

Equation 5

Object 1:

Object 2:

Object 3:

Object 4:

The ratio with which each of the first to fourth object signals from OBJECT1 to OBJECT4 is distributed between the L and R down-mix signals L and R channels can be calculated as indicated by equation (6):

Equation 6

Object 1:

Object 2:

Object 3:

Object 4:

Referring to equation (5), the sum of the portion of the third object signal OBJECT3 allocated to the pre-processed L-channel down-mix signal L and the portion of the third object signal OBJECT3 allocated to the R-channel down-mix signal R is 110, and thus it is determined that the level of the third object signal OBJECT3 has increased by 10. On the other hand, the sum of the part of the fourth object signal OBJECT4 allocated to the pre-processed signal L down-mix L-channel, and part the fourth object signal OBJECT4 allocated to the R-channel down-mix signal R is 85, and thus it is determined that the level of the fourth object signal OBJECT4 has decreased by 5. If the rendering matrix for the first to fourth object signals from OBJECT1 to OBJECT4 has a reference value of 100, and the amount by which the sum of the coefficients in each row of the rendering matrix differs from the reference value of 100, represents the value (units - dB) by which the level of the corresponding object signal was varied, It is determined that the level of the third object signal OBJECT3 increased by 10 dB, and that the level of the fourth object signal OBJECT4 has decreased by 5 dB.

Equations (5) and (6) can be reorganized into equation (7):

Equation 7

Object 1:

Object 2:

Object 3:

Object 4:

Equation (7) compares the ratio with which each of the first to fourth object signals from OBJECT1 to OBJECT4 is distributed between the down-mix signals of the L- and R-channel before preprocessing, and compares the ratio with which each of the first to fourth object signals from OBJECT1 to OBJECT4 is distributed between the down-mix signals of the L- and R-channel after pre-processing. Therefore, by using equation (7), it is easy to determine how much of each of the first to fourth object signals from OBJECT1 to OBJECT4 should be redistributed by preprocessing. For example, referring to (7), the ratio with which the second object signal OBJECT2 is distributed between the down-mix signals of the L- and R-channel changes from 40:60 to 30:70, and thus, one fourth can be determined (25%) of the part of the second object signal OBJECT2, previously distributed to the down-mix signal of the L-channel, should be shifted to the down-mix signal of the R-channel. This may become more apparent by reference to equation (8):

Equation 8

OBJECT1: 55% of the portion of OBJECT1 previously allocated to R should be shifted to L

OBJECT2: 25% of the portion of OBJECT1 previously allocated to L should be shifted to R

OBJECT3: 50% of the portion of OBJECT1 previously allocated to L should be shifted to R

OBJECT4: 50% of the portion of OBJECT1 previously allocated to R should be shifted to L.

By using equation (8), the signals L_L, L_R, R_L and R_R of FIG. 25 can be represented as indicated by equation (9):

Equation 9

The value of each object signal in equation (9) can be represented as the ratio with which the corresponding object signal is distributed between the L and R channels, using the dequantized CLD information provided by the OTT module, as indicated by equation (10):

Equation 10

The CLD information used in each analysis unit of FIG. 25 can be determined as indicated by equation (11):

Equation 11

where ε is a constant to avoid division by zero, for example, 96dB below the maximum signal input.

Thus, the CLD and ICC information used in the analysis unit to generate the L_L and L_R signals based on the L-channel down-mix signal can be determined, and the CLD and ICC information used in the analysis unit to generate the R_L and R_R signals based on the signal down-mix of the R-channel can also be determined. After the signals L_L, L_R, R_L and R_R are received, as illustrated in FIG. 25, the signals L_R and R_R can be summed, thereby obtaining a pre-processed stereo down-mix signal. If the final channel is a stereo channel, the down-mix signals of the L- and R-channels obtained by preprocessing can be output. In this case, the variation, if any, of the level of each object signal should be adjusted. For this, a predefined module that performs the functions of an ADG module can be additionally provided. Information for adjusting the level of each object signal can be calculated using the same method as used to calculate ADG information, and this is described in more detail below. Alternatively, the level of each object signal may be adjusted during the preprocessing operation. In this case, the level control of each object signal can be performed using the same method as that used to process the ADG. Alternative to the embodiment of FIG. 25, the decorrelation operation may be performed by a decorrelator and mixer rather than the PARSING 1 and PARSING 2 analysis modules, as illustrated in FIG. 26, to adjust the correlation between the L and R signals obtained by mixing. Referring to FIG. 26, Pre_L and Pre_R indicate L- and R-channel signals obtained by level control. One of the signals Pre_L and Pre_R can be input into the decorrelator and then subjected to a mixing operation performed by the mixer, thereby obtaining a signal with adjusted correlation.

The pre-processed stereo down-mix signal can be input into a multi-channel encoder. In order to provide multi-channel output compatible with object position information and playback configuration information specified by the end user, not only a pre-processed downmix signal, but also channel-based additional information for performing multi-channel decoding is required. The following describes in detail how to obtain channel-based additional information by re-examining the above example. The pre-processed down-mix signals L and R, which are input to the multi-channel decoder, can be set based on equation (5), as indicated by equation (12):

Equation 12

The ratio with which each of the first to fourth object signals from OBJECT1 to OBJECT4 is distributed on the FL-, RL-, C-, FR- and RR-channels can be determined as indicated by equation (13):

Equation 13

The preprocessed downmix signals L and R can be expanded to 5.1 channels by MPS, as illustrated in FIG. Referring to FIG. 27, the parameters of the TTT module TTT0 and the OTT modules OTTA, OTTB and OTTC may need to be calculated in units of parametric bands, even if the parametric bands are not illustrated for convenience.

TTT-module TTT0 can be used in two different modes: energy-oriented mode and prediction mode. When used in an energy-oriented mode, the TTT-module TTT0 requires two pieces of CLD information. When used in prediction mode, the TTT module TTT0 requires two pieces of CPC information and a piece of ICC information.

In order to calculate the CLD information in an energy-oriented mode, the energy ratio of the signals L ", R" and C of Fig. 27 can be calculated using equations (6), (10) and (13). The signal energy level L "can be calculated as indicated by equation (14):

Equation 14

Equation (14) may also need to be used to calculate the energy level R "or C. Further, the CLD information used in the TTT-module TTT0 can be calculated based on the energy levels of the signals L", R "and C, as indicated by equation (15):

Equation 15

Equation (14) can be established based on equation (10). Despite the fact that equation (10) defines only how to calculate the energy values for the L channel, the energy values for the R channel can be calculated using equation (10). Thus, the CLD and ICC values of the third and fourth OTT0 modules can be calculated based on the CLD and ICC values of the first and second OTT modules. However, this may not necessarily apply to all tree structures, but only to certain tree structures for decoding object signals. Information included in the object bitstream may be transmitted to each OTT module. Alternatively, information included in the object bitstream can only be transmitted to some OTT modules, and information indicating OTT modules that have not received the information can be obtained by calculation.

Parameters such as CLD and ICC information can be calculated for the OTT, OTTA, OTTB, and OTTC OTT modules by using the above method. These multi-channel parameters can be input into a multi-channel decoder and then subjected to multi-channel decoding, thereby obtaining a multi-channel signal that is properly prepared by rendering according to object position information and playback configuration information required by the end user.

Multichannel parameters may include an ADG parameter if the level of the object signals has not yet been adjusted by preprocessing. The following describes in detail the calculation of the ADG parameter by reviewing the above example again.

When the rendering matrix is set so that the level of the third object signal can be increased by 10 dB, that the level of the fourth object signal can be reduced by 5 dB, that the level of the component of the third object signal in L can be increased by 10 dB, and that the level of the component of the fourth the object signal in L can be reduced by 5 dB, the ratio of Ratio _{ADG, L} energy levels before and after adjusting the levels of the third and fourth object signals can be calculated using equation (16):

Equation 16

The ratio Ratio _{ADG, L} can be determined by substituting equation (10) into equation (16). The ratio Ratio _{ADG, R} for the R channel can also be calculated using equation (16). Each of the ratios of Ratio _{ADG, L} and Ratio _{ADG, R} represents the variation in energy of the corresponding parametric band due to the regulation of the levels of object signals. Thus, the values ADG ADG (L) and ADG (R) can be calculated using the ratios Ratio _{ADG, L} and Ratio _{ADG, R,} as indicated by equation (17):

Equation 17

After the ADG parameters ADG (L) and ADG (R) are determined, the ADG parameters ADG (L) and ADG (R) are quantized using the ADG quantization table, and quantized ADG values are transmitted. If there is a need to further fine-tune the ADG values of ADG (L) and ADG (R), the adjustment of the ADG values of ADG (L) and ADG (R) can be performed by a preprocessor, and not by an MPS decoder.

The number and spacing of parametric bands for representing object signals in an object bitstream may differ from the number and spacing of parametric bands used in a multi-channel decoder. In this case, the parametric bands of the object bitstream can be linearly converted to the parametric bands of the multi-channel decoder. More specifically, if a specific parametric band of an object bit stream covers two parametric bands of a multi-channel decoder, linear conversion can be performed so that a certain parametric strip of an object bit stream can be divided according to the ratio with which a corresponding parametric strip is distributed between two parametric bands of a multi-channel decoder . On the other hand, if more than one parametric band of the object bitstream is included in the specific parametric band of the multi-channel decoder, the parameter values of the object bitstream can be averaged. Alternatively, parametric band conversion can be performed using the existing multi-channel standard parametric band conversion table.

When object coding is used for teleconferences, the voices of different people correspond to object signals. The object decoder outputs the voices appropriately corresponding to the object signals to specific speakers. However, when several people speak at the same time, it is difficult for an object decoder to properly distribute people's voices to different speakers through decoding, and rendering people's voices can cause sound distortion and degrade sound quality. To resolve this, information indicating whether several people are talking at the same time can be included in the bitstream. Further, if it is determined based on the information that several people are talking simultaneously, the channel-based bit stream can be modified so that barely decoded signals, almost like down-mix signals, can be output to each speaker.

For example, suppose there are three people a, b, and c, and the voices of three people a, b, and c must be decoded and thereby output to speakers A, B, and C, respectively. When three people a, b and c speak at the same time, the voices of three people a, b and c can all be included in the down-mix signal, which is obtained by down-mixing the object signals, respectively, representing the voices of three people a, b and c. In this case, information regarding portions of the downmix signal appropriately corresponding to the voices of three people a, b, and c can be configured as a multi-channel bitstream. Further, the down-mix signal can be decoded using a typical object decoding method so that the voices of three people a, b and c can be output to speakers A, B and C, respectively. The output of each of the speakers A, B, and C, however, may be distorted and thus may have a lower recognition coefficient than the original down-mix signal. In addition, the voices of three people a, b, and c may be improperly separated from each other. To resolve this, information indicating that simultaneous speech fragments of three people a, b, and c are being spoken can be included in the bitstream. After that, the transcoder can generate a multi-channel bit stream so that the down-mix signal obtained by down-mixing the object signals, corresponding to the voices of three people a, b and c, can be output to each of the speakers A, B and C as is . Thus, signal distortion can be prevented.

In fact, when several people speak at the same time, it is difficult to separate the voice of each person. Therefore, the sound quality may be higher when the down-mix signal is output as is than when the down-mix signal is prepared by rendering so that the voices of different people can be separated from each other and output to different speakers. For this, the transcoder can generate a multi-channel bit stream so that the down-mix signal obtained from the simultaneous fragments of speech of several people can be output to all speakers or so that the down-mix signal can be amplified and then output to the speakers.

To indicate whether the downmix signal comes from simultaneous speech fragments of several people, the object encoder can appropriately modify the object bitstream instead of providing additional information, as described above. In this case, the object decoder can perform a typical decoding operation with the object bitstream so that the down-mix signal can be output to the speakers as is or the down-mix signal can be amplified, but not to the extent that signal distortion occurs, and then brought to the speakers.

The following describes in detail 3D information, such as HTRF, which is provided to a multi-channel decoder.

When the object encoder operates in stereo mode, the multi-channel decoder in the object decoder also operates in stereo mode. The end user can transmit 3D information, such as HRTF, which is optimized based on the spatial positions of the object signals, to a multi-channel decoder.

More specifically, when there are two object signals, i.e. OBJECT1 and OBJECT2, and two object signals OBJECT1 and OBJECT2 are located at positions 1 and 2, respectively, the rendering matrix generator or transcoder can have 3D information indicating the positions of the object signals OBJECT1 and OBJECT2. If the rendering matrix generator has 3D information indicating the positions of the object signals OBJECT1 and OBJECT2, the rendering matrix generator can transmit 3D information indicating the positions of the object signals OBJECT1 and OBJECT2 to the transcoder. On the other hand, if the transcoder has 3D information indicating the position of the object signals OBJECT1 and OBJECT2, the rendering matrix generator can only transmit index information corresponding to the 3D information to the transcoder.

In this case, a stereo signal can be generated based on 3D information specifying positions 1 and 2, as indicated by equation (18):

Equation 18

A multi-channel stereo decoder obtains stereo sound by performing decoding, with the assumption that a 5.1-channel speaker system must be used to reproduce sound, and stereo sound can be represented by equation (19):

Equation 19

The L-channel component of the object signal OBJECT1 can be represented by equation (20):

Equation 20

The component of the R-channel of the object signal OBJECT1 and the components of the L- and R-channel of the object signal OBJECT2 can all be specified by using equation (20). For example, if the ratios of the energy levels of the object signals OBJECT1 and OBJECT2 to the total energy level are a and b, respectively, the ratio of the part of the object signal OBJECT1 distributed in the FL channel to the entire object signal OBJECT1 is c, and the ratio of the part of the object signal OBJECT1 distributed in The FL channel, to the entire object signal OBJECT2 is equal to d, the ratio with which the object signals OBJECT1 and OBJECT2 are distributed in the FL channel is ac: bd. In this case, the HRTF of the FL channel can be determined as indicated by equation (21):

Equation 21

Thus, 3D information can be obtained for use in a multi-channel stereo decoder. Since 3D information for use in a multi-channel stereo decoder better represents the actual position of the object signals, stereo signals can be reproduced more vividly by stereo decoding using 3D information for use in a multi-channel stereo encoder than when performing multi-channel decoding with 3D information corresponding to five speaker positions .

As described above, 3D information for use in a multi-channel stereo decoder can be based on 3D information representing the spatial positions of the object signals and energy ratio information. Alternatively, 3D information for use in a multi-channel stereo encoder can be generated by properly performing decorrelation when summing 3D information representing the spatial positions of the object signals based on the ICC information of the object signals.

Effect processing can be performed as part of pre-processing. Alternatively, the effect processing result may simply be added to the output of a multi-channel decoder. In the first case, in order to perform processing of the effects of the object signal, the extraction of the object signal may need to be performed in addition to dividing the L-channel signal into L_L and L_R and dividing the R-channel signal into R_R and R_L.

More specifically, the object signal may first be extracted from the L and R channel signals. Then, the L-channel signal can be divided into L_L and L_R, and the R-channel signal can be divided into R_R and R_L. Effects can be processed with an object signal. Then the object signal with processing effects can be divided into components of the L- and R-channel according to the rendering matrix. After that, the L-channel component of the object signal with effects processing can be added to L_L and R_L, and the R-channel component of the object signal with effects processing can be added to R_R and L_R.

Alternatively, pre-processed L- and R-signals of the L- and R-channel may first be generated. Then, the object signal can be extracted from the pre-processed L- and R-signals of the L- and R-channel. After that, the effects processing can be performed for the object signal, and the result of the effects processing can be added back to the pre-processed L- and R-channel signals.

The spectrum of the object signal can be modified by processing effects. For example, the level of the high part or low part of the object signal can be selectively increased. For this, only a part of the spectrum corresponding to the high-tonal part or low-tonal part of the object signal can be modified. In this case, information related to objects included in the object bitstream may need to be modified accordingly. For example, if the level of the low-tone part of a certain object signal rises, the energy of the low-tone part of a certain object signal can also be increased. Thus, the energy information included in the object bitstream no longer correctly represents the energy of a particular object signal. To resolve this, the energy information included in the object bitstream can be directly modified according to the variation in energy of a particular object signal. Alternatively, the spectrum variation information provided by the transcoder can be applied to the formation of a multi-channel bitstream so that the variation in energy of a particular object signal can be reflected in the multi-channel bitstream.

28-33 illustrate diagrams for explaining combining a plurality of fragments of object-based supplemental information and a plurality of downmix signals into a plurality of supplemental information and a downmix signal. In the case of teleconferences, it is sometimes necessary to combine a plurality of fragments of object-based additional information and a plurality of downmix signals into additional information and a downmix signal. In this case, a number of factors should be considered.

Fig. 28 illustrates an object-encoded bitstream diagram. Referring to FIG. 28, an object-encoded bitstream includes a downmix signal and additional information. The downmix signal is synchronized with additional information. Therefore, the object-encoded bitstream can easily be decoded without taking into account additional factors. However, when combining multiple bit streams into a single bit stream, you need to make sure that the down-mix signal of one bit stream is synchronized with the additional information of one bit stream.

29 is a diagram for explaining a combination of a plurality of object-coded bit streams BS1 and BS2. Referring to FIG. 29, references 1, 2, and 3 indicate frame numbers. To combine multiple downmix signals into a single downmix signal, the downmix signals can be converted to pulse code modulation (PCM) signals, the PCM signals can be down-mixed in the time domain, and the down-mixed PCM signals can be converted to compression codec format. During these processes, a delay d can be generated, as illustrated in FIG. 29 (b). Therefore, when the bit stream to be decoded is obtained by combining a plurality of bit streams, it must be ensured that the down-mix signal of the bit stream to be decoded is correctly synchronized with the additional information of the bit stream to be decoded.

If the delay between the down-mix signal and the additional bitstream information is specified, the bitstream can be compensated by a predetermined amount corresponding to the delay. The delay between the downmix signal and the additional bitstream information may vary according to the type of compression codec used to generate the downmix signal. Therefore, a bit indicating the delay, if any, between the down-mix signal and the additional information of the bit stream, may be included in the additional information.

30 illustrates combining two bit streams BS1 and BS2 into one bit stream when down-mixing signals of bit streams BS1 and BS2 are generated by different types of codecs or the configuration of additional information of bitstream BS1 is different from the configuration of additional information of bitstream BS2. Referring to FIG. 30, when the down-mix signals of the bit streams BS1 and BS2 are generated by different types of codecs or the configuration of the additional information of the bitstream BS1 is different from the configuration of the additional information of the bitstream BS2, it can be determined that the bit streams BS1 and BS2 have different delays signals d1 and d2, which are the result of converting the downmix signals into time-domain signals and transforming the time-domain signal using a single compression codec. In this case, if the BS1 and BS2 bitstreams are simply summed without regard to various signal delays, the downmix signal of the BS1 bitstream may be inconsistent with the downmix signal of the BS2 bitstream and the additional information of the BS1 bitstream may be inconsistent with the additional information of the BS2 bitstream. To resolve this, the down-mix signal of the bitstream BS1, which is delayed by d1, can be further delayed so as to be synchronized with the down-mix signal of the bitstream BS2, which is delayed by d2. Then, the bit streams BS1 and BS2 can be combined using one method of the embodiment of FIG. 30. If there is more than one bit stream that needs to be combined, the one of the bit streams that has the largest delay can be used as the reference bit stream, and other bit streams can be further delayed so as to be synchronized with the reference bit stream. A bit indicating the delay between the down-mix signal and the additional information may be included in the object bit stream.

A bit indicating whether there is a signal delay in the bitstream may be provided. Only if the bit information indicates that there is a signal delay in the bitstream, information indicating the signal delay can be further provided. Thus, it is possible to minimize the amount of information required to indicate a signal delay, if any, in the bitstream.

32 illustrates a diagram for explaining how to compensate one of two bit streams BS1 and BS2 having different signal delays by the difference between different signal delays, in particular how to compensate for a BS2 bit stream that has a longer signal delay than the bit stream BS1. Referring to FIG. 32, the first to third frames of additional information of the BS1 bitstream can all be used as is. On the other hand, the first to third frames of the additional information of the bitstream BS2 may not be used as they are, since the first to third frames of the additional information of the bitstream BS2 are not synchronized with the first to third frames of the additional information of the bitstream BS1. For example, the second frame of additional information of the bitstream BS1 corresponds not only to the part of the first frame of additional information of the bitstream BS2, but also to the part of the second frame of additional information of the bitstream BS2. The proportion of the portion of the second frame of additional information of the bitstream BS2 corresponding to the second frame of additional information of the bitstream BS1 to the entire second frame of additional information of the bitstream BS2 and the proportion of the portion of the first frame of additional information of the bitstream BS2 corresponding to the second frame of additional information of the bitstream BS1 the first frame of additional information of the bitstream BS2 can be calculated, and the first and second frames of additional information of the bitstream BS2 can be averaged or inter polished based on calculation results. Thus, the first to third frames of the additional information of the bitstream BS2 can respectively be synchronized with the first to third frames of the additional information of the bitstream BS1, as illustrated in FIG. 32 (b). In this case, the additional information of the bitstream BS1 and the additional information of the bitstream BS2 can be combined using the method of the embodiment of FIG. 29. The down-mix signals of the bit streams BS1 and BS2 can be combined into a single down-mix signal without the need for delay compensation. In this case, the delay information corresponding to the delay of the signal d1 can be stored in the combined bit stream obtained by combining the bit streams BS1 and BS2.

33 illustrates diagrams for explaining how to compensate for one of two bit streams having different signal delays that has a shorter signal delay. Referring to FIG. 33, first to third frames of additional information of the BS2 bitstream can all be used as is. On the other hand, the first to third frames of the additional information of the bitstream BS1 may not be used as is, since the first to third frames of the additional information of the bitstream BS1 are accordingly not synchronized with the first to third frames of the additional information of the bitstream BS2. For example, the first frame of the additional information of the bitstream BS2 corresponds not only to the part of the first frame of the additional information of the bitstream BS1, but also to the part of the second frame of the additional information of the bitstream BS1. The proportion of the portion of the first frame of additional information of the bitstream BS1 corresponding to the first frame of additional information of the bitstream BS2 to the entire first frame of additional information of the bitstream BS1 and the proportion of the portion of the second frame of additional information of the bitstream BS1 corresponding to the second frame of additional information of the bitstream BS2 the second frame of additional information of the bitstream BS1 can be calculated, and the first and second frames of additional information of the bitstream BS1 can be averaged or inter polished based on calculation results. Thus, from the first to third frames of the additional information of the bitstream BS1, respectively, can be synchronized with the first to third frames of the additional information of the bitstream BS2, as illustrated in FIG. 33 (b). In this case, the additional information of the bitstream BS1 and the additional information of the bitstream BS2 can be combined using the method of the embodiment of FIG. 29. The down-mix signals of the bit streams BS1 and BS2 can be combined into a single down-mix signal without the need for delay compensation, even if the down-mix signals have different signal delays. In this case, the delay information corresponding to the delay of the signal d2 can be stored in the combined bit stream obtained by combining the bit streams BS1 and BS2.

If a plurality of object-encoded bit streams are combined into a single bit stream, down-mix signals of the object-encoded bit streams may need to be combined into a single down-mix signal. In order to combine a plurality of downmix signals corresponding to different compression codecs into a single downmix signal, the downmix signals can be converted to PCM signals or frequency domain signals and PCM signals or frequency domain signals can be summed in the corresponding region. Therefore, the summation result can be converted using a predefined compression codec. Different signal delays can occur according to the fact that the down-mix signals are summed during the PCM operation or summed in the frequency domain, and according to the type of compression codec. Since the decoder cannot easily recognize the various signal delays from the bitstream to be decoded, the delay information regarding the various delay signals may not need to be included in the bitstream. This delay information may represent the number of delay samples in the PCM signal or the number of delay samples in the frequency domain.

The present invention can be implemented as a computer-readable code recorded on a computer-readable recording medium. A computer-readable recording medium may be any type of recording device in which data is stored in a computer-readable manner. Examples of computer-readable recording media include ROM, RAM, CD-ROM, a magnetic tapes, floppy disks, optical storage devices, and a wave carrier (e.g., data transmission over the Internet). Computer-readable recording media can be distributed across a plurality of computing systems connected over a network, so that computer-readable code is written to and executed from them in a decentralized manner. Functional programs, code, and code segments required to implement the present invention can be readily construed by those skilled in the art.

As described above, according to the present invention, audio images are localized for each object signal by taking advantage of the advantages of encoding and decoding methods based on objects. Thus, it is possible to offer more realistic sounds during the reproduction of object signals. In addition, the present invention can be applied to interactive games and thereby can provide the user with more realistic experiences in virtual reality.

Although the present invention is specifically shown and described with reference to its exemplary embodiments, those skilled in the art should understand that various changes in form and content can be made without departing from the spirit and scope of the present invention as defined by the appended claims .

Claims (11)

1. An audio decoding method, comprising the steps of:
receiving a downmix signal and object-based additional information and control information, wherein the downmix signal contains at least two channel downmix signals and is generated by downmixing a plurality of object signals, and the control information controls the position or level of the object signal included in the signal downmix;
extracting from the objects additional information based on the distribution ratio between the channels indicating a relationship corresponding to the object signal with which the object signal is distributed over each of the downmix channel signals;
generating pre-processing information for modifying the channel down-mix signals based on the distribution ratio information between the channels and the control information; and
modifying the channel down-mix signals by applying the preprocessing information to the channel down-mix signals;
forming channel-based additional information based on object-based additional information and control information; and
form a multi-channel audio signal based on additional information based on the channels and modified channel down-mix signals. 2. The audio decoding method of claim 1, wherein the object-based additional information comprises flag information indicating whether downmix amplification information is included in the object-based additional information. 3. An audio encoding method, comprising the steps of:
generating a downmix signal by downmixing a plurality of object signals, wherein the downmix signal comprises at least two channel downmix signals;
generating information of the distribution ratio between the channels by calculating the ratio with which the object signal is distributed over each of the channel down-mix signals;
generating object-based additional information based on said plurality of object signals;
inserting distribution ratio information between the channels for modifying the downmix channel signals into the object-based additional information; and
form a bitstream by combining a downmix signal and object-based additional information. 4. The audio encoding method according to claim 3, in which the bitstream contains flag information regarding whether to transmit the distribution ratio information between the channels. 5. An audio decoding apparatus comprising:
a demultiplexer configured to extract the downmix signal and object-based additional information from the input audio signal, wherein the downmix signal comprises at least two channel downmix signals;
a parameter converter configured to extract distribution relation information between channels indicating a relationship corresponding to the object signal with which the object signal is distributed across each of the downmix channel signals, and generate preprocessing information for modifying the downmix channel signals based on the distribution ratio information between the channels ; and
a preprocessor configured to modify the downmix channel signals by applying the preprocessing information to the downmix channel signals. 6. The audio decoding device according to claim 5, wherein the demultiplexer further extracts control information indicating a position or level of an object signal included in the downmix signal, wherein the parameter converter generates additional channel-based information based on the object-based additional information and control information. 7. The audio decoding apparatus according to claim 6, further comprising a multi-channel encoder that generates a multi-channel audio signal based on additional information based on the channels and modified down-mix channel signals. 8. The processor-readable recording medium on which the program is written to perform the method according to claim 1 in the processor. 9. A computer-readable recording medium on which a computer program is recorded for executing an audio decoding method, the audio decoding method comprising the steps of:
receiving a downmix signal and object-based additional information, wherein the downmix signal comprises at least two channel downmix signals;
extracting from the objects additional information based on the distribution ratio between the channels indicating a relationship corresponding to the object signal with which the object signal is distributed over each of the downmix channel signals;
generating pre-processing information for modifying the channel down-mix signals based on the distribution ratio information between the channels; and
modifying the channel down-mix signals by applying the preprocessing information to the channel down-mix signals. 10. The computer readable recording medium according to claim 9, wherein the method for decoding audio further comprises the steps of:
receiving control information indicating a position or level of an object signal included in the downmix signal;
forming channel-based additional information based on object-based additional information and control information; and
form a multi-channel audio signal based on additional information based on the channels and modified channel down-mix signals. 11. A computer-readable recording medium on which a computer program for executing an audio encoding method is recorded, wherein the audio encoding method comprises the steps of:
generating a downmix signal by downmixing a plurality of object signals, wherein the downmix signal comprises at least two channel downmix signals;
generating information of the distribution ratio between the channels by calculating the ratio with which the object signal is distributed over each of the channel down-mix signals;
generating object-based additional information based on said plurality of object signals;
inserting distribution ratio information between the channels for modifying the downmix channel signals into the object-based additional information; and
form a bitstream by combining a downmix signal and object-based additional information.

Images