KR20140037118A - Method of processing audio signal, audio encoding apparatus, audio decoding apparatus and terminal employing the same - Google Patents

Abstract

A method for processing an audio signal, when down-mixing a first multiple input channels to a second multiple output channels, comprises the steps of: comparing the location of the first multiple input channels and the second multiple output channels; down-mixing the channel having the same location with the second multiple output channel among the first multiple input channels to the channel having the same location among the second multiple output channels; searching at least one adjacent channel for the rest channels among the first multiple input channels; deciding weighted value by considering at least one among the distance between channels, the correlation of signals, and the error from restoration; and down-mixing the rest channels among the first multiple input channels to the adjacent channels based on the decided weighted value.

Description

Method of processing audio signal, audio encoding apparatus, audio decoding apparatus and terminal employing the same}

The present invention relates to audio encoding / decoding, and more particularly, to an audio signal processing method, an audio encoding apparatus, an audio decoding apparatus, and a terminal employing the same, which can minimize sound quality degradation when reconstructing a multi-channel audio signal. .

Recently, as multimedia contents are spread, the demand of users who want to experience a more realistic and rich sound source environment is increasing. In order to meet the needs of these users, research on multi-channel audio is being actively conducted.

Multi-channel audio signals require high data compression rates depending on the transmission environment. In particular, spatial parameters are used to recover the multi-channel audio signals. At this time, distortion may occur due to the influence of the reverberation signal in the process of extracting the spatial parameter. Then, sound quality degradation may occur in recovering the multi-channel audio signal.

Accordingly, there is a need for a multi-channel audio codec technology capable of reducing or eliminating sound quality degradation that may occur when reconstructing a multi-channel audio signal using spatial parameters.

SUMMARY OF THE INVENTION An object of the present invention is to provide an audio signal processing method, an audio encoding apparatus, an audio decoding apparatus, and a terminal employing the same, capable of minimizing sound degradation when reconstructing a multi-channel audio signal.

The audio signal processing method according to an embodiment of the present invention for achieving the above object, when downmixing the first plurality of input channels to the second plurality of output channels, the position and the position of the first plurality of input channels Comparing the positions of the second plurality of output channels; Downmixing a channel having the same position as the second plurality of output channels among the first plurality of input channels to a channel having the same position among the second plurality of output channels; Searching for at least one adjacent channel with respect to the remaining channels of the first plurality of input channels; Determining a weight with respect to the found adjacent channel in consideration of at least one of an inter-channel distance, a correlation of a signal, and an error in reconstruction; And downmixing the remaining channels of the first plurality of input channels to the adjacent channel based on the determined weight.

.

1 is a block diagram showing the configuration of an audio signal processing system to which the present invention is applied.
2 is a block diagram showing a configuration of an audio encoding apparatus to which the present invention is applied.
3 is a block diagram showing a configuration of an audio decoding apparatus to which the present invention is applied.
4 illustrates an example of channel matching between a 10.2 channel audio signal and a 5.1 channel audio signal according to an embodiment of the present invention.
5 is a flowchart illustrating a downmixing method according to an embodiment of the present invention.
6 is a flowchart illustrating an upmixing method according to an embodiment of the present invention.
7 is a block diagram showing the configuration of a spatial parameter encoding apparatus according to an embodiment of the present invention.
8A and 8B are diagrams showing an example of a quantization step that is variable according to an energy value in a frequency band of each frame for each downmix channel.
9 is a diagram illustrating an example of energy distribution for each frequency band of spectral data for all channels.
10A to 10C are diagrams illustrating an example of adjusting the overall bit rate by varying a threshold.
11 is a flowchart illustrating a method of generating spatial parameters according to an embodiment of the present invention.
12 is a flowchart illustrating a method of generating spatial parameters according to another embodiment of the present invention.
13 is a flowchart for explaining an audio signal processing method according to an embodiment of the present invention.
14A to 14C are exemplary diagrams for describing operation 1110 of FIG. 11 or operation 1330 of FIG. 13.
FIG. 15 is a diagram of another example for describing operation 1110 of FIG. 11 or operation 1330 of FIG. 13.
16A to 16D illustrate another example for describing operation 1110 of FIG. 11 or operation 1330 of FIG. 13.
17 is a graph showing the total sum of angle parameters.
18 is a view for explaining calculation of angle parameters according to an embodiment of the present invention.
19 is a block diagram illustrating a configuration of an audio signal processing system through a multi-channel codec and a core codec according to an embodiment of the present invention.
20 is a block diagram showing a configuration of an audio encoding apparatus according to an embodiment of the present invention.
21 is a block diagram showing the configuration of an audio decoding apparatus according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it can be understood to include all transformations, equivalents, and substitutes included in the technical spirit and technical scope of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

Terms such as first and second may be used to describe various components, but the components are not limited by the terms. Terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terminology used in the present invention is to select the general term is widely used as possible in consideration of the function in the present invention, but this may vary according to the intention of the person skilled in the art, precedent, or the emergence of new technology. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the contents throughout the present invention, rather than the names of the simple terms.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present invention, the term "comprises" or "having ", etc. is intended to specify that there is a feature, number, step, operation, element, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, in the following description with reference to the accompanying drawings, the same or corresponding components will be given the same reference numerals and duplicate description thereof will be omitted. do.

1 is a block diagram showing the configuration of an audio signal processing system to which the present invention is applied. The audio signal processing system 100 corresponds to a multimedia device, and includes a dedicated terminal for voice communication including a telephone and a mobile phone, a broadcast or music terminal including a TV, an MP3 player, and the like, or a broadcast or music terminal for a voice communication terminal. A fusion terminal of a dedicated terminal may be included, but is not limited thereto. In addition, the audio signal processing system 100 may be used as a client, a server, or a transducer disposed between the client and the server.

Referring to FIG. 1, an audio signal processing system 100 includes an encoding device 110 and a decoding device 120. In one embodiment, the audio signal processing system 100 may include both the encoding apparatus 110 and the decoding apparatus 120. In another embodiment, the audio signal processing system 100 may include the encoding apparatus 110 and the decoding apparatus. It may include any one of the 120.

The encoding apparatus 110 receives an original signal composed of a plurality of channels, that is, a multichannel audio signal, and downmixes the original signal to generate a downmixed audio signal. The encoding apparatus 110 generates and encodes a prediction parameter. Here, the prediction parameter is a parameter applied to restore the downmixed audio signal to the original signal. Specifically, it is a value related to the downmix matrix used for downmixing the original signal, each coefficient value included in the downmix matrix, and the like. For example, the prediction parameter may comprise a spatial parameter. In addition, the prediction parameter may vary according to a product specification, a design specification, etc. of the encoding apparatus 110 or the decoding apparatus 120, and may be set to an experimentally optimized value. Here, the channel may mean a speaker.

The decoding device 120 upmixes the downmixed audio signal using the prediction parameter to generate a reconstruction signal corresponding to the multichannel audio signal as the original signal.

2 is a block diagram showing a configuration of an audio encoding apparatus to which the present invention is applied.

Referring to FIG. 2, the audio encoding apparatus 200 may include a downmixer 210, an additional information generator 220, and an encoder 230. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

The downmixer 210 receives the N multichannel audio signals and downmixes the received multichannel audio signals. The N-channel audio signal can be downmixed to produce a mono-channel audio signal or an M (where M < N) channel audio signal. For example, the 10.2 channel audio signal may be downmixed into a 3 channel audio signal or a 6 channel audio signal to correspond to a 2.1 channel audio signal or a 5.1 channel audio signal.

According to an embodiment, two channels are selected from the N channels and downmixed to generate a first monochannel, and a second monochannel is generated by downmixing another channel different from the generated first monochannel. The final monochannel audio signal or the M channel audio signal may be generated by repeating the downmixing process by adding another channel to the monochannel generated as the downmixing result.

In downmixing N-channel audio signals, it is desirable to downmix similar channels in order to downmix with minimal entropy. Therefore, the downmixing unit 210 can downmix the multi-channel audio signal at a higher compression rate by downmixing the highly correlated channels.

The additional information generator 220 generates additional information necessary to restore the multichannel from the downmixed channel. Each time the downmixing unit 210 sequentially downmixes the multichannels, the downmixing unit 210 generates additional information necessary to restore the multichannels from the downmixed channels. In this case, information for determining the strength of the downmixed two channels and information for determining the phase of the two channels may be generated.

In addition, whenever the downmixing is performed, the additional information generating unit 220 generates information indicating which channels are downmixed. When the downmixing is sequentially performed based on the correlation calculation rather than the fixed order, the downmixing order of the channels may be generated as additional information.

The additional information generator 220 repeats generation of information necessary to restore the downmixed channel in the mono channel whenever downmixing continues. For example, if a single channel is generated by sequentially downmixing 12 channels 11 times in sequence, information about the downmixing order, information for determining channel strength, and information for determining channel phase are provided. 11 times each. In addition, according to an embodiment, when generating information for determining the strength of the channel and information for determining the phase of the channel for each of a plurality of frequency bands, if the number of frequency bands is k, the strength of the channel is determined. 11 * k information may be generated, and 11 * k information for determining a phase of a channel may be generated.

The encoder 230 may encode a monochannel audio signal or an M channel audio signal generated by downmixing by the downmixer 210. When the audio output from the downmixing unit 210 is an analog signal, the analog signal is converted into a digital signal, and the symbols are encoded according to a predetermined algorithm. There is no limitation to an encoding algorithm, and any algorithm for encoding a audio signal to generate a bitstream may be used in the encoder 230. In addition, the encoder 230 may encode the additional information generated by the additional information generator 220 to recover the multichannel audio signal from the monochannel audio signal.

3 is a block diagram showing a configuration of an audio decoding apparatus to which the present invention is applied.

Referring to FIG. 3, the audio decoding apparatus 300 may include an extractor 310, a decoder 320, and an upmixer 330. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

The extractor 310 extracts the encoded audio and the encoded additional information from the received audio data, that is, the bitstream. The encoded audio may be generated by downmixing the N channels to one mono channel or M (where M <N) channel, and then encoding the audio signal according to a predetermined algorithm.

The decoder 320 decodes the encoded audio and additional information extracted by the extractor 310. The encoded audio and the additional information are decoded using the same algorithm as the algorithm used for encoding. As a result of decoding the audio, one monochannel audio signal or M multichannel audio signals are restored.

The upmixer 330 up-mixes the audio signal decoded by the decoder 720 to restore the N-channel audio signal before downmixing. At this time, the N-channel audio signal is restored based on the additional information decoded by the decoder 320.

That is, the downmix process is performed inversely with reference to the additional information, which is a spatial parameter, to upmix the downmixed audio signal into a multichannel audio signal. At this time, the channels are separated in order from the mono channel with reference to the additional information including information on the downmixing order of the channels. By determining the strength and phase of the downmixed channels according to the information for determining the strength and phase of the downmixed channels, the channels may be separated in order in the monochannel.

4 illustrates an example of channel matching between a 10.2 channel audio signal and a 5.1 channel audio signal according to an embodiment of the present invention.

If the input multichannel audio signal is a 10.2 channel audio signal, fewer downmixed multichannel audio signals, such as 7.1 channel audio signals, 5.1 channel audio signals, or 2.0 channel audio signals, are available as output multichannel audio signals. Can be.

As shown in FIG. 4, when downmixing the 10.2 channel audio signal 410 to the 5.1 channel audio signal 420, if the FL and RL channels are identified as the adjacent channels of the LW channel among the 10.2 channels, the position and correlation Alternatively, the weight may be determined in consideration of an error in restoration. According to an embodiment, when the weight for the FL channel is 0 and the weight for the RL channel is determined to be 1, the channel signal of the LW channel among the 10.2 channels may be downmixed to the RL channel among the 5.1 channels.

Meanwhile, in FIG. 4, the L channel and the Ls channel among the 10.2 channels may be allocated to the FL channel and the RL channel of the 5.1 channel located at the same position.

5 is a flowchart illustrating a downmixing method according to an embodiment of the present invention.

Referring to FIG. 5, in step 510, the number and positions of input channels are checked from the first layout information. For example, the first layout information is IC 1, IC 2, and IC (N), where the positions of the N input channels can be known.

In operation 520, the number and locations of downmixed channels, that is, output channels, are determined from the second layout information. For example, the second layout information is DC (1), DC (2), and DC (M), where the positions of M output channels can be known.

In operation 530, starting from the first channel IC 1 of the input channel, it is determined whether there is a channel having the same output position among the input channel and the output channel.

In operation 540, when there is a channel having the same output position among the input channel and the output channel, the channel signal of the input channel may be allocated to the output channel having the same position. For example, when the output positions of the input channel IC (n) and the output channel DC (m) are the same, DC (m) = DC (m) + IC (n).

In step 550, if there is no channel having the same output position among the input channel and the output channel, starting from the first channel IC 1 of the input channel, it is checked whether there is a channel adjacent to the input channel IC (n) among the output channels.

In step 560, if it is determined in step 550 that there are a plurality of adjacent channels, the channel signal of the input channel IC (n) is distributed to each adjacent channel using a predetermined weight corresponding to each identified adjacent channel. For example, if DC (i), DC (j), DC (k) is identified as an adjacent channel of the input channel IC (n) in the output channel, the input channel IC (n) and the output channel DC (i), Weights w _i , w _j , and w _k may be set for the input channel IC (n) and the output channel DC (j) and the input channel IC (n) and the output channel DC (k), respectively. DC (i) = DC (i) + w _i using the set weights w _i , w _j , w _k * IC (n), DC (j) = DC (j) + w _j * IC (n), DC (k) = DC (k) + w _k Like the IC (n), the channel signal of the input channel IC (n) can be distributed.

In addition, a weight can be set based on the following method.

In one embodiment, the weight may be determined according to the relationship between the plurality of adjacent channels and the input channel IC (n). As a relationship between a plurality of adjacent channels and the input channel IC (n), the correlation between the distance between the plurality of adjacent channels and the input channel IC (n), each channel signal of the plurality of adjacent channels and the channel signal of the input channel IC (n) At least one of a relationship and recovery errors in a plurality of adjacent channels may be applied.

In another embodiment, the weight may be determined as 0 or 1 depending on the relationship between the plurality of adjacent channels and the input channel IC (n). For example, among the plurality of adjacent channels, the adjacent channel having the closest distance to the input channel IC (n) may be determined as 1, and the remaining adjacent channels may be determined as 0. Alternatively, the adjacent channel having the highest correlation with the channel signal of the input channel IC (n) among the channel signals of the plurality of adjacent channels may be determined as 1, and the remaining adjacent channels may be determined as 0. Alternatively, the adjacent channel having the least reconstruction error among the plurality of adjacent channels may be determined as 1, and the remaining adjacent channels may be determined as 0.

In step 570, it is determined whether all the channels of the input channel have been checked.

In step 580, when all the channels of the input channel are checked, configuration information of the downmixed channels having a signal allocated in step 540 and a signal distributed in step 560 and a spatial parameter corresponding thereto are generated. do.

The downmixing method according to the above embodiment may be performed in units of channels, frames, frequency bands, or frequency bands, thereby adjusting the precision of performance improvement as necessary. Here, the frequency band is a unit of grouping samples of the audio spectrum and may have a uniform or non-uniform length reflecting a critical band. In the case of non-uniformity, the number of samples included in the frequency band from the start sample to the last sample may be gradually increased for one frame. In the case of supporting multiple bit rates, the number of samples included in each frequency band corresponding to different bit rates may be set to be the same. The number of frequency bands included in one frame or the number of samples included in one frequency band may be predetermined.

In addition, the downmixing method according to the embodiment may determine a weight used for channel downmixing, corresponding to the layout of the downmixed channel and the layout of the input channel. According to this, it is possible to adaptively adapt to various layouts, and to determine the weight in consideration of the correlation between the channel signals as well as the position of the channel or an error in reconstruction, thereby improving the reconstructed sound quality. In addition, since downmixed channels are configured in consideration of channel position, correlation between channel signals, or an error in reconstruction, if the audio decoding apparatus has the same number of downmixed channels, there is no upmixing process. Even if a user listens to only downmixed channels, there is an advantage in that subjective sound quality deterioration cannot be recognized.

6 is a flowchart illustrating an upmixing method according to an embodiment of the present invention.

Referring to FIG. 6, in step 610, configuration information of downmixed channels and a spatial parameter corresponding thereto are generated through a process as illustrated in FIG. 5.

In operation 620, the input channel audio signal is restored by performing upmixing using configuration information of the downmixed channels received in operation 610 and spatial parameters corresponding thereto.

FIG. 7 is a block diagram illustrating a configuration of a spatial parameter encoding apparatus according to an embodiment of the present invention, and may be included in the encoder 230 of FIG. 2.

Referring to FIG. 7, the spatial parameter encoding apparatus 700 may include an energy calculator 710, a quantization step determiner 720, a quantizer 730, and a multiplexer 740. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

The energy calculator 710 inputs a downmixed channel signal provided from the downmixer 210 of FIG. 2 to calculate an energy value in units of channels, frames, frequency bands, or frequency bands. Here, an example of the energy value may be a norm value.

The quantization step determiner 720 determines the quantization step by using energy values calculated in units of channels, frames, frequency bands, or frequency bands provided by the energy calculator 710. For example, the quantization step can be reduced for a channel, frame, frequency band or frequency band with a large energy value, and the quantization step can be increased for a channel, frame, frequency band or frequency band with a small energy value. In this case, two quantization steps may be set, and one quantization step may be selected according to a result of comparing the energy value with a predetermined threshold. On the other hand, when adaptively allocating a quantization step corresponding to the distribution of energy values, it is possible to select a quantization step matching the distribution of energy values. According to this, the allocation bit required for quantization can be adjusted according to the auditory importance, thereby improving sound quality. In addition, according to an embodiment, the overall bit rate may be adjusted by varying the threshold frequency while maintaining the assigned weight according to the energy distribution of each downmixing channel.

The quantization and lossless encoding unit 730 quantizes a spatial parameter in units of a channel, a frame, a frequency band, or a frequency band by using the quantization step determined by the quantization step determiner 720, and then performs lossless encoding.

The multiplexer 740 multiplexes the lossless coded downmixed audio signal together with the lossless coded spatial parameters to form a bitstream.

8A and 8B are diagrams showing an example of a quantization step that is variable according to an energy value in a frequency band of each frame for each downmix channel. Here, an example of downmixing channel 1 and channel 2 and downmixing channel 3 and channel 4 will be given. d0 represents an energy value of the downmixing channel for channel 1 and channel 2, and d1 represents an energy value of the downmixing channel for channel 3 and channel 4, respectively.

8A and 8B are examples in which two quantization steps are set. Since the hatched portion corresponds to a frequency band having an energy value equal to or greater than a predetermined threshold, it is set as a small quantization step.

FIG. 9 shows an example of energy distribution for each frequency band of spectral data for all channels, and FIGS. 10A to 10C show that the threshold frequency is changed in consideration of energy distribution while allocating weights according to energy values for each channel. It is a figure which shows the example which adjusts a bit rate.

FIG. 10A shows that the left portion, i.e., the low frequency regions 110a, 120a, 130a smaller than the threshold frequency, set the quantization step smaller, and the right portion, i.e., the high frequency region 110b larger than the threshold frequency, based on the initial threshold frequency 100a. 120b and 130b show an example in which the quantization step is set large. FIG. 10B shows an example in which the areas 140a, 150a, 160a for setting the quantization step are increased by using the increased threshold frequency 100b compared to the initial threshold frequency 100a, thereby increasing the overall bit rate. FIG. 10C illustrates an example in which the areas 170a, 180a, and 190a for setting the quantization step are reduced by using the reduced threshold frequency 100c compared to the initial threshold frequency 100a, thereby lowering the overall bit rate.

11 is a flowchart illustrating a method of generating a spatial parameter according to an embodiment of the present invention, which may be performed by the encoding apparatus 200 of FIG. 2.

Referring to FIG. 11, in operation 1110, N angle parameters are generated.

In step 1120, encoding is independently performed on the (N-1) angle parameters among the N angle parameters.

In operation 1130, the other one angle parameter is predicted from the (N-1) angle parameters.

In operation 1140, residual encoding is performed on the predicted angle parameter to generate residue of the other angle parameter.

12 is a flowchart illustrating a method of generating a spatial parameter according to another embodiment of the present invention, which may be performed by the decoding apparatus 300 of FIG. 3.

Referring to FIG. 12, in step 1210, (N-1) angle parameters among N angle parameters are received.

In operation 1220, the other angle parameter is predicted from the (N-1) angle parameters.

In operation 1230, the predicted angle parameter and the residue are added to generate the other angle parameter.

13 is a flowchart for explaining an audio signal processing method according to an embodiment of the present invention.

Referring to FIG. 13, in operation 1310, n channel signals ch1 to chn that are multichannel signals are downmixed. In detail, the n channel signals ch1 to chn may be downmixed into one mono signal DM. Operation 1310 may be performed by the downmixing unit 210 of FIG. 2.

In operation 1320, the (n-1) channel signals among the n channel signals ch1 to chn inputted are summed or the n channel signals ch1 to chn inputted. Specifically, the remaining channel signals except for the reference channel signal among the first to nth channel signals ch1 to chn may be summed, and the summed signal becomes the first summation signal described above. Alternatively, all of the first to n th channel signals ch1 to chn may be summed, and the summed signal may be the second summation signal.

In operation 1330, the aforementioned first spatial parameter may be generated by using a correlation between the first sum signal, which is the signal generated in operation 1320, and the reference channel signal. Alternatively, in operation 1330, the second spatial parameter may be generated by using the correlation between the second summation signal and the reference channel signal, which are signals generated in operation 1320.

In addition, the reference channel signal may be each of the first to nth channel signals ch1 to chn. Therefore, the number of reference channel signals may be all n, and n spatial parameters corresponding to the reference channel signal may also be generated.

Therefore, operation 1330 may further include generating n spatial parameters by using each of the first to n th channel signals ch1 to chn as a reference channel signal.

The operations of steps 1320 and 1330 may be performed by the downmixing unit 210.

In operation 1340, the spatial parameter SP generated in operation 1330 is encoded and transmitted to the decoding apparatus 300 of FIG. 3. In addition, the mono signal DM generated in operation 1310 is encoded and transmitted to the decoder 300 of FIG. 3. In detail, the encoded spatial parameter and the encoded mono signal may be included in the transport stream TS and transmitted to the decoding apparatus 300 of FIG. 3. The spatial parameter included in the transport stream TS refers to a spatial parameter set including the aforementioned first to nth spatial parameters.

Operation 1340 may be performed by the encoding apparatus 200 of FIG. 2.

14A to 14C are exemplary diagrams for describing operation 1110 of FIG. 11 or operation 1330 of FIG. 13. Hereinafter, an operation of generating the first summation signal and the first spatial parameter will be described in detail with reference to FIGS. 14A to 14C. 14A to 14C illustrate an example in which a multi-channel signal includes first to third channel signals ch1, ch2, and ch3. In addition, FIGS. 14A to 14C illustrate the summation of signals as vector summation of signals, and the summation of signals means downmixing, and there may be various downmixing methods in addition to the vector summation method.

14A, 14B, and 14C illustrate cases where the reference channel signal is the first channel signal ch1, the second channel signal ch2, and the third channel signal ch3, respectively.

Referring to FIG. 14A, when the reference channel signal is the first channel signal ch1, the additional information generator 220 sums the second and third channel signals ch2 and ch3 excluding the reference channel signal (ch2). + ch3) to generate a sum signal 1410. The spatial parameter is generated using the correlations ch1 and ch2 + ch3 between the first channel signal ch1 that is the reference channel signal and the sum signal 1410. The spatial parameter has information indicating a correlation between the reference channel signal and the sum signal and information indicating a relative signal magnitude of the reference channel signal and the sum signal.

Referring to FIG. 14B, when the reference channel signal is the second channel signal ch2, the additional information generator 220 sums the first and third channel signals ch1 and ch3 excluding the reference channel signal (ch1). + ch3) to generate a sum signal 1420. The spatial parameter is generated using the correlations ch2 and ch1 + ch3 between the second channel signal ch2 that is the reference channel signal and the summation signal 1420.

Referring to FIG. 14C, when the reference channel signal is the third channel signal ch3, the additional information generator 220 sums the first and second channel signals ch1 and ch2 excluding the reference channel signal (ch1). + ch2) to generate a sum signal 1430. The spatial parameter is generated using the correlations ch3 and ch1 + ch3 between the third channel signal ch3 and the summation signal 1430.

When the multichannel signal includes three channel signals, three reference channel signals may be generated, and three spatial parameters may be generated. The generated spatial parameter is encoded by the encoding apparatus 200 and transmitted to the decoding apparatus 300 through a network (not shown).

The mono signal DM downmixing the first to third channel signals ch1, ch2 and ch3 is the same as the sum signal of the first to third channel signals ch1, ch2 and ch3, and DM = ch1. It can be expressed as + ch2 + ch3. Therefore, the relationship of ch1 = DM-(ch2 + ch3) is established.

The decoding apparatus 300 receives and decodes the first spatial parameter, which is the spatial parameter described with reference to FIGS. 14A to 14C. The original channel signals are recovered using the decoded mono signal and the decoded spatial parameter. As described above, the relationship of ch1 = DM-(ch2 + ch3) is established, and the spatial parameter generated in FIG. 14A is a parameter and signals ch1, representing the relative magnitude of the signals ch1, ch2 + ch3. Since the parameter representing the similarity of ch2 + ch3) may be included, the ch1 and ch2 + ch3 signals may be recovered by using the spatial parameter and the mono signal DM generated in FIG. 14A. In the same way, using the spatial parameters generated in FIGS. 14B and 14C, the ch2 and ch1 + ch3 signals and the ch3 and ch1 + ch3 signals may be restored. That is, the upmixer 330 of FIG. 3 may restore all of the first to third channel signals ch1, ch2, and ch3.

FIG. 15 is a diagram of another example for describing operation 1110 of FIG. 11 or operation 1330 of FIG. 13. Hereinafter, an operation of generating the second sum signal and the second spatial parameter will be described in detail with reference to FIG. 15. FIG. 15 illustrates an example in which a multichannel signal includes first to third channel signals ch1, ch2, and ch3. In addition, in FIG. 15, the sum of the signals is illustrated taking the vector sum of the signals as an example.

Referring to FIG. 15, since the second summation signal is a sum of all of the first to third channel signals ch1, ch2, and ch3 that are multi-channel signals, the second summation signal may be added to the signal 1510 that sums the ch1 and ch2 signals. A signal (ch1 + ch2 + ch3) 1520 obtained by adding the ch3 signals becomes a second sum signal.

The spatial parameter between the first channel signal ch1 and the second sum signal 1520 is generated using the first channel signal ch1 as a reference channel signal. Specifically, a spatial parameter including at least one of the first parameter and the second parameter is determined using the correlations ch1 and ch1 + ch2 + ch3 between the first channel signal ch1 and the second sum signal 1520. Can be generated.

Then, using the second channel signal ch2 as the reference channel signal, the spatial parameter is obtained by using the correlations ch2 and ch1 + ch2 + ch3 between the second channel signal ch2 and the second sum signal 1520. Create In addition, by using the third channel signal ch3 as a reference channel signal and using the correlations ch2 and ch1 + ch2 + ch3 between the third channel signal ch3 and the second sum signal 1520, a spatial parameter is obtained. Create

The decoding apparatus 300 of FIG. 3 receives and encodes the first spatial parameter, which is the spatial parameter described with reference to FIG. 15. The original channel signals are recovered by using the encoded mono signal and the decoded spatial parameter. Here, the decoded mono signal corresponds to the sum signal ch1 + ch2 + ch3 of the multichannel signals.

Therefore, when the spatial parameter generated using the correlation between the first channel signal ch1 and the second sum signal 1520 (ch1, ch1 + ch2 + ch3) and the decoded mono signal are used, the first channel signal ( ch1) can be restored. Similarly, using the spatial parameter generated by the correlation between the second channel signal ch2 and the second sum signal 1520, ch2, ch1 + ch2 + ch3, the second channel signal ch2 Can be restored In addition, when the spatial parameter generated by the correlation (ch2, ch1 + ch2 + ch3) between the third channel signal ch3 and the second sum signal 1520 is used, the third channel signal ch3 may be restored. Can be.

16A through 16D are still another diagrams for describing operation 1110 of FIG. 11 or operation 1330 of FIG. 13.

First, in the encoding apparatus 200 of FIG. 2, the spatial parameter generated by the additional information generator 220 may include an angle parameter as the first parameter. The angle parameter is the other channel signals except for the reference channel signal which is one of the first to nth channel signals ch1 to chn and the reference channel signal among the first to nth channel signals ch1 to chn. This parameter represents a correlation between the signal magnitudes of the signals at predetermined angle values. The angle parameter may be referred to as a global vector angle (GVA). In addition, the angle parameter may be viewed as a parameter representing the relative magnitude of the reference channel signal and the first sum signal as an angle value.

In addition, the additional information generator 220 may generate n first to nth angle parameters by using each of the first to nth channel signals ch1 to chn as a reference channel signal. Hereinafter, an angle parameter generated using the k-th channel signal as the reference channel signal is referred to as a k-th angle parameter.

FIG. 16A illustrates an example in which the multichannel signal input by the encoding apparatus 200 includes first to third channel signals ch1, ch2, and ch3. 16B, 16C, and 16D illustrate cases where the reference channel signal is the first channel signal ch1, the second channel signal ch2, and the third channel signal ch3, respectively.

Referring to FIG. 16B, when the reference channel signal is the first channel signal ch1, the additional information generator 220 may select the second and third channel signals ch2 and ch3 which are the remaining channel signals except for the reference channel signal. The sum (ch2 + ch3) is performed to obtain a first angle parameter (angle 1) 1622, which is an angle parameter between the summed signal 1620 and the first channel signal ch1.

In detail, the first angle parameter 1622 is an inverse tangent of a value obtained by dividing an absolute value of the summed signal (ch2 + ch3) 1620 by an absolute value of the first channel signal ch1. Can be obtained by

Referring to FIG. 16C, the second angle parameter 1632 using the second channel signal ch2 as the reference channel signal may include the absolute value of the summed signals ch1 + ch3 1630 as the second channel signal. It can be found by inverse tangent of the value divided by the absolute value of (ch2).

Referring to FIG. 16D, the third angle parameter 1642 using the third channel signal ch3 as the reference channel signal may include an absolute value of the summed signal ch2 + ch3 1640 as the third channel signal. The value divided by the absolute value of (ch3) can be obtained by inverse tangent.

17 is a graph showing the sum of angle parameters, where the x axis represents an angle value and the y axis represents a distribution probability. In addition, the illustrated angle value corresponds to 6 degrees in one unit, for example, a value of 30 on the x-axis becomes 180 degrees.

Specifically, the sum of the n angle parameters calculated using each of the first to nth channel signals as the reference channel signal converges to a predetermined value. The predetermined value converged may vary depending on the value of n, and may be optimized through simulation or experimentally. For example, when n is 3, it may be approximately 180 degrees.

Referring to FIG. 17, when n is three, the sum of the angle parameters converges at 30 units, that is, around 1710 around 1710 as shown. Here, the graph of FIG. 14 is calculated through simulation or experimentally.

Exceptionally, the sum of the angle parameters sometimes converges at 45 units, i.e., 1720, around 1720. When the predetermined value converges in the vicinity 1720, all three channel signals are silent and each angle parameter has a value of 90 degrees. In the case of this exception, changing the value of one of the three angle parameters to zero, the sum of the angle parameters again converges to 180 degrees. When all three channel signals are muted, the downmixed mono signal also has a zero value, even when upmixing and decoding the mono signal. Accordingly, since the upmixing and decoding results do not change even when the value of the angle parameter is changed to 0, the value of the angle parameter of one of the three angle parameters may be changed to 0.

FIG. 18 is a diagram for describing calculation of angle parameters, and illustrates an example in which a multi-channel signal includes three channel signals ch1, ch2, and ch3. According to an embodiment, the spatial parameter may be generated including the residues of the first to nth angle parameters except the kth angle parameter and the kth angle parameter used to calculate the kth angle parameter. have.

Referring to FIG. 18, when the first channel signal ch1 is a reference channel signal, a first angle parameter is calculated and encoded, and the encoded first angle parameter is included in a predetermined bit region 1810 to decode the decoder (FIG. 3). Is sent to 300). When the second channel signal ch2 is the reference channel signal, the second angle parameter is calculated and encoded, and the encoded second angle parameter is included in the predetermined bit region 1830 to decode the apparatus (300 of FIG. 3). Is sent to.

When the third angle parameter is the above-described kth angle parameter, the residue of the kth angle parameter may be obtained as follows.

Since the sum of the n angle parameters converges to a predetermined value, the value of the kth angle parameter may be obtained by subtracting the value of the angle parameters except the kth angle parameter among the n angle parameters from the predetermined value. Specifically, when n is 3, the total sum of the three angle parameters converges 180 degrees unless all three channel signals are silent. Thus, the value of the third angle parameter = 180 degrees-(the value of the first angle parameter + the value of the second angle parameter). The third angle parameter may be predicted by using the relationship between the first to second angle parameters.

In detail, the additional information generator 220 of FIG. 2 predicts a value of the k th angle parameter among the first to n th angle parameters. The predetermined bit area 1870 represents a data area including a value of the predicted k-th angle parameter.

Next, the additional information generator 220 of FIG. 2 compares the predicted k-th angle parameter with the value of the original k-th angle parameter. The predetermined bit area 1850 represents a data area including the value of the third angle parameter calculated as in FIG. 13D.

Next, the additional information generator 220 of FIG. 2 generates a difference value between the predicted k-th angle parameter value 1870 and the original k-th angle parameter value 1850 as a residue of the k-th angle parameter. . The predetermined bit area 1890 represents a data area including a residue of the kth angle parameter.

The encoder (200 of FIG. 2) may include angle parameters 1810 and 1830 in the first to nth angle parameters except for the kth angle parameter, and a residue 1910 of the kth angle parameter. The spatial parameter including the included parameter is encoded and transmitted to the decoder 300 of FIG. 3.

The decoder 300 of FIG. 3 receives the spatial parameters including the angle parameters except the kth angle parameter among the first to nth angle parameters and the residue of the kth angle parameter.

The decoder 320 of the decoder 300 of FIG. 3 restores the k-th angle parameter by using the received spatial parameter and a predetermined value.

In detail, the decoder 320 subtracts values of angle parameters excluding the k-th angle parameter among the first to n-th angle parameters from a predetermined value, and compensates for the residue of the k-th angle parameter from the subtracted value. The k th angle parameter may be generated.

The residue of the kth angle parameter has a smaller data size compared to the value of the k angle parameter. Therefore, when the spatial parameters including the angle parameters except the k-th angle parameter and the residue of the k-th angle parameter among the first to n-th angle parameters are transmitted to the decoder 300 of FIG. The amount of data transmitted / received between 200 of 2 and the decoder 300 of FIG. 3 can be reduced.

On the other hand, in generating angle parameters for three channels, for example, it is possible to know which channel has performed the residual encoding by representing the angle parameters of 0, 1, and 2. That is, when all three channels are independently encoded, 2 bits * 3 = 6 bits are required. However, according to the following method, 5 bits can be represented.

First, if D = A + B * 3 + C * 9 (where% D is in the range of 0 to 26), A, B, C is C = floor (D / 9); D '= mod (D, 9); B = floor (D '/ 3); Can be found as A = mod (D '/ 3).

19 is a block diagram showing the configuration of an audio signal processing system integrating a multi-channel codec and a core codec according to an embodiment of the present invention.

The audio signal processing system 1900 illustrated in FIG. 19 includes an encoding device 1910 and a decoding device 1940. In one embodiment, the audio signal processing system 1900 may include both an encoding device 1910 and a decoding device 1940. In another embodiment, the audio signal processing system 100 may include an encoding device 1910 and a decoding device. 1940.

The encoding apparatus 1910 may include a multichannel encoder 1920 and a core encoder 1930, and the decoding apparatus 1940 may include a coder decoder 1850 and a multichannel decoder 1960.

Examples of codec algorithms used in the coder encoder 1930 and the core decoder 1950 may be AC-3, Enhancement AC-3, or AAC using a Modified Discrete Cosine Transform (MDCT) as a conversion algorithm, but is not limited thereto. It doesn't work.

20 is a block diagram showing the configuration of an audio encoding apparatus according to an embodiment of the present invention, in which a multi-channel encoder 2010 and a core encoder 2040 are integrated.

The audio encoding apparatus 2000 illustrated in FIG. 20 includes a multichannel encoder 2010 and a core encoder 2040. The multichannel encoder 2010 is a converter 2020 and a downmixer 2030. The encoder 2040 may include an envelope encoder 2050, a bit allocator 2060, a quantizer 2070, and a bitstream combiner 2080. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 20, the converter 2020 converts a PCT input in the time domain into spectral data in the frequency domain. At this time, Modified Odd Discrete Fourier Transform (MODFT) may be applied. MODFT = MDCT jMDST is used to generate the MDCT component, which eliminates the need for existing inverse transform parts and analysis filter bank parts. In addition, since the MODFT is composed of complex values, the MODFT can be obtained more accurately for the level / phase / correlation than in MDCT.

The downmixer 2030 extracts spatial parameters from the spectrum data provided from the converter 2020 and performs downmixing to generate downmixed spectrum. The extracted spatial parameters are provided to the bitstream combiner 2080.

The envelope encoder 2050 performs lossless encoding by obtaining an envelope value in units of predetermined frequency bands from the MDCT transform coefficients of the downmixed spectrum provided from the downmixer 2030. Here, the envelope may be configured from any one of power, average amplitude, norm value, and average energy obtained in units of a predetermined frequency band.

The bit allocator 2060 generates bit allocation information necessary for encoding the transform coefficients using envelope values obtained in units of frequency bands, and normalizes the MDCT transform coefficients. In this case, an envelope value quantized and losslessly encoded in each frequency band unit may be included in the bitstream and provided to the decoding apparatus 2100 of FIG. 21. With respect to bit allocation using envelope values of respective frequency bands, dequantized envelope values may be used so that the same process may be used in the encoding apparatus and the decoding apparatus. If the norm value is used as the envelope value, the masking threshold value may be calculated using the norm value for each frequency band unit, and the perceptually necessary number of bits may be predicted using the masking threshold value.

The quantization unit 2070 generates a quantization index by performing quantization on the MDCT transform coefficients of the downmixed spectrum based on the bit allocation information provided from the bit allocation unit 2060.

The bitstream combiner 2080 generates a bitstream by combining the encoded spectral envelope, the quantization index of the downmixed spectrum, and the spatial parameter.

21 is a block diagram showing the configuration of an audio decoding apparatus according to an embodiment of the present invention, in which a core decoder 2110 and a multichannel decoder 2160 are integrated.

The audio decoding apparatus 2100 illustrated in FIG. 21 includes a core decoder 2110 and a multichannel decoder 2160, and the core decoder 2110 includes a bitstream parser 2120, an envelope decoder 2130, and a bit. As the allocator 2140 and the inverse quantizer 2150, the multichannel decoder 2160 may include an upmixer 2150 and an inverse transformer 2160. Each component may be integrated into at least one or more modules and implemented as at least one or more processors (not shown).

Referring to FIG. 21, the bitstream parser 2120 parses a bitstream transmitted through a network (not shown) to extract an encoded spectral envelope, a quantization index of a downmixed spectrum, and a spatial parameter.

The envelope decoder 2130 losslessly decodes the encoded spectral envelope provided from the bitstream parser 2120.

The bit allocator 2140 is used for bit allocation required to decode the transform coefficient using the encoded spectral envelope provided from the bitstream parser 2120 in units of frequency bands. The bit allocator 2140 may operate in the same manner as the bit allocator 2060 of the audio encoding apparatus 2000.

The inverse quantization unit 2150 performs inverse quantization on the quantization index of the downmixed spectrum provided from the bitstream parser 2120 based on the bit allocation information provided from the bit allocation unit 2140, thereby spectrum of the MDCT component. Generate data.

The upmixer 2170 performs upmixing on the spectral data of the MDCT component provided from the inverse quantizer 210 by using the spatial parameter provided from the bitstream parser 2120, and the envelope decoder 2130. Denormalization is performed using the decoded spectral envelope provided by.

The inverse transform unit 2180 performs inverse transform on the upmixed spectrum provided from the upmixing unit 2170 to generate the PCM output of the time domain. In this case, an inverse MODFT may be applied to correspond to the converter (2020 of FIG. 20). To this end, spectral data of the MDST component may be generated or predicted from the spectral data of the MDCT component. Inverse MODFT may be applied by generating spectral data of the MODFT component using the spectral data of the MDCT component and the spectral data of the generated or predicted MDST component. The inverse transform unit 2180 may apply an inverse MDCT to the spectral data of the MDCT component. To this end, a parameter for compensating for an error occurring when upmixing is performed in the MDCT domain may be transmitted from the audio encoding apparatus 2000 of FIG. 20.

According to an embodiment, the multi-channel decoding may be performed in the MDCT domain for the static signal interval. Meanwhile, in a non-static signal section, for example, a transient signal section, a MODFT component may be generated by generating or predicting an MDST component from an MDCT component, and then multichannel decoding may be performed in the MODFT domain.

Meanwhile, whether the current signal corresponds to a static signal period or a non-static signal period may be checked using flag information or window information added to a bitstream in a predetermined frequency band or frame unit. For example, it may correspond to a non-static signal section when a short window is applied and a static signal section when a long window is applied.

More specifically, when the enhancement AC-3 algorithm is applied to the core codec, blksw and AHT flag information are used, and when the AC-3 algorithm is applied, the characteristics of the current signal can be checked by using the blksw flag information. .

According to FIGS. 20 and 21, by using the MODFT for time / frequency domain conversion, the complexity of the decoding stage can be reduced even when integrating a multi-channel codec and a core codec using different converter methods. In addition, even when integrating a multi-channel codec and a core codec using different converter methods, an existing synthesis filter bank part and a conversion part are unnecessary, so that an overlap add can be omitted, and thus no additional delay occurs.

The method according to the embodiments can be written in a computer executable program and can be implemented in a general-purpose digital computer operating the program using a computer readable recording medium. In addition, data structures, program instructions, or data files that can be used in the above-described embodiments of the present invention can be recorded on a computer-readable recording medium through various means. The computer-readable recording medium may include all kinds of storage devices in which data that can be read by a computer system is stored. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, floppy disks, and the like. Such as magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. The computer-readable recording medium may also be a transmission medium for transmitting a signal specifying a program command, a data structure, or the like. Examples of program instructions may include machine language code such as those produced by a compiler, as well as high level language code that may be executed by a computer using an interpreter or the like.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is clearly understood that the same is by way of illustration and example only and is not to be construed as limiting the scope of the invention as defined by the appended claims. Various modifications and variations are possible in light of the above teachings. Therefore, the scope of the present invention is shown in the claims rather than the foregoing description, and all equivalent or equivalent modifications thereof will be within the scope of the present invention.

Claims (1)

When downmixing the first plurality of input channels to the second plurality of output channels, comparing the positions of the first plurality of input channels with the positions of the second plurality of output channels;
Downmixing a channel having the same position as the second plurality of output channels among the first plurality of input channels to a channel having the same position among the second plurality of output channels;
Searching for at least one adjacent channel with respect to the remaining channels of the first plurality of input channels;
Determining a weight with respect to the found adjacent channel in consideration of at least one of a distance between channels, a correlation of a signal, and an error in reconstruction; And
And downmixing the remaining channels of the first plurality of input channels to the adjacent channel based on the determined weight.

Images