WO2015012594A1

WO2015012594A1 - Method and decoder for decoding multi-channel audio signal by using reverberation signal

Info

Publication number: WO2015012594A1
Application number: PCT/KR2014/006705
Authority: WO
Inventors: 백승권; 성종모; 이태진; 강경옥; 서정일; 김진웅; 장대영
Original assignee: 한국전자통신연구원
Priority date: 2013-07-23
Filing date: 2014-07-23
Publication date: 2015-01-29

Abstract

A method and a decoder for decoding a multi-channel audio signal by using a reverberation signal are disclosed. The decoding method, according to one embodiment, enables the quality of a multi-channel audio signal to be maintained by applying different reverberation signals in element units for the multi-channel audio signal.

Description

Method and decoder for decoding multichannel audio signals using reverberation signals

The following embodiments relate to a method and a decoder for decoding a multichannel audio signal, and more particularly, to a method and an apparatus for decoding a multichannel audio signal by applying different reverberation signals for each channel.

The decoding of the multichannel audio signal is performed using various additional information generated by the encoder. When the multi-channel audio signal is composed of N channels of audio signals, the encoder may compress the M channels into audio signals of M channels. The decoder can then recover the N channel audio signal from the M channel audio signal.

In this case, the decoder for the multi-channel audio signal may restore the N-channel audio signal by applying a reverberation component to the M-channel audio signal. At this time, if the same reverberation component is applied to the audio signals of the M channels, the sound field feeling of the audio signals of the N channels restored may be degraded.

First of all, in the case of a multichannel audio signal, sound quality is considered more important than compression efficiency, so that the deterioration of sound quality can be reduced compared to the audio signal before the original encoding by reconstructing the multichannel audio signal in consideration of the difference between a plurality of channels. A solution is required.

The following embodiments provide a method and apparatus for maintaining a sound field of a multichannel audio signal by generating and applying different reverberation signals for each channel when reconstructing the multichannel audio signal.

A decoding method performed by a decoder according to an embodiment of the present invention includes decoding an input audio signal of a core band of M channels; Extending a frequency band using a core band of the decoded input audio signal; And generating an output audio signal of N channels by upmixing the input audio signal based on the input audio signal having the extended frequency band and the reverberation signal different for each M channel.

In the extending of the frequency band by using the core band of the decoded input audio signal, the frequency band of the input audio signal may be extended by copying the core band to a high frequency band.

The reverberation signal may have uncorrelation as orthogonal to the input audio signal.

The input audio signal of the core band of the M channels may be processed in the MPS 2-1-2 mode and include an input audio signal of the core band of the M channels divided into elements corresponding to channel pair elements.

The generating of the N-channel output audio signal may generate a different reverberation signal for each element based on filter coefficients differently determined for a plurality of elements for dividing the input audio signal of the core band of M channels. .

The element is divided for each of the M channels and serves as a criterion for grouping subbands for allocating filter coefficients, and the filter coefficients may have different values for each element and for each grouped subband.

The generating of the N audio channels may include sequentially selecting a filter coefficient set according to an element corresponding to the input audio channel, and performing an all pass filter according to the selected filter coefficient set. Thus, different reverberation signals can be generated for each element.

The core band may correspond to a low frequency band in which the frequency band is not extended in the encoder.

The decoding of the input audio signal may decode the input audio signal by determining whether the input audio signal of the core band is an audio characteristic or a voice characteristic for each frame.

The generating of the N channel output audio signal may generate the N channel output audio signal by upmixing the M channel core audio signal according to the MPS 2-1-2 mode.

A decoder according to an embodiment of the present invention includes a core band decoding unit for decoding an input audio signal of a core band of M channels; A frequency band extension unit for extending a frequency band by using a core band of the decoded input audio signal; And a stereo upmixing unit configured to generate the N-channel output audio signal by upmixing the input audio signal based on the input audio signal having the extended frequency band and the reverberation signal different for each of the M channels.

The frequency band extension unit may expand the frequency band of the input audio signal by copying the core band to a high frequency band.

The stereo upmixing unit may generate different reverberation signals for each element based on filter coefficients that are differently determined for each of the plurality of elements for dividing the input audio signal of the core band of M channels.

The stereo upmixing unit sequentially selects filter coefficient sets according to elements corresponding to the input audio channel and performs an all pass filter according to the selected filter coefficient set to generate different reverberation signals for each element. Can be generated.

The core band decoding unit may decode the input audio signal by determining whether the input audio signal of the core band is an audio characteristic or a voice characteristic for each frame.

The stereo upmixer may generate a stereo output audio signal by upmixing an input audio signal in a mono form according to the MPS 2-1-2 mode.

According to an embodiment of the present invention, when reconstructing a multichannel audio signal, a different reverberation signal is generated and applied to each channel to maintain a sound field of the multichannel audio signal.

1 is a diagram illustrating a detailed configuration of a decoder for decoding an input audio signal of one channel according to an embodiment.

2 illustrates a detailed configuration of a decoder for decoding an input audio signal of N channels according to an embodiment.

3 is a diagram for describing an operation of a stereo upmixing unit for processing an input audio signal of one channel according to an exemplary embodiment.

FIG. 4 is a diagram for describing an operation of a stereo upmixing unit for processing input audio signals of N channels according to an embodiment.

5 illustrates a process of generating a reverberation signal in element units according to an exemplary embodiment.

6 illustrates a process of processing a bitstream according to an embodiment.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Referring to FIG. 1, the decoder 100 may include a core band decoding unit 101, a frequency band expansion unit 102, and a stereo upmixing unit 103. For example, the decoder 100 may process an audio signal having a bit rate in the range of 8 kbps to 192 kbps as a decoder based on a Unified Speech Audio Codec (USAC).

The core band decoding unit 101 may decode the bit rate of the input audio signal included in the bitstream. In particular, the core band decoding unit 101 may decode the input audio signal of the core band corresponding to the low frequency band to be encoded in the encoder. In other words, the core band is a frequency band used for encoding in the entire frequency band of the input audio signal, and means a low frequency band other than the high frequency band generated as the frequency band is expanded in the decoder.

The core band decoding unit 101 may use different decoding schemes according to whether each frame of the input audio signal has an audio characteristic or a voice characteristic. For example, when the input audio signal has an audio characteristic, the core band decoding unit 101 may decode the core band of the input audio signal based on a Modified Discrete Cosine Transform (MDCT) scheme. When the input audio signal has a voice characteristic, the core band decoding unit 101 may decode a core band of the input audio signal based on an ACELP (Algebraic code-excited linear prediction) scheme. Since the decoding process is performed in units of frames, when the frame units have different characteristics, the core band decoding unit 101 may switch and process the decoding processes between frames.

The frequency band extension unit 102 may extend the frequency band of the input audio signal by copying the core band to the high frequency band with respect to the input audio signal of the decoded core band. That is, the frequency band extension unit 102 may expand the frequency band by using parameter information of SBR (Spectral Band Replication). Specifically, since the output result of the core band decoding unit 101 is a result of restoring only the low frequency band which is the core band, the high frequency band may be restored by copying the core band to restore the entire frequency band of the input audio signal. Then, the frequency band of the input audio signal can be extended from the core band to the entire frequency band.

The stereo upmixing unit 103 may upmix the input audio signal having the extended frequency band to generate a stereo audio output audio signal corresponding to two channels. The stereo upmixing unit 103 may generate a stereo output audio signal by upmixing an input audio signal in a mono form. In this case, the stereo upmixing unit 103 may operate in the QMF domain. Here, the input audio signal having the extended frequency band may be input to the stereo upmixing unit 103 as a downmix signal. Then, as illustrated in FIG. 1, an output audio signal corresponding to an L (left) channel and an R (right) channel may be generated.

The operation of the stereo upmixing unit 103 may be performed according to the MPS (MPEG Surround) 2-1-2 mode. Here, in the MPS 2-1-2 mode, the stereo audio signal is downmixed by the encoder and output as a mono audio signal, and the mono audio signal is upmixed by the decoder and restored to the stereo audio signal. it means. In order to operate in the MPS 2-1-2 mode, MPS information for upmixing may be needed.

1 illustrates one decoding unit of a plurality of decoding units included in a decoder for processing a multichannel audio signal of the present invention. In FIG. 1, the positions of the frequency band extension unit 102 and the stereo upmixing unit 103 may be interchanged. In other words, as shown in FIG. 1, the output result of the frequency band extension unit 102 is input to the stereo upmixing unit 103, or unlike the output result of the stereo upmixing unit 103 as shown in FIG. 1, the frequency band extension unit 102 is output. ) Can be entered.

The output audio signal of two channels may be generated by one decoder unit. In FIG. 2, an operation of all decoding units included in the decoder will be described in detail. If the number of channels of the multi-channel audio signal to be finally restored is N, the number of decoders needs to be N / 2 in total, which is equal to M, which is the number of channels of the input audio channel.

2 is a diagram illustrating a detailed configuration of a decoder for decoding an input audio signal of M channels according to an embodiment.

Referring to FIG. 2, the decoder 200 may include M core band decoding units 201, a frequency band extension unit 202, and a stereo upmixing unit 203 to process M channel input audio signals. Can be. As described above, one core band decoding unit 201, the frequency band extension unit 202, and the stereo upmixing unit 203 constitute one decoding unit. That is, the decoder 200 is composed of M decoding units, and thus N (2M) output audio signals may be generated.

The decoding process performed by each decoding unit is the same as that described with reference to FIG. 1. In FIG. 2, the downmix signal DMX signal derived from the M frequency band extension units 202 may be d ₀ (n) to d _M-1 (n).

Referring to FIG. 2, each downmix signal DMX signal may be upmixed by the stereo upmixing unit 203 into a stereo audio output audio signal. In detail, the downmix signal d ₀ (n) may be upmixed by the stereo upmixing unit 203 into y ₀ (n) and y ₁ (n), which are stereo output audio signals.

Referring to FIG. 3, the stereo upmixing unit 300 may include an uncorrelated unit 301 and an upmixing unit 302.

The decorrelator 301 may generate a wet signal having uncorrelation with the downmix signal DMX having an extended frequency band. Here, the Wet signal is orthogonal to the downmix signal. The upmixing unit 302 may generate a stereo audio output audio signal by upmixing the downmix signal and the wet signal. In this case, the downmix signal and the wet signal may be applied to the upmixing matrix. The output audio signal may include an L channel audio signal and an R channel audio signal.

Here, the downmix signal may have a primary attribute of the input audio signal, and the wet signal may have a reverberation attribute of the input audio signal. Hereinafter, the wet signal is defined as a reverberation signal.

In FIG. 3, coefficients constituting the upmixing matrix may be calculated by additional information transmitted from the encoder. In the case of FIG. 3, the stereo upmixing unit 300 may perform upmixing according to the MPS 2-1-2 mode. For example, the stereo upmixing unit 300 may perform upmixing according to Equation 1 below.

H _LL and H _LR are coefficients for adjusting the ratio of the downmix signal and the reverberation signal to generate the output audio signal of the L channel. And, H H _RL and _RR refers to the down-mix signal and the coefficient for adjusting the ratio of the reverberation signal to produce an output audio signal of the R channel.

According to an embodiment of the present invention, a multi-channel audio signal may be decoded using a decoder implemented in the USAC method as shown in FIG. 2. In particular, the multi-channel audio signal can be decoded more effectively when it has a low bit rate.

The decoder implemented in the USAC method can process a mono input audio signal or a stereo input audio signal. Thus, a multichannel audio signal composed of a plurality of channels needs to be decoded by dividing into elements corresponding to mono or stereo.

For example, in order to process an input audio signal of 5.1 channels, a bitstream including the following elements is required.

UsacSingleChannelElement (): mono channel coding

UsacChannelPairElement (): stereo channel coding

UsacLfeElement (): Lfe (Low Frequency Effect) channel coding

That is, the decoder described in the present invention can decode the input audio signal by dividing the 5.1-channel input audio signal into a plurality of elements. Here, the input audio signal corresponding to an element such as UsacChannelPairElement () may be processed according to a stereo coding scheme, and thus may be processed by the stereo upmixing unit described with reference to FIGS. 1 to 3.

If 10 multichannel audio signals are divided into UsacChannelPairElement () and decoded at a low bit rate, the multichannel audio signals may be divided into the following elements. UsacChannelPairElement () means to upmix the input audio signal of one channel in stereo form to produce the output audio signal of two channels.

UsacChannelPairElement (): stereo channel coding by MPS 2-1-2 mode

In this case, when upmixing an input audio signal corresponding to each element, a stereo upmixing unit 400 classified according to an element may be used as shown in FIG. 4. In addition, the same upmixing unit 402 may be applied to each element. However, different decorating units 401 may be applied to each element. That is, the stereo upmixing unit 400 may maintain the sound quality of the multichannel audio signal by maintaining the difference between the channels constituting the multichannel audio signal by upmixing using different reverberation signals for each element.

Referring to FIG. 4, the downmix signal having the extended frequency band may include d _M-1 (n) to d ₀ (n), respectively. Then, M reverberation signals may be generated by different uncorrelated units 401 allocated to the plurality of elements. Here, the element may be classified as K. In the case of FIG. 4, an input audio signal divided into elements from K = 0 to K = M-1 may be upmixed and processed.

For example, the uncorrelated unit 401 D ₀ may generate wet ₀ (n) that is a reverberation signal using the downmix signal d ₀ (n). Similarly, the uncorrelated unit 401 D ₁ may generate wet ₁ (n), which is a reverberation signal, using the downmix signal d ₁ (n). The non-correlated units 401 to D ₀ to D _M-1 may have different filter characteristics for generating a reverberation signal. As such, wet _M-1 (n) may be generated in the _M reverberation signals wet ₀ (n) from the downmix signals d ₀ (n) to d _M-1 (n). As such, since different reverberation signals are generated for each element, the output audio signal generated by upmixing from the input audio signal also has different reverberation effects, and thus an output audio signal effectively reflecting the acoustic characteristics between channels can be generated. have.

As described above, the element allocated to the input audio signal may be assigned an index k. The process of generating the reverberation signal by each uncorrelated unit 401 may be performed by Equation 2.

Equation 2 refers to a basic formula to apply the All-Pass Filter. Where the filter coefficient is

Wow

to be. Index k means the grouping index for the Quadrature Mirror Filter (QMF) band. Subbands may be defined in the range of 0≤Subband≤70. Filter coefficients may be allocated to each subband grouped by index k. The number of subbands can be a total of 71 including the Hybrid QMF band. Since the filter coefficients are usually defined in four groups, the index k may have a value of 0 ≦ k ≦ 3. For example, if index k = 0, subbands 0-7, index k = 1 subbands 8-20, index k = 2, 21-29 subbands, and index k = 3, 30- 70 subbands may correspond.

In other words,

Wow

May be determined for each grouped subband and for each element. As shown in FIG. 4, when the plurality of stereo upmixing units 400 upmixes the input audio signals of M channels,

Wow

May be determined for each grouped subband and for each element. In Equation 2, Delay _K is a different delay time applied to each subband group, and filtering may be performed after a predetermined delay time is applied to each subband.

Wow

May be expressed as the coefficient of the All-Pass Filter or as one reflection coefficient. Specifically,

Wow

Is one

It can be expressed as.

If you are doing stereo upmixing on one element,

Is the same as Equation 3.

Explained in Equation 3

from

Wow

Can be extracted.

As shown in FIG. 4, when the input audio signal corresponds to M channels and there are M elements, a reverberation signal may be generated in element units as shown in FIG. 5.

The element displayed before switching in FIG. 5 is an index indicating the number of elements of the downmix signal input to the stereo upmixing unit. Then, the filter coefficient selector 501 may select a set of filter coefficients by switching according to the index element assigned to the downmix signal. The filtering unit 502 is converted from the set of the selected filter coefficients

Wow

All pass filtering can be performed using. Here, All Pass Filtering means to perform Equation 2 described above.

6 illustrates a process of processing a bitstream according to an embodiment.

The decoder may determine the number of elements to be processed from the bitstream. The element refers to a unit for distinguishing an input audio signal. At this time, in step 601, the decoder may determine whether there is one element or a plurality of elements. If there is only one element, the decoder may process the input audio signal according to the method 2.

Conversely, if there are a plurality of elements, in step 602, the decoder may determine whether there are a plurality of channel pair elements among the plurality of elements. If the channel pair elements are not plural, the decoder may process the input audio signal according to the method 2.

On the other hand, if there are a plurality of channel pair elements, in step 603, the decoder may determine whether there are a plurality of channel pair elements processed in the MPS 2-1-2 mode. Here, the process of determining whether it is in the MPS 2-1-2 mode may be based on syntax included in the bitstream. For example, when stereoConfigIndex == 1 in the syntax, the decoder may upmix the input audio signal through the stereo upmixer operating according to the MPS 2-1-2 mode.

If there are not a plurality of channel pair elements processed in the MPS 2-1-2 mode, the decoder may process the input audio signal according to the method 2. If there are a plurality of channel pair elements processed in the MPS 2-1-2 mode, the decoder may process the input audio signal according to the first method.

Here, method 1 means a process of extracting filter coefficients according to the process described with reference to FIG. 5. In this case, it is assumed that there are M elements in FIG. 5. This means that there are M input audio signals corresponding to the channel pair elements processed in the MPS 2-1-2 mode in all the input audio signals included in the bitstream.

In addition, in FIG. 6, method 2 means that the input audio signal is processed in a manner other than the decoding method described in the present invention.

6 may be implemented by the following syntax. Here, numElements means the number of elements, and elementLength corresponds to LEN of Equation 3. The USAC CPE corresponds to the channel pair element of FIG. 6.

On the other hand, if the element is a USAC CPE, the decoder can operate specifically by the following syntax.

Here, stereoConfigIndex indicates whether to operate in the MPS 2-1-2 mode. If stereoConfigIndex == 1, the stereo upmixing unit included in the decoder may upmix the input audio signal according to the MPS 2-1-2 mode.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments may be, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable arrays (FPAs), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved. Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims

In the decoding method performed by the decoder,

Decoding an input audio signal of a core band of M channels;

Extending a frequency band using a core band of the decoded input audio signal;

Generating N output audio signals by upmixing the input audio signal based on the input audio signal having the extended frequency band and the reverberation signal different for each M channel.

Decoding method comprising a.
The method of claim 1,

Extending a frequency band by using a core band of the decoded input audio signal,

And decoding a frequency band of an input audio signal by copying the core band to a high frequency band.
The method of claim 1,

The reverberation signal is,

And a decoupling as orthogonal to each other with the input audio signal.
The method of claim 1,

The input audio signal of the core band of the M channels,

A decoding method comprising an input audio signal of a core band of M channels processed in MPS 2-1-2 mode and divided into elements corresponding to channel pair elements.
The method of claim 1,

Generating the output audio signal of the N channels,

A decoding method of generating different reverberation signals for each element based on filter coefficients that are differently determined for a plurality of elements for dividing the input audio signal of the core band of M channels.
The method of claim 5,

The element is divided for each of the M channels and serves as a criterion for grouping subbands for allocating filter coefficients.

And the filter coefficients have different values for each element and for each grouped subband.
The method of claim 1,

Generating the output audio signal of the N channels,

And a filter coefficient set is sequentially selected according to an element corresponding to the input audio channel, and a different reverberation signal is generated for each element by performing an all pass filter according to the selected filter coefficient set.
The method of claim 1,

The core band,

A decoding method corresponding to a low frequency band in which the frequency band is not extended in the encoder.
The method of claim 1,

Decoding the input audio signal,

And decoding the input audio signal by determining whether the input audio signal of the core band is an audio characteristic or a voice characteristic for each frame.
The method of claim 1,

Generating the output audio signal of the N channels,

A decoding method of generating an output audio signal of N channels by upmixing an input audio signal of M channel core bands according to the MPS 2-1-2 mode.
A core band decoding unit for decoding an input audio signal of a core band of M channels;

A frequency band extension unit for extending a frequency band by using a core band of the decoded input audio signal; And

Stereo upmixing unit for generating an output audio signal of N channels by upmixing the input audio signal based on the input audio signal having the extended frequency band and different reverberation signals for every M channels.

Decoder comprising a.
The method of claim 11,

The frequency band extension unit,

A decoder that extends the frequency band of the input audio signal by copying the core band into a high frequency band.
The method of claim 11,

The reverberation signal is,

And a decoder having uncorrelation as orthogonal to the input audio signal.
The method of claim 11,

The input audio signal of the core band of the M channels,

A decoder comprising an input audio signal of a core band of M channels processed in MPS 2-1-2 mode and divided into elements corresponding to channel pair elements.
The method of claim 11,

The stereo upmixing unit,

A decoder for generating different reverberation signals for each element based on filter coefficients differently determined for a plurality of elements for dividing the input audio signal of the core band of M channels.
The method of claim 15,

The element is divided for each of the M channels and serves as a criterion for grouping subbands for allocating filter coefficients.

The filter coefficients have a different value for each subband grouped for each element.
The method of claim 11,

The stereo upmixing unit,

And a filter coefficient set is sequentially selected according to an element corresponding to the input audio channel, and a different reverberation signal is generated for each element by performing an all pass filter according to the selected filter coefficient set.
The method of claim 11,

The core band,

A decoder corresponding to a low frequency band in which the frequency band is not extended in the encoder.
The method of claim 11,

The core band decoding unit,

And a decoder to decode the input audio signal by determining whether the input audio signal of the core band is an audio characteristic or a voice characteristic for each frame.
The method of claim 11,

The stereo upmixing unit,

A decoder for upmixing an input audio signal in mono form according to MPS 2-1-2 mode to produce a stereo output audio signal.