KR20060122695A - Method and apparatus for decoding audio signal - Google Patents

Abstract

A method and an apparatus for decoding an audio signal are provided to produce to extract the spatial information of a multi-channel and produce the multi-channel. An encoding device(1) includes a spatial encoder(10). A core encoder(20) encodes a downmix audio signal. A multiplexer(30) performs the multiplexing of the encoded downmix audio signal and the spatial information. When an audio signal is inputted through the number of N of multi-channel, a downmix member(11) performs the downmix of the inputted audio signal and produces the downmix channel. The core encoder performs encoding of the received downmix audio signal and inputs the encoded downmix audio signal to the multiplexer. A spatial information extracting member(12) extracts the spatial information from the multichannel and transmits the extracted spatial information to the multiplexer.

Description

Method and apparatus for decoding audio signal {Method and Apparatus for decoding audio signal}

1 illustrates an embodiment of an apparatus for encoding and decoding a signal according to the present invention;

2 illustrates an audio bitstream structure according to the present invention.

3 is a diagram illustrating an example of a decoding method and apparatus according to the present invention.

4 illustrates a first embodiment of a virtual surround rendering process and a transformation process of spatial information according to the present invention.

5 illustrates a second embodiment of a virtual surround rendering process and a transformation process of spatial information according to the present invention.

6A and 6B illustrate an example of a channel mapping process according to the present invention.

7 is a diagram illustrating an example of filter coefficients for each channel according to the present invention.

8 to 10 illustrate in detail the process for generating filter coefficients according to the present invention, for example.

* Explanation of symbols for main parts of the drawings

1: encoding device 2: decoding device

10: spatial encoder 11: downmix unit

12: spatial information extraction unit 20: core encoder

30: multiplexer 40,300: demultiplexer

50, 310: Spatial information converter 60, 320: Audio decoder

311: Spatial information decoder 312, 400, 500: Spatial information converter

321: core decoder 322, 510: virtual surround renderer

401, 501: channel mapping unit

402, 502, 800, 900, 1000: coefficient generator

403, 503, 810, 910, 1010: Synthesizer / Domain Converter

411, 412, 511: Domain Converter 413: Renderer L

414: Renderer R 416, 417: Adder

417, 418, 513, 514: reverse domain converter

512: Renderer M

601, 602, 603, 604, 605, 611, 612, 613, 614, 615: OTT box

811, 912, 1012: Synthesizer 812, 911, 1000: Interpolator

813, 913, 1013: Domain Converter

The present invention relates to a method and apparatus for decoding an audio signal, and more particularly, to an effective method for generating a virtual surround signal from a received audio signal in processing an audio signal.

The standard for digital video and digital audio is the standard for compression and reconstruction for each signal. In addition, the standard for digital systems divides each compressed video and audio into packets of a certain size, and then multiplexes and transmits timing information and stream-related information, and vice versa. It is a standard necessary to obtain information and separate separate compressed video and audio.

Recently, various coding techniques and methods for digital audio signals have been developed, and related products have been produced. In addition, coding methods for multichannel audio signals have been developed using a psychoacoustic model, and standardization thereof has been performed.

The psychoacoustic model takes unnecessary parts in the coding process by using a method in which a human recognizes a sound, for example, a small sound following a loud sound and only a sound corresponding to a frequency of 20 Hz to 20000 Hz. By removing the signal for, we can effectively reduce the amount of data needed.

In addition, audio standard technologies such as MPEG-1 audio, MPEG-4 advanced audio coding (AAC), and MPEG-4 high-efficiency AAC (HE-AAC) have been developed and commercialized.

However, a method of decoding an audio signal for generating a virtual surround signal in an audio bitstream including spatial information has not been specifically described, and there are many problems in efficiently processing an audio signal.

SUMMARY OF THE INVENTION The present invention has been made to solve the above problems, and an object thereof is to provide a method and apparatus for decoding an audio signal that provides a virtual stereophonic effect in an audio system.

In order to achieve the above object, the present invention comprises the steps of (a) demultiplexing the received bitstream into a core codec bitstream and a spatial information bitstream; (b) decoding the core codec bitstream to produce a decoded downmix signal; (c) decoding the spatial information bitstream to generate spatial information; And (d) generating a virtual surround signal from the decoded downmix signal using the spatial information.

In addition, the present invention includes demultiplexing a received bitstream and generating downmixed signal and spatial information in the demultiplexed bitstream; Generating filter coefficients using the spatial information and the filter information; And generating a virtual surround signal from the downmixed signal using the filter coefficients.

The present invention also provides a demultiplexer for demultiplexing a received bitstream into a core codec bitstream and a spatial information bitstream; A spatial information converter which decodes the spatial information bitstream to generate spatial information, and converts the spatial information to generate a virtual surround signal; And an audio decoder which decodes the core codec bitstream to generate a decoded downmix signal, and generates a virtual surround signal from the decoded downmix signal using the converted spatial information. An apparatus for decoding an audio signal is provided.

Therefore, according to the present invention, a decoding apparatus that receives a audio bitstream generated by downmixing a multichannel to generate a downmix channel and extracting spatial information of the multichannel is not an environment in which a multichannel can be generated. Even if it is possible to decode to have a virtual surround effect.

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can be specifically realized for the above purpose.

In addition, the terms used in the present invention was selected as a general term widely used as possible now, but in some cases, the term is arbitrarily selected by the applicant, in which case the meaning is described in detail in the description of the invention, It is to be understood that the present invention is to be understood as the meaning of terms rather than the names of terms.

In the present invention, " spatial information " refers to down-mixing multichannels in an encoding apparatus and receiving signals transmitted from a decoding apparatus to perform upmixing to generate multichannels. Means information required for Although the spatial information is described based on the spatial parameters, it is to be understood that the present invention is not limited thereto. In this regard, the spatial parameter is used when generating three channels from two channels and an inter channel coherence (ICC) representing a channel level difference (CLD) representing an energy difference between two channels and a correlation between two channels. Channel prediction coefficients (CPCs), which are prediction coefficients.

In the present invention, "core codec" refers to a codec for coding an audio signal rather than spatial information, that is, a spatial parameter. In the present invention, a downmix audio signal is described as an audio signal instead of the spatial information. In addition, the core codec may include MP3, AC-3, DTS, or AAC, and may also be the case of a PCM signal without compression. If the codec function is performed on the audio signal, it may include a codec to be developed in the future as well as a codec previously developed.

In the present invention, a "channel splitting unit" means a splitting unit that divides a specific number of input channels into a specific output channel number different from the input channel number. For example, the channel dividing unit converts an output channel into three when two input channels are provided, or one input channel based on a TTT box or a TTT box. In this case, the OTT (one to two: OTT) section or OTT box for converting two output channels can be described. However, the channel dividing unit of the present invention is not limited to the TTT unit and the OTT unit, and it is apparent that both of the input channels and the output channels can be applied when the arbitrary number is obvious.

1 illustrates an embodiment of an apparatus for encoding and decoding a signal according to the present invention. For example, a diagram for describing an encoding device and a decoding device of an audio signal in MPEG surround.

Referring to FIG. 1, there is shown an encoding device 1 and a decoding device 2 for the process of audio signal processing. However, the present invention will be described with respect to the audio signal, but the present invention is applicable to the processing of all signals in addition to the audio signal.

The encoding apparatus 1 includes a spatial encoder: 10 including a downmix unit 11 and a spatial parameter estimation unit 12, and encodes a downmix audio signal. A core encoder 20 and a multiplexer 30 for multiplexing the encoded downmix audio signal and spatial information are included.

,

,...,

)로 입력되면, 다운믹스부(11)는 미리 정해진 다운믹스 정보 또는 외부 제어 명령에 따라 특정 개수의 채널로 입력된 오디오 신호의 다운믹스를 수행하여 다운믹스 채널을 생성하고, 상기 다운믹스 채널로 다운믹스 오디오 신호를 출력하면, 상기 출력된 신호는 코어 인코더(20)로 입력한다.The audio signal is N multichannel (

,

, ...,

), The downmix unit 11 generates a downmix channel by performing downmixing of an audio signal input to a specific number of channels according to predetermined downmix information or an external control command and generating a downmix channel. When the downmix audio signal is output, the output signal is input to the core encoder 20.

The downmix channel may have one channel or two channels, or may have a specific number of channels according to a downmix command. At this time, the number of downmix channels can be set. Further, it is noted that the downmix audio signal may optionally use a downmix audio signal provided directly from the outside, that is, an artistic downmix signal.

The core encoder 20 receives a downmix audio signal transmitted through a downmix channel and encodes the received downmix audio signal. The encoded downmix audio signal is input to a multiplexer 30.

The spatial information extracting unit 12 extracts spatial information from the multichannel, and transmits the extracted spatial information to the multiplexing unit 30.

The multiplexer 30 receives the downmixed audio signal encoded by the core encoder 20, and the spatial information extractor 12 receives spatial information. The encoded downmix audio signal and the spatial information are multiplexed to generate a bitstream and transmit the generated bitstream to the decoding apparatus 2. In this case, the bitstream includes a core codec bitstream and a spatial information bitstream, and the bitstream will be described with reference to FIG. 2.

The decoding device 2 is characterized in that it comprises a demultiplexer 40, a spatial information converter 50, and an audio decoder 60.

The demultiplexer 40 receives the bitstream and demultiplexes the received bitstream into a core codec bitstream and a spatial information bitstream.

The spatial information converter 50 receives the spatial information bitstream from the demultiplexer 40, decodes the received spatial information bitstream, generates spatial information, and receives filter information from the outside. Also, to generate a virtual surround signal using the generated spatial information and the received filter information, the spatial information is converted into a form that can be applied to an output signal (for example, a stereo output). . As a result, the present invention has the feature of decoding in virtual surround together with stereo decoding. Here, although an example of the filter information is described based on a head-related transfer functions (HRTF) parameter, it is obvious that the present invention is not limited thereto.

The audio decoder 60 receives the core codec bitstream from the demultiplexer 40 to generate a decoded downmix signal, and the spatial information received by the generated decoded downmix signal and the spatial information converter 50. Generates and outputs a virtual surround signal using. Here, the spatial information is found to be spatial information converted for generation of a virtual surround signal. For example, the virtual surround signal is a signal that provides a virtual stereophonic effect in an audio system having only a stereo device. In this case, it is apparent that the present invention is applicable to an audio system having only a device in which the output signal is stereo. In addition, the rendering performed by the audio decoder 60 may be applied in various forms according to a set mode.

The spatial information converter 50 and the audio decoder 60 will be described in detail with reference to FIGS. 3 to 10.

As described above, in the present invention, instead of directly transmitting a multichannel audio signal in the encoding apparatus 1, downmixing and transmitting a stereo or mono audio signal, and transmitting spatial information of the multichannel audio signal together, decoding If the device 2 includes the spatial information converting unit 50 and the audio decoder 60 according to the present invention, even if the output channel is not a multi-channel but is a mono or stereo channel, the user may apply a virtual 3D effect. It's a great way to experience it.

2 illustrates an audio bitstream structure according to the present invention. In this case, the audio bitstream may include spatial information.

Referring to FIG. 2, one frame of an audio payload includes a field including coded audio data and an ancillary data field. Here, the spatial information coded in the additional data field may be included. For example, when the audio payload is 48 to 128 kbps, the spatial information may have a range of about 5 to 32 kbps, but this is one example and there is no limitation thereto.

3 is a diagram illustrating an example of a decoding method and apparatus according to the present invention.

Referring to FIG. 3, the decoding apparatus may include a demultiplexer 300, a core decoder 321, and a virtual surround renderer that demultiplexes a received bitstream into a core codec bitstream and a spatial information bitstream. An audio decoder 320 including a renderer: 322, a spatial information decoder 311, and a spatial information converter 310 including a spatial information converter 312 are included.

The spatial information decoder 311 receives the spatial information bitstream transmitted by the demultiplexer 300, and decodes the spatial information bitstream to generate spatial information. The spatial information converter 312 receives the filter information and the spatial information and converts the spatial information into a form for generating a virtual surround signal. The converted spatial information is output to the virtual surround renderer 322. For example, the transformed spatial information may be filter coefficients.

The core decoder 321 receives the core codec bitstream transmitted by the demultiplexer 300 and outputs a decoded downmix signal. For example, when downmixing multichannels in an encoding apparatus, when downmixing to a mono channel or a stereo channel, the decoded downmix signal may be a mono channel or a stereo channel. However, in the embodiment of the present invention, the downmix channel will be described based on a mono channel or a stereo channel, but the present invention is not limited thereto, and it is understood that the present invention can be applied regardless of the number of downmix channels.

The virtual surround renderer 322 receives the decoded downmix signal transmitted by the core decoder 321, and receives the transformed spatial information transmitted by the spatial information converter 312 to virtually render the decoded downmix signal. To output the virtual surround signal. For example, when the output unit is a stereo channel, the virtual surround signal is a pseudo-surround stereo output having virtual stereo sound. Here, the case where the output unit is a stereo channel will be described by way of example, but the present invention is applicable regardless of the number of channels of the output unit.

For example, spatial information of MPEG surround includes CLD, CPC, ICC, and the like. Among these, the CLD represents a relative level difference value between the channel and the channel. In the encoding process, the downmixing process from the multichannel to the downmix channel (for example, one channel or two channels) is performed in multiple steps using a channel converter (for example, an OTT box or a TTT box). In this case, the spatial information is generated for each channel converter in each step. The decoding apparatus may decode the spatial information and generate a signal having virtual surround by applying spatial information to a downmix signal using the decoded spatial information. The present invention is characterized by extracting only spatial information for each stage and performing virtual surround rendering using the extracted spatial information, rather than spatially upmixing the downmix signal.

For example, a virtual surround rendering method includes HRTF (head-related transfer functions) (hereinafter referred to as 'HRTF') filtering. In this case, the spatial information is a value that can be applied in a hybrid filterbank domain defined in MPEG surround. In this regard, the virtual surround rendering method may be of the following types depending on the domain.

First, a hybrid filterbank is passed through a downmix signal, and virtual surround rendering is performed in a hybrid domain. There is no need to convert the domain to spatial information here.

The second method is to perform virtual surround rendering in the time domain. This method requires transforming spatial information into filter coefficients in the time domain by using the HRTF filter modeled as a FIR filter or an IIR filter in the time domain.

Third, virtual surround rendering is performed in different frequency domains. For example, a method of performing virtual surround rendering in a discrete fourier transform (DFT) domain. This method requires the conversion of spatial information to the domain. In addition, this method replaces filtering in the time domain with operations in the frequency domain, thereby enabling high-speed computation.

In the present invention, the filter coefficient means a coefficient of a specific filter. Examples of the filter coefficients include the following filter coefficients. Proto-type HRTF filter coefficients represent original filter coefficients of a specific HRTF filter, and may be expressed as GL_L or the like. The transformed HRTF filter coefficient refers to a filter coefficient after the circular HRTF filter coefficient is modified and may be expressed as GL_L '. The spatialized HRTF filter coefficients represent spatial filter coefficients obtained by spatializing the circular HRTF filter coefficients for generating a virtual surround signal, and may be expressed as FL_L1 or the like. The master rendering coefficients mean filter coefficients necessary for performing rendering, and can be expressed as HL_L. The interpolated / blurred master rendering coefficients refer to filter coefficients in which the master rendering coefficients are interpolated and / or blurred, and can be expressed as HL_L '. Here, it should be noted that the present invention is not limited to the names of the filter coefficients.

4 illustrates a first embodiment of a virtual surround rendering process and a transformation process of spatial information according to the present invention. In particular, the case where the decoded downmix signal input to the virtual surround renderer is stereo is taken as an example.

Depending on the domain to which virtual surround rendering is applied, domain conversion may vary. Since the domain for the virtual surround rendering of the downmix signal and the domain for spatial information must be matched domains, if both domains do not match, the domains must be converted to the matching domain.

The spatial information converter 400 includes a channel mapping unit 401, a coefficient generator 402, and an integrator / domain converter 403.

The channel mapping unit 401 performs channel mapping so that the input spatial information is mapped to each channel of the multichannel signal. A detailed description of the channel mapping process will be given below with reference to FIGS. 6A and 6B.

The coefficient generators 402_1, 402_2,..., 402_N of each channel of the coefficient generator 402 receive filter information through an external input, and receive channel-mapped spatial information from the channel mapping unit 402 to correspond to the corresponding channel. Generate coefficients for each channel that can virtually generate signals. For example, when the pseudo-surround renderer 410 is an HRTF filter, the coefficient generator 402 generates HRTF coefficients. A description of the coefficient for each channel will be described in detail with reference to FIG. 7.

The synthesizer / domain converter (403) receiving the coefficients for each channel integrates the coefficients for each channel for each downmix channel and converts them into domains for virtual surround rendering. Outputs In this case, the integration may be omitted as a process for reducing the amount of computation for virtual surround rendering. In addition, when the downmix signal is a stereo, a coefficient set to be applied to the left and right downmix signals is generated during coefficient generation for each channel. In this regard, a set of filter coefficients includes coefficients delivered from each channel to the magnetic channel and coefficients delivered to the relative channel.

That is, the spatial information converter 400 generates spatial coefficients transmitted to the magnetic channel of the virtual surround renderer 510 and coefficients transmitted to the relative channel by using the spatial information. For example, the spatial information converter 400 is input to the renderer L (413) and passes the coefficient (HL_L) to the left output (left out), which is the output of its own channel, and the right output (right out) which is the relative channel. Create a coefficient (HL_R) passing in In addition, the spatial information converter 400 is inputted to the renderer R (414) R (R R) to pass to the right output (right out) of its own channel output (HR_R), and to the left output (left out) of the relative channel. Generates the passing coefficient (HR_L).

The pseudo-surround renderer (410) is a domain converter (411,412), renderer L (413), renderer R (414), and adder (415, 415), and reverse domain. And an inverse domain converter (417, 418).

Domain converters 411 and 412 perform domain conversion to match both domains when the spatial information domain and the domain for performing virtual surround rendering are different.

Renderer L: 413 and Renderer R: 414 receive downmix signals on stereo channels and output left, right (integrator / domain converter: 403) outputs. right) receives a set of four filter coefficients to be applied to the downmix signal. The renderer L (413) and the renderer R (414r) are virtual surrounds in the downmix signal using four sets of filter coefficients (e.g., HL_L, HL_R, HR_L, HR_R). Perform rendering to generate the signal.

For example, the renderer L (413) is a filter coefficient set (HL_L) transmitted to a magnetic channel and a filter coefficient set (referred to) in a left channel, which is a set of filter coefficients. HL_R) is used for rendering. The renderer L (413) may be configured to include a renderer HL_L and a renderer HL_R. The renderer HL_L renders using a filter coefficient set HL_L which is passed to the left out which is the output of its own channel, and the renderer HL_R is the right output that is a relative channel. Rendering is performed using the filter coefficient set HL_R delivered to out).

In addition, the renderer R (414) is a filter coefficient set HR_R delivered to the magnetic channel and a filter coefficient set HR_L delivered to the relative channel from the right set which is a set of filter coefficients. Rendering is done using. The renderer R (414) may include a renderer HR_R and a renderer HR_L. The renderer HR_R renders using a filter coefficient set HR_R delivered to the right out, which is a channel output, and the renderer HR_L is a left channel, which is a relative channel. Rendering is performed using the filter coefficient set HR_L delivered to out). In this regard, the HL_R and HR_L are added to the relative channel at the adders 415 and 416 after cross term processing. In this case, the HL_R and HR_L may be 0 in some cases, which means that the coefficient of the cross term may have a value of zero. Here, when the HL_R and HR_L become 0, it means that no contribution is made to the corresponding path.

Inverse domain converters 417 and 418 perform domain inverse transform on the received signal to output a virtual surround stereo signal. At this time, the user can listen to the sound having a multi-channel effect, such as earphone having a stereo channel.

Hereinafter, when the input signal is a stereo downmix signal as shown in FIG. 4, the downmix signal is x, the channel-mapped coefficient is D, the external HRP is a circular HRTF filter coefficient, and the temporary multichannel signal is p. If the output signal is defined as y and expressed as a determinant, it is as follows.

,

,

,

,

,

Here, if each coefficient is a value in the frequency domain, it can be developed in the following form. First, the temporary multichannel signal may be represented by a product of channel-mapped coefficients and a downmix signal.

,

,

The temporary multichannel p is then rendered using the circular HRTF filter coefficient G as follows.

를 대입하여 y를 구할 수 있다.Where

You can find y by substituting for.

로 정의하면, 렌더링된 출력신호 y와 다운믹스 신호 x와는 다음의 관계를 갖는다.Where H

When defined as, the rendered output signal y and the downmix signal x have the following relationship.

,

,

Therefore, the product of the filter coefficients may be first processed to generate H, and then the downmix signal x may be multiplied to obtain y.

의 관계, 다음 식의 관계에 의해 얻을 수 있다. 상기 F계수는 이하 도 8에서 살펴보도록 한다.According to the above definition, the F coefficient to be described below is

Can be obtained by the following relationship. The F coefficient will be described below with reference to FIG. 8.

5 illustrates a second embodiment of a virtual surround rendering process and a transformation process of spatial information according to the present invention. In particular, the case where the decoded downmix signal input to the virtual surround renderer is mono is an example.

Referring to FIG. 5, the spatial information converter 500 includes a channel mapping unit 501, a coefficient generator 502, and an integrator / domain converter 503. Since the components of the spatial information converter 500 perform the same functions as the components of the spatial information converter 400 described with reference to FIG. 4, a detailed description thereof will be omitted herein. However, at this time, the spatial information converter 500 generates the final filter coefficients transformed into the virtual surround rendering domain. The filter coefficients include a set of filter coefficients (HM_L) for rendering a mono signal to the left channel and a set of filter coefficients (HM_R) for rendering a mono signal to the right channel when the decoded downmix signal is mono. .

The pseudo-surround renderer (510) includes a domain converter (511), a renderer M (512), and an inverse domain converter (513,514). The difference between the components of the virtual surround renderer 510 and the virtual surround renderer 410 described in FIG. 4 is that the decoded downmix signal is mono, so that it has a domain converter 511, and the renderer that performs virtual surround rendering is also a renderer. M 512 is one. The renderer M 512 receives filter coefficients from the spatial information converter 500, and performs virtual surround rendering for generating a virtual surround signal using the received filter coefficients. In this case, the filter coefficients are the filter coefficient set HM_L used to render the mono signal to the left channel and the filter coefficient set HM_R used to render the mono signal to the right channel.

On the other hand, with respect to the input of the downmix signal that is mono, the following two methods are possible when the output after virtual surround rendering is intended to obtain an output of a form such as downmix stereo.

First, the virtual surround renderer (eg, HRTF filter) does not use filter coefficients for the virtual surround effect, but uses a value used for stereo downmix. In this case, the values used in the stereo downmix may be coefficients left front = 1, right front = 0, ... for left output.

Second, in the decoding process of generating a multichannel using spatial information in the downmix channel, the decoding may be performed only up to a corresponding step to obtain a desired number of channels instead of generating the last multichannel.

Hereinafter, when the input signal is a mono downmix signal as shown in FIG. 5, the downmix signal is x, the channel-mapped coefficient is D, the circular HRTF filter coefficient is G as an external input, and the temporary multichannel signal is p. If the output signal is defined as y and expressed as a determinant, it is as follows.

,

,

,

,

,

,

Here, since the relationship between the determinants has been described in FIG. 4, it will be omitted here. 4 illustrates an example in which the input downmix signal is stereo, and FIG. 5 illustrates an example in which the input downmix signal is mono. In relation to the above-described definition, the relationship between H and GD in the present invention will be described with reference to FIG. 8.

6A and 6B illustrate an example of a channel mapping process according to the present invention.

The channel mapping process is a process of generating a value mapped to each channel on a multi-channel according to the received spatial information to the virtual surround renderer. In this case, the channel mapping process is a process performed by the channel mapping units 401 and 501.

,

,

,

,

,

등이 가능하다. 이때, 상기 채널 매핑 출력값을 생성하는 과정은 디코딩 장치에 수신된 공간 정보에 대응하는 트리 컨피규레이션(tree configuration)과 디코딩 장치에서 사용하는 공간 정보의 범위 등에 따라 가변적이다.For example, when the downmix signal is mono, channel mapping output values are generated using coefficients such as CLD1 to CLD5 and ICC1 to ICC5. For example, the channel mapping output value

,

,

,

,

,

Etc. are possible. In this case, the process of generating the channel mapping output value is variable according to a tree configuration corresponding to the spatial information received by the decoding apparatus and a range of spatial information used by the decoding apparatus.

6A illustrates an example of a first channel configuration for explaining a channel mapping process according to the present invention. At this time, the channel conversion unit constituting the channel configuration is an OTT box, the channel configuration shows a structure of 5151.

,

,

,

,

,

,

,

,

등)를 이용하여, 다운믹스 채널(m) 에서 멀티채널(L, R, C, LFE, Ls, Rs)을 생성하는 것이 가능하다.Referring to FIG. 6A, OTT boxes 601, 602, 603, 604, and 605 and spatial information (eg,

,

,

,

,

,

,

,

,

Etc.), it is possible to generate the multichannels L, R, C, LFE, Ls, Rs in the downmix channel m.

For example, when the tree structure is 5151, a method of obtaining a channel mapping output value using only the CLD is as follows.

here,

,

,

to be.

,

,

,

,

,

,

,

,

등)를 이용하여, 다운믹스 채널(m)에서 멀티채널(L, Ls, R, Rs, C, LFE)을 생성하는 것이 가능하다.Referring to FIG. 6B, OTT boxes 611, 612, 613, 614, and 615 and spatial information (eg,

,

,

,

,

,

,

,

,

Etc.), it is possible to generate the multichannels L, Ls, R, Rs, C, LFE in the downmix channel m.

For example, when the tree structure is 5152, a method of obtaining a channel mapping output value using only the CLD is as follows.

In addition, the generated channel mapping output values have different values for each frequency band, for each parameter band, and / or for each transmitted time slot. Here, if the value is significantly different between neighboring bands and time slots as boundaries, distortion may occur during virtual surround rendering. In order to prevent the generated distortion, a process of blurring in the frequency and time domains is required. The method performed to prevent the distortion is as follows. First, the above-described frequency blurring and time blurring can be used, and other methods suitable for virtual surround rendering can be used. In addition, each channel mapping sorting value may be used by multiplying a specific gain. The time blurring will be described in detail later with reference to FIG. 8.

FIG. 7 illustrates an example of filter coefficients for each channel according to the present invention. For example, the HRTF coefficient may be used as the filter coefficient.

For virtual surround rendering, the signal passing through the GL_L filter is sent to the left output for the left channel source and the signal passing through the GL_R filter to the right output. Thereafter, a process of generating the left final output (eg, Lo) and the right final output (eg, Ro) by summing all the signals received from each channel are performed.

Accordingly, the left and right channel outputs of the virtual surround rendering are as follows.

Lo = L * GL_L + C * GC_L + R * GR_L + Ls * GLs_L + Rs * GRs_L

Ro = L * GL_R + C * GC_R + R * GR_R + Ls * GLs_R + Rs * GRs_R

In the present invention, a method for obtaining L (710), C (700), R (720), Ls (730), and Rs (740) is as follows. First, L (710), C (700), R (720), Ls (730), and Rs (740) can be obtained by using a decoding method of generating a multichannel using the downmix channel using spatial information. For example, the multi-channel generating method is an MPEG surround decoding method. Second, L (710), C (700), R (720), Ls (730), and Rs (740) may be expressed as a relational expression of only spatial information.

8 to 10 illustrate a process for generating filter coefficients according to the present invention, for example, in detail.

Referring to FIG. 8, the coefficient generator 800 may include at least one coefficient generator (coef_1 generator 800_1, coef_2 generator 800_2,..., Coef_N generator 800_N). . The synthesizer / domain converter 810 may be configured to include an integrator 811, an interpolator 812, and a domain converter 813.

As the coefficient generation process using the filter information and the spatial information in the coefficient generator 800, the coefficient generation process in a specific coefficient generator (for example, the first coefficient generator is called coef_1 generator 800_1) is as follows. It can be expressed as

For example, when the downmix channel is monaural, the first coefficient generator (coef_1 generator) is a coefficient generator corresponding to the left channel on the multi-channel, and uses coefficient D_L generated from spatial information as follows. I can express it.

FL_L = D_L * GL_L (coefficient used to go from mono input to left output channel)

FL_R = D_L * GL_R (coefficient used to go from mono input to right output channel)

Here, D_L is a value generated from the spatial information in the channel mapping process of the spatial information. However, the process of obtaining the D_L may vary depending on the channel tree configuration transmitted from the encoding apparatus and received by the decoding apparatus. In relation to this, when a second coefficient generator is a coef_2 generator and a third coefficient generator is a coef_3 generator, the second coefficient generator generates FR_L and FR_R in the same manner as the coefficient generation method, and the third coefficient is generated. The generator coef_3 generator may generate FC_L, FC_R, and the like.

For example, when the downmix channel is stereo, the first coefficient generator (coef_1 generator) is a coefficient generator corresponding to the left channel on the multi-channel, and then uses the coefficients D_L1 and D_L2 generated from the spatial information. It can be expressed as

FL_L1 = D_L1 * GL_L (coefficient used to go from left input to left output channel)

FL_L2 = D_L2 * GL_L (coefficient used to go from right input to left output channel)

FL_R1 = D_L1 * GL_R (coefficient used to go from left input to right output channel)

FL_R2 = D_L2 * GL_R (coefficient used to go from right input to right output channel)

Here, when the downmix channel is stereo, at least one coefficient generator may generate various coefficients in the same manner as when the downmix channel is mono.

Hereinafter, a detailed description of a process of generating filter coefficients in the spatial information converter will be described with reference to FIGS. 8 to 10.

FIG. 8 illustrates a first embodiment of a process of generating filter coefficients in a spatial information converter according to the present invention.

First, the synthesizer 811 synthesizes the filter coefficients generated for each channel into filter coefficients for virtual surround rendering. The synthesis process in the synthesizer 811 will be described below by dividing into a mono input case and a stereo input case.

<Example of mono input>

HM_L = FL_L + FR_L + FC_L + FLS_L + FRS_L + FLFE_L

HM_R = FL_R + FR_R + FC_R + FLS_R + FRS_R + FLFE_R

<Example of stereo input>

HL_L = FL_L1 + FR_L1 + FC_L1 + FLS_L1 + FRS_L1 + FLFE_L1

HR_L = FL_L2 + FR_L2 + FC_L2 + FLS_L2 + FRS_L2 + FLFE_L2

HL_R = FL_R1 + FR_R1 + FC_R1 + FLS_R1 + FRS_R1 + FLFE_R1

HR_R = FL_R2 + FR_R2 + FC_R2 + FLS_R2 + FRS_R2 + FLFE_R2

Here, HM_L and HM_R refer to coefficients synthesized as filter coefficients for virtual surround rendering in the case of mono input, and HL_L, HR_L, HL_R and HR_R refer to coefficients synthesized as filter coefficients for virtual surround rendering in case of stereo input. do.

The interpolator / time blurer 812 may perform interpolation and time domain blurring on the filter coefficients.

The interpolation performed by the interpolator is performed to obtain spatial information that does not exist between the transmitted and generated spatial information when the transmitted and generated spatial information is wide in the time axis. For example, when spatial information exists in the nth paramSlot and the n + kth paramSlot (k> 1), the generated coefficients (eg, HL_L, HR_L, HL_R, HR_R) are not transmitted. The formula for the case of performing linear interpolation on paramSlot is as follows.

<Example of mono input>

HM_L (n + j) = HM_L (n) * a + HM_L (n + k) * (1-a)

HM_R (n + j) = HM_R (n) * a + HM_R (n + k) * (1-a)

<Example of stereo input>

HL_L (n + j) = HL_L (n) * a + HL_L (n + k) * (1-a)

HR_L (n + j) = HR_L (n) * a + HR_L (n + k) * (1-a)

HL_R (n + j) = HL_R (n) * a + HL_R (n + k) * (1-a)

HR_R (n + j) = HR_R (n) * a + HR_R (n + k) * (1-a)

Here, HM_L (n + j) and HM_R (n + j) refer to coefficients obtained by interpolating the coefficients synthesized with the input filter coefficients for virtual surround rendering. HL_L (n + j), HR_L (n + j), HL_R (n + j), HR_R (n + j) are coefficients obtained by interpolating the coefficients synthesized with the filter coefficients for the virtual surround rendering input in the case of stereo input. Means. In addition, 0 <j <k, j, k are integers, and a is a real number of 0 <a <1, which can be obtained by the following formula.

a = j / k

Therefore, the equation for the case of performing the linear interpolation on the untransmitted paramSlot is to find the value of the parameter slot existing between the n-th parameter slot and the n + k-th parameter slot. to be. According to the above equation, a value corresponding to a corresponding position is obtained on a line connecting values in two slots in a straight line.

Time blurring in the time blurr may be performed to prevent a problem in which discontinuities may occur and cause distortion when a coefficient value rapidly changes between neighboring blocks in the time domain. have. The time domain blurring may be performed in parallel with the interpolation, or the method applied may vary depending on the location. Hereinafter, an example of the equation for time domain blurring when the downmix channel is mono will be described.

HM_L (n) '= HM_L (n) * b + HM_L (n-1)' * (1-b)

HM_R (n) '= HM_R (n) * b + HM_R (n-1)' * (1-b)

That is, the filter coefficients HM_L (n-1) 'or HM_R (n-1)' in the previous block n-1 are multiplied by (1-b), and the filter coefficients generated in the current block n ( HM_L (n) or HM_R (n)) may be multiplied by b to add a 1-pole IIR filter. In this case, b is a constant value of 0 <b <1. The smaller the value of b, the larger the blurring effect. The larger the value of b, the smaller the blurring effect. The remaining filters are also applicable in the same way.

Interpolation and blurring are expressed as one equation using the equation for time domain blurring as follows.

HM_L (n + j) '= (HM_L (n) * a + HM_L (n + k) * (1-a)) * b + HM_L (n + j-1)' * (1-b)

HM_R (n + j) '= (HM_R (n) * a + HM_R (n + k) * (1-a)) * b + HM_R (n + j-1)' * (1-b)

On the other hand, if the interpolation and time domain blurring process is performed in the interpolator / time blurer 812, a filter coefficient having an energy value different from the energy of the original filter coefficient may be generated. Energy normalization work may be added to prevent this.

A domain converter 813 performs domain conversion to the virtual surround rendering application area. In this case, when the virtual surround rendering application area and the spatial information application area are the same, domain conversion may not be performed.

9 is a view illustrating a second embodiment of a process of generating filter coefficients in a spatial information converter according to the present invention.

Referring to FIG. 9, an integrator / domain converter 910 may include at least one interpolator 911_1, 911_2,..., 911_N, an integrator 912, and a domain converter 913. It characterized in that it is configured to include.

The difference from the first embodiment described with reference to FIG. 8 is that coefficients generated by each channel in the coefficient generator 900 (for example, FL_L1, FL_L2, FL_R1, and FL_R2 in the case of mono and FL_L2, FL_R1, and stereo). ), Interpolation is performed for each of the coefficients.

FIG. 10 illustrates a third embodiment of a process of generating filter coefficients in the spatial information converter according to the present invention.

FIG. 10 differs from the above-described embodiments of FIGS. 8 and 9 after performing interpolation in the interpolator 1000 for channel-mapped spatial information, and then using the interpolated value for each channel. To generate.

Accordingly, in FIG. 10, an integrator / domain converter: 1010 may include a coefficient generator 1011, an integrator: 1012, and a domain converter: 1013.

In the method of each embodiment described with reference to FIGS. 8 to 10, the channel mapped output of the spatial information is a value in the frequency domain (for example, a parameter band unit has a single value). Note that it is assumed that all of the generation processes, such as, are performed in the frequency domain. In addition, when the virtual surround rendering is also performed in the frequency domain, the domain converter does not play any role and may output the filter coefficients in the frequency domain as they are or by performing only a conversion process that matches the frequency resolution.

The present invention is not limited to the above-described embodiments, and as can be seen in the appended claims, modifications can be made by those skilled in the art to which the invention pertains, and such modifications are within the scope of the present invention.

The decoding method and apparatus for decoding an audio signal according to the present invention as described above generate a downmix channel by downmixing multichannels, and the decoding apparatus that receives the audio bitstream generated by extracting spatial information of the multichannel is multichannel. It is possible to decode to have a virtual surround effect even if the environment is not capable of generating the.

Claims (33)

(a) demultiplexing the received bitstream into a core codec bitstream and a spatial information bitstream; (b) decoding the core codec bitstream to produce a decoded downmix signal; (c) decoding the spatial information bitstream to generate spatial information; (d) generating a virtual surround signal from the decoded downmix signal using the spatial information. The method of claim 1, And a domain for generating the virtual surround signal and a domain for spatial information are the same domain. The method of claim 2, And when the domain generating the virtual surround signal is a frequency domain, converts a downmix signal into a frequency domain. The method of claim 2, And when the domain for generating the virtual surround signal is a time domain, converting a domain for spatial information into a time domain. The method of claim 1, wherein step (d) And generating filter coefficients using the spatial information and the external filter information to generate the virtual surround signal. The method of claim 5, And the filter information is a head-related transfer functions (HRTF) parameter. The method of claim 5, The generating of the filter coefficients includes a channel mapping step of mapping spatial information to each channel. The method of claim 7, wherein And generating filter coefficients for each channel by using the channel-mapped spatial information. The method of claim 8, And decoding and / or domain converting the filter coefficients of each channel for each downmix channel. The method of claim 1, wherein step (d) Domain converting the domains of the downmixed signal and spatial information for domain matching. The method of claim 1, wherein step (d) And performing inverse domain transformation after generating a virtual surround signal when domain conversion is performed for domain matching of the downmixed signal and spatial information. The method of claim 1, When the downmixed signal is mono, two filter coefficient sets are generated using spatial information, and the downmix signal is rendered using the filter coefficient sets. Decoding method. The method of claim 12, And the rendering generates a virtually rendered first output signal using the first set of coefficients and generates a second output signal using the second set of coefficients. The method of claim 1, When the downmixed signal is a stereo, four filter coefficient sets are generated using spatial information, and a downmix signal is rendered using the filter coefficient sets in each channel. Method of decoding an audio signal. The method of claim 14, And the rendering uses a first set of coefficients delivered to its own channel and a second set of coefficients delivered to its relative channel. Demultiplexing the received bitstream and generating downmixed signal and spatial information in the demultiplexed bitstream; Generating filter coefficients using the spatial information and the filter information; Generating a virtual surround signal from the downmixed signal using the filter coefficients. The method of claim 16, Receiving filter information comprising the step of decoding the audio signal. The method of claim 17, The filter information is a method of decoding an audio signal, characterized in that the HRTF parameter. A demultiplexer which demultiplexes the received bitstream into a core codec bitstream and a spatial information bitstream; A spatial information converter which decodes the spatial information bitstream to generate spatial information, and converts the spatial information to generate a virtual surround signal; And an audio decoder configured to decode the core codec bitstream to generate a decoded downmix signal and to generate a virtual surround signal from the decoded downmix signal using the converted spatial information. Device for decoding signal. The method of claim 19, wherein the spatial information converter, A spatial information decoder for decoding the spatial information bitstream; And a spatial information converter for generating an essential coefficient for generating a virtual surround signal from the spatial information. The method of claim 20, wherein the spatial information converter, And a channel mapping unit for mapping the received spatial information to each channel of the multichannel signal. The method of claim 21, wherein the spatial information converter, And a coefficient generator for generating at least one filter coefficient for each channel in the channel-mapped spatial information. The spatial information converter of claim 22, And an integrator for synthesizing the filter coefficients for each channel for each downmix channel. The spatial information converter of claim 22, And an interpolator for interpolating the filter coefficients of each channel. The spatial information converter of claim 22, And a domain converter for matching the decoded downmix signal and the domain to each of the channel filter coefficients. The method of claim 19, wherein the audio decoder, A core decoder for decoding the core codec bitstream to produce a decoded downmix signal; And a virtual surround renderer for generating a virtual surround signal using the decoded downmix signal and spatial information. 27. The system of claim 26 wherein the virtual surround renderer is: And a domain converter configured to perform domain conversion for domain matching of the downmixed signal and spatial information. 27. The system of claim 26 wherein the virtual surround renderer is: In case of domain conversion for domain matching of the downmixed signal and spatial information, an audio signal comprising an inverse domain converter for inverse domain conversion after rendering the decoded downmix signal Decoding device. 27. The system of claim 26 wherein the virtual surround renderer is: When the decoded downmix signal is mono, two filter coefficient sets are generated by using spatial information, and the decoded downmix signal is rendered by using the filter coefficient sets in each channel. A device for decoding an audio signal. The method of claim 29, wherein the virtual surround renderer, Generating a virtually rendered first output signal using the first set of coefficients, and generating a second output signal using the second set of coefficients. 27. The system of claim 26 wherein the virtual surround renderer is: When the decoded downmix signal is stereo, four filter coefficient sets are generated by using spatial information, and the decoded downmix signal is rendered by using the filter coefficient sets in each channel. A device for decoding an audio signal. The method of claim 31, wherein the virtual surround renderer, And a first renderer and a second renderer for generating a virtual surround signal using a first set of coefficients delivered to its own channel and a second set of coefficients delivered to a relative channel. The method of claim 26, And a coefficient used by the virtual surround renderer is a filter coefficient generated using spatial information and filter information.

Images