KR20060109296A - Reference channel adaptation method considering subband spatial cues for multi-channel audio signal - Google Patents

Abstract

A method of applying a variable reference channel to a multi-channel audio signal using spatial information of each sub-band is provided to represent a spatial information bit stream in an efficient manner using a variable reference channel for spatial information in consideration of characteristics of the multi-channel audio signal to improve encoding, transmission and decoding efficiency of the multi-channel audio signal. A multi-channel audio signal is down-mixed and spatial information of the multi-channel audio signal is extracted by sub-bands(315,316). Reference channels are set for respective sub-bands using the spatial information, and spatial information of the other channels is re-extracted using spatial information of the set reference channels(318). The entire bit stream is constructed using the down-mixed audio signal and the re-extracted spatial information(321).

Description

Reference channel adaptation method considering subband spatial cues for multi-channel audio signal

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a diagram illustrating a method for a human to recognize spatial information about an audio signal in the present invention.

2 is a diagram of a method of coding a multichannel audio signal using a spatial encoder and decoder in the present invention.

3A is a detailed diagram of a spatial encoder according to the first embodiment of the present invention.

3B is a flowchart of an encoding process according to the first embodiment of the present invention.

4A is a detailed diagram of a spatial encoder according to a second embodiment of the present invention.

4B is a flowchart of an encoding process according to the second embodiment of the present invention.

Fig. 5A is a detailed diagram of a spatial decoder according to the first embodiment of the present invention.

5B is a flowchart of a decoding process according to the first embodiment of the present invention.

* Explanation of symbols for main parts of the drawings

101.Remote sound source 102.Direct sound wave

104. Reflected Sound Wave 201. Multichannel Audio Signal

202. Downmix unit 203. Special parameter extraction unit

204 Spatial Encoder 205 Artistic Downmix Audio Signal

206. Mono or stereo audio signal 207. Special parameters

208.Space Decoder 302.Filter Bank

303. Reference channel selector 310. Bitstream formatter

407. Comparison selection unit 502. Bitstream decoding unit

504.Spatial information calculation unit

The present invention relates to a method for constructing a bitstream of a multichannel audio signal, and more particularly, to a method for effectively constructing a bitstream for spatial information by using a variable reference channel in multichannel audio coding.

Recently, various coding techniques and methods for digital audio signals have been developed, and related products have been produced. In addition, coding methods for multi-channel audio signals have been developed using a psychoacoustic model, and standardization thereof has been performed.

The psychoacoustic model is an unnecessary part of the coding process by using a method of recognizing a sound, for example, a small sound following a loud sound, and only a sound corresponding to a frequency of 20 Hz to 20000 Hz. By eliminating the audio signal for, the amount of data needed can be effectively reduced.

Currently, audio standard technologies such as MPEG-1 Audio (MEPG-1 Layer III), MPEG-4 Advanced Audio Coding (AAC), and MPEG-4 High-Efficiency AAC (HE-AAC) have been developed and commercialized. In addition, a method of coding a multichannel audio signal using spatial information has been developed. The multi-channel audio signal coding method uses a compressed audio signal (e.g., stereo or mono audio signal) and a low bit-rate side information (e.g., spatial information) channel. Therefore, the transmission efficiency of the multichannel audio signal is greatly improved.

However, in configuring a bitstream of a multichannel audio signal in the coding method of the multichannel audio signal, conventionally, spatial information is ordered by using a fixed reference channel (for example, a front left signal). Expressed. Therefore, the conventional method does not consider the characteristics of the audio signal at all and requires a large bit rate for coding spatial information, which has a disadvantage in that the configuration efficiency of the bitstream for the audio signal is not good.

Therefore, the present invention proposed to solve the above problems, in coding a multi-channel audio signal, a spatial information bitstream using a variable reference channel for spatial information in consideration of the characteristics of the audio signal It is an object of the present invention to provide an encoding and decoding method capable of improving the encoding, transmission and decoding efficiency of a multichannel audio signal by expressing in a efficient manner.

In order to achieve the above object, the present invention comprises the steps of downmixing the multichannel audio signal and extracting spatial information of the multichannel audio signal for each subband; Setting a reference channel for each subband by the reference channel selector by using the spatial information for each subband, and re-extracting spatial information of the remaining channels using the spatial information of the set reference channel; And encoding the downmixed audio signal and the re-extracted spatial information to construct a bitstream in a bitstream formatter. The spatial information may include a channel level difference (CLD), a channel time difference (CTD), an inter-channel coherence (ICC), a channel prediction coefficient (CPC), etc. The preset criterion is a maximum of the spatial information. The channel corresponding to either the value or the median may be set as the reference channel. In addition, the index information for the reference channel may be included in the bitstream, and the index information is included in the bitstream using the same table as the table for the spatial information or in the bitstream using a separate table. May be included.

In addition, in order to achieve the above object, the present invention comprises the steps of downmixing the multichannel audio signal and extracting spatial information of the multichannel audio signal for each subband (step (a)); The reference channel selector sets a reference channel for each subband by using the spatial information for each subband, and extracts spatial information of the remaining channels using spatial information of the set reference channel to form a bitstream ((b). Step); Extracting spatial information of the remaining channels using spatial information of the fixed reference channel to form a bitstream (step (c)); Comparing the bitstream formed in the step (b) with the bitstream formed in the step (c), and then selecting one method (step (d)); A method of encoding a channel audio signal is provided. In this case, identification information for distinguishing a case in which a reference channel is variably set from each subband and a case in which the stationary reference channel is used may be included in the bitstream. Selecting the one method may be, for example, selecting a method having a minimum bit rate.

In addition, to achieve the above object, the present invention includes the steps of receiving a core codec bitstream and spatial information bitstream for the downmixed audio signal; The core codec bitstream and the spatial information bitstream are decoded, and index information of the reference channel, which is variably set for each subband from the decoded spatial information bitstream, spatial information of the reference channel, and spatial information of the reference channel are used. And reading spatial information of the extracted remaining channel. The decoding method may include converting a downmix audio signal generated by decoding the core codec non-stream into a multi-channel using index information of the read reference channel, spatial information of the reference channel, and spatial information of the remaining channels. It may further include.

In addition, to achieve the above object, the present invention includes the steps of receiving a core codec bitstream and spatial information bitstream for the downmixed audio signal; Decoding the core codec bitstream and configuring a bitstream using a reference channel set variably for each subband from the spatial information bitstream and configuring a bitstream using a fixed reference channel. And reading the distinguishable identification information and decoding the spatial information bitstream according to the read identification information. The decoding method reads the index information of the reference channel, the spatial information of the reference channel and the spatial information of the remaining channels from the decoded spatial information bitstream, and the index information of the read reference channel, the spatial information of the reference channel, and the rest. The method may further include converting a downmix signal obtained by decoding the core codec bitstream by using spatial information of a channel into a multichannel.

In addition, in order to achieve the above object, the present invention is generated so that an audio signal includes a core codec bitstream and a spatial information bitstream, wherein the spatial information bitstream is selected by using the spatial information of the multi-channel audio signal A reference channel is set for each subband, re-extracting spatial information of the remaining channels using the set spatial information of the reference channel, and using the spatial information of the reference channel and the spatial information of the re-extracted remaining channels It provides a method for generating an audio signal, characterized in that the stream is generated.

In addition, in order to achieve the above object, the present invention extracts the spatial information of the multi-channel audio signal for each subband, and a reference channel for each subband in the reference channel selector variably using the spatial information for each subband A method of configuring a bitstream by re-extracting spatial information of the remaining channels using spatial information of the set reference channel and extracting spatial information of the remaining channels using spatial information of a fixed reference channel. The present invention provides a method for generating an audio signal, characterized in that the method is configured to compare one method and then select one method. Here, the bitstream may include identification information for distinguishing a case where a bitstream is configured using a variable reference channel set for each subband and a case where a bitstream is configured using the fixed reference channel. .

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention that can specifically realize the above object will be described.

1 shows a method for a human to recognize spatial information about an audio signal in the present invention. The coding method for a multichannel audio signal is based on the fact that a human perceives the audio signal as a three-dimensional space. I use it. Such parameters, called "spatial parameters" for indicating spatial information of a multichannel audio signal, include channel level differences (CLD), inter channel coherences (ICC), channel prediction coefficients (CPC), and channel time differences (CTD). The CLD denotes an energy difference between two channels, the ICC denotes a correlation between two channels, and the CPC denotes a prediction coefficient used when generating three channels from two channels. , CTD means time difference between two channels.

How a human perceives an audio signal spatially and how the concept of the spatial parameter is generated is shown in FIG. 1. Direct sound wave 103 from the remote source 105 arrives at the human left ear 107, and another direct sound wave 102 is diffracted around the head to the right ear 106. Will be reached. The two sound waves 102 and 103 show a difference in arrival time and energy level, and this difference generates the CLD, CPC and CTD parameters.

Also, if the reflected sound waves 104 and 105 reach both ears, or if the sound source 105 is dispersed, sound waves that do not correlate with each other will reach both ears, which will generate the ICC parameter. Spatial parameters generated on the principle as described above are known to enable a strong number of bits in transmitting a multichannel audio signal as a mono or stereo signal and then outputting the multichannel audio signal back to the multichannel. The present invention proposes a method of expressing the spatial parameters in a bitstream in a very efficient manner using a variable reference channel.

2 illustrates the principle of coding a multichannel audio signal using a spatial encoder and decoder in the present invention. As shown, the spatial encoder 204 first receives a multichannel audio signal 201. Where N is the number of input channels. The multichannel audio signal 201 is downmixed by the down-mix unit 202 to be a downmix signal 206.

In addition, the spatial information of the multichannel audio signal, that is, the spatial parameter, is extracted from the multichannel audio signal 201 by the spatial parameter extractor 203. In this case, spatial information refers to downmixing a multi-channel (eg, Left, Right, Center, Left surround, Right surround, etc.) audio signal, transmitting the downmix signal 206, and transmitting the transmitted down signal. Information about the audio signal channel used when upmixing a mix signal back to multichannel. Alternatively, the downmix signal 206 may be generated using an externally provided downmix signal, for example, an artistic down-mix signal 205.

The downmix signal 206 is encoded using a core codec coding method and then compressed and transmitted. The downmix signal 206 is also transmitted with the spatial information, that is, the spatial parameter 207. The core codec refers to a codec for coding or encoding an audio signal other than the spatial information, that is, the spatial parameter 207. The core codec may include MP3, AC-3, DTS, or AAC. If the function is performed, it may include a codec to be developed in the future as well as a codec previously developed. If the user's system can only output the downmix signal 206, the compressed downmixed signal 206 may be decoded and output 209 directly. If the system is capable of outputting a multichannel audio signal, the compressed and transmitted audio signal is decoded and then received spatial information, i.e., spatial parameter 207, of the multichannel audio signal transmitted together by the spatial decoder 208. The multichannel audio signal 210 may be converted into a multichannel audio signal 210 and output.

Instead of transmitting the multichannel audio signal directly, the method of downmixing and transmitting the downmix signal 206 as described above, and transmitting the spatial information of the multichannel audio signal, that is, the spatial parameter 207 together, is compressed and It is very advantageous in terms of transmission efficiency. In the present invention, when transmitting spatial information of the multi-channel audio signal, that is, the spatial parameter 207 together, the spatial parameter 207 is represented in a more efficient manner using a variable reference channel to form a bit string. Compression and transmission efficiency can be improved.

3A shows in detail the spatial encoder according to the first embodiment of the present invention. As shown, the multichannel audio signal 301 is downmixed and encoded 312, and the multichannel audio signal is passed through a filterbank 302 to extract spatial information from the multichannel audio signal 301. It is divided by subbands. The filter bank 302 serves to divide an audio signal across all frequency bands for each subband, and the filter bank 302 may include a sub-band filter bank or a QMF filter bank. . The spatial information is extracted for each subband of the audio signal passing through the filter bank, and the reference channel selector 303 uses the extracted spatial information to variably set a reference channel according to a preset reference. The reference channel may be set for each subband. Then, spatial information is expressed using the variably set reference channel. The spatial information expressed using the reference channel may be expressed as a comparison value with a peripheral value, for example, as a difference value with the peripheral value. The spatial information may include CLD, CTD, ICC or CPC. The preset criterion may be, for example, setting a channel having spatial information of a maximum value or a median value of the spatial information as a reference channel.

The reference channel selector 303 sets a channel having spatial information of a predetermined value, for example, the maximum value or the median value of the subband spatial information, as the reference channel using the spatial information for each subband (304). can do. The reference channel selector 303 may be configured in several steps. For example, the reference channel selector 303 may calculate spatial information using the reference channel for each subband set as described above, and quantize the calculated spatial information 306. ), The index information 307 and the quantized spatial information 308 of the reference channel can be generated. The index information 307 and the quantized spatial information 308 of the reference channel are then encoded 309 to complete the entire bitstream 311 along with the downmixed audio signal in the bitstream formatter 310. To form. The index information 307 and the quantized spatial information 308 of the reference channel form a spatial information bitstream, and the downmixed audio signal forms a core codec bitstream. The spatial information bitstream is a spatial information extracted from a multi-channel audio signal, that is, a bitstream for a spatial parameter, and may be configured with a configuration bitstream and a spatial data bitstream for the spatial information. The core codec bitstream refers to a bitstream formed by an audio signal except for the spatial information.

For example, in the case of CLD among the spatial information, first, the energy CLD for each channel is calculated from the audio signal divided for each subband through the filter bank 302. Then, the reference channel selector 303 sets a specific channel for each subband as a reference channel using the calculated energy for each channel. Energy information such as energy difference between channels is expressed using the set energy of the reference channel, and the energy information expressed as described above is quantized. Subsequently, the quantized energy information is subjected to entropy coding such as Huffman coding or the index information of the reference channel, respectively, and the entire bitstream together with the audio signal downmixed in the bitstream formatter 310. Will be configured.

For example, in the case of CTD, ICC, or CPC, the CTD, ICC, or CPC is extracted for each subband from the audio signal divided for each subband through the filter bank 302. Then, using the extracted CTD, ICC or CPC, the reference channel selector 303 sets a specific channel for each subband as a reference channel according to a preset reference. CTD, ICC or CPC is re-extracted using the configured CTD, ICC or CPC of the reference channel, and the CTD, ICC or CPC expressed as described above is quantized. The quantized CTD, ICC, or CPC is then subjected to entropy coding such as Huffman coding or index information of the reference channel, respectively, and constitutes the entire bitstream together with the downmixed audio signal in the bitstream formatter 310. Done.

3B is a flowchart of an encoding process according to the first embodiment of the present invention. To more efficiently represent spatial information, first downmix 315 the audio signal from the multichannel audio signal 314 and generate a core codec bitstream using the downmixed audio signal (317). The downmixed audio signal may comprise a mono or stereo signal. Subsequently, spatial information, e.g., spatial parameters, are extracted for each subband from the multichannel audio signal 314, and a reference channel is selected for each subband in a preset manner by using the extracted spatial information. 318). The preset method may be to set a channel corresponding to one of the maximum value or the median value of the spatial information as a reference channel. Next, the spatial information is represented using the set reference channel, and the spatial information bitstream including the spatial information expressed as described above is generated 320, and the entire codec bitstream and the spatial information bitstream are composed. The bitstream is transmitted 321.

4A shows a detailed diagram of a spatial encoder according to a second embodiment of the present invention. As shown, the multichannel audio signal 401 is downmixed and encoded 410, and the multichannel audio signal is passed through a filterbank 402 to extract spatial information from the multichannel audio signal 401. It is divided by subbands. Spatial information is extracted for each subband from the audio signal passing through the filter bank, and the extracted spatial information is expressed in two ways. In the first method, the reference channel selector 403 variably sets a reference channel for each subband according to a preset reference using spatial information extracted for each subband, and uses the variably set reference channel. The spatial information of the remaining channels is reextracted, and the encoding unit 405 encodes the spatial information and the reextracted spatial information of the reference channel to form a bitstream. The second method extracts spatial information of the remaining channels using spatial information of the fixed reference channel in the reference channel selector 404, and encodes spatial information of the reference channel and spatial information of the remaining channels in the encoding unit 406. The encoding consists of the bitstream. The reference channel selector 404 may be the same as or different from the reference channel selector 403. Next, the comparison selecting unit 407 selects one of a method of configuring a bitstream using the variable reference channel and a method of configuring a bitstream using the fixed reference channel. The comparison selecting unit 407 may select a method having a minimum bit rate by comparing bit rates of the bitstreams formed by the two methods. The bitstream by the selected method then forms the entire bitstream 409 along with the bitstream for the downmixed audio signal in the bitstream formatter 408.

4B is a flowchart of an encoding process according to a second embodiment of the present invention. In order to more efficiently represent the spatial information, first downmix 412 the audio signal from the multichannel audio signal 411, and generate a core codec bitstream using the downmixed audio signal (414). The downmixed audio signal may comprise a mono or stereo signal. Subsequently, spatial information, e.g., spatial parameters, are extracted from the multi-channel audio signal 411 for each subband (413), and a reference channel that is changed in a predetermined manner is used by using the spatial information extracted for each subband. Selected for each subband, and generates a spatial information bitstream using the selected variable reference channel (416). In addition, the spatial information extracted for each subband generates a spatial information bitstream using a fixed channel (415). Next, the spatial information bitstream generated using the variable reference channel and the spatial information bitstream generated using the fixed channel are compared 417. If generating the spatial information bitstream using the variable reference channel has a smaller bit rate, select (419) the spatial information bitstream generated using the variable reference channel, and if using the fixed channel, If generating the information bitstream has a smaller bit rate, then select 418 the spatial information bitstream generated using the fixed channel. Then, the entire bitstream consisting of the spatial information bitstream of the selected scheme and the core codec bitstream is transmitted (420).

5a shows a spatial decoder according to a first embodiment of the present invention in detail. As shown, the spatial decoder receives the entire bitstream 501 including the core codec bitstream and the spatial information bitstream, and is decoded by the bitstream decoder 502. The core codec bitstream is decoded 412 to be a downmixed audio signal 413, which may include a mono or stereo audio signal. In addition, the spatial information bitstream is decoded 503 for each subband and entered into the spatial information calculator 504. The decoding step may use, for example, a Hoffman decoding scheme. The spatial information calculator 504 calculates 507 spatial information using the index information 505 of the reference channel and the spatial information 506 expressed using the reference channel, and corresponds to the calculated spatial information. The physical quantity is calculated 408, and the physical quantity corresponding to the spatial information is output 409 for each subband. The multichannel audio signal generator 510 may then generate the multichannel audio signal 511 using the downmixed audio signal and the physical quantity corresponding to the spatial information output for each subband.

5B shows a flowchart of a decoding process according to the first embodiment of the present invention. First, the spatial decoder receives 514 the entire bitstream including the core codec bitstream and the spatial information bitstream. The core codec bitstream may then be decoded 515 and extract 517 a downmixed audio signal from the decoded core codec bitstream. The downmixed audio signal may comprise a mono or stereo audio signal. In addition, the spatial information bitstream is decoded 516, the index information of the reference channel is read 518 from the decoded spatial information bitstream, and the spatial information is calculated using the index information of the reference channel. A physical quantity corresponding to the information is calculated (519). Thereafter, the extracted downmix signal is converted into a multi-channel audio signal using the physical quantity corresponding to the calculated spatial information (520).

Although the present invention has been described in detail with reference to some embodiments, the above embodiments are presented for the purpose of understanding the present invention, and the scope of the present invention is not limited to the above embodiments. Those skilled in the art will understand that various modifications are possible without departing from the scope of the technical idea of the present invention, and the scope of the present invention should be interpreted by the appended claims.

As described above, in coding a multi-channel audio signal according to the present invention, in order to efficiently represent spatial information, the reference channel is variably changed according to a preset reference using spatial information extracted for each subband. By setting and expressing spatial information using the variable reference channel, the entire bitstream can be represented at a lower bit rate, thereby improving the encoding, transmission and decoding efficiency.

Also, by comparing the case of representing spatial information using the variable reference channel and the case of representing spatial information using a fixed channel, encoding, transmission and decoding are performed by selecting a method that can be represented with a lower bit rate. There is an effect that can improve the efficiency.

Claims (18)

A method of encoding a multichannel audio signal, (a) downmixing the multichannel audio signal and extracting spatial information of the multichannel audio signal for each subband; (b) setting a reference channel for each subband using the spatial information for each subband, and re-extracting spatial information for the remaining channels using spatial information of the set reference channel; And and (b) constructing an entire bitstream by using the downmixed audio signal and the re-extracted spatial information. The method of claim 1, The spatial information includes a channel level difference (CLD), a channel time difference (CTD), an inter-channel coherence (ICC), or channel prediction coefficients (CPC). The method of claim 1, In step (b), A channel having spatial information of the maximum value or the median value among the spatial information extracted for each subband is set as a reference channel for each subband, and the spatial information of the remaining channels is reextracted using the set reference channel. , Method of encoding multichannel audio signals. The method of claim 1, In step (b), And incorporating index information of the reference channel into the bitstream. The method of claim 4, wherein And the index information is included in the bitstream using the same table as the table for the spatial information or in the bitstream using a separate table. The method of claim 1. In step (b), And expressing the re-extracted spatial information as a comparison value with adjacent spatial information. A method of encoding a multichannel audio signal, (a) downmixing the multichannel audio signal and extracting spatial information of the multichannel audio signal for each subband; (b) setting a reference channel for each subband using the spatial information for each subband, and reconstructing spatial information of the remaining channels using spatial information of the set reference channel to form a bitstream; (c) forming a bitstream using spatial information of a reference channel in which spatial information for each subband is fixed; And (d) comparing the bitstream formed in the step (b) with the bitstream formed in the step (c), and then selecting one of the methods; encoding method of the multi-channel audio signal . The method of claim 7, wherein In step (d), When composing the entire bitstream using the bitstream of the selected method, it is possible to distinguish whether the bitstream is formed in the step (b) or the bitstream is formed in the step (c). And including the identification information. The method of claim 7, wherein In step (d), And selecting a method having a minimum bit rate among the bitstream formed in the step (b) and the bitstream formed in the step (c). In the method of decoding a multichannel audio signal, (a) receiving a core codec bitstream and a spatial information bitstream for the downmixed audio signal; And (b) index information of a reference channel, spatial information of a reference channel, and spatial information of the reference channel, which are decoded from the core codec bitstream and the spatial information bitstream and are variably set for each subband from the decoded spatial information bitstream; And reading out spatial information of the remaining channel extracted using the multi-channel audio signal. The method of claim 10, In step (b), And converting the downmix signal generated by decoding the core codec nonstream using the index information of the read reference channel, the spatial information of the reference channel, and the spatial information of the remaining channels, into a multichannel. A method for decoding a multichannel audio signal. In the method of decoding a multichannel audio signal, (a) receiving a core codec bitstream and a spatial information bitstream for the downmixed audio signal; And (b) decoding the core codec bitstream and configuring a bitstream using a reference channel set variably from the spatial information bitstream for each subband and configuring a bitstream using a fixed reference channel; And reading the identification information for distinguishing cases, and decoding the spatial information bitstream according to the read identification information. The method of claim 12, In step (b), The index information of the reference channel, the spatial information of the reference channel, and the spatial information of the remaining channels are read from the decoded spatial information bitstream, and the index information of the read reference channel, spatial information of the reference channel, and spatial information of the remaining channels are read. And converting the downmix signal obtained by decoding the core codec bitstream into a multichannel using the multi-channel audio signal. In generating an audio signal, The audio signal is generated to include a core codec bitstream and a spatial information bitstream, The spatial information bitstream sets a reference channel for each subband using spatial information of a multichannel audio signal, re-extracts spatial information of the remaining channels using spatial information of the set reference channel, and spaces of the reference channel. And the entire bitstream is generated using the information and the spatial information of the remaining extracted channels. The method of claim 14, The spatial information includes a channel level difference (CLD), a channel time difference (CTD), an inter-channel coherence (ICC), or channel prediction coefficients (CPC). The method of claim 14, The setting of the reference channel may include setting a channel having spatial information of a maximum value or a median value among the spatial information extracted for each subband as a reference channel. In generating an audio signal, Extract spatial information of a multi-channel audio signal for each subband, variably set a reference channel for each subband using the spatial information for each subband, and spatial information of the remaining channels using spatial information of the set reference channel. It is generated to select one method after comparing the method of composing the bitstream by re-extracting and the method of composing the bitstream by extracting the spatial information of the remaining channels using the spatial information of the fixed reference channel. A method of generating an audio signal. The method of claim 17, The bitstream is generated to include identification information for distinguishing a case of configuring a bitstream using a variable reference channel set for each subband and a case of configuring a bitstream using the fixed reference channel. A method of generating an audio signal, characterized in that.

Images