JP2006323314A - Apparatus for binaural-cue-coding multi-channel voice signal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for carrying out binaural-cue-coding of a multi-channel voice signal, capable of playing back an effect of an original signal with high quality only by multi-channel, in a coding processing which extracts a binaural cue and down-mixes the original signal. <P>SOLUTION: After deriving a desired vector relation between a down-mix channel and an original channel from a binaural cue at first, an accurate vector relation between a down-mix signal and a signal that is orthogonal and non-correlated thereto is simulated. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an apparatus for compressing a multi-channel audio signal by extracting a binaural cue in an encoding process to generate a downmix signal and adding the binaural cue to the downmix signal in a decoding process. The present invention is applicable to a home theater system, a car audio system, an electronic game system, and the like.

  The present invention relates to encoding multi-channel audio signals. The main purpose is to encode the digital audio signal while maintaining the audible quality of the digital audio signal to the maximum even when the bit rate is limited. Lowering the bit rate is advantageous in reducing the transmission bandwidth and storage capacity.

Conventionally, there are many methods for realizing the bit rate reduction as described above.
In the “MS stereo” method, the stereo channels L and R are represented in the form of their “sum” (L + R) and “difference” (LR). When these stereo channels are highly correlated, the “difference” signal includes less important information that can be coarsely quantized with fewer bits than the “sum” signal. In an extreme example where L = R, it is not necessary to transmit information regarding the differential signal.

  The “intensity stereo” method uses the psychoacoustic characteristics of the ears, and for the high frequency range, only the “sum” signal is transmitted along with the scale factor corresponding to the frequency, and the scale is set on the decoder side. A factor is applied to the “sum” signal to synthesize the L and R channels.

  In the method by “binaural cue coding”, a binaural cue is generated in order to form a downmix signal in the decoding process. The binaural cue is, for example, an inter-channel level / intensity difference (ILD), an inter-channel phase / delay difference (IPD), an inter-channel coherence / correlation (ICC), or the like. The relative signal power can be measured from the ILD queue, the time difference until the sound reaches both ears can be measured from the IPD queue, and the similarity can be measured from the ICC queue. In general, the balance and direction of voice can be controlled by the level / intensity cue and the phase / delay cue, and the width and spread of the voice can be controlled by the coherence / correlation cue. Together, these cues are spatial parameters that help the listener compose an auditory scene in the head.

  FIG. 1 is a diagram illustrating a typical codec using a method based on binaural cue coding. In the encoding process, the audio signal is processed for each frame. Module (100) downmixes left channel L and right channel R to produce M = (L + R) / 2. The binaural queue extraction module (102) processes L, R, and M to generate a binaural queue. The binaural extraction module (102) typically comprises a time-frequency conversion module in which L, R and M are converted into a full spectral representation, eg, FFT, MDCT, or time such as QMF. And a hybrid representation of frequency. Alternatively, M can be generated from L and R after spectral conversion by taking an average value of L and R expressed in a spectrum. The binaural cue can be obtained by comparing L, R, and M expressed as described above for each spectrum band.

The audio encoder (104) encodes the M signal and generates a compressed bit stream. Examples of audio encoders include MP3 and AAC. The binaural cues are quantized in module (106) and then multiplexed into compressed M to form a complete bitstream. In the decoding process, the demultiplexer (108) separates the M bitstreams from the binaural queue information. The audio decoder (110) decodes the M bit stream and restores the downmix signal M. The multi-channel synthesis module (112) processes the downmix signal and the dequantized binaural cue to recover the multi-channel signal.
ISO / IEC 14496-3: 2001 / AMD2, "Parametric Coding for high Quality Audio" US2003 / 0219130A1, "Coherence-based Audio Coding and Synthesis" Karls, M., Brandenburg, K., et al, "Applications of Digital Signal Processing to Audio and Acoustics", Kluwear Academic Press. JP2004 / 248989, "Encoding and Decoding Devices for Audio Signals"

  The present invention aims to improve the method based on binaural cue coding in the prior art.

  The present invention relates to a binaural cue encoding method for converting an L channel and an R channel into a time-frequency (T / F) representation using a QMF filter bank in an encoding process.

  In Non-Patent Document 1, sound spread is realized by mixing a downmix signal and a “reverberation signal”. The reverberation signal can be obtained by processing the downmix signal using Shroeder's all-pass link. However, this mixing method does not fully utilize the vector relationship between the downmix signal and the original signal.

  In Patent Literature 1, sound expansion (that is, a surround effect) is realized by inserting a “random sequence” into the ILD queue and the IPD queue. The random sequence is controlled by the ICC queue.

  In Embodiment 1 of the present invention, a desired vector relationship between the downmix channel and the original channel is first derived from the binaural cue, and then the exact vector relationship between the downmix signal and its orthogonal signal is simulated. A new mixing method is proposed.

  Embodiment 2 proposes a method of applying the channel separation method to multi-channel.

  In the present invention, in an encoding process in which binaural cues are extracted and the original signal is downmixed, the multichannel effect of the original signal can be reproduced with high quality. This can be achieved by applying the binaural cue to the downmix signal in the decoding process.

  It will be understood by those skilled in the art that the embodiments described below merely illustrate various inventive principles of the present invention, and various modifications can be made to the detailed description given below. If there is, it is easy to understand. Therefore, the present invention is limited only by the scope of the claims, and is not limited by the specific examples shown below.

  Furthermore, here only two cases of stereo-mono-stereo (hereinafter referred to as “2-1-2 case”) and 5-channel-mono-5 channel (hereinafter referred to as “5-1-5 case”) are included. However, the present invention is not limited to this. This can be generalized as an M original channel and an N downmix channel.

  FIG. 2 is a diagram showing an encoding process in the 2-1-2 case. The transform module (200) processes the original channels L (t) and R (t) to obtain respective time-frequency representations L (t, f) and R (t, f). Here, t indicates a time index, and f indicates a frequency index. The conversion module (200) is, for example, a complex QMF filter bank as used in MPEG Audio Extensions 1 and 2. L (t, f) and R (t, f) include a plurality of continuous subbands, and each subband represents a narrow frequency band of the original signal. Since the QMF filter bank has a narrow frequency band in the low frequency subband and a wide band in the high frequency subband, the QMF filter bank can be composed of a plurality of stages.

  The downmix module (202) processes L (t, f) and R (t, f) and generates a downmix signal M (t, f). In this embodiment, a simple method using “weighting” is shown.

  In the present invention, level adjustment is performed using an ILD queue. To compute the ILD queue, module (204) further processes L (t, f) and R (t, f) to generate ILD (l, b) and Border. As shown in FIG. 3, first, the time-frequency representation L (t, f) is divided into a plurality of bands (300) in the frequency direction. Each band includes a plurality of subbands. Using the psychoacoustic characteristics of the ear, the low frequency band has fewer subbands than the high frequency band. For example, when subbands are grouped into bands, the “Burk scale” or “critical band” well known in the field of psychoacoustics can be used.

L (t, f) and R (t, f) are further divided into frequency bands (l, b) at the boundary Border (302) in the time direction, whereas E _L (l, b) and E _R ( l, b) is calculated. Here, l is an index of time division, and b is an index of bandwidth. The optimal placement location of Border is the time position where the ratio of E _L (l, b) and E _R (l, b) changes rapidly. ILD (l, b) is calculated as follows.

In order to obtain the inter-channel coherence queue in the encoding process, the module (206) processes L (t, f) and R (t, f), and obtains ICC (b) using the following equation.

Further, to determine the high frequency inter-channel correlation cue for the high frequency subband (> 1.5 kHz only) in the encoding process, (208) processes L (t, f) and R (t, f), ICCH (b) is obtained using the following mathematical formula.

  As will be described later, the gain factor is derived by using ICC (l, b) in combination with ILD (l, b), and the actual signal strengths of L and R with respect to M are restored. In addition, ICC (l, b) is used to measure the phase relationship between L and R at low frequencies, which also helps to measure the degree of separation between L and R. However, at high frequencies, the effect brought about by the separation of the sounds is influenced not by the phase difference but by the similarity of the L and R waveforms. For example, when L = cos (ωt + θ) and R = cos (ωt), if the value of ω is large, the same stereoacoustic effect is brought about regardless of the value of θ. The use of ICCH (l, b) is more suitable for such waveform correlation measurement.

  All the binaural cues are part of the sub information in the encoding process. As shown in FIG. 4, the entire process for binaural cue generation can be included in the module (400) using the input / output described above.

  FIG. 5 is a diagram showing a decoding process using the binaural queue generated as described above. The conversion module (500) processes the downmix signal M (t) and converts it into a time-frequency representation M (t, f). The conversion module shown in the present embodiment is a complex QMF filter bank.

  The decorrelator (502) processes M (t, f) and generates two orthogonal signals. FIG. 6 shows two examples of the orthogonal signal generation method in the prior art. Non-Patent Document 1 uses Block (600) and uses a fractional delay all-pass filter to derive a reverberation signal that is orthogonal to the downmix signal M (t, f). Block (604) represents an all-pass filter connected in series. It is also possible to use a decorrelator other than the above. For example, in Non-Patent Document 2, Block (602) is used, and after processing M (t, f) in the common all-pass filter (606), the processed M (t, f) is a delay that is relatively prime to each other. Two comb filters (608) and (610) having characteristics are uncorrelated (mutually-prime orders). In the following description, the decorrelator (600) is assumed.

In the first embodiment of the present invention, the module (504) performs binaural queues Border, ILD (l, b), ICC (l, b), and ICCH (l, b) for each band indicated as (l, b). ), The mixing coefficients g _L (l, b), g _R (l, b), θ _L (l, b), and θ _R (l, b) are obtained. Module (506) then mixes the mixing factors g _L1 (l, b), g _L2 (l, b), g _R 1 (l, b), and g _R2 (l, b) based on the determined mixing coefficients. ) Is calculated.

In order to simplify the description, the notation of (l, b) is omitted in the following formulas.
Based on the downmix processing at the encoder, the relationship between the L, R, and M energies is derived as follows.

Conventionally, ILD and ICC are defined as follows.

For this reason, when the definitions of ILD and ICC are substituted into the equation E _M , the gain coefficients necessary to amplify M to the level of the separated channels L ′ and R ′ are as follows.

FIG. 7 is a diagram geometrically showing how “L” and “R” are separated from M in the vector relationship (Patent Document 2). In the figure, θ _L and θ _R indicate the degree of separation. For low frequencies, (θ _L + θ _R ) is set to θ = cos ⁻¹ (ICC), and for high frequencies (> 1.5 kHz) (θ _L + θ _R ) is set to θ = cos ⁻¹ ( ICCH) for the same reason as described above. Applying the trigonometric function to the vertical triangle shown in FIG.

復号化器ではオリジナルのＬおよびＲを利用できないため、相関性のない二つの信号 をモジュール（５０６）においてミックスして、上記分離をシミュレーションする。図 ８に示すように、前記相関性のない二つの信号は直交的なベクトル関係を有している。 Similarly,

Since the original L and R cannot be used in the decoder, two uncorrelated signals are mixed in the module (506) to simulate the separation. As shown in FIG. 8, the two signals having no correlation have an orthogonal vector relationship.

In Non-Patent Document 1, two uncorrelated signals are a downmix signal S1 = M and an uncorrelated signal S2 = _Mrev derived from M. In the present invention, mixing is performed by scaling M and M _rev using mixing factors g _L1 , g _L2 , g _R1 , and g _R2 , followed by vector addition. g _L1 , g _L2 , g _R1 , and g _R2 are derived from g _L , g _R , θ _L , and θ _R. This is because M _rev is an all-pass version of M, so | M | = | M _rev |.

As a premise for synthesizing the left channel L ′, the following two requirements must be satisfied.

A mixing factor for deriving the left channel can be obtained by solving the above two simultaneous equations.

Similarly, a mixing factor for deriving the right channel can be obtained.

Finally, to synthesize the two channels L ′ and R ′, a mixing factor is used as follows.

  In module (508), the separated channels L '(l, b) and R' (l, b) are inverse transformed to form time domain signals L '(t) and R' (t).

Embodiment 2 of the present invention shows a method in which the above channel separation method is applied to multi-channel. In the present embodiment, the description will be made using the 5-1-5 case. Further, the following formula is assumed as a formula for downmix.

In the above equation, L and R indicate two front (front) channels, L _S and R _S indicate two rear (rear) channels, and C indicates a central (center) channel.

In the encoding process in the case of 5-1-5 shown in FIG. 9, four binaural queue sets are generated by performing the process four times for four combinations of channels in the module (400) shown in FIG. . For example, to generate the first binaural cue set, the C channel and the intermediate downmix channel (L + 0.707SL _S + R + 0.707R _S ) for block (900) (same as module (400) in FIG. 4). Enter. Similar processing is performed in the modules (902) to (906). The generated four binaural queue sets are used to separate the downmix channel M into L, R, L _S , R _S and C in the multi-stage decoding process.

FIG. 10 is a diagram showing the multistage decoding process. Similarly to the case of 2-1-2 shown in FIG. 5, the _MMF is generated by performing QMF conversion (1000) and decorrelation processing (1002) on the downmix channel M.

The binaural cue set 1 is processed in the mixing coefficient calculation module (1004) to generate two mixing factor sets (g _L1 , g _L2 ) and (g _R1 , g _R2 ). This process is performed to separate M into C and M ₁ = (L + 0.707L _s + R + 0.707R _s ). From [Expression 15], it is easy to obtain M = 0.293C + 0.707M _1, and 0.293 and 0.707 are used as weighting values.

The binaural cue set 2 is processed in the mixing coefficient calculation module (1006) to generate two mixing factor sets (g _L3 , g _L4 ) and (g _R3 , g _R4 ). This process is performed to separate M ₁ into M ₂ = (L + R) / 2 and M ₃ = (L _s + R _s ) / 2. From [Equation 15], it is easy to obtain M ₁ = 0.586M ₂ + 0.414M ₃ , and 0.586 and 0.414 are used as weighting values.

The binaural cue set 3 is processed in the mixing coefficient calculation module (1008) to generate two mixing factor sets (g _L5 , g _L6 ) and (g _R5 , g _R6 ). This process is performed to separate M ₂ into L and R. Since M ₂ = 0.5L + 0.5R, 0.5 is used as the weighting value.

The binaural cue set 4 is processed in the mixing coefficient calculation module (1010) to generate two mixing factor sets (g _L7 , g _L8 ) and (g _R7 , g _R8 ). This process is performed to separate M ₃ into Ls and Rs. Since M ₃ = 0.5L _s + 0.5R _s , 0.5 is used as the weighting value.

  The channel mixing modules (1012) to (1020) combine mixing factors in a series of matrix operations to obtain an overall mixing factor. First of all, please note the following points.

である場合、その直交信号（＋π／２で回転）は以下のようになる。

, The quadrature signal (rotated at + π / 2) is as follows:

Therefore, in matrix form,

In order to obtain L ′, mixing factors M → M ₁ → M ₂ used for a series of channel separation processes are combined. The mixing formula for L ′ is

Ｍ‐＞Ｍ₁‐＞Ｍ₃から

Ｍ‐＞Ｍ₁‐＞Ｍ₃から

Ｍから

Similarly, other mathematical formulas for mixing can be obtained in the modules (1014) to (1020).
From M-> M _1- > M ₂

From M-> M _1- > M ₃

From M-> M _1- > M ₃

From M

  Inverse QMF modules (1022)-(1030) convert all combined channels into time domain signals.

(Other variations)
Although the present invention has been described based on the above embodiment, it is needless to say that the present invention is not limited to the above embodiment. The following cases are also included in the present invention.

  (1) Each of the above devices is specifically a computer system including a microprocessor, a ROM, a RAM, a hard disk unit, a display unit, a keyboard, a mouse, and the like. A computer program is stored in the RAM or hard disk unit. Each device achieves its functions by the microprocessor operating according to the computer program. Here, the computer program is configured by combining a plurality of instruction codes indicating instructions for the computer in order to achieve a predetermined function.

  (2) A part or all of the constituent elements constituting each of the above-described devices may be configured by one system LSI (Large Scale Integration). The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on a single chip, and specifically, a computer system including a microprocessor, ROM, RAM, and the like. . A computer program is stored in the RAM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  (3) Part or all of the constituent elements constituting each of the above devices may be configured from an IC card that can be attached to and detached from each device or a single module. The IC card or the module is a computer system including a microprocessor, a ROM, a RAM, and the like. The IC card or the module may include the super multifunctional LSI described above. The IC card or the module achieves its function by the microprocessor operating according to the computer program. This IC card or this module may have tamper resistance.

  (4) The present invention may be the method described above. Further, the present invention may be a computer program that realizes these methods by a computer, or may be a digital signal composed of the computer program.

  The present invention also provides a computer-readable recording medium such as a flexible disk, hard disk, CD-ROM, MO, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray Disc). ), Recorded in a semiconductor memory or the like. The digital signal may be recorded on these recording media.

  In the present invention, the computer program or the digital signal may be transmitted via an electric communication line, a wireless or wired communication line, a network represented by the Internet, a data broadcast, or the like.

  The present invention may be a computer system including a microprocessor and a memory, wherein the memory stores the computer program, and the microprocessor operates according to the computer program.

  In addition, the program or the digital signal is recorded on the recording medium and transferred, or the program or the digital signal is transferred via the network or the like, and executed by another independent computer system. It is good.

  (5) The above embodiment and the above modifications may be combined.

  The present invention is applicable to a home theater system, a car audio system, an electronic game system, and the like.

A typical binaural cue coding system. Spatial speech coding processing in the 2-1-2 case. Division of time and frequency to form a frequency band. Binaural queue extraction block. 2-1-2 spatial audio decoding processing in the case. Two decorrelators. A vector relationship between a pair of audio signals and their downmix signals. Signal synthesis by the sum of two orthogonal signal vectors. Part of the spatial speech coding process in the 5-1-5 case. 5-1-5 Spatial speech decoding process in case.

Explanation of symbols

200 Conversion Module 202 Downmix Module 204 ILD Module 206 ICC Module 208 ICCH Module 400 2-1 BCC Coding Module 500 QMF Filter Bank 502 Correlator Correlator 504 Mixing Coefficient Calculation Module 506 Channel Mixing Module 508 QMF ^-1 Filter Bank

Claims (15)

An apparatus for encoding a plurality of audio channels as spatial audio information,
(A) Calculate the downmix channel,
(B) converting the plurality of audio channels and downmix channels into a time-frequency representation, dividing them into intermediate frequency bands along the frequency axis;
(C) deriving a channel separation step for separating the downmix channel into individual audio channels in a multi-stage decoding process;
(D) For each channel separation step, determine a boundary in the time direction for further dividing the intermediate frequency band into frequency bands;
(E) For each channel separation step and each frequency band, calculate an inter-channel level difference queue (ILD);
(F) For each channel separation step and each frequency band, calculate an inter-channel coherence queue (ICC);
(G) A high frequency inter-channel correlation queue (ICCH) is calculated for each channel separation step and each high frequency band. The apparatus according to claim 1, wherein, for each channel separation step, one composite downmix signal composed of a plurality of signals is input, and each of the one composite downmix signal is composed of one or a plurality of signals. A device characterized in that it is divided into two composite downmix signals. The apparatus according to claim 1, wherein the boundary is arranged at a temporal position where a large change in the ILD appears in the time direction. The apparatus according to claim 1, wherein the ILD queue is a ratio of energy of two composite signals in a frequency band. The apparatus according to claim 1, wherein the ICC queue is used for measuring a phase correlation between two composite signals in a frequency band. The apparatus according to claim 1, wherein the ICCH queue is used to measure a waveform correlation, not a phase, between two composite signals in a frequency band. An apparatus for decoding spatial audio information into a plurality of audio channels,
(A) Convert the downmix channels into a time-frequency representation, divide them into intermediate frequency bands along the frequency axis,
(B) performing processing by a correlator on the downmix channel to form an inverse correlation channel of the downmix signal;
(C) For each channel separation step, input all binaural cue sets each including Border, ILD, ICC, and ICCH into a mixing coefficient calculation (MCC) module to derive a mixing factor;
(D) In the channel mixing (CM) module, by combining the mixing factors, an overall mixing factor for each channel is calculated,
(E) In the CM module, the uncorrelated signal and the overall mixing factor are mixed to generate the individual signals,
(F) A device that restores multi-channel speech by inversely transforming all individual signals from a time-frequency representation into the time domain. The apparatus according to claim 7, wherein the uncorrelated signals are orthogonal to each other. The apparatus according to claim 7, wherein the MCC generates two mixing factor sets to be added to the two composite signals output for the corresponding channel separation step, respectively. 10. Apparatus according to claim 7 and 9, wherein the mixing factor is a function of a gain factor and a degree of separation. 11. The apparatus according to claim 7, 9 and 10, wherein the gain factor and the degree of separation are calculated considering a vector relationship between two composite signals predicted from ILD, ICC and ICCH. A device characterized by that. 8. The apparatus according to claim 7, wherein the overall mixing factor is calculated by adding the mixing factors respectively derived at the corresponding channel separation stages by a series of matrix operations. The apparatus according to claim 7, wherein in the mixing process, the downmix signal is separated from the downmix signal by using an orthogonal and uncorrelated signal in a vector relationship. A device characterized by realizing a desired degree of separation between them. 14. The apparatus according to claim 7 or 13, wherein in the mixing process, a downmix signal and an uncorrelated signal are scaled using the mixing factor, and these signals are added to a vector space. Also, it is possible to generate a separation signal in which the degree of separation from the downmix signal is a desired degree and the signal strength is a desired intensity.
A device characterized by that. 15. The apparatus according to claim 12, 13 and 14, wherein in the matrix operation, a downmix signal and an uncorrelated signal of a current channel separation stage are obtained until an input downmix signal and its uncorrelated signal are reached. , Repeatedly derived as a function of the dowmix signal and uncorrelated signal of the previous channel separation stage.
A device characterized by that.

Images