KR20040102164A - Parametric representation of statial audio - Google Patents

Abstract

In summary, this application describes a psycho-acoustically motivated, parametric description of the spatial attributes of multichannel audio signals. This parametric description allows strong bitrate reductions in audio coders, since only one monaural signal has to be transmitted, combined with (quantized) parameters which describe the spatial properties of the signal. The decoder can form the original amount of audio channels by applying the spatial parameters. For near-CD-quality stereo audio, a bitrate associated with these spatial parameters of 10 kbit/s or less seems sufficient to reproduce the correct spatial impression at the receiving end.

Description

Parametric representation of statial audio

In the field of audio coding, it is generally desirable to encode an audio signal, for example, in order to reduce the bit rate of communicating the signal or the storage requirements for storing the signal without inadequately compromising the perceptual quality of the audio signal. This is an important issue when audio signals must be transmitted over limited capacity communication channels or when these signals must be stored on a storage medium having limited capacity.

Prior art solutions in audio coders that have been proposed to reduce the bit rate of stereo program material include:

' Intensity stereo '. In this algorithm, high frequencies (typically above 5 kHz) are represented by a single audio signal (ie mono) combined with time-varying and frequency-dependent coefficient factors.

' M / S Stereo '. In this algorithm, the signal is decomposed into sum (or mid, or common) and difference (or side, or non-common) signals. This decomposition is sometimes combined with major component analysis or time-varying coefficient factors. These signals are then independently coded by a transform coder or waveform coder. The amount of information reduction achieved by this algorithm strongly depends on the spatial characteristics of the source signal. For example, if the source signal is monaural, the different signal is zero and can be discarded. However, when the correlation of left and right audio signals is small (which is often the case), this regime offers little advantage.

Parametric interpretations of audio signals have been of interest for many years, especially in the field of audio coding. It has been found that transmitting (quantized) parameters describing audio signals requires very little transmission capacity to resynthesize perceptually equivalent signals at the receiving end. However, current parametric audio coders are focused on coding monaural signals, and stereo signals are often treated as dual mono.

European patent application EP 1 107 232 discloses a method for encoding a stereo signal having L and R components, wherein the stereo signal is one of the stereo components, the step of capturing parameter information and the level of the audio signal. Represented by the differences. At the decoder, the other stereo component is recovered on the basis of the encoded stereo component and parametric information.

The present invention relates to the coding of audio signals, and more particularly to the coding of multi-channel audio signals.

It is an object of the present invention to solve the problem of providing improved audio coding that produces a high perceptual quality of the recovered signal.

The above and other problems are solved by a method of coding an audio signal, which method

Generating a monaural signal comprising a combination of at least two input audio channels,

Determining a set of spatial parameters indicative of the spatial characteristics of at least two input audio channels, said set of spatial parameters comprising a parameter indicative of a measure of similarity of waveforms of at least two input audio channels,

Generating an encoded signal comprising a monaural signal and a set of spatial parameters.

It has been found by the inventors that multi-channel signals can be recovered with high perceptual quality by encoding multi-channel audio signals such as monaural audio signals and many spatial properties including measures of similarity of corresponding waveforms. . A further advantage of the present invention is to provide efficient encoding of multi-channel signals, ie signals comprising at least a first channel and a second channel, for example stereo signals, four channel signals and the like.

Thus, according to the invention, the spatial properties of the multi-channel audio signals are parameterized. For typical audio coding applications, transmitting these parameters in combination with only one monaural audio signal reduces the transmission capacity required to transmit stereoscopic signals compared to audio coders that advance the channels independently, while Maintain spatial impressions. An important issue is that although people receive the waveforms of the auditory object twice (once in the left ear and once in the right ear), only a single auditory object is perceived at a certain location (or spatial diffusion) at a particular location.

Thus, it would seem unnecessary to describe audio signals as two or more (independent) waveforms, and it would be better to describe multi-channel audio as a set of auditory objects, each with its own spatial characteristics. One difficulty that arises immediately is the fact that it is almost impossible to automatically separate individual auditory objects from a given ensemble of auditory objects, for example a music recording. This problem can be avoided by not dividing the program material in individual auditory objects, but rather by describing the spatial parameters in a manner similar to the effective (peripheral) processing of the auditory system. When spatial properties include a measure of (de) similarity of corresponding waveforms, efficient coding can be achieved while maintaining a high level of perceptual quality.

In particular, the parametric description of the multi-channel audio provided herein relates to the binaural processing model provided by Breebaart et al. This model aims to describe the effective signal processing of Binaural's auditory system. For a description of the processing model of both ears by Breebaart et al., Breebaart, J., van de Par, S. and Kohlrausch, A. (2001a). Binaural processing model based on contralateral inhibition. I. Model setup. J. Acoust. Soc. Am. 110, 1974-1088; Breebaart, J. van de Par, S. and Kohlrausch; A. (2001b). Binaural processing model based on contralateral inhibition. II.Dependence on spectral parameters. J. Acoust. Soc. Am. 110, 1989-1088; And Breebaart, J., van de Par, S. and Kohlrausch; A. (2001c). Binaural processing model based on contralateral inhibition. III. Dependence on tempora parameters. J. Acoust. Soc. Am. See 110, 1105-1117. The following brief interpretation is given to help understand the present invention.

In a preferred embodiment, the set of spatial parameters includes at least one localization queue. Particularly efficient coding is achieved, especially while maintaining a high level of recognition quality, when the spatial properties comprise one or more, preferably two localization cues, as well as a measure of the (non) similarity of the corresponding waveforms.

The term localization cue includes any suitable parameter that conveys information about the localization of the auditory objects that contribute to the audio signal, eg, the direction and / or distance of the auditory object.

In a preferred embodiment of the invention, the set of spatial parameters comprises at least two localization queues comprising a selected one of an interchannel level difference (ILD), an interchannel time difference (ITD) and an interchannel phase difference (IPD). The level difference between channels and the time difference between channels are considered to be the most important localization queues in the horizontal plane.

The measure of similarity of the waveforms corresponding to the first and second audio channels can be any suitable function that describes how similar or dissimilar the corresponding waveforms are. Thus, the measure of similarity may be a parameter that is determined from / to an increasing function of similarity, for example cross-channel cross-correlation (function).

According to a preferred embodiment, the measure of similarity corresponds to the value of the cross-correlation function at the maximum value of the cross-correlation function (known as interference). The maximum interchannel cross-correlation relationship is strongly related to the diffusion (or compression) of the recognition space of the acoustic source, ie by providing additional information not considered by the localization queues, thereby It provides parameters with a small amount of surplus information to be conveyed, thus providing efficient coding.

Alternatively, a function of increasing by other measures of similarity, for example dissimilarity of waveforms may be used. One example of such a function is 1-c, where c is a cross-correlation that can assume values between 0 and 1.

According to a preferred embodiment of the present invention, determining the set of spatial parameters indicative of the spatial characteristics comprises determining the set of spatial parameters as a function of time and frequency.

Our insight is sufficient to describe the spatial properties of any multichannel audio signal by specifying the maximum correlation as a function of ILD, ITD (or IPD) and time and frequency.

In a further preferred embodiment of the invention, the step of determining a set of spatial parameters indicative of the spatial characteristics,

Dividing each of the at least two input audio channels into a corresponding plurality of frequency bands,

For each of the plurality of frequency bands, determining a set of spatial parameters indicative of the spatial characteristics of the at least two input audio channels within the corresponding frequency band.

Thus, the incoming audio signal is (preferably) split into several band-limited signals that are spatially arranged linearly on an EPR-grade scale. Preferably, the analysis filters show partial overlap in the frequency and / or time domain. The bandwidth of these signals depends on the center frequency, depending on the ERB speed. In order, preferably for all frequency bands the following characteristics of the incoming signals:

Level difference or ILD between channels defined by the relative levels of bandwidth-limited signal stemming from the left and right signals,

The interchannel time (or phase) difference (ITD or IPD) defined by the interchannel delay (or face shift) corresponding to the position of the peak in the interchannel cross-correlation function, and

The (non) similarity of waveforms that cannot be considered by ITDs or ILDs that can be parameterized by the maximum inter-channel cross-correlation (i.e. the value of the standardized cross-correlation function at the position of the maximum peak) , Also known as interference).

The three parameters change over time; Since the binaural hearing system is very slow in its processing, the update rate of these properties is rather low (typically tens of milliseconds).

Here, the (slowly) time-changing characteristics are available to the binaural auditory system, and from these time and frequency dependent parameters, it can be assumed that the perceived auditory world is reconstructed by higher levels of the auditory system. have.

One embodiment of the present invention,

One monaural signal constituting a particular combination of input signals, and

Set of spatial parameters: preferably a parameter describing the similarity or dissimilarity of waveforms that cannot be explained by ILDs and / or IDTs for all time / frequency slots (e.g., the maximum value of cross-correlation) and It is aimed at describing a multi-channel audio signal by two localization cues (ILD and ITD or IPD). Preferably, spatial parameters are included for each additional auditory channel.

An important issue in the transmission of parameters is the accuracy of the parameter representation (ie the magnitude of the quantization errors), which is directly related to the unnecessary transmission capacity.

According to another preferred embodiment of the present invention, generating an encoded signal comprising a monaural signal and a set of spatial parameters comprises a set of quantized spatial parameters each introducing a corresponding quantization error relative to the corresponding determined spatial parameter. Generating at least one of the introduced quantization errors is controlled to depend on the value of at least one of the determined spatial parameters.

Thus, the quantization error introduced by quantization of the parameters is controlled in accordance with the sensitivity of the human auditory system with changes in these parameters. This sensitivity is highly dependent on the values of the parameters themselves. Thus, by controlling the quantization error, depending on the values of the parameters, an improved encoding is achieved.

An advantage of the present invention is to provide decoupling of monaural and binaural signal parameters in audio coders. Thus, difficulties associated with stereoscopic audio coders (e.g., audibility of inter-audit uncorrelated quantization noise compared to correlated quantization noise, or inter-hearing in parametric codes encoded in dual mono mode) Phase inconsistency) is strongly reduced.

A further advantage of the present invention is that strong bit rate reduction is achieved in audio coders due to the low update rate and low frequency resolution required for spatial parameters. The associated bit rate for coding spatial parameters is typically less than 10 kbit (see embodiment below).

A further advantage of the present invention is that it can be easily combined with existing audio coders. The proposed framework produces one monosignal that can be coded and encoded by any existing coding strategy. After monaural decoding, the system described herein regenerates a stereoscopic multichannel signal with appropriate spatial properties.

The set of spatial parameters can be used as an enhancement layer in audio coders. For example, a mono signal is transmitted when only a low bit rate is allowed, while including a spatial enhancement layer allows the decoder to reproduce stereoscopic sound.

Note that the present invention is not limited to stereoscopic signals but can be applied to any multi-channel signal including n channels (n> 1). In particular, the present invention can be used to generate n channels from one mono signal when (n-1) sets of spatial parameters are transmitted. In this case, the spatial parameters describe how to form n different audio channels from a single mono signal.

The invention can be implemented in different ways including the above method, and in the following, a method of decoding a coded audio signal, an encoder, a decoder and further generating means, each of which relates to the first method. Produces one or more benefits and advantages described, each having one or more preferred embodiments corresponding to the preferred embodiments described in connection with the first method and disclosed in the dependent claims.

It is noted that the method and the following method may be implemented in software and performed in a data processing system or other processing means caused by the execution of computer-executable instructions. The instructions may be program code means loaded into a memory, for example RAM, from a storage medium or from another computer via a computer network. Alternatively, the described features may be implemented by hardwired circuitry instead of or in combination with software.

The invention further relates to an encoder for coding an audio signal, the encoder comprising:

Means for generating a monaural signal comprising a combination of at least two input audio channels,

Means for determining a set of spatial parameters indicative of the spatial characteristics of at least two input audio channels, wherein the set of spatial parameters comprises a parameter indicative of a measure of similarity of waveforms of at least two input audio channels,

Means for generating an encoded signal comprising a monaural signal and a set of spatial parameters.

The means for generating a monaural signal, the means for determining a set of spatial parameters as well as the means for generating an encoded signal may be produced by any suitable circuit or device, for example general purpose or special purpose programmable microprocessors, digital Signal processors (DSP), application specific integrated circuits (ASIC), programmable logic arrays (PLA), field programmable gate arrays (FPGA), special purpose electronic circuits, or the like, or a combination thereof. It is noted that.

The invention further relates to a device for supplying an audio signal, the device comprising:

An input for receiving an audio signal,

An encoder as described above and next for encoding an audio signal to obtain an encoded audio signal,

An apparatus for supplying an audio signal, comprising an output for supplying an encoded audio signal.

The apparatus may be any electronic equipment or part of such equipment, for example fixed or portable computers, fixed or portable wireless communication equipment or other handheld or portable devices such as media players, recording devices, and the like. . The term portable wireless communication equipment includes mobile telephones, pagers, communicators, ie electronic organizers, smart phones, personal digital assistants (PDAs), handheld computers, and the like.

The input stage may be in analog or digital form, or any suitable circuit or device that receives a multi-channel audio signal via a wired connection, eg a line jack or in a wireless connection, eg a wireless signal, or any other suitable manner. It may include.

Similarly, the output stage can include any suitable circuit or device that supplies the encoded signal. Examples of such output stages include a network interface that provides a signal to a communication network, such as a LAN, the Internet, and communication circuitry that communicates the signal through a communication channel, such as a wireless communication channel. In other embodiments, the output stage can include a device that stores a signal on a storage medium.

The invention further relates to an encoded audio signal, wherein the signal is

A monaural signal comprising a combination of at least two audio channels,

A set of spatial parameters indicative of the spatial characteristics of at least two input audio channels, the set of spatial parameters comprising said set comprising a parameter indicative of a measure of similarity of waveforms of at least two input audio channels.

The invention also relates to a storage medium in which such encoded signals are stored. Here, the term storage medium refers to magnetic tape, optical disk, digital video disk (DVD), compact disk (CD or CD-ROM), mini-disk, hard disk, floppy disk, ferro-electric memory, electrically erasable programmable Read Only Memory (EEPROM), Flash Memory, EPROM, Read Only Memory (ROM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Ferromagnetic Memory, Optical Storage Groups, charge coupled devices, smart cards, PCMCIA cards, and the like.

The present invention further provides

More related to a method of decoding an encoded audio signal, the method comprising:

Obtaining a monaural signal from an encoded audio signal, said monaural signal comprising a combination of at least two audio channels;

Obtaining a set of spatial parameters from the encoded audio signal, said set of spatial parameters comprising a parameter indicating a measure of similarity of waveforms of at least two input audio channels,

Generating a multi-channel output signal from the monaural signal and the spatial parameters.

The present invention further

Further related to a decoder for decoding an encoded audio signal, the decoder comprising:

Means for obtaining a monaural channel from an encoded audio signal, said monaural signal comprising a combination of at least two audio channels,

Means for obtaining a set of spatial parameters from an encoded audio signal, said set of spatial parameters comprising a parameter indicating a measure of similarity of waveforms of at least two audio channels,

Means for generating a monaural signal and a multi-channel output signal from said spatial parameters.

The means may be any suitable circuit or device, for example general purpose or special-purpose programmable microprocessors, digital signal processors (DSP), application specific integrated circuits (ASIC), programmable logic arrays (PLA). It is noted that the present invention can be implemented by field-programmable gate arrays (FPGA), special purpose electronic circuits, or the like, or a combination thereof.

The invention further relates to an apparatus for supplying a decoded audio signal, the apparatus comprising:

An input for receiving an encoded audio signal,

A decoder as claimed in claim 14 for decoding said encoded audio signal to obtain a multi-channel output signal;

An output stage for supplying or reproducing a multi-channel output signal.

This device may be any electronic equipment or part of such equipment as described above.

The input stage can include any suitable circuit or device that receives the coded audio signal. Examples of such inputs include a network interface for receiving signals via a computer network, such as a LAN, the Internet, or a communication circuit for receiving signals via a communication channel, for example a wireless communication channel. In other embodiments, the input stage can include a device that reads signals from the storage medium.

Similarly, the output stage may include any suitable circuit or device for supplying a multi-channel signal in digital or analog form.

These and other aspects of the invention will become apparent and apparent from the embodiments described below with reference to the drawings.

1 shows a flowchart of a method of encoding an audio signal according to an embodiment of the present invention.

2 shows a schematic block diagram of a coding system according to an embodiment of the present invention.

3 shows a filter method for use in synthesizing an audio signal.

4 shows a decorrelator for use in synthesizing an audio signal.

1 shows a flowchart of a method of encoding an audio signal according to an embodiment of the present invention.

In an initial step S1, incoming signals L and R are divided (preferably by a bandwidth that increases with frequency) into band-pass signals indicated by reference numeral 101, so that their parameters are analyzed as a function of time. Can be. One possible method for time / frequency division is to use time-windowing following the conversion operation, but time-continuous methods may be used (eg, filter banks). The time and frequency resolution of this process is preferably employed in the signal; For temporal signals, fine time resolution (dimensions of a few milliseconds) and coarse frequency resolution are preferred, while for non-transient signals, finer frequency resolution and coarser time resolution (dimensions of tens of milliseconds) are preferred. In turn, in step S2, the level difference ILD of the corresponding subband signals is determined; In step S3, the time difference (ITD or IPD) of the corresponding subband signals is determined; In step S4 the amount of similarity or dissimilarity of waveforms that cannot be described by ILDs or ITDs is described. Analysis of these parameters is discussed below.

Step S2: Analysis of ILDs

The ILD is determined by the level difference of the signals at specific times for a given frequency band. One way to determine the ILD is to measure the root mean square (rms) values of the corresponding frequency bands of the two input channels and calculate the ratio of these rms values (preferably expressed in dB).

Step S3: Analysis of ITDs

ITD is determined by the time or phase alignment that provides the best match between the waveforms of both channels. One way to obtain an ITD is to compute a cross-correlation function between two corresponding subband signals and find the maximum. The delay corresponding to this maximum in the cross-correlation function can be used as the ITD value. The second method is to compute the analytical signals of the left and right subbands (ie, calculate the face and envelope values), and use the (average) phase difference between the channels as the IPD parameter.

Step S4: Analysis of Correlation

The correlation is obtained by first finding the ILD and ITD that provide the best match between the corresponding subbands, and then measuring the similarity of the waveforms after compensation for the ITD and / or ILD. Thus, in this framework, correlation is defined as the similarity or dissimilarity of corresponding subband signals that may not belong to ILDs and / or ITDs. A suitable measure for this parameter is the maximum value of the cross-correlation function (ie the maximum value across the set of delays). However, other measures may be used (preferably compensated for ILDs and / or ITDs), such as the relative energy of the difference signal after the ILD and / or ITD compensation compared to the sum signal of the corresponding subbands. This difference parameter is basically a linear transformation of the (maximum) correlation.

In subsequent steps S5, S6 and S7 the measured parameters are quantized. An important issue in the transmission of parameters is the accuracy of the parameter representation (ie the magnitude of quantization errors), which is directly related to the necessary transmission capacity. In this section, various issues related to the quantization of spatial parameters will be considered. The basic concept is based on quantization errors for the so-called perceptible differences (JNDs) of spatial cues. For clarity, the quantization error is determined by the human auditory system's sensitivity to changes in parameters. Because sensitivity to changes in parameters is strongly dependent on the values of the parameters themselves, we apply the following methods to determine discrete quantization steps.

Step S5: Quantization of ILDs

This is known from psychoacoustic studies that the sensitivity to changes in the ILD depends on the ILD itself. When the ILD is expressed in dB, a deviation of approximately 1 dB from the reference value of 0 dB can be detected, while changes in the numerical value of 3 dB are necessary when the reference level difference is an amount equivalent to 20 dB. Thus, the quantization errors can be greater if the signal of the left and right channels have a greater level difference. For example, this can be done by first measuring the level difference between the channels and then by nonlinear (compression) transformation of the obtained level difference and sequentially by linear quantization process or by using an index table for valid ILD values with nonlinear distribution. Can be applied. The example below provides an example of such an index table.

Step S6: Quantization of ITDs

Sensitivity to changes in ITDs can be characterized as having a constant phase threshold. This is in terms of delay times, the quantization step of the ITD should be reduced by frequency. Alternatively, if the ITD appears in the form of phase differences, the quantization steps should be frequency independent. One way to implement this is to take a fixed phase difference as the quantization step and determine the corresponding time delay for each frequency band. This ITD value is then used as the quantization step. Another method is to transmit phase differences according to the frequency-independent quantization regime. It has also been found that above a certain frequency, the human auditory system is not sensitive to ITDs in microstructured waveforms. This phenomenon can only be developed by transmitting ITD parameters down to a certain frequency (typically 2 kHz).

A third bitstream reduction method is to include ITD quantization steps that depend on ILD and / or correlation parameters of the same subband. For large ILDs, ITDs can be coded exactly less. Moreover, when the correlation is very low, it is known that human sensitivity to changes in ITD is reduced. Thus, larger ITD quantization errors can be applied when there is little correlation. An extreme example of this concept is no transmission of ITDs if the correlation is below a certain threshold and / or if the ILD is large enough for the same subband (typically about 20 dB).

Step S7: Quantization of Correlation

The quantization error of the correlation depends on (1) the correlation value itself and possibly (2) the ILD. Correlation values approximating +1 can be coded with greater accuracy (ie, smaller quantization steps), while correlation values approximating zero can be coded with lower accuracy (large quantization steps). An example of a set of non-linearly distributed correlation values is given in this embodiment. The second probability is to use quantization steps for correlations that depend on the measured ILD of the same subband: for large ILDs (ie, one channel dominates in terms of energy), quantization in correlation The errors are big. An extreme embodiment of this principle may be to transmit no correlation values for a particular subband if the absolute value of the ILD for that subband is below a certain threshold.

In step S8, the monaural signal S is generated by generating the dominant component signal from the incoming signal components by determining the dominant signal from the incoming audio signals, for example as the sum signal of the incoming signal components. Such a processor preferably uses the spatial parameters extracted to produce a mono signal, ie first by aligning the subband waveforms using ITD or IPD prior to combining.

Finally, in step S9, the coded signal 102 is generated from the monaural signal and the determined parameters. Alternatively, the sum signal and spatial parameters can be communicated as separate signals on the same or different channels.

The method can be implemented by a corresponding arrangement, for example general purpose or special purpose programmable microprocessors, digital signal processors (DSP), application specific integrated circuits (ASIC), programmable logic arrays. (PLA), field programmable gate arrays (FPGA), special purpose electronic circuits, or the like, or combinations thereof.

2 shows a schematic block diagram of a coding system according to an embodiment of the present invention. The system includes an encoder 201 and a corresponding decoder 202. The decoder 201 receives a stereo signal having two components L and R, and generates a coded signal 203 that includes the spatial parameters P and the sum signal S communicated to the decoder 202. This signal 203 can be communicated via any suitable communication channels 204. Alternatively or in addition, the signal may be stored on an erasable storage medium 214, for example a memory card, which may be transmitted from an encoder to a decoder.

Encoder 201 preferably includes analysis modules 205 and 206 for analyzing the spatial parameters of incoming signals L and R, respectively, for each time / frequency slot. The encoder includes a parameter extraction module 207 for generating quantized spatial parameters; And a combiner module 208 for generating a sum (or dominant) signal comprised of a particular combination of at least two input signals. The encoder further includes an encoding module 209 for generating the resultant coded signal 203 comprising the monaural signal and the spatial parameters. In one embodiment, this module 209 further performs one or more of the following functions: bit rate allocation, framing, lossless coding, and the like.

Synthesis (in decoder 202) is performed by applying spatial parameters to the sum signal to generate left and right output signals. Thus, the decoder 202 includes a decoding module 210 that performs the inverse operation of the module 209 and extracts the parameters P and the sum signal S from the coded signal 203. The decoder further includes a synthesis module 211 for recovering the stereo components L and R from the sum (or dominant) signal and the spatial parameters.

In this embodiment, the spatial parameter description is combined with a monaural (single channel) audio coder to encode the stereo audio signal. Although the described embodiment works on stereo signals, it should be noted that the general concept can be applied to n-channel audio signals (where n> 1).

In the analysis modules 205 and 206, the signals L and R incoming to the left and right, respectively, are divided in various time frames (e.g., each containing 2048 samples at a 44.1 kHz sampling rate), and square root hanning ( Hanning) Windows. In turn, FFTs are computed. Negative FFT frequencies are discarded and the resulting FFTs are partly divided into groups (subbands) of FFT bins. The number of FFT bins summed in subband g depends on the frequency; At higher frequencies, more bins are combined at less frequencies. In one embodiment, FFT bins corresponding to approximately 1.8ERBs (equivalent to rectangular bandwidth) are grouped, resulting in 20 subbands to represent the entire audio frequency range. The number of results (starting at the lowest frequency) of the FFT bins S [g] of each sequential subband is as follows.

Thus, the first three subbands include 4FFT bins, and the fourth subband includes 5FFT bins. For each subband, the corresponding ILD, ITD, and correlation r are computed. ITD and correlation are computed simply by setting all FFT bins belonging to different groups to zero, multiplying the resulting (band-limited) FFTs from the left and right channels, and then inverse FFT transform. The cross-correlation function of the result is scanned for peaks within the interchannel delay between -64 and +63 samples. The internal delay corresponding to the peak is used as the ITD value, and the value of the cross-correlation function at this peak is used as the interchannel correlation of this subband. Finally, the ILD is simply calculated by taking the power ratio of the left and right channels for each subband.

In the combiner module 208, the left and right subbands are summed after phase correction (temporary alignment). This phase correlation consists of following the ITD computed for that subband, delaying the left-channel subband with ITD / 2 and delaying the right-channel subband with -ITD / 2. This delay is performed in the frequency domain by appropriate changes in the phase angles of each FFT bin. In turn, the sum signal is computed by adding phase-modified versions of the left and right subband signals. Finally, to compensate for uncorrelated or correlated additions, each subband of the sum signal is multiplied by a square root (2 / (1 + r)), according to the r correlation of the corresponding subband. If necessary, the sum signal may be transformed into the time domain by (1) insertion of multiple conjugates at negative frequencies, (2) inverse FFT, (3) windowing, and (4) overlap-addition. Can be.

In the parameter extraction module 207, the spatial parameters are quantized and the ILDs (in dB) are quantized to the nearest value outside of the next set I:

The ITD quantization steps are determined by the constant phase difference of each subband of 0.1 rad. Thus, for each subband, the time difference corresponding to 0.1 rad of the subband center frequency is used as the quantization step. For frequencies above 2 kHz, no ITD information is sent.

The interchannel correlation value r is quantized to the nearest value of the following ensemble R:

This will bear the other three bits per correlation value.

If the absolute value of the (quantized) ILD of the current subband is a quantity of 19 dB, no ITD and correlation values are transmitted in this subband. If the (quantized) correlation value of a particular subband is positive, no ITD value is sent for that subband.

In this way, each frame needs up to 233 bits to transmit spatial parameters. With a frame length of 1024 frames, the maximum bit rate for transmission is an amount of 10.25 kbit / s. It should be noted that using entropy coding or different coding, this bit rate may be further reduced.

The decoder includes a combining module 211, where the stereo signal is synthesized from the received sum signal and the spatial parameters. Thus, for the purposes of this description, it is assumed that the synthesis module receives the frequency-domain representation of the sum signal as described above. This indication can be obtained by windowing the time-domain waveform and FFT operations. First, the sum signal is duplicated into left and right output signals. In turn, the correlation between the left and right signals is changed by the decorrelator. In a preferred embodiment, a decorrelator as described below can be used. In turn, each subband of the left signal is delayed by -ITD / 2, and the right signal is delayed by ITD / 2 providing the (quantized) ITD corresponding to that subband. Finally, the left and right subbands are scaled according to the ILD for that subband. In one embodiment, the modification is performed by a filter as described below. To convert the output signals to the time domain, the following steps are performed: (1) insertion of multiple conjugates at negative frequencies, (2) inverse FFT, (3) windowing, and (4) overlap-addition.

3 illustrates a filter method for use in synthesizing an audio signal. In an initial step 301, the incoming audio signal x (t) is segmented into many frames. Segmentation step 301 is divided into frames of appropriate length x _R (t), for example in the range of 500-5000 samples, into 1024 or 2048 samples.

Preferably, segmentation is performed using overlapping analysis and synthesis window functions, thereby suppressing artifacts that may be introduced at frame boundaries (eg, Princen, JP and Bradley, AB: “Analysis / synthesis filterbank design based on time domain aliasing cancellation ", IEEE transactions on Acoustics, Speech and Signal processing, ASSP 34, 1989)

In step 302, each of the frames x _R (t) is transformed into the frequency domain by applying a Fourier transform, preferably implemented as a Fast Fourier Transform (FFT). The frequency representation of the result of the n-th frame x _R (t) includes many frequency components X (k, n), where parameter n indicates the number of frames and parameter k is 0 <k <K, where frequency indicates a frequency bin or frequency component corresponding to ω _k . In general, the frequency domain components X (k, n) are complex numbers.

In step 303, the desired filter for the current frame is determined according to the received time-varying spatial parameters. The desired filter is represented as the desired filter response containing a set of K complex weight factors 0 <k <K, F (k, n) for the n-th frame. The filter response F (k, n) is two real numbers, namely According to its magnitude a (k, n) and its phase It may be indicated by.

In the frequency domain, the filtered frequency components are Y (k, n) = F (k, n) .F (k, n), that is, they are frequency components F (k, n) and filter response F of the input signal. This results in a multiplication of (k, n). As will be apparent to the skilled person, this multiplication in the frequency domain corresponds to the rise of the corresponding filter f _n (t) with the input signal frame x _n (t).

In step 304, the desired filter response F (k, n) is changed before applying it to the current frame X (k, n). In particular, the actual filter response F '(k, n) to be applied is determined as a function of the desired filter response F (k, n) and the information 308 of previous frames. Preferably, this information comprises the actual and / or desired filter response of one or more previous frames according to the following.

Thus, artifacts introduced by changes in the filter response between successive frames can be effectively suppressed, making the actual filter response dependent on the history of previous filter responses. Preferably, the actual form of transform function Φ is chosen to reduce overlap-added artifacts resulting from the dynamically-changing filter responses.

For example, the transform function φ may be a function of a single previous response function. For example, F '(k, n) = Φ ₁ [F (k, n), F (k, n-1)] or F' (k, n) = Φ ₂ [F '(k, n), F '(k, n-1)]. In other embodiments, the transform function may include a floating average over many previous response functions, eg, a filtered version of previous response functions, and the like. Preferred embodiments of the transform function Φ will be described in more detail below.

In step 305, the actual filter response F '(k, n) is determined by the frequency components X (k) of the current frame of the input signal according to Y (k, n) = F' (k, n) .X (k, n). , n) is applied to the current frame by multiplying the corresponding filter response factors F '(k, n).

At step 306, the resulting processed frequency components Y (k, n) are converted back to the time domain resulting in filtered frames y _n (t). Preferably, the inverse transform is implemented as an inverse fast Fourier transform (IFFT).

Finally, in step 307, the filtered frames are recombined into the filtered signal y (t) by an overlap-added method. An efficient implementation of such overlap addition method is disclosed in Bergmans, J. W. M .: "Digital basband transmission and recording", Kluwer, 1996.

In one embodiment, the transform function Φ of step 304 is implemented as a phase-change limiter between the current frame and the previous frame. According to this embodiment, the actual phase distortion applied to the previous sample of the corresponding frequency component The phase change δ (k) of each frequency component F (k, n) compared to is calculated as follows. In other words, .

In turn, the phase component of the desired filter F (k, n) is modified in such a way that the phase change across the frames is reduced, in which case the change can lead to overlap-added artifacts. According to this embodiment, this is achieved by ensuring that the actual phase difference does not exceed a predetermined threshold, for example by simple cutting of the following phase difference.

(One)

Threshold c may be a constant, for example, a constant between π / 8 and π / 3 rad. In one embodiment, the threshold c is not constant but may be a function of time, frequency and / or the like, for example. Moreover, as an alternative to the above limits on phase change, other phase-change-limit functions can be used.

In general, in this embodiment, the desired phase-change across the subsequent time frames for the individual frequency components is transformed by the input / output function P (δ (k)) and the actual filter response F '(k, n) Is given by

F '(k, n) = F' (k, n-1) .exp [jP (δ (k))]. (2)

Thus, according to this embodiment, a transform function P of phase change across subsequent time frames is introduced.

In another embodiment of the transformation of the filter response, the phase limiting process is driven by an appropriate measure of pitch, for example the prediction method described below. This has the advantage that phase jumps between successive frames occurring in signals such as noise can be excluded from the phase-change limiting process according to the present invention. This is advantageous because limiting such phase jumps in signals such as noise can produce more tonality of the noisy signal sound that is often perceived as synthesized or metallic.

According to this embodiment, the predicted phase error θ (k) = (k, n)- (k, n-1) -ω _k -h is calculated. Ω _k denotes a frequency corresponding to a k th frequency component, and h denotes a hop size among samples. The term hop size here means half the difference between two adjacent window centers, ie the analysis length for symmetric windows. In the following, it is assumed that the error is wrapped at intervals [−π, + π].

In turn, the prediction measure P _k for the amount of phase predictability at the k th frequency is _{calculated according} to P _k = (π− | θ (k) | / π∈ [0,1]), where | It represents the absolute value.

Thus, the measure P _k produces a value between 0 and 1 depending on the amount of phase-predictability in the k th frequency bin. When P _k is close to 1, the underlying signal can be assumed to have a high degree of tonality, ie it has a substantially sinusoidal waveform. For such a signal, phase jumps can be easily recognized, for example, by a listener of the audio signal. Therefore, phase jumps should be eliminated in this case. On the other hand, _if the value of P _k is close to zero, the underlying signal can be assumed to be noise. For noise signals, phase jumps are not easily recognized and can therefore be allowed.

Thus, the phase limit function is applied when P _k exceeds a predetermined threshold, i.e., results in a measure P _k > A, followed by the actual filter response F '(k, n).

Here, A is limited by the upper and lower bounds of P, which are +1 and 0, respectively. The exact value of A depends on the actual implementation. For example, A can be chosen between 0.6 and 0.9.

Alternatively, it is understood that any other suitable measure for estimating pitch may be used. In another embodiment, the allowed phase jump c is made in dependence on a suitable measure of pitch, e.g., the measure P _k , thereby allowing larger phase jumps if P _k is large or vice versa.

4 shows a decorrelator for use in synthesizing an audio signal, the decorrelator receiving a monaural signal x and a set of spatial parameters P comprising a parameter representing a cross-correlation r between the channels and a channel difference c; An all-pass filter 401. Note that parameter c is related to the level difference between channels by ILD = klog (c), where k is a constant, i.e., ILD is proportional to the logarithm of c.

Preferably, the all-pass filter includes a frequency-dependent delay that provides a relatively small delay at higher frequencies than at lower frequencies. This can be achieved by replacing the fixed delay of the global-pass filter with a global-pass filter that includes one period of Schroeder-phase complex (eg, MR Schroeder, "Synthesis of low-peak-"). factor signals and binary sequences with low autocorrelation ", IEEE Transact. Inf. Theor. 16: 85-89, 1970). The decorrelator further comprises an analysis circuit 402 which receives spatial parameters from the decoder and extracts the cross-channel cross-correlation r and the channel difference c. Circuit 402 determines the mixing matrix M (α, β) to be considered below. The components of the mixing matrix are fed into a conversion circuit 403 to provide input signal x and filtered signal. Receive additionally. Circuit 403 performs a blend operation according to

(3)

Resulting in output signals L and R.

The correlation between signals L and R is based on r = cos (α) It can be expressed as an angle α between vectors representing each of signals L and R in a space spanned by. As a result, any pair of vectors representing a correct angular distance has a specific correlation.

Thus, signals x and The mixed matrix M which transforms into signals L and R by a given correlation r can be expressed as follows:

(4)

Thus, the amount of all-pass filtered signal depends on the desired correlation. Moreover, the energy of the all-pass signal component is the same (but shifted 180 ° out of phase) in both output channels.

If matrix M is given by

(5)

That is, when α = 90 °, note that it corresponds to Lauridsen decorrelator when it corresponds to uncorrelated output signals r = 0.

To illustrate the problem by the matrix of equation (5), we assume a situation with the highest amplitude panning towards the left channel, i.e. when a particular signal is present only in the left channel. We further assume that the desired correlation between the outputs is zero. In this case, the output of the left channel of the transformation of equation (3) by the mixing matrix of equation (5) Create Thus, this output stage is its all-pass filtered version And the raw signal x in combination with.

However, this is an undesired situation because all-pass filters typically degrade the perceptible quality of the signal. Moreover, the addition of the raw signal and the filtered signal results in comb-filter effects such as perceived coloring of the output signal. In this extreme case, the best solution is that the left output signal consists of the input signal. This is how the correlation of the two output signals can still be zero.

In situations with more moderate level differences, the preferred situation is that a larger output channel contains relatively more raw signals, and a more flexible output channel contains relatively more filtered signals. Thus, in general, it is desirable to maximize the amount of raw signal present at the two outputs and to minimize the amount of filtered signal.

According to this embodiment, this is achieved by introducing different mixing matrices containing additional common rotations.

(6)

Where β is an additional rotation and C is a scaling matrix that ensures that the relative level difference between the output signals is equal to c. In other words,

Inserting the matrix of equation (6) into equation (3) produces the output signals generated by the matrixing operation according to this embodiment:

Thus, the output signals L and R still have an angular difference, that is, the correlation between the L and R signals depends on the signal L and the additional level of each β of the L and R signals and the desired level difference. It is not affected by scaling R.

As mentioned above, preferably, the amount of raw signal x in the summarized output of L and R should be maximized. These conditions can be used to determine the angle β according to

Create the following condition:

In summary, the present invention describes a psychoacoustically stimulated parametric description of the spatial properties of multichannel audio signals. This parametric description allows strong bit rate reductions in audio coders because only one monaural signal should be transmitted and combined with (quantized) parameters describing the spatial characteristics of the signal. The decoder can form the original amount of audio channels by applying spatial parameters. For near-CD-quality stereo audio, the bit rate associated with spatial parameters of 10 kbit or less seems sufficient to reproduce the correct spatial impression at the receiving end. This bit rate can be further scaled down by reducing the spatial and / or temporal resolution of the spatial parameters and / or processing the spatial parameters using intact compression algorithms.

The above embodiments are illustrative rather than limiting of the invention, and those skilled in the art should recognize that many alternative embodiments can be devised without departing from the scope of the appended claims.

For example, the present invention has been described in the context of an embodiment which mainly uses two localization cues ILD and ITD / IPD. In alternative embodiments, other localization queues may be used. Moreover, in one embodiment, the ILD, ITD / IPD, and inter-channel cross-correlation relationship may be determined as described above, but only the cross-channel cross-correlation relationship is transmitted with the monaural signal, thereby transmitting / saving the audio signal. The bandwidth / storage capacity required to do so can be further reduced. Alternatively, cross-channel cross-correlation and one of ILD and ITD / IPD may be transmitted. In these embodiments, this signal is synthesized from the monaural signal based only on the transmitted parameters.

In the claims, any symbols placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "one" or "one" preceding an element does not exclude the presence of a plurality of such elements.

The invention can be implemented by means of hardware and variously programmed computer means comprising a variety of unique elements. In the device claim enumerating several elements, several of these means can be embodied by one and the same item of hardware. The simple fact that certain measures are re-cited in mutually different dependent claims does not indicate that a combination of these measures cannot be used advantageously.

Claims (15)

A method of coding an audio signal, Generating a monaural signal comprising a combination of at least two input audio channels, Determining a set of spatial parameters indicative of the spatial characteristics of at least two input audio channels, wherein the set of spatial parameters comprises a parameter indicative of a measure of similarity of waveforms of at least two input audio channels. Determining a set of spatial parameters to represent; Generating an encoded signal comprising a monaural signal and a set of spatial parameters. 2. The method of claim 1, wherein determining the set of spatial parameters indicative of spatial characteristics comprises determining the set of spatial parameters as a function of time and frequency. The method of claim 2, wherein determining the set of spatial parameters indicative of the spatial characteristics comprises: Dividing each of the at least two input audio channels into a corresponding plurality of frequency bands, -For each of the plurality of frequency bands, determining a set of spatial parameters indicative of the spatial characteristics of the at least two input audio channels within the corresponding frequency band. 4. A method according to any of the preceding claims, wherein said set of spatial parameters comprises at least one localization cue. 5. The method of claim 4, wherein said set of spatial parameters comprises at least two localization cues comprising a selected one of an interchannel level difference and an interchannel time difference and an interchannel phase difference. 6. The method of claim 4 or 5, wherein the measure of similarity comprises information that cannot be described by localization queues. The method of any one of claims 1 to 6, wherein the measure of similarity corresponds to a value of the cross-correlation function at the maximum value of the cross-correlation function. 8. The method of any of claims 1 to 7, wherein generating an encoded signal comprising the monaural signal and the set of spatial parameters comprises generating a set of quantized spatial parameters; Respectively introducing a corresponding quantization error with respect to the corresponding determined spatial parameter, wherein at least one of the introduced quantization errors is controlled to depend on the value of at least one of the determined spatial parameters. An encoder for coding an audio signal, Means for generating a monaural signal comprising a combination of at least two input audio channels, Means for determining a set of spatial parameters indicative of the spatial characteristics of at least two input audio channels, said set of spatial parameters comprising parameters indicative of a measure of similarity of waveforms of at least two input audio channels Means for determining a set of spatial parameters representing the Means for generating an encoded signal comprising said monaural signal and said set of spatial parameters. A device for supplying an audio signal, An input for receiving an audio signal, An encoder as claimed in claim 9 for encoding said audio signal to obtain an encoded audio signal; And an output end for supplying the encoded audio signal. An encoded audio signal, A monaural signal comprising a combination of at least two audio channels, A set of spatial parameters indicative of spatial characteristics of at least two input audio channels, the set of spatial parameters comprising the set comprising a parameter indicative of a measure of similarity of waveforms of at least two input audio channels signal. A storage medium having stored the encoded signal claimed in claim 11. A method of decoding an encoded audio signal, Obtaining a monaural signal from the encoded audio signal, wherein the monaural signal comprises a combination of at least two audio channels; Obtaining a set of spatial parameters from the encoded audio signal, wherein the set of spatial parameters comprises a parameter indicating a measure of similarity of waveforms of at least two input audio channels; Generating a multi-channel output signal from the monaural signal and the spatial parameters. A decoder for decoding an encoded audio signal, Means for obtaining a monaural signal from the encoded audio signal, wherein the monaural signal comprises a combination of at least two audio channels; Means for obtaining a set of spatial parameters from the encoded audio signal, the set of spatial parameters comprising means for obtaining the set of spatial parameters including a parameter indicating a measure of similarity of waveforms of at least two audio channels; Means for generating a multi-channel output signal from the monaural signal and the spatial parameters. An apparatus for supplying a decoded audio signal, An input for receiving an encoded audio signal, A decoder as claimed in claim 14 for decoding said encoded audio signal to obtain a multi-channel output signal; And an output stage for supplying or reproducing the multi-channel output signal.