KR20060059882A - Audio coding - Google Patents

Abstract

A method of classifying a spectro- temporal interval of an input audio signal (x(t)) is disclosed. A spectro- temporal interval of the input audio signal is first modelled (62...71) according to a perceptual model to provide a first representation (Rep 1). The spectro-temporal interval is then modelled (62...71) using a modified noise substituted input signal according to the same perceptual model to provide a second representation (Rep 2). The spectro-temporal interval is then classified as being noise or not based on a comparison of the first and second representations.

Description

Audio coding

The present invention relates to a method of audio signal coding.

The operation of encoders such as MPEG encoders is well known. In one implementation of FIG. 1, the input PCM (Pulse Code Modulated) signal x (t) is used to generate 1024 filters 11 with respective transfer functions H ₁ ... H ₁₀₂₄ . A sub-band filter bank (SBF) 10 is provided. Each filtered signal is decimated and then fed to a scaler (SC) 12, which determines the appropriate scale factors for each band. Separately, a masking threshold and bit allocation calculator (MT / BA) 13, which typically operates in some form of psycho-acoustic model, determines the bit allocation for each frequency band, where the bit rate is Balance the distortion introduced during quantization. Then, each filtered and scaled signal is assigned to the assigned bit rate before being supplied to the multiplexer (MUX) 15 where the final audio stream AS, which includes the quantized signals, scale factors and bit allocation information, is generated. It is thus quantized (Q) 14.

It is known that some spectral and / or temporal parts of audio signals can only be represented in a very efficient manner (eg 4 kb / s to 10 kb / s) with noise model description.

Thus, with respect to FIG. 1, the input signal x (t) may be supplied to a selection element (Sel) 16 that classifies frequency bands during temporal sections as noise or non-noise. When it is determined that spectro-temporal intervals are noise, the selection element 16 instructs the multiplexer 15 not to code sub-band signals during this interval. The spectral-temporal interval of the input signal x (t) is instead modeled as a noise analyzer (NA) 17, the output of which is quantized according to the available bit rate (Q) (18).

However, the notorious problem is to determine which part of the audio signal can be represented by noise. This decision is based on the assumption that the modeling part of the noisy audio signal will not result in a reduction in quality. This will also result in an increase in the efficiency of encoding the signal.

Schulz, D., 1996, J. Audio Eng. Soc., Vol. 44, pp 593-598, "Improving audio codecs by noise substitution," shows that the statistical signal characteristics of a signal can be derived to perform the classification. Exemplary techniques published by Schultz include:

Tracing of spectral peaks in consecutive spectra

Use of predictors in the frequency domain

Use predictability in the time domain as a transversal filter.

In the latter two examples, it is assumed that there are many predictable signals, which are further tonalized and that such predictability is opposed to noise.

Other techniques are based on the analysis of spectral flatness of a frame (typically over a short duration, for example 10 ms to 20 ms). Again, the flatter the spectrum, the more noise is considered to be present.

Hel, Jay. Herre, J. Schulz, D., 1998, published in Proc. 104th convention of the Audio Eng. The statistical methods of "Extending the MPEG-4 AAC codec by perceptual noise substitution" published by Soc, Amsterdam, preprint 4720 are mentioned in the context of MPEG 4 AAC. . Here, the spectral-temporal intervals correspond to scale-factor-bands and frames, and bit rate saving is done when they are modeled by noise.

However, it will be appreciated that the prior art signal statistical criteria do not necessarily match the criteria employed by the observer, ie the possible matching between these criteria is somewhat consistent.

According to the invention, a method according to claim 1 is provided.

The present invention is based on noise classification of spectral-temporal sections of typical audio signals using a cognitive or psycho-acoustic model. The present invention is based on the predicted audibility of noise substitution, i.e. if the noise substitution is predicted to be inaudible to the observer, this does not lead to cognitive degradation.

Embodiments of the invention will now be described with reference to the accompanying drawings, by way of example.

1 illustrates a conventional MPEG encoder in which selected spectral-temporal portions of an audio signal are represented by noise model parameters.

2 illustrates the operation of an improved selection element in accordance with an embodiment of the present invention that may be operated within the encoder of FIG.

3 is a block diagram of a known psycho-acoustic based signal comparison model.

4 is a block diagram of a preferred embodiment of a psycho-acoustic based signal comparison model for use in the selection element of FIG.

FIG. 5 shows the power spectrum R _fnr (f) of the harmonic tone-complex generated by the FFT element of the model of FIG. 4. FIG.

FIG. 6 illustrates the power spectrum R _fnr (f) of Gaussian noise generated by the FFT element of the model of FIG. 4. FIG.

7 shows an encoder according to a second embodiment of the present invention.

8 illustrates the operation of a selectable element operable within the encoder of FIG.

9 (a) and 9 (b) show the modulation spectral output (P ₂₅ ) of one of the filters 25, 18 of the filterbank of the model of FIG. 4 for the harmonic tone complex and noise input signal, respectively. , ₁₈ ) and input R ₂₅ .

In a first embodiment of the present invention, an improved selection element is employed in the MPEG coder of the type shown in FIG. Determine whether or not.

Here, referring to FIG. 2, generally, an improved selection element (Sel) 16 ′ replaces noise modeling for each of a plurality of frequency bands (i) during the interval (n) of the input signal (x). Test it repeatedly. Preferably, the selection element is tested over a time period exceeding the base interval length of the encoder.

In this embodiment, the interval t (n) of the PCM format input signal x (t) surrounding the test interval n is separated into a sequence of nine short overlapping segments s1, s2 ... . Each of these segments is windowed to a square root Hanning window (or any other analysis window) in the segmentation unit 42. (It will be appreciated that a certain number of sections are not critical to implementing the present invention and for example 8 or 11 sections may also be used). At the same time, the signal x (t) for the interval t (n) is provided to the psycho-acoustic analyser 52 as input I / PI.

A Fast Fourier Transform (FFT) is applied to each time-domain windowed signal (... s1, s2 ...) to generate respective complex frequency spectral representations of the windowed signals (step 44).

For each representation and each frequency band i, the noise analyzer / synthesizer 46 provides the rest of the spectrum unchanged in the noise modeled signal for the frequency band i. This noise modeled signal is preferably based on the same model used by the noise analyzer (NA) 17 in the encoder.

This selection element then takes the inverse FFT of each noise replaced signal to obtain time domain signals (..s'1 (i), s'2 (i) ...) (step 48). In step 50, the separated segments are first recombined by windowing back into the square root hanning window (or any other composite window) and applying an overlap-add method. This results in a long PCM, signal x '(t) (i), corresponding to each segment i replacing noise over the interval t (n). The signals x '(t) (i) are then sent to the psycho-acoustic analyzer (PA) 52 as a series of test input signals I / P2 (i). In the matrix shown at the bottom of FIG. 2, a symbolic representation of the modified signal is shown, where noise is replaced in the i th frequency band. Along the horizontal axis, time is shown and along the vertical axis, the frequency band number fbnr corresponding to the scale factor bands used in the AAC encoder is shown. The dots indicate areas containing the original signal samples, and the bars indicate areas with replaced noise. Gray bars indicate areas where noise classification is applied.

A cognitive or psycho-acoustic model within the analyzer 52 is used to calculate the difference (decrease in quality) between the modified input signals I / P2 (i) and the original signal I / P1. If this cognitive difference does not exceed a certain reference value, the frequency band i for the middle spectral-temporal interval, i.e., interval n, among the nine intervals replaced by noise may actually be replaced by the noise model parameters. It is estimated. In this way, all the spectral-temporal sections are studied one by one to make a decision on noise replacement for all sections.

Using this embodiment, which makes a decision on only one of the nine replaced intervals based on the results of the perceptual model, it is more reliable for noise replacement than testing and replacing only a single interval at a time. It was found to make a decision.

After all the spectral-temporal intervals have been evaluated in this manner, the analyzer 52 indicates to the multiplexer MUX of FIG. 1 that the actual noise replacement may be made for any of the frequency bands of the interval n.

In a preferred embodiment, testing is always performed on the original signal with only noise replaced in the frequency band i being tested, i.e., the analyzer 52 has no noise from the interval n-1 to the band i-1. Even if it is determined that it has been replaced, the original signal is employed when testing band i in interval n.

The multiplexer then suitably and in particular for quantizer 18 or sub-band filter (NA) for noise analyzer (NA) with respect to savings in bitrate that can be provided by switching between noise and sub-band filter models. Select the data to be encoded from any one of the quantizer (s).

The selection element 16 'also communicates with either or both of the sub-band filters 11 and the noise analyzer 17 or the quantizers 14, 18 to switch them in and out as appropriate to the system. Reduce the overall processing performed by However, this requires that the selection element be executed before the noise analyzer 17 and sub-band filter 10 elements, resulting in undesirable lag in the encoder. Thus, in carrying out the embodiments described above, the lags need to be balanced against processing overhead.

In a particularly preferred embodiment of the first embodiment described above, the cognitive model employed in the analyzer 52 is generally a Dow, T., Fuschel, D., Colausch, A quantitative model of the "efficient" signal processing in "audible systems" published in June 1996 by Kohlrausch, A., J. Acoust. Soc. Am., Vol 99, 3615-3631. effective signal processing in the auditory system) "; And Dow, T., Kollmeier B., Kohlrausch, A., J. Acoust. Soc. Am., Vol. 102, 2892-2905, which is based on a model of "Modeling auditory processing of amplitude modulation" published in November 1997 (Figure 3).

In Dau, the input signal I / P1 or I / P2 is first transmitted through an auditory filterbank 62. It is known that each position on the basement-membrane in the cochlea of a person has certain bandpass-filter characteristics. Thus, filterbank 62 models the frequency-place transformation of the base film by generating a plurality of (x) band-pass filtered time domain signals that are fed to the next step in the model (FIG. 3). Each of the following stages operates on each of the filterbank output signals, but only processing for one of the x signals is shown).

The next step is a haircell model that includes half-wave rectification 63, low pass filtering 64 with a cutoff frequency of 1 kHz, and down sampling 65 of each filtered signal. Here, the conversion of the mechanical oscillations of the basement membrane into the receptor potentials in the inner hair cells is approximated. The next step describes the adaptive characteristics of the hearing surroundings, including feedback loops 66.

The modulated or linear filterbank 67 then describes the temporal pattern processing of the auditory system. The modulation filterbank includes a total y filters divided into two sets, each having a different scaling. The first set includes a filter with a bandwidth of 2.5 Hz, with the following filters rising up to 10 Hz with a constant bandwidth of 5 Hz. For frequencies between 10 Hz and about 1000 Hz, the second set has an algebraic scaling where the ratio Q = center frequency / bandwidth = 2 is constant such that the y filters are entirely.

In Dau, the modulation filterbank 67 provides a time-domain modulation spectrum. Thus, a matrix of x * y of such modulation spectra is generated to represent each input signal. Internal noise 68 is then added to each modulated spectral signal to model the limited performance resolution of the auditory system.

For each input signal, each matrix representation (Rep 1, Rep 2) 70 is fed to a detector 69 which determines the difference D between the two representations. This amount can be compared with a predetermined threshold to indicate whether the difference between the signals is audible.

Thus, each individual matrix cell in Dau is a time signal, i.e. for each auditory filter and each subsequent modulation filter, a template generated from I / P2 to determine whether a particular test-signal (or distortion) is audible ( time signal from I / P1 compared to the template).

Thus, applying Dau directly to the problem of determining whether noise replacement can be audible, the entire temporal structure of the signal is used in the decision process. Thus, every detail of the replaced noise token can result in predicted distortion. Indeed, listeners are not sensitive to certain details of the noise signal. In other words, each different noise token that can be replaced provides a different internal representation. Therefore, it is very unlikely that one particular replaced noise token will provide an internal representation very similar to the internal representation due to the original (unmodified) signal.

On the other hand, Figure 4 shows the main steps of a modified psycho-acoustic model on which analyzer 52 of the preferred embodiment is based. First, for brevity, it will be appreciated that the adaptation loops 66 and noise adder 68 of FIG. 3 are not used. However, one or both of these stages can be employed if desired.

However, apart from Dau's time-based solution, the embodiment of FIG. 4 converts the time domain signals generated by the hair cell model into respective frequency domain representations by a transform unit (FFT) 71. Modulation filters 67 'are then applied in the spectral domain (weighting function) to generate a plurality of modulation spectra for each of the x original signals.

More specifically, for each of the x time signals supplied to the conversion unit 71, the power spectrum R _fnr (f) is calculated for a period corresponding to about 100 ms of the input signal. Typically, the noise replaced part (if provided) is in the middle of this interval. To convert to the modulation spectrum 67 ', the weighting functions w _{mfnr , fnr} (f) are defined, where' mfnr 'is the index of the weighting function (or number of modulation filters) and' fnr 'is the filterbank ( Number of auditory filter channels from 62) and w _mfnr , _fnr (f) is a function of frequency. For low frequencies, the bandwidths of the individual filters 67 'are small and constant (e.g., 10 Hz to 50 Hz) and above a certain frequency, these filters preferably have a constant Q between 1 and 4. . The shape of the window function may be, for example, an amplitude transfer function of a Hanning window shape or a gamma-tone filter. In a preferred implementation, the minimum filter width is 50 Hz and Q = 2. You will see that the lowest frequency weighting function is centered at 0 Hz to cover only the upper half of the filter shape (everything above the maximum).

The weighting functions generate a series of numbers (P _mfnr , _fnr (f)) used as an internal representation that is sublimated and multiplied by the power spectra and fed to the _averager 70 '.

To illustrate this, FIGS. 5 and 6 show the power spectra R _fnr (f) of harmonic tone-complex and Gaussian noise, respectively, provided as inputs to the filterbank 67 '. 9 (a) and 9 (b) are diagrams of filter banks 67 'for the harmonic tone complex and noise input signals, respectively, having an input frequency R ₂₅ corresponding to FIGS. 5 and 6 and a fundamental frequency of 100 Hz. The modulation spectral output P ₂₅ , ₁₈ of one of the filters ₂₅ , _{18 is} shown. The two input signals consist of the same spectral density and total level. However, it is evident that the filters P ₂₅ , ₁₈ (f) have a higher average output level for the harmonic tone complex than for this noise signal. Thus, the summed values M ₂₅ , ₁₈ will be different. For the noise signal, M is 0.0054, while for harmonic tone complexes, M is 0.0093, approximating the factor of the two differences. The matrix of values M provides a quite different representation for noise and harmonic tone complex signals, which shows that it is possible to classify noise signals using this model.

In the model of FIG. 4, the powers P _mfnr , _fnr (f) for each modulation spectrum are summed (70 ′) to produce a value for each cell in the matrix M. In this way, the activity M (fnr, mfnr) in each modulation filter averaged over a certain time (9 frames) is determined. This average is not sensitive to the specific details of the noise signal, which avoids the problem of using the Dau model described above. The activity for each filter for one signal can then be compared with the corresponding activity M 'for another signal processed in parallel to provide a cognitive measure D of the difference between the signals. have:

The value D can then be compared to a criterion for determining whether noise replacement is allowed. It should be noted that this criterion may be frequency dependent. For example, for low frequencies, this criterion may be lower and proportional to the bandwidth of auditory filters, and for high frequencies this criterion may be constant.

In addition, the selection element 16 ′ or the analyzer 52 of FIG. 2 models the noise with more than a threshold number of adjacent frequency bands for more than successive intervals before instructing the multiplexer (MUX) to switch the noise model. This may be necessary because the required saving of the bitrate is done by swapping into the noise model only when these thresholds are exceeded.

In experiments, the embodiment described above is tested for multiple short (300 ms) segments of fixed audio. Due to the 50% to 80% of the bandwidth replaced, it has been found that audio quality comparable to the audio quality of MPEG 1 layer III can be obtained at a bit rate of 96 kbit / sec for mono audio.

In the first embodiment of the invention, the noise is repeatedly replaced and tested. For each test, the model output of the original signal is compared with the model output of the modified signal, i.e. the replaced noise. Based on this comparison, a determination is made as to whether noise is to be replaced. However, it will be appreciated that this method is computationally intensive.

An alternative method makes direct decisions on certain auditory filters 62, 67 ′ and certain time intervals suspected of being good candidate spectral-temporal intervals for noise replacement, eg, those with low energy levels. Will.

In this case, one input signal, i.e., I / P2, comprises a composite noise signal. The model output Rep 2 for this signal is then compared directly with the model output Rep 1 for the original signal. It can be seen that for a given spectral-temporal interval, Rep 2 is precomputed to reduce the computational concentration of this method.

When the difference between Rep 1 and Rep 2 is less than a certain criterion, it can be assumed that the noise can be replaced within a particular spectral-temporal interval since the input audio signal is very similar in terms of noise signal (in terms of recognition) in this interval. .

In the first embodiment, it will be appreciated that masking is originally considered in the decision process. This is useful because any spectral-temporal interval can be replaced by noise without any problem when masked. In alternative implementations, it is not immediately apparent how certain spectral-temporal interval modifications affect the model output. In order to be able to do this, it is useful to consider the extent to which candidate spectral-temporal intervals for noise replacement are masked by other signal components. This can be considered by rating the alternative detectability (det) of the spectral-temporal interval, ie the degree of masking by other components. Also, for example, low energy sections in high power signals have low detectable ratings. The product of the detectability det and the measured value D of the difference obtained during the candidate interval is assumed to be a good indicator of whether noise can be replaced.

This method is much faster than the method of the first embodiment because it requires only deviations of the masking characteristics in a single pass (instead of many) of the original input signal by the model, which can be achieved without extensive computational complexity. to be.

The invention can be applied not only to the MPEG encoder but also to any encoder which parametrically encodes the signal by noise and some other means. 7, in the second embodiment of the present invention, an improved selection element 16 " is employed within the parametric audio encoder 80 to improve discrimination between noise and non-noise spectral-temporal intervals. Let's do it. An example of such a parametric coder is the sinusoidal description of audio signals, which is well suited for the various tonal signals described in European patent application 02077727.2 filed on July 8, 2002 (agent number PHNL020598). Within the encoder, the sinusoidal analyzer 82 converts sequential segments of the input signal x (t) into the frequency domain, where each segment or frame is then amplitude, frequency and possible phase parameters C _s . Modeled using multiple sinusoids When the components of the synthesized sinusoid of the signal are removed from the input signal, the remaining signal can be estimated to contain noise, which is modeled in the noise analyzer 84 to generate the noise codes C _N. Then, each sinusoidal codes and noise codes C _S , C _N are encoded in the bitstream AS. Other components of the signal that can be coded include transient and harmonic complexes, but these are not described herein for brevity.

The present invention is implemented with the following encoder. The original input signal x (t) is first coded by default to provide a combination of noise and sinusoidal codes C _{S (1)} , C _{N (1)} , these coded segments being the element 16 of FIG. 2. 'Is provided as input I / P1 (0) of selection element 16 ".

Then, for each of the plurality of frequency bands i in a given segment n, the sinusoidal analyzer 82 does not encode sinusoidal components within the frequency band, so that the (larger) remaining signal is a noise analyzer. Coded by 84. Then, the generated candidate noise and sinusoidal codes (C _{S (i),} C _{N (i)),} each of which is provided to I / P2 (i) of the selection element (16 "). Based on the resulting distortion D, the set of candidate codes C _{S (i)} , C _{N (i)} can make the most efficient decision with respect to the bit rate and distortion that exceeds a predetermined threshold Will not have

Referring to FIG. 8, as in the first embodiment, for each of the inputs I / P1 and I / P2 (i), a plurality of segments s1, s2 and s'1 (i), s' The codes for 2 (i) are synthesized and combined using the respective Hanning window functions in units 42 'to provide time windowed signals during interval t (n) as inputs to cognitive analyzer 52, which The cognitive analyzer operates as described in connection with the first embodiment. Therefore, the analyzer 52 only makes a determination as to whether modeling of a given band in a given segment is audible with a combination of sinusoids and noise I / P1 compared to noise I / P2 (i). to provide. This is then left in the multiplexer 15 'to determine which sets of codes 1 ... i are employed across the segments (... s1, s2 ...) to signal x (t). It provides an optimal bit rate for coding.

As in the first embodiment, rather than repeatedly testing each interval for a noise-substituted version of the input signal, for a precomputed representation of the noise signal while the candidate spectral-temporal interval of the input signal is only the same interval. The comparison is made to determine whether the candidate interval is noisy.

In either case, this requires that for a parametric coder, the noise classified intervals must be represented by sinusoids or other components, such as harmonic complexes or transients, with possible savings and possible quality improvements in the bitrate. This is because the noise interval will not be represented in particular sinusoids.

In particular using this second embodiment, it will be appreciated that the specified spectral-temporal sections of the audio signal replaced by noise have the same energy as that of the conventionally modeled audio signal.

As discussed above in connection with the two embodiments, in order to improve the noise replacement operation, it has been found that it is important to first determine whether replacement is allowed by replacing noise over a longer period of time. After this, the actual final replacement is only done for a much smaller interval. Although the present invention can be implemented as such, it has generally been found that if noise is classified only in the test interval to be used for later final replacement, rather unreliable classifications result.

However, if using long temporal test intervals has proved problematic, instead of taking such long intervals for classification, a broad spectral interval (with a short duration) can also be used, with the final replacement being only Only in narrower spectral intervals.

Claims (15)

A method of classifying a spectral-temporal interval of an input audio signal x (t), First modeling (62 ... 71) said spectral-temporal interval of said input audio signal according to a cognitive model to provide a first representation (Rep 1); Second modeling (62 ... 71) said spectral-temporal interval using a noise replaced input signal modified according to said cognitive model to provide a second representation (Rep 2); And, Classifying (52) the spectral-temporal interval of the audio signal as being noise or not based on a comparison of the first and second representations. The method of claim 1, wherein the cognitive model is: First plurality of x filters 62 for providing respective band-pass filtered time-domain signals derived from the input audio signal for each of a first plurality of frequency bands; A rectifier (63) and a low pass filter (64) for processing each of the band-pass filtered signals; A converter (71) for providing a frequency spectral representation (R _fnr (f)) of the processed and filtered signals; And, Second plurality of y filters each providing respective band-pass filtered frequency-domain signals P _{fnr , mfnr} (f) derived from each of the transformed signals for each of a second plurality of frequency bands (67 '), Wherein each of the first and second representations is an x * y matrix (M, M ') of filtered frequency-domain information. The method of claim 2, Wherein each of the first and second representations comprises an x * y matrix comprising an integral of the filtered frequency-domain information. The method of claim 1, And the modified noise replaced input signal comprises a temporal section (t (n)) of the input audio signal in which frequency band (i) is replaced with a noise modeled signal. The method of claim 4, wherein The frequency bands (i) of the temporal interval t (n) of the input audio signal are repeated as a noise modeled signal to provide a series of modified input signals respectively corresponding to the candidate spectral-temporal intervals to be classified. Replacing with; Iteratively modeling the series of modified input signals to provide a series of second representations; And, And repeatedly classifying the candidate spectral-temporal intervals based on a comparison of each of the first and second series of representations. The method of claim 1, The spectral-temporal interval of the input audio signal comprises a frequency band selected during the temporal interval of the input audio signal and the modified noise-replaced input signal comprises a noise modeled signal for the frequency band. Temporal Interval Classification Method. The method of claim 6, And wherein said second modeling step is performed only once. The method of claim 6, Determining a degree to which the replacement of noise of the input signal for the selected frequency band is masked by the remainder of the input audio signal, wherein the classification step 52 comprises the first and second Classifying the spectral-temporal interval of the audio signal according to the comparison function of representations and the degree of masking. A method of coding an audio signal, Classifying (16 ', 16 ") the spectral-temporal signal of said audio signal with or without noise according to the steps of claim 1; Modeling at least a portion of the spectral-temporal interval that is classified as noise with noise model parameters (17, 84); And, Encoding (15, 15 ') said noise model parameters in a bit stream (AS). The method of claim 9, Wherein the portion of the spectral-temporal interval comprises a temporal sub-set of the spectral-temporal interval. The method of claim 9, And a portion of the spectral-temporal interval comprises a spectral sub-set of the spectral-temporal interval. The method of claim 9, And wherein the spectral-temporal interval comprises a time period of length greater than the base interval length (s1, s2) in the bit stream. A component for classifying a spectral-temporal interval of an input audio signal x (t), Means (62 ... 71) for modeling the spectral-temporal intervals of the input audio signal according to a cognitive model to provide a first representation (Rep 1); Means (62 ... 71) for modeling the spectral-temporal interval using a noise replaced input signal modified according to the cognitive model to provide a second representation (Rep 2); And, Means (52) for classifying the spectral-temporal interval of the audio signal as being noise or not based on a comparison of the first and second representations. The method of claim 13, Wherein said component is employed to determine whether a spectral-temporal interval is coded using noise model parameters. The method of claim 14, Wherein the encoder is one of a sinusoidal encoder or an MPEG type encoder.

Images