EP1741039B1 - Information signal processing by carrying out modification in the spectral/modulation spectral region representation - Google Patents

Abstract

Processing of information signals separated according to modulation and carrier components in a more controlled way is made possible by a device for processing an information signal including a unit for converting the information signal to a time/spectral representation by block-wise transforming of the information signal and a unit for converting the information signal from the time/spectral representation to a spectral/modulation spectral representation, wherein the unit for converting is designed such that the spectral/modulation spectral representation depends on both a magnitude component and a phase component of the time/spectral representation of the information signal. A unit then performs a manipulation and/or modification of the information signal in the spectral/modulation spectral representation to obtain a modified spectral/modulation spectral representation. A further unit finally forms a processed information signal representing a processed version of the information signal based on the modified spectral/modulation spectral representation.

Description

The present invention relates to the processing of audio signals, and more particularly to spectral / modulation spectral processing.

In the field of signal processing, for example in the processing of digital audio signals, signals which consist of a carrier signal component and a modulation component frequently exist. In the case of modulated signals, a representation in which the signals are decomposed into carrier and modulation components is often needed to be able to filter, encode or otherwise modify them, for example.

For purposes of audio coding, for example, it is known to subject the audio signal to a so-called modulation transformation. In this case, the audio signal is decomposed by a transformation into frequency bands. Then a decomposition in amount and phase is made. While the phase is not processed further, the amounts per subband are retransformed over a number of transformation blocks in a second transformation. The result is a frequency decomposition of the temporal envelope of the relevant subband into modulation coefficients. Audio codings that consist of such a modulation transformation are, for example, in M. Vinton and L. Atlas, "A Scalable and Progressive Audio Codec", in Proceedings of the 2001 IEEE ICASSP, 7-11. May 2001 , Salt Lake City, United States Patent Application US 2002 / 0176353A1 : Atlas et al., Scalable And Perceptually Ranked Signal Coding And Decoding, 11/28/2002 , and J. Thompson and L. Atlas, "A Non-uniform Modulation Transform for Audio Coding with Increased Time Resolution ", in Proceedings of the 2003 IEEE ICASSP, 6-10 April, Hong Kong, 2003 , described.

An overview of further different demodulation techniques over the full bandwidth of the signal to be demodulated, including asynchronous and synchronous demodulation techniques etc., is given for example by the article L. Atlas, "Joint Acoustic And Modulation Frequency," Journal of Applied Signal Processing 7 EURASIP, pp. 668-675, 2003 ,

A disadvantage of the above-mentioned audio coding schemes using a modulation transformation is the following fact. As long as no further processing steps are performed on the modulation coefficients together with the phases, the modulation coefficients form a spectral / modulation spectral representation of the audio signal which is reversible and perfectly reconstructive, ie can be reconverted back to the original audio signal in the time domain without changes. In these methods, however, the modulation coefficients are filtered in order to reduce or quantize the modulation coefficients according to psychoacoustic criteria to the smallest possible values, so that the highest possible compression rate is achieved. However, this does not generally achieve the desired goal of removing the relevant modulation components from the resulting signal or of deliberately introducing quantization noise in this component. The reason for this is that the phases of the subbands after the inverse transformation of the modified modulation coefficients are no longer consistent with the changed amounts of these subbands and still contain strong components of the modulation component of the original signal. If the phases of the subbands are recombined with the changed amounts, these modulation components or components are again introduced into the filtered or quantized signal by the phase. In other words, provides a modulation transformation followed by a modification of the modulation coefficients in the manner described above, that is to say by filtering the modulation coefficients, together with a subsequent synthesis of the phase and magnitude components, a signal which, in a renewed analysis or modulation transformation, still contains significant modulation components at those points in the spectral range. / Modulation spectral range representation that should be filtered out. Effective filtering is therefore not possible based on the detailed modulation transformation-based signal processing schemes.

US 2003/185411 discloses an apparatus for processing an audio signal in which the result of a first transformation of the audio signal is transformed again. The first transformation separates the audio signal into a magnitude and a phase spectrogram. The second transformation is applied separately to each of these two spectrograms.

There is therefore a need for an audio signal processing scheme that makes it possible to separately process modulated signals having a carrier component and a modulation component according to modulation and carrier component.

The object of the present invention is thus to provide a processing scheme for audio signals, which allows a specific processing of audio signals separately according to modulation and carrier portions.

This object is achieved by a device according to claim 1 and a method according to claim 17.

The core idea of the present invention is that it is possible to achieve a more strictly processing of audio signals separately for modulation and carrier components if the transfer of the information signal from the time / spectral representation or the time / frequency representation into the spectral / modulation spectral representation or the frequency / modulation frequency representation is performed depending on both an amount component and a phase component of the time / spectral representation of the information signal. This eliminates a recombination between phase and magnitude, and thus the reintroduction of unwanted modulation components in the time representation of the processed audio signal on the synthesis side.

Transferring the audio signal from the time / spectral representation to the spectral / modulation spectral representation, taking into account both the magnitude and the phase, involves the problem that the time / spectral representation of the information signal is in fact not only from the audio signal but also from the phase offset the time blocks to the carrier spectral component of the audio signal depends. In other words, the block-wise transformation of the audio signal from the time representation into the time / spectral representation causes the sequences of spectral values obtained per spectral component in the time / spectral representation of the audio signal to be a modulated complex Carrier, which depends only on the asynchronicity of the block repetition frequency to the carrier frequency component of the audio signal. According to the exemplary embodiments of the present invention, a demodulation of the sequence of spectral values in the time / spectral representation of the audio signal per spectral component is therefore undertaken in order to obtain a demodulated sequence of spectral values per spectral component. The subsequent conversion of the demodulated sequences of spectral values thus obtained is carried out by block-wise transforms from the time / spectral representation into the spectral / modulation spectral representation or by blockwise spectral decomposition of the same, whereby blocks of modulation values are obtained. These are manipulated or modified, such as weighted for bandpass filtering to remove the modulation component from the original audio signal with a corresponding weighting function. The result is a modified demodulated sequence of spectral values or modified demodulated time / spectral representation. On the modified demodulated sequences of spectral values thus obtained, the complex carrier is re-modulated, whereby a modified sequence of spectral values is obtained which forms part of a time / spectral representation of the processed audio signal. A return of this representation into the time representation results in a processed audio signal in the time representation or time domain, which can be changed very precisely with respect to modulation and carrier components with respect to the original audio signal.

Fig. 1

ein Blockschaltbild einer Vorrichtung zur Verarbeitung eines Audiosignals gemäß einem Ausführungsbeispiel der vorliegenden Erfindung; und

Fig. 2

eine schematische Skizze zur Veranschaulichung der Funktionsweise der Vorrichtung nach Fig. 1.

Preferred embodiments of the present invention will be explained in more detail below with reference to the accompanying drawings. Show it:

Fig. 1

a block diagram of an apparatus for processing an audio signal according to a Embodiment of the present invention; and

Fig. 2

a schematic diagram for illustrating the operation of the device according to Fig. 1 ,

Fig. 1 shows an apparatus for processing an audio signal according to an embodiment of the present invention. The device of Fig. 1 , indicated generally at 10, includes an input 12 at which it receives the audio signal 14 to be processed. The device of Fig. 1 is exemplary provided to process the audio signal 14 such that the modulation component is removed from the audio signal 14, and thus to obtain a processed audio signal with only the carrier portion. Further, the device 10 includes an output 16 for outputting the carrier portion as the processing result and the processed audio signal 18, respectively.

Internally, the device 10 is divided essentially into a part 20 for transferring the audio signal 14 from a time representation into a time / frequency representation, a means 22 for transferring the audio signal from the time / frequency representation into the frequency / modulation frequency representation, a portion 24 in which the actual processing takes place, namely the modification of the audio signal, and a part 26 for the return of the processed in the frequency / modulation frequency representation audio signal from this representation in the time representation. The said four parts are connected in series between the input 12 and the output 16 in series, the more detailed structure and operation of which will be described hereinafter.

The part 20 of the device 10 comprises a fenestration device 28 and a transformation device 30, which connect to the input 12 in this order. In particular, an input of the fenestration device 28 is connected to the input 12 to obtain the audio signal 14 as a sequence of information values. If the audio signal is still present as an analog signal, this can be converted, for example, by an A / D converter or a discrete sampling into a sequence of information or sampling values. The windowing device 28 forms blocks of the same number of information values from the sequence of information values and additionally performs a weighting with a weighting function on each block of information values, which, for example, can not correspond exclusively to a sine window or a KBD window. The blocks may overlap, such as by 50% or not. The following is merely an example of a 50% overlap. Preference is given to window functions having the property that they enable a good subband separation in the time / spectral representation and that the squares of their mutually corresponding weighting values applied in the overlap region, applied to one and the same information value, add up to one.

An output of the fenestration device 28 is connected to an input of the transformation device 30. The blocks of information values output by the windowing means 28 are received by the transformation means 30. It then subjects the transformation means 30 in blocks to a spectrally decomposing transformation, such as a DFT or other complex transformation. The transformation device 30 thus achieves block-by-block decomposition of the audio signal 14 into spectral components and thus produces, in particular per block of time, as obtained from the windowing device 28, a block of spectral values which comprises one spectral value per spectral component. Several spectral values can be combined into subbands. In the following, however, the terms subband and Spectral component used synonymously. For each spectral component or each subband, a spectral value thus results per time block, or more, if there is a subband summary, which is however not assumed below. Accordingly, the transformation means 30 outputs per spectral component or subband a sequence of spectral values which represent the time profile of this spectral component or of this subband. The spectral values output by the transformation device 30 represent a time / frequency representation of the audio signal 14.

The part 22 comprises a carrier frequency determination device 32, a mixer 34 serving as a demodulation device, a windowing device 36, and a second transformation device 38.

The windowing device 32 comprises an input which is connected to the output of the transformation device 30. It receives there the spectral value sequences for the individual subbands and divides the spectral value sequences per subband - similarly as the fenestration 28 does with respect to the audio signal 14 - into blocks and weights the spectral values of each block with a suitable weighting function. The weighting function may be one of the weighting functions already mentioned above with respect to device 28. The successive blocks in a subband may or may not overlap, again exemplarily assuming a mutual overlap of 50%. In the following it is assumed that the blocks of different sub-bands are aligned with each other, as in the following with reference to Fig. 1 will be explained in more detail. Another approach with offset between the subbands block sequences would also be conceivable. At the output, the windowing device outputs sequences of windowed spectral value blocks per subband.

The carrier frequency determination device 32 also has an input which is connected to the output of the transformation device 30 in order to obtain the spectral values of the subbands or spectral components as sequences of spectral values per subband. It is intended to find out in each subband that carrier component which results from the fact that the individual time blocks from which the individual spectral values of the subbands have been derived have a time-varying phase offset to the carrier frequency component of the audio signal. The carrier component determined per subband outputs the carrier frequency determining device 32 at its output to an input of the mixer 34, which in turn has a further input which is connected to the output of the fenestration device 36.

The mixer 34 is designed such that, per subband, it multiplies the blocks of windowed spectral values as output from the transformation means by the complex conjugate of the respective carrier component as determined by the carrier frequency determining means 30 for the respective subband, whereby Subbands or blocks of windowed spectral values are demodulated.

Demodulated subbands thus result at the output of the mixer 34 or a sequence of demodulated blocks of windowed spectral values results per subband. The output of the mixer 34 is connected to an input of the transformation device 38, so that the latter per subband each other - here exemplarily 50% - overlapping blocks of windowed and demodulated spectral values and transforms these blocks in the spectral / modulation spectral representation or spectrally decomposed to by processing all subbands or spectral components a previously modified only with regard to the demodulation of the subband spectral value sequences Frequency / modulation frequency representation of the audio signal 14 to produce. The transformation underlying the transformation device 38 per subband can be, for example, a DFT, an MDCT, MDST or the like, and in particular also the same transformation as that of the transformation device 30 Fig. 1 By way of example, it has been assumed that the transformations of both transformation devices 30, 38 are DFTs.

Accordingly, at its output, the transformation means 38 successively outputs blocks of values for each sub-band or spectral component, hereinafter referred to as modulation values, representing a spectral decomposition of the blocks of windowed and demodulated spectral values. The blocks of spectral values per subband, with respect to which the transformation means 38 performs the transformations, are aligned with respect to one another in time, so that a time interval always results in a matrix of modulation values composed of one modulation value block per subband. The modulation values are forwarded by the transformation device 38 to the part 24 which has only one signal processing device 40.

The signal processing device 40 is connected to the output of the transformation device 38 and thus receives the blocks of modulation values. In the present exemplary case, since the device 10 is for modulation-rate rejection, the signal processor 40 performs effective low-pass filtering in the frequency domain on the incoming blocks of modulation values, namely, weighting the modulation values with a function increasing from zero to higher or lower Modulation frequencies drops. The thus modified blocks of modulation values pass the signal processing means 40 to the back transfer part 26. That of the signal processing device 40 output modified blocks of modulation values represent a modified frequency / modulation frequency representation of the information signal 14, or in other words, a frequency / modulation frequency representation other than the demodulation by the mixer 34 from the frequency / modulation frequency representation of the modified information signal 18.

The re-transfer part 26 is in turn divided into two parts, namely a part for transferring the processed audio signal 18 from the frequency / modulation frequency representation as output from the signal processing means 40 to the time / frequency representation, and a part for returning the processed one Audio signal from the time / frequency representation in the time representation. The former of the two parts comprises a transformation means 42 for performing a block-wise transformation inverse to the transformation after the transformation means 38, a mixer 46 and an assembly means 44. The second part of the return part 26 comprises a transformation means 48 for performing the transformation of the transformation means 30 inverse blockwise transformation and an aggregator 50.

The inverse transformation means 42 is connected with its input to the output of the signal processing means 40 and transforms the modified blocks of modulation values partially bandwise from the spectral representation back into the time / frequency representation and thus reverses the spectral decomposition to a subsequence of modified blocks of To obtain spectral values. These modified spectral value blocks output by the inverse transformation means 42 differ from the spectral value blocks as output by the windowing device 36, but not only by the processing by the signal processing device 40 but also by the demodulation effected by the mixer 34. Therefore, the mixer 46 receives the strings of modified spectral value blocks output from the inverse transform means 42 per subband and mixes them with a complex carrier corresponding to that used at the corresponding block for demodulating the audio signal at the mixer 34 is complex conjugated to modulate the spectral value blocks again with the carrier caused by the phase offsets of the time blocks. The result, which is established at the output of the mixer 46, is a sequence of modified non-demodulated spectral value blocks per subband.

The output of the mixer 46 is connected to an input of the assembler 44. This merges the sequence of modified blocks of spectral values modulated again with the complex carrier into a uniform stream or a uniform sequence of spectral values per subband, by corresponding spectral values of adjacent or successive blocks of spectral values for a subband as described by the US Pat Mixer 46 are obtained, suitably linked together. In the case of using the above-mentioned weighting functions with the positive property that the squares of mutually corresponding weighting values add up to one in the case of overlapping, the combination consists in a simple addition of mutually associated spectral values. The result output at the output of the merging device 44 (OLA = overlap-add = overlap add-on) is composed of a modified sequence of spectral values per subband. The result that is thus output at the output of the OLA 44 are thus modified subbands or modified sequences of spectral values for all spectral components and represents a modified time / frequency representation of the information signal 14 and a time / frequency representation of the modified audio signal 18.

The transformation device 48 receives the spectral value sequences and thus in particular successively a spectral value for all subbands or spectral components or a spectral decomposition of a section of the modified audio signal 18 successively. It generates a sequence of modified time blocks from the sequence of spectral decompositions by reversing the spectral decomposition. In turn, these modified time blocks receive the merge means 50. The merge means 50 works in a similar manner to the merge means 44. It assembles the modified time blocks, which overlap by way of example by 50%, by adding corresponding information values from adjacent or successive modified time blocks. The result at the output of the combining device 50 is thus a sequence of information values representing the processed audio signal 18.

Now that the structure of the device 10 and the operation of the individual components has been described above, in the following the operation of the same reference is taking Fig. 1 and 2 discussed in more detail.

The processing of the audio signal by the device 10 begins with the reception of the audio signal 14 at the input 12. The audio signal 14 is present in a sampled form. The sampling has been carried out, for example, by means of an analog / digital converter. The sampling took place with a certain sampling frequency ω _s . The information signal 14 consequently reaches the input 12 as a sequence of sample values s _i = s (2π / ω _s · i), where s is the analog information signal, s _{i is} the information values and the index i is an index for the information values should. Below the incoming samples s _i the windowing device 28 combines 2N consecutive samples into time blocks, in this case exemplarily with a 50% overlap. For example, it combines the samples s ₀ to s _2N-1 into a time block with the index n = 0, the samples s _N to s _3N-1 into a second time block with the index n = 1, the samples s _2N to s _{4N -1} to a third time block of information values with the index n = 2, etc. together. Each of these blocks weights the fenestration 28 with a weighting function as described above. For example, if s ⁿ ₀ to s ⁿ _2N-1 are the 2N information values of the time block n, then the block output by the device 28 becomes s ⁿ ₀ → s ⁿ ₀ · g ₀ to s ⁿ _2N-1 → s ⁿ _{2N -1} · g _2N-1 , where g _i with i = 0 to 2N-1 is the weighting function.

In Fig. 2 For example, the windowing functions applied to the information values s _i are exemplarily illustrated for four consecutive time blocks n = 0, 1, 2, 3 in a diagram 70 where the time t is in arbitrary units along the x-axis and the amplitude along the y-axis the windowing functions is plotted in arbitrary units. In this way, the windowing device 28, after each N information values, forwards a new windowed time block to 2N information values to the transformation device 30. The repetition frequency of the time blocks is thus ω _s / N.

The transformation means 30 transforms the windowed time blocks into a spectral representation. In this case, the transformation device 30 performs a spectral decomposition of the time blocks of windowed information values into a plurality of predetermined subbands or spectral components. In the present case, it is assumed by way of example that the transformation is a DFT or discrete Fourier transformation. For each time block to 2N information values, the transformation means 30 generates in this exemplary Case N complex-valued spectral values for N spectral components when the audio signal is real. The complex spectral values output by the transformation device 30 represent the time / frequency representation 74 of the audio signal. The complex spectral values are in Fig. 2 illustrated by box 76. Since the transformation device 30 generates at least one spectral value per successive time block of information values per subband or spectral component, the transformation device 30 thus outputs a sequence of spectral values 76 with the frequency ω _s / N per subband or spectral component. The spectral values output to a time block are in Fig. 2 shown at 74 horizontally along the frequency axis 78 arranged. The spectral values output at a subsequent time block are directly adjacent thereto in the vertical direction along the axis 80. The axes 78 and 80 thus represent the frequency or time axis of the time / frequency representation of the audio signal 14. By way of example, only four subbands are shown in FIG. The sequence of spectral values per subband run in the exemplary representation of Fig. 2 along the columns and are shown at 82a, 82b, 82c and 82d.

It will be short again Fig. 1 Reference is made, in which the audio signal 14 is exemplified as a function that is represented by sin (bt) · (1 + μ · sin (at)), where α, for example, the modulation frequency of the envelope of the information signal indicated by the dashed line 84 14, while β represents the carrier frequency of the audio signal 14, t is the time and μ is the modulation depth. At sufficiently high sampling frequency ω _s results with this exemplary information signal by the transformation 72 per block of time a block of spectral values 76, ie a line at 74, in which primarily the spectral component or the associated spectral value at the carrier frequency β has a pronounced maximum. The spectral values for this spectral component f = β however, varies in time for successive time blocks due to the variation of the envelope 84. Accordingly, the magnitude of the spectral values of the spectral component β varies with the modulation frequency α.

However, the previous consideration ignored that the different time blocks may each have a different phase offset from the carrier frequency β due to a frequency mismatch between the time block repetition frequency ω _s / N and the carrier frequency of the audio signal 14. Depending on the phase offset, the spectral values of the spectral blocks resulting from the time blocks in transformation 72 are modulated with a carrier e ^jΔφf , where j represents the imaginary unit, f the frequency and Δφ the phase offset of the respective time block. At substantially the same carrier frequency as is the case in the present exemplary case, the phase offset Δφ increases linearly. Therefore, the spectral values of a subband due to a frequency mismatch between the time block repetition frequency and the carrier frequency also undergo a modulation with a carrier component which depends on the mismatch of the two frequencies.

Taking this into consideration, the carrier frequency determining unit 32 derives from the spectral values a (ω _b , n) the carrier component resulting from the phase offset of the time blocks or caused by the time block phase offset in the subbands, where ω _{b is} the angular frequency ω or frequency f (ω = 2πf) of the respective subband 0≤b <N among all N subbands and n is the time block or spectral block index, which is associated with time t according to n = ω _s · t. The thus determined Modulationsträgerfrequenz ω (m, f) determines the carrier frequency determining means 32 for each subband ω _b or each frequency f blockwise, where m indicates a block index, as will be explained in more detail below. For this purpose, the carrier frequency determination device 32 combines M consecutive spectral values 76 of a subband ω _b , for example the spectral values a (ω _b , 0) to a (ω _b , M-1). Among these M spectral values, it determines a phase course through phase unwrapping. It then determines, for example by means of a least squares algorithm, a straight line equation which comes closest to the phase curve. From the slope and an intercept, or a phase or initial offset of the line equation, the carrier frequency determiner 32 obtains the desired modulation carrier frequency ω _d for subband b with respect to time block m, or a spectral value block phase offset φ for subband b with respect to time block m. This determination carries out the carrier frequency determination device for all subbands over temporally identical spectral values, ie for all spectral value blocks a (ω _{b, 0} ) to a (ω _b , _M-1 ) with ω _b for all subbands 0≤b <N. In this way, for each subband ω _b , the carrier frequency determiner 32 determines a modulation carrier frequency ω _d and the spectral value block phase offset φ, block by block. The block ordering underlying the determination of the complex carriers for all subbands by the device 32 is that used by the fenestration device for fenestration. The carrier frequency determiner 32 outputs the determined values for the complex carriers to the demodulator 34.

The mixer 34 now mixes the windowed blocks of spectral values of the individual subbands as output from the windowing means 36 with the complex conjugate of the respective modulation carrier frequencies ω _d taking into account the spectral ^{value block phase offsets} φ by multiplying these subband spectral ^{value blocks} by e ^{-j · (ω_d · n + φ))} , wherein, as mentioned above, respectively a different pair of ω _d and φ is used for each subband and within each subband for the successive blocks. In this way, the mixer 34 outputs demodulated subband spectral value blocks aligned with each other ie, two-dimensional blocks of N spectral value blocks for each M demodulated spectral values.

Since the modulations in the subbands caused by the time block offsets have been removed by the demodulation by means of the mixer 34, the phase characteristic of the spectral values in the subbands within the blocks is on average flatter and substantially around phase 0. In this way it is achieved that in the subsequent transformation by the transformation means 38, the demodulated and windowed blocks of spectral values lead to a spectral decomposition in which the frequency 0 or the DC component is very well centered.

The transformation 86, following the demodulation 84 by the mixer 34, is performed by the transformation means 38 on a block-by-block basis on each sub-band or demodulated blocks of spectral values. In particular, the demodulated spectral value blocks of the N subbands are subjected in block by block to spectral decomposition by the transformation 86. The result of the spectral decomposition of the blocks of spectral values may also be referred to as a modulation frequency representation. Thus, for N aligned blocks of windowed and demodulated spectral values, the transform 86 yields a matrix of M x N modulation values representing the frequency / modulation frequency representation of the information signal 14 over the time period of the M time blocks that contributed to that matrix. The modulation matrix is in Fig. 2 exemplified at 88 for the case N = M = 4. As can be seen, the frequency / modulation frequency representation 88 has two dimensions, namely the frequency 90 and the modulation frequency 92. The individual modulation values are symbolized at 88 with box 93.

The transformation device 38 forwards the modulation matrix to the processing device 40. The processing device 40 is according to the present. Embodiment provided to filter out the modulation signal from the audio signal 14. In the present exemplary case, therefore, the processing means 40 performs low pass filtering on the modulation frequency components in the frequency / modulation frequency matrix. In Fig. 1 For illustration, at 94, a diagram is shown in which the modulation frequency is plotted along the x-axis and the amount of the modulation values is plotted along the y-axis. The diagram 94 illustrates a section of the modulation matrix 88 for the exemplary case of the audio signal 14 of FIG Fig. 1 , namely the sinus modulated sine. In particular, plot of the amounts of the modulation values along the modulation frequency for the subband having the frequency β, ie the carrier frequency, is shown in the diagram 94. By the demodulation 84 by means of the mixer 34, the modulation frequency spectrum is substantially perfectly centered - at least in the case of the FFT as the transformation 86 - or correctly aligned. In particular, the modulation frequency spectrum at the carrier frequency β two sidebands 96 and 98, which are arranged at the modulation frequency α, ie the modulation frequency of the envelope 84 of the audio signal 14. Furthermore, the modulation values of the modulation matrix 88 have a DC component 100 at the frequency β. The signal processing device 40 is now designed as a low-pass filter with a filter characteristic 102, which is shown with a dashed line, to remove the two sidebands 96 and 98 from the frequency / modulation frequency representation 88. In this way, the audio signal 14 is freed from its modulation component, after which only the carrier component remains. The modulation matrix modified in this way forwards the processing device 40 to the inverse transformation device 42. The inverse transformation means 42 processes the modified modulation matrix for each subband such that the block of modulation values for the respective subband, ie one Column in the modulation matrix 88, a to the transformation of the transformation means 38 inverse transform is subjected, so that these modulation value blocks are transferred from the frequency / modulation frequency representation back in the time / frequency representation. In this way, inverse transform means 42 generates from each such block of modulation values for each subband a block of spectral values for that subband.

From the output of the spectral values by the transformation device 30, the preceding description referred primarily to the processing of the first M spectral values or of M consecutive spectral values for each subband. However, the processing by means 32, 34, 36, 38, 40 and 42 is also repeated for subsequent blocks of every M spectral values for each of the N subbands, with an overlap of the blocks to each of M spectral values of, in the present case, by way of example 50%, ie with an overlap per subband around M / 2 spectral values. The blocks are in Fig. 2 by way of example with m = 0, m = 1 and m = 2 in the time / frequency representation 74 illustrated by exemplary arcuate windowing or weighting functions, which extend exemplarily over M = 4 spectral values in each subband. Finally, for each of these blocks m, the transformation means 38 generates a modulation matrix of M x N modulation values each, which are filtered by the signal processing means 40 in the manner described above. The inverse transformation device 42 again generates from these modified modulation matrices 88 a block of spectral values for each subband, ie a block of spectral values modified with the matrix but still demodulated.

However, the blocks of spectral values per subband output by the inverse transform means 42 differ from those obtained from the information signal 14 at the output of the windowing device 36, however not only by the processing by the processing means 40, but also by the change caused by the demodulation. The spectral value blocks are therefore modulated in the modulation device 46 again with the modulation carrier component with which they were previously demodulated. In particular, therefore, the respective blocks of spectral ^values previously multiplied by e ^{-j * (ω_d * n + φ)} are now multiplied by e ^{+ j * (ω_d * n + φ)} where n is the index of the spectral value sequence indicate the respective subband and ω_d or ω _{d is} the angular frequency of the complex determined by the means 32 for the respective spectral value block modulation carrier.

The sequences of blocks of spectral values per subband resulting after the modulation stage 46 are now combined for each subband by the merging device 44 to form a uniform stream 82a-82d of spectral values per subband, by the examples corresponding to the blocks of spectral values, in this case by 50%, overlapping each other and combining corresponding spectral values according to the weighting function used in the fenestration device 36, namely, by adding in the case of the sine or KBD windows exemplified above.

The streams of spectral values per subband resulting at the output of the merging means 44 represent the time / frequency representation of the processed audio signal 18. The streams are received by the inverse transform means 48. In each time step n, it uses the spectral values for all subbands ω _b , ie all the spectral values a (ω _b , n) where 0≤b <N, in order to perform a transformation from the frequency to the time representation in order to calculate for each n, ie with a repetition period of 2πN / ω _s to obtain a time block. These time blocks are overlaid by the merging device 50 by way of example by way of example 50% overlapping and combining mutually corresponding information values in these time blocks is brought together into a uniform stream of information values, which finally represents the processed audio signal in the time domain 18, which is output at the output 16.

The processed audio signal is in Fig. 1 at 18 in a diagram in which the x-axis is the time and the y-axis the amplitude of the audio signal 18. As can be seen, only the carrier component of the input-side audio signal 14 has remained. The modulation components or the envelope component 84 has been removed.

In other words, the embodiment of FIG Fig. 1 and 2 a processing device that used a signal adaptive filter bank to decompose signals into carriers and modulation components and used the resulting representation of the modulated signals to filter them. However, it would also be possible to perform encoding, encryption, or compression rather than filter processing in the signal processing device, or to otherwise modify the modulation matrices. In comparison with the modulation transformation methods used for audio coding described in the introduction, which carry out an amount formation, in this embodiment a demodulation is carried out per sub-band with respect to a carrier component. After estimating this subband carrier component in the carrier frequency determiner 32, the demodulation per subband is achieved by multiplication with the complex conjugate of that component. The subband signals demodulated in this way are then transformed into the modulation range by means of a further frequency decomposition by means of the window device 36 and the transformation device 38.

In the embodiment of Fig. 1 For example, DFT with 50% overlap and windowing was used as the first transformation 72, although deviations and variations are conceivable. Several blocks of the first transformation 72 were again - there with 50% overlap example - summarized by the fenestration device 36 and partially bandwise with a complex modulator, which has been determined by the carrier frequency determining means 32, demodulated by the mixer 34 and then transformed with a DFT. In the foregoing embodiment, in the carrier frequency determining means, the frequency of this modulator has been obtained from the phases of the respective blocks of the sub-band to be demodulated, namely by approximating a line through the unwrapped phase characteristic of the spectral values of the corresponding blocks. However, this can also be done differently. For example, the carrier frequency determiner 32 may approximate one plane to the phase portion of all subbands in this section per spectral block section n to n + M-1. Furthermore, it would be possible for the carrier frequency determination device 32 to carry out the determination of the complex modulator not in blocks but continuously via the stream of spectral values per subband. For this purpose, for example, the carrier frequency determining device 32 could, for example, first unwrap the phases of the sequence of spectral values of a respective subband, then low pass filter it, and then use the local enhancement of the filtered phase response to adapt the complex modulator. Accordingly, the modulation part at the mixer 46 would also be changed. More generally, the carrier frequency determiner attempts to influence the phase response by either increasing or decreasing the phase of the complex spectral values of a subband having an increasing or decreasing amount across the sequence such that an average slope of the phase of the sequence of spectral values is reduced. or the unwrapped phase curve is essentially one solid phase value, preferably the phase 0, varies around.

Once again explicit reference is made to the fact that other types than the DFT or IDFT are conceivable for the transformations 72, 86 used and the transformation means 42 and 48 which are inverse thereto. Thus, for example, but not according to the invention, the complex demodulated subband signal can also be transformed into the frequency / modulation frequency representation separately or separated spectrally, each with a real-valued transformation into real and imaginary parts. The real part then represented, after the demodulation stage, the amplitude modulation of the subband signal with respect to the carrier used for demodulation. The imaginary part then represented the frequency modulation of this carrier. In the case of the DFT and IDFT for the devices 38 and 42, respectively, the amplitude modulation component of the subband signal is reflected in the symmetric component of the DFT spectrum along the modulation frequency axis, while the frequency modulation component of the carrier corresponds to the asymmetric component of the DFT spectrum along the modulation frequency axis ,

The exemplary embodiment described above has been illustrated by way of example on a simple sine-modulated sinusoidal signal. The embodiment of Fig. 1 and 2 but is also suitable for filtering the course of the envelope of a mixture of amplitude modulated signals of any frequency, such as amplitude modulated tonal signals. The individual frequency components of the envelope are directly represented in the modulation matrix 88 for consistent processing, in contrast to the already known magnitude-phase representation according to the modulation transformation analysis method for audio coding described in the introduction to the description. Also, the filtering of frequency modulated signals low modulation depth, ie with a frequency deviation, the essential is smaller than the subband width of the first DFT is, with the embodiment of Fig. 1 and 2 possible.

The embodiment of Fig. 1 and 2 Thus, an arrangement for modulation filtering, which was expressed in other words again based on a signal adaptive transformation, a filtering in the modulation range and a corresponding inverse transformation. Without signal manipulation in the modulation range, in the present embodiment of the filtering, the arrangement is made Fig. 1 perfectly reconstructed. By introducing an appropriate spectral range filter, such as filter 102, ie, attenuating the modulation values with increasing distance from a center modulation frequency of zero, the modulation components to be removed may be attenuated as desired. However, other types of processing of audio signals in the frequency / modulation frequency representation are also conceivable. So it might also be desirable to remove only the carrier. In this case, the filtering would consist of a high-pass filtering, ie a weighting function with a modulation frequency edge at a certain modulation frequency, which weakens modulation values at lower modulation frequencies more than those at higher modulation frequencies. In yet other applications or applications, the signal processing in the signal processing device 40 could again consist of bandpass filtering, ie weighting with a weighting function falling away from a certain center modulation frequency, to separate portions of the audio signal originating from different sources, ie to achieve a source separation. Other applications in which the foregoing embodiment may be used may involve audio coding for encoding audio signals, disturbed signal reconstruction, and error concealment. In general, however, the device 10 could be used as a music effect device to special acoustic effects in the incoming audio signal. The processing in the signal processing device 40 can accordingly take on a very wide variety of forms, such as the quantization of the modulation values, the zeroing of some modulation values, the weighting of individual sections of the or all modulation values or the like. Another application would be the use of the device 10 of Fig. 1 as a watermark embedder. The watermark embedder would receive an audio signal 14, wherein the processor 40 could introduce a received watermark into the audio signal by modifying individual segments or modulation values according to the watermark. The selection of the segments or modulation values could be different or time-varying for successive modulation matrices and would be made such that by psychoacoustic masking effects the modifications due to the human acoustic watermark insertion in the resulting watermarked audio signal 18 are inaudible.

With regard to the transformation devices, it should also be pointed out that they can of course also be embodied as filter banks which produce a spectral representation through many individual bandpass filters. It should also be noted that the resulting audio signal 18 does not have to be output in the time domain representation after processing. It would also be conceivable to output the information signal, for example in a time / spectral representation or even in the spectral / modulation spectral representation. In the latter case, of course, it would then have to be ensured that the necessary modulation 46 can again be performed on the receiver side with the suitable carrier, for example by supplying the complex carriers varying for each subband and spectral value block which were used for demodulation 84. In this way, the above embodiment could be used to implement a compression method.

In particular, it should be noted that, depending on the circumstances, the inventive scheme can also be implemented in software. The implementation may be on a digital storage medium, in particular a floppy disk or a CD with electronically readable control signals, which may cooperate with a programmable computer system such that the corresponding method is executed. In general, the invention thus also consists in a computer program product with program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer. In other words, the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Claims (18)

1. Device for processing an audio signal (14), comprising

   means (20) for converting the audio signal (14) to a time/spectral representation (74) by block-wise transforming of the audio signal;

   means (22) for converting the audio signal from the time/spectral representation (74) to a spectral/modulation spectral representation (88) by means of one single frequency decomposition transform, wherein the means (22) for converting is designed such that the spectral/modulation spectral representation (88) depends on both a magnitude component and a phase component of the time/spectral representation (74) of the audio signal (14);

   means (24, 40) for manipulating the audio signal (14) in the spectral/modulation spectral representation (88) to obtain a modified spectral/modulation spectral representation; and

   means (26) for forming a processed audio signal (18) representing a processed version of the audio signal (14) based on the modified spectral/modulation spectral representation.
2. Device according to claim 1, wherein the means (20) for converting the audio signal (14) to the time/spectral representation (74) is designed to decompose the time/spectral representation into a plurality of spectral components to obtain a sequence (82a, 82b, 82c, 82d) of complex spectral values per spectral component.
3. Device according to claim 2, wherein the means (22) for converting the audio signal (14) from the time/spectral representation (74) to the spectral/modulation spectral representation (88) comprises means (36, 38) for block-wise spectral decomposition of the sequence (82a, 82b, 82c, 82d) of spectral values for a predetermined spectral component to obtain a portion of the spectral/modulation spectral representation (88).
4. Device according to claim 3, wherein the means (22) for block-wise spectral decomposition of the sequence (82a, 82b, 82c, 82d) of spectral values for a predetermined spectral component is designed to first multiply (84) the sequence (82a, 82b, 82c, 82d) of spectral values block-wise by a complex carrier such that a magnitude of a mean slope of a phase course of the sequence (82a, 82b, 82c, 82d) of spectral values is reduced block-wise to obtain demodulated blocks of spectral values, and to then spectrally decompose the demodulated blocks of spectral values block-wise to obtain the portion of the modified spectral/modulation spectral representation (88).
5. Device according to claim 4, wherein the means (22) for block-wise spectral decomposition of the sequence (82a, 82b, 82c, 82d) of complex spectral values for a predetermined spectral component comprises means (32) for block-wise varying, depending on the time/spectral representation (74) of the audio signal, the complex carrier by which the sequence (82a, 82b, 82c, 82d) of complex spectral values is multiplied block-wise.
6. Device according to claim 5, wherein the means (32) for varying is designed to block-wise unwrap phases of the spectral values in the sequence of spectral values for block-wise varying of the complex carrier to obtain a phase course, to determine a mean slope of the phase course and to determine the complex carrier based on the mean slope.
7. Device according to claim 6, wherein the means (32) for varying is further designed to determine an axis portion of the phase course from the phase course and to further determine the complex carrier based on the axis portion.
8. Device according to one of claims 4 to 7, wherein the means (26) for forming comprises:

   means (42) for back-converting the audio signal from the modified spectral/modulation spectral representation to a modified time/spectral representation to obtain modified demodulated blocks of spectral values for the predetermined spectral component;

   means (46) for block-wise multiplying the modified demodulated blocks of spectral values by a carrier complex conjugated with respect to the complex carrier to obtain modified blocks of spectral values; and

   means (44) for combining the modified blocks of spectral values to form a modified sequence of spectral values to obtain a portion of a time/spectral representation of the processed audio signal (18).
9. Device according to claim 8, wherein the means for forming further comprises:

   means for back-converting the processed audio signal (18) from the time/spectral representation to the time representation.
10. Device according to one of the preceding claims, wherein the means (40) for modifying is designed to perform weighting of the modulation components of the spectral/modulation spectral representation (88) for modulation filtering, audio coding, source separation, reconstruction of the audio signal, for error concealing or for superimposing a watermark on the audio signal.
11. Device according to claim 1, wherein the means (20) for converting the audio signal to the time/spectral representation (74) comprises:

    block formation means (28) for forming a sequence of blocks of information values from the audio signal (14); and

    means (30) for spectrally decomposing each of the sequence of blocks of information values to obtain a sequence of spectral value blocks, wherein each spectral value block comprises a spectral value (76) for each of a predetermined plurality of spectral components, so that the sequence of spectral value blocks per spectral component forms a sequence (82a-82d) of spectral values.
12. Device according to claim 11, wherein the means (22) for converting the audio signal (14) to the spectral/modulation spectral representation (88) comprises:

    means (32-38) for spectrally decomposing a predetermined sequence of the sequences (82a-82d) of spectral values to obtain a block of modulation values,

    wherein the means (24; 40) for modifying is designed to modify the block (88) of modulation values to obtain a modified block of modulation values, which is part of the modified spectral/modulation spectral representation (88).
13. Device according to claim 12, wherein the means (26) for forming is designed to back-convert (42, 44, 46) the modified block of modulation values from the spectral decomposition to obtain a modified sequence of spectral values, and to back-convert (48) a sequence of modified spectral blocks based on the modified sequence of spectral values to obtain a sequence of modified blocks of information values, and to combine (50) the modified blocks of information values to obtain the processed audio signal (18).
14. Device according to claim 13, wherein the means (20) for spectrally decomposing each of the sequence of blocks of information values is designed to first multiply each block of the sequence of blocks of information values by a window function and to then spectrally decompose it, and the means (26) for forming is designed to process the modified blocks of information values, when combining (50), such that the multiplication by the window function does not affect the processed audio signal (18).
15. Device according to claim 12, wherein the means (20) for spectrally decomposing each of the sequence of blocks of information values is designed such that it provides a sequence (82a-82d) of complex spectral values in the spectral decomposition per spectral component, and the means (32, 34, 36, 38) for spectrally decomposing the predetermined sequence of the sequences (82a-82d) of spectral values is designed to first modify (34) the predetermined sequence (82a-82d) of spectral values such that a phase of the spectral values of the predetermined sequence of spectral values is increased or reduced by an amount steadily increasing or decreasing with the sequence to obtain a phase-modified sequence of spectral values, and then to spectrally decompose (38) the phase-modified sequence of spectral values to obtain the at least one block of modulation values, and the means for forming is designed to back-convert (42) the modified block of modulation values from the spectral decomposition to obtain a modified sequence of spectral values, to modify (46) the modified sequence of spectral values inversely to the means (34) for spectrally decomposing the predetermined sequence of the sequences of spectral values such that a phase of the spectral values of the at least one sequence of spectral values is increased or reduced by an amount steadily increasing or decreasing with the sequence to obtain a modified sequence of spectral values, to back-convert (48) a sequence of modified spectral blocks based on the modified sequence of spectral values to obtain a sequence of modified blocks of information values, and to combine (50) the modified blocks of information values to obtain the processed audio signal (18).
16. Device according to one of the preceding claims, wherein the only frequency decomposition transform is one single discrete Fourier transform.
17. Method for processing an audio signal (14), comprising

    converting (20) the audio signal (14) to a time/spectral representation (74) by block-wise transforming of the information signal;

    converting (22) the audio signal from the time/spectral representation (74) to a spectral/modulation spectral representation (88) by means of one single frequency decomposition transform, wherein the conversion is performed such that the spectral/modulation spectral representation (88) depends on both a magnitude component and a phase component of the time/spectral representation (74) of the audio signal (14);

    modifying (24) the audio signal (14) in the spectral/modulation spectral representation (88) to obtain a modified spectral/modulation spectral representation; and

    forming (26) a processed audio signal (18) representing a processed version of the audio signal (14) based on the modified spectral/modulation spectral representation.
18. Computer program with a program code for performing the method according to claim 16 when the computer program runs on a computer.

Images