KR20170016488A - Apparatus and method for enhancing an audio signal, sound enhancing system - Google Patents

Abstract

A signal processor for processing the audio signal to reduce or eliminate transient and tonal portions of the processed signal, and a decorrelator for generating a first decorrelated signal and a second decorrelated signal from the processed signal, An apparatus for enhancing an audio signal is disclosed. The apparatus comprises means for weighting the first and second decorrelated signals and the signal derived by the coherence enhancement from the audio signal or audio signal using time-varying weighting factors and obtaining a 2-channel audio signal Lt; / RTI > The apparatus further includes a controller for controlling the time-varying weighting factors by analyzing the audio signal such that different portions of the audio signal are multiplied by different weighting factors and the 2-channel audio signal has a time varying decorrelation.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an apparatus and method for enhancing an audio signal,

This application relates to audio signal processing and, in particular, audio processing of mono or dual-mono signals.

The auditory scene can be modeled as a mixture of direct sound and ambient sound. Direct (or directed) sounds are emitted by sources, such as musical instruments, vocalists or speakers, and reach the receiver, e. G., The listener's ear or microphone on the shortest possible path. When direct sound is captured using a set of spaced microphones, the received signals are coherent. Conversely, ambient (or diffuse) sounds are emitted by, for example, many spaced sound sources or acoustic reflection boundaries that contribute to room reverberation, applause or noise. When using a set of spaced microphones to capture a surrounding acoustic field, the received signals are not at least partially coherent.

Monophonic sound reproduction may be considered appropriate for some playback scenarios (e.g., dance clubs) or for some types of signals (e.g., voice recordings), but most music recordings, And TV sound are stereo signals. Stereo signals may produce ambient (or diffuse) sounds and a sense of direction and width of sound sources. This is accomplished using stereo information encoded by space cues. The most important spatial cues are channel-to-channel level differences (ICLD), inter-channel time differences (ICTD) and channel-to-channel coherence (ICC). As a result, stereo signals and corresponding sound reproduction systems have more than one channel. ICLD and ICTD contribute to sense of direction. ICC causes a sense of width of sound, and in the case of ambient sounds, sound is perceived as coming from all directions.

Although there are multi-channel sound reproduction of various formats, most of the audio recording and sound reproduction systems still have two channels. Two-channel stereo sound is standard for entertainment systems and listeners are used for this. However, a stereo signal is not limited to having only two channel signals, but can have more channel signals. Similarly, monophonic signals are not limited to having only one channel signal, but may have many but the same channel signals. For example, an audio signal comprising two identical channel signals may be referred to as a dual-mono signal.

There are a number of reasons why monophonic signals are available to listeners instead of stereo signals. First, old recordings are monophonic, because stereo techniques were not used at that time. Second, bandwidth constraints on the transmission or storage medium may result in the loss of stereo information. A representative example is radio broadcast using frequency modulation (FM). Here, interference sources, multipath distortions or other transmission impairments can result in noisy stereo information, which is typically the case for the transmission of two-channel signals encoded with differential signals between both channels. It is common to discard the stereo information partially or completely if the reception conditions are poor.

Loss of stereo information may result in poor sound quality. In general, an audio signal that includes a greater number of channels may include a higher audio quality than an audio signal that includes fewer channels. Listeners may prefer to listen to audio signals containing high quality. Audio quality is often reduced for efficiency reasons, such as data rates stored at or transmitted through the media.

Therefore, it is necessary to increase (improve) the sound quality of the audio signals.

It is therefore an object of the present invention to provide an apparatus or method for enhancing audio signals and / or for increasing the sense of reproduced audio signals.

This object is achieved by an apparatus for enhancing an audio signal according to claim 1, a method for enhancing an audio signal according to claim 14 and a sound enhancement system according to claim 13 or a computer program according to claim 15, Lt; / RTI >

The invention is based on the idea that the received audio signal can be enhanced by dividing the received audio signals into at least two shares and artificially creating spatial cues by decorrelating at least one of the shares of the received signal Based on discovery. The weighted combination of sharers allows to receive audio signals that are recognized as stereo, and is therefore improved. If the decorrelation can cause annoying effects that reduce the sound quality, controlling the applied weights can be accomplished by varying the degree of decorrelation and thus the degree of enhancement so that the level of enhancement can be lowered, . Thus, it includes portions or time intervals to which low decorrelation is applied or not at all, such as in the case of speech signals, and includes portions or time intervals to which higher or higher decorrelation is applied as in the case of music signals A variable audio signal can be improved.

One embodiment of the present invention provides an apparatus for enhancing an audio signal. The apparatus includes a signal processor for processing an audio signal to reduce or eliminate transient and tonal portions of the processed signal. The apparatus further includes an decorrelator for generating a first decorrelated signal and a second decorrelated signal from the processed signal. The apparatus further comprises a coupler and a controller. The combiner combines the first de-correlated signal, the second de-correlated signal, and the signal derived by the coherence enhancement from the audio signal or audio signal, using the time-varying weight factors, Signal. The controller is configured to control the time-varying weight factors by analyzing the audio signal such that different portions of the audio signal are multiplied by different weighting factors and the 2-channel audio signal has a time varying decorrelation.

An audio signal having little or no stereo (or multi-channel) information, for example a signal with one channel or a signal with many but almost identical channel signals, It can be recognized as a stereo signal. The received mono or dual-mono audio signal may be processed differently in the different paths and the transient and / or tonal portions of the audio signal in one path are reduced or eliminated. The fact that the signal processed in this way is correlated in an uncorrelated manner and that the decorrelated signal is weighted with a second path comprising an audio signal or a signal derived therefrom, Allowing the two channels to be acquired, so that the two channels are recognized as stereo signals.

By controlling the weighting factors used to weightively combine the decorrelated signal and the audio signal (or a signal derived therefrom), a time varying decorrelation can be obtained so that it is possible, In situations where effects will result, the enhancement may be reduced or omitted. For example, it may not be desirable to improve the signal of a radio speaker or other significant sound source signals when recognizing a speaker from sources at multiple locations may cause annoying effects on the listener.

According to a further embodiment, an apparatus for enhancing an audio signal comprises a signal processor for processing an audio signal to reduce or eliminate transient and tonal portions of the processed signal. The apparatus further includes an decorrelator, a combiner, and a controller. The decorrelator is configured to generate a first decorrelated signal and a second decorrelated signal from the processed signal. The combiner is configured to weightively combine the first decorrelated signal and the signal derived by the coherence enhancement from the audio signal or the audio signal using the time-varying weight factors and obtain a 2-channel audio signal. The controller is configured to control the time-varying weight factors by analyzing the audio signal such that different portions of the audio signal are multiplied by different weighting factors and the 2-channel audio signal has a time varying decorrelation. This allows recognizing a mono signal or a signal similar to a mono signal (e.g., dual-mono or multi-mono) to be a stereo-channel audio signal.

To process the audio signal, the controller and / or the signal processor may be configured to process the representation of the audio signal in the frequency domain. The representation may comprise a plurality of or a plurality of frequency bands (subbands), each of which comprises a part of the audio signal portion of the spectrum of each of the audio signals. For each of the frequency bands, the controller can be configured to predict the recognized correlation level in the two-channel audio signal. The controller may be configured to increase the weighting factors for the portions of the audio signal (frequency bands) to allow for a higher degree of decorrelation and to reduce the weighting factors for portions of the audio signal, ≪ / RTI > For example, a portion that includes non-significant source signals such as applause or noise may be combined by a weight factor that allows a higher correlation than the portion that contains the significant source signal, Is used for portions of a signal that are recognized as direct sounds, such as a voice, an instrument, a vocalist, or a speaker.

The processor may be configured to determine, for some or all of the frequency band, spectral weights that determine whether the frequency band includes transient or tonal components and allow for the reduction of transient or tonal portions. The spectral weights and scaling factors may each include a number of possible values such that annoying effects due to binary crystals can be reduced and / or avoided.

The controller may be further configured to scale the weighting factors such that the recognized de-correlation level of the two-channel audio signal is maintained within a certain range around the target value. The range can be extended to, for example, ± 20%, ± 10% or ± 5% of the target value. The target value may be a predetermined value for the measurement of tonalities and / or transient portions such that, for example, an audio signal containing varying transient and tonal parts that change the target value is obtained. This means that if the audio signal is decoded or if it is not intended for any decorrelation with respect to a prominent sound source, such as, for example, speech, then performing a low decorrelation or even no decorrelation at all, If correlation is not correlated and / or anticorrelation is the object, it allows a high correlation. The weighting factors and / or spectral weights may be determined and / or adjusted to multiple values, or even approximately continuously.

The decorrelator may be configured to generate a first decorrelated signal based on an echo or delay of the audio signal. The controller may be configured to generate a test decorrelated signal based also on the echo or delay of the audio signal. The echoes can be performed by delaying the audio signal and combining the audio signal and its delayed version similar to a finite impulse response filter structure, and the echo can also be implemented as an infinite impulse response filter. The number of delay times and / or delays and combinations may vary. The delay time for delaying or echoing the audio signal relative to the test decorrelated signal may be shorter than, for example, the delay time so that if the audio signal is delayed or echoed to the first decorrelated signal, And derives the coefficient. In order to predict the recognized reverse correlation strength, a lower degree of correlation and hence a shorter delay time may be sufficient so that the calculation effort and / or the calculation capability can be reduced by decreasing the delay time and / or filter coefficients.

Subsequently, preferred embodiments of the present invention are described with reference to the accompanying drawings.
Figure 1 shows a schematic block diagram of an apparatus for enhancing an audio signal.
Figure 2 shows a schematic block diagram of an additional apparatus for enhancing audio signals.
FIG. 3 shows an exemplary table indicating the computing of scaling factors (weighting factors) based on the level of the perceived reverse correlation strength that is predicted.
Figure 4A shows a schematic flow diagram of a portion of a method that may be executed to partially determine weighting factors.
FIG. 4B shows a schematic flow diagram of the further steps of the method of FIG. 4A, which illustrate the case where the measurement for the recognized reverse correlation level is compared to the thresholds.
Figure 5 shows a schematic block diagram of an decorrelator that can be configured to operate as the decorrelator of Figure 1;
Figure 6a shows a schematic diagram including a spectrum of an audio signal including at least one transient (short) signal portions.
6B shows a schematic spectrum of an audio signal including a tonality component.
Figure 7A shows a schematic table illustrating possible transient processing performed by the transient processing stage.
7B shows an exemplary table illustrating possible tone processing that may be performed by tone processing stages.
Figure 8 shows a schematic block diagram of a sound enhancement system including an apparatus for enhancing audio signals.
Figure 9A shows a schematic block diagram of the processing of an input signal in accordance with background / foreground processing.
FIG. 9B illustrates splitting the input signal into foreground and background signals.
Figure 10 shows a schematic block diagram and apparatus configured to apply spectral weights to an input signal.
Figure 11 shows a schematic flow diagram of a method for enhancing an audio signal.
Figure 12 illustrates an apparatus for determining measurements for echo / decorrelation levels recognized in a mixed signal comprising a direct signal component or a dry signal component and an echo signal component.
Figures 13A-13C illustrate implementations of a loudness model processor.
Figure 14 illustrates an implementation of the loudness model processor already discussed in some aspects with respect to Figures 12, 13A, 13B, and 13C.

The same or equivalent elements or elements having the same or equivalent function are denoted by the same or equivalent reference numerals even if they occur in different drawings in the following description.

In the following description, numerous details are set forth in order to provide a more thorough description of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form, rather than being described in detail, in order to avoid obscuring embodiments of the present invention. Further, the features of the different embodiments described below may be combined with each other unless specifically stated otherwise.

Next, processing of the audio signal will be referred to. The device or components thereof may be configured to receive, provide and / or process audio signals. Each audio signal may be received, provided, or processed in the same time domain and / or frequency domain. The audio signal representation in the time domain can be transformed into a frequency representation of the audio signal, e.g., by Fourier transforms or the like. The frequency representation may be obtained, for example, by using short-term Fourier transform (STFT), discrete cosine transform and / or fast Fourier transform (FFT). Alternatively or additionally, the frequency representation may be obtained by a filterbank which may include quadrature mirror filters QMF). The frequency domain representation of the audio signal may comprise a plurality of frames each comprising a plurality of subbands as is known from Fourier transforms. Each subband includes a portion of the audio signal. Since the temporal representation and the frequency representation of the audio signal can be mutually transformed, the following description should not be restricted to that the audio signal is a time domain representation or a frequency domain representation.

1 shows a schematic block diagram of an apparatus 10 for enhancing an audio signal 102. As shown in FIG. The audio signal 102 is, for example, a mono signal or a mono type signal, e.g., a dual-mono signal, expressed in the frequency domain or time domain. The apparatus 10 includes a signal processor 110, an decorrelator 120, a controller 130 and a combiner 140. The signal processor 110 receives the audio signal 102 and processes the audio signal 102 to reduce or eliminate the transient and tonal portions of the processed signal 112 relative to the audio signal 102, 0.0 > 112 < / RTI >

The decorrelator 120 is configured to receive the processed signal 112 and to generate a first decorrelated signal 122 and a second decorrelated signal 124 from the processed signal 112. The decorrelator 120 may be configured, at least in part, to generate a first decorrelated signal 122 and a second decorrelated signal 124 by echoing the processed signal 112. The first de-correlated signal 122 and the second de-correlated signal 124 are generated such that the first de-correlated signal 122 has a time delay (echo time) that is shorter or longer than the second de- Lt; RTI ID = 0.0 > echo < / RTI > The first or second decorrelated signal 122 or 124 may also be processed without delay or echo filter.

The decorrelator 120 is configured to provide the first de-correlated signal 122 and the second de-correlated signal 124 to the combiner 140. The controller 130 receives the audio signal 102 and controls the time-varying weight factors a and b by analyzing the audio signal 102 such that different portions of the audio signal 102 are multiplied by different weighting factors a or b. . Thus, the controller 130 includes a control unit 132 configured to determine the weighting factors a and b. The controller 130 may be configured to operate in the frequency domain. The control unit 132 may be configured to convert the audio signal 102 into the frequency domain by using a short term Fourier transform (STFT), a fast Fourier transform (FFT) and / or a normal Fourier transform (FT). The frequency domain representation of the audio signal 102 may comprise a plurality of subbands as is known from Fourier transforms. Each subband includes a portion of the audio signal. Alternatively, the audio signal 102 may be a representation of a signal in the frequency domain. The control unit 132 may be configured to control and / or determine a pair of weighting factors a and b for each subband of the digital representation of the audio signal.

The combiner is configured to weight combine the signal 136 derived from the audio signal 102 using a first decorrelated signal 122, a second decorrelated signal 124, and weighting factors a and b. do. The signal 136 derived from the audio signal 102 may be provided by the controller 130. Thus, the controller 130 may include an optional guidance unit 134. [ Induction unit 134 may be configured, for example, to adapt, modify, or enhance portions of audio signal 102. In particular, the guidance unit 110 may be configured to amplify portions of the audio signal 102 that are attenuated, reduced, or removed by the signal processor 110.

The signal processor 110 also operates in the frequency domain and processes the audio signal 102 to enable the signal processor 110 to reduce or eliminate transient and tonal portions for each subband of the spectrum of the audio signal 102 . This can result in little or no processing for subbands that contain little or no transient portion or little or no (i.e., noise) tonal portions. Alternatively, the combiner 140 may receive the audio signal 102 instead of the derived signal, i.e., the controller 130 may be implemented without the inductive unit 134. The signal 136 may then be the same as the audio signal 102.

The combiner 140 is then configured to receive a weighting signal 138 comprising weighting factors a and b. The combiner 140 is further configured to obtain an output audio signal 142 that includes a first channel y1 and a second channel y2, i.e., the audio signal 142 is a two-channel audio signal.

The signal processor 110, decorrelator 120, controller 130, and combiner 140 are coupled to the audio signal 102, the signal 136 derived therefrom, and / or the processed signals 112, 122 and / ) By processing on a frame and subband basis so that the signal processor 110, decorrelator 120, controller 130 and combiner 140 process one or more frequency bands (portions of the signal) RTI ID = 0.0 > of the < / RTI >

FIG. 2 shows a schematic block diagram of an apparatus 200 for enhancing an audio signal 102. The apparatus 200 includes a signal processor 210, an decorrelator 120, a controller 230 and a combiner 240. The decorrelator 120 is configured to generate a first decorrelated signal 122, denoted rl, and a second decorrelated signal 124, denoted r2.

Signal processor 210 includes transient processing stage 211, tone processing stage 213, and combining stage 215. The signal processor 210 is configured to process the representation of the audio signal 102 in the frequency domain. The frequency domain representation of the audio signal 102 includes a plurality of subbands (frequency bands), wherein the transient processing stage 211 and the tone processing stage 213 are configured to process each of the frequency bands. Alternatively, the spectrum obtained by the frequency transformation of the audio signal 102 may be in a specific frequency range or frequency bands, e.g., 20 Hz, 50 Hz, or 100 Hz and / or 16 kHz, 18 kHz, That is, to exclude the frequency bands of the frequency bands of interest from further processing. This may allow reduced computational effort and hence faster and / or more accurate processing.

The transient processing stage 211 is configured to determine, for each of the processed frequency bands, whether the frequency band includes transitional portions. Tone processing stage 213 is configured to determine, for each of the frequency bands, whether the audio signal 102 comprises tone portions in the frequency band. The transient processing stage 211 is configured to determine spectral weighting factors 217 for frequency bands including at least transient portions, and the spectral weighting factors 217 are associated with respective frequency bands. As described in Figures 6A and 6B, transient and tonal characteristics can be identified by spectral processing. The level of transients and / or tonality may be measured by transient processing stage 211 and / or tonality processing stage 213 and converted into spectral weights. Tone processing stage 213 is configured to determine spectral weighting coefficients 219 for frequency bands including at least tone portions. The spectral weighting factors 217 and 219 may include a number of possible values and the size of the spectral weighting factors 217 and / or 219 may indicate the amount of transient and / or tone portions in the frequency band.

The spectral weighting factors 217 and 219 may include an absolute value or a relative value. For example, the absolute value may include values of transient and / or tonal acoustic energy in the frequency band. Alternatively, the spectral weighting factors 217 and / or 219 may comprise a relative value such as a value between 0 and 1, and a value of 0 indicates that the frequency band contains no or very few transient or tonal parts And a value of 1 indicates a frequency band comprising a large amount or a complete transient and / or tone part. The spectral weighting factors may be selected from among a number of values such as 3, 5, 10 or more values (steps), e.g., (0, 0.3 and 1), (0.1, 0.2, ..., One can be included. The size of the scale, the number of steps between the minimum and maximum values is at least zero, but preferably is at least one and more preferably at least five. Preferably, the values of the plurality of spectral weights 217 and 219 include at least three values including a minimum value, a maximum value, and a value between a minimum value and a maximum value. A greater number of values between the minimum and maximum values may allow for more continuous weights of each of the frequency bands. The minimum and maximum values may be scaled to a scale between 0 and 1 or other values. The maximum value can indicate the highest or lowest level of transient and / or tonality.

The combining stage 215 is configured to combine the spectral weights for each of the frequency bands as described below. The signal processor 210 is configured to apply the combined spectral weights to each of the frequency bands. For example, the spectral weights 217 and / or 219 or values derived therefrom may be multiplied by the spectral values of the audio signal 102 in the processed frequency band.

The controller 230 is configured to receive spectral weighting factors 217, 219 or information about it from the signal processor 210. The derived information may be, for example, the index number of the table, and the index number is associated with the spectral weighting factors. The controller may enhance the audio signal 102 for coherent signal portions, i.e., the portions that are not reduced or removed or only partially reduced or removed by the transient processing stage 211 and / or tone processing stage 213. [ . In short, the guidance unit 234 can amplify portions that have not been reduced or removed by the signal processor 210. [

Induction unit 234 is configured to provide a signal 236 derived from audio signal 102 marked z. The combiner 240 is configured to receive the signal z (236). The decorrelator 120 is configured to receive the processed signal 212 indicated by s from the signal processor 210.

The combiner 240 is configured to combine the signals r1 and r2 that are correlated with the weighting factors (scaling factors) a and b to obtain the first channel signal y1 and the second channel signal y2. The signal channels y1 and y2 may be coupled to the output signal 242 or may be individually output.

That is, the output signal 242 is a combination of the (typically) correlated signal z 236 and the decorrelated signal s (r 1 or r 2, respectively). The decorrelated signal is obtained in two stages, i.e., suppression (reduction or elimination) of transient and tonal signal components first, and second, inverse correlation. Suppression of transient signal components and tonal signal components is performed using spectral weighting. The signal is processed on a frame-by-frame basis in the frequency domain. Spectral weights are computed for each frequency bin (frequency band) and time frame. Thus, the audio signal is processed in the entire band, i.e. all the parts considered are processed.

The input signal of the processing may be a single-channel signal x 102 and the output signal may be a two-channel signal y = [y1, y2], where the index may be a first and a second channel, The left and right channels of the signal are displayed. The output signal y is a 2-channel signal r = [r1, r2]

(1)

(One)

(2)

(2)

Can be computed by linearly combining the single-channel signal z with the scaling factors a and b according to,

Here, "x" represents a multiplication operator of equations (1) and (2).

Equations (1) and (2) should be interpreted qualitatively, indicating that the share of signals z, r1, and r2 can be controlled (modified) by varying weighting factors. By forming inverse operations such as, for example, dividing by a reciprocal value, the same or equivalent results can be obtained by performing different operations. Alternatively or additionally, a lookup table may be used that includes values for scaling factors a and b and / or y1 and / or y2 to obtain a two-channel signal y.

The scaling factors a and / or b may be computed to monotonically decrease according to the recognized correlation strength. A predicted scalar value for the recognized strength may be used to control the scaling factors.

The decorrelated signal r comprising r1 and r2 may be computed in two stages. First, the attenuation of the transient and tonal signal components derives the signal s. Second, an inverse correlation of the signal s can be performed.

The attenuation of transient signal components and tonal signal components is performed, for example, using spectral weighting. The signal is processed on a frame-by-frame basis in the frequency domain. The spectral weights are computed for each frequency bin and time frame. The purpose of attenuation is twofold.

One. Transient or tonal signal components typically belong to so-called foreground signals, and therefore their position in the stereo image is often central.

2. The decorrelation of signals with strong transient signal components results in recognizable artifacts. The decorrelation of the signals with strong tone signal components is also advantageous if at least the frequency modulation is slow enough to be perceived as a change in frequency rather than a change in tone color due to a rich signal spectrum (possibly dissonant) Resulting in recognizable artifacts when the components (i.e., sinusoids) are frequency modulated.

The correlated signal z can be obtained by qualitatively applying the inverse of the suppression to compute the processing of the transient and tonal signal components, e.g., the signal s. Alternatively, for example, an unprocessed input signal may be used as is. Note that z may also be a 2-channel signal. In fact, many storage media (e. G., Compact discs) use two channels even when the signal is mono. A signal having two identical channels is referred to as "dual-mono ". The case where the input signal z is a stereo signal may also be present and the purpose of the processing may be to increase the stereo effect.

The recognized reverse correlation strength can be predicted similar to the recognized late echo intensity predicted using loudness calculation models, as described in EP 2 541 542 A1.

3 shows an exemplary table indicating the computing of scaling factors (weighting factors) a and b based on the predicted level of the recognized decorrelation strength.

For example, the recognized inverse correlation strength may vary between a value of zero indicating a recognized low inverse correlation level, a value of ten indicating a high inverse correlation level, Lt; RTI ID = 0.0 > a < / RTI > scalar value. The levels may be determined based on, for example, listener tests or predictive simulations. Alternatively, the value of the de-correlation level may include a range between a minimum value and a maximum value. The value of the de-correlated level to be recognized can be configured to accommodate more values than the minimum and maximum values. Preferably, the recognized correlation level can accommodate at least three different values and more preferably at least seven different values.

The weighting factors a and b to be applied based on the determined level of the recognized decorrelation may be stored in memory and accessible to the controller 130 or 230. [ As the recognized reverse correlation levels increase, the scaling factors a to be multiplied by the audio signal or a signal derived therefrom by the combiner can also be increased. The increased level of perceived reverse correlation is such that as the de-correlate levels increase, the audio signal or a signal derived therefrom already includes a higher share in the output signal 142 or 242, Can be interpreted as "what happened". As the de-correlation levels increase, the weight factor b is configured to decrease, i. E., The signals r1 and r2 generated by the decorrelator based on the output signal of the signal processor are lowered when combined in combiner 140 or 240 It can include SharePoint.

The weight factor a is shown to include a scalar value of at least 1 (the minimum value) and a maximum of 9 (the maximum value). Although the weight factor b is shown as including a scalar value in a range including a minimum value 2 and a maximum value 8, both weighting factors a and b are values within a range including a minimum value and a maximum value, And at least one value between the maximum values. As an alternative to the values of the weighting factors a and b shown in FIG. 3, as the recognized de-correlation level increases, the weighting factor a may increase linearly. Alternatively or additionally, as the recognized correlation level increases, the weight factor b may decrease linearly. Further, for the level of perceived decorrelation, the sum of the weighting factors a and b determined for the frame may be constant or nearly constant. For example, as the recognized reverse correlation level increases, the weight factor a may increase from 0 to 10, and the weight factor b may decrease from a value of 10 to a value of zero. If both of the weighting factors decrease or increase linearly, e.g., step size 1, then the sum of the weighting factors a and b may include a value of 10 for each level of the recognized decorrelation. The weighting factors a and b to be applied can be determined by simulation or experiment.

4A shows a schematic flow diagram of a portion of a method 400 that may be executed, for example, by controllers 130 and / The controller is configured to determine a measure for the de-correlation level recognized in step 410, e.g., deriving a scalar value as shown in FIG. In step 420, the controller is configured to compare the determined measurement to a threshold. If the measure is above the threshold, the controller is configured to modify or adapt the weighting factors a and / or b in step 430. In step 430, the controller is configured to decrease the weight factor b, increase the weight factor a, decrease the weight factor b, and increase the weight factor a, for a reference value for a and b. For example, the threshold may vary within frequency bands of, for example, an audio signal. For example, the threshold may include a low value for frequency bands that include significant source signals indicative of low-level decorrelation being the preferred or target. Alternatively or additionally, the threshold may include a high value for frequency bands that include non-significant source signals indicating that a high level of decorrelation is favored.

It may be desirable to increase the correlation of frequency bands containing non-significant source signals and to limit the decorrelation for frequency bands containing significant source signals. The threshold may be, for example, 20%, 50% or 70% of the range of acceptable values of the weighting factors a and / or b. For example, referring to FIG. 3, the threshold for a frequency frame containing significant source signals may be less than 7, less than 5, or less than 3. If the recognized reverse correlation level is too high, then by performing step 430, the recognized reverse correlation level can be reduced. The weighting factors a and b may be varied alone or both at a time. The table shown in Fig. 3 may be, for example, a value including initial values for weighting factors a and / or b that are initial values to be adapted by the controller.

Figure 4b illustrates an additional step of method 400 that depicts a measurement for a recognized de-correlation level (as determined at step 410) is compared to a threshold value and the measurement is less than a threshold value (step 440) Lt; / RTI > The controller is configured to either increase b for a reference to a and b to increase the recognized reverse correlation level, decrease a, increase b, and decrease a, and wherein the measure is at least a threshold value Value.

Alternatively or additionally, the controller may be configured to scale the weighting factors a and b such that the recognized de-correlation level of the two-channel audio signal is maintained within a certain range around the target value. The target value may be, for example, a threshold value, and the threshold value may vary based on the type of signal included in the frequency band for which weighting factors and / or spectral weights are determined. The range around the target value can be extended to ± 20%, ± 10%, or ± 5% of the target value. This may allow to stop adapting the weighting factors if the recognized decorrelation is approximately the target value (threshold).

FIG. 5 shows a schematic block diagram of an decorrelator 520 that may be configured to operate as decorrelator 120. The decorrelator 520 includes a first decorrelation filter 526 and a second decorrelation filter 524. Both the first de-correlation filter 526 and the second de-correlation filter 528 are configured to receive, for example, signal s 512 processed from the signal processor. The decorrelator 520 combines the processed signal 512 with the output signal 523 of the first decorrelator filter 526 to obtain a first decorrelated signal 522 And combines with the output signal 525 of the filter 528 to obtain the second decorrelated signal 524 (r2). For combination of signals, decorrelator 520 may be configured to convolute the signals with impulse responses and / or multiply the spectral values with real and / or imaginary values. Alternatively, or in addition, other operations such as division, sum, difference, etc. may be performed.

The decorrelated filters 526 and 528 may be configured to echo or delay the processed signal 512. The decorrelation filters 526 and 528 may comprise a finite impulse response (FIR) and / or an infinite impulse response (IIR) filter. For example, decorrelation filters 526 and 528 may be configured to convolve the processed signal 512 with an impulse response obtained from a noise signal that attenuates or exponentially decays over time and / or frequency . This allows to generate the decorrelated signal 523 and / or 525, which includes the echo for signal 512. The echo time of the echo signal may include, for example, a value of 50 to 1000 ms, 80 to 500 ms and / or 120 to 200 ms. The echo time can be understood as the duration required to attenuate to a small value after the excitation power is excited by the impulse, e.g., 60 dB below the initial power. Preferably, the decorrelation filters 526 and 528 comprise IIR-filters. This allows to reduce the amount of computation if at least some of the filter coefficients are set to zero such that calculations for (zero-) filter coefficients can be omitted. Optionally, the decorrelation filter may comprise more than one filter, wherein the filters are connected in series and / or in parallel.

That is, the echo includes an inverse correlation effect. The decorrelators may be configured not only to decorrelate but also to change the sonority only slightly. Technically, echoes can be viewed as a linear time invariant (LTI) system that can be characterized in terms of impulse response. The length of the impulse response is often referred to as RT60 for echo. This is the time after which the impulse response is reduced by 60 dB. Echo can be up to 1 second or even up to several seconds in length. An decorrelator may be implemented that includes different configurations for echo-like structures, but for parameters that affect the length of the impulse response.

6A shows a schematic diagram including a spectrum of an audio signal 602a including at least one transient (short) signal portions. The transient signal portion derives a broadband spectrum. The spectrum is shown as magnitudes S (f) over frequencies f, where the spectrum is subdivided into a plurality of frequency bands b1-3. The transient signal portion may be determined at one or more of the frequency bands at b1-3.

FIG. 6B shows a schematic spectrum of an audio signal 602b that includes a tonal component. An example of the spectrum is shown in seven frequency bands fb1-7. The frequency band fb4 is arranged in the middle of the frequency bands fb1-7 and includes the maximum size S (f) compared to the other frequency bands fb1-3 and fb5-7. Frequency bands with increasing distance to the center frequency (frequency band fb5) include harmonic repeats of the tonal signal with decreasing magnitudes. The signal processor may be configured to determine the tonality component, for example, by evaluating the size S (f). The increasing size S (f) of the tonality component can be integrated by the signal processor by as much as the reduced spectral weighting factors. Thus, the higher the share of transients and / or tone components in the frequency band, the less the frequency band may contribute in the processed signal of the signal processor. For example, the spectrum weight for frequency band fb4 may include a value of zero or a value close to zero, or other value indicating that frequency band fb4 is considered to have a low share.

FIG. 7A shows a schematic table illustrating possible transient processing 211 performed by a signal processor such as signal processor 110 and / or 210. The signal processor is configured to determine the amount of transient components, e.g., SHARE, in each of the frequency bands of the representation of the audio signal in the frequency domain considered. The evaluation may include determining the amount of transient components with a starter value that includes at least a minimum value (e.g., 1) and a maximum value (e.g., 15), and a higher value may include a greater amount The transient components of < / RTI > The greater the amount of transient components in the frequency band, the lower the respective spectral weight, e.g., spectral weight 217, may be. For example, the spectral weight may include a minimum value such as at least 0 and a maximum value such as maximum 1. The spectral weighting may include a plurality of values between the minimum and maximum values, and the spectral weighting may indicate a consideration factor of the frequency band for consideration factor and / or further processing. For example, a spectral weighting of zero can indicate that the frequency band is fully attenuated. Alternatively, other scaling ranges may be implemented, i.e., the table shown in FIG. 7A may be scaled by tables having different step sizes for the evaluation of the step size of the frequency band and / or spectral weight, which is an excess frequency band, and / / RTI > The spectral weights can be continuously varied.

FIG. 7B illustrates an exemplary table illustrating possible tone processing that may be performed, for example, by tone processing stage 213. As shown in FIG. The greater the amount of tonality components in the frequency band, the lower the respective spectral weight 219 may be. For example, the amount of tonality components in the frequency band may be scaled between a minimum value of 1 and a maximum value of 8, and a minimum value indicates that the frequency band contains no or little tonality components. The maximum value may indicate that the frequency band includes a large amount of tonality components. Each spectral weight, such as spectral weight 219, may also include a minimum value and a maximum value. The minimum value (e.g., 0.1) may indicate that the frequency band is almost completely or completely attenuated. The maximum value may indicate that the frequency band is nearly attenuated or fully attenuated. The spectral weight 219 may accommodate one of a plurality of values including a minimum value, a maximum value, and preferably at least one value between a minimum value and a maximum value. Alternatively, the spectral weighting may be reduced for a reduced share of tonal frequency bands such that the spectral weighting is a consideration factor.

The signal processor may be configured to combine spectral weights for transient processing and / or spectral weights for tone processing with the spectral values of the frequency band, as described for the signal processor 210. For example, for a processed frequency band, the average value of the spectral weights 217 and / or 219 may be determined by the combining stage 215. The spectral weights of the frequency bands may be combined with, or multiplied by, the spectral values of the audio signal 102, for example. Alternatively, the combining stage may be configured to compare both spectral weights 217 and 219, and / or to select lower or higher spectral weights of both, and to combine the selected spectral weights with the spectral values . Alternatively, the spectral weights may be combined differently, e.g., as sum, difference, quotient or factor.

The characteristics of the audio signal can change over time. For example, a radio broadcast signal may first include a voice signal (a prominent source signal), and then a music signal (a non-significant source signal), or vice versa. Variations within the audio signal and / or music signal may also occur. This may result in abrupt changes in spectral weights and / or weighting factors. The signal processor and / or controller may further adjust the spectral weights and / or weighting factors to limit or limit variations between two frames, for example, by limiting the maximum step size between two signal frames . ≪ / RTI > One or more frames of the audio signal may be summed over a period of time and the signal processor and / or controller may compare the spectral weights and / or weighting factors of the previous time period, e.g., one or more previous frames, May be configured to determine whether the difference in spectral weightings and / or weighting factors determined for the threshold value exceeds a threshold. The threshold value may, for example, express a value that causes annoying effects to the listener. The signal processor and / or controller may be configured to limit variations such that these annoying effects are reduced or prevented. Alternatively, instead of a difference, other mathematical expressions, such as a ratio, may also be determined to compare the weighting factors of the spectral weights and / or the previous and actual time periods.

That is, features including the amount of pitch and / or transient characteristics are assigned to each frequency band.

FIG. 8 shows a schematic block diagram of a sound enhancement system 800 including an apparatus 801 for enhancing an audio signal 102. As shown in FIG. The sound enhancement system 800 includes a signal input 106 that is configured to receive an audio signal and provide an audio signal to the device 801. The sound enhancement system 800 includes two speakers 808a and 808b. Speaker 808a is configured to receive signal y1. The speaker 808b is configured to receive the signal y2 so that the signals y1 and y2 can be transmitted to the sound wave or signals using the speakers 808a and 808b. The signal input 106 may be a wired or wireless signal input, for example, a radio antenna. Device 801 may be, for example, device 100 and / or 200.

The correlated signal z can be obtained by applying processing (a qualitative inverse of the suppression to compute the signal s) that improves transient and tonal components. The coupling performed by the coupler can be expressed linearly with y (y1 / y2) = scaling factor 1z + scaling factor 2 scaling factor (r1 / r2). The scaling factors can be obtained by predicting the recognized reverse correlation strength.

Alternatively, the signals y1 and / or y2 may be further processed before being received by the speakers 808a and / or 808b. For example, signals y1 and / or y2 may be amplified, equalized, etc., such that signals or signals derived by processing signals y1 and / or y2 are provided to the speakers 808a and / or 808b.

The artificial echo added to the audio signal can be implemented so that the level of echo is audible but not too large (intensive). Audible or annoying levels may be determined in tests and / or simulations. Too high levels do not sound well due to poor clarity, percussion sounds being blurred over time, and so on. The target level may depend on the input signal. If the input signal contains a small amount of transients and contains a small amount of tonality by the frequency modulation, the echo can be audible to a lower degree and the level can be increased. Since the decorrelators may contain similar active principles, this applies similarly to the decorrelation. Thus, the optimal strength of the decorrelator may depend on the input signal. Computing may be the same by modified parameters. The decorrelation performed in the signal processor and the controller can be performed by two decorrelators, which may be structurally identical but operate on different sets of parameters. The inverse correlation processors are not limited to two-channel stereo signals but may also be applied to channels having more than two signals. The inverse correlation may be quantified with correlation metrics that may include all maximum values for the decorrelation of all signal pairs.

The discovery of the invented method is to create a spatial cue and introduce a spatial cue into the signal so that the processed signal produces a sense of the stereo signal. The processing may be deemed designed according to the following criteria.

One. Direct sound sources with high intensity (or loudness level) are localized in the center. These are prominent direct sound sources, for example, singer or loud instruments in music recording.

2. Peripheral sounds are perceived to be diffuse.

3. A direct sound source with low intensity (i. E., Low loudness levels) is added with a degree of diffusion, possibly to a range smaller than the ambient sounds.

4. Processing should be natural and should not introduce artifacts.

The design criteria are consistent with conventional practices in the generation of audio recordings and signal characteristics of the stereo signals.

One. Significant direct sounds are typically panned to the center, i. E. They are mixed with negligible ICLD and ICTD. These signals represent high coherence.

2. Ambient sounds represent low coherence.

3. For example, when recording a large number of direct sources in an echo environment such as an opera singer with an orchestra, the amount of diffusion of each direct sound is related to the distance of the sources to the microphones, The ratio between the direct signal and the echo is reduced. Thus, sounds captured at low intensities are typically coherent (i.e., more vice versa) than direct sounds.

The processing uses spatial correlation to generate spatial information. That is, the ICC of the input signals is reduced. In only extreme cases, the decorrelation results in completely uncorrelated signals. Typically, partial decorrelation is achieved and desirable. Processing does not manipulate directional queues (i.e., ICLD and ICTD). The reason for this limitation is that no information about the original or intended location of the direct sound source is available.

According to the design criteria, the decorrelation is selectively applied to the signal components in the mixed signal as follows:

One. As discussed in Design Criteria 1, no or little correlation is applied to the signal components.

2. An inverse correlation is applied to the signal components as discussed in Design Criteria 2. This inverse correlation contributes significantly to the perceived width of the mixed signal obtained at the output of the processing.

The inverse correlation is applied to the signal components as discussed in Design Criteria 3, but to a lesser extent than for the signal components as discussed in Design Criteria 2.

This processing is illustrated using a signal model in which the input signal x is expressed as an additive mixture of the foreground signal x _a and the background signal x _b , i.e., x = x _a + x _b . As discussed in Design Criteria 1, the foreground signal includes all signal components. As discussed in Design Criteria 2, the background signal includes all signal components. All signal components as discussed in Design Criteria 3 are not exclusively assigned to any of the separate signal components but are partly included in the foreground and background signals.

The output signal y is calculated as y = y _a + y _b , where y _b is calculated by decorrelating x _b and y _a is calculated by decorrelating y _a = x _a or x _a . That is, the background signal is processed using the decorrelation and the foreground signal is not processed using the decorrelation, or is processed by the decorrelation but is processed to a lesser extent than the background signal. Figure 9B illustrates this processing.

This approach does not remain to meet the above design criteria. An additional advantage is that foreground signals can be susceptible to undesired coloration when applying decorrelation, while backgrounds can be decoded without introducing these audible artifacts. Thus, the described processing results in better sound quality compared to processing that equally applies an inverse correlation to all signal components of the mixture.

Up to now, the input signal has been decomposed into two signals, marked as "foreground signal" and "background signal, " which are individually processed and combined into an output signal. It should be noted that equivalent methods that follow the same rationale are feasible.

The signal decomposition does not necessarily have to be an audio signal, i.e., a signal that outputs signals similar in shape to the waveform over time. Instead, signal decomposition can be used as an input to the decorrelation processing and can derive any other signal representation that can subsequently be transformed into a waveform signal. An example of such a signal representation is a spectrogram computed by a short-term Fourier transform. In general, the reversible and linear transformations yield appropriate signal representations.

Alternatively, the spatial cue is selectively generated without preceding signal decomposition by generating stereo information based on the input signal x. The derived stereo information is weighted with time-varying and frequency-selective values and combined with the input signal. The time-varying and frequency-selective weighting factors are computed such that they are small in time-frequency regions where the background signal is dominant in time-frequency regions and the foreground signal is dominant. This can be formulated by quantifying the time-varying and frequency-selective ratios of the background and foreground signals. The weighting factors may be computed from the background-to-foreground ratio, for example, using monotonically increasing functions.

Alternatively, proactive signal decomposition can yield more than two separate signals.

Figures 9A and 9B illustrate separating an input signal into foreground and background signals, for example, by suppressing (reducing or eliminating) tonal transient portions in one of the signals.

Simplified processing is induced by using the assumption that the input signal is an additive mixture of the foreground signal and the background signal. Figure 9b illustrates this. Here, separation 1 represents the separation of either the foreground signal or the background signal. When the foreground signal is separated, output 1 indicates the foreground signal and output 2 the background signal. When the background signal is separated, output 1 indicates the background signal and output 2 the foreground signal.

The design and implementation of the signal separation method is based on the discovery that foreground signals and background signals have distinct characteristics. However, deviations from ideal separation, i.e., leakage of signal components of significant direct sound sources into the background signal, or leakage of peripheral signal components into the foreground signal, are acceptable and do not necessarily impair the sound quality of the final result.

For temporal characteristics, it can be observed that the temporal envelopes of the subband signals of generally foreground signals are characterized by stronger amplitude modulations than the temporal envelopes of the subband signals of background signals. Conversely, background signals are typically less transient (or percussive, i.e. more persistent) than foreground signals.

In the case of spectral characteristics, it can be observed that generally foreground signals may be more negative. Conversely, background signals are typically noisier than foreground signals.

For the phase characteristics, it is generally observed that the phase information of the background signals is more noisy than the foreground signals. The phase information for many examples of foreground signals is consistent across a number of frequency bands.

Signals featuring characteristics similar to prominent sound source signals are more likely to be foreground signals than background signals. Significant source signals are characterized by transitions between the tonal signal component and the noise signal component, wherein the tonal signal components are time-varying filtered train with strongly modulated fundamental frequency. Spectral processing may be based on these properties, and decomposition may be implemented using spectral subtraction or spectral weighting.

Spectral subtraction is performed in the frequency domain where, for example, spectra of short frames of successive (possibly overlapping) portions of the input signal are processed. The basic principle is to subtract an estimate of the magnitude spectrum of the interfering signal from the magnitude spectra of the input signals assumed to be an additive mixture of the desired signal and the interfering signal. For foreground signal separation, the desired signal is the foreground and the interfering signal is the background signal. In case of background signal separation, the desired signal is background and the interference signal is foreground signal.

Spectral weighting (or short-term spectral attenuation) follows the same principle and attenuates the interference signal by scaling the input signal representation. The input signal x (t) may be a short term Fourier transform (STFT), a filter bank, or any other means for deriving a signal representation with multiple frequency bands X (n, k) Lt; / RTI > The frequency domain representations of the input signals are processed such that the subband signals are scaled to the time varying weights G (n, k).

(3)

(3)

The result of the weighting operation Y (n, k) is a frequency domain representation of the output signal. The output time signal y (t) is computed using the inverse processing of the frequency domain transform, e.g., an inverse STFT. Figure 10 illustrates spectral weighting.

The inverse correlation refers to processing one or more identical input signals such that a plurality of output signals that are mutually (partially or completely) uncorrelated but sounds similar to the input signal are obtained. The correlation between the two signals can be measured using a correlation coefficient or a normalized correlation coefficient. The normalized correlation coefficient NCC in the frequency bands for the two signals X ₁ (n, k) and X ₂ (n, k)

(4)

(4)

Lt; / RTI >

및

는 각각 제 1 및 제 2 입력 신호의 자동 전력 스펙트럼 밀도들(PSD)이고,

는,here,

And

Are the automatic power spectral densities (PSD) of the first and second input signals, respectively,

Quot;

(5)

(5)

는 기대값 연산이고, X^*는 X의 복수 콘주게이트를 표시한다.RTI ID = 0.0 > PSD, < / RTI >

Is an expected value operation, and X ^* denotes a plurality of conjugates of X.

The decorrelation can be implemented by using decorrelation filters or by manipulating the phase of the input signals in the frequency domain. An example for decorrelated filters is an all-pass filter, which by definition does not change the magnitude spectrum of the input signals, but only changes the phase of the signals. This leads to output signals that sound neutral in that the output signals are similar to the input signals. Another example is echo, which can also be modeled as a filter or a linear time invariant system. In general, the decorrelation can be achieved by adding a plurality of delayed (and possibly filtered) copies of the input signal to the input signal. In mathematical terms, an artificial echo can be implemented with the convolution of the input signal with the impulse response of the echo (or decorrelation) system. If the delay time is small, for example, less than 50 ms, the delayed copies of the signal are not recognized as separate signals (echoes). The exact value of the delay time to derive the sense of echo is the echo threshold and depends on the spectral and temporal signal characteristics. This is, for example, smaller for impulse sounds than for sounds with a slowly increasing envelope. For current problems, it is desirable to use delay times that are less than the echo threshold.

In a general case, the decorrelation process processes the input signal with N channels and outputs a signal with M channels such that the channel signals of the output are (partially or completely) mutually uncorrelated.

In many application scenarios for the described method it is not appropriate to process the input signal uniformly and it is appropriate to activate the input signal and to control its influence based on the analysis of the input signal. An example is an FM broadcast, where the described method applies only if the failure of transmission results in complete or partial loss of stereo information. Another example is listening to a collection of music recordings, where the subset of recordings is monophonic and the other subset is stereo recordings. Both scenarios are characterized by the time-varying amount of stereo information of the audio signals. This requires control of the activation and influence of the stereo enhancement, i.e., the control of the algorithm.

Control is implemented using analysis of audio signals that estimate the spatial cues (ICLD, ICTD, and ICC, or a subset thereof) of the audio signals. The estimation can be implemented in a frequency selective manner. The output of the estimate is mapped to a scalar value that controls the activation or influence of processing. The signal analysis processes an input signal or alternatively a separate background signal.

A simple way to control the effect of processing is to reduce its effect by adding a (possibly scaled) copy of the input signal to the (possibly scaled) output signal of the stereo enhancement. The smooth transition of the control is obtained by low pass filtering the control signal over time.

9A shows a schematic block diagram of processing 900 of input signal 102 in accordance with background / foreground processing. The input signal 102 is separated so that the foreground signal 914 can be processed. In step 916, an inverse correlation is performed on the foreground signal 914. Step 916 is optional. Alternatively, foreground signal 914 may not be processed, i.e., uncorrelated. In step 922 of the processing path 920, the background signal 924 is extracted, i.e., filtered. At step 926, the background signal 924 is decoded. At step 904, the decorrelated foreground signal 918 (alternatively foreground signal 914) and the decorrelated background signal 928 are mixed to obtain an output signal 906. [ That is, Figure 9A shows a block diagram of the stereo enhancement. Foreground signals and background signals are computed. The background signal is processed by inverse correlation. Optionally, the foreground signal may be processed by the decorrelation, but to a lesser extent than the background signal. The processed signals are coupled to an output signal.

FIG. 9B shows a schematic block diagram of a processing 900 'that includes a separation step 912' of an input signal 102. The separation step 912 'may be performed as described above. The foreground signal (output signal 1) 914 'is obtained by a separation step 912'. Background signal 928 'is obtained by combining foreground signal 914', weighting factors a and / or b, and input signal 102 in combining step 926 '. The background signal (output signal 2) 928 'is obtained by the combining step 926'.

10 shows a schematic block diagram and apparatus 1000 that is configured to apply spectral weights to an input signal 1002, which may be, for example, an input signal 1002. [ The time domain input signal 1002 is divided into subbands X (1, k) ... X (n, k) in the frequency domain. The filter bank 1004 is configured to divide the input signal 1002 into N subbands. The apparatus 1000 includes N sets of N spectral weights G (n, k) configured to determine transient spectral weights and / or tone spectral weights G (1, k) ... G (n, k) for each of the N subbands in a time instance Calculation instances. The spectral weights G (1, k) ... G (n, k) are set such that the weighted subband signals X (1, k) The band signal X (1, k) ... X (n, k). Apparatus 1000 includes a reverse processing unit 1008 configured to combine the weighted subband signals to obtain a filtered output signal 1012 denoted Y (t) in the time domain. Apparatus 1000 may be part of signal processor 110 or 210. That is, FIG. 10 illustrates decomposing an input signal into a foreground signal and a background signal.

FIG. 11 shows a schematic flow diagram of a method 1100 for enhancing an audio signal. The method 1100 includes a first step 1110 in which an audio signal is processed to reduce or remove transient and tonal portions of the processed signal. The method 1100 includes a second step 1120 in which a first de-correlated signal and a second de-correlated signal are generated from the processed signal. In step 1130 of method 1100, the first de-correlated signal, the second decorrelated signal and the audio signal, or a signal derived from the audio signal by coherence enhancement, Are weighted together by using time-varying weight factors. In step 1140 of method 1100, the time-varying weight factors are controlled by analyzing the audio signal such that different portions of the audio signal are multiplied by different weighting factors and the 2-channel audio signal has a time varying decorrelation.

Details for explaining the possibility of determining the recognized correlation level based on the loudness measurement will next be described. As shown, the loudness measurement may allow to predict the recognized echo level. As noted above, echoes also mean de-correlated so that the recognized echo level can also be regarded as the recognized de-correlation level, and in the case of de-correlated echo can be less than one second, for example 500 ms And may be shorter than 250 ms or 200 ms.

Figure 12 illustrates an apparatus for determining a measure for a recognized echo level in a mixed signal comprising a direct signal component or a dry signal component 1201 and an echo signal component 102. [ The dry signal component 1201 and the echo signal component 1202 are input to the loudness model processor 1204. The loudness model processor is configured to receive the direct signal component 1201 and the echo signal component 1202 and also includes a recognition filter stage 1204a and a subsequently connected loudness calculator 1204b as illustrated in Figure 13a. do. The loudness model processor generates a first loudness measurement 1206 and a second loudness measurement 1208 as its output. Both of the loudness measurements are input to the combiner 1210 to combine the first loudness measurement 1206 and the second loudness measurement 1208 and a measurement 1212 for the finally recognized reverberation level is obtained. Depending on the implementation, the measurement for the detected level 1212 is input to a predictor 1214 for predicting the recognized echo level, based on the average of at least two measurements for the recognized loudness for different signal frames . However, the predictor 1214 of FIG. 12 transforms the measurements for the selectively and practically recognized levels into a unit range or a range of specific values, such as a range of Sone units useful for giving quantitative values associated with loudness . However, at the controller, other uses may also be used for measurements on the recognized level 1212 that have not been processed by the predictor 1214, for example, But need not be dependent on the recognized level 1212 in a kind of smoothed form or in a direct form, preferably with temporal smoothing being preferred so as not to have strongly varying level corrections of the echoed signal or gain factor g You can directly process the measurements for.

In particular, the recognition filter stage is configured to filter a direct signal component, an echo signal component, or a mixed signal component, wherein the recognition filter stage models an auditory perception mechanism of an object such as a human to generate a filtered direct signal, To obtain a mixed signal. Depending on the implementation, the recognition filter stage may comprise two filters operating in parallel, or may comprise a storage and a single filter, which may comprise three signals, echo signal, mixed signal and direct signal respectively In fact one and the same filter can be used. However, in this situation, FIG. 13A illustrates n filters that model acoustic perception mechanisms, but actually filters two signals from a group that includes two filters or echo signal components, mixed signal components, and direct signal components A single filter may be sufficient.

The loudness calculator 1204b or the loudness evaluator is configured to estimate the first loudness-related measure using the filtered direct signal and estimate the second loudness measure using the filtered echo signal or the filtered mixed signal, Signal component and echo signal component superposition.

Figure 13c illustrates four preferred modes for calculating measurements for the recognized echo level. The implementation relies on the partial loudness in which the direct signal component x and the echo signal component r are both used in the loudness model processor but the echo signal is used as a stimulus to determine the first measurement EST1 and the direct signal is used as noise . To determine the second loudness measure EST2, the situation is changed, a direct signal component is used as a stimulus, and an echo signal component is used as noise. The measurement for the recognized correction level produced by the coupler is then the difference between the first loudness measurement EST1 and the second loudness measurement EST2.

However, there are additional computationally efficient embodiments shown in lines 2, 3 and 4 of Figure 13c. These more computationally efficient measurements are dependent on calculating the total loudness of the three signals including the mixed signal m, the direct signal x and the echo signal n. The first loudness measure EST1 is the total loudness of the mixed signal or the echo signal and the second loudness measure EST2 is the sum of the direct signal component x or the sum of the mixed signal components m, according to the required calculation performed by the combiner shown in the last column of Figure 13c. Loudness, and the actual combination is as illustrated in Fig. 13C.

Figure 14 illustrates an implementation of the loudness model processor already discussed in some aspects with respect to Figures 12, 13A, 13B, and 13C. In particular, the recognition filter stage 1204a includes a time-to-frequency converter 1401 for each branch, where x [k] denotes the stimulus and n [k] denotes the noise do. The time / frequency converted signal is forwarded to an ear transfer function block 1402 (the ear-transfer function is alternatively computed prior to the time-to-frequency converter with similar results, but with a higher computational load) The output of this block 1402 is input to the excitation pattern computing block 1404 and then to the temporal integration block 1406. [ Then, at block 1408, the specific loudness in this embodiment is calculated, where block 1408 corresponds to loudness calculator block 1204b in FIG. 13A. Subsequently, integration across frequency is performed at block 1410, where block 1410 corresponds to the adder already described as 1204c and 1204d in Figure 13b. Block 1410 generates a first measurement for the first set of stimuli and noise, and a second measurement for the second set of stimuli and noise. In particular, if FIG. 13B is taken into account, the stimulus for calculating the first measurement is an echo signal and the noise is a direct signal, while in the second measurement calculation the situation is changed so that the stimulus is a direct signal component and the noise is an echo signal component to be. Thus, to generate two different loudness measurements, the procedure illustrated in FIG. 14 was performed twice. However, the change in the calculation only occurs in differently operating blocks 1408, so that the steps illustrated by blocks 1401 through 1406 need only be performed once, and in the case of the implementation shown in Figure 13c, The result of the temporal integration block 1406 may be stored to compute the estimated loudness and the second estimated loudness. For other implementations, it should be noted that block 1408 can be replaced with a separate block "total volume computing" for each branch, and in this implementation, whether one signal is considered a stimulus or noise .

While some aspects have been described in the context of an apparatus, it is evident that these aspects also represent a description of a corresponding method, wherein the block or device corresponds to a feature of a method step or method step. Similarly, aspects described in the context of a method also represent a description of a block or item or feature of the corresponding device.

Depending on the specific implementation requirements, embodiments of the present invention may be implemented in hardware or software. The implementation may be implemented in a digital storage medium, such as a floppy disk, a DVD, a CD, a ROM, a floppy disk, a CD, a CD, a CD, PROM, EPROM, EEPROM or FLASH memory.

Some embodiments in accordance with the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system to perform one of the methods described herein.

In general, embodiments of the present invention may be implemented as a computer program product having program code, and the program code is operable to perform one of the methods when the computer program product is run on a computer. The program code may be stored, for example, on a machine-readable carrier.

Other embodiments include a computer program stored on a machine readable carrier for performing one of the methods described herein.

Thus, in other words, one embodiment of the inventive method is a computer program having program code for performing one of the methods described herein when the computer program is run on a computer.

Accordingly, additional embodiments of the inventive methods are data carriers (e.g., digital storage media or computer readable media) in which computer programs for carrying out one of the methods described herein are included and recorded.

Thus, a further embodiment of the inventive method is a sequence or data stream of signals representing a computer program for performing one of the methods described herein. For example, a sequence of signals or a data stream may be configured to be transmitted over a data communication connection, for example, over the Internet.

Additional embodiments include processing means, e.g., a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

Additional embodiments include a computer in which a computer program for performing one of the methods described herein is installed.

The embodiments described above are merely illustrative of the principles of the present invention. It is understood that variations and modifications of the arrangements and details described herein will be apparent to those skilled in the art. Accordingly, it is intended that the invention be limited only by the scope of the following claims, and are not intended to be limited by the specific details presented in the manner of description and explanation of the embodiments herein.

Claims (15)

An apparatus (100; 200) for enhancing an audio signal (102)
A signal processor (110; 210) for processing the audio signal (102) to reduce or remove transient and tonal portions of the processed signal (112; 212);
An decorrelator (120; 520) for generating a first decorrelated signal and a second decorrelated signal (124; r2) from the processed signal (112; 212);
(122; 522, r1), the second decorrelated signal (124; r2), and a signal induced by the coherence enhancement from the audio signal or the audio signal (102) A combiner (140; 240) for weighting using the time-varying weight factors (a, b) and obtaining a 2-channel audio signal (142; 242); And
The audio signal 122 is multiplied by different weighting factors a and b so that different portions fb1 through fb7 of the audio signal are multiplied by different weighting factors a and b and the two- (130, 230) for controlling said time-varying weight factors (a, b)
Device. The method according to claim 1,
The controller 130 230 increases the weighting factors a, b for the portions fb1-fb7 of the audio signal 102 to allow a higher degree of decorrelation, (A, b) for the portions (fb1-fb7) of the plurality of subcarriers (102, 102) to allow a lower degree of decorrelation.
Device. 3. The method according to claim 1 or 2,
The controller 130 is configured to scale the weighting factors a, b such that the recognized de-correlation level of the two-channel audio signal 142; 242 is maintained within a certain range around the target value , Said range extending to +/- 20% of said target value,
Device. The method of claim 3,
The controller 130 230 determines the target value by echoing the audio signal 102 to obtain an echoed audio signal and compares the audio signal with the echoed audio signal 102 to obtain a comparison result Wherein the controller is configured to determine the recognized de-correlation level (232) based on the comparison result,
Device. 5. The method according to any one of claims 1 to 4,
The controller (130; 230) determines a significant sound source signal portion of the audio signal (102), and determines the weighting factor for the significant sound source signal portion relative to the portion of the audio signal (102) B, < / RTI >
Wherein the controller (130; 230) determines a non-significant source signal portion of the audio signal (102), and wherein the non-significant source B) < / RTI > for the signal portion,
Device. 6. The method according to any one of claims 1 to 5,
The controller (130; 230)
Generate a test decorrelated signal from a portion of the audio signal (102);
Derive a measure of a portion of the audio signal and a recognized correlation level from the test decorrelated signal;
And to derive the weighting factors (a, b) from the measurements for the recognized de-correlation level.
Device. The method according to claim 6,
The decorrelator (120, 520) is configured to generate the first decorrelated signal (122; r1) based on an echo of the audio signal (102) having a first echo time, the controller ) Is configured to generate the test decorrelated signal based on an echo of the audio signal (102) having a second echo time, wherein the second echo time is shorter than the first echo time
Device. 8. The method according to any one of claims 1 to 7,
The controller (130; 230) is configured to control the weighting factors (a, b) such that each of the weighting factors (a, b) comprises a value of one of the first plurality of possible values, Wherein the plurality of values includes at least three values including a minimum value, a maximum value, and a value between the minimum value and the maximum value,
The signal processor (110, 210) is configured to determine spectral weights (217, 219) for a second plurality of frequency bands each representing a portion of the audio signal (102) in the frequency domain, Each of the first plurality 217 and the second 219 includes a value of one of the third plurality of possible values and the value of the third plurality comprises at least three values including a minimum value, a maximum value, and a value between the minimum value and the maximum value ≪ / RTI >
Device. 9. The method according to any one of claims 1 to 8,
The signal processor (110; 210)
The audio signal 102 being transmitted in the frequency domain and the second plurality of frequency bands fb1-fb7 being representative of a second plurality of portions in the frequency domain between the audio signal 102 Processing the signal 102;
For each frequency band (fb1-fb7), determining a first spectral weight (217) representing a processing value for the transient processing (211) of the audio signal (102);
For each of the frequency bands (fb1-fb7), determining a second spectral weight (219) representing a processing value for tonal processing (213) of the audio signal (102);
And at least one of the first spectral weight 217 and the second spectral weight 219 for each of the frequency bands fb1 to fb7 is divided into a spectrum of the audio signal 102 in the frequency bands fb1 to fb7, Values,
Wherein each of the first spectral weight 217 and the second spectral weight 219 comprises a value of one of a third plurality of possible values and wherein the value of the third plurality comprises at least one of a minimum value, And at least three values including a value between the maximum values.
Device. 10. The method of claim 9,
Wherein for each of the second plurality of frequency bands (fb1-fb7), the signal processor (110; 210) comprises a first spectral weight (217) determined for the frequency band (fb1-fb7) (217; 219) including the smaller value is compared with the weighted value (219) in the frequency band (fb1-fb7) by comparing the weighted value (219) Is adapted to apply to the spectral values of the audio signal (102)
Device. 11. The method according to any one of claims 1 to 10,
The decorrelator 520 includes a first decorrelation filter 526 configured to filter the processed audio signal 512, s to obtain the first decorrelated signal 522, rl, And a second decorrelation filter (528) configured to filter the audio signal (512, s) to obtain a second decorrelated signal (524, r2), wherein the combiner (140; 240) The second decorrelated signal 524, r2 and the signal 136 (236) derived from the audio signal 102 or the audio signal 102 in a weighted combination of the decorrelated signal 522, r1, the second decorrelated signal 524, To obtain the 2-channel audio signal (142; 242)
Device. 12. The method according to any one of claims 1 to 11,
For a second plurality of frequency bands fb1-fb7, each of the frequency bands fb1-fb7 comprises a portion of the audio signal 102 having a first time period and being represented in the frequency domain,
The controller (130; 230) controls the weighting factors (a, b) such that each of the weighting factors (a, b) comprises a value of one of a first plurality of possible values, Comprises a minimum value, a maximum value and at least three values including a value between the minimum value and the maximum value, the value of the weighting factors (a, b) determined for the actual time period and the value of the previous time period (A, b) determined for the real time period so that the ratio or difference is reduced if the ratio or difference based on the values of the weight factors (a, b) determined for the real time period is greater than or equal to the threshold value, , ≪ / RTI >
Wherein the signal processor (110, 210) is configured to determine the spectral weights (217, 219) each comprising a value of one of a third plurality of possible values, And a value between the minimum value and the maximum value,
Device. As a sound enhancement system 800,
An apparatus (801) for enhancing an audio signal according to any one of claims 1 to 12;
A signal input (106) configured to receive the audio signal (102);
Channel audio signal y1 / y2 or a signal derived from the 2-channel audio signal y1 / y2 and the 2-channel audio signal y1 / / y2), < / RTI > and at least two speakers (808a, 808b)
Sound Enhancement System. A method (1100) for enhancing an audio signal (102)
Processing (1110) the audio signal (102) to reduce or eliminate transient and tonal portions of the processed signal (112; 212);
Generating (1120) a first decorrelated signal (122, r1) and a second decorrelated signal (124; r2) from the processed signal (112; 212);
(122, r1), the second decorrelated signal (124, r2) and a signal (102) derived from the audio signal (102) or the coherence enhancement 136; 236) weighted using the time-varying weight factors (a, b) and obtaining a 2-channel audio signal 142 (242) (1130); And
By analyzing the audio signal 102 such that different portions of the audio signal are multiplied by different weighting factors a and b and the 2-channel audio signal 142 (242) has a time varying decorrelation, (A, b). ≪ RTI ID = 0.0 >
Way. A non-temporary storage medium in which a computer program is stored,
The computer program comprising program code for performing a method for enhancing an audio signal according to claim 14 when executed by a computer,
Non-temporary storage medium.

Images