EP2402943B1 - Method and device for creating an environmental signal - Google Patents

Description

The present invention relates to audio signal processing, and more particularly to concepts for generating ambient signals (ambience signals) for loudspeakers in a multi-channel scenario for which no dedicated loudspeaker signal has been transmitted.

Multi-channel audio is becoming more and more popular. As a result, many end users now have multi-channel playback systems. This is mainly because DVDs are becoming more and more popular and that is why many DVD users now have 5.1 multi-channel equipment. Such playback systems generally consist of three speakers L (left), C (center) and R (right), which are typically located in front of the user, and two speakers Ls and Rs located behind the user, and typically one of them LFE channel, which is also called low-frequency effect channel or subwoofer. Such a channel scenario is in Fig. 10 and in Fig. 11 indicated. While the positioning of the speakers L, C, R, Ls, Rs, with respect to the user as in the FIGS. 10 and 11 should be made drawn so that the user gets the best possible listening experience, is the positioning of the LFE channel (in Figs. 10 and 11 not shown) is not as critical as the ear can not locate at such low frequencies and thus the LFE channel can be located anywhere where it does not bother due to its considerable size.

Such a multi-channel system provides several advantages over a typical stereo reproduction, which is a two-channel reproduction, such as in Fig. 9 is shown.

Even outside of the optimal central listening position results in improved stability of the front listening experience, which is also referred to as a "front image", due to the center channel. This results in a larger "sweet spot", where "sweet spot" stands for the optimal listening position.

Further, the listener has a better feeling of "immersing" in the audio scene due to the two rear speakers Ls and Rs.

However, there is a huge amount of user-owned or commonly available audio material that only exists as stereo material, so it only has two channels, the left channel and the right channel. Typical sound carriers for such stereo pieces are compact disks.

To play such a stereo material over a 5.1 multi-channel audio system, you have two options, which are recommended according to the ITU.

The first option is to play the left and right channels through the left and right speakers of the multi-channel playback system. A disadvantage of this solution, however, is that you do not exploit the variety of existing speakers, so that you do not take advantage of the presence of the center speaker and the two rear speakers advantageous.

Another option is to convert the two channels into a multi-channel signal. This can be done during playback or by a special preprocessing, which advantageously takes advantage of all six speakers of the existing example 5.1 playback system and thus leads to an improved listening experience when the upmixing or the "Upmix" of two channels to 5 or 6 channels is performed without errors.

Only then does the second option, ie the use of all the loudspeakers of the multichannel system, have an advantage over the first solution, if one commits no upmix errors. Such upmix errors can be especially troublesome when signals for the rear speakers, which are also known as ambience signals or ambient signals, are not generated without error.

One way to perform this so-called upmix process is known under the keyword "Direct Ambience Concept". The direct sound sources are reproduced by the three front channels so that they are perceived by the user at the same position as in the original two-channel version. The original two-channel version is in Fig. 9 shown schematically, using the example of various drum instruments.

Fig. 10 shows a highly mixed version of the concept, in which all the original sound sources, so the drum instruments again from the three front speakers L, C and R are played, in addition to the two rear speakers special environmental signals are output. The term "direct sound source" is thus used to describe a sound coming only and directly from a discrete sound source, such as a drum instrument or other instrument, or generally a particular audio object, as shown schematically in, for example, US Pat Fig. 9 represented by a drum instrument. Any additional sounds, such as due to wall reflections, etc. are not present in such a direct sound source. In this scenario, the sound signals coming from the two rear speakers Ls, Rs in Fig. 10 are emitted only from environmental signals that are present in the original record or not. Such environmental signals or "Ambience" signals do not belong to a single sound source, but contribute to the reproduction of the room acoustics of a recording and thus lead to the so-called "immersion" feeling of the listener.

Another alternative concept, called in-the-band concept, is in Fig. 11 shown schematically. Each type of sound, ie direct sound sources and ambient sounds, are all positioned around the listener. The position of a sound is independent of its characteristics (direct sound sources or ambient sounds) and depends only on the specific design of the algorithm, as described in eg Fig. 11 is shown. So was in Fig. 11 the upmix algorithm determines that the two instruments 1100 and 1102 are positioned laterally relative to the listener while the two instruments 1104 and 1106 are positioned in front of the user. As a result, the two rear speakers Ls, Rs now also contain portions of the two instruments 1100 and 1102 and no longer just ambient sounds, as in Fig. 10 was still the case where the same instruments were all positioned in front of the user.

The specialist publication " C. Avendano and JM Jot: "Ambience Extraction and Synthesis from Stereo Signals for Multichannel Audio Upmix", IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 02, Orlando, FL, May 2002 "discloses a frequency domain technique for identifying and extracting environmental information in stereo audio signals." This concept is based on the calculation of inter-channel coherence and a non-linear mapping function to allow time-frequency regions in the stereo signal to be determined. Surround signals are then synthesized and used to connect the rear channels or "surround" channels Ls, Rs (which are mainly ambient components. Figs. 10 and 11 ) of a multi-channel playback system.

In the technical publication " R. Irwan and Ronald M. Aarts: "A Method to Convert Stereo to Multi-channel Sound", The Proceedings of the AES 19th International Conference, Schloss Elmau, Germany, June 21-24, pp. 139-143, 2001 A method for converting a stereo signal to a multi-channel signal is presented, and the signal for the surround channels is calculated using a cross correlation technique This vector is then mapped from a two-channel representation to a three-channel representation to produce the three front channels.

The specialist publication " G. Soulodre, Ambience Based Upmixing, Spatial Coding Of Surround Sound: A Progress Report, 117th AES Convention, San Francisco, CA, USA, 2004 discloses a system that generates a multi-channel signal from a stereo signal, which is decomposed into so-called individual source streams and ambient streams, based on which a so-called "aesthetics processor" synthesizes the multi-channel output signal.

All known techniques try in various ways to extract the ambience signals or ambient signals from the original stereo signal or even to synthesize from noise or other information, which also used for the synthesis of ambience signals, which are not in the stereo signal can be. Ultimately, however, it is always about extracting information from the stereo signal or feed information into a playback scenario, which are not explicitly available, since typically only a two-channel stereo signal and possibly any additional information or meta-information are available.

In this respect, the extraction or partial extraction and partial synthesis of such ambient signals is one risky affair, as it would be annoying for a user to include information from sound sources in the surround channels which the user identifies as coming from the front, ie from the left channel, middle channel and right channel. For this reason, generating ambient signals would be very "defensive" to make sure that no user-perceived artifacts are generated. The other extreme case, if one acts too defensively in the generation of the ambient signals, is that a very quiet, or barely perceptible ambient signal is extracted, or that the ambient signal has only noise, but no longer has any special information, so that the ambient signal contributes very little to the enjoyment of listening and could actually be left out altogether in this case.

The problem with the generation of the ambient signal is thus that on the one hand generates an ambient signal that includes information that goes beyond normal noise, but that the ambient signal unobtainable artifacts leads, so that a proper measure between audibility and information content is maintained.

The US 2006/0018486 A1 discloses an efficient filter for generating an artificial environmental signal. The filter includes a transient reduction module adapted to reduce transients in an audio signal, the audio signal comprising one or more channels. Further, the filter comprises a reverberation filter adapted to receive the transient-reduced signal and to generate a reverberation signal having one or more channels and corresponding to the transient-reduced signal.

The U.S. Patent No. 4,076,969A discloses an impulse noise reduction system for detecting unwanted pop and click sound signals generated by mechanical defects on a recording medium. A few of fully or partially correlated signals are generated from the recorded sound information by means of a stereo recording device. The output signals of the stereo recording apparatus are combined so that desired sound signals generated by horizontal movement on the recording medium are subtracted and undesired signals generated by vertical movement on the recording medium are added. The resulting unwanted signal is detected as a noise signal. A control signal that starts before the leading edge of the detected unwanted sound signal and that has a width that is greater than the width of the detected unwanted sound signal is used in the system to eliminate the unwanted signals.

The specialist publication " Restoration of Historical Recordings by Means of Digital Signal Processing ", Preprint No. 2091, WA German, et al., 75th Convention 1984 refers to improving the overall quality of the records that have been degraded by multiple glitches. In particular, impulsive noises generated by scratches and cracks in the splitting material are to be eliminated. Furthermore, distortions in the frequency domain, which are generated by the recording device to be compensated. Furthermore, the continuous broadband surface noise is to be reduced.

The specialist publication " Detection and modeling of fast attack transients ", X. Rodet et al., Proc. Of Intl. Computer Music Conf. (ICMC), September 1, 2001, pages 1-4 , discloses detection and modeling of transients due to fast attacks. For this, the energy in a frequency band is examined, and a peak with triangular shape is sought, which lies between previous and subsequent plateaus.

The specialist publication " Enhancement of Audio Signaling Using Transient Detection and Modification ", AES Convention Paper, New York, Vol. 6255, 2004, pp. 2-11 discloses a processing approach that makes it possible to perform perceptually responsive modification of audio signals via the transient enhancement of transients. A frequency domain analysis is used which gives a spectral flux parameter. This parameter is compared to a threshold to derive a binary transient detection function.

The object of the present invention is to provide a concept for generating an environmental signal in which audible artifacts are reduced.

This object is achieved by a device for generating an environmental signal according to claim 1, a method for generating an environmental signal according to claim 10 or a computer program according to claim 11.

The present invention is based on the recognition that the artifacts that are most negatively perceived by listeners in ambient signals are artifacts that cause the listener to think that a direct sound source is in the rear speaker, although he or she is Sound source perceives from the front. Characteristics for the perception of direct sound sources are transient processes, ie signal fine structures in the time signal, which concern a (fast) change over a change threshold from a quiet state to a loud state or from a loud state to a quiet state, or one (Strong) energy increase above a threshold of change in special bands and especially in the upper bands within a certain time.

Such transient events are, for example, the insertion of an instrument or the impact of a percussion instrument, or the end of a sound that does not fade away slowly, but ends abruptly. A listener perceives such transient events as characteristics of direct sound sources, which according to the invention are eliminated from an ambient signal, so that the ambient loudspeakers are supplied with an environmental signal generated according to the invention which comprises no or only strongly attenuated transients.

According to the invention, it is further ensured that the suppression of a transient in the ambient signal does not lead to an excessive amplitude modulation. In fact, according to the invention, it has also been found that variations in the amplitude, that is, the loudness, even if they are not transient, are below the transient threshold, but above a certain threshold of variation, are annoying to the user and therefore when such amplitude variations are due to a transient threshold simply eliminating a transient in an ambient signal would be detected by the listener as an artifact or error.

According to the invention, a transient period is thus detected in an examination signal, in which a transient region is present in the examination signal. Then, with the help of a synthesis signal generator, a synthesis signal generated for the transient period, which is configured to generate the synthesis signal to have a shallower time course than the examination signal in the transient region, wherein the synthesis signal generator is further configured to generate the synthesis signal so that it to its intensity of a preceding or succeeding portion of the examination signal by less than a predetermined threshold. This generated synthesis signal is then used by a signal substitutor instead of the examination signal in the transient period to obtain the ambient signal.

Thus, according to the invention, the extraction of an ambient signal-like signal from a two-channel stereo input signal is improved, or a post-processing of an existing signal, e.g. already an extracted raw ambient signal is made. In the first case, the examination signal is the actual two-channel stereo signal or respectively one channel of the two-channel signal, while in the second case the examination signal is already an extracted environment signal or a presynthesized environment signal. Thus, the inventive concept is particularly useful for the upmix concept, which has also been presented as a "direct ambience concept". The concept according to the invention can also be advantageous for the "in-the-band" concept, since it also leads to an improved environmental signal which, on the one hand, no longer has any interfering artifacts, but on the other hand still contains enough information for a user to benefit from the ambient signal.

The ambient signal generation according to the invention results in that the ambient signal has no relevant parts of direct sound sources, wherein in particular no transients are contained or transients are contained only in a very strongly attenuated form. Otherwise, the listener would perceive direct sound sources behind them, resulting in conflict to the experience of the user who typically perceives sound sources only from the front.

Further, the inventive concept ensures that the surround signal is a continuous, uninterrupted, diffused sound signal, since intermittent ambient sound, which is obtained, for example, if transients were simply completely eliminated, would be considered uncomfortable by the user or even errors in the high-mix process would be perceived.

In a preferred embodiment of the present invention, to achieve a direct ambience type of upmixing process, an ambience-like signal for the rear channels is extracted from the stereo signal. In order to achieve this, for example, only the uncorrelated signal components or, as a simple solution, the difference between the original right and left channels is simply used. When the back channels are created in this way, they often have transient-like components of direct sound sources. These transients may be tones, such as, for example, musical beginnings or parts of percussive instruments. A transient perceived behind the listener while a direct sound source (to which the transient typically belongs) is positioned in front of the listener has a negative impact on the localization of the direct sound source. The direct sound source thus appears either wider than the original or, even more damaging, perceived as an independent direct sound source behind the user, both effects being very undesirable, especially for the direct ambience concept.

According to the invention, this problem is addressed by suppressing transients in the ambient-like signal, and minimizing the effect of this suppression on the remaining signal, ie, preserving the continuity of the signal by providing only limited intensity variations be admitted for the transient period.

In the preferred embodiment of the present invention, the signal generated for the transient period, before being used by the signal substituter, is mixed with the signal originally present in the transient period, which is achieved, for example, by overlapping processing. Alternatively or additionally, in order to suppress or at least reduce the discontinuities at the edges of the transient period, a cross-fading may be performed to slowly fade in a cross-fade range from the signal before the transient period to the signal in the transient period, or slowly out of the transient period hide.

In particular, the blanking out of the transient period into the original signal when no more transient is detected is preferred for an artifact-free listening impression, since it is intended to ensure that when there is a non-artifact-related examination signal, the transition from the synthesis signal to the original one Examination signal no cracking or something similar arises.

In further preferred embodiments of the present invention, a manipulation of the signal in the transient period is performed in the frequency domain by randomizing signs of spectral values or, more generally, phases of spectral values, which inevitably results in a smoothing of the temporal fine structure of this frequency domain manipulated signal. Another spectral processing is to perform a prediction on the frequency of the spectral values and then use the prediction spectral values as spectral values of the synthesis signal, since the prediction over the frequency leads to a smoothing of the corresponding time signal.

To suppress transients while maintaining or only slightly influencing, it is preferred to change the intensity of the transient period by at most plus or minus 50%, is to limit the change of spectral values from one block to another, this limitation being global , ie for all spectral values equal or selective, ie only for certain spectral values, which have a particularly large change, can take place.

Fig. 1

ein Blockschaltbild der erfindungsgemäßen Vorrichtung zum Erzeugen eines Umgebungssignals;

Fig. 2a

eine schematische Darstellung der Blockverarbeitung bei nicht-überlappenden Blöcken, jedoch mit Überblendbereich;

Fig. 2b

eine schematische Darstellung der Synthesesignalerzeugung bei überlappenden Blöcken;

Fig. 3

eine spezielle Implementierung einer Überblendung mit einer Einblendfunktion und einer Ausblendfunktion, die für Fig. 2a oder Fig. 2b eingesetzt werden kann;

Fig. 4

ein Blockschaltbild einer bevorzugten Implementierung mit einer Verarbeitung im Frequenzbereich;

Fig. 5a

eine alternative Implementierung der Frequenzbereichsverarbeitung;

Fig. 5b

eine wieder alternative Frequenzbereichsverarbeitung;

Fig. 5c

eine bevorzugte Implementierung einer Intensitäts-basierten Verarbeitung;

Fig. 6

eine Implementierung zur Erhaltung tonaler Bereiche im Synthesesignal;

Fig. 7

ein Blockschaltbild einer bevorzugten Ausführungsform basierend auf dem Hochfrequenzgehalt HFC;

Fig. 8

eine bevorzugte Implementierung der erfindungsgemäßen Vorrichtung mit zusätzlicher Funktionalität zum Erzeugen der Direktschallkanäle L, R, C;

Fig. 9

ein Stereo-Wiedergabe-Szenario;

Fig. 10

ein Multikanal-Wiedergabe-Szenario, bei dem alle Direktschallquellen durch die vorderen Kanäle wiedergegeben werden; und

Fig. 11

ein Multikanal-Wiedergabe-Szenario, bei dem Schallquellen auch durch hintere Kanäle wiedergebeben werden können.

Preferred embodiments of the present invention will be explained below in detail with reference to the accompanying drawings. Show it:

Fig. 1

a block diagram of the device according to the invention for generating an environmental signal;

Fig. 2a

a schematic representation of the block processing in non-overlapping blocks, but with cross-fade area;

Fig. 2b

a schematic representation of the synthesis signal generation in overlapping blocks;

Fig. 3

a special implementation of a fade with a fade - in function and a fade - in function, which is used for Fig. 2a or Fig. 2b can be used;

Fig. 4

a block diagram of a preferred implementation with a processing in the frequency domain;

Fig. 5a

an alternative implementation of frequency domain processing;

Fig. 5b

another alternative frequency domain processing;

Fig. 5c

a preferred implementation of intensity-based processing;

Fig. 6

an implementation for preserving tonal regions in the synthesis signal;

Fig. 7

a block diagram of a preferred embodiment based on the high frequency content HFC;

Fig. 8

a preferred implementation of the device according to the invention with additional functionality for generating the direct sound channels L, R, C;

Fig. 9

a stereo playback scenario;

Fig. 10

a multi-channel playback scenario in which all direct sound sources are played back through the front channels; and

Fig. 11

a multi-channel playback scenario in which sound sources can also be reproduced through rear channels.

Fig. 1 shows an inventive device for generating an ambient signal 10, which is suitable for broadcasting via speakers for which no separate speaker signal has been transmitted. Such speakers are typically the rear speakers or surround speakers, as in Ls, Rs in 10 and FIG. 11 For example, shown.

In the Fig. 1 The device shown comprises a transient detector 11 for detecting a transient period (at 20 in Fig. 2 shown) in which an examination signal has a transient region. Although some implementations of the transient detector are described here It should be noted that any other methods for transient detection can be used, such as those found in an MPEG4 audio encoder, in which is switched depending on a transient detection of short to long windows. Also in other areas of audio signal processing, transient detectors are used which can detect fast and strong changes in the envelope of a time signal. Exemplary magnitudes to be detected are changes in the envelope that affect changes equal to or greater than 100% of the amplitude of the envelope over a period of 1 ms.

The transient detector 11 is coupled to a synthesis signal generator 12, which is designed to generate a synthesis signal 13 which satisfies the two conditions, on the one hand the transient condition and, on the other hand, the continuity condition. The transient condition is that the synthesis signal has a shallower time course than the examination signal in the transient region, while the continuity condition is that the intensity of the synthesis signal in the transient region is less than a preset one of an intensity of a preceding or succeeding portion of the examination signal Threshold deviates. Preferably, the threshold is a relative threshold and is a value = 2.5, with values = 1.5 even being preferred. This means that the intensity of the signal in the transient region is at most 1.5 times or 0.66 times the intensity of a preceding non-transient portion or subsequent non-transient portion of the examination signal. This ensures that a transient suppression will not lead to a disturbing amplitude variation or intensity variation.

The threshold can also be realized by a confidence interval of 80% or less, which is determined based on the historical values.

Intensity measures that can be used for the present invention include the energy obtained by adding the sample squares or spectral value squares of a block, or a measure of performance that can be obtained considering the temporal block length, or a measure of weighting or unweighted adding up the magnitude of spectral values in a band, this particular measure, which is also an intensity, also being referred to as high frequency content if the band in which it is added is the upper frequency band of the examination signal, or generally higher frequencies versus lower Frequencies are more heavily weighted or have a greater impact on the end result.

The synthesis signal generator then generates a synthesis signal that is used by a signal substituter 14 to use the synthesis signal instead of the corresponding portion of the original examination signal to finally provide the ambient signal 10. The signal substitute 14 thus receives in addition to the synthesis signal via the line 13 and the examination signal via a line 15, as in Fig. 1 is indicated. The transient detector 11 receives the examination signal via an input line 16 and provides via an output line 17 a transient information to the synthesis signal generator 12, so that this generates the synthesis signal using the examination signal, which is provided to him via a line 18.

In particular embodiments of the present invention, non-overlapping block processing, as in FIG Fig. 2a represented or an overlapping block processing as in Fig. 2b shown used. In non-overlapping block processing in Fig. 2a an examination signal 21 is divided into preferably equal blocks of a specific block length. The transient detector then detects a transient 22 in the transient period 20. The transient 22 is thus in the transient period 20 of FIG Fig. 2a , which causes the transient detector 11 to provide an output signal via its output line 17, which tells the synthesis signal generator 12 that it now has to start with signal synthesis. While the blocks preceding and following the transient period 20 are immediately the corresponding parts of the surround signal 10 except for a transition in a crossfade area 23, the block of the exam signal corresponding to the transient period 20 is now synthesized by the synthesis signal generator and then substituted by the signal substituter 14 of the original block of the examination signal in the ambient signal.

In the preferred embodiments, as will be shown later, a processing of the block of the examination signal is performed, which takes place in the frequency domain. As a result, at a block boundary, the synthesis signal has a sample which may differ significantly from a sample which is the last sample of the preceding block in the examination signal. In order to eliminate such block boundary artifacts that may occur, it is used in the art Fig. 2a In the exemplary embodiment shown, it is preferred to superimpose a block before a transient period into the synthesis signal in the transient period, for example by adding the first sample of the generated synthesis signal to the eg last 10 samples of the previous block weighted according to the suppression function, for example according to FIG Fade in Fig. 3 , At the same time, the last sample of the previous block is still in accordance with the blanking in Fig. 3 the first and the first sample samples of the synthesized block, respectively, weighted according to the fade-in function are added in the transient period to provide a fade. Correspondingly, in the rear Crossfade area, so if the transient period back into the non-transient-afflicted block of the ambient signal, proceed.

To further reduce such block boundary artifacts, as shown in FIG Fig. 2b is shown, overlapping processing is preferred. The transient detector then detects at the in Fig. 2b shown embodiment block areas, which are shown with ringed numbers (1), (2), (3), (4), (5), (6). A transient is detected at 22. This causes it to be compared to Fig. 2a gives a larger transient period 20 since the transient at position 22 has been detected in both block 4 and block 5. Therefore, the synthesis signal generator 12 of Fig. 1 Generate synthesis signals for both block 4 and block 5. While for the blocks preceding the three transient period ranges A, B, C, the examination signal has no transients and thus is taken over directly into the surrounding signal, the regions A, B, C are signaled by the signal substituter 14 of FIG Fig. 1 The section A is generated by the addition of the second half of the non-transient-related block 3 of the examination signal with the first half of the synthesis signal generated for the block 4. The second part B of the transient period 20 is provided by adding the second half of the synthesis signal generated for block 4 to the first half of the synthesis signal generated for block 5 and from the signal substituter as a corresponding portion of the environmental signal 10 substituted. The third portion C of the transient period 20 is generated by adding the second half of the synthesizer signal generator generated block # 5 to the first half of the block # 6, which is no longer transient, and written into the ambient signal by the signal substituter 14.

In the Fig. 3 The skip function shown will be explained in more detail below. Thus, this skip function can be used to provide soft block transition from a non-synthesized block to a synthesized block in block processing with non-overlapping blocks, and also to provide a smooth transition from a synthesized block back to a non-synthesized block , Alternatively, a corresponding cross-fade function can also be used to overshadow back to the original examination signal, in particular when a synthesis signal has been generated by a specific specified number of blocks. Given the likelihood that the synthesis signal has moved relatively far from the probe due to the extrapolation, an abrupt return to the probe would, in some cases, result in audible artifacts. Therefore, it is preferred to slowly according to the fade-in / fade function of Fig. 3 For a block in which no transient has already been detected, a synthesis signal is generated, which consists of 90% of the last synthesized block and 10% of the current block of investigation. In the next block, the ratio could then be changed to 80%: 20%, until then after a certain number of blocks, the synthesis signal is completely hidden and the current non-transient-related examination signal is fully displayed again.

Hereinafter, a preferred implementation of a portion of the synthesis signal generator 12 based on Fig. 4 posed. For this purpose, the time signal, which represents a block of the examination signal, is converted into a frequency domain representation or a subband representation by a converter 40, which may comprise a transformation or an analysis filter bank. The spectral representation in the form of spectral coefficients or the subband signals are then optionally information, as shown at 41 replaced by an extrapolated spectral representation or extrapolated subband signals, if it is a block of the time signal in which a transient has been detected. The spectral representation is then optionally supplied, using additional information due to extrapolation, to a smoother 42, which influences the spectral values in such a way that the temporal course of the underlying signal is smoothed. In the case of a filter bank, the smoother 42 will affect the subband signals so that the timing of the signal underlying the subband signals is smoother than before smoothing. Then, in a block 43, an inverse conversion into the time domain takes place using either an inverse transformation or a synthesis filterbank to finally arrive at a timing signal 44 which is smoother than the timing signal at the input of the stage 40, however, has energy that has not been significantly affected by smoothing. Further, the smoothing has been done so that the energy of the smoothed time signal 44 does not differ from the energy of the previous time signal any more than the threshold.

Thus, in the present invention, an overall e-energy manipulation of the energy of the time signal may occur. However, only the transient is attenuated while the tonal components continue to be synthesized or past, by synthesizing the signal in the transient period by a prediction using a non-transient signal from the past.

However, if the energy - as in randomization or spectral prediction - is not touched, the smoothing has caused the energy over the block to be more evenly distributed, thus producing a smoother timing, but without losing the block's energy Sampling of the examination signal considerably to change. This is sufficient in most cases and ensures that the user hears a test signal that always satisfies the continuity condition. Only when the transient leads to a considerable increase in energy on the entire block, the smoothing alone, so the more even distribution of energy over the block, will not be sufficient and then a controlled signal limitation can be made.

Known methods, which are to avoid localization of direct sound sources in the back channels, are to delay the back channels a few milliseconds. This solution does not result in transient suppression, but attempts to "mask" the transients by using the precedence effect. The precedence effect is that the ear suspects a sound source where it first hears something from that sound source, and what you hear from that sound source can be louder or come from another direction. A disadvantage of this solution, however, is that very short sound events with sharp transients are often still audible and are then perceived twice, through a front loudspeaker and a few milliseconds later through the rear channels, which causes an unpleasant auditory impression.

Commercially available matrix decoders, such as Dolby Pro Logic II or Logic 7, have the ability to upmix non-preprocessed 2-channel stereo files in multichannel surround files, although they have not been designed immediately for this task. These matrix decoders are often incapable of rejecting transient tones in the back channels, resulting in a signal that does not meet the requirements for transient-free and amplitude-continuity.

In contrast, according to the invention, channel regions where transients occur are detected and attenuated. However, simply attenuating the entire signal at those time ranges would result in amplitude modulation of the environmental signal and would be perceived as an unpleasant or even artifact. This would therefore degrade the sense of quality of the extracted or processed environmental signal. In order to overcome this unpleasant amplitude modulation effect, according to the invention a transient suppression is produced without affecting the continuity of the synthesis signal or ambient signal. For this purpose, an input signal, e.g. a high-mix signal as obtained by a matrix up-mixer for the back channels, or a signal with similar characteristics and a similar range of application is analyzed to detect if a transient is present.

When a transient is detected, the currently processed block is replaced with a substitution signal having a flat (non-transient) time envelope. This substitution signal is either generated from previous signal portions in which no transient was present, or is generated from the currently processed block by a processing step that flattens the temporal envelope of the signal, or is generated by a combination of both methods.

The substitution signal generated by previous sections is generated, for example, by extrapolating previous energy levels of the signal or by copying / repeating previous signal sections without a transient portion of the signal.

For example, flattening of the fine time structure or the fine timing signal on the basis of the currently processed block may be performed be, as described below with reference to the Figures 5a, 5b or 5c is shown.

Thus, the absolute values of the spectral coefficients can be randomized within a limited range extending around the extrapolated spectral coefficients or amounts thereof, as will be described later Fig. 5c is pictured.

Alternatively or additionally, the phases or signs of the spectral coefficients of the processed block in which the transient is located may be randomized by a randomizer 50. For this purpose, a short-time spectrum of the considered block of the examination signal is generated, and the complex spectral values obtained there are calculated in magnitude and phase, in order then to randomize the phases of the spectral values. If a transformation is used, which can only resolve phases of +/- 180 °, which can thus provide spectral values with positive and negative signs only, then the signs can also be randomized to obtain a short-term spectrum with randomized phases / signs, which has a has a flatter time course of the corresponding time signal.

This approach is based on the fact that a rapid change in a time signal is only possible if the phases of the fundamental wave underlying this transient range and associated harmonics are in a very specific ratio. If a randomization of the phases is achieved, this leads to the fact that the transient range is smoothed, since the exact interaction of the phases of the individual sinusoidal oscillations, which are represented by the spectral values, no longer exists.

An alternative implementation is in Fig. 5b represented by a predictor 51, which is designed to perform a prediction of the short-term spectrum over the frequency. Such a predictor is in J. Herre, JD Johnston: "Exploiting Both Time and Frequency Structure in a System that Uses presented to Analysis / Synthesis Filterbank with High Frequency Resolution", 103rd AES Convention, New York 1997, Preprint 4519 ,

Again, a short-term spectrum is generated which has a transient course in its assigned time signal. Typically, using an open-loop predictor, a current spectral value of the short-term spectrum is predicted using a previous or a plurality of previous spectral values, and then the predicted spectral value could be subtracted from the actual spectral value to obtain a residual spectral value. While the residual spectral value of a typical prediction over frequency represents the value of interest and information-bearing information along with coefficients of a prediction filter, according to the invention a given prediction filter is preset and the spectral values of the short-term spectrum are replaced by the spectral values predicted using this prediction filter, while the prediction error signal is no longer used.

However, the actually erroneous predictive spectral values thus obtained now have a flatter time course than the original short-term spectrum, but still have approximately the same energy, so that both the transient condition and the continuity condition, as described in connection with the synthesis signal generator 12 of FIG Fig. 1 has been shown is satisfied. A preferred simple embodiment of the prediction filter is simply that a value of a spectral line lower in the index is used as the prediction value for a current spectral line.

In general, the extrapolated signal may be blended with the original signal after a specified period of time rather than hard switching, to avoid long-term extrapolation artefacts.

Furthermore, it is preferred, as it is based on Fig. 6 is shown to detect tonal components / bands by a detector 60 and not to be influenced by the synthesis signal generator, but to combine in a mixer / combiner 61 with transient band synthesis signals to then, after a transformation into the time domain, the could take place in block 61, to obtain a time signal with a shallower time course, which still includes the tonal bands, that is, portions that were not transient, in an unchanged form.

Thus, stationary / tonal frequency components in the input signal, which were present during the duration of the transient, for example, in only parts of the spectrum, are detected and a substitution signal is generated, which consists of an extrapolation of the past stationary / tonal signal components and the stationary / tonal signal detected in the current block. tonal frequency components.

The following is based on Fig. 5c an implementation of the present invention, which does not require an implicit and no longer explicit transient detector. For this purpose, means 53 for calculating the intensity of a block and a previous block in FIG Fig. 5c shown. A measure of the intensity of a processed signal block is, for example, the energy or radio frequency content (HFC) or other measure based on the spectral values, time samples, energy, power or other amplitude-related measure of the signal. It is then determined in a device 54 whether an intensity increases from one block to the next above a threshold. If so, the spectral values of the processed block are limited so that their intensity does not exceed the intensity of the previous signal block by more than the particular relative or exceeds absolute threshold such that at least the overall dominance of transients is reduced. This limitation takes place in a device 55 which is designed to, when a need for a limitation has been detected, ie when an implicit transient has been detected, either individually or globally limit spectral values. An individual limitation would be that an increase in energy is calculated for spectral values or for bands and the spectral values or the energy bands increase only up to a maximum energy increase and values exceeding this are cut off.

Thus, means 56 for limiting the spectral values individually or globally limits the spectral values, an individual limitation being that only the spectral values which rise above a threshold are limited and preferably limited to this threshold, while the other spectral values are not increase strongly, not be influenced. Alternatively, however, in certain cases it will be more convenient and simpler in terms of computational effort, then, if too much increase has been detected, all spectral values will be bounded by the same absolute or relative measure.

In this case, it is still preferable to post-process the limited spectral values by means 56 for post-processing, this post-processing being carried out in a randomization, as described in US Pat Fig. 5a has been described, or even in a prediction, as in Fig. 5b has been described can exist. The order of processing by means 55 and 56 may also be reversed, such that first a randomization or prediction processing is performed on a block for which a transient has been detected, only then an intensity limitation according to FIG the processing in block 55 is made.

With regard Fig. 5c It should also be noted that the block Z / F represents a time / frequency domain conversion 57, wherein a conversion from the time domain into the frequency domain may also be filtering by means of an analysis filter bank, such that in this case the spectral representation consists of subband signals and not individual spectral components.

wobei X(f) die Spektralkoeffizienten für bestimmte Frequenzen sind, und wobei w(f) Gewichtungsfaktoren für bestimmte Frequenzen sind.Hereinafter, a particularly preferred embodiment of the present invention with reference to Fig. 7 explained. The transient detector as indicated at 11 in Fig. 1 In this embodiment, means 89 for calculating the high-frequency content (HFC) for each block, which is followed by a means for calculating the long-term HFC-72. A comparator 73 then detects if there is a transient or if there is a transient period in which a transient exists. In particular, means 71 is configured to calculate the weighted radio frequency content (HFC) for each block of the original left signal and the original right signal. Alternatively, one HFC for each channel can be calculated separately. The HFC is the weighted sum of the absolute values of all the frequency lines in a block, with increasing weighting factors from lower to higher frequencies. The HFC is calculated as follows: $$\mathrm{HFC}=\mathrm{total}\left(\mathrm{X}\left(\mathrm{f}\right)\cdot \mathrm{w}\left(\mathrm{f}\right)\right).$$

where X (f) are the spectral coefficients for certain frequencies, and w (f) are weighting factors for certain frequencies.

As the weighting factors increase from lower to higher frequencies, it is ensured that in the HFC value the energy in the higher frequency components is weighted compared to the energy in the lower frequency components. An energy in higher spectral components is a better indication of a transient than energy in lower spectral components. In the implementation, all spectral components can be used to calculate the HFC. Alternatively, the calculation of the HFC can also be carried out starting from a limit which is approximately in the middle range of the spectrum, so that the low spectral coefficients play no role in the calculation of the HFC.

Further, a long term HFC average called HFC 'is calculated over at least three and preferably five preceding blocks. If it is then determined in the device 73 that the HFC in the current block deviates by a factor of more than a constant factor c from the long-term mean value HFC ', using a number> or = 1.0 as the constant factor c a transient detected. The threshold depends on the type of moving average. If the moving average is an average in which the past is weighted more heavily relative to the more recent block, ie a slower average, the threshold is closer to unity than in the case where the past is less heavily weighted into the moving average. Here the threshold would be further away from one.

When a transient is detected, as indicated to means 74 for calculating the mean by means 73, the average of the past absolute values of each frequency line (spectral coefficient) over a defined time interval, such as five blocks, is calculated. In addition, a prediction allowance interval Δ _max for the extrapolated absolute values is calculated. The extrapolated absolute values vary randomly within this interval Δ _max . To achieve this, a calculation is performed according to an equation as used in the device 75 in FIG Fig. 7 is shown. RN stands for random number, Δ _max represents the admissibility interval, SW is a spectral value as represented by the means 75 for calculating is calculated, and SW _m is the spectral value which results as an average of several past blocks as calculated by the block 74. The device 75 is therefore designed to evaluate the following equation: $$\mathrm{SW}={\mathrm{SW}}_{\mathrm{m}}+\mathrm{RN}\cdot {\mathrm{\Delta }}_{\mathrm{Max}}\mathrm{,}$$

To avoid repetitive effects that can occur when a detected transient is too long, the extrapolated values are blended in with the original values, when a fixed time interval has elapsed, such as three blocks of synthesis signals were present which then has to be returned to the original signal. However, if the transient period is shorter than three blocks, it is preferred not to perform the crossfading because then it is assumed that the extrapolated signals have not yet moved so much from the original signals. The fading can take place either before conversion into the time domain or, preferably, after conversion into the time domain, as at 76 in FIG Fig. 7 is shown to obtain the synthesis signal.

In one implementation, the inventive concept may be integrated into an environment signal extraction process or used as a separate post-processing step using an existing environmental signal, but still includes undesirable transients prior to processing in accordance with the invention.

The processing steps according to the invention can be carried out in the frequency domain per frequency line or in subbands. However, they can also be made only partially in the frequency range typically above a certain frequency limit, or else in the time domain be performed exclusively or in a combination of time and frequency domain.

Fig. 8 FIG. 12 shows a preferred embodiment of the present invention in which the ambient signal generation device is not only designed to generate ambient signals for a left surround channel output 80 and a right surround channel output 81. The device according to the invention further comprises in addition a high mixer 82 for generating signals for the left channel L, the right channel R, the center channel C and preferably also for the LFE channel, as shown in FIG Fig. 8 is shown. Both the combination of transient detector 12, synthesis generator 14 and signal substitute 16, ie the high mixer 82, are fed by a decoder 84. The decoder 84 is configured to receive and process a bit stream 85 to provide a mono signal or a stereo signal 86 on the output side. The bit stream may be an MP3 bit stream or an MP3 file, or it may be an AAC file, or may also be a representation of a parametrically encoded multi-channel signal. For example, the bitstream 85 could be a left channel, right channel, and center channel parametric representation, including one transmission channel and multiple cues for the second and third channels, this processing being known from BCC multi-channel processing. Then, the decoder 84 would be a BCC decoder which not only provides a mono or a stereo signal, but even provides a 3-channel signal, but which still does not include data on the two surround channels Ls, Rs. In one implementation, the examination signal will thus be a mono signal, a stereo signal or even a multi-channel signal in this case, but does not include separate loudspeaker signals for the surround channels Ls, Rs.

It should be noted that you can either use the same ambience signal for both surround channels or for each surround channel can calculate its own signal. In the first case, for. B. the examination signal or the ambient signal derived from a sum of left and right channel. In the other case z. For example, from the left channel, the surrounding signal for the left surround channel is calculated, and from the right channel, the surround signal for the right channel is calculated.

One aspect of the present invention relates to a device for generating an ambient signal which is suitable for transmission via loudspeakers Ls, Rs, for which no suitable loudspeaker signal is present, having the following features: a transient detector 11 for detecting a transient period 20, an examination signal having a transient region 22; a synthesis signal generator 12 for generating a synthesis signal for the transient period 20, wherein the synthesis signal generator 12 is adapted to generate a synthesis signal having a shallower time course than the examination signal in the transient period 20, and its intensity of an intensity of a preceding or succeeding section the examination signal deviates by less than a predetermined threshold; and a signal substitutor 14 for substituting the analysis signal in the transient period by the synthesis signal to obtain the environmental signal.

In one embodiment, the apparatus is configured for block processing to process successive blocks of discrete-time samples in an overlapping or non-overlapping manner.

In one embodiment, the transient detector 11 is configured to calculate intensity values for successive blocks and to detect a transient period 20 when an intensity value of a block from a previous or subsequent one Intensity value is different by more than a predetermined transient threshold.

In one embodiment, the synthesis signal generator 12 is configured to limit, for a block in the transient period 20, a plurality of spectral values representing a short-term spectrum of the block, such that their intensity is less than that of the preceding or following block or transient predetermined threshold is different.

In one embodiment, the synthesis signal generator 12 is configured to randomize complex spectral values representing a short-term spectrum of the block comprising the transient period 20 with respect to their phase or their sign.

In one embodiment, the synthesis signal generator 12 is configured to perform prediction processing 51 over frequency to obtain a prediction spectrum whose associated time signal has a flatter timing than a time signal associated with a spectrum prior to prediction processing versus frequency.

According to the invention, the transient detector 11 is configured to calculate a high-frequency content for a block of the examination signal 61, wherein the transient detector 11 is designed to be the weighted HF content with a moving average over a plurality of preceding or following Compare blocks without transient 73,
wherein the transient detector 11 is adapted to detect a transient for a block when the RF content of a current block exceeds the moving average by more than a threshold c.

In one embodiment, the transient detector is configured to use a threshold that is selected depending on the manner of calculating the moving average and closer to unity when the moving average is more in the past and farther from one, when the past comparatively weakens in the moving average.

According to the invention, the synthesis signal generator is designed to calculate an average for each spectral value of a short-term spectrum of a plurality of blocks using corresponding spectral values of the plurality of blocks 74 to obtain an average spectrum for calculating deviations for spectral values that differ for spectral values and less than a maximum deviation Δ _max , and to add the deviations and the mean spectral values to obtain a processed spectrum.

In one embodiment, the synthesis signal generator 12 is configured to calculate the synthesis signal from signal portions of the examination signal before or after the transient period, from the examination signal in the transient period after smoothing the time course thereof or from a combination of the signal portions of the examination signal and the examination signal after a smoothing.

In one embodiment, the synthesis signal generator 12 is configured to copy signal portions of the examination signal before or after the transient period.

In one embodiment, the synthesis signal generator 12 is configured to randomize extrapolated spectral values derived from the examination signal outside the transient period in a predetermined range.

In one embodiment, the synthesis signal generator 12 is configured to, when the transient period lasts longer than a predetermined time, mix synthesis signal values with signal values of the examination signal for times later than the predetermined period.

In one embodiment, the signal substitute 14 is configured to fade into a transient period from a portion prior to the transient period according to a fade function, or to fade from the transient period into a portion after the transient period according to a fade function.

In one embodiment, the synthesis signal generator 12 is configured to compute 40, 41, 42 a short-term spectrum of the synthesis signal to convert the short-term spectrum into a time representation 43 representing the synthesis signal 44.

In one embodiment, the synthesis signal generator 12 is configured to compute a short-term spectrum of the synthesis signal with subband signals 40, 41, 42, and to convert the short-term spectrum with subband signals into a timing representative of the synthesis signal 43.

In one embodiment, the synthesis signal generator 12 is configured to generate the synthesis signal such that the predetermined threshold is less than or equal to a factor of two.

In one embodiment, the synthesis signal generator 12 is configured to use a band-selective preset threshold or a single threshold for the entire spectrum.

In one embodiment, the apparatus further comprises extraction means for processing a left channel signal and a right channel signal to extract the examination signal.

In one embodiment, the apparatus comprises a 2-to-3 mixer 82 for generating a left channel, a right channel, and a center channel from a transmitted stereo or mono signal, wherein the synthesis signal generator 12 is configured to be the same To provide surround signal for the rear left or rear right channel, or to scale the examination signal so that the rear left channel and the rear right channel can receive different scaled version of the surrounding signal, or to calculate two own surround signals for two surround channels ,

The present invention further relates to a method for generating an ambient signal suitable for transmission via loudspeakers Ls, Rs for which no suitable loudspeaker signal is present, comprising the following steps: Detecting 11 a transient period 20 in which an examination signal has a transient region 22 ; Generating 12 a synthesis signal for the transient period 20, wherein the synthesis signal generator 12 is adapted to generate a synthesis signal having a shallower time course than the examination signal in the transient period 20, and the intensity of an intensity of a preceding or succeeding portion of the examination signal deviates less than a predetermined threshold; and substituting 14 the examination signal in the transient period 20 by the synthesis signal to obtain the surrounding signal.

Depending on the circumstances, the method according to the invention can be implemented in hardware or in software. The implementation may be on a digital storage medium, in particular a floppy disk or CD with electronic readable control signals that can interact with a programmable computer system such that the method is executed. In general, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer. In other words, the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Claims (10)

1. A device for generating an ambience signal suitable for being emitted via loudspeakers (Ls, Rs) for which there is no suitable loudspeaker signal, comprising:

   a transient detector (11) for detecting a transient (22) in a block of an examination signal to obtain a transient period (20) comprising the block of the examination signal;

   a synthesis signal generator (12) for generating a synthesis signal for the transient period (20), the synthesis signal generator (12) being implemented to generate a synthesis signal which comprises flatter a temporal course than the examination signal in the transient period (20) and the intensity of which deviates from an intensity of a portion of the examination signal preceding the transient period (20) or subsequent to the transient period (20) by less than a predetermined threshold; and

   a signal substituter (14) for substituting the examination signal in the transient period by the synthesis signal to obtain the ambience signal,

   wherein the transient detector (11) is implemented to calculate weighted high-frequency contents for the block of the examination signal (71), wherein the weighted high-frequency contents are a weighted sum of absolute values of all frequency lines in the block with increasing weighting factors from lower to higher frequencies;
   wherein the transient detector (11) is implemented to compare (73) the weighted high-frequency contents of the block to a floating average value (72) over weighted high-frequency contents of a plurality of preceding or subsequent blocks of the block without any transients,
   wherein the transient detector (11) is implemented to detect a transient for the block when the weighted high-frequency contents of the block exceed the floating average value by more than a threshold (c),
   wherein the synthesis signal generator is implemented to calculate (74), when generating the synthesis signal for every spectral value of a short-term spectrum of a plurality of blocks, which form the portion of the examination signal preceding the transient period (20) or subsequent to the transient period (20), an average value using corresponding spectral values of the plurality of blocks to obtain an average value spectrum for the block,
   to calculate, for spectral values of the average value spectrum for the block, deviations differing for the spectral values of the average value spectrum for the block and being smaller than a maximum deviation (Δ_max), and
   to add (75) the deviations and the average value spectral values to obtain the spectral values for the block, and
   to convert (43, 76) the spectral values for the block into a temporal representation that represents the synthesis signal (44).
2. The device according to claim 1, which is implemented for block processing in order to divide the examination signal into successive and overlapping or nonoverlapping blocks of time-discrete sample values.
3. The device according to claim 1, wherein the threshold (c) is selected depending on the type of calculation of the floating average value and is closer to one when the history has stronger an influence on the floating average value, and is further from one when the history has a comparatively smaller influence on the floating average value.
4. The device according to one of the preceding claims,
   wherein the synthesis signal generator (12) is implemented to calculate the synthesis signal from signal portions of the examination signals before or after the transient period, from the examination signal in the transient period after smoothing the temporal course thereof or from a combination of the signal portions of the examination signal and the examination signal after smoothing.
5. The device according to one of the preceding claims,
   wherein the synthesis signal generator (12) is implemented to generate the synthesis signal such that the predetermined threshold is smaller than or equal to a factor of 2.
6. The device according to one of the preceding claims,
   wherein the predetermined threshold is a band-selective preset threshold or a single threshold for the entire spectrum.
7. The device according to one of the preceding claims, further comprising:

   extracting means for processing a left channel signal and a right channel signal to extract the examination signal.
8. The device according to one of the preceding claims, further comprising:

   a two-to-three mixer (82) for generating a left channel, a right channel and a center channel from a stereo signal transmitted; and

   wherein the synthesis signal generator (12) is implemented to provide the ambience signal for a back left channel (Ls) and the ambience signal for a back right channel (Rs), wherein the back left channel and the back right channel are the channels for which there is no suitable loudspeaker signal, or

   wherein the synthesis signal generator (12) is implemented to scale the examination signal so that differently scaled versions of the ambience signal are obtained for the back left channel and the back right channel, or
   wherein the synthesis signal generator (12) is implemented to calculate a first ambience signal for the back left channel and to calculate a second ambience signal differing from the first ambience signal for the back right channel.
9. A method for generating an ambience signal suitable for being emitted via loudspeakers (Ls, Rs) for which there is no suitable loudspeaker signal, comprising the steps of:

   detecting (11) a transient (22) in a block of an examination signal to obtain a transient period (20) comprising the block of the examination signal;

   generating (12) a synthesis signal for the transient period (20), the synthesis signal generator (12) being implemented to generate a synthesis signal which comprises flatter a temporal course than the examination signal in the transient period (20) and the intensity of which deviates from an intensity of a portion of the examination signal preceding the transient period (20) or subsequent to the transient period (20) by less than a predetermined threshold; and

   substituting (14) the examination signal in the transient period (20) by the synthesis signal to obtain the ambience signal,

   wherein in the step of detecting (11) weighted high-frequency contents for the block of the examination signal are calculated (71), wherein the weighted high-frequency contents are a weighted sum of absolute values of all frequency lines in the block with increasing weighting factors from lower to higher frequencies;
   wherein in the step of detecting (11) the weighted high-frequency contents of the block are compared (73) to a floating average value (72) over weighted high-frequency contents of a plurality of preceding or subsequent blocks of the block without any transients,
   wherein in the step of detecting (11) a transient is detected for the block when the weighted high-frequency contents of the block exceed the floating average value by more than a threshold (c),
   wherein in the step of generating (12) when generating the synthesis signal for every spectral value of a short-term spectrum of a plurality of blocks, which form the portion of the examination signal preceding the transient period (20) or subsequent to the transient period (20), an average value is calculated (74) using corresponding spectral values of the plurality of blocks to obtain an average value spectrum for the block,
   wherein in the step of generating (12) for spectral values of the average value spectrum for the block, deviations differing for spectral values of the average value spectrum for the block and being smaller than a maximum deviation (Δ_max) are calculated, and
   wherein in the step of generating (12) the deviations and the average values spectral values are added (75) to obtain the spectral values for the block, and
   wherein in the step of generating (12) the spectral values for the block are converted (43, 76) into a temporal representation that represents the synthesis signal.
10. A computer program for performing a method according to claim 9, when the method runs on a computer.

Images