EP1878308B1 - Device and method for generation and processing of sound effects in spatial audio reproduction systems using a graphical user interface - Google Patents

Abstract

The present invention is based on the finding that a sound-reproduction system, which can generate a spatial sound impression in a reproduction environment, can be efficiently and intuitively controlled by means of a graphic user interface when an impulse response associated with a spatial direction with respect to the reproduction environment is graphically represented, and when the possibility is created for a user to change the impulse response graphically so that, based on the user input of a change, the changed impulse response can be graphically represented and the changed graphical representation can be detected, in order to control the sound-reproduction system. Since it is system-theoretically possible to describe all known linear signal processing operations by impulse responses, it is possible, with the inventive graphic user interface, to provide a sound mixer, through a graphical representation, with an intuitive access to sound effects depending on the direction.

Description

The present invention relates to modern audio technologies, and more particularly to the generation and processing of spatial sound impressions for sound reproduction systems.

In modern sound reproduction systems, the use of multiple loudspeakers makes it possible to precisely locate individual sound sources in the room and to give the impression within the playback environment that one would be within a simulated space, such as a sound space. A stage or a cathedral. In principle, two different playback concepts can be distinguished. In conventional surround sound reproduction, which is also customary in the field of home entertainment, the location and room information is already mixed during the sound mixing process into individual channels to be transmitted discretely, a reproduction system comprising several loudspeakers being used to reproduce the individual channels. In this case, the reproducing speakers should be in a predetermined position relative to the playback environment in order to achieve the best possible spatial impression.

More advanced systems, such as the wave-field synthesis-based space simulations, generate the drive signals for the individual speakers only during the reproduction, based on position information of a sound source with respect to the reproduction space and the space information of a reproduction environment to be simulated. As a result, much more authentic results can be achieved with respect to the location and the spatial impression, since the individual loudspeaker setup is taken into account during playback can be used to create a wavefront in the playback environment that best represents the space impression to be simulated.

For a better understanding of the present invention, the wave field synthesis technique will be discussed in more detail below.

A better natural spatial impression as well as a stronger envelope in the audio reproduction can be achieved with the help of a new technology. The basics of this technology, the so-called wave field synthesis (WFS), were researched at the TU Delft and first presented in the late 80s ( Berkhout, AJ; de Vries, D .; Vogel, P .: Acoustic control by wave-field synthesis. JASA 93, 1993 ).

Due to the enormous demands of this method on computer performance and transmission rates, wave field synthesis has rarely been used in practice. Only the advances in the areas of microprocessor technology and audio coding allow today the use of this technology in concrete applications.

The basic idea of WFS is based on the application of Huygens' principle of wave theory:

Every point, which is detected by a wave, is the starting point of an elementary wave, which spreads in a spherical or circular manner.

Applied to the acoustics can be simulated by a large number of speakers, which are arranged side by side (a so-called speaker array), any shape of an incoming wavefront. In the simplest case, a single point source to be played and a linear array of speakers, the audio signals of each speaker must be delayed and Amplitude scaling are fed so that the radiated sound fields of each speaker properly overlap. With multiple sound sources, the contribution to each speaker is calculated separately for each source and the resulting signals added together. If the sources to be reproduced are in a virtual room with reflective walls, reflections must also be reproduced as additional sources via the loudspeaker array. The effort involved in the calculation therefore depends heavily on the number of sound sources, the reflection characteristics of the room and the number of loudspeakers.

The advantage of this technique is in particular that a natural spatial sound impression over a large area of the playback room is possible. In contrast to the known techniques, the direction and distance of sound sources are reproduced very accurately. To a limited extent, virtual sound sources can even be positioned between the real speaker array and the listener.

The wave field synthesis thus allows a correct mapping of virtual sound sources over a large playback area. At the same time it offers the sound engineer and sound engineer new technical and creative potential in the creation of even complex soundscapes. Wavefield synthesis (WFS or sound field synthesis), as developed at the TU Delft in the late 1980s, represents a holographic approach to sound reproduction. The basis for this is the Kirchhoff-Helmholtz integral. This states that any sound fields within a closed volume can be generated by means of a distribution of monopole and dipole sound sources (loudspeaker arrays) on the surface of this volume. Details can be found in MM Boone, ENG Verheijen, PF. Tol, "Spatial Sound-Field Reproduction by Wave-Field Synthesis", Delft University of Technology Laboratory of Seismics and Acoustics, Journal of J. Audio Eng. Soc., Vol. 43, No. 12, December 1995 and Diemer de Vries, "Sound Reinforcement by Wavefield Synthesis: Adaptation of the Synthesis Operator to the Loudspeaker Directivity Characteristics ", Delft University of Technology Laboratory of Seismics and Acoustics, Journal of J. Audio Eng. Soc., Vol. 44, No. 12, December 1996 ,

In the wave field synthesis, a synthesis signal for each loudspeaker of the loudspeaker array is calculated from an audio signal which emits a virtual source at a virtual position, the synthesis signals being designed in amplitude and phase such that a wave resulting from the superposition of the individual the sound wave present in the loudspeaker array will correspond to the wave that would result from the virtual source at the virtual position if that virtual source at the virtual position were a real source with a real position.

Typically, multiple virtual sources exist at different virtual locations. The computation of the synthesis signals is performed for each virtual source at each virtual location, typically resulting in one virtual source in multiple speaker synthesis signals. Seen from a loudspeaker, this loudspeaker thus receives several synthesis signals, which go back to different virtual sources. A superimposition of these sources, which is possible due to the linear superposition principle, then gives the reproduced signal actually emitted by the speaker.

The possibilities of wave field synthesis can be better exploited, the more closed the loudspeaker arrays are, ie the more individual loudspeakers can be arranged as close to each other as possible. However, this also increases the computing power which a wave field synthesis unit has to accomplish, since channel information also typically has to be taken into account. This means in more detail that from each virtual source to each speaker in principle a separate transmission channel is present, and that in principle there may be the case that each virtual source leads to a synthesis signal for each speaker, or that each speaker receives a number of synthesis signals , which is equal to the number of virtual sources.

In addition, it should be noted at this point that the quality of the audio playback increases with the number of speakers provided. This means that the audio reproduction quality becomes better and more realistic as more loudspeakers are present in the loudspeaker array (s).

Spatial sound reproduction systems such as wave field synthesis make it possible to generate the sound in 360 degrees around the audience room with optimal spatial resolution. So far, these systems have been used essentially for positioning discrete sound sources and for direct sound reproduction. In addition, all known linear signal processing can be applied to the signals of the sound sources generated in this way, such. B adding reverberation. In spatial sound reproduction systems such as Wave Field Synthesis (WFS), it is still possible to generate spatial effects based on the direct sound. This happens, for example, in the room simulation, in which, for reasons of efficiency, the reproduction in a limited number of spatial directions (plane waves) can be simplified.

In a very simple case of room simulation, the same parameters are used to describe the room for all spatial directions (diffuse reverberation) and automatically generate direction-dependent spatial components (early reflections). Generating spatial effects is not only useful when it comes to the reproduction of natural space effects, since the principal possibilities of this Type of signal processing can also be used creatively elsewhere.

In wave field synthesis, a room to be sounded is irradiated by as many individual loudspeakers as possible in order to allow the reconstruction of wavefronts with the best possible accuracy. For the location of sound signals and the generation of a spatial impression usually a variety of parameters are used, which are to be determined individually for each speaker during the mixing of the sound signal.

As described above, the multichannel sound reproduction systems are characterized by extremely high complexity, so that the additional generation of spatial information or the location information during the mixing of the sound conditional to generate a variety of parameters, the speaker individually the location information or additional linear Describe signal processing steps (for generating acoustic effects). This description based on a large number of abstract mathematical parameters without directly intuitively meaning is difficult to control, especially in wave field synthesis systems.

For example, wave field synthesis offers the possibility of freely positioning sound sources on a two-dimensional listening plane. This is done by synthesizing different wavefronts depending on the position of the sound sources. User interfaces, as currently used, use a point in a plan view of the two-dimensional listening plane for positioning the sound source, the point representing the position of the sound source. Although the spatial position of the sound source is adequately visualized in this approach, the sound impression of depth (spatial impression) can not be represented simultaneously in the visualization, so there are discrepancies between the real world Perception and presentation, so that only in a few exceptional cases, a visual image is provided, which corresponds to the real sound impression or a conclusion on this allows.

For the audio authoring system Nuendo Steinberg exists a plug-in, in which temporal echo traces for a signal are displayed graphically when echo parameters of the echo waveform are entered as numerical values. The E-choverlauf can be partially influenced individually for three different frequency ranges.

The international patent application WO 2004/047485 A1 describes an audio reproduction system based on wave field synthesis and deals in particular with a possible modularization of the wave field synthesis system in order to adapt it successively to changing reproduction environments.

The authors Theile G., Wittek H., Reisinger M. describe in "Potential Wavefield Synthesis Applications in the Multichannel Stereophonic World", Processings of the AES 24th International Conference: Multichannel Audio, 26-28 June 2003 , different methods of reproducing audio signals while maintaining the spatial hearing impression. In particular, the wave field synthesis method and the parameters underlying this method will be described.

The object of the present invention is to provide a graphical user interface that allows to more efficiently control a sound reproduction system for generating a spatial sound impression.

This object is achieved by a device according to claim 1 or 20 and by a method according to claim 25 or 26.

The present invention is based on the finding that a sound reproduction system which can produce a spatial tone impression in a reproduction environment can be controlled efficiently and intuitively by means of a graphical user interface if an impulse response assigned to a spatial direction with respect to the reproduction environment or a graphic image obtained from the impulse response Representation is graphed, and when the possibility is created that a user can graphically change this representation, so based on the user change input, the changed impulse response can be graphed and the changed graphical representation can be detected to control the sound reproduction system.Da systemically possible is to describe all known linear signal processing by impulse responses, it is possible with the graphical user interface according to the invention, a sound designer via a graphic R to provide intuitive access to directional sound effects, thus increasing the efficiency and quality of control of the sound reproduction system.

By folding an original signal with impulse responses, all linear signal processing algorithms can be represented. By way of example, in a space simulation based on wave field synthesis, the signals for plane waves can be generated by convolution with corresponding spatial impulse responses associated with corresponding spatial directions. Thus, rooms can also be simulated, wherein the impulse responses used according to the invention in addition to a description by the underlying parameters are also visualized directly. The inventive new tool for sound design consists of a simultaneous visualization of all direction-dependent impulse responses corresponding to a source. The sound design takes place through direct interaction with this visualization. The processing of the visual representation is converted into a parametric description and from this the corresponding impulse responses are generated.

The direction information or a spatiality is thus impressed on a sound signal by a mathematical convolution with an impulse response, which will be briefly explained below for a better understanding of the inventive concept.

A spatial signal or reflection pattern or location information by convolution with an impulse response g (x) is impressed on a tone signal f (y), so that the combined tone signal F (x) results according to the following folding integral: $$\mathrm{F}\left(\mathrm{x}\right)\mathrm{=}\underset{\mathrm{-}\mathrm{\infty }}{\overset{\mathrm{\infty }}{\mathrm{\int }}}\mathrm{f}\left(\mathrm{y}\right)\mathrm{G}\left(\mathrm{x}\mathrm{-}\mathrm{y}\right)\mathrm{dy}\mathrm{,}$$

The impulse response g (x) generally describes the response of a system to a dirac pulse δ ( x ), that is, a pulse of infinitesimal length for which: $$\underset{\mathrm{-}\mathrm{\infty }}{\overset{\mathrm{\infty }}{\mathrm{\int }}}\delta \left(x\right)\mathit{dx}=A\mathrm{,}$$

This means that an ideal Dirac impulse is characterized by an infinitesimal length and additionally by the fact that its integral, as described above, is finite. In the case of a sound signal, this means that a Diracpuls is arbitrarily small, but carries fixed acoustic energy.

If one tests a room with a Dirac pulse, the simplest impulse response is again a Dirac pulse, which is registered with a propagation delay t for transmitting the test pulse at the place of emission of the test pulse. This is exactly the case if, in the direction in which the test pulse was emitted, there is an ideal reflector which reflects the acoustic test signal without attenuation, the transit time between the location of the emission of the source and the reflector being then exactly t / 2.

It should be noted here that in reality it is impossible to generate an ideal Diracpulse, instead, from now on also pulses whose width is finite and whose intensity is A are called Diracpulse.

Illustratively, such real pulses can be imagined, for example, from Gaussian curves of small width with area A.

If the reflector described above would absorb some of the acoustic energy, thus dampening the test signal, then the reflected Dirac pulse received after runtime t would have a smaller area B under the curve than the original pulse (B <A).

In addition to the so far described, idealized simple cases of an impulse response, it is also possible, arbitrary to get complex impulse responses. If, for example, two reflectors in different distances corresponding to the acoustic propagation times t ₁ and t ₂ are at the location of the emission of the test signal, the impulse response will consist of two dirac pulses received at the times 2 * t ₁ and 2 * t ₂ . Usually, acoustic scenes are very complex, so that a real impulse response will become a denser pulse sequence beginning with early reflections, and whose components arriving later in time, for example, describe a reverberation.

As explained above, an impulse response in the form of a Dirac pulse describes a delay or an echo. Likewise, for example, a multiple echo can be represented by a sum of dirac-shaped pulses. For realistic space simulation, in general, the impulse response that is convolved with the sound signal will be continuous, e.g. B. a time t ₀ rising sharply and then gently decaying signal that describes a multiple reflection, the signals reflected at later times are more attenuated.

In real scenarios, sound signals are additionally attenuated in a frequency-selective manner; for example, high sound signals from carpets and wall hangings are more strongly damped than deep sound signals. In order to cope with this situation, different impulse responses, for example, can be used and visualized separately for several frequency ranges or the visualization of the impulse response must include the time and frequency range.

In another embodiment of the present invention, the graphical user interface is used to represent the spatial position of a sound source relative to the sound reproduction system, and the resulting impulse responses, which individually for each loudspeaker of a display system, the spatial orientation represent the sound signal with respect to the reproducing speaker visualize.

In this case, the user can graphically change the position of the source with respect to the reproduction environment, whereby the loudspeaker-individual impulse response or the parameters for controlling the loudspeakers result automatically from the illustrated wavefront of the punctiform acoustic signal source. A sound engineer thus has the opportunity to intuitively create the complex parameters needed to control the sound reproduction system.

An essential aspect here is that the possibility is additionally created of directly changing the impulse responses by graphical interaction with the user interface, wherein it is immediately shown how the current change affects the perception of the position of the sound source. With the graphic user interface according to the invention, it is thus advantageously possible to choose whether one wants to place the sound source directly from physical reality or whether one wishes to creatively use the possibilities of changing the impulse response. In the latter case, one additionally obtains an estimate of how the manual change of the impulse responses in the perception of a listener is interpreted. A sound engineer can therefore choose between two possibilities of visual sound processing and pursue the approach that is most advantageous for the desired sound result or the spatial sound impression that is to be achieved.

In a further embodiment of the present invention, the graphical user interface according to the invention is used to display impulse responses that contain information about a room to be simulated. In this case, the display device, the impulse responses with respect to a fixed point within the playback environment represented in the spatial directions for which they also carry the spatial information.

From the display device so all relevant for the overall spatial impression data (impulse responses) are simultaneously displayed, where they are visualized as a three-dimensional image of the environment. A user thus has the advantage that he receives all the information concerning the spatial sound impression at the same time or that he can change it simultaneously, whereby at any time the changed spatial sound impression resulting from a change is displayed and can be assessed.

This makes it possible to achieve a sound impression with reverberation or desired attenuation and other signal manipulations in an intuitive manner, without having to manually change the parameters underlying the impulse responses, which requires a considerable amount of abstraction.

The graphical representation also makes it possible to carry out the design process detached from technical conditions. Thus, in general, an impulse response function will be stored discreetly, i. H. for discrete periods, there is an associated amplitude value. The intuitive operation of the graphical user interface does not need to take this into account as the relevant parameters are automatically generated based on a graphical change in the displayed impulse response.

Another advantage is that the complexity of a system can easily be increased without reducing the intuitiveness of the operation under the increased number of parameters.

In a further embodiment of the present invention, the impulse responses are allowed to be multiple Spatial directions regarding frequency selective represent or edit. This makes it possible to further increase the naturalness of the spatial impression, for example by assuming different frequency-dependent attenuation profiles for different spatial directions, which on the one hand increases the authenticity of the sound impression achieved, but on the other hand also increases the complexity of generating the parameters. In the visual representation, it is still possible to predict the achievable sound experience, and to change this creatively, for example, by a strong artificial damping is introduced at a certain frequency for a freely selectable spatial direction. These changes are immediately visible and it is possible to reliably predict the impact on the entire sound scene in the context of the overall system.

In a simple example, the same parameters can be used to describe the room for all spatial directions, which corresponds to a diffuse reverberation. Direction-dependent spatial portions (early reflections) are only then attached. This results in a specific room impulse response for each spatial direction, an unwanted deviation of the parameters for a spatial direction can be detected and corrected immediately.

A further advantage of the three-dimensional representation according to the invention is that the frequency-selective impulse response representation for each direction can easily be converted into a matrix representation by simple scanning, the further processing of which is extremely efficient.

In a further embodiment of the present invention, individual delay times are set for a given number of spatial directions, the delay times being represented as dirac-shaped impulse responses. These are relative to a fixed point in the Playback environment are shown in a three-dimensional view. It is particularly advantageous that the graphical manipulation, which allows the shifting of the dirac-shaped impulse responses with respect to a reference point, directly reflects the spatial effect visually. The dirac-shaped impulse responses corresponding to a delay just describe a reflection on an object, wherein increasing the distance of the impulse response with respect to the reference point in the graph corresponds to increasing the propagation time of the reflected signal. The direct correspondence of the graphical representation to the simulated reality can thus be used to simulate in a most efficient manner, for example, spaces within which the reproduction environment is located.

A particular advantage of this simplified type of interior design is the high degree of intuitiveness of the presentation and the associated reduced probability of errors in the control of a sound reproduction system.

In another embodiment of the present invention, the graphical user interface for a sound reproduction system is operated with a signal generator which generates loudspeaker signals for a plurality of loudspeakers mounted at different spatial positions. The high level of user-friendliness and user-friendliness of the graphical user interface makes it possible to manipulate the reproduction of signal sources in real time in such a way that the acoustic location of a sound signal, for example a singer on the stage, matches the visual impression. In this case, only a tracking of the moving sound source within the graphical user interface according to the invention is necessary, which would not be feasible by means of classical parameter input for a loudspeaker system to be controlled.

Fig. 1

ein Blockschaltbild zur Erläuterung der Funkti- onsweise der graphischen Benutzerschnittstelle;

Fig. 2

Blockschaltbild zum Festlegen und Bearbeiten der Position von Schallquellen;

Fig. 3a

ein Beispiel für eine graphische Benutzerschnitt- stelle zum Bearbeiten von Impulsantworten der Pa- rameter, die den Ort einer Schallquelle beschrei- ben;

Fig. 3b

ein weiteres Beispiel für eine graphische Benut- zerschnittstelle

Fig. 4

Hinzufügen eines räumlichen Klangeindrucks für zu einer Tonquelle;

Fig. 5

Hinzufügen eines räumlichen Klangeindrucks zu einzelnen Lautsprechersignalen;

Fig. 6

eine graphische Benutzerschnittstelle zum Anzei- gen und Ändern von Impulsantworten;

Fig. 7

eine graphische Benutzerschnittstelle zum Anzei- gen und Verändern von frequenzselektiven Impuls- antworten;

Fig. 8

eine graphische Benutzerschnittstelle zum Anzei- gen und Ändern von Zeitverzögerungen für ver- schiedene Raumrichtungen; und

Fig. 9

ein System zum Ansteuern eines Tonwiedergabesys- tems mit einer graphischen Benutzeroberfläche.

Preferred embodiments of the present invention will be explained in more detail below with reference to the accompanying drawings. Show it:

Fig. 1

a block diagram for explaining the mode of operation of the graphical user interface;

Fig. 2

Block diagram for setting and editing the position of sound sources;

Fig. 3a

an example of a graphical user interface for processing impulse responses of the parameters describing the location of a sound source;

Fig. 3b

another example of a graphical user interface

Fig. 4

Adding a spatial sound impression to a sound source;

Fig. 5

Adding a spatial sound impression to individual loudspeaker signals;

Fig. 6

a graphical user interface for displaying and changing impulse responses;

Fig. 7

a graphical user interface for displaying and modifying frequency-selective impulse responses;

Fig. 8

a graphical user interface for displaying and changing time delays for different spatial directions; and

Fig. 9

a system for driving a sound reproduction system with a graphical user interface.

Fig. 1 FIG. 2 is a block diagram illustrating the operation of a graphical user interface 10 of the present invention having a display means 12 for displaying an impulse response, means for permitting a change in the graphic display 14, means for receiving a user change input 16, and means for detecting the changed impulse response 18 ,

The display device 12 graphically presents the impulse responses to the user such that the effects of changing the impulse responses presented can be intuitively interpreted and predicted.

The device for enabling the change of the graphic display 14 has access to the display device 12 and the data visualized by it.

In order to allow for a change in the impulse responses, user input is required, which is received by the device for receiving a user change input 16, which change may be, for example, by means of a computer mouse, a touch pad, or interaction and visualization techniques from virtual reality systems.

Based on the user change input, the display device 12 can now graphically display a changed impulse response.

Through the interaction of the display device 12, the means 14 for allowing a change 14 and the means for receiving a user change input 16, an iterative change method of user input and subsequent graphical update becomes possible. This has the great advantage that the effect of a user change can be controlled directly graphically or acoustically. Explicitly performing the Changes and subsequent control by test listening within a sound reproduction system can thereby be eliminated, which contributes significantly to the cost and time savings.

From the means for detecting the changed impulse response 18, the modified impulse response is detected and stored for further use, for example. The possibility of storing the impulse response can advantageously be used to reuse an already generated impulse response, which describes a specific space to be simulated, for further projects.

• Wellenfeldsynthese Punktquelle
• Impulsantwort-Zeit Darstellung
• Impulsantwort-Zeit-Frequenz Darstellung
• Multitapdelay

It should be noted that various possibilities are conceivable to visualize impulse responses. The simplest way is to arrange the impulse responses according to their direction around the center of a rendering system. Frequency-independent processing of the amplitude profiles of the impulse responses can be carried out in the resulting "mountain range". For examples of visualization methods, reference is made to the following figures, which describe the following four variants of the visualization:

• Wave field synthesis point source
• Impulse response time representation
• Impulse response time-frequency representation
• Multitapdelay

Fig. 2 shows schematically how it is based on the in Fig. 3a or 3b shown visualization of the graphical user interface is possible to determine the position of a sound source by means of a graphical user interface according to the invention or to change an existing position so that a desired position impression is created.

In the positioning step 20, the position of a sound source relative to the reproduction environment is initially determined graphically.

The graphical user interface graphically illustrates, in the second step 22, the impulse responses representing the position of the sound source, which can be changed directly by the user.

It should be noted that, as in the following by means of Fig. 3a or 3b can be seen, both the position of the source varies, and the course of the calculated impulse responses can be manipulated directly. This additionally makes it possible to implement creative sound effects which need not be directly linked to "real" location information.

Fig. 3a or 3b shows an embodiment of a graphical user interface according to the invention for determining the spatial position of a sound source or for changing the impulse responses representing the sound source.

Shown is a point-shaped sound source 30 in the form of a sphere, a reproduction environment 32 and a wavefront 34 corresponding to the point source.

The position of the ball describes the position of the sound source 30 in space. Based on the position of the point source 30, the wavefront 34 is shown, which results from the sound radiation of the point-shaped signal source. For example, moving the point source 30 to a point in space farther from the rendering environment 32 will make the wavefront 34 flatter. If the point source 30 is moved closer to the speaker system, then the corresponding wavefront will be more curved.

According to the invention, the curvature of the wavefront can also be changed directly with the aid of two graspers 36a and 36b. This directly affects the perceived position of the point source 30, which is automatically displayed by the graphical user interface according to the invention.

The graphical user interface in Fig. 3a or 3b also shows a delay radius 38 which serves to avoid acausal states in the reproduction of a wave field synthesis based system where the position of wavefront 34 is determined by the delay radius. The delay radius 38 corresponds to a basic delay, which requires a wave field synthesis system, and which corresponds to the distance of the loudspeaker farthest from the center of the system. The basic delay makes it possible to position sources arbitrarily within and outside the speaker system / reconstruction area or the reproduction environment 32.

Like it Fig. 3a or 3b shows, the position of the wavefront is defined by the intersection of the connecting line between the system center point and the position of the sound source 30 with the delay radius. The thus determined position of the wavefront 34 is thus equivalent to a vanishing delay, since the delay radius 38 determines just the minimum delay time to be observed. With the graphical user interface according to the invention, it is possible to arbitrarily position a sound source and to change its wavefront or the impulse response representing the wavefront.

With regard to the propagation delays, it should be noted that, in the case of a real sound field, a real signal propagation time depends on the distance of the sound source to the listening space. This is determined by the distance between the sound source position and the center of the Playback system. In the creation of imaginary auditory scenes, this runtime is usually not desirable because it restricts the positioning options of the sources, as this, for example, temporal relationships can be changed in a music recording. This delay can therefore be deactivated in wave field synthesis systems, which may be required for an authentic sound impression. This important additional parameter is represented in the graphic user interface according to the invention as a circle 40, the position of the circle 40 on the connecting line between the system center and the sound source 30 visualizing the set delay time.

Im in Fig. 3a or 3b In the case shown, the circle 40 is located directly at the boundary of the delay radius 38, the illustrated transit time has its minimum possible value, which corresponds to the basic delay of the wave field synthesis system. If the case of a real sound propagation time / deceleration is to be simulated, the position of the circle 40 would be directly below the sphere representing the sound source 30, it being understood that all intermediate values can additionally be displayed and adjusted. By means of the graphical user interface according to the invention, therefore, the important delay time parameters can be set and changed intuitively, which further increases the creative freedom and, moreover, increases the efficiency of the design process in spatial sound reproduction.

In addition, the graphical user interface according to the invention has the advantage of extremely great flexibility, so that further parameters can be easily added, for example the area of circle 40 could describe a ratio of diffused sound to direct sound, which is considered by a listener to be another feature for the removal of a sound source Listening position is understood, changing this ratio, for example, by a Moving the circle 40 or changing its surface could be implemented.

In accordance with the position of a virtual sound source S relative to the individual loudspeaker positions L _1..n , the wave field synthesis _algorithm calculates the impulse response IR _L1..Ln for each loudspeaker involved (amplitude, delay). If, at a time t, these impulse responses are lined up next to one another, the peaks result in a sampled version of the wavefront emanating from the virtual sound source. In another graphical processing step (see Fig. 3a ) from the wavefront can be simplified and displayed with interaction elements. If the user now interacts with these elements, the graphical representation of the wavefront changes. This representation change can be impressed on the individual impulse responses IR _L1..Ln in the next step

Generally speaking, the graphical user interface allows the manipulation of impulse responses, which are preferably to be computed for each individual loudspeaker which illuminates the reproduction volume 32.

At the in Fig. 3b In the exemplary embodiment shown, the graphic user interface makes it possible to manipulate impulse responses which are to be calculated for each individual loudspeaker which illuminates the reproduction volume 32. The representation of the impulse responses results directly from the representation of the graphical user interface, for which purpose a connection line 42 between the sound source 30 and an imaginary loudspeaker at the edge of the reproduction volume 32 is shown by way of example. The impulse response to be calculated is given directly by the shape of the wavefront at the location at which the connecting line 42 intersects the wavefront 34. The spatial position of a sound source 30, as it is in Fig. 3a or 3b can be seen for each individual speaker translated into a time delay and an amplitude. The amplitude results directly from the height of the graphical representation of the wavefront 34, wherein the time delay is also determined by the intersection of the line 42 with the wavefront 34, wherein for the determination of the time delay, the length of the cut sections of the line 42 is relevant.

As an alternative to the previously described manipulation forms implemented in the graphical user interface, a number of other alternative scenarios are easily implementable.

So z. B. the wavefront representation 34 in the figure by two balls or handles 36a and 36b limited. The manipulation of the wave front at these points ultimately affects the delays or the time delays of the speakers of the wave field synthesis system involved in the synthesis. Other handles on the illustrated wavefront 34 could be used, for example, to change the loudspeaker amplitudes. Thus, the simple adjustment of a window to avoid edge effects as well as the definition of a point with maximum amplitude. This point can then give the sound source, at least with respect to the intensity, a frequency-independent directional characteristic.

For example, the size of the sound source descriptive ball 30 can be used to represent the volume of a sound source. The above-mentioned manipulation of the direct sound / diffuse sound ratio can also be displayed again here. If the volume of the direct sound corresponds to the size of the ball 30, z. For example, a distant source of sound tends to be quieter and thus corresponds to a small sphere. A link with the distance-dependent calculation of the volume of a sound source is thus easily realized by this representation.

With the graphic user interface according to the invention in FIG Fig. 3a or 3b Thus, it is possible to intuitively and generally understand the mathematical function that embodies the impulse response in such a way that the impulse response can be purposefully manipulated so that a desired directional impression is created.

While the possibilities of the graphical user interface are out Fig. 3a or 3b for positioning a sound source, that is, for determining a sound impression, which reproduces the location of the sound source, is based on the Fig. 4-8 explained that the graphical user interface according to the invention is also suitable to visualize such impulse responses and to allow their change, which cause a sound impression that corresponds to that of a room to be simulated, such as a cathedral.

In order to make this possible, there are two principal possibilities, which are based on the Fig. 4 and 5 will be explained below.

Fig. 4 shows a possibility in which first in a positioning step 50, the sound sources are arranged in space, as for example with reference to Fig. 3a or 3b has been described. The impulses are assigned impulse responses for each sound source.

Since the sound source is in a defined spatial position with respect to the reproduction environment, a spatial sound impression of the sound source can be imprinted directly when it is in a spatial direction with respect to the reproduction environment for which a specific spatial sound impression is to be simulated.

In this case, in a space simulation step 52, an impulse response function is generated for each sound source and spatial direction, which is sent to a reproduction system together with the Sound source in a transfer step 54 must be transmitted in order to achieve the desired spatial sound impression during playback.

Like it Fig. 5 Alternatively, it is also possible to first determine the position of the sound sources in a positioning step 60, in which impulse responses are generated for loudspeakers for each sound source, which describe the position. The impression of space that is to be created in a listening direction can, since the loudspeakers used in the reproduction system are also assigned to fixed spatial directions, also be generated by additionally generating for each loudspeaker in a room simulation step 62 an impulse response which contains the information about the in the auditory system Direction of the relevant speaker contains space.

In a transfer or storage step 64, the sound source system must then be transmitted to the sound reproduction system and a position impulse response and a room impulse response must be transmitted for each individual loudspeaker. Due to the flexibility of the graphic user interface according to the invention, the allocation of a spatial sound impression either individually to each sound source can be done or it can groups of sound sources, which are arranged in a similar spatial direction with respect to the playback environment, summarized to represent a plurality of discrete spatial directions, whereby in the Playback the required computing capacity is reduced.

An embodiment of the graphical user interface according to the invention showing the manipulation of an impulse response in an impulse response time representation is shown in FIG Fig. 6 shown.

For this purpose, the spatial directions with respect to a reproduction environment 70 are subdivided into eight discrete sectors 72a-72h. For each of the sectors 72a-72h, therefore, a common Spatial impression achieved by means of an impulse response time representation. For visualization, the envelopes of the eight impulse responses used for space simulation are extruded to surfaces. These surfaces are arranged in the form of an octagon and connected to a common surface 74. The height of the area above the area defined by sectors 72a-72i corresponds to the amplitude of the impulse response. The distance from the center of the replay environment 70 represents the time, therefore, events occurring at the end of the impulse response are farther from the center of the replay environment 70.

With this representation, the amplitude curves of the room impulse responses over time can be represented according to their spatial direction. The change takes place interactively by moving interaction elements 76a, b and c exemplified here. It is thus possible to grasp the entire spatial sound situation at a glance and to recognize and eliminate deviations from the desired behavior.

For example, should the reverberation time from all directions usually be almost the same for a real space. In the example of Fig. 6 However, the reverberation time in the direction of the sector 72h is reduced, which can be easily recognized by the asymmetry of the total area 74, so that the difference to the real, evenly reverberant room can be recognized immediately.

Fig. 7 describes a representation of spatial impulse responses in a time-frequency representation. Shown is the rendering environment 80 and eight time-frequency representations of impulse responses 82a-82h associated with eight discrete spatial directions relative to the rendering environment 80.

It is generally with the embodiment according to the invention in Fig. 7 possible, both the time and the To visualize frequency components of impulse responses related to their spatial directions and to manipulate them. The timeline of the visualization runs outward from the center of the rendering environment 80 so that more distant points describe later events. The eight areas 82a-82h, which represent the impulse responses in the form of a waterfall diagram, can be changed, for example, by means of interaction elements 86a-86c. The exemplified interaction elements 86a-86c allow the manipulation of the amplitude frequency response at a certain time, in the example shown here at the beginning of the impulse response. In the case shown here, low frequencies are located farther left and high frequencies farther to the right, so that it can be seen immediately that in the spatial simulation, the low frequencies begin with a higher amplitude and end longer than the high frequencies. This complex relationship, which can be stored in the form of a matrix, for example by describing the areas 82a-82h, can be intuitively recorded and changed here.

The type of representation also makes it possible to attach additional effects or to recognize their effect, for example, strong reflections from certain spatial directions would be visible as elevations on the surfaces of the corresponding spatial impulse response in this representation.

It is thus apparent from the simultaneous view of the time and frequency component, which frequency components are reflected. With a shift of the interaction elements 86a-86c to a corresponding point in the impulse response, this reflection can be processed both in terms of time and frequency, so that the large number of parameters underlying the visualization can be conveniently and efficiently sampled and stored.

Fig. 8 shows another example of a graphical user interface according to the invention, in which the impulse responses the individual spatial directions consist of discrete peaks. Shown are a rendering environment 90, eight discrete spatial directions 92a-92i, and five exemplary delta-shaped impulse responses 94a-94e.

Since peak- or delta-shaped impulse responses correspond to time delays of a sound signal, direction-dependent multi-tap delays can thus be realized. The wavefronts 94a-94e represent echoes from the spatial directions assigned to them. Their distance to the center of the playback volume indicates the time of the repetition of the original signal. According to the invention, for example, by means of an interaction element 96 in the form of a sphere, the position of the repetitions can be influenced by radial movements of the impulse responses from or to the center of the system. At the same time, the amplitude of the repetitions can be influenced by the height of the wavefronts in the vertical direction.

The advantage of the high level of intuitiveness of the graphical user interface according to the invention becomes particularly clear here, since the position of the delta-shaped peaks describes the delay time of an echo, which is acoustically equivalent to a reflective wall with predetermined attenuation, which is located at the position of the impulse responses.

In an expanded variant of the graphic user interface according to the invention, a time-frequency representation can also be implemented here in order to additionally impress an individual frequency response on each echo.

Fig. 9 describes a system for visualizing and editing spatial sound effects 100, which is composed of a signal processing part 102 and a visualization and interaction part 104.

According to the invention, the signal processing consists in that incoming audio signals 106 are folded by means of a mathematical convolution 108 with the impulse responses determined by means of the visualization and interaction part 104 in order to generate therefrom audio signals 110 which carry the sound impression of a room to be simulated. The visualization and interaction part 104 has a display means for displaying calculated impulse responses 112, means for receiving a user change input 114, means for permitting a change of the graphical display 116, and means for detecting the changed impulse response 118. The means for receiving a user change input 114 comprises an interaction device 120 and a means for implementing the interaction 122. The means for permitting a change in the graphical display of the impulse response 116 comprises an output device 124 for displaying the original impulse response and an image calculation unit 126 for visualizing the original impulse response ,

From the means for receiving a user change input and the means for permitting a change in the graphical display of the impulse response 116, a visual model 112 is generated based on parameters describing the impulse responses and thus containing the information about the space to be simulated. If a suitable visual model has been created by repeated interaction and visualization, the means for detecting the changed impulse response 118 extracts the parameters on which the visualization is based, and transmits them as impulse responses to the signal processor 102.

In a preferred embodiment of the present invention, the signal processing comprises the convolution of N input signals with n impulse responses to n output signals. N can be of z. B. eight signals in the generation from reverb effects for wave field synthesis to a very large number in the generation of a whole wave field. If multiple effects or sources are generated simultaneously, the output signals for each effect or source must be summed up at the end.

The impulse responses required for the signal processing are thus generated with the aid of the visualization and interaction part of the system. From an impulse response, sound-relevant parameters can be generated. It must be distinguished whether it is room signals or direct signals.

For room signals, different methods can be applied. The values obtained can then be graphically displayed as described in the section on visualization. Using the graphics and the built-in interaction elements, the parameters can be changed and processed into a new impulse response.

In the case of direct sound positioning, parameters can also be obtained from the interface. However, these can only be converted into impulse responses for the loudspeaker channels by the application of the wave field synthesis algorithm. The parameters are thus at a more abstract level. But the structure of the block diagram in Fig. 9 does not change.

With the help of this system, all spatial sound effects from room simulation to multi-tap delays can be visualized and edited. This concept can be used in all conventional multi-channel systems up to wave field synthesis. It offers a universal solution for spatial sound effects and their intuitive use for the user.

As is illustrated by the described embodiments, a significant advantage of the graphical user interface according to the invention is that complex mathematical parameters are made intuitively accessible. This makes it possible to generate or set these parameters, with it being possible in particular to keep an eye on the entire sound event at all times. It is particularly advantageous that in the described embodiments, which are based on 3D visualizations, the direction in which the playback environment is considered, can be varied, so that an emerging sound impression can be even better predicted that this judges from different spatial directions becomes.

Although in the presentation in Fig. 1 the graphic user interface has individual discrete function blocks, such a division is only to be understood as an example, in principle any combinations and summaries of the individual function blocks are possible. So z. For example, it will be understood that the display 12 is combined with the means 14 for altering the graphical display, as is partially the case in the illustrated embodiments, where the possibility of modification is already implemented as part of the display, for example in the form of the handles 36a and 36b in Fig. 3a or 3b ,

In the device for receiving a user change input, methods other than those shown in the exemplary embodiments are also conceivable in principle. The user input may be by means of a mouse, a touchscreen, or any other means of moving a cursor on a screen. The direct input of discrete change steps by means of a keyboard can also be represented, for example in the case of a discretized representation of an impulse response, where in defined time ranges the value of the impulse response in discrete steps can be adjusted, which is easily possible for example by means of a conventional keyboard.

The representation of the wavefronts or impulse responses and the possibility of manipulating the same are to be understood as examples only, any other suitable representation of impulse response functions is also possible in order to allow setting or generation of a spatial impression according to the invention. For example, it would be conceivable to represent a common impulse response function when considering different spatial directions, which to some extent prescribes the fundamental spatial character, which is therefore the same for all spatial directions. A direction-dependent spatial sound character could advantageously be represented by the fact that for each spatial direction only the difference to the common impulse response function is displayed, so that one easily gets an impression of how the observed spatial direction differs in its spatial properties from the overall sound image (middle sound image).

An order of processing the impulse response functions that describe the position of a sound source or the spatial impression is not fixed. It is possible to first position all the sound sources in the room and then create a spatial impression, as well as to first define the room to be simulated in order to subsequently position the sound sources within the room.

Accordingly, the processing steps for a system for driving a sound reproduction system which comprises a graphic user interface according to the invention and a signal generator for supplying loudspeaker signals differ. On the one hand, it is possible to impress on each sound source, which is located in a defined spatial direction, spatial information by folding with a spatial impulse response function, and then in a further step speaker-individually a convolution with impulse responses which describe the position of the sound sources relative to the playback volume.

Alternatively, it is possible first to edit the sound source speaker individually, d. H. to generate individual loudspeaker signals by convolution of the sound signal with the impulse responses describing the position of the sound source, and then loudspeaker individually perform another convolution that creates the spatial impression, wherein the speakers, which are arranged in a fixed geometric direction to the playback environment, folded with a spatial impulse response which correspond to the room impression to be simulated in the direction of the loudspeakers.

The form of the graphic elements shown in the embodiments for visualizing the individual essential components, such as the position of the sound source or the form of an impulse response, are to be understood as preferred embodiments, but the operation according to the invention is also ensured if the art the geometric representation differs with respect to the shape, depending on the purpose, a different shape could even have a functional character, i. H. describe different properties of a sound source, for example.

The signal processing, which is represented individually for each speaker by convolution of a sound signal with an impulse response function, can be implemented both continuously and discretely, although alternative mathematical methods are also possible for imparting the spatial impression which an impulse response describes to a sound signal.

In the embodiments shown above, in order to create a spatial impression, the space enclosing the reproduction environment is subdivided into eight discrete spatial directions, wherein a spatial sound character can be determined individually for each spatial direction. This is to be understood only as an example, of course, any other numbers of spatial directions are possible, in principle, the number of upward directions is not limited, so that it is easily possible according to the invention to improve the overall sound impression even further.

Depending on the circumstances, the inventive method of using a graphical user interface for using a sound reproduction system may be implemented in hardware or in software. The implementation can be carried out on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which can cooperate with a programmable computer system such that the method according to the invention for checking the success of a deconing process is carried out. In general, the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer. In other words, the invention can thus be realized as a computer program with a program code for carrying out the method when the computer program runs on a computer.

Claims (20)

1. Graphic user interface (10) for a sound-reproduction system, which is formed so as to generate in a reproduction environment (32; 70; 80; 90) a spatial sound impression, comprising:

   display means (12) for graphically displaying impulse responses (34; 74; 82a - 82h; 94a - 94e), which are associated with spatial directions of the reproduction environment (32; 70; 80; 90), the impulse responses, relative to the reproduction environment, being represented in the spatial directions they are allocated to;

   means for allowing changing (14) the graphical display of the impulse responses (34; 74; 82a - 82h; 94a - 94e) by the user, a change of the graphical display of the impulse responses (34; 74; 82a - 82h; 94a - 94e) being enabled at predetermined points;

   means for receiving (16) a user input of a change, in order to graphically represent changed impulse responses by the display means (12); and

   means for detecting the changed impulse responses (18).
2. Graphic user interface according to claim 1, wherein the display means (12) is formed so as to represent the impulse responses (74; 82a - 82h; 94a - 94e) as time-dependent evolutions of an intensity value.
3. Graphic user interface according to claim 2, wherein the display means (12) is formed so as to represent the time-dependent evolutions of the impulse responses (74; 82a - 82h; 94a - 94e) such that same are divided into discrete time periods, an intensity value being associated with each time period.
4. Graphic user interface according to one of the previous claims, wherein the display means (.12) is formed so as to represent the impulse responses (82a - 82h) as a function of the frequency.
5. Graphic user interface according to claim 4, wherein the display means (12) is formed so as to represent the frequency evolutions of the impulse responses (74; 82a - 82h; 94a - 94e) such that same are divided into discrete frequency segments, an intensity value being associated with each frequency segment.
6. Graphic user interface according to one of the previous claims, wherein the display means (12) is formed so as to graphically represent the impulse responses (82a - 82h) as functions of the time and as functions of the frequency in a three-dimensional representation, the function values being represented as a height over a two-dimensional surface, one side of which has the time as a measure and a second side of which following the first side has the frequency as a measure.
7. Graphic user interface according to one of the previous claims, wherein the display means (12) is formed so as to display in addition a graphical representation of the reproduction environment (32; 70; 80; 90) in a three-dimensional representation, the impulse responses (34; 74; 82a - 82h; 94a - 94e) regarding the reproduction environment (32; 70; 80; 90) being represented in the spatial directions, which the impulse responses (34; 74; 82a - 82h; 94a - 94e) are associated with.
8. Graphic user interface according to one of the previous claims, wherein the means for allowing changing (14) the graphical display of the impulse responses (34; 74; 82a - 82h; 94a - 94e) is formed so as to allow changing the graphical representation of the impulse responses (34; 74; 82a - 82h; 94a - 94e) at any arbitrary point of the graphical representation of the impulse responses (34; 74; 82a - 82h; 94a - 94e).
9. Graphic user interface according to one of the previous claims, wherein the means for allowing changing (14) the graphical display of the impulse responses (94a - 94e) is formed so as to allow shifting the impulse responses (94a - 94e) in the time as a change of the graphical display of the impulse responses (94a - 94e).
10. Graphic user interface according to one of the previous claims, wherein the means for receiving a user input of a change (16) is formed so as to receive signals of a computer mouse, a touch pad, a touch screen, a track ball or a keyboard.
11. Graphic user interface according to one of the previous claims, wherein the means for detecting (18) the changed impulse responses is formed so as to scan, for detecting, the changed impulse responses graphically represented and to store the scanned values in a memory.
12. Graphic user interface according to one of the previous claims, wherein the display means (12) is formed so as to graphically display impulse responses (74; 82a - 82h; 94a - 94e), which contain information about a space to be simulated.
13. Graphic user interface according to one of the previous claims, wherein the display means (12) is formed so as to graphically display impulse responses (34), which contain information about the position of a sound source (30) with respect to the reproduction environment (32).
14. Controlling apparatus for a sound-reproduction system, which is formed so as to generate a spatial sound impression in a reproduction environment, comprising:

    a graphic user interface (10) according to any one of claims 1 to 13; and

    a signal generator (102) for providing loudspeaker signals (110) for loudspeakers of a plurality of loudspeakers that can be placed at different spatial positions.
15. Controlling apparatus according to claim 14, wherein the signal generator (102) has combination means (108) for combining sound signals (106) with the changed impulse responses, the sound signals (106) being intended for loudspeakers arranged at spatial positions corresponding to the spatial directions, which the impulse responses are associated with, in order to obtain loudspeaker signals (110), the combination means (108) being formed so as to combine such that the loudspeaker signals (110) contain the information about the space to be simulated.
16. Controlling apparatus according to claim 15, wherein the signal generator (102) has combination means (108) for combining the sound signals (106) with the changed impulse responses, in order to obtain a loudspeaker signal (110), the combination means (108) being formed so as to combine such that the loudspeaker signals (110) contain the information about the relative positions of a sound source associated with the sound signals (106).
17. Controlling apparatus according to one of claims 15 or 16, wherein the combination means (108) is formed so as to alias, during the combination, the sound signals (106) with the changed impulse responses.
18. Method for using a sound-reproduction system, which is formed so as to generate a spatial sound impression in a reproduction environment (32; 70; 80; 90), comprising:

    graphically displaying impulse responses (34; 74; 82a - 82h; 94a - 94e) associated with spatial directions of the reproduction environment (32; 70; 80; 90), the impulse responses, relative to the reproduction environment, being represented in the spatial directions they are allocated to;

    allowing changing the graphical display of the impulse responses (34; 74; 82a - 82h; 94a - 94e) by the user, a change of the graphical display of the impulse responses (34; 74; 82a - 82h; 94a - 94e) being enabled at predetermined points;

    receiving a user input of a change, in order to graphically represent changed impulse responses; and

    detecting the changed impulse responses.
19. Method for controlling a sound-reproduction system, comprising the steps of the method according to claim 18 and additionally comprising:

    providing loudspeaker signals for a plurality of loudspeakers, which can be placed at different spatial positions based on the changed impulse responses.
20. Computer program with a program code for performing the method according to claim 18 or 19 when the computer program is executed on a computer.

Images