Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
  • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
• • G—PHYSICS
  • G10—MUSICAL INSTRUMENTS; ACOUSTICS
  • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
  • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
  • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
  • G10L19/16—Vocoder architecture
  • G10L19/18—Vocoders using multiple modes
  • G10L19/20—Vocoders using multiple modes using sound class specific coding, hybrid encoders or object based coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
  • H04R3/00—Circuits for transducers, loudspeakers or microphones
  • H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
  • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
  • H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04S—STEREOPHONIC SYSTEMS 
  • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
  • H04S2420/13—Application of wave-field synthesis in stereophonic audio systems

Definitions

• the present invention is in the field of wave field synthesis and relates in particular to devices and methods for generating, storing or editing an audio representation of an audio scene.
• WFS Wave-Field Synthesis
• wave field synthesis Due to the enormous demands of this method on computer performance and transmission rates, wave field synthesis has so far only rarely been used in practice. It is only the advances in the areas of microprocessor technology and audio coding that allow this technology to be used in concrete applications. The first products in the professional sector are expected next year. The first wave field synthesis applications for the consumer sector are also expected to be launched in a few years.
• Every point that is captured by a wave is the starting point for an elementary wave that propagates in a spherical or circular manner.
• a large number of loudspeakers that are arranged next to each other can be used to simulate any shape of an incoming wavefront.
• the audio signals of each loudspeaker have to be fed with a time delay and amplitude scaling in such a way that the emitted sound fields of the individual loudspeakers overlap correctly. If there are several sound sources, the contribution to each loudspeaker is calculated separately for each source and the resulting signals are added. If the sources to be reproduced are in a room with reflecting walls, then reflections must also be reproduced as additional sources via the loudspeaker array the. The effort involved in the calculation therefore depends heavily on the number of sound sources, the reflection properties of the recording room and the number of speakers.
• the particular advantage of this technique is that a natural spatial sound impression is possible over a large area of the playback room.
• the direction and distance of sound sources are reproduced very precisely.
• virtual sound sources can even be positioned between the real speaker array and the listener.
• wave field synthesis works well for environments whose properties are known, irregularities occur when the nature changes or when the wave field synthesis is carried out on the basis of an environment condition that does not correspond to the actual condition of the environment.
• the technique of wave field synthesis can also be used advantageously to complement a visual perception with a corresponding spatial audio perception.
• the focus in production in virtual studios has been to provide an authentic visual impression of the virtual scene.
• the acoustic impression that goes with the image is usually imprinted on the audio signal by manual work steps in what is known as post-production, or is classified as too complex and time-consuming to implement and is therefore neglected. This usually leads to a contradiction of the individual sensations, which leads to the fact that the designed space, i. H. the designed scene, which is perceived as less authentic.
• the audio material for a film for example, consists of a large number of audio objects.
• An audio object is a sound source in the film setting. If, for example, you think of a film scene in which two people face each other and are in a dialogue, and at the same time e.g. For example, if a rider and a train are approaching, a total of four sound sources exist in this scene over a certain period of time, namely the two people, the approaching rider and the approaching train. If it is assumed that the two people who are in dialogue do not speak at the same time, then at least two audio objects are likely to be active at a time, namely the rider and the train, if both people are currently silent.
• an audio object presents itself in such a way that the audio object describes a sound source in a film setting that is active or “alive” at a certain point in time. This means that an audio object is further characterized by a start time and an end time.
• the rider and the train are active throughout the setting, and when both approach, the listener will notice this by making the rider and the train noisier and possibly - in an optimal wave field synthesis setting - as well change the positions of these sound sources accordingly
• the two speakers in dialogue are constantly generating new audio objects, since whenever a speaker stops speaking the current audio object has ended and when the other speaker starts speaking, a new audio object begins which in turn ends when the other S precher stops speaking, and when the first speaker starts speaking again, a new audio object is started again.
• Existing wave field synthesis rendering devices exist which are able to generate a certain number of loudspeaker signals from a certain number of input channels, with knowledge of the individual positions of the loudspeakers in a wave field synthesis loudspeaker array.
• the wave field synthesis renderer is to a certain extent the "heart" of a wave field synthesis system that correctly calculates the loudspeaker signals for the many loudspeakers of the loudspeaker array in terms of amplitude and phase, so that the user not only has an optimal optical impression but also an optimal one has an acoustic impression.
• Playback systems usually have fixed speaker positions, such as in the case of 5.1 the left channel nal ("left”), the middle channel (“center”), the right channel (“right”), the surround left channel (“Surround left”) and the surround right channel (“Surround right”)
• the ideal sound image the sound engineer is looking for is limited to a small number of seats, the so-called sweet spot, although the use of phantom sources between the 5.1 positions described above results in certain cases to improvements, but not always satisfactory results.
• the sound of a film usually consists of dialogues, effects, atmospheres and music. Each of these elements is mixed taking into account the limitations of 5.1 and 7.1 systems. Typically, the dialogue is mixed in the center channel (in 7.1 systems also on a half-left and a half-right position). This implies that when the actor moves across the screen, the sound does not follow. Movement sound object effects can only be realized if they move quickly, so that the listener is unable to recognize when the sound passes from one speaker to another.
• Lateral sources also cannot be positioned due to the large audible gap between the front and surround speakers so that objects cannot move slowly from back to front and vice versa.
• Surround loudspeakers are also placed in a diffuse array of loudspeakers and thus produce a sound image that represents a kind of envelope for the listener. Therefore, precisely positioned sound sources behind the listeners are avoided in order to avoid the unpleasant sound interference field that is associated with such precisely positioned sources.
• Wave field synthesis as a completely new way of building up the sound field that is heard by the listener overcomes these essential shortcomings. The consequence for cinema applications is that an accurate sound image can be achieved without restrictions with regard to a two-dimensional positioning of objects. This opens up a wide variety of possibilities in the design and mixing of sound for cinema purposes. Due to the complete sound image reproduction, which is achieved by the technique of wave field synthesis, sound sources can now be positioned freely. Furthermore, sound sources can be placed as focused sources inside the listener room as well as outside the listener room.
• stable sound source directions and stable sound source positions can be generated using point-shaped radiating sources or plane waves.
• sound sources can be moved freely inside, outside or through the listening room.
• the sound design ie the activity of the sound engineer
• the coding format and the number of speakers ie 5.1 systems or 7.1 systems, determine the reproduction setup.
• a special sound system requires a special encoding format.
• the channels are of no concern to a viewer / listener. He does not care which sound system a sound is generated from, whether an original sound description was object-oriented, was channel-oriented, etc. The listener also does not care whether and how an audio setting was mixed. All that counts for the listener is the sound impression, i.e. whether he likes a sound setting for a film or a sound setting without a film or not.
• the sound engineers are responsible for the sound mixing. Due to the channel-oriented paradigm, sound engineers are "calibrated" to work channel-oriented. For them it is actually the goal to mix the six channels for a cinema with a 5.1-sound system, for example audio signals recorded in a virtual studio and mix the final 5.1 or 7.1 loudspeaker signals, for example, not channel objects, but channel orientation, so in this case an audio object typically has no start time or no end time a signal for a loudspeaker to be active from the first second of the film to the last second of the film, due to the fact that one of the (few) loudspeakers of the typical cinema sound system always produces any sound since it is always there may be a sound source that is broadcast over the special speaker, even if it is just background music.
• existing wave field synthesis rendering units are used to work oriented so that they have a certain number of input channels, from which, when the audio signals and associated information are input into the input channels, the loudspeaker signals for the individual loudspeakers or loudspeaker groups of a wave field synthesis loudspeaker array are generated.
• the technique of wave field synthesis leads to the fact that an audio scene is much more "transparent", namely in that in principle an unlimited number of audio objects viewed via a film, ie viewed via an audio scene, can be present
• Channel-oriented wave field synthesis rendering devices can become problematic if the number of audio objects in an audio scene exceeds the typically always predetermined maximum number of input channels of the audio processing device.
• the object of the present invention is to create a concept for generating, storing or editing an audio representation of an audio scene, which has a high level of acceptance on the part of the users for whom corresponding tools are intended.
• This object is achieved by a device for generating, storing or editing an audio representation of an audio scene according to claim 1, a method for generating, storing or editing an audio representation of an audio Dioscene according to claim 15 or a computer program according to claim 16 solved.
• the present invention is based on the knowledge that for audio objects as they occur in a typical film setting, only an object-oriented description can be processed clearly and efficiently.
• the object-oriented description of the audio scene with objects that have an audio signal and to which a defined start and a defined end time are assigned correspond to the typical conditions in the real world, in which it is rare for a sound to be heard anyway Time is there. Instead, it is common, for example in a dialogue, that a dialogue partner begins to speak and stops speaking, or that noises typically have a beginning and an end.
• the object-oriented audio scene description which assigns each sound source its own object in real life, is adapted to the natural conditions and therefore optimal in terms of transparency, clarity, efficiency and intelligibility.
• an imaging device is used to map the object-oriented description of the audio scene onto a plurality of input channels of an audio processing device, such as, for example, a wave field synthesis rendering unit.
• the imaging device is designed to assign a first audio object to an input channel, and to assign a second audio object, the start time of which reads after an end time of the first audio object, to the same input channel, and a third audio object, the start time of which after the start time of the first audio object and before the end time of the first audio object is to assign another one of the plurality of input channels.
• This time allocation which assigns audio objects that occur simultaneously to different input channels of the wave field synthesis rendering unit, and which assigns audio objects that occur sequentially, has been found to be extremely channel-efficient.
• the user e.g. the sound engineer
• the user can get a quick overview of the complexity of an audio scene at a certain point in time without having to laboriously search from a variety of input channels to find out which object is currently active or which object is not currently active.
• the user can easily manipulate the audio objects, as in the object-oriented representation, using his or her usual channel controls.
• FIG. 1 shows a block diagram of the device according to the invention for generating an audio representation
• Fig. 2 is a schematic representation of a user interface for the concept shown in Fig. 1;
• FIG. 3a shows a schematic illustration of the user interface parts from FIG. 2 according to an exemplary embodiment of the present invention
• FIG. 3b shows a schematic illustration of the user interface from FIG. 2 according to another exemplary embodiment of the present invention
• FIG. 4 shows a block diagram of a device according to the invention in accordance with a preferred exemplary embodiment
• FIG. 5 shows a temporal representation of the audio scene with different audio objects
• FIG. 6 shows a comparison of a 1: 1 conversion between object and channel and an object-channel assignment according to the present invention for the audio scene shown in FIG. 5.
• the device according to the invention comprises a device 10 for providing an object-oriented description of the audio scene, the object-oriented description of the audio scene comprising a plurality of audio objects, with at least one audio signal, a start time and an end time being assigned to an audio object.
• the device according to the invention further comprises an audio processing device 12 for generating a plurality of loudspeaker signals LSi 14, which is channel-oriented and which generates the plurality of loudspeaker signals 14 from a plurality of input channels EKi.
• an imaging device 18 for mapping the object-oriented description of the audio scene onto the plurality of input channels 16 of the channel-oriented audio signal processing device 12 , wherein the imaging device 18 is designed to assign a first audio object to an input channel, such as EKI, and to assign a second audio object whose start time is after an end time of the first audio object to the same input channel, such as the input channel EKI, and to assign a third audio object whose start time is after the start time of the first audio object and before the end time of the first audio object to another input channel of the plurality of input channels, such as the input channel EK2.
• the imaging device 18 is thus designed so that audio objects that do not overlap in time are assigned to the same input channel. assign, and to assign overlapping audio objects to different parallel input channels.
• the audio objects are further specified in such a way that they are assigned a virtual position.
• This virtual position of an object can change during the lifetime of the object, which would correspond to the case in which, for example, a rider approaches a scene center, in such a way that the rider's gallop becomes louder and, in particular, comes closer and closer to the auditorium.
• an audio object includes not only the audio signal that is assigned to this audio object and a start time and an end time, but also a position of the virtual source that can change over time and possibly further properties of the audio object, such as whether it should have point source properties or whether it should emit a plane wave, which would correspond to a virtual position with an infinite distance to the viewer.
• Further properties for sound sources, ie for audio objects, are known in the art and can be taken into account depending on the equipment of the channel-oriented audio signal processing device 12 from FIG. 1.
• the structure of the device hierar ⁇ constructed chically, to the effect that the channel-based audio signal processing apparatus dioumbleen for receiving Au ⁇ is not directly combined with the means for providing, but is combined with the same via the exhaust school.
• the device shown in FIG. 1 is further provided with a user interface, as shown at 20 in FIG. 2.
• the user interface 20 is designed to have one user interface channel per input channel and preferably one manipulator for each user interface channel.
• the user interface 20 is coupled via its user interface input 22 to the imaging device 18 in order to receive the assignment information from the imaging device, since the occupancy of the input channels EKI to EKm is to be displayed by the user interface 20.
• the user interface 20 On the output side, if the user interface 20 has the manipulator feature for each user interface channel, it is coupled to the device 10 for providing.
• the user interface 20 is designed to provide manipulated audio objects of the device 10 for provision via its user interface output 24 with respect to the original version, which thus receives a changed audio scene, which is then returned to the imaging device 18 and, accordingly, distributed over the input channels Channel-oriented audio signal processing device 12 is provided.
• the user interface 20 is designed as a user interface, as shown in FIG. 3a, that is to say as a user interface, which always only shows the current objects.
• the user interface 20 is configured to be structured as in FIG. 3b, that is to say in such a way that all objects are always represented in an input channel.
• a time line 30 is shown which comprises objects A, B, C in chronological order, where for object A comprises a start time 31a and an end time 31b.
• the end time 31b of the first object A coincides with a start time of the second object B, which in turn has an end time 32b, which in turn coincides with a start time of the third object C, which in turn has an end time 33b.
• the start times 32a and 33b correspond to the end times 31b and 32b and are not shown in FIGS. 3a, 3b for reasons of clarity.
• a mixer channel symbol 34 is shown on the right in FIG. 3a, which comprises a slider 35 and stylized buttons 36, via the properties of the Audio signal of object B or virtual positions etc. can be changed.
• the time stamp in FIG. 3a which is represented by 37
• the stylized channel representation 34 would not display object B, but rather object C.
• the user interface in FIG. B. an object D would take place simultaneously with the object B, represent another channel, such as the input channel i + 1.
• 3a provides the sound engineer with a simple overview of the number of parallel audio objects at a time, that is to say the number of active channels that are displayed at all. Inactive input channels are not displayed at all in the embodiment of the user interface 20 of FIG. 2 shown in FIG. 3a.
• the input channel i to which the channels assigned in chronological order belong, is represented in triplicate, once as object channel A, another time as object channel B and again another time as object channel C.
• the channel such as input channel i for object B (reference symbol 38 in FIG. B. highlight in color or brightness to give the sound engineer on the one hand a clear overview of which object is currently being fed on the channel i in question, and which objects z. B.
• the user interface 20 of FIG. 2 and in particular the versions thereof in FIGS. 3a and 3b are thus designed to provide a visual representation as desired for the “assignment” of the input channels of the channel-oriented audio signal processing device that is generated by the imaging device 18 becomes.
• FIG. 5 shows an audio scene with different audio objects A, B, C, D, E, F and G. It can be seen that objects A, B, C and D overlap in time. In other words, these objects A, B, C and D are all active at a certain point in time 50. In contrast, object E does not overlap with objects A, B. Object E only overlaps with objects D and C, as can be seen at a point in time 52. The object F and the object D are again overlapping, as was the case at a point in time 54. B. can be seen. The same applies to objects F and G, which, for. B. overlap at a time 56 while object G does not overlap with objects A, B, C, D and E.
• a simple and in many respects disadvantageous channel assignment would be to assign each audio object to an input channel in the example shown in FIG. so that the 1: 1 conversion on the left in the table in Fig. 6 would be obtained.
• a disadvantage of this concept is that many input channels are required or that if there are many audio objects, which is very quickly the case in a film, the number of input channels of the wave field synthesis rendering unit is the number of virtual sources that can be processed in one limits the real film setting, which is of course not desirable, since technology limits should not impair the creative potential.
• this 1: 1 implementation is very confusing, in that, although at some point each input channel typically receives an audio object, that when a particular audio scene is viewed, relatively few input channels are typically active, but the user cannot easily determine this , because he must always have an overview of all audio channels.
• this concept of the 1: 1 assignment of audio objects to input channels of the audio processing device means that in order to limit the number of audio objects as little or not as possible, audio processing devices which have a very high number of input channels must be provided, which leads to an immediate increase in the computing complexity, the required computing power and the required storage capacity of the audio processing device in order to calculate the individual loudspeaker signals, which directly results in a higher price of such a system.
• the parallel audio objects A, B, C and D are sequentially assigned to the input channels EKI, EK2, EK3 and EK4.
• the object E no longer has to be assigned to the input channel EK5, as in the left half of FIG. 6, but They can be assigned to a free channel, such as the input channel EKI or, as indicated by the brackets, the input channel EK2.
• object F which can in principle be assigned to all channels except the input channel EK4.
• object G which can also be assigned to all channels except the channel to which object F was previously assigned (in the example the input channel EKI).
• the imaging device 18 is designed to always occupy channels with the lowest possible atomic number and to always occupy adjacent input channels EKi and EKi + 1 so that no holes arise.
• this "neighborhood feature" is not essential since a user of the audio authoring system according to the present invention is indifferent to whether he is currently using the first or the seventh or any other input channel of the audio processing device as long as he is through the the user interface according to the invention is enabled to manipulate precisely this channel, for example by means of a controller 35 or by buttons 36 of a mixer channel representation 34 of the current channel.
• the user interface channel i does not necessarily have to discuss the input channel i, but it can also do so a channel assignment takes place in such a way that the user interface channel i corresponds, for example, to the input channel EKm, while the user interface channel i + 1 corresponds to the input channel k, etc.
• the user interface channel remapping thus avoids channel holes, so that the sound engineer can always immediately and clearly see the current user interface channels displayed side by side.
• the concept of the user interface according to the invention can of course also be transferred to an existing hardware mixing console which includes actual hardware controls and hardware buttons which a Tomhoff will operate manually in order to achieve an optimal audio mix.
• An advantage of the present invention is that even such a sound mixer, which is typically very familiar and loved by the sound mixer, can also be used, for example by B. by indicators typically present on the mixing console, such as LEDs, the current channels are always clearly marked for the sound engineer.
• the present invention is also flexible in that it can deal with cases where the wave field synthesis speaker setup used for production is different from the reproduction setup e.g. B. deviates in a cinema. Therefore, according to the invention, the audio content is encoded in a format that can be processed by different systems.
• This format is the audio scene, i. H. the object-oriented audio representation and not the loudspeaker signal representation.
• the preparation process is understood as an adaptation of the content to the reproduction system.
• not only a few master channels but an entire object-oriented scene description are processed in the wave field synthesis reproduction process.
• the scenes are prepared for each reproduction. This is typically carried out in real time in order to adapt to the current situation.
• this adaptation takes into account the number of loudspeakers and their positions, the characteristics of the reproduction system, such as the frequency response, the sound pressure level etc., the room acoustic conditions or other image reproduction conditions.
• a major difference in the wave field synthesis mix compared to the channel-based approach of current systems consists in the freely available positioning of the sound objects.
• the position of the sound sources is relatively encoded. This is important for mixed concepts that belong to a visual content, such as cinema films, since positioning of the sound sources with respect to the image is attempted to be approximated by a correct system setup.
• the wave field synthesis system requires absolute positions for the sound objects, which is given to this audio object in addition to the audio signal of an audio object in addition to the start time and the end time of this audio object.
• the aim of the re-engineering of the post-production process is to minimize user training and integrate the integration of the new system according to the invention in the be ⁇ standing knowledge of the user.
• all tracks or objects that are to be prepared at different positions will exist within the master file / distribution format, which in contrast to conventional production facilities that are optimized to reduce the number of tracks during the production process.
• the wave field synthesis authoring tool according to the present invention is implemented as a workstation which has the possibility of recording the audio signals of the final mix and converting them to the distribution format in another step.
• the first is that all audio objects or tracks still exist in the final master.
• the second aspect is that positioning is not done in the mixing console. This means that so-called authoring is one of the last steps in the production chain.
• the wave field synthesis authoring system that is to say the device according to the invention for generating an audio representation
• the device according to the invention for generating an audio representation is implemented as an independent workstation, which can be integrated into different production environments by feeding audio outputs from the mixer into the system.
• the mixer represents the user interface, which is coupled to the device for generating the audio representation of an audio scene.
• FIG. 4 The system according to the invention according to a preferred embodiment of the present invention is shown in FIG. 4.
• the same reference numerals as in Fig. 1 or 2 indicate the same elements.
• the basic system design ba- is based on the goal of modularity and the possibility of integrating existing mixing consoles into the inventive wave field synthesis authoring system as user interfaces.
• a central controller 120 which communicates with other modules, is formed in the audio processing device 12. This enables the use of alternatives for certain modules as long as they all use the same communication protocol.
• the system shown in FIG. 4 is considered a black box, one generally sees a number of inputs (from the provision device 10) and a number of outputs (loudspeaker signals 14) as well as the user interface 20.
• the actual WFS renderer 122 Integrated in this black box next to the The user interface is the actual WFS renderer 122, which performs the actual wave field synthesis calculation of the loudspeaker signals using various input information.
• a room simulation module 124 is provided, which is designed to carry out certain room simulations that are used to generate room properties of a recording room or to manipulate room properties of a recording room.
• an audio recording device 126 and a recording playback device are provided.
• the device 126 is preferably provided with an external input.
• the entire audio signal is either already object-oriented or still provided and fed in in a channel-oriented manner. Then the audio signals do not come from the scene protocol, which then only performs control tasks.
• the fed-in audio data is then possibly converted into an object-oriented representation by the device 126 and then fed internally to the imaging device 18, which then carries out the object / channel mapping. All audio connections between the modules can be switched by a matrix module 128 in order to connect corresponding channels to corresponding channels as required by the central controller 120.
• the user has the option of feeding 64 input channels with signals for virtual sources into the audio processing device 12, so there are 64 input channels EK1-EK in this exemplary embodiment.
• Existing consoles can thus be used as user interfaces for premixing the virtual source signals.
• the spatial mixing is then carried out by the wave field synthesis authoring system and in particular by the heart, the WFS renderer 122.
• the complete scene description is stored in the provision device 10, which is also referred to as a scene protocol.
• the main communication or the required data traffic is carried out by the central controller 120.
• Changes in the scene description such as can be achieved, for example, by the user interface 20 and in particular by a hardware mixing console 200 or a software GUI, that is to say a graphical software user interface 202, are made via a user interface controller 204 of the provision device 10 fed as a changed scene record.
• the imaging device 18 assigns each sound object to a processing channel (input channel) in which the object exists for a specific time.
• a processing channel input channel
• a number of objects exist in chronological order on a specific channel, as has been illustrated with reference to FIGS. 3a, 3b and 6.
• the wave field synthesis renderer has to do the objects don't know yourself. It simply receives signals in the audio channels and a description of the way in which these channels have to be processed.
• the provision device with the scene protocol that is to say with knowledge of the objects and the assigned channels, can transform the object-related metadata (for example the source position) to channel-related metadata and transmit the same to the WFS renderer 122.
• the communication between other modules is carried out by special protocols in such a way that the other modules contain only necessary information, as is shown schematically by the function protocols block 129 in FIG. 4.
• the control module also supports hard disk storage of the scene description. It preferably differentiates between two file formats.
• a file format is an author format where the audio data is stored as uncompressed PCM data.
• session-related information such as a grouping of audio objects, that is to say of sources, layer information, etc., is also used to be stored in a special file format based on XML.
• the other type is the distribution file format.
• audio data can be stored in a compressed manner, and there is no need to additionally store the session-related data.
• the audio objects still exist in this format and that the MPEG-4 standard can be used for distribution.
• the one or more wave field synthesis renderer modules 122 are usually supplied with virtual source signals and a channel-oriented scene description.
• a wave field synthesis renderer calculates the driver signal for each speaker, i.e. a speaker signal of the speaker signals 14 of Fig. 4.
• the wave field synthesis renderer will also calculate signals for sobwoofer speakers, which are also required to the wave field synthesis system to support at low frequencies.
• Room simulation signals from the room simulation module 124 are rendered using a number (usually 8 to 12) of static plane waves. Based on this concept, it is possible to integrate different solutions for room simulation. Without using the room simulation module 124, the wave field synthesis system generates already acceptable sound images with stable perception of the source direction for the listening area.
• a room simulation model is used which reproduces wall reflections, which are modeled, for example, in such a way that a mirror source model is used to generate the early reflections.
• These mirror sources can in turn be treated as audio objects of the scene protocol or can actually only be added by the audio processing device itself.
• the recording / playback tools 126 are a useful addition. Sound objects that are ready for mixing in a conventional manner during premixing, so that only the spatial mixing needs to be performed, can be done from the conventional mixer an audio object Playback device.
• an audio recording module which records the output channels of the mixer in a time code-controlled manner and stores the audio data on the playback module.
• the playback module is received a start time code to play a particular audio object in connection with a respective output channel which is supplied to the playback device 126 by the imaging device 18.
• the recording / playback device can start and stop the playback of individual audio objects independently of one another, depending on the description of the start time and the stop time which is assigned to an audio object.
• the audio content can be taken from the playback device module and exported to the distribution file format.
• the distribution file format thus contains a finished scene report of a completely mixed scene.
• the aim of the user interface concept according to the invention is to implement a hierarchical structure which is adapted to the tasks of the cinema mixing process.
• an audio object is understood as a source that exists as a representation of the individual audio object for a given time.
• a start time and a stop / end time are typical for a source, i.e. for an audio object.
• the source or audio object requires system resources during the time the object or source "lives".
• Each sound source preferably includes metadata in addition to the start time and the stop time.
• This metadata is "type” (a plane wave or point source at a given time), "direction”, “volume”, “mute” and “flags” for directional loudness and directional delay. All of these metadata can be used automatically ,
• the authoring system according to the invention also serves the conventional channel concept in that, for. B. Ob- objects that are "alive” over the entire film or generally over the entire scene also get their own channel. This means that these objects are in principle simple channels in a 1: 1 implementation, as set out in FIG. 6 will represent.
• At least two objects can be grouped. For each group it is possible to choose which parameters should be grouped and how they should be calculated using the master of the group. Groups of sound sources exist for a given time, which is defined by the start time and the end time of the members.
• groups are to use them for standard virtual surround setups. These could be used for virtual fading out of a scene or for virtual zooming in on a scene. Alternatively, the grouping can also be used to integrate surround reverberation effects and record them in a WFS mix.
• Pre Dubs can be simulated in the audio workstation using layers. Layers can also be used to change display attributes during the authoring process, for example to show or hide different parts of the current mixed item.
• a scene consists of all the components previously discussed for a given period of time. This period could be a film reel or z. B. be the entire film, or else only z. B. a film section of certain duration, such as five minutes.
• the scene consists of a number of layers, groups and sources that belong to the scene.
• the complete user interface 20 should include both a graphics software part and a hardware part to allow haptic control.
• the user interface could also be completely implemented as a software module for cost reasons.
• a design concept for the graphic system is used, which is based on so-called "spaces". There are a small number of different spaces in the user interface. Each space is a special editing environment that shows the project from a different approach, with all tools for There are no more windows to look at, all the tools needed for an environment are in the space.
• the adaptive mixing space already described with reference to FIGS. 3a and 3b is used. It can be compared to a conventional mixer that only shows the active channels.
• audio object information is also presented instead of the pure channel information.
• these objects are assigned to input channels of the WFS rendering unit by the imaging device 18 of FIG. 1.
• timeline space which provides an overview of all input channels. Each channel is represented with its corresponding objects. The user has the option of object-to-channel mapping to be used, although automatic channel assignment is preferred for reasons of simplicity.
• Another space is the positioning and editing space, which shows the scene in a three-dimensional view. This space should enable the user to record or edit movements of the source objects. Movements can be generated using, for example, a joystick or using other input / display devices, as are known for graphic user interfaces.
• each room is described by a specific parameter set that is stored in a room preset library.
• different types of parameter sets as well as different graphical user interfaces can be used.
• the method according to the invention for generating an audio representation can be implemented in hardware or in software.
• the implementation can take place on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which can cooperate with a programmed computer system in such a way that the method according to the invention is carried out.
• the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer.
• the invention is thus also a computer program with a program code for executing the method when the computer program runs on a computer.

Landscapes

• Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Acoustics & Sound (AREA)
• Signal Processing (AREA)
• Health & Medical Sciences (AREA)
• Multimedia (AREA)
• Human Computer Interaction (AREA)
• Audiology, Speech & Language Pathology (AREA)
• Computational Linguistics (AREA)
• General Health & Medical Sciences (AREA)
• Otolaryngology (AREA)
• Mathematical Physics (AREA)
• Stereophonic System (AREA)
• Indexing, Searching, Synchronizing, And The Amount Of Synchronization Travel Of Record Carriers (AREA)
• Optical Recording Or Reproduction (AREA)

Priority Applications (1)

Applications Claiming Priority (4)

Publications (2)

Family

ID=34178382

Family Applications (1)

Country Status (7)

Families Citing this family (39)

Family Cites Families (12)

• 2003
  • 2003-09-25 DE DE10344638A patent/DE10344638A1/de not_active Ceased
• 2004
  • 2004-08-02 CN CNB2004800264019A patent/CN100508650C/zh active Active
  • 2004-08-02 JP JP2006522307A patent/JP4263217B2/ja active Active
  • 2004-08-02 WO PCT/EP2004/008646 patent/WO2005017877A2/fr active IP Right Grant
  • 2004-08-02 AT AT04763715T patent/ATE390824T1/de not_active IP Right Cessation
  • 2004-08-02 EP EP04763715A patent/EP1652405B1/fr active Active
  • 2004-08-04 US US10/912,276 patent/US7680288B2/en active Active

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events