EP1652405B1 - Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio - Google Patents

Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio Download PDF

Info

Publication number
EP1652405B1
EP1652405B1 EP04763715A EP04763715A EP1652405B1 EP 1652405 B1 EP1652405 B1 EP 1652405B1 EP 04763715 A EP04763715 A EP 04763715A EP 04763715 A EP04763715 A EP 04763715A EP 1652405 B1 EP1652405 B1 EP 1652405B1
Authority
EP
European Patent Office
Prior art keywords
audio
user interface
channel
scene
time instant
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP04763715A
Other languages
German (de)
English (en)
Other versions
EP1652405A2 (fr
Inventor
Sandra Brix
Frank Melchior
Jan Langhammer
Thomas Röder
Kathrin Reichelt
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority to EP04763715A priority Critical patent/EP1652405B1/fr
Publication of EP1652405A2 publication Critical patent/EP1652405A2/fr
Application granted granted Critical
Publication of EP1652405B1 publication Critical patent/EP1652405B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
    • G10L19/04—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using predictive techniques
    • G10L19/16—Vocoder architecture
    • G10L19/18—Vocoders using multiple modes
    • G10L19/20—Vocoders using multiple modes using sound class specific coding, hybrid encoders or object based coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers, loudspeakers or microphones
    • H04R3/12—Circuits for transducers, loudspeakers or microphones for distributing signals to two or more loudspeakers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/13—Application of wave-field synthesis in stereophonic audio systems

Definitions

  • the present invention is in the field of wave field synthesis and more particularly relates to apparatus and methods for generating, storing or manipulating an audio representation of an audio scene.
  • WFS Wave Field Synthesis
  • Applied to the acoustics can be simulated by a large number of speakers, which are arranged side by side (a so-called speaker array), any shape of an incoming wavefront.
  • a so-called speaker array any shape of an incoming wavefront.
  • the audio signals of each speaker must be fed with a time delay and amplitude scaling so that the radiated sound fields of each speaker properly overlap.
  • the contribution to each speaker is calculated separately for each source and the resulting signals added together. If the sources to be reproduced are in a room with reflective walls, reflections must also be reproduced as additional sources via the loudspeaker array. The cost of the calculation therefore depends heavily on the number of sound sources, the reflection characteristics of the recording room and the number of speakers.
  • the advantage of this technique is in particular that a natural spatial sound impression over a large area of the playback room is possible.
  • the direction and distance of sound sources are reproduced very accurately.
  • virtual sound sources can even be positioned between the real speaker array and the listener.
  • wavefield synthesis works well for environments whose characteristics are known, irregularities occur when the texture changes, or when wave field synthesis is performed based on environmental conditions that do not match the actual nature of the environment.
  • the technique of wave field synthesis can also be used advantageously to supplement a visual perception with a corresponding spatial audio perception.
  • production in virtual studios focused on providing an authentic visual impression of the virtual scene.
  • the matching to the image acoustic impression is usually impressed by manual operations in the so-called post-production subsequently the audio signal or classified as too complex and time-consuming in the realization and therefore neglected. This usually leads to a contradiction of the individual sense sensations, which leads to the space being designed, i. H. the designed scene is perceived as less authentic.
  • the audio material is a movie of a plurality of audio objects.
  • An audio object is a sound source in the film setting. For example, when thinking of a movie scene in which two people and are in a dialogue, while z. For example, when approaching a rider and a train, there are a total of four sound sources in this scene over a period of time, namely the two persons, the approaching rider, and the approaching train. If it is assumed that the two persons in dialogue are not talking at the same time, then at least two audio objects are likely to be active at any one time, namely the rider and the train, if at that time both persons are silent.
  • an audio object is such that the audio object describes a sound source in a film setting that is active or "alive" at a particular time. This means that an audio object is still characterized by a start time and an end time. For example, in the previous example, the tab and the train are active throughout the setting. As the two approach, the listener will perceive this by the louder sounds of the rider and the train becoming louder, and possibly - in an optimal wave field synthesis setting - also changing the positions of those sound sources accordingly.
  • the two speakers in dialogue are constantly generating new audio objects, because whenever a speaker stops speaking the current audio object is over and when the other speaker starts to speak, a new audio object is started, which in turn ends is when the other speaker stops talking, and then when the first speaker starts speaking again, a new audio object is started again.
  • the wave-field synthesis renderer is effectively the "heart" of a wave-field synthesis system that computes the loudspeaker signals for the many loudspeakers of the loudspeaker array in amplitude and phase so that the user not only has an optimal visual impression but also an optimal acoustic impression.
  • Rendering systems usually have fixed speaker positions, such as the left channel in the case of 5.1 ("left"), the center channel, the right channel, the surround left channel (“surround left”) and the surround right channel (“surround right”).
  • fixed (few) positions the ideal sound image sought by the sound engineer is limited to a small number of seats, the so-called sweet spot.
  • phantom sources between the above-mentioned 5.1 positions leads in certain cases to improvements, but not always to satisfactory results.
  • the sound of a movie usually consists of dialogues, effects, atmospheres and music. Each of these elements is mixed in consideration of the limitations of 5.1 and 7.1 systems. Typically, the dialog is merged into the center channel (in 7.1 systems also in a half-left and a half-right position). This implies that when the actor moves across the screen, the sound does not follow. Motion sound effects can only be realized if they move quickly, so that the listener is unable to detect when the sound is passing from one speaker to another.
  • Lateral sources also can not be positioned due to the large audible gap between the front and surround speakers, so that objects can not move slowly from back to front and vice versa.
  • surround speakers are placed in a diffuse array of loudspeakers, thus creating a sound image that is a kind of envelope for the listener. Therefore, accurately positioned sound sources behind the listeners are avoided to avoid the unpleasant sonic interference field associated with such accurately positioned sources.
  • Wave field synthesis as a completely new way of building the sound field perceived by the listener overcomes these essential shortcomings.
  • the consequence for cinema applications is that an accurate sound image can be achieved without limitations with regard to a two-dimensional positioning of objects. This opens up a wide variety of possibilities in the design and mixing of sound for cinema use. Due to the complete sound image reproduction, which is achieved by the technique of wave field synthesis, now sound sources can be freely positioned. Further, sound sources may be placed as focused sources within the audience room as well as outside the audience room.
  • stable sound source directions and stable sound source positions can be generated using point-shaped radiating sources or plane waves.
  • sound sources can be moved freely within, outside, or through the audience room.
  • the sound design ie the activity of the sound engineer
  • the coding format or the number of speakers ie 5.1 systems or 7.1 systems
  • a special sound system also requires a special encoding format.
  • the channels do not matter to a viewer / listener. He does not care which sound system produces a sound, whether an original sound description was object-oriented, channel-oriented, etc. The listener also does not care if and how an audio setting has been mixed. All that counts for the listener is the sound impression, so whether he likes a sound setting for a film or a sound setting without a film or not.
  • existing wave-field synthesis rendering units are used to channel-orientate work so that they have a certain number of input channels, from which, when in the input channels the audio signals and associated information are entered, the speaker signals for each speaker or groups of speakers of a wave field synthesis speaker array are generated.
  • the technique of wave field synthesis makes an audio scene much more "transparent" in that, in principle, an unlimited number of audio objects viewed over a movie, that is, viewed over an audio scene, may be present.
  • this can become problematic if the number of audio objects in an audio scene exceeds the typically always predetermined maximum number of input channels of the audio processing device.
  • the object of the present invention is to provide a concept for generating, storing or editing an audio presentation of an audio scene that has a high acceptance on the part of the users for whom corresponding tools are intended.
  • the present invention is based on the finding that for audio objects, as they occur in a typical film setting, only an object-oriented description can be processed clearly and efficiently.
  • the object-oriented description of the audio scene with objects that have an audio signal and to which a defined start and a defined end time are assigned correspond to the typical conditions in the real world, in which it is rare that a noise the whole There is time. Instead, it is customary, for example in a dialogue, for a dialogue partner to start talking and ceasing to speak or for sounds to typically have a beginning and an end.
  • the object-oriented audio scene description which assigns each sound source in real life its own object, adapted to the natural conditions and therefore optimal in terms of transparency, clarity, efficiency and clarity.
  • z. Tonauer for example, who want to create an audio presentation of an audio scene, who want to incorporate their creative potential to "synthesize" an audio presentation of a audio scene in a movie theater possibly taking into account special audio effects, because of the channel paradigm accustomed typically to work with either hardware or software-implemented mixers, which are a consistent implementation of the channel-oriented operation.
  • hardware- or software-implemented mixers each channel has knobs, knobs, etc., which manipulate the audio signal in that channel.
  • a balance is made between the object-oriented audio presentation that is life-affirming, and the channel-oriented presentation provided to the sound engineer is achieved in that an imaging device is used to image the object-oriented description of the audio scene to a plurality of input channels of an audio processing device, such as a wave field synthesis rendering unit.
  • the mapping means is adapted to assign a first audio object to an input channel and to assign a second audio object whose start time is after an end time of the first audio object to the same input channel and a third audio object whose start time is after the start time of the first audio object and before the end time of the first audio object is to assign to another of the plurality of input channels.
  • This timing which assigns concurrent audio objects to different input channels of the wave field synthesis rendering unit, but which assigns sequentially occurring audio objects to the same input channel, has proven to be extremely channel efficient.
  • the user for example the sound engineer, can get a quick overview of the complexity of an audio scene at a specific time without having to search laboriously from a plurality of input channels, which object is currently active or which object is currently not active.
  • the user can easily perform a manipulation of the audio objects as in object-oriented representation by his usual channel controller.
  • inventive concept which is based on the mapping of the object-oriented audio approach into a channel-oriented rendering approach, thus meets all requirements.
  • object-oriented description of an audio scene as it has been done, is best adapted to nature and therefore efficient and clear.
  • the habits and needs of the users are taken into account, in that the technique is directed to the users and not vice versa.
  • Fig. 1 shows a block diagram of a device according to the invention for generating an audio presentation of an audio scene.
  • the inventive apparatus comprises means 10 for providing an object-oriented description of the audio scene, wherein the object-oriented description of the audio scene comprises a plurality of audio objects, wherein an audio object is assigned at least an audio signal, a start time and an end time.
  • the device according to the invention further comprises an audio processing device 12 for generating a plurality of loudspeaker signals LSi 14 which is channel-oriented and which generates the plurality of loudspeaker signals 14 from a plurality of input channels EKi.
  • An imaging device 18 for mapping the object-oriented description of the audio scene onto the plurality of input channels 16 of the channel-oriented audio signal processing device 12 is located between the providing device 10 and the channel-oriented audio signal processing device, which is embodied, for example, as a WFS rendering unit the mapping means 18 is adapted to assign a first audio object to an input channel, such as EK1, and to assign a second audio object whose start time is after an end time of the first audio object to the same input channel, such as the input channel EK1, and a third one Audio object whose start time is after the start time of the first audio object and before the end time of the first audio object to assign another input channel of the plurality of input channels, such as the input channel EK2.
  • the mapping device 18 is thus designed to assign temporally non-overlapping audio objects to the same input channel, and to assign overlapping audio objects to different parallel input channels.
  • the audio objects are further specified to be associated with a virtual position.
  • This virtual position of an object may change during the lifetime of the object, which would correspond to the case where, for example, a rider approaches a scene center, such that the rider's gallop gets louder and louder and closer to the auditorium.
  • an audio object includes not only the audio signal associated with that audio object and a start time and an end time, but additionally a position of the virtual source which may change over time and, optionally, other properties of the audio object such as ob It should have point source characteristics or whether it should emit a plane wave, which would correspond to a virtual position with infinite distance to the viewer.
  • further properties for sound sources, ie for audio objects are known, depending on the features of the channel-oriented audio signal processing device 12 of Fig. 1 can be considered.
  • the structure of the device is hierarchical in that the channel-oriented audio signal processing means for receiving audio objects is not directly combined with the means for providing, but combined with the same via the mapping means.
  • the entire audio scene is to be known and stored only in the means for providing, but the mapping device, and still less the channel-oriented audio signal processing device, must already have knowledge of the overall audio setting. Instead Both the mapping device 18 and the audio signal processing device 12 operate under the direction of the audio scene provided by the providing device 10.
  • the in Fig. 1 shown device further provided with a user interface, as shown in Fig. 2 at 20 is shown.
  • the user interface 20 is configured to have one user interface channel per input channel, and preferably one manipulator for each user interface channel.
  • the user interface 20 is coupled via its user interface input 22 to the mapping device 18 to obtain the mapping information from the mapping device, since the assignment of the input channels EK1 to EKm is to be displayed by the user interface 20.
  • the user interface 20 On the output side, if the user interface 20 has the manipulator feature for each user interface channel, it is coupled to the device 10 for providing.
  • the user interface 20 is configured to provide, via its user interface output 24 with regard to the original version, manipulated audio objects of the device 10 for providing, thus receiving a changed audio scene which is then returned to the mapping device 18 and, correspondingly to the input channels Channel-oriented audio signal processing device 12 is provided.
  • the user interface 20 is designed as a user interface, as shown in FIG Fig. 3a that is, as a user interface that always represents only the current objects.
  • the user interface 20 is configured to operate as in FIG Fig. 3b to be constructed, so that always all objects are displayed in an input channel.
  • a time line 30 is shown, which comprises the objects A, B, C in chronological order, wherein the object A includes a start time 31a and an end time 31b.
  • the object A includes a start time 31a and an end time 31b.
  • the end time 31b of the first object A coincides with a start time of the second object B, which in turn has an end time 32b, which in turn coincides coincidentally with a start time of the third object C, which in turn has an end time 33b.
  • the start times 32a and 33b correspond to the end times 31b and 32b and are in the Fig. 3a, 3b not shown for clarity.
  • a mixer channel icon 34 is shown, which includes a slider 35 and stylized buttons 36, over the properties of the audio signal of the object B or virtual positions, etc. can be changed.
  • the stylized channel representation 34 would not display the object B but the object C.
  • the user interface in FIG Fig. 3a would then, if z. B. an object D would take place simultaneously to the object B, another channel, such as the input channel i + 1, represent.
  • another channel such as the input channel i + 1, represent.
  • FIG. 3a The illustration shown provides the sound engineer with a simple overview of the number of parallel audio objects at a time, ie the number of active channels that are even displayed. Non-active input channels are used at the in Fig. 3a shown embodiment of the user interface 20 of Fig. 2 not displayed at all.
  • the input channel i to which the channels assigned temporally in chronological order belong, is shown in triplicate, once as an object channel A, on another occasion as an object channel B and again as an object channel C according to the invention. It is preferred to use the channel, such as the input channel i for the object B (reference numeral 38 in FIG Fig. 3b ) z. B. highlight color or brightness, on the one hand to give the sound engineer a clear overview of which object is currently being fed to the respective channel i, and which objects z. B.
  • the user interface 20 of Fig. 2 and in particular the expressions thereof in Fig. 3a and Fig. 3b are thus designed to provide a visual representation as desired for the "occupancy" of the input channels of the channel-oriented audio signal processing device generated by the imaging device 18.
  • FIG. 5 a simple example of the functionality of the imaging device 18 of Fig. 1 given.
  • Fig. 5 shows an audio scene with different audio objects A, B, C, D, E, F and G.
  • the objects A, B, C and D overlap in time.
  • these objects A, B, C and D are all active at a particular time 50.
  • the object E does not overlap with the_objects A, B.
  • the object E overlaps only with the objects D and C, as can be seen at a time 52.
  • overlapping the object F and the object D as it is at a time 54 z. B. can be seen.
  • the same applies to the objects F and G, the z. B. overlap at a time 56, while the object G does not overlap with the objects A, B, C, D and E.
  • FIG. 5 shown example assign each audio object to an input channel so that the 1: 1 conversion left in the table in Fig. 6 would be obtained.
  • a disadvantage of this concept is that many input channels are needed or that, when there are many audio objects, which is the case very quickly in a movie, the number of input channels of the wave field synthesis rendering unit is the number of processable virtual sources in one limited in the real film setting, which of course is not desirable because technology limits should not affect the creative potential.
  • this 1: 1 conversion is very confusing, in that at some point each input channel typically receives an audio object, but when a particular audio scene is viewed, typically relatively few input channels are active, but the user can not easily determine this because he always has all the audio channels at a glance.
  • this concept of 1: 1 assignment of audio objects to input channels of the audio processor means that in order to minimize or limit the number of audio objects, audio processing equipment having a very high number of input channels must be provided Immediately increase the computational complexity, the required computing power and the required storage capacity of the audio processing device to calculate the individual loudspeaker signals, which directly results in a higher price of such a system.
  • FIG Fig. 6 The inventive assignment object channel of in Fig. 5 as shown by the imaging device 18 according to the present invention is shown in FIG Fig. 6 shown in the right section of the table.
  • the parallel audio objects A, B, C and D are sequentially assigned to the input channels EK1, EK2, EK3 and EK4.
  • the object E no longer has to, as in the left half of Fig. 6 assigned to the input channel EK5, but can be assigned to a free channel, such as the input channel EK1 or, as indicated by the bracket, the input channel EK2.
  • the same applies to the object G which can also be assigned to all channels except the channel to which the object F was previously allocated (in the example the input channel EK1).
  • the imaging device 18 is designed to always occupy channels with the lowest possible atomic number, and to occupy always adjacent input channels EKi and EKi + 1, so that no holes.
  • this "neighborhood feature" is not essential, as it does not matter to a user of the audio authoring system according to the present invention whether he is currently serving the first, seventh, or any other input channel of the audio processing device, as long as he is through the inventive user interface is enabled to manipulate precisely this channel, for example by a controller 35 or by buttons 36 a mixer channel display 34 of the current channel.
  • the user interface channel i need not necessarily discuss the input channel i, but it can also be so far a channel assignment done, such that the user interface channel i z. B. the input channel EKm corresponds, while the user interface channel i + 1 corresponds to the input channel k etc.
  • the user interface concept of the present invention may also be applied to an existing hardware mixing console that includes actual hardware controls and hardware buttons that a master will manually operate to achieve optimal audio mixing.
  • An advantage of the present invention is that even such a sound mixer typically very familiar and heart-growing hardware mixing console can also be used by z. B. by typically present on the mixing console indicators, such as LEDs are always the current channels for the sound engineer clearly marked.
  • the present invention is further flexible in that cases can also be dealt with where the wave field synthesis loudspeaker setup used for production is covered by the reproduction setup e.g. B. differs in a movie theater. Therefore, according to the invention, the audio content is encoded in a format that can be processed by various systems. This format is the audio scene, i. H. the object-oriented audio presentation and not the speaker signal presentation.
  • the treatment process is understood as an adaptation of the content to the reproduction system.
  • not only a few master channels but an entire object-oriented scene description are processed in the wave field synthesis reproduction process.
  • the scenes are processed for each reproduction. This is typically done in real time to adapt to the current situation.
  • this adaptation takes into account the number of speakers and their positions, the characteristics of the reproduction system, such as the frequency response, the sound pressure level, etc., the room acoustics conditions or other image reproduction conditions.
  • a major difference in the wave field synthesis mix compared to the channel-based approach of current systems consists in the freely available positioning of the sound objects.
  • the position of the sound sources is relatively coded. This is important for blending concepts that pertain to visual content, such as movies, because positioning of the sound sources relative to the image is attempted by a proper system setup.
  • the wave field synthesis system requires absolute positions for the sound objects, which is given as additional information about the audio signal of an audio object to this audio object in addition to the start time and the end time of this audio object.
  • the goal of re-engineering the post-production process is to minimize user training and integrate the integration of the new system of the invention into existing users' knowledge.
  • all tracks or objects to be rendered at different positions will exist within the master file / distribution format, unlike conventional production facilities that are optimized to reduce the number of tracks during the production process.
  • the wave field synthesis authoring tool according to the present invention is implemented as a workstation having the ability to record the audio signals of the final mix and to convert them to the distribution format in another step.
  • two aspects are considered according to the invention. The first is that all audio objects or tracks still exist in the final master. The second aspect is that the positioning is not performed in the mixing console. This means that the so-called authoring, so the Tonmeister post-processing is one of the last steps in the production chain.
  • the wave field synthesis authoring system ie the inventive device for generating an audio presentation, is implemented as a standalone workstation that can be integrated into different production environments by feeding audio outputs from the mixer into the system.
  • the mixer represents the user interface coupled to the device for generating the audio presentation of an audio scene.
  • Fig. 4 The system according to the invention according to a preferred embodiment of the present invention is disclosed in Fig. 4 shown. Same reference numerals as in Fig. 1 or 2 indicate the same elements.
  • the basic system design is based on the goal of modularity and the ability to integrate existing mixing consoles into the inventive wave field synthesis authoring system as user interfaces.
  • a central controller 120 is formed, which communicates with other modules. This allows the use of alternatives for certain modules as long as they all use the same communication protocol. If that is in Fig. 4
  • the system shown is considered a black box, one generally sees a number of inputs (from the provisioning means 10) and a number of outputs (loudspeaker signals 14) and the user interface 20.
  • the actual WFS Renderer 122 Integrated in this black box next to the user interface is the actual WFS Renderer 122, which performs the actual wave field synthesis calculation of the loudspeaker signals using various input information.
  • a space simulation module 124 configured to perform certain room simulations that are used to create room characteristics of a recording room or to manipulate room characteristics of a recording room.
  • an audio recording device 126 and a recording reproduction device are provided.
  • the device 126 is preferably provided with an external input.
  • the entire audio signal is either already object-oriented or channel-oriented provided and fed. Then the audio signals do not come from the scene protocol, which then only perceives control tasks.
  • the input audio data is then converted from the device 126, if necessary, into an object-oriented representation and then supplied internally to the imaging device 18, which then performs the object / channel mapping.
  • All audio connections between the modules are switchable by a matrix module 128 to connect corresponding channels to corresponding channels as required by the central controller 120.
  • the user has the ability to feed 64 input channels of virtual source signals to the audio processor 12, thus having 64 input channels EK1-EKm in this embodiment. This allows existing consoles to be used as user interfaces for premixing the source virtual signals. Spatial mixing is then performed by the wave-field synthesis authoring system and, in particular, by the heart, WFS renderer 122.
  • the complete scene description is stored in the provisioning facility 10, also referred to as a scene log.
  • the main communication or the required data traffic is performed by the central controller 120.
  • Changes in the scene description such as may be achieved by the user interface 20, and in particular by a hardware mixing console 200 or a software GUI, that is, a graphical software user interface 202, are provided via a user interface controller 204 of the providing device 10 as a modified scene protocol fed.
  • the mapping device 18 assigns each switch object to a render channel (input channel) in which the object exists for a certain time.
  • a render channel input channel
  • a number of objects exist in chronological order on a particular channel as determined by the Fig. 3a, 3b and 6 has been shown.
  • the wavefield synthesis renderer must retrieve the objects do not know yourself. It simply receives signals in the audio channels and a description of how these channels need to be processed.
  • the scene protocol providing means that is, the knowledge of the objects and the associated channels, may perform a transformation of the object-related metadata (eg, the source position) to channel-related metadata and transmit them to the WFS renderer 122.
  • the communication between other modules is performed by special protocols in such a way that the other modules contain only necessary information, as indicated schematically by the block function protocols 129 in FIG Fig. 4 is shown.
  • the control module further supports the hard disk storage of the scene description. It preferably distinguishes between two file formats.
  • a file format is an author format where the audio data is stored as uncompressed PCM data.
  • session-related information such as a grouping of audio objects, that is, sources, layer information, etc., is also used to be stored in a special file format based on XML.
  • the other type is the distribution file format.
  • audio data can be stored in a compressed manner and there is no need to additionally store the session-related data.
  • the audio objects still exist in this format and that the MPEG-4 standard can be used for distribution.
  • the one or more wave-field synthesis renderer modules 122 are typically supplied with source virtual signals and a channel-oriented scene description.
  • a wave field synthesis renderer calculates according to the wave field synthesis theory, the driver signal for each speaker, so a speaker signal of the speaker signals 14 of Fig. 4 .
  • the wave field synthesis renderer will also compute signals for Sobwoofer speakers, which are also needed to support the wave field synthesis system at low frequencies.
  • Room simulation signals from the room simulation module 124 are rendered using a number (typically 8 to 12) of static plane waves. Based on this concept, it is possible to integrate different solutions for room simulation. Without using the space simulation module 124, the wave-field synthesis system already produces acceptable sound images with stable perception of the source direction for the listening area.
  • a space simulation model is used that reproduces wall reflections modeled, for example, such that a mirror source model is used to generate the early reflections.
  • These mirror sources may be treated as audio objects of the scene protocol, or may actually be added by the audio processor itself.
  • the record / playback tools 126 are a useful complement. Scaling objects that are ready to be blended in a conventional manner during premixing, so that only spatial mixing needs to be performed, can be converted from the conventional mixer to an audio object reproducing apparatus be fed.
  • an audio recording module which records the output channels of the mixer in a time code controlled manner and stores the audio data on the playback module.
  • the rendering module will receive a start time code to play a particular audio object in conjunction with a respective output channel supplied to the player 126 from the imaging device 18.
  • the recording / playback device can independently start and stop the playback of individual audio objects, depending on the description of the start time and the stop time point associated with an audio object.
  • an audio object is considered as a source that exists as a representation of the individual audio object for a given time.
  • a start time and a stop / end time are typical for a source, ie for an audio object.
  • the source or audio object needs resources of the system during the time that the object or source "lives".
  • each sound source includes metadata in addition to the start time and the stop time.
  • metadata are "type” (a level wave or point source at a given time), "direction,” “volume,” “mute,” and “flags” for directional loudness and directional delay. All of this metadata can be used automatically.
  • the authoring system according to the invention also serves the conventional channel concept in that z. B. objects, who are "alive” throughout the film or generally over the entire scene, also get their own channel. This means that these objects are in principle simple channels in 1: 1 implementation as they are based on Fig. 6 is presented.
  • At least two objects may be grouped. For each group it is possible to choose which parameters should be grouped and how they should be calculated using the master of the group. Groups of sound sources exist for a given time, which is defined by the start time and the end time of the members.
  • groups are to use them for standard virtual surround setups. These could be used for fading out of a scene or zooming in on a scene. Alternatively, the grouping can also be used to integrate surround reverb effects and record into a WFS mix.
  • a layer or layer In order to structure a mixture or a scene, in a preferred embodiment of the present invention groups and sources are arranged in different layers. Using layers, pre-dubs can be simulated in the audio workstation. Layers can also be used to change display attributes during the authoring process, for example to show or hide different parts of the current mix item.
  • a scene consists of all previously discussed components for a given period of time. This period could be a film spool or z. B. be the entire movie, or else only z. B. a movie section of certain duration, such as five minutes.
  • the scene again consists of a number of layers, groups, and sources that belong to the scene.
  • the complete user interface 20 should include both a graphics software part and a hardware part to allow haptic control.
  • the user interface could also be fully implemented as a software module for cost reasons.
  • a design concept for the graphical system is used, which is based on so-called "spaces".
  • spaces There are a small number of different spaces in the user interface.
  • Each space is a special editing environment that presents the project from a different approach, with all the tools needed for a space. Therefore, you no longer have to pay attention to different windows. All tools needed for an environment are in the appropriate space.
  • Fig. 3a and 3b described adaptive mixing space used. It can be compared to a conventional mixer that only displays the active channels.
  • the adaptive mixing space also presents audio object information instead of pure channel information. As has been shown, these objects are represented by the imaging device 18 of FIG Fig. 1 Input channels assigned to the WFS rendering unit.
  • timeline space which provides an overview of all input channels. Each channel is displayed with its corresponding objects. The user has the option of object-to-channel assignment although for simplicity, automatic channel assignment is preferred.
  • Another space is the positioning and editing space, which shows the scene in a three-dimensional view. This space is to enable the user to record or edit movements of the source objects. Movements can be generated using, for example, a joystick or other input / display devices known for graphical user interfaces.
  • each room is described by a particular set of parameters stored in a Room Presets library. Depending on the room model, different types of parameter sets as well as different graphical user interfaces can be used.
  • the inventive method for generating an audio representation in hardware or in software can be implemented.
  • the implementation may be on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which may interact with a programmed computer system so as to carry out the inventive method.
  • the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method according to the invention, when the computer program product runs on a computer.
  • the invention is thus also a computer program with a program code for carrying out the method when the computer program runs on a computer.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Mathematical Physics (AREA)
  • General Health & Medical Sciences (AREA)
  • Otolaryngology (AREA)
  • Stereophonic System (AREA)
  • Indexing, Searching, Synchronizing, And The Amount Of Synchronization Travel Of Record Carriers (AREA)
  • Optical Recording Or Reproduction (AREA)

Claims (16)

  1. Dispositif de génération, de mise en mémoire ou de traitement d'une représentation audio d'une scène audio, aux caractéristiques suivantes:
    un moyen de traitement audio (12) destiné à générer une pluralité de signaux de haut-parleur à partir d'une pluralité de canaux d'entrée (EK1, EK2, ..., EKm) (16);
    un moyen (10) destiné à préparer une description de la scène audio orientée sur l'objet, la description de la scène audio orientée sur l'objet comprenant une pluralité d'objets audio, à un objet audio étant associés un signal audio, un moment de début et un moment de fin; et
    un moyen de reproduction (18) destiné à reproduire la description de la scène audio orientée sur l'objet sur la pluralité de canaux d'entrée du moyen de traitement audio, le moyen de reproduction étant réalisé de manière à attribuer un premier objet audio à un canal d'entrée, et à attribuer un deuxième objet audio dont le moment de début se situe après un moment de fin du premier objet audio au même canal d'entrée, et à attribuer un troisième objet audio dont le moment de début se situe après le moment de début du premier objet audio et avant le moment de fin du premier objet audio à un autre de la pluralité de canaux d'entrée.
  2. Dispositif selon la revendication 1, dans lequel le moyen de traitement audio (12) comprend un moyen de synthèse de champ d'onde (122) réalisé de manière à calculer, en connaissance des positions d'une pluralité de haut-parleurs, la pluralité de signaux de haut-parleur pour les haut-parleurs.
  3. Dispositif selon la revendication 1 ou 2, dans lequel il est par ailleurs associé une position virtuelle à un objet audio, et dans lequel le moyen de traitement audio (12) est réalisé de manière à tenir compte, lors de la génération de la pluralité de signaux de haut-parleur, des positions virtuelles des objets audio.
  4. Dispositif selon l'une des revendications précédentes, dans lequel le moyen de traitement audio (12) est couplé exclusivement par l'intermédiaire du moyen de reproduction (18) au moyen (10) destiné à préparer, pour recevoir des données d'objet audio à traiter.
  5. Dispositif selon l'une des revendications précédentes, dans lequel un nombre de canaux d'entrée du moyen de traitement audio est prédéterminé et est inférieur à un nombre admis d'objets audio dans la scène audio, au moins deux objets audio qui ne se recouvrent pas dans le temps étant présents.
  6. Dispositif selon l'une des revendications précédentes, présentant par ailleurs une interface d'utilisateur (20), l'interface d'utilisateur présentant un nombre de canaux d'interface d'utilisateur séparés, un canal d'interface d'utilisateur étant associé à un canal d'entrée du moyen de traitement audio, et l'interface d'utilisateur (20) étant couplée au moyen de reproduction (80), pour identifier à un moment l'objet audio qui est précisément attribué au canal d'interface d'utilisateur.
  7. Dispositif selon la revendication 6, dans lequel l'interface d'utilisateur (20) est réalisé de manière à identifier les canaux d'interface d'utilisateur qui sont associés à des canaux d'entrée du moyen de traitement audio auxquels est précisément attribué un objet audio.
  8. Dispositif selon la revendication 7, dans lequel l'interface d'utilisateur est réalisé sous forme de console de mélange de matériel présentant, pour chaque canal d'interface d'utilisateur, un moyen de manipulation de matériel, et dans lequel à chaque moyen de manipulation de matériel est associé un indicateur, pour identifier un canal d'interface d'utilisateur précisément actif.
  9. Dispositif selon la revendication 7, dans lequel l'interface d'utilisateur présente une interface d'utilisateur graphique qui est réalisée de manière à n'afficher sur un appareil d'affichage électrique que les canaux d'interface d'utilisateur auxquels est associé un canal d'entrée du moyen de traitement audio auquel est précisément attribué un objet audio.
  10. Dispositif selon l'une des revendications 6 à 9, dans lequel l'interface d'utilisateur (20) présente par ailleurs un moyen de manipulation d'un canal d'interface d'utilisateur qui est réalisé de manière à manipuler un objet audio attribué au canal d'entrée du moyen de traitement audio (12) correspondant au canal d'interface d'utilisateur, l'interface d'utilisateur étant couplé au moyen (10) destiné à préparer, pour remplacer un objet audio par une version manipulée de ce dernier, et le moyen de reproduction (18) étant réalisé de manière à attribuer, au lieu de l'objet audio, la version manipulée de ce dernier à un canal d'entrée du moyen de traitement audio (12).
  11. Dispositif selon la revendication 10, dans lequel le moyen de manipulation est réalisé de manière à modifier la position, le type ou le signal audio d'un objet audio.
  12. Dispositif selon l'une des revendications 6 à 9, dans lequel l'interface d'utilisateur est réalisé de manière à représenter, pour un canal d'interface d'utilisateur, une occupation dans le temps, l'occupation dans le temps représentant une séquence dans le temps d'objets audio attribués à un canal d'interface d'utilisateur, et l'interface d'utilisateur étant par ailleurs réalisé de manière à marquer, dans l'occupation dans le temps, un moment actuel (37).
  13. Dispositif selon la revendication 12, dans lequel l'interface d'utilisateur (20) est réalisé de manière à représenter l'occupation dans le temps comme faisceau horaire présentant les objets audio attribués proportionnellement à leur longueur ainsi qu'un indicateur (37) se déplaçant au fur et à mesure que le temps avance.
  14. Dispositif selon l'une des revendications précédentes,
    dans lequel le moyen (10) destiné à préparer est réalisé de manière à permettre un regroupement d'objets audio de sorte que les objets audio regroupés soient marqués par une information de groupe en ce qui concerne leur appartenance à un groupe, et
    le moyen de reproduction (18) étant réalisé de manière à conserver les informations de groupe, de sorte qu'une manipulation d'une propriété de groupe ait un effet sur tous les membres du groupe, indépendamment du canal d'entrée du moyen de traitement audio auquel sont associés les objets audio du groupe.
  15. Procédé de génération, de mise en mémoire ou de traitement d'une représentation audio d'une scène audio, aux étapes suivantes:
    générer (12) une pluralité de signaux de haut-parleur à partir d'une pluralité de canaux d'entrée (EK1, EK2, ..., EKm) (16);
    préparer (10) une description de la scène audio orientée sur l'objet, la description de la scène audio orientée sur l'objet comprenant une pluralité d'objets audio, à un objet audio étant associés un signal audio, un moment de début et un moment de fin; et
    reproduire (18) la description de la scène audio orientée sur l'objet sur la pluralité de canaux d'entrée du moyen de traitement audio en attribuant un premier objet audio à un canal d'entrée, et en attribuant un deuxième objet audio dont le moment de début se situe après un moment de fin du premier objet audio au même canal d'entrée, et en attribuant un troisième objet audio dont le moment de début se situe après le moment de début du premier objet audio et avant le moment de fin du premier objet audio à un autre de la pluralité de canaux d'entrée.
  16. Programme d'ordinateur avec un code de programme pour réaliser le procédé selon la revendication 15 lorsque le programme est exécuté sur un ordinateur.
EP04763715A 2003-08-04 2004-08-02 Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio Active EP1652405B1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP04763715A EP1652405B1 (fr) 2003-08-04 2004-08-02 Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP03017785 2003-08-04
DE10344638A DE10344638A1 (de) 2003-08-04 2003-09-25 Vorrichtung und Verfahren zum Erzeugen, Speichern oder Bearbeiten einer Audiodarstellung einer Audioszene
EP04763715A EP1652405B1 (fr) 2003-08-04 2004-08-02 Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio
PCT/EP2004/008646 WO2005017877A2 (fr) 2003-08-04 2004-08-02 Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio

Publications (2)

Publication Number Publication Date
EP1652405A2 EP1652405A2 (fr) 2006-05-03
EP1652405B1 true EP1652405B1 (fr) 2008-03-26

Family

ID=34178382

Family Applications (1)

Application Number Title Priority Date Filing Date
EP04763715A Active EP1652405B1 (fr) 2003-08-04 2004-08-02 Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio

Country Status (7)

Country Link
US (1) US7680288B2 (fr)
EP (1) EP1652405B1 (fr)
JP (1) JP4263217B2 (fr)
CN (1) CN100508650C (fr)
AT (1) ATE390824T1 (fr)
DE (1) DE10344638A1 (fr)
WO (1) WO2005017877A2 (fr)

Families Citing this family (39)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050058307A1 (en) * 2003-07-12 2005-03-17 Samsung Electronics Co., Ltd. Method and apparatus for constructing audio stream for mixing, and information storage medium
DE102005008342A1 (de) * 2005-02-23 2006-08-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Speichern von Audiodateien
DE102005008343A1 (de) * 2005-02-23 2006-09-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Liefern von Daten in einem Multi-Renderer-System
DE102005008333A1 (de) * 2005-02-23 2006-08-31 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Steuern einer Wellenfeldsynthese-Rendering-Einrichtung
DE102005027978A1 (de) * 2005-06-16 2006-12-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Erzeugen eines Lautsprechersignals aufgrund einer zufällig auftretenden Audioquelle
US8027477B2 (en) * 2005-09-13 2011-09-27 Srs Labs, Inc. Systems and methods for audio processing
JP5265517B2 (ja) * 2006-04-03 2013-08-14 ディーティーエス・エルエルシー オーディオ信号処理
PL2067138T3 (pl) * 2006-09-18 2011-07-29 Koninl Philips Electronics Nv Kodowanie i dekodowanie obiektów audio
EP2575129A1 (fr) 2006-09-29 2013-04-03 Electronics and Telecommunications Research Institute Appareil et procédé de codage et de décodage d'un signal audio à objets multiples ayant divers canaux
US10296561B2 (en) 2006-11-16 2019-05-21 James Andrews Apparatus, method and graphical user interface for providing a sound link for combining, publishing and accessing websites and audio files on the internet
US9361295B1 (en) 2006-11-16 2016-06-07 Christopher C. Andrews Apparatus, method and graphical user interface for providing a sound link for combining, publishing and accessing websites and audio files on the internet
KR102149019B1 (ko) * 2008-04-23 2020-08-28 한국전자통신연구원 객체기반 오디오 컨텐츠의 생성/재생 방법 및 객체기반 오디오 서비스를 위한 파일 포맷 구조를 가진 데이터를 기록한 컴퓨터 판독 가능 기록 매체
KR101724326B1 (ko) * 2008-04-23 2017-04-07 한국전자통신연구원 객체기반 오디오 컨텐츠의 생성/재생 방법 및 객체기반 오디오 서비스를 위한 파일 포맷 구조를 가진 데이터를 기록한 컴퓨터 판독 가능 기록 매체
ES2963744T3 (es) * 2008-10-29 2024-04-01 Dolby Int Ab Protección de recorte de señal usando metadatos de ganancia de audio preexistentes
TWI383383B (zh) * 2008-11-07 2013-01-21 Hon Hai Prec Ind Co Ltd 音頻處理系統
EP2205007B1 (fr) * 2008-12-30 2019-01-09 Dolby International AB Procédé et appareil pour le codage tridimensionnel de champ acoustique et la reconstruction optimale
US8396576B2 (en) * 2009-08-14 2013-03-12 Dts Llc System for adaptively streaming audio objects
US9305550B2 (en) * 2009-12-07 2016-04-05 J. Carl Cooper Dialogue detector and correction
DE102010030534A1 (de) * 2010-06-25 2011-12-29 Iosono Gmbh Vorrichtung zum Veränderung einer Audio-Szene und Vorrichtung zum Erzeugen einer Richtungsfunktion
US9026450B2 (en) 2011-03-09 2015-05-05 Dts Llc System for dynamically creating and rendering audio objects
US8971917B2 (en) * 2011-04-04 2015-03-03 Soundlink, Inc. Location-based network radio production and distribution system
KR102548756B1 (ko) 2011-07-01 2023-06-29 돌비 레버러토리즈 라이쎈싱 코오포레이션 향상된 3d 오디오 오서링과 렌더링을 위한 시스템 및 툴들
US9078091B2 (en) * 2012-05-02 2015-07-07 Nokia Technologies Oy Method and apparatus for generating media based on media elements from multiple locations
WO2013167164A1 (fr) * 2012-05-07 2013-11-14 Imm Sound S.A. Procédé et appareil de reproduction sonore 3d ne dépendant pas du schéma ni du format
US9264840B2 (en) * 2012-05-24 2016-02-16 International Business Machines Corporation Multi-dimensional audio transformations and crossfading
US9613660B2 (en) 2013-04-05 2017-04-04 Dts, Inc. Layered audio reconstruction system
US9666198B2 (en) 2013-05-24 2017-05-30 Dolby International Ab Reconstruction of audio scenes from a downmix
KR101761569B1 (ko) 2013-05-24 2017-07-27 돌비 인터네셔널 에이비 오디오 현장의 코딩
WO2014187990A1 (fr) 2013-05-24 2014-11-27 Dolby International Ab Codage efficace de scènes audio comprenant des objets audio
EP3005353B1 (fr) * 2013-05-24 2017-08-16 Dolby International AB Codage efficace de scènes audio comprenant des objets audio
CN106961647B (zh) 2013-06-10 2018-12-14 株式会社索思未来 音频再生装置以及方法
CN104240711B (zh) 2013-06-18 2019-10-11 杜比实验室特许公司 用于生成自适应音频内容的方法、系统和装置
EP3503095A1 (fr) * 2013-08-28 2019-06-26 Dolby Laboratories Licensing Corp. Amélioration hybride de la parole codée du front d'onde et de paramètres
WO2015150384A1 (fr) * 2014-04-01 2015-10-08 Dolby International Ab Codage efficace de scènes audio comprenant des objets audio
CA3210174A1 (fr) * 2014-05-30 2015-12-03 Sony Corporation Appareil de traitement de l'information et methode de traitement de l'information
EP3254477A1 (fr) 2015-02-03 2017-12-13 Dolby Laboratories Licensing Corporation Construction audio adaptative
US11096004B2 (en) * 2017-01-23 2021-08-17 Nokia Technologies Oy Spatial audio rendering point extension
GB201719854D0 (en) * 2017-11-29 2018-01-10 Univ London Queen Mary Sound effect synthesis
GB201800920D0 (en) 2018-01-19 2018-03-07 Nokia Technologies Oy Associated spatial audio playback

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH01279700A (ja) * 1988-04-30 1989-11-09 Teremateiiku Kokusai Kenkyusho:Kk 音響信号処理装置
JPH04225700A (ja) * 1990-12-27 1992-08-14 Matsushita Electric Ind Co Ltd オーディオ再生装置
JPH06246064A (ja) * 1993-02-23 1994-09-06 Victor Co Of Japan Ltd テレビゲーム機用付加装置
JP3492404B2 (ja) * 1993-12-24 2004-02-03 ローランド株式会社 音響効果装置
US7085387B1 (en) * 1996-11-20 2006-08-01 Metcalf Randall B Sound system and method for capturing and reproducing sounds originating from a plurality of sound sources
KR100637292B1 (ko) * 1997-11-29 2006-10-24 코닌클리케 필립스 일렉트로닉스 엔.브이. 가변 속도의 샘플링된 디지털 오디오 정보를 일련의 균일한 크기를 갖는 복수의 블록에 인터페이스하는 방법 및 장치와, 기록-인터페이스 연결에 의해 제조된 단일 저장매체
US6054989A (en) * 1998-09-14 2000-04-25 Microsoft Corporation Methods, apparatus and data structures for providing a user interface, which exploits spatial memory in three-dimensions, to objects and which provides spatialized audio
GB2349762B (en) * 1999-03-05 2003-06-11 Canon Kk Image processing apparatus
US7149313B1 (en) * 1999-05-17 2006-12-12 Bose Corporation Audio signal processing
EP1209949A1 (fr) * 2000-11-22 2002-05-29 Technische Universiteit Delft Système de reproduction sonore avec synthèse du champ d' ondes en utilisant un panneau en modes distribués
GB0127778D0 (en) * 2001-11-20 2002-01-09 Hewlett Packard Co Audio user interface with dynamic audio labels
US20030035553A1 (en) * 2001-08-10 2003-02-20 Frank Baumgarte Backwards-compatible perceptual coding of spatial cues

Also Published As

Publication number Publication date
ATE390824T1 (de) 2008-04-15
EP1652405A2 (fr) 2006-05-03
WO2005017877A2 (fr) 2005-02-24
JP4263217B2 (ja) 2009-05-13
JP2007501553A (ja) 2007-01-25
US7680288B2 (en) 2010-03-16
DE10344638A1 (de) 2005-03-10
WO2005017877A3 (fr) 2005-04-07
US20050105442A1 (en) 2005-05-19
CN100508650C (zh) 2009-07-01
CN1849845A (zh) 2006-10-18

Similar Documents

Publication Publication Date Title
EP1652405B1 (fr) Dispositif et procede de production, de mise en memoire ou de traitement d'une representation audio d'une scene audio
DE10328335B4 (de) Wellenfeldsyntesevorrichtung und Verfahren zum Treiben eines Arrays von Lautsprechern
EP1844628B1 (fr) Procede et dispositif d'amorçage d'une installation de moteur de rendu de synthese de front d'onde avec objets audio
EP1844627B1 (fr) Dispositif et procédé pour simuler un système de synthèse de champ d'onde
EP1872620B9 (fr) Dispositif et procede pour commander une pluralite de haut-parleurs au moyen d'une interface graphique d'utilisateur
EP1878308B1 (fr) Dispositif et procede de production et de traitement d'effets sonores dans des systemes de reproduction sonore spatiale a l'aide d'une interface graphique d'utilisateur
EP1851998B1 (fr) Dispositif et procédé pour fournir des données dans un système a dispositifs de rendu multiples
DE10254404B4 (de) Audiowiedergabesystem und Verfahren zum Wiedergeben eines Audiosignals
DE19950319A1 (de) Verfahren zum Synthetisieren eines dreidimensionalen Schallfeldes
DE102006017791A1 (de) Wiedergabegerät und Wiedergabeverfahren
WO2007009599A1 (fr) Dispositif et procede de commande d'une pluralite de haut-parleurs a l'aide d'un dsp
DE10321980B4 (de) Vorrichtung und Verfahren zum Berechnen eines diskreten Werts einer Komponente in einem Lautsprechersignal
DE102006010212A1 (de) Vorrichtung und Verfahren zur Simulation von WFS-Systemen und Kompensation von klangbeeinflussenden WFS-Eigenschaften
DE102005008333A1 (de) Vorrichtung und Verfahren zum Steuern einer Wellenfeldsynthese-Rendering-Einrichtung
EP3756363A1 (fr) Dispositif et procédé pour matriçage audio spatial à base d'objet
DE2850490A1 (de) Einrichtung zur mehrdimensionalen signalverteilung
WO2006089685A1 (fr) Procede et dispositif pour memoriser des fichiers audio
DE112021003592T5 (de) Informationsverarbeitungsvorrichtung, Ausgabesteuerverfahren und Programm
DE2503778B2 (de) Tonuebertragungssystem mit mindestens vier kanaelen und mit einer tonaufnahmeeinrichtung
DE102010009170A1 (de) Verfahren zum Verarbeiten und/oder Mischen von Tonspuren
WO2001033907A2 (fr) Systeme de traitement du son a canaux multiples
DD140828A1 (de) Anordnung zur aufnahme und bearbeitung von stereofonen signalen

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050527

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

DAX Request for extension of the european patent (deleted)
RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWAN

RIN1 Information on inventor provided before grant (corrected)

Inventor name: MUENNICH, KATHRIN

Inventor name: ROEDER, THOMAS

Inventor name: LANGHAMMER, JAN

Inventor name: MELCHIOR, FRANK

Inventor name: BRIX, SANDRA

RIN1 Information on inventor provided before grant (corrected)

Inventor name: REICHELT, KATHRIN

Inventor name: ROEDER, THOMAS

Inventor name: LANGHAMMER, JAN

Inventor name: MELCHIOR, FRANK

Inventor name: BRIX, SANDRA

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

Ref country code: CH

Ref legal event code: EP

REF Corresponds to:

Ref document number: 502004006676

Country of ref document: DE

Date of ref document: 20080508

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

ET Fr: translation filed
REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080626

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080901

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080707

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20081230

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080831

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080626

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080831

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080802

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080802

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080927

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080326

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20080627

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230524

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20230823

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230824

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230821

Year of fee payment: 20

Ref country code: DE

Payment date: 20230822

Year of fee payment: 20