EP1525776A1 - Dispositif de correction de niveau dans un systeme de synthese de champ d'ondes - Google Patents

Dispositif de correction de niveau dans un systeme de synthese de champ d'ondes

Info

Publication number
EP1525776A1
EP1525776A1 EP04739176A EP04739176A EP1525776A1 EP 1525776 A1 EP1525776 A1 EP 1525776A1 EP 04739176 A EP04739176 A EP 04739176A EP 04739176 A EP04739176 A EP 04739176A EP 1525776 A1 EP1525776 A1 EP 1525776A1
Authority
EP
European Patent Office
Prior art keywords
amplitude state
virtual source
source
wave field
virtual
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP04739176A
Other languages
German (de)
English (en)
Other versions
EP1525776B1 (fr
Inventor
Thomas Röder
Thomas Sporer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Forderung der Angewandten Forschung eV
Publication of EP1525776A1 publication Critical patent/EP1525776A1/fr
Application granted granted Critical
Publication of EP1525776B1 publication Critical patent/EP1525776B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S3/00Systems employing more than two channels, e.g. quadraphonic
    • H04S3/002Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S2420/00Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/13Application of wave-field synthesis in stereophonic audio systems
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04SSTEREOPHONIC SYSTEMS 
    • H04S7/00Indicating arrangements; Control arrangements, e.g. balance control

Definitions

  • the present invention relates to wave field synthesis systems and in particular to the reduction or elimination of level artifacts in wave field synthesis systems.
  • WFS Wave-Field Synthesis
  • Every point that is captured by a wave is the starting point of an elementary wave that propagates in a spherical or circular manner.
  • a large number of loudspeakers that are arranged next to each other can be used to simulate any shape of an incoming wavefront.
  • the audio signals of each loudspeaker must be fed with a time delay and amplitude scaling in such a way that the emitted sound fields of the individual loudspeakers are superimposed correctly.
  • the contribution is calculated separately be ⁇ to each speaker and adds the resulting signals for each source.
  • reflections can also be reproduced as additional sources via the loudspeaker array . The effort involved in the calculation therefore depends heavily on the number of sound sources, the reflection properties of the recording room and the number of speakers.
  • the particular advantage of this technique is that a natural spatial sound impression is possible over a large area of the playback room.
  • the direction and distance of sound sources are reproduced very precisely.
  • virtual sound sources can even be positioned between the real speaker array and the listener.
  • wave field synthesis works well for environments whose properties are known, irregularities do occur when the nature changes or when the wave field synthesis is carried out on the basis of an environment condition that does not match the actual nature of the environment.
  • the technique of wave field synthesis can also be used advantageously to complement a visual perception with a corresponding spatial audio perception.
  • the focus in production in virtual studios has been to convey an authentic visual impression of the virtual scene.
  • the acoustic impression that goes with the image is usually imprinted on the audio signal by manual work steps in what is known as post-production, or is classified as too complex and time-consuming to implement and is therefore neglected. This usually leads to a contradiction of the individual sensations, which leads to the fact that the designed space, i. H. the designed scene, which is perceived as less authentic.
  • “Hearing with the ears of the camera” is to be made possible in order to make a scene appear more real.
  • the aim here is to achieve the highest possible correlation between the sound event location in the image and the hearing event location in the surround field.
  • Camera parameters such as Zoom, should be included in the sound design as well as a position of two loudspeakers L and R.
  • tracking data of a virtual studio are written into a file together with an associated time code by the system.
  • picture, sound and time code are recorded on a MAZ.
  • the camdump file is transferred to a computer, which generates control data for an audio workstation and outputs it via a MIDI interface in sync with the image from the MAZ.
  • the actual audio processing such as positioning the sound source in the surround field and inserting early reflections and reverberation takes place within the audio workstation.
  • the signal is processed for a 5.1 surround speaker system.
  • Camera tracking parameters as well as positions of sound sources in the recording setting can be recorded in real film sets. Such data can also be generated in virtual studios.
  • an actor or presenter stands alone in a recording room.
  • he stands in front of a blue wall, which is also known as a blue box or blue panel.
  • a pattern of blue and light blue stripes is applied to this blue wall.
  • the special thing about this pattern is that the stripes are of different widths and thus result in a multitude of stripe combinations. Due to the unique stripe combinations on the blue wall, it is possible to determine exactly in which direction the camera is looking when the post-processing is replaced by a virtual background. With the help of this information, the computer can determine the background for the current camera viewing angle. Sensors on the camera are also evaluated, which record and output additional camera parameters.
  • Typical parameters of a camera which are recorded by means of sensors, are the three degrees of translation x, y, z, the three degrees of rotation, which can also be called roll, tilt, pan. are drawn, and the focal length or the zoom, which is synonymous with the information about the opening angle of the camera.
  • a tracking system can be used that consists of several infrared cameras that determine the position of an infrared sensor attached to the camera. This also determines the position of the camera.
  • a real-time computer can now calculate the background for the current image. The blue hue that the blue background had was then removed from the image, so that the virtual background is imported instead of the blue background.
  • wave field synthesis In the audio area, the technology of wave field synthesis (WFS) can be used to achieve good spatial sound for a large range of listeners.
  • wave field synthesis is based on the principle of Huygens, according to which wave fronts can be shaped and built up by superimposing elementary waves. According to a mathematically exact theoretical description, an infinite number of sources at infinitely small distances would have to be used to generate the elementary waves. In practice, however, many loudspeakers are finally used at a finite distance apart. Each of these loudspeakers is controlled according to the WFS principle with an audio signal from a virtual source, which has a specific delay and a specific level. Levels and delays are usually different for all speakers.
  • the wave field synthesis system works on the basis of the Huygens principle and reconstructs a given waveform, for example a virtual source, which is arranged at a certain distance from a demonstration area or to a listener in the demonstration area by a large number of single waves.
  • the wave field synthesis algorithm thus receives information about the actual position of a single speaker from the speaker array, in order to then calculate a component signal for this single speaker, which this speaker must then ultimately emit so that the listener overlays the speaker signal from one speaker with the speaker signals of the other active ones Loudspeaker a reconstruction results in the listener having the impression that he is not being "sonicated" by many individual speakers, but only by a single speaker at the position of the virtual source.
  • each virtual source for each loudspeaker ie the component signal of the first virtual source for the first loudspeaker, the second virtual source for the first loudspeaker, etc.
  • the contribution from each virtual source for each loudspeaker is calculated in order to then add up the component signals to finally get the actual speaker signal.
  • the overlaying of the loudspeaker signals of all active loudspeakers at the listener would result in the listener not having the impression that he is being emitted by a large array of loudspeakers, but rather that the sound he hears only comes from three sound sources positioned at special positions, which are the same as the virtual sources.
  • the component signals are usually calculated by applying a delay and / or a scaling factor to the audio signal assigned to a virtual source, depending on the position of the virtual source and the position of the loudspeaker, at a certain point in time, in order to delay and / or scale it Obtain audio signal of the virtual source, which represents the loudspeaker signal immediately if only one virtual source is present, or which after addition with further component signals for the loudspeaker under consideration from other virtual sources then contributes to the loudspeaker signal for the loudspeaker under consideration.
  • Typical wave field synthesis algorithms work regardless of how many speakers are in the speaker array.
  • the theory underlying wave field synthesis is that any sound field can be ne infinitely high number of individual speakers can be reconstructed exactly, the individual individual speakers being arranged infinitely close to one another. In practice, however, neither the infinitely high number nor the infinitely close arrangement can be realized. Instead, there is a limited number of speakers, which are also arranged at certain predetermined distances from each other. This means that in real systems only an approximation to the actual waveform is achieved, which would take place if the virtual source were actually available, i.e. would be a real source.
  • the loudspeaker array can only be viewed when viewing a cinema, e.g. B. is arranged on the side of the cinema screen.
  • the wave field synthesis module would generate loudspeaker signals for these loudspeakers, the loudspeaker signals for these loudspeakers normally being the same as for corresponding loudspeakers in a loudspeaker array that not only extends over the side of a cinema, for example, on which the screen is arranged, but also which is also located on the left, right and behind the audience room.
  • This "360 °" speaker array will of course provide a better approximation to an exact wave field than just a one-sided array, for example in front of the audience.
  • a wave field synthesis module typically receives no feedback as to how many speakers are present or whether it is a one-sided or multi-sided or even a 360 ° array or not.
  • a wave field synthesis device calculates a speaker signal for a speaker based on the position of the speaker and regardless of which other speakers are still present or not. This is a major strength of the wave field synthesis algorithm in that it can be optimally modularly adapted to different circumstances by simply giving the coordinates of the existing loudspeakers in very different demonstration rooms.
  • it is disadvantageous that, in addition to the poorer reconstruction of the current wave field that may be acceptable, considerable level artifacts occur.
  • the level of the signal from the virtual source which results at the listener's ear is reduced has, as it were, 40 component signals of the now missing speakers "missing".
  • the alternative case can also occur in which e.g. B. are first left and right of the listener speakers that are driven in a certain constellation in phase opposition, so that the speaker signals from two opposite speakers cancel each other due to a certain delay calculated by the wave field synthesis device. Is now in a reduced system such. B. without the speakers on one side of the listener, the virtual source suddenly appears much louder than it should be.
  • Wave field synthesis devices are also able to emulate several different types of sources.
  • a prominent source form is the point source, where the level decreases proportionally 1 / r, where r is the distance between a listener and the position of the virtual source.
  • Another source form is a source that emits plane waves. Here the level remains constant regardless of the distance to the listener, since plane waves can be generated by point sources that are arranged at an infinite distance.
  • the level change in two-dimensional loudspeaker arrangements corresponds to the natural level change except for a negligible error.
  • Absolute level errors result which result from using a finite number of speakers instead of the theoretically required infinite number of speakers as set forth above.
  • the object of the present invention is to provide a concept for level correction for wave field synthesis systems which is suitable for mobile sources.
  • the present invention is based on the knowledge that the inadequacies of a wave field synthesis system with a (practically realizable) finite number of loudspeakers can at least be alleviated if a level correction is carried out, in that either the audio signal assigned to a virtual source before the wave field synthesis or the component signals for various loudspeakers, which are based on a virtual source, are manipulated after the wave field synthesis using a correction value in order to reduce a deviation between a desired amplitude state in a demonstration area and an actual amplitude state in the demonstration area.
  • the target amplitude state results from the fact that depending on the position of the virtual source, and z. B.
  • a target level is determined as an example of a target amplitude state, and that an actual level as an example of an actual amplitude state of the listener is determined. While the target amplitude state is determined independently of the actual grouping and type of the individual speakers only on the basis of the virtual source or their position, the actual situation is taken into account the positioning, type and control of the individual speakers of the speaker array are calculated.
  • the sound level at the ear of the listener can be determined at the optimum point within the demonstration area on the basis of a component signal from the virtual source, which is emitted via a single loudspeaker.
  • the level at the ear of the listener at the optimal point within the demonstration area can also be determined for the other component signals that go back to the virtual source and are emitted via other loudspeakers, in order to then summarize these levels to the actual actual level at the ear of the To receive the handset.
  • the transfer function of each individual loudspeaker as well as the level of the signal at the loudspeaker and the distance of the listener at the point under consideration within the demonstration area from the individual loudspeaker can be taken into account.
  • the transmission characteristics of the loudspeaker can be assumed to work as an ideal point source.
  • the directional characteristic of the individual loudspeaker can also be taken into account for more complex implementations.
  • a major advantage of the concept according to the invention is that in an embodiment in which sound levels are considered, only multiplicative scaling occurs, in that for a quotient between the target level and the actual level, which gives the correction value, not the absolute level of the listener or the absolute level of the virtual source is required. Instead, the correction factor depends only on the position of the virtual source (and thus on the positions of the individual speakers) and the optimal point within the demonstration area. However, these quantities are in view of the position of the optimal point and the positions and transfer characteristics of each Speakers are fixed and do not depend on a track being played.
  • the concept according to the invention can be implemented in a computing time-efficient manner as a look-up table, in that a look-up table is generated and used which comprises position correction factor value pairs, for all or a substantial part of possible virtual positions.
  • a look-up table is generated and used which comprises position correction factor value pairs, for all or a substantial part of possible virtual positions.
  • no online setpoint determination, actual value determination and setpoint / actual value comparison algorithm is then to be carried out.
  • These algorithms which can be time-consuming in some cases, can be dispensed with if the look-up table is accessed on the basis of a position of a virtual source in order to determine from there the correction factor valid for this position of the virtual source.
  • a virtual source with a certain calibration level would be placed in a certain virtual position.
  • a wave field synthesis module would calculate the loudspeaker signals for the individual loudspeakers in order to finally measure the level actually arriving on the listener due to the virtual source.
  • a correction factor would then be determined such that it at least reduces the deviation from the target level to the actual level or preferably brings it to 0.
  • This correction factor would then be stored in the look-up table in association with the position of the virtual source, so gradually, that is generate the entire lookup table for many positions of the virtual source, for a specific wave field synthesis system in a special demonstration room.
  • manipulation based on the correction factor there are several options for manipulation based on the correction factor.
  • this automatically means that all component signals that originate from this manipulated virtual source are also weighted accordingly, in comparison to the case in which no correction has been made in accordance with the present invention.
  • the correction factor does not necessarily have to be identical for all component signals. However, this is largely preferred in order not to impair the relative scaling of the component signals to one another, which is necessary for the reconstruction of the actual wave situation.
  • An advantage of the present invention is that with relatively simple measures, at least during operation, a level correction can be carried out in such a way that the listener, at least with regard to the volume of a virtual source that he perceives, does not notice that the infinitely many that are actually required are not Speakers are available, but only a limited amount of speakers.
  • Another advantage of the present invention is that even if a virtual source moves at a constant distance (e.g. from left to right) with respect to the viewer, this source for the viewer who, for example, in the middle in front of the Screen sits, is always the same loud and is not even louder and once quieter, which would be the case without correction.
  • Another advantage of the present invention is that it provides the option of offering less expensive wave field synthesis systems with a smaller number of loudspeakers, which nevertheless do not involve any level artifacts, particularly for moving sources, that is to say for a listener with regard to the level problem work just as well as more complex wave field synthesis systems with a large number of speakers. Levels that are too low can also be corrected according to the invention for holes in the array.
  • FIG. 1 shows a block diagram of the device according to the invention for level correction in a wave field synthesis system
  • FIG. 2 shows a basic circuit diagram of a wave field synthesis environment as can be used for the present invention
  • FIG. 3 shows a more detailed illustration of the wave field synthesis module shown in FIG. 2;
  • FIG. 4 shows a block diagram of a device according to the invention for determining the correction value according to FIG an exemplary embodiment with a look-up table and, if appropriate, interpolation device;
  • FIG. 5 shows a further exemplary embodiment of the device for determining FIG. 1 with setpoint / actual value
  • FIG. 6a shows a block diagram of a wave field synthesis module with an embedded manipulation device for manipulating the component signals
  • 6b shows a block diagram of a further exemplary embodiment of the present invention with an upstream manipulation device
  • FIG. 7a shows a sketch for explaining the desired amplitude state at an optimal point in a demonstration area
  • FIG. 8 shows a basic block diagram of a wave field synthesis system with a wave field synthesis module and loudspeaker array in a demonstration area.
  • the wave field synthesis system has a speaker array 800 placed with respect to a demonstration area 802.
  • the speaker array shown in Fig. 8 which is a 360 ° array, includes four array sides 800a, 800b, 800c and 800d.
  • the demonstration area 802 e.g. B. a cinema
  • the viewer who is sitting at the so-called optimal point P in the demonstration area 802, would see the front, that is, the screen.
  • Each loudspeaker array consists of a number of different individual loudspeakers 808, each of which is controlled with its own loudspeaker signals, which are provided by a wave field synthesis module 810 via a data bus 812, which is only shown schematically in FIG. 8.
  • the wave field synthesis module is designed to use the information about e.g. B.
  • loudspeaker information (LS information)
  • loudspeaker signals for the individual loudspeakers 808, each of which is generated by the audio tracks for virtual sources to which position mations are assigned are derived in accordance with the known wave field synthesis algorithms.
  • the wave field synthesis module can also receive further inputs, such as information about the room acoustics of the demonstration area, etc.
  • the following statements on the present invention can in principle be carried out for each point P in the demonstration area.
  • the optimum point can thus be anywhere in the demonstration area 802.
  • it is preferred to place the optimal point or the optimal line in the middle or at the center of gravity of the wave field synthesis system, which is generated by the loudspeaker sub-arrays 800a, 800b, 800c , 800d is defined to assume.
  • a more detailed illustration of the wave field synthesis module 800 is given below with reference to FIGS. 2 and 3 with reference to the wave field synthesis module 200 in FIG. 2 and to the arrangement shown in detail in FIG. 3.
  • FIG. 2 shows a wave field synthesis environment in which the present invention can be implemented.
  • the center of a wave field synthesis environment is a wave field synthesis module 200, which comprises various inputs 202, 204, 206 and 208 and various outputs 210, 212, 214, 216.
  • Various audio signals for virtual sources are fed to the wave field synthesis module via inputs 202 to 204. So the input 202 receives z. B. an audio signal from virtual source 1 and associated position information from the virtual source.
  • the audio signal 1 would be e.g. B. the language of an actor who moves from a left side of the screen to a right side of the screen and possibly additionally away from the viewer or towards the viewer.
  • the audio signal 1 would then be the actual language of this actor, while the position information as a function of time represents the current position of the first actor in the recording setting at a certain point in time.
  • the audio signal n would be the language of, for example, another actor who moves the same or different than the first actor.
  • the current position of the other actor to whom the audio signal n is assigned is communicated to the wave field synthesis module 200 by position information synchronized with the audio signal n.
  • different virtual sources exist depending on the recording setting, the audio signal of each virtual source being supplied to the wave field synthesis module 200 as a separate audio track.
  • a wave field synthesis module feeds a plurality of speakers LSI, LS2, LS3, LSm by outputting speaker signals via the Outputs 210 to 216 to the individual speakers.
  • the positions of the individual loudspeakers in a playback setting, such as a cinema, are communicated to the wave field synthesis module 200 via the input 206.
  • the wave field synthesis module 200 In the cinema hall there are many individual loudspeakers grouped around the cinema audience, which are preferably arranged in arrays in such a way that there are loudspeakers in front of the audience, for example behind the screen, as well as behind the audience and to the right and left of the audience.
  • other inputs can be communicated to the wave field synthesis module 200, such as information about the room acoustics, etc., in order to be able to simulate the actual room acoustics prevailing during the recording set-up in a cinema hall.
  • the loudspeaker signal which is supplied to the loudspeaker LSI via the output 210, for example, will be a superimposition of component signals of the virtual sources, in that the loudspeaker signal for the loudspeaker LSI is a first component which is based on the virtual source 1. a second component that originates from virtual source 2 and an nth component that originates from virtual source n.
  • the individual component signals are linearly superposed, ie added after their calculation, in order to simulate the linear superposition at the ear of the listener, who in a real setting will hear a linear superimposition of the sound sources perceptible by him.
  • the wave field synthesis module 200 has a strongly parallel structure in such a way that, starting from the audio signal for each virtual source and starting from the position information for the corresponding virtual source, delay information V and scaling factors SFi are first calculated, which are based on the position information and the position of the one under consideration loudspeaker chers, e.g. B. depend on the loudspeaker with the order number j, i.e. LSj.
  • Known algorithms which are implemented in devices 300, 302, 304, 306, calculate the delay information Vi and a scaling factor SFi based on the position information of a virtual source and the position of the speaker j in question. Based on the delay information Vj .
  • a discrete value AW ⁇ (t A ) for the component signal Ki j is calculated in a loudspeaker signal ultimately obtained for a current time t ⁇ .
  • This is done by means 310, 312, 314, 316, as shown schematically in FIG. 3. 3 also shows, so to speak, a "flash light recording" at time t A for the individual component signals.
  • the individual component signals are then summed by a summer 320 to determine the discrete value for the current time t A of the loudspeaker signal for loudspeaker j, which then for the output (for example the output 214 if the loudspeaker j is the loudspeaker LS3), the loudspeaker can be supplied.
  • a value that is valid due to a delay and scaling with a scaling factor at a current point in time is first calculated individually for each virtual source, after which all component signals for a loudspeaker are summed due to the different virtual sources. If, for example, there were only one virtual source, the summer would be omitted, and the signal present at the output of the summer in FIG. B. correspond to the signal output by the device 310 when the virtual source 1 is the only virtual source.
  • FIG. 1 shows a block diagram of the device according to the invention for level correction in a wave field synthesis system, which has been explained with reference to FIG. 8.
  • the wave field synthesis system comprises the wave field synthesis module 810 and the loudspeaker array 800 for the sound supply of the demonstration area 802, the wave field synthesis module 810 being designed to receive an audio signal assigned to a virtual sound source and source position information assigned to the virtual sound source, and component signals for those taking into account speaker position information Speakers should be calculated on the basis of the virtual source.
  • the device according to the invention initially comprises a device 100 for determining a correction value based on a desired amplitude state in the demonstration area, the desired amplitude state depending on a position of the virtual source or a type of the virtual source, and the correction value also being based on a Actual amplitude state is based in the demonstration area, which depends on the component signals for the loudspeakers due to the virtual source.
  • the device 100 has an input 102 for obtaining a position of the virtual source when e.g. B. has a point source characteristic, or to obtain information about a type of source when the source z. B. is a source for generating plane waves.
  • B. is a source for generating plane waves.
  • the distance of the listener from the source to determine the actual state is not necessary, since the source is intended in the model due to the generated plane waves. is infinitely far from the listener and has a position-independent level.
  • the device 100 is designed to output a correction value 104 on the output side, which is sent to a device 106 for manipulating an audio signal assigned to the virtual source (which is obtained via an input 108) or for manipulating component signals for the loudspeakers on the basis of a virtual source (the can be obtained via an input 110). If the alternative of manipulating the audio signal, which is provided via the input 108, is carried out, an output 112 results in a manipulated audio signal which, according to the invention, then instead of the original audio signal which is provided at the input 108, is input into the wave field synthesis Module 200 is fed to generate the individual speaker signals 210, 212, ..., 216.
  • manipulated component signals are obtained on the output side, which still have to be summed up loudspeaker-wise (device 116), and if necessary manipulated component signals from other virtual sources, which are provided via further inputs 118.
  • the device 116 again delivers the loudspeaker signals 210, 212, ..., 216.
  • the alternatives of the upstream manipulation (output 112) or the embedded manipulation (output 114) shown in FIG. 1 are used alternatively to one another can.
  • the weighting factor or correction value which is provided via the input 104 in the device 106, is split to a certain extent, so that partly an upstream manipulation and partly an embedded manipulation is carried out.
  • the upstream manipulation would thus consist in manipulating the audio signal of the virtual source, which is fed into a device 310, 312, 314 or 316, before it is fed in.
  • the embedded manipulation on the other hand, would consist in manipulating the component signals output by devices 310, 312, 314 and 316, respectively, prior to their summation in order to obtain the actual loudspeaker signal.
  • FIGS. 6a and 6b show the embedded manipulation by the manipulation device 106, which is drawn in FIG. 6a as a multiplier.
  • a wave field synthesis device which for example consists of blocks 300, 310 or 302, 312, or 304, 314 and 306 or 316 of FIG. 3, supplies the component signals Ku, K i2 , K i3 for the loudspeaker LSI or the component signals K n ⁇ , K ⁇ 2 and K n3 for the loudspeaker LSn.
  • the first index of Kij indicates the loudspeaker
  • the second index indicates the virtual source from which the component signal originates.
  • the virtual source 1 is expressed, for example, in the component signal Ku, ..., K n ⁇ .
  • a multiplication of the component signals leading to the source 1, that is, the component signals, the index j of which indicates the virtual source 1 take place with the correction factor Fi.
  • the manipulation device here is connected upstream of the wave field synthesis device and is effective to correct the audio signals of the sources with the corresponding correction factors in order to obtain manipulated audio signals for the virtual sources, which are then fed to the wave field synthesis device in order to obtain the component signals which are then are summed up by the respective component summation devices in order to obtain the loudspeaker signals LS for the corresponding loudspeakers, such as, for example, the loudspeaker Si.
  • the device 100 for determining the correction value is designed as a look-up table 400, which stores position-correction factor-value pairs.
  • the device 100 is preferably also provided with an interpolation device 402, on the one hand to keep the table size of the lookup table 400 within a limited range, and on the other hand also for current positions of a virtual source that are fed into the interpolation device via an input 404, at least using one or more adjacent position correction factor value pairs stored in the lookup table, which are supplied to the interpolation device 402 via an input 406 to generate an interpolated current correction factor at an output 408.
  • the interpolation device 402 can also be omitted, so that the device 100 for determining FIG.
  • the look-up table performs direct access to the look-up table using position information supplied at an input 410 and a corresponding correction factor at an output 412 delivers. If the current position information that is assigned to the audio track of the virtual source does not exactly correspond to a position information that can be found in the look-up table, the look-up table can also be assigned a simple rounding-off / rounding-up function in order to store the closest one stored in the table To take the base value instead of the current base value.
  • the device for determining can be designed to actually carry out a setpoint-actual value comparison.
  • the device 100 of FIG. 1 comprises a setpoint amplitude State determination device 500 and an actual amplitude state determination device 502 in order to deliver a desired amplitude state 504 and an actual amplitude state 506, which a comparison unit direction 508, which, for example, calculates a quotient from the desired amplitude state 504 and the actual amplitude state 506 in order to generate a correction factor 510 which is supplied to the device 106 for manipulation, which is shown in FIG. 1, for further use .
  • the correction value can also be stored in a look-up table.
  • the target amplitude state calculation is designed to determine a target level at the optimum point for a virtual source configured at a specific position or in a specific type.
  • the desired amplitude state determination device 500 does not, of course, require any component signals, since the desired amplitude state is independent of the component signals.
  • component signals are fed to the actual amplitude determination device 502, which, depending on the embodiment, can also receive information about the speaker positions and information about speaker transmission functions and / or information about directional characteristics of the speakers by one Determine the current situation as well as possible.
  • the actual amplitude state determination device 502 can also be designed as an actual measuring system in order to determine an actual level situation at the optimal point for certain virtual sources at certain positions.
  • FIG. 7a shows a diagram for determining a target amplitude state at a predetermined point, which is designated in Fig. 7a with "optimal point" and which is in the demonstration area 802 of Fig. 8.
  • Virtual source 700 is shown as a point source that generates a sound field with concentric wavefronts for the virtual source 700 the level L v of the virtual source 700 is known.
  • the target amplitude state or, if the amplitude state is a level state, the target level at point P in the demonstration area is readily obtained in that level L P at point P is equal to the quotient of L v and a distance r
  • the point P has to the virtual source 700.
  • the target amplitude state can thus be easily determined by calculating the level L v of the virtual source and by calculating the distance r from the optimal point to the virtual source.
  • a coordinate transformation of the virtual coordinates into the coordinates of the screening room or a coordinate transformation of the screening room coordinates of point P into the virtual coordinates must typically be carried out, which is known to those skilled in the field of wave field synthesis.
  • the virtual source is an infinitely distant virtual source, which generates plane waves at point P
  • the distance between point P and the source is not required to determine the desired amplitude state, since this goes towards infinite anyway. In this case, only information about the type of source is required.
  • the target level at point P is then equal to the level which is assigned to the plane wave field which is generated by the virtual source which is infinitely distant.
  • Fig. 7 shows a diagram for explaining the actual amplitude state.
  • different loudspeakers 808 are drawn in FIG. 7b, all of which are fed with their own loudspeaker signal which, for. B. has been generated by the wave field synthesis module 810 of FIG. 8.
  • each loudspeaker is modeled as a point source that outputs a concentric wave field.
  • the law of the concentric wave field is again that the level drops according to 1 / r.
  • a corresponding procedure can also be carried out for the other loudspeakers of the loudspeaker array, so that there is a number of “partial level values” for point P, which represent a signal contribution from the virtual source under consideration, which is transmitted from the individual loudspeakers to the listener at point P
  • partial level values represent a signal contribution from the virtual source under consideration, which is transmitted from the individual loudspeakers to the listener at point P
  • a correction value which is preferably multiplicative, but which is in principle could be additive or subtractive.
  • the desired level for a point ie the desired amplitude state
  • the optimal point or the point in the demonstration area that is being viewed is sensibly in the middle of the wave field synthesis system. At this point it should be pointed out that an improvement is achieved even if the point on which the calculation of the desired amplitude state is based does not correspond directly to the point which was used to determine the actual amplitude state.
  • a target amplitude state is determined for any point in the demonstration area and that an actual amplitude state is also determined for any point in the demonstration area, whereby however, it is preferred that the point to which the actual amplitude state is related be in a zone around the point for which the target amplitude state has been determined, this zone preferably being less than 2 meters for normal cinema applications. For best results, these points should essentially coincide.
  • the level practically generated by superimposition at this point which is called the optimal point in the demonstration area.
  • the levels of the individual speakers and / or sources are then corrected with this factor according to the invention.
  • the method according to the invention for level correction can be implemented in hardware or in software.
  • the implementation can take place on a digital storage medium, in particular a floppy disk or CD with electronically readable control signals, which can cooperate with a programmable computer system in such a way that the method is carried out.
  • the invention thus also consists in a computer program product with a program code stored on a machine-readable carrier for carrying out the method for level correction when the computer program product is on a Calculator expires.
  • the invention can thus be implemented as a computer program with a program code for carrying out the method if the computer program runs on a computer.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)
  • Radar Systems Or Details Thereof (AREA)
  • Variable-Direction Aerials And Aerial Arrays (AREA)
  • Circuit For Audible Band Transducer (AREA)
EP04739176A 2003-05-15 2004-05-11 Dispositif de correction de niveau dans un systeme de synthese de champ d'ondes Expired - Lifetime EP1525776B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
DE10321986A DE10321986B4 (de) 2003-05-15 2003-05-15 Vorrichtung und Verfahren zum Pegel-Korrigieren in einem Wellenfeldsynthesesystem
DE10321986 2003-05-15
PCT/EP2004/005045 WO2004103024A1 (fr) 2003-05-15 2004-05-11 Dispositif de correction de niveau dans un systeme de synthese de champ d'ondes

Publications (2)

Publication Number Publication Date
EP1525776A1 true EP1525776A1 (fr) 2005-04-27
EP1525776B1 EP1525776B1 (fr) 2006-04-19

Family

ID=33440866

Family Applications (1)

Application Number Title Priority Date Filing Date
EP04739176A Expired - Lifetime EP1525776B1 (fr) 2003-05-15 2004-05-11 Dispositif de correction de niveau dans un systeme de synthese de champ d'ondes

Country Status (7)

Country Link
US (1) US7751915B2 (fr)
EP (1) EP1525776B1 (fr)
JP (1) JP4617311B2 (fr)
CN (1) CN100551134C (fr)
AT (1) ATE324023T1 (fr)
DE (2) DE10321986B4 (fr)
WO (1) WO2004103024A1 (fr)

Families Citing this family (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102004057500B3 (de) 2004-11-29 2006-06-14 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zur Ansteuerung einer Beschallungsanlage und Beschallungsanlage
DE102005008369A1 (de) 2005-02-23 2006-09-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Simulieren eines Wellenfeldsynthese-Systems
DE102005008366A1 (de) * 2005-02-23 2006-08-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Ansteuern einer Wellenfeldsynthese-Renderer-Einrichtung mit Audioobjekten
DE102005008343A1 (de) 2005-02-23 2006-09-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Liefern von Daten in einem Multi-Renderer-System
DE102005008333A1 (de) * 2005-02-23 2006-08-31 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Steuern einer Wellenfeldsynthese-Rendering-Einrichtung
DE102005008342A1 (de) 2005-02-23 2006-08-24 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Speichern von Audiodateien
US8031891B2 (en) * 2005-06-30 2011-10-04 Microsoft Corporation Dynamic media rendering
DE102005033239A1 (de) * 2005-07-15 2007-01-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Steuern einer Mehrzahl von Lautsprechern mittels einer graphischen Benutzerschnittstelle
DE102005033238A1 (de) * 2005-07-15 2007-01-25 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zum Ansteuern einer Mehrzahl von Lautsprechern mittels eines DSP
DE102006010212A1 (de) * 2006-03-06 2007-09-20 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung und Verfahren zur Simulation von WFS-Systemen und Kompensation von klangbeeinflussenden WFS-Eigenschaften
US8180067B2 (en) * 2006-04-28 2012-05-15 Harman International Industries, Incorporated System for selectively extracting components of an audio input signal
DE102006039162A1 (de) 2006-08-21 2008-02-28 Lear Corp., Southfield Schutzverfahren für Lautsprechersysteme
US8036767B2 (en) 2006-09-20 2011-10-11 Harman International Industries, Incorporated System for extracting and changing the reverberant content of an audio input signal
EP2092516A4 (fr) * 2006-11-15 2010-01-13 Lg Electronics Inc Procédé et appareil de décodage de signal audio
US8265941B2 (en) 2006-12-07 2012-09-11 Lg Electronics Inc. Method and an apparatus for decoding an audio signal
KR100943215B1 (ko) * 2007-11-27 2010-02-18 한국전자통신연구원 음장 합성을 이용한 입체 음장 재생 장치 및 그 방법
WO2011044064A1 (fr) * 2009-10-05 2011-04-14 Harman International Industries, Incorporated Système pour l'extraction spatiale de signaux audio
US9271081B2 (en) * 2010-08-27 2016-02-23 Sonicemotion Ag Method and device for enhanced sound field reproduction of spatially encoded audio input signals
DE102011082310A1 (de) * 2011-09-07 2013-03-07 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Vorrichtung, Verfahren und elektroakustisches System zur Nachhallzeitverlängerung
WO2013068402A1 (fr) 2011-11-10 2013-05-16 Sonicemotion Ag Procédé d'implémentations pratiques de reproduction de champs sonores basé sur des intégrales de surface en trois dimensions
KR101719837B1 (ko) * 2012-05-31 2017-03-24 한국전자통신연구원 음장 합성 신호 생성 장치 및 방법
RU2602346C2 (ru) * 2012-08-31 2016-11-20 Долби Лэборетериз Лайсенсинг Корпорейшн Рендеринг отраженного звука для объектно-ориентированной аудиоинформации
EP3076500B1 (fr) 2013-11-27 2018-10-10 NGK Insulators, Ltd. Dispositif électroluminescent de type résonateur externe
DE102022129642A1 (de) * 2022-11-09 2024-05-16 Holoplot Gmbh Verfahren zur richtungsabhängigen Korrektur des Frequenzganges von Schallwellenfronten

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH04132499A (ja) 1990-09-25 1992-05-06 Matsushita Electric Ind Co Ltd 音像制御装置
GB2294854B (en) * 1994-11-03 1999-06-30 Solid State Logic Ltd Audio signal processing
US6205224B1 (en) * 1996-05-17 2001-03-20 The Boeing Company Circularly symmetric, zero redundancy, planar array having broad frequency range applications
US5798922A (en) * 1997-01-24 1998-08-25 Sony Corporation Method and apparatus for electronically embedding directional cues in two channels of sound for interactive applications
DE19706137A1 (de) * 1997-02-18 1998-08-20 Marc Wehberg Holophonie- Ein Verfahren zur echten dreidimensionalen Raumklangerzeugung
DE19739425A1 (de) * 1997-09-09 1999-03-11 Bosch Gmbh Robert Verfahren und Anordnung zur Wiedergabe eines sterophonen Audiosignals
JP4132499B2 (ja) 1999-11-08 2008-08-13 株式会社アドバンテスト 半導体試験用プログラムデバッグ装置
GB0124352D0 (en) * 2001-10-11 2001-11-28 1 Ltd Signal processing device for acoustic transducer array
DE10254404B4 (de) * 2002-11-21 2004-11-18 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Audiowiedergabesystem und Verfahren zum Wiedergeben eines Audiosignals
US7336793B2 (en) * 2003-05-08 2008-02-26 Harman International Industries, Incorporated Loudspeaker system for virtual sound synthesis

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2004103024A1 *

Also Published As

Publication number Publication date
CN100551134C (zh) 2009-10-14
US20060109992A1 (en) 2006-05-25
US7751915B2 (en) 2010-07-06
ATE324023T1 (de) 2006-05-15
JP4617311B2 (ja) 2011-01-26
CN1792117A (zh) 2006-06-21
DE10321986B4 (de) 2005-07-14
DE10321986A1 (de) 2004-12-09
EP1525776B1 (fr) 2006-04-19
JP2007502589A (ja) 2007-02-08
WO2004103024A1 (fr) 2004-11-25
DE502004000439D1 (de) 2006-05-24

Similar Documents

Publication Publication Date Title
DE10328335B4 (de) Wellenfeldsyntesevorrichtung und Verfahren zum Treiben eines Arrays von Lautsprechern
EP1525776B1 (fr) Dispositif de correction de niveau dans un systeme de synthese de champ d'ondes
EP1671516B1 (fr) Procede et dispositif de production d'un canal a frequences basses
DE10254404B4 (de) Audiowiedergabesystem und Verfahren zum Wiedergeben eines Audiosignals
EP1972181B1 (fr) Dispositif et procédé de simulation de systèmes wfs et de compensation de propriétés wfs influençant le son
EP1782658B1 (fr) Dispositif et procede de commande d'une pluralite de haut-parleurs a l'aide d'un dsp
EP1872620B9 (fr) Dispositif et procede pour commander une pluralite de haut-parleurs au moyen d'une interface graphique d'utilisateur
EP1851998B1 (fr) Dispositif et procédé pour fournir des données dans un système a dispositifs de rendu multiples
DE10321980B4 (de) Vorrichtung und Verfahren zum Berechnen eines diskreten Werts einer Komponente in einem Lautsprechersignal
EP2754151B1 (fr) Dispositif, procédé et système électroacoustique de prolongement d'un temps de réverbération
EP1518443B1 (fr) Dispositif et procede pour determiner une position de reproduction
DE102005027978A1 (de) Vorrichtung und Verfahren zum Erzeugen eines Lautsprechersignals aufgrund einer zufällig auftretenden Audioquelle
DE10254470A1 (de) Vorrichtung und Verfahren zum Bestimmen einer Impulsantwort und Vorrichtung und Verfahren zum Vorführen eines Audiostücks

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050223

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL HR LT LV MK

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: 7H 04S 7/00 B

Ipc: 7H 04S 3/00 A

REG Reference to a national code

Ref country code: HK

Ref legal event code: DE

Ref document number: 1075166

Country of ref document: HK

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT;WARNING: LAPSES OF ITALIAN PATENTS WITH EFFECTIVE DATE BEFORE 2007 MAY HAVE OCCURRED AT ANY TIME BEFORE 2007. THE CORRECT EFFECTIVE DATE MAY BE DIFFERENT FROM THE ONE RECORDED.

Effective date: 20060419

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060511

REF Corresponds to:

Ref document number: 502004000439

Country of ref document: DE

Date of ref document: 20060524

Kind code of ref document: P

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060531

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060531

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20060525

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060719

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060719

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060730

DAX Request for extension of the european patent (deleted)
PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060919

ET Fr: translation filed
REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

RAP2 Party data changed (patent owner data changed or rights of a patent transferred)

Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWAN

NLT2 Nl: modifications (of names), taken from the european patent patent bulletin

Owner name: FRAUNHOFER-GESELLSCHAFT ZUR

Effective date: 20070117

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070122

BERE Be: lapsed

Owner name: FRAUNHOFER-GESELLSCHAFT ZUR FORDERUNG DER ANGEWAN

Effective date: 20060531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060720

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060719

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061020

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20060511

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20060419

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080531

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20080531

REG Reference to a national code

Ref country code: HK

Ref legal event code: WD

Ref document number: 1075166

Country of ref document: HK

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 13

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20230519

Year of fee payment: 20

Ref country code: FR

Payment date: 20230517

Year of fee payment: 20

Ref country code: DE

Payment date: 20230519

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230522

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 502004000439

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MK

Effective date: 20240510

REG Reference to a national code

Ref country code: GB

Ref legal event code: PE20

Expiry date: 20240510

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20240510

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20240510