EP2255359B1 - Device and method for acoustic indication - Google Patents
Device and method for acoustic indication Download PDFInfo
- Publication number
- EP2255359B1 EP2255359B1 EP09721864.8A EP09721864A EP2255359B1 EP 2255359 B1 EP2255359 B1 EP 2255359B1 EP 09721864 A EP09721864 A EP 09721864A EP 2255359 B1 EP2255359 B1 EP 2255359B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- objects
- signal
- acoustic
- signals
- loudspeaker
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims description 18
- 230000015572 biosynthetic process Effects 0.000 claims description 17
- 238000003786 synthesis reaction Methods 0.000 claims description 17
- 230000005236 sound signal Effects 0.000 claims description 13
- 238000004590 computer program Methods 0.000 claims description 7
- 230000003068 static effect Effects 0.000 claims description 4
- 230000001419 dependent effect Effects 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 238000001514 detection method Methods 0.000 description 5
- 230000003247 decreasing effect Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 3
- 238000012545 processing Methods 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 230000004913 activation Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 238000002474 experimental method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 238000010009 beating Methods 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 231100001261 hazardous Toxicity 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000012913 prioritisation Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000001020 rhythmical effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 210000002023 somite Anatomy 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
- 238000010626 work up procedure Methods 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/13—Aspects of volume control, not necessarily automatic, in stereophonic sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/13—Application of wave-field synthesis in stereophonic audio systems
Definitions
- the present invention relates to an apparatus and method for acoustically displaying a position of an object in a playback room.
- Exemplary embodiments include in particular acoustic displays for use on ships.
- the DE 101 55 742 A1 discloses an apparatus and method for generating spatially located warning and information signals.
- visual signals emitted wherein the respective acoustic and visual signal sources are mounted in a specific for the respective warning or information position.
- a conventional stereo system of a vehicle is adapted as a "virtual reality" system.
- a moving sound source is simulated realistically, the emitted information being varied in connection with the movement of the virtual sound source (eg imitation of the Doppler effect simulating an approaching siren of an emergency vehicle or intensifying the volume at the approach of a dangerous object).
- a further advantageous use of this technique results from its integration into a lane recognition system.
- a warning signal for example Nagelbandrattern
- the US Pat. No. 6,097,285 discloses an apparatus for emitting acoustic signals in a vehicle compartment for notifying a driver of a vehicle of the presence of objects at predetermined locations around the vehicle.
- the driver is alerted to the presence of objects in predetermined detection zones around the vehicle by the presence of various audible sounds corresponding to each detection zone.
- an object present in a detection zone is caused to produce a unique, audibly different sound within the cabin to alert the driver of the presence of the particular object.
- the present invention has the object to obtain a device or a method for the acoustic display of a position of an object.
- an acoustic display device according to claim 1, an apparatus for scanning an environment according to claim 11, a method for acoustic display according to claim 13 or a computer program according to claim 14.
- the core idea of the present invention is that a plurality of loudspeakers is spatially arranged so differently in a reproduction room that different positions can be acoustically represented by different activation of the loudspeakers.
- a signal allocation device is designed to assign an acoustic signal to the object
- a loudspeaker drive device is designed to determine one or more loudspeaker signals for the multiplicity of loudspeakers.
- the one or more loudspeaker signals are arranged to indicate the position of the object, wherein the one or more loudspeaker signals are based on the acoustic signal associated with the object by the signal assigning means.
- the one or more loudspeaker signals are determined such that upon playback of the one or more loudspeaker signals, the position of the object in the playback room is displayed acoustically.
- Embodiments of the present invention also relate to how sensor signals can be displayed more easily by means of intelligent acoustic displays and thus both the security can be improved and the running costs can be reduced.
- Another idea of the present invention is based on the fact that an essential part of the information in many detectors is a location.
- a detector for example, a radar, a depth sounder, nautical charts or weather maps come into consideration and the location refers to, for example, a direction as well as a distance to the object.
- a sound field generated which encodes as precisely as possible this information in a natural way.
- the core idea of the present invention is that a plurality of loudspeakers is spatially arranged so differently in a reproduction room that different positions can be acoustically represented by different activation of the loudspeakers.
- a signal allocation device is designed to assign an acoustic signal to the object
- a loudspeaker drive device is designed to determine one or more loudspeaker signals for the multiplicity of loudspeakers.
- the one or more loudspeaker signals are arranged to indicate the position of the object, wherein the one or more loudspeaker signals are based on the acoustic signal associated with the object by the signal assigning means.
- the one or more loudspeaker signals are determined such that upon playback of the one or more loudspeaker signals, the position of the object in the playback room is displayed acoustically.
- Embodiments of the present invention also relate to how sensor signals can be displayed more easily by means of intelligent acoustic displays and thus both the security can be improved and the running costs can be reduced.
- Another idea of the present invention is based on the fact that an essential part of the information in many detectors is a location.
- a detector for example, a radar, a depth sounder, nautical charts or weather maps come into consideration and the location refers to, for example, a direction as well as a distance to the object.
- a sound field is generated, for example by means of several speakers, which encodes this information as precisely as possible in a natural way.
- the workup includes on the one hand the detection of moving objects, such as ships and aircraft, and also the detection of static objects, such as the coastline, buoys or islands.
- moving objects such as ships and aircraft
- static objects such as the coastline, buoys or islands.
- the audio signal can optionally be converted into an audio signal by means of a text-to-speech identification, so that the text signal of the transponder becomes audible.
- Such objects are z. B. determines buoys or beacons, whose identifying information appear, for example, on the radar as text.
- Objects can still be classified according to their hazard potential. For example, objects that come closer (from the front or faster from the back) or cross the ship's path of movement may be classified as more dangerous than objects that run parallel to the ship or are moving away from the ship. Objects that are farther away are generally considered less dangerous than those that are near or approaching at a high relative speed. Depending on the risk can thus be assigned to the objects a different identifier tone, with the identification tone for example in pitch or in the pulse repetition frequency and increase as the danger increases. Thus, a higher tone may mean greater danger or increasing volume may imply an increasing danger. Similarly, a faster beating clock pulse may mean a rising or a higher hazard than a lower clock pulse (for example, if the signature sound is represented as a rhythmic clock pulse).
- the audio signals of the objects thus generated are then reproduced, for example, by the above-mentioned WFS or ZAP, whereby automatically far distant objects become quieter.
- non-hazardous objects are completely blanked out (not shown) so as not to overload the helmsman or the listener with too much information.
- the replay location may appear at the same distance as the actual distance, ie, if the object is one kilometer away by radar, the audio object is perceptible at one kilometer distance (1: 1 mapping).
- the playback location is scaled accordingly so that, for example, a 1: 100 mapping is made and an object one kilometer away is acoustically perceptible or reproduced by an acoustic signal (virtual sound source) approximately ten meters away.
- the former the 1: 1 image
- Very distant objects would only be audible very late due to the speed of sound, and furthermore, in a 1: 1 representation, very distant objects are hardly distinguishable by distance.
- Exemplary embodiments thus pursue the goal of coding objects with audio signals, so that they can be located as well as possible.
- the audio signals should be sufficiently broadband, for example, because a sine wave is difficult to perceive. Accordingly, narrowband noise or speech should be used to identify objects, not sinusoidal ones.
- pulsed signals are emitted instead of continuous signals (eg a continuous tone). The pulse rate can rise similarly to parking sensors in cars with increasing risk.
- the audio signals should sound pleasant when the danger is sufficiently low.
- the danger threshold above which there is a serious danger or below which there is no risk or hardly any danger, is set variably in accordance with the circumstances, for example.
- the danger threshold can optionally also be adapted by the user.
- the size and speed of a ship or the speeds of the other objects play a role.
- the threshold value can be determined, for example, from the ratio of the time duration to a predicted collision to a braking time of the ship.
- the pleasant sound of the audio signals can be achieved, for example, by using a low center frequency of the narrowband noise or a low pulse frequency (rare representation) for unidentified objects (eg objects that pose no danger).
- a spectral hue of narrowband noise can be used where high frequencies have less energy than deep (cut with pink noise bandpass). For identified objects, this is done Rarely reporting reached, eg. B. at first contact to then send only a minute away a new signal.
- the reporting signal may optionally be selected to be precisely located and distinguishable from ambient noise. Moreover, it is advantageous if the reporting signal has a pleasant sound, so that even with long trips, the system is permanently accepted.
- An essential advantage of acoustic, spatially resolving displays is that, unlike optical displays, they can be used by one person simultaneously with the natural environment.
- the natural environment may include, for example, driving on sight or listening to ships and buoys. Thus, a so-called augmented reality can be generated.
- Embodiments are particularly advantageous because they provide an important synergy effect between acoustic and visual display. Namely, the audible indication is always reported and perceived, whereby prioritization for danger may occur while the visual indication requires the attention of the personnel on the bridge. For example, a helmsman sees an object on the radar screen only when he looks at the radar screen. At the same time, however, he no longer looks out of the window and thus loses some of the information about what is happening in his immediate surroundings. Acoustic displays allow him to simultaneously use the information from the radar and the view from the window. Especially in the case of non-self-identifying objects, however, the experienced evaluator is able to classify an object from the radar image (eg as a ship, island or picture disturbance). Thus, in the interaction of the acoustic perception (there is an object) and the view on the radar screen to control an important synergy effect. For remote, self-identifying objects, the identification can be read at any time by looking at the radar screen.
- Fig. 1 shows a schematic representation of an apparatus for acoustic display 100 having an input 105, above the position information of an object in the Device 100 can be entered.
- the device 100 further has outputs for a plurality of loudspeaker signals LS (eg for a first loudspeaker signal LS1, a second loudspeaker signal LS2, a third loudspeaker signal LS3, ..., an nth loudspeaker signal LSn).
- the input for the position information 105 is designed to signal objects with their position to a signal allocation device 110.
- the signal allocation device 110 is designed to assign an acoustic signal to the objects, wherein the signal allocation device 110 optionally accesses a signal database 140 in order to assign different signals to different objects, for example on the basis of their potential dangers.
- the respectively assigned signal may, for example, depend on whether the object is moving, if so at what speed, or if it is immovable.
- the device 100 has a loudspeaker drive device 120, which receives from the signal allocation device 110 the position of the object and the acoustic signal in order to determine one or more loudspeaker signals LS for a multiplicity of loudspeakers and these via the outputs for the loudspeaker signals LS1,. .. to spend LSn.
- the loudspeaker driver 120 is configured to determine the one or more loudspeaker signals LS based on the acoustic signal assigned to the object. The determination is carried out in such a way that, when the one or more loudspeaker signals LS are reproduced, the position of the object in the reproduction room is indicated acoustically.
- a listener or user then takes the position (eg, distance and direction) of the object as a virtual sound source position.
- one embodiment relates to the reproduction of information of a radar device which determines positions of objects.
- information from, for example, other sources, such as echosounders, or other sensors are implemented in a similar manner.
- loudspeakers on the bridge of the ship below windows may be arranged on all walls. These loudspeakers, for example, can all be equipped with their own amplifiers or with A / D converters (analog-to-digital converters) and can also be individually controlled.
- Fig. 2 shows a schematic representation of a playback room 210 with three speakers 220a, 220b and 220c and a radar 230.
- the radar 230 is connected to the input 105 and provides position information about objects in a vicinity of the playback room 210. For example, the radar 230 is formed to the Position of the object 200 to the device 100 for acoustic display to pass.
- the three speakers 220a, 220b, 220c are also connected to the outputs for the loudspeaker signals LS of the acoustic display device 100.
- a first speaker 220a is connected to the output for the first speaker signal LS1
- a second speaker 220b is connected to the output for the second speaker signal LS2
- a third speaker 220c is connected to the output for the third speaker signal LS3.
- the acoustic display device 100 evaluates the position information of the object 200 received from the radar 230 to generate three loudspeaker signals LS1, LS2, LS3 for the first, second and third loudspeakers 220a, 220b, 220c. The determination is made such that the position of the object 200 is audible to the listener in the playback room 210, which is at a position P, for example. For this purpose, first the device 100 determines an acoustic signal for the object 200 as a function of the position of the object 200. The position is determined by the distance d and the direction, which can be given for example by an angle ⁇ . Next, the apparatus 100 calculates loudspeaker signals LS for the first to third loudspeakers 220a to 220c.
- This may include, for example, scaling the signal level and delaying the signal so that the listener at position P perceives the object 200 according to its position. In the in the Fig. 2 For example, this may be done such that the third loudspeaker 220c provides the strongest signal, during which the first loudspeaker 220a provides only a small signal and the second loudspeaker 220b does not provide a signal.
- radar 230 may further be coupled to a sonar device, which, for example, the underwater topography abskannt and signals any existing shoals, which are also acoustically displayed.
- a sonar device for example, the underwater topography abskannt and signals any existing shoals, which are also acoustically displayed.
- Fig. 3a and 3b show possible variations of the acoustic signal as a function of the distance of the object and the associated danger potential.
- a dependency of a frequency f of the signal on the distance d of the object 200 is shown.
- a critical distance d c is less than that, there is an increased danger which requires an increased attention of the helmsman.
- This transition from a safe to a dangerous state for example, be signaled in a changing acoustic signal.
- the frequency f of the signal are close or only slightly above a fundamental frequency f 0 , wherein the thus defined frequency range is perceived by the helmsman as safe.
- the object reduces the distance so that it comes below the critical distance d c , the frequency f of the acoustic signal can suddenly rise sharply, so that the increasing danger is signaled to the helmsman.
- the increase in frequency can optionally also increase monotonously with decreasing distance of the object without causing a sudden change in the critical distance and a constantly increasing danger potential for the helmsman becomes perceptible.
- the acoustic signal or the frequency f of the acoustic signal may include on the one hand, the audio frequency or else the clock frequency, for example, if the acoustic signal indicates a particular clock in a particular frequency (repetition rate of the clocks). Even with the clock signal can increase with decreasing distance, the clock frequency, so that acoustically an increasing risk potential for the helmsman is perceived.
- Fig. 3b shows an embodiment in which the signal level S is shown as a function of time t.
- the distance between two adjacent clocks decreases, so that the clock frequency increases, so that an approaching object will signal.
- the decreasing pitch can be combined by the fact that the signal pulses louder and / or the frequencies of the signal pulses are changed.
- the change of the signal may, for example, have a shift of the center frequency to higher frequencies, so that the increasing danger potential is also perceptible in the frequency level or audio frequency of the signal pulses.
- the amplitude or volume of the signal can increase as the potential dangers increase.
- the acoustic signals are barely perceptible, so that the helmsman is not disturbed by the acoustic signals.
- Fig. 4 11 shows an embodiment in which a plurality of loudspeakers 220, a first loudspeaker 220a,..., a fourth loudspeaker 220d,..., a ninth loudspeaker 220i,... have a twelfth loudspeaker 2201.
- the loudspeakers 220 are arranged around the position P of a listener so that the position of an object 200 or the direction of the object 200 becomes noticeable by the fact that only one loudspeaker is active.
- the position of the active loudspeaker corresponds at the same time in the direction of the object 200. This is particularly advantageous when the position P in the reproduction room 210 is fixed.
- a first object 200a at a distance d1 and a second object 200b at a distance d2 from the point of view P are perceived by the fourth loudspeaker 220d generating a first sound signal S1 and the ninth loudspeaker 220i generating a second sound signal S2.
- the listener at the position P takes the first object 200a and the second object was then according to their positions.
- the speaker can be selected, which is the shortest distance to the connecting line between the respective object and the Position P has. That would be the fourth loudspeaker 220d for the first object 200a and the ninth loudspeaker 220i for the second object 200b. All other speakers are further away from the respective connection lines (measured as a vertical distance) and, for example, can not be active in this embodiment (do not generate a sound signal).
- the respective adjacent loudspeakers between which the connecting line between the first object 200a and the position P runs, to be active.
- other neighbors speakers may be active. This means that, for example, in further embodiments not only the fourth loudspeaker 220d is active, but at the same time the third loudspeaker 220c and / or the second loudspeaker 220b and / or the fifth loudspeaker 220e can also be active.
- the amplitude / phase should be selected such that for a listener at position P, the object 200 will be acoustically perceivable at its respective position.
- Acoustic perceptibility means that the object 200 is perceived as a virtual sound source, wherein the distance in addition to the volume can also be signaled by a different clock frequency or frequency (as in the example Fig. 3a, b was shown).
- Fig. 5 shows an embodiment in which the speakers are arranged in the context of a wave field synthesis system, so that the device for acoustic display 100, a first loudspeaker array 221a, a second loudspeaker array 221b and a third loudspeaker array 221c drives.
- Each of the three loudspeaker arrays 221a, 221b, 221c has, for example, a multiplicity of loudspeakers which, for B.
- each loudspeaker in a respective array can be controlled individually, so that the three arrays, which may be arranged, for example, on the sidewalls of the reproduction room 210, synthesize a wave field which would produce an object 200 as a virtual sound source in the reproduction room 210.
- the device 100 can in turn be coupled to a radar device or a sonar device 230 which transmits the device 100 the position of the respective objects.
- the object itself does not need to be a sound source, but instead a sound signal is specifically assigned to the object. In this sense, therefore, the acoustic display differs according to embodiments of conventional audio playback systems.
- Wave field synthesis is an audio reproduction method developed at the TU Delft for the spatial reproduction of complex audio scenes.
- the spatially correct rendering is not limited to a small area, but extends over a wide viewing area.
- WFS is based on a well-founded mathematical-physical basis, namely the principle of Huygens and the Kirchhoff-Helmholtz integral.
- a WFS reproduction system consists of a large number of loudspeakers (so-called secondary sources).
- the loudspeaker signals are formed from delayed and scaled input signals. Since many audio objects (primary sources) are typically used in a WFS scene, many such operations are required to generate the loudspeaker signals. This requires the high computing power required for wave field synthesis.
- WFS also offers the possibility of realistically mapping moving sources. This feature is used in many WFS systems and is for example for use in the cinema, virtual reality applications or live performances of great importance.
- a primary goal is the development of signal processing algorithms for the playback of moving sources using WFS.
- the real-time capability of the algorithms is an important condition.
- the most important criterion for evaluating the algorithms is the objective perceived audio quality.
- WFS is a very expensive audio reproduction process in terms of processing resources. This is mainly due to the large number of speakers in a WFS setup and the often high number of virtual sources used in WFS scenes. For this reason, the efficiency of the algorithms to be developed is of paramount importance.
- Wave field synthesis systems have the advantage, in comparison to conventional multi-speaker systems, that exact positioning becomes possible as a result and exact positioning can also be determined at different positions within the reproduction space 210.
- a basic structure of a wave field synthesis system is shown and has a speaker array 221 placed with respect to a reproduction space 210. Specifically, this includes in Fig. 6 shown speaker array, which is a 360 ° array, four array sides 221a, 221b, 221c and 221d.
- speaker array which is a 360 ° array, four array sides 221a, 221b, 221c and 221d.
- Each loudspeaker array 221 consists of a number of different individual loudspeakers 708, each of which is driven by its own loudspeaker signals LS, which are transmitted by a wave field synthesis module 710 via an in-line loudspeaker signal LS Fig. 6 only schematically shown data bus 712 are provided.
- the position information is determined, for example, by a sensor for determining the position of objects (eg the radar) and provided to the wave field synthesis module via the input 105.
- the wave field synthesis module may also receive further inputs, such as information about the room acoustics of the playback room 210, etc.
- the signal allocator 110 is configured to associate acoustic signals to a plurality of objects 200
- the loudspeaker driver 120 is configured to generate component signals for each of the plurality of objects 200 Combine component signals to speaker signals LS, so that the plurality of objects 200 are acoustically perceptible at different positions.
- the various objects can appear or be perceived as virtual sources (sound sources) for the listeners.
- boundary conditions are considered in the ships.
- the boundary conditions include, for example, requirements for the frequency of the messages, possible positions of the loudspeakers, the required sound pressure level, the characterization of the noise (for example from the engine) and a specification of the control signals for the acoustic display.
- optimal message signals can then be generated taking into account typical spatial sounds on the ships.
- the acoustic drive includes techniques such as binaural coding or the wave field synthesis described above.
- the different techniques are used on test rigs in ships (or one-to-one models of the bridge and / or the control room). For example, psychoacoustic experiments can provide clues.
- Embodiments use reporting signals that are as well as possible to locate in the ship environment, but at the same time sound as pleasant as possible. In this case test setups in the laboratory or else a one-to-one model from the bridge and / or the control station or in vehicles as well as psychoacoustic experiments are useful.
- Further embodiments also provide a connection of sensors and information, for example, from Radar, sounder and nautical charts are received, to the audible indicator.
- An essential part of the connection is the selection of the relevant objects, which should be displayed for example by means of acoustic display.
- the described systems can also be applied in automobiles, i. Further embodiments also include corresponding driver assistance systems in the car. For example, vehicles approaching laterally (eg when changing lanes) can be signaled acoustically.
- the signal allocation device 110 is designed to assign an acoustic signal to the object 200 even if the object 200 itself is not a sound source.
- the signal allocation device 110 further has an input 105 which can be coupled to a sensor 230 for determining the position of the object 200, wherein the sensor 230 is configured to transmit the position of the object 200 to the signal allocation device 110.
- the sensor 230 has a radar or sonar.
- the loudspeaker drive device 120 is designed to detect precisely one loudspeaker signal LS for exactly one loudspeaker 220d, wherein the loudspeaker 220d can be placed in the reproduction space 210 in the direction of the object 200.
- exactly one loudspeaker signal LS drives exactly one other loudspeaker 220 when the object 200 changes position.
- the signal allocation device 110 is designed to assign the object 200 an acoustic signal in a predetermined minimum bandwidth, so that the acoustic signal is clearly acoustically perceptible.
- the inventive scheme can also be implemented in software.
- the implementation may be on a digital storage medium, in particular a floppy disk or a CD with electronically readable control signals, which may interact with a programmable computer system such that the corresponding method is executed.
- the invention thus also consists in a computer program product with program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer.
- the invention can thus be described as a computer program with a program code for carrying out the method be realized when the computer program runs on a computer.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Circuit For Audible Band Transducer (AREA)
- Measurement Of Velocity Or Position Using Acoustic Or Ultrasonic Waves (AREA)
- Radar Systems Or Details Thereof (AREA)
Description
Die vorliegende Erfindung bezieht sich auf eine Vorrichtung und ein Verfahren zur akustischen Anzeige einer Position eines Objekts in einem Wiedergaberaum. Ausführungsbeispiele umfassen insbesondere akustische Displays für den Einsatz auf Schiffen.The present invention relates to an apparatus and method for acoustically displaying a position of an object in a playback room. Exemplary embodiments include in particular acoustic displays for use on ships.
Auf der Brücke bzw. im Maschinenleitstand von mittleren und großen Schiffen befinden sich häufig viele optische Anzeigen (z. B. von Sensoren), die zum einen die Technik des Schiffes überwachen und zum anderen Informationen über die Umgebung über und unter Wasser und insbesondere über Hindernisse liefern. Zur Steuerung des Schiffes befinden sich daher in der Regel mehrere Personen auf der Brücke bzw. auf dem Leitstand. Mit steigender Anzahl von meldenden Sensoren wird es immer wichtiger unterscheidbare Signale zu erzeugen, wobei beispielsweise zwischen Warnungen und Hinweisen zu unterscheiden ist. Neben der optischen Anzeige ist insbesondere eine akustische Meldung wünschenswert. Während selten auftretende Meldungen mittels Sprachausgabe unterstützt werden können, ist die Meldung häufig vorkommender Meldungen, wie sie z. B. Radargeräte oder Echolote liefern, deutlich komplexer. Ein Stand der Technik aus dem Automobilbau wären Abstandssensoren, die Piepstöne variabler Frequenz wiedergeben. Beispielsweise kann die Frequenz mit sinkendem Abstand bei Annäherung an ein Hindernis variabel sein. Bei Schiffen ist dies nicht ausreichend aussagefähig, da bewegliche Hindernisse sich in jeder Richtung befinden und bewegen können.On the bridge or in the machine control center of medium and large ships are often many visual displays (eg of sensors), which monitor the one hand, the technology of the ship and on the other hand, information about the environment above and below water and in particular over obstacles deliver. For control of the ship are therefore usually several people on the bridge or on the control room. As the number of reporting sensors increases, it becomes more and more important to generate distinct signals, for example distinguishing between warnings and cues. In addition to the visual display in particular an acoustic message is desirable. While rarely occurring messages can be supported by means of voice output, the message is frequently occurring messages, such as. As radars or echosounders, much more complex. A state of the art in automotive engineering would be distance sensors that emit beeps of variable frequency. For example, the frequency may be variable with decreasing distance when approaching an obstacle. For ships, this is not sufficiently meaningful because moving obstacles are in any direction and can move.
Die
Die
Ausgehend von diesem Stand der Technik liegt der vorliegenden Erfindung die Aufgabe zugrunde, eine Vorrichtung oder ein Verfahren zur akustischen Anzeige einer Position eines Objekts zu beschaffen.Based on this prior art, the present invention has the object to obtain a device or a method for the acoustic display of a position of an object.
Diese Aufgabe wird durch eine Vorrichtung zur akustischen Anzeige nach Patentanspruch 1, eine Vorrichtung zum Abtasten einer Umgebung nach Patentanspruch 11, ein Verfahren zur akustischen Anzeige nach Patentanspruch 13 oder ein Computerprogramm nach Patentanspruch 14 gelöst.This object is achieved by an acoustic display device according to
Der Kerngedanke der vorliegenden Erfindung besteht darin, dass in einem Wiedergaberaum eine Vielzahl von Lautsprechern räumlich derart unterschiedlich angeordnet ist, dass durch unterschiedliche Ansteuerung der Lautsprecher unterschiedliche Positionen akustisch darstellbar sind. Insbesondere ist eine Signalzuordnungseinrichtung ausgebildet, um dem Objekt ein akustisches Signal zuzuordnen und eine Lautsprecheransteuereinrichtung ist ausgebildet, um eine oder mehrere Lautsprechersignale für die Vielzahl von Lautsprechern zu ermitteln. Das eine oder die mehreren Lautsprechersignale sind derart beschaffen, dass dadurch die Position des Objekts angezeigt wird, wobei das eine oder die mehreren Lautsprechersignale auf dem durch die Signalzuordnungseinrichtung dem Objekt zugeordneten akustischen Signal basiert. Die einen oder mehreren Lautsprechersignale werden so ermittelt, dass bei der Wiedergabe des einen oder der mehreren Lautsprechersignale die Position des Objekts in dem Wiedergaberaum akustisch angezeigt wird.The core idea of the present invention is that a plurality of loudspeakers is spatially arranged so differently in a reproduction room that different positions can be acoustically represented by different activation of the loudspeakers. In particular, a signal allocation device is designed to assign an acoustic signal to the object, and a loudspeaker drive device is designed to determine one or more loudspeaker signals for the multiplicity of loudspeakers. The one or more loudspeaker signals are arranged to indicate the position of the object, wherein the one or more loudspeaker signals are based on the acoustic signal associated with the object by the signal assigning means. The one or more loudspeaker signals are determined such that upon playback of the one or more loudspeaker signals, the position of the object in the playback room is displayed acoustically.
Ausführungsbeispiele der vorliegenden Erfindung beziehen sich ferner darauf, wie mittels intelligenter akustischer Displays Sensorsignale leichter darstellbar und damit sowohl die Sicherheit verbessert als auch laufende Kosten reduziert werden können. Eine weitere Idee der vorliegenden Erfindung beruht darauf, dass ein wesentlicher Teil der Information bei vielen Meldern eine Ortsangabe ist. Als Melder kommen beispielsweise ein Radar, ein Echolot, Seekarten oder Wetterkarten in Betracht und die Ortsangabe bezieht sich dabei beispielsweise auf eine Richtung als auch auf eine Entfernung zu dem Objekt. Zur Meldung oder zur Darstellung der Richtung und der Entfernung wird beispielsweise mittels mehrerer Lautsprecher ein Schallfeld erzeugt, welches möglichst präzise diese Information in natürlicher Weise kodiert.Embodiments of the present invention also relate to how sensor signals can be displayed more easily by means of intelligent acoustic displays and thus both the security can be improved and the running costs can be reduced. Another idea of the present invention is based on the fact that an essential part of the information in many detectors is a location. As a detector, for example, a radar, a depth sounder, nautical charts or weather maps come into consideration and the location refers to, for example, a direction as well as a distance to the object. To report or display the direction and the distance, for example, by means of several speakers a sound field generated, which encodes as precisely as possible this information in a natural way.
Der Kerngedanke der vorliegenden Erfindung besteht darin, dass in einem Wiedergaberaum eine Vielzahl von Lautsprechern räumlich derart unterschiedlich angeordnet ist, dass durch unterschiedliche Ansteuerung der Lautsprecher unterschiedliche Positionen akustisch darstellbar sind. Insbesondere ist eine Signalzuordnungseinrichtung ausgebildet, um dem Objekt ein akustisches Signal zuzuordnen und eine Lautsprecheransteuereinrichtung ist ausgebildet, um eine oder mehrere Lautsprechersignale für die Vielzahl von Lautsprechern zu ermitteln. Das eine oder die mehreren Lautsprechersignale sind derart beschaffen, dass dadurch die Position des Objekts angezeigt wird, wobei das eine oder die mehreren Lautsprechersignale auf dem durch die Signalzuordnungseinrichtung dem Objekt zugeordneten akustischen Signal basiert. Die einen oder mehreren Lautsprechersignale werden so ermittelt, dass bei der Wiedergabe des einen oder der mehreren Lautsprechersignale die Position des Objekts in dem Wiedergaberaum akustisch angezeigt wird.The core idea of the present invention is that a plurality of loudspeakers is spatially arranged so differently in a reproduction room that different positions can be acoustically represented by different activation of the loudspeakers. In particular, a signal allocation device is designed to assign an acoustic signal to the object, and a loudspeaker drive device is designed to determine one or more loudspeaker signals for the multiplicity of loudspeakers. The one or more loudspeaker signals are arranged to indicate the position of the object, wherein the one or more loudspeaker signals are based on the acoustic signal associated with the object by the signal assigning means. The one or more loudspeaker signals are determined such that upon playback of the one or more loudspeaker signals, the position of the object in the playback room is displayed acoustically.
Ausführungsbeispiele der vorliegenden Erfindung beziehen sich ferner darauf, wie mittels intelligenter akustischer Displays Sensorsignale leichter darstellbar und damit sowohl die Sicherheit verbessert als auch laufende Kosten reduziert werden können. Eine weitere Idee der vorliegenden Erfindung beruht darauf, dass ein wesentlicher Teil der Information bei vielen Meldern eine Ortsangabe ist. Als Melder kommen beispielsweise ein Radar, ein Echolot, Seekarten oder Wetterkarten in Betracht und die Ortsangabe bezieht sich dabei beispielsweise auf eine Richtung als auch auf eine Entfernung zu dem Objekt. Zur Meldung oder zur Darstellung der Richtung und der Entfernung wird beispielsweise mittels mehrerer Lautsprecher ein Schallfeld erzeugt, welches möglichst präzise diese Information in natürlicher Weise kodiert.Embodiments of the present invention also relate to how sensor signals can be displayed more easily by means of intelligent acoustic displays and thus both the security can be improved and the running costs can be reduced. Another idea of the present invention is based on the fact that an essential part of the information in many detectors is a location. As a detector, for example, a radar, a depth sounder, nautical charts or weather maps come into consideration and the location refers to, for example, a direction as well as a distance to the object. To report or display the direction and the distance, a sound field is generated, for example by means of several speakers, which encodes this information as precisely as possible in a natural way.
Im Zusammenwirken mit den bisher verwendeten optischen Anzeigen von Radar und Echolot ist es dabei sinnvoll, bei der akustischen Darstellung der Umgebung nur die wichtigsten oder die wichtigen Objekte zu augmentieren. Dies sind Objekte, die sich beispielsweise nähern oder deren Kurs den Kurs des Schiffes kreuzen, so dass die Gefahr der Kollision besteht.In conjunction with the previously used optical displays of radar and echosounder, it makes sense to augment only the most important or important objects in the acoustic representation of the environment. These are objects that approach, for example, or whose course crosses the course of the ship, so there is a risk of collision.
Basierend auf Wiedergabesystemen für räumliche Audiosignale im Unterhaltungsbereich und im Bereich der virtuellen Realität ist es damit möglich, auch in kleinen Räumen die Wände virtuell verschwinden zu lassen, so dass die Position eines Objekts (Entfernung und Richtung) auch außerhalb des Wiedergaberaums präzise zu hören ist.Based on spatial audio playback systems in the entertainment area and in the area of virtual reality, it is possible to make the walls virtually disappear even in small rooms, so that the position of an object (distance and direction) can be heard precisely even outside the playback room.
Bei der Ansteuerung der Lautsprecher ergeben sich prinzipiell zwei Möglichkeiten:
- (i) Wellenfeldsynthese (WFS): bei diesem System befinden sich die Lautsprecher beispielsweise in einem konstanten Abstand und die Berechnung der einzelnen Signale für die Lautsprecher erfolgt nach den bekannten WFS-Algorithmen. Objekte aus einem Radarsignal werden dabei als akustische Objekte in entsprechender Richtung und Entfernung wiedergegeben. Die Objekte erscheinen somit als virtuelle Schallquellen und können durch einen Hörer lokalisiert werden. Alle Personen auf der Brücke können dabei beispielsweise die Objekte am gleichen Ort wahrnehmen. Möglich ist auch, dass nicht nur ein einzelnes Objekt, sondern dass auch mehrere Objekte gleichzeitig akustisch dargestellt werden, wobei jedem Objekt beispielsweise ein anderes oder optional auch ein gleiches akustisches Signal zugeordnet werden kann.
- (ii) Zeit- und Amplitudenpanning (ZAP): bei diesem Verfahren wird ein akustisches Schallsignal in der Amplitude und Phase für die einzelnen Lautsprecher derart geändert, dass das akustische Signal aus einer bestimmten Richtung und in einer bestimmten Entfernung erscheint. Es ist bei diesem System möglich, einen größeren bzw. unterschiedliche Abstände zwischen den Lautsprechern zu erlauben. Diese Methode hat gegenüber der WSF den Vorteil, dass weniger Lautsprecher erforderlich sind, aber den Nachteil, dass der akustische Ort einer Schallquelle weniger präzise wahrgenommen wird. Eventuell kann der wahrgenommene Ort der Schallquelle auch etwas von dem Ort der hörenden Person abhängen.
- (i) Wave Field Synthesis (WFS): in this system, for example, the speakers are at a constant distance and the individual signals for the loudspeakers are calculated according to the well-known WFS algorithms. Objects from a radar signal are reproduced as acoustic objects in the corresponding direction and distance. The objects thus appear as virtual sound sources and can be localized by a listener. For example, all persons on the bridge can perceive the objects in the same place. It is also possible that not only a single object but also several objects are displayed acoustically at the same time, wherein each object, for example, another or optionally also a same acoustic signal can be assigned.
- (ii) Time and Amplitude Panning (ZAP): In this method, an acoustic sound signal becomes amplitude and phase for the individual speakers changed so that the acoustic signal from a certain direction and at a certain distance appears. It is possible with this system to allow a larger or different distances between the speakers. This method has the advantage over the WSF of requiring less loudspeakers, but the disadvantage of perceiving the acoustic location of a sound source less precisely. Eventually, the perceived location of the sound source may also depend somewhat on the location of the person listening.
Um ein Radarsignal akustisch darzustellen, wird es zunächst akustisch aufgearbeitet. Die Aufarbeitung umfasst dabei zum einen das Erkennen von beweglichen Objekten, wie beispielsweise Schiffe und Flugzeuge, und ferner das Erkennen von statischen Objekten, wie beispielsweise die Küstenlinie, Bojen oder Inseln. Bei Objekten, die einen Transponder enthalten und sich mit einem Text (Textmitteilung oder allgemein Daten) identifizieren, kann das Audiosignal optional mittels einer Text-zu-Sprache-Identifikation in ein Audiosignal umgesetzt werden, so dass das Textsignal des Transponders hörbar wird. Solche Objekte sind z. B. bestimmt Bojen oder Leuchtfeuer, deren identifizierende Information beispielsweise auf dem Radar als Text erscheinen.To acoustically represent a radar signal, it is first processed acoustically. The workup includes on the one hand the detection of moving objects, such as ships and aircraft, and also the detection of static objects, such as the coastline, buoys or islands. For objects that contain a transponder and identify themselves with a text (text message or general data), the audio signal can optionally be converted into an audio signal by means of a text-to-speech identification, so that the text signal of the transponder becomes audible. Such objects are z. B. determines buoys or beacons, whose identifying information appear, for example, on the radar as text.
Objekte können weiterhin entsprechend ihres Gefahrenpotentials klassifiziert werden. Dabei können beispielsweise Objekte, die näher kommen (von Vorne oder schneller von Hinten) oder den Bewegungspfad des Schiffes kreuzen, als gefährlicher klassifiziert werden, als Objekte, die parallel zum Schiff verlaufen oder sich vom Schiff entfernen. Objekte, die weiter entfernt sind, gelten in der Regel als weniger gefährlich als solche Objekte, die nahe sind bzw. die sich in einer großen Relativgeschwindigkeit nähern. Je nach Gefahr kann somit den Objekten ein unterschiedlicher Kennungston zugewiesen werden, wobei sich der Kennungston beispielsweise in Tonhöhe oder in der Impulsfolgefrequenz unterscheidet und sich erhöhen, wenn die Gefahr zunimmt. Somit kann ein höherer Ton eine größere Gefahr bedeuten oder eine zunehmende Lautstärke eine steigende Gefahr implizieren. Ähnlich kann ein schneller schlagender Taktimpuls eine steigende oder eine höhere Gefahr bedeuten als ein niedriger Taktimpuls (wenn beispielsweise der Kennungston als ein rhythmischer Taktimpuls dargestellt wird).Objects can still be classified according to their hazard potential. For example, objects that come closer (from the front or faster from the back) or cross the ship's path of movement may be classified as more dangerous than objects that run parallel to the ship or are moving away from the ship. Objects that are farther away are generally considered less dangerous than those that are near or approaching at a high relative speed. Depending on the risk can thus be assigned to the objects a different identifier tone, with the identification tone for example in pitch or in the pulse repetition frequency and increase as the danger increases. Thus, a higher tone may mean greater danger or increasing volume may imply an increasing danger. Similarly, a faster beating clock pulse may mean a rising or a higher hazard than a lower clock pulse (for example, if the signature sound is represented as a rhythmic clock pulse).
Die so erzeugten Audiosignale der Objekte werden dann beispielsweise durch die oben genannte WFS oder ZAP wiedergegeben, wodurch automatisch weit entfernte Objekte leiser werden.The audio signals of the objects thus generated are then reproduced, for example, by the above-mentioned WFS or ZAP, whereby automatically far distant objects become quieter.
Bei weiteren Ausführungsbeispielen werden in besonderen Umgebungen, wie beispielsweise Schifffahrtsstraßen, ungefährliche Objekte komplett ausgeblendet (nicht wiedergegeben), um den Steuermann oder den Hörer nicht mit zuviel Information zu überlasten.In other embodiments, in particular environments, such as shipping lanes, non-hazardous objects are completely blanked out (not shown) so as not to overload the helmsman or the listener with too much information.
Ferner kann bei Ausführungsbeispielen der Wiedergabeort in gleicher Entfernung erscheinen wie die tatsächliche Entfernung ist, d. h. wenn das Objekt laut Radar ein Kilometer entfernt ist, ist das Audioobjekt in ein Kilometer Entfernung wahrnehmbar (1:1-Abbildung). Alternativ wird der Wiedergabeort entsprechend skaliert, so dass beispielsweise eine 1:100-Abbildung vorgenommen wird und ein ein Kilometer entferntes Objekt akustisch durch ein ca. zehn Meter entferntes akustisches Signal (virtuelle Schallquelle) wahrnehmbar ist oder wiedergegeben wird. Ersteres (die 1:1-Abbildung) weist beispielsweise den Vorteil auf, dass bei der WFS keinerlei Parallaxenfehler auftreten, so dass die Entfernung des Objekts nur noch durch die Lautstärke kodiert wird und nicht mehr durch die gekrümmte Wellenform. Sehr weit entfernte Objekte wären allerdings infolge der Schallgeschwindigkeit erst sehr spät hörbar und außerdem sind bei einer 1:1-Darstellung sehr weit entfernte Objekte kaum entfernungsmäßig unterscheidbar.Further, in embodiments, the replay location may appear at the same distance as the actual distance, ie, if the object is one kilometer away by radar, the audio object is perceptible at one kilometer distance (1: 1 mapping). Alternatively, the playback location is scaled accordingly so that, for example, a 1: 100 mapping is made and an object one kilometer away is acoustically perceptible or reproduced by an acoustic signal (virtual sound source) approximately ten meters away. For example, the former (the 1: 1 image) has the advantage that no parallax errors occur in the WFS, so that the distance of the object is coded only by the volume and not by the curved waveform. Very distant objects, however, would only be audible very late due to the speed of sound, and furthermore, in a 1: 1 representation, very distant objects are hardly distinguishable by distance.
Ausführungsbeispiele verfolgen somit das Ziel Objekte mit Audiosignalen zu kodieren, damit sie möglichst gut ortbar sind. Um dies zu erzielen, sollten die Audiosignale hinreichend breitbandig sein, da beispielsweise ein Sinuston nur schwer wahrnehmbar ist. Dementsprechend sollten eher Schmalbandrauschen oder Sprache zur Identifizierung von Objekten - nicht aber ein Sinuston - benutzt werden. Um in dichten Umgebungen, wie beispielsweise Schifffahrtsstraßen, eine hohe Anzahl von Objekten wiedergeben zu können und darüber hinaus auch akustisch wahrnehmen zu können, werden statt kontinuierlicher Signale (z. B. ein Dauerton) gepulste Signale ausgesendet. Die Pulsfrequenz kann dabei ähnlich zu Parksensoren bei Autos mit steigender Gefahr steigen. Um eine dauerhafte Nutzung zu ermöglichen, sollten die Audiosignale angenehm klingen, wenn die Gefahr hinreichend niedrig ist. Die Gefahrenschwelle, oberhalb derer eine ernsthafte Gefahr bzw. unterhalb derer kein oder kaum Gefahrenpotential besteht, wird dabei beispielsweise variabel entsprechend den Gegebenheiten eingestellt. Die Gefahrenschwelle kann optional auch durch den Nutzer angepasste werden. Zum Beispiel spielen die Größe und die Geschwindigkeit eines Schiffes oder aber auch die Geschwindigkeiten der anderen Objekte eine Rolle. Der Schwellwert kann beispielsweise aus dem Verhältnis der Zeitdauer bis zu einer vorausberechneten Kollision zu einer Bremszeit des Schiffes ermittelt werden.Exemplary embodiments thus pursue the goal of coding objects with audio signals, so that they can be located as well as possible. To achieve this, the audio signals should be sufficiently broadband, for example, because a sine wave is difficult to perceive. Accordingly, narrowband noise or speech should be used to identify objects, not sinusoidal ones. In order to be able to reproduce a high number of objects in dense environments, such as, for example, shipping lanes, and in addition to be able to perceive acoustically, pulsed signals are emitted instead of continuous signals (eg a continuous tone). The pulse rate can rise similarly to parking sensors in cars with increasing risk. In order to allow for permanent use, the audio signals should sound pleasant when the danger is sufficiently low. The danger threshold, above which there is a serious danger or below which there is no risk or hardly any danger, is set variably in accordance with the circumstances, for example. The danger threshold can optionally also be adapted by the user. For example, the size and speed of a ship or the speeds of the other objects play a role. The threshold value can be determined, for example, from the ratio of the time duration to a predicted collision to a braking time of the ship.
Der angenehme Klang der Audiosignale kann beispielsweise dadurch erreicht werden, dass bei nicht identifizierten Objekten (z. B. Objekte, die keine Gefahr darstellen) eine niedrige Mittenfrequenz des Schmalbandrauschens oder eine niedrige Pulsfrequenz (seltene Darstellung) verwendet wird. Alternativ kann auch eine spektrale Färbung des Schmalbandrauschens verwendet werden, bei der hohe Frequenzen weniger Energie haben als tiefe (Ausschneiden mit Bandpass aus rosa Rauschen). Bei identifizierten Objekten wird dies durch seltenes Melden erreicht, z. B. bei Erstkontakt um dann nur im Minutenabstand ein neues Signal zu senden.The pleasant sound of the audio signals can be achieved, for example, by using a low center frequency of the narrowband noise or a low pulse frequency (rare representation) for unidentified objects (eg objects that pose no danger). Alternatively, a spectral hue of narrowband noise can be used where high frequencies have less energy than deep (cut with pink noise bandpass). For identified objects, this is done Rarely reporting reached, eg. B. at first contact to then send only a minute away a new signal.
Das meldende Signal kann optional derart ausgewählt werden, dass es präzise zu orten ist und von Umgebungsgeräuschen unterscheidbar ist. Außerdem ist es vorteilhaft, wenn das meldende Signal einen angenehmen Klang hat, so dass auch bei langen Fahrten das System dauerhaft akzeptiert wird. Ein wesentlicher Vorteil akustischer, räumlich auflösender Displays ist, dass sie anders als optische Displays simultan mit der natürlichen Umwelt von einer Person verwendet werden können. Die natürliche Umwelt kann dabei beispielsweise ein Fahren auf Sicht oder auch das Hören von Schiffen und Bojen umfassen. Somit kann eine sogenannte augmented reality erzeugt werden.The reporting signal may optionally be selected to be precisely located and distinguishable from ambient noise. Moreover, it is advantageous if the reporting signal has a pleasant sound, so that even with long trips, the system is permanently accepted. An essential advantage of acoustic, spatially resolving displays is that, unlike optical displays, they can be used by one person simultaneously with the natural environment. The natural environment may include, for example, driving on sight or listening to ships and buoys. Thus, a so-called augmented reality can be generated.
Ausführungsbeispiele sind insbesondere deshalb vorteilhaft, da sie einen wichtigen Synergieeffekt zwischen akustischer und optischer Anzeige liefern. Die akustische Anzeige wird nämlich immer gemeldet und wahrgenommen, wobei eine Priorisierung nach Gefahr erfolgen kann, währenddessen die optische Anzeige die Aufmerksamkeit des Personals auf der Brücke erfordert. Ein Steuermann sieht beispielsweise nur dann ein Objekt auf dem Radarschirm, wenn er auf den Radarschirm blickt. Gleichzeitig sieht er aber nicht mehr aus dem Fenster und verliert damit einen Teil der Information, was in seinem näheren Umfeld passiert. Akustische Displays ermöglichen ihm gleichzeitig die Information aus dem Radar und den Blick aus dem Fenster zu nutzen. Gerade bei nicht sich selbst identifizierenden Objekten ist aber der erfahrene Auswerter in der Lage, aus dem Radarbild ein Objekt zu klassifizieren (z. B. als Schiff, Insel oder Bildstörung). Somit steckt in dem Zusammenwirken aus der akustischen Wahrnehmung (da ist ein Objekt) und dem Blick auf dem Radarschirm zur Kontrolle ein wichtiger Synergieeffekt. Bei weit entfernten, sich selbst identifizierenden Objekten kann durch einen Blick auf den Radarschirm jederzeit die Identifikation gelesen werden.Embodiments are particularly advantageous because they provide an important synergy effect between acoustic and visual display. Namely, the audible indication is always reported and perceived, whereby prioritization for danger may occur while the visual indication requires the attention of the personnel on the bridge. For example, a helmsman sees an object on the radar screen only when he looks at the radar screen. At the same time, however, he no longer looks out of the window and thus loses some of the information about what is happening in his immediate surroundings. Acoustic displays allow him to simultaneously use the information from the radar and the view from the window. Especially in the case of non-self-identifying objects, however, the experienced evaluator is able to classify an object from the radar image (eg as a ship, island or picture disturbance). Thus, in the interaction of the acoustic perception (there is an object) and the view on the radar screen to control an important synergy effect. For remote, self-identifying objects, the identification can be read at any time by looking at the radar screen.
Ausführungsbeispiele der vorliegenden Erfindung werden nachfolgend Bezug nehmend auf die beiliegenden Zeichnungen näher erläutert. Es zeigen:
- Fig. 1
- eine schematische Darstellung einer Vorrichtung zur akustischen Anzeige gemäß einem Ausführungsbeispiel der vorliegenden Erfindung;
- Fig. 2
- eine Darstellung eines erfindungsgemäßen Systems mit einem Sensor zur Bestimmung der Position eines Objekts;
- Fig. 3a und 3b
- Darstellungen von ortsabhängigen Signalen, um eine ansteigende Gefahr akustisch wahrzunehmen;
- Fig. 4
- ein Ausführungsbeispiel mit einer Vielzahl von Lautsprechern zur akustischen Darstellung von zwei getrennten Objekten;
- Fig. 5
- eine schematische Darstellung eines Wiedergaberaums mit einem WFS-Modul; und
- Fig. 6
- ein prinzipielles Blockschaltbild eines Wellenfeldsynthesesystems mit Wellenfeldsynthesemodulen und Lautsprecherarrays in einem Wiedergaberaum.
- Fig. 1
- a schematic representation of an apparatus for acoustic display according to an embodiment of the present invention;
- Fig. 2
- a representation of a system according to the invention with a sensor for determining the position of an object;
- Fig. 3a and 3b
- Representations of location-dependent signals to acoustically perceive an increasing danger;
- Fig. 4
- an embodiment with a plurality of speakers for the acoustic representation of two separate objects;
- Fig. 5
- a schematic representation of a playback room with a WFS module; and
- Fig. 6
- a schematic block diagram of a wave field synthesis system with wave field synthesis modules and speaker arrays in a playback room.
Bezüglich der nachfolgend Beschreibung sollte beachtet werden, dass bei den unterschiedlichen Ausführungsbeispielen gleiche oder gleichwirkende Funktionselemente gleiche Bezugszeichen aufweisen und somit die Beschreibung dieser Funktionselemente in den verschiedenen, in den nachfolgend dargestellten Ausführungsbeispielen untereinander austauschbar sind.With regard to the following description, it should be noted that in the different embodiments, the same or equivalent functional elements have the same reference numerals and thus the description of these functional elements in the various, in the following illustrated embodiments are interchangeable.
Ferner weist die Vorrichtung 100 eine Lautsprecheransteuereinrichtung 120 auf, die von der Signalzuordnungseinrichtung 110 die Position des Objekts und das akustische Signal erhält, um daraus ein oder mehrere Lautsprechersignale LS für eine Vielzahl von Lautsprechern zu ermitteln und diese über die Ausgänge für die Lautsprechersignale LS1, ..., LSn auszugeben. Die Lautsprecheransteuereinrichtung 120 ist ausgebildet, um die einen oder die mehreren Lautsprechersignale LS basierend auf das akustische Signal, welches dem Objekt zugeordnet wurde, zu ermitteln. Die Ermittlung wird derart ausgeführt, dass bei Wiedergabe des einen oder der mehreren Lautsprechersignale LS die Position des Objekts in dem Wiedergaberaum akustisch angezeigt wird. Ein Hörer (oder Nutzer) nimmt dann die Position (z. B. Entfernung und Richtung) des Objekts als Position einer virtuellen Schallquelle war.Furthermore, the
Ein Ausführungsbeispiel betrifft wie gesagt die Wiedergabe von Informationen eines Radargeräts, das Positionen von Objekten ermittelt. Zusätzlich zum oder anstatt des Radars können auch Informationen beispielsweise aus anderen Quellen, wie Echolots, oder anderen Sensoren in ähnlicher Weise umgesetzt werden. Bei diesem Ausführungsbeispiel, das im Folgenden beispielhaft genauer beschrieben werden soll, können beispielsweise Lautsprecher auf der Brücke des Schiffes unterhalb Fenster (eventuell auch zusätzlich oberhalb der Fenster) an allen Wänden angeordnet sein. Diese Lautsprecher können beispielsweise alle mit eigenen Verstärkern oder mit A/D-Umsetzer (Analog-Digital-Wandlern) ausgestattet sein und können darüber hinaus einzeln angesteuert werden. Es ist dabei besonders vorteilhaft, wenn eine möglichst komplette Umschließung des Personals auf der Brücke mit Lautsprechern erreicht wird, wobei für die zivile Seefahrt eine ebene Umschließung (Kreis) und für militärische Anwendungen eventuell auch eine räumliche Umschließung (Halbkugel) nützlich ist bzw. angestrebt wird. Die Umschließung braucht dabei nicht vollständig zu sein und kleinere Lücken in der Umschließung, die beispielsweise durch vorhandene Türen gegeben sind, wären auch möglich.As mentioned above, one embodiment relates to the reproduction of information of a radar device which determines positions of objects. In addition to or instead of the radar, information from, for example, other sources, such as echosounders, or other sensors are implemented in a similar manner. In this embodiment, which will be described in more detail below by way of example, loudspeakers on the bridge of the ship below windows (possibly also additionally above the windows) may be arranged on all walls. These loudspeakers, for example, can all be equipped with their own amplifiers or with A / D converters (analog-to-digital converters) and can also be individually controlled. It is particularly advantageous if the most complete possible enclosure of the staff is achieved on the bridge with speakers, for the civilian navigation a flat enclosure (circle) and for military applications, possibly a spatial enclosure (hemisphere) is useful or sought , The enclosure does not need to be complete, and smaller gaps in the enclosure, such as those provided by existing doors, would also be possible.
Die Vorrichtung zur akustischen Anzeige 100 wertet die Positionsangaben des Objekts 200, die sie von dem Radargerät 230 erhält, aus, um daraus drei Lautsprechersignale LS1, LS2, LS3 für den ersten, zweiten und dritten Lautsprecher 220a, 220b, 220c zu ermitteln. Die Ermittlung geschieht derart, dass die Position des Objekts 200 für den Zuhörer in dem Wiedergaberaum 210, der sich beispielsweise an einer Position P befindet, hörbar ist. Dazu ermittelt zunächst die Vorrichtung 100 ein akustisches Signal für das Objekt 200 in Abhängigkeit der Position des Objekts 200. Die Position ist durch die Entfernung d und die Richtung, die beispielsweise über einen Winkel α angegeben werden kann, bestimmt. Als nächstes berechnet die Vorrichtung 100 Lautsprechersignale LS für den ersten bis dritten Lautsprecher 220a bis 220c. Das kann beispielsweise ein Skalieren des Signalpegels und ein Verzögern des Signals umfassen, so dass der Zuhörer an der Position P das Objekt 200 entsprechend seiner Position wahrnimmt. Bei dem in der
Das in der
Die
In der
Der Anstieg in der Frequenz kann optional auch monoton mit abnehmender Entfernung des Objekts zunehmen, ohne dass es zu einem plötzlichen Änderung bei der kritischen Distanz kommt und ein ständig steigerndes Gefahrenpotential für den Steuermann wahrnehmbar wird.The increase in frequency can optionally also increase monotonously with decreasing distance of the object without causing a sudden change in the critical distance and a constantly increasing danger potential for the helmsman becomes perceptible.
Das akustische Signal oder die Frequenz f des akustischen Signals kann dabei zum einen die Tonfrequenz umfassen oder aber auch die Taktfrequenz, wenn beispielsweise das akustische Signal einen bestimmten Takt in einer bestimmten Frequenz (Wiederholrate der Takte) angibt. Auch bei dem Taktsignal kann mit abnehmender Entfernung die Taktfrequenz zunehmen, so dass dadurch akustisch ein zunehmendes Gefahrenpotential für den Steuermann wahrnehmbar wird.The acoustic signal or the frequency f of the acoustic signal may include on the one hand, the audio frequency or else the clock frequency, for example, if the acoustic signal indicates a particular clock in a particular frequency (repetition rate of the clocks). Even with the clock signal can increase with decreasing distance, the clock frequency, so that acoustically an increasing risk potential for the helmsman is perceived.
Im Allgemeinen ist es vorteilhaft, wenn in einem gefahrlosen Zustand die akustischen Signale kaum wahrnehmbar sind, so dass der Steuermann durch die akustischen Signale nicht gestört wird.In general, it is advantageous if in a safe state, the acoustic signals are barely perceptible, so that the helmsman is not disturbed by the acoustic signals.
Beispielsweise können, wie in der
Alternativ ist es ebenfalls möglich, dass die jeweils benachbarten Lautsprecher, zwischen denen die Verbindungslinie zwischen dem ersten Objekt 200a und der Position P entlang läuft, aktiv sind. Es können darüber hinaus noch weitere Nachbarlautsprecher aktiv sein. Dies bedeutet, dass beispielsweise bei weiteren Ausführungsbeispielen nicht nur der vierte Lautsprecher 220d aktiv ist, sondern gleichzeitig auch der dritte Lautsprecher 220c und/oder der zweite Lautsprecher 220b und/oder der fünfte Lautsprecher 220e aktiv sein können. Wenn jedoch mehrere Lautsprecher gleichzeitig aktiv sind, um die Position eines der Objekte 200 darzustellen, ist die Amplitude/Phase derart zu wählen, dass für einen Zuhörer an der Position P das Objekt 200 an seiner jeweiligen Position akustisch wahrnehmbar wird. Akustische Wahrnehmbarkeit bedeutet dabei, dass das Objekt 200 als eine virtuelle Schallquelle wahrgenommen wird, wobei die Entfernung neben der Lautstärke auch durch eine verschiedene Taktfrequenz oder Tonfrequenz signalisiert sein kann (wie es beispielsweise in der
Der Aufbau eines WFS-System ist im Allgemeinen sehr komplex und basiert auf die Wellenfeldsynthese. Die Wellenfeldsynthese ist ein an der TU Delft entwickeltes Audio-Reproduktionsverfahren zur räumlichen Wiedergabe komplexer Audioszenen. Im Gegensatz zu den meisten existierenden Verfahren zur Audio-Reproduktion ist die räumlich korrekte Wiedergabe nicht auf ein kleines Gebiet beschränkt, sondern erstreckt sich über ein ausgedehntes Wiedergabegebiet. WFS basiert auf einer fundierten mathematisch-physikalischen Grundlage, namentlich dem Prinzip von Huygens und dem Kirchhoff-Helmholtz-Integral.The structure of a WFS system is generally very complex and based on wave field synthesis. Wave field synthesis is an audio reproduction method developed at the TU Delft for the spatial reproduction of complex audio scenes. In contrast to most existing audio reproduction techniques, the spatially correct rendering is not limited to a small area, but extends over a wide viewing area. WFS is based on a well-founded mathematical-physical basis, namely the principle of Huygens and the Kirchhoff-Helmholtz integral.
Typischerweise besteht ein WFS-Reproduktionssystem aus einer großen Zahl von Lautsprechern (sog. sekundärer Quellen). Die Lautsprechersignale werden aus verzögerten und skalierten Eingangssignalen gebildet. Da in einer WFS-Szene typischerweise viele Audioobjekte (primäre Quellen) verwendet werden, sind sehr viele solcher Operationen zur Erzeugung der Lautsprechersignale erforderlich. Dies bedingt die hohe für die Wellenfeldsynthese benötigte Rechenleistung.Typically, a WFS reproduction system consists of a large number of loudspeakers (so-called secondary sources). The loudspeaker signals are formed from delayed and scaled input signals. Since many audio objects (primary sources) are typically used in a WFS scene, many such operations are required to generate the loudspeaker signals. This requires the high computing power required for wave field synthesis.
Neben den oben genannten Vorteilen bietet die WFS auch die Möglichkeit, bewegte Quellen realistisch abzubilden. Dieses Feature wird in vielen WFS-Systemen genutzt und ist beispielsweise für den Einsatz im Kino, Virtual-Reality-Applikationen oder Live-Aufführungen von großer Bedeutung.In addition to the advantages mentioned above, WFS also offers the possibility of realistically mapping moving sources. This feature is used in many WFS systems and is for example for use in the cinema, virtual reality applications or live performances of great importance.
Jedoch verursacht die Wiedergabe bewegter Quellen eine Reihe charakteristischer Fehler, die im Falle statischer Quellen nicht auftreten. Die Signalverarbeitung eines WFS-Wiedergabesystems hat dabei einen bedeutenden Einfluss auf die Wiedergabequalität.However, the playback of moving sources causes a number of characteristic errors that do not occur in the case of static sources. The signal processing of a WFS playback system has a significant influence on the playback quality.
Ein primäres Ziel ist die Entwicklung von Signalverarbeitungsalgorithmen für die Wiedergabe bewegter Quellen mittels WFS. Dabei ist die Echtzeitfähigkeit der Algorithmen eine wichtige Bedingung. Das wichtigste Kriterium zur Bewertung der Algorithmen ist die objektive wahrgenommene Audio-Qualität.A primary goal is the development of signal processing algorithms for the playback of moving sources using WFS. The real-time capability of the algorithms is an important condition. The most important criterion for evaluating the algorithms is the objective perceived audio quality.
WFS ist wie gesagt ein bezüglich Verarbeitungsressourcen sehr aufwändiges Verfahren zur Audio-Reproduktion. Dies wird vor allem durch die große Zahl von Lautsprechern in einem WFS-Setup sowie die oftmals hohe Zahl virtueller Quellen, die in WFS-Szenen verwendet werden, bedingt. Aus diesem Grund ist die Effizienz der zu entwickelnden Algorithmen von herausragender Bedeutung.As I said, WFS is a very expensive audio reproduction process in terms of processing resources. This is mainly due to the large number of speakers in a WFS setup and the often high number of virtual sources used in WFS scenes. For this reason, the efficiency of the algorithms to be developed is of paramount importance.
Wellenfeldsynthesesysteme haben im Vergleich zu üblichen Mehrlautsprechersystemen den Vorteil, dass dadurch eine exakte Positionierung möglich wird und die exakte Positionierung auch an verschiedenen Positionen innerhalb des Wiedergaberaums 210 bestimmbar ist.Wave field synthesis systems have the advantage, in comparison to conventional multi-speaker systems, that exact positioning becomes possible as a result and exact positioning can also be determined at different positions within the
In
Jedes Lautsprecherarray 221 besteht aus einer Anzahl von verschiedenen Einzellautsprechern 708, die jeweils mit eigenen Lautsprechersignalen LS angesteuert werden, die von einem Wellenfeldsynthesemodul 710 über einen in
Bei Ausführungsbeispielen, die WFS oder auch ZAP zur Ansteuerung der Lautsprecher nutzen, ist die Signalzuordnungseinrichtung 110 ausgebildet ist, um mehreren Objekten 200 akustische Signale zuzuordnen und die Lautsprecheransteuereinrichtung 120 ist ausgebildet, um für jedes der mehreren Objekte 200 Komponentensignale zu erzeugen und die Komponentensignale zu Lautsprechersignalen LS zu kombinieren, so dass die mehreren Objekte 200 akustisch an verschiedenen Positionen wahrnehmbar sind. Die verschiedenen Objekte können dabei wie oben beschrieben als virtuelle Quellen (Schallquellen) für die Zuhörer erscheinen oder wahrgenommen werden.In embodiments using WFS or ZAP to drive the loudspeakers, the
Ausführungsbeispiele können beispielsweise wie folgt ergänzt oder modifiziert werden. So werden bei weiteren Ausführungsbeispielen ebenfalls Randbedingungen in den Schiffen berücksichtigt. Die Randbedingungen umfassen beispielsweise Anforderungen an die Häufigkeit der Meldungen, mögliche Positionen der Lautsprecher, der nötige Schalldruckpegel, die Charakterisierung des Störschalls (z. B. vom Motor) sowie eine Spezifikation der Ansteuersignale für die akustische Anzeige.Exemplary embodiments can be supplemented or modified, for example, as follows. Thus, in further embodiments also boundary conditions are considered in the ships. The boundary conditions include, for example, requirements for the frequency of the messages, possible positions of the loudspeakers, the required sound pressure level, the characterization of the noise (for example from the engine) and a specification of the control signals for the acoustic display.
Unter Nutzung einer Datenbank können dann optimale Meldesignale unter Berücksichtigung typischer räumlicher Klänge auf den Schiffen erzeugt werden.Using a database, optimal message signals can then be generated taking into account typical spatial sounds on the ships.
Bei Ausführungsbeispielen umfasst die akustische Ansteuerung Techniken wie beispielsweise die binaurale Kodierung oder die oben beschriebene Wellenfeldsynthese. Dabei werden die verschiedenen Techniken anhand von Testaufbauten in Schiffen (oder Eins-zu-Eins-Modellen von der Brücke und/oder des Leitstandes) genutzt. Psychoakustische Experimente können beispielsweise entsprechende Hinweise liefern.In embodiments, the acoustic drive includes techniques such as binaural coding or the wave field synthesis described above. The different techniques are used on test rigs in ships (or one-to-one models of the bridge and / or the control room). For example, psychoacoustic experiments can provide clues.
Ausführungsbeispiele setzen Meldesignale ein, die im Schiffsumfeld möglichst gut zu orten sind, gleichzeitig aber möglichst angenehm klingen. Hierbei sind Testaufbauten im Labor oder aber ein Eins-zu-Eins-Modell von der Brücke und/oder des Leitstandes oder in Fahrzeugen sowie psychoakustische Experimente nützlich.Embodiments use reporting signals that are as well as possible to locate in the ship environment, but at the same time sound as pleasant as possible. In this case test setups in the laboratory or else a one-to-one model from the bridge and / or the control station or in vehicles as well as psychoacoustic experiments are useful.
Weitere Ausführungsbeispiele liefern ferner eine Anbindung von Sensoren und Informationen, die beispielsweise von Radar, Echolot und Seekarten erhalten werden, an die akustische Anzeige. Ein wesentlicher Teil der Anbindung ist dabei die Auswahl der relevanten Objekte, die beispielsweise mittels akustischer Anzeige dargestellt werden sollten.Further embodiments also provide a connection of sensors and information, for example, from Radar, sounder and nautical charts are received, to the audible indicator. An essential part of the connection is the selection of the relevant objects, which should be displayed for example by means of acoustic display.
Ausführungsbeispiele umfassen zusammenfassend beispielsweise die folgenden Aspekte:
- (a) Einsatz akustischer Displays in Schiffen;
- (b) Anbindung von Radar, Echolot und Seekarten an akustische Displays;
- (c) Anbindung von Wetterkarten an akustische Displays;
- (d) Anbindung von Funkbojen an akustische Displays für Schiffe;
- (e) Auswahl von Objekten nach Wichtigkeit, insbesondere bezüglich des Ortes und der relativen oder der absoluten Geschwindigkeit des Schiffes sowie der Objekte (Schiffe, Unterwasserhindernisse, etc.); und
- (f) Auswahl wohlklingender Meldesignale.
- (a) use of acoustic displays in ships;
- (b) connection of radar, depth sounder and nautical charts to acoustic displays;
- (c) connection of weather maps to acoustic displays;
- (d) connecting radio buoys to acoustic displays for ships;
- (e) selecting objects according to importance, in particular with respect to the location and the relative or absolute speed of the ship as well as the objects (ships, underwater obstacles, etc.); and
- (f) Selection of melodious message signals.
Schließlich können die beschriebenen Systeme auch in Automobilen angewendet werden, d.h. weitere Ausführungsbeispiele umfassen ebenfalls entsprechende Systeme zur Fahrerassistenz im Auto. Beispielweise können damit seitlich sich nähernde Fahrzeuge (z. B. beim Spurwechsel) akustisch signalisiert werden.Finally, the described systems can also be applied in automobiles, i. Further embodiments also include corresponding driver assistance systems in the car. For example, vehicles approaching laterally (eg when changing lanes) can be signaled acoustically.
Bei einem Ausführungsbeispiel der Vorrichtung 100 ist die Signalzuordnungseinrichtung 110 ausgebildet, um dem Objekt 200 auch dann ein akustisches Signal zuzuordnen, wenn das Objekt 200 selbst keine Schallquelle ist.In one embodiment of the
Bei einem Ausführungsbeispiel der Vorrichtung 100 hat die Signalzuordnungseinrichtung 110 ferner einen Eingang 105, der mit einem Sensor 230 zur Positionsbestimmung des Objektes 200 koppelbar ist, wobei der Sensor 230 ausgebildet ist, die Position des Objekts 200 an die Signalzuordnungseinrichtung 110 zu übermitteln. Insbesondere weist der Sensor 230 ein Radar oder Sonar auf.In one embodiment of the
Bei einem Ausführungsbeispiel der Vorrichtung 100 ist die Lautsprecheransteuereinrichtung 120 ausgebildet, um genau ein Lautsprechersignal LS für genau einen Lautsprecher 220d zu ermitteln, wobei der Lautsprecher 220d im Wiedergaberaum 210 in Richtung des Objekts 200 platzierbar ist. Optional steuert das genau eine Lautsprechersignal LS genau einen anderen Lautsprecher 220 an, wenn das Objekt 200 seine Position ändert.In one embodiment of the
Bei einem Ausführungsbeispiel der Vorrichtung 100 ist die Signalzuordnungseinrichtung 110 ausgebildet, um dem Objekt 200 ein akustisches Signal in einer vorbestimmten Mindestbandbreite zuzuordnen, so dass das akustische Signal deutlich akustisch wahrnehmbar ist.In one embodiment of the
Insbesondere wird darauf hingewiesen, dass abhängig von den Gegebenheiten das erfindungsgemäße Schema auch in Software implementiert sein kann. Die Implementierung kann auf einem digitalen Speichermedium, insbesondere einer Diskette oder einer CD mit elektronisch auslesbaren Steuersignalen erfolgen, die so mit einem programmierbaren Computersystem zusammenwirken können, dass das entsprechende Verfahren ausgeführt wird. Allgemein besteht die Erfindung somit auch in einem Computerprogrammprodukt mit auf einem maschinenlesbaren Träger gespeicherten Programmcode zur Durchführung des erfindungsgemäßen Verfahrens, wenn das Computerprogrammprodukt auf einem Rechner abläuft. In anderen Worten ausgedrückt kann die Erfindung somit als ein Computerprogramm mit einem Programmcode zur Durchführung des Verfahrens realisiert werden, wenn das Computerprogramm auf einem Computer abläuft.In particular, it should be noted that, depending on the circumstances, the inventive scheme can also be implemented in software. The implementation may be on a digital storage medium, in particular a floppy disk or a CD with electronically readable control signals, which may interact with a programmable computer system such that the corresponding method is executed. In general, the invention thus also consists in a computer program product with program code stored on a machine-readable carrier for carrying out the method according to the invention when the computer program product runs on a computer. In other words, the invention can thus be described as a computer program with a program code for carrying out the method be realized when the computer program runs on a computer.
Claims (14)
- Device (100) for acoustic display of the positions of at least two different objects (200) in a reproduction space (210), at least three loudspeakers (220) being arranged at spatially different positions in the reproduction space (210) so that different spatial positions may be represented acoustically by driving the loudspeakers (220) differently, the device (100) for acoustic display comprising:signal associating means (110) configured to associate an acoustic signal with each of the objects (200) in dependence on the distance of the respective object (200) and on the danger potential associated therewith,wherein the signal associating means (110) additionally comprises an input (105) which may be coupled to a sensor (230) for determining the positions of the objects (200), and the sensor (230) is configured to transmit the position of the objects (200) to the signal associating means (110); andloudspeaker driving means (120) configured to establish one or several loudspeaker signals (LS) for the at least three loudspeakers (220),wherein the one or several loudspeaker signals (LS) by which the positions of the objects (200) are displayed are based on the acoustic signals associated with the objects (200) by the signal associating means (110), and wherein the one or several loudspeaker signals (LS) may be established such that upon reproduction of the one or several loudspeaker signals (LS), the positions of the objects (200) in the reproduction space (210) are displayed acoustically,the device (100) for acoustic display further comprising a signal database (140) connected to the signal associating means (110), the signal database (140) being configured to provide different acoustic signals for the objects (200), the associated acoustic signals depending on whether the respective object (200) is movable or static.
- Device (100) in accordance with claim 1, wherein acoustic signals in the signal database (140) are classified in accordance with a danger potential, and the signal associating means (110) is configured to associate acoustic signals from different classes with different objects (200) in accordance with their potential dangers.
- Device (100) in accordance with claim 2, wherein acoustic signals of a higher danger potential comprise a higher audio frequency or a higher clock frequency.
- Device (100) in accordance with claim 2 or claim 3, wherein an acoustic signal of a high danger potential is associated with an object (200) at a smaller distance, and an acoustic signal of a smaller danger potential is associated with an object (200) at a greater distance.
- Device (100) in accordance with one of the preceding claims, wherein at least one of the objects (200) comprises a relative speed to the reproduction space (200), and wherein the associated acoustic signal is dependent on the relative speed.
- Device (100) in accordance with one of the preceding claims, wherein the loudspeaker driving means (120) is configured to establish several loudspeaker signals (LS) for the multitude of loudspeakers (220), the multitude of loudspeakers (220) at least partly surrounding a position in the reproduction space (210) in one plane.
- Device (100) in accordance with one of the preceding claims, wherein at least one of the objects (200) identifies itself by a textual notification, the sensor (230) being configured to pass on the textual notification to the input (105), and the device (100) additionally comprising a text-to-speech module configured to convert the textual notification to an audio signal and to pass it on to the loudspeaker driving means (120).
- Device (100) in accordance with one of the preceding claims, wherein the signal associating means (110) is configured to associate acoustic signals with several objects (200), and wherein the loudspeaker driving means (120) is configured to generate component signals for each of the several objects (200) and to combine the component signals to form loudspeaker signals (LS) so that the several objects (200) are acoustically perceivable at different positions.
- Device (100) in accordance with one of the preceding claims, wherein the loudspeaker driving means (120) is configured to encode the distance (d) of the object (200) by an audio frequency or clock frequency such that the distance of the object (200) is perceivable at a predetermined scale.
- Device (100) in accordance with one of the preceding claims, wherein the loudspeaker driving means (120) comprises a wave field synthesis system, the wave field synthesis system being configured to reproduce the acoustic signal associated with the object (200) as a virtual source.
- Device for scanning an environment, comprising:a sensor (230) for determining positions of objects (200) in the environment; anda device (100) for acoustic display in accordance with one of claims 1 to 10 that is coupled to the sensor (230) and receives the positions of the objects (200) from the sensor (230).
- Device in accordance with claim 12, wherein the sensor (230) comprises a radar or sonar.
- Method for acoustic display of the positions of at least two different objects (200) in a reproduction space (210), at least three loudspeakers (220) being arranged at spatially different positions in the reproduction space (210) so that different positions may be represented acoustically by driving the loudspeakers (220) differently, comprising:associating different acoustic signals with the objects (200) in dependence on the distance of the respective object (200) and the danger potential associated therewith, wherein said associating of the different acoustic signals with the objects using a signal database (140) comprises associating the acoustic signals in dependence on whether the respective object (200) is movable or static; and establishing one or several loudspeaker signals (LS) for the at least three loudspeakers (220), whereinthe one or several loudspeaker signals (LS) are established such that upon reproduction of the one or several loudspeaker signals (LS), the positions of the objects (200) in the reproduction space (210) are displayed acoustically.
- Computer program comprising a program code for performing the method in accordance with claim 13 when the computer program runs on a computer.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US3820908P | 2008-03-20 | 2008-03-20 | |
PCT/EP2009/001963 WO2009115299A1 (en) | 2008-03-20 | 2009-03-17 | Device and method for acoustic indication |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2255359A1 EP2255359A1 (en) | 2010-12-01 |
EP2255359B1 true EP2255359B1 (en) | 2015-07-15 |
Family
ID=40673888
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP09721864.8A Active EP2255359B1 (en) | 2008-03-20 | 2009-03-17 | Device and method for acoustic indication |
Country Status (6)
Country | Link |
---|---|
US (1) | US20110188342A1 (en) |
EP (1) | EP2255359B1 (en) |
JP (1) | JP2011516830A (en) |
KR (1) | KR20100116223A (en) |
CN (1) | CN101978424B (en) |
WO (1) | WO2009115299A1 (en) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101285391B1 (en) * | 2010-07-28 | 2013-07-10 | 주식회사 팬택 | Apparatus and method for merging acoustic object informations |
CN105578380B (en) * | 2011-07-01 | 2018-10-26 | 杜比实验室特许公司 | It is generated for adaptive audio signal, the system and method for coding and presentation |
EP2727383B1 (en) | 2011-07-01 | 2021-04-28 | Dolby Laboratories Licensing Corporation | System and method for adaptive audio signal generation, coding and rendering |
DE102011082310A1 (en) * | 2011-09-07 | 2013-03-07 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus, method and electroacoustic system for reverberation time extension |
KR101308588B1 (en) * | 2012-02-28 | 2013-09-23 | 주식회사 부국하이텍 | Radar system and method for displaying sound wave of target in the same |
WO2013142657A1 (en) * | 2012-03-23 | 2013-09-26 | Dolby Laboratories Licensing Corporation | System and method of speaker cluster design and rendering |
CN108028048B (en) | 2015-06-30 | 2022-06-21 | 弗劳恩霍夫应用研究促进协会 | Method and apparatus for correlating noise and for analysis |
GB2542846A (en) * | 2015-10-02 | 2017-04-05 | Ford Global Tech Llc | Hazard indicating system and method |
PT109485A (en) * | 2016-06-23 | 2017-12-26 | Inst Politécnico De Leiria | METHOD AND APPARATUS TO CREATE A THREE-DIMENSIONAL SCENARIO |
US20210182487A1 (en) * | 2017-11-01 | 2021-06-17 | Sony Corporation | Information processing apparatus, information processing method, and program |
DE102019006679A1 (en) * | 2019-09-23 | 2021-03-25 | Mbda Deutschland Gmbh | System and method for situation recognition with regard to mobile objects located in a monitoring room |
CN112911354B (en) * | 2019-12-03 | 2022-11-15 | 海信视像科技股份有限公司 | Display apparatus and sound control method |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6097285A (en) * | 1999-03-26 | 2000-08-01 | Lucent Technologies Inc. | Automotive auditory feedback of changing conditions outside the vehicle cabin |
DE10155742A1 (en) * | 2001-10-31 | 2003-05-22 | Daimler Chrysler Ag | Virtual reality warning and information system for road vehicle produces visual signals localized in space and individual signal sources may be represented in given positions |
EP1584901A1 (en) * | 2004-04-08 | 2005-10-12 | Wolfgang Dr. Sassin | Apparatus for the dynamic optical, acoustical or tactile representation of the sourrounding of a vehicle |
Family Cites Families (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS6299879U (en) * | 1985-12-13 | 1987-06-25 | ||
JP2580117B2 (en) * | 1985-12-18 | 1997-02-12 | ソニー株式会社 | Sound field expansion device for playback device |
JPS6325666U (en) * | 1986-03-13 | 1988-02-19 | ||
FR2744871B1 (en) * | 1996-02-13 | 1998-03-06 | Sextant Avionique | SOUND SPATIALIZATION SYSTEM, AND PERSONALIZATION METHOD FOR IMPLEMENTING SAME |
US5979586A (en) * | 1997-02-05 | 1999-11-09 | Automotive Systems Laboratory, Inc. | Vehicle collision warning system |
WO2001055833A1 (en) * | 2000-01-28 | 2001-08-02 | Lake Technology Limited | Spatialized audio system for use in a geographical environment |
JP4304845B2 (en) * | 2000-08-03 | 2009-07-29 | ソニー株式会社 | Audio signal processing method and audio signal processing apparatus |
FR2842064B1 (en) * | 2002-07-02 | 2004-12-03 | Thales Sa | SYSTEM FOR SPATIALIZING SOUND SOURCES WITH IMPROVED PERFORMANCE |
US20050222844A1 (en) * | 2004-04-01 | 2005-10-06 | Hideya Kawahara | Method and apparatus for generating spatialized audio from non-three-dimensionally aware applications |
US8494861B2 (en) * | 2004-05-11 | 2013-07-23 | The Chamberlain Group, Inc. | Movable barrier control system component with audible speech output apparatus and method |
JP2006005868A (en) * | 2004-06-21 | 2006-01-05 | Denso Corp | Vehicle notification sound output device and program |
JP2006019908A (en) * | 2004-06-30 | 2006-01-19 | Denso Corp | Notification sound output device for vehicle, and program |
DE102005008333A1 (en) * | 2005-02-23 | 2006-08-31 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Control device for wave field synthesis rendering device, has audio object manipulation device to vary start/end point of audio object within time period, depending on extent of utilization situation of wave field synthesis system |
US20060256976A1 (en) * | 2005-05-11 | 2006-11-16 | House William N | Spatial array monitoring system |
JP4914057B2 (en) * | 2005-11-28 | 2012-04-11 | 日本無線株式会社 | Ship obstacle warning device |
US7898423B2 (en) * | 2007-07-31 | 2011-03-01 | At&T Intellectual Property I, L.P. | Real-time event notification |
-
2009
- 2009-03-17 CN CN2009801100998A patent/CN101978424B/en not_active Expired - Fee Related
- 2009-03-17 WO PCT/EP2009/001963 patent/WO2009115299A1/en active Application Filing
- 2009-03-17 JP JP2011500111A patent/JP2011516830A/en active Pending
- 2009-03-17 EP EP09721864.8A patent/EP2255359B1/en active Active
- 2009-03-17 US US12/922,910 patent/US20110188342A1/en not_active Abandoned
- 2009-03-17 KR KR1020107021102A patent/KR20100116223A/en not_active IP Right Cessation
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6097285A (en) * | 1999-03-26 | 2000-08-01 | Lucent Technologies Inc. | Automotive auditory feedback of changing conditions outside the vehicle cabin |
DE10155742A1 (en) * | 2001-10-31 | 2003-05-22 | Daimler Chrysler Ag | Virtual reality warning and information system for road vehicle produces visual signals localized in space and individual signal sources may be represented in given positions |
EP1584901A1 (en) * | 2004-04-08 | 2005-10-12 | Wolfgang Dr. Sassin | Apparatus for the dynamic optical, acoustical or tactile representation of the sourrounding of a vehicle |
Also Published As
Publication number | Publication date |
---|---|
CN101978424A (en) | 2011-02-16 |
JP2011516830A (en) | 2011-05-26 |
WO2009115299A1 (en) | 2009-09-24 |
CN101978424B (en) | 2012-09-05 |
KR20100116223A (en) | 2010-10-29 |
US20110188342A1 (en) | 2011-08-04 |
EP2255359A1 (en) | 2010-12-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP2255359B1 (en) | Device and method for acoustic indication | |
DE60217809T2 (en) | Safety device for vehicles with a multi-channel audio system | |
EP3005732B1 (en) | Device and method for spatially selective audio playback | |
DE102012208825A1 (en) | 3D AUDIO DEVICE | |
EP1878308B1 (en) | Device and method for generation and processing of sound effects in spatial audio reproduction systems using a graphical user interface | |
WO2013045374A1 (en) | Method for the computer-assisted processing of the vicinity of a vehicle | |
DE102011082886A1 (en) | Stereo sound reproduction system | |
AT517657B1 (en) | Method and device for warning road users for a rail vehicle by means of sound or light signals | |
DE102015221361A1 (en) | Method and device for driver assistance | |
EP2171700A1 (en) | Method to provide information to an observer about an emergency vehicle in operation, and corresponding equipment | |
DE102009057981B4 (en) | Method for controlling the acoustic perceptibility of a vehicle | |
DE19801205A1 (en) | Acoustic obstacle warning arrangement for motor vehicle | |
DE102014217732B4 (en) | Method for assisting a driver of a motor vehicle, device and system | |
EP3652964A1 (en) | Localized outputting of information | |
EP2226217B1 (en) | Information system for passenger of a vehicle using acoustic and tactile means | |
EP2182744B1 (en) | Replaying a sound field in a target sound area | |
EP1552720B1 (en) | Method for simulating a movement by means of an acoustic reproduction device, and sound reproduction arrangement therefor | |
DE102016114413A1 (en) | Device for generating object-dependent audio data and method for generating object-dependent audio data in a vehicle interior | |
WO2023016924A1 (en) | Method and system for generating noises in an interior on the basis of extracted and classified real noise sources, and vehicle which is acoustically transparent to specific target noises and which comprises a system of this type | |
DE102014210932A1 (en) | System and method for a vehicle for the acoustic detection of a traffic situation | |
DE102015226045A1 (en) | Method and control unit for reproducing an audio signal in a vehicle | |
DE102018203661A1 (en) | Method and apparatus for testing directional hearing in a vehicle | |
DE102014222822A1 (en) | Means of transport with a location device and a signaling device | |
EP3742281A1 (en) | Assistance system and method for supporting a surgeon | |
DE102019004587A1 (en) | Device and method for acoustic environmental presentation for a vehicle |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20101005 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA RS |
|
DAX | Request for extension of the european patent (deleted) | ||
17Q | First examination report despatched |
Effective date: 20140130 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20150202 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO SE SI SK TR |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 737158 Country of ref document: AT Kind code of ref document: T Effective date: 20150815 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502009011260 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20150715 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151015 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151016 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20151116 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 8 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502009011260 Country of ref document: DE |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
26N | No opposition filed |
Effective date: 20160418 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: LU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20160317 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160331 Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160317 Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160331 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 9 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 737158 Country of ref document: AT Kind code of ref document: T Effective date: 20160317 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20160317 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 10 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20090317 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20150715 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230524 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20240321 Year of fee payment: 16 Ref country code: GB Payment date: 20240322 Year of fee payment: 16 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20240319 Year of fee payment: 16 |