DE102009050667A1 - System for the notification of localized information - Google Patents

Abstract

The invention essentially consists in adding bone conduction speakers and directional sensors to a protective helmet, so that the wearer can be informed of localized information in his environment via auditory signals in a spatially directed manner - relative to the orientation of the skull. The notification of the user takes place via various context-dependent auditory signals calculated in real time from the position of the user, the orientation of his head and the position of the information, which are output via these bone conduction speakers, the perceived sound source corresponding to the position of the information. Depending on whether the user has a mobile phone, PDA, tablet PC or other portable computer with him, he is notified in different ways about a changing state of the software or the system. The bone sound transmission ensures that the notification signal can be perceived independently of ambient noise. The invention is likely to find its main application in the domains of location-based services, augmented reality, human-machine interfaces (HMI) and industry.

Description

The invention relates to a system for notification or publication of localized information, that is, information that is assigned to a specific location or can be addressed by it.

The invention described here relates primarily to location-based services as mobile, location-based information services that provide the end user with selective information on mobile devices, such as. PDA, Tablet PC or mobile phone, with the help of position, time and personal data. Such information can be, for example, user manuals for complex installations, maintenance orders for machines, plans or also warning notices which only become visible on the mobile terminal when one is actually in the vicinity of them. Conceivable applications of such technology are in addition to large-scale industrial plants, construction sites, mines, etc.

Due to the changing position or the temporal and spatial context of the end user, it is necessary to notify the user about the new and updated information in his environment. Currently applied notification mechanisms are the requirements of a mobile, and constantly changing environmental influences, such as. Noise, light, danger zones, protective clothing, etc., exposed users do not do justice. In addition, previous notification technologies disrupt or hinder the user and his environment in his workflow.

An auditive notification is done so far by means of warning or advisory tone through the loudspeaker of a terminal. The disadvantages here are that adequate volume and intelligibility is dependent on ambient noise, the distance of the loudspeaker to the ear, as well as the attenuation through clothing, etc. Too soft notification can either be overheard or not understood. Too loud a notification is perceived by the user or his environment as disturbing and obtrusive. Although the wearing of headphones or headsets allows a flawless and directional audio playback, but represents a security risk because ambient noise can no longer be sufficiently perceived. Information about spatial proximity / distance or direction to the information can in turn not be adequately communicated via speakers on a mobile device.

A visual notification has so far taken place by means of a message on a display. The disadvantage here is that the terminal is not always in the field of view of the user or that reading the display in many situations endangers the safety of the user, such as a forklift driver in a warehouse, on a scaffolding, etc.

A notification in the form of a vibration alarm, that is, by means of vibration of the terminal, has the disadvantage that a vibration is only noticeable when the terminal is worn by the user directly on the body or under protective or work clothing. Information about spatial proximity / distance or direction for information can not be conveyed herewith.

When visualizing within Augmented Reality (AR) displays, ie when additional information on visible objects is either displayed in the image of a video camera or the view of reality is augmented directly by means of special see-through displays, a physical and visual disadvantage occurs disadvantageously Restriction by wearing / using an AR display.

If airborne sound transmission is not possible or appropriate, bone sound is often used to transmit sound. This is the case, for example, when hearing protection is worn, but also when using it under water, and when it is necessary to perceive ambient noise as well. In the hearing impaired with damage to the middle ear, appropriate hearing aids are also used. Bone sound refers to the transmission of sound vibrations or vibrations through the skull surrounding the auditory organ, bypassing the middle ear. By attaching two speakers directly to the skull bone tones can be stereoscopically reproduced. This achieves a spatial perception effect. For the use of stereoscopic bone noise, in particular Walker et. al .: Evaluation of bone conduction headsets for use in multitalk communication environments.
¹ http://sonify.psych.gatech.edu/publications/pdfs/2005HFES-WalkerStanleyIyer.pdf

In addition, the SWAN (System for Wearable Audio Navigation) system was developed at the Georgia Institute of Technology. It is a prototypical, auditory guidance system for blind or visually impaired people, such as. B. Firefighters in the smoke. The system is designed to guide people through auditory information generated in real time based on sensor data. It uses visual sensor information from four head mounted cameras to generate auditory signals. In essence, an attempt is made to replace a non-existent or impaired body sense, that is, seeing here, by another, ie hearing here.
http://sonify.psych.gatech.edu/publications/pdfs/2005ICAD-WalkerLindsay.pdf

The object underlying the invention is now to provide a system for the notification of localized information, in which the above-mentioned disadvantages are avoided as possible.

This object is achieved by the features of claim 1 according to the invention. The further claims relate to preferred embodiments of the invention.

The invention consists essentially in a protective helmet to expand bone sound speakers and directional sensors, so that the carrier on auditory signals spatially directed - relative to the orientation of the skull - can be pointed to localized information in his environment. The user's notification is made via various context-dependent real-time from the user's position, the orientation of his head and the position of the information calculated auditory signals that are output through these bone-noise speakers, the sound source thereby corresponds to the position of the information. Depending on whether the user is carrying a mobile phone, PDA, tablet PC or other portable computer, it is notified in different ways of a changing state of the software or system. Bone sound transmission ensures that the signal for notification can be perceived independently of ambient noise. The invention is expected to find its main application in the domains of Location Based Services, Augmented Reality, Human Machine Interfaces (HMI) and Industry.

The invention will be explained in more detail with reference to embodiments shown in the drawing.

It shows

1 a representation for explaining the invention with respect to the position of the speakers and the direction sensor in a protective helmet,

2 a schematic representation of the inventive one embodiment of the overall system and

3 a schematic diagram for explaining the notification using the example of a control system.

1 shows in two views a conventional protective helmet with three cross straps and an adjustable peripheral belt. According to the invention, a bone sound loudspeaker unit L is located in the area above the ears of protective helmet wearer on the left and right in the adjustable circumferential belt, and a directional sensor RS is installed in the area between the helmet roof and the transverse belts.

The direction sensor RS contains a digital compass and an inclination sensor and detects the alignment of the helmet with the aid of the earth's magnetic field. The inclination sensor is necessary, since otherwise the orientation of the head can not be calculated unambiguously from the measured geomagnetic field.

By means of the bone sound speakers, which are worn directly on the skull, it is possible to spatially inform the user about local information in his environment.

2 shows a schematic representation of an embodiment of the overall system according to the invention with a helmet-typical arrangement of the bone sound speaker units L and the direction sensor RS which are connected to a wearable computing unit D either via a cable or via radio, where bone sound speaker units L in the case of a radio link includes a radio receiver and an amplifier or these components are also housed between the helmet roof and the cross straps. From the direction sensor RS, sensor data thus arrive at the arithmetic unit D and 2-channel audio data 2 to the bone sound speaker units L.

The arithmetic unit D may be a mobile telephone, a PDA (personal digital assistant) or also a wrist-worn device (Wrist Computing Device) or a clothing integrated device (wearable computing device).

• – Ermitteln der absoluten Position des Benutzers, z. B. im Freien mittels GPS-Daten 4 oder in Gebäuden mit WLAN Triangulation
• – Auslesen des Richtungssensors, also Neigung und Kompass, im Schutzhelm
• – Errechnen der tatsächlichen absoluten Ausrichtung des Kopfes
• – Beziehen von Daten 3 für verortete Informationen über eine Datenfunkverbindung zu einem Geoinformationssystem, etwa über UMTS oder WLAN
• – Anzeigen von verorteten Informationen wie bspw. Dokumenten und Karten am Display
• – Interaktion des Benutzers etwa über Touchdisplay, z. B. eine Bestätigung von Meldungen, ein Abrufen von Information
• – Ausgeben eines Stereo-Audiosignals an die Knochenschall-Lautsprechereinheiten L im Schutzhelm. Ggf. ist noch eine Audioverstärker-Einheit vorzuschalten, da Knochenschallautsprecher einen 40 bis 50 dB höheren Audiopegel als Luftschallautsprecher benötigen.
• – Erzeugung des Audiosignals. Dieses wird abhängig von der Position des Benutzers und der Ausrichtung seines Kopfes in Echtzeit relativ zu der virtuellen Soundquelle am Ort der zu notifizierenden verorteten Information räumlich richtig berechnet. Mit Hilfe dieses generierten 3D Audio Sounds wird die Orientierung des Trägers gewährleistet.

The arithmetic unit D has to fulfill the following tasks:

• Determine the absolute position of the user, e.g. B. outdoors using GPS data 4 or in buildings with wireless triangulation
• - Reading the direction sensor, so tilt and compass, in the helmet
• - Calculate the actual absolute orientation of the head
• - Obtaining data 3 for localized information about a data radio connection to a geographic information system, such as UMTS or WLAN
• - Display of localized information such as documents and maps on the display
• - user interaction via touch display, eg. As confirmation of messages, retrieval of information
• Outputting a stereo audio signal to the bone sound speaker units L in the protective helmet. Possibly. is still an audio amplifier unit vorzuschalten because bone sound speakers need a 40 to 50 dB higher audio level than airborne loudspeakers.
• - Generation of the audio signal. This is spatially correctly calculated depending on the position of the user and the orientation of his head in real time relative to the virtual sound source at the location of the located information to be notified. This generated 3D audio sound ensures the orientation of the wearer.

These signals are calculated in real time to produce a spatially correctly perceived audio impression.

Optionally, further information about the audio signal is transmitted in addition to the direction to a located information. For example, the distance to an information is not only about the volume of the audio signal, what v. a. In the industrial environment with corresponding environmental noise is not effective, but also on the pitch or the frequency of a repeating signal, that is analogous to a Geiger counter, shown.

In addition, various information categories, such as warnings, virtual barriers or general information, are optionally assigned to different sound patterns.

By way of example, some advantageous examples of application of the invention are shown below:

- As an extension of existing interfaces:

Since the position of a user is known to the system and this is clearly identified, existing control systems can be expanded by virtual interfaces. The user also operates a controller via his portable computer. Through the inventive system for notification, he can be made aware that a change of state on his mobile computer, such as triggered by a scheme, requires his input that further virtual controls are present, and information, manuals or maintenance books are located, etc. , please refer 3 ,

- As an auditory supplement to warning devices:

Existing warning and information lamps can be extended by a directional, audible signal. Since such a notification is reproduced spatially correct, the attention of the user can be controlled purposefully.

- In auditive navigation and guidance systems:

By means of auditory spatially directed signals, the user can be directed to a certain point in the room or even be kept away with warnings of restricted areas / danger areas.

- For the notification of location based services:

The user is notified of the presence of located information when he is in their sphere of influence, that is, for example, within a defined radius to an information. This allows messages and information to be left in a specific location and retrieved.

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• http://sonify.psych.gatech.edu/publications/pdfs/2005HFES-WalkerStanleyIyer.pdf [0008]
• http://sonify.psych.gatech.edu/publications/pdfs/2005ICAD-WalkerLindsay.pdf [0009]

Claims (5)

System for notification of localized information, comprising a protective helmet of the two bone sound speakers (L) and a direction sensor (RS) and with a computing unit (D) through which data ( 3 ) for localized information, position data ( 4 ) of a user and sensor data ( 1 ) and stereo audio signals ( 2 ) to the bone sound speakers (L) and in which a notification of different context-dependent from the user's position, the orientation of his head and the position of the information calculated stereo audio signals ( 2 ) are output via the bone sound speakers so that the perceived sound source corresponds to the position of the information. System according to Claim 1, in which the arithmetic unit (D) is present in such a way that - from received GPS data ( 4 ) and / or by Wi-Fi triangulation the absolute position of a user is determined, - that from sensor data ( 1 ) of the directional sensor (RS) an actual absolute alignment of the head of the user is calculated, - that data ( 3 ) for localized information can be received via a data radio connection to a geographic information system and can be displayed on a display, that at least one interaction of the user with respect to the located information is feasible, and that the stereo audio signals 2 ) and output to the bone sound speaker units (L), which is calculated in real time depending on the position of the user and the orientation of his head relative to the virtual sound source at the location of the located information to be notified. System according to Claim 1 or 2, in which the arithmetic unit (D) is present in such a way that the stereo audio signals ( 2 ) information about the distance between the user and the information to be notified in the form of the pitch of an acoustic signal or the frequency of a repeating acoustic signal can be generated. System according to claim 1 or 2, in which the arithmetic unit (D) is present in such a way that different information categories can be assigned different sound patterns. System according to one of the preceding claims, in which the arithmetic unit (D) is present in such a way that the user automatically notifies of the presence of located information when he is in the sphere of influence of information.

Images