WO2003016110A2 - Method for assisting the user of a piece of equipment in locating a control element - Google Patents

Method for assisting the user of a piece of equipment in locating a control element Download PDF

Info

Publication number
WO2003016110A2
WO2003016110A2 PCT/EP2002/005650 EP0205650W WO03016110A2 WO 2003016110 A2 WO2003016110 A2 WO 2003016110A2 EP 0205650 W EP0205650 W EP 0205650W WO 03016110 A2 WO03016110 A2 WO 03016110A2
Authority
WO
WIPO (PCT)
Prior art keywords
control element
operator
occupant
recognition system
vehicle
Prior art date
Application number
PCT/EP2002/005650
Other languages
German (de)
French (fr)
Other versions
WO2003016110A3 (en
WO2003016110A8 (en
Inventor
Josef Bärenweiler
Günter Fendt
Helmut Riedel
Original Assignee
Conti Temic Microelectronic Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Conti Temic Microelectronic Gmbh filed Critical Conti Temic Microelectronic Gmbh
Publication of WO2003016110A2 publication Critical patent/WO2003016110A2/en
Publication of WO2003016110A3 publication Critical patent/WO2003016110A3/en
Publication of WO2003016110A8 publication Critical patent/WO2003016110A8/en

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Q—ARRANGEMENT OF SIGNALLING OR LIGHTING DEVICES, THE MOUNTING OR SUPPORTING THEREOF OR CIRCUITS THEREFOR, FOR VEHICLES IN GENERAL
    • B60Q3/00—Arrangement of lighting devices for vehicle interiors; Lighting devices specially adapted for vehicle interiors
    • B60Q3/10—Arrangement of lighting devices for vehicle interiors; Lighting devices specially adapted for vehicle interiors for dashboards
    • B60Q3/16—Circuits; Control arrangements
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B21/00—Teaching, or communicating with, the blind, deaf or mute
    • G09B21/001—Teaching or communicating with blind persons
    • G09B21/006—Teaching or communicating with blind persons using audible presentation of the information
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Definitions

  • the invention relates to a method for supporting an operator of a device in finding an operating element.
  • control elements are increasingly designed to be multifunctional and arranged in a wide variety of positions on the vehicle, so that an occupant may even be distracted by searching for an control element in safety-critical traffic situations.
  • a speech recognition system By means of a speech recognition system, the acoustic signals in the vicinity of the device are monitored for a search query by the operator regarding a control element to be found.
  • a conventional speech recognition system can be used for this.
  • a signaling control unit activates a signaling which is perceptible to the operator and which tells the operator the position of the operating element displays.
  • the signaling control unit can thus activate a lighting device in the vicinity of the control element, in particular switch the background lighting of the control element in question, for example into a flashing operating state for a predetermined period of time.
  • a prepared voice signal can be output, which describes the position of the corresponding control element.
  • the position of the hands of the operator is additionally detected by means of a sensor, and voice signals are generated in such a way that they indicate how the operator has to move his hands or one of the hands in order to arrive at the operating element sought .
  • a sensor for example, for detecting seat occupancy or occupant position in vehicles, in order to accordingly release occupant protection devices or to suppress their triggering.
  • a number of such interior monitoring sensors offer sufficient image resolution for detecting the position of the hand or hands of the operator or can be further developed in a corresponding manner without great effort.
  • the signaling control unit which here receives a search request from the operator regarding the control element to be found by the speech recognition system, has a memory in which the position of the control elements of the vehicle is stored and which contains the signal from a sensor for describing the actual position of the hand or hands of the operator can be fed, which accordingly generates a voice signal to the occupant as a function of the difference between the actual position of the hand and the hands of the occupant and the positions of the control element to be found in the memory.
  • Figure 1 vehicle with occupants and optical signaling of the control element sought.
  • FIG. 1 shows an occupant 1 in a vehicle 2, here the driver, for example, the method also being applicable to other occupants in vehicles, be they passengers, passengers in the rear or passengers in a bus, plane or train.
  • the vehicle 2 has a speech recognition system 4 with a microphone 3 for detecting the acoustic signals in the vehicle 2, wherein speech recognition systems already present in the vehicle 2 can be used for this, provided that these offer the possibility of corresponding adaptation to monitoring of a search query by the occupant 1 with respect to a control element to be found.
  • other devices can also be equipped with a corresponding speech recognition system 4 and this method can be used to assist the operator in finding an operating element.
  • a signaling control device here in a functional unit with the speech recognition system - likewise identified as 4 - activates a signaling 7a which is perceptible to the occupant 1 and which the occupant 1 detects the position of the control element 6a displays.
  • the signaling control unit 4 controls an illumination device 7a in the vicinity of the control element 6a, more precisely here even the background illumination of the control element 6a sought, and preferably for a certain period of time by means of a flashing signal, so that it is independent of the connection general vehicle lighting
  • FIG. 2 shows a sketch of the method by means of bidirectional voice control, in which the speech recognition system 4 provides the occupant with a speech signal 7b describing the position of the control element 6b sought after the search query 12.
  • This can be a generally prepared speech signal, which is read out from a memory and played back when such a search query is recognized and which indicates the position of the operating element 6b to the occupant 1 on the basis of instructions, for example the front row on the center console.
  • this descriptive voice signal 7b can also be used in addition to optical signaling, as shown in FIG. 1.
  • This method is particularly preferably supplemented, in which the position of the hand or hands 11 of the occupant is detected by means of a sensor 5 arranged here in the upper corner between the windshield and the roof of the vehicle.
  • a sensor 5 arranged here in the upper corner between the windshield and the roof of the vehicle.
  • these interior sensors 5 can also be used to detect the position of the occupant's hand or hands 11.
  • This is used in DE 197 53 160 C1, for example, to recognize an impending accident situation of the vehicle, to record the speed of movement of the driver's hands and to derive an impending accident situation from this if the driver moves his hands in a panic.
  • the position of the hands 11 of the occupant 1 is to be used to locate an operating element 6b by responding to a query by the occupant 1 regarding an operating element to be found
  • a speech signal 7b is output by the loudspeaker 8, which indicates how the occupant 1 has to move his hands 11 in order to arrive at the control element 6b sought.
  • the hand 11 of the occupant is guided step by step to the operating element 6b on the basis of simple predetermined speech signals 7b, and the actual position of the hand 11 of the occupant 1 detected by the sensor 5 is brought into agreement with the position of the operating element 6b to be found in the memory.
  • a PMD sensor is particularly suitable as a sensor 5 for detecting the position of the hands 11 of the occupant 1, which, in addition to an optical image by means of phase modulation, can also assign a distance signal to the respective image point and thus generate a three-dimensional image of the interior of the vehicle.
  • the voice signal 7b can also explain the mode of operation of the control element 6b in more detail, which appears particularly useful in the case of multifunction control elements.
  • the way in which the functional units are arranged after the control element 6b can also be described, possibly in response to a corresponding supplemented search query 12.
  • a voice recognition system 4 in a vehicle 2 by reading in a program stored on a portable storage medium, allows this preinstalled voice recognition system 4 to be equipped with the additional data about the position of the operating elements 6 of the vehicle 2 and the voice signals agreed for the search query, so that even in different speech recognition systems, an occupant 1, for example from a rental car company, is provided with corresponding requests for the preparation of the speech recognition system 4 for carrying out the method, provided that this is not pre-assembled in the vehicle.
  • Appropriate data supplementation and search queries mean that the occupant 1 can obtain an almost complete presentation of the control elements 6 of the vehicle 2.
  • Storage volume can be implemented in the language processing system. It can also be used for a plurality of devices in each case by means of a speech recognition system.

Abstract

The invention relates to a method for assisting the user of a piece of equipment, particularly a vehicle, in locating a control element, whereby acoustic signals in the vicinity of the equipment, for example in the passenger compartment of the vehicle, are monitored by means of a speech recognition system on a search request from said user, regarding a control element to be located. When such a search request is detected by the speech recognition system of a signalling control unit, a signal, perceptible to the user and indicating to said user the position of the control element, is activated. Said signal can be an optical signal. In this case, a lighting device, preferably a backlighting, in the vicinity of the control element is activated. Alternatively, a speech signal, stored in a memory and prepared for the corresponding control element, is activated. Ideally, a sensor for detecting the position of the user's hand can also be used. In this case, speech signals are produced, in order to indicate to the user how he should move his hand to reach the control element. Vehicles can then be especially equipped with a sensor for monitoring the passenger compartment.

Description

Verfahren zur Unterstützung eines Bedieners eines Gerätes beim Auffinden eines Bedienelements Method for assisting an operator of a device to find an operating element
Die Erfindung betrifft ein Verfahren zur Unterstützung eines Bedieners eines Gerätes beim Auffinden eines Bedienelements.The invention relates to a method for supporting an operator of a device in finding an operating element.
Heutige und insbesondere auch zukünftige Fahrzeuge sowie andere zu bedienende Ge- rate, wie Fahrkartenautomaten oder vergleichbare Selbstbedienungsautomaten weisen eine Vielzahl von Bedienelemente auf, die insbesondere für ein mit diesem Gerät, bspw. Fahrzeugtyp nicht näher bekannten Bediener nicht immer selbsterklärend und aufgrund ihrer Vielzahl oft verwirrend sind. So werden in Fahrzeugen bspw. Bedienelemente zunehmend multifunktional ausgelegt und an den verschiedensten Positionen am Fahrzeug angeordnet, so dass ein Insasse eventuell sogar in sicherheitskritischen Verkehrssituationen mit der Suche eines Bedienelements abgelenkt wird.Today's and in particular also future vehicles as well as other devices to be operated, such as ticket machines or comparable self-service machines, have a large number of operating elements which are not always self-explanatory and often confusing due to their large number, in particular for an operator who is not known with this device, for example, vehicle type are. In vehicles, for example, control elements are increasingly designed to be multifunctional and arranged in a wide variety of positions on the vehicle, so that an occupant may even be distracted by searching for an control element in safety-critical traffic situations.
Aus diesem Grund werden bereits Spracherkennungssysteme angeboten, mit denen automatisch bestimmte Bedienelemente des Fahrzeuges, insbesondere Wahlfunktionen des Telefons oder des Radios durch Sprachbefehle des Bedieners an das Spracherken- nungssystem ersetzt werden, so dass der Insasse die entsprechenden Bedienelemente nicht mehr manuelle bedienen muss. Nachteil dieser sprachgesteuerten Systeme ist jedoch, dass diese aufgrund der hohen Komplexität nur dafür geeignete und vorbereitete Anwendungsgruppen geeignet sind. Um jedoch den Insassen, insbesondere in der Kennenlernphase des Fahrzeuges, beim Auffinden der Bedienelemente zu unterstützen, wird nachfolgendes Verfahren vorgeschlagen. Dieses Verfahren eignet sich auch für die Unterstützung von Bedienern anderer Geräte beim Auffinden eines Bedienelements.For this reason, speech recognition systems are already offered, with which certain control elements of the vehicle, in particular dialing functions of the telephone or the radio, are automatically replaced by voice commands from the operator to the speech recognition system, so that the occupant no longer has to operate the corresponding control elements manually. The disadvantage of these voice-controlled systems, however, is that, due to their high level of complexity, they are only suitable and prepared application groups. However, the following method is proposed in order to assist the occupant in locating the control elements, in particular in the familiarization phase of the vehicle. This procedure is also suitable for assisting operators of other devices to find a control element.
Mittels eines Spracherkennungssystems werden die akustischen Signale im Nahbereich des Gerätes auf eine Suchanfrage des Bedieners hinsichtlich eines aufzufindenden Bedienelements überwacht. Dazu kann ein herkömmliches Spracherkenπungssystem die- nen. Beim Erkennen einer solchen Suchanfrage des Bedieners durch das Spracherken- nungssystem wird von einer Signalisierungssteuereinheit eine für den Bediener wahrnehmbare Signalisierung aktiviert, die dem Bediener die Position des Bedienelements anzeigt. Hinsichtlich der für den Bediener wahrnehmbaren Signalisierung ergeben sich nun mehrere Möglichkeiten.By means of a speech recognition system, the acoustic signals in the vicinity of the device are monitored for a search query by the operator regarding a control element to be found. A conventional speech recognition system can be used for this. When such a search request by the operator is recognized by the speech recognition system, a signaling control unit activates a signaling which is perceptible to the operator and which tells the operator the position of the operating element displays. With regard to the signaling perceptible to the operator, there are now several possibilities.
So kann die Signalisierungssteuereinheit eine Beleuchtungseinrichtung im Nahbereich des Bedienelements aktivieren, insbesondere die Hintergrundbeleuchtung des jeweils gesuchten Bedienelements, beispielsweise in einen blinkenden Betriebszustand für eine vorgegebene Zeitspanne schalten.The signaling control unit can thus activate a lighting device in the vicinity of the control element, in particular switch the background lighting of the control element in question, for example into a flashing operating state for a predetermined period of time.
Alternativ dazu bietet sich die Ausgabe eines vorbereiteten Sprachsignals an, welches die Position des entsprechenden Bedienelements beschreibt.Alternatively, a prepared voice signal can be output, which describes the position of the corresponding control element.
In einer besonders bevorzugten Weiterbildung wird jedoch zusätzlich mittels eines Sen- sor,s die Position der Hände des Bedieners erfasst und Sprachsignale derart erzeugt, dass diese angeben, wie der Bediener seine Hände bzw. eine der Hände bewegen muss, um zum gesuchten Bedienelement zu gelangen. Dafür bietet sich die Verwendung eines Innenraumüberwachungssensors an, wie er beispielsweise zur Erkennung der Sitzbelegung oder Insassenposition in Fahrzeugen durchaus bereits üblich ist, um dementspre- chend Insassenschutzeinrichtungen frei zu geben bzw. deren Auslösung zu unterdrücken. Eine Reihe solcher Innenraumüberwachungssensoren bietet eine ausreichende Bildauflösung zur Erkennung der Position der Hand bzw. der Hände des Bedieners oder kann ohne größeren Aufwand in entsprechender Art und Weise weitergebildet werden.In a particularly preferred development, however, the position of the hands of the operator is additionally detected by means of a sensor, and voice signals are generated in such a way that they indicate how the operator has to move his hands or one of the hands in order to arrive at the operating element sought , For this, the use of an interior monitoring sensor lends itself, as is already customary, for example, for detecting seat occupancy or occupant position in vehicles, in order to accordingly release occupant protection devices or to suppress their triggering. A number of such interior monitoring sensors offer sufficient image resolution for detecting the position of the hand or hands of the operator or can be further developed in a corresponding manner without great effort.
Die Signalisierungssteuereinheit, die hier vom Spracherkennungssystem eine Suchanfra- ge des Bedieners hinsichtlich des aufzufindenden Bedienelements zugeführt wird, weist einen Speicher auf, in dem die Position der Bedienelemente des Fahrzeugs abgespeichert ist und welcher von einem Sensor das Signal zur Beschreibung der ist-Position der Hand bzw. Hände des Bedieners zuführbar ist, die dementsprechend ein Sprachsignal an den Insassen in Abhängigkeit von der Differenz zwischen Ist-Position der Hand und Hän- de des Insassen und den im Speicher entnehmbaren Positionen des aufzufindenden Bedienelements erzeugt.The signaling control unit, which here receives a search request from the operator regarding the control element to be found by the speech recognition system, has a memory in which the position of the control elements of the vehicle is stored and which contains the signal from a sensor for describing the actual position of the hand or hands of the operator can be fed, which accordingly generates a voice signal to the occupant as a function of the difference between the actual position of the hand and the hands of the occupant and the positions of the control element to be found in the memory.
Die Erfindung soll nachfolgend anhand von Ausführungsbeispielen und Figuren näher erläutert werden. Kurze Beschreibung der Figuren:The invention will be explained in more detail below on the basis of exemplary embodiments and figures. Brief description of the figures:
Figur 1 Fahrzeug mit Insassen und optischer Signalisierung des gesuchten Be- dienelements.Figure 1 vehicle with occupants and optical signaling of the control element sought.
Figur 2 Fahrzeug mit Insassen und in Abhängigkeit der Position der Hände gesteuerter sprachgeführter Unterstützung des Insassen beim Auffinden des Bedienelements. Die Figur 1 zeigt einen Insassen 1 in einem Fahrzeug 2, hier beispielsweise der Fahrer, wobei das Verfahren auch für andere Insassen in Fahrzeugen, seien es Beifahrer, Mitreisende im Fond oder Passagiere in einem Bus, Flugzeug oder in einer Bahn, anwendbar ist. Das Fahrzeug 2 weist ein Spracherkennungssystem 4 mit einem Mikrofon 3 zur Er- fassung der akustischen Signale im Fahrzeug 2 auf, wobei hierfür bereits im Fahrzeug 2 vorhandene Spracherkennungssysteme genutzt werden können, sofern diese die Möglichkeit einer entsprechenden Anpassung an eine Überwachung auf eine Suchanfrage des Insassen 1 hinsichtlich eines aufzufindenden Bedienelements aufweisen. Grundsätzlich können aber auch andere Geräte mit einem entsprechenden Spracherkennungssys- tem 4 ausgestattet und dieses Verfahren zur Unterstützung des Bedieners beim Auffinden eines Bedienelements genutzt werden.Figure 2 vehicle with occupants and depending on the position of the hands controlled voice-guided support of the occupant in locating the control element. FIG. 1 shows an occupant 1 in a vehicle 2, here the driver, for example, the method also being applicable to other occupants in vehicles, be they passengers, passengers in the rear or passengers in a bus, plane or train. The vehicle 2 has a speech recognition system 4 with a microphone 3 for detecting the acoustic signals in the vehicle 2, wherein speech recognition systems already present in the vehicle 2 can be used for this, provided that these offer the possibility of corresponding adaptation to monitoring of a search query by the occupant 1 with respect to a control element to be found. In principle, however, other devices can also be equipped with a corresponding speech recognition system 4 and this method can be used to assist the operator in finding an operating element.
Beim Erkennen einer solchen Suchanfrage 12 durch das Spracherkennungssystem 4, wird von einer Signalisierungssteuereinrichtung, hier in Funktionseinheit mit dem Spracherkennungssystem - ebenfalls als 4 gekennzeichnet -, eine für den Insassen 1 wahr- nehmbare Signalisierung 7a aktiviert, die dem Insassen 1 die Position des Bedienelements 6a anzeigt. Die Signalisierungssteuereinheit 4 steuert dabei in diesem Ausführungsbeispiel in Figur 1 eine Beleuchtungseinrichtung 7a im Nahbereich des Bedienelements 6a, genauer hier sogar die Hintergrundbeleuchtung des gesuchten Bedienelements 6a an, und zwar vorzugsweise für eine bestimmte Zeitdauer mittels eines Blinksig- nals, so dass unabhängig von der Zuschaltung der allgemeinen Fahrzeugbeleuchtung dieWhen such a search request 12 is recognized by the speech recognition system 4, a signaling control device, here in a functional unit with the speech recognition system - likewise identified as 4 - activates a signaling 7a which is perceptible to the occupant 1 and which the occupant 1 detects the position of the control element 6a displays. In this exemplary embodiment in FIG. 1, the signaling control unit 4 controls an illumination device 7a in the vicinity of the control element 6a, more precisely here even the background illumination of the control element 6a sought, and preferably for a certain period of time by means of a flashing signal, so that it is independent of the connection general vehicle lighting
Signalisierung immer erkennbar bleibt. Dank der blinkenden Hintergrundbeleuchtung 7a des Bedienelements 6a ist dieses vom Insassen schnell wahrnehmbar und kann ohne längere Verzögerung im Fahrzeug 2 aufgefunden werden.Signaling always remains recognizable. Thanks to the flashing background lighting 7a of the control element 6a, this can be quickly perceived by the occupant and can be found in the vehicle 2 without a longer delay.
Figur 2 zeigt nun noch skizzenhaft das Verfahren mittels bidirektionaler Sprachsteue- rung, bei dem dem Insassen auf seine Suchanfrage 12 vom Spracherkennungssystem 4 ein die Position des gesuchten Bedienelements 6b beschreibendes Sprachsignal 7b bereitgestellt wird. Dies kann ein allgemeines vorbereitetes Sprachsignal sein, welches beim Erkennen einer solchen Suchanfrage aus einem Speicher ausgelesen und abgespielt wird und dem Insassen 1 anhand von Hinweisen, beispielsweise vordere Reihe auf der Mittelkonsole die Position des Bedienelements 6b angibt. Selbstverständlich kann dieses beschreibende Sprachsignal 7b auch ergänzend zu einer optischen Signalisierung, wie in Figur 1 gezeigt, verwendet werden.FIG. 2 shows a sketch of the method by means of bidirectional voice control, in which the speech recognition system 4 provides the occupant with a speech signal 7b describing the position of the control element 6b sought after the search query 12. This can be a generally prepared speech signal, which is read out from a memory and played back when such a search query is recognized and which indicates the position of the operating element 6b to the occupant 1 on the basis of instructions, for example the front row on the center console. Of course, this descriptive voice signal 7b can also be used in addition to optical signaling, as shown in FIG. 1.
Besonders bevorzugt wird dieses Verfahren zudem ergänzt, in dem mittels eines Sensors 5 hier angeordnet in der oberen Ecke zwischen Windschutzscheibe und Dach des Fahr- zeugs die Position der Hand bzw. Hände 11 des Insassen erfasst wird. Heute bzw. bereits in 'naher Zukunft werden Innenraumüberwachungssensoren zur Verfügung stehen-, die zur Erkennung der Sitzbelegung oder Sitzposition des Insassen im Fahrzeug vorgesehen sind. Diese Innenraumsensoren 5 können in vielen Fällen auch zur Erkennung der Position der Hand bzw. Hände 11 des Insassen genutzt werden. Dies wird in der DE 197 53 160 C1 beispielsweise zum Erkennen einer bevorstehenden Unfallsituation des Fahr- Zeuges genutzt, um die Bewegungsgeschwindigkeit der Hände des Fahrers zu erfassen und daraus eine bevorstehende Unfallsituation abzuleiten, wenn der Fahrer panikartig seine Hände bewegt. Hinsichtlich messtechnischer Probleme wird daher auf diese Schrift als Beispiel verwiesen. Im vorliegenden Anwendungsfall soll jedoch die Position der Hände 11 des Insassen 1 zum Auffinden eines Bedienelements 6b genutzt werden, in dem auf eine Suchanfrage des Insassen 1 hinsichtlich eines aufzufindenden BedienelementsThis method is particularly preferably supplemented, in which the position of the hand or hands 11 of the occupant is detected by means of a sensor 5 arranged here in the upper corner between the windshield and the roof of the vehicle. Today or already in 'the near future interior monitoring sensors are stand- available which are provided for recognizing the occupancy of the occupant in the vehicle. In many cases, these interior sensors 5 can also be used to detect the position of the occupant's hand or hands 11. This is used in DE 197 53 160 C1, for example, to recognize an impending accident situation of the vehicle, to record the speed of movement of the driver's hands and to derive an impending accident situation from this if the driver moves his hands in a panic. With regard to metrological problems, reference is therefore made to this document as an example. In the present application, however, the position of the hands 11 of the occupant 1 is to be used to locate an operating element 6b by responding to a query by the occupant 1 regarding an operating element to be found
6b, ein Sprachsignal 7b vom Lautsprecher 8 ausgegeben wird, welches angibt, wie der Insasse 1 seine Hände 11 bewegen muss, um zum gesuchten Bedienelement 6b zu gelangen. Dabei wird die Hand 11 des Insassen anhand einfacher vorgegebener Sprachsignale 7b schrittweise zum Bedienelement 6b geführt und so die vom Sensor 5 erfasste Ist-Position der Hand 11 des Insassen 1 in Übereinstimmung mit dem im Speicher entnehmbaren Position des aufzufindenden Bedienelements 6b in Übereinstimmung gebracht. Als Sensor 5 zur Erfassung der Position der Hände 11 des Insassen 1 eignet sich insbesondere ein PMD-Sensor, der neben einem optischen Bild mittels Phasenmodulation zusätzlich auch ein Entfernungssignal dem jeweiligen Bildpunkt zuordnen kann und so ein dreidimensionales Bild vom Innenraum des Fahrzeuges erzeugt. Alternative Sensorkonzepte, notfalls mit einem an der Hand 11 anzubringenden Bezugspunkt sind ebenfalls denkbar. Neben der Position des Bedienelements 6b kann das Sprachsignal 7b auch die Funktionsweise des Bedienelements 6b näher erläutern, was insbesondere bei Multifunk- tionsbedienelementen sinnvoll erscheint. Auch können die dem Bedienelement 6b nach- geordneten Funktionseinheiten in ihrer Funktionsweise beschrieben werden, ggf. auf eine entsprechende ergänzte Suchabfrage 12 hin. In besonders bevorzugter Weise erlaubt ein Spracherkennungssystem 4 in einem Fahrzeug 2 durch Einlesen eines auf einem tragbaren Speichermedium abgelegten Programms, dieses vorinstallierte Spracherkennungssystem 4 mit den zusätzlichen Daten über die Position der Bedienelemente 6 des Fahr- zeugs 2 und die für die Suchabfrage vereinbarten Sprachsignale aufzurüsten, so dass ggf. sogar in unterschiedlichen Spracherkennungssystemen einem Insassen 1 beispielsweise von einer Mietwagenfirma entsprechende Anträge zur Vorbereitung des Spracherkennungssystems 4 zur Durchführung des Verfahrens zur Verfügung gestellt wird, sofern dies im Fahrzeug nicht fest vormontiert ist. Durch entsprechende Datenergänzung und Suchanfragen kann der Insasse 1 quasi eine gesamte Vorstellung der Bedienelemente 6 des Fahrzeuges 2 erhalten. Grundsätzlich können aber auch andere Geräte mit einem entsprechenden Spracherkennungssystem 4 ausgestattet und dieses Verfahren zur Unterstützung des Bedieners beim Auffinden eines Bedienelements genutzt werden. Insbesondere für Blinde kann dieses Verfahren als Zwischenschritt zu einer vollständigen Sprachsteuerung von Geräten äu- ßerst hilfreich sein. So können beispielsweise die Position bestimmter Geräte in einem definierten Gebiet, bspw. einer Wohnung, einem Hotelzimmer oder ähnlichem abgespeichert und vom (blinden oder auch sehenden) Bediener mittels Suchanfrage abgefragt werden. Auch für diese Anwendung scheint eine Ergänzung weiterer Informationen neben der Position, also beispielsweise die Ausgabe bestimmter Bedienungsanweisungen, Sicherheitshinweise oder Nutzungsmöglichkeiten sehr sinnvoll und mit entsprechendem6b, a speech signal 7b is output by the loudspeaker 8, which indicates how the occupant 1 has to move his hands 11 in order to arrive at the control element 6b sought. The hand 11 of the occupant is guided step by step to the operating element 6b on the basis of simple predetermined speech signals 7b, and the actual position of the hand 11 of the occupant 1 detected by the sensor 5 is brought into agreement with the position of the operating element 6b to be found in the memory. A PMD sensor is particularly suitable as a sensor 5 for detecting the position of the hands 11 of the occupant 1, which, in addition to an optical image by means of phase modulation, can also assign a distance signal to the respective image point and thus generate a three-dimensional image of the interior of the vehicle. Alternative sensor concepts, if necessary with a reference point to be attached to the hand 11, are also conceivable. In addition to the position of the control element 6b, the voice signal 7b can also explain the mode of operation of the control element 6b in more detail, which appears particularly useful in the case of multifunction control elements. The way in which the functional units are arranged after the control element 6b can also be described, possibly in response to a corresponding supplemented search query 12. In a particularly preferred manner, a voice recognition system 4 in a vehicle 2, by reading in a program stored on a portable storage medium, allows this preinstalled voice recognition system 4 to be equipped with the additional data about the position of the operating elements 6 of the vehicle 2 and the voice signals agreed for the search query, so that even in different speech recognition systems, an occupant 1, for example from a rental car company, is provided with corresponding requests for the preparation of the speech recognition system 4 for carrying out the method, provided that this is not pre-assembled in the vehicle. Appropriate data supplementation and search queries mean that the occupant 1 can obtain an almost complete presentation of the control elements 6 of the vehicle 2. In principle, however, other devices can also be equipped with a corresponding speech recognition system 4 and this method can be used to assist the operator in finding an operating element. For blind people in particular, this method can be extremely helpful as an intermediate step to complete voice control of devices. For example, the position of certain devices in a defined area, for example an apartment, a hotel room or the like, can be stored and queried by the (blind or also seeing) operator by means of a search query. For this application, too, additional information in addition to the position, for example the output of certain operating instructions, safety instructions or possible uses, seems to be very useful and with the appropriate information
Speichervolumen im Sprachverarbeitungssystem realisierbar. Dabei kann auch mittels eines Spracherkennungssystems jeweils für eine Mehrzahl von Geräten gemeinsam genutzt werden. Storage volume can be implemented in the language processing system. It can also be used for a plurality of devices in each case by means of a speech recognition system.

Claims

Patentansprücheclaims
1 ) Verfahren zur Unterstützung eines Bedieners (1) zumindest eines Gerätes, insbe- sondere eines Fahrzeugs (2), beim Auffinden eines Bedienelements (6a, 6b), indem a) mittels eines Spracherkennungssystems (4) akustische Signale (2) im Nahbereich des Geräts auf eine Suchabfrage (12) des Bedieners (1) hinsichtlich eines aufzufin-1) Method for supporting an operator (1) of at least one device, in particular a vehicle (2), when locating an operating element (6a, 6b) by a) using acoustic signals (2) in the vicinity of the Device on a search query (12) by the operator (1) with regard to a
•• denden Bedienelements (6a, 6b) überwacht werden b) und beim Erkennen einer solchen Suchanfrage (12) durch das Spracherkennungs- System (4) von einer Signalisierungssteuereinheit (4) eine für den Bediener (1) wahrnehmbare Signalisierung (7a, 7b) aktiviert wird, die dem Bediener (1) die Position des gesuchten Bedienelements (6a, 6b) anzeigt. •• the control elements (6a, 6b) are monitored b) and when such a search request (12) is recognized by the speech recognition system (4) from a signaling control unit (4), a signaling (7a, 7b) perceptible to the operator (1) is activated, which indicates to the operator (1) the position of the control element (6a, 6b) sought.
2) Verfahren nach Anspruch 1 , wobei die Signalisierungssteuereinheit (4) eine Beleuchtungseinrichtung (7a) im Nahbereich des Bedienelements (6a) aktiviert.2) Method according to claim 1, wherein the signaling control unit (4) activates a lighting device (7a) in the vicinity of the control element (6a).
3) Verfahren nach Anspruch 2, wobei als Beleuchtungseinrichtung (7a) eine Hintergrundbeleuchtung des jeweils gesuchten Bedienelements (6a) aktiviert wird.3) Method according to claim 2, wherein a backlight of the control element (6a) in question is activated as the lighting device (7a).
4) Verfahren nach einem der Ansprüche 2 oder 3, wobei die Beleuchtungseinrichtung (7a) blinkt.4) Method according to one of claims 2 or 3, wherein the lighting device (7a) flashes.
5) Verfahren nach Anspruch 1 , wobei ein die Position des Bedienelements (6b) be- schreibendes Sprachsignal (7b) aktiviert wird.5) Method according to claim 1, wherein a voice signal (7b) describing the position of the control element (6b) is activated.
6) Verfahren nach Anspruch 1 , wobei mittels eines Sensors (5) die Position der Hände (11 ) des Bedieners (1 ) erfasst und Sprachsignale erzeugt werden, die angeben, wie der Insasse (1 ) eine Hand (11) bewegen muss, um zum Bedienelement (6b) zu gelangen.6) Method according to claim 1, wherein the position of the hands (11) of the operator (1) is detected by means of a sensor (5) and speech signals are generated which indicate how the occupant (1) has to move a hand (11) in order to to get to the control element (6b).
7) Spracherkennungssystem (4) für Fahrzeuge (2), welches akustische Signale im Innenraum des Fahrzeugs (2) auf eine Suchabfrage des Insassen (1) hinsichtlich eines aufzufindenden Bedienelements (6a, 6b) überwacht. 8) Kraftfahrzeug mit einem Spracherkennungssystem (4) mit Mitteln zur für den Insassen (1) wahrnehmbaren Signalisierung zur Durchführung des Verfahrens nach einem der vorangehenden Ansprüche.7) Voice recognition system (4) for vehicles (2), which monitors acoustic signals in the interior of the vehicle (2) for a search query by the occupant (1) with regard to an operating element (6a, 6b) to be found. 8) Motor vehicle with a speech recognition system (4) with means for the occupant (1) perceivable signaling for performing the method according to any one of the preceding claims.
9) Verwendung eines Innenraumüberwachungssensors, insbesondere eines Sensors zur Erkennung der Sitzbelegung und/oder Insassenposition, von Fahrzeugen (2) zur9) Use of an interior monitoring sensor, in particular a sensor for detecting the seat occupancy and / or occupant position, of vehicles (2)
Erkennung der Position der Hand/Hände (11 ) des Insassen (1).Detection of the position of the hand / hands (11) of the occupant (1).
10) Signalisierungssteuereinheit (4), a) welcher von einem Spracherkennungssystem eine Suchanfrage des Bedieners (1 ) "• hinsichtlich eines aufzufindenden Bedienelements (6b) zuführbar ist, b) welches einen Speicher aufweist, indem die Position der Bedienelemente (6b) des10) signaling control unit (4), a) which a search query of the operator (1) " • can be supplied from a speech recognition system with regard to a control element (6b) to be found, b) which has a memory by the position of the control elements (6b) of the
Gerätes (2) abgespeichert ist, c) welcher von einem Sensor ein Signal zur Beschreibung der Ist-Position der Hand/Hände (11) des Bedieners (1) zuführbar ist, d) und welches ein Sprachsignal (7b) an den Bediener (1) in Abhängigkeit von der Ist- Position der Hand/Hände (11) des Bedieners und der dem Speicher entnehmbarenDevice (2) is stored, c) which can be supplied by a sensor with a signal describing the actual position of the hand (s) (11) of the operator (1), and d) and which a voice signal (7b) to the operator (1 ) depending on the actual position of the hand / hands (11) of the operator and that which can be removed from the memory
Position des aufzufindenden Bedienelements (6b) erzeugt. Position of the control element to be found (6b) generated.
PCT/EP2002/005650 2001-08-16 2002-05-23 Method for assisting the user of a piece of equipment in locating a control element WO2003016110A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE10140097.7 2001-08-16
DE10140097A DE10140097B4 (en) 2001-08-16 2001-08-16 Method for assisting an operator of a device in finding the position of a control element

Publications (3)

Publication Number Publication Date
WO2003016110A2 true WO2003016110A2 (en) 2003-02-27
WO2003016110A3 WO2003016110A3 (en) 2003-11-20
WO2003016110A8 WO2003016110A8 (en) 2003-12-18

Family

ID=7695552

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2002/005650 WO2003016110A2 (en) 2001-08-16 2002-05-23 Method for assisting the user of a piece of equipment in locating a control element

Country Status (2)

Country Link
DE (1) DE10140097B4 (en)
WO (1) WO2003016110A2 (en)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10321964A1 (en) * 2003-05-15 2004-12-09 Webasto Ag Vehicle roof with an operating device for electrical vehicle components and method for operating electrical vehicle components
DE102004045885A1 (en) * 2004-09-22 2006-03-23 Volkswagen Ag Operating element for a motor vehicle
DE102005035992A1 (en) * 2005-08-01 2007-02-08 GM Global Technology Operations, Inc., Detroit Information system for a motor vehicle
DE102005040853A1 (en) * 2005-08-29 2007-03-08 Webasto Ag Control unit for roof lining area of interior of passenger car, has housing area formed of flat plastic part, and switching units arranged behind part when viewed from car interior and have capacitive sensor operated in contactless manner
SG133461A1 (en) * 2005-12-06 2007-07-30 Sin Etke Technology Co Ltd On-line voice help system and method for automobile
WO2015130567A1 (en) * 2014-02-28 2015-09-03 Bombardier Inc. Method, system, and executable program product for controlling passenger services
US10112716B2 (en) 2014-02-28 2018-10-30 Bombardier Inc. Method, system, and executable program product for controlling lighting
US10144512B2 (en) 2014-02-28 2018-12-04 Bombardier Inc. Method, system, and executable program product for controlling passenger services

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102005052843A1 (en) * 2005-11-05 2007-05-10 GM Global Technology Operations, Inc., Detroit Speech recognition system for use in motor vehicle, has operating unit e.g. pushbutton, for activation and deactivation of system, and optical indicator that is activated in activated condition of system
DE102012001149B4 (en) * 2012-01-21 2016-04-28 Audi Ag Method and device for generating a synthetic, spatially located, acoustic feedback signal upon actuation of a control element
DE102019211751A1 (en) * 2019-08-06 2021-02-11 Volkswagen Aktiengesellschaft Method for the acoustic display of a position of a vehicle element in a vehicle interior of a vehicle, as well as a sound generation system
DE102019212429A1 (en) 2019-08-20 2021-02-25 Volkswagen Aktiengesellschaft Method for providing individual user guidance for at least one functional component of a motor vehicle for a user in the motor vehicle, as well as an electronic user guidance system
DE102021003861A1 (en) 2021-07-27 2021-10-28 Daimler Ag Method and system for setting a de-escalating posture of a vehicle occupant when a police check is detected

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE3511026A1 (en) * 1985-03-27 1986-10-02 Blaupunkt-Werke Gmbh, 3200 Hildesheim FRONT PANEL WITH CONTROL ELEMENTS OF A DEVICE FOR MESSAGE TECHNOLOGY
CA2049173C (en) * 1990-08-16 1996-06-25 Shoji Fujimoto Speech recognition apparatus
DE69230419T2 (en) * 1991-05-31 2000-07-20 Koninkl Philips Electronics Nv Device with a human-machine interface
EP1164465A3 (en) * 1991-05-31 2002-01-09 Koninklijke Philips Electronics N.V. A method of processing user inputs
US5956682A (en) * 1991-06-06 1999-09-21 Lj Laboratories, Llc Picture frame with associated audio messages and position sensitive or speech recognition device
ES2125505T3 (en) * 1994-06-06 1999-03-01 Volkswagen Ag MULTIFUNCTIONAL INSTALLATION FOR THE ACTIVATION OF DIFFERENT ADJUSTMENT MEMBERS IN A VEHICLE AND VEHICLE WITH A MULTIFUNCTIONAL INSTALLATION OF THIS TYPE.
US6012029A (en) * 1995-09-29 2000-01-04 Cirino; Sepideh S. Voice activated system for locating misplaced items
DE19709518C5 (en) * 1997-03-10 2006-05-04 Harman Becker Automotive Systems Gmbh Method and device for voice input of a destination address in a real-time route guidance system
DE19753160C1 (en) * 1997-11-29 1999-04-15 Bosch Gmbh Robert Arrangement for detecting an impending accident situation for a motor vehicle
US6201540B1 (en) * 1998-01-07 2001-03-13 Microsoft Corporation Graphical interface components for in-dash automotive accessories
DE19845030A1 (en) * 1998-09-30 2000-04-20 Siemens Ag Imaging system for reproduction of medical image information
US6240347B1 (en) * 1998-10-13 2001-05-29 Ford Global Technologies, Inc. Vehicle accessory control with integrated voice and manual activation
DE19907795A1 (en) * 1999-02-24 2000-09-14 Daimler Chrysler Ag Method for supporting a user of a motor vehicle in the operation of components of the motor vehicle and an associated device
DE10008197C1 (en) * 2000-02-23 2001-07-26 Kostal Leopold Gmbh & Co Kg Multi-function operating device for automobile equipment, has operating surface with incorporated operating elements represented via head-up display provided on windscreen

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE10321964B4 (en) * 2003-05-15 2008-05-29 Webasto Ag Vehicle roof with an operating device for electrical vehicle components and method for operating electrical vehicle components
DE10321964A1 (en) * 2003-05-15 2004-12-09 Webasto Ag Vehicle roof with an operating device for electrical vehicle components and method for operating electrical vehicle components
US7342485B2 (en) 2003-05-15 2008-03-11 Webasto Ag Motor vehicle roof with a control means for electrical motor vehicle components and process for operating electrical motor vehicle components
DE102004045885A1 (en) * 2004-09-22 2006-03-23 Volkswagen Ag Operating element for a motor vehicle
DE102005035992A1 (en) * 2005-08-01 2007-02-08 GM Global Technology Operations, Inc., Detroit Information system for a motor vehicle
EP1749688A3 (en) * 2005-08-01 2010-03-31 GM Global Technology Operations, Inc. Information system for a motor vehicle
DE102005040853A1 (en) * 2005-08-29 2007-03-08 Webasto Ag Control unit for roof lining area of interior of passenger car, has housing area formed of flat plastic part, and switching units arranged behind part when viewed from car interior and have capacitive sensor operated in contactless manner
SG133461A1 (en) * 2005-12-06 2007-07-30 Sin Etke Technology Co Ltd On-line voice help system and method for automobile
WO2015130567A1 (en) * 2014-02-28 2015-09-03 Bombardier Inc. Method, system, and executable program product for controlling passenger services
CN106068225A (en) * 2014-02-28 2016-11-02 庞巴迪公司 For controlling method, system and the executable program product of passenger services
US10093419B2 (en) 2014-02-28 2018-10-09 Bombardier Inc. Method, system, and executable program product for controlling passenger services
US10112716B2 (en) 2014-02-28 2018-10-30 Bombardier Inc. Method, system, and executable program product for controlling lighting
US10144512B2 (en) 2014-02-28 2018-12-04 Bombardier Inc. Method, system, and executable program product for controlling passenger services

Also Published As

Publication number Publication date
DE10140097A1 (en) 2003-02-27
WO2003016110A3 (en) 2003-11-20
WO2003016110A8 (en) 2003-12-18
DE10140097B4 (en) 2005-04-28

Similar Documents

Publication Publication Date Title
EP1658200B1 (en) Vehicle system
EP1977926B1 (en) Control unit and control method
DE102017216837A1 (en) Gesture and facial expression control for a vehicle
EP1031456B1 (en) Procedure and device to support a motor vehicle user when operating vehicle components
DE102011112370A1 (en) motor vehicle
DE112014006409T5 (en) Vehicle control device and vehicle control method
WO2003016110A2 (en) Method for assisting the user of a piece of equipment in locating a control element
DE102010049081A1 (en) Method for operating a motor vehicle and motor vehicle
DE102014204980B4 (en) Method and apparatus for restricting or forcibly activating motor vehicle functions
DE102017011827A1 (en) Method for operating an occupant protection device
DE112017007207T5 (en) Passenger support device, method and program
EP2727773A1 (en) Method for activating a voice interaction with a passenger of a motor vehicle and voice interaction system for a vehicle
DE202013006009U1 (en) Driver assistance system
DE102012004601A1 (en) Assistance system for rider for offering and/or signaling information to passenger of motor car, has control device for controlling output device such that occupants offer and/or signal information in context-sensitive manner
DE102012203795A1 (en) Device for adjusting and/or readjusting of mirror in motor car, has acoustic echo sensor that is integrated in headrest of seat of driver so as to collect data regarding head position of driver relative to headrest
WO2016124278A1 (en) Determination of a position of a non-vehicle object in a vehicle
DE102017103391A1 (en) Method for improving the user-friendliness of a vehicle
DE102016211034A1 (en) Information providing device for a vehicle
DE10308415B3 (en) Seat setting control process for vehicles involves filming and storing person's seated position and using control unit to set seat accordingly
DE102018217634A1 (en) Method for operating a vehicle and vehicle
DE102017206876A1 (en) Method and device for outputting a status message in a motor vehicle with voice control system
DE102016200048B4 (en) Warning device and method for warning motor vehicle occupants
DE10012788B4 (en) Driver assistance device for a motor vehicle, in particular parking aid
DE102020114924A1 (en) MOTOR VEHICLE
DE102019008318A1 (en) Method for controlling at least one driver assistance system

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): JP

Kind code of ref document: A2

Designated state(s): JP US

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB IE IT LU MC NL PT SE TR

121 Ep: the epo has been informed by wipo that ep was designated in this application
CFP Corrected version of a pamphlet front page
CR1 Correction of entry in section i

Free format text: IN PCT GAZETTE 09/2003 UNDER (71) DELETE "PATENTE & LIZENZEN"

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase

Ref country code: JP

WWW Wipo information: withdrawn in national office

Country of ref document: JP