WO2014108160A2

WO2014108160A2 - User interface for the contactless selection of a device function

Info

Publication number: WO2014108160A2
Application number: PCT/EP2013/003880
Authority: WO
Inventors: Michael SCHLITTENBAUER; Martin Roehder; Heiko Maiwand; Nathaniel COSER; Lorenz Bohrer; Alexander Sebastian Strauch
Original assignee: Audi Ag; Volkswagen Ag
Priority date: 2013-01-08
Filing date: 2013-12-20
Publication date: 2014-07-17
Also published as: DE102013000081A1; WO2014108160A3; DE102013000081B4

Abstract

The invention relates to a method for operating a user interface (10) of a device, in particular of a motor vehicle, in which an optical sensor device (26) observes, in a contact-free manner, a hand (24) of a user and in accordance with the direction of movement (32, 34) of the hand (24), a function (40, 42) of the device (12) is selected and is activated. The aim of the invention is to provide a user interface for a device with which the user can control the device even with paying minimum attention. Said sensor device (10) can detect whether the user introduces the hand (24) into a predefined spatial region (30), and after detecting the hand (24) in the spatial region (30) activates a function selection. A sliding movement of the hand (24) in the direction of the edge of the spatial area (30) is detected and a direction of movement (32, 34) defined by the sliding movement is determined. Finally, a function (40, 42) associated with the direction of movement (32, 34) is selected.

Description

Operator interface for contactless selection of a device function

DESCRIPTION: The invention relates to a method for operating a control point of a device. By means of an optical sensor device of the user interface, a position of a hand of a person is determined without contact and a function of the device is selected and activated as a function of a movement of the hand. The invention also includes a motor vehicle with an operating interface for activating at least one function of the motor vehicle. Furthermore, the invention includes a computer program product by means of which it is possible to carry out the method according to the invention with a mobile terminal, that is, for example, a smartphone or a tablet PC, and a corresponding mobile terminal.

The non-contact observation of a hand of an operator and the movement of the hand-dependent control of a device are known from US 2011/0286676 A1 in connection with a gesture recognition. The system described in this document generates 3D image data about a body part of a person in a motor vehicle by means of a TOF camera (TOF - Time of Flight) and performs gesture recognition on the basis of the 3D image data. A distinction is made between dynamic and static gestures. When a gesture is detected by the system, a function associated with that gesture is activated in the motor vehicle.

Gesture recognition devices have the advantage that an operation of a device without buttons is possible. Keys are locally defined and usually must be located by an operator prior to operation. In particular, in connection with the operation of a motor vehicle while driving this is dangerous for a driver, because he is distracted from the traffic. Gesture recognition reduces attention loss by eliminating the need to search for a switch. The disadvantage of a gesture recognition, however, is that the operator has to remember the different gestures. Another drawback is that it can be difficult to ensure robust gesture recognition in varying light conditions.

The invention is based on the object to provide a user interface for a device, with which an operator can operate the device with little attention.

The object is achieved by a method according to claim 1, a motor vehicle according to claim 7, a computer program product according to claim 9 and a mobile terminal according to claim 10. Advantageous developments of the invention are given by the subclaims.

The method according to the invention is based on the approach described at the beginning, by means of an optical sensor device of an operator interface, to contactlessly determine a position of a hand of an operator and to select and activate a function of the device as a function of a movement of the hand. Unlike an elaborate gesture recognition device, however, in the method according to the invention it is sufficient for an operator to move his hand through a predetermined spatial area, e.g. swings left or right to select a specific function.

The optical sensor device works by the inventive method, so to speak, similar to a light barrier. This is achieved as follows. The sensor device first detects whether the operator grasps his hand into a predefined spatial area. After detecting the hand in the room area then a function selection is activated. In this function selection, a wiping movement of the hand is detected to an edge of the room area. The wiping movement defines a direction of movement. This is determined by the sensor device accordingly. Depending on the direction of movement, a specific function associated with the direction of movement is then selected. This can then be activated either immediately or automatically after confirmation by the operator. For example, pivoting the hand to the left may trigger a first function, eg accepting a telephone call, and pivoting the hand to the right a second function, eg refusing the telephone call. For a more accurate determination of the direction of movement, a current position of the hand in the spatial area can be defined as the starting position, and then a wiping movement of the hand starting from the starting position to an edge of the spatial area can be detected. Due to the starting position and the wiping movement emanating from this then the direction of movement is particularly clearly defined.

The method according to the invention has the advantage that the operator, without loss of attention, makes a binary selection with a single wiping motion (activate function or not). Another advantage is that the operation of the device is accelerated, if otherwise the function would otherwise first have to be selected from an operating menu of the device in order to then activate it. In the case of the method according to the invention, the actual detection of the position of the hand and its movement can be carried out in a known manner. Thus, a system used for this purpose does not need to be structurally adapted in a particular way.

A further advantage results when, when activating the function selection via a display device of the operator interface of the operator is displayed, by which movement, ie which direction of movement, which function can be selected. Thus, at least one possible direction of movement and in each case one of the possible directions of movement assigned, selectable function is displayed. The user interface is then without prior knowledge and thus very intuitive to use.

Another advantage arises when particularly frequently required functions or functions that are used in particularly driving-critical situations can be made selectable via a simple wiping movement of the hand. For this purpose, it must be possible to set by an operator himself which function is selected when moving the hand in a specific direction of movement within the spatial area. For this purpose, it is provided that in a configuration mode of the user interface, a user input regarding a specific function is received by the user interface and, depending on the user input, an assignment of the function to a possible direction of movement is determined. Such an assignment thus represents an abbreviation (short cut) for The function selection is also possible. It is also possible for several functions in different directions of movement.

A further advantage results if the ZuOrdnungsvorschrift by which at least one function is assigned to a possible direction of movement, is determined depending on a current operating state of the device. Other functions are then offered when a telephone is currently active than when music is playing. To make this possible, several prepared ZuOrdnungsvorschriften are provided, for example, one for the telephony and one for music playback, and one of a change of the operating state, one of them as the current valid ZuOrdnungsvorschrift selected. Thus, a context-dependent user interface is realized in an advantageous manner. For example, if the method is performed in an operator interface of a motor vehicle, an operator may be allowed to hand-grip into the space area and then determine whether to answer the telephone call (for example, pivoting the hand) to the left in the room area) or to reject the call (swiveling the hand to the right in the room area). If, on the other hand, the phone does not ring, but if, for example, music is being played back from a music file, it can be provided by a corresponding other assignment specification that the hand swiveling to the left is a previously played song and the hand is swiveling to the right to play one in the Playlist triggers subsequent song. By such a context-dependent assignment of functions to directions of movement, there is the advantage that the control of several different components of a device is made possible with a single user interface and constant operating movements of an operator. It has proven to be particularly advantageous if the optical sensor device observes the hand by means of a 3D camera and determines a position and movement of the hand on the basis of 3D image data of the SD camera. Thus, it is possible that all three spatial directions can be evaluated as the possible directions of movement for the selection of functions. In addition, the arrangement of the 3D Kanriera is independent of a movement plane of the wiping movements. In particular, it is also possible to detect movements away from or towards the 3D camera. A further advantage results if the sensor device additionally detects a respective finger position of at least one finger of the hand after the hand is detected within the spatial area and the function selection is activated only if the at least one finger has a respective predetermined finger position. In other words, the operator interface responds only when the operator forms a predetermined character with his hand within the space area. This avoids accidental selection of functions. In addition or alternatively to the finger position, the orientation of the rotation of the entire hand can also be checked to see whether it is inclined in a predetermined manner. The recognition of the finger position and the inclination of the hand can be realized on the basis of a skeleton or hand model adapted to a shape of the hand recognizable in the image data. If the model conforms to the shape of the imaged hand, then a digital representation of the finger position and the position of the hand is available with the parameters of the model.

As already stated, the invention also includes a motor vehicle. The motor vehicle according to the invention has an operating interface for activating at least one function of the motor vehicle, wherein the operating interface identifies an optical sensor device and is designed to carry out an embodiment of the method according to the invention. In particular, in the motor vehicle according to the invention the operating interface is part of an infotainment system of the motor vehicle. The motor vehicle according to the invention has the advantage that it is also possible for a driver to select and activate the functions of the motor vehicle during a journey without requiring a great deal of attention from the driver. In the motor vehicle according to the invention, it has proved to be particularly advantageous if the sensor device has a TOF camera for observing the hand. Variable light conditions then have no particularly disturbing influence on the motion detection. In contrast to this, the use of a simple infrared camera, for example, can lead to an error detection when warm sunlight is received. In contrast, the TOF camera uses a light beam modulated in the light intensity (eg visible light or infrared light), so that foreign light sources can be reliably distinguished from the light source of the TOF camera. The invention further includes a computer program product, ie a program stored on at least one storage medium. This is designed, when executed by a processor device of a mobile terminal, for example, a smartphone, a tablet PC, a personal digital assistant (PDA) or a notebook, on the basis of camera data of a camera of the mobile terminal, an embodiment of the method according to the invention perform. The computer program product according to the invention thus has the advantage that the mobile terminal can be operated by simple wiping movements of one hand, if they are performed in a detection range of the camera of the mobile terminal. The computer program product is preferably provided in a form known from the prior art in connection with so-called "apps" (applications - application programs) for mobile terminals.

Finally, the invention also encompasses a mobile terminal in which an embodiment of the computer program product according to the invention is provided. The mobile terminal according to the invention may in particular be a smartphone, a tablet PC, a notebook or a personal digital assistant.

In the following, the invention is explained once again with reference to a concrete embodiment. This shows:

1 shows a block diagram of an operating interface which may be installed in an embodiment of the motor vehicle according to the invention, and FIG. 2 shows a sketch of an operating procedure as it is made possible for an operator on the basis of an embodiment of the method according to the invention.

The examples shown represent preferred embodiments of the invention.

In Fig. 1 of a motor vehicle, such as a passenger car, an optical sensor device 10 and a re-input device 12 are shown. The optical sensor device 10 forms an operating The playback device 12 can be, for example, an infotainment system, an audio system, a navigation system, a television system, a telephone system, a combination instrument or a head-up display. The sensor device 10 comprises a measuring device 14 and a calculation unit 16. The measuring device 14 comprises an optical sensor 18, which may be, for example, a TOF camera or PMD camera. The optical sensor 18 may also be, for example, a stereo camera. In the example shown in FIG. 1, it has been assumed that the optical sensor 18 is a PMD camera. The optical sensor 18 may be arranged, for example, in a headliner of the motor vehicle.

The optical sensor 8 can be configured in a manner known per se, ie a light source 20 for electromagnetic radiation, for example an infrared light, illuminates a detection area 22, for example a space above a center console of the motor vehicle. If there is an object in it, for example a hand 24 of the driver of the motor vehicle, then the radiation emitted by the light source 20 is reflected back by the hand 24 to a sensor array 26. By means of the sensor array 26 then 3D image data can be generated, which indicate 3D coordinates to individual surface elements of the hand 24. The 3D image data are transmitted from the measuring device 14 to the calculation unit 16. The calculation unit 16 may be, for example, a control unit of the motor vehicle. In the calculation unit 16, the signals are evaluated and then the evaluated data are made available to the vehicle, for example, by being transmitted to the reproduction device 12. By means of the calculation unit 16 limbs, such as, for example, the hand 24, can be segmented from the 3D image data, whereby, for example, the position of a fingertip in the detection area 22 can be determined. Here, known per se segmentation algorithms can be used. The 3D image data of the sensor array 26 of the optical sensor 18 can also represent a sequence of successive 3D images, ie movements of the hand 24 can also be detected with the optical sensor 18. By tracking the trajectory, for example, the fingertip or the center of gravity in this 3D image sequence, in particular By tracking the position and speed of the fingertip, a motion gesture indicated by the fingertip can be extrapolated from the trajectory. FIG. 2 shows how an embodiment of the method according to the invention can be implemented with the optical sensor device shown in FIG. The optical sensor 26 is directed to an interior of the motor vehicle, so that in a detection area 28 of the optical sensor 26 therein objects are detected and to individual surface areas of the objects 3D image data, so for example coordinates, to their position in space by the measuring device 14th be generated and transferred to the calculation unit. The detection area 28 may, for example, include a space area above a center console of the motor vehicle.

An operator, for example the driver of the motor vehicle, has moved his hand 24 into the detection area 28. The calculation unit 26 recognizes from the 3D image data of the optical sensor 26 that it is a moving object in the shape of a hand and follows a movement trajectory of the hand 24 in the detection area 28. The calculation unit 16 checks whether the hand is in a predetermined space area 30 is located. The space region 30 may be, for example, a cuboid volume region, the edges of which may each independently have dimensions which are, for example, in a range from 20 cm to 50 cm.

If the calculation unit 16 recognizes that the hand 24 is in the space area 30, a selection mode for a function selection is activated. It is thereby possible for the operator to select a function of the vehicle by means of a pivoting movement of the hand 24 in possible directions of movement 32, 34 (here, for example, either to the right or to the left), for example accepting a telephone call or changing a radio station, and this function then activate. When the function selection is activated, the calculation unit generates on a screen 36 a display 38 by means of which selectable functions 40, 42 are displayed. The functions are represented in FIG. 2 by symbolic function names (F1, F2). In addition, it may be provided that an image of a slide switch 44 is displayed, which intuitively suggests to the operator that from her now a movement with the hand 24 in one of the possible Movement directions 32, 34 is expected to select one of the functions 40, 42.

The selection is possible for the operator by a simple swipe with the hand 24 after the function selection has been activated. The detection of the wiping movement is performed by the computing unit 16 based on the 3D image data of the optical sensor 26. For this purpose, the current position of the hand 24 after activating the function selection is set as the starting position PO. Subsequently, the computing unit 16 observes a change in the coordinates of a reference point on the hand 24, e.g. that reference point whose position has defined the starting position PO. As soon as the reference point has a distance to the starting position PO which is greater than a predetermined threshold, this is interpreted as a selection operation by the operator, i. H. the direction of movement described by the starting point PO and the current position of the hand 24 is compared with one of the possible directions of movement 32, 34. If it is detected by the calculation unit that the hand 24 has been moved in one of the possible directions of movement 32, 34, then the corresponding function 40, 42 is selected and the selection of the display device 12 is signaled. As a result, the selected function can then be activated in the playback device 12.

It is thus possible, for example, for a previous track to be played with a gesture to the left for the operation of a radio or a player, and a track being switched further to the right with a gesture. Instead of the shown binary swipe gesture (left / right) also swipe gestures "hoch'T'runter" or a one-sided swipe gesture in only one direction or even, for example, six swipe gestures (up / down, right / left, forward / back) as possible The assignment of the possible directions of movement 32, 34 to selectable functions 40, 42 can also be context-dependent, ie the selectable functions 40, 42 can be changed.

The example shows how a clear and intuitive selection of functions can be made possible, which can be performed after a short learning period for a passenger by heart and without distraction from the traffic. The operator can thus with a loss of attention with a single operating gesture make a selection, in particular a binary selection. In addition, the operation can also be accelerated.

Claims

CLAIMS:

1. A method for operating an operating interface (10) of a device, in particular of a motor vehicle, wherein an optical sensor device (26) of the operating interface (10) non-contact a hand (24) of an operator observed and in dependence on a direction of movement (32, 34) of the hand (24) a function (40, 42) of the device (12) is selected and activated,

characterized in that

is detected by the sensor device (10), whether the operator with the hand (24) into a predefined space area (30), and after detecting the hand (24) in the space area (30) a function selection is activated, in which a wiping movement the hand (24) is detected towards an edge of the spatial area (30) and a movement direction (32, 34) defined by the wiping movement is determined and a function (40, 42) associated with the movement direction (32, 34) is selected ,

2. The method of claim 1, wherein in activating the function selection of the operator on a display device (36) of the Bedienschnitt- point (10) at least one possible direction of movement (32, 34) and each one of the possible direction of movement (32, 34) associated, selectable function (40, 42) is displayed.

3. The method of claim 1 or 2, wherein an assignment of at least one selectable function (40, 42) to a possible direction of movement (32,

34) is set in a configuration mode of the operator interface (10) by the operator interface (10) in response to a user input.

4. The method according to any one of the preceding claims, wherein an assignment rule, by which at least one function (40, 42) of a possible direction of movement (32, 34) is assigned, depending on a current operating state of the device (12) is determined and For this, the ZuOrdnungsvorschrift is selected from several prepared ZuOrdnungsvorschriften.

5. The method according to any one of the preceding claims, wherein in the optical sensor device (10) the hand (24) by means of a 3D camera beo- is observed and a position and / or movement of the hand (24) on the basis of 3D image data of the 3D camera (26) are determined.

6. The method according to any one of the preceding claims, wherein by the sensor device (10) after detecting the hand (24) has a respective

Fingerstellung of at least one finger of the hand (24) and / or alignment of a rotational position of the hand (24) is detected and the function selection is activated only if the at least one finger each predetermined finger position and / or the hand (24) has a predetermined inclination having.

7. Motor vehicle with an operator interface (10) for activating at least one function of the motor vehicle, wherein the user interface (10) comprises an optical sensor device (10) and is adapted to perform a method according to one of the preceding claims.

8. Motor vehicle according to claim 7, wherein the sensor device (10) comprises a TOF camera (26) for observing the hand.

9. Computer program product with a program stored on at least one storage medium, which is designed to perform a method according to one of claims 1 to 6 when executing the program by a processor device of a mobile terminal based on camera data of a camera of the mobile terminal.

10. Mobile terminal, which has a computer program product according to claim 9.