WO2016067082A1

WO2016067082A1 - Method and device for gesture control in a vehicle

Info

Publication number: WO2016067082A1
Application number: PCT/IB2015/001966
Authority: WO
Inventors: Frank Schliep; Oliver Kirsch; Christian Sinn; Stephen WINTON; Xin Jiang
Original assignee: Visteon Global Technologies, Inc.
Priority date: 2014-10-22
Filing date: 2015-10-22
Publication date: 2016-05-06
Also published as: DE202015105611U1

Abstract

The invention relates to a method for gesture control in a vehicle interior (1), wherein according to the invention image data (I) of at least one position of an occupant (4) and at least one gesture of the occupant (4) are detected by the detection unit (2), which is designed as a 3D camera, in a detection region (E) of the detection unit (2). The invention further relates to a device (V) for gesture control in a vehicle interior (1).

Description

Method and device for gesture control in a vehicle

description

The invention relates to a method for gesture control in one

Vehicle interior according to the preamble of claim 1. Furthermore, the invention relates to a device for gesture control in

Vehicle interior.

So-called TOF cameras (short for: time-of-flight camera) are known from the prior art, which determine distances to objects with a transit time method. Here, the TOF camera is based on an active one

Illumination of a scene, for example by infrared light, and determines a duration of the light used for illumination by means of sensors. On this basis, a distance of the illuminated object from the TOF camera is then determined.

Object of the present invention is to provide a comparison with the prior art improved method and an improved apparatus for

Indicate gesture control in a vehicle with a detection unit.

With regard to the method, the object is achieved according to the invention with the features specified in claim 1. With regard to the device, the object is achieved according to the invention with the features specified in claim 10.

Advantageous embodiments of the invention are the subject of

Dependent claims. In the method for gesture control in the vehicle interior according to the invention by a trained as a 3D camera

Detection unit image data of at least one position of an occupant and at least one gesture of the occupant in a detection range of

Capture unit are detected.

The advantages achieved by the invention are, in particular, that the detection unit formed as a 3D camera combines both the position of a head of the occupant, head movements and gesture movements are combined and analyzed, so that an ergonomic

Working and operating vehicle functions and / or

Vehicle components based on an eye, head and / or

Gesture control of the occupant in the vehicle are possible or is. The registration unit is part of an interactive system

User interface for interactions and controls formed by gestures with or without contacting a solid of the vehicle, the gestures are carried out in particular by a person. In one possible embodiment of the method, a 3D point cloud is generated on the basis of the captured image data for determining the position, gesture and at least one distance of the occupant. In particular, the 3D point cloud makes it possible to determine a distance and position of an object and the occupant located in the detection area in a simple manner.

In a further embodiment of the method, to verify the position and gesture, these are compared with a generated and given head-shoulder template or an upper body template. Based on these templates, the position and gesture can in particular be evaluated quickly. A further embodiment of the method provides that the detection area is subdivided into at least two interaction areas for recognizing and tracking a movement and / or a gesture of the occupant. Thus, in particular, the movement and / or gesture of the head and the hand of the occupant can be detected spaced from each other.

One possible embodiment of the method provides that the movement and / or gesture of a head and a hand of the occupant in the same

Interaction area are recorded or will. This allows, for example, a quick activation or deactivation of the

Vehicle application or vehicle function. Here, the movement and / or gesture of the head and the hand of the occupant can be determined clearly and thus quickly.

In a development of the method, the movement and gesture executed by the occupant will be detected at the same time. For example, the position of the head is detected and rigid in combination with a 3D

Hand gesture of the occupant triggered the vehicle application in a fast manner.

In a further embodiment of the method, the movement and / or gesture of the head and the hand carried out by the occupant are detected in at least one of the two interaction areas in different time periods. In particular, a workload and an execution effort of the occupant, in particular of the driver, may interact with and / or control at least one of them

Vehicle component can be reduced. Thus, the driver can perform at intervals the movement of the head and then the hand. In a further embodiment of the method, the interaction and / or control is / are determined by detecting a line of sight of the occupant. In particular, this allows the driver to easily or simply activate or deactivate interaction with the vehicle application.

In one possible embodiment of the method, the viewing direction of eyes and / or eyelids of the eyes of the occupant is or are determined. This can be especially a tiredness and a

Attention of the driver to be monitored.

A device for gesture control in a vehicle interior comprises a detection unit which displays image data of at least one position of an occupant and at least one gesture of the occupant in one

Detected detection range of the detection unit, wherein the

Detection unit is designed as a 3D camera. In particular, the detection unit is designed as a TOF camera (short for: time-of-flight camera). The driver's gesture control is particularly accurate and targeted with the trained as a 3D camera or TOF camera

Capture unit detectable. Here, a body of the gesture-executing occupant or certain body parts of the occupant and a background serves as a reference area for the TOF camera.

A further embodiment of the device provides that the

Detection unit electrically connected to a multifunction display and / or integrated into a unit of the multi-function display.

In particular, the gesture and movement of the occupant provided for operating the multifunction display can be specifically detected by the detection unit. In one possible refinement of the device, the detection unit is data-technologically coupled to a control unit for processing and evaluating acquired image data, wherein the control unit compares the acquired image data with predetermined image data for evaluating the gesture or a state of the occupant. For example, due to the comparison of acquired image data with predetermined image data, a fast execution

different vehicle functions allows. In addition, the evaluation of acquired image data to increase driving safety, in particular for monitoring the driver while driving, and the

Seat occupancy recognition provided.

A further embodiment of the device provides that the

Detection unit with a lighting unit for illuminating the

Detection area is signal coupled, the

Lighting unit, for example, event-driven, for example, at predetermined intervals, emitted light in the infrared spectral range. In this case, a glare of the occupant is avoided, wherein the detection area for precise determination of the position or the gesture of the occupant is sufficiently illuminated.

Embodiments of the invention will be explained in more detail with reference to drawings.

Showing:

1 shows schematically a perspective view of a

Vehicle interior with a device for

Gesture control, Figure 2 schematically shows a block diagram of the device in a

possible embodiment, FIG. 3 schematically shows a block diagram of the device in a further embodiment, and

FIG. 4 shows schematically a block diagram of the device in one

possible design.

Corresponding parts are the same in all figures

Provided with reference numerals.

1 shows a vehicle interior 1 of a vehicle not shown in detail with a device V, comprising a detection unit 2, for gesture control in the vehicle.

The detection unit 2 is designed here as a 3D camera.

In particular, the detection unit 2 is a TOF camera, a

Stereo camera or other suitable for the vehicle interior 1 SD camera is formed.

In the vehicle interior 1, the detection unit 2 is in one

Dashboard 3, preferably in close proximity to one

Multifunction display 3.1 arranged such that an occupant 4,

in particular at least one upper body region of the occupant 4, can be detected, wherein the occupant 4, in particular a movement of his

Upper body area, can be determined.

The multi-function display 3.1 is in particular an infotainment system. The detection unit 2 comprises a detection area E which is directed into the vehicle interior 1, wherein the detection area E detects at least one position of a head 4.1 of the occupant 4 and a movement performed by the occupant 4 as a gesture.

As a gesture, for example, a movement of the head 4.1 and / or a movement of a hand 4.2 and / or an arm of the

Occupant 4 detected to trigger an associated with the movement and / or gesture of the head 4.1 and / or the hand 4.2 application.

For this purpose, the detection unit 2 in the form of the TOF camera is connected to a control unit 6, which detected with the detection unit 2

Image data I processed and evaluated.

Furthermore, the detection unit 2 is, for example, as one

Thermal imaging camera, in particular as an infrared camera, formed with an active illumination of a scene using infrared light and determined by a transit time measurement of a light used for illumination, a distance of an illuminated object. Such an illuminated object is, for example, the upper body portion or the head portion of the occupant 4.

In this case, a scene or gesture, in particular the temporal position of the head 4.1 and / or the hand 4.2, in the detection area E of

Detecting unit 2 detected. This scene or gesture is going through

corresponding camera data settings and camera properties, for example, the TOF camera using a 3D point cloud to determine the distance of the illuminated object by an amplitude evaluation of the intensity of the reflected and detected light signals in

Detection area E recorded. Preferably, the detection unit 2 and a lighting unit 5 are arranged in the dashboard 3 and signal technology coupled that the light at predetermined intervals light, especially in the infrared spectral range, emitting lighting unit 5 the detection area E of

Detection unit 2 illuminated.

Furthermore, the detection unit 2 comprises such an optical

Resolution that the detection unit 2 within the

Detection area E local areas or local zones, such as the head 4.1, a face, in particular eyes and / or nose, and / or the hand 4.2 of the occupant 4, can capture, for example, a

Viewing direction of the eyes to detect a movement of the head 4.1 and / or a gesture of the hand 4.2 of the occupant 4. The detection unit 2 is in particular electrically connected to the multifunction display 3.1 and / or in a structural unit of

Multifunction display 3.1 integrated.

By this arrangement, it is possible in an advantageous manner, with a largely high precision, the movement and / or gesture, in particular an extrapolation of the movement and / or gesture of the occupant 4 to detect relative to the multifunction unit 3.1 within the detection area E. Detection and / or tracking of the movement and / or gesture of the occupant 4 is performed by the detection unit 2 in the same

Detection area E, wherein the head 4.1 and the hand 4.2 of the occupant 4 independently of each other can trigger the associated with the executed movement and / or gesture application. Here, the detection range E of the detection unit 2 in at least two interaction areas, in which in each case the movement and / or gesture of the head 4.1 and the hand 4.2 of the occupant 4 from each other

be divided at a distance.

Alternatively, a combination of the movement and / or gesture of the head 4.1 and the hand 4.2 of the occupant 4 relative to one another, in particular in the same interaction region of the detection area E, be detectable, for example, a desired application,

in particular function of the vehicle, to activate or deactivate.

For applications in which the movement and / or gesture of the head 4. 1 and that of the hand 4. 2 are to be recorded in the same interaction area, different possibilities for processing are covered

Image data I provided.

One possibility for the rapid activation or deactivation of the application desired by the occupant 4 is based on the processing of at least simultaneously acquired image data I from the occupant 4

Movements and / or gestures in the same interaction area of the

Detection range E.

Another possibility is based on the processing of in

Image data I detected a different movement and / or gesture of the head 4.1 and the hand 4.2 of the occupant 4.

By the first possibility, the application of the vehicle

For example, two movements and / or gestures executed in the same interaction area and associated with the application can be activated or deactivated, whereby, for example, the position of the head 4.1 is detected and in combination with a 3D rigid hand gesture of the occupant 4 the application is triggered.

The second possibility can be at least one workload of the

Inmates 4, in particular of a driver, for interaction with at least one vehicle component and / or for controlling the vehicle component reduce, wherein the interaction and / or control near the head 4.1 and / or the hand 4.2 of the occupant 4 takes place. The interaction with and / or the control of the at least one

Vehicle component is detected by detecting the line of sight of the eyes of the occupant 4 within the interaction area of

Detection area E determined, the interaction and / or control based on the associated with the application and detected motion and / or gesture, in particular hand gesture of the occupant 4, takes place.

Alternatively, the interaction and / or control may be determined by the combination of the movement associated with the application and / or gesture of the head 4.1 and the hand 4.2. For example, based on the extrapolation of the hand gesture, in particular finger gesture, and a subsequent movement and / or gesture of the head 4.1, in particular nodding, the interaction and / or the control of the at least one

Vehicle component done. The position of the hand 4.2 of the occupant 4 in front of the head 4.1 of the occupant 4 in the same interaction area of the detection area E of

Detection unit 2 can be detected, for example, as the associated with the application movement and / or gesture of the occupant 4. To effectively monitor a behavior of the occupant 4 is the

Detection unit 2 directed into the vehicle interior 1, in particular outside a (main) axis. In order to be able to detect a head area of the occupant 4 outside the (main) axis, a method based on a 3D model for the partial recognition of the head 4.1 is provided. Preferably, in particular, the position of the head 4.1 and / or the eye and / or a nose vector and / or facial features of the occupant 4 can be detected thereby. This detection takes place in particular on the basis of a filter algorithm, for example a so-called Kalman filter, for processing statically estimated and / or predicted values.

FIGS. 2 to 4 each show a block diagram of an embodiment of the device V for detecting and / or detecting the head 4.1 of FIG

Inmates 4 shown. In addition, a sequence of the method for detecting and / or detecting the head 4.1 of the occupant 4 is shown.

For illuminating the detection area E, the detection unit 2 is electrically coupled to the illumination unit 5. For processing and

Evaluation of acquired image data I in the detection area E is the

Detection unit 2 electrically coupled to the control unit 6. These image data I include, for example, movements, gestures and / or

Facial features and facial structures of the occupant 4 and / or

Positions of body parts of the occupant 4 and / or other objects in the vehicle interior 1.

The following is the procedure for gesture control in detail

described. By means of the detection unit 2, in particular the 3D point cloud of the TOF camera, the image data I, in particular of the upper body region of the occupant 4, in the detection area E are determined on the basis of local coordinates and light reflections of the object. In this case, in particular protruding facial features and / or facial structures are detected on the basis of detected amplitude maps.

The device V comprises, for example, an interface S1 to

Application programming of the detection unit 2.

The detection and / or detection of the head 4.1 of the occupant 4 is based on a combination of 2D and 3D Algohthmen to largely appropriate information from the 3D point cloud and the

Extract amplitude map.

Furthermore, the device V comprises a connection interface S2, which detected the from the 3D point cloud and amplitude mapping

Image data I includes and is provided for noise filtering the captured image data I.

To different distances of different objects, in particular steering wheel and / or window of the vehicle, within the

Detection area E of the detection unit 2 to be able to determine in the vehicle interior 1, the device V comprises a

Segmentation interface S3. This will be in the 3D point cloud

Foreground divided into segments, object boundaries detected and compared within the local coordinates in the control unit 6 with each other.

A verification interface S4 of the device V is provided for checking the position of the head 4.1 within at least one segment, wherein the position of the head 4.1 by a head-shoulder detector S4.1 is determined. Here, the segments are each compared with a head-shoulder template S4.2 or a reference.

For a most accurate comparison within the local

Coordinates, in particular 3D coordinates, the head-and-shoulders template S4.2 is aligned to the segment such that the head-shoulder template S4.2 scales at least with a segment size, the head-shoulder template S4.2 based on a rotation is aligned. In the segment with a largely high agreement to the head and shoulder template S4.2 are at least heuristic

Assumptions S4.4 further coordinates for completing the position of the head 4.1 is determined. Alternatively, the position of the head 4.1 is determined based on an upper body template S4.3, in which at least body parts of the upper body of the occupant 4 are compared with pre-stored forms of the upper body of a human. Subsequently, with a match based on heuristic assumptions S4.4, further coordinates for completing the position of the head 4.1 are determined. For example, a head of the pre-stored shape of the upper body has an elliptical shape.

In a possible embodiment of the method is by a

Evaluation interface S5 estimated the position and viewing direction of the head 4.1 of the occupant 4 within a current freeze frame, with a largely suitable for estimation algorithm S5.1 is determined in advance.

When recognizing the header within a

Foreground segment, the position and direction of the head 4.1 can largely on the basis of acquired image data I of the current still image be determined. This is done via an output interface S5.2 as a function of the algorithm S5.1 the evaluation interface S5, wherein the captured image data I then as a motion and

Position sequence S5.3 are stored.

In the case of insufficient image data acquisition I of the detection unit 2 and / or unsuccessful estimation of the head area of the occupant 4, based on the stored movement and / or

Position sequences S5.3 determines a subsequent position and / or viewing direction of the head 4.1 of the occupant 4.

Alternatively, the position and / or viewing direction of the head 4.1 can be determined based on the nose vector. In addition, a viewing direction of the eye can differ from the viewing direction of the head 4.1, wherein the viewing direction of the eye can be determined, for example, by detecting at least one pupil.

For monitoring a condition of the occupant 4, in particular a tiredness of the driver, for example, a number of eyelids, intervals between the respective eyelid beats and / or duration of a

Lidschlages detected and compared with predetermined information, for example, in relation to eye behavior. In addition, such captured image data I are stored in a memory interface S6 or in the control unit 6, wherein the captured image data I can be retrieved at any time. A decision is made on the basis of a preprogrammed in the control unit 6 decision tree, which automatically from the stored image data I at least one suitable method, in particular detection by nose vector and / or

Eye position, selects to capture the line of sight of the occupant 4. Furthermore, individual information about head size, face geometry and / or head-to-shoulder distance can be different

Inmates 4 are stored, which are available at any time.

The method can also be used to improve driving experience and / or driving safety, and / or to monitor the driver and / or

Seat occupancy detection can be used.

LIST OF REFERENCE NUMBERS

Vehicle interior

2 detection unit

3 dashboard

3.1 Multifunction display

4 inmates

4.1 head

4.2 Hand

5 lighting unit

6 control unit

V device

E detection range

I image data

51 interface

52 processing interface

53 Segmentation interface

54 verification interface

54.1 Head-shoulder detector

54.2 Head and shoulder template

54.3 Upper body template

54.4 heuristic assumptions

55 evaluation interface

55.1 Algorithm

55.2 Output interface

55.3 Movement and Position Sequence

55.4 Position and viewing direction

56 memory interface

Claims

claims

1. A method for gesture control in a vehicle interior (1) by a detection unit (2),

characterized in that formed by the 3D camera as

Detection unit (2) image data (I) at least one position of an occupant (4) and at least one gesture of the occupant (4) in a detection area (E) of the detection unit (2) are detected. 2. The method according to claim 1,

characterized in that on the basis of the acquired image data (I), a 3D point cloud is generated to determine the position, gesture and at least one distance of the occupant (4). 3. The method according to claim 1 or 2,

characterized in that to check the position and gesture, these are compared with a generated head-shoulder template (S4.2) or an upper body template (S4.3). 4. The method according to any one of the preceding claims,

characterized in that the detection area (E) in at least two interaction areas for the detection and tracking of a

Movement and / or a gesture of a head (4.1) and a

Hand (4.2) of the occupant (4) is divided.

5. The method according to claim 4,

characterized in that the movement and / or gesture of the occupant (4) is / are detected in the same interaction area. 6. The method according to claim 4 or 5,

characterized in that the executed by the occupant (4) Movement and gesture are recorded at the same time.

7. The method according to claim 4,

characterized in that the movement and / or gesture of the head (4.1) and the hand (4.2) performed by the occupant (4) are detected in at least one of the two interaction areas in different time periods.

8. The method according to any one of the preceding claims,

characterized in that the interaction and / or control based on the detection of a line of sight of the occupant (4) is determined or will.

9. The method according to claim 8,

characterized in that the viewing direction of eyes and / or eyelids of the eyes of the occupant are determined.

10. Device (V) for gesture control in a vehicle interior (1) with a detection unit (2),

characterized in that the detection unit (2) image data (I) at least one position of an occupant (4) and at least one gesture of the occupant (4) in a detection area (E) of

Detection unit (2) detected, wherein the detection unit (2) is designed as a 3D camera.

1 . Device (V) according to claim 10,

characterized in that the detection unit (2) is designed as a TOF camera.

12. Device (V) according to claim 10 or 11,

characterized in that the detection unit (2) with a Multifunction display (3.1) connected and / or integrated into a unit of the multi-function display (3.1)

13. Device (V) according to one of claims 10 to 12,

characterized in that the detection unit (2) with a control unit (6) for processing and evaluation detected

Image data (I) is coupled.

14. Device (V) according to one of claims 10 to 13,

characterized in that the control unit (6) compares instantaneous image data (I) with predetermined image data for evaluating the gesture or a state of the occupant (4).

15. Device (V) according to one of claims 10 to 14,

characterized in that the detection unit (2) is coupled to a lighting unit (5) for illuminating the detection area (E).