WO2017167484A1

WO2017167484A1 - Method and device for localising the head of a vehicle occupant

Info

Publication number: WO2017167484A1
Application number: PCT/EP2017/052894
Authority: WO
Inventors: Anke SCHWARZ; Ernst Schermann
Original assignee: Robert Bosch Gmbh
Priority date: 2016-03-31
Filing date: 2017-02-09
Publication date: 2017-10-05
Also published as: CN108885340A; DE102016205307A1; EP3436866A1

Abstract

The invention relates to a method for localising the head (108) of a vehicle occupant (106) wearing a pair of glasses (112). Said method consists of reading and processing information representing the glasses (112) in order to locate the head (108).

Description

Description title

Method and device for locating a head of a

passenger

State of the art

The invention is based on a device or a method according to the preamble of the independent claims. The subject of the present invention is also a computer program.

The head orientation of a driver of a vehicle plays an important role in safety and comfort functions. For example, based on the

Kopforientierung a lane change intent of the driver are predicted. Furthermore, the head orientation can be an important parameter for the visual

Assessment of attention or fatigue of the driver.

Various methods of head pose estimation are known,

For example, 2-D image-based methods or methods in which a head pose can be estimated using depth data in real time.

Disclosure of the invention

Against this background, with the approach presented here, a method for locating a head of a vehicle occupant, a device using this method, and finally a corresponding one

Computer program presented according to the main claims. The measures listed in the dependent claims are advantageous Further developments and improvements of the device specified in the independent claim possible.

A method for locating a head of a vehicle occupant wherein the vehicle occupant wears spectacles is provided, the method comprising the steps of:

Reading a spectacle information representing the spectacles; and

Process the eyeglass information to locate the head.

The vehicle occupant may be a driver or passenger of a vehicle. For example, the eyeglass information may represent a position or position of the eyeglasses. For example, the spectacle information can be processed to determine a position or position of the head depending on the position or position of the spectacles.

The approach presented here is based on the knowledge that with the aid of data regarding a pair of spectacles worn by a vehicle occupant, a head of the vehicle occupant can be located in the vehicle interior. Thus, the head of the vehicle occupant can be located accurately and reliably even if the face of the vehicle occupant is partially hidden by the glasses.

According to one embodiment, in the reading step, a position and additionally or alternatively a position of the spectacles can be read in as the spectacle information. A position of the spectacles can be understood as data representing a location of the spectacles in the vehicle interior. A position of the glasses can be understood as meaning an orientation of the spectacles in the vehicle interior. For example, the position or location of the glasses can be made using a three-dimensional

Coordinate system have been determined. This embodiment enables accurate and reliable location of the head in response to location of the goggles. According to a further embodiment, at least one vehicle-occupant-specific parameter can be read in during the reading-in step. In the step of processing, the eyeglass information can be read using the

vehicle occupant-specific parameters are processed. A vehicle-occupant-specific parameter can be understood, for example, as a parameter with the aid of which the vehicle occupant can be identified. The vehicle occupant-specific parameter may indicate, for example, whether the vehicle occupant is wearing glasses or not. Furthermore, the vehicle occupant-specific parameter, such as a usual or average position or position of the spectacles in the vehicle interior, may represent. By this embodiment, the accuracy in

Localizing the head can be increased.

It is advantageous if, in the reading step, a reference position and / or reference position of the spectacles is read in as the vehicle occupant-specific parameter. In the step of processing, the head may be adjusted by comparing the position and / or position of the glasses with the reference position and / or

Reference location are localized. The reference position or positions may be, for example, a usual or average position or position of the spectacles when the vehicle occupant wears the spectacles in the vehicle. In particular, the reference position or positions may be a position or attitude of the goggles that corresponds to a position or attitude of the head with sufficient vigilance or attention of the vehicle occupant. By this embodiment, the head with low

Computing effort can be located reliably and accurately.

Furthermore, it is advantageous if, in the step of processing, the spectacle information is processed in order to determine a position and / or position of the head. This embodiment allows an unambiguous localization of the head.

According to a further embodiment, in the reading-in step, information generated using at least one image of the vehicle occupant may be read in as the spectacle information. This enables continuous generation of the spectacle information. The method may also include a step of capturing the image of the

Vehicle occupant by a detection device for detecting the

Vehicle occupants include. The detection device may be, for example, a camera for monitoring the vehicle interior. By this embodiment, the image of the vehicle occupant can be detected.

Furthermore, in the reading-in step, information generated using at least one algorithm may be read in as the eyeglass information. This embodiment enables a fast and efficient

Generate the spectacle information.

The method may further include a step of generating a

Attention of the vehicle occupant

Attention value depending on a result of processing the eyewear information include. Additionally or alternatively, in the step of generating, a drive signal for driving a vehicle control device may be generated depending on the result of the processing. This allows a reliable determination of the attention of the vehicle occupant or a reliable control of the vehicle.

This method can be implemented, for example, in software or hardware or in a mixed form of software and hardware, for example in a control unit.

The approach presented here also provides a device which is designed to implement the steps of a variant of a method presented here

to implement, control or implement appropriate facilities. Also by this embodiment of the invention in the form of a device, the object underlying the invention can be solved quickly and efficiently.

For this purpose, the device may comprise at least one computing unit for processing signals or data, at least one memory unit for storing signals or data, at least one interface to a sensor or a Actuator for reading sensor signals from the sensor or for outputting data or control signals to the actuator and / or at least one

Communication interface for reading or outputting data embedded in a communication protocol. The arithmetic unit may be, for example, a signal processor, a microcontroller or the like, wherein the memory unit is a flash memory, an EPROM or a

magnetic storage unit can be. The communication interface can be designed to read or output data wirelessly and / or by line, wherein a communication interface that can read or output line-bound data, for example, electrically or optically read this data from a corresponding data transmission line or output in a corresponding data transmission line.

In the present case, a device can be understood as meaning an electrical device which processes sensor signals and outputs control and / or data signals in dependence thereon. The device may have an interface, which may be formed in hardware and / or software. In the case of a hardware-based embodiment, the interfaces can be part of a so-called system ASIC, for example, which contains a wide variety of functions of the device. However, it is also possible that the interfaces are their own integrated circuits or at least partially consist of discrete components. In a software training, the interfaces may be software modules that are present, for example, on a microcontroller in addition to other software modules.

In an advantageous embodiment, the device is used to control a driver assistance system of a vehicle. For this purpose, the device may, for example, sensor signals such as image signals of at least one camera for monitoring the interior of the vehicle, in particular a

Head area of the vehicle occupant, environment sensor signals,

Steering angle sensor signals or Raddrehzahlsensorsignale access.

Also of advantage is a computer program product or computer program with program code which is stored on a machine-readable carrier or storage medium such as a semiconductor memory, a hard disk memory or an optical disk Memory may be stored and for carrying out, implementing and / or controlling the steps of the method according to one of the above

described embodiments, in particular when the program product or program is executed on a computer or a device.

Embodiments of the invention are illustrated in the drawings and explained in more detail in the following description. It shows:

1 shows a schematic representation of a vehicle with a device according to an embodiment;

Fig. 2 is a schematic representation of a head of a vehicle occupant of Fig. 1;

Fig. 3 is a schematic representation of a device according to a

Embodiment;

4 is a schematic illustration of a procedure for locating a head using a device according to an embodiment; and

5 is a flowchart of a method according to a

Embodiment.

In the following description of favorable embodiments of the present invention are for the in the various figures

represented and similar elements acting the same or similar

Reference numeral used, wherein a repeated description of these elements is omitted.

1 shows a schematic representation of a vehicle 100 with a device 102 according to an exemplary embodiment. The vehicle 100 is equipped with a detection device 104, which is designed to be a vehicle occupant 106, here a driver of the vehicle 100, in particular capture an area of a head 108 of the vehicle occupant 106 and send an image signal 110 representing at least one captured image of the vehicle occupant 106 to the device 102. The detection device 104 is, for example, a camera for monitoring a vehicle interior of the vehicle 100

Vehicle occupant 106 wears goggles 112. Device 102 is configured to detect goggles 112 using image signal 110 and to locate head 108 within the vehicle interior, depending on a result of detecting goggles 112.

According to an optional embodiment, the device 102 is

designed to detect a position or position of the spectacles 112 when detecting the spectacles 112 and to determine a position or position of the head 108 as a function of the position or position of the spectacles 112.

According to another optional embodiment, the device 102 is configured to generate and send a drive signal 114 to a vehicle controller 116 depending on a result of locating the head 108. The vehicle control unit 116 is configured, for example, to use the drive signal 114 to provide a driver assistance function for the driver

Assisting the vehicle occupant 106 in guiding the vehicle 100 to control. Optionally, the device 102 is configured to generate an attention value 118 representing an attention of the vehicle occupant 106, depending on the result of the location, and to the vehicle

Vehicle control unit 116 to send. Here, the vehicle control unit 116 is configured to use the vehicle 100 using the

Attention worth 118 to control.

Fig. 2 shows a schematic representation of a head 108 of a

The vehicle occupant 106 of FIG. 1. The head 108 shown in FIG. 2 is, for example, an image of the head 108 taken by the detection device. Shown is a frontal view of the head 108 where the eyeglasses 112 can be seen. For example, the device is configured to adjust the position or attitude of the glasses 112 from the image represented by the image signal using a three-dimensional coordinate system extract. For this purpose, the device determines the position or position of the spectacles 112 relative to an x, y or z axis. Based on x, y, and z coordinates representing the position or attitude of the glasses 112, the device determines x, y, and z coordinates representing the position or location of the head 108.

FIG. 3 shows a schematic representation of a device 102 according to an exemplary embodiment. The device 102 may be a device described above with reference to FIGS. 1 and 2. The

Device 102 comprises a read-in unit 310, which is designed to read in spectacle information 315 representing the spectacles and to a

Forward processing unit 320. The processing unit 320 is configured to locate the head of the vehicle occupant using the eyeglass information 315 and to generate a localization value 325 representing a result of the locating. Of the

Localization value 325 represents, for example, the position or position of the head determined as a function of the position or position of the spectacles.

According to one embodiment, the read-in unit 310 is configured to further read in the image signal 110 via an interface to the detection device and to forward it to an optional generation unit 330. The generating unit 330 is configured to use the

Image signal 110 to generate the eyeglass information 315. Here reads the

The reading unit 310 reads the eyeglass information 315 via the generating unit 330 e. According to an embodiment, the generating unit 330 is configured to extract the eyeglass information 315 from the image signal 110 using at least one algorithm.

Optionally, the generating unit 330 is configured to further include the

Localization value 325 received from the processing unit 320, below

Using the localization value 325 to generate the drive signal 114 or attention value 118 and output the drive signal 114 or attention value 118 to an interface to the vehicle control unit. According to a further exemplary embodiment, the read-in unit 310 furthermore serves to read in at least one vehicle-occupant-specific parameter 335. The read-in unit 310 forwards the parameter 335 to the processing unit 320, wherein the processing unit 120 is designed to generate the

Further, localization value 325 may be generated using parameter 335. The parameter 335 represents, for example, a reference position or reference position of the glasses. A reference position or positions may be understood to mean a desired position or desired position of the spectacles, which may correspond, for example, to an ordinary or average position or position of the spectacles. In this case, the processing unit 320 uses the parameter 335 to determine the position or attitude of the head by comparing the position or attitude of the goggles with the reference position or positions. Such a spectacle orientation estimate can be robust

Head orientation estimation can be performed.

4 shows a schematic illustration of a procedure for locating a head using a device according to an exemplary embodiment, for example a device as described above with reference to FIGS. 1 to 3. Here, a block 402 represents the driver of the vehicle, a block 402 an input, a block 404 a sensor for detecting the

Driver, a block 406 a spectacle detection, a block 408 a

Eyeglass segmentation, a block 410 an optional driver ID, a block 412 at least one optional further parameter, a block 414 a

For example, a block 416 is a computing unit, a block 418 is an output, a block 420 is a driver's line of sight, a block 422

Drowsiness level of the driver, a block 424 a driver's attention and a block 426 an airbag adjustment.

FIG. 5 shows a flowchart of a method 500 according to a

Embodiment. The method 500 may, for example, be carried out in conjunction with a device described above with reference to FIGS. 1 to 4. In this case, the spectacle information is read in a step 510. In step 520, the eyewear information is processed to locate the head of the vehicle occupant. Steps 510, 520 may be performed continuously.

In the case of severe occlusion of the head, it may be difficult to robustly determine head orientation using existing algorithms.

For example, a pair of (sun) glasses can greatly obscure a meaningful eye region important for a head pose estimation. Even with additional occlusions in the mouth area, it may be difficult or even impossible to perform a robust determination of head orientation.

By means of the method 500, it is now possible to assist the driver with more safety and comfort functions. A robust head orientation is a basic variable for these functions.

Current fatigue estimation systems are usually based on non-visual parameters such as a steering angle or data from

Tracking sensors. Such parameters are significantly further away in the signal chain than directly measurable physiological changes. Furthermore, the steering behavior often does not allow a clear conclusion on the fatigue of the driver. For example, when driving with a trailer, a steering angle signal similar to when driving with fatigue can be generated.

The above-described method for Kopfposenschätzung works, however, even if the driver wears glasses or sunglasses. In this case, the driver's eyes are hidden. By determining the head orientation based on the spectacle orientation, the head orientation can nevertheless be determined robustly.

In one embodiment, the head position or orientation is determined using 3-D data from a depth image of the driver. The glasses are detected from the depth image and segmented. The orientation of the glasses is determined and from this the head position and orientation are closed. The position of the glasses reflects a robust head position and orientation.

For this purpose, the driver is continuously detected by a sensor. The sensor captures both 2-D near-infrared data and depth data. From the obtained data, the glasses are detected and segmented. These

Eyeglass segmentation can be done through simple algorithms available

be implemented. The determination of the position and orientation of the spectacles is much simpler than the determination of the position and orientation of the head, since spectacles, especially in the depth image, have specific features such as a planar surface or detectable edges.

Optionally, an existing person ID becomes additional

Person-specific parameters taken into account, such as a familiar position or orientation of the driver's glasses. Alternatively, an average position or orientation of a pair of glasses can be used. Using these parameters, the position or orientation of the glasses determines the position or orientation of the head robust. Optionally, for example, a

Viewing direction, a level of drowsiness or a level of attention of the driver are determined.

If an exemplary embodiment comprises a "and / or" link between a first feature and a second feature, then this is to be read so that the embodiment according to one embodiment, both the first feature and the second feature and according to another embodiment either only first feature or only the second feature.

Claims

claims

A method (500) for locating a head (108) of a

A vehicle occupant (106), wherein the vehicle occupant (106) wears spectacles (112), the method (500) comprising the steps of:

Reading in (510) one of the glasses (112) representing

Eyeglass information (315); and

Processing (520) the eyewear information (315) to locate the head (108).

2. The method (500) according to claim 1, wherein in the step of

Reading in (510) a position and / or position of the spectacles (112) as the spectacle information (315) is read.

3. The method (500) according to one of the preceding claims, wherein in the step of reading (510) at least one

vehicle occupant-specific parameter (335) is read in, wherein in the step of processing (520) the eyewear information (315) is processed using the vehicle-occupant-specific parameter (335).

4. The method (500) according to claim 2 and 3, wherein in the step of

Read in (510) a reference position and / or reference position of the spectacles (112) as the vehicle occupant-specific parameter (335), wherein in the step of processing (520) the head (108) by comparing the position and / or position of the spectacles (112 ) is localized with the reference position and / or reference position. The method (500) according to one of the preceding claims, wherein in the step of processing (520) the eyewear information (315) is processed to determine a position and / or position of the head (108).

Method (500) according to one of the preceding claims, in which, in the reading-in step (510), information generated using at least one image of the vehicle occupant (106) is read in as the spectacle information (315).

The method (500) of claim 6 including a step of capturing the image of the vehicle occupant (106) by a vehicle

Detecting means (104) for detecting the vehicle occupant (106).

Method (500) according to one of the preceding claims, wherein in the reading-in step (510) an information generated using at least one algorithm is read in as the spectacle information (315).

The method (500) according to one of the preceding claims, comprising a step of generating an attention of

Vehicle occupant (106) representing

Attention value (118) and / or a drive signal (114) for driving a vehicle control device (116) depending on a result of the processing (520) of the eyewear information (315).

Device (102) comprising units (310, 320, 330) configured to execute and / or drive the method (500) according to one of the preceding claims.

A computer program configured to execute and / or drive the method (500) according to any one of claims 1 to 9.

A machine readable storage medium storing the computer program of claim 11.