GB2595895A

GB2595895A - Method for detecting safety relevant driving distraction

Info

Publication number: GB2595895A
Application number: GB2008797.9A
Authority: GB
Inventors: Mörti Peter; Höfler Margit; Güzel Kalayci Elem
Original assignee: Virtual Vehicle Research GmbH
Current assignee: Virtual Vehicle Research GmbH
Priority date: 2020-06-10
Filing date: 2020-06-10
Publication date: 2021-12-15
Also published as: GB202008797D0

Abstract

A method to detect driver distraction based on changes in control actions with which the driver controls the vehicle, for example the frequency of steering manoeuvres. An environmental information complexity value EIC is estimated 2 by analysing images from an outward facing camera 1 and weighting driving critical information elements (e.g. road curvature, traffic, signs, pedestrians). Recorded inputs of the control action rate 4 under baseline non-distracted driver conditions are used to determine 3 a preferred control action rate CAND for the complexity value to which the current observed control action rate CAobs is then compared 5. Based on the difference between the observed rate CAobs and the expected rate if the driver were not distracted CAND, a warning or driving intervention is outputted.

Description

Method for Detecting Safety Relevant Driving Distraction

Background of the invention

Driver distraction is one of the most common factors reported when it comes to crashes or safety critical events: Previous research has shown that 5 to 25% of accidents can be attributed to the involvement of driver distraction [1]. Typically, visual distraction (when the gaze is off the road) and cognitive distraction (when the mind is off the road) are distinguished [1] [2]. It has been shown that, when drivers are engaged in complex visual and/or manual secondary tasks, they have a nearcrash/crash risk three-times higher than attentive drivers [3]. Automated detection of driver distraction is therefore an important road safety method.

Several methods to detect driver distraction have been developed. Most common algorithms attempt to detect driver distraction work via driver gaze analysis where a camera observes the human driver to determine gaze behavior toward the road or away from it.

A new method is proposed that does not rely on driver gaze but on changes in control actions dependent on environmental information complexity. This invention describes a method measuring information complexity of the environment and establishing a driver's baseline control action variability under non-distracted driving. Significant deviations from the baseline control action variability indicate evidence for distracted driving.

State of the Art Several devices and methods to measure driver distraction are presented that mainly measure distraction via the direction of the gaze or the position of the head or other physiological data such as heart rate or EEG. For instance, some inventions observe the driver's gaze via an external camera in relation to a primary preview region to issue an alert (W02019144880 and U520190236386) or determine driver distraction based on the gaze direction (JP2010131401, U520160214618) the proportion/duration of the off-road gaze (EP1878604, U520160180677 and CN105711517) or detect eye closure and/or gaze angle as measures of fatigue and distraction (WO/2007/092512). Others derive driver distraction by sampling information via visual or optical sensors at an on-board system of the vehicle (W02018085804) or measure driver's distraction and drowsiness by processing the driver's face and pose as a function of speed and maximum allowed travel distance (U520140139655). CN106205052 presents a method in which distraction is determined via analysing the head and face postures and eye features as well as the vehicle speed and a warning signal is provided in case visual attention is not concentrated. The analysis of eye movements in order to detect distraction is also used in combination with driver's gesture (CN106354259) or head movements (EP2314207) or to extract a feature parameter set based on collected eye motion behavior during a driving main and driving auxiliary task (CN108537161). CN110262084 describe sunglasses that assesses driver distraction via a six-axis accelerometer embedded in the nose bridge based on the head motion of the driver and provides a distraction warning signal based on GPS data. Other approaches describe methods that detect and predict behavioral and cognitive distraction via measuring the eye movements and cardiac health via a heart rate sensor (U520190038204) or assess driver distraction by the characteristics of the EEG frequency spectrum of a driver (CN107334481).

Other inventions that do not use gaze or face direction but focus on information from the vehicle such as the transverse driving speed at fixed time intervals (CN109649399) or a lateral jerk when a vehicle is traveling (US10479371). CN103661375 describes a method in which distraction is detected according to the traveling track of an automobile and the operation information of the driver (i.e., yaw angle of the automobile and the rotating speed signal of a steering wheel). W02017093716 describes a human machine interface that reduces its complexity when driver distraction based on one or several signals calculated from e.g., speed, driver drowsiness, road condition, traffic conditions, such as traffic jams or navigation data, is detected.

[4] explain differences in intra-individual steering wheel control action rates across environmental complexity conditions when drivers are driving under distracted versus non-distracted conditions. In contrast, in the present invention a method is proposed that measures environmental information complexity and establishes a driver's baseline control action variability under non-distracted driving. Significant deviations from this baseline control-action variability indicate evidence for distracted driving. In contrast to previously proposed methods, this method does not utilize the drivers gaze behavior and explicitly takes into account environmental information complexity.

Description of the Invention

The present innovation describes a method to detect driver distraction without optical observation of the drivers' gaze. Rather driver distraction detection is based on observing changes of driver control actions under specific conditions. The method consists of a three-stage process. First, the environmental information complexity is estimated preferably by digital image processing methods enumerating and weighing the driving critical information elements in the driving scene including the road curvature, traffic, traffic signs, pedestrians, etc. that need to be considered by the driver on a per-time basis. Thereby the weights for the driving critical information elements are commensurate to the amount of response required. An element with high information complexity requires multiple actions such as deciding whether a pedestrian is a potential safety threat or identifying the need for a lateral avoidance maneuver. An element with low information complexity requires few driver actions, such as processing the information on a speed limitation sign. Under higher speed, the traffic information complexity increases such that drivers would reduce their speed to keep an acceptable rate of information complexity.

Weights may be defined and shared a priori or online for different types and classes of elements and their level of/contribution to criticality or complexity in traffic conditions (e.g. determined by filed studies, literature, connected vehicle network, onboard scene interpretation, etc.).

Second, a driver's "preferred" control action rate is measured for non-distracted driving under certain levels of environmental information complexity. This is done by recording driving data such as vehicle speed, acceleration and breaking activity or steering wheel angle data. Once this connection between preferred control actions and given environmental information complexity is formed, the value of the mismatch between the actual observed and driver preferred control action rate is taken as evidence for safety critical distracted driving and can be used to trigger warnings or other safety interventions.

Drivers exhibit control over the vehicle through various control actions such as speed control and steering. Thereby, drivers respond to the informational complexity in their environment such as curvature of the road, traffic, pedestrians, road signs and more. Generally, in more complex environments such as in cities, drivers have to process more information than in simpler environments such as on country roads without much traffic or traffic signs. In addition, drivers have their inter-individually different driving styles when perceiving and processing the environmental information but also when controlling the vehicle according to their individual styles. Accordingly, drivers exhibit steering patterns that are characteristic for a given driver across different drives but also differentiate them from other drivers. For example, in experiments it was found that the steering wheel reversal rate of distracted drivers per second increases by about 20% from 0.9 per second to 1.1 [4], see figure 2. Therefore, in summary, the individual control actions during non-distracted driving (CAD) that a person exhibits when driving, are a function of the environmental information complexity (Etc) and the personal driving style for non-distracted driving (PDSND) plus some influences that are here referred to as error (e).

C AND = f (Eic, P DSND) e CAN° is the "preferred" control action rate that a specific driver adopts under non-distracted driving. The observed control action rate for a given environmental complexity and preferred driving style is: CA abs = (Ern, PDs) + e By just observing the control actions, it is not clear yet whether the driver is distracted or not. However, by comparing the observed with the preferred control action rate, evidence for safety relevant distracted driving (EsDoo) is detected when CADDs exceeds CAND under high levels of Eic: EsDDD: CAobs(EIC) > CAN (Ern and PDSND) and Ere = high The environmental information complexity estimator (2) takes information from the outward facing camera (1), which covers the field of view of a human driver in driving direction, and optionally available map information about the road environment and identifies critical information elements for the current driving situation. Critical information elements include the road curvature, traffic, traffic signs, or pedestrians or any other traffic related object. The information elements are subsequently weighted by the amount of response that is required to appropriately respond to them. A pedestrian that is showing ambivalent behavior concerning crossing the street ahead of the vehicle is associated with high information complexity as it requires multiple glances and a subsequent decisions process. Therefore, this element receives a high complexity weight. In contrast, a simple speed limitation sign on the road without any other elements represents low information complexity. Thereby the driving speed represents an important factor such that under higher speed, the traffic information complexity increases and that drivers usually reduce their speed to keep an acceptable rate of information complexity. Therefore Eir can be expressed as: Plc = f (Odt tl Where f(t) is a function of complexity weights over time.

The preferred control action estimator (3) gets input about the current information complexity estimator and the drivers current control action rate from the control action observer (4) and builds an association between them under non-distracted driving conditions. With sufficient evidence, this association is fortified to a preferred control action rate, i.e. a control action rate that the driver would exhibit spontaneously under these EIC conditions.

In a specific implementation of this invention that considers the drivers' steering control action rates, two different types of steering maneuvers (i.e., exploratory and compensatory) can be differentiated depending on their frequency and amplitude. Exploratory movements can be defined as higher frequency steering maneuvers at lower amplitude, compensatory movements as lower frequency steering maneuvers at higher amplitude. These characteristics of both types of steering maneuvers are measured for given EIC under non-distracted driving conditions. During a period of distracted driving a driver's steering maneuvers exhibit a change in the characteristics of the steering movements. This discrepancy between control actions under non-distracted driving and distracted driving is identified by the discrepancy detector (5) which receives input about the current driver control action rate (4) and compares it to the preferred control action rate for the current EIC. A discrepancy is indicative for distracted driving.

The discrepancy detector (5) receives input about the current driver control action rate (4) and compares it to the preferred control action rate for the current EIC. If a discrepancy is detected this is evidence for distracted driving.

While the present invention has been described with respect to the illustrated embodiment, it is recognized that numerous modifications and variations in addition to those mentioned herein will occur to those skilled in the art. Accordingly, it is intended that the invention not be limited to the disclosed embodiment, but that it have the full scope permitted by the language of the following claims.

For a better understanding of the invention figures are given. These figures show: Fig. 1: graphical depiction of the method Fig. 2: Steering Wheel Reversal Rates for Distracted and Non-Distracted Driving (example) References [1] European Community, "Driver Distraction 2018." 2018, [Online]. Available: https://ec.europa. eu/transport/road_safety/sites/roadsafety/files/pdf/ersosynthesis2018-driv erdistraction.pdf.

[2] Y. Liang and J. Lee, "Driver Cognitive Distraction Detection Using Eye Movements," 2007, pp. 285-300.

[3] S. Klauer, T. Dingus, T. Neale, J. Sudweeks, and D. Ramsey, -The Impact of Driver Inattention on Near-Crash/Crash Risk: An Analysis Using the 100-Car Naturalistic Driving Study Data," vol. 594, Jan. 2006.

[4] M. Heller and P. Moertl, "Toward Driver State Models that Explain Interindividual Variability of Distraction for Adaptive Automation," p. 15.

Claims

Claims 1. Method to detect safety relevant driving distraction of a human driver in a road vehicle comprising in a first step to estimate the environmental information complexity by detecting and analysing the type and the number of driving critical information elements in digital images of an outward facing camera 1 and assigning the weighted sum of information complexity values to an environmental information complexity value for each image according to the type and number of detected elements, in a second step to extract the preferred and most likely control action rates for a given driver 3 and for a given environmental information complexity value 2 by processing the recorded input of a driver's control action rates 4 under baseline and non-distracted conditions, in a third step to compare the value of the actually observed control action rates with the value of the preferred and most likely control action rates under non-distracted conditions of the driver and calculate the difference between the two values for control actions 5, in a fourth step to initiate a warning or intervention procedure based on the difference of the two values for control actions, where a low critical warning also comprises advertisement of evidence of detected safety critical distraction to other vehicle systems or the driver and high critical driving interventions.