CN116829415A

CN116829415A - Driver monitoring method for monitoring activities of a driver of a vehicle and vehicle for carrying out said method

Info

Publication number: CN116829415A
Application number: CN202080108236.0A
Authority: CN
Inventors: 安娜·佩佐德; 比拉尔·科翰
Original assignee: Wuhan Lotus Cars Co Ltd
Current assignee: Wuhan Lotus Cars Co Ltd
Priority date: 2020-12-29
Filing date: 2020-12-29
Publication date: 2023-09-29
Also published as: TWI818395B; TW202241742A; EP4271591A4; WO2022141041A1; EP4271591A1

Abstract

The present disclosure relates to a driver monitoring method for monitoring an activity of a vehicle driver, comprising the steps of: estimating the fatigue degree of the driver and/or the waiting time of the driver, and establishing the driving context and/or the vehicle state of the vehicle; outputting one of a plurality of recommended actions for changing the activity of the driver based on the degree of fatigue, the waiting time, a power of the recommended action, the driving context, and/or the vehicle state; estimating a response of the driver to the recommended action; and estimating and updating the success rate of the recommended action. The disclosure also relates to a vehicle comprising a controller for executing a corresponding method, an input device and an output device.

Description

Driver monitoring method for monitoring activities of a driver of a vehicle and vehicle for carrying out said method

Technical Field

The invention relates to a driver monitoring method for monitoring the activity of a vehicle driver, comprising the steps of: estimating a fatigue level (fatigue level of the driver) of the driver and/or a waiting time (time spent waiting by the driver) of the driver, establishing a driving context (driving context) and/or a vehicle status (vehicle status) of the vehicle, outputting one of a plurality of recommended actions (one of a plurality of recommended actions) for changing an activity of the driver based on the fatigue level, the waiting time, a power of the recommended action, the driving context and/or the vehicle status, estimating a response of the driver to the recommended action, and estimating and updating a success rate of the recommended action. The invention also relates to a vehicle comprising a controller for executing a corresponding method, an input device and an output device.

Background

The current methods and corresponding systems for monitoring a driver are designed to detect the activity of a vehicle driver so that dangerous or undesirable activity can be detected and reacted to so that the driver's sense of safety (safety) and well-being (wellbearing) can be improved.

For example, a driver's fatigue or drowsiness or a situation in which the driver is forced to remain idle and wait for certain events may be detected and reacted to. Obviously, the term activity should be understood in a broad sense and may be meant to indicate a tired driver's behavior, such as unstable control trails of the vehicle or too long closure of the driver's eyes. Further, the driver's monitoring behavior may indicate a waiting situation where the vehicle may be stationary and the driver may not be engaged in any significant control activities related to the vehicle's track.

Once a corresponding activity is detected, predefined responses may be generated to reduce the risk of drowsy driver-related accidents or to increase the well-being of a waiting driver. However, the known methods are insensitive to whether the predefined reactions are truly appropriate for the respective driver and achieve the desired results of reducing the driving volume of the drowsy driver or increasing the driver's well-being in idle conditions. This represents a problem, because the known method is ineffective in reducing the driving load of the drowsy driver and improving his well-being, or worse, the driver may actually find the plurality of predefined reactions to be a nuisance to her and as a result may stop the execution of the method altogether.

Disclosure of Invention

It is an object of the present invention to provide an improved method and an improved vehicle which overcome the above-mentioned problems.

This object is achieved by a driver monitoring method for monitoring an activity of a vehicle driver according to claim 1 and a vehicle according to claim 10, comprising a controller, an input device and an output device for performing a corresponding method. Preferred embodiments of the present invention suffer from a number of appurtenances.

According to claim 1, there is provided a driver monitoring method for monitoring an activity of a vehicle driver, the driver monitoring method comprising the steps of:

estimating a degree of fatigue of the driver and/or a waiting time of the driver;

establishing a driving context and/or a vehicle state of the vehicle;

outputting one of a plurality of recommended actions for changing the activity of the driver based on the degree of fatigue, the waiting time, a power of the recommended action, the driving context, and/or the vehicle state;

estimating a response of the driver to the recommended action; and

The success rate of the recommended action is estimated and updated.

The term monitoring should be understood in a broad sense, including monitoring of the driver's behaviour (monitoring) and advice to change or enhance the driver's behaviour (monitoring).

The recommended actions may include changing a setting of the vehicle, such as its interior lighting or seat position. They may include playing certain music or initiating some games to be played by the driver. The recommended actions may include suggestions of activities that the driver may perform, such as walking away, going to rest, or performing some sort of meditation (medium). The recommended actions may be directly output or performed, or the driver may be prompted to confirm a recommended action (a suggested action) before he actually outputs or performs. Generally, the output of a recommended action (a recommended action) may include outputting optical and/or visual information and/or changing the physical configuration of the vehicle.

The success rate of a given recommended action may be a function of the ratio of the number of times a driver accepts a recommended action to the total number of actions recommended to a driver.

According to the present invention, a particular driver may be identified such that the method may be adapted to each unique driver of a given vehicle. The driver's recognition may be based on facial recognition steps and corresponding hardware such as a driver monitoring camera.

In the case of a driver using a plurality of vehicles, the plurality of vehicles may be interconnected such that the method may be performed on any vehicle driven by the same driver. Thus, the method is not limited to a single vehicle.

The presently described method may be continuously performed while the driver is driving the vehicle. Alternatively, the execution of the method may be initiated by some predefined signal, such as the occurrence of a potential driving error, such as abrupt steering and/or breaking or crossing some road markings.

The method may be performed in parallel, i.e. simultaneously and with respect to different recommended actions, such that more than one recommended action may be output to the driver.

According to the present invention, dangerous driving is reduced, the effectiveness of the method of combating fatigue effectively, improving drowsiness of the driver, and improving the user experience by adapting a number of measures to the unique driver. In addition, the overall well-being of the driver may be enhanced because the driver may be supported to promote a potential fatigue waiting period (wearing waiting periods).

In contrast to current driver monitoring systems, this solution integrates upgrade states and provides countermeasures (such as modes), firstly, depending on the type of current activity or state of the driver and her fatigue level, and secondly, a previously established success rate (previously established success rate of a proposed solution or recommended action) of proposed solutions or recommended actions. Based on these, the algorithm may learn and adapt to the most efficient methods to counteract the fatigue of an individual and/or improve her overall well-being. To achieve this, the method may comprise a driver identification step, wherein the identity of the driver is established. Thus, the method may be adapted to a variety of drivers and their personal characteristics and preferences.

The degree of fatigue may be some fraction that is associated with the actual fatigue of the driver and may be a fatigue function that is indicative of the driver's behavior, such as yawning, eye movement, and/or false driving behavior. This behavior can be detected by the presently described method and can be used to estimate or calculate the degree of fatigue.

The present invention provides a context-based recommendation algorithm in which a vehicle may take some of the currently described remedial actions (remedies) if a driver's fatigue condition or a driver's waiting condition or pattern is detected.

In a preferred embodiment of the invention, the driver's fatigue level and/or driver's waiting time is estimated by a driver monitoring camera and/or by estimating lateral and longitudinal vehicle control by the driver.

In another preferred embodiment of the invention, establishing the driving context and/or vehicle status of the vehicle comprises: a current workday, an inter-day period, a driving time, a weather, a road type, a driving history, a plurality of advanced driver assistance system responses, a battery level, and/or a location of the vehicle is established.

In another preferred embodiment of the present invention, the plurality of recommended actions includes suggesting a refresh mode and/or a rest mode. Many other modes may be applied in addition or instead. Suggestions of a mode or actions associated with a mode may include changing some vehicle settings, outputting some information, prompting the driver to confirm and initiate the output of some information, and/or other method steps that can improve driving safety and increase the driver's well-being. The patterns may be understood as a collection of various recommended actions, where each pattern may be associated with a particular driver's fatigue and/or driver activity.

In another preferred embodiment of the invention, the refresh mode is output when the driver's fatigue level is lower than the rest mode. If the fatigue level exceeds a threshold or fatigue is deemed to be possible due to detection of a long driving time, e.g. on a rural road, a corresponding countermeasures (a corresponding counteracting measure), i.e. a recommended action, is proposed.

In another preferred embodiment of the present invention, the rest mode comprises:

suggesting a rest area (a rest area), in particular a rest area with a battery charging station;

suggesting a timer for outputting an alert, outputting an alert based on the state of charge of the battery, playing music and/or providing a massage to signal the driver to the end of the rest mode;

it is suggested to perform noise cancellation, suppress extraneous messages, set ambient light and/or temperature.

In another preferred embodiment of the present invention, the refresh mode includes:

an entertainment event is suggested.

The refresh mode may be output as a recommended action, particularly when the driver monitoring method has estimated that the driver has spent more than a predetermined amount of time waiting and/or has not significantly controlled the vehicle. This may occur when the driver is involved in a traffic jam or is waiting for other passengers to get on. Alternatively or additionally, the refresh mode may be output when the driver monitoring method has estimated a low level of fatigue.

The outputting of recommended actions in the refresh mode may include outputting a number of meditation instructions so that the driver may utilize the waiting time to increase mental well-being. In this case, the vehicle may be configured to provide a relaxed environment, for example by playing suitable music or natural sounds, providing suitable ambient lighting and/or a massaging function. Similarly, guidance to perform a light yoga (e.g., a breath, pranayama) may be output in the refresh mode.

In principle, any type of information that may be deemed helpful by the driver may be output in the refresh mode. This may include playing an electronic book, enabling a game such as a game player or a card game on a computer system of the vehicle or suggesting to walk in a nearby area. The driver may accept an appropriate recommended action and the method ensures that the success rate of the selected recommended action is updated accordingly.

In another preferred embodiment of the present invention, estimating the driver's response to the recommended action includes: it is established which recommended actions have been accepted by the driver and to what extent.

Thus, each time the driver accepts the recommended action, the success rate of accepting the recommended action may be increased. Thus, the driver's response to the recommended action is estimated. In future situations, the method may suggest some recommended actions more frequently if the success rate of the plurality of recommended actions is higher than the success rate of other actions. A recommended action that has been frequently selected or accepted by a driver and improves the fatigue level of a driver may be stored and/or rated accordingly and may be suggested as a favorite, for example more frequently in future situations, especially in similar context and/or when similar fatigue levels are detected.

The method may monitor how long a recommended action has been accepted or performed by the driver to determine how much of the recommended action has been accepted.

In another preferred embodiment of the present invention, estimating the driver's response to the recommended action includes: after the recommended action has been completed, the driver's fatigue level is estimated.

The invention is also directed to a vehicle comprising a controller, an input device and an output device for performing the method according to any of claims 1 to 9. The vehicle may be designed to be able to perform any of the presently outlined method steps, i.e. the vehicle may comprise any hardware necessary to perform the presently described method steps.

Accordingly, the present invention provides benefits of a context-aware and intelligent method for monitoring a driver, including improved driving safety, improved effectiveness of countermeasure methods, and improved driver satisfaction due to a highly personalized and adaptable monitoring method. In addition, the invention enhances the perceptibility of an intelligent vehicle.

Drawings

Other features, details, and advantages of the invention are disclosed in the claims, in the following description of the drawings, and in the drawings themselves. The drawings show:

fig. 1 is a flow chart of some steps of a method according to the invention.

Detailed Description

In one method step shown in fig. 1, the context of the vehicle may be checked. This may correspond to establishing a driving context and/or a vehicle status of the vehicle. In this step, the road type on which the vehicle is located may be retrieved from some accessible map data and/or from a number of vehicle sensors, such as a camera capturing the surroundings of the vehicle. Further data, such as time of day, traffic density, driver behavior and/or regularity of certain driver behaviors, may be obtained from vehicle sensors, from vehicle storage devices such as hard drives and/or from other sources connected to the vehicle and accessible to the method of the invention, such as the internet and/or suitable cloud storage sources.

In a further method step, the fatigue level of the driver and/or the waiting time of the driver can be estimated from data retrieved from a driver monitoring camera and/or from sensors measuring the driving characteristics of the vehicle.

If it is found that the driving context of the vehicle and the current activity of the driver are indicative of some undesired situation, such as a drowsy driver driving the car in an unsafe manner, or a driver waiting in the car while the car is stationary, some of a plurality of recommended actions may be output according to the method. The plurality of recommended actions aim to change the activity of the driver based on the degree of fatigue, the waiting time, a power of the recommended actions, the driving context, and/or the vehicle state.

Initially, the success rate of the untried recommended action may be set to some predetermined value, which may initially be the same for all recommended actions. Once the driver's response to the recommended action is estimated, the success rate of the recommended action may be estimated and updated. Thus, after the method is performed multiple times and for various recommended actions, the success rates of the multiple recommended actions may be automatically updated to reflect their actual effectiveness for a particular driver.

As an example, the method may detect that a drowsy driver is driving the vehicle. This may result, for example, from the driver closing his eyes more regularly and for a longer period of time than the average value and/or from the driver performing a false control of the vehicle. If the method establishes that the vehicle is traveling, a rest may be recommended. Drowsiness of the driver may be detected based on direct driver behavior such as eye movements of the driver. Alternatively, it may be detected from indirect information, such as when a driving context causing drowsiness is expected, for example from regular driving behaviors or driving trajectories detected and stored by the method, or when the environment causing drowsiness is actually detected, for example, if the current route of the vehicle is a rural road with a low traffic density.

Alternatively, the method may detect that the car is stationary and the driver does not have any substantial control over the car, which means that the driver may be waiting for some event, such as the arrival of another passenger of the vehicle. In this case, other actions may be performed, such as issuing yoga instructions.

Once the recommended action has been completed or rejected by the driver, the driver's activity or status may be checked again, wherein the driver's response to the recommended action is estimated for its effectiveness. If the given recommended action is found to be valid, such as the driver accepting the recommended action and her drowsiness is intended to decrease after the recommended action, the success rate of the recommended action may be increased from a previous value to a new higher value and vice versa. Thus, the success of a proposal (proposal) can be effectively checked. If no drowsy or waiting driver is found in the vehicle, no proposal may be output according to the method.

In practice, the present invention includes algorithms that are capable of retrieving information about the status of the driver and the current context of the vehicle. Based on the retrieved data, the algorithms recommend appropriate means or actions for improving the current state of the driver and evaluating the success or failure of the plurality of recommended actions.

If a similar situation is detected on another occasion, i.e. similar driver status and vehicle environment, a previously successful action may be recommended again. The similarity of the various conditions may mean a similar route of travel of the vehicle and/or some conventional phenomenon or repetitive event, such as repeating similar waiting times to allow more passengers to get to the vehicle at certain boarding points. The algorithms may be designed to iteratively learn the personal preferences of a driver and the efficiency of a given suggested action.

The present invention addresses two important issues with respect to vehicles driven by drivers.

In one aspect, studies have shown that many drivers spend a significant amount of time waiting behind the steering wheel (in some cases about 75 minutes a week on average), such as when receiving a child from school, when taking a child to a club, when waiting for others during an activity such as shopping or a doctor's consultation. Especially during epidemics, it is preferable to wait in the vehicle, for example, a doctor's consultation, disease detection, and the like. The driver may use the wait time for, for example, reading, participating in some form of internet activity, such as using YouTube or Facebook, listening to music, planning a meal, etc. However, few drivers take advantage of this time to actually improve their well-being.

On the other hand, a significant number of vehicle accidents are caused by drowsy drivers who continue to control the vehicle far beyond a safe state of operating any machine.

The present invention provides a method for simultaneously detecting drowsiness and waiting period of a driver so that a plurality of corresponding problems can be improved and the condition of the driver can be improved.

In a practical example, the method may detect fatigue of a driver and may suggest a switch to the refresh mode, including a plurality of possible suggested actions associated with the refresh mode. The driver may accept the advice and the vehicle may take appropriate action. After a given amount of time, for example after 15 minutes, the fatigue level of the driver can be estimated again and still an elevated fatigue level can still be detected. In this case, it may be found that the suggested action was unsuccessful. As a result, the method may output suggested actions associated with the rest mode. The driver may be prompted to confirm the suggested selection. In the event that the driver accepts the advice, the method may indicate the next possible rest area on a navigation device so that the driver may select an appropriate rest area. The method may consider the state of charge of a vehicle battery when suggesting a rest area. If the state of charge is below a given value (e.g., 60%), the method may suggest an area to charge the vehicle. Preferably, a rest area that can be reached within a given time (e.g. 15 minutes) can be suggested. Once the zone has been reached, the method may suggest that the driver set a timer or consider completing charging with a wakeup timer. During rest, the method may provide noise cancellation (noise cancellation) and extraneous message suppression (irrelevant message suppression) in addition to setting the vehicle to include pleasant ambient light, proper temperature, etc. Once the timer has expired, the method slowly wakes up the driver, such as by playing quiet music and/or providing a massage. The method may then check again the driver's fatigue level and update the success rate of the recommended action accordingly. Over time, the algorithm of the method establishes the driver's preferences and characteristics from the driver's reaction to outgoing recommendations, i.e., whether the driver accepts or rejects the recommendations, from the current context (i.e., workday, daytime period, weather, driving time, etc.), and from the success or failure of the proposed mode, i.e., the re-estimated fatigue level.

The invention is not limited to the embodiments described presently, but can be varied in different ways. All the features defined in the claims, in the description and in the drawings including structural details and spatial arrangements may be relevant to the invention.

Claims

1. A driver monitoring method for monitoring an activity of a vehicle driver, characterized by: the driver monitoring method includes the steps of:

establishing a driving context and/or a vehicle state of the vehicle;

estimating a response of the driver to the recommended action; and

The success rate of the recommended action is estimated and updated.

2. The driver monitoring method according to claim 1, characterized in that: the driver's fatigue level and/or driver waiting time is estimated by a driver monitoring camera and/or by estimating lateral and longitudinal vehicle control by the driver.

3. A driver monitoring method according to any of the preceding claims, characterized in that: establishing the driving context and/or vehicle status of the vehicle includes: a current workday, an inter-day period, a driving time, a weather, a road type, a driving history, a plurality of advanced driver assistance system responses, a battery level, and/or a location of the vehicle is established.

4. A driver monitoring method according to any of the preceding claims, characterized in that: the plurality of recommended actions includes suggesting a refresh mode and/or a rest mode.

5. The driver monitoring method according to claim 4, characterized in that: the refresh mode is output when the driver's fatigue level is lower than the rest mode.

6. The driver monitoring method according to claim 4 or 5, characterized in that: the rest mode includes:

suggesting a rest area, in particular a rest area provided with a battery charging station;

7. The driver monitoring method according to any one of claims 4 to 6, characterized in that: the refresh mode includes: an entertainment event is suggested.

8. A driver monitoring method according to any of the preceding claims, characterized in that: estimating the driver's response to the recommended action includes: it is established which recommended actions have been accepted by the driver and to what extent.

9. A driver monitoring method according to any of the preceding claims, characterized in that: estimating the driver's response to the recommended action includes estimating a degree of fatigue of the driver after the recommended action has been completed.

10. A vehicle comprising a controller, an input device and an output device for performing the method according to any one of claims 1 to 9.