TW202241742A - Driver monitoring method for monitoring the activity of a vehicle driver and vehicle for executing said method - Google Patents

Abstract

The present invention pertains to a driver monitoring method for monitoring the activity of a vehicle driver comprising the steps of estimating the fatigue level of the driver and/or a time spent waiting by the driver, establishing the driving context of the vehicle and/or the vehicle status, outputting one of a plurality of recommended actions for altering the activity of the driver based on the fatigue level, the time spent waiting, a success rate of the recommended action, the driving context and/or the vehicle status, evaluating the response of the driver to the recommended action and evaluating and updating the success rate of the recommended action. The invention also pertains to a vehicle comprising a controller, input means and output means for executing a corresponding method.

Description

Driver monitoring method for monitoring activity of vehicle driver and vehicle for performing the method

The present invention relates to a driver monitoring method for monitoring the activities of a driver of a vehicle, comprising the steps of: assessing the driver's fatigue level (fatigue level of the driver) and/or the driver's waiting time (time spent waiting by the driver), establishing the vehicle's driving context (driving context) and/or vehicle status (vehicle status), outputting one of a plurality of recommended actions for changing the driver's activities ) Evaluate the driver's response to the recommended action and evaluate and update the success rate of the recommended action based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context and/or the vehicle state. The invention also relates to a vehicle comprising a controller, an input device and an output device for carrying out a corresponding method.

Current methods and corresponding systems for monitoring drivers are designed to detect the activity of a driver of a vehicle so that dangerous or undesired activities can be detected, reacted to, and the driver's sense of safety and well-being (wellbeing) can be improved.

For example, a driver's fatigue or drowsiness, or a situation in which the driver is forced to remain idle and wait for some event, can be detected and reacted to. Clearly, the term activity is to be understood in a broad sense and may mean behavior indicative of a tired driver, such as erratic control behavior of the vehicle or prolonged closure of the driver's eyes. Furthermore, the driver's monitoring behavior may indicate a waiting situation in which the vehicle may be stationary and the driver may not be making any significant control activity related to the vehicle's track.

Once a corresponding activity is detected, a number of predefined responses can be generated to reduce the risk of an accident related to a drowsy driver or to increase the well-being of a waiting driver. However, known methods are uncertain as to whether the predefined responses are actually suitable for the respective driver and achieve the desired results of reducing the amount of driving a drowsy driver or increasing the driver's well-being in idling situations. Sensitive. This represents a problem because known methods are ineffective in reducing the amount of driving and improving the driver's well-being in a drowsy driver, or worse, the driver may actually find that multiple predictors The defined response is a nuisance to her and her and as a result may stop the practice of the method altogether.

It is an object of the present invention to provide an improved method and an improved vehicle which overcome the above-mentioned problems.

This object is achieved by a driver monitoring method according to claim 1 for monitoring the activity of a driver of a vehicle and a vehicle according to claim 10 comprising a controller, an input device for carrying out a corresponding method and an output device. The preferred embodiments of the invention are subject to numerous provisos.

According to claim 1, there is provided a driver monitoring method for monitoring the activities of a driver of a vehicle, the driver monitoring method comprising the steps of: assessing a degree of fatigue of the driver and/or a waiting time of the driver; establishing the driving context and/or vehicle state of the vehicle; outputting one of a plurality of recommended actions for changing the driver's activity based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context and and/or the vehicle status; evaluating the driver's response to the recommended action; and evaluating and updating the success rate of the recommended action.

The term monitoring should be understood in a broad sense, including surveillance of the driver's behavior and suggestions for changing or enhancing the driver's behavior.

The list of recommended actions may include changing the vehicle's settings, such as its interior lighting or seat position. They may include playing some music or initiating some games to be played by the driver. The recommended actions may include suggestions for activities that the driver can perform, such as going for a walk, taking a nap, or doing some kind of meditation. The suggested actions can be output or executed directly, or the driver can be prompted to confirm a suggested action before he actually outputs or executes. In general, the output of a recommended action may include outputting optical and/or visual information and/or changing the physical configuration of the vehicle.

The success rate for a given recommended action may be a function of the ratio of the number of times a driver takes a recommended action to the total number of times suggested actions are taken for a driver.

According to the invention, a specific driver can be identified so that the method can be applied to each unique driver of a given vehicle. The identification of the driver can be done based on some facial recognition steps and corresponding hardware such as a driver monitoring camera.

In the case of a driver using multiple vehicles, the multiple vehicles may be interconnected such that the method may be performed on any vehicle driven by the same driver. Therefore, the method is not limited to a single vehicle.

The presently described method may be continuously performed while the driver is driving the vehicle. Alternatively, execution of the method may be initiated by some predefined signal, such as the occurrence of a potential driving error, such as swerving and/or breaking or crossing some road markings.

The method can be performed in parallel, ie simultaneously and with respect to different suggested actions, so that more than one suggested action can be output to the driver.

According to the invention, dangerous driving is reduced, by effectively combating fatigue, increasing the effectiveness of methods for combating driver drowsiness, and by adapting measures to the unique driver to enhance the user experience. In addition, the driver's overall well-being can be enhanced since the driver can be supported to increase potential wearing waiting periods.

Compared to current driver monitoring systems, this solution integrates many escalated states and offers many countermeasures (such as many modes), firstly, depending on the type of current activity or state of the driver and her level of fatigue, and secondly, The previously established success rate of a proposed solution or recommended action. Based on these, the algorithm can learn and adapt the most effective ways to counteract an individual's fatigue and/or improve her overall well-being. To achieve this, the method may include a driver identification step, in which the identity of the driver is established. Therefore, the method can be adapted to various drivers and their individual characteristics and preferences.

The fatigue level may be some score that correlates to the driver's actual fatigue and may be a fatigue function indicative of the driver's behavior, such as yawning, eye movements, and/or erroneous driving behavior. This behavior can be detected by the presently described method and can be used to assess or calculate the fatigue level.

The present invention provides a context-based recommendation algorithm, wherein if a driver's fatigue condition or a driver's waiting condition or attitude is detected, a vehicle can perform some of the currently described remedies .

In a preferred embodiment of the invention, the driver's fatigue level and/or the driver's waiting time are assessed by a driver monitoring camera and/or by evaluating the lateral and longitudinal vehicle control by the driver .

In another preferred embodiment of the present invention, establishing the driving context and/or vehicle state of the vehicle includes: establishing a current working day, a time period, a driving time, a weather, a road type, a driving history, ADAS responses, a battery level and/or a location of the vehicle.

In another preferred embodiment of the present invention, the plurality of recommended actions include suggesting a refresh mode and/or a rest mode. Numerous further modes may be applied additionally or alternatively. Suggestions for a mode or actions associated with a mode may include changing some vehicle settings, outputting some information, prompting the driver to confirm and initiate the output of some information and/or improving driving safety and increasing the driver's sense of well-being many other method steps. The patterns can be understood as a collection of various recommendable actions, where each pattern can be associated with a particular driver's fatigue level and/or driver activity.

In another preferred embodiment of the present invention, the refresh mode is output when the driver's fatigue level is lower than the rest mode. If the fatigue level exceeds a threshold or fatigue is considered possible due to detection of eg a long driving time on a country road, a corresponding counteracting measure, ie a recommended action, is proposed.

In another preferred embodiment of the present invention, the resting mode includes: suggesting a resting area (a resting area), especially a resting area with a battery charging station; suggesting setting a timer for outputting an alarm , output an alarm based on the state of charge of the battery, play music and/or provide a massage to signal the end of the rest mode to the driver; suggest implementing noise cancellation, suppressing irrelevant messages, setting ambient light and/or temperature.

In another preferred embodiment of the present invention, the refreshing mode includes: suggesting an entertainment activity.

The refresh mode may be output as a recommended action, particularly when the driver monitoring method has assessed that the driver has spent more than a predetermined amount of time waiting and/or not having significant control of the vehicle. This may occur when the driver is stuck in a traffic jam or is waiting for other passengers to board. Alternatively or additionally, the refresh mode may be output when the driver monitoring method has assessed a low level of fatigue.

The output of the recommended actions in the refreshing mode may include outputting many meditation instructions, so that the driver can use the waiting time to increase psychological well-being. In this case, the vehicle can be set to provide a relaxing environment, for example by playing suitable music or nature sounds, providing suitable ambient lighting and/or a massage function. Similarly, instructions for performing light yoga (such as Pranayama) can be output in the refresh mode.

In principle, any type of information that may be considered helpful by the driver can be output in this refresh mode. This may include playing an electronic book, enabling a game such as chess or a card game on the vehicle's computer system, or suggesting a walk in a neighborhood. The driver can accept an appropriate recommended action, and the method ensures that the success rate of the selected recommended action is updated accordingly.

In another preferred embodiment of the present invention, evaluating the driver's response to the recommended actions includes establishing which recommended actions have been accepted by the driver and to what extent.

Therefore, each time the driver accepts the recommended action, the success rate for taking the recommended action can be increased. Accordingly, the driver's response to the recommended action is evaluated. In future situations, the method may suggest some recommended actions more frequently if the success rate of the multiple recommended actions is higher than the success rate of other actions. A recommended action that has been frequently selected or accepted by a driver and improves a driver's fatigue level can be stored and/or rated accordingly and can be suggested as a favorite, for example in future situations More frequently, especially in similar contexts and/or when similar levels of fatigue are detected.

The method may monitor how long a recommended action has been accepted or performed by the driver in order to determine to what extent the recommended action has been accepted.

In another preferred embodiment of the present invention, evaluating the driver's response to the recommended action includes: evaluating the driver's fatigue level after the recommended action has been completed.

The present invention is also directed to a vehicle comprising a controller, an input device and an output device for executing the method according to any one of claims 1 to 9. The vehicle may be designed to be able to perform any of the presently outlined method steps, ie the vehicle may comprise any hardware necessary to perform the presently described method steps.

Thus, the present invention provides the benefits of a context-aware and intelligent method for monitoring a driver, including increased driving safety, improved effectiveness of countermeasures and due to a highly personalized and Improved driver satisfaction through adaptive monitoring methods. In addition, the present invention enhances the perception of an intelligent vehicle.

In one method step shown in FIG. 1 , the context of the vehicle can be checked. This may correspond to establishing a driving context and/or vehicle state for the vehicle. In this step, the type of road the vehicle is on can be retrieved from some accessible map data and/or from vehicle sensors such as a camera capturing the vehicle's surroundings. Further data, such as time of day, traffic density, driver behavior, and/or the regularity of certain driver behaviors, may be obtained from vehicle sensors, from vehicle storage devices such as hard drives, and/or from connections to The vehicle and other sources accessible by the method of the present invention are available, such as the Internet and/or suitable cloud storage sources.

In another method step, the driver's fatigue level and/or the driver's waiting time may be estimated based on data retrieved from a driver monitoring camera and/or from sensors measuring the driving characteristics of the vehicle .

If both the driving context of the vehicle and the current activity of the driver are found to indicate some undesirable situation, such as a drowsy driver driving the car in an unsafe manner, or a driver waiting in the car , while the car is stationary, some recommended actions among multiple recommended actions can be output according to the method. The recommended actions are intended to alter the driver's activities based on the fatigue level, the waiting time, a success rate of the recommended actions, the driving context, and/or the vehicle state.

Initially, the success rate of untried recommended actions may be set to some predetermined value, which may initially be the same for all recommended actions. Once the driver's response to the recommended action is evaluated, the success rate of the recommended action can be evaluated and updated. Therefore, after performing the method multiple times and for various recommended actions, the success rates of the plurality of recommended actions can be automatically updated to reflect their actual effectiveness for a particular driver.

As an example, the method may detect that a drowsy driver is driving the car. This can result, for example, from the fact that the driver closes his eyes more regularly and for a longer period of time than on average and/or from the fact that the driver controls the vehicle incorrectly. If the method establishes that the vehicle is moving, a break may be recommended. Driver drowsiness may be detected based on direct driver behavior such as the driver's eye movements. Alternatively, it may be detected from indirect data, such as when a drowsiness-inducing driving context is expected, for example from regular driving behavior or driving trajectories detected and stored by the method, or when drowsiness-inducing circumstances are actually detected , for example, if the vehicle's current route is a rural road with low traffic density.

Alternatively, the method may detect that the car is stationary and the driver does not have any substantial control of the car, which indicates that the driver may be waiting for some event, such as the arrival of another passenger of the vehicle. In this case, other actions may be performed, such as issuing yoga instructions.

Once the recommended action has been completed or declined by the driver, the driver's activity or status may be checked again, wherein the driver's response to the recommended action is evaluated for its effectiveness. If the given recommended action is found to be effective, for example, the driver accepts the recommended action, and her drowsiness decreases after taking the recommended action, the success rate of the recommended action can be increased from a previous value to a new higher value and vice versa. Therefore, the success of a proposal can be efficiently checked. If no drowsy or waiting driver is found in the vehicle, no proposal can be output according to the method.

In fact, the algorithms included in the present invention are capable of retrieving information about the state of the driver and the current context of the vehicle. Based on the retrieved data, the algorithms recommend appropriate means or actions for improving the driver's current state and evaluate the success of the recommended actions.

If a similar situation is detected on another occasion, ie a similar driver state and vehicle environment, a previously successful action can be recommended again. The similarity of situations may mean a similar travel route of the vehicle and/or some routine or recurring event, such as repeating similar waiting times for more passengers to come to the vehicle at certain pick-up locations. The algorithms can be designed to iteratively learn a driver's personal preferences and the efficiency of given suggested actions.

The present invention addresses two important issues regarding vehicles driven by multiple drivers.

On the one hand, studies have shown that many drivers spend a significant amount of time (in some cases averaging about 75 minutes per week) behind the wheel waiting, for example, when picking up children from school, When waiting for other people during an event such as a medical appointment. Especially during epidemics, it is better to wait in the vehicle, eg for a doctor's appointment, for disease testing, and the like. The driver may use this waiting time for things like reading, engaging in some form of Internet activity such as using YouTube or Facebook, listening to music, planning a meal, and the like. However, few drivers use this time to actually improve their well-being.

On the other hand, a considerable number of vehicle accidents are caused by drowsy drivers who continue to control the vehicle far beyond a safe state to operate any machinery.

The present invention provides a method for simultaneously detecting drowsiness and waiting period of a driver, so that corresponding problems can be improved and the condition of the driver can be improved.

In a practical example, the method can detect a driver's fatigue and can recommend switching to the refresh mode, including a number of possible suggested actions associated with the refresh mode. The driver can accept the suggestion and the vehicle can take appropriate action. After a given amount of time, such as after 15 minutes, the driver's fatigue level may be assessed again, and still an elevated fatigue level may still be detected. In such cases, it may be found that the suggested action was not successful. As a result, the method can output a number of suggested actions associated with the rest mode. The driver may be prompted to confirm the suggested choice. In case the driver accepts the suggestion, the method can indicate the next possible rest area on a navigation device so that the driver can select a suitable rest area. The method may take into account the state of charge of a vehicle battery when suggesting a rest area. If the state of charge is below a certain given value (eg, 60%), the method may propose an area to charge the vehicle. Preferably, a rest area that can be reached within a given time (eg 15 minutes) can be suggested. Once the area has been reached, the method may advise the driver to set a timer or consider a wake-up timer to complete charging. During breaks, the method may provide noise cancellation and irrelevant message suppression in addition to setting the vehicle to conditions including pleasant ambient light, suitable temperature, etc. Once the timer has expired, the method slowly wakes the driver up by, for example, playing quiet music and/or providing a massage. Then, the method can recheck the driver's fatigue level and update the success rate of the recommended action accordingly. Over time, the method's algorithm learns from a driver's reaction to recommendations from a number of outputs, i.e. whether the driver accepts or rejects the recommendations, from the current context (i.e. weekday, time of day, weather, Driving time, etc., and from the success of the proposed mode, namely the reassessed fatigue level, to establish the driver's preferences and characteristics. Therefore, many future proposals can be improved, and the driver can experience to a highly personalized and efficient vehicle.

Thus, the present invention provides the benefits of a context-aware and intelligent method for monitoring a driver, including increased driving safety, improved effectiveness of countermeasures, and due to a highly personalized and Improved driver satisfaction with adaptive monitoring methods. In addition, the present invention enhances the perception of an intelligent vehicle.

The invention is not limited to the presently described embodiments but can be varied in different ways. All features defined in the claims, the description and the drawings including structural details and spatial arrangements may be relevant to the invention.

(none)

Further features, details and advantages of the invention are disclosed in the claims, in the following description of the drawings and in the drawings themselves. The diagram shows: [FIG. 1]: A flowchart of some steps of the method according to the present invention.

Claims (10)

A driver monitoring method for monitoring the activities of a driver of a vehicle, the driver monitoring method comprising the steps of: Assess the driver's fatigue level and/or the driver's waiting time; establish the driving context and/or vehicle status of the vehicle; outputting one of a plurality of recommended actions for changing the driver's activity based on the fatigue level, the waiting time, a success rate of the recommended action, the driving context and/or the vehicle state; assess the driver's response to the recommended action; and The success rate of the recommended action is evaluated and updated. The driver monitoring method according to claim 1, wherein the driver's fatigue level and/or the driver's fatigue level is evaluated by a driver monitoring camera and/or by evaluating the lateral and longitudinal vehicle control performed by the driver waiting time. According to the driver monitoring method described in any one of the preceding claims, wherein establishing the driving context and/or vehicle status of the vehicle includes: establishing a current working day, a time period of a day, a driving time, a weather, a road type, A driving history, ADAS responses, a battery level and/or a location of the vehicle. The driver monitoring method according to any one of the preceding claims, wherein the plurality of recommended actions include suggesting a refresh mode and/or a rest mode. The driver monitoring method according to claim 4, wherein the refresh mode is output when the driver's fatigue level is lower than the rest mode. According to the driver monitoring method described in claim 4 or 5, wherein the rest mode includes: Suggest a rest area, especially one with a battery charging station; It is recommended to set a timer for outputting an alarm, output an alarm based on the state of charge of the battery, play music and/or provide a massage to signal the end of the rest mode to the driver; It is recommended to perform noise cancellation, suppress extraneous messages, set ambient light and/or temperature. The driver monitoring method according to any one of claims 4 to 6, wherein the refreshing mode includes: suggesting an entertainment activity. A driver monitoring method according to any one of the preceding claims, wherein evaluating the driver's response to the recommended actions includes establishing which recommended actions have been accepted by the driver and to what extent. The driver monitoring method according to any one of the preceding claims, wherein assessing the driver's response to the recommended action comprises assessing the driver's fatigue level after the recommended action has been completed. A vehicle, comprising a controller, an input device and an output device, for executing the method according to any one of claims 1 to 9.

Images