JP2023072271A - Driver state determination method and device - Google Patents

Abstract

To provide a driver state determination method and a device that can determine an abnormal state of a driver with high accuracy.SOLUTION: A driver state determination method includes an open or closed eye determination step S2, S3 of detecting an open or closed state of both left and right eyes of a driver, a driving operation amount acquisition step S1 of acquiring at least one of amounts of driving operation for an operation device of a vehicle performed by the driver at predetermined timing, and a driver abnormal state determination step S5, S10, S11 of determining whether the driver is in an abnormal state. The driver abnormal state determination step S10, S11 determines that the driver is in the abnormal state when both the left and right eyes of the driver are in a closed eye state and the driver's non-driving operation state continues for a first predetermined time T1, and determines that the driver is in the abnormal state when one eye of the driver is in the closed eye state and specific operation to an extent not hindering the driving is not detected within a second predetermined time T2 despite a request for the specific operation to the driver.SELECTED DRAWING: Figure 5

Description

The present invention relates to a driver condition determination method and device for detecting an abnormal condition of a driver driving a vehicle.

2. Description of the Related Art Conventionally, there has been known a driver assistance device that accurately grasps the state of consciousness of a driver, particularly the arousal state of the driver, and assists the driving operation of a vehicle according to the arousal state.
A driver's abnormal state including drowsiness level and poor wakefulness is usually determined through head-related information such as the driver's head posture and face state obtained by imaging means such as a camera.
Among the head-related information, the head posture is detected via, for example, the driver's face orientation and head position (three-dimensional coordinates). (eyelid state) and the like.

The driver monitoring device of Patent Document 1 includes captured image acquisition means for acquiring a captured image of a driver, recognition stability determination means for determining recognition stability by the captured image acquisition means, and attention alerting to the driver. and an action requesting means for requesting the driver to perform a predetermined action. The recognition stability determining means activates the attention calling means when the recognition stability is high and the degree of eye opening is low, and the recognition stability is improved. When it is low, the driver is requested to perform a non-image motion to detect the motion without relying on image processing, and the arousal level of the driver is determined based on the presence or absence of this non-image motion.

JP 2015-095162 A

In the driver monitoring device of Patent Document 1, when the imaging conditions such as the in-vehicle imaging environment in which the driver is imaged and the operation performance of the captured image acquisition means are not satisfied, the driving is performed based on the non-image operation by the driver. It is possible to determine the abnormal state (defective state of alertness) of a person.
However, in the driver monitoring device of Patent Document 1, there is a possibility that the abnormal state of the driver cannot be determined even when the image capturing condition is satisfied.

The technique disclosed in Patent Literature 1 determines whether the driver is in an abnormal state based on the driver's degree of eye opening when the imaging conditions are satisfied in imaging the driver. Therefore, when an image of the driver with one eye closed is captured by the captured image acquisition means, the other eye is open, so it is determined that the other eye is open.
However, when a driver develops facial nerve paralysis such as blepharospasm caused by a brain disease (cerebral infarction, stroke, etc.), a state in which one eye is open and the other eye is closed may occur.

On the other hand, when there is high illumination in front of the vehicle in the direction of travel, such as during the daytime in summer, or when dust or the like enters the eyes of the driver, even if the driver is well awake, only one eye is closed. occurs.
Therefore, even if the one-eye-closed state of the driver is captured by the imaging means and accurately detected, the acquisition of the detected one-eye-closed state is unambiguously regarded as a symptom of the driver's disease. can't That is, it is not easy to determine the abnormal state of the driver with high accuracy.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a driver's condition determination method, apparatus, etc., capable of determining an abnormal condition of a driver with high accuracy.

According to a first aspect of the present invention, there is provided a driver state determination method for detecting an abnormal state of a driver of a vehicle, comprising: an open/closed eye determination step of detecting whether the left and right eyes of the driver are open or closed; a driving operation amount acquiring step of acquiring at least one amount of driving operation by a driver with respect to the operating device of the vehicle; and a driver abnormal state determining step of determining whether the driver is in an abnormal state, The driver's abnormal state determination step determines that the driver is in an abnormal state when the driver's left and right eyes are closed and the driver's non-operating state continues for a first predetermined time. When the driver is in an abnormal state when one eye of the driver is closed and the specific operation is not detected within the second predetermined time even if the driver is requested to perform a specific operation that does not interfere with driving. It is characterized by determining that there is

In this driver state determination method, at least one of an open/closed eye determination step of detecting whether the left and right eyes of the driver are open or closed, and a driving operation amount of acquiring at least one of a driving operation amount of the operation device of the vehicle by the driver at a predetermined timing. Since the operation amount acquisition step and the driver abnormal state determination step for determining whether the driver is in an abnormal state are included, the driver's symptoms in an abnormal state (hereinafter referred to as abnormal symptoms) such as closed eyes and an abnormal state are included. A driver's operation (hereinafter referred to as an abnormal operation), that is, no driving operation, can be used as a condition for determining whether the driver is in an abnormal state.
The driver's abnormal state determination step is for determining that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver's non-operating state continues for a first predetermined time. , the abnormal state of the driver can be determined with high accuracy by the two abnormality determination conditions of the abnormal symptom and the abnormal operation. In the driver's abnormal state determination step, the specific operation is detected within a second predetermined time even if the driver is in a state where one eye is closed and the driver is requested to perform a specific operation that does not hinder driving. If not, it is determined that the driver is in an abnormal state. Therefore, it is possible to reliably determine the abnormal state of the driver with one eye closed based on the actual operation of the driver without hindering the driving of the vehicle. .

The invention according to claim 2 is characterized in that, in the invention according to claim 1, the driver's abnormal state determination step provides notification when both the left and right eyes of the driver are closed and when one eye is closed. there is
According to this configuration, it is possible to more reliably determine whether the driver is in an abnormal state in a situation where both the left and right eyes are not open.

The invention of claim 3 is the invention of claim 1 or 2, further comprising a head posture acquiring step of acquiring the face orientation and the head position of the driver, and the driver abnormal state determining step includes: The driver's head posture acquired in the posture acquisition step is an abnormal posture, the driver has one eye closed, and the driver has not operated the vehicle for a third predetermined time period shorter than the first predetermined time period. It is characterized by determining that the driver is in an abnormal state when the condition continues for a long period of time.
According to this configuration, the abnormal state of the driver can be reliably and quickly determined using the abnormal posture as a parameter.

According to a fourth aspect of the present invention, there is provided a driver state determination device for detecting an abnormal state of a driver of a vehicle, wherein an open/closed eye determination means detects whether the left and right eyes of the driver are open or closed; Driving operation amount acquisition means for acquiring at least one amount of driving operation of the operation device of the vehicle by the driver; and driver abnormal state determination means for determining whether the driver is in an abnormal state, The driver's abnormal state determination means determines that the driver is in an abnormal state when the left and right eyes of the driver are closed and the driver's non-operating state continues for a first predetermined time. When the driver is in an abnormal state when one eye of the driver is closed and the specific operation is not detected within the second predetermined time even if the driver is requested to perform a specific operation that does not interfere with driving. It is characterized by determining that there is

In this driver state determination device, an open/closed eye determining means for detecting whether the left and right eyes of the driver are open or closed; Since the operation amount acquisition means and the driver abnormality determination means for determining whether or not the driver is in an abnormal state are provided, it is possible to detect whether the driver is abnormally closing his/her eyes, which is an abnormality symptom, and not driving, which is an abnormal operation. It can be used as a condition for judging the state.
The driver's abnormal condition determination means determines that the driver is in an abnormal condition when the driver's left and right eyes are closed and the driver's non-operating state continues for a first predetermined time. , the abnormal state of the driver can be determined with high accuracy by the two abnormality determination conditions of the abnormal symptom and the abnormal action. The driver's abnormal state determination means detects the specific operation within a second predetermined time even if the driver is in a closed eye state and the specific operation is requested to the extent that the driver does not hinder driving. If not, it is determined that the driver is in an abnormal state. Therefore, it is possible to reliably determine the abnormal state of the driver with one eye closed without hindering the driving of the vehicle.

According to the driver condition determination method and apparatus of the present invention, the abnormal condition of the driver can be determined with high accuracy by determining the symptoms of the driver's disease when one eye is closed.

1 is a block diagram of a driver's state determination device according to Embodiment 1; FIG. It is an explanatory view concerning a monitoring support device. It is an explanatory view concerning a braking assistance device. It is explanatory drawing which concerns on a stop assistance apparatus. 7 is a flowchart showing closed-eye determination control processing. 9 is a flowchart showing head posture determination control processing;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings.
The following description exemplifies the application of the present invention to a vehicle driver state determination device, and does not limit the present invention, its application, or its use.

Embodiment 1 of the present invention will be described below with reference to FIGS. 1 to 6. FIG.
The driver state determination device M determines the state of consciousness of the driver, in particular, the abnormal state in which the driving operation of the vehicle driven by the driver (hereinafter also referred to as own vehicle) is difficult, and determines the abnormal state of the driver. It is a driving support device that can autonomously make an emergency stop.
Hereinafter, the abnormal state of the driver will be described as a state of poor wakefulness (including the onset of a disease and falling asleep) in which the driver is not consciously awake and is physically disoriented and unable to drive. .
Also, the following description includes the description of the driver state determination method.

As shown in FIG. 1, the driver state determination device M includes a plurality of detection means 1 capable of detecting the operation behavior of the driver, the running behavior of the vehicle, etc. A driver assistance device 2 that assists driving and an ECU (Electronic Control Unit) 3 that performs various calculations based on input signals input from the detection means 1 and outputs various command signals to the driver assistance device 2, etc. It is a component.

First, the detection means 1 will be explained.
As shown in FIG. 1, the detection means 1 includes a camera 11 that captures an image of the driver, a vehicle speed sensor 12 that detects the speed of the vehicle, and a steering angle sensor 13 that detects the steering angle by operating a steering wheel (not shown). , a gripping sensor 14 for detecting gripping of the steering wheel by the driver, an accelerator sensor 15 for detecting the amount of depression of an accelerator pedal (not shown), and a brake sensor 16 for detecting the amount of depression of a brake pedal (not shown). , and a hazard switch 17 for operating a hazard lamp (not shown). Each of the sensors 12 to 16 corresponds to a driving operation amount acquiring means for acquiring the driving operation amount at a predetermined timing, that is, just before the so-called determination time.

The camera 11 is composed of, for example, a CCD (Charge Coupled Device) camera, and is mounted at a position facing the driver in the upper center of an instrument panel (not shown).
The camera 11 is formed so as to be able to image the head including the face of the driver seated in the driver's seat and wearing the seat belt from the front. The image of the driver's head is extracted from the image of the driver's face, and the orientation direction of the face (face direction), the facial expression of the driver is identified, and the eye iris (iris) is enlarged and detected to determine the line-of-sight direction. , the eyelid portion is enlarged and detected to detect the opening degree and open/closed state of the left and right eyes of the driver, the three-dimensional coordinate position of the driver's head (center of gravity), and the like.
The camera 11 is sufficient as long as it can acquire at least a necessary captured image, and may be equipped with two cameras, a front camera and an upper camera installed directly above the driver's seat, or three or more cameras.

Next, the driver assistance device 2 will be described.
As shown in FIG. 1, the driver support device 2 includes a monitoring support device 21 that notifies the driver of a warning state, a braking support device 22 that performs a braking operation on the vehicle, and an emergency warning device for the vehicle. It is composed of a stop support device 23 and the like that cause the vehicle to stop.

When the monitoring support device 21 recognizes the state of consciousness, that is, the so-called arousal state of the driver, and determines that recovery is necessary, the monitoring support device 21 notifies the driver to encourage recovery of the driver's consciousness.
When the monitoring support device 21 receives a distraction signal or a doze estimation signal from the ECU 3, it issues an alarm to the driver from one or a plurality of speakers (not shown) provided in the vehicle compartment.
As shown in FIG. 2, the speedometer section 21a arranged on the instrument panel is provided with a display section 21b capable of displaying an alarm. When the monitoring support device 21 receives the inattentive signal or the dozing estimation signal from the ECU 3, the display unit 21b displays, for example, a warning of "Caution ahead".

The braking support device 22 is configured to issue an alarm when the own vehicle is approaching another preceding vehicle, and to perform emergency braking when the braking limit distance at which collision can be avoided is reached.
As shown in FIG. 3, the braking support device 22 issues an alarm to the driver from the speaker when the own vehicle approaches the preceding other vehicle B to the position of the own vehicle A1.
The separation distance between the other vehicle B and the own vehicle A1 is a distance corresponding to the time (for example, 0.8 sec) in which the driver can normally avoid a collision by his/her own driving operation (manual operation). .
The braking support device 22 automatically performs emergency braking when the own vehicle approaches the preceding other vehicle B to the position of the own vehicle A2. The separation distance between the other vehicle B and the own vehicle A2 is the limit distance at which a collision can be avoided by emergency braking corresponding to the maximum braking force.

Further, as shown in FIG. 3 , when the braking support device 22 receives the inattentive signal or the drowsiness estimation signal from the ECU 3 , the vehicle A1 is positioned closer to the other vehicle B in which the own vehicle is running than the position of the own vehicle A1. When the host vehicle A3 approaches the distant position, the speaker issues an alarm to the driver. Here, the separation distance between the other vehicle B and the own vehicle A3 is set to the time (for example, 1.2 seconds) during which the driver can avoid the collision by his/her own driving operation even if the driver is in a distracted state. corresponding distance.

When receiving the dozing presumed signal from the ECU 3, the stop support device 23 issues an alarm to the driver from the speaker, and autonomously makes an emergency stop of the own vehicle after receiving the dozing confirmation signal.
As shown in FIG. 4, when the stop assistance device 23 receives the drowsiness presumed signal corresponding to the state of both eyes closed at the position of the own vehicle X1, it activates the speaker to issue an alarm, and at the position of the own vehicle X2. A doze confirmation signal corresponding to the driving inactivity state or the like is received. In the following description, when both eyes are closed, the eye is simply closed, and when only one eye is closed, the one eye is closed, unless otherwise specified.
Next, in the case of the own vehicle X3, automatic driving is started even if manual driving by the driver has been carried out until then. After the own vehicle that has started automatic driving moves to a safe zone such as the shoulder of the road, it is autonomously stopped at the position of the own vehicle X4.

In addition, when receiving an abnormal signal from the ECU 3, the stop support device 23 autonomously stops the host vehicle in an emergency. As shown in FIG. 4, the stop assistance device 23 receives the driver's abnormality signal at the position of the own vehicle X2. Next, in the case of the own vehicle X3, automatic driving is started even if manual driving by the driver has been carried out until then. After the own vehicle that has started automatic driving moves to a safe zone such as the shoulder of the road, it is autonomously stopped at the position of the own vehicle X4. Thereafter, the stop support device 23 makes an emergency call to an emergency center or the like, for example, after the vehicle stop holding period has elapsed for a predetermined period of time. These autonomous emergency stop controls are hereinafter referred to as abnormal sequence control.

Next, ECU3 is demonstrated.
The ECU 3 recognizes signs of the driver when there is an abnormality (hereinafter referred to as signs of abnormality), physical conditions of the driver when there is an abnormality (hereinafter referred to as abnormal conditions), and actions of the driver when there is an abnormality (hereafter referred to as abnormal actions). ) are used to make each judgment. The ECU 3 includes a CPU (Central Processing Unit), a ROM, a RAM, an in-side interface, an out-side interface, and the like. Various programs and data such as brake control and display control are stored in the ROM, and a processing area used when the CPU performs a series of processes is provided in the RAM.

As shown in FIG. 1, the ECU 3 includes a head posture determination unit 31 that determines the state of the driver's head posture, and an eye state determination unit 32 that determines the open/closed state of each of the left and right eyes of the driver based on the degree of eye opening. (open/closed eye determination means), and a driver state determination unit 33 for determining a driver state that may be a hindrance to driving the vehicle.

The head posture determination unit 31 determines the head posture state of the driver based on the captured image information captured by the camera 11 . When the head posture determination unit 31 detects that the head posture state of the driver is one of downward, sideways, prostrate, and backward tilt, the head posture determination unit 31 determines that the head posture has collapsed.
When the head posture collapse is determined, 1 is substituted for the count value of the first counter C1. The initial value of the first counter C1 is zero.

For example, when the absolute value of the face angle in the pitch direction is equal to or greater than a predetermined threshold, the head posture determination unit 31 outputs a downward determination signal. Outputs a fall (lateral lean) determination signal. Further, for example, when the absolute value of the face angle in the pitch direction is equal to or greater than a predetermined threshold and the current head position is equal to or greater than a value obtained by subtracting the head position threshold from the reference head position, A lunge determination signal is output, and if the face angle in the pitch direction is equal to or greater than a predetermined threshold and is smaller than the current head position with respect to the value obtained by adding the head position threshold to the reference head position, the backward tilt determination signal is output. Output. Each threshold associated with the angle and head position for determining whether the driver is lying down, lying down, lying down, or tilting backward corresponds to the head posture determination threshold for determining whether the driver's head posture is abnormal.

The eye state determination unit 32 determines whether the left and right eyes of the driver are in the closed state, and whether only one of the left and right eyes is in the closed state.
The eye condition determination unit 32 detects, for example, the upper and lower eyelids of the driver captured by the camera 11, and detects the degree of opening of the left and right eyes of the driver based on the number of pixels between the edges of the eyelids. .
When the degree of openness of each of the detected left and right eyes is equal to or greater than a predetermined determination threshold value, it is determined that the eyes are open, and the degree of openness of each of the detected left and right eyes is less than the predetermined threshold value. In this case, it is determined that the eyes are closed.

In addition, the eye state determination unit 32 determines that the one eye is closed when the degree of openness of one eye is equal to or greater than a predetermined determination threshold value and the degree of openness of the other eye is less than a predetermined determination threshold value. .
When it is determined that both the left and right eyes of the driver are closed, 1 is substituted for the count value of the second counter C2. The initial value of the second counter C2 is zero.

The driver state determination unit 33 includes a driver state determination unit 33a that determines whether or not the driver is directing his or her line of sight in a direction other than forward in the direction of travel, and an inattentive state determination unit 33a that determines whether or not the driver is drowsy. a doze determination unit 33b for determining the confirmed state of doze in which the driver is detected as dozing off; I have.

The inattentiveness determination unit 33a will be described.
The inattentive determination unit 33a determines whether or not the driver is looking aside from the face direction information, line-of-sight direction information, and vehicle information of the driver. A distraction signal is output to the device 21 and the braking support device 22 .
When the count value of the yaw direction line-of-sight direction of the driver is outside the range of the yaw direction threshold value, it is determined that the driver is looking aside. The yaw direction threshold is changed according to the vehicle speed or the radius of curvature during vehicle travel. Further, when the count value of the line-of-sight direction in the pitch direction of the driver is outside the range of the pitch direction threshold value, it is determined that the driver is looking aside. The pitch direction threshold is changed according to the vehicle speed.

The yaw direction line-of-sight direction count value and the pitch direction line-of-sight direction count value are calculated by a long-time inattentive continuation count or a short-time inattentive cumulative count. The long-time inattentive continuation count is the number of times the determination results for each processing cycle continue in time series. In addition, the cumulative count of inattentiveness for a short period of time is the cumulative count in a state in which the determination result for each processing cycle satisfies the conditions within a predetermined time period before that.

The line-of-sight direction information in the yaw direction and the pitch direction is determined using a reliability threshold whose reliability is set in advance. The reliability of each item of line-of-sight direction information is set based on the definition of the captured image of the driver (definition of the contours of the face, eyelids, black eyes, etc.).
When the reliability of each line-of-sight direction information is low and each line-of-sight direction is invalid, the face direction information in the yaw direction and the pitch direction are interpolated, respectively. When the reliability of the face direction information in the yaw direction and the pitch direction is low and each face direction is invalid, interpolation is performed using the previous effective line-of-sight direction.

Next, the doze determination unit 33b will be described.
The dozing determination unit 33b determines whether or not the driver is dozing off from the duration of eye closure, vehicle information, etc., and if it is estimated that the driver is dozing off, the dozing estimation signal is sent to the monitoring support device 21. , and output to the braking support device 22 and the stop support device 23 , and when it is determined that the driver is dozing off, a doze confirmation signal is output to the stop support device 23 . Note that this doze determination is performed using the eye-closed state in which both eyes are closed.

The drowsiness determination unit 33b estimates the drowsiness state when the continuous eye-closed time of the driver exceeds a predetermined time threshold, or when the short eye-closed time repeatedly exceeds the predetermined time threshold within a predetermined time. .
In addition, the dozing determination unit 33b determines whether the driver's continuous eye closure time exceeds a predetermined time threshold in a situation where the vehicle is traveling at a constant speed or higher after the doze warning, or the continuous eye closure and driving non-operating state. is greater than the time threshold, establish the doze state.

The driving non-operation state is judged as improper driving operation during running by comparing the detection signal of the detection means 1 with a judgment threshold value. Specifically, it is determined based on at least one operation state among non-steering of the steering wheel, non-gripping of the steering wheel, non-operation of accelerator and brake. It should be noted that an extremely small determination threshold value may be set for each operating member to determine an inappropriate driving operation.

Next, the abnormality determination section 33c will be described.
The abnormality determination unit 33c detects an abnormal state in which it is difficult for the driver to operate the vehicle when the driver is in a closed-eyes state and the head posture is disturbed, or when the driver is in a state where the head posture is disturbed and the driver does not operate the vehicle. It is determined that

The inventor of the present invention has found that the symptoms of the driver in an abnormal situation can be detected using closed eyes as a parameter, and the physical state of the driver in an abnormal situation can be detected using head posture collapse as a parameter. It was found that the driver's behavior can be detected as a parameter of driving inaction. Based on this knowledge, in the present embodiment, detection of eye closure and head posture collapse and detection of head posture deformation and driving inactivity are used as determination conditions.

Further, the abnormality determination unit 33c determines that the driver is in an abnormal state when the left and right eyes are closed and the driving non-operation state continues for the first predetermined time T1, and the driver is in the one-eye-closed state. If the execution of the specific operation is not detected within the second predetermined time T2 even if the execution of the specific operation that does not hinder driving is requested, it is determined that the driver is in an abnormal state.
As a result, the abnormal state of the driver with one eye closed is determined without interfering with the driving of the vehicle.

Here, the specific operation to the extent that it does not interfere with driving is, for example, a steering angle operation corresponding to play of the steering wheel (approximately ±5°), a brake pedal depression operation corresponding to 0.1 G, and the like.
Further, such a specific operation can be performed by, for example, an audio instruction from a speaker or a display on the display unit 21b, such as "press the hazard switch 17" or "steer the steering wheel slightly to the right". Requested by the driver from the side.

Next, based on the flowchart of FIG. 5, the closed-eye determination control process will be described.
Note that Si (i=1, 2, . . . ) indicates steps for each process.

First, as shown in FIG. 5, the ECU 3 receives outputs from various detection means 1 such as the camera 11 and various information such as the second counter C2 (S1), and proceeds to S2.
In S2, it is determined whether or not at least one eye of the driver is closed.
If the result of the determination in S2 is that at least one eye of the driver is closed, the driver has his or her eyes closed for some reason, so the process proceeds to S3. As a result of the determination in S2, if at least one eye of the driver is not closed, both the left and right eyes are open, so the process returns.

In S3, it is determined whether or not the driver has one eye closed.
If the result of determination in S3 is that the driver has one eye closed, it is not possible to identify whether the driver is in a good wakefulness state or a poor wakefulness state with one eye closed, so the process proceeds to S4.
If the result of determination in S3 is that the driver does not have one eye closed, both the left and right eyes are closed, so the process proceeds to S7.

In S4, after the vehicle requests the driver to perform a specific operation that does not hinder driving, the process proceeds to S5. Simultaneously with the execution request for the specific operation in S4, a timer (not shown) is started.
In S5, it is determined whether or not the driver has performed the specific operation within the second predetermined time T2 after requesting the execution of the specific operation.

As a result of the determination in S5, if the specific operation is executed by the driver within the second predetermined time T2 after the execution request for the specific operation is made, the driver is in a well-awakened state, so the timer is reset and the process returns. . As a result of the determination in S5, if the driver does not perform the specific operation within the second predetermined time T2 after requesting the execution of the specific operation, the driver is in a poorly awakened state, so the timer is reset and the process proceeds to S6. do. In S6, after executing the sequence control at the time of abnormality, the process returns. In S6, 0 is substituted for the count values of the first and second counters C1 and C2.

In S7, 1 is added to the current count value of the second counter C2, and then the process proceeds to S8.
In S8, it is determined whether or not the count value of the second counter C2 is greater than or equal to the second determination threshold value L2. As a result of the determination in S8, if the count value of the second counter C2 is equal to or greater than the second determination threshold value L2, both the right and left eyes are closed, and the onset of a disease or drowsiness is presumed, so the process proceeds to S9.
If the result of determination in S8 is that the count value of the second counter C2 is less than the second determination threshold value L2, it is impossible to estimate the onset of a disease or doze under the present circumstances, so the process returns.
In S9, after issuing an alarm, the process proceeds to S10.

In S10, it is determined whether or not the first determination condition is satisfied.
The first determination conditions are the following three conditions.
(1) The driver's head posture is in an abnormal state. (2) The driver has not operated the vehicle for the first predetermined time period T1. (3) The driver has closed his eyes for a certain period of time. When all the conditions are met, the first determination condition is met.

As a result of the determination in S10, if the first determination condition is satisfied, the driver does not start driving even if an alarm is issued, so the process proceeds to S11. As a result of the determination in S10, if the first determination condition is not satisfied, one of the conditions (1) to (3) is not satisfied and an abnormal state cannot be determined, so the process returns.

In S11, it is determined whether or not a cancel signal for stopping the alarm has been generated.
As a result of the determination in S11, if a cancel signal has been generated, the driver has performed a cancel signal generating operation, so 0 is assigned to the count values of the first and second counters C1 and C2 (S12), and the process returns. As a result of the determination in S11, if the cancel signal is not generated, the driver cannot operate to generate the signal to cancel the warning, so the process proceeds to S6.
In this embodiment, when the hazard switch 17 is turned on, a cancel signal is output. By outputting the cancel signal, the alarm is stopped, and the abnormality determination is stopped.

Next, head posture determination control processing will be described based on the flowchart of FIG. This head posture determination control process is controlled in parallel independently of the closed eye determination control process.
Si (i=21, 22...) indicates the steps for each process.

First, as shown in FIG. 6, the ECU 3 receives the output from the detection means 1 such as the camera 11 and the hazard switch 17 (S21), and proceeds to S22.
In S22, it is determined whether or not the driver's head posture is broken.
If the result of the determination in S22 is that the driver's head posture is unsteady, it is in an abnormal state, so the process proceeds to S23. If the result of determination in S22 is that the driver's head posture has not collapsed, the process returns because the state is not abnormal.

In S23, it is determined whether or not the driving operation is appropriate.
If the result of the determination in S23 is that the driving operation is appropriate, the driver is intentionally driving with an unsteady posture, and the process returns. If the result of determination in S23 is that the driving operation is inappropriate (driving non-operating state), the process proceeds to S24. Determination of the driving operation is detected by at least one of the steering wheel, accelerator, and brake. In this embodiment, the determination is made using the operation of the steering wheel.

In S24, it is determined whether or not the vehicle is traveling at a speed equal to or higher than a predetermined determination threshold.
As a result of the determination in S24, if the vehicle speed is equal to or greater than the determination threshold value, the driver does not operate the vehicle even though the vehicle is running. S25), and shift to S26. If the vehicle speed is less than the determination threshold as a result of the determination in S24, the vehicle is already stopped or slowing down, so the process returns.

In S26, it is determined whether or not the count value of the first counter C1 is greater than or equal to the first determination threshold value L1.
If the result of determination in S26 is that the count value of the first counter C1 is equal to or greater than the first determination threshold value L1, an abnormal condition is estimated regardless of the condition of the driver's eyes, so the process proceeds to S27.
If the result of determination in S26 is that the count value of the first counter C1 is less than the first determination threshold value L1, the abnormal state cannot be estimated at present, so the process returns.

In S27, after generating an alarm, the process proceeds to S28.
In S28, it is determined whether or not a cancel signal for stopping the alarm has been generated.
As a result of the determination in S28, if a cancel signal has been generated, the driver has performed an operation to generate the cancel signal, so 0 is assigned to the count values of the first and second counters C1 and C2 (S29), and the process returns. . As a result of the determination in S28, if the cancel signal is not generated, the driver cannot operate the hazard switch 17 to generate the cancel signal, so the process proceeds to S30. In S30, after executing the sequence control at the time of abnormality, the process returns. In S30, 0 is substituted for the count values of the first and second counters C1 and C2.

Next, the operation and effect of the driver condition determination method and the device M will be described.
According to this driver state determination method, at least one of the open/closed eye determination steps S2 and S3 for detecting the open/closed state of the left and right eyes of the driver and the operation amount of the vehicle operation device by the driver at a predetermined timing is acquired. and the driver's abnormal state determination steps S5, S10, and S11 for determining whether or not the driver is in an abnormal state. Operation can be used as a condition for judging an abnormal state of the driver.
The driver's abnormal state determination steps S10 and S11 are performed to determine that the driver is in an abnormal state when the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined time period T1. , the abnormal state of the driver can be determined with high accuracy by the two abnormality determination conditions of the abnormal symptom and the abnormal action. The driver's abnormal state determination step S5 is performed when the driver has one eye closed and the specific operation is not detected within the second predetermined time T2 even if the driver is requested to perform a specific operation that does not hinder driving. Since it is determined that the driver is in an abnormal state, the abnormal state of the driver with one eye closed can be reliably determined based on the actual operation of the driver without hindering the driving of the vehicle.

In the driver's abnormal state determination steps S4 and S27, in order to notify when both the left and right eyes of the driver are closed and when one eye is closed, the abnormal state of the driver is detected when both the left and right eyes are not open. can be determined more reliably.

According to the driver state determination device M, at least one of the eye state determination unit 32 that detects the open/closed state of the left and right eyes of the driver and the amount of driving operation of the vehicle by the driver on the operation device of the vehicle at a predetermined timing is acquired. Since the detection means 1 and the abnormality determination unit 33c for determining whether or not the driver is in an abnormal state are included, the condition for determining the abnormal state of the driver is the closing of the eyes, which is a symptom at the time of abnormality, and the non-driving operation, which is the action at the time of abnormality. can be
The abnormality determining unit 33c determines that the driver is in an abnormal state when the left and right eyes of the driver are closed and the driver does not operate the vehicle for the first predetermined time period T1. The abnormal state of the driver can be determined with high accuracy by the two abnormality determination conditions of the operation in the event of an abnormality. The abnormality determination unit 33c detects the specific operation within the second predetermined time T2 even if the driver has one eye closed and the specific operation that does not hinder driving is requested to the driver. is an abnormal state, it is possible to reliably determine the abnormal state of the driver with one eye closed based on the driver's actual operation without hindering the driving of the vehicle.

Next, a modified example in which the above embodiment is partially changed will be described.
1) In the above embodiment, an example was described in which it is determined that the driver is in an abnormal state when the specific operation is not detected within the second predetermined time T2 when one eye of the driver is closed. In addition to the conditions, the driver's head posture acquired by the camera 11 is an abnormal posture, one eye of the driver is closed, and the driver's non-operating state is shorter than the first predetermined time T1. A determination condition may be added to determine that the driver is in an abnormal state when the three predetermined time T3 continues.
As a result, the abnormal state of the driver can be reliably and quickly determined using the abnormal posture as a parameter.

2) In addition, those skilled in the art can implement various modifications to the above embodiment without departing from the scope of the present invention, and the present invention includes such modifications. be.

1 detection means 11 camera 32 eye condition determination unit 33c abnormality determination unit M driver condition determination device

Claims (4)

In a driver state determination method for detecting an abnormal state of a vehicle driver,
an open/closed eye determination step of detecting the open/closed state of both left and right eyes of the driver;
a driving operation amount acquiring step of acquiring at least one amount of driving operation of the operation device of the vehicle by the driver at a predetermined timing;
a driver abnormal state determination step of determining whether the driver is in an abnormal state;
The driver's abnormal state determination step includes:
determining that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined period of time;
When the driver is in an abnormal state when one eye of the driver is closed and the specific operation is not detected within a second predetermined time even if the driver is requested to perform a specific operation that does not hinder driving. A driver state determination method characterized by determining that 2. The driver's state determination method according to claim 1, wherein, in the driver's abnormal state determination step, notification is given when both the left and right eyes of the driver are closed and when one eye is closed. a head posture acquiring step of acquiring the driver's face orientation and head position;
The driver's abnormal state determination step includes:
The head posture of the driver acquired in the head posture acquiring step is an abnormal posture, one eye of the driver is closed, and the driver's non-operating state is shorter than a first predetermined time. 3. The method for judging a driver's condition according to claim 1, further comprising the step of judging that the driver is in an abnormal condition if the condition continues for a third predetermined time. In a driver state determination device for detecting an abnormal state of a vehicle driver,
open/closed eye determination means for detecting the open/closed state of both left and right eyes of the driver;
a driving operation amount acquiring means for acquiring at least one amount of driving operation of the operating device of the vehicle by the driver at a predetermined timing;
a driver abnormal state determination means for determining whether the driver is in an abnormal state,
The driver's abnormal state determination means includes:
determining that the driver is in an abnormal state when both the left and right eyes of the driver are closed and the driver has not operated the vehicle for a first predetermined period of time;
When the driver is in an abnormal state when one eye of the driver is closed and the specific operation is not detected within a second predetermined time even if the driver is requested to perform a specific operation that does not hinder driving. A driver's state determination device characterized by determining that

Images