JP2023072268A - Driver state determination method and device - Google Patents

Abstract

To provide a driver state determination method and a device that can determine an abnormal state of a driver early with high accuracy by varying time to determine abnormality between a closed eye state and a non-closed eye state.SOLUTION: A driver abnormal state determination step S19 determines that a driver is in an abnormal state when a closed eye state is not detected in a closed eye determination step S12, if a head posture is an abnormal posture and a situation where a driving operation amount falls below a predetermined threshold corresponding to non-driving operation continues for a period corresponding to a first determination threshold for non-closed eye L1, and determines that the driver is in the abnormal state when the closed eye state is detected in the closed eye determination step S12, if the head posture is an abnormal posture with a smaller degree of abnormality than the above abnormal posture and the situation where the driving operation amount falls below the predetermined threshold corresponding to non-driving operation continues for a period corresponding to a first determination threshold for closed eye L1 shorter than the period corresponding to the first determination threshold for non-closed eye L1.SELECTED DRAWING: Figure 6

Description

The present invention relates to a driver condition determination method and device for detecting an abnormal condition of a driver driving a vehicle.

2. Description of the Related Art Conventionally, there has been known a driver assistance device that accurately grasps the state of consciousness of a driver, particularly the arousal state of the driver, and assists the driving operation of a vehicle according to the arousal state.
In addition, in the event of a sudden change in the driver's physical condition that makes it impossible for the driver to continue driving, a so-called abnormal condition, the vehicle automatically moves to the side of the road and stops autonomously. Devices are also known.

Normally, the driver's abnormal state including sleepiness level is determined through head-related information such as the driver's head posture and face state acquired by imaging means such as a camera.
Of the head-related information, the head posture is detected via, for example, the driver's face orientation and head position (three-dimensional coordinates), and the face state is detected, for example, from the driver's facial expression and eye opening state. (eyelid state) and the like.

The drowsiness detection device of Patent Document 1 derives an eye opening degree based on an imaging means for imaging a driver's face and a captured image, and determines a higher arousal level as the time when the opening degree is equal to or less than a specified ratio is shorter. Arousal level estimating means for estimating, line-of-sight estimating means for estimating the line-of-sight direction of the driver, and in the case of downward looking in which the line of sight is directed downward from the line of sight to the gaze point, a decrease in arousal level due to downward looking is offset. and drowsiness detection means for detecting drowsiness of the driver by comparing the corrected wakefulness with a predetermined threshold value. When the drowsiness of the driver is detected, an alarm is issued to eliminate the drowsiness of the driver.

Japanese Patent No. 4840146

The drowsiness detection device of Patent Document 1 can avoid erroneous determination that the eyes of the driver are closed even though the eyes are actually open, and can limit unnecessary alarms.
However, with the drowsiness detection device of Patent Document 1, even if it is possible to determine drowsiness in consideration of the driver's downward gaze, there is a possibility that the abnormal state of the driver cannot be accurately determined.
For example, if the determination threshold for detecting an abnormal state of the driver is set low, in addition to the annoyance of frequent warnings, there is a risk of causing an unexpected emergency stop for other vehicles.

Therefore, in order to avoid erroneous determination of an abnormal state, eye closure, which is a sign of the driver in an abnormal state, and head posture collapse, which is a physical state of the driver in an abnormal state, are detected, respectively, and these two abnormal states are determined. It is conceivable to determine an abnormal state of the driver when both conditions are met.
However, since closed-eyes determination control and head posture determination control are independently processed in parallel, the head posture determination time when the eyes are closed is set to the same time as the head posture determination time when the eyes are not closed. If it is set, the judgment time when the eyes are closed may be unnecessarily prolonged, which may lead to delays in emergency stoppages and emergency calls.
That is, it is not easy to quickly and accurately determine the abnormal state of the driver.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a driver condition determination method and apparatus capable of quickly and accurately determining an abnormal condition of a driver.

A driver state determination method according to claim 1 is a driver state determination method for detecting an abnormal state of a driver of a vehicle, comprising: a closed-eye determination step of detecting that a person's eyes are closed; a driving operation amount acquiring step of acquiring at least one amount of driving operation of the vehicle by the driver at a predetermined timing; and a driver's abnormal state determination step for determining whether or not the driver's abnormal state determination step includes the head posture acquisition step when the eye-closed state of the driver is not detected in the closed-eyes determination step. If the head posture of the driver acquired in step 1 is an abnormal posture and the driving operation amount is below a predetermined threshold value for a first predetermined time period, it is determined that the driver is in an abnormal state, and the eyes are closed. When the closed eye state of the driver is detected in the determining step, the head posture of the driver acquired in the head posture acquiring step is an abnormal posture with a smaller degree of abnormality than the abnormal posture, and the driving operation. It is characterized by determining that the driver is in an abnormal state when the situation in which the amount is below a predetermined threshold continues for a second predetermined period of time shorter than the first predetermined period of time.

This driver state determination method includes a head posture acquisition step of acquiring the face orientation and head position of the driver, an eye closure determination step of detecting that the driver's eyes are closed, and at a predetermined timing: A driving operation amount acquisition step of acquiring at least one amount of driving operation of the vehicle by the driver, and a driver abnormal state determination step of determining whether the driver is in an abnormal state. The physical condition of the driver (hereinafter referred to as abnormal state) is head posture collapse, and the driver's symptoms in abnormal conditions (hereinafter referred to as abnormal symptoms) are eye closure and driver behavior in abnormal conditions (hereinafter referred to as abnormal conditions). It is possible to use the driver's non-operating operation as a condition for determining whether the driver is in an abnormal state.
The driver's abnormal state determination step includes, when the closed-eyes state of the driver is not detected in the closed-eyes determination step, the head posture of the driver acquired in the head posture acquisition step is an abnormal posture, and Since it is determined that the driver is in an abnormal state when the driving operation amount remains below a predetermined threshold for a first predetermined period of time, the abnormal state and the abnormal action are determined even if the driver is in the non-eyes-closed state. As a condition, the abnormal state of the driver can be determined with high accuracy.
In the driver abnormal state determination step, when the closed-eye state of the driver is detected in the closed-eye determination step, the head posture of the driver acquired in the head posture acquisition step is higher than the abnormal posture. When the state in which the driver is in an abnormal posture with a small degree of abnormality and the driving operation amount is less than a predetermined threshold continues for a second predetermined time period shorter than the first predetermined time period, the driver is determined to be in an abnormal state. The abnormal state of the driver can be determined in a short time based on the abnormal symptom and the abnormal state.

The invention of claim 2 is characterized in that, in the invention of claim 1, the driver's abnormal condition determination step determines that the head posture is abnormal based on at least the face angle of the driver. It is characterized by
According to this configuration, it can be easily determined that the head posture of the driver is abnormal.

The invention of claim 3 is characterized in that, in the invention of claim 1 or 2, the closed-eyes determination step is characterized in that an alarm is issued after a predetermined time has passed since the closed-eyes state of the driver is detected. there is
According to this configuration, the driver can be made to recognize that the eyes are closed.

According to a fourth aspect of the present invention, there is provided a driver state determination device for detecting an abnormal state of a driver of a vehicle, comprising head posture acquisition means for acquiring the face orientation and head position of the driver; closed-eye determination means for detecting that a person's eyes are closed; driving operation amount acquisition means for acquiring at least one amount of driving operation of the vehicle by the driver at a predetermined timing; and when the driver is in an abnormal state. a driver's abnormal state determination means for determining whether or not the driver's abnormal state determination means determines whether or not the closed-eyes state of the driver is detected by the driver's abnormal state determination means; If the head posture of the driver acquired in step 1 is an abnormal posture and the driving operation amount is below a predetermined threshold value for a first predetermined time period, it is determined that the driver is in an abnormal state, and the eyes are closed. When the determining means detects the eye-closed state of the driver, the head posture of the driver acquired by the head posture acquiring means is an abnormal posture with a smaller degree of abnormality than the abnormal posture, and the driving operation. It is characterized by determining that the driver is in an abnormal state when the situation in which the amount is below a predetermined threshold continues for a second predetermined period of time shorter than the first predetermined period of time.

In this driver state determination device, in a driver state determination device for detecting an abnormal state of a driver of a vehicle, head posture acquisition means for acquiring the face orientation and head position of the driver; closed-eye determination means for detecting that the is in an eye-closed state; driving operation amount acquiring means for acquiring at least one amount of driving operation of the vehicle by the driver at a predetermined timing; and determining whether the driver is in an abnormal state Since it has a driver's abnormal state determination means for determining the abnormal state of the driver, the head posture collapse, which is an abnormal state, the closed eye, which is an abnormal sign, and the driving inactivity, which is an abnormal operation, are used as the conditions for determining the abnormal state of the driver. can do.
The driver's abnormal state determining means determines that the driver's head posture acquired by the head posture acquiring means is an abnormal posture and the above-mentioned Since it is determined that the driver is in an abnormal state when the driving operation amount remains below a predetermined threshold for a first predetermined period of time, the abnormal state and the abnormal action are determined even if the driver is in the non-eyes-closed state. As a condition, the abnormal state of the driver can be determined with high accuracy.
The driver's abnormal state determination means is configured such that, when the closed-eyes determination means detects the closed-eyes state of the driver, the head posture of the driver acquired by the head posture acquisition means is higher than the abnormal posture. When the state in which the driver is in an abnormal posture with a small degree of abnormality and the driving operation amount is less than a predetermined threshold continues for a second predetermined time period shorter than the first predetermined time period, the driver is determined to be in an abnormal state. The abnormal state of the driver can be determined in a short time based on the abnormal symptom and the abnormal state.

According to the driver state determination method and apparatus of the present invention, the abnormal state of the driver can be determined early and with high accuracy by differentiating the determination time of the abnormal state between the eye closed state and the non-closed eye state.

1 is a block diagram of a driver's state determination device according to Embodiment 1; FIG. It is an explanatory view concerning a monitoring support device. It is an explanatory view concerning a braking assistance device. It is explanatory drawing which concerns on a stop assistance apparatus. 7 is a flowchart showing closed-eye determination control processing. 9 is a flowchart showing head posture determination control processing;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail based on the drawings.
The following description exemplifies the application of the present invention to a vehicle driver state determination device, and does not limit the present invention, its application, or its use.

Embodiment 1 of the present invention will be described below with reference to FIGS. 1 to 6. FIG.
The driver state determination device M determines the state of consciousness of the driver, in particular, the abnormal state in which the driving operation of the vehicle driven by the driver (hereinafter also referred to as own vehicle) is difficult, and determines the abnormal state of the driver. It is a driving support device that can autonomously make an emergency stop.
Hereinafter, the abnormal state of the driver will be described as a state in which the driver is not consciously awake and is physically out of posture, making it impossible to drive (including the onset of a disease and falling asleep).
Also, the following description includes the description of the driver state determination method.

As shown in FIG. 1, the driver state determination device M includes a plurality of detection means 1 capable of detecting the operation behavior of the driver, the running behavior of the vehicle, etc. A driver assistance device 2 that assists driving and an ECU (Electronic Control Unit) 3 that performs various calculations based on input signals input from the detection means 1 and outputs various command signals to the driver assistance device 2, etc. It is a component.

First, the detection means 1 will be explained.
As shown in FIG. 1, the detection means 1 includes a camera 11 that captures an image of the driver, a vehicle speed sensor 12 that detects the speed of the vehicle, and a steering angle sensor 13 that detects the steering angle by operating a steering wheel (not shown). , a gripping sensor 14 for detecting gripping of the steering wheel by the driver, an accelerator sensor 15 for detecting the amount of depression of an accelerator pedal (not shown), and a brake sensor 16 for detecting the amount of depression of a brake pedal (not shown). , and a hazard switch 17 for operating a hazard lamp (not shown). Each of the sensors 12 to 16 corresponds to a driving operation amount acquiring means for acquiring the driving operation amount at a predetermined timing, that is, just before the so-called determination time.

The camera 11 is composed of, for example, a CCD (Charge Coupled Device) camera, and is mounted at a position facing the driver in the upper center of an instrument panel (not shown).
The camera 11 is formed so as to be able to image the head including the face of the driver seated in the driver's seat and wearing the seat belt from the front. The captured head image is obtained by extracting the image of the driver's face, specifying the orientation direction of the face (face direction), identifying the driver's facial expression, and detecting the eye iris (iris) in an enlarged manner to determine the line-of-sight direction. , the eyelid portion is magnified and detected to detect the opening degree and open/close state of the driver's eyes, the three-dimensional coordinate position of the driver's head (center of gravity), and the like.
The camera 11 is sufficient as long as it can acquire at least a necessary captured image, and may be equipped with two cameras, a front camera and an upper camera installed right above the driver's seat, or three or more cameras.

Next, the driver assistance device 2 will be described.
As shown in FIG. 1, the driver support device 2 includes a monitoring support device 21 that notifies the driver of a warning state, a braking support device 22 that performs a braking operation on the vehicle, and an emergency warning device for the vehicle. It is composed of a stop support device 23 and the like that cause the vehicle to stop.

When the monitoring support device 21 recognizes the state of consciousness, that is, the so-called arousal state of the driver, and determines that recovery is necessary, the monitoring support device 21 notifies the driver to encourage recovery of the driver's consciousness.
When the monitoring support device 21 receives a distraction signal or a doze estimation signal from the ECU 3, it issues an alarm to the driver from one or a plurality of speakers (not shown) provided in the vehicle compartment.
As shown in FIG. 2, the speedometer section 21a arranged on the instrument panel is provided with a display section 21b capable of displaying an alarm. When the monitoring support device 21 receives the inattentive signal or the dozing estimation signal from the ECU 3, the display unit 21b displays, for example, a warning of "Caution ahead".

The braking support device 22 is configured to issue an alarm when the own vehicle is approaching another preceding vehicle, and to perform emergency braking when the braking limit distance at which collision can be avoided is reached.
As shown in FIG. 3, the braking support device 22 issues an alarm to the driver from the speaker when the own vehicle approaches the preceding other vehicle B to the position of the own vehicle A1.
The separation distance between the other vehicle B and the own vehicle A1 is a distance corresponding to the time (for example, 0.8 sec) in which the driver can normally avoid a collision by his/her own driving operation (manual operation). .
The braking support device 22 automatically performs emergency braking when the own vehicle approaches the preceding other vehicle B to the position of the own vehicle A2. The separation distance between the other vehicle B and the own vehicle A2 is the limit distance at which a collision can be avoided by emergency braking corresponding to the maximum braking force.

Further, as shown in FIG. 3 , when the braking support device 22 receives the inattentive signal or the drowsiness estimation signal from the ECU 3 , the vehicle A1 is positioned closer to the other vehicle B in which the own vehicle is running than the position of the own vehicle A1. When the host vehicle A3 approaches the distant position, the speaker issues an alarm to the driver. Here, the separation distance between the other vehicle B and the own vehicle A3 is set to the time (for example, 1.2 seconds) during which the driver can avoid the collision by his/her own driving operation even if the driver is in a distracted state. corresponding distance.

When receiving the dozing presumed signal from the ECU 3, the stop support device 23 issues an alarm to the driver from the speaker, and autonomously makes an emergency stop of the own vehicle after receiving the dozing confirmation signal.
As shown in FIG. 4, when the vehicle stop support device 23 receives the drowsiness estimation signal corresponding to the eye-closed state at the position of the own vehicle X1, it activates the speaker to generate an alarm, and drives the vehicle at the position of the own vehicle X2. A doze confirmation signal corresponding to a non-operating state or the like is received. Next, in the case of the own vehicle X3, automatic driving is started even if manual driving by the driver has been carried out until then. After the own vehicle that has started automatic driving moves to a safe zone such as the shoulder of the road, it is autonomously stopped at the position of the own vehicle X4.

In addition, when receiving an abnormal signal from the ECU 3, the stop support device 23 autonomously stops the host vehicle in an emergency. As shown in FIG. 4, the stop assistance device 23 receives the driver's abnormality signal at the position of the own vehicle X2. Next, in the case of the own vehicle X3, automatic driving is started even if manual driving by the driver has been carried out until then. After the own vehicle that has started automatic driving moves to a safe zone such as the shoulder of the road, it is autonomously stopped at the position of the own vehicle X4. Thereafter, the stop support device 23 makes an emergency call to an emergency center or the like, for example, after the vehicle stop holding period has elapsed for a predetermined period of time. These autonomous emergency stop controls are hereinafter referred to as abnormal sequence control.

Next, ECU3 is demonstrated.
The ECU 3 recognizes signs of the driver when there is an abnormality (hereafter referred to as signs of abnormality), physical conditions of the driver when there is an abnormality (hereafter referred to as abnormal states), and behaviors of the driver when there is an abnormality (hereafter referred to as behaviors when there is an abnormality). ) are used to make each determination. The ECU 3 includes a CPU (Central Processing Unit), a ROM, a RAM, an in-side interface, an out-side interface, and the like. Various programs and data such as brake control and display control are stored in the ROM, and a processing area used when the CPU performs a series of processes is provided in the RAM.

As shown in FIG. 1, the ECU 3 includes a head posture determination unit 31 that determines the state of the driver's head posture, and an eye state determination unit 32 (closed-eye determination means) that determines whether the left and right eyes of the driver are open or closed. , and a driver state determination unit 33 for determining a driver state that can be a hindrance to driving the vehicle.

The head posture determination unit 31 determines the head posture state of the driver based on the captured image information captured by the camera 11 . When the head posture determination unit 31 detects that the head posture state of the driver is one of downward, sideways, prostrate, and backward tilt, the head posture determination unit 31 determines that the head posture has collapsed.
When the head posture collapse is determined, 1 is substituted for the count value of the first counter C1. The initial value of the first counter C1 is zero.

For example, when the absolute value of the face angle in the pitch direction is equal to or greater than a predetermined threshold, the head posture determination unit 31 outputs a downward determination signal. Outputs a fall (lateral lean) determination signal. Further, for example, when the absolute value of the face angle in the pitch direction is equal to or greater than a predetermined threshold and the current head position is equal to or greater than a value obtained by subtracting the head position threshold from the reference head position, A lunge determination signal is output, and if the face angle in the pitch direction is equal to or greater than a predetermined threshold and is smaller than the current head position with respect to the value obtained by adding the head position threshold to the reference head position, the backward tilt determination signal is output. Output.

Each threshold associated with the angle and head position for determining whether the driver is lying down, lying down, lying down, or tilting backward corresponds to the head posture determination threshold for determining whether the driver's head posture is abnormal.
Two types of thresholds are prepared: a closed-eyes head posture determination threshold used when the eyes are closed and a non-closed-eyes head posture determination threshold used when the eyes are not closed.
The closed-eyes head posture determination threshold is set to a smaller value than the non-eyes-closed head posture determination threshold, aiming at early determination of an abnormal state.

The eye state determination unit 32 determines whether or not the driver's eyes are closed.
The eye condition determination unit 32 detects, for example, the upper and lower eyelids of the driver captured by the camera 11, and detects the degree of eye opening of the driver based on the number of pixels between the edges of the eyelids. If the detected eye openness is equal to or greater than a predetermined determination threshold, it is determined that the eyes are open, and if the detected eye openness is less than the predetermined determination threshold, it is determined that the eyes are closed. When the eye-closed state is determined, 1 is substituted for the count value of the second counter C2. The initial value of the second counter C2 is zero.
Note that in the present embodiment, closed-eye determination is made only when both left and right eyes of the driver are determined to be closed, and in other cases, closed-eye determination is not made.

The driver state determination unit 33 includes a driver state determination unit 33a that determines whether or not the driver is directing his or her line of sight in a direction other than forward in the direction of travel, and an inattentive state determination unit 33a that determines whether or not the driver is drowsy. a doze determination unit 33b for determining the confirmed state of doze in which the driver is detected as dozing off; I have.

The inattentiveness determination unit 33a will be described.
The inattentive determination unit 33a determines whether or not the driver is looking aside from the face direction information, line-of-sight direction information, and vehicle information of the driver. A distraction signal is output to the device 21 and the braking support device 22 .
When the count value of the yaw direction line-of-sight direction of the driver is outside the range of the yaw direction threshold value, it is determined that the driver is looking aside. The yaw direction threshold is changed according to the vehicle speed or the radius of curvature during vehicle travel. Further, when the count value of the line-of-sight direction in the pitch direction of the driver is outside the range of the pitch direction threshold value, it is determined that the driver is looking aside. The pitch direction threshold is changed according to the vehicle speed.

The yaw direction line-of-sight direction count value and the pitch direction line-of-sight direction count value are calculated by a long-time inattentive continuation count or a short-time inattentive cumulative count. The long-time inattentive continuation count is the number of times the determination results for each processing cycle continue in time series. In addition, the cumulative count of inattentiveness for a short period of time is the cumulative count in a state in which the determination result for each processing cycle satisfies the conditions within a predetermined time period before that.

The line-of-sight direction information in the yaw direction and the pitch direction is determined using a reliability threshold whose reliability is set in advance. The reliability of each item of line-of-sight direction information is set based on the definition of the captured image of the driver (definition of the contours of the face, eyelids, black eyes, etc.).
When the reliability of each line-of-sight direction information is low and each line-of-sight direction is invalid, the face direction information in the yaw direction and the pitch direction are interpolated, respectively. When the reliability of the face direction information in the yaw direction and the pitch direction is low and each face direction is invalid, interpolation is performed using the previous effective line-of-sight direction.

Next, the doze determination unit 33b will be described.
The dozing determination unit 33b determines whether or not the driver is dozing off from the duration of eye closure and vehicle information, etc., and if it is estimated that the driver is dozing off, the dozing estimation signal is sent to the monitoring support device 21. , and output to the braking support device 22 and the stop support device 23 , and when it is determined that the driver is dozing off, a doze confirmation signal is output to the stop support device 23 .

The drowsiness determination unit 33b estimates the drowsiness state when the continuous eye-closed time of the driver exceeds a predetermined time threshold, or when the short eye-closed time repeatedly exceeds the predetermined time threshold within a predetermined time. .
In addition, the dozing determination unit 33b determines whether the driver's continuous eye closure time exceeds a predetermined time threshold in a situation where the vehicle is traveling at a constant speed or higher after the doze warning, or the continuous eye closure and driving non-operating state. is greater than the time threshold, establish the doze state.

The driving non-operation state is judged as improper driving operation during running by comparing the detection signal of the detection means 1 with a judgment threshold value. Specifically, it is determined based on at least one operation state among non-steering of the steering wheel, non-gripping of the steering wheel, non-operation of accelerator and brake. It should be noted that an extremely small determination threshold value may be set for each operating member to determine an inappropriate driving operation.

Next, the abnormality determination section 33c will be described.
The abnormality determination unit 33c uses, as the determination thresholds, a first non-closed-eye determination threshold L1 (first predetermined time) used when the eyes are not closed and a first closed-eye determination threshold L1 (second predetermined time) used when the eyes are closed. Two types are prepared. The eye-closed first determination threshold value L1 is set to a value smaller than the eye-non-closed first determination threshold value L1 with the aim of early determination of an abnormal state.

When the eyes are not closed, the abnormality determination unit 33c determines that a situation in which the driver's head posture determined by the head posture determination unit 31 is in an abnormal posture and the amount of driving operation is below a predetermined determination threshold corresponding to an inappropriate operation is non-existent. If it continues for a period corresponding to the first determination threshold value L1 for closed eyes (first predetermined time), it is determined that the driver is in an abnormal state. The abnormality determination unit 33c determines that, when the eyes are closed, the head posture of the driver determined by the head posture determination unit 31 is an abnormal posture with a smaller degree of abnormality than the abnormal posture, and the amount of driving operation corresponds to an inappropriate operation. continues for a period corresponding to the closed-eyes first determination threshold L1 (second predetermined time) shorter than the non-closed-eyes first determination threshold L1, it is determined that the driver is in an abnormal state.

The present inventor is able to detect abnormal symptoms using closed eyes as a parameter, detect abnormal states using head posture collapse as a parameter, and detect abnormal actions using driving inactivity as a parameter. I got the knowledge that it is possible.
Based on this knowledge, in the present embodiment, when the driver does not close his/her eyes, the abnormal state of the driver is determined using the non-closed-eyes head posture determination threshold value and the non-closed-eyes first determination threshold value L1. When the eyes are closed, an abnormality signal is output to the driver support device 2 by determining the abnormal state of the driver using the closed-eyes head posture determination threshold value and the first closed-eyes determination threshold value L1.
Note that the non-closed eyes include a state in which closing of both eyes cannot be detected technically and a state in which at least one eye is open.

Next, based on the flowchart of FIG. 5, the closed-eye determination control process will be described.
Note that Si (i=1, 2, . . . ) indicates steps for each process.

First, as shown in FIG. 5, the ECU 3 receives outputs from various detection means 1 such as the camera 11 and various information such as the second counter C2 (S1), and proceeds to S2.
In S2, it is determined whether or not the driver's eyes are closed.
If the result of the determination in S2 is that the driver has closed eyes, the closed eyes, which is a sign of the driver in an abnormal state (disease onset or drowsiness), has been detected, so the process proceeds to S3. If the result of determination in S2 is that the driver does not have his/her eyes closed, at least one eye is open or the reliability of the imaging information is low, so the process returns.

In S3, 1 is added to the current count value of the second counter C2, and then the process proceeds to S4.
In S4, it is determined whether or not the count value of the second counter C2 is greater than or equal to the second determination threshold value L2. As a result of the determination in S4, if the count value of the second counter C2 is equal to or greater than the second determination threshold value L2, it means that the eyes are closed, and it is estimated that the onset of a disease or drowsiness is occurring, so the process proceeds to S5.
If the result of determination in S4 is that the count value of the second counter C2 is less than the second determination threshold value L2, it is impossible to estimate the onset of a disease or dozing off at present, so the process returns.
In S5, after issuing an alarm, the process returns.

Next, head posture determination control processing will be described based on the flowchart of FIG. This head posture determination control process is controlled in parallel independently of the closed eye determination control process.
Si (i=11, 12...) indicates the steps for each process.

First, as shown in FIG. 6, the ECU 3 receives the output from the detection means 1 such as the camera 11 and the hazard switch 17 (S11), and proceeds to S12.
In S12, it is determined whether or not the count value of the second counter C2 is greater than zero.
As a result of the determination in S12, if the count value of the second counter C2 is greater than 0, the eyes of the driver are closed. Move to S15. As a result of the determination in S12, if the count value of the second counter C2 is 0 or less, it means that the driver has non-closed eyes. First determination threshold values L1 for non-closed eyes are set (S14), and the process proceeds to S15.

In S15, it is determined whether or not the driver's head posture has collapsed.
Here, the head posture of the driver is determined using the closed-eyes head posture determination threshold when the eyes are closed, and when the eyes are not closed, a non-closed-eyes head posture determination threshold larger than the closed-eyes head posture determination threshold is set. is determined using Therefore, when the eyes are closed, it is easier to determine head posture collapse than when the eyes are not closed.
If the result of the determination in S15 is that the driver's head posture is unsteady, it is in an abnormal state, so the process proceeds to S16. If the result of determination in S15 is that the driver's head posture has not collapsed, the process returns because the state is not abnormal.

In S16, it is determined whether or not the driving operation by the driver is appropriate.
If the result of the determination in S16 is that the driving operation is appropriate, the driver is intentionally driving in an unsteady posture, and the process returns. If the result of determination in S16 is that the driving operation is inappropriate (no driving operation), the process proceeds to S17. Determination of the driving operation is detected by at least one of the steering wheel, accelerator, and brake. In this embodiment, the determination is made using the operation of the steering wheel.

In S17, it is determined whether or not the vehicle is traveling at a speed equal to or higher than a predetermined determination threshold.
As a result of the determination in S17, if the vehicle speed is equal to or greater than the determination threshold value, the driver does not operate the vehicle even though the vehicle is running. S18), and shifts to S19. If the vehicle speed is less than the determination threshold as a result of the determination in S17, the vehicle is already stopped or slowing down, so the process returns.

In S19, it is determined whether or not the count value of the first counter C1 is greater than or equal to the first determination threshold value L1.
Here, when the eyes are closed, the non-driving operation, which is the abnormal operation, is determined using the closed-eyes first determination threshold value L1, and when the eyes are not closed, the non-closed-eyes first determination threshold value larger than the closed-eyes first determination threshold value L1 is used. determined using L1. Therefore, when the eyes are closed, it is easier to determine that the driver is not operating, which is an abnormal operation, than when the eyes are not closed. As a result of the determination in S19, if the count value of the first counter C1 is equal to or greater than the first determination threshold value L1, an abnormal state is estimated regardless of the state of the driver's eyes, so the process proceeds to S20.
If the result of determination in S19 is that the count value of the first counter C1 is less than the first determination threshold value L1, the abnormal state cannot be estimated at present, so the process returns.

In S20, after generating an alarm, the process proceeds to S21.
In S21, it is determined whether or not a cancel signal for stopping the alarm has been generated.
As a result of the determination in S21, if a cancel signal has been generated, the driver has performed an operation to generate the cancel signal, so 0 is assigned to the count values of the first and second counters C1 and C2 (S22), and the process returns. . As a result of the determination in S21, if the cancel signal is not generated, the driver cannot cancel with the hazard switch 17, so the process proceeds to S23.
In S23, after executing the sequence control at the time of abnormality, the process returns. In S23, 0 is substituted for the count values of the first and second counters C1 and C2.

Next, the operation and effect of the driver condition determination method and the device M will be described.
According to this driver state determination method, head posture acquisition steps S1 and S11 for acquiring the driver's face orientation and head position, eye closure determination step S2 for detecting that the driver's eyes are closed, S12, driving operation amount acquisition steps S1 and S11 for acquiring at least one amount of vehicle operation by the driver at a predetermined timing, and driver abnormality determination step S19 for determining whether or not the driver is in an abnormal state. Therefore, it is possible to set the conditions for determining the abnormal state of the driver, namely, head posture collapse, which is an abnormal state, closed eyes, which is an abnormal sign, and non-driving operation, which is an abnormal action.
The driver's abnormal state determination step S19 determines whether the driver's head posture acquired in the head posture acquisition step S11 is an abnormal posture and the driving operation amount is determined when the closed-eyes state of the driver is not detected in the closed-eyes determination step S12. is less than a predetermined threshold value corresponding to non-driving operation for a period corresponding to the non-closed-eyes first determination threshold value L1, it is determined that the driver is in an abnormal state. The abnormal state of the driver can be determined with high accuracy using the time state and the abnormal operation as determination conditions. In the driver's abnormal state determination step S19, when the closed-eye state of the driver is detected in the closed-eye determination step S12, the degree of abnormality of the driver's head posture acquired in the head posture acquisition step S11 is higher than that of the abnormal posture. A case where a situation in which the driver is in a small abnormal posture and the amount of driving operation is below a predetermined threshold value corresponding to no driving operation continues for a period corresponding to the closed-eyes first determination threshold value L1, which is shorter than the period corresponding to the non-closed-eyes first determination threshold value L1. Since it is determined that the driver is in an abnormal state, when the eyes are closed, the abnormal state of the driver can be determined in a short time based on the abnormal symptom and the abnormal state.

Since the driver's abnormal state determination step S19 determines that the head posture is abnormal based on at least the face angle of the driver, it can be easily determined that the driver's head posture is abnormal. can do.

In the eyes-closed determination steps S4 and S5, an alarm is issued after a period of time corresponding to the second determination threshold value L2 has elapsed after the eyes-closed state of the driver is detected, so that the driver can recognize that the eyes are closed.

According to the present driver state determination device M, in the driver state determination device M for detecting an abnormal state of the driver of the vehicle, the camera 11 for acquiring the driver's face direction and head position, and the driver's eyes An eye state determination unit 32 that detects that the eyes are closed, sensors 12 to 16 that acquire at least one amount of vehicle driving operation by the driver at a predetermined timing, and a driver that determines whether or not the driver is in an abnormal state. Since the abnormal state determination unit 33c is provided, it is possible to use the abnormal state of the head posture collapse, the abnormal symptom of closed eyes, and the abnormal operation of the driver's non-operating operation as the conditions for determining the abnormal state of the driver. . When the eye state determination unit 32 does not detect the closed eyes state of the driver, the abnormal state determination unit 33c determines that the head posture of the driver acquired by the camera 11 is an abnormal posture and the amount of driving operation is non-driving. If the condition below the corresponding predetermined threshold continues for a period corresponding to the eye non-closed first determination threshold L1, it is determined that the driver is in an abnormal state. The abnormal state of the driver can be determined with high accuracy using the motion as a determination condition. When the eye state determination unit 32 detects that the driver's eyes are closed, the abnormal state determination unit 33c determines that the head posture of the driver acquired by the camera 11 is an abnormal posture with a smaller degree of abnormality than the abnormal posture, and If the state in which the amount of driving operation falls below a predetermined threshold value corresponding to no driving operation continues for a period corresponding to the closed-eyes first determination threshold value L1, which is shorter than the period corresponding to the non-closed-eyes first determination threshold value L1, the driver is abnormal. Therefore, when the eyes are closed, the abnormal state of the driver can be determined in a short period of time based on the abnormal symptom and the abnormal state.

In addition, those skilled in the art can implement various modifications to the above-described embodiment without departing from the scope of the present invention, and the present invention includes such modifications.

1 detection means 11 camera 32 eye condition determination unit 33c abnormality determination unit M driver condition determination device

Claims (4)

In a driver state determination method for detecting an abnormal state of a vehicle driver,
a head posture acquiring step of acquiring the driver's face orientation and head position;
a closed-eye determination step of detecting that the driver's eyes are closed;
a driving operation amount obtaining step of obtaining at least one driving operation amount of the vehicle by the driver at a predetermined timing;
a driver abnormal state determination step of determining whether the driver is in an abnormal state;
The driver's abnormal state determination step includes:
When the closed-eyes state of the driver is not detected in the closed-eyes determination step, the head posture of the driver acquired in the head posture acquisition step is an abnormal posture and the amount of driving operation is less than a predetermined threshold. determining that the driver is in an abnormal state when the situation continues for a first predetermined time;
When the eyes-closed state of the driver is detected in the closed-eyes determination step, the head posture of the driver acquired in the head posture acquisition step is an abnormal posture with a smaller degree of abnormality than the abnormal posture, and A method for determining a driver's state, comprising determining that the driver is in an abnormal state when a situation in which the amount of driving operation is below a predetermined threshold continues for a second predetermined period of time shorter than the first predetermined period of time. 2. The driver according to claim 1, characterized in that, in the driver's abnormal condition determination step, it is determined that the head posture is an abnormal posture based on at least the face angle of the driver. State determination method. 3. The driver state determination method according to claim 1, wherein, in said closed eyes determination step, an alarm is issued after a predetermined period of time has passed since the closed eyes state of the driver is detected. In a driver state determination device for detecting an abnormal state of a vehicle driver,
head posture acquiring means for acquiring the driver's face orientation and head position;
closed-eye determination means for detecting that the driver's eyes are closed;
driving operation amount acquisition means for acquiring at least one amount of driving operation of the vehicle by the driver at a predetermined timing;
a driver abnormal state determination means for determining whether the driver is in an abnormal state;
The driver's abnormal state determination means includes:
When the closed-eye determination means does not detect the closed-eye state of the driver, the head posture of the driver acquired by the head posture acquisition means is an abnormal posture, and the amount of driving operation is less than a predetermined threshold. determining that the driver is in an abnormal state when the situation continues for a first predetermined time;
When the eye-closed determination means detects the eye-closed state of the driver, the head posture of the driver acquired by the head posture acquisition means is an abnormal posture with a smaller degree of abnormality than the abnormal posture, and A driver's state determination device that determines that the driver is in an abnormal state when a situation in which a driving operation amount is below a predetermined threshold continues for a second predetermined time that is shorter than the first predetermined time.

Images