JP2017049636A - Driver state detection device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a driver state detection device capable of rapidly determining a consciousness loss state of a driver with a simple configuration by determining the consciousness loss state of the driver with a posture of the driver as a parameter to determine a consciousness state.SOLUTION: A driver state detection device 1 for detecting a consciousness loss state of a driver of a vehicle includes: a first camera 3 capable of imaging a head of a driver from the front of the driver; a second camera 4 capable of imaging the head of the driver from above the driver; and a state determination section 8c capable of determining a driver state on the basis of first and second captured images of the first and second cameras 3, 4. When both of the first and second captured images of the first and second cameras 3, 4 include the image of a face of the driver or when both of the first and second captured images of the first and second cameras 3, 4 do not include the image of the face of the driver, the state determination section 8c determines that the driver is in a consciousness loss state.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a driver state detection device, and more particularly to a driver state detection device that detects a loss of consciousness of a driver of a vehicle.

Conventionally, a driver's state, in particular, a state in which the driver is conscious, is accurately grasped, and driving operation is supported according to the driver's conscious state.
The emergency evacuation device for a vehicle disclosed in Patent Document 1 includes a physiological state estimation unit that estimates a driver's physiological state, a driver instruction input unit that outputs an execution start command for automatic stop control, and automatic stop control for the driver. A consciousness confirmation unit that confirms the execution of the vehicle and an automatic stop control unit that automatically stops the vehicle. The display position of the driver instruction input unit is changed according to the driver's consciousness level by the physiological state estimation unit. When the vehicle is in the state of loss of consciousness, the automatic stop control is executed with the consciousness confirmation process omitted.
The physiological state estimation unit estimates the driver's consciousness state based on the detected biological information such as the brain wave, heart rate, sweating volume, blood pressure, and components in exhaled breath.

When determining a driver's low arousal state such as a dozing state, the driver's face is imaged from the front of the driver by an imaging means such as a camera, and the degree of eye opening of the driver is detected based on the captured image (Patent Document 2).
Also, paying attention to the fact that the corrected steering angle operation increases in the low awake state while the vehicle is running, the corrected steering amount of the steering wheel and the number of corrected steering operations are compared with a predetermined judgment value, and the corrected steering amount and correction executed. When the number of times of steering is larger than a determination value, a decrease in the driver's arousal level is estimated (Patent Document 3).

Japanese Patent No. 5527411 JP 2008-99884 A JP 2011-39735 A

The emergency evacuation device for a vehicle disclosed in Patent Literature 1 can estimate the driver's consciousness state based on biological information such as the driver's brain wave, heart rate, sweat rate, blood pressure, and components in exhalation.
However, since the vehicle emergency evacuation device disclosed in Patent Document 1 requires a plurality of biological information sensors, a holding unit for these sensors, a harness, and the like, the vehicle operability by the driver may be reduced. Moreover, it is not practical to apply to general vehicles from the viewpoint of increasing the number of parts.

By the way, in the low consciousness state where the driver's consciousness level is low, the low wakefulness state that can be returned to the awakening state by an alarm (stimulation) or the like that is a consciousness confirmation process, There is a state of loss of consciousness that is difficult to wake up and difficult for the driver to drive.
In the state estimation device of Patent Literature 2, there is a possibility that the degree of eye opening cannot be detected depending on the posture of the driver (sitting state with respect to the seat).
In other words, when the driver is in a state of loss of consciousness that is difficult to operate, the driver's face is not always facing the front toward the camera. Can not do it.
In addition, when the driver is in a state of loss of consciousness, even if an alarm is given for a predetermined time in order to wake up the consciousness, the consciousness is not awakened, so that the start of execution of driving assistance such as automatic stopping may be delayed.
Even in the state estimation device of Patent Document 3, since the decrease in the arousal level is estimated by confirming the corrected steering angle operated by the driver, similar to Patent Document 2, it can be expected to promptly perform driving assistance. Can not.

Therefore, as a result of verifying the relationship between the posture of the driver seated in the driver's seat and the driver's consciousness state level, the present inventor is extremely dissatisfied with the normal state when the driver's driving operation is difficult. It has been found that the posture is a forward leaning posture or a backward leaning posture.
This is because the driver is restrained with respect to the driver's seat when the seat belt is worn, and thus the posture of the driver who has lost his or her consciousness is mainly specified in the two forms described above by the pressing force of the seat belt on the driver's seat. It is guessed.

  An object of the present invention is to provide a driver state detection device or the like that can quickly determine a driver's loss of consciousness state with a simple configuration.

  The driver state detection device according to claim 1 is a driver state detection device that detects a loss of consciousness of the driver of the vehicle. The first imaging means capable of imaging the driver's head from the front of the driver, and the driver A second imaging unit capable of imaging the driver's head from above, and a state determination unit capable of determining the driver state based on the captured images of the first and second imaging units, the state determination unit comprising: When the captured images of the first and second imaging means both include an image of the driver's face, or when the captured images of the first and second imaging means do not include the image of the driver's face, It is characterized by determining that the driver is in an unconscious state.

In this driver state detection device, first imaging means capable of imaging the driver's head from the front of the driver, second imaging means capable of imaging the driver's head from above the driver, first and first 2 is provided with a state determination unit capable of determining the driver state based on the captured image of the image pickup unit, so that the face direction in which the driver's face is directed can be determined by the minimum number of image pickup units. Based on the direction, the forward leaning posture or the backward leaning posture of the driver can be determined.
When the state determination means includes both the captured images of the first and second imaging means including the driver's face image, or both the captured images of the first and second imaging means include the driver's face image. When it is not, since it is determined that the driver is in an unconscious state, it is possible to determine an unconscious state in which it is difficult for the driver to drive based on the actual posture of the driver.

According to a second aspect of the present invention, in the first aspect of the invention, the state determination unit includes a captured image of the first imaging unit including an image of a driver's face, and a captured image of the second imaging unit is a driver's face. When the image of the face is not included, the driver is determined to be in a conscious awakening state.
According to this configuration, the driver's consciousness awakening state can be accurately determined with a simple configuration.

A third aspect of the invention is characterized in that, in the first or second aspect of the invention, the state determining means causes the vehicle to stop urgently when it is determined that the driver is in an unconscious state.
According to this configuration, the vehicle can be safely stopped even when the driver is in an unconscious state.

According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, when the state determination means determines that the driver is in an unconscious state, the emergency determination state is notified to the outside. It is a feature.
According to this configuration, even when the driver is in an unconscious state, the driver can be rescued by reporting the emergency abnormal state to the outside.

  According to the driver state detection device of the present invention, by determining the driver's loss of consciousness state using the driver's posture as a consciousness state determination parameter, the driver's loss of consciousness state can be quickly determined with a simple configuration. be able to.

1 is a block diagram of a driver state detection device according to a first embodiment. The driver's seating model in a conscious awakening state is shown, (a) is a front view, (b) is a side view, and (c) is a plan view. The driver's seating model of a 1st consciousness loss state is shown, (a) is a front view, (b) is a side view, (c) is a top view. The seating model of the driver | operator in the 2nd consciousness loss state is shown, (a) is a front view, (b) is a side view, (c) is a top view. The captured image of the driver | operator of the conscious awakening imaged with the 1st, 2nd camera is shown, (a) is a 1st captured image, (b) is an example of a 2nd captured image. The captured image of the driver | operator of the 1st loss of consciousness imaged with the 1st, 2nd camera is shown, (a) is a 1st captured image, (b) is an example of a 2nd captured image. The captured image of the driver | operator of the 2nd loss of consciousness imaged with the 1st, 2nd camera is shown, (a) is a 1st captured image, (b) is an example of a 2nd captured image. It is a flowchart which shows a driver | operator state detection process. It is a flowchart which shows a driver | operator state determination process.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The following description exemplifies a case where the present invention is applied to a vehicle, and does not limit the present invention, its application, or its use.

Embodiment 1 of the present invention will be described below with reference to FIGS.
The driver state detection device 1 is a driving support device that can determine a driver's consciousness state, particularly a loss of consciousness state in which it is difficult for the driver to drive the vehicle (also referred to as the host vehicle).
As shown in FIG. 1, the driver state detection device 1 includes a vehicle exterior camera 2 that captures a surrounding landscape including the front of the vehicle, and a vehicle interior that captures the head of the driver seated in the driver seat while wearing a seat belt. The camera includes first and second cameras 3 and 4, a navigation system 5, an operation control unit 6, a communication unit 7, an ECU (Electronic Control Unit) 8, and the like.

  The outside camera 2 is mounted in the vicinity of a rearview mirror (not shown) on the lower surface of the front end side of the roof panel, for example, and can be a CCD (Charge Coupled Device) that can capture the front of the vehicle body as a still image or a moving image through a front window glass (not shown). ) Consists of cameras.

  The first camera 3 is constituted by a CCD camera, and is mounted, for example, at a driver-facing position above an instrument panel (not shown). As shown in FIGS. 2 to 4, the first camera 3 is formed such that the head including the face of the driver who is seated in the driver's seat and wears the seat belt can be imaged from the front in front. The image is configured so that the image of the driver's face can be cut out and the orientation of the face, identification of the driver's facial expression, the iris of the eye (iris) can be enlarged and the pupil size and gaze direction can be specified, etc. Has been.

  The second camera 4 is constituted by a CCD camera, and is mounted on, for example, a roof panel (not shown) directly above the driver's seat. As shown in FIGS. 2 to 4, the second camera 4 is formed so that the upper surface of the head including the face of the driver sitting on the driver's seat and wearing the seat belt can be imaged from directly above, and the imaged head The partial image is configured to be able to identify the face portion of the driver and the portion other than the face.

The navigation system 5 is a system that provides vehicle route guidance.
The navigation system 5 is configured to be able to detect current position information of the host vehicle by receiving position signals from a plurality of GPS satellites via an external information center 9.
The navigation system 5 includes a map database that stores road map data, a traffic rule database that stores traffic rule data (both not shown), and the like.
Thereby, the navigation system 5 performs route guidance to the driver to the destination using the current position data of the own vehicle, the road map data of the map database, and the traffic rule data of the traffic rule database.

The driving control unit 6 is a drive mechanism or a steering mechanism for executing traveling control of the host vehicle.
The operation control unit 6 includes an engine control unit, a steering actuator, a brake actuator, a shift actuator, and the like.
When an emergency command is input from the ECU 8, the operation control unit 6 executes traveling control of the host vehicle. This traveling control is, for example, emergency evacuation to a safety zone such as a road edge side by an emergency stop or automatic driving. Since the automatic driving is a known technique that is executed based on detection information of the preceding vehicle, the oncoming vehicle, the traveling lane, and the like, detailed description thereof is omitted.

  The communication unit 7 includes an antenna (not shown) and the like, and is configured to be able to transmit and receive with the information center 9. When an emergency command is input from the ECU 8, the communication unit 7 transmits an emergency rescue signal for notifying an emergency abnormal state, the vehicle ID of the host vehicle, the host vehicle position information, and the driver's posture state (forward leaning). The information center 9 is urgently notified of information such as the prone state of the posture or the back-facing state of the tilted posture.

Next, the ECU 8 will be described.
As shown in FIG. 1, the ECU 8 includes an image processing unit 8a, an emotion estimation unit 8b that determines the driver's emotion, a state determination unit 8c that determines the driver's loss of consciousness, and the like.
The ECU 8 is an electronic control unit including a CPU, a ROM, a RAM, and the like, and performs various arithmetic processes by loading an application program stored in the ROM into the RAM and executing it by the CPU.

The image processing unit 8a performs image processing of image data captured by the outside camera 2 and the first and second cameras 3 and 4, temporary storage of image data, compression / decompression processing of image data, and synthesis of a plurality of captured images. It is configured to be possible.
Based on the captured image of the camera 21 outside the vehicle, a preceding vehicle preceding the own vehicle, an oncoming vehicle traveling in the opposite lane, a traveling lane in which the own vehicle is traveling, a roadside zone, and the like are detected.
The preceding vehicle and the oncoming vehicle are extracted by pattern matching with a vehicle determination template stored in advance. The white line part of the driving lane and the roadside band is estimated by using the fact that edges that are high frequency components are generated at both ends of the white line by differentiating the brightness of the front image in the horizontal direction. The white line is extracted from the estimated white line portion based on the threshold value determined from the brightness and the contrast with the road surface, the threshold value of the white line width, and the like.

The image processing unit 8a detects the driver's head seated on the driver's seat and the driver's face based on the first captured image from the first camera 3, and performs face recognition processing for each first image data.
For example, the face recognition recognizes the position of the eyes, the position of the nose, the position of the mouth, and the positional relationship thereof, and compares the numerical data with the face determination data stored in advance to recognize the face of the driver. I do. Further, based on the first image data whose face is recognized by the first camera 3, the number of gaze direction changes and the pupil diameter are detected.
The image processing unit 8a detects the driver's head and the driver's face seated in the driver's seat based on the second image captured by the second camera 4, and performs face recognition processing for each second image data.
For example, the face recognition recognizes the position of the eyes, the position of the nose, the position of the mouth, and the positional relationship thereof, and compares the numerical data with the face determination data stored in advance to recognize the face of the driver. I do.

The emotion estimation unit 8b determines the current emotion of the driver based on the face image data from the first camera 3. A facial expression master image ordered corresponding to each emotion of the driver is prepared in advance, and when the facial image data and the facial expression master image are compared and matched with a specific facial expression master image, the driver's emotion is the specific emotion It is estimated that. The estimated driver's feeling is corrected by the number of gaze direction changes and the pupil diameter.
When the driver's emotion is deteriorated based on the determination result of the driver's emotion, various in-vehicle devices are controlled in a direction to improve the driver's emotion (see Japanese Patent Application No. 2015-15334).

The state determination unit 8c determines a loss of consciousness state in which it is difficult for the driver to drive the vehicle based on the face recognition results of the first and second cameras 3 and 4.
When the first captured image includes the driver's facial image and the second captured image does not include the driver's facial image, the state determination unit 8c When both the first and second captured images do not include the driver's facial image, the driver is in a first consciousness state in which the driver is leaning forward, and both the first and second captured images include the driver's facial image. When the driver determines the second state of loss of consciousness in a backward tilted posture, and when one of the first and second consciousness loss states is determined, the driving control unit 6 and the communication unit 7 An emergency command is output.

Here, based on FIGS. 2-7, the conscious state of the driver | operator in boarding is demonstrated.
The driver's consciousness level includes the consciousness-awakening state in which the vehicle can be driven normally, the low-awakening state in which the vehicle can return to the consciousness-awakening state by stimulating alarms, etc. There is a state of loss of consciousness that is difficult and difficult for the driver to drive.
As shown in FIGS. 2 (a) to 2 (c), when the driver is in a consciousness awakening state, the driver directs his / her line of sight in the vehicle traveling direction. In addition, even in a low arousal state due to sleepiness, since it does not become an extreme forward leaning posture or a backward leaning posture, it can be classified into a driver posture substantially similar to the conscious awakening state. At this time, as shown in FIG. 5A, the driver's face is recognized in the first captured image, and as shown in FIG. 5B, the driver's head upper surface is displayed in the second captured image. Recognized.

As shown in FIGS. 3 (a) to 3 (c), when the driver is in the first loss of consciousness, the driver is in a prone and extremely leaning posture with his head held on the steering wheel. . At this time, as shown in FIG. 6A, the upper surface of the driver's head is recognized in the first captured image, and as shown in FIG. 6B, the driver's head is displayed in the second captured image. The back is recognized.
Further, as shown in FIGS. 4A to 4C, when the driver is in the second consciousness loss state, the driver assumes an extremely backward tilted posture in which the waist is shifted forward. At this time, as shown in FIG. 7A, the driver's face is recognized in the first captured image, and as shown in FIG. 7B, the driver's face is recognized in the second captured image. ing.
Thus, the driver's consciousness loss state can be accurately determined using the driver's posture, in other words, the direction in which the driver's face is directed as an index.

Next, the driver state detection process will be described based on the flowcharts of FIGS. Si (i = 1, 2,...) Indicates a step for each process.
First, as shown in FIG. 8, in the driver state detection device 1, outputs from various devices such as a driver seat seating sensor and a seat belt mounting sensor are input (S1), and the driver seat seating and seat belt mounting are performed. The process proceeds to S2 as a condition.
In S2, a driver state determination process is performed. In the driver state determination process, it is determined whether the driver's consciousness state is any of the consciousness awakening state, the first consciousness loss state, or the second consciousness loss state.

After the driver state determination process, the process proceeds to S3 to determine whether the driver's consciousness state is the first consciousness loss state or the second consciousness loss state.
As a result of the determination in S3, when the driver's consciousness state is the first consciousness loss state or the second consciousness loss state, the process proceeds to S4 to stop the vehicle urgently. As a result of the determination in S3, if the driver's consciousness state is not the first consciousness loss state or the second consciousness loss state, the process returns.
After the emergency stop of the vehicle, the process proceeds to S5, a rescue request is notified to the information center 9, and the process returns.

Next, the driver state determination process will be described.
As shown in the flowchart of FIG. 9, the heads of the seated drivers are respectively imaged by the first and second cameras 3 and 4 (S10), and the process proceeds to S11.
In S11, it is determined whether or not the driver's head is recognized in the first captured image captured by the first camera 3, in other words, whether or not the driver's head is present.
As a result of the determination in S11, when the driver's head is present in the first captured image, the process proceeds to S12 to determine whether the driver's face is present in the first captured image.

As a result of the determination in S12, if the driver's face is present in the first captured image, the process proceeds to S13, and whether the driver's face is present in the second captured image captured by the second camera 4 or not. Determine whether.
As a result of the determination in S13, when the driver's face is present in the second captured image, the driver is in the extremely backward tilted posture shown in FIGS. Determine the disappearance state and end.
As a result of the determination in S13, when the driver's face is not present in the second captured image (the head is present), the driver is in the normal posture shown in FIG. 2 and FIG. Then, the state of conscious awakening is determined and the process ends.

As a result of the determination in S12, when the driver's face is not present in the first captured image (the head is present), the process proceeds to S16, and the driver's face is captured in the second captured image captured by the second camera 4. It is determined whether or not a face exists.
As a result of the determination in S16, if the driver's face is present in the second captured image, the driver returns to S10 because the driver is in a unique posture.
If the driver's face is not present in the second captured image (the head is present) as a result of the determination in S16, the driver is in the extreme forward leaning posture shown in FIG. 3 and FIG. It shifts, it determines with a 1st consciousness loss state, and complete | finishes.

Next, the operation and effect of the driver state detection device 1 will be described.
According to the driver state detection device 1, the first camera 3 that can image the driver's head from the front of the driver, the second camera 4 that can image the driver's head from above the driver, 1 and a state determination unit 8c capable of determining the driver state based on the first and second captured images of the second cameras 3 and 4, so that the face direction to which the driver's face is directed is minimized. The determination can be made by the first and second cameras 3 and 4, and the forward leaning posture or the backward leaning posture of the driver can be determined based on the face direction. When the state determination unit 8c includes both the first and second captured images of the first and second cameras 3 and 4 including the image of the driver's face, or the first and second of the first and second imaging means. When both the captured images do not include the image of the driver's face, it is determined that the driver is in a state of loss of consciousness. Can be determined.

  When the first captured image of the first camera 3 includes an image of the driver's face and the second captured image of the second camera 4 does not include the image of the driver's face, the state determination unit 8c recognizes the driver. Since it is determined that the state is an awakening state, the driver's awareness state can be accurately determined with a simple configuration.

  When the driver determines that the driver is in an unconscious state, the state determiner 8c stops the vehicle urgently, so that the vehicle can be stopped safely even if the driver is in an unconscious state.

  When the driver determines that the driver is in a state of loss of consciousness, the state determination unit 8c reports the emergency abnormal state to the outside. The driver can be rescued.

Next, a modified example in which the embodiment is partially changed will be described.
1) In the above-described embodiment, the example in which the first camera having a higher performance than the second camera is used for estimating the driver's emotion in order to detect the driver's facial expression has been described. By omitting, it is possible to reduce the cost by making both the first and second cameras functions to perform only face recognition.

2) In the above embodiment, the example in which the low wakefulness state is included in the consciousness wakefulness state has been described. However, when the consciousness wakefulness state and the low wakefulness state are separated and the low wakefulness state is detected, the driver's consciousness wakefulness is detected. It can also be used in combination with a conscious awakening mechanism.

3) In addition, those skilled in the art can implement the present invention with various modifications added without departing from the spirit of the present invention, and the present invention includes such modifications. is there.

DESCRIPTION OF SYMBOLS 1 Driver state detection apparatus 3 1st camera 4 2nd camera 8 ECU
8c State determination unit

Claims (4)

In the driver state detection device that detects the loss of consciousness of the driver of the vehicle,
First imaging means capable of imaging the driver's head from the front of the driver;
A second imaging means capable of imaging the driver's head from above the driver;
A state determination unit capable of determining a driver state based on the captured images of the first and second imaging units;
The state determination means is configured such that when both the captured images of the first and second imaging means include an image of the driver's face, or both the captured images of the first and second imaging means are images of the driver's face. When the vehicle is not included, it is determined that the driver is in a state of loss of consciousness.   When the captured image of the first imaging unit includes an image of the driver's face and the captured image of the second imaging unit does not include the image of the driver's face, the state determination unit The driver state detection device according to claim 1, wherein the driver state detection device is determined to be.   3. The driver state detection device according to claim 1, wherein when the driver determines that the driver is in a state of loss of consciousness, the driver determines the state of emergency.   The driver state detection device according to any one of claims 1 to 3, wherein when the driver determines that the driver is in a state of loss of consciousness, the driver determines the emergency abnormal state to the outside. .