JP2000280780A - Driver state detecting system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce a mental sense of unease imparted to a driver, and to improve physical safety by controlling starting/stopping of a face recognizing device including irradiation of an infrared LED arranged to support driving of the driver according to, for example, a vehicle operational state and a vehicle peripheral situation. SOLUTION: A driver state detecting system has an infrared LED 3 for irradiating a face of a driver D in a vehicle 1 by the infrared illuminating light and a CCD camera 4 for picking up the illuminated face of the driver D. A recognizing device 15 controls an LED driving processing part 10 and a camera driving processing part 11, and detects a state of the driver D by a recognizing processing part 12 on the basis of a picked-up face image of the driver D to perform warning output processing by a warning output part 13 according to a state of the driver D. When a state is not either of an engine stopping state and an idling state in a controller 17, the vehicle 1 travels, and it is judged that driver state detecting processing is required, so that a driving staring command is transmitted to the recognizing device 15.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a driver state detection system which assists driving of a driver by capturing a driver's face and detecting the driver's state from the image.

[0002]

2. Description of the Related Art In order to prevent an accident caused by a driver (driver) falling asleep or looking aside, the driver's face is constantly illuminated and an image is taken by an imaging device such as a CCD camera. An image processing unit (CPU) performs image processing such as binarization processing based on the image, thereby recognizing a driver's eye position, blinking, opening degree, and the like, thereby recognizing the driver's state. There is a known driver state detection system.

According to this driver state detection system,
The recognition device, for example, when detecting the driver's dozing driving state from the determination of the number of eye blinks or the degree of opening based on the image processing, or when detecting the inattentive driving state from the determination of the position of the eyeball, The driver is alerted by outputting an alarm such as an alarm sound to the driver.

In the driver state detection system having the above-described recognition device, it is necessary to be able to cope with changes in brightness in a driving environment due to day and night, weather, and the like. For example, direct sunlight can create shadows on the driver's face,
There is a possibility that the detection of the state of the driver by image processing will be hindered.

[0005] Therefore, in order to be able to take images day and night, and to reduce the influence of external disturbance light on the face (illuminance change during the day and night, direct sunlight, etc.), the light for illuminating the driver's face is used as ambient light. Infrared illumination light, which is hardly affected, is used, and by constantly irradiating the infrared illumination light to the driver's face, the accuracy of detecting the state of the driver is improved.

[0006]

However, according to the driver state detection system having the conventional recognition device,
This means that the driver is constantly irradiating the driver with infrared illumination light. If this infrared ray is continuously radiated to the eyes, it may cause visual impairment depending on the intensity. .

[0007] Even in non-emergency situations or when the driver is aware, the durability of the components of the recognition device deteriorates because the face image is always taken by the CCD camera and the CPU executes the image processing. However, there is a fear that the cost may increase.

[0008] The present invention has been made in view of the above-mentioned circumstances, and performs start / stop control of a face recognition device including irradiation of an infrared LED according to, for example, the operating state of a vehicle and the situation around the vehicle. It is an object of the present invention to provide a driver state detection system capable of reducing a feeling of psychological anxiety given to a driver, improving physical safety, and further improving durability of a recognition device.

[0009]

According to a first aspect of the present invention, there is provided a driver status detection system for detecting a status of a driver driving a vehicle, wherein a driver's face is imaged while being illuminated. A recognition device that recognizes the state of the driver based on the obtained image, a traveling state detection unit that detects a traveling state of the vehicle, and control that performs drive control of the recognition device in accordance with the detected traveling state. Means.

[0010] In the first invention, the recognition device includes an illuminator for illuminating an infrared light emitting device to illuminate a driver's face with infrared light, an imaging device for imaging the face illuminated by the illuminator, A recognition unit that recognizes the state of the driver based on an image captured by the device, wherein the control unit controls the turning on and off of the infrared light emitting device of the lighting unit in accordance with the detected traveling state. Means to do so.

According to the second aspect of the present invention, in a driver state detection system for detecting a state of a driver driving a vehicle, an image is taken while illuminating a driver's face, and an obtained image is obtained. A recognition device for recognizing a state of the driver based on the vehicle, a surrounding situation detecting means for detecting a situation around the vehicle, and a control means for controlling driving of the recognition device in accordance with the detected surrounding situation. I have.

[0012] In the second invention, the recognizing device includes an illuminating section for illuminating an infrared light emitting device to illuminate a driver's face with infrared light, an imaging device for imaging the face illuminated by the illuminating section, A recognition unit that recognizes the state of the driver based on an image captured by the device, wherein the control unit controls the turning on and off of the infrared light emitting device of the lighting unit according to the detected surrounding situation. Means to do so.

[0013]

Embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is a block diagram showing a schematic configuration of a driver state detection system 2 according to the present embodiment mounted in a vehicle 1. As shown in FIG.

Referring to FIG. 1, a driver state detection system 2 includes an infrared LED, which is an infrared light emitting device for irradiating the face of a driver D in a vehicle 1 with infrared illumination light.
3 and a CCD camera 4 as an imaging device for imaging the face of the driver D illuminated by the infrared illumination light.

The driver status detecting system 2 includes an LED drive processing unit 1 for turning on / off the infrared LED 3.
0, a camera drive processing unit 11 for performing a drive process (face image capturing process) of the CCD camera 4, an image such as a binarization process based on a face image of the driver D captured by the CCD camera 4, a pattern matching based on grayscale information, and the like. A recognition processing unit (CPU) 1 that recognizes the position of the eyes of the driver D, blinks, the degree of opening, and detects the state of the driver by performing the processing.
2 and a recognition device 15 having an alarm output unit 13 for performing an alarm output process such as an alarm sound in accordance with the detected driving state of the driver D.

Further, the driver state detection system 2
A controller 17 is provided for controlling the drive processing timing for turning on / off the infrared LED 3 of the ED drive processing unit 10 and controlling the drive processing timing (imaging timing) of the camera drive processing unit 11 for the CCD camera 10.

The controller 17 includes a rotation speed detection sensor 20 for detecting the rotation speed of the drive engine of the vehicle 1,
An angle sensor 21 that detects a change in the steering wheel rotation angle when the driver D operates the steering wheel H, and detects a horn when the horn is emitted from the other vehicles around the own vehicle 1 to the own vehicle 1. The sound sensors 22 are connected to each other, and detection signals from the sensors 20 to 22 are sent to the controller 17.

Next, the overall operation of the driver state detection system 2 according to this embodiment will be described.

The driver state detection system 2 of the present embodiment
According to this, the controller 17 instructs the recognition device 15 to start driving, that is,
An LED lighting control command is transmitted to the LED driving processing unit 10 and a CCD camera driving command is transmitted to the camera driving processing unit 11.

As a result, the infrared LED 3 is turned on by the lighting drive processing of the LED drive processing unit 10 to illuminate the face of the driver D, and the camera drive processing unit 1 is illuminated in the illuminated state.
By the processing of step 1, the CCD camera 4 is driven, and images of the face of the driver D are continuously photographed. Then, image processing is performed via the recognition processing unit 12 based on the captured face image of the driver D, and the state detection processing of the driver D is performed.

At this time, the controller 17 performs the processing shown in FIG. 2 according to the detection signal (rotation speed) sent from the rotation speed detection sensor 20.

That is, based on the engine speed of the running vehicle 1 sent from the speed sensor 20, the controller 17 determines whether the engine is stopped or waits for a traffic signal or a red signal. It is determined whether or not the vehicle is idling while stopped (step S
1).

If the result of the determination in step S1 is NO, that is, if neither the engine stopped state nor the idling state is present, it is determined that the host vehicle 1 is running and it is necessary to perform the driver state detection process, and The process of S1 is repeated.

On the other hand, if the result of the determination in step S1 is YES,
That is, for example, when the running vehicle 1 is stopped due to traffic congestion or waiting for a traffic light by a red light and the engine is in an idling state, or when the own vehicle 1 is completely stopped and the engine is in a stopped state, The controller 17 issues a drive stop command to the recognition device 15, that is, an LED turn-off control command to the LED drive processing unit 10 and a camera drive processing unit 11.
, A CCD camera driving stop command is transmitted (step S2).

As a result, the infrared LED 3 is turned off by the turn-off driving process of the LED drive processing unit 10,
The driving of the D camera 4 stops, and the imaging of the face of the driver D stops.

Subsequently, the controller 17 starts driving the engine based on the engine speed sent from the speed sensor 20 or ends the idling state due to the elimination of traffic congestion or a change in the green signal of the signal. It is determined whether or not this is the case (step S3).

If the result of the determination in step S3 is NO, that is, if the engine is in either the engine stopped state or the idling state, the host vehicle 1 is in the stopped state or in the traffic congestion road / signal waiting state, and the driver state detection processing is performed. Is determined to be unnecessary, and the process of step S3 is repeated.

On the other hand, if the result of the determination in step S3 is YES,
That is, for example, when the host vehicle 1 that is stopped has started running (engine driving has started), or when the idling state has ended due to the elimination of traffic congestion or a change in the green signal of the signal, the controller 17 executes the LED driving process. LE in part 10
D lighting control command and C
Each of the CD camera drive commands is transmitted (step S
4), end the process.

As a result, the infrared LED 3 is turned on again by the lighting drive processing of the LED drive processing section 10 to illuminate the face of the driver D, and the CCD camera 4 is driven again by the processing of the camera drive processing section 11 in the illuminated state. Thus, the image of the face of the driver D is continuously photographed. Then, the image processing of the recognition processing unit 12 based on the captured face image of the driver D is restarted, and the state detection processing of the driver D is performed again.

That is, according to the present embodiment, the host vehicle 1
Is stopped (stopped), or in a state of traffic congestion, signal waiting, or the like, the degree of necessity of performing the face recognition processing of the driver D is determined to be low, and the driving of the recognition device 15 is stopped, that is, the infrared LED 3 is turned off. Since it is possible to stop imaging of the CCD camera 4 and stop recognition processing of the recognition processing unit 12, the infrared LED 3 is prevented from constantly illuminating the face of the driver D, and the driver D's psychological anxiety And the physical safety of the driver D can be improved.

Further, according to the present embodiment, the infrared ray LE
D3, CCD camera 4 and recognition processing unit (CPU) 12
Can be driven only when the degree of necessity is high (during driving) without constantly driving
It is possible to avoid wasteful use of the infrared LED 3, the CCD camera 4, and the CPU 12, improve the durability thereof, and reduce the cost of the driver state detection system 2.

On the other hand, in the present embodiment, the controller 17
Performs the processing shown in FIG. 3 by interrupt processing in addition to the drive start / stop processing of the recognition device 15 shown in FIG. 2 according to the rotation speed from the rotation speed detection sensor 20.

That is, the controller 17 determines the engine speed of the running vehicle 1 sent from the speed detection sensor 20 and the timer function of the controller itself.
The speed of the host vehicle 1 is a predetermined speed (for example,
(At least 80 km) is determined (step S10).

If the result of determination in step S10 is NO, that is, if the vehicle 1 is traveling on a general road,
It is determined that the driver state detection processing needs to be performed, and the processing of step S10 is repeated.

On the other hand, for example, when the vehicle 1 is traveling on a highway, the result of the determination in step S10 is YES.
The controller 17 transmits an LED turn-off control command to the LED drive processing unit 10 and transmits a CCD camera drive stop command to the camera drive processing unit 11 (step S11).

As a result, the infrared LED 3 is extinguished by the extinguishing drive processing of the LED drive
The driving of the D camera 4 stops, and the imaging of the face of the driver D stops.

Next, the controller 17 determines whether or not a steering angle change signal has been transmitted from the angle sensor 21 (step S12).

If the result of the determination in step S12 is NO, that is, if the steering angle change signal is not transmitted from the angle sensor 21, the driver D does not rotate the steering wheel, and there is no need to perform the driver state detection processing. And the process of step S12 is repeated.

On the other hand, as a result of the determination in step S12, YE
S, that is, when the driver D rotates the steering wheel and the steering angle change signal is transmitted from the angle sensor 21, the controller 17 sends the LED drive processing unit 10
A D-light control command is transmitted, and a CCD camera drive command is transmitted to the camera drive processing unit 11 (step S1).
3) End the interrupt processing and return the processing to main.

As a result, the face image of the driver D illuminated by the re-lighting of the infrared LED 3 is continuously photographed by the CCD camera 4, and the state detection processing of the driver D based on the image processing of the recognition processing unit 12 is performed again. .

That is, according to the present embodiment, the host vehicle 1
While the vehicle is traveling, for example, while traveling on a highway, while the driver D does not perform the steering wheel angle operation, it is determined that the degree of necessity of performing the face recognition processing of the driver D is weak, and the driving of the recognition device 15 is performed. Stop, ie infrared LED
3 can be realized, the imaging of the CCD camera 4 can be stopped, and the recognition processing of the recognition processing unit 12 can be stopped. Of the recognition device 15 (infrared LED 3,
It is possible to avoid continuous driving of the CCD camera 4 and the recognition processing unit 12), thereby contributing to improvement in durability and cost reduction.

Further, in this embodiment, the controller 1
7 is when the driving of the recognition device 15 shown in FIG. 2 is stopped (judgment in steps S2 to S3 → NO) and when the driving of the recognition device 15 shown in FIG. 3 is stopped (steps S11 to S11).
In the determination of NO → 12), for example, the process shown in FIG. 4 is performed by an interrupt process.

That is, the controller 17 determines whether or not a detection signal based on horn for the vehicle 1 has been sent from the sound sensor 22 (step S2).
0).

If the result of the determination in step S20 is NO, that is, if it is determined that the horn is not sounding for the host vehicle 1 from other vehicles around the host vehicle 1, the host vehicle 1 detects the driver state. It is determined that there is no need to perform the process, and the process of step S20 is repeated.

On the other hand, as a result of the determination in step S20, YE
S, that is, when it is determined that a horn has sounded to the host vehicle 1 from another vehicle around the host vehicle 1, the controller 17 instructs the recognition device 15 to start driving, that is, the LED drive processing unit 10. , And transmits a CCD camera drive command to the camera drive processing unit 11 (step S21).

As a result, the infrared LED 3 is turned on by the lighting drive processing of the LED drive processing unit 10, and
The driving of the D camera 4 starts, and the imaging of the face of the driver D starts.

Subsequently, the controller 17 determines whether the horn of the own vehicle 1 from the other vehicle has been completed (step S22).

When the result of the determination in step S22 is NO, that is, when the horn is sounding on the host vehicle 1, it is determined that the driver state detection processing needs to be continuously performed, and the processing in step S22 is performed. repeat.

On the other hand, as a result of the determination in step S22, YE
S, that is, when the horn output to the own vehicle 1 is completed, the controller 17 transmits an LED turn-off control command to the LED drive processing unit 10 and a CCD camera drive stop command to the camera drive processing unit 11, respectively. (Step S23), the interrupt process is terminated, and the process returns to the main process.

That is, according to the present embodiment, when the horn is sounded from the other vehicle to the own vehicle 1 while the driving of the recognition device 15 is stopped, it is necessary to perform the face recognition processing of the driver D. It is determined that the degree is high, and the driving of the recognition device 15 is started (that is, the infrared LED
3, the start of imaging by the CCD camera 4 and the start of recognition processing by the recognition processing unit 12). In other cases, the necessity of the face recognition processing is determined to be low, and the infrared Since it is possible to turn off the LED 3 and the like, it is possible to reduce the psychological anxiety of the driver D caused by the constant irradiation of the infrared LED 3, to improve the physical safety thereof, and to make each component (infrared ray LED 3, CCD camera 4, and recognition processing unit 1
2) can be avoided to contribute to improvement of durability and cost reduction.

In the embodiment described above, the sensor 20
Although the control of the recognition device 15 is performed in accordance with the detection signals from the detection devices 15 to 22, the present invention is not limited to this.
The imaging process of the CCD camera 4 and the recognition process of the recognition processing unit 12 may be performed at appropriate time intervals (periodically).

That is, as shown in FIG. 5, the controller 17 determines whether or not a predetermined time (standby time) has elapsed by a timer function (step S30).
If the time has not elapsed (NO in step S30), the timer is updated in step S31, the process returns to the determination in step S30, and the above-described determination process is repeated.

On the other hand, if the standby time has elapsed, the controller transmits an LED lighting control command to the LED drive processing unit 10 and a CCD camera drive command to the camera drive processing unit 11 (step S32). As a result, the infrared LED 3 is turned on by the lighting drive processing of the LED drive processing unit 10, and the driving of the CCD camera 4 is started to start capturing the face of the driver D.

Based on the face image of the driver D captured by turning on the infrared LED 3 and driving the CCD camera 4, the recognition processing unit 12 performs image processing, and performs face recognition processing (state detection processing) of the driver D. This is performed for a fixed time (step S33).

At this time, if the recognition processing unit 12 recognizes that the state of the driver D is normal, that is, that the driver D does not fall asleep, even after performing the state detection processing based on the face state of the driver D for a certain period of time, The recognition device 15 determines whether a preset recognition time has elapsed (step S3).
4).

If the result of determination in step S34 is NO, that is, if the recognition time has not elapsed, or if it is determined that driver driving state detection processing based on face recognition processing is necessary (step S34 → NO) In step S35, the timer is updated, and the process returns to step S34 to repeat the above-described determination process.

On the other hand, as a result of the determination in step S34, YE
S, that is, when the recognition time has elapsed, the controller 17 issues a drive stop command to the recognition device 15, that is,
An LED turn-off control command is sent to the LED drive processor 10 and a CCD camera drive stop command is sent to the camera drive processor 11 (step S36).

As a result, the infrared LED 3 is extinguished by the extinguishing drive processing of the LED drive
The driving of the D camera 4 stops, and the imaging of the face of the driver D stops.

After the processing in step S36, the controller 1
7 returns to the timer processing of step S30, and repeats the processing of steps S30 to S36 described above.

That is, according to the present embodiment, the driving of the recognition device 15 is started every time a preset standby time elapses,
During the standby time, the driving of the recognition device 15 can be stopped, that is, the infrared LED 3 can be turned off, the imaging of the CCD camera 4 can be stopped, and the recognition processing of the recognition processing unit 12 can be stopped. Driver D
To reduce the psychological anxiety about the body, improve its physical safety, and avoid the continuous driving of each component (infrared LED 3, CCD camera 4, recognition processing unit 12, etc.) of the recognition device 15 and endurance. It can contribute to improvement of performance and cost reduction.

In this embodiment, the controller 17
Performs the processes shown in FIGS. 2 to 4 so that the recognition device 15 is driven non-continuously (such as turning on the infrared LED 3) according to its necessity. Any one of the processes in FIGS. 2 to 4 may be performed.

In this embodiment, the infrared light L
Although the description has been made mainly on the control of turning on and off the ED, the present invention is not limited to this. The lighting intensity control of the infrared LED is performed periodically, according to the operation state of the own vehicle, the surrounding situation, and the like. You may.

Since the effect on the human eye changes depending on the intensity of the infrared illumination, the influence can be reduced by adjusting the intensity of the illumination.

For example, according to the external lighting conditions, the control is performed to reduce the intensity of the illumination when the outside is bright, or to increase the intensity of the illumination only when it is desired to perform recognition with high accuracy. You may.

Further, in this embodiment, by detecting the number of revolutions of the engine of the own vehicle 1, the own vehicle 1
The operation state of the vehicle 1 is detected and specified, for example, whether the vehicle is stopped, (2) waiting for a signal or traffic, or (3) high-speed driving, and the recognition device 15 is driven in accordance with the specified operation state. Although the start / stop control is performed, the present invention is not limited to this.

For example, as shown in FIG. 6, VICS (Vehicle I) installed at predetermined intervals on the road side of the road.
nformation Communication
The traffic information, traffic congestion information, and the like may be obtained from the System 23, and the current operation state of the host vehicle 1 and the situation around the host vehicle may be detected and specified according to the obtained traffic information, traffic congestion information, and the like. In this case, VIC is provided to the controller 17A.
S (such as traffic information and congestion information transmitted from S)
A function of wirelessly receiving an M multiplex signal, an optical signal, or the like) may be provided.

In this embodiment, the controller 17
Is a driving timing control for turning on / off the infrared LED 3 of the LED driving processing unit 10 in accordance with a running state of the host vehicle 1 and a surrounding state such as a horn of the host vehicle 1, and a CCD camera of the camera driving processing unit 11. Although the drive processing timing (imaging timing) control is performed for each of the LED 10, the present invention is not limited to this.
Only the drive processing timing control for turning on / off light may be performed.

[0069]

As described above, according to the driver state detection system of the present invention, the control of turning on / off the infrared light emitting device and the adjustment of the intensity according to the operating state of the own vehicle or the situation around the own vehicle. Since the driving control (start / stop control) of the recognition device including the control can be executed, it is possible to prevent the infrared light emitting device from constantly illuminating the driver's face, thereby reducing the driver's psychological anxiety. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a schematic configuration of a driver state detection system detection device according to an embodiment of the present invention.

FIG. 2 is a schematic flowchart illustrating an example of a process of a controller illustrated in FIG. 1;

FIG. 3 is a schematic flowchart illustrating an example of a process of a controller illustrated in FIG. 1;

FIG. 4 is a schematic flowchart illustrating an example of a process of a controller illustrated in FIG. 1;

FIG. 5 is a schematic flowchart showing an example of a process of the controller shown in FIG. 1;

FIG. 6 is a block diagram showing a schematic configuration of a driver state detection system according to a modification of the embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Vehicle 2 Driver state detection system 3 Infrared LED 4 CCD camera 10 LED drive processing unit 11 Camera drive processing unit 12 Recognition processing unit 13 Alarm output unit 15 Recognition device 17, 17A Controller 20 Rotation speed detection sensor 21 Angle sensor 22 Sound sensor 23 VICS D driver

Claims (4)

[Claims] 1. A driver status detection system for detecting a status of a driver driving a vehicle, wherein the recognition device recognizes the status of the driver based on an image obtained by capturing an image while illuminating a face of the driver; A driver state detection system, comprising: a driving state detection unit that detects a driving state of a vehicle; and a control unit that controls driving of the recognition device according to the detected driving state. 2. An illumination device for illuminating a driver's face with infrared light by turning on an infrared light emitting device, an imaging device for imaging a face illuminated by the illumination unit, and an image captured by the imaging device. A recognition unit that recognizes the state of the driver based on the detected image, and the control unit includes a unit that performs on / off control of the infrared light emitting device of the illumination unit in accordance with the detected traveling state. The driver status detection system according to claim 1. 3. A driver status detection system for detecting a status of a driver driving a vehicle, wherein the recognition device recognizes the status of the driver based on an image obtained by capturing an image while illuminating a face of the driver; A driver state detection system comprising: a peripheral state detection unit that detects a state around a vehicle; and a control unit that controls driving of the recognition device in accordance with the detected peripheral state. 4. The recognition device according to claim 1, wherein the recognition unit illuminates an infrared light emitting device to illuminate a driver's face with infrared light, an imaging device for imaging the face illuminated by the illumination unit, and an image captured by the imaging device. A recognition unit that recognizes the state of the driver based on the detected image, and the control unit includes a unit that controls on and off of the infrared light emitting device of the illumination unit according to the detected surrounding situation. The driver status detection system according to claim 3.