WO2019030855A1

WO2019030855A1 - Drive incapability state determination device and drive incapability state determination method

Info

Publication number: WO2019030855A1
Application number: PCT/JP2017/028931
Authority: WO
Inventors: 慶友樹小川; 敬平野; 政信大澤
Original assignee: 三菱電機株式会社
Priority date: 2017-08-09
Filing date: 2017-08-09
Publication date: 2019-02-14
Also published as: JP6594595B2; JPWO2019030855A1

Abstract

This drive incapability state determination device is provided with: an image acquisition unit (101) for acquiring a captured image of the interior of a vehicle; a face detection unit (102) for detecting the face of a driver from the acquired captured image; a determination area acquisition unit (103) for acquiring a determination area image from the acquired captured image; and a first state determination unit (104) by which, if a state in which the driver's face cannot be detected continues for a first threshold time, the state of the driver is determined on the basis of a difference between the determination area image in the acquired captured image and a determination area image in a comparison image.

Description

Inoperable state determination device and inoperable state determination method

The present invention relates to a technology for determining an inoperable state of a driver driving a vehicle.

There is a technique for determining a driver's inoperable state from a captured image obtained by imaging a driver driving a vehicle. One of the methods of detecting the driver's inoperable state from the captured image is a method of detecting the inoperable state when a state in which the driver's face can not be detected from the captured image is continuously detected for a predetermined time.
For example, in Patent Document 1, the driver's face is sequentially detected from a captured image captured by a camera mounted on the vehicle, and the driver's face detected while the vehicle is traveling is out of a predetermined range of the image In some cases, an inoperable state detection device is disclosed that detects that the driver is inoperable.

JP, 2016-27452, A

In the inoperable state detection device described in Patent Document 1 described above, when the driver's face is blocked by, for example, the driver's hand or a towel cover, the driver's face image is within a predetermined range Therefore, there is a problem that the driver may erroneously detect that the driver can not drive the vehicle.

The present invention has been made to solve the above-described problems, and prevents the driver from being erroneously determined to be inoperable when the driver's face is blocked by a shield. The purpose is

In the driving impossible state determination device according to the present invention, an image acquisition unit that acquires a captured image in a vehicle, a face detection unit that detects a driver's face from the captured image acquired by the image acquisition unit, and an image acquisition unit The captured image acquired by the determination area acquisition unit when the determination area acquisition unit that acquires the determination area image from the acquired captured image and the state in which the face detection unit can not detect the driver's face continue for the first threshold time And a state determination unit that determines the state of the driver based on the difference between the determination area image of the inside and the determination area image of the comparison image.

According to the present invention, when the face of the driver is blocked by the shield, it is possible to prevent the driver from being erroneously determined to be inoperable.

FIG. 1 is a block diagram showing a configuration of an inoperable state determination device according to Embodiment 1. 2A and 2B are diagrams showing an example of processing of the face detection unit and the determination area acquisition unit of the inoperable state determination device according to the first embodiment. FIGS. 3A and 3B are diagrams showing an example of a hardware configuration of the inoperable state determination device according to the first embodiment. FIG. 4A, FIG. 4B, FIG. 4C, and FIG. 4D are figures which show the determination processing of the 1st state determination part of the operation impossible state determination apparatus based on Embodiment 1. FIG. 5 is a flowchart showing the operation of the inoperable state determination device according to the first embodiment. 5 is a flowchart showing an operation in a warning state of the inoperable state determination device according to the first embodiment. FIG. 7 is a diagram showing a display example based on the determination result of the inoperable state determination device of the invention according to the first embodiment.

Hereinafter, in order to explain the present invention in more detail, a mode for carrying out the present invention will be described according to the attached drawings.
Embodiment 1
FIG. 1 is a block diagram showing the configuration of the inoperable state determination device 100 according to the first embodiment.
The inoperable state determination device 100 first performs a process of detecting the driver's face, and when the face is not successfully detected, a captured image of the first driver's seat acquired from the imaging device in the vehicle compartment and The image is captured in advance by the imaging device in the vehicle compartment, and the image of the second driver's seat obtained by imaging the driver's seat on which the driver is not seated is compared. The inoperable state determination device 100 determines whether or not the driver is inoperable based on the comparison result of the captured image of the first driver's seat and the image of the second driver's seat.

Although details will be described later, the inoperable state determination device 100 calculates the difference between the captured image of the first driver's seat and the captured image of the second driver's seat. If the calculated difference is equal to or greater than the reference value, the inoperable state determination device 100 determines that the driver is not inoperable. On the other hand, when the calculated difference is less than the reference value, the inoperable state determination device 100 determines that the driver is inoperable.
As a result, if the driver's face is not successfully detected because the driver's face is covered with the driver's hand or a cover such as a towel, the driver's face can not be detected. It is possible to prevent an erroneous determination that the vehicle is in an inoperable state.

As shown in FIG. 1, the inoperable state determination apparatus 100 includes an image acquisition unit 101, a face detection unit 102, a determination area acquisition unit 103, a first state determination unit (state determination unit) 104, a comparison image storage unit 105, A face information acquisition unit 106 and a second state determination unit 107 are provided.
A camera 200, a warning device 300 and a vehicle control device 400 are connected to the inoperable state determination device 100.
The camera 200 is an imaging device for imaging the interior of a vehicle equipped with the inoperable state determination device 100. The camera 200 is, for example, an infrared camera. The camera 200 is installed at a position where at least the head of the driver sitting on the driver's seat can be photographed. Also, the camera 200 is configured of one or more cameras.

The warning device 300 generates information indicating a warning or information indicating a warning by voice or voice and display for the driver of the vehicle based on the determination result of the inoperable state determination device 100. The warning device 300 performs output control of information indicating a generated alert or information indicating a warning. The warning device 300, for example, a speaker, or a speaker and a display make an audio output or an audio output and display information indicating a warning or information indicating a warning based on the output control.
The vehicle control device 400 controls the traveling of the vehicle based on the determination result of the inoperable state determination device 100.

The components of the inoperable state determination device 100 will be described.
The image acquisition unit 101 acquires a captured image captured by the camera 200. The captured image is an image captured such that at least the head of the driver is viewed when the driver is seated in the driver's seat in a general driving posture. The image acquisition unit 101 outputs the acquired captured image to the face detection unit 102 and the determination area acquisition unit 103.
The face detection unit 102 analyzes the captured image acquired by the image acquisition unit 101 to detect a distribution of luminance values. The face detection unit 102 performs a process of detecting the driver's face from the captured image by comparing the detected distribution of the luminance values with the distribution of the luminance values for matching obtained in advance by learning. Here, the distribution of the luminance values for comparison is obtained statistically by learning the distribution of luminance values in the case where a human face is imaged using a plurality of captured images in which a human face is imaged. Shows a distribution pattern of various luminance values. When the face detection unit 102 can not detect the driver's face, the face detection unit 102 outputs information indicating failure in face detection to the first state determination unit 104 and the determination area acquisition unit 103. In addition, when the face detection unit 102 detects a driver's face, the face detection unit 102 outputs, to the face information acquisition unit 106, information indicating success in face detection and a captured image.

When information indicating failure in face detection is input from the face detection unit 102, the determination area acquisition unit 103 determines an image of the determination area (hereinafter referred to as a determination area image) from the captured image acquired by the image acquisition unit 101. get. Here, the determination area is an area including all the headrests of the driver's seat. Further, the determination area is an area set in advance based on the position of the driver's seat and the position of the camera 200 for each vehicle type. Further, the determination area is an area determined in consideration of the adjustment width of the driver's seat in the front-rear direction, the adjustment width of the driver's seat inclination, and the adjustment width of the driver's seat's headrest in the vertical direction. . The determination area acquisition unit 103 outputs the acquired determination area image to the first state determination unit 104.

FIG. 2 is a diagram showing an example of processing of the face detection unit 102 and the determination area acquisition unit 103 of the inoperable state determination apparatus 100 according to the first embodiment.
FIG. 2A illustrates an example of determination in a captured image obtained by capturing the driver's seat from the front, and FIG. 2B illustrates an example of a determination region in a captured image obtained by capturing the driver's seat from the oblique front. 2A and 2B show an example in which the face detection unit 102 succeeds in detecting the face Xa of the driver X.
The face detection unit 102 detects the face Xa of the driver X from the captured image A shown in FIGS. 2A and 2B.
The determination area acquisition unit 103 acquires an image of the determination area B, which is an area determined in advance, from the captured image A. The determination area B is an area including all the headrests H of the driver's seat. The determination area acquisition unit 103 outputs the acquired image of the determination area B to the first state determination unit 104.

When the information indicating failure in face detection is input from the face detection unit 102, the first state determination unit 104 determines the state of the driver. The first state determination unit 104 compares the determination area image in the captured image used in the face detection input from the determination area acquisition unit 103 with the determination area image in the comparison image captured in advance. The first state determination unit 104 determines the state of the driver based on the obtained comparison result. Here, the comparison image is an image captured in advance by the camera 200, and is a captured image obtained by capturing a driver's seat on which the driver is not seated.

Specifically, when the face detection unit 102 fails in face detection, the face detection unit 102 repeatedly performs face detection until the face detection is successful. The first state determination unit 104 continuously inputs information indicating a failure in face detection from the face detection unit 102 continuously for a time determined to require attention (hereinafter referred to as a first threshold time). Then, the difference between the determination area image in the captured image used for face detection and the determination area image in the comparison image is calculated. The first state determination unit 104 determines, based on the calculated difference and the reference value, whether the driver needs a warning (hereinafter, referred to as a warning state). Furthermore, the first state determination unit 104 determines that the driver is inoperable when the alert state continues for the second threshold time. Here, the second threshold time is a time obtained by adding a time for determining that the driver is inoperable to the first threshold time.
When the first state determination unit 104 determines that the driver is in the alert state or determines that the driver is in the inoperable state, the determination result is output to the warning device 300 or the vehicle control device 400. Output.

The comparison image storage unit 105 stores a comparison image to be referred to when the first state determination unit 104 compares the determination region images. The comparison image is an image captured in advance by the camera 200, and is a captured image obtained by capturing a driver's seat on which the driver is not seated.

The face information acquisition unit 106 analyzes the captured image when the face detection unit 102 receives the information indicating the success of the face detection and the captured image used in the face detection, and configures the driver's face. Obtain location information of elements such as eyes, nose and mouth. The face information acquisition unit 106 acquires face information for determining the inoperable state, such as the degree of eye opening of the driver and the face orientation of the driver, from the acquired position information of the components of the face of the driver. The face information acquisition unit 106 outputs the acquired face information to the second state determination unit 107.

The second state determination unit 107 detects, for example, the degree of eye opening or the face orientation from the face information acquired by the face information acquisition unit 106. The second state determination unit 107 determines whether or not the driver is incapable of driving based on the acquired degree of eye opening or face orientation. If the second state determination unit 107 determines that the driver is inoperable, the second state determination unit 107 outputs the determination result to the warning device 300 or the vehicle control device 400.

Next, a hardware configuration example of the inoperable state determination device 100 will be described.
FIG. 3A and FIG. 3B are diagrams showing an example of a hardware configuration of the inoperable state determination device 100. As shown in FIG.
Each function of the image acquisition unit 101, the face detection unit 102, the determination area acquisition unit 103, the first state determination unit 104, the face information acquisition unit 106, and the second state determination unit 107 in the inoperable state determination device 100 is processed It is realized by a circuit. That is, the inoperable state determination device 100 includes a processing circuit for realizing the above functions. The processing circuit may be the processing circuit 100a which is dedicated hardware as shown in FIG. 3A, or may be the processor 100b which executes a program stored in the memory 100c as shown in FIG. 3B. Good.

As shown in FIG. 3A, the image acquisition unit 101, the face detection unit 102, the determination area acquisition unit 103, the first state determination unit 104, the face information acquisition unit 106, and the second state determination unit 107 are dedicated hardware. In some cases, the processing circuit 100a may be, for example, a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. The thing corresponds. Even if each function of each unit of image acquisition unit 101, face detection unit 102, determination area acquisition unit 103, first state determination unit 104, face information acquisition unit 106 and second state determination unit 107 is realized by a processing circuit Alternatively, the functions of the respective units may be integrated and realized by one processing circuit.

As shown in FIG. 3B, when the image acquisition unit 101, the face detection unit 102, the determination region acquisition unit 103, the first state determination unit 104, the face information acquisition unit 106, and the second state determination unit 107 are the processor 100b. The function of each unit is realized by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 100c. The processor 100b reads out and executes the program stored in the memory 100c, whereby the image acquisition unit 101, the face detection unit 102, the determination area acquisition unit 103, the first state determination unit 104, the face information acquisition unit 106, and The respective functions of the second state determination unit 107 are realized. That is, when the image acquisition unit 101, the face detection unit 102, the determination area acquisition unit 103, the first state determination unit 104, the face information acquisition unit 106, and the second state determination unit 107 are executed by the processor 100b, A memory 100c is provided for storing a program that results in the steps shown in FIG. 5 and FIG. 6 described later. Further, these programs are obtained by computerizing the procedure or method of the image acquisition unit 101, the face detection unit 102, the determination area acquisition unit 103, the first state determination unit 104, the face information acquisition unit 106, and the second state determination unit 107. It can be said that the

Here, the processor 100 b refers to, for example, a central processing unit (CPU), a processing device, an arithmetic device, a processor, a microprocessor, a microcomputer, or a digital signal processor (DSP).
The memory 100c may be, for example, a nonvolatile or volatile semiconductor memory such as a random access memory (RAM), a read only memory (ROM), a flash memory, an erasable programmable ROM (EPROM), or an electrically EPROM (EEPROM). It may be a hard disk, a magnetic disk such as a flexible disk, or an optical disk such as a mini disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), or the like.

Note that hardware functions of the image acquisition unit 101, face detection unit 102, determination area acquisition unit 103, first state determination unit 104, face information acquisition unit 106, and second state determination unit 107 are partially dedicated. It may be realized by hardware, and a part may be realized by software or firmware. As described above, the processing circuit 100a in the inoperable state determination device 100 can realize each of the functions described above by hardware, software, firmware, or a combination thereof.

Next, details of the first state determination unit 104 will be described with reference to FIG.
FIG. 4 is a diagram showing a determination process of the first state determination unit 104 of the inoperable state determination apparatus 100 according to the first embodiment. 4A shows an example of the image of the determination area in the comparison image stored in the comparison image storage unit 105, and FIG. 4B to FIG. 4D show an example of a captured image at the time of face detection and an image of the determination area in the captured image. FIG. 4A shows an image of a determination area in a comparison image obtained by imaging a driver's seat where a driver is not seated. 4B shows the driver's face not blocked by the shield, FIG. 4C shows the driver's face blocked by the shield, and FIG. 4D shows the driver's face tilted forward. It shows the state of doing.

The image of the determination area C in the comparative image of FIG. 4A and the images of the determination areas Ba, Bb, and Bc in the captured image A at face detection in FIGS. 4B to 4D are images obtained by capturing the same area.
First, the comparison between the comparative image of FIG. 4A and the captured image at face detection of FIG. 4B will be described. The first state determination unit 104 compares the image of the determination area Ba in the captured image A with the image of the determination area C in the comparison image, and calculates the difference between the two images in which the headrest is captured. Do. The first state determination unit 104 calculates the difference between the image of the determination area C in the comparison image and the determination area Ba in the captured image A based on the difference between the area Ha and the area Hb in which the headrest is imaged.
The first state determination unit 104 uses at least one of the number of pixels at which the headrest is imaged, the length of the outline at which the headrest can be visually recognized, and the luminance of the determination area image. calculate.

Next, comparison between the comparison image of FIG. 4A and the captured image at face detection of FIG. 4C will be described.
The first state determination unit 104 compares the image of the determination area Bb in the captured image A with the image of the determination area C in the comparison image, and calculates the difference between the two images in which the headrest is captured. Do. The first state determination unit 104 calculates the difference between the image of the determination area C of the comparison image and the image of the determination area Bb of the captured image A based on the difference between the area Ha and the area Hc where the headrest is imaged. .

Next, the comparison between the comparative image of FIG. 4A and the captured image at the time of face detection of FIG. 4D will be described. The first state determination unit 104 compares the image of the determination area Bc in the captured image A with the image of the determination area C in the comparison image, and calculates the difference between the two images in which the headrest is captured. Do. The first state determination unit 104 calculates the difference between the image of the determination area C of the comparative image and the image of the determination area Bc of the captured image A based on the difference between the area Ha and the area Hd in which the headrest is imaged. .

The first state determination unit 104 generates an image of the determination area C in the comparison image of FIG. 4A and an image of the determination area Ba in the captured image A of FIG. 4B in which the driver's face is not blocked by the shield. The ratio of the area Ha in which the headrest is captured in the image of the determination area C in the comparison image is 100%, and the ratio of the area Hb in which the headrest is captured in the image of the determination area Ba in the captured image A Is calculated as, for example, 20%. The first state determination unit 104 calculates a difference ratio (80%) of the ratio of the area Ha (100%) to the ratio of the area Hb (20%), and the calculated difference ratio is a predetermined reference value. It is determined that (for example, 50%) or more. The first state determination unit 104 determines from the determination result that the headrest of the captured image A is sufficiently hidden by the driver's head or the like, and that there is a large difference as compared with the comparison image.

On the other hand, the first state determination unit 104 displays the image of the determination area C in the comparison image of FIG. 4A and the image of the determination area Bb in the captured image A of FIG. 4C in which the driver's face is blocked by the shield. In the comparison image, the proportion of the area Ha in which the headrest is captured in the image of the determination area C in the comparison image is 100%, and the area Hc in the image of the determination area Bb in the captured image A The ratio is calculated as, for example, 20%. The first state determination unit 104 calculates a difference ratio (80%) of the ratio of the area Ha (100%) to the ratio of the area Hc (20%), and the calculated difference ratio is a reference value previously determined. It is determined that (for example, 50%) or more. The first state determination unit 104 determines from the determination result that the headrest of the captured image A is sufficiently hidden by the driver's head or the like, and that there is a large difference as compared with the comparison image.

In addition, the first state determination unit 104 controls the image of the determination area C in the comparison image of FIG. 4A and the image of the determination area Bc in the captured image A of FIG. 4D in which the driver's face is inclined forward. In the comparison, the ratio of the area Ha obtained by imaging the headrest to the image of the determination area C in the comparison image is 100%, and the ratio of the area Hd acquired by imaging the headrest to the image of the determination area Bc in the imaged image A Calculated as 80%. The first state determination unit 104 calculates the difference ratio (20%) of the ratio of the area Ha (100%) to the ratio of the area Hd (80%), and the calculated difference ratio is a predetermined reference value. It determines that it is less than (for example, 50%). The first state determination unit 104 determines from the determination result that the headrest of the captured image A is not hidden by the driver's head or the like, and there is no large difference compared to the comparison image.

If the calculated difference ratio is less than a predetermined reference value, the first state determination unit 104 determines that the driver needs to call attention.
For example, in comparison between the image of the determination area C in the comparative image of FIG. 4A and the image of the determination area Ba in the captured image A of FIG. 4B, the first state determination unit 104 determines the difference ratio between the area Ha and the area Hb. Is greater than the reference value, it is determined that the driver X is not in a state requiring a warning.
Similarly, in comparison between the image of the determination area C in the comparative image of FIG. 4A and the image of the determination area Bb in the captured image A of FIG. 4C, the first state determination unit 104 determines the difference between the area Ha and the area Hc. Since the ratio is equal to or higher than the reference value, it is determined that the driver X is not in a state requiring attention.
On the other hand, for example, in the comparison between the determination area C in the comparison image of FIG. 4A and the image of the determination area Bc in the captured image A of FIG. The first state determination unit 104 determines that the driver X needs to call attention.

Next, the operation of the inoperable state determination device 100 will be described.
The operation of the inoperable state determination device 100 is performed when the first state determination unit 104 does not determine that the driver is in the alerting state, and the first state determination unit 104 is controlled by the driver. The operation in the case where it has already been determined to be in the awake state will be described separately.
First, the operation of the inoperable state determination device 100 when the first state determination unit 104 does not determine that the driver is in the alert state will be described with reference to the flowchart in FIG. 5.
FIG. 5 is a flowchart showing the operation of the inoperable state determination apparatus 100 according to the first embodiment. It is assumed that the first state determination unit 104 does not determine that the driver is in the alert state as a premise of performing the process of the flowchart in FIG. 5.

When the image acquisition unit 101 acquires a captured image (step ST1), the face detection unit 102 performs processing for detecting a driver's face from the acquired captured image (step ST2). The face detection unit 102 determines whether or not the driver's face has been successfully detected from the captured image in the process of step ST2 (step ST3). If the face detection is successful (step ST3; YES), the face detection unit 102 outputs, to the face information acquisition unit 106, information indicating that the face detection is successful and the captured image. The face information acquisition unit 106 acquires driver's face information from the captured image input from the face detection unit 102 (step ST4), and outputs the face information to the second state determination unit 107. The second state determination unit 107 executes a process of determining whether the driver is incapable of driving based on the driver's face information acquired in step ST4 (step ST5). Thereafter, the flowchart ends the process.

On the other hand, if face detection has failed (step ST3; NO), the face detection unit 102 outputs information indicating that face detection has failed to the determination area acquisition unit 103 and the first state determination unit 104. The determination area acquisition unit 103 acquires an image of the determination area from the captured image acquired in step ST1 (step ST6), and outputs the image to the first state determination unit 104. When the information indicating that the face detection has failed is input from the face detection unit 102, the first state determination unit 104 determines whether a counter (not shown) has been activated (step ST7). If the counter has not been activated (step ST7; NO), the first state determination unit 104 starts counting the counter (step ST8), and counts up the counter (step ST9). On the other hand, when the counter has been started (step ST7; YES), the flowchart proceeds to the process of step ST9.

The first state determination unit 104 refers to the count value of the counter and determines whether the first threshold time has elapsed (step ST10). If the first threshold time has not elapsed (step ST10; NO), the flowchart ends the process. On the other hand, when the first threshold time set in advance has elapsed (step ST10; YES), the first state determination unit 104 compares the ratio of the area in which the headrest is captured with the area of the determination area acquired in step ST6. The difference between the image and the ratio of the area in which the headrest is captured is calculated (step ST11).

The first state determination unit 104 determines whether the difference calculated in step ST11 is equal to or greater than the reference value (step ST12). If the difference is equal to or greater than the reference value (step ST12; YES), the first state determination unit 104 determines that the driver is not in an inoperable state, and ends the process. On the other hand, when it is determined that the difference is less than the reference value (step ST12; NO), the first state determination unit 104 determines that the driver is in the alert state (step ST13). The first state determination unit 104 outputs the determination result of step ST13 to the warning device 300 or the vehicle control device 400 (step ST14), and ends the process.

Next, the operation of the inoperable state determination device 100 when the first state determination unit 104 has determined that the driver is in the alert state will be described with reference to the flowchart in FIG. 6.
FIG. 6 is a flowchart showing the operation in the alert state of the inoperable state determination device 100 according to the first embodiment. As a premise that the process of the flowchart of FIG. 6 is performed, it is assumed that the first state determination unit 104 determines in step ST13 of the flowchart of FIG.
When the image acquisition unit 101 acquires a captured image (step ST21), the face detection unit 102 performs processing for detecting the driver's face from the acquired captured image (step ST22). The face detection unit 102 determines whether or not the driver's face has been successfully detected from the captured image in the process of step ST22 (step ST23). If the face detection is successful (step ST23; YES), the face detection unit 102 outputs information indicating that the face detection is successful to the first state determination unit 104. When information indicating that the face detection has succeeded is input, the first state determination unit 104 cancels the driver's attention (step ST24).

On the other hand, when face detection has failed (step ST23; NO), the face detection unit 102 outputs information indicating that face detection has failed to the first state determination unit 104. When the information indicating that the face detection has failed is input, the first state determination unit 104 counts up the counter (step ST25). The first state determination unit 104 refers to the count value of the counter and determines whether the second threshold time has elapsed (step ST26). If the second threshold time has not elapsed (step ST26; NO), the flowchart ends the process.

On the other hand, when the second threshold time has elapsed (step ST26; YES), the first state determination unit 104 determines that the driver is incapable of driving (step ST27). The first state determination unit 104 outputs the determination result of step ST27 to the warning device 300 or the vehicle control device 400 (step ST28). The first state determination unit 104 resets the counter (step ST29), and ends the process.
The flowcharts of FIG. 5 and FIG. 6 show the processing of one cycle, and when the driver is driving, the processing of the flowchart is repeatedly performed.

As described above, the inoperable state determination apparatus 100 according to the first embodiment includes the image acquisition unit 101 that acquires a captured image in a vehicle, and a face detection unit that detects the driver's face from the acquired captured image. 102, the determination area acquisition unit 103 for acquiring a determination area image from the acquired captured image, and the determination in the acquired captured image when the state in which the driver's face can not be detected continues for the first threshold time The first state determination unit 104 is configured to determine the state of the driver based on the difference between the area image and the determination area image in the comparison image.
Therefore, the inoperable state determination apparatus 100 according to the first embodiment compares the difference between the determination area image in the captured image and the determination area image in the comparison image with the reference value to find that the driver's face is at the normal position. It can be determined that it is located. When the driver's face is blocked by the shield, it is possible to prevent the driver from being erroneously determined to be inoperable.

Further, according to the first embodiment, when the difference is less than the reference value, the first state determination unit 104 determines that the driver is in the alerting state, and the driver is in the alerting state. Since the driver is determined to be inoperable when it is determined that there is a situation where the face detection unit 102 can not detect the driver's face continues for the second threshold time, the driver's The inoperable state can be determined more accurately.

In the inoperable state determination apparatus 100 shown in the first embodiment described above, when it is determined that the driver needs to be alerted, or when it is determined that the driver is inoperable, external The warning device 300 gives a warning by voice.
In addition, the warning device 300 may display on the display that it is determined that the driving is not possible, and may display a button for receiving an input for canceling the determination of the driving impossible.
FIG. 7 is a diagram showing an example of notifying the driver of the determination result of the inoperable state determination device 100 of the invention according to the first embodiment.
The display 301 displays a determination result 302 of the inoperable state determination device 100 and a button 303 for canceling the determination result. The determination result 302 is displayed as, for example, “determining as inoperable state”. Further, the button 303 is displayed as, for example, “reset to normal state”.

The driver refers to the determination result 302 displayed on the display 301, and when the driver is not in the inoperable state, pressing the button 303 causes the inoperable state determination device 100 to cancel the determination result. it can.

In the present invention, within the scope of the invention, modifications of optional components of the embodiment or omission of optional components of the embodiment is possible.

The inoperable state determination device according to the present invention is suitable for being used in a driver monitoring system or the like for which improvement in determination accuracy is required.

Reference Signs List 100 non-operational state determination device, 101 image acquisition unit, 102 face detection unit, 103 determination area acquisition unit, 104 first state determination unit, 105 comparison image storage unit, 106 face information acquisition unit, 107 second state determination unit .

Claims

An image acquisition unit that acquires a captured image in the vehicle;
A face detection unit that detects a driver's face from the captured image acquired by the image acquisition unit;
A determination area acquisition unit that acquires a determination area image from the captured image acquired by the image acquisition unit;
When the state where the face detection unit can not detect the driver's face continues for the first threshold time, the determination area image in the captured image acquired by the determination area acquisition unit and the determination area in the comparison image And a state determination unit that determines the state of the driver based on a difference with an image.
The said comparative image is a captured image which imaged the driver's seat in which the said driver | operator is not seated, The driving impossible state determination apparatus of Claim 1 characterized by the above-mentioned.
The driving impossible state judging device according to claim 1 or 2, wherein the state judging unit judges that the driver is not in a state of being alerted when the difference larger than a reference value is detected.
The driving impossible state judging device according to claim 1 or 2, wherein the state judging unit judges that the driver is in a state of being alerted when the difference is smaller than a reference value.
The state determination unit determines that the driver is in a state of alerting, and the driver drives when the state in which the face detection unit can not detect the driver's face continues for a second threshold time. The inoperable state determination apparatus according to claim 4, wherein the inoperable state is determined.
The image acquisition unit acquires a captured image in the vehicle;
Detecting a face of the driver from the captured image acquired by the face detection unit;
A determination area acquisition unit acquiring a determination area image from the acquired captured image;
The difference between the determination area image in the acquired captured image and the determination area image in the comparison image when the state determination unit can not detect the driver's face continues for the first threshold time Determining the state of the driver based on the condition.