WO2019220572A1 - Driving inability detection device and driving inability detection method - Google Patents

Abstract

An image acquisition unit (11) acquires a captured image of a driver. An image determination unit (13) calculates a first degree of deviation between a normal image which has been registered in a database (12) and the captured image acquired by the image acquisition unit (11). An eye detection unit (14) detects the driver's eyes from the captured image. When the first degree of deviation is equal to or higher than a first threshold value, a state determination unit (15) determines that the driver is in a driving inability state if a period of time in which the eye detection unit (14) cannot detect the driver's eyes continues for a prescribed period or more, and determines that the driver is in a normal state if the period of time in which the eye detection unit (14) cannot detect the driver's eyes does not continue for the prescribed period or more. A registration unit (16) registers the captured image acquired by the image acquisition unit (11) as a normal image in the database (12) when it is determined that the driver is in a normal state.

Description

Inoperability detection device and inoperability detection method

The present invention relates to an inoperability detecting device and an inoperability detecting method for detecting that the driver is in an inoperable state.

For example, the driving disability detection device described in Patent Literature 1 sequentially detects the driver's head based on the image of the driver's seat taken by the imaging device, and when the detected head is out of a predetermined range, Detects that the driver is in an inoperable state. Thereby, the said driving | operation impossible detection apparatus can detect having fallen into the driving | operation impossible state, such as a driver | operator losing consciousness, during driving | running | working of a vehicle.

JP 2016-27452 A

Since the head of a driver who has a habit of driving with a posture deviating from the standard position is located outside the predetermined range, it is impossible to detect whether or not the driver is in an inoperable state. In view of this, the inoperability detecting device described in Patent Document 1 detects whether or not the inoperable state is detected for a driver whose driving posture is deviated from the standard position by, for example, causing the driver to adjust a predetermined range. It is possible. However, since the driving impossibility detecting device described in Patent Document 1 is configured as described above, a predetermined range used for detecting whether or not driving is impossible is automatically determined in consideration of individual differences in driving posture. There was a problem that it could not be adjusted.

The present invention has been made to solve the above-described problems, and an object thereof is to detect an inoperable state in consideration of individual differences in driving posture.

The driving impossibility detecting device according to the present invention includes: an image acquisition unit that acquires a captured image of a driver of a vehicle; and a normal captured image when the driver is in a normal state among the captured images acquired by the image acquisition unit. A database to be registered as an image, an image determination unit that calculates a degree of divergence between a normal image registered in the database and a captured image acquired by the image acquisition unit, and an image acquisition unit The eye detection unit that detects the driver's eyes from the captured image and the first divergence calculated by the image determination unit are compared with a first threshold, and when the first divergence is greater than or equal to the first threshold, If the time during which the driver's eyes cannot be detected by the detection unit continues for a predetermined time or longer, the driver determines that the driver is incapable of driving, and the time during which the driver's eyes cannot be detected by the eye detection unit is predetermined. A state determination unit that determines that the driver is in a normal state if it does not continue for more than a period of time, and when the state determination unit determines that the driver is in a normal state, the captured image acquired by the image acquisition unit is a normal image And a registration unit for registering in the database.

According to the present invention, the captured image when the driver is determined to be in the normal state is registered in the database as a normal image and used for subsequent determinations. Therefore, driving considering individual differences in driving posture An impossible state can be detected.

It is a block diagram which shows the structural example of the driving impossible detection apparatus which concerns on Embodiment 1. FIG. 6 is a diagram illustrating an example of a method of calculating a first divergence degree by the image determination unit of Embodiment 1. FIG. 4 is a flowchart illustrating an operation example of the inoperability detecting apparatus according to the first embodiment. It is a block diagram which shows the structural example of the driving impossible detection apparatus which concerns on Embodiment 2. FIG. 6 is a flowchart illustrating an operation example of the inoperability detecting apparatus according to the second embodiment. FIG. 10 is a block diagram illustrating a configuration example of an inoperability detecting device according to a third embodiment. 10 is a flowchart illustrating an operation example of the inoperability detecting apparatus according to the third embodiment. 10 is a flowchart illustrating an operation example of the inoperability detecting apparatus according to the fourth embodiment. 9A and 9B are diagrams illustrating a hardware configuration example of the inoperability detecting device according to each embodiment.

Hereinafter, in order to explain the present invention in more detail, modes for carrying out the present invention will be described with reference to the accompanying drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram illustrating a configuration example of the inoperability detecting apparatus 10 according to the first embodiment. The driving impossibility detection device 10 and the imaging device 1 connected to the driving impossibility detection device 10 are mounted on a vehicle.

The imaging device 1 is a visible light camera or an infrared camera installed in the vehicle. Note that the imaging apparatus 1 may be configured by a plurality of cameras installed in the vehicle. The imaging device 1 is connected to the inoperability detecting device 10 via a bus or the like. This imaging device 1 images the inside of the vehicle including the driver, and outputs the captured image to the image acquisition unit 11.

The inoperability detection apparatus 10 includes an image acquisition unit 11, a database 12, an image determination unit 13, an eye detection unit 14, a state determination unit 15, and a registration unit 16.

The image acquisition unit 11 acquires a captured image captured by the imaging device 1 and outputs the captured image to the image determination unit 13, the eye detection unit 14, and the registration unit 16. The captured image output by the image acquisition unit 11 to the image determination unit 13 or the like includes at least a range where the face is likely to be located when the driver is seated in the driver's seat. . By widening this range, it is possible to cope with various driving postures.

In the database 12, a captured image when the driver is in a normal state among the captured images acquired by the image acquisition unit 11 is registered as a normal image.

“Normal state” means that normal operation is possible. The inoperable state refers to a state in which the driver has lost consciousness due to a sudden illness and the like and is in a state in which normal driving cannot be performed. When the driver is unable to drive, the driver's face often moves from the normal driving position and does not return to the normal driving position.

The image determination unit 13 compares the normal image registered in the database 12 with the captured image acquired by the image acquisition unit 11 and calculates the first degree of divergence between the two images. The image determination unit 13 outputs the calculated first divergence degree to the state determination unit 15. The image determination unit 13 calculates a first divergence degree indicating the degree of divergence between the luminances of both images. When the first divergence degree is large, there is a high possibility that the driver's face has moved from the position during normal driving, and there is a suspicion that the driver is unable to drive.

FIG. 2 is a diagram illustrating an example of a first divergence calculation method by the image determination unit 13 according to the first embodiment. 2, the normal image _{p 1, p 2, ···,} p n is the registered normal image database 12, the time _{t 1,} t 2, · · ·, constant captured in the order of _{t n} It is a captured image for a time (for example, the latest 60 seconds). The image determination unit 13 compares the plurality of normal images p ₁ , p ₂ ,..., _{Pn one} by one in time series order, and finally adds the luminance in the corresponding pixel unit, thereby finally One reference image b _n is generated.
First, the image determining unit 13, as a reference image _{b 1} first normal image _{p 1.} Subsequently, the image determination unit 13 weights and adds the luminance of each pixel of the next normal image p ₂ and the luminance of each pixel of the reference image b ₁ to generate the reference image b ₂ . Subsequently, the image determination unit 13 weights and adds the luminance of each pixel of the next normal image p ₃ and the luminance of each pixel of the reference image b ₂ to generate the reference image b ₃ . . Image determination unit 13, by repeating this process, finally generating one reference image b _n.

b _n = (1−α) · p _n + α · b _n−1 (1)

Here, n is an integer of 2 or more, and α is a weight. α may be a fixed value regardless of the time, or may be a value that changes according to the time. When α is a value that changes according to time, it is preferable that the brightness of the captured image captured at a newer time than the brightness of the captured image captured at the old time is easily reflected in the reference image b _n. .

The image determining unit 13, and the reference image b _n which is finally produced, as compared to the current captured image acquired by the image acquisition unit 11 calculates the luminance difference between pixels. Then, the image determination unit 13 calculates an average luminance difference from the luminance difference in units of pixels. The image determination unit 13 sets the calculated average luminance difference as the first divergence degree.
Alternatively, the image determination unit 13 may calculate a variance value by using a plurality of average luminance differences calculated in the past and the average luminance difference calculated this time, and may use the calculated variance value as the first divergence degree. By using the variance value as the first divergence degree, it is possible to eliminate the influence of disturbance on the captured image.

The eye detection unit 14 detects the driver's eyes from the captured image acquired by the image acquisition unit 11. That is, when the driver's eyes are reflected in the captured image, the eye detection unit 14 determines that the eyes have been detected regardless of whether the eyes are open or closed. On the other hand, the eye detection unit 14 determines that the eyes cannot be detected when the driver's face is removed from the captured image and the eyes are not visible, or when eye detection fails. The eye detection unit 14 outputs the eye detection result to the state determination unit 15. As described above, when the driver is unable to drive, the driver's face often moves from the normal driving position and does not return to the normal driving position. Therefore, when the driver is unable to drive, it is often impossible to detect eyes from the captured image.

Note that the eye detection unit 14 may determine the presence or absence of eye detection using the reliability of image recognition when the eyes are detected from the captured image. For example, the eye detection unit 14 determines that the eye has been detected when the reliability of image recognition when the eye is detected is greater than or equal to a predetermined value, and the reliability is less than the predetermined value. It is determined that the eyes cannot be detected.

The state determination unit 15 determines whether the driver is in a normal state or incapable of driving using the first divergence calculated by the image determination unit 13 and the eye detection result by the eye detection unit 14. To do. When the state determination unit 15 determines that the driver is in a normal state, the state determination unit 15 outputs a determination result to the registration unit 16.

Specifically, the first divergence calculated by the image determination unit 13 is compared with a predetermined first threshold. The state determination unit 15 determines that the driver is suspected of being incapable of driving when the first divergence degree is equal to or greater than the first threshold, and when the first divergence degree is less than the first threshold, the driver Is determined to be in a normal state. When the state determination unit 15 determines that there is a suspicion that the driver is incapable of driving, the time during which the eyes of the driver cannot be detected by the eye detection unit 14 is determined in advance (hereinafter referred to as “predetermined time”). It is determined whether or not to continue. The predetermined time is, for example, 2 seconds. When the time during which the driver's eyes cannot be detected continues for a predetermined time or longer, the state determination unit 15 determines that the driver is incapable of driving, and the time during which the driver's eyes cannot be detected is less than the predetermined time. It is determined that the driver is in a normal state.

When the state determination unit 15 determines that the driver is in a normal state, the registration unit 16 registers the captured image acquired by the image acquisition unit 11 in the database 12 as a normal image.

FIG. 3 is a flowchart showing an operation example of the inoperability detecting apparatus 10 according to the first embodiment. When the ignition switch of the vehicle is turned on and the supply of power to the inoperability detecting device 10 is started, the inoperability detecting device 10 periodically repeats the operation shown in the flowchart of FIG.

In step ST1, the image acquisition unit 11 acquires a captured image from the imaging device 1, and outputs the acquired image to the image determination unit 13, the eye detection unit 14, and the registration unit 16.

In step ST2, the image determination unit 13 determines whether or not a sufficient amount of normal images for calculating the first divergence degree is registered in the database 12. The amount sufficient for calculating the first divergence degree is, for example, a normal image for 60 seconds. If the image determination unit 13 determines that a sufficient amount of normal images for calculating the first divergence degree is registered in the database 12 (step ST2 “YES”), the process proceeds to step ST3. On the other hand, if the image determination unit 13 determines that a sufficient amount of normal images for calculating the first divergence degree is not registered in the database 12 (step ST2 “NO”), the process proceeds to step ST9. At this time, the image determination unit 13 considers that the driver is in a normal state for a certain period (for example, 60 seconds) after the ignition switch is turned on, and uses the current captured image acquired in step ST1 as a normal image. The registration unit 16 is instructed to register in the database 12.

In step ST3, the image determination unit 13 acquires a sufficient amount of normal images from the database 12 for calculating the first divergence degree. Then, the image determination unit 13 calculates the first divergence based on the luminance of the normal image acquired from the database 12 and the luminance of the current captured image acquired in step ST1, and outputs the first divergence degree to the state determination unit 15. .

In step ST4, when the first divergence calculated by the image determination unit 13 is greater than or equal to the first threshold (step ST4 “YES”), the state determination unit 15 is suspected that the driver is in an inoperable state. And the process proceeds to step ST5. On the other hand, if the first divergence calculated by the image determination unit 13 is less than the first threshold (step ST4 “NO”), the state determination unit 15 proceeds to step ST8.

In step ST5, the eye detection unit 14 detects an eye from the current captured image acquired in step ST1, and outputs a detection result indicating the presence or absence of detection to the state determination unit 15.

In step ST6, the state determination unit 15 proceeds to step ST7 when the eye can be detected by the eye detection unit 14 (step ST6 “YES”), and to step ST10 when the eye cannot be detected (step ST6 “NO”). move on.

In step ST7, the state determination unit 15 resets the duration. This duration time is a time period in which there is a suspicion that the driver is in an inoperable state and an eye cannot be detected by the eye detection unit 14.

In step ST8, the state determination unit 15 determines that the driver is in a normal state. Then, the state determination unit 15 instructs the registration unit 16 to register the current captured image acquired in step ST1 in the database 12 as a normal image.

In step ST9, when the registration unit 16 receives an instruction from the image determination unit 13 in step ST2 or receives an instruction from the state determination unit 15 in step ST8, the registration unit 16 performs normal operation on the current captured image acquired in step ST1. The image is registered in the database 12 as an image.
Each time the operation shown in the flowchart of FIG. 3 is performed once, one captured image is registered in the database 12. Therefore, in the initial state where no normal image is registered in the database 12, the operations of steps ST1, ST2, and ST9 are performed until the normal image registered in the database 12 in step ST2 has a sufficient amount for calculating the first divergence degree. Will be repeated.

In step ST10, the state determination unit 15 counts the duration time because there is a suspicion that the driver is in an inoperable state and the eyes cannot be detected by the eye detection unit 14.

In step ST11, the state determination unit 15 determines whether or not the duration time is equal to or longer than a predetermined time. If state determination unit 15 determines that the duration is greater than or equal to the predetermined time (step ST11 “YES”), it proceeds to step ST12 and determines that the duration is less than the predetermined time (step ST11 “NO”). Then, the operation shown in the flowchart of FIG.
The state determination unit 15 continues to hold the duration until it is determined that the driver is in a normal state while the operation shown in the flowchart of FIG. 3 is repeated.

In step ST12, the state determination unit 15 determines that the driver is in an inoperable state.

As described above, the inoperability detecting apparatus 10 according to the first embodiment includes the image acquisition unit 11, the database 12, the image determination unit 13, the eye detection unit 14, the state determination unit 15, and the registration unit 16. The image acquisition unit 11 acquires a captured image of the vehicle driver. The database 12 registers, as a normal image, a captured image when the driver is in a normal state among the captured images acquired by the image acquisition unit 11. The image determination unit 13 calculates a first divergence degree indicating the degree of divergence between the normal image registered in the database 12 and the captured image acquired by the image acquisition unit 11. The eye detection unit 14 detects the driver's eyes from the captured image acquired by the image acquisition unit 11. The state determination unit 15 compares the first divergence calculated by the image determination unit 13 with a first threshold, and when the first divergence is equal to or greater than the first threshold, the eye detection unit 14 performs the driver's eyes. If the time during which the vehicle cannot be detected continues for a predetermined time or more, the driver determines that the vehicle is in an inoperable state. If the time during which the eyes of the driver cannot be detected by the eye detection unit 14 does not continue for the predetermined time or longer, the driver is in a normal state. Is determined. When the state determination unit 15 determines that the driver is in a normal state, the registration unit 16 registers the captured image acquired by the image acquisition unit 11 in the database 12 as a normal image. With this configuration, the inoperability detecting device 10 can register normal images in the database 12 and use them for subsequent determinations, and therefore can detect an inoperable state in consideration of individual differences in driving posture. In addition, since the inoperability detecting device 10 detects the inoperable state by combining the first deviation degree between the normal image and the current captured image and the presence / absence of eye detection, the normal state and the inoperable state are accurately discriminated. can do.

Further, the image determination unit 13 according to the first embodiment calculates the first divergence degree based on the luminance of the normal image registered in the database 12 and the luminance of the captured image acquired by the image acquisition unit 11. Thereby, when the brightness of the normal image and the brightness of the current captured image are deviated, the driving impossibility detection device 10 moves from the position during normal driving and becomes incapable of driving. It can be determined that there is a suspicion.

Embodiment 2. FIG.
FIG. 4 is a block diagram illustrating a configuration example of the inoperability detecting apparatus 10 according to the second embodiment. The inoperability detecting apparatus 10 according to Embodiment 2 has a configuration in which a vehicle device 2 and a vehicle information acquisition unit 21 are added to the inoperability detecting apparatus 10 of Embodiment 1 shown in FIG. 4, parts that are the same as or correspond to those in FIG. 1 are given the same reference numerals, and descriptions thereof are omitted.

The vehicle device 2 is, for example, a steering sensor, a speedometer, a gear position sensor, an accelerator position sensor, a brake pressure sensor, an accelerometer, a thermometer, a hygrometer, a seat position sensor, or a GPS (Global Positioning System) receiver. The vehicle device 2 is connected to the inoperability detecting device 10 via a CAN (Controller Area Network) or the like. The vehicle device 2 outputs vehicle information including information such as a steering angle, a vehicle speed, or a gear position to the vehicle information acquisition unit 21.

The vehicle information acquisition part 21 acquires the vehicle information which shows the state of a vehicle from the vehicle apparatus 2, and outputs it to the image determination part 13a and the registration part 16a.

In the database 12a, a captured image when the driver is in a normal state among the captured images acquired by the image acquisition unit 11 is registered in association with the vehicle information acquired by the vehicle information acquisition unit 21. That is, in the database 12a, a normal image when the driver is in a normal state and vehicle information indicating the state of the vehicle when the normal image is captured are associated and registered.

FIG. 5 is a flowchart showing an operation example of the inoperability detecting apparatus 10 according to the second embodiment. When the ignition switch of the vehicle is turned on and the supply of power to the inoperability detecting device 10 is started, the inoperability detecting device 10 periodically repeats the operation shown in the flowchart of FIG.

In step ST1, the image acquisition unit 11 acquires a captured image from the imaging device 1, and outputs the acquired image to the image determination unit 13a, the eye detection unit 14, and the registration unit 16a.

In step ST21, the vehicle information acquisition unit 21 acquires vehicle information from the vehicle device 2 and outputs the vehicle information to the image determination unit 13a and the registration unit 16a.

In step ST2a, the image determination unit 13a determines whether or not an amount of normal images sufficient for calculating the first divergence degree is registered in the database 12a. At this time, the image determination unit 13a selects a normal image associated with vehicle information that matches the current vehicle information acquired in step ST21 as the normal image used for calculating the first divergence degree. For example, the image determination unit 13a selects a normal image captured at the steering angle and vehicle speed that match the current steering angle and vehicle speed. Note that the current vehicle information and the vehicle information already registered in the database 12a do not need to be exactly the same, but may be similar to each other as long as they can be regarded as substantially matching. If the image determination unit 13a determines that a sufficient amount of normal images for calculation of the first divergence degree is registered in the database 12a (step ST2a “YES”), the process proceeds to step ST3. On the other hand, if the image determination unit 13a determines that a sufficient amount of normal images for calculation of the first divergence degree is not registered in the database 12a (step ST2a “NO”), the process proceeds to step ST9a. At this time, the image determination unit 13a considers that the driver is in a normal state for a certain period (for example, 60 seconds) after the ignition switch is turned on, and sets the current captured image acquired in step ST1 as a normal image. The registration unit 16a is instructed to associate the normal image with the vehicle information acquired in step ST21 and register it in the database 12a.

In step ST3, the image determination unit 13a acquires from the database 12a a normal image associated with vehicle information that matches the current vehicle information acquired in step ST21, and calculates a first degree of divergence. Steps ST4 to ST8 and ST10 to ST12 after step ST3 are the same operations as steps ST4 to ST8 and ST10 to ST12 of FIG.

In step ST9a, when the registration unit 16a receives an instruction from the image determination unit 13a in step ST2a, or receives an instruction from the state determination unit 15 in step ST8, the registration unit 16a acquires the current captured image acquired in step ST1. A normal image is registered, and the normal image and the current vehicle information acquired in step ST21 are associated with each other and registered in the database 12a.

As described above, the inoperability detecting apparatus 10 according to the second embodiment includes the vehicle information acquisition unit 21 that acquires vehicle information indicating the state of the vehicle. The database 12a registers a normal image in association with vehicle information indicating the state of the vehicle when the normal image is captured. The image determination unit 13a calculates a first degree of divergence using a normal image associated with vehicle information that matches the vehicle information acquired by the vehicle information acquisition unit 21 among normal images registered in the database 12a. With this configuration, the inoperability detecting device 10 can detect the inoperable state with higher accuracy in consideration of the difference in the state of the vehicle in addition to the individual difference in the driving posture.

Embodiment 3 FIG.
FIG. 6 is a block diagram illustrating a configuration example of the inoperability detecting apparatus 10 according to the third embodiment. The driving disability detection device 10 according to the third embodiment has a configuration in which a face movement determination unit 31 is added to the driving disability detection device 10 of the first embodiment shown in FIG. In FIG. 6, the same or corresponding parts as those in FIG.

The face movement determination unit 31 compares the normal image registered in the database 12 with the captured image acquired by the image acquisition unit 11, and calculates the second degree of divergence between the two images. The face movement determination unit 31 outputs the calculated second deviation degree to the state determination unit 15b. The face movement determination unit 31 calculates a second divergence degree indicating the degree of divergence between the face positions of both images. When the second divergence degree is large, there is a high possibility that the driver's face is moving from the normal driving position, and there is a suspicion that the driver is in a driving impossible state.

Here, an example of a method of calculating the second deviation degree by the face movement determination unit 31 will be described.
The face movement determination unit 31 detects a face position from each of a plurality of captured images captured in the most recent time zone from the present to a certain time before (for example, 10 seconds before). Then, the face movement determination unit 31 creates a determination histogram that represents the distribution of face positions over the last 10 seconds. Note that the captured image for the latest 10 seconds may include an image that has been determined as a normal image by the state determination unit 15b. Also, the captured image captured in the time zone from the present to a certain time ago may be temporarily stored in the database 12 or may be temporarily stored in the face movement determination unit 31.

In addition, the face movement determination unit 31 acquires a plurality of normal images captured in a time period from the time when the ignition switch is turned on to the predetermined time before the present time among the normal images registered in the database 12. . For example, when the current time is the time when 10 minutes have passed since the ignition switch was turned on, the face movement determination unit 31 acquires a plurality of normal images for the past 9 minutes and 50 seconds from the database 12. The face movement determination unit 31 detects a face position from each of a plurality of normal images acquired from the database 12. Then, the face movement determination unit 31 creates a reference histogram representing the distribution of face positions for the past 9 minutes and 50 seconds. Note that since the number of target images differs between the determination histogram and the reference histogram, the face movement determination unit 31 sets the frequencies of both histograms to relative values.

The face movement determination unit 31 compares the determination histogram for the latest 10 seconds with the reference histogram for the past 9 minutes and 50 seconds, and calculates the second divergence degree. For example, the face movement determination unit 31 calculates a correlation value between two histograms using a histogram intersection method, and sets the reciprocal of the correlation value as the second divergence degree. The face movement determination unit 31 determines the movement of the face position using a plurality of normal images and a plurality of captured images, so that when the driver changes his / her posture while driving, it is erroneously determined to be incapable of driving. Can be prevented.

The face movement determination unit 31 creates a histogram for determination by using the captured images captured in the latest 10 seconds at the next time (for example, when 10 minutes and 1 second have elapsed since the ignition switch was turned on). In addition, the face movement determination unit 31 creates a reference histogram using normal images captured from the time when the ignition switch is turned on until 9 minutes 51 seconds among normal images registered in the database 12. To do. When creating the reference histogram, the face movement determination unit 31 assigns a small weight to the frequency indicating the face position in the normal image for the past 9 minutes and 50 seconds, and determines the face position in the newly added normal image at 9 minutes and 51 seconds. Give a high weight to the frequency shown. Accordingly, the face position of the normal image captured at a newer time than the face position of the normal image captured at the old time is easily reflected in the reference histogram.

FIG. 7 is a flowchart showing an operation example of the inoperability detecting apparatus 10 according to the third embodiment. When the ignition switch of the vehicle is turned on and the supply of power to the inoperability detector 10 is started, the inoperability detector 10 periodically repeats the operation shown in the flowchart of FIG. Steps ST1 to ST3 and ST5 to ST12 in FIG. 7 are the same as the operations in steps ST1 to ST3 and ST5 to ST12 in FIG.

In step ST31, the face movement determination unit 31 acquires, from the database 12, a normal image captured in a time zone from when the ignition switch is turned on until the predetermined time before the present time. Then, the face movement determination unit 31 is captured in the most recent time zone from the present to the predetermined time before, including the face position in the normal image acquired from the database 12 and the current captured image acquired in step ST1. Based on the face position in the captured image, the second divergence degree is calculated and output to the state determination unit 15b.

In step ST <b> 32, the state determination unit 15 b compares the first divergence calculated by the image determination unit 13 with the first threshold and is determined in advance as the second divergence calculated by the face movement determination unit 31. The second threshold value is compared. When the first divergence is greater than or equal to the first threshold and the second divergence is greater than or equal to the second threshold (step ST32 “YES”), the state determination unit 15b becomes incapable of driving. The process proceeds to step ST5. On the other hand, when the first deviation degree is less than the first threshold value or when the second deviation degree is less than the second threshold value (step ST32 “NO”), the state determination unit 15b proceeds to step ST8.

As described above, the driving disability detection device 10 according to the third embodiment includes the driver's face position in the normal image registered in the database 12 and the driver's face position in the captured image acquired by the image acquisition unit 11. A second divergence degree indicating the degree of divergence is calculated. When the first divergence calculated by the image determination unit 13 is greater than or equal to the first threshold and the second divergence calculated by the face movement determination unit 31 is greater than or equal to the second threshold. In addition, if the eye detection unit 14 cannot detect the driver's eyes for a predetermined time or more, it is determined that the driver is in an inoperable state. On the other hand, the state determination unit 15b determines that the driver is in a normal state if the eye detection unit 14 cannot detect the driver's eyes for a predetermined time or longer. With this configuration, when the face position of the normal image and the face position of the captured image deviate from each other, the driving impossibility detection device 10 moves from the position during normal driving and becomes incapable of driving. Can be determined to be suspected. In addition, since the inoperability detecting device 10 detects the inoperable state by combining the first and second divergence degrees between the normal image and the current captured image, and the presence or absence of eye detection, the normal state and the inoperable state are detected. Can be more accurately discriminated.

In addition, you may combine the structure of the vehicle equipment 2 of Embodiment 2, and the vehicle information acquisition part 21 with respect to the driving | operation impossible detection apparatus 10 of Embodiment 3. FIG.

Embodiment 4 FIG.
Since the configuration of the inoperability detecting apparatus 10 according to the fourth embodiment is the same as the configuration shown in FIG. 6 of the third embodiment in the drawing, FIG. 6 is used below.

FIG. 8 is a flowchart showing an operation example of the inoperability detecting apparatus 10 according to the fourth embodiment. When the ignition switch of the vehicle is turned on and power supply to the inoperability detecting device 10 is started, the inoperability detecting device 10 periodically repeats the operation shown in the flowchart of FIG. Steps ST1 to ST3, ST5 to ST12, ST31, and ST32 in FIG. 8 are the same as the operations in steps ST1 to ST3, ST5 to ST12, ST31, and ST32 in FIG.

In step ST41, when the face movement determination unit 31 can detect the driver's face position from the current captured image acquired in step ST1 (step ST41 “YES”), the process proceeds to step ST31 and the face position cannot be detected. If so (step ST41 "NO"), the process proceeds to step ST42.

In step ST <b> 42, the state determination unit 15 b uses the second deviation degree that should have been calculated by the face movement determination unit 31 when the face movement determination unit 31 cannot detect the driver's face position in the captured image. It is determined that only the threshold determination using the first divergence calculated by the image determination unit 13 in step ST3 is performed without performing the determination. When it is determined that only the threshold determination using the first divergence degree is performed, the state determination unit 15b is more than the case where the threshold determination is performed using the first divergence degree and the second divergence degree (step ST32). 1 Threshold value is changed to a larger value.

In step ST43, when the first divergence calculated by the image determination unit 13 is greater than or equal to the first threshold after the change (step ST43 “YES”), the state determination unit 15b enters a state in which the driver cannot drive. The process proceeds to step ST5. On the other hand, when the first divergence calculated by the image determination unit 13 is less than the changed first threshold (step ST43 “NO”), the state determination unit 15b proceeds to step ST8.

As described above, when the face movement determination unit 31 cannot detect the driver's face position in the captured image, the state determination unit 15b according to the fourth embodiment uses the first divergence calculated by the image determination unit 13 as the first degree of deviation. When the first deviation degree is equal to or greater than the first threshold value when compared with the threshold value, it is determined that the driver is in an inoperable state if the eye detection unit 14 continues for a predetermined time or longer during which the driver's eyes cannot be detected. If the time during which the eyes of the driver cannot be detected by the eye detection unit 14 does not continue for a predetermined time or more, it is determined that the driver is in a normal state. The inoperability detecting device 10 can accurately determine the normal state and the inoperable state by not performing the threshold determination using the inaccurate second degree of deviation.

Moreover, the state determination part 15b of Embodiment 4 changes a 1st threshold value to a large value, when the face movement determination part 31 cannot detect a driver | operator's face position in a captured image. As a result, the driving impossibility detecting device 10 does not determine that there is a suspicion that the driver is in an inoperable state only by a slight movement of the driver's face position, and is in an inoperable state when the driver moves more greatly. It will be determined that there is doubt. Therefore, the inoperability detection apparatus 10 determines that there is an excessive suspicion that the inoperability is excessive when only the threshold determination using the first divergence is performed without performing the threshold determination using the second divergence. Can be prevented.

In addition, you may combine the structure of the vehicle equipment 2 of Embodiment 2, and the vehicle information acquisition part 21 with respect to the driving impossible detection apparatus 10 of Embodiment 4. FIG.

In the first to fourth embodiments, when there are a plurality of drivers who drive the same vehicle, the driving impossibility detecting device 10 performs personal authentication based on a captured image or the like, and builds a database 12 for each driver. That's fine. Then, the inoperability detecting device 10 may determine whether the driver is in a normal state or in an inoperable state using a normal image in the database 12 corresponding to the driver.

Finally, the hardware configuration of the inoperability detecting apparatus 10 according to each embodiment will be described.
9A and 9B are diagrams illustrating a hardware configuration example of the inoperability detecting apparatus 10 according to each embodiment. The databases 12 and 12a in the inoperability detecting apparatus 10 are the memory 102. Functions of the image acquisition unit 11, the image determination units 13 and 13 a, the eye detection unit 14, the state determination units 15 and 15 b, the registration units 16 and 16 a, the vehicle information acquisition unit 21, and the face movement determination unit 31 in the driving disability detection device 10. Is realized by a processing circuit. That is, the inoperability detecting device 10 includes a processing circuit for realizing the above function. The processing circuit may be the processing circuit 100 as dedicated hardware, or may be the processor 101 that executes a program stored in the memory 102.

As shown in FIG. 9A, when the processing circuit is dedicated hardware, the processing circuit 100 includes, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit). ), FPGA (Field Programmable Gate Array), or a combination thereof. The functions of the image acquisition unit 11, the image determination units 13 and 13 a, the eye detection unit 14, the state determination units 15 and 15 b, the registration units 16 and 16 a, the vehicle information acquisition unit 21, and the face movement determination unit 31 are a plurality of processing circuits 100. Alternatively, the functions of the respective units may be combined and realized by one processing circuit 100.

As shown in FIG. 9B, when the processing circuit is the processor 101, the image acquisition unit 11, the image determination units 13, 13a, the eye detection unit 14, the state determination units 15, 15b, the registration units 16, 16a, and the vehicle information acquisition The functions of the unit 21 and the face movement determination unit 31 are realized by software, firmware, or a combination of software and firmware. Software or firmware is described as a program and stored in the memory 102. The processor 101 reads out and executes a program stored in the memory 102, thereby realizing the function of each unit. That is, the inoperability detecting apparatus 10 includes a memory 102 for storing a program that, when executed by the processor 101, results in the steps shown in the flowchart of FIG. In addition, this program is a procedure of the image acquisition unit 11, the image determination units 13 and 13 a, the eye detection unit 14, the state determination units 15 and 15 b, the registration units 16 and 16 a, the vehicle information acquisition unit 21, and the face movement determination unit 31. It can also be said that the method is executed by a computer.

Here, the processor 101 is a CPU (Central Processing Unit), a processing device, an arithmetic device, a microprocessor, or the like.
The memory 102 may be a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), or a nonvolatile or volatile semiconductor memory such as a flash memory, a hard disk, a flexible disk, or the like. The magnetic disk may be a CD (Compact Disc) or a DVD (Digital Versatile Disc).

Part of the functions of the image acquisition unit 11, the image determination units 13 and 13 a, the eye detection unit 14, the state determination units 15 and 15 b, the registration units 16 and 16 a, the vehicle information acquisition unit 21, and the face movement determination unit 31. May be realized by dedicated hardware, and a part thereof may be realized by software or firmware. As described above, the processing circuit in the inoperability detecting apparatus 10 can realize the above-described functions by hardware, software, firmware, or a combination thereof.

In the present invention, within the scope of the invention, free combinations of the respective embodiments, modification of arbitrary components of the respective embodiments, or omission of arbitrary components of the respective embodiments are possible.

The inoperability detecting apparatus according to the present invention detects an inoperable state in consideration of individual differences in driving posture, and is therefore suitable for use in an inoperable detecting apparatus used in an occupant monitoring system for monitoring an occupant. ing.

DESCRIPTION OF SYMBOLS 1 Image pick-up device, 2 Vehicle equipment, 10 Unoperability detection apparatus, 11 Image acquisition part, 12, 12a database, 13, 13a Image determination part, 14 Eye detection part, 15, 15b State determination part, 16, 16a Registration part, 21 Vehicle information acquisition unit, 31 face movement determination unit, 100 processing circuit, 101 processor, 102 memory.

Claims (7)

1. An image acquisition unit for acquiring a captured image of the driver of the vehicle;
   A database that registers a captured image as a normal image when the driver is in a normal state among the captured images acquired by the image acquisition unit;
   An image determination unit that calculates a first divergence degree indicating a degree of divergence between the normal image registered in the database and the captured image acquired by the image acquisition unit;
   An eye detection unit that detects the eyes of the driver from the captured image acquired by the image acquisition unit;
   When the first deviation degree calculated by the image determination unit is compared with a first threshold value and the first deviation degree is equal to or greater than the first threshold value, the eye detection unit cannot detect the driver's eyes. Is continued for a predetermined time or more, it is determined that the driver is in an inoperable state, and if the time during which the eyes of the driver cannot be detected by the eye detection unit does not continue for the predetermined time or longer, the driving is performed. A state determination unit that determines that the person is in a normal state;
   A driving disability detection device comprising: a registration unit that registers the captured image acquired by the image acquisition unit as a normal image in the database when the state determination unit determines that the driver is in a normal state.
2. A vehicle information acquisition unit that acquires vehicle information indicating a state of the vehicle;
   The database registers a normal image in association with vehicle information indicating a state of the vehicle when the normal image is captured;
   The image determination unit calculates a first degree of divergence using a normal image associated with vehicle information that matches vehicle information acquired by the vehicle information acquisition unit among normal images registered in the database. The inoperability detecting device according to claim 1.
3. The said image determination part calculates the said 1st deviation degree based on the brightness | luminance of the normal image registered into the said database, and the brightness | luminance of the captured image acquired by the said image acquisition part. The inoperability detecting device as described.
4. Face movement determination for calculating a second divergence degree indicating a degree of divergence between the driver's face position in the normal image registered in the database and the driver's face position in the captured image acquired by the image acquisition unit. Part
   The state determination unit is configured such that the first divergence calculated by the image determination unit is equal to or greater than the first threshold, and the second divergence calculated by the face movement determination unit is equal to or greater than a second threshold. In this case, when the time during which the eyes of the driver cannot be detected by the eye detection unit continues for the predetermined time or longer, it is determined that the driver is incapable of driving, and the eyes of the driver are determined by the eye detection unit. 4. The driving disability detection device according to claim 3, wherein the driver is determined to be in a normal state unless the time during which the vehicle cannot be detected continues for the predetermined time or longer.
5. When the face movement determination unit cannot detect the driver's face position in the captured image, the state determination unit compares the first divergence calculated by the image determination unit with the first threshold, When the degree of deviation is equal to or greater than the first threshold, if the time during which the eyes of the driver cannot be detected by the eye detection unit continues for the predetermined time or longer, the driver determines that the driver is in an inoperable state. 5. The inability to drive according to claim 4, wherein if the time during which the eyes of the driver cannot be detected by the eye detection unit does not continue for the predetermined time or longer, the driver is determined to be in a normal state. Detection device.
6. 6. The driving impossibility detection according to claim 5, wherein the state determination unit changes the first threshold to a large value when the face movement determination unit cannot detect the driver's face position in the captured image. apparatus.
7. An image acquisition unit acquiring a captured image of the driver of the vehicle;
   An image determination unit refers to a database that registers, as a normal image, a captured image when the driver is in a normal state among the captured images acquired by the image acquisition unit, and a normal image registered in the database Calculating a first degree of deviation indicating a degree of deviation from a captured image acquired by the image acquisition unit;
   An eye detection unit detecting the driver's eyes from the captured image acquired by the image acquisition unit;
   The state determination unit compares the first divergence calculated by the image determination unit with a first threshold, and when the first divergence is equal to or greater than the first threshold, When the time during which the eyes cannot be detected continues for a predetermined time or more, the driver determines that the driver is incapable of driving, and the time during which the eyes of the driver cannot be detected by the eye detection unit continues for the predetermined time or longer. Otherwise, determining that the driver is in a normal state;
   When the state determination unit determines that the driver is in a normal state, the registration unit registers the captured image acquired by the image acquisition unit as a normal image in the database. .

Images