JP7450715B2 - Inoperability determination device and inoperability determination method - Google Patents

Description

The present disclosure relates to a technique for determining that a driver is in an inoperable state in which he or she cannot drive a vehicle.

Patent Document 1 discloses a technique for detecting that a driver is unable to drive. The technology of Patent Document 1 detects the driver's head position using a captured image of the driver, and detects a state in which the head position is outside a predetermined range or the direction of the face is outside the predetermined direction. If the condition continues for a certain period of time or more, the driver is determined to be unable to drive.

Japanese Patent Application Publication No. 2018-173996

However, even if the head position or the direction of the face does not deviate from a predetermined position or range, the driver may be unable to drive. For example, a driver may faint or become stiff due to an epileptic attack. The technique disclosed in Patent Document 1 has a problem in that it is not possible to determine whether the vehicle is inoperable in such a stationary state.

The present disclosure has been made to solve the above-mentioned problems, and aims to provide a technology for determining whether a driver is unable to drive when the driver is stationary.

A first driveability determination device of the present disclosure analyzes a captured image of a camera mounted on a vehicle and captures the driver of the vehicle, thereby obtaining information related to the driver's head, including positional information of the driver's face area. an image analysis unit that acquires characteristic information; and an immobility determination unit that determines whether or not the driver is in an immobile state based on a change in the characteristic information between two frames having a certain temporal relationship in a photographed video. , an inability determining unit that determines that the driver is in an inoperable state when the duration of the immobile state of the driver is equal to or greater than a second threshold; Based on the position information, the overlapping area ratio of the face area between two frames is calculated, and when the overlapping area ratio is equal to or higher than the first threshold value, it is determined that the driver is in an immobile state, and the feature information is The immobility determining unit determines whether, between the two frames, the overlapping area ratio is greater than or equal to a first threshold, and the amount of change in face direction, which is the amount of change in the direction of the driver's face, is less than a third threshold. In this case, it is determined that the driver is in an immobile state.
The second driveability determination device of the present disclosure analyzes images taken by a camera mounted on a vehicle that photographs the driver of the vehicle, thereby obtaining information related to the driver's head, including positional information of the driver's face area. an image analysis unit that acquires characteristic information; and an immobility determination unit that determines whether or not the driver is in an immobile state based on a change in the characteristic information between two frames having a certain temporal relationship in a photographed video. , an inability determining unit that determines that the driver is in an inoperable state when the duration of the immobile state of the driver is equal to or greater than a second threshold; Based on the position information, the overlapping area ratio of the face area between the two frames is calculated, and when the overlapping area ratio is equal to or higher than the first threshold value, it is determined that the driver is in an immobile state, and the disability determination unit From the position of the driver's face area acquired by the analysis unit, the moving distance, which is the total distance the driver's face area has moved within a certain time period, which is longer than the time between two frames, is calculated, and the movement distance is calculated. Based on the distance, the driver is determined to be unable to drive.

The driving incapacity determination method of the present disclosure obtains characteristic information about the driver's head, including positional information of the driver's face area, by analyzing video captured by a camera mounted on a vehicle that photographs the driver of the vehicle. Based on the change in characteristic information between two frames that have a certain temporal relationship in the photographed video, it is determined whether or not the driver is in an immobile state, and the duration of the driver's immobile state is determined. is greater than or equal to the second threshold, the driver is determined to be unable to drive, and based on the position information of the face area in each of the two frames, the overlapping area ratio of the face area between the two frames is calculated, and the overlapping area ratio of the face area between the two frames is calculated. When the area ratio is greater than or equal to the first threshold, it is determined that the driver is in an immobile state , and the feature information includes information on the direction of the driver's face, and between the two frames, the superimposed area ratio is the first When the face direction change amount, which is the amount of change in the direction of the driver's face, is equal to or greater than the threshold value and less than the third threshold value, it is determined that the driver is in an immobile state .

According to the technology of the present disclosure, in order to determine that the driver is unable to drive when the duration of the immobile state of the driver is equal to or greater than the second threshold value, it is possible to determine the inoperable state in the immobile state of the driver. I can do it. Objects, features, aspects, and advantages of the present disclosure will become more apparent from the following detailed description and accompanying drawings.

1 is a diagram showing the configuration of an inoperability determination device according to a first embodiment; FIG. 2 is a flowchart showing the overall processing of the inoperability determination device according to the first embodiment. 3 is a flowchart showing immobility determination processing of the inoperability determination device according to the first embodiment. FIG. 3 is a diagram showing a change in a face area between two frames. 2 is a flowchart showing an inoperability determination process of the inoperability determination device according to the first embodiment. FIG. 3 is a diagram showing the configuration of an inoperability determination device according to a second embodiment. 12 is a flowchart showing immobility determination processing of the inoperability determination device according to the second embodiment. FIG. 7 is a diagram showing the configuration of an inoperability determination device according to a third embodiment. 13 is a flowchart showing immobility determination processing of the inoperability determination device according to the third embodiment. FIG. 7 is a diagram showing the configuration of an inoperability determination device according to a fourth embodiment. 12 is a flowchart showing immobility determination processing of the inoperability determination device according to the fourth embodiment. FIG. 3 is a diagram showing how a vehicle travels around a curve. FIG. 3 is a diagram showing the direction of the driver's face in a conscious state. FIG. 3 is a diagram showing the direction of the face of an unconscious driver. It is a figure showing the composition of the inoperability judgment device of Embodiment 5. 12 is a flowchart showing immobility determination processing of the inoperability determination device according to the fifth embodiment. FIG. 7 is a diagram showing the configuration of an inoperability determination device according to a sixth embodiment. 12 is a flowchart showing the inoperability determination process of the inoperability determination device according to the sixth embodiment. FIG. 2 is a diagram showing a hardware configuration of an inoperability determination device. FIG. 2 is a diagram showing a hardware configuration of an inoperability determination device. It is a figure showing the example of composition of the drivability judgment device of Embodiment 6 including a vehicle and a server.

The inoperability determining device of each embodiment described below determines that the driver of the vehicle is in an inoperable state when the driver of the vehicle continues to be immobile. Hereinafter, a vehicle in which a driver, whose inability to drive state is to be determined by the inoperability determination device, is boarded will be simply referred to as a vehicle. Moreover, in each embodiment described below, the same reference numerals are given to the same or corresponding components. Therefore, components labeled with the same reference numerals are the same or correspond between different embodiments.

<A. Embodiment 1>
<A-1. Configuration>
FIG. 1 is a diagram showing the configuration of an inoperability determination device 101 and its peripheral devices according to the first embodiment. The driving impossibility determination device 101 is connected to the imaging device 50 and is configured to be able to use it.

The image capturing device 50 is a device that captures an image of a driver, and is mounted on a vehicle. The imaging device 50 includes a camera 51 and an LED 52. The camera 51 mainly images the driver's head including the face. The camera 51 does not take pictures only at a specific moment, but continuously takes pictures for a certain period of time. Each frame of the video shot by the camera 51 is output to the image analysis section 10A. The frame rate of the captured video is, for example, 10 fps or 30 fps. The LED 52 irradiates the driver's face with light so that the camera 51 can clearly photograph the driver's face.

The driveability determination device 101 is typically mounted on a vehicle, but <G. Hardware Configuration> As will be described later in ``Hardware Configuration'', each component of the inoperability determination device 101 does not necessarily need to be installed in the vehicle. The inoperability determination device 101 includes an image analysis section 10A, an immobility determination section 20A, and an inoperability determination section 30A.

The image analysis unit 10A acquires characteristic information (hereinafter simply referred to as "characteristic information") regarding the driver's head by analyzing each frame of the image captured by the camera 51. Specifically, the image analysis section 10A includes a face position detection section 11. The face position detection unit 11 analyzes each frame of the image captured by the camera 51, detects a face area in each frame, and obtains position information of the face area. That is, the feature information includes position information of the driver's face area.

The immobility determining unit 20A determines whether the driver is in an immobile state based on the characteristic information. Typically, a driver makes some head movements while driving to check various objects or people around the vehicle, such as nearby vehicles or obstacles. However, when the body becomes rigid for some reason, such as during an epileptic attack, the movement of the head becomes noticeably smaller than normal. In this specification, a state in which the driver does not move to an extent that is unnatural for a state in which the driver is driving is referred to as an "immobile state."

Specifically, the immobility determination unit 20A includes a superimposition ratio calculation unit 21. The superimposition ratio calculation unit 21 calculates the superimposition area ratio of the face area between two frames from the position of the face area in each frame of the photographed video detected by the face position detection unit 11. Here, the two frames whose face regions are compared are typically two consecutive frames in time series, but are not limited to this, as long as they have a certain temporal relationship. For example, the superimposition ratio calculation unit 21 may calculate the superimposition area ratio of the face area in two frames selected every two frames or every ten frames.

The immobility determining unit 20A determines that the driver is in an immobile state when the overlapping area ratio of the face area between two frames is greater than or equal to a predetermined first threshold Th1. The immobility determination unit 20A repeats the immobility determination process for each new frame of the image captured by the camera 51.

The inability determining unit 30A determines that the driver is in an inoperable state in which driving is not possible when the time period during which the driver is determined to be in an immobile state by the immobility determining unit 20A continues for a predetermined second threshold Th2 or more. It is determined that The second threshold Th2 is, for example, 90 seconds.

<A-2. Operation＞
FIG. 2 is a flowchart showing the overall processing of the inoperability determination device 101. The overall processing of the inoperability determination device 101 will be described below with reference to FIG. The flow shown in FIG. 2 starts, for example, when the vehicle starts traveling, and is repeated while the vehicle is traveling. Further, it is assumed that the imaging device 50 always takes pictures, for example, after the accessory power of the vehicle is turned on.

First, the image analysis unit 10A acquires a captured image of the camera 51, analyzes the captured image, and obtains characteristic information regarding the driver's head in each frame of the captured image (step S101). More specifically, in this embodiment, the face position detection unit 11 acquires the position of the driver's face area in each frame.

Next, the immobility determining unit 20A determines whether the driver is in an immobile state based on the image analysis result of the image analyzing unit 10A (step S102). Details of this step will be described later with reference to FIG.

Next, the inability determining unit 30A determines whether the driver is in an inoperable state based on the determination result of the immobility determining unit 20A (step S103). Details of this step will be described later with reference to FIG. The inoperability determination device 101 then returns to the process of step S101.

FIG. 3 is a flowchart showing details of the immobility determination process (step S102 in FIG. 2) in the immobility determination unit 20A. The immobility determination process will be described below with reference to FIG. First, the superimposition ratio calculation unit 21 acquires the position of the driver's face area in each frame from the face position detection unit 11, and calculates the overlap area ratio of the face area between two specific frames.

FIG. 4 shows the overlapping area of face regions in two frames. In FIG. 4, the x-axis is set in the horizontal direction and the y-axis is set in the vertical direction. The position coordinates of the face area in the previous frame 202 are represented by (X', Y'), and the position coordinates of the face area in the current frame 201 are represented by (X, Y). From these positional coordinates of the face area, the overlapping ratio calculation unit 21 can obtain the overlapping area 203 of the face area in both frames. Then, the superimposition ratio calculating unit 21 calculates the superimposition area ratio from the area of the face region and the area of the superimposition region 203.

The immobility determining unit 20A determines whether the overlapping area ratio is equal to or greater than a threshold Th1 (step S1021). For example, assuming that the driver does not move at all, the position of the driver's face area will be exactly the same between two frames and will be completely overlapped, so the overlap area ratio will be 1. The larger the movement of the driver's face area, the smaller the overlapping area ratio becomes. Therefore, when the superimposed area ratio is equal to or greater than the threshold Th1, the immobility determining unit 20A determines that the driver is in an immobile state, and counts up the immobility determination time T0 (step S1022). On the other hand, if the overlapping area ratio is less than the threshold Th1, the immobility determining unit 20A determines that the driver is not in an immobile state, and resets the immobility determination time T0 (step S1023). With this, the immobility determination unit 20A ends the immobility determination process.

FIG. 5 is a flowchart showing details of the impossibility determination process (step S103 in FIG. 2) in the impossibility determination section 30A. The impossibility determination process will be described below with reference to FIG. First, the impossibility determination unit 30A determines whether the immobility determination time T0 is equal to or greater than the threshold Th2 (step S1031). The immobility determination time T0 is counted up by the immobility determination unit 20A in step S1022 of FIG. 3, or reset in step S1023. If T0≧Th2 in step S1031, the inability determining unit 30A determines that the driver remains immobile and is in an inoperable state (step S1032). On the other hand, if T0<Th2 in step S1031, the inability determining unit 30A determines that the driver is not in an inoperable state (step S1033). With this, the impossibility determining section 30A ends the impossibility determining process.

<A-3. Effect>
The driving impossibility determination device 101 of the first embodiment analyzes a captured image of a camera 51 that is mounted on a vehicle and photographs the driver of the vehicle, and obtains information related to the driver's head, including positional information of the driver's face area. The image analysis unit 10A that acquires feature information calculates the overlapping area ratio of the driver's face area between the two frames based on the change in feature information between two frames having a certain temporal relationship of the photographed video. The immobility determination unit 20A determines whether or not the driver is in an immobile state when the first threshold value Th1 or more is reached, and the immobility determination unit 20A determines whether the driver is in an immobile state when the duration of the immobility state of the driver is equal to or more than a second threshold value. It includes an incapacity determining section 30A that determines that the state is in an incapacitated state.

With the above configuration, the driving impossibility determination device 101 can determine whether the driver is in an immobile state based on the overlapping area ratio of the driver's face area between two frames of the image captured by the camera 51. I can do it. According to the inoperability determining device 101, if the driver remains immobile for a certain period of time or more, it can be determined that the driver is in an inoperable state.

<B. Embodiment 2>
<B-1. Configuration>
FIG. 6 is a diagram showing the configuration of the inoperability determining device 102 and its peripheral devices according to the second embodiment. The driving impossibility determination device 102 is connected to the imaging device 50, the output device 60, and the vehicle control device 70, and is configured to be able to use these. Hereinafter, parts of the inoperability determining device 102 that are different from the inoperable determining device 101 will be mainly explained.

The output device 60 is a device mounted on the vehicle that outputs the determination result of the impossibility determination section 30A. In the example of FIG. 6, the output device 60 includes a display device 61 and a speaker 62, but it is sufficient to include at least one of these. When the inability determining unit 30A determines that the driver is in an inoperable state, for example, a message "The driver is incapable of driving." is displayed on the display device 61 or output as a sound from the speaker 62. This is expected to allow fellow passengers in the vehicle or passersby outside the vehicle to understand that the driver is unable to drive, and to take appropriate measures.

The vehicle control device 70 acquires the determination result of the impossibility determining section 30A, and performs automatic driving control of the vehicle based on the determination result. When the inability determination unit 30A determines that the driver is in an inoperable state, the vehicle control device 70 finds an emergency evacuation space in front of the vehicle and makes an emergency stop of the vehicle in the emergency evacuation space, thereby ensuring the safety of the vehicle occupants. secure.

The inoperability determination device 102 includes an image analysis section 10B, an immobility determination section 20B, and an inoperability determination section 30A. The image analysis section 10B includes a face direction detection section 12 in addition to the configuration of the image analysis section 10A of the first embodiment. The face direction detection unit 12 analyzes each frame of the image captured by the camera 51 and detects the direction of the driver's face in each frame. That is, in the second embodiment, the feature information includes information on the direction of the driver's face.

The immobility determination unit 20B includes a facial direction change calculation unit 22 in addition to the configuration of the immobility determination unit 20A of the first embodiment. The face direction change calculation unit 22 calculates the amount of change in the face direction between two frames (hereinafter referred to as the “face direction change amount”) from the face direction in each frame of the photographed video detected by the face direction detection unit 12. calculate. Here, the two frames for which the face direction change calculation unit 22 calculates the amount of change in the face direction are the same as the two frames for which the superimposition ratio calculation unit 21 calculates the overlap area ratio of the face area. The face direction change amount is expressed by three angles: yaw angle, pitch angle, and roll angle.

The immobility determining unit 20B determines that the driver is in an immobile state when the overlapping area ratio of the face area is greater than or equal to the first threshold Th1 and the face direction change amount is less than a predetermined third threshold. In this embodiment, since the amount of change in face direction is taken into consideration in determining whether the driver is in an immobile state, the face position hardly changes between frames, but if the face direction is changing, the driver is incorrectly determined to be in an immobile state. This can be prevented. Examples of actions in which the face position changes little between frames but the face direction changes include actions such as looking in a rearview mirror and visually checking the display screen of a navigation device.

<B-2. Operation＞
FIG. 7 is a flowchart showing details of the immobility determination process (step S102 in FIG. 2) in the immobility determination unit 20B. The immobility determination process will be described below with reference to FIG. First, as in the first embodiment, the superimposition ratio calculation unit 21 calculates the superimposition area ratio of the face area between two specific frames. Then, the immobility determination unit 20B determines whether the superimposed area ratio is equal to or greater than the threshold Th1 (step S1021). If the superimposed area ratio is less than the threshold Th1 in step S1021, the immobility determination unit 20B resets the immobility determination time T0 as in the first embodiment (step S1023). If the superimposed area ratio is equal to or greater than the threshold Th1 in step S1021, the face direction change calculation unit 22 calculates the face direction change amount between two specific frames. Then, the immobility determining unit 20B determines whether the face direction change amount is less than the threshold Th3 (step S1021A).

If the face direction change amount is less than the threshold Th3 in step S1021A, the immobility determining unit 20B determines that the driver is in an immobile state, and counts up the immobility determination time T0 (step S1022). On the other hand, if the face direction change amount is equal to or greater than the threshold Th3 in step S1021A, the immobility determining unit 20B determines that the driver is not in an immobile state, and resets the immobility determination time T0 (step S1023). With this, the immobility determination unit 20B ends the immobility determination process.

<B-3. Effect>
In the driveability determination device 102 of the second embodiment, the feature information acquired by the image analysis unit 10B includes information on the direction of the driver's face, and the immobility determination unit 20B determines whether the superimposed area ratio is the highest between two frames. It is determined that the driver is in an immobile state when the face direction change amount, which is the amount of change in the direction of the driver's face, is equal to or more than one threshold value and less than a third threshold value. Therefore, according to the driveability determining device 102, when the face position hardly changes between frames but the face direction changes, it is possible to prevent the driver from being erroneously determined to be in an immobile state.

<C. Embodiment 3>
<C-1. Configuration>
FIG. 8 is a diagram showing the configuration of the inoperability determination device 103 and its peripheral devices according to the third embodiment. The driving impossibility determination device 103 is connected to the imaging device 50, the output device 60, and the vehicle control device 70, and is configured to be able to use these. Hereinafter, the parts of the inoperability determining device 103 that are different from the inoperable determining device 102 will be mainly explained.

The inoperability determination device 103 includes an image analysis section 10C, an immobility determination section 20C, and an inoperability determination section 30A. The image analysis section 10C includes a line-of-sight direction detection section 13 in addition to the configuration of the image analysis section 10B of the second embodiment. The line-of-sight direction detection unit 13 analyzes each frame of the image captured by the camera 51 and detects the driver's line-of-sight direction in each frame. That is, in the third embodiment, the feature information includes information on the driver's line of sight direction.

The immobility determination unit 20C includes a line-of-sight direction change calculation unit 23 in addition to the configuration of the immobility determination unit 20B of the second embodiment. The line-of-sight direction change calculation unit 23 calculates the amount of line-of-sight direction change between two frames from the line-of-sight direction in each frame of the photographed video detected by the line-of-sight direction detection unit 13. Here, the two frames for which the line-of-sight direction change calculation unit 23 calculates the amount of change in the line-of-sight direction are the same as the two frames for which the superimposition ratio calculation unit 21 calculates the overlap area ratio of the face area.

The immobility determination unit 20C determines when the overlapping area ratio of the face area is equal to or higher than the first threshold Th1, the amount of change in face direction is less than a predetermined third threshold, and the amount of change in line of sight direction is less than a fourth predetermined threshold. It is determined that the driver is not moving. In this embodiment, since the amount of change in the direction of the driver's gaze is taken into consideration in determining the immobile state, the face position and face direction hardly change between frames, but if the direction of the driver's gaze changes, the driver is determined to be in the immobile state. Misjudgment can be prevented.

<C-2. Operation>
FIG. 9 is a flowchart showing details of the immobility determination process (step S102 in FIG. 2) in the immobility determination unit 20C. Step S1021 and step S1021A are the same as in the second embodiment. If the immobility determining unit 20B determines that the face direction change amount is less than the threshold Th3 (Yes in step S1021A), the line-of-sight direction change calculation unit 23 calculates the line-of-sight direction change amount between two specific frames. Then, the immobility determination unit 20C determines whether the amount of change in the line of sight direction is less than the threshold Th4 (step S1021B). If the line-of-sight direction change amount is less than the threshold Th4 in step S1021B, the immobility determining unit 20C determines that the driver is in an immobile state, and counts up the immobility determination time T0 (step S1022). On the other hand, if the face direction change amount is equal to or greater than the threshold Th4 in step S1021B, the immobility determining unit 20C determines that the driver is not in an immobile state, and resets the immobility determination time T0 (step S1023). With this, the immobility determination unit 20B ends the immobility determination process.

In the above description, the driving impossibility determining apparatus 102 of the second embodiment in which the line-of-sight direction detection section 13 and the line-of-sight direction change calculation section 23 are added is referred to as the driving impossibility determining apparatus 103 of the third embodiment. However, the line-of-sight direction detection section 13 and the line-of-sight direction change calculation section 23 may be added to the driveability determination device 101 of the first embodiment.

<C-3. Effect>
In the driving impossibility determination device 103 of the third embodiment, the feature information includes information on the direction of the driver's line of sight, and the immobility determination unit 20C determines whether the overlapping area ratio is greater than or equal to the first threshold value and the face direction between the two frames. When the amount of change is less than the third threshold and the amount of change in the direction of the line of sight, which is the amount of change in the direction of the driver's line of sight, is less than the fourth threshold, it is determined that the driver is in an immobile state. Therefore, according to the driving impossibility determination device 103, when the face position and face direction hardly change between frames, but the line of sight direction changes, it is possible to prevent the driver from being erroneously determined to be in an immobile state.

<D. Embodiment 4>
<D-1. Configuration>
FIG. 10 is a diagram showing the configuration of the inoperability determination device 104 and its peripheral devices according to the fourth embodiment. The driving impossibility determination device 104 is connected to the imaging device 50, the output device 60, the vehicle control device 70, and the vehicle information communication device 80, and is configured to be able to use these devices. Vehicle information communication device 80 transmits vehicle information to drivability determination device 104 . The vehicle information includes information regarding the driving status of the vehicle or information regarding the operation. Examples of the vehicle information include a vehicle speed signal, a steering angle signal, shift position information, seatbelt wearing information, turn signal operation information, an accelerator signal, or a brake signal.

Hereinafter, the parts of the inoperability determining device 104 that are different from the inoperable determining device 103 will be mainly explained. The inoperability determination device 104 includes an image analysis section 10C, an immobility determination section 20D, and an inoperability determination section 30A.

The immobility determination unit 20D includes an unconsciousness determination unit 24 in addition to the configuration of the immobility determination unit 20C of the third embodiment. The unconscious determination unit 24 determines that the change in the face direction is induced by the behavior of the vehicle when the driver's face direction changes, that is, when the amount of change in the driver's face direction is equal to or greater than a third threshold, Determine whether the driver is unconscious. The immobility determination unit 20D determines that the driver is in an immobile state when the unconsciousness determination unit 24 determines that the driver is in an unconscious state even if the amount of change in the driver's face direction is equal to or greater than the third threshold value.

An example of a change in the direction of the face induced by the behavior of a vehicle is when the head changes vertically due to vibrations of the vehicle. Humans tend to maintain their posture when they are conscious. Therefore, when the vehicle body vibrates up and down depending on the road surface condition, the relationship between the vertical movement of the head and the amount of change in the pitch direction of the face differs depending on whether the driver is conscious or not. Therefore, the unconsciousness determination unit 24 acquires the position of the face area from the face position detection unit 11 and the face direction from the face direction detection unit 12. If the index value determined by the variance of the position Y is greater than or equal to the threshold value, it can be determined that the driver is in an unconscious state.

Another example of changes in facial orientation induced by vehicle behavior is head movement during vehicle turns. FIG. 11 shows the vehicle 204 turning to the left. At this time, the direction of change in the steering angle is to the left as shown by arrow 209 in FIG. 12, and for the driver 205, centrifugal force is generated in the right direction opposite to the direction in which the steering angle changes, as shown by arrow 208. When the driver is conscious, he turns his head in the direction of travel of the vehicle 204 to confirm safety, so his face moves in the direction shown by arrow 206 and then in the direction shown by arrow 207. In this way, if the steering angle changes to the left, the yaw angle of the face also tilts to the left in accordance with the change in the steering angle. In other words, the face yaws in the direction of the change in steering angle.

On the other hand, when the driver is unconscious, his or her face tilts to the right due to centrifugal force, as shown in FIG. 13. In other words, the face rolls in the opposite direction to the direction of change of the steering angle. Therefore, the unconsciousness determination unit 24 acquires information on the steering angle of the vehicle from the vehicle information communication device 80, acquires the direction of the driver's face from the face direction detection unit 12, and determines that the rolling direction of the driver's face is equal to the steering angle. If the direction is opposite to the direction of change, it can be determined that the driver is in an unconscious state.

Further, if the roll angle of the face is equal to or greater than a predetermined threshold value, the unconsciousness determining unit 24 can determine that the driver is in an unconscious state.

<D-2. Operation＞
FIG. 14 is a flowchart showing details of the immobility determination process (step S102 in FIG. 2) in the immobility determination unit 20D. Step S1021 and step S1021A are the same as in the third embodiment. When the immobility determination unit 20D determines that the amount of change in facial direction is equal to or greater than the threshold Th3 (No in step S1021A), the unconsciousness determination unit 24 determines whether the driver is unconscious (step S1021AB).

If the driver is not in an unconscious state in step S1021AB, the immobility determining unit 20D determines that the driver is not in an immobile state, and resets the immobility determination time T0 (step S1023). On the other hand, if the driver is unconscious in step S1021AB, the immobility determination unit 20D proceeds to the process of step S1021B. The processing after step S1021B is the same as in the third embodiment.

<D-3. Effect>
In the driveability determining device 104 of the fourth embodiment, the immobility determining section 20D includes an unconscious determining section 24 that determines whether or not the driver is unconscious based on the direction of the driver's face. If so, it is determined that the driver is in an immobile state even if the face direction change amount is equal to or greater than the third threshold between two frames. Therefore, according to the driveability determining device 104, when the direction of the unconscious driver's face changes significantly with the behavior of the vehicle, it is possible to prevent the driver from erroneously determining that the driver is not in an immobile state.

The unconsciousness determining unit 24 can determine whether or not the driver is unconscious based on the variance of the vertical position of the driver's face area and the variance of the pitch angle in the direction of the driver's face. . This can prevent an erroneous determination that the driver is not in an immobile state when the pitch angle in the direction of the unconscious driver's face changes significantly due to vibrations of the vehicle.

Alternatively, the unconsciousness determining unit 24 can determine whether the driver is unconscious based on the direction of change in the steering angle of the vehicle and the direction of the driver's face. Thereby, when the direction of the unconscious driver's face changes significantly due to the centrifugal force acting on the vehicle, it is possible to prevent an erroneous determination that the driver is not in an immobile state.

<E. Embodiment 5>
<E-1. Configuration>
FIG. 15 is a diagram showing the configuration of the inoperability determination device 105 and its peripheral devices according to the fifth embodiment. The driving impossibility determination device 105 is connected to the imaging device 50, the output device 60, the vehicle control device 70, and the vehicle information communication device 80, and is configured to be able to use these devices. The inoperability determination device 105 includes an image analysis section 10C, an immobility determination section 20E, and an inoperability determination section 30A. The configuration of immobility determining section 20E is similar to immobility determining section 20D of the fourth embodiment.

The line-of-sight direction detection unit 13 analyzes each frame of the image captured by the camera 51 and detects the driver's line-of-sight direction in each frame. The line-of-sight direction change calculation unit 23 calculates the amount of line-of-sight direction change between two frames from the line-of-sight direction in each frame of the photographed video detected by the line-of-sight direction detection unit 13. The immobility determining unit 20E determines that the amount of change in the line-of-sight direction is less than the fourth threshold as one of the conditions for determining that the driver is in an immobile state.

However, the line-of-sight direction detection unit 13 cannot always detect the driver's line-of-sight direction. For example, if the driver is wearing sunglasses or his eyes are hidden by his hair, the line-of-sight direction detection unit 13 cannot detect the line-of-sight direction. In such a case, the line-of-sight direction detection unit 13 outputs information indicating that the line-of-sight direction cannot be detected to the immobility determination unit 20E. Naturally, the line-of-sight direction change calculation unit 23 cannot calculate the amount of change in the line-of-sight direction. Therefore, the immobility determination unit 20E excludes the amount of change in the direction of the line of sight from the conditions for determining that the driver is in an immobility state, and similarly to the immobility determination unit 20B of the second embodiment, the overlapping area ratio of the face area and the face direction. Immobility determination processing is performed based on the amount of change.

Since the immobility determination process based on the amount of change in the direction of the line of sight cannot be performed, the accuracy of the immobility determination may be reduced, and there is a risk that the driver may be erroneously determined to be in an immobile state. Therefore, when the line-of-sight direction detection unit 13 cannot detect the line-of-sight direction, the immobility determination unit 20E sets the first threshold Th1, which is the threshold of the overlapping area ratio of the face area, and the immobility determination time, compared to the case where the line-of-sight direction detection unit 13 cannot detect the line-of-sight direction. By increasing the second threshold value Th2, which is the threshold value, it becomes difficult to determine that the vehicle is in an immobile state, and furthermore, it becomes difficult to determine that the vehicle is in an inoperable state. Note that here, the immobility determination unit 20E may increase only one of the first threshold Th1 and the second threshold Th2.

<E-2. Operation>
FIG. 16 is a flowchart showing details of the immobility determination process (step S102 in FIG. 2) in the immobility determination unit 20E. Hereinafter, the immobility determination process of the immobility determination unit 20E will be explained along FIG. 16. First, the immobility determining unit 20E determines whether the line of sight direction can be detected (step S1020A). When the immobility determination unit 20E obtains the driver's line of sight direction in each frame of the image captured by the camera 51 from the line of sight direction detection unit 13, it determines that the line of sight direction can be detected. Further, when the immobility determination unit 20E obtains information indicating that the line-of-sight direction cannot be detected from the line-of-sight direction detection unit 13, it determines that the line-of-sight direction cannot be detected.

If the line of sight direction can be detected in step S1020A, the immobility determination unit 20E sets Th1A to the first threshold Th1, which is the threshold for the overlapping area ratio of the face area, and sets Th1A to the second threshold Th2, which is the threshold for the immobility determination time T0. Th2A is set (step S1020B). On the other hand, if the line of sight direction cannot be detected in step S1020B, the immobility determining unit 20E sets Th1B to the first threshold Th1, which is the threshold for the overlapping area ratio of the face area, and sets Th1B to the second threshold Th1, which is the threshold for the immobility determination time T0. Th2B is set as the threshold Th2 (step S1020C). Here, Th1A<Th1B and Th2A<Th2B. That is, when the line-of-sight direction detecting unit 13 cannot detect the line-of-sight direction, it is less likely to be determined to be in an immobile state, and furthermore, it is less likely to be determined to be in an inoperable state, compared to a case where the line-of-sight direction can be detected.

Subsequent steps S1021, S1021A, and S1021AB are the same as in the fourth embodiment. In step S1021A, if the face direction change amount is less than the threshold Th3, or if the driver is unconscious in step S1021AB, the immobility determination unit 20E determines whether the line of sight direction can be detected in the same manner as in step S1020A ( Step S1021AC). If the line-of-sight direction cannot be detected in step S1021AC, the immobility determination unit 20E skips step S1021B and counts up the immobility determination time T0 (step S1022). On the other hand, if the line-of-sight direction can be detected in step S1021AC, the immobility determination unit 20E proceeds to the process of step S1021B. The processing after step S1021B is the same as in the fourth embodiment.

<E-3. Effect>
In the driveability determination device 105 of Embodiment 5, if the image analysis unit 10C cannot acquire the direction of the driver's line of sight from the photographed video, the immobility determination unit 20E excludes the amount of change in the direction of the line of sight from the conditions for determining the immobility state. , at least one of the first threshold value and the second threshold value is made larger than in the case where the image analysis unit 10C can acquire the direction of the driver's line of sight from the captured video. Thereby, according to the driving impossibility determination device 105, it is possible to take into account the decrease in accuracy of immobility determination due to not taking into account the amount of change in the line of sight direction, and to suppress erroneously determining that the driver is in an immobile state.

<F. Embodiment 6>
In Embodiments 1 to 5, if the driver remains immobile for a certain period of time or more, the driver is determined to be unable to drive. However, in the sixth embodiment, by adding other conditions to the conditions for determining the inoperable state, the possibility that the driver mistakenly determines that the driver is in the inoperable state when the driver is not in the inoperable state is further reduced. .

<F-1. Configuration>
FIG. 17 is a diagram showing the configuration of the inoperability determination device 106 and its peripheral devices according to the sixth embodiment. The driving impossibility determination device 106 is connected to the imaging device 50, the output device 60, the vehicle control device 70, and the vehicle information communication device 80, and is configured to be able to use these devices. The inoperability determination device 106 includes an image analysis section 10D, an immobility determination section 20E, and an inoperability determination section 30B.

The image analysis section 10D includes an eye opening/closing degree detection section 14 in addition to the configuration of the image analysis section 10C of the fifth embodiment. The eye opening/closing degree detecting unit 14 acquires a video shot by the camera 51, analyzes each frame of the shot video, and detects the degree of opening/closing of the driver's eyes in each frame. The eye opening/closing degree detection section 14 outputs the detected degree of eye opening/closing to the disability determining section 30B.

The impossibility determining section 30B includes an eye-closing frequency counting section 31, a face area movement distance calculating section 32, and an abnormal operation detecting section 33. The eye-closing frequency counting section 31 counts the number of times the driver's eyes are closed within a certain period of time based on the degree of opening/closing of the driver's eyes in each frame of the photographed video, which is obtained from the eye-opening/closing degree detection section 14 . It is preferable that the eye-closed number counting section 31 counts the number of times the eyes are closed during the time when the immobility determining section 20E determines that the driver is in an immobile state. Therefore, for example, when the immobility determination time T0 starts counting up, the eye closure count unit 31 starts counting the number of eye closures, and when the immobility determination time T0 is reset, it resets the count of the eye closure times, and the immobility determination time T0 is reset. When the second threshold value Th2 or more is reached, counting of the number of times the eyes are closed ends. In this case, the counting time of the number of times the eyes are closed becomes equal to the second threshold Th2. Note that the count time of the number of times the eyes are closed may be less than the second threshold Th2.

The face area movement distance calculation unit 32 acquires position information of the driver's face area in each frame of the photographed video from the face position detection unit 11. Then, the face area movement distance calculation unit 32 calculates the movement distance of the driver's face area within a certain period of time from the position information of the driver's face area in each frame of the photographed video. Here, the movement distance of the face area does not mean the difference between the positions of the face area at the start and end of the calculation time, but rather the total distance that the face area has moved within the calculation time. The calculation time of the movement distance of the face area is at least several times longer than the time between two frames compared in the immobility determination process of the immobility determination unit 20E, and is set similarly to the time for counting the number of times the eyes are closed.

The abnormal operation detection unit 33 acquires vehicle information from the vehicle information communication device 80, and detects whether an abnormal operation is performed on the vehicle based on the vehicle information. An example of abnormal operation is repeatedly changing lanes without turning on the turn signal. This can be determined from the operation information of the direction indicator and the steering angle. Further, the abnormal operation detection unit 33 can detect sudden acceleration or deceleration as an abnormal operation from an accelerator signal, a brake signal, or a vehicle speed signal.

<F-2. Operation>
FIG. 18 is a flowchart showing the impossibility determination process by the impossibility determination unit 30B of the sixth embodiment. The impossibility determination process will be described below along the flowchart of FIG. First, the impossibility determination unit 30B determines whether the immobility determination time T0 is equal to or greater than the threshold Th2 (step S1031). If T0<Th2 in step S1031, the inability determining unit 30B determines that the driver is not in an inoperable state (step S1033). These processes are similar to the impossibility determination process by the impossibility determination unit 30A.

Even if T0≧Th2 in step S1031, the inability determining unit 30B does not immediately determine that the driver is in an inoperable state, and performs the following processing. First, the incapacity determining unit 30B determines whether the number of times n of eyes closed by the driver within a certain period of time counted by the eyes closed count unit 31 is less than a predetermined fifth threshold Th5 (step S1031A). For example, the count time of the number of closed eyes n is 90 seconds, and the fifth threshold Th5 is, for example, 6. If n≧Th5 in step S1031A, the inability determining unit 30B determines that the driver is not in an inoperable state (step S1033). In other words, the inability determining unit 30B does not determine that the vehicle is in an inoperable state if the driver is blinking normally.

If n<Th5 in step S1031A, the impossibility determination unit 30B determines whether the face area movement distance d calculated by the face area movement distance calculation unit 32 is less than a predetermined sixth threshold Th6. (Step S1031B). If d≧Th6 in step S1031B, the inability determining unit 30B determines that the driver is not in an inoperable state (step S1033). If the driver's face moves slowly or repeats the same movement, the movement distance of the face area is small between the two frames compared in the immobility determination process of the immobility determination unit 20E, and the overlapping area ratio is It is assumed that the driver is determined to be in an immobile state due to the first threshold value being greater than or equal to the first threshold value. However, in this step, by determining the movement distance of the face area in a time longer than the time between the two frames compared in the immobility determination process, even if the immobility state is incorrectly determined as described above, the driver This prevents erroneous determination that the vehicle is in an inoperable state.

If d<Th6 in step S1031B, the impossibility determining unit 30B determines whether the abnormal operation detecting unit 33 has detected an abnormal operation (step S1031C). If no abnormal operation is detected in step S1031C, the inability determining unit 30B determines that the driver is not in an inoperable state (step S1033). On the other hand, if an abnormal operation is detected in step S1031C, the inability determining unit 30B determines that the driver is in an inoperable state (step S1032). With this, the impossibility determining unit 30B ends the impossibility determining process.

In this way, the disability determining unit 30B determines whether the driver is unable to drive by considering three conditions: the number of times of eye closure n, the face frame movement distance d, and the abnormal operation in addition to the condition of the immobility determination time T0. It is determined that the vehicle is in an inoperable state only when there is an extremely high possibility that the vehicle is in an inoperable state. Therefore, it is possible to prevent a driver who is not in an inoperable state from being erroneously determined to be in an inoperable state. In the above, the disability determining unit 30B considers three conditions: the number of times of eye closure n, the face frame movement distance d, and the abnormal operation, but it is only necessary to consider at least one of these conditions.

<F-3. Effect>
In the driving disability determination device 106 of the sixth embodiment, the image analysis unit 10D acquires the driver's eye opening/closing degree from the photographed video, and the disability determination unit 30B acquires the driver's eye opening/closing degree acquired by the image analysis unit 10D. Based on the degree of opening/closing, the number of times the driver's eyes are closed within a certain period of time is calculated, and based on the number of times the driver's eyes are closed, the driver is determined to be unable to drive. Therefore, even if the driver is in an immobile state, it is possible to prevent the driver from erroneously determining that he or she is blinking normally as being unable to drive.

In the driving disability determination device 106 of the sixth embodiment, the disability determination unit 30B calculates the moving distance of the driver's face area within a certain period of time from the position of the driver's face area acquired by the image analysis unit 10D, Based on the distance traveled, the driver is determined to be unable to drive. Therefore, when the face moves slowly, it is possible to prevent the driver from being erroneously determined to be unable to drive.

In the driving impossibility determination device 106 of the sixth embodiment, the impossibility determination unit 30B performs an abnormality determination in which the vehicle operation is determined to be abnormal based on the vehicle operation information, and also determines the driver's condition based on the result of the abnormality determination. Determined to be inoperable state. Therefore, it is possible to prevent the driver from being erroneously determined to be unable to drive when no abnormal operation of the vehicle is detected.

<G. Hardware configuration>
Image analysis sections 10A, 10B, 10C, 10D, immobility judgment sections 20A, 20B, 20C, 20D, 20E, and impossibility judgment sections 30A, 30B in the above-mentioned inoperability judgment devices 101, 102, 103, 104, 105, 106 is realized by a processing circuit 81 shown in FIG. That is, the processing circuit 81 includes image analysis units 10A, 10B, 10C, and 10D, immobility determination units 20A, 20B, 20C, 20D, and 20E, and disability determination units 30A and 30B (hereinafter referred to as "image analysis units, etc."). Equipped with. Dedicated hardware may be applied to the processing circuit 81, or a processor that executes a program stored in memory may be applied. The processor is, for example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

When the processing circuit 81 is dedicated hardware, the processing circuit 81 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Circuit). Gate Array), or a combination of these. The functions of each section, such as the section .

When the processing circuit 81 is a processor, the functions of the image analysis section and the like are realized by a combination of software and the like (software, firmware, or software and firmware). Software etc. are written as programs and stored in memory. As shown in FIG. 20, a processor 82 applied to a processing circuit 81 realizes the functions of each part by reading and executing a program stored in a memory 83. That is, when the driving impossibility determination devices 101, 102, 103, 104, 105, and 106 are executed by the processing circuit 81, they analyze the video captured by the camera 51 and include the position information of the driver's face area. Obtaining feature information regarding the driver's head; and determining whether or not the driver is in an immobile state based on changes in feature information between two frames having a certain temporal relationship in the photographed video. and a step of determining that the driver is unable to drive if the duration of the driver's immobile state is equal to or greater than a second threshold value. A memory 83 is provided. In the immobility determination, it is determined that the driver is in an immobility state when the overlapping area ratio of the driver's face area between two frames is equal to or greater than a first threshold value. In other words, this program can be said to cause a computer to execute the procedure or method of the image analysis section or the like. Here, the memory 83 includes nonvolatile memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). or volatile semiconductor memory, HDD (Hard Disk Drive), magnetic disk, flexible disk, optical disk, compact disk, mini disk, DVD (Digital Versatile Disk) and its drive device, or any storage medium that will be used in the future. There may be.

Above, the configuration in which each function of the image analysis section and the like is realized by either hardware or software has been described. However, the present invention is not limited to this, and a configuration may be adopted in which a part of the image analysis section or the like is realized by dedicated hardware, and another part is realized by software or the like. For example, in the inoperability determination device 101, the function of the image analysis unit 10A is realized by a processing circuit as dedicated hardware, and for other parts, the processing circuit 81 as the processor 82 executes the program stored in the memory 83. The function can be realized by reading and executing it.

As described above, the processing circuit can realize each of the above functions using hardware, software, etc., or a combination thereof.

The driveability determination devices 101, 102, 103, 104, 105, and 106 are typically in-vehicle devices, but include PNDs (Portable Navigation Devices), communication terminals (for example, mobile terminals such as mobile phones, smartphones, and tablets), The present invention can also be applied to a system constructed by appropriately combining functions of applications installed on these, servers, etc. In this case, each function or each component of the inoperability determination devices 101, 102, 103, 104, 105, and 106 described above may be distributed and arranged in each device constructing the system, or may be arranged in any one of the devices. The equipment may be centrally located.

As an example, FIG. 21 shows a configuration example of the drivability determination device 106 including a vehicle and a server. In FIG. 21, the image analysis section 10D and the impossibility determination section 30B are arranged in the vehicle, and the immobility determination section 20E is arranged in the server.

Note that it is possible to freely combine each embodiment, or to modify or omit each embodiment as appropriate. The above description is in all aspects exemplary. It is understood that countless variations not illustrated may be envisioned.

10A, 10B, 10C, 10D Image analysis unit, 11 Face position detection unit, 12 Face direction detection unit, 13 Gaze direction detection unit, 14 Eye opening/closing degree detection unit, 20A, 20B, 20C, 20D, 20E Immobility determination unit, 21 Superimposition ratio calculation unit, 22 Face direction change calculation unit, 23 Gaze direction change calculation unit, 24 Unconsciousness determination unit, 30A, 30B Impossibility determination unit, 31 Eye closure count unit, 32 Face area movement distance calculation unit, 33 Abnormal operation detection unit , 50 imaging device, 51 camera, 52 LED, 60 output device, 61 display device, 62 speaker, 70 vehicle control device, 80 vehicle information communication device, 101, 102, 103, 104, 105, 106 driveability determination device, 204 vehicle.

Claims (11)

an image analysis unit that acquires characteristic information regarding the driver's head, including positional information of the driver's face area, by analyzing a captured image of a camera mounted on the vehicle and photographing the driver of the vehicle; ,
an immobility determining unit that determines whether or not the driver is in an immobile state based on a change in the characteristic information between two frames having a certain temporal relationship of the photographed video;
an inability determining unit that determines that the driver is in an inoperable state when the duration of the immobile state of the driver is equal to or greater than a second threshold;
The immobility determination unit calculates an overlapping area ratio of the facial area between the two frames based on position information of the facial area in each of the two frames, and the overlapping area ratio is equal to or higher than a first threshold value. In this case, it is determined that the driver is in an immobile state,
The feature information includes information on the direction of the driver's face,
The immobility determination unit is configured to determine whether the overlapping area ratio is equal to or greater than the first threshold and the amount of change in face direction, which is the amount of change in the direction of the driver's face, is less than a third threshold between the two frames. , determining that the driver is in the immobile state;
Drivability determination device. The characteristic information includes information on the direction of the driver's line of sight,
The immobility determination unit is configured to determine, between the two frames, that the superimposed area ratio is greater than or equal to the first threshold, the amount of change in face direction is less than the third threshold, and the amount of change in the direction of the driver's line of sight is the amount of change in the direction of the driver's line of sight. determining that the driver is in the immobile state when the amount of direction change is less than a fourth threshold;
The inoperability determination device according to claim 1. The immobility determination unit includes an unconsciousness determination unit that determines whether the driver is unconscious based on the direction of the driver's face, and when determining that the driver is unconscious, the two frames determining that the driver is in the immobile state even if the face direction change amount is equal to or greater than the third threshold value;
The inoperability determination device according to claim 1. The unconsciousness determination unit determines whether or not the driver is unconscious based on a variance in the vertical position of the driver's face area and a variance in pitch angle in the direction of the driver's face. ,
The inoperability determination device according to claim 3. The unconsciousness determining unit determines whether the driver is unconscious based on a direction of change in the steering angle of the vehicle and a direction of the driver's face.
The inoperability determination device according to claim 3. If the image analysis unit cannot obtain the direction of the driver's line of sight from the captured video,
The immobility determination unit excludes the amount of change in the direction of the line of sight from the determination condition of the immobility state, and the image analysis unit determines at least one of the first threshold value and the second threshold value based on the driver's line of sight based on the photographed video. to be larger than when you can get the direction of ,
The inoperability determination device according to claim 2. The image analysis unit obtains the degree of eye opening/closing of the driver from the captured video,
The inability determining unit calculates the number of times the driver's eyes are closed within a certain period of time based on the degree of opening and closing of the driver's eyes acquired by the image analysis unit, and also based on the number of times the driver's eyes are closed. It is determined that the vehicle is unable to drive.
The inoperability determination device according to claim 1. an image analysis unit that acquires characteristic information regarding the driver's head, including positional information of the driver's face area, by analyzing a captured image of a camera mounted on the vehicle and photographing the driver of the vehicle; ,
an immobility determining unit that determines whether or not the driver is in an immobile state based on a change in the characteristic information between two frames having a certain temporal relationship of the photographed video;
an inability determining unit that determines that the driver is in an inoperable state when the duration of the immobile state of the driver is equal to or greater than a second threshold;
The immobility determination unit calculates an overlapping area ratio of the facial area between the two frames based on position information of the facial area in each of the two frames, and the overlapping area ratio is equal to or higher than a first threshold value. In this case, it is determined that the driver is in an immobile state,
The impossibility determination unit determines the total amount of movement of the driver's face area within a certain period of time that is longer than the time between the two frames from the position of the driver's face area acquired by the image analysis unit. calculating a travel distance that is a distance of , and determining that the driver is in the driving incapacity state based on the travel distance;
Drivability determination device. The incapacity determining unit performs an abnormality determination in which the operation of the vehicle is determined to be abnormal based on the operation information of the vehicle, and determines the state of the driver to be in an inoperable state based also on the result of the abnormality determination.
The inoperability determination device according to claim 1. outputting the determination result of the impossibility determination unit to a vehicle control device that performs automatic driving control of the vehicle;
The inoperability determination device according to claim 1. By analyzing the video captured by a camera mounted on a vehicle that photographs the driver of the vehicle, characteristic information regarding the driver's head, including positional information of the driver's face area, is acquired, and information on the captured video is analyzed. determining whether or not the driver is in an immobile state based on a change in the characteristic information between two frames having a certain temporal relationship;
determining that the driver is unable to drive if the duration of the immobility state of the driver is equal to or greater than a second threshold;
The overlapping area ratio of the facial area between the two frames is calculated based on the positional information of the facial area in each of the two frames, and when the overlapping area ratio is equal to or higher than a first threshold, the driver is determined to be in an immobile state,
The feature information includes information on the direction of the driver's face,
Between the two frames, if the superimposed area ratio is greater than or equal to the first threshold and the amount of change in face direction, which is the amount of change in the direction of the driver's face, is less than the third threshold, the driver It is determined that it is in an immobile state,
How to determine if you are unable to drive.

Images