JP6614286B2 - Driver inoperability detection device - Google Patents

Description

  The present invention relates to a driver inoperability state detection device that detects a state in which a driver has become inoperable.

  Conventionally, it has been a problem that a driver falls into an inoperable state due to a sudden illness or the like during driving of the vehicle, leading to an accident. Therefore, it has been proposed to detect such an inoperable state of the driver and prevent an accident.

  In the emergency evacuation device described in Patent Literature 1, the driver's line-of-sight information, blink information, face orientation information, and the like are recognized from the captured image of the driver, and the driver's heartbeat information, respiratory information, brain wave information, and the like are measured. . Then, in the device described in Patent Document 1, using the acquired various information, when the driver's state such as the driver's consciousness reduction degree, arousal degree, posture, etc. is recognized and the driver state is low, Emergency evacuation support is provided.

JP 2014-19301 A

  Since the apparatus described in Patent Literature 1 acquires various information and recognizes the state of the driver, the process for detecting the inoperable state of the driver is complicated.

  In view of the above circumstances, it is a main object of the present invention to provide a driver inoperability state detection device that can easily detect that a driver has become inoperable.

In order to solve the above-described problem, a first invention disclosed in the present application is a driver inoperability state detection device, which is based on an image of a driver's seat taken by an imaging device mounted on a vehicle. A head detection means for detecting the position of the head above the head in three dimensions, a face orientation detection means for detecting the orientation of the driver's face relative to the front of the vehicle based on the image of the driver's seat, and the vehicle running Further, the position of the head detected by the head detecting means is out of the predetermined range, and the direction of the driver's face relative to the front of the vehicle detected by the face orientation detecting means is below the threshold value. And a state detecting means for detecting that the driver is in an inoperable state.
In addition, the second invention disclosed in the present application is a head that three-dimensionally detects the position of the head above the driver's neck based on the image of the driver's seat imaged by the imaging device mounted on the vehicle. A detection unit, a face direction detection unit that detects a driver's face direction relative to the front of the vehicle based on an image of the driver's seat, and a head position detected by the head detection unit while the vehicle is traveling is predetermined. State detection that detects that the driver is incapable of driving when the driver's face direction with respect to the front of the vehicle detected by the face direction detection means is above the threshold value. Means.

  According to the first invention or the second invention, the driver's head is sequentially detected based on the image obtained by imaging the driver's seat. When the driver is driving the vehicle normally, the driver's head often falls within a predetermined range of the driver's seat image. On the other hand, when sudden illness develops and the driver's consciousness is lost, the driver's head may be out of the predetermined range. In general, when a sudden illness occurs and the driver's consciousness is lost, the driver's face is often larger than a threshold value or directed downward or upward. Therefore, as in the first aspect of the invention, the driver is incapable of driving when the head is out of the predetermined range and the face direction of the driver is below the threshold value (downward threshold value). By detecting this, it is possible to easily detect the inoperable state of the driver. Further, as in the second invention, when the head is out of the predetermined range and the face direction of the driver is directed upward from the threshold value (upward threshold value), the driver is incapable of driving. By detecting, it is possible to easily detect the inoperable state of the driver.

  Further, another invention disclosed in the present application is a driver inoperable state detection device, which is based on an image of a driver's seat taken by an imaging device mounted on a vehicle, and a head above the driver's neck From the position of the head detected by the head detection means, the trajectory acquisition means for acquiring the trajectory of the head from the position of the head detected by the head detection means, and the trajectory acquired by the trajectory acquisition means. And a state detecting means for detecting that the driver is in an inoperable state.

  According to another invention disclosed in the present application, the head of the driver is sequentially detected based on the image of the driver's seat, and the locus of the head is acquired from the detected position of the head. When a sudden illness develops and the driver is unable to drive, the driver's head often moves from the driving position and does not return to the driving position. Therefore, the inoperable state of the driver can be easily detected based on the trajectory of the driver's head.

The block diagram which shows the structure of a driving impossible state detection apparatus. The figure which shows the vehicle interior which mounts an operation impossible state detection apparatus. The figure which shows the driver's seat of the vehicle carrying a driving impossible state detection apparatus. The block diagram which shows the function of a control apparatus. The figure which shows the change of the attitude | position when sudden disease develops. The figure which shows the attitude | position when taking a thing. The figure which shows the change of the attitude | position when sudden disease develops. The figure which shows the attitude | position when looking aside. The figure which shows the attitude | position when taking a thing. The figure which shows the face orientation when sudden illness develops. The figure which shows the change of face orientation when looking aside. The figure which shows the shaking of the head accompanying generation | occurrence | production of external force. The figure which shows the amplitude range of the shaking of the head determined to be a driving impossible state. The figure which shows the change of the facial expression when sudden illness develops. The figure which shows the state which faced the normal state and the white of eyes. The flowchart which shows the process sequence which detects a driving | operation impossible state. A subroutine showing a processing procedure for detecting posture collapse. The figure which shows the aspect which alert | reports a posture collapse level to a driver. The figure which shows the frame range of the frame-out determination which concerns on the modification 1. FIG. The figure which shows the frame range of the frame-out determination which concerns on the modification 2. FIG. The figure which shows the frame range of the frame-out determination which concerns on the modification 3. FIG. The figure which shows the frame range of the frame-out determination which concerns on the modification 4. FIG. The figure which shows the frame range and determination time of the frame-out determination which concern on 2nd Embodiment.

  Hereinafter, embodiments embodying a driver inoperability detection device will be described with reference to the drawings. In the present embodiment, the driver's inoperable state includes a state in which the driver develops sudden illness and becomes unconscious and cannot perform driving operation, and a driver develops a sudden illness such as a heart attack and is conscious but cannot move the body. Therefore, it includes a state in which the driving operation cannot be performed. In the following embodiments, parts that are the same or equivalent to each other are denoted by the same reference numerals in the drawings, and the description of the same reference numerals is used.

(First embodiment)
First, the structure of the detection apparatus 100 (driver inoperable state detection apparatus) according to the present embodiment will be described with reference to FIGS. The detection device 100 includes a control device 50, a driver state recognition device 20, a vehicle information recognition device 30, a traveling environment recognition device 40, an HMI (Human Machine Interface) 80, and a storage device 52, and detects an inoperable state of the driver. To do. And the detection apparatus 100 transmits the instruction | command which stops a vehicle safely to the vehicle control apparatus 90, when a driver confirms that it is a driving impossible state and there is no response.

  The driver state recognition device 20 includes a plurality of driver cameras 21 (imaging devices), a seat belt sensor 22 (amount detection means), and a seating surface sensor 23 (seat pressure detection means). The driver camera 21 is a CCD camera, for example, and images the driver's seat illuminated by an illumination device such as a near infrared LED. As shown in FIGS. 2 and 3, the driver camera 21 is mounted on the meter panel 14, the approximate center of the lower end of the rearview mirror 16, and the left and right A pillars 17 toward the driver. Instead of the meter panel 14, a driver camera 21 may be installed on the dashboard 13 (shown by a broken line) or on a steering column. Further, instead of the lower end of the rearview mirror 16, it may be installed at the left end or the right end (shown by a broken line) of the rearview mirror 16. These four driver cameras 21 constitute a driver status monitor, which captures several tens of images per second from the front side of the driver sitting on the driver's seat 11.

  The seat belt sensor 22 is a sensor that detects the amount of withdrawal of the seat belt 12. Specifically, the seat belt sensor 22 is an encoder that detects a rotation angle of a motor that feeds and winds the seat belt 12. The seat surface sensor 23 is a sensor that detects the pressure distribution of the seat portion 11a of the seat 11 of the driver's seat.

  The vehicle information recognition device 30 includes a vehicle speed sensor 31, a steering angle sensor 32, an accelerator sensor 33, and a brake sensor 34. The vehicle speed sensor 31 is a sensor that detects the speed of the vehicle 10. The steering angle sensor 32 is a sensor that detects the steering angle of the handle 15. The accelerator sensor 33 is a sensor that detects an accelerator opening, that is, an operation amount of an accelerator pedal. The brake sensor 34 is a sensor that detects an operation amount of a brake pedal.

  The travel environment recognition device 40 includes a front / rear camera 41, a front / rear sensor 42, a car navigation device 43, and a G sensor 44. The front / rear camera 41 is a camera that images the front of the vehicle 10 including the white line of the road, and a camera that images the rear and rear sides of the vehicle 10. The front / rear sensor 42 is a sensor such as an ultrasonic sensor, a laser radar, or a millimeter wave radar, and detects an object in front or rear of the vehicle 10 and acquires a distance between the vehicle 10 and an object in front or rear. Based on the distance between the vehicle 10 acquired by the front / rear sensor 42 and the front or rear vehicle, the relative speed between the front vehicle and the rear vehicle can be calculated.

  The car navigation device 43 calculates the current position of the vehicle 10 by using the GPS signal received by the GPS receiver and information acquired by various sensors including the G sensor, and guide routes from the current position to the destination Is calculated. The G sensor 44 is, for example, a sensor that is installed on the seat 11 and detects three-dimensional acceleration in the front-rear, left-right, and upper-lower direction of the vehicle 10. In addition, the G sensor 44 may be a sensor included in the car navigation device 43. When the vehicle 10 includes a vehicle operation management system (AVOS), the G sensor 44 is a sensor included in the AVOS. May be. That is, the G sensor 44 may be shared if there is one installed for other purposes.

  The control device 50 is a microcomputer including a CPU, ROM, RAM, I / O, and the like, and acquires various information from the driver state recognition device 20, the vehicle information recognition device 30, the traveling environment recognition device 40, the storage device 52, and the HMI 80. To do. The control device 50 and the various devices are connected by wired communication such as CAN or wireless communication such as LAN or Bluetooth (registered trademark). In addition, the control device 50 realizes the functions of the image analysis unit 60, the learning unit 51, and the state detection unit 70 by the CPU executing various programs stored in the ROM, and detects the inoperable state of the driver. . Detailed explanation of each means will be described later.

  The HMI 80 (attitude notification means, confirmation means) includes a display 81, a speaker 82, and a cancel switch 83. The display 81 is a display of the car navigation device 43 or an in-vehicle display provided in the meter panel 14. The display 81 may be a touch display including a liquid crystal panel or an organic EL panel. The display 81 reports the degree of the driver's posture collapse based on the driver's posture detected from the image. Specifically, the display 81 displays the status of the driver's posture in five stages. The posture collapse level 5 with the highest degree of collapse is a level at which it is determined that the driver has developed a sudden illness and is unable to maintain the driving posture, that is, a driving impossible state. Since the driver can check his / her driving posture by looking at the posture status displayed on the display 81, when the posture collapse level approaches 5, the driving posture can be corrected before being determined as being incapable of driving. .

  The speaker 82 is an in-vehicle speaker shared with the car navigation device 43, the audio device, and the like. When the inoperable state of the driver is detected, the speaker 82 confirms to the driver by voice whether the inoperable state. The display 81 may display a screen for confirming the inoperable state. Further, the speaker 82 may notify the driver of the posture collapse level by voice.

  The cancel switch 83 is a switch that stops detection of the inoperable state. When the cancel switch 83 is operated once, detection of the inoperable state is stopped for one trip. Further, when the cancel switch 83 is operated during the trip, the detection of the inoperable state is stopped while the cancel switch 83 is operated or for a certain period of time (about several seconds) after the cancel switch 83 is operated. Therefore, if the driver operates the cancel switch 83 in advance when taking an action, there is no possibility that the driver is erroneously detected as being inoperable even if the driver's posture collapses.

  Next, various functions realized by the control device 50 will be described with reference to FIG. The image analysis means 60 includes a head detection means 61, a trajectory acquisition means 62, a convulsions detection means 63, an inclination detection means 64, a face orientation detection means 65, and a white eye detection means 66.

  The head detection unit 61 sequentially detects the head above the driver's neck based on the driver's seat image captured by the driver camera 21. Specifically, the head detection unit 61 extracts an edge representing the outline of the driver's head from the driver's seat image each time a driver's seat image is captured by the driver camera 21, and is an area surrounded by the extracted edges. Is detected as the head.

  The trajectory acquisition unit 62 acquires the trajectory of the driver's head from the position of the driver's head sequentially detected by the head detection unit 61. For example, the trajectory acquisition unit 62 uses the center of the driver's head detected in each image as the head position, and connects the head positions in each image to acquire the head trajectory.

  The convulsions detecting means 63 detects convulsions of the driver, that is, involuntary contraction of the muscles of the trunk part below the head and neck of the driver. Specifically, the convulsions detection means 63 extracts edges representing the contours of the driver's head and torso in each image, and when the extracted edges vibrate regularly (periodically) in successive images, Detect that the driver is spasm.

  The inclination detecting means 64 detects the inclination θ of the head with respect to the driver's body based on the driver's seat image. Specifically, the inclination detection means 64 detects the regions surrounded by the edges representing the contours of the head and the body as the head and the body, respectively, and detects the central axes of the head and the body. And the inclination detection means 64 makes the inclination of the central axis of the head with respect to the central axis of the body part the inclination θ of the head. The center axis of the body part is detected from the body part in which the direction of the body part is determined by matching the prepared body direction pattern with the detected body part direction to determine the body part direction. Further, the feature axis such as the eyes, nose and mouth of the face included in the head is extracted from the central axis of the head, and is detected from the three-dimensional arrangement of the feature points of the face. When the head is tilted forward, the distance between the facial feature point and the front of the vehicle approaches, and when the head turns back, the distance between the facial feature point and the front of the vehicle increases. When detecting the central axis of the head, the distance between facial feature points in the front-rear direction of the vehicle may be used.

  Alternatively, the inclination detection unit 64 detects the seat belt 12 of the driver's seat from the image of the driver's seat, and detects the inclination θ of the head with respect to the trunk from the positional relationship between the seat belt 12 and the head. Since the body part of the driver is restrained by the seat belt 12, the position of the body part can be estimated from the position of the seat belt 12.

  The face direction detection means 65 detects the direction of the driver's face relative to the front of the vehicle 10 based on the driver's seat image. The face direction detection unit 65 detects the inclination of the face with respect to the vertical plane facing the front surface of the vehicle 10 as the face direction.

  The white eye detection unit 66 includes a facial expression detection unit 67 and a white eye degree calculation unit 68, and detects a state in which the driver has white eyes. Here, the state where the whites are removed is not limited to the state where the whites are completely removed as shown in FIG. 15C, but the black eye area is smaller than a predetermined amount as shown in FIG. 15B. Including state. That is, the state where the whites are peeled is a state where the visual field is narrower than a predetermined range due to the biased black eyes.

  The facial expression detection means 67 detects the eye contour and black eye area of the driver based on the driver seat image. Here, the outline of the eyes of the driver is a boundary line between the eyelids and the eyes. The black eye region is a region having lightness lower than that of the white eye in a region inside the outline of the eye, and is not limited to black, but is a region having a color such as blue, brown, or gray. The facial expression detection means 67 detects the opening of the driver's mouth from the edge representing the extracted mouth outline.

  The white eye degree calculation means 68 calculates the white eye degree of the driver's eyes based on the eye contour and the black eye area detected by the facial expression detection means 67.

  Specifically, the white eye degree calculation means 68 calculates the white eye degree from the ratio between the vertical length Lw + Lb of the region surrounded by the outline of the eye and the vertical length Lb of the black eye region (see FIG. 15). The smaller the length Lb with respect to the length Lw + Lb, the greater the degree of white eye. Alternatively, the white eye degree calculation means 68 calculates the white eye degree based on the distance Lb from the upper part of the eye contour to the lowermost part of the black eye region. The smaller the distance Lb, the greater the degree of white eye. Alternatively, the white eye degree calculating means 68 calculates the white eye degree based on the ratio of the area of the white eye region obtained by removing the area of the black eye region from the area of the whole eye region surrounded by the outline of the eye and the area of the black eye region. calculate. The smaller the area of the black eye region relative to the area of the white eye region, the greater the degree of white eye.

  Alternatively, the white eye degree calculation unit 68 calculates the white eye degree based on the flatness ratio of the black eye region. In a state where the whites are peeled off, the black eye region faces upward, so that the flatness of the black eye region increases apparently, and the degree of white eye increases as the flatness of the black eye region increases. Alternatively, the white eye degree calculation unit 68 calculates the white eye degree based on the distance Lc from the center line in the vertical direction of the region surrounded by the outline of the eye to the lowest part of the black eye region.

  The learning means 51 learns the head inclination θ detected by the inclination detection means 64 when the driver is not in an inoperable state. The learning means 51 learns the face orientation detected by the face orientation detection means 65 when the driver is not in an inoperable state. Further, the learning means 51 learns the amplitude of the head shake detected by the head detecting means 61 when the driver is not in an inoperable state. That is, the learning means 51 learns the driver's driving posture habit. When there are a plurality of drivers who drive the vehicle 10, the driver learns the habit of driving posture for each driver.

  The state detection unit 70 includes a frame-out state detection unit 71, a posture collapse state detection unit 72, a direction collapse state detection unit 73, a shaking state detection unit 74, and a white-eye state detection unit 75.

  The frame-out state detection means 71 determines that the vehicle 10 is out of frame while the vehicle 10 is traveling, and detects that the driver is in an inoperable state when the vehicle is out of frame. Specifically, the frame-out state detection unit 71 detects that the driver is inoperable when the head of the driver detected by the head detection unit 61 is out of the image range FA. Here, the range FA is a predetermined range in an image captured by the driver camera 21. During normal driving, the driver's head does not deviate from the range FA. The range FA may be the entire captured image.

  When the driver is driving the vehicle 10 normally, as shown in FIG. 6, the driver's head often falls within the image range FA even if the driver takes an action. On the other hand, when sudden illness develops and the driver's consciousness is lost, as shown in FIG. 5, the head of the driver may be out of the range FA. Therefore, the frame-out state detection means 71 detects that the driver is in an inoperable state when the head of the driver is out of the image range FA.

  At this time, the frame-out state detection unit 71 can improve the accuracy of detecting the inoperable state of the driver in consideration of the trajectory acquired by the trajectory acquisition unit 62 before the head is out of the range FA. By using the trajectory of the head when the driver's head could not be detected within the range FA due to image ambiguity and when the driver's head moved and could not be detected within the range FA Since the determination can be made, the detection accuracy of the inoperable state of the driver is improved.

  In addition, when the driver temporarily moves the head, or when the head is not detected due to unclearness of the image, the head can often be detected again near the final position of the trajectory. Therefore, when the head is no longer detected by the head detection unit 61, the head detection unit 61 searches for the vicinity of the final position of the trajectory acquired by the trajectory acquisition unit 62. In this way, even when the driver's head is not detected, the head detection can be efficiently performed again using the head trajectory.

  The posture collapse state detection means 72 determines whether the driver is out of posture while the vehicle 10 is traveling, and detects that the driver is incapable of driving when the posture is out of alignment. Specifically, the posture collapse state detection unit 72 detects that the driver is incapable of driving when the head inclination θ detected by the inclination detection unit 64 is larger than a threshold Th1 (relative inclination threshold). .

  Normally, the body of the driver is restrained by the seat 11 and the seat belt 12 of the driver's seat, so that the body is relatively difficult to move even if the driver's consciousness is lost. On the other hand, since the driver's head is often not restrained, it is necessary to maintain the position of the head with the driver's intention. Therefore, when sudden illness develops and the driver's consciousness disappears, the driver cannot maintain the position of the head, and as shown in FIG. 7, the head often tilts greatly in either direction with respect to the trunk. .

  On the other hand, when the driver looks aside while traveling, the driver generally looks with the neck rotated, so that the inclination of the head with respect to the trunk is often small as shown in FIG. In addition, when the driver takes a position away from the driver's seat, the driver generally consciously tilts the torso, so that the inclination θ of the head with respect to the torso is small as shown in FIG. There are many. Therefore, the posture collapse state detection means 72 detects that the driver is incapable of driving when the head inclination θ is larger than the threshold Th1. At this time, if the posture collapsed state detection means 72 further detects that the driver is in an inoperable state on the condition that the driver's face is not facing the front of the vehicle 10, erroneous detection of the inoperable state can be suppressed. .

  The direction change state detection means 73 determines that the driver's face is broken while the vehicle 10 is traveling, and detects that the driver is incapable of driving when the face is broken. Specifically, the direction change state detection unit 73 detects that the face direction with respect to the front of the vehicle 10 detected by the face direction detection unit 65 exceeds the threshold Th2 (face direction threshold) beyond the time T2 (direction change determination time). When it is larger, it is detected that the driver is in an inoperable state.

  In general, when a sudden illness occurs, the driver cannot maintain the face orientation, and the face orientation with respect to the front of the vehicle 10 remains broken as shown in FIG. On the other hand, when the driver looks aside while driving, the driver generally returns immediately even if the face is changed. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  Alternatively, the direction change state detection unit 73 may detect that the driver is driving when the face direction with respect to the front of the vehicle 10 detected by the face direction detection unit 65 is larger than the threshold Th2 and the driver releases the handle 15. It detects that it is impossible. Whether the driver has released the handle 15 may be detected from an image, or may be detected by a pressure sensor or the like installed on the handle 15.

  In general, when a sudden illness develops, the driver cannot maintain the orientation of the face, the orientation of the face with respect to the front of the vehicle 10 collapses as shown in FIG. 10, and the driver often releases the handle 15. On the other hand, when the driver looks aside while traveling, as shown in FIG. 11, the driver generally changes the direction of the face while holding the handle 15. In addition, when the driver looks aside, the head may not be tilted but the neck may be turned to change the face direction. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  Alternatively, the direction change state detection unit 73 is incapable of driving when the face direction detected by the face direction detection unit 65 is larger than the threshold value Th2 and the accelerator opening is larger than a predetermined opening. Detect that.

  In general, when a driver looks aside while traveling, the accelerator is often not stepped on for safety. Therefore, when the face direction with respect to the front of the vehicle 10 is larger than the threshold Th2 and the accelerator opening is larger than the predetermined opening, the face direction is not broken due to looking aside, but the face direction is broken due to sudden illness. Probability is high. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  Alternatively, the direction change state detection unit 73 has been subjected to the accelerator operation and the brake operation for a time longer than the time Th3 (operation determination time) when the face direction detected by the face direction detection unit 65 is greater than the threshold Th2. If not, it is detected that the driver is in an inoperable state.

  In general, when a sudden illness develops, the orientation of the face with respect to the front of the vehicle 10 is lost, and the accelerator operation and the brake operation are not performed for a time longer than the time T3. On the other hand, when the driver looks aside while traveling, the driver generally changes the face direction and frequently performs an accelerator operation or a brake operation within time T3. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  The swing state detection means 74 determines the swing state of the driver's head accompanying the external force while the vehicle 10 is traveling, and the driver is incapable of driving when the head is shaking differently than usual. Detect that. Specifically, the swing state detection means 74 has the amplitude Am1 of the head shake detected by the head detection means 61 from the time when the external force is applied to the vehicle 10 until the time T5 (swing determination time) elapses. When it is smaller than (first amplitude) or larger than amplitude Am2 (second amplitude), it is detected that the driver is in an inoperable state. The amplitude Am2 is larger than the amplitude Am1.

  As shown in FIG. 12, when an external force is applied to the vehicle 10, vibration is transmitted to the driver after a predetermined time difference. Normally, when there is a driver's consciousness, when an external force (specifically, an external force in the vertical direction) is applied to the vehicle 10, the head of the driver has an amplitude within the range of amplitude Am1 to amplitude Am2, as shown in FIG. Shake. On the other hand, when the driver develops a sudden illness and the body is stiff, the amplitude of the head shake is smaller than that in the normal state. In addition, when the driver develops a sudden illness and the body is relaxed, the amplitude of the shaking of the head becomes larger than that in the normal state. Therefore, the shaking state detection means 74 detects the inoperable state of the driver in the case described above.

  Time T5 is the time from when an external force is applied to the vehicle 10 until the driver's movement becomes unrelated to the external force. The amplitude Am1 and the amplitude Am2 are functions of time, and FIG. 13 shows an example thereof. In the determination of the shaking state, the minimum value of the amplitude Am1 and the maximum value of the amplitude Am2 from when an external force is applied until the time T5 elapses may be simply used as threshold values.

  The white-eye state detection unit 75 determines white eyes while the vehicle 10 is traveling, and detects that the driver is incapable of driving when the white-eye detection unit 66 detects a state where white eyes are peeled off. Specifically, the white-eye state detection unit 75 detects that the driver is incapable of driving when the white-eye degree calculated by the white-eye degree calculation unit 68 is larger than a threshold Th3 (white-eye threshold).

  Normally, when the driver can drive, the driver does not peel off his white eyes. On the other hand, when the driver develops a sudden illness, the driver may have white eyes as shown in FIG. Therefore, the white-eye state detection means 75 detects that the driver is in an inoperable state when a state in which white eyes have been peeled is detected.

  Each threshold value and each determination value used by each state detection unit are stored in the storage device 52 (storage unit). Further, the storage device 52 stores the head inclination θ, the face orientation, and the head shaking amplitude learned by the learning means 51. In the storage device 52, personal information including the medical history and age of the driver is registered. When there are a plurality of drivers, personal information of each driver is registered. Further, the storage device 52 is registered with the posture of the driver that is not determined as being incapable of driving and the posture of the driver that is determined as being incapable of driving. The posture of the driver who is not determined to be incapable of driving is, for example, a normal driving posture or a posture improved during driving. The posture of the driver that is determined as being incapable of driving is, for example, the posture that a driver who has illness makes during an attack. The driver captures the posture desired to be registered in the driver's seat in advance with the driver camera 21 and registers it in the storage device 52.

  Next, a processing procedure for detecting an inoperable state of the driver will be described with reference to a flowchart of FIG. This processing procedure is executed by the control device 50.

  First, it is determined whether the vehicle speed of the vehicle 10 is V or less (S10). V may be 0 km / h (stop) or a sufficiently low speed (for example, 1 km / h) to be regarded as a stop. When the vehicle speed is equal to or lower than V (S10: YES), the determination of S10 is repeatedly executed until it is determined that the vehicle speed is higher than V. When the vehicle speed is higher than V (S10: N0), it is determined that the vehicle is traveling, and the detection process of the inoperable state of the driver is started, and frame out determination is first performed.

  Here, in the determination of S10, it is also determined whether or not the driver is performing a driving operation. When the driving operation is being performed, the determination of S10 is repeatedly performed so that the driving operation is performed while traveling. If not, the detection process of the inoperable state of the driver may be started. For example, it is determined whether the vehicle speed of the vehicle 10 is V2 (for example, 50 km / h) or higher, the steering angle detected by the steering angle sensor 32 is higher than a predetermined angle, or the steering angular velocity is higher than a predetermined angular velocity. V2 is a value that can be considered that the driver is operating the accelerator pedal, a predetermined angle, and a predetermined angular velocity are values that can be considered that the driver is operating the steering wheel. If at least one of the above three conditions is satisfied, it is determined that the driver is performing a driving operation, and the driver's inoperable state detection process is not started.

  In determination of frame-out, first, the driver's head and torso are detected from the driver's seat image (S11). At this time, the facial feature point included in the head of the driver is detected to authenticate the driver. Driver authentication may be performed in advance by communication with a mobile terminal such as a smartphone, or may be performed by communication with a key of the vehicle 10 in which personal information is registered.

  Next, it is determined whether or not the head position has been detected (S12). If the head position cannot be detected (S12: NO), the head trajectory information is acquired from the head position recorded in the process of S19 described later, and it is determined whether the trajectory information indicates frame out. (S13). That is, it is determined whether the non-detection of the head position is non-detection due to the head being out of the imaging range or non-detection due to unclearness of the image.

  If the trajectory information of the head does not indicate frame out (S13: NO), it is determined as non-detection due to temporary obscuration of the image and the process returns to S10. On the other hand, when the trajectory information of the head indicates frame out (S13: YES), it is determined that the head has gone out of the imaging range. In this case, information on the seat belt sensor 22 and the seating surface sensor 23, which are used auxiliary to confirm that the head is outside the imaging range, is acquired (S14).

  Next, it is determined whether or not the frame is in a frame-out state continuously for a time T0 (frame-out determination time) (S15). Specifically, the head position is out of the range FA for more than time T0, and the pull-out amount of the seat belt 12 exceeds the first pull-out amount than the pull-out amount detected when the seat belt 12 is worn. In addition, it is determined whether or not the high pressure portion is biased toward the end of the seat 11a in the pressure distribution of the seat 11a. Then, when the above three conditions are satisfied, it is determined that the frame is out of time T0. At this time, the pull-out amount per amount detection time detected by the seat belt sensor 22, that is, the pull-out speed of the seat belt 12 may be set to be larger than the second pull-out amount.

  Information on the seat belt sensor 22 and the seat surface sensor 23 may not be used. In other words, the process of S14 is not executed, and the process of S15 may continue to determine whether or not the head position is out of the range FA continuously for time T0 or more.

  Here, the time T0 is set based on the personal information registered in the storage device 52. For example, the time T0 is shortened for an older person than for a younger person. In addition, a person with a specific medical history has a shorter time T0 than a person without a specific medical history. Furthermore, the time T0 is changed according to the state of the driver and the driving environment. In a driving environment where there is an indication that the driver is unable to drive, where the driver is likely to be unable to drive, or where there is a high risk of a collision when the driver is disabled T0 is shortened to make it easier to detect an inoperable state of the driver.

  Specifically, when the head position recorded in the process of S19 is oscillating with an amplitude larger than a predetermined amplitude, that is, when the head is swaying, there is a high probability of being inoperable. The time T0 is shortened. Also, the higher the moving speed of the head, the higher the possibility of posture collapse due to sudden illness rather than posture collapse when taking a thing. Therefore, in the acquired head trajectory information, the time T0 is shortened as the moving speed of the head position increases. In the case of posture collapse due to a sudden illness, the moving speed of the head often increases as the head approaches the end of the range FA. Therefore, when the moving speed of the head increases as the head approaches the end of the range FA, there is a high possibility that the posture is not collapsed due to a sudden illness but is not collapsed when taking a thing. Therefore, the time T0 is shortened when the moving speed of the head increases as the recorded head position approaches the end of the range FA. In addition, when convulsions are detected, the probability of becoming inoperable is high, so the time T0 is shortened.

  Moreover, in order to avoid a collision, it is necessary to start appropriate vehicle control earlier as the vehicle speed increases, so the time T0 is shortened as the vehicle speed of the vehicle 10 increases. In addition, in order to avoid a collision, it is necessary to start appropriate vehicle control earlier as the TTC (collision margin time) obtained by dividing the inter-vehicle distance from the preceding vehicle by the relative speed with the preceding vehicle is shorter. The shorter the time T0 is, the shorter it is. Further, when driving support control such as ACC (Adaptive Cruise Control) or LKA (Lane Keep Assist) is executed in the vehicle 10, the driver may break the posture for a long time, so the time T 0 is extended. Furthermore, the time T0 may be shortened on days of the week and times of day when statistically appearing sudden attacks such as heart attacks are likely to occur.

  If the frame is not continuously out for time T0 or more (S15: NO), the process proceeds to S21. If the frame is continuously out of time T0 or more (S15: YES), it is detected that the driver is in an inoperable state, and the driver is confirmed to be in an inoperable state. Specifically, the detection of the inoperable state is notified by voice from the speaker 82, display on the display 81, blinking of an indicator (not shown), and the like, and it is determined whether or not there is a response from the driver within a predetermined time ( S16).

  If a driver touches the touch display, the driver's voice, the operation of the vehicle 10 such as the steering wheel 15 or the brake, or the operation of a specific switch is detected within a predetermined time, a response is received from the driver. Determine (S16: NO). If none of the above is detected, it is determined that there is no response from the driver (S16: YES).

  When it is determined that there is a response from the driver, the driver recognizes that driving by the driver is possible by a voice from the speaker 82 or a display on the display 81 (S17). On the other hand, when it is determined that there is no response from the driver, an instruction is issued to the vehicle control device 90 so that appropriate braking and steering are performed and the vehicle is safely stopped. In addition, in order to notify the surrounding vehicles of the danger, an instruction is given to the vehicle control device 90 to light the headlight and listen to the horn (S18). Further, the danger is also notified to other passengers of the vehicle 10.

  Next, when it is determined that the head position has been detected in the process of S12 (S12: YES), the positions of the head and the torso are recorded (S19). The trajectory information of the head can be acquired from the position of the head recorded in each image.

  Subsequently, it is determined whether or not the head position is out of a preset range FA (S20). If the head position is out of the range FA even if it is within the imaging range (S20: YES), the process proceeds to S15 and frame out is determined.

  Subsequently, based on the positional relationship between the head and the torso, it is determined whether or not the posture of the driver is a posture determined to be an inoperable state registered in advance in the storage device 52 (S21). If the posture of the driver is determined to be an inoperable state (S21: YES), it is detected that the driver is in an inoperable state, and the process proceeds to S16.

  When the posture of the driver is not the posture determined to be the inoperable state (S21: NO), it is determined whether or not the driver posture is the posture not determined as the inoperable state registered in the storage device 52 (S22). If the posture of the driver is not determined to be an inoperable state (S22: YES), the process returns to S10. When the posture of the driver is different from the posture that is not determined as being incapable of driving (S22: NO), the posture collapse is next determined.

  First, it is determined whether or not posture collapse has been detected (S23). Specifically, the posture collapse is detected by the processing of the subroutine of FIG. First, the head tilt and the head tilt direction are calculated (S231). Subsequently, the inclination of the body part and the direction of the inclination of the body part are calculated (S232). Subsequently, an angle formed by the calculated inclination of the trunk and the inclination of the head, that is, the inclination θ of the head with respect to the trunk is calculated (S233). The calculated head inclination θ is learned when an inoperable state of the driver is not detected. Then, when the calculated head inclination θ is larger than the threshold Th1 (relative inclination threshold), posture collapse is detected (S234). Then, when posture collapse is not detected (S23: NO), the process proceeds to the determination of collapse of the face direction in S25.

  When posture collapse is detected (S23: YES), it is determined whether or not the head inclination θ is larger than the threshold Th1 and the face is not facing the front of the vehicle 10 (S24).

  At this time, simply, it may be continued only for the time T1 or more, and the condition may be only that the head inclination θ is larger than the threshold Th1 or that the face does not face the front of the vehicle 10. In addition, at this time, the calculated head inclination direction and body inclination direction do not change continuously for the time T1 or more, that is, the head position and the body position are within the range UA (non-moving determination). Range). The range UA is a range in which it can be considered that the head and the body are not moving. Alternatively, the calculated head inclination θ may be a condition that is greater than the learned head inclination beyond the determination value D1 (inclination determination value). Alternatively, it may be a condition that the handle 15 has not been operated for a time longer than the time T3 (operation determination time).

  Here, similarly to time T0, time T1 is set based on personal information registered in storage device 52, and is changed according to the vehicle speed, TTC, and whether or not driving support control is executed. The threshold value Th1 is reduced when convulsions are detected. Further, similarly to the time T0, the time T1 may be shortened on a day of the week or a time zone in which a sudden illness such as a heart attack that is statistically likely occurs.

  If the head inclination θ is greater than the threshold Th1 and the face does not face the front of the vehicle 10 after the time T1 (S24: YES), it is detected that the driver is unable to drive, S16 Proceed to the confirmation process. If the head inclination θ is not greater than the threshold Th1 for a time T1 or longer, or if the face is facing the front of the vehicle (S24: NO), then the determination of face collapse is performed.

  First, it is determined whether or not a face orientation collapse has been detected (S25). Specifically, the direction of the driver's face relative to the front of the vehicle is detected. Then, when the detected face orientation is larger than the threshold Th2 (face orientation threshold), the collapse of the face orientation is detected. When the collapse of the face direction is not detected (S25: NO), the process proceeds to the determination of the shaking state in S28. The detected face orientation is learned when the driver's inoperability is not detected.

  When the collapse of the face direction is detected (S25: YES), it is determined whether or not the face direction is larger than the threshold value Th2 continuously for the time T2 (the collapse direction determination time) (S26).

  Here, similarly to time T0, time T2 is set based on personal information registered in storage device 52, and is changed according to vehicle speed, TTC, and whether or not driving support control is executed. Further, similarly to the time T0, the time T2 may be shortened on a day of the week or a time zone on which a sudden illness such as a heart attack that is statistically likely occurs. The threshold value Th2 is reduced when convulsions are detected.

  If the face orientation is greater than the threshold value Th2 for more than the time T2 (S26: YES), it is detected that the driver is in an inoperable state, and the process proceeds to the confirmation process of S16.

  If the face orientation is not greater than the threshold Th2 for a time T2 or longer (S26: NO), then the driver has released the handle 15 for a time T3 or longer, or the accelerator opening is predetermined open. It is determined whether or not there is an accelerator operation and a brake operation (S27). If at least one of the three conditions in S27 is satisfied (S27: YES), it is detected that the driver is in an inoperable state, and the process proceeds to S16. When none of the three conditions in the process of S27 is satisfied (S27: NO), the determination of the shaking state is performed next.

  Here, in the determination of S26, it is also possible to make it a condition that at least one of the three conditions in the process of S27 is satisfied. Further, the determination in S26 and S27 may be made on the condition that the detected face orientation is larger than the learned face orientation by exceeding the determination value D1 (inclination determination value). In general, since the driver's hand remains above the driver's neck, the driver will not be disabled, so the driver's hand may be below the driver's neck. .

  Next, in the determination of the shaking state, it is determined whether or not the head wobbling that is different from the normal is detected with respect to the external force (S28). Specifically, the amplitude of the head shake is smaller than the amplitude Am1 (first amplitude) or the amplitude Am2 (second amplitude) from the time when the external force is applied to the vehicle 10 until the time T5 (swing determination time) elapses. ) Is determined.

  At this time, until the time T5 elapses after the external force is applied to the vehicle 10, the head vibrates with an amplitude different from normal, and after the time T5 elapses, the position of the head is within the range UA. This may be a condition. In other words, the head may be oscillated according to the external force, and the head position may not be changed after the influence of the external force is eliminated. Further, when it is not detected that the driver is incapable of driving, the head swing amplitude is learned, and the detected head swing amplitude is larger than the learned head swing amplitude. It is good also as a condition that it exceeds the judgment value D2 (amplitude judgment value). Moreover, it is good also as a condition that the handle | steering-wheel 15 is not operated continuously for time T3 or more.

  When the head wobbling that is different from the normal force is detected with respect to the external force (S28: YES), it is detected that the driver is in an inoperable state, and the process proceeds to S16. When the head wobbling that is different from the normal is not detected with respect to the external force (S28: NO), the white-eye state is determined next.

  First, it is determined whether or not a state with white eyes is detected (S29). Specifically, when the calculated white eye degree is larger than the threshold value Th3 (white eye threshold value), it is determined that a state where white eyes have been peeled is detected. Here, the degree of white eyes of both eyes of the driver is calculated, and it is determined that the state of having white eyes is detected on the condition that the degree of white eyes of both eyes is greater than the threshold Th3. However, when only one eye is detected, or when a state where the white of the eye is removed is simply detected, the detection of the state where the white of the eye is removed may be determined based on the degree of the white of the one eye.

  If it is determined that the state where the white of the eyes has been peeled is not detected (S29: NO), the driver determines that the frame out determination, the posture collapse determination, the face direction collapse determination, the shaking state determination, and the white eye state determination are all performed. Since it is not detected that the vehicle is in an inoperable state, the process returns to S10.

  If it is determined that a state with white eyes is detected (S29: YES), it is determined whether or not the degree of white eyes is greater than the threshold Th3 by continuing for a time T4 (white eye determination time) (S30). At this time, it may be a condition that the handle 15 is not operated continuously for time T3 or more.

  Here, similarly to the time T0, the time T4 is set based on the personal information registered in the storage device 52, and is changed according to the vehicle speed and TTC. The threshold value Th3 is reduced when convulsions are detected. Further, similarly to the time T0, the time T4 may be shortened on a day of the week or a time zone in which a sudden illness such as a heart attack that is statistically likely occurs.

  When the white eye degree is not greater than the threshold Th3 continuously for the time T4 or longer (S30: NO), the process returns to S10. When the white eye degree continues for a time T4 or more and is greater than the threshold value Th3 (S30: YES), the driver's inoperable state is detected, and the process proceeds to S16. This process is complete | finished above.

  Based on the detected head inclination θ with respect to the torso, the detected face orientation, and the detected head position, the degree of driver's posture collapse is displayed on the display 81 as shown in FIG. The posture collapse level increases as the detected head inclination θ increases. In addition, the posture collapse level is increased as the detected face orientation increases. Further, the posture collapse level is increased as the detected head position is further away from the standard position during driving. The standard position during driving is the position of the head when the vehicle 10 is started, or the average position of the head when it is not detected that the driver is unable to drive.

  According to this embodiment described above, the following effects are obtained.

  -When the driver's head is out of the range FA, the driver's inoperable state can be easily detected by detecting that the driver is in an inoperable state. Furthermore, when the trajectory until the head is out of the range FA is taken into account, it is possible to improve the accuracy of detecting the inoperable state of the driver.

  When the driver takes a position away from the driver's seat, the head often returns to the range FA even if the head is temporarily away from the range FA. Therefore, by making the condition that the driver's head is out of the range FA beyond the time T0, erroneous detection of the inoperable state of the driver can be suppressed.

  If the range FA is the entire image, it is detected that the driver is inoperable when the driver's head is not present in the image, so that the detection process can be simplified.

  -When the driver's posture collapses due to a sudden illness, it is considered that the pull-out amount of the seat belt 12 exceeds the first pull-out amount than the pull-out amount at the time of wearing. In addition, even when the driver's head is not detected, the position of the driver's head is out of the imaging range when the pull-out amount of the seat belt 12 is greater than the pull-out amount when the seat belt 12 is worn. It can be seen that Therefore, when the pull-out amount of the seat belt 12 is larger than the pull-out amount when the seat belt 12 is worn, the inoperable state of the driver can be detected with high accuracy.

  ・ When sudden illness develops, the driver's posture often collapses more rapidly than when taking things. Therefore, when a sudden illness develops, the seat belt 12 is often pulled out more rapidly than when taking things. Therefore, the erroneous detection of the inoperable state of the driver can be suppressed on condition that the amount of withdrawal of the seat belt 12 per amount detection time is larger than the second amount of withdrawal.

  When the driver's posture collapses due to a sudden illness, the high pressure portion in the pressure distribution of the seat portion 11a of the driver's seat is considered to be biased toward the end portion of the seat portion 11a. Even when the driver's head is not detected, the driver's head is out of the imaging range when the high-pressure portion of the pressure distribution of the driver's seat 11a is biased toward the end of the seat. It can be seen that it exists at the position. Therefore, when the high pressure portion in the pressure distribution of the seat portion 11a is biased toward the end portion of the seat portion 11a, it is possible to detect the inoperable state of the driver with high accuracy.

  When the head vibration greater than the predetermined amplitude is detected, that is, when the head is shaking, the time required for determining the driver's inoperable state can be shortened by reducing the time T0. As a result, the execution of the vehicle control when the driver is unable to drive can be quickly started.

  -The time required for determining the inoperable state of the driver can be shortened by shortening the time T0 as the moving speed of the head increases.

  -When the head moves closer to the end of the range FA, the time required for determining the inoperable state can be reduced by shortening the time required for determining the inoperable state when the moving speed of the head increases. .

  -When the inclination θ of the head with respect to the torso is larger than the threshold Th1, there is a high possibility that the posture is not collapsed due to sudden illness rather than the posture collapse when taking things. Therefore, when the inclination θ of the head with respect to the body portion is larger than the threshold value Th1, it is possible to detect the driver's inoperable state with high accuracy by detecting that the driver is in an inoperable state.

  -If the driver is conscious of the action of tilting his / her head with respect to the body part while driving, it is considered that the driver keeps his face facing the front of the vehicle 10 for safety. Therefore, by making the condition that the face is not facing the front of the vehicle 10, it is possible to suppress erroneous detection of the driver's inoperable state.

  ・ If the driver is conscious of the action of tilting the head with respect to the body while driving, the driver will immediately return the head position to the original position. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by making the condition that the head is greatly inclined with respect to the body part beyond the time T1.

  -Suppresses erroneous detection of driver inoperability by providing that the position of the head and torso does not change while the head is tilted greatly with respect to the torso over time T1. it can.

  -The condition that the driver tilts his / her head with respect to the torso is determined on condition that the detected head tilt θ is greater than the learned head tilt beyond the determination value D1. Even if it is held, it is possible to suppress erroneous detection of an inoperable state of the driver.

  -When the face direction with respect to the front of the vehicle 10 exceeds the threshold Th2 beyond the time T2, there is a high possibility that the face direction is not broken due to a side look or the like, but the face direction is broken due to a sudden illness. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  If the face direction relative to the front of the vehicle 10 is greater than the threshold Th2 and the driver has released the handle 15, the face direction may not be lost due to looking aside, but the face may be broken due to a sudden illness. high. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  -When the face direction with respect to the front of the vehicle 10 is larger than the threshold Th2 and the accelerator opening is larger than the predetermined opening degree, the face direction may not be broken due to looking aside, but the face direction may be broken due to sudden illness. High nature. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  -If the face direction with respect to the front of the vehicle 10 is greater than the threshold Th2 and the accelerator or brake operation is not performed for a time longer than the time T3, the face direction is not collapsed due to looking aside but is caused by a sudden illness There is a high possibility of collapse. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  In general, if the driver is in an operable state, the steering wheel operation is performed within time T3. Therefore, it is possible to suppress erroneous detection of the inoperable state of the driver by assuming that the steering wheel is not operated for a time longer than time T3.

  In general, if the driver can drive, the accelerator is not depressed greatly for a time longer than time T3. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by setting that the accelerator opening is larger than the predetermined opening for a time longer than the time T3.

  -If the driver is conscious of changing the direction of the face of the vehicle 10 while traveling, the driver will immediately return the face to the front for safety reasons. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by providing that the face orientation with respect to the front of the vehicle 10 exceeds the threshold Th2 beyond the time T2.

  In general, when a driver develops a sudden illness, the driver's hand remains above the driver's neck and the driver is not disabled. Therefore, it is possible to suppress erroneous detection of the driver's inoperable state on condition that the driver's hand is below the driver's neck.

  By detecting that the driver is incapable of driving when the amplitude of the head shake is smaller than the amplitude Am1 or larger than the amplitude Am2 until the time T5 elapses after the external force is applied to the vehicle. The driver's inoperable state can be easily detected.

  In general, when the driver is not conscious, the head vibrates according to the external force, and the head does not move when the external force is no longer affected. On the other hand, when the driver vibrates the head with a scissors, the head vibrates regardless of the influence of external force. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by assuming that the head vibrates according to the external force and the position of the head does not change after the influence of the external force is eliminated.

  Even when the driver has a habit of shaking his / her head, on condition that the detected amplitude of the head shake is larger than the learned amplitude of the head shake beyond the determination value D2. It is possible to suppress erroneous detection of an inoperable state of the driver.

  -When the state where the white of the eye is peeled is detected, it is possible to detect the inoperable state of the driver with high accuracy by detecting that the driver is in an inoperable state.

  The driver's eye contour and black eye area are detected based on the image. Then, based on the detected eye contour and black eye area, the white eye degree is calculated, and when the white eye degree is larger than the threshold value Th3, it is detected that the driver is in an inoperable state. Therefore, it is possible to detect with high accuracy the state in which the driver has white eyes, and thus to detect with high accuracy the inoperable state of the driver.

  -When the driver can drive, even if the degree of white eyes temporarily increases with the line of sight upward, the degree of white eyes does not remain high. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by making it a condition that the degree of white eye is greater than the threshold Th3 beyond the time T4.

  -By assuming that the degree of white eye on both sides is greater than the threshold Th3, even if the eye patch is worn on one eye, or one eye is a prosthetic eye, When the other eye does not have a white eye, the driver's inoperable state is not detected. Therefore, it is possible to suppress erroneous detection of the inoperable state of the driver.

  The ratio between the length Lw + Lb in the vertical direction of the eye and the length Lb in the vertical direction of the black eye area has a correlation with the ratio of the white eye area to the entire eye area, and thus the vertical length Lw + Lb of the eye and the black eye The white eye degree can be calculated from the ratio to the length Lb in the vertical direction of the region.

  -The smaller the distance Lb from the upper part of the eye contour to the lowermost part of the black eye region, the greater the degree of white eye. Therefore, the white eye degree can be calculated from the distance Lb from the upper part of the eye contour to the lowermost part of the black eye region.

  The white eye degree can be calculated from the ratio of the area of the white eye region to the area of the black eye region.

  -When the white of the eye is peeled off, the black eye area faces upward, so that the flatness of the black eye area is apparently increased. Therefore, the white eye degree can be calculated from the flatness ratio of the black eye region.

  -The larger the distance Lc from the vertical center line of the entire eye to the lowest part of the black eye region, the smaller the black eye region and the higher the white eye degree. Therefore, the white eye degree can be calculated from the distance Lc from the center line to the lowermost part of the black eye region.

  In general, when a driver has convulsions, there is a higher probability of being incapable of driving than when convulsions are not occurring. Therefore, when convulsions are detected, it is possible to easily detect the inoperable state of the driver by reducing the threshold values Th1 and Th2.

  -By shortening the times T0, T1, T2, and T4 as the vehicle speed increases, the time required for determining the driver's inoperable state decreases as the vehicle speed increases, and appropriate vehicle control execution can be started earlier.

  By shortening the times T0, T1, T2, and T4 as the TTC is shorter, the shorter the TTC, the shorter the time required for determining the driver's inoperable state, and the appropriate vehicle control can be started earlier.

  -By setting the times T0, T1, T2, and T4 based on personal information including the driver's medical history and age, the time required to determine the inoperable state can be set according to the characteristics of the driver individual. .

  When the driving support control is being executed in the vehicle, it is possible to suppress erroneous detection of the inoperable state by extending the times T0, T1, and T2.

  -The driver can recognize his / her posture by notifying the driver of the degree of collapse of the driver's posture. For this reason, the driver can correct his / her posture so that even if the driving posture collapses, the driver is not detected as being incapable of driving. Thereby, the erroneous detection of a driving impossible state can be suppressed.

  ・ When a driver's inoperable state is detected, by confirming to the driver whether the driver is inoperable, if the driver's inoperable state is erroneously detected, execution of vehicle control to safely stop the vehicle can be avoided. .

(Modification 1 of the first embodiment)
In the frame-out determination by the frame-out state detection means 71, the shape of the range FA that is a predetermined range may be an elliptical shape as shown in FIG. The elliptical shape is a shape in which the width direction of the vehicle 10 is a long axis and the height direction of the vehicle 10 is a short axis.

  Looking at the movements of the driver's heads during driving, the driver's heads move in the right and left directions when turning right and left, so they move within an elliptical shape with the left and right direction as the major axis. . Therefore, by setting the shape of the range FA to the above-described elliptical shape, it is possible to suppress erroneous detection of an inoperable state even when the driver moves the head in a normal driving operation.

(Modification 2 of the first embodiment)
The detection apparatus 100 may include a deforming unit that deforms the range FA according to the intention of the user including the driver or the driver information regarding the driver. The deformation means may be realized as a function provided in the state detection means 70. The deformation includes enlargement, reduction, and change of shape of the range FA. The deformation means deforms the range FA in accordance with the input operation of the HMI 80 by the user. The HMI 80 may include an enlargement switch or a reduction switch, and the display 81 may be a touch display.

  As shown in FIG. 20, when the driver has a driving posture that deviates from the standard position or when it is difficult to detect the inoperable state, the driver operates the enlargement switch or the like to operate the range FA. Is expanded to the range FAa.

  In addition, when the driver is an elderly person or a person with chronic illness, the driver himself or his / her family can easily detect the inoperable state by operating a reduction switch or the like to reduce the range FA to the range FAb. Good. When the driver is a bus or taxi driver, the manager of the bus or taxi may reduce the range FA to the range FAb by operating a reduction switch or the like in order to facilitate detection of the inoperable state. In FIG. 20, the ranges FA, FAa, and FAb are elliptical, but may be rectangular.

  Further, the deforming means may enlarge or reduce the range FA according to the driver information regardless of the user's intention. The driver information is information including at least one of the driver's age, medical history, and driving posture, and is stored in the storage device 52. The driving posture wrinkle may be obtained by learning the position of the head of the driver by the learning means 51, or may be obtained by the driver capturing and registering the driving posture in advance.

  For example, the deformation means expands the range FA to the range FAa when the driver has a habit of driving posture, and reduces the range FA to the range FAb when the driver is an elderly person or a person with a medical history. In addition, the deformation means may expand or contract the range FA according to the vehicle speed, TTC, or the presence or absence of driving support control, or the day of the week or the time of day when a sudden illness such as a statistically occurring heart attack is likely to occur. Then, the range FA may be reduced.

  Note that the deformation means changes the range FA according to at least one of the user's intention and driver information, and may change the range FA according to both the user's intention and driver information. Further, the deformation means may perform only one of enlargement and reduction of the range FA. Further, the deforming means may change the shape of the range FA according to the user's intention and driver information.

  As described above, the range FA is deformed according to the user's intention or the driver information, so that the inoperable state can be detected according to the user's intention or the driver information. For example, when there is a relatively high possibility of being in an inoperable state, the inoperable state can be easily detected. In addition, when the driver has a wrinkle in which the driving posture deviates from the standard position, the erroneous detection of the inoperable state can be suppressed by changing the range FA according to the wrinkle.

(Modification 3 of the first embodiment)
The detection apparatus 100 includes moving means for moving the range FA on the image in accordance with the intention of the user including the driver or driver information related to the driver. The moving means may be realized as a function provided in the state detecting means 70. The moving means moves the range FA in response to an input operation of the HMI 80 by the user, for example, a drag operation on the touch display. In other words, the second embodiment is different from the second modification in that the range FA is changed while the range FA is changed according to the user's intention or driver information.

  As shown in FIG. 21, when the driver has a wrinkle whose driving posture deviates from the standard position, the range FA is moved to a position corresponding to the wrinkle to make the range FAc, thereby preventing erroneous detection of the inoperable state. it can. In FIG. 21, the range FA has a rectangular shape, but may have an elliptical shape. Further, the modification example 2 and the modification example 3 may be combined to move the range FA and the range FA may be modified.

(Modification 4 of the first embodiment)
The driver camera 21 includes a stereo camera. Then, as shown in FIG. 22, the range FA is set as a three-dimensionally set range that extends not only in the width direction of the vehicle 10 but also in the front-rear direction. The stereo camera is installed on the left and right ends of the rearview mirror 16, for example. The range FA is not limited to a cubic shape, but may be an elliptical spherical shape or the like. When the range FA is an elliptical sphere, the major axis of the elliptical sphere is in the width direction of the vehicle 10. Further, Modification 2 or 3 may be applied to Modification 4.

  By using a stereo camera, the position of the head of the driver can be acquired in three dimensions. That is, not only the position of the head in the width direction of the vehicle 10 but also the position of the head in the front-rear direction of the vehicle 10 can be acquired. Therefore, by setting the range FA three-dimensionally, not only when the driver is inoperable and falls in the left-right direction, but also when the driver is inoperable and falls forward, frame out determination Thus, it is possible to detect the inoperable state of the driver.

(Second Embodiment)
Hereinafter, the difference between the detection apparatus 100 according to the second embodiment and the detection apparatus 100 according to the first embodiment will be described. In the detection apparatus 100 according to the second embodiment, the method of frame-out determination by the frame-out state detection unit 71 is different from the detection apparatus 100 according to the first embodiment.

  In the present embodiment, the frame-out state detection means 71 has three ranges of a range FA1 (first predetermined range), a range FA2 (second predetermined range), and a range FA3 as a range FA that is a predetermined range in frame-out determination. Set. The range FA1 is the smallest range. The range FA2 is larger than the range FA1, and includes a range FA1 and a range outside the range FA1 (the end side of the image). The range FA3 is larger than the range FA2 and includes a range FA2 and a range outside the range FA2. In the present embodiment, the centers of the range FA1, the range FA2, and the range FA3 are a common center. In the present embodiment, three ranges are set as the predetermined ranges in the frame-out determination, but at least two ranges may be set.

  Further, the frame-out state detection means 71 sets a frame-out determination time for each predetermined range. That is, the frame-out state detection unit 71 sets times T01, T02, and T03 as frame-out determination times for each of the ranges FA1, FA2, and FA3.

  The frame-out state detection means 71 is incapable of driving the driver on condition that the head of the driver is out of at least one predetermined range beyond the frame-out determination time corresponding to the predetermined range. Detect that.

  That is, in the present embodiment, (1) the driver's head is out of range FA1 beyond time T01, (2) the driver's head is out of range FA2 over time T02, (3) There are three conditions that the head of the driver is out of range FA3 beyond time T03. The frame-out state detection means 71 detects that the driver is in an inoperable state when at least one of the three conditions (1) to (3) is satisfied.

  Therefore, in the flowchart of FIG. 16, in the process of S15, at least one of the three conditions (1) to (3) described above is used instead of determining whether or not the state is out of the range FA for a time T0 or more. It is determined whether one condition is satisfied. If at least one condition is satisfied, the process proceeds to S16. If none of the three conditions is satisfied, the process proceeds to S21.

  Generally, when the driver takes a position away from the driver's seat, the staying time is shortened as the driver moves away from the normal driving position for safety. Therefore, the frame-out state detection unit 71 sets the frame-out determination time to be shorter as it corresponds to a wider predetermined range. That is, the frame-out state detection unit 71 sets the frame-out determination time so that time T01> time T02> time T03.

  For example, when the time T01 is 3 seconds and the time T02 is 1 second, even if 3 seconds have not passed since the driver's head has gone out of the range FA1, 1 has elapsed since the driver's head has gone out of the range FA2. When the second elapses, an inoperable state of the driver is detected.

  Note that the times T01, T02, and T03 are set based on the personal information registered in the storage device 52, and are changed according to the vehicle speed, TTC, and whether or not the driving support control is executed, similarly to the time T0. Further, the times T01, T02, and T03 may be shortened on the day of the week and the time of day when a sudden illness such as a heart attack that statistically appears is likely to occur, similarly to the time T0. Furthermore, the times T01, T02, and T03 may be shortened according to the detection of the amplitude of the head position, the moving speed of the head, and the convulsions, similar to the time T0.

  In FIG. 23, the range FA1, the range FA2, and the range FA3 have an elliptical shape, but the range FA1, the range FA2, and the range FA3 may have a rectangular shape. Furthermore, the range FA1, the range FA2, and the range FA3 may be respectively deformed or moved by applying the second and third modifications of the first embodiment. Further, the range FA1, the range FA2, and the range FA3 may be set three-dimensionally by applying the fourth modification of the first embodiment.

  According to the second embodiment described above, a plurality of predetermined ranges having different widths and overlapping central portions are set, and a frame-out determination time is set for each predetermined range. That is, the frame-out determination time is set according to the degree of deviation from the normal driving position. Therefore, the inoperable state of the driver can be detected according to the degree of deviation from the normal driving position.

  And, as the degree of deviation from the normal driving position is larger, the frame-out determination time is set shorter, so that erroneous detection of the inoperable state is suppressed, and when the driver becomes inoperable, the early Can be detected.

(Other embodiments)
The driver camera 21 may be a part of the four cameras mounted in the vehicle interior. There may be at least one driver camera 21.

  The frame-out state detection unit 71 may detect that the driver is in an inoperable state based on the trajectory acquired by the trajectory acquisition unit 62. When a driver develops sudden illness and becomes unable to drive, the driver's head often moves from the driving position and does not return to the driving position. It is possible to detect an inoperable state.

  The direction-displacement state detection means 73 is further provided when the driver's face is facing downward larger than the threshold Th2d (downward threshold) or upward larger than the threshold Th2u (upward threshold). It may be detected that the driver is in an inoperable state. In general, when a sudden illness occurs and the driver's consciousness is lost, the driver's face often turns downward larger than the threshold Th2d or faces larger upward than the threshold Th2u. Therefore, it can be detected that the driver is in an unmanageable state when the driver's face is large downward or large upward.

  The shaking state detection means 74 detects that the head detected by the head detection means 61 exceeds the time T6 (return determination time) when an external force is applied to the vehicle 10 while the vehicle 10 is traveling. When the vehicle is tilted in the direction, it may be detected that the driver is in an inoperable state. Normally, when there is a driver's consciousness, when an external force (specifically, an external force in the left-right direction and the front-rear direction) is applied to the vehicle 10, the driver's head tilts in the direction of the external force, but returns to its original location within time T6. Return. On the other hand, if the driver develops a sudden illness and is not conscious, the driver's head is less resistant to the external force and remains tilted in the direction of the external force over time T6. Therefore, the shaking state detection means 74 can detect the inoperable state of the driver in the case described above.

  The white-eye state detecting means 75 further detects that the driver is in an inoperable state even when the opening of the mouth (specifically, the opening in the vertical direction) detected by the facial expression detecting means 67 is larger than the open determination amount. May be. When a driver develops a sudden illness and has white eyes, the mouth is often opened. Therefore, even when the opening of the driver's mouth is larger than the open determination amount, it may be detected that the driver is in an inoperable state.

  When the frame out determination, the posture collapse determination, the face collapse determination, the shaking state determination, and the white eye state determination are all performed, the detection accuracy of the inoperable state of the driver is the highest, but at least one determination may be performed. Moreover, you may perform combining any number of determinations. In that case, it is preferable to perform the priority in the order of frame-out determination, posture collapse determination, face direction collapse determination, shaking state determination, and white-eye state determination.

  For example, when combining the posture collapse determination and the swing state determination, if the posture failure determination does not detect that the driver is in an inoperable state, performing the shake state determination increases the driver's inoperability state. It can be detected with accuracy.

  In addition, when combining the face direction determination and the shaking state determination, if the driver does not detect that the driver is in an inoperable state by the face direction determination, the driver's inoperable state Can be detected with high accuracy.

  In addition, when combining the shaking state determination and the white-eye state determination, if the white-eye state determination is performed when the driver state is not detected in the shaking state determination, the driver's inoperability state is increased. It can be detected with accuracy.

  The learning unit 51 may learn the driver's posture when it is erroneously detected that the driver is in an inoperable state. That is, although it is detected that the driver is incapable of driving, the posture of the driver when there is a response from the driver may be learned. Then, the learned posture may be a posture that is not determined as an inoperable state.

  A statistical value of each threshold value and each determination value may be stored in the storage device 52 and used as an initial value. The statistical value of each threshold value and each determination value is obtained by statistically calculating each threshold value and each determination value corresponding to each of a plurality of vehicle drivers. Moreover, it is good to make it statistics in an information center by transmitting each threshold value and each determination value which are set corresponding to the driver from the vehicle 10 to an information center.

  -After confirming whether the driver is incapable of driving and receiving a response from the driver, the driver may recognize that the driver can drive for a certain period of time. Moreover, you may make it perform the process which detects the driving | operation impossible state of a driver at the time interval (for example, once per hour) which the driver set.

The external force applied to the vehicle 10 may be detected by the seat sensor 23 other than the G sensor 44.
(Appendix 1)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection unit that detects that the driver is in an inoperable state when the head detected by the head detection unit is out of a predetermined range of the image while the vehicle is running. (71),
An inoperable state detection device for a driver, comprising: deformation means for deforming the predetermined range in accordance with an intention of a user including the driver or driver information related to the driver.
(Appendix 2)
The driver's inoperable state detection device according to supplementary note 1, further comprising a moving unit that moves the predetermined range in accordance with a user's intention including the driver or driver information related to the driver.
(Appendix 3)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection unit that detects that the driver is in an inoperable state when the head detected by the head detection unit is out of a predetermined range of the image while the vehicle is running. (71),
An inoperable state detection device for a driver, comprising: moving means for moving the predetermined range in accordance with a user's intention including the driver or driver information related to the driver.
(Appendix 4)
The driver's inoperable state detection device according to any one of appendices 1 to 3, wherein the driver information includes at least one of the driver's age, medical history, and driving posture.
(Appendix 5)
5. The driver inoperable state detection device according to any one of appendices 1 to 4, wherein the shape of the predetermined range is an elliptical shape having a major axis in a width direction of the vehicle.
(Appendix 6)
The state detection means detects that the driver is in an inoperable state on the condition that the head is out of the predetermined range beyond a frame-out determination time (T0). The inoperable state detection device for a driver according to any one of the above.
(Appendix 7)
As the predetermined range, at least a first predetermined range and a second predetermined range including the first predetermined range and a range outside the first predetermined range are set,
The frame out determination time is set for each predetermined range,
The state detecting means is that the driver is in an inoperable state on the condition that the head is out of at least one predetermined range beyond the frame-out determination time corresponding to the predetermined range. The inoperable state detection device for a driver according to appendix 6, wherein
(Appendix 8)
The driver inoperable state detection device according to appendix 7, wherein the frame-out determination time is set to be shorter as the predetermined range is wider.
(Appendix 9)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection unit that detects that the driver is in an inoperable state when the head detected by the head detection unit is out of a predetermined range of the image while the vehicle is running. (71)
The state detection means detects that the driver is in an inoperable state on the condition that the head is out of the predetermined range beyond a frame-out determination time (T0),
As the predetermined range, at least a first predetermined range and a second predetermined range including the first predetermined range and a range outside the first predetermined range are set,
The frame out determination time is set for each predetermined range,
The state detecting means is that the driver is in an inoperable state on the condition that the head is out of at least one predetermined range beyond the frame-out determination time corresponding to the predetermined range. Detect
The inoperable state detection device for a driver in which the frame-out determination time is set to be shorter as the predetermined range is wider.
(Appendix 10)
The imaging device is a stereo camera;
The driver's inoperable state detection device according to any one of appendices 1 to 9, wherein the predetermined range is a three-dimensionally set range.
(Appendix 11)
A trajectory acquisition means (62) for acquiring the trajectory of the head from the position of the head detected by the head detection means;
While the vehicle is running, the state detecting means detects that the head detected by the head detecting means is out of the predetermined range of the image and that the head is out of the predetermined range. The driver inoperable state detection device according to any one of appendices 1 to 10, wherein the driver is detected to be in an inoperable state based on the trajectory acquired by the trajectory acquisition unit.
(Appendix 12)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection unit that detects that the driver is in an inoperable state when the head detected by the head detection unit is out of a predetermined range of the image while the vehicle is running. (71),
Trajectory acquisition means (62) for acquiring the trajectory of the head from the position of the head detected by the head detection means;
While the vehicle is running, the state detecting means detects that the head detected by the head detecting means is out of the predetermined range of the image and that the head is out of the predetermined range. Based on the trajectory acquired by the trajectory acquisition means, it is detected that the driver is in an inoperable state,
When the head is no longer detected by the head detection means, the head detection means detects an inoperable state of a driver that searches the vicinity of the final position of the trajectory acquired by the trajectory acquisition means.
(Appendix 13)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device mounted on the vehicle;
Trajectory acquisition means (62) for acquiring the trajectory of the head from the position of the head detected by the head detection means;
Frame out state detection means (71) for detecting that the driver is in an inoperable state based on the trajectory acquired by the trajectory acquisition means during traveling of the vehicle,
When the head is no longer detected by the head detection means, the head detection means detects an inoperable state of a driver that searches the vicinity of the final position of the trajectory acquired by the trajectory acquisition means.
(Appendix 14)
Comprising an amount detection means (22) for detecting the amount of withdrawal of the seat belt of the driver seat;
The state detection means is provided on the condition that the pull-out amount detected by the amount detection means is greater than the first pull-out amount detected when the seat belt is worn, exceeding the first pull-out amount. The driver inoperable state detecting device according to any one of appendices 6 to 9, which detects that the inoperable state is present.
(Appendix 15)
The state detection means detects that the driver is in an inoperable state on condition that the withdrawal amount per amount detection time detected by the amount detection means is larger than the second withdrawal amount. The inoperable state detection device for a driver according to claim 1.
(Appendix 16)
Seat pressure detecting means (23) for detecting the pressure distribution in the seat portion of the driver seat,
The state detection means is that the driver is in an inoperable state on condition that the high pressure portion in the pressure distribution detected by the seat pressure detection means is biased toward the end of the seat (11a). The inoperable state detection device for a driver according to any one of supplementary notes 6 to 9, 14, and 15 to be detected.
(Appendix 17)
Any one of appendices 6 to 9 and 14 to 16 for shortening the frame-out determination time when the position of the head sequentially detected by the head detecting means vibrates with an amplitude larger than a predetermined amplitude The inoperable state detection device for a driver according to claim 1.
(Appendix 18)
The driver inoperable state detection device according to any one of appendices 6 to 9 and 14 to 17, wherein the frame-out determination time is shortened as the moving speed of the head sequentially detected by the head detection unit increases.
(Appendix 19)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection means for detecting that the driver is in an inoperable state when the head detected by the head detection means is out of a predetermined range of the image during traveling of the vehicle. (71)
The state detection means detects that the driver is in an inoperable state on the condition that the head is out of the predetermined range beyond a frame-out determination time (T0),
An inoperable state detection device for a driver that shortens the frame-out determination time as the moving speed of the head sequentially detected by the head detecting means increases.
(Appendix 20)
Additional notes 6 to 9 for shortening the frame-out determination time when the moving speed of the head increases as the position of the head sequentially detected by the head detecting means approaches the end of the predetermined range. The driver inoperable state detection device according to any one of 14 to 19.
(Appendix 21)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection means for detecting that the driver is in an inoperable state when the head detected by the head detection means is out of a predetermined range of the image during traveling of the vehicle. (71)
The state detection means detects that the driver is in an inoperable state on the condition that the head is out of the predetermined range beyond a frame-out determination time (T0),
When the head position sequentially detected by the head detection means approaches the end of the predetermined range, the driver's inoperable state reduces the frame-out determination time when the moving speed of the head increases. Detection device.
(Appendix 22)
Convulsions detecting means (63) for detecting convulsions of the driver,
The driver's inoperable state detection device according to any one of appendices 6 to 9 and 14 to 21, which shortens the frame-out determination time when the convulsions are detected by the convulsions detection unit.
(Appendix 23)
The driver inoperable state detection device according to any one of appendices 6 to 9 and 14 to 22, which shortens the frame-out determination time as the vehicle speed of the vehicle increases.
(Appendix 24)
The driver cannot operate according to any one of appendices 6 to 9 and 14 to 23, which shortens the frame-out determination time as the collision margin time obtained by dividing the inter-vehicle distance from the preceding vehicle by the relative speed with the preceding vehicle is shorter. State detection device.
(Appendix 25)
Storage means (52) in which personal information including the medical history and age of the driver is registered;
The driver inoperable state detection device according to any one of appendices 6 to 9 and 14 to 24, which sets the frame-out determination time based on personal information registered in the storage unit.
(Appendix 26)
Head detection means (61) for sequentially detecting the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
A frame-out state detection unit that detects that the driver is in an inoperable state when the head detected by the head detection unit is out of a predetermined range of the image while the vehicle is running. (71),
Storage means (52) in which personal information including medical history and age of the driver is registered,
The state detection means detects that the driver is in an inoperable state on the condition that the head is out of the predetermined range beyond a frame-out determination time (T0),
An inoperable state detection device for a driver that sets the frame-out determination time based on personal information registered in the storage means.
(Appendix 27)
27. The driver inoperable state detection device according to any one of appendices 6 to 9 and 14 to 26, which extends the frame-out determination time when driving support control is being executed in the vehicle.
(Appendix 28)
The inoperable state detection device for a driver according to any one of appendices 1 to 27, wherein the inoperable state is a state in which the driver has developed a sudden illness.

  DESCRIPTION OF SYMBOLS 10 ... Vehicle, 21 ... Driver camera, 61 ... Head detection means, 62 ... Trajectory acquisition means, 63 ... Convulsion detection means, 64 ... Inclination detection means, 65 ... Face direction detection means, 66 ... White eye detection means, 71 ... Frame Out state detection means, 72 ... posture collapse state detection means, 73 ... direction collapse state detection means, 74 ... shaking state detection means, 75 ... white eye state detection means.

Claims (3)

Head detection means (61) for detecting in three dimensions the position of the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10); ,
Face orientation detection means (65) for detecting the orientation of the driver's face relative to the front of the vehicle based on the driver seat image;
While the vehicle is running, the position of the head detected by the head detection means is out of a predetermined range, and the face of the driver with respect to the front of the vehicle detected by the face orientation detection means is A state detecting means (70, 71, 73) for detecting that the driver is in an inoperable state when the direction is lower than a threshold value (Th2d). Inoperable state detection device. Head detection means (61) for detecting in three dimensions the position of the head above the driver's neck based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10); ,
Face orientation detection means (65) for detecting the orientation of the driver's face relative to the front of the vehicle based on the driver seat image;
While the vehicle is running, the position of the head detected by the head detection means is out of a predetermined range, and the face of the driver with respect to the front of the vehicle detected by the face orientation detection means is And a state detecting means (70, 71, 73) for detecting that the driver is in an inoperable state when the direction is higher than a threshold value (Th2u). Inability state detection device. In the inoperable state detection device for a driver according to claim 1 or claim 2,
The state detection means is configured such that the position of the head exceeds the determination time (T0) and is out of the predetermined range, and the orientation of the face with respect to the front of the vehicle exceeds the determination time and exceeds the threshold. And a driver inoperable state detecting device that detects that the driver is in an inoperable state.

Images