JP6607287B2 - Driver inoperability detection device - Google Patents

Description

  The present invention relates to a driver inoperability state detection device that detects a state in which a driver has become inoperable.

  Conventionally, it has been a problem that a driver falls into an inoperable state due to a sudden illness or the like during driving of the vehicle, leading to an accident. Therefore, it has been proposed to detect such an inoperable state of the driver and prevent an accident.

  In the emergency evacuation device described in Patent Document 1, various information is acquired from a physiological measurement device or the like, and the driver state such as a driver's consciousness degree, arousal degree, or posture is recognized, and the driver state is lowered. In addition, emergency evacuation assistance is provided. The physiological measurement device is a device that measures a driver's heartbeat information, respiratory information, brain wave information, and the like.

JP 2014-19301 A

  The device described in Patent Document 1 uses the driver's heart rate information, respiratory information, brain wave information, etc. to recognize the driver state, but measures the driver's heart rate information, respiratory information, brain wave information, etc. in the vehicle. In order to do so, a complicated device is required, which is difficult to realize.

  In view of the above circumstances, it is a main object of the present invention to provide a driver inoperability state detection device that can easily detect that a driver has become inoperable.

In order to solve the above problems, the invention disclosed in the present application is an inoperable state detection device for a driver, which detects a driver's head based on an image of a driver's seat taken by an image pickup device mounted on a vehicle. And the amplitude of the shaking of the head detected by the head detecting means from when an external force is applied to the vehicle until the shaking judgment time elapses during the running of the vehicle. And a swing state detecting means for detecting that the driver is in an inoperable state when the amplitude is smaller than the amplitude or larger than the second amplitude larger than the first amplitude.

According to the invention disclosed by the present application , the head of the driver is detected based on an image obtained by imaging the driver's seat. Usually, when there is a driver's consciousness, when an external force is applied to the vehicle, the head of the driver swings with an amplitude within a range from the first amplitude to the second amplitude. On the other hand, when the driver develops a sudden illness and the body is stiff, the amplitude of the head shake is smaller than that in the normal state. In addition, when the driver develops a sudden illness and the body is relaxed, the amplitude of the shaking of the head becomes larger than that in the normal state.

  Therefore, when the amplitude of the head shake is smaller than the first amplitude or larger than the second amplitude after the external force is applied to the vehicle until the shake determination time elapses, the driver is incapable of driving. By detecting, it is possible to easily detect the inoperable state of the driver.

The invention according to claim 1 is a driver inoperable state detection device that detects a driver's head based on an image of a driver's seat imaged by an imaging device mounted on a vehicle. detection means, during travel of the vehicle, when an external force is applied to the vehicle, the head detected by the head detecting means, when the inclined Iteiru beyond restoration determination time, the driver comprising a you detected state detecting means to be a non-operational state, the.

According to the first aspect of the present invention, the head of the driver is detected based on the image obtained by imaging the driver's seat. Usually, if there is a consciousness of the driver, when an external force is applied to the vehicle, the driver of the head can have inclined, back to the original location in the return determination time. In contrast, if there is no consciousness developed sudden illness driver, the driver's head, resistance to external force is reduced, it will remain had inclined beyond restoration determination time.

  Therefore, when a lateral external force is applied to the vehicle and the driver's head is tilted in the direction of the external force beyond the return determination time, it is detected that the driver is in an inoperable state. The inoperable state of can be easily detected.

The block diagram which shows the structure of a driving impossible state detection apparatus. The figure which shows the vehicle interior which mounts an operation impossible state detection apparatus. The figure which shows the driver's seat of the vehicle carrying a driving impossible state detection apparatus. The block diagram which shows the function of a control apparatus. The figure which shows the change of the attitude | position when sudden disease develops. The figure which shows the attitude | position when taking a thing. The figure which shows the change of the attitude | position when sudden disease develops. The figure which shows the attitude | position when looking aside. The figure which shows the attitude | position when taking a thing. The figure which shows the face orientation when sudden illness develops. The figure which shows the change of face orientation when looking aside. The figure which shows the shaking of the head accompanying generation | occurrence | production of external force. The figure which shows the amplitude range of the shaking of the head determined to be a driving impossible state. The figure which shows the change of the facial expression when sudden illness develops. The figure which shows the state which faced the normal state and the white of eyes. The flowchart which shows the process sequence which detects a driving | operation impossible state. A subroutine showing a processing procedure for detecting posture collapse. The figure which shows the aspect which alert | reports a posture collapse level to a driver.

  Hereinafter, an embodiment embodying a driver inoperability state detection device will be described with reference to the drawings. In the present embodiment, the driver's inoperable state includes a state in which the driver develops sudden illness and becomes unconscious and cannot perform driving operation, and a driver develops a sudden illness such as a heart attack and is conscious but cannot move the body. Therefore, it includes a state in which the driving operation cannot be performed.

  First, the structure of the detection apparatus 100 (driver inoperable state detection apparatus) according to the present embodiment will be described with reference to FIGS. The detection device 100 includes a control device 50, a driver state recognition device 20, a vehicle information recognition device 30, a traveling environment recognition device 40, an HMI (Human Machine Interface) 80, and a storage device 52, and detects an inoperable state of the driver. To do. And the detection apparatus 100 transmits the instruction | command which stops a vehicle safely to the vehicle control apparatus 90, when a driver confirms that it is a driving impossible state and there is no response.

  The driver state recognition device 20 includes a plurality of driver cameras 21 (imaging devices), a seat belt sensor 22 (amount detection means), and a seating surface sensor 23 (seat pressure detection means). The driver camera 21 is a CCD camera, for example, and images the driver's seat illuminated by an illumination device such as a near infrared LED. As shown in FIGS. 2 and 3, the driver camera 21 is mounted on the meter panel 14, the approximate center of the lower end of the rearview mirror 16, and the left and right A pillars 17 toward the driver. Instead of the meter panel 14, a driver camera 21 may be installed on the dashboard 13 (shown by a broken line) or on a steering column. Further, instead of the lower end of the rearview mirror 16, it may be installed at the left end or the right end (shown by a broken line) of the rearview mirror 16. These four driver cameras 21 constitute a driver status monitor, which captures several tens of images per second from the front side of the driver sitting on the driver's seat 11.

  The seat belt sensor 22 is a sensor that detects the amount of withdrawal of the seat belt 12. Specifically, the seat belt sensor 22 is an encoder that detects a rotation angle of a motor that feeds and winds the seat belt 12. The seat surface sensor 23 is a sensor that detects the pressure distribution of the seat portion 11a of the seat 11 of the driver's seat.

  The vehicle information recognition device 30 includes a vehicle speed sensor 31, a steering angle sensor 32, an accelerator sensor 33, and a brake sensor 34. The vehicle speed sensor 31 is a sensor that detects the speed of the vehicle 10. The steering angle sensor 32 is a sensor that detects the steering angle of the handle 15. The accelerator sensor 33 is a sensor that detects an accelerator opening, that is, an operation amount of an accelerator pedal. The brake sensor 34 is a sensor that detects an operation amount of a brake pedal.

  The travel environment recognition device 40 includes a front / rear camera 41, a front / rear sensor 42, a car navigation device 43, and a G sensor 44. The front / rear camera 41 is a camera that images the front of the vehicle 10 including the white line of the road, and a camera that images the rear and rear sides of the vehicle 10. The front / rear sensor 42 is a sensor such as an ultrasonic sensor, a laser radar, or a millimeter wave radar, and detects an object in front or rear of the vehicle 10 and acquires a distance between the vehicle 10 and an object in front or rear. Based on the distance between the vehicle 10 acquired by the front / rear sensor 42 and the front or rear vehicle, the relative speed between the front vehicle and the rear vehicle can be calculated.

  The car navigation device 43 calculates the current position of the vehicle 10 by using the GPS signal received by the GPS receiver and information acquired by various sensors including the G sensor, and guide routes from the current position to the destination Is calculated. The G sensor 44 is, for example, a sensor that is installed on the seat 11 and detects three-dimensional acceleration in the front-rear, left-right, and upper-lower direction of the vehicle 10. In addition, the G sensor 44 may be a sensor included in the car navigation device 43. When the vehicle 10 includes a vehicle operation management system (AVOS), the G sensor 44 is a sensor included in the AVOS. May be. That is, the G sensor 44 may be shared if there is one installed for other purposes.

  The control device 50 is a microcomputer including a CPU, ROM, RAM, I / O, and the like, and acquires various information from the driver state recognition device 20, the vehicle information recognition device 30, the traveling environment recognition device 40, the storage device 52, and the HMI 80. To do. The control device 50 and the various devices are connected by wired communication such as CAN or wireless communication such as LAN or Bluetooth (registered trademark). In addition, the control device 50 realizes the functions of the image analysis unit 60, the learning unit 51, and the state detection unit 70 by the CPU executing various programs stored in the ROM, and detects the inoperable state of the driver. . Detailed explanation of each means will be described later.

  The HMI 80 (attitude notification means, confirmation means) includes a display 81, a speaker 82, and a cancel switch 83. The display 81 is a display of the car navigation device 43 or an in-vehicle display provided in the meter panel 14. The display 81 may be a touch display including a liquid crystal panel or an organic EL panel. The display 81 reports the degree of the driver's posture collapse based on the driver's posture detected from the image. Specifically, the display 81 displays the status of the driver's posture in five stages. The posture collapse level 5 with the highest degree of collapse is a level at which it is determined that the driver has developed a sudden illness and is unable to maintain the driving posture, that is, a driving impossible state. Since the driver can check his / her driving posture by looking at the posture status displayed on the display 81, when the posture collapse level approaches 5, the driving posture can be corrected before being determined as being incapable of driving. .

  The speaker 82 is an in-vehicle speaker shared with the car navigation device 43, the audio device, and the like. When the inoperable state of the driver is detected, the speaker 82 confirms to the driver by voice whether the inoperable state. The display 81 may display a screen for confirming the inoperable state. Further, the speaker 82 may notify the driver of the posture collapse level by voice.

  The cancel switch 83 is a switch that stops detection of the inoperable state. When the cancel switch 83 is operated once, detection of the inoperable state is stopped for one trip. Further, when the cancel switch 83 is operated during the trip, the detection of the inoperable state is stopped while the cancel switch 83 is operated or for a certain period of time (about several seconds) after the cancel switch 83 is operated. Therefore, if the driver operates the cancel switch 83 in advance when taking an action, there is no possibility that the driver is erroneously detected as being inoperable even if the driver's posture collapses.

  Next, various functions realized by the control device 50 will be described with reference to FIG. The image analysis means 60 includes a head detection means 61, a trajectory acquisition means 62, a convulsions detection means 63, an inclination detection means 64, a face orientation detection means 65, and a white eye detection means 66.

  The head detection unit 61 sequentially detects the head above the driver's neck based on the driver's seat image captured by the driver camera 21. Specifically, the head detection unit 61 extracts an edge representing the outline of the driver's head from the driver's seat image each time a driver's seat image is captured by the driver camera 21, and is an area surrounded by the extracted edges. Is detected as the head.

  The trajectory acquisition unit 62 acquires the trajectory of the driver's head from the position of the driver's head sequentially detected by the head detection unit 61. For example, the trajectory acquisition unit 62 uses the center of the driver's head detected in each image as the head position, and connects the head positions in each image to acquire the head trajectory.

  The convulsions detecting means 63 detects convulsions of the driver, that is, involuntary contraction of the muscles of the trunk part below the head and neck of the driver. Specifically, the convulsions detection means 63 extracts edges representing the contours of the driver's head and torso in each image, and when the extracted edges vibrate regularly (periodically) in successive images, Detect that the driver is spasm.

  The inclination detecting means 64 detects the inclination θ of the head with respect to the driver's body based on the driver's seat image. Specifically, the inclination detection means 64 detects the regions surrounded by the edges representing the contours of the head and the body as the head and the body, respectively, and detects the central axes of the head and the body. And the inclination detection means 64 makes the inclination of the central axis of the head with respect to the central axis of the body part the inclination θ of the head. The center axis of the body part is detected from the body part in which the direction of the body part is determined by matching the prepared body direction pattern with the detected body part direction to determine the body part direction. Further, the feature axis such as the eyes, nose and mouth of the face included in the head is extracted from the central axis of the head, and is detected from the three-dimensional arrangement of the feature points of the face. When the head is tilted forward, the distance between the facial feature point and the front of the vehicle approaches, and when the head turns back, the distance between the facial feature point and the front of the vehicle increases. When detecting the central axis of the head, the distance between facial feature points in the front-rear direction of the vehicle may be used.

  Alternatively, the inclination detection unit 64 detects the seat belt 12 of the driver's seat from the image of the driver's seat, and detects the inclination θ of the head with respect to the trunk from the positional relationship between the seat belt 12 and the head. Since the body part of the driver is restrained by the seat belt 12, the position of the body part can be estimated from the position of the seat belt 12.

  The face direction detection means 65 detects the direction of the driver's face relative to the front of the vehicle 10 based on the driver's seat image. The face direction detection unit 65 detects the inclination of the face with respect to the vertical plane facing the front surface of the vehicle 10 as the face direction.

  The white eye detection unit 66 includes a facial expression detection unit 67 and a white eye degree calculation unit 68, and detects a state in which the driver has white eyes. Here, the state where the whites are removed is not limited to the state where the whites are completely removed as shown in FIG. 15C, but the black eye area is smaller than a predetermined amount as shown in FIG. 15B. Including state. That is, the state where the whites are peeled is a state where the visual field is narrower than a predetermined range due to the biased black eyes.

  The facial expression detection means 67 detects the eye contour and black eye area of the driver based on the driver seat image. Here, the outline of the eyes of the driver is a boundary line between the eyelids and the eyes. The black eye region is a region having lightness lower than that of the white eye in a region inside the outline of the eye, and is not limited to black, but is a region having a color such as blue, brown, or gray. The facial expression detection means 67 detects the opening of the driver's mouth from the edge representing the extracted mouth outline.

  The white eye degree calculation means 68 calculates the white eye degree of the driver's eyes based on the eye contour and the black eye area detected by the facial expression detection means 67.

  Specifically, the white eye degree calculation means 68 calculates the white eye degree from the ratio between the vertical length Lw + Lb of the region surrounded by the outline of the eye and the vertical length Lb of the black eye region (see FIG. 15). The smaller the length Lb with respect to the length Lw + Lb, the greater the degree of white eye. Alternatively, the white eye degree calculation means 68 calculates the white eye degree based on the distance Lb from the upper part of the eye contour to the lowermost part of the black eye region. The smaller the distance Lb, the greater the degree of white eye. Alternatively, the white eye degree calculating means 68 calculates the white eye degree based on the ratio of the area of the white eye region obtained by removing the area of the black eye region from the area of the whole eye region surrounded by the outline of the eye and the area of the black eye region. calculate. The smaller the area of the black eye region relative to the area of the white eye region, the greater the degree of white eye.

  Alternatively, the white eye degree calculation unit 68 calculates the white eye degree based on the flatness ratio of the black eye region. In a state where the whites are peeled off, the black eye region faces upward, so that the flatness of the black eye region increases apparently, and the degree of white eye increases as the flatness of the black eye region increases. Alternatively, the white eye degree calculation unit 68 calculates the white eye degree based on the distance Lc from the center line in the vertical direction of the region surrounded by the outline of the eye to the lowest part of the black eye region.

  The learning means 51 learns the head inclination θ detected by the inclination detection means 64 when the driver is not in an inoperable state. The learning means 51 learns the face orientation detected by the face orientation detection means 65 when the driver is not in an inoperable state. Further, the learning means 51 learns the amplitude of the head shake detected by the head detecting means 61 when the driver is not in an inoperable state. That is, the learning means 51 learns the driver's driving posture habit. When there are a plurality of drivers who drive the vehicle 10, the driver learns the habit of driving posture for each driver.

  The state detection unit 70 includes a frame-out state detection unit 71, a posture collapse state detection unit 72, a direction collapse state detection unit 73, a shaking state detection unit 74, and a white-eye state detection unit 75.

  The frame-out state detection means 71 determines that the vehicle 10 is out of frame while the vehicle 10 is traveling, and detects that the driver is in an inoperable state when the vehicle is out of frame. Specifically, the frame-out state detection unit 71 detects that the driver is inoperable when the head of the driver detected by the head detection unit 61 is out of the image range FA. Here, the range FA is a predetermined range in an image captured by the driver camera 21. During normal driving, the driver's head does not deviate from the range FA. The range FA may be the entire captured image.

  When the driver is driving the vehicle 10 normally, as shown in FIG. 6, the driver's head often falls within the image range FA even if the driver takes an action. On the other hand, when sudden illness develops and the driver's consciousness is lost, as shown in FIG. 5, the head of the driver may be out of the range FA. Therefore, the frame-out state detection means 71 detects that the driver is in an inoperable state when the head of the driver is out of the image range FA.

  At this time, the frame-out state detection unit 71 can improve the accuracy of detecting the inoperable state of the driver in consideration of the trajectory acquired by the trajectory acquisition unit 62 before the head is out of the range FA. By using the trajectory of the head when the driver's head could not be detected within the range FA due to image ambiguity and when the driver's head moved and could not be detected within the range FA Since the determination can be made, the detection accuracy of the inoperable state of the driver is improved.

  In addition, when the driver temporarily moves the head, or when the head is not detected due to unclearness of the image, the head can often be detected again near the final position of the trajectory. Therefore, when the head is no longer detected by the head detection unit 61, the head detection unit 61 searches for the vicinity of the final position of the trajectory acquired by the trajectory acquisition unit 62. In this way, even when the driver's head is not detected, the head detection can be efficiently performed again using the head trajectory.

  The posture collapse state detection means 72 determines whether the driver is out of posture while the vehicle 10 is traveling, and detects that the driver is incapable of driving when the posture is out of alignment. Specifically, the posture collapse state detection unit 72 detects that the driver is incapable of driving when the head inclination θ detected by the inclination detection unit 64 is larger than a threshold Th1 (relative inclination threshold). .

  Normally, the body of the driver is restrained by the seat 11 and the seat belt 12 of the driver's seat, so that the body is relatively difficult to move even if the driver's consciousness is lost. On the other hand, since the driver's head is often not restrained, it is necessary to maintain the position of the head with the driver's intention. Therefore, when sudden illness develops and the driver's consciousness disappears, the driver cannot maintain the position of the head, and as shown in FIG. 7, the head often tilts greatly in either direction with respect to the trunk. .

  On the other hand, when the driver looks aside while traveling, the driver generally looks with the neck rotated, so that the inclination of the head with respect to the trunk is often small as shown in FIG. In addition, when the driver takes a position away from the driver's seat, the driver generally consciously tilts the torso, so that the inclination θ of the head with respect to the torso is small as shown in FIG. There are many. Therefore, the posture collapse state detection means 72 detects that the driver is incapable of driving when the head inclination θ is larger than the threshold Th1. At this time, if the posture collapsed state detection means 72 further detects that the driver is in an inoperable state on the condition that the driver's face is not facing the front of the vehicle 10, erroneous detection of the inoperable state can be suppressed. .

  The direction change state detection means 73 determines that the driver's face is broken while the vehicle 10 is traveling, and detects that the driver is incapable of driving when the face is broken. Specifically, the direction change state detection unit 73 detects that the face direction with respect to the front of the vehicle 10 detected by the face direction detection unit 65 exceeds the threshold Th2 (face direction threshold) beyond the time T2 (direction change determination time). When it is larger, it is detected that the driver is in an inoperable state.

  In general, when a sudden illness occurs, the driver cannot maintain the face orientation, and the face orientation with respect to the front of the vehicle 10 remains broken as shown in FIG. On the other hand, when the driver looks aside while driving, the driver generally returns immediately even if the face is changed. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  Alternatively, the direction change state detection unit 73 may detect that the driver is driving when the face direction with respect to the front of the vehicle 10 detected by the face direction detection unit 65 is larger than the threshold Th2 and the driver releases the handle 15. It detects that it is impossible. Whether the driver has released the handle 15 may be detected from an image, or may be detected by a pressure sensor or the like installed on the handle 15.

  In general, when a sudden illness develops, the driver cannot maintain the orientation of the face, the orientation of the face with respect to the front of the vehicle 10 collapses as shown in FIG. 10, and the driver often releases the handle 15. On the other hand, when the driver looks aside while traveling, as shown in FIG. 11, the driver generally changes the direction of the face while holding the handle 15. In addition, when the driver looks aside, the head may not be tilted but the neck may be turned to change the face direction. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  Alternatively, the direction change state detection unit 73 is incapable of driving when the face direction detected by the face direction detection unit 65 is larger than the threshold value Th2 and the accelerator opening is larger than a predetermined opening. Detect that.

  In general, when a driver looks aside while traveling, the accelerator is often not stepped on for safety. Therefore, when the face direction with respect to the front of the vehicle 10 is larger than the threshold Th2 and the accelerator opening is larger than the predetermined opening, the face direction is not broken due to looking aside, but the face direction is broken due to sudden illness. Probability is high. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  Alternatively, the direction change state detection unit 73 has been subjected to the accelerator operation and the brake operation for a time longer than the time Th3 (operation determination time) when the face direction detected by the face direction detection unit 65 is greater than the threshold Th2. If not, it is detected that the driver is in an inoperable state.

  In general, when a sudden illness develops, the orientation of the face with respect to the front of the vehicle 10 is lost, and the accelerator operation and the brake operation are not performed for a time longer than the time T3. On the other hand, when the driver looks aside while traveling, the driver generally changes the face direction and frequently performs an accelerator operation or a brake operation within time T3. Therefore, the direction change state detection means 73 detects that the driver is in an inoperable state in the above case.

  The swing state detection means 74 determines the swing state of the driver's head accompanying the external force while the vehicle 10 is traveling, and the driver is incapable of driving when the head is shaking differently than usual. Detect that. Specifically, the swing state detection means 74 has the amplitude Am1 of the head shake detected by the head detection means 61 from the time when the external force is applied to the vehicle 10 until the time T5 (swing determination time) elapses. When it is smaller than (first amplitude) or larger than amplitude Am2 (second amplitude), it is detected that the driver is in an inoperable state. The amplitude Am2 is larger than the amplitude Am1.

  As shown in FIG. 12, when an external force is applied to the vehicle 10, vibration is transmitted to the driver after a predetermined time difference. Normally, when there is a driver's consciousness, when an external force (specifically, an external force in the vertical direction) is applied to the vehicle 10, the head of the driver has an amplitude within the range of amplitude Am1 to amplitude Am2, as shown in FIG. Shake. On the other hand, when the driver develops a sudden illness and the body is stiff, the amplitude of the head shake is smaller than that in the normal state. In addition, when the driver develops a sudden illness and the body is relaxed, the amplitude of the shaking of the head becomes larger than that in the normal state. Therefore, the shaking state detection means 74 detects the inoperable state of the driver in the case described above.

  Time T5 is the time from when an external force is applied to the vehicle 10 until the driver's movement becomes unrelated to the external force. The amplitude Am1 and the amplitude Am2 are functions of time, and FIG. 13 shows an example thereof. In the determination of the shaking state, the minimum value of the amplitude Am1 and the maximum value of the amplitude Am2 from when an external force is applied until the time T5 elapses may be simply used as threshold values.

  The white-eye state detection unit 75 determines white eyes while the vehicle 10 is traveling, and detects that the driver is incapable of driving when the white-eye detection unit 66 detects a state where white eyes are peeled off. Specifically, the white-eye state detection unit 75 detects that the driver is incapable of driving when the white-eye degree calculated by the white-eye degree calculation unit 68 is larger than a threshold Th3 (white-eye threshold).

  Normally, when the driver can drive, the driver does not peel off his white eyes. On the other hand, when the driver develops a sudden illness, the driver may have white eyes as shown in FIG. Therefore, the white-eye state detection means 75 detects that the driver is in an inoperable state when a state in which white eyes have been peeled is detected.

  Each threshold value and each determination value used by each state detection unit are stored in the storage device 52 (storage unit). Further, the storage device 52 stores the head inclination θ, the face orientation, and the head shaking amplitude learned by the learning means 51. In the storage device 52, personal information including the medical history and age of the driver is registered. When there are a plurality of drivers, personal information of each driver is registered. Further, the storage device 52 is registered with the posture of the driver that is not determined as being incapable of driving and the posture of the driver that is determined as being incapable of driving. The posture of the driver who is not determined to be incapable of driving is, for example, a normal driving posture or a posture improved during driving. The posture of the driver that is determined as being incapable of driving is, for example, the posture that a driver who has illness makes during an attack. The driver captures the posture desired to be registered in the driver's seat in advance with the driver camera 21 and registers it in the storage device 52.

  Next, a processing procedure for detecting an inoperable state of the driver will be described with reference to a flowchart of FIG. This processing procedure is executed by the control device 50.

  First, it is determined whether the vehicle speed of the vehicle 10 is V or less (S10). V may be 0 km / h (stop) or a sufficiently low speed (for example, 1 km / h) to be regarded as a stop. When the vehicle speed is equal to or lower than V (S10: YES), the determination of S10 is repeatedly executed until it is determined that the vehicle speed is higher than V. When the vehicle speed is higher than V (S10: N0), it is determined that the vehicle is traveling, and the detection process of the inoperable state of the driver is started, and frame out determination is first performed.

  Here, in the determination of S10, it is also determined whether or not the driver is performing a driving operation. When the driving operation is being performed, the determination of S10 is repeatedly performed so that the driving operation is performed while traveling. If not, the detection process of the inoperable state of the driver may be started. For example, it is determined whether the vehicle speed of the vehicle 10 is V2 (for example, 50 km / h) or higher, the steering angle detected by the steering angle sensor 32 is higher than a predetermined angle, or the steering angular velocity is higher than a predetermined angular velocity. V2 is a value that can be considered that the driver is operating the accelerator pedal, a predetermined angle, and a predetermined angular velocity are values that can be considered that the driver is operating the steering wheel. If at least one of the above three conditions is satisfied, it is determined that the driver is performing a driving operation, and the driver's inoperable state detection process is not started.

  In determination of frame-out, first, the driver's head and torso are detected from the driver's seat image (S11). At this time, the facial feature point included in the head of the driver is detected to authenticate the driver. Driver authentication may be performed in advance by communication with a mobile terminal such as a smartphone, or may be performed by communication with a key of the vehicle 10 in which personal information is registered.

  Next, it is determined whether or not the head position has been detected (S12). If the head position cannot be detected (S12: NO), the head trajectory information is acquired from the head position recorded in the process of S19 described later, and it is determined whether the trajectory information indicates frame out. (S13). That is, it is determined whether the non-detection of the head position is non-detection due to the head being out of the imaging range or non-detection due to unclearness of the image.

  If the trajectory information of the head does not indicate frame out (S13: NO), it is determined as non-detection due to temporary obscuration of the image and the process returns to S10. On the other hand, when the trajectory information of the head indicates frame out (S13: YES), it is determined that the head has gone out of the imaging range. In this case, information on the seat belt sensor 22 and the seating surface sensor 23, which are used auxiliary to confirm that the head is outside the imaging range, is acquired (S14).

  Next, it is determined whether or not the frame is in a frame-out state continuously for a time T0 (frame-out determination time) (S15). Specifically, the head position is out of the range FA for more than time T0, and the pull-out amount of the seat belt 12 exceeds the first pull-out amount than the pull-out amount detected when the seat belt 12 is worn. In addition, it is determined whether or not the high pressure portion is biased toward the end of the seat 11a in the pressure distribution of the seat 11a. Then, when the above three conditions are satisfied, it is determined that the frame is out of time T0. At this time, the pull-out amount per amount detection time detected by the seat belt sensor 22, that is, the pull-out speed of the seat belt 12 may be set to be larger than the second pull-out amount.

  Information on the seat belt sensor 22 and the seat surface sensor 23 may not be used. In other words, the process of S14 is not executed, and the process of S15 may continue to determine whether or not the head position is out of the range FA continuously for time T0 or more.

  Here, the time T0 is set based on the personal information registered in the storage device 52. For example, the time T0 is shortened for an older person than for a younger person. In addition, a person with a specific medical history has a shorter time T0 than a person without a specific medical history. Furthermore, the time T0 is changed according to the state of the driver and the driving environment. In a driving environment where there is an indication that the driver is unable to drive, where the driver is likely to be unable to drive, or where there is a high risk of a collision when the driver is disabled T0 is shortened to make it easier to detect an inoperable state of the driver.

  Specifically, when the head position recorded in the process of S19 is oscillating with an amplitude larger than a predetermined amplitude, that is, when the head is swaying, there is a high probability of being inoperable. The time T0 is shortened. Also, the higher the moving speed of the head, the higher the possibility of posture collapse due to sudden illness rather than posture collapse when taking a thing. Therefore, in the acquired head trajectory information, the time T0 is shortened as the moving speed of the head position increases. In the case of posture collapse due to a sudden illness, the moving speed of the head often increases as the head approaches the end of the range FA. Therefore, when the moving speed of the head increases as the head approaches the end of the range FA, there is a high possibility that the posture is not collapsed due to a sudden illness but is not collapsed when taking a thing. Therefore, the time T0 is shortened when the moving speed of the head increases as the recorded head position approaches the end of the range FA. In addition, when convulsions are detected, the probability of becoming inoperable is high, so the time T0 is shortened.

  Moreover, in order to avoid a collision, it is necessary to start appropriate vehicle control earlier as the vehicle speed increases, so the time T0 is shortened as the vehicle speed of the vehicle 10 increases. In addition, in order to avoid a collision, it is necessary to start appropriate vehicle control earlier as the TTC (collision margin time) obtained by dividing the inter-vehicle distance from the preceding vehicle by the relative speed with the preceding vehicle is shorter. The shorter the time T0 is, the shorter it is. Further, when driving support control such as ACC (Adaptive Cruise Control) or LKA (Lane Keep Assist) is executed in the vehicle 10, the driver may break the posture for a long time, so the time T 0 is extended. Furthermore, the time T0 may be shortened on days of the week and times of day when statistically appearing sudden attacks such as heart attacks are likely to occur.

  If the frame is not continuously out for time T0 or more (S15: NO), the process proceeds to S21. If the frame is continuously out of time T0 or more (S15: YES), it is detected that the driver is in an inoperable state, and the driver is confirmed to be in an inoperable state. Specifically, the detection of the inoperable state is notified by voice from the speaker 82, display on the display 81, blinking of an indicator (not shown), and the like, and it is determined whether or not there is a response from the driver within a predetermined time ( S16).

  If a driver touches the touch display, the driver's voice, the operation of the vehicle 10 such as the steering wheel 15 or the brake, or the operation of a specific switch is detected within a predetermined time, a response is received from the driver. Determine (S16: NO). If none of the above is detected, it is determined that there is no response from the driver (S16: YES).

  When it is determined that there is a response from the driver, the driver recognizes that driving by the driver is possible by a voice from the speaker 82 or a display on the display 81 (S17). On the other hand, when it is determined that there is no response from the driver, an instruction is issued to the vehicle control device 90 so that appropriate braking and steering are performed and the vehicle is safely stopped. In addition, in order to notify the surrounding vehicles of the danger, an instruction is given to the vehicle control device 90 to light the headlight and listen to the horn (S18). Further, the danger is also notified to other passengers of the vehicle 10.

  Next, when it is determined that the head position has been detected in the process of S12 (S12: YES), the positions of the head and the torso are recorded (S19). The trajectory information of the head can be acquired from the position of the head recorded in each image.

  Subsequently, it is determined whether or not the head position is out of a preset range FA (S20). If the head position is out of the range FA even if it is within the imaging range (S20: YES), the process proceeds to S15 and frame out is determined.

  Subsequently, based on the positional relationship between the head and the torso, it is determined whether or not the posture of the driver is a posture determined to be an inoperable state registered in advance in the storage device 52 (S21). If the posture of the driver is determined to be an inoperable state (S21: YES), it is detected that the driver is in an inoperable state, and the process proceeds to S16.

  When the posture of the driver is not the posture determined to be the inoperable state (S21: NO), it is determined whether or not the driver posture is the posture not determined as the inoperable state registered in the storage device 52 (S22). If the posture of the driver is not determined to be an inoperable state (S22: YES), the process returns to S10. When the posture of the driver is different from the posture that is not determined as being incapable of driving (S22: NO), the posture collapse is next determined.

  First, it is determined whether or not posture collapse has been detected (S23). Specifically, the posture collapse is detected by the processing of the subroutine of FIG. First, the head tilt and the head tilt direction are calculated (S231). Subsequently, the inclination of the body part and the direction of the inclination of the body part are calculated (S232). Subsequently, an angle formed by the calculated inclination of the trunk and the inclination of the head, that is, the inclination θ of the head with respect to the trunk is calculated (S233). The calculated head inclination θ is learned when an inoperable state of the driver is not detected. Then, when the calculated head inclination θ is larger than the threshold Th1 (relative inclination threshold), posture collapse is detected (S234). Then, when posture collapse is not detected (S23: NO), the process proceeds to the determination of collapse of the face direction in S25.

  When posture collapse is detected (S23: YES), it is determined whether or not the head inclination θ is larger than the threshold Th1 and the face is not facing the front of the vehicle 10 (S24).

  At this time, simply, it may be continued only for the time T1 or more, and the condition may be only that the head inclination θ is larger than the threshold Th1 or that the face does not face the front of the vehicle 10. In addition, at this time, the calculated head inclination direction and body inclination direction do not change continuously for the time T1 or more, that is, the head position and the body position are within the range UA (non-moving determination). Range). The range UA is a range in which it can be considered that the head and the body are not moving. Alternatively, the calculated head inclination θ may be a condition that is greater than the learned head inclination beyond the determination value D1 (inclination determination value). Alternatively, it may be a condition that the handle 15 has not been operated for a time longer than the time T3 (operation determination time).

  Here, similarly to time T0, time T1 is set based on personal information registered in storage device 52, and is changed according to the vehicle speed, TTC, and whether or not driving support control is executed. The threshold value Th1 is reduced when convulsions are detected. Further, similarly to the time T0, the time T1 may be shortened on a day of the week or a time zone in which a sudden illness such as a heart attack that is statistically likely occurs.

  If the head inclination θ is greater than the threshold Th1 and the face does not face the front of the vehicle 10 after the time T1 (S24: YES), it is detected that the driver is unable to drive, S16 Proceed to the confirmation process. If the head inclination θ is not greater than the threshold Th1 for a time T1 or longer, or if the face is facing the front of the vehicle (S24: NO), then the determination of face collapse is performed.

  First, it is determined whether or not a face orientation collapse has been detected (S25). Specifically, the direction of the driver's face relative to the front of the vehicle is detected. Then, when the detected face orientation is larger than the threshold Th2 (face orientation threshold), the collapse of the face orientation is detected. When the collapse of the face direction is not detected (S25: NO), the process proceeds to the determination of the shaking state in S28. The detected face orientation is learned when the driver's inoperability is not detected.

  When the collapse of the face direction is detected (S25: YES), it is determined whether or not the face direction is larger than the threshold value Th2 continuously for the time T2 (the collapse direction determination time) (S26).

  Here, similarly to time T0, time T2 is set based on personal information registered in storage device 52, and is changed according to vehicle speed, TTC, and whether or not driving support control is executed. Further, similarly to the time T0, the time T2 may be shortened on a day of the week or a time zone on which a sudden illness such as a heart attack that is statistically likely occurs. The threshold value Th2 is reduced when convulsions are detected.

  If the face orientation is greater than the threshold value Th2 for more than the time T2 (S26: YES), it is detected that the driver is in an inoperable state, and the process proceeds to the confirmation process of S16.

  If the face orientation is not greater than the threshold Th2 for a time T2 or longer (S26: NO), then the driver has released the handle 15 for a time T3 or longer, or the accelerator opening is predetermined open. It is determined whether or not there is an accelerator operation and a brake operation (S27). If at least one of the three conditions in S27 is satisfied (S27: YES), it is detected that the driver is in an inoperable state, and the process proceeds to S16. When none of the three conditions in the process of S27 is satisfied (S27: NO), the determination of the shaking state is performed next.

  Here, in the determination of S26, it is also possible to make it a condition that at least one of the three conditions in the process of S27 is satisfied. Further, the determination in S26 and S27 may be made on the condition that the detected face orientation is larger than the learned face orientation by exceeding the determination value D1 (inclination determination value). In general, since the driver's hand remains above the driver's neck, the driver will not be disabled, so the driver's hand may be below the driver's neck. .

  Next, in the determination of the shaking state, it is determined whether or not the head wobbling that is different from the normal is detected with respect to the external force (S28). Specifically, the amplitude of the head shake is smaller than the amplitude Am1 (first amplitude) or the amplitude Am2 (second amplitude) from the time when the external force is applied to the vehicle 10 until the time T5 (swing determination time) elapses. ) Is determined.

  At this time, until the time T5 elapses after the external force is applied to the vehicle 10, the head vibrates with an amplitude different from normal, and after the time T5 elapses, the position of the head is within the range UA. This may be a condition. In other words, the head may be oscillated according to the external force, and the head position may not be changed after the influence of the external force is eliminated. Further, when it is not detected that the driver is incapable of driving, the head swing amplitude is learned, and the detected head swing amplitude is larger than the learned head swing amplitude. It is good also as a condition that it exceeds the judgment value D2 (amplitude judgment value). Moreover, it is good also as a condition that the handle | steering-wheel 15 is not operated continuously for time T3 or more.

  When the head wobbling that is different from the normal force is detected with respect to the external force (S28: YES), it is detected that the driver is in an inoperable state, and the process proceeds to S16. When the head wobbling that is different from the normal is not detected with respect to the external force (S28: NO), the white-eye state is determined next.

  First, it is determined whether or not a state with white eyes is detected (S29). Specifically, when the calculated white eye degree is larger than the threshold value Th3 (white eye threshold value), it is determined that a state where white eyes have been peeled is detected. Here, the degree of white eyes of both eyes of the driver is calculated, and it is determined that the state of having white eyes is detected on the condition that the degree of white eyes of both eyes is greater than the threshold Th3. However, when only one eye is detected, or when a state where the white of the eye is removed is simply detected, the detection of the state where the white of the eye is removed may be determined based on the degree of the white of the one eye.

  If it is determined that the state where the white of the eyes has been peeled is not detected (S29: NO), the driver determines that the frame out determination, the posture collapse determination, the face direction collapse determination, the shaking state determination, and the white eye state determination are all performed. Since it is not detected that the vehicle is in an inoperable state, the process returns to S10.

  If it is determined that a state with white eyes is detected (S29: YES), it is determined whether or not the degree of white eyes is greater than the threshold Th3 by continuing for a time T4 (white eye determination time) (S30). At this time, it may be a condition that the handle 15 is not operated continuously for time T3 or more.

  Here, similarly to the time T0, the time T4 is set based on the personal information registered in the storage device 52, and is changed according to the vehicle speed and TTC. The threshold value Th3 is reduced when convulsions are detected. Further, similarly to the time T0, the time T4 may be shortened on a day of the week or a time zone in which a sudden illness such as a heart attack that is statistically likely occurs.

  When the white eye degree is not greater than the threshold Th3 continuously for the time T4 or longer (S30: NO), the process returns to S10. When the white eye degree continues for a time T4 or more and is greater than the threshold value Th3 (S30: YES), the driver's inoperable state is detected, and the process proceeds to S16. This process is complete | finished above.

  Based on the detected head inclination θ with respect to the torso, the detected face orientation, and the detected head position, the degree of driver's posture collapse is displayed on the display 81 as shown in FIG. The posture collapse level increases as the detected head inclination θ increases. In addition, the posture collapse level is increased as the detected face orientation increases. Further, the posture collapse level is increased as the detected head position is further away from the standard position during driving. The standard position during driving is the position of the head when the vehicle 10 is started, or the average position of the head when it is not detected that the driver is unable to drive.

  According to this embodiment described above, the following effects are obtained.

  -When the driver's head is out of the range FA, the driver's inoperable state can be easily detected by detecting that the driver is in an inoperable state. Furthermore, when the trajectory until the head is out of the range FA is taken into account, it is possible to improve the accuracy of detecting the inoperable state of the driver.

  When the driver takes a position away from the driver's seat, the head often returns to the range FA even if the head is temporarily away from the range FA. Therefore, by making the condition that the driver's head is out of the range FA beyond the time T0, erroneous detection of the inoperable state of the driver can be suppressed.

  If the range FA is the entire image, it is detected that the driver is inoperable when the driver's head is not present in the image, so that the detection process can be simplified.

  -When the driver's posture collapses due to a sudden illness, it is considered that the pull-out amount of the seat belt 12 exceeds the first pull-out amount than the pull-out amount at the time of wearing. In addition, even when the driver's head is not detected, the position of the driver's head is out of the imaging range when the pull-out amount of the seat belt 12 is greater than the pull-out amount when the seat belt 12 is worn. It can be seen that Therefore, when the pull-out amount of the seat belt 12 is larger than the pull-out amount when the seat belt 12 is worn, the inoperable state of the driver can be detected with high accuracy.

  ・ When sudden illness develops, the driver's posture often collapses more rapidly than when taking things. Therefore, when a sudden illness develops, the seat belt 12 is often pulled out more rapidly than when taking things. Therefore, the erroneous detection of the inoperable state of the driver can be suppressed on condition that the amount of withdrawal of the seat belt 12 per amount detection time is larger than the second amount of withdrawal.

  When the driver's posture collapses due to a sudden illness, the high pressure portion in the pressure distribution of the seat portion 11a of the driver's seat is considered to be biased toward the end portion of the seat portion 11a. Even when the driver's head is not detected, the driver's head is out of the imaging range when the high-pressure portion of the pressure distribution of the driver's seat 11a is biased toward the end of the seat. It can be seen that it exists at the position. Therefore, when the high pressure portion in the pressure distribution of the seat portion 11a is biased toward the end portion of the seat portion 11a, it is possible to detect the inoperable state of the driver with high accuracy.

  When the head vibration greater than the predetermined amplitude is detected, that is, when the head is shaking, the time required for determining the driver's inoperable state can be shortened by reducing the time T0. As a result, the execution of the vehicle control when the driver is unable to drive can be quickly started.

  -The time required for determining the inoperable state of the driver can be shortened by shortening the time T0 as the moving speed of the head increases.

  -When the head moves closer to the end of the range FA, the time required for determining the inoperable state can be reduced by shortening the time required for determining the inoperable state when the moving speed of the head increases. .

  -When the inclination θ of the head with respect to the torso is larger than the threshold Th1, there is a high possibility that the posture is not collapsed due to sudden illness rather than the posture collapse when taking things. Therefore, when the inclination θ of the head with respect to the body portion is larger than the threshold value Th1, it is possible to detect the driver's inoperable state with high accuracy by detecting that the driver is in an inoperable state.

  -If the driver is conscious of the action of tilting his / her head with respect to the body part while driving, it is considered that the driver keeps his face facing the front of the vehicle 10 for safety. Therefore, by making the condition that the face is not facing the front of the vehicle 10, it is possible to suppress erroneous detection of the driver's inoperable state.

  ・ If the driver is conscious of the action of tilting the head with respect to the body while driving, the driver will immediately return the head position to the original position. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by making the condition that the head is greatly inclined with respect to the body part beyond the time T1.

  -Suppresses erroneous detection of driver inoperability by providing that the position of the head and torso does not change while the head is tilted greatly with respect to the torso over time T1. it can.

  -The condition that the driver tilts his / her head with respect to the torso is determined on condition that the detected head tilt θ is greater than the learned head tilt beyond the determination value D1. Even if it is held, it is possible to suppress erroneous detection of an inoperable state of the driver.

  -When the face direction with respect to the front of the vehicle 10 exceeds the threshold Th2 beyond the time T2, there is a high possibility that the face direction is not broken due to a side look or the like, but the face direction is broken due to a sudden illness. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  If the face direction relative to the front of the vehicle 10 is greater than the threshold Th2 and the driver has released the handle 15, the face direction may not be lost due to looking aside, but the face may be broken due to a sudden illness. high. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  -When the face direction with respect to the front of the vehicle 10 is larger than the threshold Th2 and the accelerator opening is larger than the predetermined opening degree, the face direction may not be broken due to a side look or the like, but the face direction may be broken due to a sudden illness. High nature. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  -If the face direction with respect to the front of the vehicle 10 is greater than the threshold Th2 and the accelerator or brake operation is not performed for a time longer than the time T3, the face direction is not collapsed due to looking aside but is caused by a sudden illness There is a high possibility of collapse. Therefore, by detecting that the driver is inoperable in the above case, the inoperable state of the driver can be detected with high accuracy.

  In general, if the driver is in an operable state, the steering wheel operation is performed within time T3. Therefore, it is possible to suppress erroneous detection of the inoperable state of the driver by assuming that the steering wheel is not operated for a time longer than time T3.

  In general, if the driver can drive, the accelerator is not depressed greatly for a time longer than time T3. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by setting that the accelerator opening is larger than the predetermined opening for a time longer than the time T3.

  -If the driver is conscious of changing the direction of the face of the vehicle 10 while traveling, the driver will immediately return the face to the front for safety reasons. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by providing that the face orientation with respect to the front of the vehicle 10 exceeds the threshold Th2 beyond the time T2.

  In general, when a driver develops a sudden illness, the driver's hand remains above the driver's neck and the driver is not disabled. Therefore, it is possible to suppress erroneous detection of the driver's inoperable state on condition that the driver's hand is below the driver's neck.

  By detecting that the driver is incapable of driving when the amplitude of the head shake is smaller than the amplitude Am1 or larger than the amplitude Am2 until the time T5 elapses after the external force is applied to the vehicle. The driver's inoperable state can be easily detected.

  In general, when the driver is not conscious, the head vibrates according to the external force, and the head does not move when the external force is no longer affected. On the other hand, when the driver vibrates the head with a scissors, the head vibrates regardless of the influence of external force. Therefore, it is possible to suppress erroneous detection of an inoperable state of the driver by assuming that the head vibrates according to the external force and the position of the head does not change after the influence of the external force is eliminated.

  Even when the driver has a habit of shaking his / her head, on condition that the detected amplitude of the head shake is larger than the learned amplitude of the head shake beyond the determination value D2. It is possible to suppress erroneous detection of an inoperable state of the driver.

  -When the state where the white of the eye is peeled is detected, it is possible to detect the inoperable state of the driver with high accuracy by detecting that the driver is in an inoperable state.

  The driver's eye contour and black eye area are detected based on the image. Then, based on the detected eye contour and black eye area, the white eye degree is calculated, and when the white eye degree is larger than the threshold value Th3, it is detected that the driver is in an inoperable state. Therefore, it is possible to detect with high accuracy the state in which the driver has white eyes, and thus to detect with high accuracy the inoperable state of the driver.

  -When the driver can drive, even if the degree of white eyes temporarily increases with the line of sight upward, the degree of white eyes does not remain high. Therefore, it is possible to suppress erroneous detection of the inoperable state of the driver by making it a condition that the white eye degree is greater than the threshold Th3 beyond the time T4.

  -By assuming that the degree of white eye on both sides is greater than the threshold Th3, even if the eye patch is worn on one eye, or one eye is a prosthetic eye, When the other eye does not have a white eye, the driver's inoperable state is not detected. Therefore, it is possible to suppress erroneous detection of the inoperable state of the driver.

  The ratio between the length Lw + Lb in the vertical direction of the eye and the length Lb in the vertical direction of the black eye area has a correlation with the ratio of the white eye area to the entire eye area, and thus the vertical length Lw + Lb of the eye and the black eye The white eye degree can be calculated from the ratio to the length Lb in the vertical direction of the region.

  -The smaller the distance Lb from the upper part of the eye contour to the lowermost part of the black eye region, the greater the degree of white eye. Therefore, the white eye degree can be calculated from the distance Lb from the upper part of the eye contour to the lowermost part of the black eye region.

  The white eye degree can be calculated from the ratio of the area of the white eye region to the area of the black eye region.

  -When the white of the eye is peeled off, the black eye area faces upward, so that the flatness of the black eye area is apparently increased. Therefore, the white eye degree can be calculated from the flatness ratio of the black eye region.

  -The larger the distance Lc from the vertical center line of the entire eye to the lowest part of the black eye region, the smaller the black eye region and the higher the white eye degree. Therefore, the white eye degree can be calculated from the distance Lc from the center line to the lowermost part of the black eye region.

  In general, when a driver has convulsions, there is a higher probability of being incapable of driving than when convulsions are not occurring. Therefore, when convulsions are detected, it is possible to easily detect the inoperable state of the driver by reducing the threshold values Th1 and Th2.

  -By shortening the times T0, T1, T2, and T4 as the vehicle speed increases, the time required for determining the driver's inoperable state decreases as the vehicle speed increases, and appropriate vehicle control execution can be started earlier.

  By shortening the times T0, T1, T2, and T4 as the TTC is shorter, the shorter the TTC is, the shorter the time required for determining the driver's inoperable state is, and the execution of appropriate vehicle control can be started earlier.

  -By setting the times T0, T1, T2, and T4 based on personal information including the driver's medical history and age, the time required to determine the inoperable state can be set according to the characteristics of the driver individual. .

  When the driving support control is being executed in the vehicle, it is possible to suppress erroneous detection of the inoperable state by extending the times T0, T1, and T2.

  -The driver can recognize his / her posture by notifying the driver of the degree of collapse of the driver's posture. For this reason, the driver can correct his / her posture so that even if the driving posture collapses, the driver is not detected as being incapable of driving. Thereby, the erroneous detection of a driving impossible state can be suppressed.

  ・ When a driver's inoperable state is detected, by confirming to the driver whether the driver is inoperable, if the driver's inoperable state is erroneously detected, execution of vehicle control to safely stop the vehicle can be avoided. .

(Other embodiments)
The driver camera 21 may be a part of the four cameras mounted in the vehicle interior. There may be at least one driver camera 21.

  The frame-out state detection unit 71 may detect that the driver is in an inoperable state based on the trajectory acquired by the trajectory acquisition unit 62. When a driver develops sudden illness and becomes unable to drive, the driver's head often moves from the driving position and does not return to the driving position. It is possible to detect an inoperable state.

  The direction-displacement state detection means 73 is further provided when the driver's face is facing downward larger than the threshold Th2d (downward threshold) or upward larger than the threshold Th2u (upward threshold). It may be detected that the driver is in an inoperable state. In general, when a sudden illness occurs and the driver's consciousness is lost, the driver's face often turns downward larger than the threshold Th2d or faces larger upward than the threshold Th2u. Therefore, it can be detected that the driver is in an unmanageable state when the driver's face is large downward or large upward.

  The shaking state detection means 74 detects that the head detected by the head detection means 61 exceeds the time T6 (return determination time) when an external force is applied to the vehicle 10 while the vehicle 10 is traveling. When the vehicle is tilted in the direction, it may be detected that the driver is in an inoperable state. Normally, when there is a driver's consciousness, when an external force (specifically, an external force in the left-right direction and the front-rear direction) is applied to the vehicle 10, the driver's head tilts in the direction of the external force, but returns to its original location within time T6. Return. On the other hand, if the driver develops a sudden illness and is not conscious, the driver's head is less resistant to the external force and remains tilted in the direction of the external force over time T6. Therefore, the shaking state detection means 74 can detect the inoperable state of the driver in the case described above.

  The white-eye state detecting means 75 further detects that the driver is in an inoperable state even when the opening of the mouth (specifically, the opening in the vertical direction) detected by the facial expression detecting means 67 is larger than the open determination amount. May be. When a driver develops a sudden illness and has white eyes, the mouth is often opened. Therefore, even when the opening of the driver's mouth is larger than the open determination amount, it may be detected that the driver is in an inoperable state.

  When the frame out determination, the posture collapse determination, the face collapse determination, the shaking state determination, and the white eye state determination are all performed, the detection accuracy of the inoperable state of the driver is the highest, but at least one determination may be performed. Moreover, you may perform combining any number of determinations. In that case, it is preferable to perform the priority in the order of frame-out determination, posture collapse determination, face direction collapse determination, shaking state determination, and white-eye state determination.

  For example, when combining the posture collapse determination and the shake state determination, if the posture failure determination does not detect that the driver is in an inoperable state, performing the shake state determination increases the driver's inoperability state. It can be detected with accuracy.

  In addition, when combining the face direction determination and the shaking state determination, if the driver does not detect that the driver is incapable of driving in the face direction determination, the driver cannot drive Can be detected with high accuracy.

  In addition, when combining the shaking state determination and the white-eye state determination, if the white-eye state determination is performed when the driver state is not detected in the shaking state determination, the driver's inoperability state is increased. It can be detected with accuracy.

  The learning unit 51 may learn the driver's posture when it is erroneously detected that the driver is in an inoperable state. That is, although it is detected that the driver is incapable of driving, the posture of the driver when there is a response from the driver may be learned. Then, the learned posture may be a posture that is not determined as an inoperable state.

  A statistical value of each threshold value and each determination value may be stored in the storage device 52 and used as an initial value. The statistical values of the threshold values and the determination values are statistics of the threshold values and the determination values corresponding to the drivers of the plurality of vehicles. Moreover, it is good to make it statistics in an information center by transmitting each threshold value and each determination value which are set corresponding to the driver from the vehicle 10 to an information center.

  -After confirming whether the driver is incapable of driving and receiving a response from the driver, the driver may recognize that the driver can drive for a certain period of time. Moreover, you may make it perform the process which detects the driving | operation impossible state of a driver at the time interval (for example, once per hour) which the driver set.

  The external force applied to the vehicle 10 may be detected by the seat sensor 23 other than the G sensor 44.

  DESCRIPTION OF SYMBOLS 10 ... Vehicle, 21 ... Driver camera, 61 ... Head detection means, 62 ... Trajectory acquisition means, 63 ... Convulsion detection means, 64 ... Inclination detection means, 65 ... Face direction detection means, 66 ... White eye detection means, 71 ... Frame Out state detection means, 72 ... posture collapse state detection means, 73 ... direction collapse state detection means, 74 ... shaking state detection means, 75 ... white eye state detection means.

Claims (7)

Head detection means (61) for detecting the head of the driver based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
During driving of the vehicle, when an external force is applied to the vehicle, the driver is driven when the head detected by the head detecting means is tilted beyond the return determination time (T6). A state detecting means (74) for detecting that the state is impossible,
It said state detecting means is longer time than operation determination time, on condition that the vehicle steering wheel (15) is not operated, inoperable for you detect driver that the driver is an operation disabled state condition Detection device. Facial expression detection means (67) for detecting the eye contour and black eye area of the driver based on the image;
White eye degree calculating means (68) for calculating the degree of white eyes of the driver based on the eye contour and the black eye area detected by the facial expression detecting means;
If the driver does not detect that the driver is in an inoperable state by the state detector, the driver cannot operate when the white eye degree calculated by the white eye degree calculator is greater than a white eye threshold (Th3). The inoperable state detection device for a driver according to claim 1, further comprising: a white-eye state detecting unit (75) for detecting that the state is a state. Head detection means (61) for detecting the head of the driver based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
During driving of the vehicle, when an external force is applied to the vehicle, the driver is driven when the head detected by the head detecting means is tilted beyond the return determination time (T6). A state detecting means (74) for detecting an impossible state;
Facial expression detection means (67) for detecting the eye contour and black eye area of the driver based on the image;
White eye degree calculating means (68) for calculating the degree of white eyes of the driver based on the eye contour and the black eye area detected by the facial expression detecting means;
If the driver does not detect that the driver is in an inoperable state by the state detector, the driver cannot operate when the white eye degree calculated by the white eye degree calculator is greater than a white eye threshold (Th3). and the white eye state detecting means (75) for detecting that a state, non-operational state detection device of the Ru driver which comprises a.   The posture informing means (80) for notifying the driver of the degree of collapse of the posture of the driver based on the position of the head detected by the head detecting means. Driver inoperability detection device. Head detection means (61) for detecting the head of the driver based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
During driving of the vehicle, when an external force is applied to the vehicle, the driver is driven when the head detected by the head detecting means is tilted beyond the return determination time (T6). A state detecting means (74) for detecting an impossible state;
On the basis of the position of the head detected by the head detection unit, a posture informing means for informing the collapse degree of orientation of the driver to the driver (80), non-operational state detecting device driver comprising a. The driver inoperability state detection device according to any one of claims 1 to 5 , further comprising confirmation means (80) for confirming whether the driver is inoperable when an inoperability state of the driver is detected. Head detection means (61) for detecting the head of the driver based on the image of the driver's seat imaged by the imaging device (21) mounted on the vehicle (10);
During driving of the vehicle, when an external force is applied to the vehicle, the driver is driven when the head detected by the head detecting means is tilted beyond the return determination time (T6). A state detecting means (74) for detecting an impossible state;
If the driver non-operational state has been detected, the operation disabled state detecting device driver comprising a confirmation unit (80) to see if inoperable to the driver.

Images