JP2019074965A - Device and system for detecting driving incapacity - Google Patents

Abstract

To provide a detection technique capable of reducing missed detection of driving incapacity of a driver.SOLUTION: A driving incapacity detection device comprises an acquisition unit, first detection unit, and second detection unit. The acquisition unit acquires an image captured by a camera configured to capture images of the interior of a host vehicle. The first detection unit detects postures of a driver and a passenger based on the captured image. The second detection unit detects driving incapacity of the driver based on a result of the detection by the first detection unit.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a detection technique for detecting an inoperable state of a driver.

  In recent years, the driver's health condition has suddenly changed, and there have been multiple accidents in which it is difficult to continue driving. In order to prevent such an accident, development of a device for automatically detecting a state in which the driver can not operate has been advanced.

  For example, the driver's inoperable state detection device proposed in Patent Document 1 detects the driver's head based on the image of the driver's seat captured by the driver camera, and the inclination of the head relative to the driver's trunk is relative When it is larger than the inclination threshold value, it is detected that the driver is inoperable.

JP, 2016-9255, A (summary, paragraph 0024)

  However, in some cases, the driver may become inoperable while the head has a small inclination relative to the driver's torso. In such a case, the driver's inoperable state detection device proposed in Patent Document 1 can not detect the driver's inoperable state.

  An object of this invention is to provide the detection technique which can reduce the detection leak of the inoperable state of a driver in view of the said subject.

  The inoperable state detection device according to the present invention includes an acquisition unit for acquiring a captured image obtained by a camera for capturing a vehicle interior of the host vehicle, an attitude of a driver and an attitude of an occupant other than the driver based on the captured image. The configuration includes a first detection unit to be detected and a second detection unit to detect an inoperable state of the driver based on a detection result of the first detection unit.

  In the inoperable state detection device according to the first configuration, the second detection unit determines whether the driver is inoperable based on the posture of the driver detected by the first detection unit. Based on the postures of the occupants other than the driver detected by the first detection unit when it is determined that the driver is not in an inoperable state by the first determination unit that determines And a second determination unit that determines whether the driver is inoperable, and the first determination unit determines that the driver is inoperable and the second In each of the cases where it is determined by the determination unit that the driver is inoperable, the driver may be configured to detect the inoperable state (second configuration).

  In the inoperable state detection device having the first or second configuration, the first detection unit executes the process of detecting the attitude of the driver and the process of detecting the attitudes of occupants other than the driver in parallel. Configuration (third configuration).

  In the inoperable state detection device according to any one of the first to third configurations, the first detection unit is configured to detect the attitude of the driver and the attitude of the occupant in the front passenger seat of the vehicle (fourth Configuration).

  In the inoperable state detection device of the fourth configuration, the second detection unit is configured to detect the posture of the occupant in the front passenger seat of the host vehicle from the state in which the first detection unit detects the posture of the occupant. When the detection unit shifts to a state in which the posture of the occupant in the front passenger seat of the host vehicle can not be detected, the driver may be configured to detect the inoperable state (fifth configuration).

  The inoperable state detection / prediction system of the present invention has a configuration (sixth configuration) including a camera for photographing the interior of the host vehicle and the inoperable state detection device having any one of the first to fifth configurations. is there.

  According to the present invention, it is possible to reduce detection leaks of the driver's inoperable state.

Block diagram showing a configuration example of the operation impossible state detection device A diagram showing an example of an image captured by a vehicle interior shooting camera Flow chart showing an operation example of the operation impossible state detection device Flow chart showing another operation example of the operation impossible state detection device A diagram showing an example of an image captured by a vehicle interior shooting camera A diagram showing an example of an image captured by a vehicle interior shooting camera A diagram showing an example of an image showing the result of background subtraction processing The figure which shows an example of the image which shows the result of a head detection process

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing an example of the configuration of the inoperable state detection device. The inoperable state detection device 10 detects the inoperable state of the driver from the photographed image obtained by the in-vehicle photographing camera 1. The driver's inoperable state includes, for example, when the driver loses consciousness due to a sudden illness or when the driver is sleeping. The in-room photographing camera 1 and the inoperable state detection device 10 are mounted on a vehicle. The inoperable state detection system includes the in-vehicle photographing camera 1 and the inoperable state detection device 10. Hereinafter, a vehicle on which the inoperable state detection device 10 is mounted is referred to as a "own vehicle".

  The passenger compartment photographing camera 1 is a monocular camera for photographing the passenger compartment of a vehicle from one direction. For example, the in-room photographing camera 1 may be installed at a central front portion of a ceiling of the vehicle so that not only an occupant sitting in a driver's seat or a passenger's seat but also an occupant sitting in a rear seat can be photographed. An image captured by the in-room imaging camera 1 when the in-room imaging camera 1 is installed at the front center of the ceiling of the host vehicle is, for example, as shown in FIG.

  A near infrared camera is employed as the in-vehicle shooting camera 1 so that the in-vehicle shooting camera 1 can detect the inoperable state of the driver even at night, and the shooting area of the in-room shooting camera 1 is close It is preferable to install a near-infrared lighting device that emits infrared light in the interior of a vehicle.

  The non-operational state detection device 10 can be configured by hardware such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA), or a combination of hardware and software. When software is used to configure the inoperable state detection device 10, a block diagram for a part implemented by software represents a functional block diagram of the part. A function implemented using software may be described as a program, and the function may be implemented by executing the program on a program execution device. Examples of the program execution device include a computer provided with a central processing unit (CPU), a random access memory (RAM), and a read only memory (ROM).

  The inoperable state detection device 10 includes an acquisition unit 11, a first detection unit 12, and a second detection unit 13.

  The acquisition unit 11 acquires an analog or digital photographed image from the vehicle interior photographing camera 1 continuously in time with a predetermined cycle (for example, a 1/30 second cycle). That is, a group of photographed images acquired by the acquiring unit 11 is a moving image photographed by the in-vehicle photographing camera 1. Then, when the acquired captured image is analog, the acquisition unit 11 converts the analog captured image into a digital captured image (A / D conversion). The acquisition unit 11 outputs the acquired captured image or the acquired and converted captured image to the first detection unit 12. One captured image output from the acquisition unit 11 is one frame image.

  The first detection unit 12 detects the posture of the driver and the posture of the occupant in the front passenger seat of the host vehicle based on the captured image acquired by the acquisition unit 11.

  The second detection unit 13 detects an inoperable state of the driver. Specifically, the second detection unit 13 includes a first determination unit 13a and a second determination unit 13b. The first determination unit 13a determines, based on the posture of the driver detected by the first detection unit 12, whether or not the driver is inoperable. The second determination unit 13b is based on the posture of the occupant in the front passenger seat of the vehicle detected by the first detection unit 12 when the first determination unit 13a determines that the driver is not in an inoperable state. To determine whether the driver is inoperable. The second detection unit 13 determines that the first determination unit 13a determines that the driver is inoperable, and the second determination unit 13b determines that the driver is inoperable. Detects the driver's inoperable state.

  FIG. 3A is a flowchart showing an operation example of the inoperable state detection device 10. The inoperable state detection device 10 starts the operation of the flowchart shown in FIG. 3A in a state where power is supplied.

  First, the inoperable state detection device 10 determines whether the door of the vehicle has been switched from the locked state to the unlocked state when the engine is off (step S10). Therefore, from the outside of the inoperable state detection device 10, the inoperable state detection device 10 transmits a signal indicating that the door of the vehicle is in a locked state or an unlocked state and a signal informing whether the engine is on or off receive. When the host vehicle is an electric vehicle, a state where power is not supplied to the motor drive device that supplies power to the motor that rotates the drive wheels corresponds to engine off.

  When the door of the host vehicle is switched from the locked state to the unlocked state when the engine is off, the acquiring unit 11 acquires a background image shown in FIG. 4 (step S20). After acquiring the background image, the acquisition unit 11 continues acquiring the frame image at a predetermined cycle.

  Next, the first detection unit 12 determines a frame image to be processed (step S30).

  Next, the first detection unit 12 sets a first detection area for detecting the driver's head on the processing target image and a second detection area for detecting the occupant's head at the front passenger seat of the vehicle. (Step S40). FIG. 5 is an image in which the outer frame F1 of the first detection area and the outer frame F2 of the second detection area are set on the background image shown in FIG. Although the outer frame F1 of the first detection area and the outer frame F2 of the second detection area are rectangular in this embodiment, they may have other shapes. Further, although the second detection area is set in step S40 in the present embodiment, the second detection area may be set in step S90.

  Furthermore, in step S40, the first detection unit 12 performs background difference processing within the first detection area of the processing target image to try to extract a driver. Specifically, the first detection unit 12 compares the background image, which is a frame image, with the processing target image, which is a frame image and is acquired by the acquisition unit 11 after the background image, and performs processing based on the threshold value. The first detection area of the target image is divided into a foreground area and a background area, and driver extraction is attempted using the foreground area in the first detection area. This reduces the burden of background subtraction processing. Further, by removing vehicle movable parts such as a steering wheel, a seat belt retractor, a shift knob and the like from the first detection area, it is possible to prevent the vehicle movable parts from being erroneously distinguished as a foreground area by moving.

  The first detection unit 12 may generate, for example, an image in which the boundary between the foreground area and the background area is emphasized as a result of the background difference processing, and may output the image to an external display device. FIG. 6 is an example of an image generated as a result of background subtraction processing in which the boundary line B1 between the extracted driver and the background area is emphasized.

  Also, unlike the present embodiment, the first detection unit 12 may execute background subtraction processing on the entire area of the processing target image. Thereby, the background difference processing can be executed on the processing target image without omission.

  In step S50 following step S40, the first detection unit 12 determines whether the driver has been extracted. If the driver can not be extracted, the process returns to step S30.

  On the other hand, if the driver can be extracted, the first detection unit 12 detects the posture of the driver (step S60).

  Specifically, the first detection unit 12 executes the head detection process of the driver on the first detection area (area surrounded by the outer frame F1) of the processing target image, and the head detection process of the driver Detect the driver's attitude based on the result. Although the specific method of the head detection process is not particularly limited, it is possible to cite, for example, a method of executing machine learning by the AdaBoost algorithm using a head feature (HOG) feature of the head (Histograms of Oriented Gradients).

  Note that the first detection unit 12 may generate, for example, an image in which the detection portion of the head is emphasized as a result of the head detection process, and may output the image to an external display device. FIG. 7 is an example of an image generated as a result of the head detection process, including an image including a frame F3 indicating the detection position of the head of the driver.

  In step S70 following step S60, the first determination unit 13a determines whether the driver is inoperable based on the driver's posture (particularly, the driver's head position) detected by the first detection unit 12 Determine if If the first determination unit 13a determines that the driver is inoperable (YES in step S80), the second detection unit 13 detects the inoperable state of the driver and notifies other services (step S110), and then returns to step S20. Examples of other services include an execution unit that executes an automatic stop of the own vehicle, a driver, an execution unit that executes an anomaly notification to at least one of the passenger and the outside of the vehicle, and the like.

  If it is determined by the first determination unit 13a that the driver is not in an inoperable state (NO in step S80), the driver may be in an inoperable state with a normal posture, so the first detection may be performed. The section 12 detects the posture of the passenger in the front passenger seat of the host vehicle (step S90). First, the first detection unit 12 executes background subtraction processing in the second detection area of the processing target image to try to extract an occupant in the front passenger seat of the vehicle. Specifically, the first detection unit 12 compares the background image, which is a frame image, with the processing target image, which is a frame image and is acquired by the acquisition unit 11 after the background image, and performs processing based on the threshold value. The second detection area of the target image is divided into a foreground area and a background area, and driver extraction is attempted using the foreground area in the second detection area. Then, if the occupant in the front passenger seat of the own vehicle can be extracted, the first detection unit 12 detects the posture of the occupant in the front passenger seat of the own vehicle. Specifically, the first detection unit 12 executes head detection processing of the occupant in the front passenger seat of the own vehicle with respect to the second detection area (area surrounded by the outer frame F2) of the processing target image. Based on the result of head detection processing of the driver, the posture of the occupant in the front passenger seat of the host vehicle is detected.

  The second determination unit 13b determines whether or not the driver is inoperable by determining whether or not the posture of the occupant in the front passenger seat of the host vehicle detected by the first detection unit 12 is normal. Is determined (step S100). If the second determination unit 13b determines that the posture of the passenger in the front passenger seat of the host vehicle is not normal, the second determination unit 13b determines that the driver is inoperable and the second detection unit 13 Detects the driver's inoperable state and notifies other services (step S110), and then returns to step S30. On the other hand, when it is determined by the second determination unit 13b that the posture of the passenger in the front passenger seat of the vehicle is normal, the second determination unit 13b determines that the driver is not inoperable and the driver can not operate. It transfers to step S120, without a state being notified to another service.

  In step S120, the inoperable state detection device 10 determines whether the door of the host vehicle has been switched from the unlocked state to the locked state when the engine is off. If the event that the door of the host vehicle is switched from the unlocked state to the locked state does not occur when the engine is off, the process returns to step S30. On the other hand, when the door of the host vehicle is switched from the unlocked state to the locked state when the engine is off, the inoperable state detection device 10 ends the operation of the flowchart shown in FIG. 3A. Note that when power is supplied to the inoperable state detection device 10 after the inoperable state detection device 10 ends the operation of the flowchart shown in FIG. 3A, the inoperable state detection device 10 performs the operation of the Start.

  When the driver is inoperable, for example, the passenger in the passenger seat of the vehicle touches the driver, the passenger in the passenger seat of the vehicle faces the driver, or is in the passenger seat of the vehicle It is assumed that the occupant touches the traveling device such as the steering wheel. For this reason, when the driver is in the inoperable state, the posture of the occupant in the passenger seat of the own vehicle is likely to be not normal, that is, different from the case where the driver is not inoperable. . Therefore, by providing the processes of steps S90 and S100, it is possible to reduce the detection leak of the inoperable state of the driver.

  The inoperable state detection device 10 may perform the operation of the flowchart shown in FIG. 3B instead of performing the operation of the flowchart shown in FIG. 3A. In the flowchart shown in FIG. 3B, if the driver can not be extracted in the determination of step S50, the process returns to step S20, and in the determination of step S120, an event that the door of the own vehicle is switched from the unlocked state to the locked state If it does not occur, the point of returning to step S20 and the point of returning to step S20 after step S110 are different from the flowchart shown in FIG. 3A, and the other points are the same as the flowchart shown in FIG. 3A. In the flowchart shown in FIG. 3B, since the background image is updated by the loop in the flowchart, when the seat position or the position of the seat belt retractor is changed, the acquisition unit 11 can acquire the background image on which the change is reflected. .

  In the flow chart shown in FIG. 3A or the flow chart shown in FIG. 3B, the order of step S80 and steps S90 and S100 is interchanged to determine whether the driver can not drive at the posture of the occupant who is in the assistant's seat of the vehicle. If it is determined that the driver is not incapable of driving based on the posture of the occupant in the front passenger seat of the host vehicle, it may be determined whether or not the driver is inoperable based on the posture of the driver.

  Here, when the passenger in the front passenger seat of the host vehicle touches the traveling device such as the driver or the steering wheel, the head of the passenger in the front passenger seat of the host vehicle is surrounded by the second detection area (the outer frame F2 It may move to the outside of the 2nd detection field (field surrounded by outer frame F2) from the inside of field). In this case, the state in which the first detection unit 12 detects the posture of the occupant in the front passenger seat of the host vehicle shifts to a state in which the detection can not be performed. Therefore, in the flow chart shown in FIG. 3A or the flow chart shown in FIG. 3B, when the first detection unit 12 detects the posture of the occupant in the front passenger seat of the own vehicle in step S100 Alternatively, the second determination unit 13b may determine that the posture of the occupant in the front passenger seat of the host vehicle is not normal. As a result, it is possible to reduce detection leaks of the driver's inoperable state.

  Further, in the present embodiment, when it can be determined that the driver is inoperable by only the attitude of the driver, the driver can not operate without performing the determination based on the attitude of the occupant in the front passenger seat of the host vehicle. Is detected. Thus, when it can be determined that the driver is inoperable by only the attitude of the driver, it is possible to quickly detect that the driver is inoperable.

  Note that, unlike the present embodiment, it may be determined whether or not the driver is inoperable based on both the driver's attitude and the attitude of the occupant in the front passenger seat of the vehicle.

  Various technical features disclosed in the present specification can be modified in various ways without departing from the spirit of the technical creation in addition to the above embodiment. Also, the plurality of modifications shown in the present specification may be implemented in combination as far as possible.

  In the embodiment described above, although the first detection unit 12 detects the posture of the occupant in the front passenger seat of the own vehicle, the occupant in the front passenger seat of the own vehicle instead of the posture of the occupant in the front passenger seat of the own vehicle In addition to the attitude of the driver, the first detection unit 12 detects the attitude of the occupant in the back seat of the vehicle, and the second detection unit 13 also detects the attitude of the driver based on the attitude of the occupant in the back seat of the vehicle. An inoperable state may be detected. However, the posture of the occupant in the rear seat of the own vehicle may be difficult to detect because it is hidden behind the driver's seat or the assistant's seat of the own vehicle, so the posture of the occupant in the passenger seat of the own vehicle may be difficult to detect. It is desirable to at least detect

  In the embodiment described above, after detecting the posture of the driver in step S60, the posture of the occupant who is in the assistant's seat of the host vehicle is detected in step S90, for example, step S90 is abolished and the first detection unit 12 in step S60. The process of detecting the attitude of the driver and the process of detecting the attitude of the occupant in the front passenger seat of the vehicle may be performed in parallel. In this case, since the process of always detecting the driver's attitude on the same image and the process of detecting the attitude of the occupant in the assistant's seat of the vehicle are executed in parallel, the attitude of the occupant in the assistant's seat of the vehicle The image processing is performed as compared to the case where the process of detecting the posture of the occupant in the front passenger seat of the vehicle is performed for the first time when the result of the process of detecting the vehicle is required (when NO is determined in step S80). Procedure can be standardized (simplified).

  In the embodiment described above, for example, if the occupant in the front passenger seat of the host vehicle is sleeping with the normal posture, the driver's inoperable state can not be detected even if the processing in steps S90 and S100 is performed. Therefore, in order to improve the detection accuracy of the driver's inoperable state, for example, a gaze detection device may be provided which detects the gaze of the driver and the occupant of the passenger's seat of the host vehicle. If the line of sight of the driver is in the undetectable state because the line of sight of the driver is fixed or closed, and if the line of sight of the occupant in the front passenger seat of the vehicle is undetectable because the line is closed or closed, YES in step S100. Even if it is determined that the condition is determined to be, the process may move to step S110.

DESCRIPTION OF SYMBOLS 1 in-room imaging | photography camera 10 inoperable state detection apparatus 11 acquisition part 12 1st detection part 13 2nd detection part 13a 1st determination part 13b 2nd determination part

Claims (6)

An acquisition unit that acquires a captured image obtained by a camera that captures the interior of the host vehicle;
A first detection unit that detects an attitude of a driver and an attitude of an occupant other than the driver based on the photographed image;
A second detection unit that detects an inoperable state of the driver based on the detection result of the first detection unit;
An unoperable state detection device comprising: The second detection unit is
A first determination unit that determines whether or not the driver is inoperable based on the posture of the driver detected by the first detection unit;
When the first determination unit determines that the driver is not in an inoperable state, the driver is in an inoperable state based on the posture of the occupant other than the driver detected by the first detection unit. A second determination unit that determines whether or not
Equipped with
When the first determination unit determines that the driver is inoperable, and when the second determination unit determines that the driver is inoperable, the driver can not operate. The non-operational state detection device according to claim 1, which detects   The non-operational state detection according to claim 1 or 2, wherein the first detection unit executes the process of detecting the attitude of the driver and the process of detecting the attitudes of occupants other than the driver in parallel. apparatus.   The non-drivable state detection device according to any one of claims 1 to 3, wherein the first detection unit detects a posture of the driver and a posture of a passenger in a front passenger seat of the host vehicle.   The second detection unit detects that the first detection unit detects the posture of the occupant in the front passenger seat of the own vehicle, and the first detection unit detects the posture of the occupant in the front passenger seat of the own vehicle The inoperable state detection device according to claim 4, wherein the inoperable state of the driver is detected when shifting to a state in which the attitude can not be detected. A camera that captures the interior of the vehicle;
An inoperable state detection and prediction system comprising the inoperable state detection device according to any one of claims 1 to 5.