JP6312647B2 - Danger avoidance device and computer program - Google Patents

Description

  The present invention relates to a danger avoidance device that avoids danger caused by flash blindness of a vehicle driver.

  Conventionally, when a vehicle driver's pupil becomes smaller, a device that prevents halation from occurring in the image of the in-vehicle camera by reducing the amount of light received by the in-vehicle camera from the outside has been proposed ( Patent Document 1). Further, there has been proposed a system that acquires an image of a vehicle driver and gives a warning to the driver when it is determined that the driver is drowsy based on the acquired image (Patent Document 2).

Japanese Patent No. 4687573 JP 2014-5131001 A

  However, no danger avoidance measures have been taken by detecting that the driver is temporarily blinded by the headlights of the oncoming vehicle (referred to as “flash blindness”). Flash blindness refers to the temporary disappearance of vision due to the glare of light entering the eye. A strong and sharp light enters the eye, but the vision is lost because the pupil contracts despite the dark surroundings. The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a danger avoidance device that detects that the driver has become flash blind and performs control for avoiding danger.

The risk avoidance device according to the present invention includes a measuring unit that measures the size of the driver's pupil from a captured image including the driver's eyes, and a plurality of the driver's pupil sizes repeatedly measured by the measuring unit. And a calculation unit that calculates a change rate based on the calculated average value and the size of the driver's pupil, and when the calculated change rate is greater than a threshold value, the driver's pupil contracts. A determination unit having a contraction determination unit that determines that the driver's pupil is contracting, and the determination unit determines that the driver's pupil is contracting, for avoiding danger to the vehicle that the driver drives And an output unit that outputs a control signal.

In the present invention, when it is determined that the driver's pupil is contracted, a control signal for avoiding danger is output, so that danger can be avoided. In addition, since it is determined whether or not the pupil has contracted based on the change rate of the size of the pupil, it is possible to accurately capture the contraction of the pupil even if there are individual differences in the size of the eyes.

  In the danger avoidance device according to the present invention, the output unit includes a headlight control unit that outputs a control signal for performing a control for switching the headlight of the vehicle to a high beam or a control for blinking the headlight. Features.

  In the present invention, by controlling the headlights, the driver of the oncoming vehicle is notified of the possibility of danger, so that danger can be avoided.

  In the risk avoidance device according to the present invention, when the output unit continuously determines that the driver's pupil is contracted, the output unit counts the duration time; and And a switching signal output unit that outputs a control signal for switching the vehicle to the safety mode when the duration exceeds a predetermined length.

  In the present invention, when the driver's flash blind state continues for a predetermined time or more, the vehicle is switched to the safety mode, so that further danger can be avoided.

  In the risk avoidance device according to the present invention, the determination unit includes an oncoming vehicle determination unit that determines whether there is an oncoming vehicle, and the contraction determination unit determines that the driver's pupil is contracted. However, when the oncoming vehicle determining unit determines that there is no oncoming vehicle, it is determined that the driver's pupil is not contracted.

  In the present invention, it is possible to confirm the presence or absence of an oncoming vehicle from an image taken in front of the vehicle, and when the driver's pupil contraction is not the oncoming vehicle, it is possible to avoid unnecessary danger avoidance measures. Become.

The computer program according to the present invention measures the size of the driver's pupil from a photographed image including the driver's eyes , and calculates the average value of the measured plurality of the pupil sizes of the driver. calculating a rate of change based on the size of the pupil of the average value and the driver, the calculated change rate is, the larger than the threshold, it is determined that a pupil of the driver is contracted, the driver When it is determined that the pupil is contracted, the computer is caused to perform a process of outputting a control signal for avoiding danger to the vehicle driven by the driver.

In the present invention, when it is determined that the pupil of the driver is contracted, a control signal for avoiding danger is output, so that danger can be avoided. In addition, since it is determined whether or not the pupil has contracted based on the change rate of the size of the pupil, it is possible to accurately capture the contraction of the pupil even if there are individual differences in the size of the eyes.

  In the present invention, it is possible to avoid the danger when the driver becomes flash blind.

It is a block diagram which shows an example of a vehicle control system. It is a block diagram which shows the function structure of a danger avoidance apparatus. It is a block diagram which shows an example of the hardware constitutions of a danger avoidance apparatus. It is a flowchart which shows the procedure of the main process of a danger avoidance apparatus. It is a flowchart which shows the procedure of contraction determination processing. 10 is a flowchart illustrating a procedure of main processing in the second embodiment. 10 is a block diagram illustrating a functional configuration of a danger avoidance device according to Embodiment 3. FIG. 10 is a flowchart illustrating a procedure of shrinkage determination processing according to the third embodiment. It is a flowchart which shows the procedure of an oncoming vehicle determination process. 15 is a flowchart illustrating a procedure of main processing in the fourth embodiment.

Embodiment 1
Hereinafter, embodiments will be described with reference to the drawings. FIG. 1 is a block diagram illustrating an example of a vehicle control system. The vehicle control system includes a danger avoidance device 1, a DMS (Driver Monitoring System) 2, an automatic operation control device 3, and a headlight control device 4. The danger avoidance device 1, the DMS 2, the automatic operation control device 3, and the headlight control device 4 are connected to a network N such as a CAN (Controller Area Network) so that they can communicate with each other. The danger avoidance device 1 detects that the driver is flash blind and performs control to avoid the driver's danger. The DMS 2 includes a camera for photographing the driver. The DMS 2 detects looking aside and drowsiness from the direction of the driver's face and the movement of the eyelids. The DMS 2 warns the driver according to the detected situation. The automatic driving control device 3 detects, for example, the current position of the vehicle, the lane in which the vehicle travels, and the positions of other vehicles around the vehicle as needed, and a vehicle control ECU (Electronic Control Unit) (not shown) for safe operation of the vehicle. ) Automatically controls the vehicle such as steering, engine, and brake. The headlight control device 4 switches off the headlight, switches on and off, and switches between the high beam and the low beam at the time of lighting. The DMS 2, the automatic operation control device 3, and the headlight control device 4 can be configured by a known technique.

  FIG. 2 is a block diagram showing a functional configuration of the danger avoidance device 1. The danger avoidance device 1 includes a measurement unit 11a, a determination unit 11b, and an output unit 11e. The determination unit 11b includes a calculation unit 11c and a contraction determination unit 11d. The output unit 11e includes a headlight control unit 11f, a timer unit 11g, and a switching signal output unit 11h.

  The measurement unit 11a acquires a driver's face image from the DMS 2 and extracts an image of the driver's eyes by image processing. The size of the pupil is measured from the positional relationship between the extracted image and the DMS2 camera. The measurement unit 11a performs measurement every time a face image is acquired from the DMS2.

  The determination unit 11b determines whether or not the driver's pupil is contracted. The calculation unit 11c stores the pupil size measured by the measurement unit 11a as needed, and calculates an average value of the stored pupil sizes. Furthermore, the calculation unit 11c calculates the rate of change between the average value of the pupil size and the size of the most recent pupil. The contraction determination unit 11d determines that the driver's pupil is contracted when the size of the pupil is small at a change rate equal to or greater than the threshold. Otherwise, the contraction determination unit 11d determines that the driver's pupil is not contracted.

  When the determination unit 11b determines that the driver's pupil is contracted, the output unit 11e outputs a control signal for avoiding danger to the automatic driving control device 3 or the headlight control device 4. The headlight control unit 11f alerts the driver of the oncoming vehicle by controlling the headlight. Specifically, the headlight is switched from the low beam to the high beam, the headlight is blinked, and the like. Thereby, danger can be avoided. The control for avoiding the danger is not limited to this. Further, the output destination of the control signal is not limited to the automatic operation control device 3 or the headlight control device 4. The control signal may be output to an oncoming vehicle or a device outside the vehicle. For example, a warning message may be output to a speaker or display provided in the oncoming vehicle. When the determination unit 11b continuously determines that the driver's pupil is contracted, the time measuring unit 11g measures the duration. The switching signal output unit 11h outputs a switching signal indicating that the vehicle is switched to the safety mode to the automatic driving control device 3 when the time counted by the time measuring unit 11g exceeds a predetermined length. The switching to the safety mode is, for example, reducing the traveling speed or stopping the vehicle in an emergency parking zone beside the road. The predetermined length is appropriately determined by experiments or the like. The length is not a constant value and may be changed according to the speed of the vehicle. This is because the danger is greater as the speed of the vehicle is faster, and the danger is smaller as the speed is slower.

  The threshold used for the determination by the contraction determination unit 11d is obtained as follows. The change in pupil diameter can be calculated based on a physiological nonlinear differential equation. The calculation can represent the change in pupil diameter as a function of the intensity of light reaching the eye.

Here, D is the diameter of the pupil expressed in millimeters, Φ (t−τ) is the intensity of light reaching the retina at time t, and can also be expressed as Φ = IA. That is, it is the pupil area A (mm ² ) times the brightness I (lumens / mm ² ) of the light reaching the eyes. τ is the pupil delay time, which is the delay time due to nerve transmission between the moment when the light pulse reaches the retina and the initial reaction of the iris, and is the delay of nerve-muscle excitation and activation. dM, dD and dt are the differential coefficients of the function M, the pupil diameter D and the time t.

  Based on the above formula, the intensity of light that causes flash blindness and the size of the pupil are obtained by experiments and the like, and the change rate of the pupil serving as a threshold is determined.

  FIG. 3 is a block diagram illustrating an example of a hardware configuration of the danger avoidance apparatus 1. The danger avoidance device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a mass storage device 14, and a communication unit 15. Each component is connected by a bus B.

  The CPU 11 controls each part of the hardware according to the control program 1P stored in the ROM 12, and realizes each function part shown in FIG. The RAM 13 is, for example, SRAM (Static RAM), DRAM (Dynamic RAM), or flash memory. The RAM 13 temporarily stores data generated when the CPU 11 executes the program. The mass storage device 14 is, for example, a hard disk, an SSD (Solid State Drive), or the like. The mass storage device 14 stores a measurement result of the size of the driver's pupil as a measurement result table 141. Further, the control program 1P may be stored in the mass storage device 14.

  The pupil size stored in the measurement result table 141 may be stored in association with the measured time. Then, past items may be deleted for a predetermined time or longer. This is because it is possible to prevent determination errors due to changes in the surrounding environment. For example, when driving on a highway with relatively many road lights, and when driving on a rural village road where there are few road lights, the size of the driver's pupil in normal conditions Because it will be different.

  Next, information processing performed in the danger avoidance device 1 will be described. FIG. 4 is a flowchart showing a procedure of main processing of the danger avoidance apparatus 1. CPU11 of the danger avoidance apparatus 1 performs shrinkage | contraction determination (step S1). The CPU 11 determines whether or not the contraction determination result is contracted (step S2). When the CPU 11 determines that the result of the contraction determination is contracted (YES in step S2), the CPU 11 outputs a control signal (step S3) and ends the process. If the CPU 11 determines that the result of the contraction determination is not contracted (NO in step S2), the process ends. The CPU 11 repeats the main process.

  FIG. 5 is a flowchart showing the procedure of the shrinkage determination process. CPU11 acquires a driver's face picture from DMS2 (Step S11). The CPU 11 extracts the pupil image from the face image acquired by the image processing, and measures the size of the driver's pupil from the extracted image and the shooting distance (step S12). CPU11 calculates the average value of the size of the pupil memorize | stored in the measurement result table 141 (step S13). The CPU 11 calculates the rate of change from the size of the most recent pupil measured in step S12 and the calculated average value (step S14). The CPU 11 determines whether or not the rate of change is negative and the absolute value is greater than or equal to a threshold value (step S15). That is, the CPU 11 determines whether or not the average value has rapidly decreased. When the CPU 11 determines that the change rate is negative and the absolute value is equal to or greater than the threshold (YES in step S15), the CPU 11 sets the determination to be contracted (step S16), and ends the process. When the change rate is 0 or more or the absolute value is less than the threshold value (NO in step S15), the CPU 11 stores the result measured in step S12 in the measurement result table 141 (step S17). The CPU 11 sets the determination as not contracted (step S18) and ends the process.

  The first embodiment has the following effects. Since it is determined that the flash blindness is caused by the contraction of the driver's pupil, the determination can be made with high accuracy. Since the determination is made based on the contraction rate of the size of the pupil, the determination can be made with high accuracy regardless of the characteristics of the driver. When it is determined that the driver's pupil contracts and is in a flash blind state, a danger can be avoided by outputting a control signal.

Embodiment 2
The second embodiment relates to a mode of outputting a switching signal for switching the vehicle to the safety mode when flash blindness continues for a predetermined time or more. Since the hardware configuration and the like of the danger avoidance device 1 are the same as those in the first embodiment, the following description will mainly focus on differences from the first embodiment.

  FIG. 6 is a flowchart showing a procedure of main processing in the second embodiment. CPU11 of the danger avoidance apparatus 1 performs shrinkage | contraction determination (step S21). The CPU 11 determines whether or not the result of the contraction determination is contracted (step S22). When the CPU 11 determines that the determination result is contracted (YES in step S22), the CPU 11 outputs a control signal (step S23). The CPU 11 starts measuring time (step S24). The CPU 11 determines whether or not the time (duration) during which it is determined that it is contracting is greater than a predetermined length (step S25). If the CPU 11 determines that the duration is longer (larger) than the predetermined length (YES in step S25), it outputs a switching signal (step S26) and ends the process. If the CPU 11 determines that the duration is less than or equal to the predetermined length (NO in step S25), the process ends. When the CPU 11 determines that the determination result is not contracted (NO in step S22), the CPU 11 determines whether the time is being measured (step S27). If the CPU 11 determines that the time is being measured (YES in step S27), the time is stopped (step S28), and the process is terminated. If the CPU 11 determines that the time is not being measured (NO in step S27), the process ends.

  Embodiment 2 has the following effects. When the driver's pupil contraction continues, a switching signal is output and the mode is switched to the safety mode, so that danger can be avoided.

Embodiment 3
The third embodiment relates to a mode for improving the detection accuracy of the flash blindness of the driver. FIG. 7 is a block diagram showing a functional configuration of the danger avoidance apparatus 1 according to the third embodiment. The same functional blocks as those of the first embodiment are denoted by the same reference numerals and description thereof is omitted. The determination unit 11b of the danger avoidance device 1 includes an oncoming vehicle determination unit 11i. The oncoming vehicle determination unit 11i determines whether there is an oncoming vehicle. Even if the determination unit 11b determines that the driver's pupil is contracted, even if the contraction determination unit 11d determines that the oncoming vehicle determination unit 11i determines that there is no oncoming vehicle, the driver's pupil contracts. The determination result indicating that it has not been output is output to the output unit 11e. This is because even if the driver's pupil contracts, flash blindness due to headlights does not occur when there is no oncoming vehicle. Therefore, the danger avoidance is regarded as unnecessary and the output of the control signal and the switching signal is suppressed.

  FIG. 8 is a flowchart showing the procedure of shrinkage determination processing in the third embodiment. Steps S31 to S35 are the same as steps S11 to S15 of the first embodiment (FIG. 5), and thus description thereof is omitted. When the CPU 11 determines that the rate of change is negative and the absolute value is greater than or equal to the threshold (YES in step S35), the CPU 11 performs oncoming vehicle determination (step S36). The CPU 11 determines whether the oncoming vehicle is present or not (step S37). If the CPU 11 determines that the oncoming vehicle is present (YES in step S37), the CPU 11 sets the determination to be contracted (step S38) and ends the process. When the rate of change is 0 or more or the absolute value is less than the threshold (NO in step S35), the CPU 11 stores the result measured in step S32 in the measurement result table 141 (step S39). The CPU 11 sets the determination as not contracted (step S40) and ends the process. When the CPU 11 determines that the oncoming vehicle determination result is that there is no oncoming vehicle (NO in step S37), the CPU 11 sets the determination as not contracted (step S40) and ends the process.

  FIG. 9 is a flowchart showing the procedure of the oncoming vehicle determination process. CPU11 of the danger avoidance apparatus 1 acquires the image which image | photographed the vehicle front (step S41). CPU11 extracts a light source from the acquired image by image processing (step S42). The CPU 11 determines whether the light source is a headlight (step S43). The CPU 11 determines whether or not the determination result is a headlight (step S44). If the determination result is a headlight (YES in step S44), the CPU 11 determines that there is an oncoming vehicle (step S45). When the determination result is not a headlight (NO in step S44), the CPU 11 determines that determination = no oncoming vehicle (step S46). The CPU 11 ends the process. The value set for determination is passed as the return value of the caller.

  In step S42, a headlight, a tail lamp, and a vehicle width lamp are extracted as light sources by template matching. In step S43, it is determined whether or not the headlight is based on the extracted color of the light source. If the color of the light source is within a predetermined range close to white, the headlight of the oncoming vehicle is determined. If the color is within a predetermined range close to red, the headlight is determined to be the tail lamp of the preceding vehicle. The light source extraction processing and the determination processing for determining whether or not the headlight is described here is an example, and other known techniques may be used. Moreover, although the presence or absence of the oncoming vehicle was determined using the image which reflected the front of the vehicle, it is not restricted to it. Other known techniques may be used. For example, it is possible to install a millimeter wave radar device in a vehicle and determine the presence or absence of an oncoming vehicle from the size and speed of an object detected by the radar device.

  Embodiment 3 has the following effects. Since the occurrence of flash blindness is determined based on the contraction of the driver's pupil and the presence or absence of an oncoming vehicle, detection accuracy is improved.

Embodiment 4
The fourth embodiment is a modification of the third embodiment. The fourth embodiment relates to a mode in which the oncoming vehicle determination is not included in the contraction determination. Although the functional configuration of the danger avoidance device 1 is substantially the same as that of the third embodiment (FIG. 7), the oncoming vehicle determination unit 11i is not included in the determination unit 11b, and is the same as the measurement unit 11a, the determination unit 11b, and the output unit 11e. It is an independent configuration.

  FIG. 10 is a flowchart showing the procedure of the main process in the fourth embodiment. CPU11 of the danger avoidance apparatus 1 performs shrinkage | contraction determination (step S51). The CPU 11 determines whether or not the contraction determination result is contracted (step S52). When the CPU 11 determines that the determination result is contracted (YES in step S52), the CPU 11 performs oncoming vehicle determination (step S53). The CPU 11 determines whether the oncoming vehicle is present or not (step S54). When the CPU 11 determines that the oncoming vehicle is present (YES in step S54), the CPU 11 outputs a control signal (step S55) and ends the process. When the CPU 11 determines that the contraction determination result is not contracted (NO in step S52) or when the oncoming vehicle determination result determines that there is no oncoming vehicle (NO in step S54), the process ends. The shrinkage determination process in step S51 is the same as that in the first embodiment (FIG. 5), and the oncoming vehicle determination process in step S53 is the same as that in the third embodiment (FIG. 9).

  Embodiment 4 has the following effects. Since the occurrence of flash blindness is determined based on the contraction of the driver's pupil and the presence or absence of an oncoming vehicle, detection accuracy is improved.

The technical features (components) described in each embodiment can be combined with each other, and new technical features can be formed by combining them.
The embodiments disclosed herein are illustrative in all respects and should not be considered as restrictive. The scope of the present invention is defined not by the above-mentioned meaning but by the scope of the claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

1 Danger avoidance device 11 CPU
11a Measurement unit 11b Determination unit 11c Calculation unit 11d Contraction determination unit 11e Output unit 11f Headlight control unit 11g Timing unit 11h Switching signal output unit 11i Oncoming vehicle determination unit 12 ROM
13 RAM
14 Mass storage device 141 Measurement result table 15 Communication unit 1P Control program 3 Automatic operation control device 4 Headlight control device

Claims (5)

A measurement unit that measures the size of the driver's pupil from a captured image including the driver's eyes;
An average value of the plurality of driver pupil sizes repeatedly measured by the measurement unit, a calculation unit that calculates a change rate based on the calculated average value and the pupil size of the driver, and the calculated change rate Is a determination unit having a contraction determination unit that determines that the driver's pupil is contracted when greater than the threshold ,
An output unit that outputs a control signal for avoiding danger to a vehicle driven by the driver when the determining unit determines that the pupil of the driver is contracting; To avoid danger. The output unit, according to claim 1, characterized in that it comprises a headlight control unit for outputting a switching control, or a control signal for causing the control for flashing the headlights to the high beam headlights of the vehicle Danger avoidance device. The output unit is
When the determination unit continues to determine that the driver's pupil is contracting, a time measuring unit for measuring the duration,
When the continuation time exceeds a predetermined length, the risk of claim 1 or claim 2, characterized in that it has a switching signal output unit for outputting a control signal for switching the vehicle to a safe mode Avoidance device. The determination unit
It has an oncoming vehicle determination unit that determines whether there is an oncoming vehicle,
Even if the contraction determination unit determines that the driver's pupil is contracted, the driver's pupil is not contracted when the oncoming vehicle determination unit determines that there is no oncoming vehicle. The danger avoidance device according to any one of claims 1 to 3 , wherein the risk avoidance device is determined. Measure the size of the driver's pupil from the captured image including the driver's eyes,
Calculate the rate of change based on the measured average value of the driver's pupil size, and the calculated average value and the driver's pupil size,
The calculated change rate is, the larger than the threshold, it is determined that a pupil of the driver is deflated,
A computer program for causing a computer to perform a process of outputting a control signal for avoiding danger to a vehicle driven by the driver when it is determined that the driver's pupil is contracted.

Images