JP5411653B2 - Sleepiness determination device - Google Patents

Description

  The present invention relates to a drowsiness determination device that detects drowsiness of a driver.

  In recent years, various comprehensive safety systems have been proposed and put into practical use for widely handling from before the occurrence of a collision (crash) to after the occurrence of a collision. For example, a system has been proposed in which a driver monitor that monitors the driver's condition detects the driver's face direction and eye open / closed state, and issues a warning or activates a brake to avoid a collision. For example, Japanese Patent Application Laid-Open No. 2008-65776 (Patent Document 1) discloses a technique of a dozing detection device that detects opening and closing of eyes and detects dozing when the continuous eye closing time is equal to or longer than the first determination time. Yes. This device measures the continuous eye opening time and the continuous eye closing time so that the dozing state can be accurately determined even when the open / closed state of the driver's eyes changes only for a short time. When the eyes are opened from the closed eye, when the continuous eye opening time from the eye closing becomes equal to or longer than the first determination time, the continuous eye closing time before the eye opening is reset. When the eyes are closed again within a time shorter than the first determination time after the eyes are closed, the continuous eyes closing time is measured after the eyes closed before the eyes are opened. As a result, even if the driver's eyes are in an open state due to a momentary blink of the driver or the influence of light, dozing is accurately detected.

JP 2008-65776 A (4th to 8th paragraphs, etc.)

  In Patent Document 1, the degree of eye opening is determined based on a change in the proportion of the size of the eyeball, and the determination of dozing is performed according to the proportion of time in the open / closed state. However, there are large individual differences in the proportion of eyeball size and the proportion of time in the open / closed state. Also, if the driver is fighting sleepiness, the accuracy may be reduced. Another technique is to detect the driver's line of sight and determine the degree of arousal, but for accurate line-of-sight detection, the driver must wear dedicated glasses, and a simple system using image processing is required. It is difficult to build.

  Accordingly, there is a need for a technique that can absorb individual differences and accurately detect driver drowsiness with a simple system using image processing.

The characteristic configuration of the drowsiness determination device according to the present invention created in view of the above problems is as follows.
An image acquisition unit for acquiring a captured image including a driver's face imaged at a predetermined interval;
A face detector that detects wrinkles of the driver based on the captured image;
A blink information calculation unit that calculates blink information indicating a time-series change in the opening / closing state of the driver's heel,
From the blink information, a fractal dimension calculation unit that calculates a fractal dimension that represents the characteristics of changes in the blink information;
A drowsiness determination unit that determines the degree of drowsiness of the driver based on a reference fractal dimension and a fractal dimension of the driver that are acquired according to the degree of drowsiness of a plurality of subjects and statistically set. It is in.

  The inventors have confirmed that the fractal dimension increases or decreases in response to sleepiness according to an experimental analysis based on an actual vehicle running test performed on a plurality of subjects. It was also confirmed that the fractal dimension value was less affected by individual differences among subjects. Therefore, the degree of drowsiness of the driver is satisfactorily determined based on the statistically set reference fractal dimension and the fractal dimension of the driver. As described above, it has also been confirmed that the degree of opening of the driver's eyes and the ratio of the opening and closing time vary greatly between individuals. Therefore, in the drowsiness determination device described in Patent Document 1 and the like, the determination criterion needs to be set according to each driver. This criterion is calculated in the drowsiness determination device after the driver starts driving. That is, drowsiness cannot be determined until the calculation of the criterion is completed, and the calculation load increases because the criterion is calculated. In addition, the accuracy of the judgment criteria may vary from driver to driver. On the other hand, according to this feature configuration, it is possible to perform sleepiness determination with high accuracy based on the unified reference fractal dimension, and the calculation load is small. In other words, a drowsiness determination device that absorbs individual differences and accurately detects driver drowsiness is provided by a simple system using image processing.

Further, the fractal dimension calculation unit of the drowsiness determination device according to the present invention,
A cross data generation unit that extracts the time at which the blink information that changes in time series intersects a reference value set in a range of the blink information as impulse-like cross data;
A cross data smoothing unit that smoothes the cross data and converts it into smooth cross data that is continuous data;
An autocorrelation function computing unit for deriving a power spectrum of the smooth cross data and computing an autocorrelation function indicating a correlation value as a function of a time shift amount from the power spectrum;
When the time shift amount and the correlation value are expressed by logarithm, the fractal dimension is calculated based on the slope of the linear component of the change in the real part of the autocorrelation function in a predetermined range on the side where the time shift amount is small. It is preferable to include a self-similar dimension calculation unit.

  The cross data can be generated by a general-purpose comparison operation, and the smooth cross data can be generated using a low-pass filter. The power spectrum can be calculated by fast Fourier transform, and the autocorrelation function can be calculated by inverse fast Fourier transform. Fast Fourier transform, inverse fast Fourier transform, and logarithmic transform processing are also provided in a computation subset including a microcomputer program, a computation module of a logical operation processor such as a DSP (digital signal processor), and the like, and has high practicality. In addition, since the calculation of the slope of the linear component is a linear calculation, the calculation load is light. Thus, since the fractal dimension is calculated through highly practical calculation, the drowsiness determination apparatus is configured by a simple system.

  In the sleepiness determination apparatus according to the present invention, it is preferable that a plurality of the reference fractal dimensions are set, and the sleepiness determination unit determines the degree of sleepiness of the driver according to each reference fractal dimension.

  As described above, experimental analysis by the inventors has confirmed that the fractal dimension increases or decreases in response to sleepiness. Therefore, if the reference fractal dimension is set in a plurality of stages according to the strength of sleepiness, the sleepiness determination unit can determine the degree of sleepiness as well as whether or not the driver is sleepy. As a result, an appropriate operation is selected when a warning or alarm operation is executed in response to the sleepiness determination result.

  Here, it is preferable that the reference fractal dimension is set to a higher value as drowsiness is higher. Experimental analysis by the inventors has confirmed that the higher the drowsiness, the greater the fractal dimension. Therefore, when the reference fractal dimension is set to a higher value as drowsiness is higher, the degree of drowsiness of the driver is determined better.

Block diagram schematically showing the system configuration of the vehicle Block diagram schematically showing the configuration of the drowsiness determination device Diagram showing the relationship between the driver and camera Explanatory drawing showing the concept of the face detection method Explanatory drawing which shows the concept which calculates the opening degree of eyes Graph showing an example of blink information The graph which shows an example of the cross data corresponding to the blink information of FIG. Graph showing the relationship between blink information, cross data, and smooth cross data Graph showing the relationship between smooth cross data and autocorrelation function The figure which shows an example of the calculation method of the logarithmized autocorrelation function slope Flow chart showing a series of processes for sleepiness detection

  Hereinafter, the case where the embodiment of the drowsiness determination device of the present invention is applied to a safety system that improves the safety of an occupant on a vehicle will be described as an example. FIG. 1 is a block diagram schematically showing the system configuration of a vehicle equipped with such a safety system. Safety systems include a collision safety system (passive safety system) that reduces damage when a collision occurs, a preventive safety system (active safety system) that reduces damage by predicting a collision in advance, There is a comprehensive system that fuses both. 1 includes a collision determination system 50 that uses a millimeter wave radar 51 and a camera 52 to determine the possibility of a collision between another object such as another vehicle and the host vehicle, and includes a comprehensive safety system. The system configuration which functions as is illustrated. The collision determination system 50 informs the driver of the possibility of a collision, executes a collision avoidance operation such as the operation of an emergency brake, reduces the impact at the time of collision by changing the seat posture or strengthening the tension of the seat belt. To get ready.

  The drowsiness determination device also functions as one system constituting an overall safety system. As a specific example, when driver drowsiness is detected by the driver monitor ECU 9, a warning is issued through the buzzer 7 and the speaker 20c of the monitor device 20 as a first stage. A lamp on the meter panel or the like may be blinked to visually notify the user. Further, if the driver's condition does not improve, the alarm brake is operated through the brake system 40 as a second step, and a sensible warning is given. In addition, various notifications can be performed via the collision determination system 50.

  The monitor device 20 is a display device of the navigation system 30. The monitor device 20 includes a display unit 20a that functions as a GUI (graphic user interface) of the navigation system 30, a touch panel 20b, and a speaker 20c for voice guidance. The navigation system 30 is a system that supports driving by performing route guidance and the like. The current position is specified by an unillustrated GPS device or an autonomous navigation device such as an unillustrated gyro, and the driving of the driver is supported by comparing it with a map stored in the system.

  The brake system 40 detects the amount of operation of a brake pedal operated by a driver by a brake sensor 42, and has an electric motor having a brake assist function for increasing the braking force by applying a braking force to the vehicle via an actuator 41. Brake system. Also included is an anti-lock braking system (ABS) that suppresses brake locking, an electronic stability control (ESC) that suppresses vehicle side-slip during cornering, and a BBW (brake-by-wire) system. The brake system 40 can also function as a safety system. For example, it can function as a brake assist that increases the hydraulic pressure of the brake regardless of the operation amount of the brake pedal operated by the driver, or an emergency brake or a warning brake that automatically applies the brake.

  As shown in FIGS. 1 and 2, the drowsiness determination device 10 of the present invention is configured as a driver monitor ECU (electronic control unit) 9. The ECU 9 is connected to various systems and sensors (not shown) in addition to the navigation system 30, the brake system 40, and the collision determination system 50 described above via a CAN (controller area network) 60 that is an in-vehicle network.

  As shown in FIG. 1, the driver monitor ECU 9 constituting the drowsiness determination device 10 has a central processing unit (CPU) 2 as a core, a program memory 3, a work memory 4, an image processing module 5, an audio processing module 6, and other non-existence. The illustrated peripheral circuit is configured. The CPU 2 executes various arithmetic processes using programs and parameters stored in the program memory 3. Further, the CPU 2 temporarily stores data or the like in the work memory 4 as necessary to execute the calculation. Here, an example in which the program memory 3 and the work memory 4 are memories different from the CPU 2 is shown, but they may be integrated in the same package as the CPU 2. Further, in this example, the CPU 2 is the core, but the ECU 9 may be configured with another logic operation processor or logic circuit such as a DSP (digital signal processor) as the core. Many general-purpose products equipped with arithmetic modules such as a low-pass filter (LPF) and Fourier transform are provided for the DSP, and high-speed arithmetic can be performed at low cost. Each function part which comprises the drowsiness determination apparatus 10 is implement | achieved by cooperation with hardware and software (program).

  As shown in FIG. 3, the camera 1 is installed on the cover of the steering column 91 and photographs the head of the driver 100 from the front through the gap of the steering wheel 92. The camera 1 uses an image sensor such as a charge coupled device (CCD) or a CMOS image sensor (CIS) to shoot a two-dimensional image of 10 to 30 frames per second in time series, and digitally converts it into moving image data (captured image). ). A two-dimensional image of each frame is stored in a frame memory included in the image processing module 5, and image processing can be performed for each frame in cooperation with the CPU 2. The camera 1 may be an infrared camera, and in this case, an infrared lamp or infrared strobe that projects the head (face) of the driver 100 may be provided. Further, it is sufficient that the camera 1 is installed so that the head of the driver 100 can be photographed from the front side. The camera 1 may be installed in the meter panel 93 or in a rearview mirror, not on the cover of the steering column 91. .

  The image acquisition unit 11 of the drowsiness determination device 10 is a functional unit that acquires frame data (captured image) including the face of the driver 100 captured by the camera 1 at a predetermined interval of 1/10 second to 1/30 second. . That is, the image acquisition unit 11 acquires a captured image at a sampling rate of 10 Hz to 30 Hz. The face detection unit 12 is a functional unit that detects the face of the driver 100 using frame data. The face detection unit 12 includes a contour detection unit 12a, a face part detection unit 12b, a center calculation unit 12c, and a face direction calculation unit 12d.

  Since the frame data includes not only the face of the driver 100 but also the background, the face detection processing is executed by the contour detection unit 12a. Specifically, the contour is detected using a known edge detection technique using the difference in contrast with the background, and the face width E is detected as shown in FIG. You may utilize the difference of the picked-up image acquired in time series. Next, the face part detection unit 12b detects the position of the face part, specifically the eyes, nose, mouth, ears, and the like. Also in this case, a known edge detection technique or the like can be used. Naturally, characteristics such as a filter and an operator used for edge detection are set according to each functional unit.

  When the face parts are detected and the positions of the respective face parts are detected, the center calculation unit 12c calculates the center line C of the face using the fact that the face parts are arranged substantially symmetrically. Using the calculated center line as a reference, the left / right ratio of the face width E detected by the contour detection unit 12a and the face orientation degree based on the left / right ratio are calculated by the face direction calculation unit 12d. The aside look determination unit 19 determines that the driver 100 looks aside based on the degree of face orientation or is depressed due to falling asleep. Based on the determination result, the driver monitor ECU 9 issues a warning via the buzzer 7 or issues a warning message via the voice processing module 6.

  Next, sleepiness determination processing by the sleepiness determination device 10 will be described. When the eye is detected by the face part detection unit 12b, the opening degree is calculated by the blink information calculation unit 13 using the image of the eye region. The blink information calculation unit 13 extracts an image of a region including the eye (eye region) as shown in FIG. 5 based on the eye position detected by the face part detection unit 12b. Upper eyelids and lower eyelids, eyeballs and the like are detected by a known edge detection technique and the like, and the opening degree of the eye is calculated based on distances (opening distances) A and B between the lower side of the upper eyelids and the lower eyelids. In FIG. 5, the distance A is the maximum eye opening distance (maximum eye opening distance) of the driver 100, and the distance B is the eye opening distance (current distance) obtained each time based on the captured image. The opening degree H of the eye is obtained by “H = B / A”, where the maximum eye opening distance A is the denominator and the current distance B is the numerator. Note that the opening degree H is the resolution of the effective number of division results. According to the experiments by the inventors, it has been confirmed that drowsiness can be determined even with binary data of opening and closing, that is, opening degree H = 1 and opening degree H = 0.

  When the eye is most open, that is, when the current distance B between the upper eyelid and the lower eyelid is the maximum eye opening distance A, the opening degree H is “1”. When the eyes are closed, that is, when the upper eyelid and the lower eyelid are in contact with each other and the current distance B is “0”, the opening degree H is “0”. Here, when the maximum eye opening distance A is reset to the initial value at the start of driving and updated to the maximum value of the current distance B detected by the blink information calculation unit 13, a value corresponding to the same driver 100 is obtained. Is preferable. Generally, when driving is started, the driver 100 is often in an awake state, and the eyes are also considered to be firmly open. Therefore, the maximum eye opening distance A is set to an appropriate value soon after the start of operation. However, it is not necessary to be limited to this method. Even if the maximum eye opening distance A is not updated to the maximum value and the opening degree H may exceed “1”, the distance is finally normalized between “1” and “0”. Also good.

  The blink information calculation unit 13 obtains the opening degree H for each captured image (frame) acquired in time series. For example, when the camera 1 takes a two-dimensional image of 10 frames per second in time series and the image acquisition unit 11 acquires all frames, the opening degree H is obtained every 0.1 seconds. The time-series change in the opening degree H is converted into continuous data by a curved line or a curve approximated to an analog change by linear interpolation or the like. FIG. 6 shows an approximate curve by linear interpolation. FIG. 6 exaggerates the “blink” in which the eyes are greatly opened from the closed state and closed again. As shown in FIG. 6, the information indicating the time-series change in the opening degree H, that is, the change in the opening / closing state of the heel of the driver 100 is blink information W. That is, the blink information calculation unit 13 calculates blink information indicating a change in the opening / closing state of the driver's 100 eyelids.

  The fractal dimension calculation unit 14 is a functional unit that calculates a fractal dimension from the blink information W. The fractal dimension calculation unit 14 includes functional units of a cross data generation unit 14a, a cross data smoothing unit 14b, an autocorrelation function calculation unit 14c, a logarithmic conversion unit 14d, and a self-similar dimension calculation unit 14e. Is done. The cross data generation unit 14a is a functional unit that detects a point (time) at which the blink information W intersects a predetermined reference value TH set between [0, 1] that is a range of the blink information W. That is, the time at which the blink information W that changes with the passage of time intersects the reference value TH is specified, and the impulse-shaped cross data X is generated. As shown in FIG. 7, the cross data X is generated as impulse-like flag data with “1” at the time of intersection and “0” at the time of non-intersection. The reference value TH is preferably set to a value of about 0.2 to 0.5 in terms of the opening 20% to 50% and the blink information W range.

  Next, the cross data smoothing unit 14b multiplies the cross data X by a low-pass filter a predetermined number of times to generate smooth cross data S that is pseudo analog data. FIGS. 8A to 8C show examples of blink information W, cross data X, and smooth cross data S in a longer time than those in FIGS. As will be described later, the smooth cross data S is shifted by a predetermined slide width ST and cut out every predetermined analysis unit time AT. For example, when the analysis unit time AT is about 60 seconds, the slide width ST is about 7.5 seconds, and when the analysis unit time AT is about 6 to 10 seconds, the slide width ST is 0.75 to 1. It is suitable for about 2 seconds. Then, one fractal dimension is calculated in the extracted analysis unit time AT. In addition, it has been confirmed by the inventors' experiments that sleepiness can be determined even when the analysis unit time AT is as short as about 13.6 seconds.

  The autocorrelation function calculation unit 14c performs fast Fourier transform (FFT) processing on the smoothed cross data S in the extracted analysis unit time AT to generate FFT data FD. Next, the autocorrelation function calculation unit 14c takes the conjugate complex number FDcon of the FFT data FD and multiplies the conjugate complex number FDcon and the FFT data FD to derive the power spectrum P. Further, the autocorrelation function calculation unit 14c performs an inverse fast Fourier transform (inverse FFT) process on the power spectrum P to derive an autocorrelation function AF. FIG. 9A is an example of the same smooth cross data S as FIG. 8C, and FIG. 9B shows the real part of the autocorrelation function AF derived from the smooth cross data S. . The autocorrelation is a measure for measuring how well the smooth cross data S in the analysis unit time AT is matched with that before the shift when the data is shifted in time. The autocorrelation function AF (τ) represents the autocorrelation (correlation value) as a function of the time shift magnitude τ.

  The logarithmic conversion unit 14d expands the real part of the autocorrelation function AF shown in FIG. 9B on a logarithmic scale as shown in FIG. 9C. Then, the slope a of the autocorrelation function AF in a region where the time shift amount τ of the real part of the autocorrelation function AF plotted on the logarithmic scale is small is obtained. Preferably, the slope a of the autocorrelation function AF from τ0 where the time shift amount τ is zero to a predetermined time shift amount τ1 is obtained. As an example, when the resolution in the time axis direction in the analysis unit time AT is about 1000 points, the slope a is obtained using the autocorrelation function AF values of the first about 10 points. For example, as shown in FIG. 10, the slopes a1 to a9 of nine straight lines formed between 10 points τ0 to τ1 are obtained, and the average value thereof is calculated as the slope a of the autocorrelation function AF. The linear portion of the log-plotted graph, one or more decades, is a fractal (self-similarity). Here, the first 10 points are always regarded as straight lines. In calculating the slope a, the autocorrelation function AF at τ0 to τ1 is linearly approximated by a known Hough transform, least square method, RANSAC (random sample consensus), etc., and the slope of the approximated straight line is calculated. Also good.

The self-similar dimension calculation unit 14e calculates the self-similar dimension D using the slope a of the autocorrelation function AF. This self-similar dimension D is a fractal dimension. In the present embodiment, as shown in the following equation, the self-similar dimension D is calculated by subtracting the slope a from 1.
D = 1-a

  According to the results of experiments by the inventors for a plurality of subjects, it was found that the fractal dimension D increases when the subject's sleepiness is high. It was also found that the fractal dimension D is an index representing sleepiness with little influence on individual differences among subjects. Therefore, it is possible to determine the drowsiness of the driver 100 by absorbing individual differences by statistical information processing or the like and setting the reference fractal dimension Dref as a reference for drowsiness. The reference fractal dimension Dref is preset in the program memory 3 or the like. The sleepiness determination unit 15 determines the degree of sleepiness of the driver 100 based on the reference fractal dimension Dref and the fractal dimension D of the driver 100.

  The reference fractal dimension Dref can be set by statistically processing the degree of sleepiness determined based on the determination standard (face evaluation) by the determiner with respect to the facial expression of the subject, and the fractal dimension D of the subject at that time. . In addition, a sensor for acquiring biological information such as an α wave component of an electroencephalogram is attached to the subject, and the degree of sleepiness determined based on the detection result of the sensor and the fractal dimension D of the subject at that time are statistically processed. It can also be set.

  As described above, the drowsiness determination device 10 of the present embodiment can determine the drowsiness of the driver 100 without using information other than blink such as the driver's 100 expression, voice, driving behavior, line of sight, and biological information. is there. Naturally, it is not necessary for the driver 100 to wear a detection device for detecting the line of sight or biological information. Further, since complicated operations such as Lyapunov exponent derivation used in chaos analysis are not required, high-speed real-time processing can be realized with a relatively inexpensive configuration. Further, it is not necessary to perform a complicated operation such as a correlation integration method for deriving the fractal dimension. A general-purpose DSP or CPU can be derived using a comparison operation, filtering, Fourier transform, or the like modularized by hardware or software. Further, as described above, since the reference fractal dimension Dref is set in advance by collecting data for a plurality of subjects, a learning function is not required.

  Hereinafter, the procedure of sleepiness determination by the vehicle sleepiness determination device 10 while the driver 100 is driving the vehicle will be described using the flowchart of FIG. The processes of the image acquisition unit 11, the face detection unit 12, and the blink information generation unit 13 will be omitted as appropriate, and the description will focus on the processing of the fractal dimension calculation unit 14.

  As described above, the eye is detected by the face detection unit 12 based on the captured image captured by the camera 1 and acquired by the image acquisition unit 11, and the eye opening distance (the maximum eye opening distance A and the current distance B) is acquired ( # 1). Based on the maximum eye opening distance A and the current distance B, the blink information calculation unit 13 calculates the eye opening H (= B / A) normalized to the [0, 1] section (# 2). The blink information calculation unit 13 obtains the opening degree H for each captured image (frame) acquired in time series. The change in the time series opening degree H is converted into blink information W, which is continuous data, for example by linear interpolation (# 3). Next, the cross data generation unit 14a detects a point (time) at which the blink information W intersects a predetermined reference value TH set between [0, 1], which is the range of the blink information W, and detects the cross data. X is generated (# 4). Subsequently, the cross data smoothing unit 14b generates the smooth cross data S, which is pseudo analog data, by multiplying the cross data X by a predetermined number of low-pass filters (# 5). Processes # 1 to # 5 so far may be performed without synchronizing with subsequent processes. That is, when the smooth cross data S is accumulated in the work memory 4 and the smooth cross data S is sufficient, the process # 6 and subsequent steps may be repeated.

  When the smooth cross data S is accumulated, the autocorrelation function calculator 14c cuts out the smooth cross data S for a predetermined analysis unit time AT (# 6). Then, the autocorrelation function calculation unit 14c performs fast Fourier transform (FFT) processing on the smoothed cross data S in the extracted analysis unit time AT to generate FFT data FD (# 7). Next, the autocorrelation function calculator 14c takes the conjugate complex number FDcon of the FFT data FD (# 8), and multiplies the conjugate complex number FDcon and the FFT data FD to derive the power spectrum P (# 9). Further, the autocorrelation function calculation unit 14c performs an inverse fast Fourier transform (inverse FFT) process on the power spectrum P to derive an autocorrelation function AF (# 10).

  The logarithmic conversion unit 14d plots the real part of the autocorrelation function AF on the logarithmic scale (# 11). The logarithmic conversion unit 14d or the autosimilar dimension calculation unit 14e obtains the slope a of the autocorrelation function AF in the region where the time shift amount τ of the real part of the autocorrelation function AF plotted on the logarithmic scale is small (# 12). . Subsequently, the self-similar dimension calculation unit 14e calculates the fractal dimension D (= 1-a) using the gradient a of the autocorrelation function AF (# 13).

  The sleepiness determination unit 15 determines the degree of sleepiness of the driver 100 based on the reference fractal dimension Dref and the fractal dimension D of the driver 100 (# 14). For example, when the fractal dimension D of the driver 100 is greater than or equal to the reference fractal dimension Dref, the ECU 9 outputs a drowsiness detection signal (# 15). As a result, the buzzer 7 is sounded or a warning message is issued from the speaker 20c of the monitor device 20. Further, the determination in the process # 14 may be a determination of a plurality of stages instead of the two alternatives exemplified in FIG. That is, the degree of sleepiness of the driver 100 may be determined by a plurality of reference fractal dimensions Dref, and a warning or warning corresponding to the degree of sleepiness may be performed.

  Next, it is determined whether or not the smooth cross data S is accumulated including a predetermined slide width ST (# 16). If accumulated, the analysis unit time AT is shifted by the slide width ST. (# 17). Then, the smooth cross data S for the new analysis unit time AT is cut out, the fractal dimension D is obtained, and the sleepiness of the driver 100 is determined. When the smooth cross data S is not accumulated including the predetermined slide width ST (# 16: No branch), the process # 1 to # 5 is executed to generate the smooth cross data S.

  As described above, according to the present invention, there is provided a technique for detecting a driver's sleepiness with high accuracy by absorbing individual differences by a simple system using image processing. This technology can be applied to a comprehensive vehicle safety system that widely deals before and after a collision occurs.

10: Sleepiness determination device 11: Image acquisition unit 13: Blink information calculation unit 14: Fractal dimension calculation unit 14a: Cross data generation unit 14b: Cross data smoothing unit 14c: Autocorrelation function calculation unit 14e: Self-similarity dimension calculation unit 15: Drowsiness determination unit 100: Driver AF: Autocorrelation function D: Fractal dimension Dref: Reference fractal dimension P: Power spectrum S: Smooth cross data TH: Reference value set in the range of blink information W: Blink information X: Cross data a: slope of linear component of change of real part of autocorrelation function τ: time shift amount

Claims (4)

An image acquisition unit for acquiring a captured image including a driver's face imaged at a predetermined interval;
A face detector that detects wrinkles of the driver based on the captured image;
A blink information calculation unit that calculates blink information indicating a time-series change in the opening / closing state of the driver's heel,
From the blink information, a fractal dimension calculation unit that calculates a fractal dimension that represents the characteristics of changes in the blink information;
A drowsiness determination unit that determines the degree of drowsiness of the driver based on a reference fractal dimension and a fractal dimension of the driver acquired according to the degree of drowsiness of a plurality of subjects and statistically set;
A drowsiness determination device comprising: The fractal dimension calculator is
A cross data generation unit that extracts the time at which the blink information that changes in time series intersects a reference value set in a range of the blink information as impulse-like cross data;
A cross data smoothing unit that smoothes the cross data and converts it into smooth cross data that is continuous data;
An autocorrelation function computing unit for deriving a power spectrum of the smooth cross data and computing an autocorrelation function indicating a correlation value as a function of a time shift amount from the power spectrum;
When the time shift amount and the correlation value are expressed by logarithm, the fractal dimension is calculated based on the slope of the linear component of the change in the real part of the autocorrelation function in a predetermined range on the side where the time shift amount is small. The drowsiness determination device according to claim 1, further comprising a self-similar dimension calculation unit.   The drowsiness determination device according to claim 1, wherein a plurality of the reference fractal dimensions are set, and the drowsiness determination unit determines the degree of drowsiness of the driver according to each reference fractal dimension.   The sleepiness determination apparatus according to claim 3, wherein the reference fractal dimension is set to a higher value as sleepiness is higher.

Images