JP2009028239A - Drowsy driving detector - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a drowsy driving detector capable of highly accurately detecting the arousal level of a driver. <P>SOLUTION: An arousal level estimation circuit 209 estimates an arousal level of a driver based on a heart rate equivalent signal detected by an electrocardiographic signal detector 206, an eye closing rate calculated for every preset predetermined time by an eye closing rate calculation circuit 203 based on an opening/closing state of driver's eyes detected by an image processing circuit 202 and, a determination criterion value of the heart rate equivalent signal acquired by a learning of a determination criterion value learning circuit 208 of the heart rate equivalent signal when the eye closing rate calculated by the eye closing rate calculation circuit 203 is the preset predetermined value or less. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a drowsy driving detection apparatus that detects a drowsy driving state by combining a plurality of physical data of a driver.

  Conventionally, as a technique for detecting and determining a dozing state, for example, a technique described in the following document is known (see Patent Document 1).

In Document 1, it is determined whether or not the blink time obtained by integrating the blink time over a predetermined time every predetermined time exceeds a predetermined value, and based on this determination result, the doze according to the degree of danger Techniques for detecting conditions are described.
JP-A-6-219181

  However, in the technique adopted in the above-mentioned document 1, in consideration of the practicality of the system, when the eye opening / closing is detected in a non-contact manner such as reflected light of a photodiode or image recognition, a speedometer or There is a problem that the detection accuracy deteriorates because a slow blink that may be misrecognized even when looking downward, such as the screen of the navigation system.

  Therefore, the present invention has been made in view of the above, and an object of the present invention is to provide a drowsy driving detection device capable of accurately detecting the level of the driver's arousal level.

  In order to achieve the above object, the means for solving the problems of the present invention includes a heart rate equivalent signal detection means for detecting a heart rate equivalent signal corresponding to the heartbeat of the driver, and an eye for detecting the open / closed state of the driver's eye. Calculated by the open / close detection means, the closed eye rate calculation means for calculating the closed eye rate every predetermined time based on the open / closed state of the eyes detected by the open / closed eye detection means, and the closed eye rate calculation means A heart rate equivalent signal determination reference value learning means for learning a judgment reference value of a heart rate equivalent signal detected by the heart rate equivalent signal detection means when the closed eye rate is equal to or less than a predetermined value set in advance; and the heart rate equivalent signal detection Based on the heart rate equivalent signal detected by the means, the judgment reference value obtained by learning of the heart rate equivalent signal judgment reference value learning means, and the eye closure rate calculated by the eye closure rate calculation means, the driver's awakening Every time And having a wakefulness level estimation means for estimating a bell.

  According to the present invention, it is possible to accurately detect the level of the driver's arousal level.

  DESCRIPTION OF THE PREFERRED EMBODIMENTS The best embodiment for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a diagram showing the configuration requirements of a snooze driving detection device according to the present invention and a snooze driving prevention device using the snooze driving detection device. In FIG. 1, the drowsy driving detection device includes a heartbeat equivalent signal detection means 101 for detecting a driver's heartbeat equivalent signal, an eye opening / closing detection means 102 for detecting the opening / closing of the driver's eyes, and an eye opening / closing detection means 102. The closed eye rate calculation means 103 for calculating the closed eye rate at predetermined time intervals based on the open / closed state of the eye detected by the determination of the heart rate equivalent signal when the closed eye rate calculated by the closed eye rate calculation means 103 is a predetermined value or less. Based on the relationship between the heart rate equivalent signal determination reference value learning unit 104 that learns the reference value, the heart rate equivalent signal detected by the heart rate equivalent signal detection unit 101, and the closed eye rate calculated by the closed eye rate calculation unit 103, the driver And an arousal level estimation means 105 for estimating the arousal level.

  By adopting such a configuration, the driver's eye open / close data and heart rate data are integrated to detect and determine the driver's drowsiness state, thereby determining the conflict state where the driver resists drowsiness. It becomes possible to detect with high accuracy, and it is possible to accurately distinguish from a drowsiness state that hinders driving operation.

  Further, the snooze driving prevention device provides the driver with a warm-up effect according to the arousal level estimated by the awakening level estimation means 105 of the snooze driving detection device in addition to the above-described nap driving detection device. Awakening promotion means 106 that promotes awakening and a control means 107 that controls the operation of the awakening promotion means 106 are configured.

  By adopting such a configuration, the arousal stimulus presented to the driver is changed according to the arousal level detected by the drowsiness driving detection device, and the driver's awakening is effectively promoted to prevent a drowsy driving. can do.

  FIG. 2 is a diagram showing a system configuration of a dozing operation detection apparatus according to the first embodiment of the configuration requirements of the present invention shown in FIG. 1 and a dozing operation prevention apparatus using this apparatus.

  In FIG. 2, a TV camera (imaging unit) 201 that captures a driver's face image that functions as the eye open / close detection unit 102, and an image captured by the TV camera 201 is processed using a conventionally known image processing technique. An image processing circuit 202 that detects the opening and closing of the eye, a closed eye rate calculation circuit 203 that functions as the closed eye rate calculation unit 103, and a plus pole 204 and a minus pole 205 that are installed on the steering that functions as the heart rate equivalent signal detection unit 101. , An electrocardiogram signal detection device 206 for detecting an electrocardiogram signal applied from the plus electrode 204 and the minus electrode 205, and a pulsation interval for calculating a pulsation interval based on the electrocardiogram signal detected by the electrocardiogram signal detection device 206. The closed eye rate calculated by the calculation circuit 207, the closed eye rate calculation circuit 203, and the pulsation interval calculated by the pulsation interval calculation circuit 207 , A heartbeat signal determination reference value learning circuit 208 that functions as a heartbeat equivalent signal determination reference value learning means 104 for learning a heartbeat signal determination reference value, and a wakefulness level estimation circuit that functions as a wakefulness level estimation means 105 209, a control device 210 that functions as the control means 107 that operates in response to the estimation result of the arousal level estimation circuit 209, and an operation that is controlled under the control of the control device 210 and functions as the wakefulness promotion means 106 A seat vibrator 211 that gives vibration stimulus to the person, an alarm device 212 that emits an alarm sound, an electric seat belt 213, a hazard lamp 214, an auto air conditioner 215, a scent generator 216 that sends the set scent into the vehicle, and a predetermined amount of oxygen High-concentration oxygen-added air supply device 217 for supplying the added air into the vehicle, and It is configured to include an audio system 218.

  The eye opening / closing detection means 102 uses an optical sensor capable of detecting eye opening / closing with a simple configuration in addition to a method using an image processing technique. In this method, for example, using the technique described in Japanese Patent Application Laid-Open No. 63-258226, it is attached to the lower part of the frame of the eyeglass as shown by 301 in FIG. 3A, and as shown in FIG. The light emitting diode 302 irradiates the eye with near-infrared light, the difference in reflectance between the upper eyelid and the eyeball is detected as a difference in current value with the photodiode 303, and the detected difference is amplified with the amplifier 304 and then with the comparator 305. Compared with a predetermined voltage, the open / close state of the eye is detected based on the comparison result.

  The closed eye rate calculation circuit 203 calculates the closed eye rate per predetermined time similarly to the method using the image processing technique.

  In addition to the method of using biological signals collected by the electrodes of the plus pole 204 and the minus pole 205 installed in the steering of this embodiment, the heart rate equivalent signal can be detected by a type in which electrodes are attached to the chest. In recent years, there is a chest belt type heart rate monitor developed for sports science. In this heart rate monitor, the electrodes and the transmitter that transmits the detected signal are compactly gathered, and the heartbeat signal is received from the chest relatively easily by receiving the detection signal at the receiver installed at a location slightly away from the body. Can be detected.

  Next, with reference to the flowchart of FIG. 4, the process flow of the method for learning the determination reference value of the heart rate equivalent signal using the closed eye rate in the above configuration will be described. As shown in FIG. 5, the processing content is a method of setting the heart rate when the driver is in an appropriate tension state as a guarantee that the driver is not in a state of reduced arousal level, as shown in FIG. This is specifically shown.

  Returning to FIG. 4, first, it is determined whether or not a predetermined time for learning the reference heart rate has elapsed (step S401). This predetermined time is a time sufficient for learning the reference value of the heart rate in the normal state of the driver after the start of driving, and it is appropriate that the predetermined time is usually about 30 minutes.

  Next, since it is naturally determined that the predetermined time has not elapsed immediately after the activation of the present system, it is subsequently determined whether or not the reference heart rate has been updated a predetermined number of times (step S402). Even in this determination process, if it is immediately after the system is started, it has not been performed a predetermined number of times, so the eye closure rate in section A is calculated while measuring the predetermined time in section A (step S403). Similarly, the representative value of the instantaneous heart rate in the section A is calculated (step S404). It is determined whether or not a predetermined time of section A has elapsed (step S405), and the above processing is repeated until the predetermined time has elapsed.

  The processing content of the eye closure rate calculation performed in step S403 will be specifically described using an example using an image processing technique. The eye feature amount obtained as the two-dimensional grayscale data shown in FIG. 6, for example, the open / closed eye is detected based on the size of the vertical width, and the time interval between open → close and close → open is obtained to determine the eye close time per time calculate. This eye-closed time is excluded from a certain time corresponding to the blinking time, for example, about 0.15 seconds or less, and is added to the predetermined time of the section A.

  The reason why eye closure time is not included here is that the frequency of blinking varies greatly among individuals, and there are people who increase blinking and those who decrease blinking even in the process of lowering alertness. It is. However, it has been found that the occurrence of slow blinks (for example, exceeding about 0.15 seconds) has a high correlation with a decrease in the degree of arousal and the individual difference in the occurrence frequency is small. Therefore, the eye closure rate in the section A is calculated in this way.

  In addition, since the technique described in Japanese Patent Laid-Open No. 10-143669 can be used for the eye position detection process in the previous stage in which the opening / closing eye detection in the image processing is performed, the description in this embodiment is omitted.

  Next, the processing contents of the representative value calculation of the instantaneous heart rate in the previous step S404 will be specifically described. Usually, the heart rate is judged to be large or small on the basis of the number of beats per minute, but when measuring the beat as an electrocardiographic signal, for example, an electrocardiogram as shown in FIG. Even when the R wave 701 that is most easily captured by the signal is disturbed due to noise or the like, the peak point of the disturbed R wave 702 may not be captured, and the heart rate may not be counted accurately.

  Further, in the case of a driver having an arrhythmia as shown in FIGS. 7B and 7C, it becomes noise when learning the reference heart rate at the normal time. Therefore, as shown in FIG. 8, 60 seconds is divided by the heart rate interval RRi (seconds) and replaced with the instantaneous heart rate. In such a process, the interval RRI3 spread due to the lack of an electrocardiogram signal as shown in FIG. As shown in 4, 7, and 9, since it differs greatly from the data before and after that, it can be easily excluded when determining the representative value of the instantaneous heart rate for a predetermined time (section A).

  Specifically, except for people with heart disease etc., the irregular instantaneous heart rate differs greatly when it occurs, but the frequency of occurrence is less, so the median value than the average value of all instantaneous heart rates It is possible to learn an accurate reference heart rate by using. In this way, the eye closure rate in section A and the representative value of the instantaneous heart rate are calculated through the processing shown in steps S403, S404, and S405 of FIG.

  Returning to FIG. 4, after a predetermined time of the section A has elapsed, it is determined whether or not the closed eye rate of the section A is less than the predetermined value a (step S406). This predetermined value a is determined based on the relationship that the closed eye rate is always small when the driver's arousal level does not decrease, and the heart rate that changes in the region where the closed eye rate is small as shown in FIG. learn.

  As a result of the determination, if it is determined that the closed eye rate in the section A is less than the predetermined value a, after learning the representative value of the instantaneous heart rate calculated in the section A (step S407), the learning frequency is counted up. This is performed (step S408). Thereafter, the process returns to the previous step S401 and the same processing is repeatedly executed.

  In this processing loop, if the closed eye rate is temporarily calculated to be large due to erroneous detection of opening and closing, it is determined in the previous step S406 that the closed eye rate in section A is equal to or greater than a predetermined value a. The process returns to the previous step S401 without learning the reference heart rate of A. While repeating the processing of the same loop, learning of the reference heart rate is repeated with a guarantee that the eye-closing rate in the section A is less than the predetermined value a and that the driver's arousal level has not decreased.

  Here, the learning method in the previous step S408 will be specifically described. While driving, the heart rate may increase due to temporary tension or the like, but normally it becomes a tension state suitable for driving, and the heart rate is stabilized in a lowered state. It is desirable to learn a lower heart rate if there is a guarantee that the arousal level has not decreased. Therefore, in step S408, a value smaller than the representative value of the instantaneous heart rate of the section A that is performed a plurality of times is learned.

  The number of learning times in step S408 defines the time, and is set to a time sufficient for learning the reference heart rate during driving. For example, if section A is about 30 seconds and the number of learning is 10 times, then 5 minutes is considered sufficient for learning the reference heart rate during driving. Therefore, in this example, after the start of processing, if there is no disturbance that increases the closed eye rate, learning is completed in a minimum of 5 minutes. The completion is determined by the determination process in step S402. If the learning is completed as a result of the determination, the reference heart rate is determined for each driver and the learning is terminated (step S409).

  In addition, when there is some trouble in the recognition of the open / closed eyes, the state in which the closed eye rate in the section A is equal to or greater than the predetermined value a is repeated, and the reference heart rate set in step S401 exceeds the maximum time to be learned. At the same time, a message indicating abnormality in the open / closed eye determination is transmitted to the driver, and at the same time, the system is requested to be reset (step S410), so that it is possible to cope with a state where the driver's arousal level is lowered.

  Next, referring to the flowchart of FIG. 9, when the driver's arousal level is lowered, the level of arousal level that significantly impedes driving is lowered, and it is difficult to drive the vehicle safely. The flow of processing of a method for accurately determining whether or not the patient is in a state of conflict with sleepiness will be described.

  In FIG. 9, first, the closed eye rate of the section B is calculated (step S901), and the representative value of the instantaneous heart rate of the section B is calculated (step S902). Thereafter, it is determined whether or not a predetermined time corresponding to the section B has elapsed (step S903), and the above processing loop is repeatedly executed until the predetermined time has elapsed. Since this process is the same as when the reference heart rate is learned, detailed description thereof is omitted. It should be noted that the section for determining the lowered state of the driver's arousal level may be matched with the section for learning the reference heart rate, but it is not necessary to match it, so the section is described by changing from A to B.

  If the predetermined time has elapsed as a result of the determination in the previous step S903, it is determined whether or not the closed eye rate is equal to or greater than a predetermined value a (step S904). If the closed eye rate is not greater than or equal to the predetermined value a, the driver returns to the previous step S901 and repeats the same processing, assuming that the driver is not in a state of reduced arousal. When this determination state is represented by a time chart, a section B shown in the upper half of FIG.

  Returning to FIG. 9, when it is determined in step S904 that the closed eye rate in section B is greater than or equal to the predetermined value a, the representative value of the instantaneous heart rate in section B has increased from the previously determined determination reference value. The representative value of the instantaneous heart rate of the section B is obtained by performing the determination (step S905) and the determination (step S906) whether or not the determination reference value is smaller than the determination reference value. It is determined whether it is within the range. As a result of the determination, if it is determined that the representative value of the instantaneous heart rate has not increased or decreased with respect to the reference heart rate, the driver returns to the previous step S901 and assumes similar processing, assuming that the driver is not in a state of reduced wakefulness. repeat. If this determination state is also represented by a time chart, it becomes a section B shown in the lower half of FIG. In addition, when this state is represented by the relationship between the heart rate and the eye closure rate, it is as shown in FIG. 11, which is a region (zone) where erroneous detection of eye open / closed determination is prevented by heart rate determination.

  Returning to FIG. 9, it is determined that the closed eye rate in section B is equal to or greater than the predetermined value a, and it is determined in the determination result in step S905 that the representative value of the instantaneous heart rate in section B has increased from the range of the determination reference value. If so, it is estimated that the driver is in conflict with sleepiness (step S908). When this determination state is represented by a time chart, a section B shown in the lower half of FIG.

  Moreover, when this state is shown by the relationship between the heart rate and the closed eye rate, it is as shown in FIG. In this conflict zone with drowsiness, one long closed eye (for example, about 1 second or more) does not occur. Therefore, the upper right area where the heart rate increases and the eye closure rate is large is also set as a false detection prevention zone. . As a result, when accumulating the eye closure time in the section B, one long eye closure while excluding the eye closure time not exceeding a certain time corresponding to the blinking time (for example, about 0.15 seconds or less). However, by using the method by calculating the second closed eye rate as a non-target, it is possible to further improve the determination accuracy of the conflict zone with sleepiness.

  Returning to FIG. 9, it is determined in step S904 that the closed eye rate in section B is greater than or equal to the predetermined value a, and in step S906, it is determined that the representative value of the instantaneous heart rate in section B has decreased from the range of the determination reference value. If this is the case, it is estimated that the driver is in a dozing state in which the degree of arousal is so low as to hinder driving (step S907). When this determination state is represented by a time chart, a section B shown in the lower half of FIG. Moreover, this state is shown in FIG. 15 by showing the relationship between the heart rate and the closed eye rate.

  When it is determined based on only the closed eye rate whether or not the vehicle is in a dozing state that hinders driving, the determination cannot normally be made unless the closed eye rate is significantly increased. However, as in the first embodiment, by adding to the determination that the heart rate is decreasing, it is not always necessary to set a larger criterion for determining the closed eye rate. It is possible to estimate. Therefore, it is possible to more quickly detect a drowsiness that would hinder driving.

  In this way, the driver's arousal level is detected by distinguishing between a state of conflict with sleepiness and a snoozing state that hinders driving, so that the first suitable for each state as shown in FIG. It becomes possible to operate the awakening promotion means or the second awakening promotion means.

  Compared to the case of transition from a moderate tension state to a drowsy driving state that interferes with driving through a struggle state with drowsiness, the driver suddenly interferes with driving without a conflicting state with drowsiness. In the case of shifting to a drowsy driving state that causes the driver to wake up, the driver's arousal level is effectively promoted by presenting the driver with a stronger stimulus. As a result, a drowsy driving detection device and a drowsy driving prevention device system that are more comfortable and receptive to the driver and safer are provided.

  As described above, in the first embodiment, it is possible to accurately detect the conflict state in which the driver resists drowsiness, and the conflict state in which the driver resists drowsiness and driving are hindered. A serious dozing state can be accurately distinguished and detected.

  By using the heart rate equivalent signal as the beat interval and calculating the instantaneous heart rate from this beat interval, the heart rate equivalent signal can be measured in a highly robust state, and the high arousal level estimation accuracy can be maintained. Can do.

  A heartbeat equivalent signal can be measured with higher robustness by calculating the median value of the data of the instantaneous heart rate for a predetermined time and using it as a representative value. Arousal level can be estimated.

  By detecting the opening / closing of the driver's eyes using image recognition technology, it is possible to detect the opening / closing of the eyes with high accuracy without bothering the driver. On the other hand, by detecting the opening / closing of the eyes of the driver by the detection means including the light emitting diode 302, the photodiode 303, the amplifier 304, and the comparator 305, the opening / closing of the eyes can be detected by a relatively simple method.

  By calculating the closed eye rate by adding the closed eye time for a predetermined time or more for each section, it is possible to estimate the arousal level with high accuracy using the closed eye rate. By calculating the closed eye rate by adding the closed eye time that is equal to or longer than the first predetermined time and equal to or shorter than the second predetermined time for each section, it is possible to improve the detection accuracy of the conflict state against drowsiness.

  Even if the closed eye rate exceeds a predetermined value, if the value of the heart rate equivalent signal is within the range of the determination reference value, it is estimated that the arousal level is not lowered, thereby making an erroneous determination due to the closed eye rate. While being suppressed, it is possible to maintain the estimated performance of the state of reduced arousal level.

  When the closed eye rate exceeds a predetermined value and the value of the heart rate equivalent signal decreases from the criterion value, driving is performed by estimating that the arousal level is greatly reduced to the extent that the driving is disturbed. It is possible to accurately detect a dozing state that is not suitable for the above.

  When the eye closure rate exceeds a predetermined value and the value of the signal corresponding to the heartbeat increases from the criterion value, the arousal level is lowered, and it is estimated that the driver is in a state of conflict with drowsiness. In the process of declining, the driver can accurately detect the state of conflict with drowsiness, and whether or not the conflict occurred before falling into a dozing state that would impede driving Can be detected.

  When the closed eye rate calculated by integrating the eye closure time not less than the first predetermined time and not more than the second predetermined time for each section exceeds a predetermined value, and the value of the heart rate equivalent signal increases from the determination reference value In this process, it is estimated that the driver is in conflict with drowsiness by lowering the level of arousal level. Can be detected.

  As the awakening promotion means for promoting the driver's awakening, for example, a first awakening promotion means giving priority to comfort as shown in FIG. 16 and a second awakening promotion means giving priority to urgency are different. In a state where the driver is struggling with sleepiness in response to the estimation result of the arousal level, the first awakening means is activated to present a comfortable stimulus that does not feel annoying, In a drowsy state that hinders driving, the second awakening promotion means is activated to present a strong and highly urgent stimulus to effectively awaken the driver and enhance the effect of preventing drowsy driving In addition, it is possible to provide a snooze driving prevention device that is highly receptive to the driver.

  Next, a second embodiment of the present invention will be described. The feature of the second embodiment is that, compared to the first embodiment, the section C in which the eye-closing rate determination section having a higher time resolution than the heart rate change is set shorter than the section B of the first embodiment. The determination in D is added, and the reduced state of the driver's arousal level is estimated even more quickly, and the rest is the same as in the first embodiment.

  Next, with reference to the flowcharts of FIGS. 17 and 18, the processing procedure of the above-described feature estimation method employed in the second embodiment will be described. In FIG. 17, first, the closed eye rate of each of the sections D, C, and B is sequentially calculated (steps S1701, S1702, and S1703). For example, as shown in FIGS. 19 and 21, the relation between the sections B, C, and D is set to the relation of the time length of the section B ÷ 5 = section C and the section B ÷ 10 = section D. The shortest section D is used for estimating a drowsiness state that would impede driving when a decrease in heart rate is determined, and a longer section C is a case where an increase in heart rate is determined. It is for estimating the state of conflict with sleepiness.

  Returning to FIG. 17, next, the representative value of the instantaneous heart rate in the section B is calculated in the same manner as in the first embodiment (step S1704). Thereafter, it is determined whether or not a predetermined time in the section D has passed (step S1705) and whether or not a predetermined time in the section C has passed (step S1706). In the series of processes shown in steps S1705 and S1706, as shown in the time charts of FIGS. 19, 20, and 21, the determination in the section B is performed once, and the determination of the closed eye rate is less than the predetermined value a (False), Since this is a determination flow when the representative value of the heart rate increases or decreases (True) from the range of the determination reference value and shifts to the next section B, it will be described later.

  If the determination results of steps S1705 and S1706 indicate that the predetermined times of section D and section C have not elapsed, it is determined whether or not the predetermined time of section B has elapsed as in the first embodiment (step 1). S1709). As a result of the determination, if it is determined that the predetermined time has elapsed, the heartbeat reduction and increase flag is turned OFF (step S1710). This is a process for initializing the determination in the previous B section, and the state of the heart rate in the current section B is whether or not any flag is set again in the series of processing flow shown in FIG. judge.

  Next, moving to the flowchart shown in FIG. 18, it is determined whether or not the representative value of the instantaneous heart rate has increased from the range of the determination reference value (step S1801) and whether or not it has decreased (step S1802). Do. When it is determined that the representative value of the instantaneous heartbeat is within the range of the determination reference value in the processing of both steps, that is, when neither increases nor decreases, the process returns to step S1701 in FIG. Move on to judgment.

  On the other hand, as a result of the determination in step S1801 in FIG. 8, if it is determined that the representative value of the instantaneous heart rate has increased from the reference value range, the heart rate increase flag is turned ON (step S1806). Thereafter, it is determined whether or not the closed eye rate is equal to or greater than a predetermined value a (step S1807). If it is determined that the closed eye rate is equal to or greater than the predetermined value a, the determination so far is performed as in the first embodiment. As a result, it is estimated that the driver is in conflict with sleepiness (step S1808). On the other hand, if it is determined that the closed eye rate is less than the predetermined value a, the process returns to step S1701 in FIG.

  FIG. 19 shows this state in a time chart. That is, in this section B, the calculation of the closed eye rate in section C and the determination of the closed eye ratio in section C are not less than a predetermined value b of the closed eye ratio corresponding to section C while calculating the closed eye rate in section B. It is determined in step S1707 and step S1708 in FIG.

  In the example of the timing chart shown in FIG. 19, in the determination of the closed eye rate in the first section C in step S <b> 1708 of FIG. 17, it is determined that the closed eye rate is less than the predetermined value b, and then the process of step S <b> 1709 is performed. The process returns to step S1701. Thereafter, the eye closure rate in the next section C is calculated. On the other hand, if it is determined in step S1708 that the eye closure rate in the second section C is greater than or equal to the predetermined value b, it is estimated that the driver is in conflict with drowsiness at this stage ( Step S1808).

  In addition, as shown in the example of the timing chart shown in FIG. 20, even when it is determined that all of the eye closure rates in the section C are less than the predetermined value b (False), the eye closure rate and the heart rate in the section B In the determination, since the same processing as in the first embodiment is repeated, there is no possibility that the detection timing is delayed or the estimation accuracy of the arousal level is changed.

  Next, returning to FIG. 18, if it is determined in the determination result in step S1802 that the representative value of the instantaneous heart rate has decreased below the range of the determination reference value, the heart rate reduction flag is set to ON (step S1803). Thereafter, it is determined whether or not the closed eye rate is less than the predetermined value a (step S1804). If it is determined that the closed eye rate is less than the predetermined value a, the process returns to step S1701 in FIG. This time, the calculation of the closed eye rate in the section D and the determination of the closed eye ratio in the section D are performed by the processing in steps S1705 and S1706 to determine whether or not the closed eye rate corresponding to the section D is equal to or greater than a predetermined value c.

  FIG. 21 shows a series of processing flows in a time chart. In this way, by using the heart rate change in the previous section B as a preliminary flag, it is possible to more effectively combine the determination of the heart rate and the closed eye rate with different determination resolutions. On the other hand, if it is determined in the determination result of the previous step S1804 that the closed eye rate is greater than or equal to the predetermined value a, the driver's arousal level is lowered to such an extent as to impede driving and is estimated to be in a doze state. (Step S1805).

  In general, as shown in 2201 and 2202 in FIG. 22, doze driving falls from a moderate tension state to a drowsy state that interferes with drowsiness. In the case of illness such as sleep apnea syndrome and overwork, as shown by 2301 in FIG. 23, the patient does not necessarily go through the state of conflict with sleepiness. Sometimes it falls into a state. Even in such a case, since it is possible to determine whether or not there is a conflict state with sleepiness, it is possible to estimate a dozing state to the extent that the driving is disturbed with high accuracy at an early stage.

  FIG. 24 and FIG. 25 show experimental values of 60-minute time-series data of closed eye rate and heart rate in a conflict state with sleepiness reproduced in a tabletop experiment and a drowsiness state that does not resist sleepiness and hinders driving. FIG.

  The experimental data shown in both figures are obtained by letting the subject perform a task that requires a certain level of concentration, but during a time period after lunch that tends to be relatively sleepy even with biological rhythms. The data shown tells the patient to endure the task no matter how sleepy, and gives a feeling of fear that a strong stimulus will be presented if the eye closure rate of 30% is exceeded during the task, indicating a state of conflict with sleepiness It is a reproduction.

  On the other hand, the data shown in FIG. 25 was obtained without giving any instructions. In the sleep test on the table, even if it becomes sleepy, there is no strong irritation to yourself. Unlike many people, they often fall asleep without conflicting with sleepiness.

  The details of FIGS. 24 and 25 will be described. Both figures (a) show the time change of the eye-closed rate every 30 seconds, and both figures (b) show the representative values of heart rate for 5 minutes from the start of the experiment. As a reference, the change in the representative value of the heart rate every 5 minutes is shown.

  As can be seen from a comparison between FIG. 24 and FIG. 25, it can be seen that in the data having a conflict state with drowsiness in FIG. 24, not only the closed eye rate is slightly increased but also the heart rate is increased. On the other hand, in the data of FIG. 25 that does not conflict with sleepiness and falls into a dozing state that hinders driving, the heart rate also tends to decrease unilaterally as the eye closure rate increases. By looking at the correlation between the closed eye rate and the heart rate in this way, it is possible to accurately detect whether or not there was a conflict with sleepiness in the process of transitioning from moderate tension to a dozing state that hinders driving. It is also obtained from experimental data that this can be done.

  As described above, in the second embodiment, in addition to the effects obtained in the first embodiment, the value of the heart rate equivalent signal increases from the determination reference value, and the closed eye rate does not exceed the predetermined value. In the next section, the eye closure rate is calculated by shortening the section time of one section, and the arousal level is estimated based on the closed eye ratio calculated by shortening the section time of one section. It is possible to detect a state in which the driver is struggling with drowsiness earlier than 1.

  When the value of the heart rate equivalent signal increases from the determination reference value and the closed eye rate does not exceed the predetermined value, the closed eye rate is calculated by shortening the interval time of one interval in the next interval, and the interval time of the 1 interval is calculated. By estimating the arousal level based on the eye closure rate calculated by shortening, it is possible to detect a state where the driver is struggling with sleepiness earlier than in the first embodiment.

  In addition, when the value of the heart rate equivalent signal is smaller than the determination reference value and the closed eye rate does not exceed the predetermined value, the closed eye rate is calculated by reducing the interval time of one interval in the next interval, and the interval of 1 interval is calculated. By estimating the arousal level based on the closed eye rate calculated by shortening the time, it is possible to detect a state where the driver is struggling with sleepiness earlier than in the first embodiment.

It is a figure which shows the structure of the dozing driving | running | working detection apparatus of this invention. It is a figure which shows the system configuration | structure of the dozing operation detection apparatus which concerns on Example 1 of this invention. It is a figure which shows the structure of an eye opening / closing detection means. It is a flowchart which shows the procedure of the process which concerns on Example 1 of this invention. It is a figure which shows the learning area | region of the heart rate in the relationship between a closed eye rate and a heart rate. It is a figure which shows the example of calculation of the closed eye rate using an image processing technique. It is a wave form diagram which shows the example of an electrocardiogram signal. It is a figure which shows the relationship between a heartbeat interval and an instantaneous heart rate. It is a flowchart which shows the procedure of the process which concerns on Example 1 of this invention. It is a timing chart in the judgment state of the processing concerning Example 1 of the present invention. It is the figure which represented the state shown in FIG. 10 by the relationship between a closed eye rate and a heart rate. It is a timing chart in the judgment state of the processing concerning Example 1 of the present invention. FIG. 13 is a diagram showing the state shown in FIG. It is a timing chart in the judgment state of the processing concerning Example 1 of the present invention. It is the figure which represented the state shown in FIG. 14 by the relationship between a closed eye rate and a heart rate. It is a figure which shows the correspondence of awakening promotion means and a driver | operator's state. It is a flowchart which shows the procedure of the process which concerns on Example 2 of this invention. It is a flowchart which shows the procedure of the process which concerns on Example 2 of this invention. It is a timing chart in the judgment state of the processing concerning Example 2 of the present invention. It is a timing chart in the judgment state of the processing concerning Example 2 of the present invention. It is a timing chart in the judgment state of the processing concerning Example 2 of the present invention. It is a figure which shows the mode of a driver | operator's state transition. It is a figure which shows the mode of the other transition of a driver | operator's state. It is a figure which shows the experimental result of the relationship between the eye-closing rate and heart rate in the conflict state with sleepiness. It is a figure which shows the experimental result of the relationship between a closed eye rate and the heart rate in the state which cannot resist sleepiness.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Heart rate equivalent signal detection means 102 ... Opening / closing detection means 103 ... Eye closure rate calculation means 104 ... Heart rate equivalent signal determination reference value learning means 105 ... Arousal level estimation means 106 ... Arousal promotion means 107 ... Control means 201 ... TV camera 202 ... Image processing circuit 203 ... Eye closure rate calculation circuit 204 ... Positive pole 205 ... Negative pole 206 ... Electrocardiogram signal detection device 207 ... Beat interval calculation circuit 208 ... Heart rate signal determination reference value learning circuit 209 ... Arousal level estimation circuit 210 ... Control device 211 ... Seat vibrator 212 ... Alarm device 213 ... Electric seat belt 214 ... Hazard lamp 215 ... Auto air conditioner 216 ... Scent generator 217 ... High-concentration oxygenated air supply device 218 ... Car audio system 302 ... Light emitting diode 303 ... Photodiode 304 ... Amplification 305 ... comparator

Claims (13)

A heart rate equivalent signal detection means for detecting a heart rate equivalent signal corresponding to the heart rate of the driver;
Eye opening / closing detection means for detecting the opening / closing state of the driver's eyes;
An eye-closing rate calculating unit that calculates an eye-closing rate every predetermined time based on the eye open / closed state detected by the eye opening / closing detection unit;
Heart rate equivalent signal determination reference value learning for learning a determination reference value of a heart rate equivalent signal detected by the heart rate equivalent signal detection unit when the eye closure rate calculated by the eye closure rate calculation unit is equal to or less than a predetermined value set in advance. Means,
Based on the heart rate equivalent signal detected by the heart rate equivalent signal detection means, the judgment reference value obtained by learning of the heart rate equivalent signal judgment reference value learning means, and the eye closure rate calculated by the eye closure rate calculation means. A drowsiness driving detection apparatus comprising: a wakefulness level estimating means for estimating a driver's wakefulness level. The doze driving according to claim 1, wherein the heart rate equivalent signal detecting means detects a beat interval, calculates an instantaneous heart rate based on the beat interval, and uses the instantaneous heart rate instead of the heart rate. Detection device. The heart rate equivalent signal detecting means stores the instantaneous heart rate for a predetermined time set in advance as one section, calculates a median value of stored data of the one section, calculates a representative value of the instantaneous heart rate, The sleepiness detection device according to claim 2, wherein a representative value of the heart rate is used. The eye opening / closing detection means includes: an imaging camera that captures an image of the driver's eye; and an eye opening / closing operation based on a result obtained by processing an image of the driver's eye captured by the imaging camera using an image recognition technique. The state detection is performed, The dozing driving detection device according to any one of claims 1 to 3. The eye open / close detection means comprises:
A light-emitting diode that irradiates the driver's eyes with infrared light; and
A photodiode that receives infrared light reflected from the eyes of the driver;
The doze driving detection device according to any one of claims 1 to 3, further comprising a detection circuit that detects an open / closed state of an eye based on reflected light obtained by the photodiode. The wakefulness level estimation unit obtains the value of a heart rate equivalent signal by learning of the heart rate equivalent signal determination reference value learning unit, with the closed eye rate calculated by the eye closure rate calculation unit exceeding a predetermined value set in advance. The drowsiness driving detection apparatus according to any one of claims 1 to 5, wherein the drowsiness detection level of the driver is estimated not to be lowered when the level is within the range of the determination reference value. The wakefulness level estimation unit obtains the value of a heart rate equivalent signal by learning of the heart rate equivalent signal determination reference value learning unit, with the closed eye rate calculated by the eye closure rate calculation unit exceeding a predetermined value set in advance. 7. The method according to claim 1, wherein the driver's arousal level is estimated to be lowered to an extent that hinders driving when the determination criterion value is decreased. Snooze driving detection device. The wakefulness level estimation unit obtains the value of a heart rate equivalent signal by learning of the heart rate equivalent signal determination reference value learning unit, with the closed eye rate calculated by the eye closure rate calculation unit exceeding a predetermined value set in advance. The driver's arousal level is estimated that the driver has fallen to some extent in drowsiness and a conflict state when the determination reference value increases. The dozing driving detection device according to the item. The eye-closing rate calculating means sets a predetermined time set in advance as one section, and when the eye-closed state detected by the eye open / close detection means is equal to or more than a first predetermined time set in advance, The drowsiness driving detection apparatus according to any one of claims 1 to 8, wherein the eye closure rate is calculated by integrating time for each section. The eye-closing rate calculation means sets a predetermined time set in advance as one section, and the eye-closed state detected by the eye open / close detection means is equal to or more than a preset first predetermined time and is set in advance. The drowsiness driving detection apparatus according to any one of claims 1 to 9, wherein, when the time is equal to or shorter than the second predetermined time, the eye closing rate is calculated by adding the eye closing time for each section. The wakefulness level estimation unit obtains the value of a heart rate equivalent signal by learning of the heart rate equivalent signal determination reference value learning unit, with the closed eye rate calculated by the eye closure rate calculation unit exceeding a predetermined value set in advance. The drowsiness driving detection according to claim 10, wherein the driver's arousal level is estimated that the driver has fallen to a certain degree in drowsiness and a conflict state when the determination reference value increases. apparatus. The closed eye rate calculating means has a calculated closed eye rate equal to or less than a predetermined value set in advance, and a value of a heart rate equivalent signal is greater than a judgment reference value obtained by learning of the heart rate equivalent signal judgment reference value learning means. In this case, the drowsiness driving detection apparatus according to any one of claims 9 to 11, wherein the eye closure rate is calculated by shortening the section time of the one section in the next section. The closed eye rate calculating means has a calculated closed eye rate equal to or less than a predetermined value set in advance, and a value of a heart rate equivalent signal is smaller than a judgment reference value obtained by learning of the heart rate equivalent signal judgment reference value learning means. In the case, the drowsiness driving detection apparatus according to any one of claims 9 to 12, wherein the eye closure rate is calculated by shortening the section time of the one section in the next section.

Images