JP6720821B2 - Blink detector - Google Patents

Description

The present invention relates to an eye blink detection device, and more particularly, to a blink detection device that detects eye blink so as to detect a drowsiness state of a person from a blink mode.

Various attempts have been made to detect a person's drowsiness state by a device in order to prevent an accident caused by a drowsiness during driving of a vehicle, and it is known to detect blinking of the eyes as one means of drowsiness detection. It is known that the drowsiness determination based on the blinking state uses parameters such as the blinking frequency, the average value of the blinking intervals, and the standard deviation of the blinking intervals. Further, assuming an eyelid opening/closing feature amount representing drowsiness based on the above parameters, and setting a threshold value for activating a drowsiness warning or safety support control for the eyelid opening/closing feature amount, such a threshold value is It is proposed in Patent Document 1 below that not only the eyelid opening/closing feature amount is set to a boundary that deviates from the normal region in one direction, but also the eyelid opening/closing feature amount is set to a boundary that deviates from the normal region in another direction. Has been done. This is because, in general, when the person becomes drowsy, the movement of the eyelids slows down, but sometimes the eyelids intentionally move violently in an attempt to awaken against them.

JP, 2016-146978, A

Blinking is an action that closes the eyelid for a very short time, but the characteristic movement in opening and closing the eyelid, such as blinking, is that the movement of the eyelid is sequentially taken by photographing the eye area with a camera or attaching electrodes to the vicinity of the eye. Based on the data of "eye opening degree versus time" acquired by tracking, the time point when the eye opening degree falls below a predetermined threshold value, or the time point when the eye opening degree rises above the threshold value again, or both of these time points are collected. It is judged based on what you do. In this case, the above-mentioned threshold value is conventionally set to a relatively low eye opening degree suitable for accurately capturing a normal blink. However, when a person feels drowsiness, he starts to blink with his eyelids closed only to a relatively high degree of eye opening, which may be called half-blink. Therefore, in order to more accurately grasp the blink in the state where the person is drowsy, it is conceivable to raise the above threshold and capture not only the normal blink but also the half blink. However, increasing the above threshold increases the degree of misunderstanding that the irregular opening and closing of the eyelid, which is a noise for the detection of blinks, is not the true blink in the opening and closing of the eyelids, and particularly the judgment of the normal awake state. I am afraid to make a mistake.

The present invention focuses on the above-mentioned problems in the determination of blinks, and accurately grasps the blinks at the time of normal awakening, and when a person starts drowsiness and starts half-blinking, it is also possible to accurately grasp it. An object of the present invention is to provide a blink detection device that can be used.

In order to solve the above problems, the present invention is, in the blink detection unit that detects a blink based on the data of the time versus eye opening degree, a relatively high open eye of a relatively low first threshold value and opening the eye of The second threshold is set, and it is determined that there was a normal blink when the eye opening degree crosses the first threshold value above the low level after crossing the first threshold value below the high level, and the eye opening level is determined. Characterized in that it is determined that there was a half-blink when the second threshold was crossed above low without crossing the first threshold below high after crossing the second threshold below high. It proposes a blink detection device.

The determination that there was a half-blink is based on the data of the time versus the eye opening speed obtained by temporally differentiating the time versus the eye opening degree data, and the eye opening speed is set to a predetermined negative threshold value lower than high. It may be ascertained by crossing the negative threshold above low after crossing and crossing a predetermined positive threshold above low before crossing the positive threshold below high.

Alternatively, the determination that there is a half-blink is based on the data of the time versus the eye opening acceleration obtained by further temporally differentiating the data of the time versus the eye opening speed, and the eye opening acceleration has a predetermined negative threshold value. After crossing below the high, crossing the negative threshold above the low, crossing a predetermined positive threshold above the low and then crossing the positive threshold below high, and crossing the predetermined positive threshold. Being confirmed by crossing the positive threshold below the high after crossing above the low, and then crossing the negative threshold above the low after crossing a predetermined negative threshold below the high. May be.

As described above, in the blink detection unit that detects a blink based on the data of the time versus eye opening degree, open eyes of a relatively low first threshold value and opening the eye of sets a relatively high second threshold value, It is determined that there was a normal blink when the degree of eye opening crosses the first threshold to a value higher than low after crossing the first threshold, and the degree of eye opening is lower than the second threshold to a value higher than high. If it is determined that there is a half-blink when the second threshold is crossed higher than low without crossing the first threshold lower than high after crossing, then the degree of eye opening is the first. When a normal blink such that it falls below the threshold value is performed, while accurately grasping it by the relatively low first threshold value that is clearly separated from the noisy eyelid opening and closing, in a situation where a half-blink occurs, By setting the second threshold value that is relatively high, it can be captured without letting it escape, and the degree of arousal of a person can be accurately determined by blinking when the person is drowsy even during normal awakening.

If the degree of eye opening simply crosses the second threshold value from high to low and then from low to high, the second threshold value is set to be relatively high. There is a concern that it may be erroneous due to, but the determination that there was a half-blink, based on the data of the time-opening speed obtained by differentiating the time-opening degree data temporally, the eye-opening speed Crossing the predetermined negative threshold below the high and then crossing the negative threshold above the low, and further crossing the predetermined positive threshold above the low and then crossing the positive threshold below high. By confirming the characteristics of the change in the eye-opening speed associated with the blink, the half-blink can be more accurately determined.

Further, the determination that there was a half-blink, based on the data of the time-to-eye opening acceleration obtained by further temporally differentiating the time-to-eye opening speed data, the eye-opening acceleration is higher than a predetermined negative threshold value. After crossing low, the negative threshold is crossed above low, and further, the positive threshold is crossed above low, then the positive threshold is crossed below high, and the predetermined positive threshold is crossed below low. Ascertained by crossing high and then crossing the positive threshold below high and then crossing a predetermined negative threshold below high and then across the negative threshold above low. For example, by verifying the characteristics of the change in the eye-opening acceleration with the blink, which is more prominent than the characteristics of the change in the eye-opening speed with the blink, the half-blink can be determined more accurately.

In the graph showing an example of the data of the degree of eye opening versus time obtained by low-pass filtering the electrical signal showing the temporal change of the eyelid opening and closing obtained from the electrode or the like attached to the vicinity of the eye of the camera is there. It is a graph which shows the characteristic of the time-to-eye-opening degree which appears in a blink, time-to-eye-opening speed, and time-to-eye-opening acceleration. It is a flowchart which shows the process of the signal processing which grasp|ascertains a normal blink and a half blink as the start time and the end time based on the data of the eye opening degree. FIG. 4 is a partial view showing a part of the changed portion of the flowchart shown in FIG. 3, and is a diagram showing a point of adding a confirmation process based on time versus eye opening speed or time versus eye opening acceleration to the signal processing shown in FIG. 3. It is a flowchart which shows the process of the signal processing which grasps|sees a half blink as its start time point and its end time point based on the data of time vs. eye opening speed. It is a flowchart which shows the process of the signal processing which grasps|sees a half blink as the start time and the end time based on the data of the time-to-eye-opening acceleration.

By low-pass filtering the electric signal of the temporal change of the eyelid opening and closing obtained by photographing the human eye with a camera or attaching an electrode near the eye, the time vs. eye opening degree as illustrated in FIG. 1 can be obtained. Data is obtained. Then, with respect to the data of the degree of eye opening versus the time taken, a threshold value that is a relatively low degree of eye opening that is clearly separated from the noisy eyelid opening and that is reliably crossed by a change in the degree of eye opening in a normal blink when a person is awake. If Gf is set, it is possible to accurately grasp the normal blinking state that appears at the time of awakening by collecting the change points of the eye opening degree that appear as deep depressions D1 and D3 in the figure as a pair of time points T2 and T3. You can

On the other hand, when a person becomes drowsy and a half-blink appears as a shallow depression D2 in the figure, such a change in the degree of eye opening is not detected by the threshold Gf, but a relatively high threshold for detecting such a half-blink. If Gm is set, the threshold value Gm is inferior to the threshold value Gf in that it discriminates the noisy eyelid opening/closing, but when a half-blink occurs, it is collected as a pair of time points T1 and T2. be able to.

As shown in the upper part of FIG. 2, typically, the change in the degree of eye opening during blinking is a downward acceleration including a first process C1 and Ab in which the degree of eye opening is accelerated at a downward speed accompanied by downward acceleration. The second step C2 of lowering the eye opening degree until the downward speed reaches the maximum value Vb while returning to 0, and the third step C2 of decreasing the eye opening degree by adding the upward acceleration to the downward speed to reduce the downward speed from the maximum value Vb. The fourth process of gradually decreasing the downward speed and softly arriving at the minimum value of the eye opening while returning the upward acceleration added to C3 and As to 0, C4, adding the upward acceleration to the eyelid and raising the eye opening degree from the minimum value Fifth step C5, increasing the eye opening degree until the upward speed reaches the maximum value Vs while returning the upward acceleration added to As to 0. The sixth step C6, adding the downward acceleration to the upward speed and increasing the upward speed to the maximum value. The seventh process C7 is to increase the degree of eye opening while lowering Vs, and the eighth process C8 is to gradually reduce the upward speed while returning the downward acceleration added to Ab to 0 and finish the blink.

The blink detection operation for detecting the normal blink and the half-blink by setting the two threshold values Gf and Gm as described above may be performed according to the procedure shown in the flowchart of FIG. The digital arithmetic processing by the computer as shown in the flowchart is repeatedly performed in a fine cycle of about several tens of milliseconds.

When the operation of the blink detection device is started, the eye opening degree G of the eyelid is detected by a camera, an electrode, or the like in step (S) 1, and the obtained electric signal indicating the progress of the eye opening degree versus time is obtained in step 2. Low-pass filtering that removes noise-like signal changes is performed, and the data of the degree of eye opening versus time as illustrated in FIG. 1 is obtained. Next, the signal processing proceeds to step 3 and it is determined whether the flag F1 is 1 or not. This kind of flag is reset to 0 at the start of the operation of the apparatus, and in this case, the flag F1 is set to 1 in step 5 described later, so that the answer to step 3 is no (N) until then. Yes, signal processing proceeds to step 4.

In step 4, it is judged whether or not the eye opening degree G is equal to or more than a threshold value as shown by Gm in FIG. If the blinking is not performed or the degree of eye opening is not lower than Gm even during the blinking, the answer is yes (Y), during which the signal processing bypasses step 5 and proceeds to step 6. .. On the other hand, if a blink occurs and the degree of eye opening G drops below the threshold value Gm, then the answer to step 4 changes from yes to no, and the signal processing proceeds to step 5. In step 5, the flag F1 is set to 1, the time T at that moment is stored as the time T1, and the signal processing proceeds to step 6.

In step 6, it is determined whether the flag F4 is 1. Since the flag F4 is set to 1 in step 15 which will be described later, the answer to step 6 is no, the signal processing returns to step 1 and the detection of the degree of eye opening is continued. In this case, when the signal processing proceeds to step 3 by obtaining the low-pass filtered data of the detected time vs. the degree of eye opening, the answer to step 3 is YES, so the signal processing proceeds to step 7 from this and the flag is set. It is determined whether F2 is 1. Since the flag F2 is set to 1 in step 10 which will be described later, the answer to step 7 is no and the signal processing proceeds to step 8.

In step 8, it is judged whether or not the eye opening degree G is equal to or more than a threshold value as shown by Gf in FIG. Even after the eye-opening degree G crosses the threshold value Gm downward, if the eye-opening degree is still lower than Gf during the blink, the answer is yes, and the signal processing proceeds to step 9 to open the eyes. It is determined whether the degree G is greater than or equal to the threshold value Gm. Since the signal processing reaches step 9 after the eye opening degree G becomes equal to or less than the threshold value Gm, the answer to step 9 is NO for the time being, and the signal processing proceeds to step 6 from this. If the answer is no, the signal processing returns to step 1 and detection of the degree of eye opening is continued.

If the blink is a normal blink that closes across the threshold value Gf like the depression D1 in FIG. 1, the answer of step 8 turns to yes or no while the signal processing proceeds to step 9 to no. The process proceeds to step 10, the flag F2 is set to 1, and the time T at that moment is stored as the time T2. As a result, the signal processing which has returned to step 1 through step 6 proceeds from step 7 to step 11.

In step 11, it is determined whether the flag F3 is 1. Since the flag F3 is set to 1 in step 13 described later, the answer to step 11 is no, and the signal processing proceeds to step 12. In step 12, it is determined whether or not the eye opening degree G is equal to or greater than the threshold value Gf. If the blink is a normal blink that crosses the threshold value Gf and closes largely after opening across the threshold Gf as in the case of the depression D1 in FIG. 1, the answer to step 12 is NO for a while, and the signal processing returns to step 1 via step 6, but soon. When the eye opening degree G returns to the threshold value Gf or more, the answer to step 12 is changed from no to yes, at which time the signal processing proceeds to step 13, and the time T at that moment is stored as the time T3. Further, since the flag F3 is set to 1 here, the signal processing which has returned to step 1 through step 6 from here proceeds from step 11 to step 14 thereafter.

In step 14, it is determined whether or not the eye opening degree G is equal to or greater than the threshold value Gm. If the blink is a normal blink that is wide across the threshold Gf, opens across the threshold Gf, and then opens across the threshold Gm, as in the depression D1 in FIG. 1, the answer in step 14 is initially no for the time being, Although the signal processing returns to step 1 through step 6, the answer of step 14 turns from no to yes, so that the signal processing proceeds to step 15 and the time T at that moment is stored as the time T4. Furthermore, since the flag F4 is set to 1 at this time, the signal processing which has proceeded to step 6 from here reaches step 16 when the answer to step 6 is YES.

However, on the other hand, when the blink is a half-blink that closes across the threshold Gm but does not close enough to cross the threshold Gf as indicated by the depression D2 in FIG. 1, the answer to step 8 is to change from yes to no step. Answer 9 turns from no to yes. At this time, the signal processing proceeds to step 17, the flag Fc is set to 1, the time T at that moment is stored as T2, and the signal processing proceeds to step 16 from this.

In step 16, the suffix n is incremented by 1 starting from 0 which was reset at the start of operation of the device, then the signal processing proceeds to step 18 and it is judged whether the flag Fc is 1 or not. If the flag Fc remains 0 here, it indicates that there was a normal blink that closes across the threshold Gf, and if the flag Fc is set to 1, it is a half blink that closes across the threshold Gm. Indicates that there was. Therefore, when the answer to step 18 is NO, the signal processing proceeds to step 19, where the time point T2 is the start time point Tns (n is the suffix) of the nth blink, and the time point T3 is the end of the nth blink. It is time Tne. On the other hand, when the answer to step 18 is yes, the signal processing proceeds to step 20, where the time point T1 is the start time point Tns of the nth blink and the time point T2 is the end point Tne of the nth blink. When the signal processing proceeds to step 19, the flags F1, F2, F3 and F4 are reset to 0 in the following step 21, and when the signal processing proceeds to step 20, the flags F1, F2 and Fc are continued in the following step 22. Are reset to 0, and the signal processing proceeds to step 23.

In step 23, it is judged whether or not the suffix n has reached a predetermined limit value nf. The determination of the degree of awakening or drowsiness based on the blinking state is performed based on the blinking frequency over a predetermined number, the average value of the blinking intervals, the standard deviation of the blinking intervals, and the like. The value of the suffix n determines the number of blinks to be observed, and the answer in step 23 is NO until the predetermined number n of blinks are observed, so that the start point Tns and the end point Tne of the blink are obtained. The signal processing is repeated while sequentially increasing the suffix n until it reaches a predetermined number nf.

In the signal processing shown in the flowchart of FIG. 3 above, the start time Tns and the end time Tne of the half-blink are also determined by the eye opening degree crossing a predetermined threshold. In order to make a determination based on the degree of eye opening, the threshold value of the degree of eye opening must be raised, and therefore the degree of false determination that the noiseless eyelid opening/closing is blinking is not blinking. Therefore, when a half-blink is detected by the degree of eye opening, it is possible to more accurately confirm the half-blink based on the mode of the change in the lid opening/closing speed and the change in the acceleration. FIG. 4 is a partial view showing a modification to be added to the flowchart of FIG. 4, steps 1, 3, 18, and 23 are steps 1, 3, 18, and 23 in FIG. 1, step 2-1 is a partial modification of step 2 in FIG. It is a subroutine shown in FIG. The signal processing by such a subroutine may be executed in parallel with the signal processing for the next blink detection in the main routine according to the flowchart of FIG.

When the signal processing of the subroutine A shown in FIG. 5 is added to the signal processing shown in FIG. 3, the low-pass filtered data of the eye opening degree G is stored in step 2-1 and the signal processing of the subroutine A is performed. When it is started, in step 101, the eye opening degree G between the time points T1 and T2 stored in step 2-1 is temporally differentiated, and the differential value f(T) representing the time versus the eye opening speed is calculated. The time course thereof is as shown in the middle part of FIG.

The signal processing then proceeds to step 102, where it is determined whether the flag F11 is 1 or not. Since the flag F11 is set to 1 in step 104 described later and reset to 0 until then, the answer to step 102 is no, the signal processing proceeds to step 103, and the differential value f(T) is obtained. Is determined to be equal to or greater than a predetermined threshold Vd. The threshold value Vd is a negative value as shown in the middle part of FIG. Initially, the answer is yes, and signal processing bypasses step 104 and proceeds to step 105 where it is determined whether flag F14 is one. Since the flag F14 is set to 1 in step 113 described later and has been reset to 0 until then, the answer to step 105 is no for the moment, and the signal processing returns from this to step 102.

When the downward speed of the eyelids increases and falls below the threshold value Vd (increases as the downward speed), the answer in step 103 changes from yes to no, so the signal processing proceeds to step 104 and the flag F11 is set to 1 , And the time at this moment is stored as time T1. From this, the signal processing returns to step 102 through step 105, but since the flag F11 is set to 1, the signal processing thereafter proceeds from step 102 to step 106.

In step 106, it is determined whether the flag F12 is 1. Since the flag F12 is set to 1 in step 108 described later and is reset to 0 until then, the answer to step 106 is no, the signal processing proceeds to step 107, and the differential value f(T) is obtained. Is determined to be equal to or greater than a predetermined threshold Vd. If the threshold value Vd is set to an appropriate negative value as shown in the middle part of FIG. 2, the answer is no for the time being, and the signal processing returns to step 102 through step 105, but eventually the eye opening speed at time T2. Rises (decreases as a downward speed) across the threshold Vd, and the answer in step 107 changes from no to yes, so the signal processing proceeds to step 108 and the flag F12 is set to 1 at this moment. The time is stored as time T2. From this, the signal processing returns to step 102 through step 105, but since the flag F12 is set to 1, the signal processing thereafter proceeds from step 106 to step 109.

In step 109, it is determined whether the flag F13 is 1. The flag F13 is set to 1 in step 111, which will be described later, and has been reset to 0 until then, so the answer to step 109 is no for the moment, and the signal processing proceeds to step 110, where the differential value f(T). Is determined to be equal to or greater than a predetermined threshold Vu. If the threshold value Vu is set to an appropriate positive value as shown in the middle part of FIG. 2, the answer is no for the time being and the signal processing returns to step 102 via step 105, but eventually the eye opening speed at time T3. Rises across the threshold Vu and the answer in step 110 turns from no to yes, so signal processing now proceeds to step 111, flag F13 is set to 1 and the instant time is stored as time T3. .. From this, the signal processing returns to step 102 through step 105, but since the flag F13 is set to 1, the signal processing thereafter proceeds from step 109 to step 112.

In step 112, it is determined whether the differential value f(T) is equal to or greater than the threshold value Vu. The answer is yes for a while, and the signal processing returns to step 102 through step 105, but at time T4, the eye opening speed decreases across the threshold value Vu, and the answer to step 112 changes from yes to no. Then, the signal processing proceeds to step 113, the flag F14 is set to 1, and the time at this moment is stored as the time T4.

As described above, time points T1 and T2 at which the eye opening speed crosses the threshold value Vd and time points T3 and T4 at which the eye opening speed crosses the threshold value Vu are obtained. Since the flag F14 is set to 1, the signal processing proceeds from step 105 to step 114. In step 114, the intermediate value between the time points T1 and T2 at the valley of the eye opening speed is set as the start time point Tns of the n-th blink (half blink), and the intermediate value between the time points T3 and T4 at the mountain of the eye opening speed is the end point thereof. Tns. The signal processing then proceeds to step 115, resets the flags F11, F12, F13 and F14 to 0 and ends.

Even when the signal processing of the subroutine B shown in FIG. 6 is added to the signal processing shown in FIG. 3, the low-pass filtered data of the eye opening degree G is stored in step 2-1 and the signal processing of the subroutine B is performed. When started, in step 201, the eye opening degree G between the time points T1 and T2 stored in step 2-1 is second-order differentiated in time to calculate a differential value g(T) representing the time-dependent eye opening acceleration. It The time course thereof is as shown in the lower part of FIG.

The signal processing then proceeds to step 202, and it is determined whether the flag F21 is 1 or not. Since the flag F21 is set to 1 in step 204 described later and reset to 0 until then, the answer to step 202 is no for the moment, and the signal processing proceeds to step 203 and the second-order differential value g( It is determined whether T) is greater than or equal to a predetermined threshold Ad. The threshold value Ad is a negative value as shown in the lower part of FIG. Initially, the answer is yes, and signal processing bypasses step 204 and proceeds to step 205 where it is determined whether flag F28 is one. Since the flag F28 is set to 1 in step 225 which will be described later and has been reset to 0 until then, the answer to step 205 is no for the moment, and the signal processing returns to step 202.

When the downward acceleration of the eyelids increases and falls below the threshold value Ad (increases as the downward acceleration), the answer in step 203 changes from yes to no, so the signal processing proceeds to step 204 and the flag F21 is set to 1 , And the time at this moment is stored as time T1. From this, the signal processing returns to step 202 via step 205, but since the flag F21 is set to 1, the signal processing thereafter proceeds from step 202 to step 206.

At step 206, it is judged if the flag F22 is 1 or not. Since the flag F22 is set to 1 in step 208 described later and reset to 0 until then, the answer to step 206 is no for the moment, and the signal processing proceeds to step 207 and the second derivative value g( It is determined whether T) is greater than or equal to a predetermined threshold Ad. If the threshold value Ad is set to an appropriate negative value as shown in the lower part of FIG. 2, the answer is no for the time being, and the signal processing returns to step 202 via step 205, but eventually the eye opening acceleration at time T2. Rises (decreases as downward acceleration) across the threshold Ad, and the answer in step 207 turns from no to yes, so the signal processing proceeds to step 208 and the flag F22 is set to 1 at this moment. The time is stored as time T2. From this, the signal processing returns to step 202 through step 205, but since the flag F22 is set to 1, the signal processing thereafter proceeds from step 206 to step 209.

In step 209, it is determined whether the flag F23 is 1 or not. Since the flag F23 is set to 1 in step 211 described later and reset to 0 until then, the answer to step 209 is no for the moment, and the signal processing proceeds to step 210 and the second-order differential value g( It is determined whether T) is greater than or equal to a predetermined threshold Au. If the threshold Au is set to an appropriate positive value as shown in the lower part of FIG. 2, the answer is no for the time being and the signal processing returns to step 202 via step 205, but eventually the eye opening acceleration at time T3. Rises across the threshold Au and the answer in step 210 changes from no to yes, so signal processing now proceeds to step 211, flag F23 is set to 1 and the instant time is stored as time T3. .. From this, the signal processing returns to step 202 through step 205, but since the flag F23 is set to 1, the signal processing thereafter proceeds from step 209 to step 212.

In step 212, it is determined whether the flag F24 is 1. Since the flag F24 is set to 1 in step 214 described later and has been reset to 0 until then, the answer to step 212 is no for the moment, and the signal processing proceeds to step 213 and the second differential value g( It is determined whether T) is greater than or equal to a predetermined threshold Au. If the threshold value Au is set to an appropriate positive value as shown in the lower part of FIG. 2, the answer is yes for a while and the signal processing returns to step 202 via step 205, but eventually the eye-opening acceleration at time T4. Falls below the threshold Au and the answer in step 213 turns from yes to no, so signal processing now proceeds to step 214, flag F24 is set to 1 and the instant time is stored as time T4. .. From this, the signal processing returns to step 202 via step 205, but since the flag F24 is set to 1, the signal processing thereafter proceeds from step 212 to step 215.

In step 215, it is determined whether the flag F25 is 1. Since the flag F25 is set to 1 in step 217 which will be described later and has been reset to 0 until then, the answer to step 215 is no for the moment, and the signal processing proceeds to step 216 and the second-order differential value g( It is determined whether T) is greater than or equal to a predetermined threshold Au. If the threshold value Au is set to an appropriate positive value as shown in the lower part of FIG. 2, the answer is no for the time being, and the signal processing returns to step 202 via step 205, but eventually the eye-opening acceleration at time T5. Increases across the threshold Au, and the answer in step 216 turns from no to yes, so signal processing now proceeds to step 217 where flag F25 is set to 1 and the instant time is stored as time T5. .. From this, the signal processing returns to step 202 via step 205, but since the flag F25 is set to 1, the signal processing thereafter proceeds from step 215 to step 218.

In step 218, it is determined whether the flag F26 is 1 or not. Since the flag F26 is set to 1 in step 220, which will be described later, and has been reset to 0 until then, the answer to step 218 is no for the moment, and the signal processing proceeds to step 219 and the second-order differential value g( It is determined whether T) is greater than or equal to a predetermined threshold Au. If the threshold value Au is set to an appropriate positive value as shown in the lower part of FIG. 2, the answer is yes for a while and the signal processing returns to step 202 through step 205, but eventually the eye-opening acceleration at time T6. Falls below the threshold Au and the answer in step 219 turns from yes to no, so signal processing now proceeds to step 220, flag F26 is set to 1 and the instant time is stored as time T6. .. From this, the signal processing returns to step 202 via step 205, but since the flag F26 is set to 1, the signal processing thereafter proceeds from step 218 to step 221.

In step 221, it is determined whether the flag F27 is 1 or not. Since the flag F27 is set to 1 in step 223 described later and reset to 0 until then, the answer to step 221 is no for the moment, and the signal processing proceeds to step 222 and the second-order differential value g( It is determined whether T) is greater than or equal to a predetermined threshold Ad. If the threshold value Ad is set to an appropriate negative value as shown in the lower part of FIG. 2, the answer is yes for a while, and the signal processing returns to step 202 through step 205, but eventually the eye-opening acceleration at time T7. Decreases across the threshold Ad (increases as downward acceleration), and the answer in step 222 changes from yes to no, so the signal processing proceeds to step 223 and the flag F27 is set to 1 at this moment. Is stored as time T7. From this, the signal processing returns to step 202 via step 205, but since the flag F27 is set to 1, the signal processing thereafter proceeds from step 221 to step 224.

In step 224, it is determined whether the second-order differential value g(T) is equal to or larger than a predetermined threshold Ad. If the threshold value Ad is set to an appropriate positive value as shown in the lower part of FIG. 2, the answer is no for the time being and the signal processing returns to step 202 via step 205, but eventually the eye opening acceleration at time T8. Increases across the threshold Ad (decreases as downward acceleration) and the answer in step 224 turns from no to yes, so signal processing now proceeds to step 225 and flag F28 is set to 1 at this moment. The time is stored as time T8. From this, the signal processing proceeds to step 205, and since the flag F28 is set to 1, the signal processing proceeds to step 226.

As described above, time points T1 and T2 where the eye opening acceleration crosses the threshold value Ad, time points T3 and T4 when the threshold value Vu crosses, time points T5 and T6 when the threshold value Vu crosses again, and time points T7 and T8 when the threshold value Ad crosses again. In step 226, the intermediate value between the time points T1 and T2 at the first valley of the eye opening acceleration and the time points T3 and T4 at the first mountain are used as the start time point Tns of the n-th blink (half-blink), and the eye opening is performed. An intermediate value between the time points T5 and T6 that hit the second peak of the acceleration and an intermediate value between time points T7 and T8 that hit the second valley are set as the end time point Tns. The signal processing is ended by resetting the flags F21, F22, F23, F24, F25, F26, F27 and F28 to 0 in step 227.

Although the present invention has been described in detail above with reference to some embodiments, it will be apparent to those skilled in the art that various modifications can be made to such embodiments within the scope of the present invention.

Claims (1)

In the blink detection device that detects a blink based on a time versus eye opening degree of the data, and sets the opening eye of a relatively low first threshold value and opening the eye of a relatively high second threshold value, the eye opening degree is It was determined that there was a normal blink when crossing the first threshold above the low level after crossing the first threshold below the high level, and the degree of eye opening crossed the second threshold below the high level. Later it is determined that there was a half-blink when crossing the second threshold higher than low without crossing the first threshold below high, the determination that there was a half-blink is the time vs. eye opening. Based on the data of time versus eye opening speed obtained by temporally differentiating the data of, the eye opening speed crosses a predetermined negative threshold below high and then crosses the negative threshold above low, and A blink detection device, characterized in that it is confirmed by crossing a predetermined positive threshold value higher than low and then crossing the positive threshold value lower than high .

Images