JP2002279411A - Device for detecting attentiveness in driving and device for detecting dozing at the wheel - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve accuracy of detecting driving attentiveness and dozing at the wheel. SOLUTION: A TV camera 7 images the face image of driver 5 and a circuit for detecting eye position 15 detects an eye position from the overall image. For each imaged data, eye detection by the circuit 15 is determined to be successful or not and the number of successes is counted up and then an operational circuit of OK time rate for eye detection 19 calculates a number of counted up OK eye detection for a predetermined time measured by a timer, that is a rate of OK eye detection. From the calculated results of the rate of OK eye detection, a circuit for detecting attentiveness in driving 21 detects attentiveness in driving of the driver.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a driving attention detection device and a drowsy driving detection device for processing driver's face image data to detect a driving attention and a dozing state.

[0002]

2. Description of the Related Art For example, Japanese Patent Application Laid-Open Nos. 10-40361 and 10-1436 disclose conventional dozing driving detecting devices.
There is one described in JP-A-69. These publications detect the position of the eyes from the driver's face image,
After that, open / closed eye detection is performed while tracking the eyes, and it is determined that the time during which the eyes are closed has exceeded a predetermined value, and the dozing state is detected.

[0003]

However, in the above-described conventional technique, the face movement becomes large and fast under a situation where driving attention is high and safety confirmation is frequently performed. It becomes difficult to keep catching correctly, and eye detection accuracy and eye tracking accuracy deteriorate. For this reason, when considered per unit time, it is inevitable to perform dozing detection using a small open / closed eye determination result or an erroneous eye opening value, and it becomes impossible to maintain high dozing detection accuracy, and a false alarm is generated. there's a possibility that.

[0004] Therefore, an object of the present invention is to improve driving attention and dozing detection accuracy.

[0005]

According to a first aspect of the present invention, there is provided a driving attention detecting apparatus for detecting driving attention by processing image data of a driver's face. Photographing means for photographing the face image of the person, eye position detecting means for detecting the position of the eye from the entire image photographed by the image photographing means, and said eye position detecting means for each of the photographed image data. Determining whether the eye detection by the eye is successful, counting the number of successful eye detection OK number integrating means, a predetermined time measuring means for measuring a predetermined time, and a predetermined time measurement value by the predetermined time measuring means An eye detection OK rate calculation means for calculating an eye detection OK rate based on the eye detection OK number integration value by the eye detection OK number integration means, and a driver based on the calculation result of the eye detection OK rate calculation means. of It is constituted to have a driving attentiveness detecting means for detecting the willpower.

According to a second aspect of the present invention, there is provided a driving attention detecting apparatus for detecting driver's attention by processing image data of a driver's face. An eye position detecting means for detecting the position of the eye from the entire image taken by the photographing means, and an area including at least one eye based on the eye position detected by the eye position detecting means from the entire image is set. A predetermined area setting means, a predetermined area inner eye position detecting means for detecting an eye position in a predetermined area set by the predetermined area setting means, and a position of the predetermined area inner eye for each of the photographed image data. The detection means determines whether or not the eye detection is successful, and counts the number of successful eye detection OK number integration means, a predetermined time measurement means for measuring a predetermined time, and the predetermined time measurement means. A detection OK ratio calculating means of the eye for calculating a detection OK rate of the eye based on the detected OK number cumulative value of the eye by detecting OK number integrating means of the eye with respect to a predetermined time measurement value,
Driving attention detection means for detecting the driver's attention from the calculation result of the eye detection OK rate calculation means is provided.

According to a third aspect of the present invention, in the configuration of the second aspect, the setting of the predetermined area by the predetermined area setting means is performed by:
The configuration is such that release means for releasing by manual operation is provided.

According to a fourth aspect of the present invention, in the configuration of the second aspect, the setting of the predetermined area by the predetermined area setting means is performed.
The configuration is such that release means is provided for releasing based on the calculation result of the eye detection OK rate by the eye detection OK rate calculation means.

According to a fifth aspect of the present invention, there is provided a driving attention detecting apparatus for detecting driver's attention by processing image data of a driver's face, and an image photographing means for photographing the driver's face image. An eye position detecting means for detecting the position of the eye from the entire image taken by the photographing means, and an eye tracking area set based on the eye position detected by the eye position detecting means; Eye tracking processing means for updating the eye tracking area every time, and eye tracking for accumulating the eye tracking time while checking whether the eye tracking processing by the eye tracking processing means is performed correctly. A time integrating means, a predetermined time measuring means for measuring a predetermined time, and an eye tracking time rate based on an eye tracking time integrated value by the eye tracking time integrating means with respect to a predetermined time measured value by the predetermined time measuring means. Eye It is constituted to have the tracking time ratio calculating means, and a driving attentiveness detecting means for detecting the driver's attention from the operation result of tracking time rate calculation means of the eye.

According to a sixth aspect of the present invention, in the configuration according to any one of the first to fifth aspects, a signal of a vehicle state detection sensor is input to the driving attention detecting means, and the driving attention is adjusted according to the running state of the vehicle. The configuration is such that the criterion is switched.

According to a seventh aspect of the present invention, in the configuration according to any one of the first to sixth aspects, the stored contents of the personal data are input to the driving attention detecting means, and the driving attention is changed according to the personal information of the driving behavior. The configuration is such that the criterion is switched.

According to an eighth aspect of the present invention, in the configuration according to any one of the first to seventh aspects, the signal of the cabin light environment measuring sensor is input to the driving attention detecting means to reduce the deterioration of the light environment in the vehicle compartment. The configuration is such that it is estimated to determine that an eye cannot be detected.

According to a ninth aspect of the present invention, there is provided a drowsiness driving detecting apparatus for processing a driver's face image data to detect a drowsiness driving, wherein the image capturing means captures an image of the driver's face, and the image capturing means. Eye position detection means for detecting the position of the eye from the entire image captured, and for each of the captured image data, determine whether the eye position detection means has successfully detected the eye, the number of times of success Means for counting the number of detected eyes of the eye for counting up, means for measuring a predetermined time for measuring a predetermined time, and integration of the number of OKs for detecting the number of eyes of the eye by the means for integrating the number of detected eyes of the eye with respect to a predetermined time measured by the predetermined time An eye detection OK rate calculating means for calculating an eye detection OK rate based on the value; a driving attention detecting means for detecting a driver's attention from a calculation result of the eye detection OK rate calculating means; An eye opening value calculating means for calculating an eye opening value when the eye position is successfully detected by the position detecting means; and an open / closed eye determination using the value obtained by the eye opening value calculating means. And a drowsiness driving detecting means for detecting a drowsiness according to the driving attention detected by the driving attention detecting means based on a result of the open / closed eyes determination by the opening / closing eyes determining means. .

According to a tenth aspect of the present invention, there is provided a drowsiness driving detecting apparatus for processing a driver's face image data to detect a drowsiness driving, wherein the image capturing means captures an image of the driver's face and the image capturing means. Eye position detecting means for detecting the position of the eye from the entire image captured by the user, and a predetermined area for setting an area including at least one eye based on the eye position detected by the eye position detecting means from the entire image Setting means;
Eye position detection means for detecting the position of the eye within the predetermined area set by the predetermined area setting means, and eye detection by the predetermined area inside eye position detection means for each of the photographed image data Is determined, whether the number of successful detections of the eye is counted up, a predetermined time measuring means for measuring a predetermined time, and the eye with respect to a predetermined time measurement value by the predetermined time measuring means. An eye detection OK rate calculation means for calculating an eye detection OK rate based on the eye detection OK number integration value by the detection OK number integration means, and a driver's attention based on the calculation result of the eye detection OK rate calculation means. A driving attention detecting means for detecting, an eye opening value calculating means for calculating an eye opening value when the eye is successfully detected by the eye position detecting / determining means in the predetermined area, and an eye opening degree Value calculation means Eye open / closed eye determination means for performing open / closed eye determination using the value obtained by the method, and a dozing detection corresponding to the driving attention detected by the driving attention detecting means based on the result of the open / closed eye determination by the eye open / close determination means And a drowsiness detecting means for performing the dozing operation.

According to an eleventh aspect of the present invention, there is provided a drowsiness driving detecting apparatus for processing a driver's face image data to detect a drowsiness driving, wherein the image capturing means captures the driver's face image, and the image capturing means. Eye position detection means for detecting the position of the eye from the entire image taken by, and set the eye tracking area based on the eye position detected by this eye position detection means,
Eye tracking processing means for updating the eye tracking area for each of the photographed image data, and integrating the eye tracking time while confirming whether the eye tracking processing by the eye tracking processing means is performed correctly. Eye tracking time integrating means,
A predetermined time measuring means for measuring a predetermined time; and an eye tracking time for calculating an eye tracking time rate based on an eye tracking time integrated value by the eye tracking time integrating means with respect to a predetermined time measured value by the predetermined time measuring means. Rate calculating means, driving attention detecting means for detecting a driver's attention from the calculation result of the eye tracking time rate calculating means, and an eye opening value when the eye tracking processing is performed correctly. Based on the open / closed eye determination unit that performs open / closed eye determination using the value obtained by the calculated eye openness value calculation unit, the value obtained by the eye openness value calculation unit, And a drowsiness detecting means for detecting a drowsiness according to the driving attention detected by the driving attention detecting means.

[0016]

According to the first aspect of the invention, since the driving attention is detected by calculating the eye detection OK rate in a predetermined time, it is possible to determine the state in which the face frequently moves by safety confirmation or the like. It is possible to detect driving attention with high accuracy.

According to the second aspect of the present invention, the eye position detection from the entire image is limited to the initial state, and the subsequent processing is performed within a predetermined area including the detected eye. Can be performed in a short time, and the eye detection accuracy can be improved.

According to the third aspect of the present invention, the release means for manually releasing the setting of the predetermined area by the predetermined area setting means is provided so that the position of the eye from the entire image can be arbitrarily detected. It is possible to cope with a case where the position of the eyes is shifted due to the collapse of the posture of the driver or the replacement of the driver.

According to the fourth aspect of the present invention, there is provided a canceling means for canceling the setting of the predetermined area by the predetermined area setting means based on the calculation result of the eye detection OK rate, so that the eye position detection from the entire image is automatically performed. Since it can be performed, it is possible to cope with a case where the position of the eyes is shifted due to the driver's posture collapse or the driver's replacement.

According to the fifth aspect of the present invention, the eye position detection from the entire image is limited to the initial state, and the subsequent processing is switched to the eye tracking processing, so that the eye detection processing is performed in a short time. The eye tracking time rate having the same meaning as the eye detection OK rate used in claims 1 and 2 can be calculated with high accuracy.

According to the sixth aspect of the present invention, the signal of the vehicle state detection sensor is input to the driving attention detecting means, and the driving attention judgment reference is switched according to the traveling state of the vehicle. It is possible to distinguish between a case where the frequency of confirmation is high and a case where the frequency of safety confirmation is relatively low such as traveling on a highway, and it is possible to further improve the detection accuracy of driving attention.

According to the invention of claim 7, since the stored contents of the personal data are inputted to the driving attention detecting means and the driving attention judgment criteria are switched according to the personal information of the driving behavior, the driving attention can be changed in various driving scenes. It is possible to cope with individual differences in the driver's safety confirmation frequency and the confirmation method according to the driver, and it is possible to further improve the detection accuracy of driving attention.

According to the eighth aspect of the present invention, the signal from the cabin light environment measuring sensor is input to the driving attention detecting means, and the deterioration of the light environment in the car cabin is estimated to determine that the eye cannot be detected. As a result, the detection accuracy of driving attention can be further improved.

According to the ninth aspect of the present invention, since the driving OK is detected by calculating the eye detection OK rate in a predetermined time, it is possible to determine a state in which the face is frequently moving by safety confirmation, etc. Driving attention can be detected with a high degree of accuracy, and a drowsiness detection can be performed in accordance with the driving attention. By increasing the dozing detection level when is decreased, the dozing detection accuracy can be improved.

According to the tenth aspect, the position detection of the eyes from the entire image is limited to the initial state, and the subsequent processing is performed within a predetermined area including the detected eyes. Can be performed in a short time, eye detection accuracy can be improved, and dozing detection according to driving attention can be performed. Is reduced, and when the frequency of movement of the face decreases, the dozing detection level can be increased to improve the dozing detection accuracy.

According to the eleventh aspect of the invention, the eye position detection from the entire image is limited to the initial state, and the subsequent processing is switched to the eye tracking processing, so that the eye detection processing is performed in a short time. It is possible to calculate, with high accuracy, an eye tracking time rate having the same meaning as the eye detection OK rate used in claims 1 and 2, and to detect a drowsiness according to driving attention. Since it is possible to do so, it is possible to improve the dozing detection accuracy by lowering the dozing detection level when the frequency of movement of the face is high in safety confirmation or the like and raising the dozing detection level when the frequency of moving the face is low.

[0027]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is an overall configuration diagram of a driving attention detecting device according to a first embodiment of the present invention. The instrument panel 3 of the automobile 1 is provided with a TV camera 7 as image photographing means for photographing the face of the driver 5 from the front. The TV camera 7 includes an A / D converter 9, an image memory 11, an image data calculation circuit 13, an eye position detection circuit 15 as eye position detection means, an eye detection OK number integrating means, and an eye detection OK rate. Eye detection OK including calculation means
A time rate calculation circuit 19 and a driving attention detecting circuit 21 as driving attention detecting means are connected in order.

An analog image captured by the TV camera 7 is input to the image memory 11 as digital input image data via the AD converter 9 and stored.

The image data operation circuit 13 includes the image memory 1
Pre-processing is performed on the image data stored in No. 1 to convert the data into a state suitable for computer processing. The eye position detection circuit 15 detects the position of the eye of the driver 5 on the image data converted by the image data calculation circuit 13. The eye detection OK time rate calculation circuit 19 calculates the eye detection OK time rate of each image data taken in time series. The driving attention detection circuit 21 detects the driving attention based on the calculation result of the eye detection OK time rate calculation circuit 19.

The image data calculation circuit 13, the eye position detection circuit 15, the eye detection OK time rate calculation circuit 19, and the driving attention detection circuit 21 are constituted by microcomputers including a CPU, a ROM, a RAM, and the like. This microcomputer has a built-in timer (not shown) as a predetermined time measuring means for measuring a predetermined time.

Next, the flow of the operation of the driving attention detecting device having the above configuration will be described with reference to the flowchart of FIG. 2 and FIGS.
This will be described with reference to FIG. An image captured by the TV camera 7 is input to the image data calculation circuit 13 via the A / D converter 9 and the image memory 11. Here, a timer for measuring a predetermined time is started (step 201),
Eye position detection from the entire image is performed by the eye position detection circuit 15, and whether or not the detection is successful is determined by the eye detection OK time rate calculation circuit 19 (step 202).

An example of eye position detection from the whole image will be described with reference to FIG. 3. As shown in FIG. 3A, when the driver 5 gazes ahead of the vehicle, there is an area where the eye exists. The position of the eye can be detected by defining the shape of the eye and the relative positional relationship with the eyebrow, nose, mouth, and the like.
On the other hand, as shown in FIG. 3B, when the driver 5 is looking at a place other than the front of the vehicle due to safety confirmation or the like, eyes cannot be detected under the above-defined conditions.

If the eye position has been successfully detected from the entire image in step 202, the eye detection OK counter in the eye detection OK time rate calculation circuit 19 is counted up (step 203). Conversely, when the eye position detection fails in step 202, it is determined that the driver 5 has not looked forward, and it is determined whether a predetermined time by a timer has elapsed (step 204).

If the predetermined time has not elapsed, the flow returns to step 202 to repeat the same processing.
If it is determined in step 204 that the predetermined time has elapsed, the eye detection OK rate determined based on the eye detection OK integrated value within the predetermined time is calculated by the eye detection OK time rate calculation circuit 19. (Step 205), and the driving attention of the driver 5 is detected using the calculation result (Step 20).
6).

A method for detecting the driving attention will be described with reference to FIGS. FIG. 4 shows an eye detection result for each frame of the predetermined section N. What is displayed as "OK" is a case where the driver 5 is looking forward as shown in FIG. 3A, and the position of the eye can be detected without delay, and is displayed as "NG". FIG. 3B shows a case where the driver 5 is in a state of looking at something other than the front as shown in FIG.

The calculation of the eye detection OK rate within the predetermined time is performed by integrating the number of times of the eye position detection OK within the predetermined time. FIG. 5 shows the time-series change. The horizontal axis in FIG. 5 indicates the elapsed time for each predetermined section, and the vertical axis indicates the OK detection frequency of the eye within the predetermined section.

FIG. 5 shows the predetermined section N in FIG. 4 in terms of the OK detection frequency of the eye. An eye detection OK rate in a predetermined section as shown in FIG. 5 (eye detection OK in a predetermined section)
By using the frequency), found time ratio that the driver 5 gazing at the front, therefore, is in a state in which the driver 5 is performed always safety check paying attention to the surroundings detection OK rate of the eye, 100 It does not show a value close to%.

However, in an environment where the degree of awakening of the driver 5 is reduced and the frequency of safety confirmation is reduced, since the face movement is reduced, the state in which the detection OK rate of the eyes in the predetermined section continues to be high. Therefore, it can be determined that the driver's 5 attention is reduced, and in this case, it can be said that the driver 5 is in an environment where the driver easily falls asleep. When the driver's attention is lowered in this way, a warning is output to the driver 5 by sound or display display or the like.

After the driving attention is detected in step 206 in FIG. 2, the OK detection counter for the eyes is cleared and the timer is reset (step 207). Move on to detection of attention.

According to the above-described first embodiment, since the driving attention of the driver 5 is detected after calculating the eye detection OK frequency in the predetermined section N, the face is frequently checked by safety confirmation or the like. Can be determined, and the driving attention can be detected with high accuracy.

Next, the operation flow of the driving attention detecting device according to the second embodiment of the present invention will be described with reference to the flowchart shown in FIG. 6 and the explanatory diagram of FIG. This second
1 has the same configuration as that of FIG. 1 according to the first embodiment, but in this case, the eye position detection circuit 15 includes at least one eye based on the position of the eye detected from the entire image. The apparatus includes a predetermined area setting means for setting an area, and an eye position detecting means for detecting an eye position in the predetermined area set by the predetermined area setting means.

The flowchart in FIG. 6 is different from the flowchart in FIG. 2 in that steps 602 and 603 are added to limit the eye position detection processing from the entire image to the initial state. By setting the predetermined area S including the eyes while looking forward as shown in Fig. 3 and limiting the eye detection range, the speed of the eye position detection process is increased and the eye detection accuracy is improved. Let me.

Here, the contents of the steps of adding and changing the flowchart of FIG. 6 with respect to the flowchart of FIG. 2 will be described. After the program is started, a timer for measuring a predetermined time is started in the same manner as in the flowchart of FIG. 2 (step 601), and the eye position is detected from the entire image by the eye position detection circuit 15 (step 60).
2) The predetermined area S is set by the eye position detection circuit 15 based on the detected eye position (step 603).

Thereafter, the eye position detection is repeated for the set predetermined area S, and it is determined by the eye detection OK time rate calculation circuit 19 whether the eye position detection in the predetermined area S has succeeded. (Step 604). Here, if successful detection of the position of the eye, by performing the count-up of the detection OK counter eye (step 605) to determine whether a predetermined time has elapsed (step 606), failure to detect the position of the eyes in opposite If so, the process proceeds to a determination as to whether a predetermined time has elapsed (step 606). Subsequent processing is the same operation as the flowchart of FIG.
Description is omitted.

As described above, according to the second embodiment, the position detection of the eyes from the entire image is limited to only the initial state, and the subsequent processing is performed within the predetermined area S including the detected eyes. The eye position detection process can be performed in a short time, and the eye detection accuracy can be improved.

Next, application example 1 of the second embodiment described above.
Will be described with reference to the flowchart of FIG. The flowchart of FIG. 8 adds step 804 to the flowchart of FIG. As a configuration,
A reset switch is provided as canceling means for canceling (cancelling) the setting of the predetermined area S by manual operation. By canceling the setting of the predetermined area S, it is possible to cope with a case where the position of the eyes is shifted due to the collapse of the posture of the driver 5 or the replacement of the driver 5.

Here, the contents of additional steps of the flowchart of FIG. 8 with respect to the flowchart of FIG. 6 will be described. By operating the reset switch when the posture of the driver 5 is lost or when the driver 5 is replaced, the operation of the reset switch is recognized in step 804, and the process returns to step 802 to return the eye from the entire image. Is performed again, and the process proceeds to the setting of the predetermined area S including the eyes (step 803).

Next, the operation flow of the application example 2 of the second embodiment will be described with reference to the flowcharts of FIGS. 9 and 10. The flowcharts of FIGS. 9 and 10 are different from the flowchart of FIG.
8, Step 1002 and Step 1003 are added,
By allowing the setting of the predetermined area to be automatically canceled (canceled) based on the calculation result of the eye detection OK rate, the position of the eye shifts due to the collapse of the posture of the driver 5 or the replacement of the driver 5. It can respond in the event that it happens. That is, in this case, detection OK time rate calculation circuit 19 of the eye includes releasing means for releasing the setting of the predetermined region S.

Here, the contents of additional steps of the flowcharts of FIGS. 9 and 10 with respect to the flowchart of FIG. 6 will be described. When the posture of the driver 5 has collapsed or when the driver 5 has been replaced, the fact that the eye does not constantly enter the predetermined area S that has already been set is detected at predetermined time intervals. It is determined whether the rate is 0 or not (step 1002). When it is determined that the eye detection OK rate is 0,
The number of occurrences of the situation is counted up (step 10).
03).

If it is determined that the number of occurrences of the predetermined time when the above-mentioned eye detection OK rate is 0 has exceeded the predetermined value (step 904), the eye detection OK rate "0" counter is cleared (step 904). 908), the position of the eye from the entire image is detected again (step 902), and the process proceeds to the setting of the predetermined area S including the eye (step 903).

FIG. 11 is an overall configuration diagram of a driving attention detecting device according to a third embodiment of the present invention. This device is an eye detection OK time rate calculation circuit 1 in the configuration of FIG.
Instead of 9, an eye tracking time rate calculation circuit 23 is provided as eye tracking time rate calculation means. The eye tracking time rate calculation circuit 23 updates the eye tracking area for each captured image data, and determines whether the eye tracking processing by the eye tracking processing means is performed correctly. An eye tracking time integrating means for integrating the eye tracking time while checking is included. The eye tracking time rate calculating means calculates an eye tracking time rate based on an eye tracking time integrated value by the eye tracking time integrating means with respect to a predetermined time measurement value of a timer which is a predetermined time measuring means.

Next, the operation flow of the driving attention detecting device according to the third embodiment will be described with reference to the flowchart of FIG. 12 and the explanatory diagrams of FIGS. 13, 14, and 15. First, a timer for measuring a predetermined time is started (step 1101), and it is determined whether the predetermined time has elapsed (step 1102). If it is determined that the predetermined time has not elapsed, the eye tracking time rate calculation circuit 23 determines whether or not an eye tracking area is set (step 1103).

The eye tracking area is defined as a horizontally elongated small area T including one eye as shown in FIG. The tracking area T has a size that can respond to slight movement of the eye position such as vibration of the vehicle. If it is determined in step 1103 that the eye tracking area has not been set, an eye position detection process from the entire image is performed (step 1104).

Subsequently, it is determined whether or not the eye position has been correctly detected (step 1105). For example, when the driver is looking sideways, the eye detection is not normally performed normally, so the process returns to step 1102 to perform the eye position detection processing for the next image data. Conversely, if the eye is correctly detected in step 1105, an eye tracking area is set (step 1106).

Here, an eye tracking method will be described with reference to FIG. Immediately after the detection of the eye position, that is, in the first frame, as shown in FIG. 13A, an eye tracking area is set with the detected eye position at the center. Thereafter, it is determined whether the tracking state of the eye is normal (step 110).
7) If it is determined to be normal, the tracking time of the eye is integrated (step 1108), and the process returns to step 1102.

Once the eye tracking area is set, the processing from step 1102 to step 1103 to step 1107 is repeated. One rotation of this loop means that the next captured image data, that is, eye tracking has been started and the processing of the second frame has been started.
The result is as shown in FIG. The eye tracking area T in FIG. 13B is located at the position set in the first frame, whereas the current eye position is image data captured in the second frame. As a result, the center point of the eye is shifted from the tracking area T of the eye. However, eye tracking can be performed as long as the eye is within the eye tracking area T.

Thereafter, if the eye tracking processing is performed normally, the same processing is repeated. FIG. 13 (c),
(D) shows the positional relationship between the eye position and the eye tracking region T in the face image data captured in the third and fourth frames, respectively. Step 110
If it is determined in step 7 that the eye tracking is not performed correctly, the eye tracking area T is cleared (step 1109).
Return to step 1102.

It is determined whether or not the eye has been tracked correctly by determining, in the eye tracking area T, the characteristic amount (size, shape, density value, etc.) of the eye when the driver 5 is looking ahead of the vehicle. This can be done based on whether or not the data possessed exists. In step 1109, the tracking area of the eye is cleared, and step 1102 is performed.
Returning to step 1103, when the process proceeds to step 1103, the process proceeds to step 1104 to perform the eye position detection process from the entire image again.

When this series of processing is repeated,
If it is determined in step 1102 that the time measured by the timer has elapsed, the eye tracking time rate in the predetermined time is calculated by the eye tracking time rate calculation circuit 23 (step 1110). This calculation method will be described with reference to FIG.

Until the elapse of the predetermined time is determined in step 1102, the eye position detection loop A which turns when the eye is lost even if the eye enters the initial state or the eye tracking state.
Then, there is an eye tracking loop B that turns when the eye detection is successful and the eye tracking is performed normally. In the white and black bars shown in FIG. 14, the white portion indicates the time during loop A in the eye detection process, and the black portion indicates the loop B in the eye tracking process. Time.

In step 1108 of FIG. 12, the eye tracking time of a predetermined section is integrated, and step 1110 is performed.
Calculates the ratio of the integrated value of the eye tracking time to the predetermined time. After calculating the eye tracking time rate for the predetermined time, the driving attention is detected (step 1111). This detection processing is based on the change in the eye tracking time rate in the elapsed time for each predetermined section shown in FIG. Is going.

As in the first embodiment, when the awakening degree of the driver 5 decreases, the frequency of safety confirmation and the like decreases and the face movement decreases. Are getting higher. Thereafter, the integrated value of the eye tracking time is cleared, the timer is reset (step 1112), and the eye tracking time ratio for each predetermined time is calculated.

According to the third embodiment described above, eye position detection from the entire image is limited to only the initial state, and the subsequent processing is switched to eye tracking processing. Therefore, the eye tracking time rate having the same meaning as the eye detection OK rate used in the first embodiment can be calculated with high accuracy.

FIG. 16 is an overall configuration diagram of a driving attention detecting device according to a fourth embodiment of the present invention. In this embodiment, a vehicle speed sensor, accelerator, brake, and steering sensors for detecting the operation state of the driver 5 and a vehicle state detection sensor 25 including a navigation system are added to the configuration of the first embodiment. Things. By inputting a signal from the vehicle state detection sensor 25 to the driving attention detection circuit 21, it is possible to distinguish between a city area traveling with a high frequency of safety confirmation and a highway traveling relatively infrequently with a safety confirmation frequency, thereby driving the vehicle. It is possible to further improve the detection accuracy of the attention.

FIG. 17 is an overall configuration diagram of a driving attention detecting device according to a fifth embodiment of the present invention. In this embodiment, a personal data memory 27 is added to the configuration of the fourth embodiment shown in FIG. The signal of the vehicle state detection sensor 25 and the contents stored in the personal data memory 27 are input to the driving attention detection circuit 21 so that the driving attention determination criterion can be switched according to the driving state of the vehicle and personal information of the driving behavior. And the driver 5 according to each driving scene
By responding to individual differences in the frequency and method of safety confirmations,
Further, it is possible to improve the detection accuracy of driving attention.

In the configuration shown in FIG. 17, only the contents stored in the personal data memory 27 may be input to the driving attention detecting circuit 21 without providing the vehicle state detecting sensor 25.

FIG. 18 is an overall configuration diagram of a driving attention detecting device according to a sixth embodiment of the present invention. This embodiment differs from the configuration of the fourth embodiment shown in FIG. 16 in that a vehicle interior light environment including an illuminance sensor, a barometer for estimating the weather, a compass, a clock with a calendar function for grasping the date and time, and the like. The sensor 29 is added. The signal of the vehicle state detection sensor 25 and the signal of the vehicle interior light environment sensor 29 are input to the driving attention detection circuit 21, and the deterioration of the light environment in the vehicle interior is estimated in accordance with the running state of the vehicle. By determining that the cause of the failure is other than the movement of the face of the driver 5, it is possible to further improve the detection accuracy of the driving attention.

A vehicle interior light environment sensor 29 is additionally provided in the configuration of FIG.
The contents stored in the personal data memory 27 and the signal from the vehicle interior light environment sensor 29 may be input to the driving attention detecting circuit 21. Also, in the configuration of FIG. 18, only the signal of the vehicle interior light environment sensor 29 may be input to the driving attention detection circuit 21 without providing the vehicle state detection sensor 25. Further, FIG.
Each of the configurations added to FIG. 1 from FIG. 18 to FIG. 18 is a third configuration for calculating the eye tracking time rate shown in FIG.
May be added to the above embodiment.

FIG. 19 is an overall configuration diagram of a dozing driving detecting apparatus according to a seventh embodiment of the present invention, and FIG. 20 is a flowchart showing the operation thereof. This embodiment is shown in FIG.
The driving attention detecting device according to the third embodiment shown in FIG. 1 includes an eye opening value calculating unit that calculates an eye opening value when the eye position detection circuit 15 successfully detects the eye position. Eye open / closed eye calculation circuit 31; open / closed eye determination circuit 33 as open / closed eye determination means for performing open / closed eye determination using a value obtained by the eye openness value calculation circuit 31; A drowsiness detection circuit 35 is additionally provided as a drowsiness detection means for detecting a drowsiness in accordance with the driving attention detected by the driving attention detection circuit 21 based on the open / closed eye determination result of 33.

The flowchart of FIG. 20 is obtained by adding steps 1807, 1810, and 1811 to the flowchart of the driving attention detecting device according to the third embodiment shown in FIG. 12, and is adapted to the dozing driving detecting device. Things.

The components added to FIG. 11 in FIG. 19 may be added to the first embodiment shown in FIG. 1. Therefore, the step processing added in FIG.
The present embodiment can also correspond to the first embodiment and the second embodiment. 16 to FIG.
19 may be additionally provided in FIG.

In the seventh embodiment, only steps different from those of the third embodiment shown in the flowchart of FIG. 12 will be described. In step 1807, a calculation process of the opening degree value of the data of the eye captured in the tracking area of the eye is performed. In step 1810, open / closed eye determination is performed using the eye opening value obtained in step 1807. In step 1811, dozing detection is performed according to the driving attention detected every predetermined time in step 1813.

By matching the detection result of the driving attention with the detection of the drowsiness, the accuracy of the drowsiness detection is increased when the frequency of the safety confirmation and the like is reduced and the eye tracking time rate is increased, and the frequency of the safety confirmation and the like is increased. When the eye tracking time rate is high and the drowsiness detection rate is low, a high drowsiness detection accuracy can be ensured, and a false alarm when outputting an alarm can be prevented.

The method of detecting the position of the eye, the method of tracking the eye, the calculation of the eye opening value, the method of determining the open / closed eyes, and the method of determining the arousal level are described in Japanese Patent Application Laid-Open No. 10-40361 and It is well known from Japanese Patent Application Laid-Open No. Hei 10-143669, and a detailed description thereof is omitted.

[Brief description of the drawings]

FIG. 1 is an overall configuration diagram of a driving attention detecting device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing the operation of the first embodiment of the present invention.

FIGS. 3A and 3B are explanatory diagrams regarding an eye position detection method according to the first embodiment of the present invention; FIG. 3A is an image in which the driver is gazing ahead; FIG. It is an image in a state of watching.

FIG. 4 is an explanatory diagram related to a method of calculating an eye detection OK rate showing a result of eye detection in a predetermined section according to the first embodiment of this invention;

FIG. 5 is an explanatory diagram showing an OK detection frequency of an eye in a predetermined section with respect to an elapsed time for each predetermined section according to the first embodiment of the present invention.

FIG. 6 is a flowchart showing an operation of the driving attention detecting device according to the second embodiment of the present invention.

FIGS. 7A and 7B are explanatory diagrams relating to a predetermined area setting method according to a second embodiment of the present invention, wherein FIG. 7A is an image in which the driver is gazing forward, and FIG. and that is an image of the state.

FIG. 8 is a flowchart illustrating an operation of an application example 1 of the second embodiment of the present invention.

FIG. 9 is a flowchart showing an operation of an application example 2 of the second embodiment of the present invention.

FIG. 10 is a flowchart showing an operation of an application example 2 of the second embodiment of the present invention.

FIG. 11 is an overall configuration diagram of a driving attention detecting device according to a third embodiment of the present invention.

FIG. 12 is a flowchart showing the operation of the third embodiment of the present invention.

FIG. 13 is an explanatory diagram relating to a method for setting and tracking an eye tracking area according to a third embodiment of the present invention.

FIG. 14 is an explanatory diagram related to a method of calculating an eye tracking time rate showing an eye detection processing time and an eye tracking processing time in a predetermined section according to the third embodiment of the present invention.

FIG. 15 is an explanatory diagram showing an eye tracking time rate in a predetermined section with respect to an elapsed time for each predetermined section according to the third embodiment of the present invention.

FIG. 16 is an overall configuration diagram of a driving attention detecting device according to a fourth embodiment of the present invention.

FIG. 17 is an overall configuration diagram of a driving attention detecting device according to a fifth embodiment of the present invention.

FIG. 18 is an overall configuration diagram of a driving attention detecting system according to a sixth embodiment of the present invention.

FIG. 19 is an overall configuration diagram of a dozing driving detection device showing a seventh embodiment of the present invention.

FIG. 20 is a flowchart showing the operation of the seventh embodiment of the present invention.

[Description of Signs] 5 Driver 7 TV camera (image photographing means) 15 Eye position detection circuit (eye position detection means, predetermined area setting means, eye position detection means in predetermined area) 19 Eye detection OK time rate Arithmetic circuit (eye detection OK number integration means, eye detection OK rate calculation means) 21 Driving attention detection circuit (driving attention detection means) 23 Eye tracking time rate calculation circuit (Eye tracking processing means, Eye tracking) Time integration means, eye tracking time rate calculation means) 25 vehicle state detection sensor 27 personal data memory 29 vehicle interior light environment measurement sensor 31 eye opening value calculation circuit (eye opening value calculation means) 33 open / closed eye determination circuit (Open / closed eye determination means) 35 Drowsiness detection circuit (Drowsiness detection means)

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification code FI Theme coat ゛ (Reference) G06T 7/20 300 G06T 7/60 150B 7/60 150 G08G 1/16 F G08G 1/16 A61B 3/10 BF term (reference) 3D037 FA05 4C038 PP05 PQ04 PS07 5B057 AA16 BA02 BA30 CA02 CA08 CA12 CA16 CB02 CB08 CB12 CB16 CC03 CE09 DA07 DA08 DC04 DC06 5H180 AA01 CC04 LL20 5L096 AA03 AA06 FA04 FA18 FA04

Claims (11)

[Claims] 1. A driving attention detecting apparatus for detecting driver's attention by processing image data of a driver's face, comprising: an image photographing means for photographing the driver's face image; An eye position detecting means for detecting the position of the eye from the entire image; and, for each of the photographed image data, determining whether or not the eye position detecting means has successfully detected the eye, and counting the number of times of success. OK for detecting eye
Number integration means, predetermined time measurement means for measuring a predetermined time, and an eye detection OK rate based on the eye detection OK number integration value by the eye detection OK number integration means with respect to a predetermined time measurement value by the predetermined time measurement means And a driving attention detecting means for detecting a driver's attention from the calculation result of the eye detecting OK rate calculating means. . 2. A driver's attention detecting device that detects driver's attention by processing image data of a driver's face, wherein said driver's face image is captured by said driver. Eye position detection means for detecting the position of the eye from the entire image, and predetermined area setting means for setting an area including at least one eye based on the position of the eye detected by the eye position detection means from the entire image An eye position detecting means for detecting the position of the eye within the predetermined area set by the predetermined area setting means; and an eye position detecting means for the eye within the predetermined area for each of the photographed image data. It is determined whether or not the detection has succeeded, and the number of detected OK eyes of the eye for counting up the number of times of success, a predetermined time measuring means for measuring a predetermined time, and a predetermined time measurement value by the predetermined time measuring means. An eye detection OK rate calculation means for calculating an eye detection OK rate based on the eye detection OK number integration value by the eye detection OK number integration means, and an eye detection OK
A driving attention detecting means for detecting a driver's attention from a calculation result of the rate calculating means. 3. The driving attention detecting device according to claim 2, further comprising a release unit for releasing the setting of the predetermined region by the predetermined region setting unit by a manual operation. 4. The apparatus according to claim 2, further comprising a canceling means for canceling the setting of the predetermined area by the predetermined area setting means based on the calculation result of the eye detection OK rate by the eye detection OK rate calculation means. Driving attention detection device. 5. A driver's attention detecting device for processing driver's face image data to detect driver's attention, an image photographing means for photographing the driver's face image, and photographing by the image photographing means. Eye position detecting means for detecting the position of the eye from the entire image, and an eye tracking area is set based on the eye position detected by the eye position detecting means, and the eye tracking is performed for each of the captured image data. Eye tracking processing means for updating the area, and eye tracking time integration means for integrating the eye tracking time while checking whether the eye tracking processing by the eye tracking processing means is performed correctly, A predetermined time measuring means for measuring a predetermined time; and an eye tracking time for calculating an eye tracking time rate based on an eye tracking time integrated value by the eye tracking time integrating means with respect to a predetermined time measured value by the predetermined time measuring means. Rate operator A driving attention detection device, comprising: a step; and a driving attention detection unit that detects a driver's attention from a calculation result of the eye tracking time rate calculation unit. 6. The driving attentiveness detection means receives a signal from a vehicle state detecting sensor and switches the driving attentiveness determination criterion according to the traveling state of the vehicle. A driving attention detecting device according to any one of the above. 7. The driving attention detecting means according to claim 1, wherein the stored contents of the personal data are inputted to the driving attention detecting means, and the driving attention judgment criteria are switched in accordance with the personal information of the driving behavior. A driving attention detection device according to any one of the claims. 8. A driving attention detecting means receives a signal from a vehicle interior light environment measurement sensor, estimates deterioration of the vehicle interior light environment, and determines that an eye cannot be detected. The driving attention detecting device according to any one of claims 1 to 7, wherein: 9. A drowsy driving detection device for processing a driver's face image data to detect a drowsy driving, wherein: an image photographing means for photographing the driver's face image; and the entire image photographed by the image photographing means. An eye position detecting means for detecting the position of the eye from the eye, and an eye for judging whether or not the eye position detecting means has successfully detected the eye for each of the photographed image data, and counting up the number of successes. Detection OK
Number integration means, predetermined time measurement means for measuring a predetermined time, and an eye detection OK rate based on the eye detection OK number integration value by the eye detection OK number integration means with respect to a predetermined time measurement value by the predetermined time measurement means , An eye detection OK rate calculation means for calculating the eye detection OK rate calculation means, a driving attention detection means for detecting a driver's attention from the calculation result of the eye detection OK rate calculation means, and an eye position detection by the eye position detection means An eye opening value calculating means for calculating an eye opening value when the opening is successful; an opening / closing eye determining means for performing open / closed eye determination using a value obtained by the eye opening value calculating means; A drowsy driving detection device, comprising: a drowsiness driving detection device that detects a drowsiness according to the driving attention detected by the driving attention detection device based on a result of the open / closed eye determination by the eye determination device. 10. A drowsy driving detection device for processing a driver's face image data to detect a drowsy driving, wherein an image photographing means for photographing the driver's face image, and an entire image photographed by the image photographing means. Eye position detecting means for detecting the position of the eye from; predetermined area setting means for setting an area including at least one eye based on the position of the eye detected by the eye position detecting means from the entire image; The position detection unit detects the position of the eye within the predetermined region set by the predetermined region setting unit, and the eye detection by the position detection unit within the predetermined region is performed for each of the captured image data. Eye detection OK number integrating means for judging whether or not successful, and counting up the number of successes, predetermined time measuring means for measuring a predetermined time, and a predetermined time measurement value by the predetermined time measuring means An eye detection OK rate calculating means for calculating an eye detection OK rate based on the eye detection OK number integration value with respect to the eye detection OK number integration means;
A driver's attention detecting means for detecting the driver's attention from the calculation result of the K rate calculating means, and an eye opening value is calculated when the eye is successfully detected by the eye position detecting and determining means in the predetermined area. Eye opening value calculation means, open / closed eye determination means for performing open / closed eye determination using a value obtained by the eye opening value calculation means, and based on the open / closed eye determination result of the open / closed eye determination means, A drowsiness detection device comprising: a drowsiness detection means for detecting a drowsiness according to the driving attention detected by the driving attention detection means. 11. A drowsy driving detection device for processing a driver's face image data to detect a drowsy driving, wherein: an image photographing means for photographing the driver's face image; and an entire image photographed by the image photographing means. An eye position detecting means for detecting the position of the eye from the eye, and setting an eye tracking area based on the eye position detected by the eye position detecting means, and setting the eye tracking area for each of the captured image data. An eye tracking processing means to be updated; an eye tracking time integrating means for integrating the eye tracking time while checking whether the eye tracking processing by the eye tracking processing means is performed correctly; and a predetermined time. A predetermined time measuring means for measuring an eye tracking time rate based on an eye tracking time integrated value by the eye tracking time integrating means with respect to a predetermined time measured value by the predetermined time measuring means. Means, a driving attention detecting means for detecting the driver's attention from the calculation result of the eye tracking time rate calculating means, and calculating an eye opening value when the eye tracking processing is correctly performed. An eye opening value calculating unit, an opening / closing eye determining unit that performs open / closed eye determination using a value obtained by the eye opening value calculating unit, and A drowsiness detection device comprising: a drowsiness detection means for detecting a drowsiness according to the driving attention detected by the driving attention detection means.