JPH11195200A - Operating state detecting device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a detecting device in which dozing-off operating state or looking-aside operating state of a driver can be detected always stably and properly. SOLUTION: This device is provided with a TV camera 11 for inputting the picture data of a face, an A/D converter 13, a picture memory 15 for storing input picture data converted into digital values, a picture data arithmetic circuit 17, an eye position detecting circuit 19 for detecting eye positions, a vehicle state detecting circuit 21 for detecting the vehicle state at the time of vehicle traveling, an eye reference position storage circuit 23 for judging whether or not probability that a driver watches ahead is high based on the detection of the vehicle state detecting circuit 21, determining and storing the eye positions when it is judged that the probability is high as eye reference positions, and judging whether or not the eye positions detected at the time of vehicle traveling are made coincident with a condition predetermined for the reference positions, and an alarming device 25 for issuing alarm when they are not made coincident.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an eye position detecting device for detecting a dozing state or an inattentive state of a driver of a vehicle, a boat operator, an operator of a plant, or the like.

[0002]

2. Description of the Related Art Conventional apparatuses of this type include, for example, Japanese Patent Application Laid-Open No. Hei 3-260900 and Japanese Patent Application Laid-open No. Hei 6-251273.
Is disclosed in Japanese Patent Application Laid-Open Publication No. HEI 9-203 (1995).

[0003] In the former conventional example, when the face direction angle of the face with respect to the reference position when the driver is driving normally deviates by more than the permissible angle for more than the permissible time, the driver may fall asleep. It is determined that the driver is looking aside and if the relative speed with respect to the inter-vehicle distance to the preceding vehicle is exceeded, a warning is issued to the driver.

In the latter conventional example, a frequency distribution of a driver's line of sight change is calculated, and when the maximum line of sight frequency exceeds an allowable value, it is determined that the driver is driving dimly and a warning is output. .

[0005]

However, the driver's eye position and line of sight change due to, for example, vertical, lateral, lateral, and forward accelerations of the vehicle while the vehicle is running. If you drive for a long time, your driving posture will change, so it will not be constant. In the case of a seat such as a bus or a truck where the seat back is nearly upright, the holdability is poor, and the upper body swings left and right on a curved road, and is not constant. For example, it has been difficult to accurately detect the position and the line of sight of the driver's eyes.

The present invention has been made in view of such a problem, and an object of the present invention is to provide a detection device capable of always stably and appropriately detecting a driver's drowsy driving state and inattentive driving state. .

[0007]

According to a first aspect of the present invention, there is provided an image input unit for inputting image data of a face, and an eye position is detected from the input image data. An eye position detecting means, a vehicle state detecting means for detecting a vehicle state when the vehicle is traveling, and a front side for determining whether or not the probability that the driver is gazing at the front of the vehicle is high based on the detection of the vehicle state detecting section. Gaze determination means, storage means for storing the detected eye position as a reference position when it is determined that the probability that the driver is gazing at the front of the vehicle is high, and the eye position detected during traveling of the vehicle is A comparison means for comparing whether or not a predetermined condition is met with respect to the reference position is provided.

According to a second aspect of the present invention, in the driving state detecting device according to the first aspect, the vehicle state detecting means detects whether a brake pedal is depressed and a steering angle of the steering device. The forward gaze determination means is characterized in that when the brake pedal is depressed and the steering angle is substantially at the neutral position, it is determined that the probability that the driver gazes ahead of the vehicle is high.

According to a third aspect of the present invention, in the driving state detecting device according to the first aspect, the vehicle state detecting means detects whether the accelerator pedal is depressed and the steering angle of the steering device. The forward gaze determining means determines that the probability that the driver is gazing ahead of the vehicle is high when the accelerator pedal is depressed and the steering angle is substantially at the neutral position.

According to a fourth aspect of the present invention, there is provided the driving state detecting apparatus according to the first aspect, wherein the vehicle state detecting means detects acceleration / deceleration before and after the vehicle traveling direction and a steering angle of the steering device. The forward gaze determination means determines that the probability that the driver is gazing ahead of the vehicle is high when the steering angle is substantially at the neutral position and the detected acceleration / deceleration becomes a predetermined acceleration / deceleration. It is characterized by doing.

According to a fifth aspect of the present invention, in the driving state detecting apparatus according to the first aspect, it is determined whether or not the detected eye position stays at the same position within a predetermined time. Providing a stay determination means for determining, the vehicle state detection means detects the steering angle of the steering device, the forward fixation determination means, the detected eye position stays at the same position within a predetermined time, and When the steering angle is at the neutral position, it is determined that the probability that the driver gazes ahead of the vehicle is high.

According to a sixth aspect of the present invention, there is provided an image input device for inputting image data of a face, an eye position detecting device for detecting an eye position from the input image data, and wherein the detected eye position is determined in advance. Providing stay determination means for determining whether or not the vehicle is staying at the same position within a predetermined time, and determining whether or not the probability that the driver is gazing ahead of the vehicle based on the determination of the stay determination means is high. Forward gaze determination means, and the eye position is detected when it is determined that the probability that the eye position stays at the same position within a predetermined time and the driver gazes ahead of the vehicle is high. It is characterized by comprising storage means for storing as a reference position, and comparison means for comparing whether or not the eye position detected during running of the vehicle matches a predetermined condition with respect to the reference position.

According to a seventh aspect of the present invention, there is provided the driving state detecting device according to any one of the first to sixth aspects, wherein the storage means for storing the reference position of the eye position comprises: It is characterized in that the eye position when the forward gaze determination means first determines that the probability of gaze is high after turning on the ignition switch is stored as a reference position.

According to an eighth aspect of the present invention, there is provided the driving state detecting device according to any one of the first to sixth aspects, wherein the storage means for storing the reference position of the eye position is provided so that the driver can drive the vehicle in front of the vehicle. From the time when the forward gaze determination means determines that the probability of gazing is high, storing a reference position of the eye position based on the plurality of eye positions detected by the eye position detection means within a predetermined time. I do.

According to a ninth aspect of the present invention, there is provided the driving state detecting device according to any one of the first to sixth aspects, wherein the storage means for storing the reference position of the eye position is provided so that the driver can move forward in the vehicle. The reference position is updated and stored each time the forward gaze determining means determines that the probability of gazing is high.

According to a tenth aspect of the present invention, there is provided the driving state detecting apparatus according to any one of the first to ninth aspects, further comprising an alarm unit for calling a driver's attention based on a comparison result by the comparing unit. It is characterized by the following.

[0017]

According to the first aspect of the present invention, the eye position of the driver when it is determined that the probability that the driver is gazing ahead of the vehicle based on the detection of the vehicle state is high as the eye reference position. Since it is stored and compared with the eye reference position to determine whether the eye position matches a predetermined condition, it is determined whether the vehicle is in a drowsy driving state or an inattentive driving state. Becomes possible.

According to the second aspect of the present invention, when the brake pedal is depressed and the steering angle is substantially at the neutral position as the vehicle state, it is determined that the probability that the driver gazes ahead of the vehicle is high. Thereby, an effect equivalent to that of the first aspect is obtained.

According to the third aspect of the present invention, when the accelerator pedal is depressed and the steering angle is substantially at the neutral position as the vehicle state, it is determined that the probability that the driver is gazing ahead of the vehicle is high. Thereby, an effect equivalent to that of the first aspect is obtained.

According to the present invention, when the steering angle is substantially at the neutral position and the detected acceleration / deceleration becomes a predetermined acceleration / deceleration as the vehicle state, the driver moves forward of the vehicle. By judging that the probability of gazing is high, the same effect as that of the first aspect can be obtained.

According to the fifth aspect of the invention, the driver gazes ahead of the vehicle when the eye position stays at the same position within a predetermined time and the steering angle is at the neutral position. By judging that the probability is high, the same effect as that of the first aspect can be obtained.

According to the invention described in claim 6, the probability that the driver gazes ahead of the vehicle is determined based on the determination as to whether the eye position is staying at the same position within a predetermined time. It is determined whether or not the eye position is high, and if it is determined that the forward gaze probability is high, the eye position at that time is stored as the eye reference position. Then, whether or not the detected eye position matches a predetermined condition with respect to the eye reference position is determined to determine whether the vehicle is in a drowsy driving state or an inattentive driving state. It becomes possible.

According to the invention of claim 7, according to claim 1,
And an effect equivalent to that of any one of the inventions is obtained.
Since the eye position when initially determining that the driver has a high probability of gazing ahead of the vehicle after turning on the ignition switch is stored as the eye reference position, for example, the position of the driver's eyes is relatively stable. It is effective when applied to the case of driving a passenger car.

According to the invention described in claim 8, according to claim 1,
And an effect equivalent to that of any one of the inventions is obtained.
Since it is determined that the probability that the driver is gazing at the front of the vehicle is high, the eye reference position is set based on the plurality of eye positions within a predetermined time, so that the accuracy of the eye reference position is further improved.

According to the ninth aspect of the present invention, the first aspect is provided.
And an effect equivalent to that of any one of the inventions is obtained.
Since the eye reference position is updated each time it is determined that the probability that the driver is gazing at the front of the vehicle is high, if the position of the driver's eye such as a truck is not stable, the eye reference position is constantly updated. By doing so, it becomes possible to accurately determine the reference position of the eye.

According to the tenth aspect of the invention, the same effects as those of any of the first to ninth aspects can be obtained, and when it is determined that the vehicle is in the dozing driving state or the inattentive driving state, the driver is alerted by an alarm. Because it wakes up, safety is improved.

[0027]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [First Embodiment] A first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a functional block diagram of the operating state detecting device of the present embodiment. This operating state detecting device is provided with an image input means CL1.
Eye position detecting means CL2 by image data processing, vehicle state detecting means CL3, forward gaze determining means CL4,
Storage means CL5, comparison means CL6, alarm means CL7
And

The image input means CL1 inputs image data of the driver's face. The eye position detecting means CL2
Detects the position of the eye by selecting the eye from the input image data. On the other hand, the vehicle state detecting means CL3 determines the vehicle state during traveling including the driver, that is, whether or not the brake pedal is depressed, the steering angle of the steering device, whether or not the accelerator pedal is depressed, and the acceleration / deceleration before and after the vehicle traveling direction. It is possible to detect. The forward fixation judging means CL4
Are eye position detecting means CL2 and vehicle state detecting means CL
It is determined whether or not the probability that the driver gazes ahead of the vehicle is high based on the detection information of No. 3. And the storage means C
L5 stores the position of the eye when it is determined that the probability that the driver is gazing at the front of the vehicle is high as the eye reference position. The comparing means CL6 compares whether or not the position of the eye detected during traveling of the vehicle matches a predetermined condition with respect to the eye reference position. And the alarm means CL7 is
When the eye position does not satisfy the predetermined condition with respect to the reference position based on the comparison result by the comparing means CL6, a warning is issued to call attention to the driver.

FIG. 2 is a configuration block diagram of the present embodiment. As shown in the figure, a TV camera 11 as an image input means CL1 is installed on an instrument panel in front of a driver's seat, and photographs a driver's face from the front. The input image of the TV camera 11 is stored in the image memory 15 as digital input image data via the A / D converter 13. The image memory 15 includes a density detection unit for a pixel column, a point extraction unit for extracting a point of the pixel column, and a curve group in the horizontal direction of the face by continuously connecting points in the pixel column direction of an adjacent pixel column. It comprises an image data operation circuit 17 to be extracted. Then, the eye position detecting means CL2, the comparing means CL
6, eye position detection circuit 19 as eye position retention judgment means, vehicle state detection circuit 21 as vehicle state detection means CL3 and forward gaze judgment means CL4, storage means CL5
And a warning device 25 as warning means CL7.

The details of a method of recognizing the position of the eye by processing the input image taken by the TV camera 11 by the image data calculation circuit 17 will be described later.

Next, the vehicle state detection circuit 2 having the above configuration
1. The operation flow of the eye reference position storage circuit 23 and the alarm device 25 will be described with reference to the flowchart of FIG.

In step S31 of FIG. 3, the vehicle state detection circuit 21 detects a vehicle state including the driver. When the detected vehicle state is as shown in (a) to (e) below, the eye reference position storage circuit 23 estimates that the driver is gazing ahead of the vehicle.

(A) While the vehicle is traveling straight and during the braking operation, it is presumed that the driver is gazing forward because he / she feels danger ahead and steps on the brake pedal. At this time, the detection of the straight traveling state of the vehicle is detected by the steering angle of the steering, and the braking is detected by the signal of a switch (SW) that emits a signal when the brake pedal is depressed.

(B) While the vehicle is traveling straight and accelerating, the driver determines that there is an inter-vehicle distance ahead and that the driver does not hit the accelerator pedal, and steps on the accelerator pedal. It is presumed that the user is gazing at the front with a high probability to adjust while watching. At this time, the detection of the straight traveling state of the vehicle is the same as in the case (a) described above, and the acceleration operation is detected by a signal of an accelerator pedal SW or a throttle opening signal which generates a signal when the accelerator pedal is depressed.

(C) When the vehicle is traveling straight and the acceleration or deceleration of the vehicle exceeds a predetermined value, it is estimated that the driver is gazing forward. At this time, the detection of the straight traveling state of the vehicle is the same as in the case of (a), and the acceleration or deceleration is detected by a signal of an acceleration sensor attached to the vehicle. Further, acceleration or deceleration can be detected by differentiating the vehicle speed.

(D) A case where the position of the eye stays within a predetermined range of the image for a predetermined time.

(E) The vehicle is traveling straight, and the position of the eye stays within a predetermined range of the image for a predetermined time.

Next, in step S32, if the vehicle state detected in the previous step S31 corresponds to any of the above (a) to (e), the process proceeds to the next step S33.
If none of (a) to (e) applies, the process returns to the start.

In step S33, the eye position detection circuit 19
Performs eye position detection. Then, it is determined whether or not the eye reference position (reference coordinates in FIG. 3) is already stored (set) by the eye reference position storage circuit 23 (step S3).
4) If not set, the process proceeds to the next step S35, and the position of the eye obtained in step S33 is stored as an eye reference position. On the other hand, if the eye reference position has already been set, the process skips step S35 and proceeds to step S36.

The flow from step S36 is a flow for performing an alarm process in the case of falling asleep or looking aside. here,
The flow in the case of detecting an inattentive driving state and issuing an alarm by the alarm device 25 is shown.

That is, in step S36, the eye reference position stored in the previous step S35 is set as the neutral position of the eye,
The position of the eye within the reduced area is detected. This reduced area is
It is fixed as a predetermined area based on the neutral position of the eye.

In the next step S37, the eye position detection circuit 19 determines whether or not the position of the driver's eye has been within the reduced area for a predetermined time, and if so, determines that it is not the inattentive driving. Then, the process returns to step S34 to execute the subsequent processing. On the other hand, if the driver's eye position does not exist and goes outside the reduced area, it is determined that the driver is looking aside, and the alarm device 25 issues a warning with a lamp, a buzzer, or the like, and calls the driver's attention (step S38). .

In the present embodiment, once the eye reference position is determined in step S35, the eye reference position is determined until the ignition switch is turned off, and the stored eye reference position is fixed. Therefore, it is effective in the case of a passenger car in which the position of the driver's eyes is relatively stable.

Next, a method of recognizing the position of the eye by image processing will be described with reference to FIGS. FIG. 6 shows a flow of the image processing.

In step S110 of FIG. 6, the TV camera 1
1 captures an image of the vehicle interior including the driver's face,
The density of the pixel is converted into a digital value by the -D converter 13. Next, in step S120, the image memory 15
The image data corresponding to the image of one frame is stored.

In step S130, a vertical pixel row of the face is selected from the huge amount of image data stored in the image memory 15 as shown in FIG. Dozens of columns at equal intervals in the center of the screen, one for every 10 pixels in the X direction
A row of pixels is set. One pixel column has 48 pixels in the Y direction.
It has 0 pixels, and can transmit the density values of all pixels for each pixel column to the image data calculation circuit 17.

In step S140, the density value of each pixel row is subjected to arithmetic processing to obtain a pixel to be an extraction point (Y coordinate value).
To identify. A large change in the density value on the pixel row is obtained, and a local peak thereof is set as an extraction point.

FIG. 7 shows a detailed procedure for obtaining an extraction point. That is, in step S141 of FIG. 7, the density values of all the pixels for each pixel column are called in order from the left side of FIG.

In step S142, the density values on the pixel row are arithmetically averaged to obtain an arithmetic average density value from which a high frequency component has been removed from the density value change curve on the pixel row. For example, from the pixel column X _a shown in FIG. 10 arithmetic mean density value as shown in FIG. 11 (a) it is extracted. The horizontal axis in FIG. 11 is Y
It is a coordinate value, and the vertical axis is a density value.

In step S143, the arithmetic mean density value is differentiated. In the arithmetic mean density value shown in FIG.
1 (b).

In step S144, a Y coordinate value corresponding to the dark peak of the arithmetic average density value is obtained from the obtained differential value. The Y coordinate value at which the differential value is inverted from negative to positive becomes one Y coordinate value representing each of the eyes, eyebrows, mouth, etc., corresponding to the peak of darkness. The Y coordinate value at which the differential value is inverted from positive to negative is a peak position of brightness corresponding to an intermediate position such as an eye or an eyebrow, and is not used here.

In the arithmetic mean density value shown in FIG. 11A, the differential value shown in FIG. 11B is obtained, and Y coordinate values p1, p2, p3, p4, and p5 at which the differential value is inverted from negative to positive are extracted. Is done. Among these, the Y coordinate values p1, p2, and p5 where the peak of the preceding differential value is equal to or smaller than the threshold value (shaded area) are set as extraction points. By determining the peak of the differential value at the near side, wrinkles and irregularities on the face do not become extraction points.

Then, in a step S145, it is determined whether or not the pixel row is the last pixel row (the rightmost pixel row in FIG. 10). If it is the last pixel row, the extraction point has been specified in all the pixel rows, so the process proceeds to the next step S150 (FIG. 6). If it is not the last pixel row, the process proceeds to step S141 (FIG. 7), and identification of the extraction point based on the density value data of the next pixel row is started.

In this way, as shown in FIG. 12, about 0 to 4 extraction points A1, A2, A3, A
4 is specified. The pixel-column X _C, extraction point A1, A2,
Extraction points A1, A2, A3, and A4 are specified in the pixel row _Xd .

In step S150, the extracted points obtained for each pixel column are connected in the X direction and grouped.

FIG. 8 shows a detailed procedure of the grouping process. That is, in step S151 of FIG. 8, the extraction point data of the pixel row is called in order.

In steps S152 and S153, the Y coordinate values are compared for the extraction points of the adjacent pixel rows. If the difference between the Y coordinate values of the extraction points of the adjacent pixel rows is 10 pixels or less (within a predetermined range), the process proceeds to step S154, where connected data is formed. On the other hand, when it is determined that the difference exceeds the predetermined range (grouping cannot be performed), step S155 is performed.
Proceed to and no linked data is formed.

The connected data is classified according to the group number sequentially assigned to each generated group and the number of the pixel column (corresponding to the X coordinate) to which the leftmost extracted point belongs to the group, and the number of connected extracted points It is formed by storing (the number of pixel columns) and the Y coordinate value in a memory, and rewriting the stored contents every time a pixel column is processed.

In step S155, it is determined whether the pixel row is the last pixel row. If it is the last pixel column, the grouping has been determined for all the pixel columns, and the process proceeds to the next step S160 (FIG. 6). If it is not the last pixel row, the process proceeds to step S151 (FIG. 8), and the determination of grouping based on the extraction point of the next pixel row is started.

In step S160, the image data calculation circuit 17 (see FIG. 2) removes unnecessary linked data,
The candidate point data is formed only for the finally required connection data.

FIG. 9 shows a detailed procedure for forming the candidate point data. That is, in step S161 of FIG. 9, the linked data is called one by one from the memory in order of the group number. Next, in steps S162 and S163, the number N of linked data of the linked data is evaluated. Number of connected data N is 5
If the number is equal to or more than the predetermined value (the predetermined value or more), the next step S16
Proceed to step 4 to form candidate point data. If the number N of connected data is less than 5 (for example, a nostril), step S1
Proceeding to 65 does not form candidate point data.

For example, from the group of extracted points shown in FIG. 12, as shown in FIG.
Groups G1 to G of six extraction points each corresponding to a mouth
6 are left. Here, the remaining six groups are renumbered in order from left to right and from top to bottom.

The candidate point data is a set of coordinate values representing each of the remaining extracted point groups by one candidate point on the screen, that is, the Y values of the extracted points belonging to that group.
It is composed of the average value (height) of the coordinate values and the X coordinate value of the center of the pixel rows at both left and right ends of the group.

For example, the extraction point groups G1 to G6 shown in FIG. 13A are represented by six candidate points G1 to G6 on the screen shown in FIG. 13B.

In step S165, it is determined whether or not the data is the last connected data. If it is the last consolidated data,
Since all the connected data have been evaluated, the process proceeds to the next step S170 (FIG. 6). If it is not the last connected data, the process proceeds to step S161 (FIG. 9), and evaluation of the next connected data is started.

In step S170, the eye position detection circuit 1
9 (see FIG. 2) evaluates the candidate point data and specifies one eye candidate point from among the plurality of candidate points.

For example, in the case of six candidate points G1 to G6 shown in FIG. 13B, the candidate point G having a large (low) Y coordinate value is set.
5, a central detection zone C is formed around G6.
Outside the detection zone C, left and right detection zones L and R are set with the candidate points G1 and G3 having small (high) Y coordinate values and the candidate points G2 and G4 as centers.

Then, 2 included in the left detection zone L
The candidate point G3 for one eye is specified based on the positional relationship between the two candidate points G1 and G3.

As described above, according to the present embodiment, the condition of the vehicle state in which the driver is presumed to be gazing at the front of the vehicle is set in advance as described above, and when the condition is met. The reference position of the eyes is determined from the position of the eyes of the driver, and if the position of the eyes deviates from within the reduction area set based on the driver's eyes, the driver determines that the driver is in a dozing driving state or an inattentive driving state. Judgment can be made.

Further, once the reference position of the eye is determined, the determined reference position is fixed during driving, so it is effective to apply this method to the case of driving a passenger car in which the position of the driver's eye is relatively stable. .

[Second Embodiment] A second embodiment of the present invention will be described with reference to FIG. FIG. 4 is a flowchart showing the entire operation of the present embodiment. This embodiment differs from the first embodiment in the method of determining the reference position of the eye.

First, in step S41 of FIG. 4, the vehicle state detection circuit 21 detects the vehicle state including the driver. The conditions under which the driver is supposed to be gazing at the front of the vehicle are the same as those in (a) to (e) of the first embodiment.

Next, it is determined whether or not the probability that the driver gazes ahead of the vehicle is high based on the detected vehicle state (step S42). If it is high, the process proceeds to the next step S43, and if it is not high, the process returns to the start.

In the next step S43, the eye position detection circuit 19 detects the position of the eye. Next, in step S44, it is determined whether or not a predetermined time has elapsed since the detection of the vehicle state in which it is estimated that the driver is gazing ahead of the vehicle. If not, the next step S45 is performed.
It is determined whether or not the reference position of the eye has been determined, and if not determined, in step S46, the reference position of the eye is determined from the positions of the eyes of the plurality of drivers detected within a predetermined time, Remember. On the other hand, if the predetermined time has elapsed in step S44, the timer is set to 0 in step S51, the reference position of the eye is cleared in step S52, and the process returns to step S41.

After the reference position of the eye is stored in step S46, the timer is started (step S46).
47), the position of the eye in the reduced area is detected during a predetermined time (step S48). If it is determined in step S45 that the reference position of the eye has been determined, step S45 is also performed.
This step S48 is executed by skipping steps 46 and S47.

When the position of the eye detected in step S49 goes out of the reduced area and is determined to be an inattentive driving state, the warning device 25 issues a warning and calls the driver's attention (step S50). On the other hand, if it is determined in step S49 that the vehicle is not in the inattentive driving state, the process returns to step S44. Here, if the predetermined time has elapsed, the timer is set to 0 (step S51), the reference position of the eye is cleared (step S52), and the process returns to step S41.

According to this embodiment, according to this embodiment, if it is estimated that the driver is gazing ahead of the vehicle by detecting the state of the vehicle, the position of the driver's eyes within a predetermined time is detected. Since the eye reference position is determined from the detected positions of the plurality of eyes, the accuracy of the determined driver eye reference position is further improved.

[Third Embodiment] A third embodiment of the present invention will be described with reference to FIG. FIG. 5 is a flowchart showing the entire operation of this embodiment.

In this embodiment, as a result of detecting the operation state,
The operation in the case where it is determined that the driver is not gazing at the front of the vehicle (step S69 described later) and the point of constantly updating the setting of the eye reference position (step S67) are described in the first embodiment (FIG. 3). ), And others are the same. Therefore, the same steps as those in the first embodiment are denoted by the same reference numerals, and the description of the overlapping steps will be omitted.

If it is determined in step S32 of FIG. 5 that the driver is not gazing at the front of the vehicle, the process proceeds to step S6.
9 (return to start in FIG. 3 of the first embodiment).
Conversely, when it is determined that the driver is gazing at the front of the vehicle, step S shown in FIG.
Go to 33.

In step S69, it is determined whether the reference position of the eye has already been set or not set. If the setting has been completed, the process proceeds to step S36 and step S67. On the other hand, if the reference position of the eye has not been set, the process returns to the start again.

If it is determined in step S67 that the driver is not looking aside, the flow returns to the start and the flow is repeated. If the condition that the driver gazes in front of the vehicle is satisfied in step S32, the eye reference Update location.

As described above, according to this embodiment, the reference position of the eye when the driver is estimated to be gazing ahead of the vehicle is constantly updated and determined. When the position is not stable, the reference position of the eye can be determined accurately by constantly updating the reference position of the eye.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a first embodiment of the present invention.

FIG. 2 is a configuration block diagram of a first embodiment of the present invention.

FIG. 3 is a flowchart showing all operations of the first embodiment.

FIG. 4 is a flowchart showing the entire operation of the second embodiment.

FIG. 5 is a flowchart showing the entire operation of the third embodiment.

FIG. 6 is a flowchart from input of image data to recognition of an eye position.

FIG. 7 is a flowchart of a process for obtaining an extraction point on each pixel column of an image.

FIG. 8 is a flowchart of a process for forming linked data of extraction points.

FIG. 9 is a flowchart of a process of selecting a candidate point from linked data.

FIG. 10 is an explanatory diagram related to a process in a flowchart.

FIG. 11 is an explanatory diagram related to a process in a flowchart.

FIG. 12 is an explanatory diagram related to a process in a flowchart.

FIG. 13 is an explanatory diagram related to a process in a flowchart.

[Explanation of symbols]

 CL1 Image input means CL2 Eye position detection means CL3 Vehicle state detection means CL4 Forward gaze determination means CL5 Storage means CL6 Comparison means CL7 Warning means 11 TV camera 13 A / D converter 15 Image memory 17 Image data calculation circuit 19 Eye position detection circuit (Eye position retention judging means) 21 Vehicle state detection circuit 23 Eye reference position storage circuit 25 Alarm device

Claims (10)

[Claims] 1. An image input unit for inputting image data of a face, an eye position detecting unit for detecting an eye position from the input image data, a vehicle state detecting unit for detecting a vehicle state during traveling of the vehicle, Forward gaze determining means for determining whether or not the driver has a high probability of gazing at the front of the vehicle based on the detection of the vehicle state detecting means; and determining that the probability of the driver gazing at the front of the vehicle is high. Storage means for storing the detected eye position as a reference position at the time of comparison; andcomparing means for comparing whether or not the eye position detected during traveling of the vehicle matches a predetermined condition with respect to the reference position. An operating state detection device comprising: 2. The driving state detection device according to claim 1, wherein the vehicle state detection unit detects whether a brake pedal is depressed and a steering angle of a steering device, and the front gaze determination unit includes: A driving state detection device that determines that the probability that the driver is gazing ahead of the vehicle is high when the brake pedal is depressed and the steering angle is substantially at the neutral position. 3. The driving state detection device according to claim 1, wherein the vehicle state detection unit detects whether or not an accelerator pedal is depressed, and a steering angle of a steering device, and the front gaze determination unit includes: A driving state detection device that determines that the probability that the driver gazes ahead of the vehicle is high when the accelerator pedal is depressed and the steering angle is substantially at the neutral position. 4. The driving state detection device according to claim 1, wherein the vehicle state detection unit detects acceleration / deceleration in the vehicle traveling direction and a steering angle of a steering device, and the front gaze determination unit. Driving, when the steering angle is substantially at the neutral position and when the detected acceleration / deceleration becomes a predetermined acceleration / deceleration, it is determined that the probability that the driver is gazing ahead of the vehicle is high. State detection device. 5. The driving state detecting device according to claim 1, further comprising a stay determination unit that determines whether the detected eye position stays at the same position within a predetermined time. The vehicle state detecting means detects a steering angle of a steering device, and the forward gaze determining means determines that the detected eye position stays at the same position within a predetermined time and the steering angle is in a neutral position. A driving state detection device that determines that the probability that the driver is gazing at the front of the vehicle at a certain time is high. 6. An image input unit for inputting image data of a face, an eye position detecting unit for detecting an eye position from the input image data, and wherein the detected eye position is the same within a predetermined time. Providing stay determination means for determining whether or not staying at the position, forward gaze determination means to determine whether the probability that the driver gazes in front of the vehicle is high based on the determination of the stay determination means, Storage means for storing the detected eye position as a reference position when it is determined that the eye position stays at the same position within a predetermined time and the probability of the driver gazing ahead of the vehicle is high. And a comparing means for comparing whether or not the eye position detected during running of the vehicle matches a predetermined condition with respect to the reference position. 7. The driving state detection device according to claim 1, wherein the storage unit that stores the reference position of the eye position has a high probability that the driver is gazing ahead of the vehicle. The driving state detecting device is characterized in that the eye position determined when the front gaze determining means first determines after turning on the ignition switch is stored as a reference position. 8. The driving state detection device according to claim 1, wherein the storage unit that stores the reference position of the eye position has a high probability that the driver is gazing ahead of the vehicle. And a reference position of the eye position based on the plurality of eye positions detected by the eye position detection means within a predetermined time from when the front gaze determination means determines the driving state. 9. The driving state detection device according to claim 1, wherein the storage unit that stores the reference position of the eye position has a high probability that the driver is gazing ahead of the vehicle. And the reference position is updated and stored each time the forward-looking determination means determines. 10. The driving state detection device according to claim 1, further comprising: an alarm unit for calling a driver's attention based on a comparison result by the comparison unit. State detection device.