JP2002352400A - Face image processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a driver's face parts and to store the positions of the face parts used as the reference of the angle of the driver's face. SOLUTION: A control part 50 detects the positions of the driver's nose and eyes based on an image generated by a CCD image sensor. When a vehicle- to-vehicle distance is a prescribed threshold or less and a steering angle is a prescribed threshold or less, the control part 50 compares the present positions of the nose and eyes with the first positions of the nose and eyes and counts the number of times when the difference is within a prescribed range. When the count value reaches the prescribed number of times, the first positions of the nose and eyes are made reference positions and written in a face part position memory 40.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a face image processing apparatus, and more particularly to a face image processing apparatus suitable for detecting a driver falling asleep or looking aside.

[0002]

2. Description of the Related Art In Japanese Patent Application Laid-Open No. H11-195200, it is determined whether or not the probability that a driver is gazing at the front of a vehicle is high based on a vehicle state during traveling of the vehicle. When the probability is high, a driving state detection device (hereinafter, referred to as “conventional technology”) that stores the eye position as a reference position and detects the driving state of the driver by comparing the eye position when the vehicle is traveling with the reference position. Proposed.

In the prior art, specifically, when a driver depresses an accelerator or a brake and acceleration / deceleration exceeds a predetermined value, and in addition to these conditions, even when the steering angle is substantially at a neutral position, the driver can move forward. Is a condition for watching. If it is determined that these conditions are satisfied and the probability that the driver gazes ahead is high, the position of the driver's eyes at this time is stored as the reference position.

[0004] However, in practice, even if the condition of depressing the accelerator or the like and the condition that the steering angle is at the neutral position are satisfied, the driver may not be gazing forward. For example, if there is no vehicle ahead and the vehicle is traveling on a straight road with good visibility, the driver does not need to consider the inter-vehicle distance with other vehicles, so he can easily look aside and the probability of gazing ahead is low. Lower.

Therefore, even if the prior art is used,
There has been a problem that the position of the eyes when the driver is looking ahead cannot always be detected, and the driving state of the driver cannot be accurately detected.

SUMMARY OF THE INVENTION The present invention has been proposed to solve the above-described problem, and it is intended to detect a driver's face part and store the position of the face part as a reference of the driver's face angle. It is an object of the present invention to provide a face image processing apparatus that can perform the processing.

[0007]

According to the first aspect of the present invention,
Image capturing means for capturing a driver's face to generate a face image, and face component detecting means for performing image processing based on the face image generated by the image capturing means to detect a driver's face component;
Vehicle state quantity detecting means for detecting a vehicle state quantity; state estimating means for estimating whether or not a driver is facing forward based on the vehicle state quantity detected by the vehicle state quantity detecting means; Storage means for storing the position of the face part detected by the face part detection means when the driver estimates that the driver is facing forward.

[0008] The imaging means captures an image of the driver's face while being installed inside the vehicle. The face part detecting means is configured to output the driver's nose, eyes,
Detect facial parts such as mouth and eyebrows. Here, it is preferable that the imaging means is installed so as to image the driver's face that has received light obliquely from the front, or to image the driver's face obliquely from the front. The reason for this is that if the imaging unit and the light source are installed in front of the driver, it may be difficult to accurately detect the driver's face component by the face component detection unit.

The vehicle state quantity detecting means detects a vehicle state quantity of a vehicle driven by the driver. The vehicle state quantity is not limited to one parameter, and two or more parameters are preferable. The parameters constituting the vehicle state quantity include, for example, the following distance and the relative speed with respect to the preceding vehicle, the steering angle, the own vehicle speed, the accelerator depression amount, the brake depression amount, and the like. Note that a lateral acceleration or a yaw rate can be used instead of the steering angle.

The state estimating means estimates whether or not the driver is facing forward based on the vehicle state quantity. here,
The more parameters that make up the vehicle state quantity, the higher the probability that the driver is facing forward can be estimated.

The storage unit stores the position of the face part detected by the face part detection unit when the state estimation unit estimates that the driver is facing forward. The position of the face part stored in this way is used to determine whether the driver's face is looking aside or falling asleep. That is, since it is possible to reliably detect the state where the driver is facing forward, it is possible to accurately store the reference position of the face for determining the angle of the face.

According to a second aspect of the present invention, the vehicle state quantity detecting means detects at least an inter-vehicle distance to a preceding vehicle and a steering angle, and the state estimating means detects the vehicle state quantity detecting means. When the detected inter-vehicle distance to the preceding vehicle is equal to or less than a predetermined value and the steering angle detected by the vehicle state quantity detecting means is equal to or less than a predetermined value, it is estimated that the driver is facing forward. .

Preferably, the vehicle state quantity detecting means detects at least the following distance and the steering angle with respect to the preceding vehicle in order to estimate whether or not the driver is facing forward. Then, it is determined whether the inter-vehicle distance to the preceding vehicle is equal to or less than a predetermined value and the steering angle is equal to or less than the predetermined value because the inter-vehicle distance to the preceding vehicle decreases when the vehicle is traveling on a straight road. This is because the driver has a high probability of gaze ahead.

According to a third aspect of the present invention, the vehicle state amount detecting means detects an accelerator or brake depression amount and a steering angle, and the state estimating means detects the vehicle state amount detecting means. When the depressed amount of the accelerator or the brake is substantially zero and the steering angle detected by the vehicle state amount detecting means is equal to or smaller than a predetermined value, it is estimated that the driver is facing forward.

The reason why it is determined whether the accelerator depression amount is substantially zero and the steering angle is equal to or smaller than a predetermined value is as follows. Drivers are generally more likely to turn off the accelerator before stepping on the brake, and if the accelerator is turned off while the vehicle is traveling on a straight road, the driver will This is because the probability of gazing at is high.

The reason why it is determined whether the brake depression amount is substantially zero and the steering angle is equal to or smaller than a predetermined value is as follows. When starting the own vehicle, the driver turns off the brake and then depresses the accelerator. This is because the driver has a high probability of turning off the brake after confirming the start or acceleration of the vehicle ahead, and in such a case, it is very likely that the driver turns forward.

[0017]

Embodiments of the present invention will be described below in detail with reference to the drawings.

[First Embodiment] As shown in FIG. 1, a face image processing apparatus according to a first embodiment includes a light source 11 for irradiating a driver with light, and a CC for imaging a driver's face.
A D image sensor 12, an inter-vehicle distance sensor 21 for detecting a distance to a preceding vehicle, a steering angle sensor 22 for detecting a steering angle, and a CRT for displaying an operation during processing, a measurement result and the like.
(Cathode Ray Tube) 31, a face part position memory 40 for storing the position of each part of the driver's face (hereinafter referred to as "face parts"), and a control unit 50 for detecting the face parts of the driver and controlling the whole. And

The light source 11 emits light under the control of the control unit 50, and emits light to the face of the driver sitting in the driver's seat of the vehicle. The CCD image sensor 12 generates a captured image in accordance with the imaging light from the driver, which is a subject, and supplies the captured image to the control unit 50.

Here, the CCD image sensor 12 captures the driver's face that has received light obliquely from the front.
Alternatively, it is installed so that the driver's face is imaged from obliquely forward. Therefore, when the light source 11 is installed so as to emit light obliquely from the front of the driver, the CCD
The image sensor 12 may be installed in any direction, such as directly in front of or obliquely in front of the driver. When the CCD image sensor 12 is installed so as to capture the driver's face diagonally from the front, the light source 11 may irradiate the driver with light from any direction. Light source 11 and CC
The reason for installing the D image sensor 12 as described above is as follows.
This is because the left edge and the right edge of the driver's nose line are made different to facilitate the detection of the nose position, which will be described later.

The inter-vehicle distance sensor 21 is composed of, for example, an optical pulse radar. The inter-vehicle distance sensor 21 transmits pulsed light and receives reflected light, and detects the distance to the vehicle ahead from the round trip time of the pulse from transmission to reception. Then, the following distance sensor 21 supplies the detection result to the control unit 50. The inter-vehicle distance sensor 21 may emit the radio wave radar and detect the inter-vehicle distance to the preceding vehicle based on the round-trip time of the radio wave radar, and is not particularly limited.

The steering angle sensor 22 includes, for example, a slot disk (not shown) attached to the steering main shaft and rotating, and two sets of LEDs and a phototransistor (photo interrupter) (not shown) fixed across the slot disk. It is configured. When the steering main shaft rotates, the photo interrupter generates a pulse signal at each predetermined steering angle in accordance with transmission / blocking of light by the slot disk. Therefore, the steering angle sensor 22 generates a pulse signal indicating the steering angle according to the driver's steering and supplies the pulse signal to the control unit 50.

The CRT 31 is connected to the control unit 50, and displays a binarized face image of the driver detected by the control unit 50, and displays processing contents and processing results of the control unit 50.

The face part position memory 40 is constituted by, for example, a nonvolatile RAM. The control unit 50 performs control to write the positions of the driver's nose and eyes to the face part position memory 40 as initial values, and to read the initial values from the face part position memory 40 as necessary.

The control unit 50 controls the CCD image sensor 12
Then, the position of the driver's nose is detected based on the captured image generated in step (1), and the position of the eye is detected from the position of the nose. Further, the control unit 50 determines a first threshold value for determining whether or not the inter-vehicle distance with the preceding vehicle is short, and determines whether the steering angle is substantially at the neutral position, that is, whether the steering angle is near zero degree. And a second threshold value for performing the operation. Further, the control unit 5
0 is the third threshold value indicating the maximum allowable value of the absolute value of the difference between the initial value of the nose position and the actually detected nose position, the initial value of the eye position, and the detected eye position. And a fourth threshold value indicating the maximum allowable value of the absolute value of the difference.

The control unit 50 has a counter function for counting the number of times the positions of the nose and eyes are detected when the inter-vehicle distance to the preceding vehicle and the steering angle satisfy predetermined conditions.

In the face image processing apparatus configured as described above, the control unit 50 executes the processing from step ST1 shown in FIG. 2 to store the reference position of the face part.

In step ST1, the control unit 50 controls the CCD image sensor 1 to capture the driver's face image.
2 to generate a driver's face image. Then, the control unit 50 stores the face image generated by the CCD image sensor 12, and proceeds to step ST2.

In step ST2, the control unit 50 detects the position of the driver's nose based on the driver's face image stored in the memory. Specifically, the control unit 50 extracts a face area from the face image, extracts a left edge and a right edge from the extracted face area using different thresholds, and extracts the extracted left edge and right edge. A region based on the right edge is extracted as a candidate for a nose ridge.

The control unit 50 selects the most probable candidate from the extracted candidates for the nose line as a nose line using, for example, a condition relating to the vertical length of the candidate region. If the condition is not satisfied, the control unit 50 detects the nose line. Shall not be able to do so.
When extracting using the condition regarding the vertical length of the candidate area, a candidate longer than an appropriate threshold value may be selected, and the longest one from the selected candidates may be selected as a nose line. When the control unit 50 detects the nose muscle as described above, the control unit 50 stores the coordinate position of each pixel constituting the nose muscle, and proceeds to step ST3. The control unit 50 stores the coordinate position of the nose detected first as an initial value of the nose position.

In step ST3, the control unit 50 detects the position of the eye based on the position of the nose detected in step ST2. Specifically, the control unit 50 assumes the detected nose muscle as a line segment, and estimates the eye existence region from the midpoint of the nose muscle. Then, the position of the eye is detected based on the region in which the eye is present, the coordinates of each pixel constituting the position of the eye are stored, and the process proceeds to step ST4. The control unit 50 stores the coordinate position of the eye detected first as the initial value of the eye position.

In step ST4, the control unit 50 compares the inter-vehicle distance with the preceding vehicle detected by the inter-vehicle distance sensor 21 with a first predetermined threshold value. If not more than the threshold value of step ST5
When the inter-vehicle distance to the preceding vehicle is not less than or equal to the first threshold, the process returns to step ST1.

The detection of the inter-vehicle distance with respect to the preceding vehicle is performed when the inter-vehicle distance with respect to the preceding vehicle is equal to or less than the first threshold value, and the driver needs to pay attention to the preceding vehicle. This is because the probability of gazing at is high.

In step ST5, the control unit 50 compares the steering angle detected by the steering angle sensor 22 with a second predetermined threshold value. When the current steering angle is equal to or smaller than the second threshold, that is, when the current steering angle is near zero degree, the process proceeds to step ST6, and when the current steering angle is not equal to or smaller than the second threshold, the process returns to step ST1. .

In step ST6, the control unit 50 calculates the absolute value ε _n of the difference between the initial value of the nose position and the currently detected nose position, and the absolute value ε _n is equal to or smaller than the third threshold value. Is determined. Further, the control unit 50 calculates the absolute value ε _e of the difference between the initial value of the eye position and the currently detected eye position, and determines whether the absolute value ε _e is equal to or less than a fourth threshold. When the absolute value ε _n is equal to or less than the third threshold value and the absolute value ε _e is also equal to or less than the fourth threshold value, the control unit 50 proceeds to step ST7, and selects one of the absolute value ε _e and the absolute value ε _n If it is not less than the corresponding threshold value, the process returns to step ST1.

In step ST7, the control unit 50 sets the number of times the process has shifted from step ST6 to step ST7, that is, the absolute value ε _n is equal to or smaller than the third threshold value and the absolute value ε _e
Is also counted when it is less than or equal to the fourth threshold value. Here, it is assumed that the state in which the absolute value ε _n and the absolute value ε _e are equal to or smaller than the corresponding threshold values is temporally continuous. Therefore, the control unit 50 resets the count value when any one of the absolute value ε _n and the absolute value ε _e is not smaller than the corresponding threshold value even once.
Start counting from zero. When the count value has exceeded the predetermined number, the control unit 50 proceeds to step ST8,
If the count value does not exceed the predetermined number, step S
Return to T1.

In step ST8, the control unit 50 stores the initial values of the positions of the nose and eyes in the face part position memory 40 as reference positions, and ends the processing. The control unit 50
Is not an initial value of the position of the nose and eyes, facial part the average value of the position of the nose and the eye when the absolute value epsilon _n and the absolute value epsilon _e is equal to or less than a predetermined threshold corresponding to each as a reference value It may be stored in the position memory 40.

Further, the control unit 50 can estimate the state in which the driver is facing forward with a higher probability by determining whether or not the inter-vehicle distance to the vehicle ahead is reduced. At this time, specifically, as shown in FIG.
During the process of step 5, the process of step ST9 may be performed.

In step ST9, the control unit 50 determines whether or not the rate of decrease in the inter-vehicle distance with the preceding vehicle is equal to or greater than a fifth threshold value. Here, the control unit 50 sets the parameter equivalent to the rate of decrease in the inter-vehicle distance to the vehicle in front as:
The relative speed with respect to the preceding vehicle obtained by differentiating the inter-vehicle distance with the preceding vehicle with respect to time is used. Then, the control unit 50 determines whether the relative speed with respect to the preceding vehicle is equal to or more than a fifth threshold value,
If it is equal to or greater than the fifth threshold, the process proceeds to step ST5,
If not, the process returns to step ST1. Then, the control unit 50 proceeds to step ST5 in the same manner as in FIG.
Execute the following processing.

The reason for determining whether the rate of decrease in the inter-vehicle distance with the preceding vehicle is equal to or greater than the fifth threshold value is as follows. Because of the rapid approach, the driver must step on the brakes, and in such a case the driver is necessarily more likely to look ahead.

As described above, the face image processing apparatus according to the first embodiment can be used when the inter-vehicle distance to the vehicle in front is equal to or smaller than the first threshold and the steering angle is equal to or smaller than the second threshold. Is
The driver estimates that he is looking ahead, and stores the positions of the driver's nose and eyes at this time. With this, the face image processing apparatus can store the exact reference position of the nose and eyes of the face image, and therefore can use the reference position to determine whether the driver is looking aside or falling asleep. it can.

Further, the face image processing apparatus estimates the state in which the driver is facing forward with a higher probability by adding the inter-vehicle distance reduction ratio judgment in addition to the inter-vehicle distance judgment and the steering angle judgment. The position of the face part at that time can be stored as the reference position.

[Second Embodiment] Next, a second embodiment of the present invention will be described. The same parts as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

As shown in FIG. 4, the face image processing apparatus according to the second embodiment has the same structure as the face image processing apparatus according to the first embodiment, but also includes a wheel speed when the wheel rotates. Is provided. Although the wheel speed sensors 23 are provided corresponding to the respective wheels of the vehicle, one wheel speed sensor 23 is representatively shown in FIG.
Is shown. The control unit 50 stores a sixth threshold for determining whether the rate of increase in the inter-vehicle distance is large and a seventh threshold for determining whether the vehicle is stopped.

In the face image processing apparatus configured as described above, the control unit 50 executes the processing from step ST11 to step ST20 shown in FIG. 5 in order to store the reference position of the face part. Step ST1
In steps 1 to ST13, the control unit 50 executes steps ST1 to ST3 described in the first embodiment.
The same processing is performed.

In step ST14, the control unit 50 compares the inter-vehicle distance with the preceding vehicle detected by the inter-vehicle distance sensor 21 with a first predetermined threshold value. If it is equal to or less than the threshold value of step ST
The process proceeds to 15, and if the inter-vehicle distance to the preceding vehicle is not equal to or less than the first threshold, the process returns to step ST11.

In step ST15, the control unit 50 determines whether the rate of increase in the inter-vehicle distance with respect to the preceding vehicle is equal to or greater than a sixth threshold. Here, the control unit 50 uses a relative speed with respect to the preceding vehicle obtained by time-differentiating the following distance with respect to the preceding vehicle as a parameter equivalent to a rate at which the following distance increases with respect to the preceding vehicle. Then, the control unit 50
It is determined whether the relative speed with respect to the preceding vehicle is equal to or greater than the sixth threshold. If the relative speed is equal to or less than the sixth threshold, the process proceeds to step ST16 assuming that the vehicle is rapidly separated from the preceding vehicle. At this time, the vehicle is not rapidly separated from the preceding vehicle, and the process returns to step ST1.

As described above, the distance between the vehicle and the preceding vehicle is the first.
Is determined to be equal to or less than the threshold value and the rate at which the inter-vehicle distance increases is equal to or greater than the sixth threshold value for the following reason.

When the preceding vehicle starts moving in a state where the preceding vehicle is stopped and the own vehicle is stopped at a certain distance from the preceding vehicle, the driver needs to pay attention to the front in order to start the own vehicle as well. There is. That is, in this case, the driver gazes ahead with a very high probability. Therefore, the control unit 50 detects a state in which the inter-vehicle distance to the vehicle in front is equal to or less than the first threshold value and the rate of increase in the inter-vehicle distance is equal to or more than the sixth threshold value in order to detect this state. ing.

In step ST16, the controller 50 determines whether or not the current wheel speed (own vehicle speed) detected by the wheel speed sensor 23 is equal to or higher than a seventh threshold value. When the current wheel speed is not higher than the seventh threshold, the process returns to step ST11. Here, the control unit 50 excludes a state in which the own vehicle is stopped by performing the process of step ST16.

The reason for excluding the case where the own vehicle is stopped is that if the own vehicle does not start even if the preceding vehicle starts,
This is because such a state is excluded because the driver is not likely to look ahead and is likely to look aside.

The control unit 50 performs each process from step ST17 to step ST19 in the same manner as each process from step ST5 to step ST7 in the first embodiment. In step ST19, the control unit 50
Returns to step ST11 when the count value does not exceed the predetermined number.

Then, in step ST20, the control unit 5
0 is a face part using the initial values of the positions of the nose and eyes as reference positions.
Write to position memory 40. In addition, the control unit 50
Value ε _nAnd the absolute value ε_eIs less than the corresponding threshold
The average value of the positions of the nose and eyes when it is down is calculated as the reference position.
May be written in the face part position memory 40.

As described above, in the face image processing apparatus according to the second embodiment, the inter-vehicle distance to the vehicle in front is less than or equal to the first threshold, and the rate of increase in the inter-vehicle distance to the vehicle in front is the sixth.
If the vehicle speed is not zero and the steering angle is equal to or less than the second threshold, it is estimated that the driver is looking forward, and the positions of the driver's nose and eyes at this time are Is stored as a reference position.

As a result, the face image processing apparatus performs the inter-vehicle distance increase judgment and the own vehicle speed judgment as compared with the first embodiment. Can be estimated. As a result, it is possible to further improve the accuracy of the nose and eye detection positions serving as the reference positions.

Although the present embodiment has been described assuming that the wheel speed and the own vehicle speed are equivalent, it goes without saying that the exact own vehicle speed may be calculated based on the wheel speed of each wheel. .

[Third Embodiment] Next, a third embodiment of the present invention will be described. The same parts as those in the above-described embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

As shown in FIG. 6, the face image processing apparatus according to the third embodiment has the same structure as the face image processing apparatus according to the first embodiment. And a timer 33 for measuring a predetermined time.

The control unit 50 determines whether or not the inter-vehicle distance to the vehicle ahead detected by the inter-vehicle distance sensor 21 has become equal to or less than a first threshold value. The alarm 32 is controlled to emit a sound. The control unit 50 uses the timer 33 to measure a predetermined time.

In the face image processing apparatus configured as described above, the control unit 50 executes the processing from step ST21 to step ST29 shown in FIG. 7 to store the reference position of the face part.

In step ST21, the control unit 50 determines whether the alarm 32 has issued an alarm, waits in step ST21 until an alarm is issued, and shifts to step ST22 when an alarm is issued.

In step ST22, the control unit 50 causes the timer 33 to start timing when the alarm 32 issues an alarm, and determines whether a predetermined time has elapsed based on the timing of the timer 33. Step ST22 until elapse
Then, when a predetermined time has elapsed, the process proceeds to step ST23.

When the alarm 32 issues an alarm, the driver inevitably looks ahead in order to confirm the situation at the time of the alarm. Therefore, by detecting a predetermined time after the alarm 32 issues an alarm, it is possible to detect a state in which the driver gazes ahead.

Then, the control unit 50 determines in step ST23
The processes from to ST25 are performed in the same manner as the processes from ST1 to ST3 in the first embodiment. Further, the control unit 50 determines in step S
About each process from T26 to step ST28,
The processing is performed in the same manner as the processing from step ST5 to step ST7 in the first embodiment. Step ST
At 28, if the count value does not exceed the predetermined number, the control unit 50 returns to step ST23.

Then, in step ST29, the control unit 5
0 is a face part using the initial values of the positions of the nose and eyes as reference positions.
Write to position memory 40. In addition, the control unit 50
Value ε _nAnd the absolute value ε_eIs less than the corresponding threshold
The average value of the position of the nose and eyes when it is down is the reference value
May be stored in the face part position memory 40.

As described above, in the face image processing apparatus according to the third embodiment, a predetermined time has elapsed since the alarm 32 which indicates that the inter-vehicle distance to the preceding vehicle has become short has sounded. If the steering angle is equal to or smaller than the second threshold, the driver estimates that the driver is looking ahead, and stores the positions of the driver's nose and eyes at this time as reference positions. This allows
The face image processing apparatus can improve the accuracy of the detection positions of the nose and the eye, which are the reference positions, as compared with the related art.

[Fourth Embodiment] Next, a fourth embodiment of the present invention will be described. The same parts as those in the above-described embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted.

The face image processing apparatus according to the fourth embodiment has substantially the same configuration as that of the first embodiment, as shown in FIG. 8, and replaces the inter-vehicle distance sensor 21 shown in FIG. In addition, an accelerator depression amount measuring unit 24 is provided.

The accelerator depression amount measuring unit 24 detects how much the driver depresses the accelerator, and supplies the detected accelerator depression amount to the control unit 50.

In the face image processing apparatus configured as described above, the control unit 50 stores the reference positions of the face parts in the steps ST31 to ST31 shown in FIG.
The processing up to 38 is executed.

In step ST31, the control unit 50 determines the accelerator depression amount of the driver by the accelerator depression amount measuring unit 24. If the accelerator depression amount is not zero, the control unit 50 waits in step ST31 until the accelerator is turned off. Becomes zero when step ST
Move to 32.

The reason why the accelerator depression amount is determined to be zero is that the probability that the driver turns off the accelerator before stepping on the brake is high. This is for detecting a state where is high.

Then, the control unit 50 determines in step ST32
The processes from to ST34 are performed in the same manner as the processes from ST1 to ST3 in the first embodiment. Further, the control unit 50 determines in step S
Regarding each process from T35 to step ST37,
The processing is performed in the same manner as the processing from step ST5 to step ST7 in the first embodiment. Step ST
At 37, if the count value does not exceed the predetermined number, the control unit 50 returns to step ST32.

Then, in step ST38, the control unit 5
0 is a face part using the initial values of the positions of the nose and eyes as reference positions.
Write to position memory 40. In addition, the control unit 50
Value ε _nAnd the absolute value ε_eIs less than the corresponding threshold
The average value of the position of the nose and eyes when it is down is the reference value
May be written to the face part position memory 40.

As described above, in the face image processing apparatus according to the fourth embodiment, when the accelerator depression amount is zero and the steering angle is equal to or smaller than the second threshold, the driver looks forward. The position of the driver's nose and eyes at this time is stored as a reference position. Thereby, the face image processing apparatus can improve the accuracy of the detection positions of the nose and the eyes, which are the reference positions, as compared with the related art.

The face image processing apparatus may include a brake depression amount measuring unit 25, for example, as shown in FIG. 10, instead of the accelerator depression amount measuring unit 24. The brake depression amount measuring unit 25 detects how much the driver depresses the brake, and supplies the detected brake depression amount to the control unit 50.

In the face image processing apparatus configured as described above, the control unit 50 executes the processing of step ST39 as shown in FIG. 11, instead of the processing of step ST31 described above.

In step ST39, the control unit 50 determines the amount of depression of the brake by the driver by the brake depression amount measurement unit 24. If the brake depression amount is not zero, the control unit 50 waits in step ST39 until the brake is turned off. Becomes zero when step ST
Move to 32.

The reason for determining whether or not the brake depression amount is zero is that when the host vehicle starts from a stopped state, the driver normally depresses the brake after depressing the accelerator. That is, since the driver has a high probability of turning off the brake after confirming the start or acceleration of the vehicle ahead, by detecting this state, the state in which the driver gazes ahead is detected.

As described above, the face image processing apparatus estimates that when the brake depression amount is zero and the steering angle is equal to or smaller than the second threshold, the driver is looking forward.
The positions of the driver's nose and eyes at this time are stored as reference positions. Thereby, the face image processing apparatus can improve the accuracy of the detection positions of the nose and the eyes, which are the reference positions, as compared with the related art.

The present invention is not limited to the first to fourth embodiments, and various design changes can be made within the scope of the invention described in the claims. .

For example, according to the flowchart shown in FIG. 3, the control unit 50 performs the processing in the order of the following distance determination, the determination of the reduction rate of the following distance, and the determination of the steering angle. However, the processing may be performed in a different order. That is, the control unit 50 is not limited to the order of each flowchart in each embodiment,
The processing may be performed in a different order.

In the above-described embodiment, the nose and eyes are detected as the face parts, but face parts such as the mouth and eyebrows may be detected. In addition, the method of detecting the nose and eyes is not limited to the above-described embodiment, and it goes without saying that other known methods may be used.

Further, the control unit 50 may be configured by installing a program for executing the above-described processing in a computer.

[0085]

The present invention estimates whether or not a driver is facing forward based on the vehicle state quantity detected by the vehicle state quantity detecting means, and estimates that the driver is facing forward. By storing the position of the detected face part, the state where the driver is facing forward is reliably detected, and the reference position of the face part for determining the angle of the driver's face is accurately determined. Can be memorized.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a face image processing apparatus according to a first embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation procedure of a control unit.

FIG. 3 is a flowchart illustrating an operation procedure of a control unit.

FIG. 4 is a block diagram illustrating a configuration of a face image processing device according to a second embodiment of the present invention.

FIG. 5 is a flowchart illustrating an operation procedure of a control unit.

FIG. 6 is a block diagram showing a configuration of a face image processing device according to a third embodiment of the present invention.

FIG. 7 is a flowchart illustrating an operation procedure of a control unit.

FIG. 8 is a block diagram showing a configuration of a face image processing device according to a fourth embodiment of the present invention.

FIG. 9 is a flowchart illustrating an operation procedure of a control unit.

FIG. 10 is a block diagram showing another configuration of the face image processing apparatus according to the fourth embodiment of the present invention.

FIG. 11 is a flowchart illustrating an operation procedure of a control unit.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 12 CCD image sensor 21 Inter-vehicle distance sensor 22 Steering angle sensor 23 Wheel speed sensor 24 Acceleration depression amount measurement unit 25 Brake depression amount measurement unit 40 Face part position memory 50 Control unit

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Shinichi Kojima 41-Cho, Yokomichi, Nagakute-cho, Aichi-gun, Aichi Prefecture Inside Toyota Central Research Institute Co., Ltd. (72) Inventor Yuji Uchiyama Oji-cho, Nagakute-cho, Aichi County, Aichi Prefecture No. 41, Yokomichi, Toyota Central Research Laboratory Co., Ltd. (72) Inventor Nobuyuki Shiraki, No. 41, Chuchu, Yoji, Nagakute-cho, Aichi-gun, Aichi Prefecture F-term in Toyota Central Research Laboratory Co., Ltd. 3D037 FA05 FA24 FA25 FA26 5H180 AA01 CC04 LL20 5L096 BA02 BA04 CA02 FA06 FA69 JA11

Claims (3)

[Claims] 1. An image pickup means for picking up a driver's face to generate a face image, and a face part detection for performing image processing based on the face image generated by the image pickup means to detect a driver's face part Means, a vehicle state quantity detecting means for detecting a vehicle state quantity, and a state estimating means for estimating whether or not the driver is facing forward based on the vehicle state quantity detected by the vehicle state quantity detecting means, A storage unit for storing a position of the face part detected by the face part detection unit when the state estimation unit estimates that the driver is facing forward; 2. The vehicle state quantity detecting means detects at least an inter-vehicle distance to a vehicle ahead and a steering angle, and the state estimating means detects a distance between the vehicle and the preceding vehicle detected by the vehicle state quantity detecting means. 2. The face according to claim 1, wherein when the inter-vehicle distance is equal to or less than a predetermined value and the steering angle detected by the vehicle state quantity detecting means is equal to or less than a predetermined value, it is estimated that the driver is facing forward. Image processing device. 3. The vehicle state amount detecting means detects an accelerator or brake depression amount and a steering angle, and the state estimating means detects an accelerator or brake depression detected by the vehicle state amount detecting means. Quantity is almost zero,
3. The face image processing apparatus according to claim 1, wherein when the steering angle detected by the vehicle state quantity detection means is equal to or smaller than a predetermined value, the driver is estimated to be facing forward.