KR100234590B1 - Apparatus and method for curing face-burn - Google Patents

Abstract

Provided are a face image processing apparatus and a face image processing method capable of more accurately determining the opening and closing of an eye of a photographed box even when the face image state changes.

Opening detection means for detecting the opening of the eye of the photographing box by the imaging information of the camera, and using the threshold value to classify the detection result of the opening detection means into a plurality of groups, and at the same time, the opening detection means Time measurement means for integrating time for which the detection result of the data belongs to the designated group in the plurality of groups, threshold value evaluation means for evaluating the validity of the threshold value based on the measurement result of the time measurement means, and the threshold value evaluation means. Threshold value generating means for resetting the threshold value based on the evaluation result, and opening / closing determination means for limiting opening and closing of the eye of the shooting box using the threshold value.

Description

Facial image processing device and facial image processing method

The present invention relates to a face image processing apparatus and a face image processing method which determine an opening / closing of an eye of a photographed box by simultaneously processing a picture of a person's face with a camera.

Conventionally, a vehicle driver's face is imaged by a camera installed in a vehicle cabin, the face obtained by this is image-processed, eyes are extracted from them, and opening / closing of the eyes is detected, thereby driving the driver's glance or drowsiness. It is proposed to detect the back and the like. As an example of such a device, there is a display in Japanese Patent Laid-Open No. 6-32154. In this publication, a face image is processed to set an area in which the eye in the image is considered to be located, and the black area in this area is determined by the eye. Therefore, the open or closed state of the eye is determined according to whether or not the black region is larger than the predetermined threshold value. A glance or drowsiness of the vehicle driver is detected by the open / closed state of the eye.

However, in the apparatus of the publication, a threshold value as a reference for determining the open state or closed state of the eye is fixed.

Therefore, when the distance between the camera and the shooting box is changed or the state of the face image is changed by changing the direction of the face, it is difficult to accurately detect the open or closed state of the eye at the fixed threshold.

SUMMARY OF THE INVENTION An object of the present invention is to provide a face image processing apparatus and a face image processing method capable of more accurately determining the opening / closing of an eye of a photographing box as a means for solving such a problem. do.

The face image processing apparatus according to the present invention uses a camera that captures a face image, opening detection means for detecting the opening of an eye of a photographing box based on the imaging information of the camera, and threshold values for detection results of the opening detection means. And time threshold means for integrating and measuring the time which the detection result of the open detection means belongs to within the specified group within the specified time, and the threshold value is determined by the measurement result of the time measurement means. A threshold value evaluating means for evaluating the validity of the subject, a threshold value setting means for resetting the threshold value based on the evaluation result of the threshold value evaluating means, and an eye opening and closing for determining whether the eye of the subject is opened or closed using the threshold value. The determination means is provided.

Further, the designated group of the face image processing apparatus according to the present invention is two groups in the plurality of groups classified by the threshold value, and the threshold value evaluating means is the ratio of the measurement result to the two groups measured by the time measuring means. By calculating the deviation between the predetermined constant and the predetermined value, the threshold setting means updates the value of the threshold in a direction in which the deviation decreases.

Further, the facial image processing apparatus according to the present invention sets the initial value of the threshold at a position separated by a predetermined amount from the average value of the detection result of the threshold setting means and the opening detection means, and the detection output of the opening detection means is within a predetermined time. If the threshold value is not crossed, the threshold value is updated in the direction of the average value.

The face image processing apparatus according to the present invention further includes an open value calculating means for calculating a representative value when the eye of the shooting box is opened by the detection result of the opening detection means, and the detection result of the opening detection means. A closed value calculating means for calculating a representative value when the imager's eyes are in the closed state, and a threshold value limiting means for setting a variable range of the threshold value based on the calculation result of the open value calculating means and the closed value calculating means. It is.

The face image processing apparatus according to the present invention further includes period detecting means for detecting that the apparatus has elapsed for a certain period from the operation time, and permits updating of a variable range of the threshold value until the predetermined period elapses, and at the same time. After the elapse of, the threshold limit control means for prohibiting the updating of the threshold variable range is provided.

The face image processing apparatus according to the present invention further includes a camera for photographing a face image, opening detection means for detecting the opening of an eye of a photographed box based on the imaging information of the camera, and detection results of the opening detection means at predetermined time intervals. A tap water measuring means for prompting a tap water distribution, a threshold value setting means for setting a threshold value within a range where the tap water distribution becomes substantially flat and updating the threshold value by updating the tap water distribution, and using a threshold value Eye opening and closing determination means for determining the opening and closing of the eye of the imager.

In addition, the facial image processing method according to the present invention comprises the steps of detecting the opening of the eye of the shooting box according to the imaging degree of the camera for imaging the face image, classifying the detection result into a plurality of groups using a threshold value; Integrating a time when the detection result of the eye open within a predetermined time belongs to the designated group in the plurality of groups, evaluating the validity of the threshold value based on the measurement result of the time, and thresholding the evaluation result Resetting the value, and determining the opening and closing of the eye of the subject using the threshold value.

In addition, the face image processing method according to the present invention comprises the steps of detecting the opening of the eye of the photographing box by the imaging information of the camera for imaging the face image, and measuring the distribution of the tap water of the detection result of the opening of the eye every fixed time. And setting a threshold value within a range in which the tap water distribution is substantially flat, updating the threshold value by updating the tap water distribution, and determining the opening and closing of the eye of the shooting box by using the threshold value. do.

1 is a block diagram showing the configuration of Embodiment 1 according to the present invention;

2 is a time chart illustrating the operation of Embodiment 1 according to the present invention.

3 is another time chart illustrating the operation of Embodiment 1 according to the present invention.

4 is another time chart illustrating the operation of Embodiment 1 according to the present invention.

5 is a flowchart showing a procedure for detecting an output value of eye patch evaluation.

6 is an explanatory diagram showing a description of the output value of the eye open / close evaluation.

Fig. 7 is an explanatory diagram showing a description of the pattern determination operation of the second embodiment according to the present invention.

8A to 8F are explanatory views showing the explanation of the operation of the threshold evaluation means according to the third embodiment of the present invention.

9 is an explanatory diagram showing the state of update of a threshold value.

10 is a time chart illustrating the operation of Embodiment 4 according to the present invention.

11 is another time chart illustrating the operation of Embodiment 4 according to the present invention.

12 is a time chart illustrating the operation of Embodiment 4 according to the present invention.

13A to 13F are explanatory views showing the description of the operation of the threshold evaluation means according to the fourth embodiment of the present invention.

14 is a time chart showing the state of update of a threshold value.

FIG. 15 is an explanatory diagram showing a description of the operation of the fifth embodiment according to the present invention; FIG.

16 is a block diagram showing a configuration of Embodiment 6 according to the present invention.

17 is a block diagram showing a configuration of Embodiment 7 according to the present invention.

18 is a block diagram showing a configuration of Embodiment 8 according to the present invention;

19 is a block diagram showing a configuration of Embodiment 9 according to the present invention.

20 is an explanatory diagram showing a threshold setting method of Embodiment 10 according to the present invention.

21 is an explanatory diagram showing yet another threshold setting method of Embodiment 10 according to the present invention;

Fig. 22 is a block diagram showing the constitution of Embodiment 10 according to the present invention.

<Explanation of symbols for main parts of drawing>

1: camera 2: image input means

3: eye area determining means 4: eye opening and closing evaluation means

5: time measuring means 6: threshold evaluation means

7, 21: threshold setting means 8: opening and closing determination means

9: maximum open value recording means 10: maximum open value filter

11: minimum closing value recording means 12: minimum opening value filter

13, 15: threshold limit means 14: open value filter

16, 18: threshold limit control means 17: timer

19: minimum closing value recording counter 20: frequency measurement means

Example 1

In Example 1, detection of opening of an eye of a photographing box by imaging information of a camera that captures a face image, classifying the detection result into a plurality of groups using a threshold value, and the detection result Classifying the data into a plurality of groups using a threshold value, integrating the time when the detection result of the eye opening within a predetermined time belongs to a designated group in the plurality of groups, and the threshold value based on the measurement result of the time. A method of processing a face image comprising the step of evaluating the validity of the image, the step of resetting the threshold value based on the evaluation result, and determining the opening / closing of the eye of the photographing box using the threshold value. A facial image processing apparatus that performs the method will be described next.

1 is a block diagram showing a schematic configuration of a face image processing apparatus of Example 1, and FIGS. 2 to 4 are time charts illustrating the operation of Example 1, and FIG. 2 is a state in which a threshold is reasonable. 3 illustrates a state in which the threshold value is excessively moved toward the open side, and FIG. 4 illustrates a state in which the threshold value is excessively moved toward the closed side.

Before giving a detailed description of Embodiment 1, the basic operation will be briefly described.

The face image of the vehicle driver is obtained by the camera. The face image is processed to detect the eyes of the vehicle driver, which is a photographed box, to obtain an output by opening the eyes. This is indicated as the open and close evaluation output of the eye in FIGS. 2 to 4. The output of the eye opening is compared with the closed threshold of the two thresholds, and when it is located on the open side than the closed threshold, it is determined that the eyes are open, and when it is located on the closed side more than the closed threshold. It is judged that the eyes are closed. Thus, opening and closing of the eye is determined.

On the other hand, the threshold value is not updated but is sequentially updated by the change in the face image. The threshold is first determined for its validity, not valid, and updated in the desired direction.

For example, in FIG. 3, the threshold value is excessively moved toward the open side. In such a state, even if the eyes of the vehicle driver are not closed, it is determined even in the closed state.

4, the threshold is excessively shifted toward the closed side. In such a state, opening and closing of the eyes of the vehicle driver cannot be determined.

In Example 1, it processes as follows to determine which state in FIGS. First, the eye open / close evaluation output value is classified into a plurality of groups using two threshold values, an open threshold and a closed threshold. An arbitrary group is selected from among them, and the time when the eye-opening evaluation output value belongs to the designated group selected within a predetermined time is measured. In the first embodiment, the designated group is a group A which is an open area than the open threshold and a group B which is an open area than the closed threshold.

Next, the measured value of group A and the measured value of group B are compared. In the case of FIG. 3, the measured value of the group A is very small compared with the measured value of the group B. FIG. In addition, in the case of FIG. 4, the measured value of the group A and the measured value of the group B are substantially the same.

On the other hand, when the threshold of FIG. 2 is valid, the measured value of the group A is slightly smaller than the measured value of the group B. FIG.

That is, in Example 1, the deviation between the measured value of the group A and the measured value of the group B is calculated, and the closed threshold value is controlled so that the deviation is also shown in FIG. In addition, the open threshold changes with the change of the closed threshold, and the interval between the closed threshold and the open threshold is constant.

Next, Example 1 will be described in detail.

In FIG. 1, reference numeral 1 denotes a camera installed in a vehicle cabin for capturing a face of a vehicle driver, and reference numeral 2 denotes an A / D (analog / digital) by receiving an image signal which is imaging information of the camera 1. Image input means for converting and outputting a digital gradation image, and reference numeral 3 designates an eye region where the driver's eyes are considered to be among the images which received the digitally modified image from the image input means 2). Eye area determination means, and reference numeral 40 is an eye open / close evaluation means for the open detection means for outputting the detection result by the opening of the eyes of the vehicle driver, for example, the eye specified by the eye area determination means 3. In the area, the value by the size of the black area is output as shown in the eye-opening evaluation output value in FIG. This is a value due to the opening of the eyes of the vehicle driver.

Reference numeral 5 denotes a time measurement means, which classifies the detection result of the eye open / close evaluation means 4 into a plurality of groups using two threshold values, an open threshold value and a closed threshold value shown in FIG. .

The plurality of groups include an area open to the open threshold, an area closer to the open threshold than the open threshold, an area open to the closed threshold, and an area closed to the stepped threshold and two. Area between thresholds and the like.

In the time measuring means 5 of the first embodiment, group A, which is an area that is open more than an open threshold value, and group B, which is an area more open than a closed threshold value, are selected as a designated group, The detection result of the opening / closing evaluation means 4 calculates by integrating the time which belongs to group A or group B. Reference numeral 6 evaluates the validity of the threshold value based on the measurement result of the time measuring means 5 in the threshold value evaluation means. Reference numeral 7 resets the threshold value to a more desirable value by the evaluation result of the threshold value evaluation means 6 in the threshold value setting means. The threshold value set by the threshold value setting means 7 is given to the time measurement means 5 mentioned above and the opening / closing determination means described later. Reference numeral 8 denotes an eye opening and closing determination means for determining the opening and closing of the eyes, wherein the value due to the opening of the eyes of the vehicle driver, which is given by the eye opening and closing evaluation means 4 as the opening and closing determination means, The given threshold value is input, and the opening / closing of the driver's eyes is determined by comparing the value of the eye opening with the closed threshold. The operation of the first embodiment will be described. 5 is a flowchart for detecting an eye occlusion evaluation output value. The flowchart specifies the eye area in the face image picked up by the camera 1, determines the black area in the eye area as the eye, and detects the size of the black area as the eye opening.

In step (1), the video signal captured by the camera (1) is embedded in the image input means (2). The image input means 2 A / D converts the video signal. In step 2, the digital gradation image converted by the image input means 2 is input to the frame memory. In step (3), the image information stored in the frame memory is detected, and the two-value image is formed by a threshold value which is two suitable values. In step (4), the white pixels are searched from the horizontal search start line of the face in the horizontal direction of the face. In step 5, the width contour of the face is specified at the end of the white pixel area when the number of consecutive white pixels is counted and in the subsequent step 6 the number of consecutive white pixels is maximum.

In step 7, the horizontal contour of the specified face determines the horizontal coordinates of the eye area where the eye is considered to be present. In step 8, regions of consecutive black pixels are searched. In step 10, the eye region is specified by the positional relationship of the found black pixel region or the number of vertical black pixels. In step 11, consecutive black pixels are detected in the specified eye area, and this is detected and output as the eye opening. That is, as shown in FIG. 6, the eye region is searched in the vertical direction, and the search and output is performed so that the number of consecutive black pixels is maximum. That is, as shown in FIG. 6, the eye region is searched in the vertical direction, and a search line in which the number of consecutive black pixels is maximum is detected. In this case, the search line and the number of consecutive black pixels above are detected as the opening of the eye. Doing. However, the search line (bar) has a larger number of continuous black pixels than the search line (D). However, the search line d is selected by setting a specified condition that omits the hair or the eyebrows.

The eye opening thus obtained increases the number of continuous black pixels in the open state and decreases the number of continuous black pixels in the closed state, resulting in a value due to eye opening.

In this flowchart, step (1) is image input means, steps (7) to (10) are eye area determination means, and step (11) constitutes open detection means.

The eye opening obtained as described above is displayed as, for example, an eye-opening evaluation output value in FIG. 2. This value slightly fluctuates depending on the degree of opening of the driver's eyes, and is output as a value in the closed direction when the driver closes the eyes.

The eye open / close evaluation output values are input to the time measuring means 5 and classified into a plurality of groups consisting of two thresholds, an open threshold and a closed threshold. In the time measuring means 5, the time during which the eye open / close evaluation output value belongs to the group A, that is, the time of the oblique portion in FIG. 2 is accumulated and measured every fixed time. Similarly, the time which the eye open / close evaluation output value belongs to group B within a predetermined time is also integrated.

This result is given to the threshold value evaluation means 6, where the validity of the threshold value is evaluated. The above-mentioned validity evaluation is evaluated by the deviation between the measured value of the group A and the measured value of the group B measured by the time measuring means 5 as described above. The deviation is given to the threshold value setting means 7.

The threshold setting means 7 controls and resets the value of the closed threshold value by the deviation calculated by the threshold evaluation means 6. The open threshold changes equally by updating the closed threshold.

2 is a case where the setting of the threshold is valid, the open threshold is below the fluctuation of the eye-opening evaluation output value, and the closed threshold is set to the closed side again than the open threshold. Since the opening and closing of the eye is determined based on the closed threshold value, in this state, the opening and closing of the eye is not wrongly determined by the fluctuation of the eye opening and closing evaluation output value when the eye is in the open state. In this case, the measurement deviation between group A and group B (group B-group A) is slightly larger than zero.

3 is a state in which the threshold value is excessively biased toward the open side. In this case, when the eye is in the open state, the closed threshold value is applied to the fluctuation of the open / closed evaluation output value of the eye, and the eye is incorrectly judged to be in the closed state even when the eye is open.

In this state, the deviation (group B-group A) is a very large value. In such a case, the threshold setting means 7 updates and resets the value of the closed threshold so that the deviation becomes a value slightly larger than zero as in the case of FIG.

4 is a state in which the threshold value is excessively moved toward the closed side. In this case, since the closed threshold value is set to the closed side again than the value when the eyes are closed, the opening and closing of the eye can be determined.

In this state, the deviation (group B-group A) is almost zero. In such a case, the value of the closed threshold value is reset so that the threshold value setting means 7 becomes a value slightly larger than 0 as in the case of FIG.

The update of the closed threshold value is performed every fixed time. That is, the evaluation by the threshold evaluation means 6 updates a fixed value every fixed time in the direction approaching the position of a preferable threshold value.

As described above, the reset two threshold values are given to the time measuring means 5, and used for group classification, and given to the opening / closing determination means 8, and the eye-opening evaluation output value and the closed threshold value are By comparison, the opening and closing of the eyes is determined.

Therefore, according to the first embodiment, the threshold value is updated to an appropriate value due to the change of the picked-up image, so that the accuracy of eye open / close judgment can be improved.

In Example 1, the maximum number of continuous black pixels of the eye region extracted as the eye opening / closing evaluation was explained, but the present invention is not limited thereto. For example, Japanese Patent Application Laid-Open No. 7 The eye open / close evaluation method described in -193647 may be applied.

In the first embodiment, two groups, group A and group B, were selected as the group designated by using two threshold values, the open threshold value and the closed threshold value. However, one threshold value and one specified group may be used.

That is, only the closed threshold value shown in Example 1 may be used. Specifically, the opening / closing evaluation output values of the eyes are classified into a plurality of groups, that is, a group of an open area rather than the closed threshold value, using the closed threshold values shown in FIGS. 2 to 4. Therefore, as the designated group, for example, the open area, that is, group B, is selected rather than the closed threshold value. At this time, the measured value integrated by the time measuring means 5 becomes a value slightly smaller than 1 in the state of FIG. In addition, in the case of FIG. 3, it becomes much smaller than 1, and in case of tap water, it becomes almost 1.

Therefore, the threshold value setting means 7 may update the value of the closed threshold value so that the measured value of the time measurement means 5 is slightly smaller than one.

This can simplify the operation and increase the processing speed compared with that described in the above-described first embodiment.

Example 2

Example 2 is substantially the same as Example 1, and differs only in the eye open / close determination method of the open / close determination means 3. That is, in Example 1, the opening and closing of the eye was determined by comparing the closed threshold value and the open / closed evaluation output value of the eyes. In Example 2, the closed and closed threshold value and the open side threshold value were compared with the open / closed evaluation output value of the eyes. Is determined. 7 is an explanatory diagram showing the operation of the opening and closing determination means according to the second embodiment.

In Example 2, when the eye open / close evaluation output value becomes closed rather than the closed threshold value, it determines with the eyes closing from closing. In addition, when the eye open / close evaluation output value becomes the open side rather than the open side threshold value, it is determined that the eyes are closed to open. In other words, hysteresis is provided to determine the opening and closing of the eye.

Therefore, even if the vehicle driver slowly opens and closes the eye and the eye open / close evaluation output value is swinging, the determination result of the open / close determination means 8 does not cause charting.

Example 3

Example 3 evaluates the threshold more accurately. The configuration of the third embodiment is almost the same as that of the first embodiment, and only the operation of the threshold evaluation means 6 is different from the first embodiment. 8 is an explanatory diagram showing the operation of the threshold value evaluation means according to the third embodiment.

FIG. 8A shows an example of the tap water distribution with respect to the output of the open state of the eye of the eye open / close evaluation means 4 at a given time. Next, the integration time of group A and group B is shown in FIG. 8B and FIG. 8C, respectively. The integration time of group A is the partial area shown to the right of the open threshold, and the integration time of group B is the partial area shown to the right of the closed threshold, and the integration time of group B is the closed threshold. Rather, it is the partial area shown in the right direction. Here, the parameter for the closed threshold changes from the closed side to the open side. In addition, the open threshold changes at regular intervals from the closed threshold. The integration time of group A of group A and group B at this time is divided by the integration time of group B. 8F shows the result of its division. The divided value decreases only as the closed threshold moves toward the open side. This monotonous change value is preferable as a comparison target.

In the third embodiment, the threshold evaluation means 6 calculates the ratio of the measurement result of the group A to the measurement result of the group B as described above, and compares this ratio with a predetermined constant value to calculate the deviation. The calculated deviation is input to the threshold setting means 7, and the closed threshold is reset in the direction in which the deviation decreases. The predetermined constant value is an empirically obtained value, in which the eye open / close evaluation output value and the threshold value are in a positional relationship as shown in FIG. 2.

Therefore, in Example 3, the threshold value can be evaluated more accurately by comparing the monotonically changing value with the predetermined value.

In addition, in the above-described embodiment, as shown in Fig. 9, the threshold value is updated by a predetermined value every predetermined time. In this example, seven updates are required until the desired threshold is reached.

On the other hand, in the third embodiment, the predetermined value is compared with the monotonically changing value, and the predetermined value is compared to calculate the deviation, and the deviation is calculated, so that the deviation monotonically separates the distance between the desired threshold value and the current threshold value. Display. In other words, if the deviation is large, the preferable threshold value and the current threshold value are largely separated, and if the puncture is small, it can be said that the preferable threshold value and the current threshold value almost match.

Therefore, the threshold value setting means 7 may update by the magnitude | size of a deviation, and may make it the current threshold value match with a preferable threshold value in 7 updates.

Alternatively, as the desired threshold is separated from the current threshold, the amount of change by seven updates may be increased. For example, there is a method of changing the threshold value by an amount that resolves 80% of the deviation by the seventh update, and then changing the threshold value by a constant value every fixed time.

In this way, the thresholds can be quickly matched by the desired thresholds.

Example 4

In Example 1, Group A and Group B were used as the designated group.

The fourth embodiment shows an example in which the designated group is changed, and basically the same as the first embodiment. 10 to 12 are time charts illustrating the operation of the fourth embodiment.

10 shows a state where the threshold is in a reasonable position. In the fourth embodiment, as a predetermined group, group A, which is an open area, and group C, which is an area between the open and closed threshold values, are used as the predetermined group.

Since the basic operation is the same as in the first embodiment, only the evaluation of the threshold value is described here.

In Fig. 10, the time measuring means 5 calculates and measures the group C while integrating and measuring the time when the eye open / close evaluation output value belongs to the loop A within a predetermined time. At this time, the group A and the group C are also calculated and measured. The deviation (group A-group C) of the measured values of group A and group C at this time becomes a value slightly smaller than the measured value of group A.

11 is a state in which the threshold value is excessively biased toward the open side. At this time, the measured value of group A decreases and the measured value of group C increases. Therefore, the deviation (group A-group C) becomes negative in the extreme case.

12 is a state in which the threshold is biased to the closed side. At this time, while the measured value of the group A increases, in the extreme case, the measured value of the group C becomes zero. The deviation (group A-group C) is therefore the measurement of group A itself in extreme cases.

Therefore, the threshold setting means 7 receives these deviations, and updates and resets the threshold values so that the deviations are slightly smaller than the measured values of the group A, that is, the state shown in FIG.

Also in the fourth embodiment, the hysteresis may be provided in the eye open / close determination as in the second embodiment described above.

Next, also in Example 4, the meaning which can improve the precision of a threshold value warning like Example 3 mentioned above is demonstrated.

FIG. 13 is an explanatory diagram showing another operation of the threshold value evaluation means according to the fourth embodiment, which is the same as that shown in FIG.

FIG. 13A shows an example of an eye open state output number distribution of the eye open / close evaluation means 4 at a given time. Next, the integration times of the groups A and C are shown in Figs. 13B and 13C, respectively. The integration area of group A is a partial area shown to the right of the open threshold, and the integration time of group C is a partial area divided into a closed threshold and an open threshold. Here we change the closed threshold as a parameter. In addition, the open threshold is also changed at regular intervals from the closed threshold. The change of integration time of group A and group C at this time is as shown to (d) and (e). Next, the integration time of group C is divided by the integration time of group A. Reference numeral f denotes the result of the division. This divided value monotonously increases as the closed threshold moves toward the open side. This monotonous change value is preferable as a comparison target.

In Example 4, in the threshold value evaluation means 6, the ratio of the measurement result of the group A and the measurement result of the group C is calculated as described above, and the deviation is calculated by comparing this ratio with a predetermined constant value. Doing. The calculated deviation is input to the threshold value setting means 7, and the closed threshold value is reset in the direction in which the deviation decreases. The predetermined constant value is an empirically obtained value in which the eye-opening evaluation output value and the threshold value are in a positional relationship as shown in FIG.

Therefore, in Example 4, the threshold value can be evaluated more accurately by comparing the monotonically changing value with the predetermined value.

In addition, in the above-mentioned Examples 1 to 4, the difference between the constant value and the two threshold values used as the threshold value evaluation means 6 is set so as to set an appropriate value by the output characteristics of the eye opening / closing evaluation means 4 to be used. Just do it.

Example 5

Embodiment 5 relates to an initial setting of a threshold value, in which the threshold value is quickly set to a desired value after the device is started.

Fig. 14 shows how the threshold value is updated. In Examples 1 to 4 described above, Group A, which is an open area than the open threshold, Group B, which is an open area than the closed threshold, and an area between the closed and open thresholds. You are using group C.

Therefore, in the case of FIG. 14, the measured values of any group also become zero, and evaluation of the validity of the threshold value becomes impossible. In this case, for example, if it has been updated by a certain value every open time, the threshold value gradually moves away from the eye open / close evaluation output value, and the eye open / close cannot be determined.

Example 5 prevents such an irregularity from occurring.

Specifically, the initial value of the threshold is set on the closed side of the average value of the eye-opening evaluation output value, and when the eye-opening evaluation output value does not cross any of the two threshold values within a predetermined time, the threshold value is updated to the open side. have.

The shape is shown in FIG. In Fig. 15, it can be seen that the threshold value is quickly updated to the desired position after the device is started.

In this embodiment, the predetermined time is about 10 seconds. In addition, the initial value of a threshold value experiences previously set the likely value to the closed side rather than the average value of the eye-opening evaluation output value.

Therefore, according to the fifth embodiment, the threshold value can be quickly updated to the position of the desired threshold value after long-term start-up.

Alternatively, the average value of the eye open / close evaluation output values may be calculated at a fixed time after the device is started, and the initial value of the threshold value may be set at a position closed by a certain amount from the average value.

In addition, at a fixed time after the device is started, the value of the closest side of the eye-closure evaluation output value may be detected and the initial value of the threshold may be set at a position slightly closed than the value.

In the above-described embodiment, the initial value of the threshold value is set to the closed side than the average value of the eye-opening evaluation output value, and the threshold value is updated to the open side if the eye-opening evaluation output value does not cross the threshold value within a predetermined time.

Conversely, the initial value of the threshold value may be set to the open side than the average value of the eye-opening evaluation output value, and the threshold value may be updated to the closed side if the eye-opening evaluation output value does not cross the threshold value within a predetermined time.

However, as described in the above embodiment, the preferable threshold value is located on the closed side by a certain amount than the average value of the eye-opening evaluation output value. Therefore, it should be noted that the time until the threshold value is updated to the desired position is slightly longer than the initial setting of the threshold value on the closed side than the average value of the eye-opening evaluation output value.

Example 6

In the above-described embodiment, the updating is performed so that accurate eye open / close determination is possible even when the image pickup conditions of the vehicle driver change. However, if the driver tries to close his eyes due to sleepiness or the like, the threshold value is updated in accordance with this, and there is a possibility that the threshold value is excessively shifted to the closed side in comparison with each castle.

In the sixth embodiment, a range in which the threshold value can be updated, that is, a variable range is set, and the threshold value is prevented from deviating from the preferred range.

16 is a block diagram showing a configuration of the sixth embodiment. In the drawings, reference numerals 1 to 8 are the same as or equivalent to those described in the above-described embodiment. The reference numeral 9 denotes the maximum open value for recording the maximum open value which is the maximum open side value of the operation opening / closing evaluation means 4 output when it is determined that the eye is opened by the opening / closing determination means 8 within a predetermined time. With recording means, the maximum open value is a representative value when the eye is in the open state. In the reference numeral 10, the recording value of the maximum open value recording means 9 is input, every fixed time. As the maximum open value filter for performing the filtering process, the open value calculation means is configured together with the maximum open value recording means 9. Reference numeral 11 denotes a minimum closed value in which the output of the eye open / close evaluation means 4 records the minimum closed value at which the output of the eye open / close evaluation means 4 is determined to be in the closed state within the predetermined time. As the value recording means, the minimum closed value is a representative value when the eyes are in the closed state. Reference numeral 12 designates a closed value calculating means together with the minimum closed value recording means 11 as a minimum closed value filter in which the recording value of the minimum closed value recording means 11 is input and performs a filtering process every fixed time. Reference numeral 13 denotes a threshold limit means for inputting the output of the maximum open value filter 10 and the minimum closed value filter 12 to filter the maximum open value obtained within a predetermined time and the value obtained within a predetermined time. The variable range of the threshold value set by the threshold value setting means 7 is set by the value which filtered the closed value.

In Example 6, the method of setting the variable range of a threshold value is demonstrated. Other basic operations are the same as in the above-described embodiment, and thus are omitted here.

The maximum open value recording means 9 records the maximum open value of the output of the eye open / close evaluation means 4 when it is determined that the eyes of the vehicle driver are open within a predetermined time. This recorded value is sequentially given to the maximum open value filter 10 every fixed time. In the maximum open value filter 10, the filtering process is performed every fixed time longer than the fixed time set by the maximum open value recording means 9. As shown in FIG. Here, the plurality of maximum open values given by the maximum open value recording means 9 are filtered, and the average upper limit value of the output fluctuation of the eye open / close evaluation means 4 is calculated in the open state.

Similarly, the maximum closed value recording means 11 records the minimum closed value of the output of the eye open / close evaluation means 4 when it is determined that the eyes of the vehicle driver are in the closed state within a certain time. The recorded value is sequentially given to the most closed value filter 12 every predetermined time. The filtering process is performed every fixed time longer than the fixed time set by the minimum closed value recording means 11. Here, the plurality of minimum closed values provided by the minimum closed value recording means 11 are filtered to calculate an average lower limit value of the output fluctuation of the eye open / close evaluation means 4 in the closed state.

Thereby, the average output range of the eye open / close evaluation means 4 accompanying the opening and closing of the eyes of the vehicle driver is obtained. In the threshold limiting means 13, the closed variable range is set by the following equation using the average output range.

The lower limit at which the closed threshold becomes the minimum closed side is obtained by the equation (1). That is, it is based on the minimum closed value and is multiplied by an integer of 1 to the average output range. In addition, the upper limit value at which the closed threshold becomes the maximum open side is obtained by the expression (2). Equation 2 is the same as Equation 1 above, but differs from an integer multiplied by an average output range. Here, the constant 1 and the constant 2 select an appropriate value in accordance with the output characteristics of the eye open / close evaluation means 4 to be used, but since the formula 1 sets the lower limit and the formula 2 sets the upper limit, the constant 2 represents the constant 1 Larger value is selected.

The threshold limiting means 13 determines whether or not the closed threshold set by the threshold setting means 7 is within a range between the upper limit value and the lower limit value of the closed threshold value obtained by the above-described equation. If the closed threshold value is within the calculated range, the threshold value set by the threshold value setting means 7 is given to the time measurement means 5 and the opening / closing determination means 8 as it is. In the case of deviation from the calculated variable range, the closed threshold value is limited to a value of an upper limit value or a lower limit value to the opening / closing determination means 8 of the time measurement means 5. When the closed threshold is out of the variable range to the open side, it is limited to the upper limit, and when the closed threshold is out of the variable range, it is limited to the lower limit.

In addition, since the open threshold is on the open side by a fixed amount than the closed threshold, a limit is placed on the variable range of the closed threshold so that the open threshold is equally limited.

Therefore, according to the sixth embodiment, even when the eyes of the driver and the driver are about to be closed, the threshold value is not updated to an inappropriate position, and the eye can be determined accurately.

Further, in the sixth embodiment, the maximum open value when the open / close determination means 8 is determined to be in the open state within a predetermined time, and the minimum closed value when the open / close determination means 8 is determined to be closed within the fixed time. I have to record.

Therefore, since the vehicle driver keeps closing the eyes within a predetermined time and the output of the eye opening / closing evaluation means 4 is small, it is not determined that the eyes are closed, and therefore, recording the value in such a state as the maximum open value. none.

Similarly, since the vehicle driver keeps eyes open within a certain time and the output of the eye open / close evaluation means 4 is not judged to be closed, the recording of the value as the minimum closed value in such a state is necessary. none.

Therefore, according to the sixth embodiment, the average output range of the eye open / close evaluation means 4 can be calculated accurately.

Example 7

Example 7 is a modified example of Example 6, but in Example 6, the most open value is used as the representative value when the eyes are in the open state, whereas in Example 7, the eye is opened as the representative value when the eyes are in the open state. The output of the eye open / close evaluation means 4 when it is a state uses the average value.

17 is a block diagram showing a configuration of the seventh embodiment. In the drawings, the same or corresponding parts as in the above-described embodiment are denoted by the same reference numerals. Reference numeral 14 constitutes an open value calculation means with an open value filter for filtering the output of the eye open / close evaluation means 4 when the open / close determination means 8 is determined to be open within a predetermined time. . Reference numeral 15 denotes a threshold limit means for receiving the output of the open value filter 14 and the minimum closed value filter 12 and setting a variable range of the closed threshold value.

Since the basic operation of the seventh embodiment is the same as that of the sixth embodiment described above, only differences are mainly described here.

The open value filter 14 filters the output value of the eye open / close evaluation means 4 when the open / close determination means 8 determines that the eye is open in a certain time, and the vehicle driver opens the eye. An average value of hereinafter (hereinafter referred to as an open value) is output. The threshold limiting means 15 calculates the variable range of the closed threshold by using the filtered open value from the open value filter 14 and the filtered minimum closed value from the minimum closed value filter 12.

The lower limit of the closed threshold is obtained by equation (3). In other words, the difference between the average open value and the minimum closed value is multiplied by an integer 3 based on the minimum closed value. In addition, the upper limit of the closed threshold is obtained by equation (4). Equation 4 is the same as Equation 3, but differs from an integer multiplied by the difference between the average open value and the minimum closed value. The constants 3 and 4 here select an appropriate value in accordance with the output characteristics of the eye open / close evaluation means 4 to be used, and Equation 3 sets the lower limit, and Equation (4) sets the upper limit, so that the constant 4 Is chosen to be a large value compared to the integer 3.

The threshold value limiting means 15 determines whether or not the closed threshold value set by the threshold value setting means 7 is within a range between the upper limit value and the lower limit value of the closed threshold value obtained by the above-described equation. If the closed threshold value is within the calculated range, the threshold value set by the threshold value setting means 7 is given to the time measurement means 5 and the opening / closing determination means 8 as it is. If it is out of the calculated variable range, the value of the closed threshold value is limited to any one of an upper limit value and a lower limit value, and is given to the time measuring means 5 and the opening / closing determination means 8. When the closed threshold value deviates to the open side than the variable range, it is limited to the upper limit value, and when the closed threshold value deviates to the closed side than the variable range, it is limited to the lower limit value.

Therefore, in Example 7, it has the same effect as described in Example 6.

In Example 7, when calculating the average output range of the eye open / close evaluation means 4, the open value which is an average value instead of the maximum open value in the open state of an eye is used.

Therefore, even if the noise is superimposed on the output of the eye open / close evaluation means 4, since the filtering process is performed, the effect due to the noise can be reduced as compared with the sixth embodiment.

[Example 8]

In the eighth embodiment, the variable range of the closed threshold value set in the sixth embodiment above is made effective after a predetermined time after the device is started.

18 is a block diagram showing a configuration of the eighth embodiment. In the figure, reference numeral 16 denotes a threshold value limiting control means for updating the variable range of the threshold value from the time when the apparatus is operated until a certain period has elapsed, and prohibiting the change of the variable range of the threshold value after the predetermined period has elapsed. Reference numeral 17 denotes a timer as a period detecting means for detecting that a certain period has elapsed since the device was started.

Since the basic operation of the eighth embodiment is the same as that of the above-described embodiment, only the operation of enabling the threshold value of the variable range to be valid is described here.

It is generally believed that the vehicle driver is alert when the device is activated. It is also thought that there is no need to blink once for five minutes. In the eighth embodiment, the timer 17 detects that five minutes, which is a predetermined period, have elapsed since the device was started. Thus, the threshold limit control means 16 allows the updating of the variable range calculated by the threshold limit means 5 until the five minutes have elapsed.

When five minutes have elapsed since the device was started, the timer 17 transmits the signal to the threshold limit control means 16. In response to this, the threshold value limit control means 16 prohibits updating of the variable range of the threshold value control means 15.

Therefore, according to the eighth embodiment, the variable range of the closed threshold value can be set in the period in which the vehicle driver is considered to be aroused.

In addition, since the variable range of the closed threshold value is allowed to be updated for five minutes after the device is operated, the variable range can be updated in five minutes, thereby obtaining a preferable variable range.

In addition, after 5 minutes have elapsed since the device was started, the upper limit and the lower limit of the closed threshold value are prevented from being updated, so that the closed threshold value is not updated to an undesirable value.

In addition, although Example 8 was demonstrated based on Example 7, it can apply also to Example 6.

Example 9

Example 9 is a modification of Example 8 mentioned above. 19 is a block diagram showing a configuration of the eighth embodiment. In the figure, reference numeral 18 denotes a threshold limit that permits the updating of the variable range of the threshold value from the operation of the apparatus until the period of time is hardened, and prohibits the change of the variable range of the threshold value after the elapse of the predetermined period. It is a control means and 19 is a minimum closed value recording counter as a period detection means which detects that a fixed period has passed since the apparatus was operating.

The basic operation of the ninth embodiment is the same as the above-described embodiment, and only an operation for validating the setting of the variable range of the threshold value will be described here.

It is generally believed that the vehicle driver is awake when the device is activated. Also, I don't think there's anything I didn't forget once in five minutes. However, in the case of a person having a small number of blinks, since the number of open or close values obtained in 5 minutes is small, it is difficult to obtain an average output range of the eye open / close evaluation means 4 more accurately than a person having many blinks.

In Example 9, the minimum closed value recording counter 19 measures how many times the vehicle driver has blinked since the device was started. That is, the output of the minimum closed device recording means 11 is input to the minimum closed value recording counter 19, and the minimum closed value recording means adds a count value by one each time the 11 closes the minimum closed value. Doing.

The minimum closed value recording counter 19 determines that the minimum closed value counter 19 has elapsed for a period of time from the start of the apparatus to the threshold limit control means 18 when the count value reaches a predetermined constant value. Here, the predetermined period is a period in which the vehicle driver blinks by the predetermined number of times at the minimum closed value recording counter 19.

The threshold limit control means 18 permits the updating of the variable range calculated by the threshold limit means 15 while the vehicle driver blinks for a certain number of times.

If the vehicle driver blinks a certain number of times from starting the device, the minimum closed value recording counter 19 determines that a certain period has elapsed and transmits the signal to the threshold limit control means 18. In response to this, the threshold limit control means 18 prohibits updating of the variable range of the threshold value control means 15.

In addition, the constant value set in the minimum closed value recording counter 19 is set to the number of only necessary minimum number of open values and minimum closed values to be able to accurately and accurately determine the average output range of the eye open / close evaluation means 4. do.

Therefore, according to the ninth embodiment, in addition to the effect of the eighth embodiment, even if the number of blinks varies depending on the individual difference, the variable range of the closed threshold value can be set to a preferable range.

In addition, although Example 9 was demonstrated based on Example 7, it is applicable also to Example 6.

Example 10

Embodiment 10 relates to evaluation of a threshold value, and provides an evaluation method and apparatus separate from the above-described embodiment.

20 shows an example of the distribution of the number of taps of the output of the eye open / close evaluation means within a certain time. According to this figure, the distribution of the tap water is based on the distribution of the output of the eye open / close evaluation means when the eyes are in the open state, the distribution of the output of the open / close evaluation means of the eyes when the eyes are in the closed state, It is divided into three types with the distribution of the output of the eye open / close evaluation means in the middle of changing from the closed state to the open state. In addition, according to this figure, as shown by the broken line in a figure, it looks like there are two mountain shapes.

Therefore, the two mountain-shaped intersections, i.e., the intersection portions of the broken lines, are preferable opening and closing determination thresholds.

The opening / closing determination threshold determined in this way is in a part corresponding to the middle of the opening / closing operation of the eye, i.e., almost flat in the drawing, and it is possible to reliably determine whether the eye is open or closed.

Therefore, if there is a deviation between the currently open / close determination threshold value and the preferred open / close determination threshold value calculated by the above-described method, the open / close determination threshold value is updated in the direction of reducing the deviation.

FIG. 21 illustrates an example of the distribution of the number of taps in the output of the eye open / close evaluation means within a predetermined time, and illustrates a method of evaluating the open / close determination threshold value different from that described above.

In Fig. 21, an approximate straight line is drawn in a certain area existing on the closed side rather than the point where the number of taps is maximized. Therefore, the intersection between the approximate straight line and the number of degrees 0 in a certain area is reversed.

The opening / closing determination threshold determined in this way is a portion corresponding to the middle of the opening / closing operation of the eye, and it is possible to reliably determine the open or closed state of the eye.

Therefore, if there is a deviation between the currently open / close determination threshold value and the preferable open / close determination threshold value calculated by the above-described method, the open / close determination threshold value is updated in a direction to eliminate this deviation.

In addition, evaluation of the opening / closing determination threshold value is performed by the above-mentioned deviation. That is, the smaller the deviation, the more favorable the evaluation.

Next, an apparatus for realizing the above-described method of FIGS. 20 and 21 will be described.

22 is a block diagram showing the configuration of the tenth embodiment. In the drawings, the same reference numerals are assigned to the same or corresponding parts as the above-described embodiment. Reference numeral 20 denotes a threshold value setting means for setting the opening / closing determination threshold value based on the distribution of the number of outputs of the eye opening / closing evaluation means 4 at predetermined time intervals.

The basic operation of the tenth embodiment is the same as that of the first embodiment. That is, the video information of the camera 1 is input to the image input means 2, A / D conversion is performed, and the eye area is determined by the method described in the above-described embodiment in the eye area determination means 3, and the eyes are opened and closed. In the evaluation means 4, the eye open / close evaluation output value by the opening of the eyes of the vehicle driver is output.

The eye open / close evaluation output value is given to the tap water measuring means 20, and the tap water distribution as shown in FIG. 20 and FIG.

The threshold value setting means 21 calculates a preferable opening / closing determination threshold value by the above-mentioned method based on the said tap water distribution. This opening and closing determination threshold is compared with the currently set opening and closing determination threshold and the deviation is calculated. Thus, the threshold setting means 21 updates and resets the opening / closing determination threshold in the direction in which the deviation decreases, and gives the opening / closing determination means 8 the reset opening / closing determination threshold.

In addition, the update of the opening / closing determination threshold value is examined every fixed time, that is, every time the new tap water distribution is measured.

Therefore, according to the tenth embodiment, the opening / closing determination threshold value is set and updated according to the frequency distribution of the eye opening / closing evaluation output values, so that the opening and closing of the eyes of the vehicle driver can be accurately determined regardless of the change in the individual vehicle and the imaging environment.

In addition, in Example 10, although only one opening / closing determination threshold value was provided as a threshold value, two threshold values may be prepared like the above-mentioned embodiment. In this case, the open threshold and the close threshold may be set on the basis of the open / close determination threshold described in the tenth embodiment.

Alternatively, the open / close determination threshold value associated with the present embodiment may be set as the closed threshold value, and a constant value may be set by adding a constant value to the threshold value.

In addition, you may combine Examples 1-10 based on the spirit of the above-mentioned invention.

As described above, the face image processing apparatus and the face image processing method have been described, but these are not limited to the above-described embodiments, and various forms can be provided within the spirit of the invention.

Provides a face image processing device and a face image processing method that can more accurately determine the opening and closing of the eye of the shooting box when the state of the face image changes, so that the distance between the camera and the shooting box changes or the direction of the face is changed. By this change, even when the state of the face image changes, the open state or closed state of the eye can be detected accurately at the fixed threshold value.

Claims (4)

The camera which picks up a face image, the opening detection means which detects the opening of the eye of a photographed box by the imaging information of the said camera, and the detection result of the opening detection means are classified into a plurality of groups using a threshold value. Time-measuring means for integrating time the detection result of the opening detection means belonging to the designated group in the plurality of groups within a predetermined time; threshold value evaluating means for evaluating the validity of the threshold value based on the measurement result of the time-measuring means; And threshold setting means for resetting the threshold value based on the evaluation result of the threshold value evaluating means, and eye opening / closing determination means for determining the opening and closing of the eye of the photographing box using the threshold value. Device. The method according to claim 1, wherein the open value calculating means for calculating a representative value when the eye of the shooting box is in the open state by the detection result of the opening detecting means, and the detection result of the opening detecting means And a closed value calculating means for calculating a representative value when the eyes are in a closed state, and a threshold value limiting means for setting a variable range of the threshold value based on the calculation result of the open value associating means and the closed value calculating means. Facial image processing apparatus. The method according to claim 2, wherein the period detecting means for detecting that the predetermined period has elapsed since the operation of the apparatus and the updating of the variable range of the threshold value are allowed until the predetermined period has elapsed, A face image processing apparatus comprising threshold limit control means for prohibiting updating of a variable range. A camera for capturing a face image, opening detection means for detecting the opening of the eye of the photographing box by the imaging information of the camera, and tap water measuring means for measuring the distribution of the tap water of the detection result of the opening detection means at a predetermined time; Threshold setting means for setting a threshold value within a range where the distribution becomes substantially flat and updating the threshold value by updating the distribution of the number of water, and eye opening / closing determination means for judging opening and closing of the subject's eye using the threshold value. A facial image processing apparatus, comprising: