JP2009025894A - Face image detection method and face image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face image detection method and a face image processor for improving the detecting precision of a face image. <P>SOLUTION: The unidirectional luminance distribution B<SB>n</SB>of a pickup image is generated, and a first region G<SB>n</SB>whose luminance value correction is not executed and a second region H<SB>n</SB>whose luminance value is lower than that of the first region G<SB>n</SB>, and whose luminance value correction is not executed are decided based on the generated luminance distribution B<SB>n</SB>. Then, a predicted luminance distribution J<SB>n</SB>as the distribution of predicted luminance values in the second region H<SB>n</SB>is calculated based on the luminance value of the first region G<SB>n</SB>, and the luminance value (actual luminance value) S<SB>n</SB>and a predicted luminance value T<SB>n</SB>at the representative point Pn of the second region are compared, and the luminance value of the second region H<SB>n</SB>is corrected based on this comparison. Thus, it is possible to execute the luminance value correction based on the luminance value of a portion (first region) of the face. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face image detection method and a face image processing apparatus for detecting a human face.

Conventionally, in such a field, for example, a human face is imaged using a near-infrared camera, and a face image is detected by performing image processing based on the captured image data. Specifically, in the captured image, a bright area compared to the surrounding area is assumed to be a face. In such a technique, for example, when the half of the face is dark due to the incidence of ambient light, etc., the brightness on the left and right sides of the face is compared with the center axis of the face as a boundary to determine the level of contrast, and the brightness on the higher contrast side The luminance value is corrected by turning the value as the luminance value on the low contrast side and turning back the center axis of the face (see, for example, Patent Document 1).
JP 2005-49979 A

  However, in the prior art described in Patent Document 1, since the luminance value is corrected by folding back symmetrically with the center axis of the face as a boundary, the incident position of the disturbance light changes, and the contrast When the high and low boundaries deviate from the center axis of the face, there is a problem that the corrected face image cannot be detected accurately.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a face image detection method and a face image processing apparatus that improve the detection accuracy of face images.

  The face image detection method according to the present invention is a face image detection method for detecting a face image based on a captured image, a luminance distribution generation step for generating a luminance distribution in one direction of the image, and a luminance based on the luminance distribution. A correction area determination step of determining a first area in which the correction of the value is not performed and a second area in which the luminance value is lower than that of the first area and in which the correction of the luminance value is performed, and the luminance distribution in the first area An expected luminance distribution calculating step for calculating an expected luminance distribution that is an expected luminance value distribution, a luminance value comparing step for comparing the luminance value and the predicted luminance value at a predetermined position in the second region, and a luminance value comparing step And a luminance value correction step of correcting the luminance value of the second region based on the comparison result of (2).

  In addition, a face image processing apparatus according to the present invention is a face image processing apparatus that detects a face image based on a captured image, based on the luminance distribution generating means for generating a luminance distribution in one direction of the image, and the luminance distribution. A correction area determination means for determining a first area where the luminance value is not corrected and a second area where the luminance value is lower than the first area and where the luminance value is corrected; and the luminance distribution in the first area A predicted luminance distribution calculating unit that calculates a predicted luminance distribution that is a predicted luminance value distribution based on the predicted luminance value, a luminance value comparing unit that compares the luminance value and the predicted luminance value at a predetermined position in the second region, and a luminance value And a luminance value correcting unit that corrects the luminance value of the second region based on a comparison result by the comparing unit.

  According to such a face image detection method and a face image processing apparatus, a first region that generates a luminance distribution in one direction of a captured image, and that does not perform luminance value correction based on the generated luminance distribution, and A second region having a luminance value lower than that of the first region and performing luminance value correction is determined. Based on the luminance value of the first area, an expected luminance distribution that is a distribution of the expected luminance value in the second area is calculated, and the luminance value (actual luminance value) and the expected luminance at the representative point of the second area are calculated. The values are compared, and the luminance value of the second region is corrected based on the comparison. As a result, an expected luminance distribution is generated based on the luminance value of the first region where correction is not performed, and the predicted luminance value and the luminance value (actually measured value) of the representative point of the second region are compared with each other. Therefore, it is possible to execute the luminance value correction based on the luminance value of a part of the face (first region). Therefore, even when the contrast boundary is not along the center axis of the face, the luminance value can be corrected with high accuracy.

  Here, the representative point is preferably a position where the maximum luminance value is detected in the second region. It can be assumed that the position where the maximum luminance value is detected is the position where the luminance value is detected with the highest accuracy in the second region. As a result, the position where the maximum luminance value is detected is used as a representative point, the expected luminance value and the luminance value at this representative point are compared, and the luminance value of the second region can be corrected based on the comparison result. Become. Therefore, the face detection accuracy can be improved.

  According to the face image detection method and the face image processing apparatus of the present invention, an expected brightness value is calculated based on the brightness value of a part of the face, and based on a comparison between the expected brightness value and the actually measured brightness value, By executing the brightness value correction, it is possible to execute the brightness value correction accurately even when the contrast boundary is not along the center axis of the face, and the face detection accuracy can be improved. .

  Preferred embodiments of the present invention will be described below with reference to the drawings. In the description of the drawings, the same or corresponding elements are denoted by the same reference numerals, and redundant description is omitted. FIG. 1 is a block diagram illustrating a face image processing apparatus according to an embodiment of the present invention, and FIG. 2 illustrates an example of a face image captured by a face image capturing camera. FIG.

  A face image processing apparatus 10 shown in FIG. 1 is mounted on a vehicle or the like and detects a face image of a driver D. The face image processing apparatus 10 includes a face image capturing camera 12 that captures a face image of the driver D, and a face image processing electronic control unit (hereinafter referred to as “face”) that performs image processing based on image signals from the face image capturing camera 12. 14) (referred to as “image processing ECU”).

  The face image capturing camera 12 is set on, for example, the upper surface of the column cover 16 and acquires the face image of the driver D. The face image capturing camera 12 is, for example, a near-infrared camera, and receives near-infrared light irradiated from a near-infrared projection device (not shown) and receives reflected light reflected on an object (a face of the driver D). The face of the person D is imaged. For example, the face image capturing camera 12 can acquire an image of 30 frames per second.

  The face image processing ECU 14 includes a CPU that performs arithmetic processing, a ROM and a RAM that serve as the storage unit 18, an input signal circuit, an output signal circuit, a power supply circuit, and the like. Detect face images. The detected face image is used for, for example, determination of arousal level of driver D, determination of looking aside, and the like.

  Here, as shown in FIG. 2, the face image processing apparatus 10 according to the present embodiment performs correction of the luminance value when, for example, the luminance value of a part of the face of the driver D decreases, It has a function to improve image detection accuracy. Then, the CPU of the face image processing ECU 14 executes a program stored in the storage unit 18 to thereby execute a luminance distribution generation unit 20, a correction area determination unit 22, an expected luminance curve generation unit 24, a luminance value comparison unit 26, A luminance value correction unit 28 and an image processing unit 30 are configured. In addition to the program for operating the CPU, the storage unit 18 stores a plurality of three-dimensional face models. The storage unit 18 functions as a storage unit that temporarily stores captured image data.

The luminance distribution generation unit 20 calculates a luminance value at each point of the image input from the face image capturing camera 12 and creates a luminance distribution. The luminance distribution generating unit 20, a plurality of rows extending the face of the driver D in the lateral direction (hereinafter, referred to as "row") is divided into A _n, to create a luminance distribution for each row A _n. FIG. 3 is a graph showing the luminance distribution generated based on the captured face image. In Figure 3, which shows the luminance distribution B _n in row A _n in FIG. 2, the horizontal axis indicates the position in the lateral direction (horizontal direction) of the face shows the brightness values on the vertical axis. L ₀ indicated by the alternate long and short dash line indicates the center axis L _{0 of the} face. The center axis L ₀ of the face passes through the center of the face of the driver D and extends in the vertical direction. The face center axis L ₀ is set based on an image acquired when the brightness value of the face image is sufficient and the brightness value correction is not necessary. F ₁ and F ₂ indicated by broken lines indicate left and right ends F ₁ and F ₂ of the rectangular frame F set so as to surround the face area. The rectangular frame F is set based on an image acquired when the brightness value of the face image is sufficient and there is no need to correct the brightness value. The luminance distribution generating unit 20, and generates the luminance distribution for all rows A _n which are arranged in the vertical direction.

Based on the luminance distribution _Bn generated by the luminance distribution generation unit 20, the correction region determination unit 22 includes a region that does not require correction of luminance values (hereinafter referred to as “first region”) _Gn, and the first region luminance value than a region G _n requires correction of low luminance value (hereinafter, referred to as "second region") determines and H _n. The correction region setting unit 22 refers to the luminance distribution _Bn , determines the level of the luminance value (contrast level), sets the higher luminance value as the first region _Gn, and sets the lower luminance value to the first region _Gn . setting a second region _{H n.}

The determination of the first region G _n and the second region H _n may be performed, for example, by comparison with a reference value set based on past data. In this case, the luminance value is greater than the reference value. A high region is set as a first region, and a region having a luminance value equal to or lower than a reference value is set as a second region. The correction area determination unit 22 may set the first area G _n and the second area H _n based on the change rate (gradient) of the luminance distribution B _n . In this case, the change rate is large. to set position as the boundary between the first region _{G n,} and the second region _{H n.} It should be noted that the “region that does not require the correction of the luminance value (first region)” has a sufficient luminance value, and features such as facial parts (eyes, nose, mouth), etc., without executing the correction of the luminance value. This is an area where points can be detected. Further, the “region requiring the correction of the luminance value (second region)” is a region where the luminance value is insufficient and a feature point such as a face part cannot be detected accurately.

In the data shown in FIG. 3, the boundary between the first region G _n and the second region H _n is the center axis L _{0 of the} face, but the first region G _n and the second region H _n boundaries, not necessarily the center axis L ₀ of the face. For example, if the brightness value in the 1/3 range on the right side of the face is higher than the reference value and the brightness value in the 2/3 range on the left side of the face is less than or equal to the reference value, 1/3 of the right side of the face range, is set as the first region G _n, 2 ranging thirds of the left face is set as the second region H _n.

Expected brightness curve generating unit 24 based on the luminance distribution B _n in the first region G _n, and generates a predicted brightness curve (expected luminance distribution) J _n shown in FIG. FIG. 4 is a graph showing an expected luminance curve. In the expected luminance curve generating unit 24, for example, by calculating the rate of change in the luminance distribution B _n, it detects a rising portion K _n indicating the end of the face area. Expected brightness curve generating unit 24 sets the normal distribution (Gaussian distribution) curve along the detected rising portion K _n, generates the normal distribution curve as expected brightness curve J _n.

Luminance value comparison section 26 compares the second region _H luminance value at the representative point _{P n} of the _n (real luminance value) _{S n} and the expected luminance value _{T n,} the expected luminance ratio (correction _{coefficient) R} n (= expected Luminance value _Tn / luminance value _Sn ) is calculated. Here, the representative point P _n is a position where the maximum luminance value is detected in the second region H _n . In FIG. 4, the position in the horizontal direction of the face corresponding to the maximum luminance value S _n in the second region H _n becomes the representative point P _n , and the point on the expected luminance curve J _{n at} the representative point P _n is the expected luminance value T _n .

Luminance value correcting unit 28, based on the expected luminance ratio R _n is a comparison result of the luminance value comparing unit 26, corrects the luminance value of the second region. Specifically, to perform the correction of the luminance value by multiplying the expected luminance ratio R _n to the luminance distribution B _n. FIG. 5 is a graph showing the luminance distribution corrected based on the expected luminance curve. In Figure 5, which shows the luminance distribution C _n corrected in row A _n in FIG. 2, the horizontal axis indicates the horizontal position of the face shows the brightness values on the vertical axis. In Figure 5, the luminance value of the second region H _n are those corrected by expected luminance ratio R _n. For example, when the predicted luminance ratio R _n is “3.0”, the luminance value is corrected by multiplying the luminance distribution B _n (actual luminance value) by three.

FIG. 6 is a diagram illustrating an example of a face image captured by the face image capturing camera, and is a diagram illustrating the face image after performing the brightness correction. The image processing unit 30 performs image processing based on the corrected image data. When the correction is not executed, image processing is performed based on the image signal input from the face image capturing camera 12. The image processing unit 30 performs image processing, extracts a face contour edge E ₁ indicating a contour line of the face of the driver D, and detects a face region E of the driver D. The image processing unit 30 detects a portion brighter than the surroundings as the face region E. In addition, the image processing unit 30 calculates the face orientation angle of the driver D and extracts facial feature points (for example, eyes, nostrils, mouth, etc.).

  Next, a face image detection method using the face image processing apparatus 10 configured as described above will be described. FIG. 7 is a flowchart showing an operation procedure of main processing executed by the face image processing ECU. The main process is repeatedly executed at a predetermined timing while the face image processing apparatus 10 is activated. First, the face image processing ECU 14 inputs an image signal obtained by capturing the face of the driver D from the face image capturing camera 12 (S1).

  Next, the face image processing ECU 14 performs a brightness value correction process (S2). FIG. 8 is a flowchart showing an operation procedure of luminance value correction processing executed by the face image processing ECU. In the luminance value correction process, the face image processing ECU 14 determines whether or not the face area E has been detected last time (S11). If it is determined that the face area E has been detected last time, the process proceeds to step S12. If it is not determined that the face area E has been detected last time, the luminance value correction process is terminated. The process proceeds to step S3 of the main process shown in FIG.

In step S12, the face image processing ECU 14 inputs data related to the previous face area E. Then, the face image processing ECU14 produces a luminance distribution _{B n} for each row _{A n} on the basis of the data related to the input face area E (S13, luminance distribution generating step).

Subsequently, the face image processing ECU 14 determines the level of the luminance value based on the luminance distribution _Bn , and the first region _Gn that does not require correction of the luminance value and the second region that requires correction of the luminance value. Is determined, and it is determined whether or not it is necessary to execute correction of the luminance value (S14, correction area determination step). If it is determined that it is not necessary to execute the correction of the brightness value, the brightness value correction process is terminated, and the process proceeds to step S3 of the main process shown in FIG. If it is determined that it is necessary to correct the luminance value, the process proceeds to step S15.

In step S15, the face image processing ECU14 creates a predicted luminance curve _{J n} in the lateral direction of the face based on the luminance distribution _{B n} in the first region _{G n} (S15, predicted luminance distribution calculation step). Then, the face image processing ECU14, based on the luminance distribution _{B n,} extracts the maximum brightness value _{S n} of the second region _{H n,} determines the representative point _{P n} (S16).

Subsequently, the face image processing ECU14 refers to the expected luminance curve _{J n,} extracts the predicted luminance value _{T n} at the representative point _{P n} (S17). Next, the face image processing ECU 14 calculates an expected luminance ratio R _n at the representative point P _n (S18, luminance value comparison step).

Subsequently, the face image processing ECU14 executes the luminance value correction of the second region _{H n} (S19, luminance value correction step). Specifically, the face image processing ECU 14 multiplies the luminance distribution B _n (actual luminance value) by the expected luminance ratio R _n to calculate a corrected luminance value. The face image processing ECU 14 proceeds to step S3 of the main process after executing the brightness value correction process for all rows A _n (see FIG. 2) adjacent in the vertical direction of the face.

  As shown in FIG. 7, the face image processing ECU 14 extracts a face contour edge (S3) and detects a face region after the brightness value correction process in step S2 based on the data after the brightness value correction (S3). ). Next, the face image processing ECU 14 calculates the face orientation angle of the driver D (S5), and extracts facial feature points (S6).

According to such a face image processing apparatus 10 and the face image detecting method, and generates a luminance distribution B _n along rows A _n of the captured image, based on the generated luminance distribution B _n, the luminance value correction the first region G _n and the luminance value than the first region G _n not run determines second region H _n to perform low luminance value correction. Then, along the luminance distribution B _n of the first region G _n, to set the normal distribution as expected luminance curve, calculate the expected intensity ratio R _n at the representative point P _n of the second region H _n, the expected brightness Based on the ratio R _n , the luminance value of the second region H _n is corrected. For this reason, luminance value correction is performed based on the luminance value of a part of the face (first region). As a result, even when the contrast boundary is not along the center axis of the face, the luminance value can be corrected with high accuracy.

Here, since the maximum luminance value in the second region H _n is a representative point P _n of the detected position, the detection accuracy of the face image is further improved. Position where the maximum luminance value is detected, because it can most accurately luminance value in the second region H _n is assumed to be position detection, the position at which the maximum luminance value is detected as a representative point, By calculating the expected brightness ratio R _n at the representative point and correcting the brightness value of the second region H _n based on the expected brightness ratio R _n , the detection accuracy of the face image can be further improved.

  As mentioned above, although this invention was concretely demonstrated based on the embodiment, this invention is not limited to the said embodiment. In the above embodiment, the face image of the driver who steers the vehicle is detected. For example, when detecting the face image of other drivers and operators who handle airplanes, railways, plants, machines, etc. The invention may be applied. Further, the present invention may be applied to a case where a face image capturing camera is installed at an entrance of a building and the like and a visitor's face image is detected.

  In the above embodiment, the luminance distribution is generated in the horizontal direction of the face. However, the luminance distribution may be generated in the vertical direction of the face or in other directions.

In the above embodiment, an expected brightness ratio R _n that is a ratio between the brightness value (actual brightness value) at the representative point P _n and the expected brightness value is calculated, and the brightness value is corrected based on the expected brightness ratio R _n. However, the correction of the luminance value may be executed by calculating the difference between the predicted luminance value and the luminance value and adding the calculated difference.

  In the above embodiment, the position where the maximum luminance value is detected in the second region is used as the representative point. However, the representative point may be set based on the difference between the predicted luminance value and the luminance value.

  Further, the luminance value may be corrected in detail by dividing the first area and the second area into a plurality of areas.

Further, in the above embodiment, it is set expected luminance ratio R _{n (correction} factor) for each row A _n, by using the correction coefficient of one row A _n, perform the brightness value correction of other rows An May be.

  In the above-described embodiment, the normal distribution is the expected luminance curve, but the predicted luminance distribution may be calculated using another approximate expression. In short, an expected luminance distribution may be generated based on the luminance distribution of the first region where luminance value correction is not performed.

It is a block diagram which shows the face image processing apparatus which concerns on embodiment of this invention. It is a figure which shows an example of the face image imaged with the face image pick-up camera, and shows the face image before luminance correction execution. It is a graph which shows the luminance distribution produced | generated based on the imaged face image. It is a graph which shows an estimated brightness | luminance curve. It is a graph which shows the luminance distribution corrected based on the expected luminance curve. It is a figure which shows an example of the face image imaged with the face image imaging camera, and shows the face image after brightness correction execution. It is a flowchart which shows the operation | movement procedure of the main process performed by face image process ECU. It is a flowchart which shows the operation | movement procedure of the luminance value correction process performed by face image process ECU.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Face image processing apparatus, 12 ... Face image pick-up camera, 14 ... Face image processing ECU, 16 ... Column cover, 18 ... Memory | storage part, 20 ... Luminance distribution generation part, 22 ... Correction area | region determination part, 24 ... Expected luminance curve Generation unit 26 ... Luminance value comparison unit 28 ... Luminance value correction unit 30 ... Image processing unit A _n ... Column extending in the horizontal direction of the face area, B _n ... Luminance distribution, C _n ... Luminance after correction distribution, D ... driver, E ... face area, E 1 _... facial contour edges, G n _... areas that do not require correction (first region), the area in need of H n _... correction (second region) , J _n ... predicted luminance curve (expected luminance distribution), K _n ... rising edge, P _n ... representative point, R _n ... expected luminance ratio (correction coefficient), S _n ... luminance value at representative point, T _n ... representative point Expected brightness value at.

Claims (3)

In a face image detection method for detecting a face image based on a captured image,
A luminance distribution generation step for generating a luminance distribution in one direction of the image;
A correction area determination step for determining a first area where the correction of the luminance value is not performed based on the luminance distribution and a second area where the luminance value is lower than that of the first area and where the correction of the luminance value is performed;
An expected luminance distribution calculating step of calculating an expected luminance distribution which is a distribution of expected luminance values in the one direction predicted based on the luminance distribution in the first region;
A luminance value comparison step of comparing a luminance value and a predicted luminance value at a representative point of the second region;
A face image detection method comprising: a brightness value correction step of correcting the brightness value of the second region based on a comparison result in the brightness value comparison step.   The face image detection method according to claim 1, wherein the representative point is a position where a maximum luminance value is detected in the second region. In a face image processing apparatus that detects a face image based on a captured image,
A luminance distribution generating means for generating a luminance distribution in one direction of the image;
A correction area determination unit that determines a first area that does not execute correction of a luminance value based on the luminance distribution and a second area that has a luminance value lower than that of the first area and performs correction of the luminance value;
Expected luminance distribution calculating means for calculating an expected luminance distribution that is a distribution of expected luminance values in one direction predicted based on the luminance distribution in the first region;
A luminance value comparing means for comparing the luminance value at the representative point of the second region and the expected luminance value;
A face image processing apparatus comprising: a brightness value correcting unit that corrects a brightness value of the second region based on a comparison result by the brightness value comparing unit.

Images