JP2008234619A - Face authenticating device and face authenticating method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face authenticating device superior in obtaining a face image suitable for face authentication. <P>SOLUTION: This face authenticating device determines whether a walking person is already registered or not, by collating the face image acquired from the walking person with preregistered dictionary information, for example, when authenticating the walking person. A change and a head portion candidate are detected for a high-gradation input image acquired from a camera, and are then converted into a low-gradation input image by conversting a face luminance for the head portion candidate region, and a face pattern is detected for the obtained low-gradation input image. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention, for example, uses a pedestrian as an authentication target, and compares the face image acquired from the pedestrian with pre-registered dictionary information to determine whether the pedestrian is a pre-registered person. The present invention relates to an apparatus and a face authentication method.

In general, this type of face authentication apparatus acquires a face image of a person to be authenticated in advance using a camera, a sensor, or the like, and stores facial feature information obtained from the acquired face image as dictionary information in a dictionary storage unit. Remember.
At the time of verification (at the time of authentication recognition), the face image of the person to be authenticated is acquired again using a camera, a sensor, or the like, and the facial feature information obtained from the acquired face image is stored in the dictionary storage means in advance. Evaluate whether it is similar to the stored dictionary information, and if the similarity is high, determine that the person is a registered person (person), and if the similarity is low, No).

  Cameras and sensors that are image acquisition means used in such face authentication devices automatically optimize control parameters such as gain, shutter speed, iris, and white balance based on the brightness and hue of the entire input image. Many are equipped with technology to adjust.

As a method of performing gain control for a face, a face authentication apparatus that photographs a person's face with a camera and determines whether the photographed face is a registered person or not is A / D converted. There is known a technique for controlling the gain value at the time of making a value suitable for face authentication (see, for example, Patent Document 1).
JP 2005-202731 A

  However, the person to be authenticated is a person (passerby) who moves without stopping in front of the camera. For this reason, there is a case where a person moves out of the field of view of the camera while trying to obtain a gain value suitable for face authentication while gradually changing the gain value.

  Accordingly, an object of the present invention is to provide a face authentication apparatus and a face authentication method that are excellent in obtaining a face image suitable for face authentication.

  A face authentication apparatus according to an embodiment of the present invention includes an image acquisition unit that acquires an image including at least a face of a moving person as a high gradation input image, and a high gradation input image obtained at regular intervals from the image acquisition unit. Change detection means for detecting a moving object area as a change area, head candidate detection means for detecting a human head candidate area from an image of the change area detected by the change detection means, and this head candidate detection Luminance conversion means for generating a low gradation input image from a high gradation input image by performing luminance conversion on the image of the head candidate area detected by the means, and a low gradation input generated by the luminance conversion means Face detection means for detecting a human face candidate area with respect to the head candidate area of the image, and an image of the face candidate area detected by the face detection means is compared with previously registered dictionary information. The person has and a determining face authentication means whether a pre-registered person by.

  Further, in the face authentication device according to an embodiment of the present invention, the change detection unit acquires a horizontal differential image and a vertical differential image from the high gradation input image obtained at regular intervals from the image acquisition unit, and acquires the acquired A change region is detected by calculating a difference between incremental code frames using a horizontal differential image and a vertical differential image. In the calculation of the difference between incremental code frames, the obtained horizontal differential image and vertical differential image are A correction process for removing the influence of light shot noise generated by the image acquisition means is performed by performing a calculation by dividing by the square root of the luminance of the high gradation input image.

  In the face authentication apparatus of the present invention, the change detection means detects a change area by calculating a difference between luminance frames for a high gradation input image obtained at regular intervals from the image acquisition means. In the calculation of the difference between the luminance frames, a correction for removing the influence of the light shot noise generated in the image acquisition unit by performing the calculation of dividing the luminance time difference by the square root of the luminance of the high gradation input image. Processing is performed.

  According to the present invention, since the high gradation input image is processed without switching the gain value, the brightness of the face can be easily controlled for the moving person, and thus a face image suitable for face authentication can be obtained. It is possible to provide a face authentication device and a face authentication method that improve face authentication performance.

  Further, according to the present invention, since the light shot noise is not detected as a change and the performance of the change detection is improved, the performance in the subsequent processing is improved, and as a result, the face authentication performance is improved. And a face authentication method.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 12 shows an example in which the output from the camera is A / D converted. FIG. 12A is an analog output level histogram for the analog output signal of the camera and the face area. The gain values g1, g2, and g3 (g3>g2> g1) are given to the A / D converter, and the luminance histograms for the input image and the face area obtained from FIG. (B), (c), and (d).

  In the example of FIG. 12, as shown in FIG. 12 (b), the face region has a gain value g1 that makes the luminance of the minimum gradation value for all pixels, or the face value as shown in FIG. 12 (d). The gain value g3 in which the luminance of the region is the maximum gradation value for all pixels is not desirable, and is set to a gain value g2 that fits in the gradation of the input image as shown in FIG. It is desirable to do.

  Specifically, the gain value is changed in stages, the face pattern is detected from the input image, the evaluation value is calculated from the luminance histogram of the detected face candidate region, and the gain value that maximizes the evaluation value is selected. .

  As the evaluation value, the following values are considered in consideration of the whiteout ratio and the blackout ratio.

-Max (white skip rate, black crush rate)
The whiteout ratio is a ratio of the number of pixels in which brightness is saturated (a brightness value is the maximum value of the gradation of the input image) in a predetermined area (in the above case, a face area). The blackout ratio is a ratio of pixels in a predetermined area whose luminance value is the minimum gradation value of the input image.

  In addition, as a change detection method for detecting a moving region (change region) from an input image sequence obtained every fixed time from the camera, a method using a luminance interframe difference calculation or an incremental code interframe difference calculation is used. The method used is known.

The difference between luminance frames is the change image DIFF _t (x, y) as follows from the input image I _t (x, y) at time t and the input image I _t-1 (x, y) at time t−1. ).

First, a difference image DI _t (x, y) represented by the following equation 1 is calculated from the input image.

Next, the calculated difference image DI _t (x, y) is compared with the threshold value TH_FD, and a comparison result shown in the following Expression 2 is obtained.

DIFF _t (x, y) = 1 indicates that a change is detected in the pixel (x, y). DIFF _t (x, y) = 0 indicates that no change is detected in the pixel (x, y).

  Since such luminance inter-frame difference directly uses the luminance of the input image It (x, y), it is easily affected by flicker, lighting on / off, and the like, and an object (a person to be authenticated) ) And a background object have a small luminance difference, it is difficult to detect the object.

On the other hand, it is conceivable to use the time difference of the incremental code image calculated from It (x, y) instead of directly calculating the time difference for the input image It (x, y).
Incremental codes are, for example, literature (Yutaka Sato, Shunichi Kaneko, Satoru Igarashi, “Robust Object Detection and Separation Based on Peripheral Incremental Sign Correlation Images,” IEICE, vol. J84-D-II, No. 12). As shown in FIG. 6, the magnitude relationship between the luminance of the pixel of interest and the surrounding pixels is encoded.

As the incremental code, for example, DS_X _t (x, y) obtained by encoding the difference between the pixel of interest in the horizontal direction and the difference between the pixel of interest in the vertical direction is encoded. DS_Y _t (x, y) is used.

DS_X _t (x, y) and DS_Y _t (x, y) are obtained by calculations shown in the following equations 3-1, 3-2, 4-1, and 4-2.

The incremental sign inter-frame difference image DIFF_S _t (x, y) is calculated as in the following Expression 5-1, Expression 5-2, and Expression 5-3.

  Incremental code frame difference is based on the magnitude relationship with surrounding pixels, so it is robust against illumination fluctuations, and even when the brightness difference between the background and the object is small, the advantage of high object detection performance There is.

With gain control for the face, the following can occur:
(1) When a person moves, gain values and face images suitable for face authentication may not be obtained.

  (2) When there are a plurality of persons at the same time in the field of view of the camera and the illuminance on the face differs greatly, if gain control is performed on the face area of one person, the face of the other person An input image suitable for face authentication cannot be obtained for the area.

  The above (1) will be described. It is assumed that the person stops in front of the camera and waits until the authentication process is completed. Therefore, the face can be photographed while gradually changing the gain value until a face image suitable for face authentication is obtained.

  However, the person targeted by the present invention is a person (passerby) who moves without stopping in front of the camera. For this reason, there is a case where a person moves out of the field of view of the camera while trying to obtain a gain value suitable for face authentication while gradually changing the gain value.

  The above (2) will be described. For example, consider the situation shown in FIGS. 13A and 13B. FIG. 13A shows a situation where a plurality of persons M1 and M2 are moving in the direction of the arrow in an environment in which a plurality of spot light illuminators L1 and L2 are arranged at regular intervals.

  In this case, the analog video output of the camera CA is as shown in FIG. 13B, and the brightness of the faces of the persons M1 and M2 are greatly different. In such a case, if gain control is performed on the face area of the person M1, many white jumps occur in the face area of the person M2, and an input image suitable for face authentication cannot be obtained for the person M2. On the other hand, if gain control is performed on the face area of the person M2, a large amount of blackening occurs in the face area of the person M1, and an input image suitable for face authentication cannot be obtained for the person M1.

  14A and 14B show specific examples when the brightness adjustment is performed on the face areas of the persons M1 and M2 in FIGS. 13A and 13B. FIG. 14A shows the result of gain control performed on the face area of the person M1. In this case, many whiteouts occur in the face area of the person M2, and an input image suitable for face authentication is not obtained. FIG. 14B shows the result of gain control performed on the face area of the person M2. In this case, many black areas are generated in the face area of the person M1, and an input image suitable for face authentication is not obtained.

  In addition, according to the method for detecting a change area based on the difference between incremental code frames, an area having no texture and no motion may be erroneously detected as a change area due to the influence of light shot noise.

  Here, light shot noise is noise generated in an image sensor used in a camera, as is well known. In general, even when light is applied to an image sensor at a constant illuminance for a certain period of time, the output of the image sensor varies. Optical shot noise is observed as this variation.

As shown in FIG. 15A, when the output S when light of a predetermined time is irradiated with light at a constant illuminance is plotted for time t, the result is as shown in FIG. 15B. This is a signal including a standard deviation noise component centered on the average value S_mean. The standard deviation is represented by the following formula 6.

  When a histogram is generated for the value S in FIG. 15B, a normal distribution with an average value S_mean and a standard deviation is obtained as shown in FIG. 15C.

In the input image obtained by A / D converting the output of the image sensor, the standard deviation σ _t (x, y) of the light shot noise is proportional to the square root of the luminance, as shown in the following Equation 7. Here, I _t (x, y) is the luminance of the input image.

Brightness value _I t (x, _{y 0)} of a straight line y = _{y o} of the input image of FIG. 16A to the vertical axis, graph the horizontal axis in the x-axis of the input image is as shown in Figure 16B. I _t (x, _{y 0)} images obtained by differentiating in x DI_X _t (x, _{y 0)} in the vertical axis, graph the horizontal axis in the x-axis of the input image is as shown in Figure 16C. In FIG. 16A, M indicates a person.

As is apparent from FIGS. 16A, 16B, and 16C, in the bright wall region E1, there are many pixels whose values are larger than the threshold value TH_SD or smaller than the threshold value -TH_SD due to noise. Incremental codes DS_X _t (x, y) and DS_Y _t (x, y) in such a bright wall region frequently change over time.

For this reason, as shown in FIG. 17, in the high luminance region, it is affected by noise, and for example, the wall region E1 and the ceiling region E2 are erroneously detected as change regions. In the difference image DIFF_S _t (x, y) in FIG. 17, pixels in which a change is detected are represented by black and diagonal lines, and pixels in which no change is detected are represented by white.

It is conceivable to increase the threshold value TH_SD so as not to be affected by light shot noise. However, in the example of FIG. 16, the contour portion of the person is regarded as flat without an edge, that is, the contour portion of the person is “0” in the incremental codes DS_X _t (x, y) and DS_Y _t (x, y). It becomes impossible to detect the movement of the person as a change.

  Since the light shot noise in the input image is proportional to the luminance value of the input image, the maximum value of the luminance value of the input image increases as the number of bits n per pixel of the input image increases. The inter-frame difference is easily affected by light shot noise.

  Therefore, according to the face authentication apparatus and the face authentication method described in the present embodiment, a face image suitable for face authentication is obtained, and as a result, face authentication performance is improved. Further, according to the face authentication device and the face authentication method described in the present embodiment, the light shot noise is not detected as a change, the change detection performance is improved, and as a result, the face authentication performance is improved.

  The outline of the present embodiment will be briefly described below. In the present embodiment, for example, as shown in FIG. 2, a camera 11 is disposed on a wall surface 2 such as a passage 1 where a person (pedestrian) M moves in the direction of the arrow in the figure, and at least the face of the person M is set at regular intervals. Get an image containing. The face area of the person M is detected and tracked from the obtained image series. It is determined from the obtained face image whether or not there is a person in which the moving person M is registered in advance.

As an object to which the present invention is applied, for example, it is installed near the entrance of a room where entry restriction is to be performed, and when it is determined that the person is allowed to enter the room in advance, it is used for unlocking the room key, etc. It is possible.
Alternatively, it may be installed in a store or the like, registered with a person requiring attention in advance, and used to issue an alarm when it is determined that the passing person is a person requiring attention.

Hereinafter, the first embodiment will be described.
FIG. 1 schematically shows a configuration of a face authentication apparatus according to the first embodiment. This face authentication apparatus includes a video camera (hereinafter simply referred to as a camera) 11, an A / D converter 12, a high gradation input image buffer 13, a change detector 14, a head candidate area detector 15, and a face luminance converter. 16, a low gradation input image buffer 17, a face detection dictionary (memory) 18, a face pattern detection unit 19, a face tracking unit 20, a face authentication dictionary (memory) 21, and a face authentication unit 22.

  The video camera 11 acquires an image including at least the face of the pedestrian M. The A / D conversion unit 12 converts an analog video signal obtained from the camera 11 into a digital high gradation input image (an input image having a first gradation). The high gradation input image buffer 13 temporarily stores the high gradation input image obtained from the A / D conversion unit 12.

  The change detection unit 14 detects the area of the moving object as a change area from the high gradation input image stored in the high gradation input image buffer 13. That is, the change detection unit 14 detects a change region between a plurality of frames constituting the high gradation input image. The head candidate area detection unit 15 detects a human head candidate area from the image of the change area detected by the change detection unit 14. The face luminance conversion unit 16 performs luminance conversion on the image of the head candidate region detected by the head candidate region detection unit 15, thereby converting the high gradation input image to the low gradation input image (second gradation degree). Input image).

  The low gradation input image buffer 17 temporarily stores the low gradation input image generated by the face luminance conversion unit 16. The face detection dictionary 18 stores a face detection dictionary pattern used for detecting a face candidate area. The face pattern detection unit 19 detects a human face candidate area from the head candidate area of the low gradation input image stored in the low gradation input image buffer 17. The face tracking unit 20 tracks the face candidate area detected by the face pattern detection unit 19. The face authentication dictionary 21 registers (stores) a plurality of dictionary information in advance. The face authentication unit 22 compares the face area image (face image) obtained by the face tracking unit 20 with dictionary information registered in advance in the face authentication dictionary 21, and the person M is a person registered in advance. Determine whether.

Hereinafter, each component will be described in detail.
For example, as shown in FIG. 2, the camera 11 is installed on a wall surface 2 such as a passage 1 where a person (pedestrian) M moves in the direction of the arrow in the figure, and includes at least a face from the front direction of the moving person M. An image is taken at regular intervals, and is mainly composed of an image sensor.

  The analog video signal from the A / D converter 12 and the camera 11 is converted into a high gradation input image of m bits per pixel and stored in the high gradation input image buffer 13. Here, for example, “14” is used as m.

  The change detection unit 14 detects a moving object region as a change region from the high gradation input image in the high gradation input image buffer 13, and the detection result is that the pixel value “1” is changed and the pixel value “0” is detected. "Is output as a change image in which there is no change.

  The head candidate area detection unit 15 detects an area that seems to be a person's head from the change image detected by the change detection unit 14. Specifically, for example, contour extraction is performed on the change region, and head candidates are detected from the curvature of the extracted contour.

  The face luminance conversion unit 16 generates a low gradation input image from the high gradation input image for the head candidate region from the head candidate region detection unit 15 and stores it in the low gradation input image buffer 17. Here, the low gradation input image is an image of n (<m) bits per pixel, and for example, “8” is used as n.

  Note that the conversion from a high gradation input image to a low gradation input image is made suitable for face pattern detection and face authentication. Specifically, for example, in the low gradation input image, conversion is performed so as not to cause overexposure or underexposure in the face area.

  The face pattern detection unit 19 performs pattern matching with the face detection dictionary pattern in the face detection dictionary 18 for the head candidate region of the low gradation input image generated by the face luminance conversion unit 16, thereby performing face matching. A probable area is detected as a face candidate area.

The face tracking unit 20 temporally tracks the face candidate area detected by the face pattern detection unit 19.
The face authentication process 22 determines whether the person M is a person registered in advance by comparing the face image obtained from the face tracking unit 20 with the dictionary information in the face authentication dictionary 21.

FIG. 3 schematically shows the overall processing flow, which will be described below.
First, in the image input process of step S1, an analog video signal from the camera 11 is input. Next, in the A / D conversion process of step S2, the A / D converter 12 converts the analog video signal from the camera 11 into a high gradation input image of m bits per pixel, and stores it in the high gradation input image buffer 13. Store.

  Next, in the change detection process in step S <b> 3, the change detection unit 14 detects the area of the moving object from the high gradation input image in the high gradation input image buffer 13 as the change area. The detection result is output as a change image in which the pixel value “1” changes and the pixel value “0” does not change.

  Next, in the head candidate region detection process in step S4, the head candidate region detection unit 15 detects a region that seems to be a person's head from the change image detected by the change detection unit 14. Specifically, contour extraction is performed on the change region, and head candidates are detected from the curvature of the extracted contour.

  Next, in the face luminance conversion process in step S5, the face luminance conversion unit 16 generates a low gradation input image from the high gradation input image for the head candidate region from the head candidate region detection unit 15, Stored in the key input image buffer 17.

  Note that the conversion from a high gradation input image to a low gradation input image is made suitable for face pattern detection and face authentication. Specifically, in the low gradation input image, conversion is performed so as not to cause overexposure or underexposure in the face area.

  Next, in the face pattern detection process of step S6, the face pattern detection unit 19 detects the face in the face detection dictionary 18 for the head candidate region of the low gradation input image generated by the face luminance conversion unit 16. By performing pattern matching with the dictionary pattern, a face-like area is detected as a face candidate area.

  Next, in the face tracking process in step S7, the face tracking unit 20 temporally tracks the face candidate area detected by the face pattern detection unit 19. When the face tracking unit 20 obtains a certain number (N_IMG) of face images for the person whose face is being tracked (step S8), the face authentication process in step S9 is performed.

  In the face authentication process in step S9, the person M is a person registered in advance by collating the face image obtained from the face tracking unit 20 with the dictionary information in the face authentication dictionary 21 in the face authentication process 22. Determine whether.

  Hereinafter, the change detection process in step S3, the face luminance conversion process in step S5, the face pattern detection process in step S6, the face tracking process in step S7, and the face authentication process in step S9 will be described in detail.

First, the change detection process in step S3 will be described.
In the change detection process, a change region is detected using a normalized incremental code interframe difference. The normalized incremental code interframe difference is an improvement of the above-described incremental code interframe difference. Hereinafter, calculation of the normalized incremental code interframe difference will be described.

First, from the high gradation input image I _t (x, y) at time t, the horizontal differential image DI_X _t (x, y) and the vertical differential image DI_Y _t (x, y) are expressed by the following equations 8-1 and 8-2. ) To get.

Next, in order to avoid the influence of light shot noise, correction processing shown in the following formulas 9-1 and 9-2 is executed.

Note that α _sd described in the above equations 9-1 and 9-2 is a constant.

  Note that the following formulas A-1 and A-2 may be used instead of the formulas 9-1 and 9-2.

N_DI_Xt (x, y) = (αsd / SD (It (x, y))) * DI_Xt (x, y) [Formula A-1]
N_DI_Yt (x, y) = (αsd / SD (It (x, y))) * DI_Yt (x, y) [Formula A-2]
Note that SD (It (x, y)) described in the above formulas A-1 and A-2 is an approximation of sqrt (It (x, y)), and specifically, as follows: Realize.

SD (I) calculation example 1
Prepare SD_ARRAY (I) (I = 0, 1, 2, .., 2 ^m − 1) with the number of elements of 2 ^m gradation of the high gradation input image, and SD_ARRAY (I) = sqrt (I) Hold the value in SD_ARRAY (I) so that (I = 0, 1, 2, .., 2 ^m − 1).

  For example, when m = 12, n = 8:

SD_ARRAY (0) = 0
SD_ARRRY (1) = 1
SD_ARRRY (2) = 1.14
SD_ARRRY (3) = 1.73
SD_ARRRY (4) = 2
….
SD_ARRAY (4095) = 63.99
In Formula A-1 and Formula A-2, calculation is performed according to the following Formula B.

SD (It (x, y)) = SD_ARRAY (It (x, y)) [Formula B-1]
SD (I) calculation example 2
Calculate as piecewise linear approximation as SD (I). Specifically, it is approximated by a dotted line (partial line) in FIG. FIG. 18 shows an example in the case of m = 12. The solid line in FIG. 18 is sqrt (I). This curve is divided by points A, B, C and D in FIG. 18 and approximated by a straight line indicated by a dotted line connecting them in order. In a situation where the accuracy of calculation is not so necessary, the number of points used may be reduced when creating a piecewise straight line such as a straight line connecting A, B, and D, for example.

Next, as shown in the following formulas 10-1 and 10-2, the corrected images N_DI_X _t (x, y) and N_DI_Y _t (x, y) are compared with a predetermined threshold TH_SD. As a result, the normalized incremental code images N_DS_X _t (x, y) and N_DS_Y _t (x, y) are acquired, respectively.

Further, N_DS_X _t-1 (x, y) and N_DS_Y _t-1 for the high gradation input image I _t (x, y) at time _t−1 are calculated in the same manner.

Next, change images N_DIFF_S_X _t-1 (x, y) and N_DIFF_S_Y _t-1 (x, y) are calculated by the following formulas 11-1 and 11-2.

Finally, a normalized increment code inter-frame difference image N_DIFF_S _t (x, y) is calculated by the following equation 12.

Next, the face luminance conversion process in step S5 will be described.
The face luminance conversion process generates a low gradation input image from a high gradation input image for the head candidate region. That is, the brightness histogram of the high gradation input image is calculated for the central region of the head candidate region. If the obtained luminance histogram is as shown in FIG. 4A, for example, the maximum and minimum luminance values of the luminance histogram are obtained as I_max and I_min, respectively. In the tone input image, the luminance conversion is performed so that the minimum gradation value “0” and the maximum gradation value “2 ⁿ −1” are obtained.

In this luminance conversion, luminance conversion as shown in the following Expression 12 is performed only for the head candidate region. Here, I _{in in the following} equation 13 is the luminance value of the high gradation input image, and 1 _out is the luminance value of the low gradation input image.

  An example of luminance conversion is shown in FIG. When the luminance histogram of the high gradation input image for the central region of the head candidate region is (b) in FIG. 5, as a result of the face luminance conversion, the central region of the head candidate region of the low gradation input image obtained as a result of the face luminance conversion is obtained. The luminance histogram is as shown in FIG. FIG. 5A is an analog output level histogram for the analog output signal of the camera 11 and the face area.

Next, the face pattern detection process in step S6 will be described.
Face pattern detection is performed on the low gradation input image of the head candidate area obtained by the face luminance conversion process. Examples of face pattern detection methods include literature (Mita, Kaneko, Hori, “Proposal of Spatial Difference Probability Template Suitable for Authentication of Images Containing Minute Differences”, Proceedings of the 9th Meeting of Image Sensing Symposium, SSII03, The technique described in 2003) can be used. This is a technique in which a face detection dictionary pattern is created in advance from a face learning pattern, and an area having a high similarity with the face detection dictionary pattern is searched from an input image.

  For example, the face detection dictionary pattern is created from a large number of face images facing the front. The face detection dictionary pattern created in this way is stored in the face detection dictionary 18 and used in the face pattern detection, whereby a face candidate region that is substantially front-facing is detected. The detected face candidate area is represented as a square face candidate rectangle FE as shown in FIG.

Next, the face tracking process in step S7 will be described.
In the face tracking process, the face candidate area in the current frame is obtained from the face candidate area in the previous frame. FIG. 7 shows the flow of the face tracking process. Here, the previous frame is an input image at time t−1, and the current frame is an input image at time t.

  First, an optical flow is extracted for the face candidate area of the previous frame (step S21). Specifically, the face candidate area of the previous frame is divided into small blocks, and the position of each block in the current frame is calculated using template matching or the like. FIG. 8 shows an example of the optical flow of the face candidate area FE.

The small block W from which the optical flow is extracted preferably has both edges in the x direction and the y direction. This is because if an optical flow is extracted from a flat region having no edge, an erroneous optical flow is extracted. Specifically, an optical flow is extracted from a small block W that satisfies the following expression (14).

Here, T _x (W) and T _y (W) are the average of the absolute value of the differentiation for x of the small block W and the average of the absolute value of the differentiation for y, respectively. V (W) is the average luminance of the small block W, and TH_TEXTURE is a threshold value.

In the calculation of E _texture (W), the average of the absolute value of the differentiation is divided by the square root of the luminance average of the small block W because the average value of the absolute value of the differentiation due to the light shot noise in the bright small block. This is to prevent the optical flow from being calculated for a flat region not including an edge.

T _x (W), T _y (W), and V (W) are expressed as follows. Here, Aw in the following formulas 15-1, 15-2, and 15-3 is the area of the small block W.

In addition, you may replace the said Formula 14 with the following formula C.

Etexuture (W) = min (Tx (W), Ty (W)) / SD (V (W))> TH_TEXTURE [Formula C]
SD (V (W)) is an approximation of sqrt (V (W)). The calculation is performed by the same method as described in the specific example of the SD (I) calculation method described above.

Here, the center point of the small block of the face candidate area of the previous frame and the point of the current frame corresponding to the center point are defined as follows.

The center point of the small block in the face candidate area of the previous frame is as shown in Equation 17 below.

The point of the current frame corresponding to the center point is as shown in the following equation (18).

Next, face deformation parameter calculation is performed (step S22). That is, the following conversion formula is calculated.

For example, the following equations 20-1 and 20-2 are used as conversion equations.

  This is a conversion formula for scaling and translation of the x-axis and y-axis, respectively.

Next, A _x , B _x , A _y , B _y are predicted using regression analysis, and the obtained values are set as A ′ _x , B ′ _x , A ′ _y , B ′ _y , and the following equation 21− 1. The calculation of Expression 21-2 is performed.

When the values obtained by Expression 21-1 and Expression 21-2 are both smaller than a predetermined threshold, it is determined that face deformation parameters have been calculated (step S23), and the face candidate area of the previous frame enclosing rectangle R ^t-1 after calculating a circumscribed rectangle R ^t of the face candidate region of the current frame, the position of the face candidate region of the current frame (step S24), and ends the process.

R ^t-1 of the upper left and the coordinates of the points of the lower right, ^{respectively, (LEFT t-1, TOP} t-1), (RIGHT t-1, and BOTTOM ^t-1, of the upper left point and the lower right of the ^{R t} Assuming that the coordinates are (LEFT ^t , TOP ^t ) and (RIGHT ^t , BOTTOM ^t ), the coordinates of the upper left and lower right points of R ^t are the following expressions 22-1, 22-2, and 22-3. , Converted as shown in Equation 22-4.

  If the calculation of the deformation parameter fails (step S23), that is, if the values obtained by Expression 21 are both equal to or larger than a predetermined threshold, the face candidate region of the previous frame is expanded by a certain size. Pattern detection is performed (step S25). If a face candidate area is detected from the face pattern (step S26), it is set as a face candidate area in the current frame (step S27), and the process is terminated. If the face candidate area is not detected by the face pattern (step S26), the face candidate area of the previous frame is set as the face candidate area of the current frame (step S28).

  Next, it is checked whether or not the number of frames in which the face candidate area of the previous frame is continuously the face candidate area of the current frame is greater than or equal to a predetermined threshold T_NOT_DETECTED (step S29). If it is equal to or greater than the threshold value T_NOT_DETECTED, it is assumed that the person M has disappeared from the field of view of the camera 11, and the face tracking is stopped (step S30), and the process is terminated.

Next, the face authentication process in step S9 will be described.
In the face authentication process, it is determined whether or not the face image obtained by the face tracking process is a person registered in the face authentication dictionary 21 in advance. Specifically, for example, as described in Japanese Patent Laid-Open No. 2001-266152, a facial image of a registrant (pedestrian) is prepared in advance, and unique feature information of the face extracted from the facial image is stored in a dictionary. It is stored (registered) in the face recognition dictionary 21 as information. Then, the face area image (feature information) obtained by the face tracking process and the dictionary information registered in advance in the face recognition dictionary 21 are collated to obtain the similarity between them, and the obtained similarity is set in advance. If it is greater than or equal to the threshold, it is determined that the person (pedestrian) M is a person registered in advance, and if the obtained similarity is not greater than or equal to the threshold, the person M is determined not to be a person registered in advance.

  As described above, according to the first embodiment, change detection and head candidate detection are performed for a high gradation input image, then face luminance conversion is performed for the head candidate region, and the obtained low By performing face pattern detection on the gradation input image, the high gradation input image is processed without switching the gain value, so that it is possible to easily control the brightness of the face for a moving person. On the other hand, as described above, if it is assumed that an image is acquired while switching a plurality of gain values, it is difficult to perform luminance control for a moving person.

  In addition, even when there are a plurality of persons as shown in FIG. 13 and the illuminances of the faces of the persons differ greatly, the brightness control may be appropriately performed on the faces of the individual persons. In the face image obtained by applying the gain value switching method in the case of FIG. 13, appropriate brightness control is performed only for one person as shown in FIG. 14A or FIG. 14B. On the other hand, in the first embodiment, since brightness control is performed on each face separately, a result of appropriate brightness control as shown in FIG. 9 is obtained.

Further, in the calculation of the difference between the incremental code frames for detecting the change, the differential image is divided by the square root of the luminance of the input image, and the normalized incremental code interframe difference which is less affected by the light shot noise is used. 10B obtained from the situation of FIG. 10B, and DI_X _t (x, y) shown in FIG. 10C obtained from FIG. 10B are converted into N_DI_S _t (x, y) shown in FIG. Noise component is reduced. Note that FIGS. 10A to 10C correspond to FIGS. 16A to 16C, and thus detailed descriptions of FIGS. 10A to 10C are omitted.
As a result, the difference image N_DIFF_S _t (x, y) is as shown in FIG. 11, and the light shot noise is not detected as a change. Thereby, the performance in the subsequent processing is improved, and as a result, the authentication performance as face authentication is improved.

Next, a second embodiment will be described.
The apparatus configuration and the process flow of the second embodiment are basically the same as those of the first embodiment described above, and therefore the description thereof will be omitted. Only the parts different from the first embodiment will be described. . In the second embodiment, in the change detection process in step S3, the change area is detected using the time difference normalized luminance inter-frame difference. The time difference normalized luminance inter-frame difference is an improvement of the above-described luminance inter-frame difference so as not to be affected by light shot noise. Hereinafter, calculation of time difference normalized luminance inter-frame difference will be described. To do.

First, from the input image I _t (x, y) at time t and the input image I _t-1 (x, y) at time t−1, a difference image DI _t (x, y) shown in the following Expression 23 is calculated. .

Next, the calculated difference image DI _t (x, y) is normalized by the following equation 24. Here, α _d in the following equation 24 is a constant.

Next, as shown in the following equation 25, the normalized difference image N_DI _t (x, y) is compared with the threshold value TH_FD to obtain a change image N_DIFF _t (x, y).

N_DIFF _t (x, y) = 1 indicates that a change is detected in the pixel (x, y). N_DIFF _t (x, y) = 0 represents that no change was detected in the pixel (x, y).

  As described above, according to the second embodiment, in the difference between luminance frames in which change is detected, the time difference in which the luminance time difference is divided by the square root of the luminance of the input image and is less susceptible to light shot noise. By using the normalized luminance inter-frame difference, compared to the luminance inter-frame difference, it becomes less susceptible to light shot noise, and the change detection performance is improved. Thereby, the performance in the subsequent processing is improved, and as a result, the authentication performance as face authentication is improved.

1 is a block diagram schematically showing the configuration of a face authentication device according to a first embodiment of the present invention. The schematic diagram explaining the installation state of a camera. The flowchart which shows the flow of the whole process roughly. The figure which shows the brightness | luminance histogram of a high gradation input image. The figure which shows the brightness | luminance histogram of the low gradation input image converted from the high gradation input image. The figure explaining face brightness control. The figure explaining a face candidate area | region. The flowchart which shows the flow of a face tracking process. The figure explaining the optical flow about a face candidate area | region. The figure explaining the result of face luminance control. The figure explaining normalization of an edge image. The figure explaining normalization of an edge image. The figure explaining normalization of an edge image. The figure explaining normalization of an edge image. The figure explaining the process of the difference between normalized increment code frames. The figure explaining the example of A / D conversion by a gain value. The figure explaining the example where brightness control of a face is difficult. The figure explaining the example where brightness control of a face is difficult. The figure explaining the result of face luminance control. The figure explaining the result of face luminance control. The figure explaining optical shot noise. The figure explaining optical shot noise. The figure explaining optical shot noise. The figure explaining the influence by the optical shot noise of a 1st derivative. The figure explaining the influence by the optical shot noise of a 1st derivative. The figure explaining the influence by the optical shot noise of a 1st derivative. The figure explaining the process of the difference between incremental code frames. The figure for demonstrating the calculation of SD (I).

Explanation of symbols

  M, M1, M2 ... Pedestrian (moving person), 11 ... Camera (image acquisition means), 12 ... A / D converter (A / D conversion means, image acquisition means), 13 ... High gradation input image buffer, DESCRIPTION OF SYMBOLS 14 ... Change detection part (change detection means), 15 ... Head candidate area | region detection part (head candidate detection means), 16 ... Face luminance conversion part (luminance conversion means), 17 ... Low gradation input image buffer, 18 ... Face detection dictionary (memory), 19 ... Face pattern detection unit (face detection means), 20 ... Face tracking unit (face tracking means), 21 ... Face authentication dictionary (memory, dictionary storage means), 22 ... Face authentication unit (face) Authentication means).

Claims (20)

Image acquisition means for acquiring an image including at least a face of a moving person as a high gradation input image;
A change detecting means for detecting a moving object area as a change area from a high gradation input image obtained at regular intervals from the image acquisition means;
Head candidate detection means for detecting a human head candidate area from the image of the change area detected by the change detection means;
Luminance conversion means for generating a low gradation input image from a high gradation input image by performing luminance conversion on the image of the head candidate area detected by the head candidate detection means;
Face detection means for detecting a human face candidate area with respect to the head candidate area of the low gradation input image generated by the luminance conversion means;
Face authentication means for determining whether or not the person is a pre-registered person by collating the image of the face candidate area detected by the face detection means with pre-registered dictionary information;
A face authentication apparatus comprising:   The change detection means acquires a horizontal differential image and a vertical differential image from a high gradation input image obtained at regular intervals from the image acquisition means, and uses the acquired horizontal differential image and vertical differential image to increment code frames. A change area is detected by calculating a difference. In calculating the difference between the incremental sign frames, the obtained horizontal differential image and vertical differential image are divided by the square root of the luminance of the high gradation input image. The face authentication apparatus according to claim 1, wherein correction processing for removing the influence of light shot noise generated by the image acquisition unit is performed.   The change detection means detects a change area by calculating a difference between luminance frames for a high gradation input image obtained at regular intervals from the image acquisition means. A correction process for removing the influence of light shot noise generated by the image acquisition means is performed by performing a calculation of dividing the time difference of luminance by the square root of the luminance of the high gradation input image. Item 2. The face authentication device according to Item 1. A horizontal differential image and a vertical differential image are acquired from a high-gradation input image obtained at regular intervals, and a change area is detected by calculating an incremental code interframe difference using the acquired horizontal differential image and vertical differential image. An image processing apparatus that
In the calculation of the difference between the incremental code frames, the influence of light shot noise generated at the time of image acquisition is eliminated by performing the calculation by dividing the acquired horizontal differential image and vertical differential image by the square root of the luminance of the high gradation input image. An image processing apparatus characterized by performing a correction process for the purpose. An image processing apparatus for detecting a change area by calculating a difference between luminance frames for a high gradation input image obtained at regular intervals,
In the calculation of the difference between luminance frames, a correction process is performed to eliminate the influence of light shot noise that occurs at the time of image acquisition by calculating the time difference of luminance by the square root of the luminance of the high gradation input image. An image processing apparatus. An image acquisition step of acquiring an image including at least a face of a moving person as a high gradation input image;
A change detection step for detecting a moving object region as a change region from a high gradation input image obtained at regular intervals from this image acquisition step;
A head candidate detection step for detecting a human head candidate region from the image of the change region detected by the change detection step;
A luminance conversion step of generating a low gradation input image from the high gradation input image by performing luminance conversion on the image of the head candidate region detected by the head candidate detection step;
A face detection step of detecting a human face candidate region with respect to the head candidate region of the low gradation input image generated by the luminance conversion step;
A face authentication step for determining whether or not the person is a pre-registered person by comparing the image of the face candidate area detected by the face detection step with pre-registered dictionary information;
A face authentication method comprising: A horizontal differential image and a vertical differential image are acquired from a high-gradation input image obtained at regular intervals, and a change area is detected by calculating an incremental code interframe difference using the acquired horizontal differential image and vertical differential image. An image processing method for
In the calculation of the difference between the incremental code frames, the influence of light shot noise generated at the time of image acquisition is eliminated by performing the calculation by dividing the acquired horizontal differential image and vertical differential image by the square root of the luminance of the high gradation input image. An image processing method characterized by performing a correction process for the purpose. An image processing method for detecting a change area by calculating a difference between luminance frames for a high gradation input image obtained at regular intervals,
In the calculation of the difference between luminance frames, a correction process is performed to eliminate the influence of light shot noise that occurs at the time of image acquisition by calculating the time difference of luminance by the square root of the luminance of the high gradation input image. An image processing method. Detecting means for detecting a change area between a plurality of frames constituting the input image of the first gradation, and detecting a head candidate area from the change area;
Conversion means for converting the luminance of the head candidate region and generating an input image having a second gradation lower than the first gradation from the input moving image having the first gradation;
Face pattern detection means for detecting a face pattern from the input image of the second gradation,
An image processing apparatus comprising:   The detection means performs a correction process based on a luminance value of the input image or an average value of the luminance on the feature amount calculated from the differential value of the input image, calculates a difference between frames, and calculates the change area. The image processing apparatus according to claim 9, wherein the image processing apparatus is detected. An image processing apparatus for calculating a difference between frames,
An image processing apparatus, wherein a feature value calculated from a differential value of an input image is subjected to correction processing based on a luminance value of the input image or an average value of the luminance values, and an inter-frame difference is calculated.   The spatial differential image is obtained from the input image, the spatial differential image is divided by a value calculated from the luminance of the input image, and a normalized incremental code interframe difference is calculated. Image processing device.   13. The image processing apparatus according to claim 12, wherein the value calculated from the luminance of the input image is a value calculated by a function that outputs a larger value as the average input luminance value increases.   12. The image processing according to claim 11, wherein a spatial differential image is obtained from the input image, the spatial differential image is divided by a square root of the luminance of the input image, and the normalized incremental code interframe difference is calculated. apparatus.   The image processing apparatus according to claim 11, wherein the time difference in luminance of the input image is divided by a value calculated from the luminance of the input image to calculate a time difference normalized luminance inter-frame difference.   The image processing apparatus according to claim 15, wherein the value calculated from the luminance of the input image is a value calculated by a function that outputs a larger value as the average input luminance value increases.   The image processing apparatus according to claim 11, wherein a time difference in luminance of the input image is divided by a square root of the luminance of the input image to calculate a time difference normalized luminance inter-frame difference.   The average value of the differential value of the small area W included in the input image is divided by the value calculated from the average value of the luminance of the input image, whether or not the optical flow is extracted is determined, and the optical flow is extracted. The image processing apparatus according to claim 11, wherein a small area is determined.   19. The image according to claim 18, wherein the value calculated from the average luminance value of the input image is a value calculated by a function that outputs a larger value as the average input luminance value increases. Processing equipment.   The average value of the differential value of the small area W included in the input image is divided by the square root of the average value of the luminance of the input image to determine whether or not to extract the optical flow, and the small area from which the optical flow is extracted is determined. The image processing apparatus according to claim 11, wherein the determination is performed.

Images