JP5955031B2 - Face image authentication device - Google Patents

Description

  The present invention relates to a face image authentication device, and more particularly to a face image authentication device that authenticates a user based on a pre-registered face image.

  A face image authentication device that authenticates a person by collating face images extracts an image of an area including the face of the person from the monitoring image as a face area image, and extracts the extracted face area image and a plurality of registered face images registered in advance. The collation score with each, that is, the similarity is calculated. Then, the person of the face area image is determined to be the person with the highest similarity among the persons of the registered face images having a certain degree of similarity. However, when a person is determined based on a face area image extracted from one monitoring image, the face area image and the registered face image of the person depending on the facial expression, face orientation, etc. when the monitoring image is captured The degree of similarity becomes lower and the person cannot be correctly determined.

  Thus, Patent Document 1 proposes a person specifying device that specifies a person by capturing a face image of the target person a plurality of times. This person specifying device captures the face image of the target person a plurality of times, and calculates the similarity with each registered face image for each captured face image. Then, for each registered face image, an average value, total value, or multiplication value of a plurality of similarities calculated is obtained as an evaluation value, and the evaluation value of the registered face image having the highest evaluation value exceeds a predetermined reference value. In this case, the target person is identified as the person of the registered face image.

JP 2005-242890 A

Since the person specifying device described in Patent Literature 1 specifies a target person based on a plurality of face images, it is possible to reduce the failure of personal authentication as compared with a case where a person is determined from one face image.
On the other hand, in order to authenticate a person walking in the monitoring area, the monitoring images are sequentially taken, and the similarity between the face area image extracted from each of the taken monitoring images and each of the plurality of registered face images is calculated in time series. A so-called walk-through type face image authentication apparatus is conventionally used. The walk-through type face image authentication device extracts face area images from images obtained by photographing a person walking at a plurality of points or time points, and calculates the similarity between these face area images and registered face images. In such a walk-through type face image authentication device, the illumination condition for the face of the person differs depending on the position of the person when the monitoring image is taken, and affects the similarity between the face area image and the registered face image. There is a case. Therefore, when the person identification device described in Patent Document 1 is applied to the walk-through type face image authentication device, the evaluation value becomes equal to or less than a predetermined reference value due to the influence of the similarity when the illumination conditions are bad. May not be determined correctly.

  In order to reduce the influence of the illumination conditions, for example, it is conceivable to irradiate the monitoring area with high-intensity illumination from the side of the imaging device that captures the monitoring image, but in that case, a separate illumination device is required. In addition, when visible light is used as the illumination, there is a problem that it is dazzling for a person walking in the monitoring area.

  An object of the present invention is to provide a face image authentication apparatus capable of reducing the influence of illumination conditions and authenticating a person with high accuracy.

  The present invention for solving this problem provides a face image authentication apparatus. Such a face image authentication apparatus includes an imaging unit that sequentially acquires a monitoring image obtained by photographing a person in a monitoring area, a storage unit that stores a registered face image of a registered person in advance, and the monitoring image every time a monitoring image is acquired. A face detection means for extracting a face area image including a face from the face, a face tracking means for tracking the face area image from the sequentially acquired monitoring images, a light source direction estimation means for estimating the light source direction from the monitoring images, and a face area Classifying means for classifying images into a plurality of groups for each light source direction in substantially the same direction, and for each of the plurality of groups, a group similarity indicating the degree of similarity between the face area included in the group and the registered face image is obtained. The group selection means for selecting the group having the highest group similarity, the group similarity of the selected group is compared with a predetermined authentication threshold, and the person in the monitoring area and the registered person are Having a determining means for determining whether or not one person.

  Further, in the face image authentication device according to the present invention, the storage unit stores the registered face images of a plurality of registered persons in advance, and the group selection unit obtains the similarity between the face area image and the plurality of registered face images, A registered face image having the highest degree of similarity is identified for each face area image, a registered face image having the largest number of times of identification is selected for each group, and the selected registered face image and the faces included in the group are selected. It is preferable that the average value of the similarity with the region image is set as the group similarity.

  In the face image authentication device according to the present invention, the storage unit stores in advance registered face images of a plurality of registered persons, and the group selection unit includes a plurality of registered face images for each of the face area images included in the group. It is preferable that the similarity value of the registered face images having the highest average similarity value for each face area image is determined as the group similarity.

  The face image authentication apparatus according to the present invention has an effect of being able to authenticate a person with high accuracy by reducing the influence of illumination conditions.

It is a schematic block diagram of the face image authentication apparatus to which this invention is applied. It is a schematic diagram showing the installation example of an imaging part in case a face image authentication apparatus is installed in the entrance of an office building. (A)-(d) is a figure which shows the example of an image when an object person is each irradiated from the specific position. It is a schematic diagram of the history table memorize | stored in a memory | storage part. It is the schematic diagram which looked at the entrance of the office building in which the face image authentication apparatus shown in FIG. It is a schematic diagram of a frame collation result. It is a schematic diagram showing the object person who moves a monitoring area in time series. (A)-(c) is a graph for demonstrating the similarity with respect to the object person shown in FIG. It is a flowchart which shows operation | movement of the face image authentication apparatus by this invention.

Hereinafter, a face image authentication apparatus according to an embodiment of the present invention will be described with reference to the drawings.
In the authentication in the face image authentication device, there are cases where the authentication of the registrant fails due to the influence of lighting conditions or the like. Therefore, the face image authentication apparatus to which the present invention is applied sequentially acquires monitoring images obtained by photographing the monitoring areas, and extracts face area images including the same person's face from the sequentially acquired monitoring images. Then, the extracted face area image is classified into a plurality of groups for each face area image whose light source direction is substantially the same direction, a group most suitable for face matching processing is selected, and the face area image of the selected group is used. Perform face matching processing. Thereby, the face image authentication device attempts to authenticate a person with high accuracy by reducing the influence of illumination conditions.

  FIG. 1 is a diagram showing a schematic configuration of a face image authentication apparatus 10 to which the present invention is applied. As illustrated in FIG. 1, the face image authentication device 10 includes an imaging unit 100, an output unit 200, and an image processing unit 300. Hereinafter, each part of the face image authentication device 10 will be described in detail.

The imaging unit 100 is a monitoring camera that captures a predetermined monitoring area, and is, for example, a photoelectric conversion element (for example, a CCD sensor, a C-MOS, or the like) that is arranged two-dimensionally and outputs an electrical signal corresponding to the amount of received light. And an imaging optical system for forming an image of the monitoring region on the photoelectric conversion element.
The imaging unit 100 is installed so that the face of a person who passes through the monitoring area (a person in the monitoring area to be collated by the face image authentication apparatus 10 is referred to as a target person) can be sequentially photographed. Then, the imaging unit 100 acquires a monitoring image obtained by capturing the monitoring area at predetermined time intervals (for example, 200 msec). The imaging unit 100 is connected to the image processing unit 300 and passes the acquired monitoring image to the image processing unit 300.

  The monitoring image can be a grayscale or color multi-tone image. In the present embodiment, the monitoring image is a color image having 1280 pixels by 960 pixels and having 8-bit luminance resolution for each of the RGB colors. However, a monitor image having a resolution and gradation other than this embodiment may be used.

  FIG. 2 schematically shows an installation example of the imaging unit 100 when the face image authentication device 10 is installed at the entrance of an office building. As shown in FIG. 2, for example, when the face image authentication device 10 is installed at the entrance 253 of an office building, the imaging unit 100 includes a wall on which the entrance 253 is installed so as to include a passage leading to the entrance 253 in the monitoring area. It is attached to the top or ceiling of the camera with the shooting direction slightly downward and toward the passage. As a result, the imaging unit 100 can capture the target person heading toward the entrance 253 (heading in the traveling direction 254) at a predetermined time interval. FIG. 2 shows a state in which the imaging unit 100 sequentially captures the same target person 250, 251, 252 heading toward the entrance 253 at times t, t + 1, t + 2.

  As shown in FIG. 2, a downlight 260 is installed on the ceiling of the passage leading to the entrance 253 so that the irradiation direction is downward. When viewed from the imaging unit 100, the target person 250 is positioned on the back side of the downlight 260 at the time t, and the front surface of the face is illuminated by the downlight 260. At time (t + 1), the target person 251 is positioned immediately below the downlight 260 and is irradiated from the overhead by the downlight 260. At time (t + 2), the target person 252 is positioned on the near side of the downlight 260 and is irradiated from behind by the downlight 260. The passage shown in FIG. 2 is sufficiently bright due to a lighting device (not shown) or natural light, and the face image authentication device 10 can sufficiently identify the target person, but the closer the target person is to the downlight 260, the more the face of the target person is. It is assumed that the light amount corresponding to the portion is more dominant in the downlight 260.

3A to 3D show examples of images when the target person is irradiated from a specific position.
FIG. 3A shows an image 350 obtained by capturing the target person 250 at time t in FIG. In the image 350, the target person 250 is irradiated with light from the downlight 260 from above and the entire face is brightened by the downlight 260 and a lighting device (not shown) or natural light. In general, in the face collation process, it is determined whether or not the target person is a person of the registered face image based on whether or not the face image of the target person is similar to a registered face image stored in advance. Therefore, it is preferable that the target person's face image and the registered face image appropriately represent the positions, shapes, and textures of the skin portions such as the eyes, nose and mouth. In the image 350, there is no obstacle to the collation process in the face portion of the target person, and the position, shape, and texture of the skin portion, etc., which are characteristic parts of the face appear appropriately. Suitable for processing.

  FIG. 3B shows an image 351 obtained by capturing the target person 251 at time (t + 1) in FIG. In the image 351, the target person 251 is irradiated with the light of the downlight 260 from substantially above. Since the human face has irregularities and the concave part does not receive light from directly above, it becomes a shadow, so only the forehead, upper cheeks and nose are brightened, and it is dark near the orbit, from the lower cheeks to the chin It has become. As described above, the image 351 is bright and dark on the face, and the facial texture information is different from the original one. In addition, there is a possibility that this light / dark border is erroneously extracted as a characteristic part of the face. Therefore, the image 351 is not suitable for face collation processing.

  FIG. 3C shows an image 352 obtained by capturing the target person 252 at time (t + 2) in FIG. In the image 352, the target person 252 is illuminated from behind, the downlight 260 is backlit, and the target person 252 has the entire face dark. Therefore, the tone of the face portion is lost due to so-called blackening, and the image texture 352 may lose face texture information, and is not suitable for face matching processing. Even if the face portion of the target person 252 is brightened by a separate brightness conversion process, the tone is lost, so the necessary contrast cannot be obtained, and the face texture information is lost. Not suitable for verification processing.

  FIGS. 3A to 3C show the case where the downlight 260 is installed on the ceiling and irradiated with light from a substantially vertical direction, but the same thing is applied with light from a substantially horizontal direction. Sometimes it happens. FIG. 3D shows an image 353 obtained by capturing the target person when sunlight is radiated from a substantially horizontal direction with respect to the traveling direction 254. In the image 353, light hits from the left side of the target person, the left half face is bright but the right half face is dark, and a difference in brightness occurs along the midline of the face. For this reason, the image 353 is not suitable for the face collation process as is the case with the image 351 in FIG. In this way, not only the light from above like a downlight but also the window where the target person is photographed, and the light from the lateral direction such as sunlight inserted through the window has an adverse effect on the face matching process. There is a risk that.

  The output unit 200 includes a communication interface connected to, for example, an electric lock or an external device that controls the electric lock, and a control circuit thereof. When the output unit 200 receives a signal indicating successful authentication of the target person from the image processing unit 300, the output unit 200 outputs a signal requesting, for example, unlocking of the electric lock to the connected device.

  The image processing unit 300 is configured by, for example, a so-called computer. Then, the image processing unit 300 extracts an image of an area including a person's face from the monitoring image received from the imaging unit 100 as a face area image, and estimates a light source direction viewed from the person's face from the extracted face area image. . Then, each face area image is classified into a plurality of groups for each face area image whose estimated light source direction is substantially the same direction, a group most suitable for face matching processing is selected, and the face area image of the selected group is used. To perform face matching. For this purpose, the image processing unit 300 includes a storage unit 310 and a collation unit 320. Furthermore, the collation unit 320 includes a face detection unit 321, a face tracking unit 322, a light source direction estimation unit 323, a classification unit 324, and a face collation unit 325.

  The storage unit 310 includes a semiconductor memory such as a ROM and a RAM, or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 310 stores in advance a computer program and various parameters for controlling the face image authentication apparatus 10. The storage unit 310 stores in advance a registered face image showing the face of the registered person for each of one or more registered persons (hereinafter referred to as registered persons) in association with the registered person ID of the registered person. To do. The storage unit 310 also stores a history table for managing information related to the target person generated by the image processing. Details of the history table will be described later. The storage unit 310 stores at least one standard three-dimensional face shape model.

  Each unit of the verification unit 320 is a functional module implemented by a microprocessor, a memory, a peripheral circuit thereof, and software operating on the microprocessor. Alternatively, these means may be integrated by firmware. Moreover, you may comprise some or all of these means with an independent electronic circuit, firmware, a microprocessor, etc. Hereinafter, each means of the collation part 320 is demonstrated in detail.

Each time the face detection unit 321 receives a monitoring image from the imaging unit 100, the face detection unit 321 detects a face area that is an area in which the face of the target person is reflected from the received monitoring image, and extracts a face area image.
In order to detect a face area, the face detection unit 321 uses, for example, an inter-frame difference process or a background difference process, and changes with time in luminance values in a plurality of monitoring images acquired by the imaging unit 100 are performed. Extract regions. Then, the face detection unit 321 extracts, from the extracted change areas, a change area that seems to be a person from a feature amount such as the size of the change area as a person area. Then, the face detection unit 321 performs edge pixel extraction on the extracted person region using a Sobel filter or the like so that the inclination direction of the luminance change can be known. Then, the face detection unit 321 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels, and uses the area surrounded by the edge distribution as a face area. Extract. In this case, the face detection unit 321 can detect an elliptical edge distribution using, for example, a generalized Hough transform.

Alternatively, the face detection unit 321 may detect a face area using an Adaboost classifier. This method is described in a paper by P. Viola and M. Jones “Rapid Object Detection Using a Boosted Cascade of Simple Features” (Proc. The IEEE International Conference on Computer Vision and Pattern Recognition, vol.1, pp.511-518, 2001). ) Can be referred to.
Then, the face detection unit 321 creates a face region image by cutting out the detected face region from the monitoring image, and outputs the face region image and the coordinate information of the face region in the monitoring image to the face tracking unit 322.

The face tracking unit 322 performs a tracking process using a known tracking technique on the face area detected by the face detection unit 321 from each of a plurality of monitoring images continuously acquired at a predetermined time interval, Track face area image.
For example, the face tracking unit 322 detects the center of gravity of the face area detected from the latest monitoring image (hereinafter referred to as the face area of the current frame) and the face area detected from the monitoring image one frame before (hereinafter referred to as the previous area). The distance of the center of gravity position (referred to as a face area of the frame) is obtained, and when the distance is equal to or less than a predetermined threshold, the face area is associated with the same person. The face area in the monitoring image before and after the movement when the target person moves a certain distance when moving away from the imaging unit 100 and when the target person moves the same distance when moving closer to the imaging unit 100. The position difference is different. Therefore, for example, by setting a predetermined threshold as the size of the face area, it is appropriately determined whether the face area of the current frame and the face area of the previous frame are from the same person regardless of the position of the target person in the monitoring area. Can be evaluated. When a plurality of face areas are extracted, it is checked whether or not face areas with the closest distance between the centroid positions correspond to each other.

  Alternatively, the face tracking unit 322 may perform face area tracking processing using a method such as an optical flow or a particle filter.

The face tracking unit 322 updates the history table stored in the storage unit 310 when the face area tracking process is performed. FIG. 4 shows an example of the history table. As shown in FIG. 4, the history table 400 manages history data for each target person. The history data is a set of data represented in each row of the history table 400, and the face tracking unit 322 includes a trial number 401, a target person ID 402, a start time 403, a tracking flag 404, face image data 405, and the like among the history data. The tracking position information 406 is updated.
In the history table 400 shown in FIG. 4, the trial number 401 is an identification number for identifying the history data of the target person being tracked from other history data, and while the target person exists in the monitoring area. That is, the same identification number continues to be assigned while being tracked by the face tracking means 322. The target person ID 402 is an identification number for identifying the target person being tracked from other target persons, and corresponds to the trial number 401 on a one-to-one basis. The target person ID 402 is given so as to be uniquely determined. For example, a random number may be generated based on the time when the face tracking unit 322 starts tracking. Alternatively, since the target person ID 402 and the trial number 401 correspond one-to-one, either one may be omitted. The start time 403 represents the time when the face detection unit 321 first extracts a face area image for the target person and the face tracking unit 322 starts the tracking process.
The tracking flag 404 is a flag indicating whether or not the tracking of the target person is continued, and is turned on when the face tracking unit 322 starts tracking the target person, and is turned off when the tracking is finished. That is, when the tracking flag 404 is ON, the target person exists in the monitoring area, and when the tracking flag 404 is OFF, the target person has already been authenticated and has entered the room, or left without being authenticated. It will be that.
The face image data 405 is data of all face area images created by the face detection unit 321 and associated with the target person by the face tracking unit 322 in the tracking process. The tracking position information 406 is time information indicating the coordinate information of the face area in the monitoring image from which each face image data stored as the face image data 405 is cut out and the time when the tracking process is performed.

  If the face area of the current frame of interest cannot be associated with the face area of the previous frame, the face tracking unit 322 shows the target person newly entering the monitoring area in the face area of the current frame. Assuming that the target person is newly added to the history table 400, initialization processing is performed. That is, the face tracking unit 322 assigns a new trial number 401 and a target person ID 402 to the history data, records the current time as the start time 403, and sets the tracking flag 404 to ON. Further, the face tracking unit 322 records a face area image created from the face area of the current frame as the face image data 405, and records the coordinate information and the current time of the face area as the tracking position information 406. Further, the face tracking unit 322 sets the authentication flag 408 to OFF, and outputs newly added history data to the light source direction estimating unit 323 and the face matching unit 325.

  On the other hand, when the face tracking unit 322 can associate the face area of the current frame of interest with the face area of the previous frame, history data about the target person has already been created in the history table 400. Therefore, the face tracking unit 322 adds the face area image created from the face area of the current frame to the face image data 405 of the corresponding history data in the history table 400, and also adds the face area image of the face area to the tracking position information 406. An update process for adding coordinate information and the current time is performed. The face tracking unit 322 outputs the history data to the light source direction estimating unit 323 and the face collating unit 325.

  Further, the face tracking unit 322 turns off the tracking flag 404 of the corresponding history data in the history table 400 when there is a face area of the previous frame that is not associated with all face areas of the current frame. Then, the tracking process for the target person is terminated.

（１）式において、ρ(x,y)は、位置(x,y)における顔表面の反射率、l₀は光源係数、lは光源方向ベクトルを表す。またn(x,y)は、位置(x,y)における顔表面の法線方向ベクトルを表し、その位置(x,y)に対応する３次元顔形状モデル上の位置及び顔の向きに基づいて求められる。 The light source direction estimation means 323 is the latest face of the history data output from the face tracking means 322 and whose history flag 400 in the history table 400 shown in FIG. 4 is ON and the authentication flag 408 is OFF. From the region image and the three-dimensional face shape model stored in advance in the storage unit 310, the light source direction of the light emitted to the face shown in the face region image is estimated. Hereinafter, the light source direction is the direction of the light source viewed from the face of a person shown in the face area image. Various known methods can be used as a method for estimating the light source direction. For example, the light source direction estimation unit 323 estimates the light source direction from the luminance distribution in the face area image by the following method.
First, it is assumed that the face surface is a uniform diffusing surface (Lambertian surface) in which the intensity of light diffused by the surface is proportional to the cosine of the angle with the normal of the surface. In this case, assuming that the horizontal direction of the face area image is the x axis, the vertical direction is the y axis, and the upper left point is the origin, the luminance E (x, y) at the position (x, y) on the face area image is It is considered that the expression is determined by the three-dimensional shape of the face, the direction of the light source, and the reflectance of the face surface.

In equation (1), ρ (x, y) represents the reflectance of the face surface at the position (x, y), l ₀ represents a light source coefficient, and l represents a light source direction vector. N (x, y) represents the normal vector of the face surface at the position (x, y), and is based on the position on the 3D face shape model corresponding to the position (x, y) and the face orientation. Is required.

Here, it is assumed that the skin of the face is composed of the same component regardless of the location, and ρ (x, y) in Equation (1) has a constant value α. In this case, equation (1) is an equation with the light source coefficient l ₀ and the light source direction l as unknowns. Therefore, the light source direction estimating means 323 sets the equation (1) as a simultaneous equation for each pixel in the region corresponding to the facial skin in the face region image, and solves the simultaneous equations for each orientation adjusted shape model. The light source coefficient l ₀ and the light source direction l can be obtained for each. Note that the constant value α can be set to an average value, a mode value, a median value, or a value in the vicinity of the luminance value of the area corresponding to the skin of the face in the face area image.

Alternatively, the light source direction estimation means 323 prepares a standard face image obtained by photographing a person's face in various light source directions or an equivalent face image obtained by simulation, and pattern matching between them and the face area image. And the light source direction may be estimated by determining the most matching face image. Furthermore, the light source direction estimation unit 323 may be configured to know in advance the positional relationship between the illumination light source and the camera that acquired the face area image, or the relationship between the direction of the illumination light emitted from the illumination light source and the shooting direction of the camera. The light source direction may be determined based on the relationship.
The light source direction estimation unit 323 outputs the estimated light source direction to the classification unit 324 as light source direction information.

  Based on the light source direction information received from the light source direction estimating unit 323, the classifying unit 324 classifies each face area image into a plurality of groups for each face area image having the same light source direction. For example, as shown in FIGS. 3A to 3C, the classification unit 324 includes a “front” group that is a group of face area images irradiated from the front and a face area image irradiated from directly above. The group is classified into a “directly above” group that is a group and a “back” group that is a group of face area images irradiated from behind.

FIG. 5 schematically shows a view of the entrance of the office building where the face image authentication apparatus 10 shown in FIG. 2 is installed as seen from the right side in the traveling direction 254. The target persons 250 to 252 and the downlight 260 illustrated in FIG. 5 correspond to the target persons 250 to 252 and the downlight 260 illustrated in FIG. 2.
As shown in FIG. 5, at time t, the downlight 260 is positioned forward and upward as viewed from the target person 250, and the angle θ of the downlight 260 with respect to the traveling direction 254 is less than 90 °. On the other hand, at time (t + 1), the downlight 260 is positioned substantially immediately above the target person 251 and the angle θ of the downlight 260 with respect to the traveling direction 254 is approximately 90 °. At time (t + 2), the downlight 260 is positioned rearward and upward as viewed from the target person 250, and the angle θ of the downlight 260 with respect to the traveling direction 254 is greater than 90 °.

The classifying unit 324 obtains the angle θ of the downlight 260 with respect to the traveling direction 254 from the light source direction estimated by the light source direction estimating unit 323, and determines the face area image as a front group, an immediately above group, or a rear group according to the obtained angle θ. Classify into: Note that the boundary value of the angle θ included in the immediately above group is an angle that becomes a boundary at which a shadow is formed on the face area image by the light source. For example, the classifying unit 324 classifies face area images having an angle θ of 85 ° ≦ θ ≦ 92 ° into the immediately above group, classifies face area images having an angle θ of θ <85 °, and sets the angle θ to be the front group. Face area images satisfying θ> 92 ° are classified into rear groups. In addition, when the angle θ is 90 ° or less, if the angle θ is not sufficiently less than 90 °, the entire face as shown in FIG. 3A is not bright, but when the angle θ is 90 ° or more, If the angle exceeds 90 °, the entire face tends to be dark as shown in FIG. Therefore, the boundary value between the immediately upper group and the rear group is set to 92 °, which is closer to 90 ° than 85 ° which is the boundary value between the front group and the immediately above group.
The classifying unit 324 may classify the face area image more finely than the three groups of the front group, the immediately above group, and the rear group. For example, as shown in an image 353 in FIG. 3D, classification is performed based on a component in the left and right direction with respect to the traveling direction 254 in the light source direction in consideration of irradiation of sunlight from the left and right direction with respect to the traveling direction 254. May be.
The classifying unit 324 outputs group information indicating the group into which the face area image is classified to the face matching unit 325.

  The face matching unit 325 includes a group selection unit 326 and a determination unit 327, selects a group most suitable for the face matching process from the groups classified by the classification unit 324, and uses the face area image of the selected group. To perform face matching. Also, the verification history 407 and the authentication flag 408 in the history table 400 are updated.

The matching history 407 holds a frame matching result corresponding to the tracking position information 406 of each frame. FIG. 6 shows an example of the frame matching result 600.
The acquisition time 601 is the time when the face area image of the current frame is acquired.
The light source direction group 602 represents a group of each face area image classified by the classification unit 324, and the group information output from the classification unit 324 is set.
The authentication candidate person 603 has the highest similarity (first place) among the similarities (matching scores) between the face area image tracked by the face tracking unit 322 and each of the plurality of registered face images stored in the storage unit 310. This is the identification ID of the registered person.
The authentication candidate similarity 604 is a matching score for specifying the authentication candidate person 603.

  Returning to FIG. 4, the authentication flag 408 is a flag indicating whether or not the authentication of the target person has succeeded. That is, when the determination unit 327 determines that the target person is one of the registered persons, it is set to ON, and the authentication result is not obtained by the determination unit 327 as in the state where the tracking of the target person has just started. In the case of a state or when the determination means 327 determines that the target person is not a registered person, it is set to OFF.

Of the history data output from the face tracking unit 322, the group selection unit 326 displays the latest face area of the history data in which the tracking flag 404 of the history table 400 illustrated in FIG. 4 is ON and the authentication flag 408 is OFF. The similarity between the face shown in the image and the face shown in each registered face image stored in advance in the storage unit 310 is calculated. The group selection means 326 calculates the similarity using a known face matching technique. For example, the group selecting unit 326 calculates the sum of squares of the difference in luminance value between each pixel included in the face area image and the corresponding pixel of the registered face image while shifting the positions of the face area image and the registered face image, and calculates the calculated square. The reciprocal of the normalized value obtained by dividing the smallest sum of the sum by the number of pixels included in the face area image is obtained as the similarity.
The group selection unit 326 specifies the registered person of the registered face image having the highest similarity with the face area image to be processed, that is, the authentication candidate person, and stores the information (registered person identification ID) in the authentication candidate person 603. Then, the similarity is stored in the authentication candidate similarity 604. Further, the group selection unit 326 stores the group information output from the classification unit 324 in the light source direction group 602.

Further, the group selecting unit 326 includes each of a plurality of groups classified by the classifying unit 324 based on the similarity between each face area image included in the group and each registered face image. A registered face image showing a face most similar to the face shown in the face area image is selected as a group candidate face image of the group. Then, the registered person of the selected group candidate face image is set as a group candidate person. For example, for each group, the group selection means 326 sets the registered face image that is identified most frequently as the face image of the authentication candidate person as the group candidate face image, and sets the group candidate face image as the group candidate person. In that case, the group selection unit 326 sets the authentication candidate person recorded most as the authentication candidate person 603 for the same light source direction group 602 as the group candidate person. Alternatively, the registered person of the registered face image having the highest average value, median value, or quartile of each similarity with each face area image included in the group may be set as the group candidate person.
Next, the group selection unit 326 calculates the group similarity for each of the plurality of groups classified by the classification unit 324 from the authentication candidate similarity 604 recorded for the authentication candidate person who is a group candidate person. The group similarity represents the degree of similarity between each face area image included in the group and the group candidate face image, and the higher the authentication candidate similarity 604 for each authentication candidate person that is a group candidate person, the higher the group similarity degree. . For example, any one of the average value, the median value, and the quartile of all the authentication candidate similarities 604 for the authentication candidate persons that are the group candidate persons is set as the group similarity.
Then, the group selection unit 326 selects the group having the highest calculated group similarity as the group that performs the determination process.

The group selection processing by the group selection unit 326 will be described in detail with reference to FIGS.
FIG. 7 is a schematic diagram showing the target person passing through the monitoring area in time series. Figure 7 shows a target person toward the traveling direction 721, the imaging unit 100 sequentially imaging the same target person 701-709 a at time t ₁ ~t ₉ respectively. In addition, three downlights 711 to 713 are installed along the traveling direction 721 on the ceiling of the monitoring area. The downlights 711 to 713 irradiate the target persons 701, 704, and 707 from the upper front side, irradiate the target persons 702, 705, and 708 from directly above, and irradiate the target persons 703, 706, and 709 from the upper rear side.

8A to 8C are graphs for explaining the similarity to the target person shown in FIG. In the following description, it is assumed that the registered face images of the three registered persons A, B, and C are stored in the face image authentication apparatus 10.
FIG. 8A is a graph showing the authentication candidate person specified by the group selection unit 326 for the target persons 701 to 709 and the calculated similarity. In FIG. 8A, the horizontal axis represents time, and the vertical axis represents similarity. Times 801 to 809 are times t _{1 to} t ₉ , respectively, and correspond to times when the target persons 701 to 709 in FIG. Each x mark at each time represents the similarity between the face area image for the target person and the registered face images for the registered persons A, B, and C at each time in order from the left side. A person represented as A, B, or C above the cross is a registered person having the highest similarity at each time, and the registered person is specified as an authentication candidate person by the group selection unit 326. That is, the registered person A at time t ₁ , t ₂ , t ₄ , t ₆ , t ₇ , the registered person B at time t ₃ , t ₅ , t ₈ , and the registered person C at time t ₉ are authentication candidates. Identified as a person.
In the example shown in FIG. 8A, the target person is illuminated from the front at times t ₁ , t ₄ , and t ₇ , and the face area image at that time is an image suitable for face matching processing. Therefore, the following description will be made assuming that the registered person A identified as the authentication candidate person at the times t ₁ , t ₄ , and t ₇ is a true target person who passes through the monitoring area.

FIG. 8B is a graph in which the similarities of the persons specified as authentication candidate persons at each time of FIG. 8A are rearranged for each authentication candidate person. In FIG. 8B, the horizontal axis represents each authentication candidate person, and the vertical axis represents the similarity. The authentication candidate person 811 is the registered person A, and the similarity at the times t ₁ , t ₂ , t ₄ , t ₆ , t ₇ is expressed in order from the left side, and the authentication candidate person 812 is the registered person B, in order from the left side. The similarity at time t ₃ , t ₅ , t ₈ is represented, the authentication candidate person 813 is the registered person C, and the similarity at time t ₉ is represented.
As shown in FIG. 8B, the average similarity 821 for the registered person A, that is, the authentication candidate person 811, is the average similarity 822 for the registered person B, that is, the authentication candidate person 812, or the registered person. C, that is, lower than the average value 823 of the similarity for the authentication candidate person 813. This is because the similarity similarity average value 821 for the authentication candidate person 811 is extremely low similarity obtained from the face area image at time t ₂ when the target person is irradiated from directly above and face texture information cannot be extracted appropriately. This is because of the influence of the degree.
As shown in FIG. 8B, when the monitoring area is illuminated by a downlight or the like, there is a possibility that a face area image that cannot properly extract facial texture information may be obtained. Therefore, it is determined that the authentication candidate person having the highest average similarity is the target person among all the authentication candidate persons specified for each face area image acquired while the target person exists in the monitoring area. Then, there is a risk of erroneous determination.

FIG. 8C is a graph in which the similarities of the persons specified as the authentication candidate person at each time of FIG. 8A are rearranged for each group classified by the classifying unit 324. In FIG. 8C, the horizontal axis represents each group, and the vertical axis represents similarity. The group 831 is a forward group, and the similarities at times t ₁ , t ₄ , and t ₇ are represented in order from the left side, and the group 832 is a direct upper group, and the similarities at times t ₂ , t ₅ , and t ₈ in order from the left side. The group 833 is a rear group, and the degrees of similarity at the times t ₃ , t ₆ , and t ₉ are shown in order from the left side.
If the similarities are classified in this way, each group may include similarities for a plurality of authentication candidate persons. Therefore, as described above, the group selection unit 326 sets the person most frequently identified as an authentication candidate person in each group as the group candidate person of that group. In FIG. 8C, the registered person A is a group candidate person for the front group 831, and the registered person B is a group candidate person for the immediately above group 832. For the rear group 833, registered persons A, B, and C are identified as authentication candidate persons once each. In order to improve the accuracy of the collation process in consideration of “similarity of others” that cannot be avoided with a certain probability in face image authentication, it is specified as an authentication candidate person for a predetermined number of times (for example, twice) or more. It is preferable that the person who is present is a group candidate person. Therefore, no group candidate person is selected for the rear group 833.
As illustrated in FIG. 8C, the average value 841 of the similarity for the group candidate person A of the front group 831 is higher than the average value 842 of the similarity for the group candidate person B of the immediately above group 832. This is because the similarity for the front group 831 is calculated from a face area image from which all face texture information can be appropriately extracted, and the similarity for the immediately above group 832 is a face area from which all face texture information cannot be extracted appropriately. This is because it is calculated from the image.
In the example illustrated in FIG. 8C, the group selection unit 326 selects the front group 831 having the highest average degree of similarity for the group candidate person as the group for performing the determination process.

  The determination unit 327 compares the at least one face area image included in the group selected by the group selection unit 326 with the group candidate face image, so that the target person passing in the monitoring area and the registered face image are reflected. It is determined whether or not the person who is present is the same person. For this purpose, the determination unit 327 determines whether the group similarity of the group selected by the group selection unit 326 is greater than or equal to the authentication threshold, for example. The authentication threshold is a reference for authenticating whether or not the face included in each face area image of the selected group is the face of a group candidate person. If the group similarity is equal to or higher than the authentication threshold value, the determination unit 327 determines that the face included in each face area image of the group is the face of the registered person specified as the group candidate person, and within the monitoring area It is determined that the target person who passes is the registered person. Then, in the corresponding history table 400, the tracking flag 404 is set to OFF and the authentication flag 408 is set to ON, and a signal indicating successful authentication is output to the output unit 200.

  On the other hand, when the group similarity is less than the authentication threshold even though the tracking is continued for a predetermined time (for example, 2 seconds) or more, the determination unit 327 determines the authentication flag 408 of the history table 400 corresponding to the authentication failure. Is set to OFF. Even when the group candidate person is not selected immediately after the tracking is started or when the matching history 407 has not been acquired for a predetermined frame (for example, 30 frames), the determination unit 327 stores the corresponding history table 400. The authentication flag 408 is set to OFF.

  Hereinafter, the operation of the authentication process performed by the face image authentication apparatus 10 to which the present invention is applied will be described with reference to the flowchart shown in FIG. The flow of operations described below is controlled by a control unit (not shown) that operates on the microprocessor constituting the image processing unit 300 and controls the face image authentication apparatus 10 as a whole. Note that the history table 400 is initialized to be blank before the processing described below is performed. The operation described below is performed every time one monitoring image is acquired.

  First, the face image authentication apparatus 10 acquires a monitoring image obtained by capturing a monitoring area by the imaging unit 100 and sends the monitoring image to the matching unit 320 of the image processing unit 300 (step S901). Next, the face detection unit 321 of the collation unit 320 detects a face area in which the face of the target person is reflected from the acquired monitoring image, and cuts out the face area from the monitoring image to create a face area image ( Step S902). Next, the face detection unit 321 determines whether or not one or more face regions have been detected (step S903). If no face area is detected, the process proceeds to step S901, and the processes in steps S901 to S903 are repeated until the face area is detected.

  On the other hand, if one or more face areas are detected and a face area image is created in step S903, the face tracking unit 322 tracks all the detected face areas in the history table 400 shown in FIG. Association with the target person ID 402 for which the flag 404 is ON is performed (step S904). Then, the face tracking unit 322 determines whether there is a target person ID 402 for which the tracking flag 404 is ON that is not associated with any of the face areas of the current frame (step S905). If there is no face area that is not associated with any face area of the current frame, the process proceeds to step S907. On the other hand, when there is a target person ID 402 that is not associated with any of the face areas of the current frame and whose tracking flag 404 is ON, the target person of the target person ID 402 has entered the room with successful authentication, Alternatively, it is considered that the user has left the monitoring area without being authenticated. Therefore, in that case, the face tracking unit 322 turns off the tracking flag 404 of the target person ID 402 that is not associated with any of the face areas of the current frame, and thereafter does not perform the tracking process for the face area (step S906).

  Then, the face tracking unit 322 determines whether there is any face area in the current frame that could not be associated with the target person ID 402 for which the tracking flag 404 is ON (step S907). If there is something that could not be associated with the target person ID 402 for which the tracking flag 404 is ON, the face tracking unit 322 stores history data regarding the target person in the face area that could not be associated in the history table 400. Add a new one. That is, the face tracking unit 322 assigns a new trial number 401 and target person ID 402 to the history data for the target person in the face area that could not be associated, and records the current time as the start time 403, and the tracking flag 404 Is set to ON (step S908). The face tracking unit 322 records the face area image created from the face area of the current frame as the face image data 405 and also records the coordinate information and the acquisition time of the face area that could not be associated as the tracking position information 406. Then, the authentication flag 408 is set to OFF.

  When the face area of the current frame can be associated with the target person ID 402 whose tracking flag 404 is ON, the face tracking unit 322 displays the coordinate information and acquisition time of the face area that can be associated with each other. An update process for adding to the tracking position information 406 of the corresponding history data is performed. Then, the face tracking unit 322 outputs the updated history data to the light source direction estimating unit 323 and the face collating unit 325.

The following steps S909 to S914 are performed for each history data in which the tracking flag 404 is ON and the authentication flag 408 is OFF. The reason why the history data whose authentication flag 408 is ON among the history data whose tracking flag 404 is ON is not to be processed is that the authentication has already been performed.
The light source direction estimation unit 323 estimates the light source direction for the latest face area image of the history data to be processed, and outputs the estimated light source direction to the classification unit 324 as light source direction information (step S909). Next, based on the light source direction information received from the light source direction estimating unit 323, the classifying unit 324 generates a plurality of latest face area images of the history data to be processed for each face area image whose light source direction is substantially the same. Sort into groups (step S910).

Next, the face matching unit 325 stores the frame matching result 600 in the matching history 407. Each frame matching result 600 has a one-to-one correspondence with the tracking position information 406 in each frame, and the time information of the tracking position information 406 is stored in the acquisition time 601. The acquisition time 601 may be omitted as being the same as the time information of the tracking position information 406.
Next, the group selection means 326 calculates the similarity between all the registered face images stored in the storage unit 310 for the latest face area image of the history data to be processed, and the similarity is the highest. The registered person of the registered face image is specified as an authentication candidate person. Then, the information is stored in the authentication candidate person 603, and the similarity is stored in the authentication candidate similarity 604 (step S911). Further, the group selection unit 326 stores the group information classified by the classification unit 324 in the light source direction group 602.

Next, the group selection unit 326 specifies a group candidate person for each group, calculates a group similarity, and selects a group having the highest calculated group similarity as a group for performing the determination process (step S912). .
When the tracking by the face tracking unit 322 has just started, the number of frames accumulated in the matching history 407 is small, and the group selected by the group selection unit 326 is greatly influenced by the result of one frame. It is necessary not to do. For this reason, it is determined whether or not the number of frames necessary for the determination process is accumulated in the collation history 407, for example, in step S912, and the number of frames necessary for the determination process is not accumulated in the collation history 407. It is preferable that the determination process is prohibited and the process proceeds to step S901. The number of frames necessary for the determination process is appropriately set according to the installation environment of the face image authentication device 10.

  Next, the determination unit 327 determines whether or not the group similarity of the group selected by the group selection unit 326 is greater than or equal to the authentication threshold (step S913). When the group similarity is less than the authentication threshold, if there is still history data in which the tracking flag 404 is ON and the authentication flag 408 is OFF, the process proceeds to step S909, and if not, the process proceeds to step S901. On the other hand, when the group similarity is equal to or higher than the authentication threshold, the determination unit 327 determines that the target person who passes through the monitoring area is the registered person specified as the group candidate person. Then, in the corresponding history table 400, the tracking flag 404 is set to OFF and the authentication flag 408 is set to ON, and a signal indicating successful authentication is output to the output unit 200 (step S914).

  When a signal indicating successful authentication is output to the output unit 200 in step S <b> 914, the face collating unit 325 unlocks the electric lock via the output unit 200. As a result, the target person who has succeeded in authentication can enter the room. It is assumed that the electric lock is locked immediately after the entry of the target person can be confirmed or automatically after a predetermined time has elapsed.

  Note that the similarity calculation processing shown in step S911 may be performed before the light source direction estimation processing and classification processing shown in steps S909 and S910.

  As described above, the face image authentication apparatus to which the present invention is applied classifies face area images into a plurality of groups for each face area image whose estimated light source direction is substantially the same direction, and each group is similar to the group. The degree is calculated, and the group having the highest group similarity is selected. Then, the face image authentication device performs face matching processing based on the similarity between each face area image of the selected group and the registered face image. Thereby, the face image authentication device can reduce the influence of the illumination condition and authenticate the person with high accuracy.

  The preferred embodiments of the present invention have been described above, but the present invention is not limited to these embodiments. For example, in the present embodiment, the face detection unit cuts out the face region from the monitoring image to create a face region image, and each unit of the collation unit performs each process on the face region image. The face detection means may not create a face area image, and each means of the collation unit may perform each process on the face area in the monitoring image.

  In the present embodiment, as the face matching process, an example of performing so-called 1-to-N matching in which the face image of the target person is matched with all registered face images stored in the storage unit has been described. May be a so-called one-to-one collation in which the face image of the target person is collated with a specific registered face image. In that case, the face collating means acquires a registered person ID input by the user via an input unit (not shown) provided with a card reader, a numeric keypad, etc., and corresponds the face image of the target person to the registered person ID. Match the registered face image. Then, the group selection unit calculates the group similarity of each group using the registered person with the registered person ID as the authentication candidate person and the group candidate person, and selects the group for which the determination process is to be performed.

In addition, when the light source direction when each registered face image is acquired is known in advance, each registered face image and the light source direction are stored in association with each other, and the classifying unit stores each face area image of the authentication candidate person. You may classify | categorize into the 1st group corresponding to the light source direction of a registration face image, and a 2nd group other than that. In this case, before the classification unit performs the classification process, the group selection unit performs the similarity calculation process and the authentication candidate person identification process. Then, the classification means includes a face area image whose light source direction is substantially the same as the light source direction of the registered face image of the authentication candidate person, that is, the difference between the light source direction of the registered face image of the authentication candidate person and the light source direction of the face area image. Are classified into the first group, and face region images whose difference is greater than or equal to the predetermined value are classified into the second group. Note that the difference between the light source directions can be, for example, a scalar amount as a difference between the light source direction vectors or a scalar amount as a difference between unit vectors of the light source direction vectors. The predetermined value is appropriately set according to the installation environment of the face image authentication apparatus, and can be set to 1/2 of the scalar quantity of the unit vector, for example.
Thus, by classifying each face area image according to whether or not the light source direction is substantially the same as the light source direction of the registered face image, the face area image having a high similarity to the registered face image and the registered face image Can be classified more accurately, and the face matching process can be performed with higher accuracy.
If the light source direction when each registered face image is acquired is not known in advance, the light source direction estimating means obtains the light source direction of each registered face image and associates each registered face image with the light source direction in the storage unit. You may make it memorize | store. Thereby, even when the light source direction is not known when the registered face image is acquired, the light source direction of the registered face image can be specified.

  Further, a person who is frequently identified as an authentication candidate person has a high degree of similarity with the person to be collated, and there is a high possibility that the person is a person to be collated. Therefore, it is possible to accept the person more by performing the face matching process using only the registered face image of the person most frequently identified as the authentication candidate person and the face area image whose light source direction is substantially the same direction. Can be increased. Therefore, the classifying unit includes the first group corresponding to the light source direction of the registered face image of the person most frequently identified as the authentication candidate person for each face area image associated by the face tracking unit, and the others. The second group may be classified. In that case, for example, after the face matching process for a predetermined frame (for example, 10 frames) is performed on the face area image associated by the face tracking unit, the classifying unit is the largest number of times specified as an authentication candidate person. Identify many people. The classifying unit classifies the face area images in which the difference between the light source direction of the registered face image of the identified person and the light source direction of the face area image is less than a predetermined value into the first group, and the difference is predetermined. Face area images that are greater than or equal to the value are classified into the second group.

  For example, when a photograph such as an employee ID card is used as the registered face image, each photograph is likely to be taken under the same lighting conditions. In such a case, by performing the face matching process using only more registered face images and face area images having substantially the same light source direction, the illumination condition of the face area image used for the face matching process is registered. Can be adapted to the lighting conditions. Therefore, the classifying means classifies each face area image with a first group corresponding to the most frequent direction in which the light source directions of all registered face images stored in the storage unit are most concentrated, and other second groups. May be classified. In that case, for example, the classifying unit divides all directions from the origin in the space defining the light source direction vector into a plurality of blocks, and among the plurality of blocks, the frequency that includes the light source direction of the registered face image is the highest. The block is a mode block, and the center direction of the mode block is a mode direction. Each block can be a range having a width of 5 ° in the latitude direction and the longitude direction, for example. Then, the classifying unit classifies the face area images in which the difference between the obtained most frequent direction and the light source direction of the face area image is less than a predetermined value into the first group, and the face area in which the difference is not less than the predetermined value. Classify the images into a second group.

  In addition, the face area image having a high similarity to the registered face image in the past face matching process is an image suitable for the face matching process, and the illumination condition is highly likely to be favorable. Therefore, by using only such a face area image and a face area image whose light source direction is substantially the same direction, face collation processing can be performed with higher accuracy. Therefore, the determination unit stores, in the storage unit, the light source direction of the face area image having the highest similarity to the registered face image for each person determined to be the same person as the person shown in the registered face image. deep. The classifying unit corresponds to the most frequent direction in which the light source directions of the face area images having the highest similarity to the registered face image are most concentrated among the face area images of the persons who have succeeded in authentication in the past. Each face area image of the current target person may be classified into a first group and a second group other than the first group. In that case, the classification means uses the same method as that for obtaining the most frequent direction of the light source directions of all registered face images, and the mode of the light source direction of each face area image having the highest similarity to the registered face image. Find the direction. Then, the face area images in which the difference between the obtained most frequent direction and the light source direction of the face area image is less than a predetermined value are classified into a first group, and face area images in which the difference is greater than or equal to a predetermined value are Classify into groups.

  Further, in the monitoring area of the face image authentication device, when a plurality of downlights are installed at regular intervals along the traveling direction of the target person, the light source direction estimating means The light source direction of each face area image may be estimated based on the temporal change in luminance of the face portion of the area image. In that case, the light source direction estimating means calculates any one of the average value and the median of the brightness of the face part for each face area image, and arranges the calculated average value or median in time series to obtain a local maximum value. And the minimum value is extracted. Then, the face area image corresponding to the local maximum value is determined to receive light from the front and the light source direction is estimated to be forward, and the face area image corresponding to the local minimum value is determined to be receiving backlight. Estimate the direction backward. The classifying unit displays each face area image associated by the face tracking unit with a first group corresponding to the front light source direction, a second group corresponding to the rear light source direction, and a third group other than the first group. Classify into groups. In this case, the classifying unit classifies the face area image temporally close to the face area image whose light source direction is estimated to be forward into the first group, and temporally matches the face area image whose light source direction is estimated to be backward. The face area image close to is classified into the second group, and the other face area images are classified into the third group. For example, the classifying unit classifies each face area image into groups by dividing the face area image into three groups temporally between the face area image with the light source direction in front and the face area image with the light source direction in the rear. Each face area image may be classified into two groups, a first group corresponding to the front light source direction and a second group corresponding to the rear light source direction, or four or more groups. You may classify.

  Further, the group selection means selects a predetermined number (for example, 5) of face area images in descending order of similarity for each group classified by the classification means, and the authentication candidate person having the largest number of face area images selected. May be a group candidate for that group. In that case, the group selection means further uses only the similarity of the face area images selected as a predetermined number of face area images in descending order of similarity among the similarities of the authentication candidate persons that are the group candidate persons. Thus, the group similarity may be calculated. Thereby, although the number of sheets with the highest similarity is large, it is possible to suppress selection of an authentication candidate person having a low degree of similarity as a group candidate person.

  The determination unit calculates a ratio of the face area image identified as the authentication candidate person with respect to all the face area images of the group selected by the group selection unit as a registered person detection rate, and calculates the calculated registered person detection rate. The authentication threshold may be determined accordingly. In this case, the determination unit lowers the authentication threshold as the registered person detection rate increases. Thus, by adaptively determining the authentication threshold value, the face image authentication device can ensure the user's acceptance even in a situation where the user authentication is difficult due to a change in facial expression, face orientation, and the like.

Further, in the embodiments described so far, the description has been made on the assumption that face images of a plurality of registered persons (for example, registered persons A, B, and C) are stored in the storage unit.
The face image authentication apparatus according to the present invention can achieve the same effect even if there is only one registered person.
That is, when the downlight is located almost directly above or behind the target person, a face area image that is not suitable for authentication is obtained because of the luminance distribution, and even if the similarity with the registered face image of the person is obtained, Only low. For this reason, if all the similarity between the face area image at each time and the registered face image is determined in a batch without considering the light source direction, the low similarity obtained from the face area image not suitable for authentication is used. The degree of adverse effects adversely affects the authentication threshold and does not exceed the authentication threshold. If the authentication threshold value is lowered in order to avoid the rejection of the principal, it is not preferable to lower the authentication threshold because the authentication succeeds and accepts another person when another person tries to authenticate.
Even in such a case, the face image authentication device according to the present invention classifies the face area images in consideration of the light source direction, and the similarity obtained from the face area images of the group in the light source direction suitable for authentication is high. Is used to calculate the group similarity and determine authentication. Therefore, the face image authentication apparatus can obtain correct results while avoiding the person's rejection.

In the embodiments described so far, the light source direction estimating means uses the face area image extracted by the face detecting means, and the direction of the light source viewed from the face of the target person in the face area image. Was going to be estimated. Instead of this, the light source direction estimation means may estimate the light source direction viewed from the entire body of the target person, not from the local face area but from the entire monitoring image acquired by the imaging unit. In other words, like a sundial, it is possible to detect the direction of the shadow extending from the vicinity of the target person's feet in the monitoring image and determine that the light source is in the opposite direction.
For example, when a shadow extends from the target person's feet toward the target person's back, the light source direction can be estimated to be in front of and above the target person. Similarly, if a shadow extends from the subject's feet to the front of the subject person, the light source direction can be estimated to be behind and above the subject person, and if almost no shadow can be detected, the light source direction is the subject It can be estimated that it is almost directly above the person. In this method, the approximate light source direction can be estimated with a lower load than the method described in the description of the light source direction estimating means.

  As described above, those skilled in the art can make various modifications in accordance with the embodiment to be implemented within the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Face image authentication apparatus 100 Imaging part 200 Output part 300 Image processing part 310 Storage part 320 Collation part 321 Face detection means 322 Face tracking means 323 Light source direction estimation means 324 Classification means 325 Face collation means 326 Group selection means 327 Determination means

Claims (2)

An imaging unit that sequentially acquires a monitoring image obtained by photographing a person in the monitoring area;
A storage unit for storing registered face images of a plurality of registered persons in advance;
Face detection means for extracting a face area image including a face from the monitoring image each time the monitoring image is acquired;
Face tracking means for tracking the face area image in the monitoring images acquired sequentially;
Light source direction estimating means for estimating a light source direction from the monitoring image;
Classification means for classifying the face area image into a plurality of groups for each light source direction in substantially the same direction;
For each of the plurality of groups, obtain a similarity between the face area image included in the group and the plurality of registered face images, identify a registered face image having the highest similarity for each of the face area images, Select the registered face image with the most specified number of times for each group, and obtain a group similarity that is an average value of the similarity between the selected registered face image and the face area image included in the group, A group selection means for selecting the group having the highest group similarity,
A determination unit that compares the group similarity of the selected group with a predetermined authentication threshold to determine whether the person in the monitoring area and the registered person are the same person;
A face image authentication apparatus characterized by comprising: An imaging unit that sequentially acquires a monitoring image obtained by photographing a person in the monitoring area;
A storage unit for storing registered face images of a plurality of registered persons in advance;
Face detection means for extracting a face area image including a face from the monitoring image each time the monitoring image is acquired;
Face tracking means for tracking the face area image in the monitoring images acquired sequentially;
Light source direction estimating means for estimating a light source direction from the monitoring image;
Classification means for classifying the face area image into a plurality of groups for each light source direction in substantially the same direction;
For each of the plurality of groups, a similarity with the plurality of registered face images is obtained for each of the face area images included in the group, and the registered face having the highest average value of the similarities for each face area image A group selection means for obtaining a group similarity that is the average value of the images and selecting a group having the highest group similarity;
A determination unit that compares the group similarity of the selected group with a predetermined authentication threshold to determine whether the person in the monitoring area and the registered person are the same person;
A face image authentication apparatus characterized by comprising:

Images