JP5871765B2 - Face image authentication device - Google Patents

Description

  The present invention relates to a face image authentication apparatus that authenticates a person based on a face image.

  Conventionally, in order to authenticate a walking person, an image of an area including a person's face is extracted as a face area image from an image of the walking person, and the extracted face area image and a plurality of registered persons registered in advance A so-called walk-through type face image authentication apparatus has been developed that calculates a collation score, that is, a degree of similarity with each registered face image, which is a face image of a person, in order of time series.

  The walk-through type face image authentication device extracts face area images from images obtained by photographing a person walking at a plurality of points or time points, and calculates the similarity between these face area images and registered face images. And the person of the registration face image whose similarity is more than a certain level is specified for each point or time point. Further, the maximum value or the average value of the similarity at each point or each time point is calculated for each specified person. Finally, the person is authenticated based on the maximum value or the average value calculated in this way (Patent Document 1).

  When acquiring similarities to registered face images in time series for face area images of the same person extracted over a plurality of images, that is, sequentially acquiring similarities for a plurality of recent face area images of the same person It is desirable that any similarity is high. In this case, an accurate authentication result can be obtained even if a maximum value or an average value of a plurality of similarities is used.

  However, in a situation where the faces of passing persons are sequentially photographed, variations may occur in facial expressions, face orientations, shadows due to lighting conditions, and the like that occur naturally during walking. When such a change occurs, even if the person is the person, the similarity is low, while the person of the face reflected in the face area image is determined to be a person other than the person, so-called “other's similarity” May occur, and the similarity of others may be higher than the similarity of the principal. This leads to the acceptance of others, and it is inappropriate to simply make a determination based on a plurality of maximum similarity values. In addition, when determining based on the average value of a plurality of similarities, the average value of the plurality of similarities may not exceed the authentication threshold, and the person may not be determined correctly.

  Therefore, an authentication device that authenticates a person based on the result of a plurality of collation processes is disclosed. This authentication apparatus stores a plurality of registered face images for each registered person, calculates the similarity with each registered face image for the input verification target person's collation image, and the similarity is within a predetermined rank. The registered person with the largest number of registered face images is set as the representative for the collation image. The authentication device executes a similar process every time a verification image is input to determine a representative, and when a specific registered person becomes a representative for a predetermined number of times or becomes a representative for a predetermined number of times or more. If this is the case, it is determined that the specific registered person is most likely to be an authentication subject (Patent Document 2).

JP 2004-118359 A JP 2008-59533 A

  The authentication device disclosed in Patent Literature 2 can accurately authenticate a person based on the result of a plurality of collation processes. However, when people with very similar faces, such as twins, are registered as registered persons, the registered persons who are representatives of each matching image are dispersed, and a specific registered person is There is a possibility that you will not be a representative continuously or you will not be a representative more than a predetermined number of times. In that case, there is a possibility that the authentication fails even though the person to be authenticated is registered.

  An object of the present invention is to provide a face image authentication apparatus capable of authenticating a person with high accuracy even when persons having similar faces are registered.

  The present invention for solving this problem provides a face image authentication apparatus. Such a face image authentication device, for each of a plurality of registered persons registered in advance, the correspondence between registered face information representing the facial features of the registered person and complementary persons that are other registered persons related to the registered person A storage unit that stores the relationship; an image acquisition unit that sequentially acquires an input image obtained by capturing a predetermined shooting region; a face detection unit that extracts a face region including a human face from the input image; and the extracted face region The input face information that represents the facial features included is compared with each of the registered face information of multiple registered persons, and among the multiple registered persons, registered persons with registered face information similar to the input face information are identified as authentication candidate persons. Based on the number of times the registered person or any of the complementary persons associated with the registered person is identified as an authentication candidate person in a plurality of input images with different shooting times. The face displayed in the input image has the same judgment means judges that the face of the registered person, the.

  Further, in the face image authentication device according to the present invention, the identity determination means determines whether the number of candidates for the same registered person in a plurality of input images or the ratio of the number of candidates to the number of frames of the plurality of input images is a criterion. When it satisfies, it is preferable to determine that the person is the registered person.

  Further, in the face image authentication device according to the present invention, the identity determination means performs determination based on the number of candidates for a registered person who has obtained the highest similarity as a result of matching with respect to a plurality of input images among a plurality of registered persons. Preferably it is done.

  The face image authentication apparatus according to the present invention further includes an operation unit and a setting unit that associates a complementary person with a registered person according to an operation from the operation unit, and the setting unit includes a plurality of instructions specified by the operation unit. When the degree of similarity of registered face information of a registered person is equal to or greater than a predetermined threshold, it is preferable to associate one registered person with the other registered person as a complementary person.

  The face image authentication apparatus according to the present invention has an effect of being able to authenticate a person with high accuracy even when persons having similar faces are registered.

It is a schematic block diagram of the face image authentication apparatus to which this invention is applied. It is a schematic diagram showing the installation example of an imaging part in case a face image authentication apparatus is installed in the entrance of an office building. It is a schematic diagram of the registration table memorize | stored in a memory | storage part. It is a schematic diagram of the collation table memorize | stored in a memory | storage part. It is a schematic diagram of the collation history which comprises a part of collation table shown in FIG. (A) is a schematic diagram which shows the example of the registration face image of a registered person, (b) is a schematic diagram which shows the example of a face area image. It is a flowchart which shows operation | movement of the face image authentication apparatus by this invention. It is a flowchart which shows the operation | movement of the authentication process performed by a face collation means.

Hereinafter, a face image authentication apparatus according to an embodiment of the present invention will be described with reference to the drawings.
The face image authentication apparatus to which the present invention is applied stores registered face images that are face images of a plurality of registered persons registered in advance. At this time, when there is another registered person related to the registered person (hereinafter referred to as a complementary person for convenience), the registered person is associated with the complementary person. As a complementary person, for example, a person who can be determined to have a face similar to the registered person in the face collation processing of the face image authentication apparatus, such as a twin, is set. The face image authentication device calculates the similarity between each of the face images tracked in the input image obtained by sequentially capturing the walking person and each of the registered face images stored in advance. The registered person of the registered face image for which the highest similarity is calculated among the similarities calculated for the most recent face images has a similarity for each of the most recent face images. It is determined whether or not the walking person is the registered person on the basis of the sum of the number of times of becoming the first place and the number of times that the complementary person of the registered person is ranked first. In this way, the face image authentication device can share the number of times that the degree of similarity is the highest between the registered person and the complementary person, so that even if a person whose face is similar to each other is registered, It is possible to suppress the dispersion of registered persons who are ranked first and to authenticate persons with high accuracy.

  FIG. 1 is a diagram showing a schematic configuration of a face image authentication apparatus 10 to which the present invention is applied. As illustrated in FIG. 1, the face image authentication device 10 includes an imaging unit 100, an output unit 200, a display unit 300, an operation unit 400, and an image processing unit 500. Hereinafter, each part of the face image authentication device 10 will be described in detail.

The imaging unit 100 is a surveillance camera that captures a predetermined area. Hereinafter, the predetermined area that the imaging unit 100 captures is referred to as an imaging area. The imaging unit 100 is, for example, a two-dimensional array of photoelectric conversion elements (for example, a CCD sensor, a C-MOS, etc.) that outputs an electrical signal corresponding to the amount of received light, and an image of a shooting area on the photoelectric conversion element. An image forming optical system for forming an image.
The imaging unit 100 is installed so that a person's face passing through the imaging area can be sequentially captured. Then, the imaging unit 100 acquires an input image obtained by imaging the imaging area at predetermined time intervals (for example, 200 msec). The imaging unit 100 is connected to the image processing unit 500 and passes the acquired input image to the image processing unit 500.

  FIG. 2 schematically shows an installation example of the imaging unit 100 when the face image authentication device 10 is installed at the entrance of an office building. As shown in FIG. 2, for example, when the face image authentication device 10 is installed at the entrance 253 of an office building, the imaging unit 100 has a wall in which the entrance 253 is installed so that a shooting area includes a passage leading to the entrance 253. It is attached to the top or ceiling of the camera with the shooting direction slightly downward and toward the passage. As a result, the imaging unit 100 can image a person heading for the entrance 253 (heading in the traveling direction 254) at a predetermined time interval. FIG. 2 shows a state in which the imaging unit 100 sequentially captures the same persons 250, 251, and 252 heading for the entrance 253 at times t, t + 1, and t + 2.

  The input image can be a grayscale or color multi-tone image. In this embodiment, the input image is a color image having 1280 pixels wide × 960 pixels vertically and having a luminance resolution of 8 bits for each of the RGB colors. However, an input image having a resolution and gradation other than this embodiment may be used. Hereinafter, for convenience, the latest input image may be referred to as the current frame, and the previous input image may be referred to as the previous frame.

  The output unit 200 includes a communication interface connected to, for example, an electric lock or an external device that controls the electric lock, and a control circuit thereof. When the output unit 200 receives a signal indicating successful authentication from the image processing unit 500, the output unit 200 outputs a signal requesting, for example, unlocking of the electric lock to the connected device.

  The display unit 300 is connected to the image processing unit 500 and includes a display device such as a liquid crystal display or an organic EL display. The display unit 300 displays various information received from the image processing unit 500 and notifies the administrator.

  The operation unit 400 is an input interface including input devices such as a keyboard and a mouse, receives an operation from an administrator, and outputs a signal corresponding to the operation to the image processing unit 500. Further, the display unit 300 and the operation unit 400 may be integrated by configuring the display unit 300 with a touch panel display.

  The image processing unit 500 is configured by, for example, a so-called computer. The image processing unit 500 authenticates the person based on the input image received from the imaging unit 100. For this purpose, the image processing unit 500 includes a storage unit 510 and a collation unit 520. Furthermore, the collation unit 520 includes a setting unit 521, a face detection unit 522, a face tracking unit 523, and a face collation unit 524.

  The storage unit 510 includes a semiconductor memory such as a ROM and a RAM, or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 510 stores in advance a computer program and various parameters for controlling the face image authentication apparatus 10. Storage unit 510 also stores a registration table for managing information related to registered persons (hereinafter referred to as registered persons) to be used for verification. Details of this registration table will be described later. The storage unit 510 also stores a collation table for managing information about a person generated by image processing. Details of this collation table will be described later.

  Each means of the collation unit 520 is a functional module implemented by a microprocessor, a memory, a peripheral circuit thereof, and software operating on the microprocessor. Alternatively, these means may be integrated by firmware. Moreover, you may comprise some or all of these means with an independent electronic circuit, firmware, a microprocessor, etc. Hereinafter, each means of the collation part 520 is demonstrated in detail.

  The setting unit 521 causes the display unit 300 to display a screen necessary for the administrator to issue an instruction to register, update, or delete each piece of information in the registration table stored in the storage unit 510. In addition, the setting unit 521 registers, updates, or deletes each piece of information in the registration table stored in the storage unit 510 according to a signal corresponding to the operation from the administrator received from the operation unit 400.

FIG. 3 shows an example of a registration table. As illustrated in FIG. 3, the registration table 310 is managed for each registered person, and includes a registered person ID 311, registered face information 312, and a complementary person ID 313. The registered person ID 311 is an identification number for identifying the registered person from other registered persons.
The registered face information 312 is information representing the facial features of a registered person for use in matching, and is included in the registered face image that is the registered person's face image and the registered face image extracted from the registered face image. A face feature amount which is a feature amount peculiar to the face of a person. Either one of the registered face image and the face feature amount may be omitted. The face feature amount can be, for example, the position and shape of characteristic parts of the face such as eyes, nose, and mouth in the registered face image, and texture information of these characteristic parts or skin parts. Furthermore, as the texture information, for example, the luminance dispersion value, fractal dimension, spatial frequency distribution, and the like of each part can be used. The setting means 521 can use various known methods for calculating a face feature amount from a registered face image. For example, the setting unit 521 performs edge extraction processing on the face image and extracts edge pixels having a large luminance difference from surrounding pixels. The setting unit 521 specifies the position of each part by checking whether or not the feature amount obtained based on the position and pattern of the edge pixel satisfies a predetermined condition for the part such as the eyes, nose, and mouth. By doing so, the position of a characteristic part on the face can be extracted and used as a face feature amount.
The complementary person ID 313 is a registered person ID for a complementary person, which is another registered person related to the registered person, and stores a registered person ID for the complementary person when the registered person has a complementary person. The complementary person is another registered person who is judged to have a similar face to the registered person in the face matching process of the face image authentication apparatus 10 such as a twin and may not be distinguished. The registered person is selected from the registered persons and associated.

Each time the face detection unit 522 receives an input image from the imaging unit 100, the face detection unit 522 detects a face area that is an area in which a person's face is captured from the received input image, and extracts a face area image.
In order to detect the face area, the face detection unit 522 uses, for example, an inter-frame difference process or a background difference process, and changes with time in luminance values in a plurality of input images acquired by the imaging unit 100 are performed. Extract regions. Then, the face detection unit 522 extracts, from the extracted change areas, a change area that seems to be a person from a feature amount such as the size of the change area as a person area. Then, the face detection unit 522 extracts edge pixels so that the inclination direction of the luminance change can be understood from the extracted person region using a Sobel filter or the like. Then, the face detection means 522 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels, and uses the area surrounded by the edge distribution as a face area. Extract. In this case, the face detection unit 522 can detect an elliptical edge distribution using, for example, a generalized Hough transform.

Alternatively, the face detection unit 522 may detect a face area using an Adaboost classifier. This method is described in a paper by P. Viola and M. Jones “Rapid Object Detection Using a Boosted Cascade of Simple Features” (Proc. The IEEE International Conference on Computer Vision and Pattern Recognition, vol.1, pp.511-518, 2001). ) Can be referred to.
Then, the face detection unit 522 creates a face region image by cutting out the extracted face region from the input image, and outputs the face region image and coordinate information of the face region in the input image to the face tracking unit 523.

The face tracking unit 523 performs a tracking process using a known tracking technique on the face area extracted from the face detection unit 522 over a plurality of input images continuously acquired at a predetermined time interval, and the same person The face area image is tracked by associating face areas in which the face is reflected.
For example, the face tracking unit 523 obtains the distance between the centroid position of the face area extracted from the current frame and the centroid position of the face area extracted from the previous frame, and when the distance is equal to or less than a predetermined threshold, The face area is associated with the same person. Note that when the person moves a certain distance when moving away from the imaging unit 100 and when the person moves the same distance when moving closer to the imaging unit 100, the facial region of the input image before and after the movement The difference in position is different. Therefore, for example, by setting a predetermined threshold as the size of the face area, it is appropriately evaluated whether the face area of the current frame and the face area of the previous frame are from the same person regardless of the position of the person in the shooting area. can do. When a plurality of face areas are extracted, it is checked whether or not face areas with the closest distance between the center of gravity positions are associated with each other.

  Alternatively, the face tracking unit 523 may perform face area tracking processing using a method such as an optical flow or a particle filter.

  When the face tracking unit 523 newly detects a face area or associates a face area, the face tracking unit 523 updates the collation table stored in the storage unit 510. FIG. 4 shows an example of the collation table. As shown in FIG. 4, the matching table 410 manages the series data for each person being tracked. The series data is a set of data represented in each row of the collation table 410, and the face tracking unit 523 updates the series ID 411, the update time 412, the tracking flag 413, and the tracking position information 414 in the series data.

In the collation table 410 shown in FIG. 4, the series ID 411 is an identification number for identifying the person being tracked from another person, and while the person exists in the imaging region, that is, by the face tracking means 523. While being tracked, the same identification number continues to be assigned. The series ID 411 is assigned so as to be uniquely determined. For example, a random number may be generated based on the time when the face tracking unit 523 starts tracking.
The update time 412 represents the time when the face detection unit 522 extracted the latest face area image for the person.
The tracking flag 413 is a flag indicating whether or not the tracking of the person is continued. The tracking flag 413 is turned on when the face tracking unit 523 starts tracking the person, and turned off when the tracking is finished. That is, when the tracking flag 413 is ON, the person exists in the shooting area, and when the tracking flag 413 is OFF, the person has already been authenticated and has entered the room, or has left the shooting area without being authenticated. It will be.
The tracking position information 414 is time information indicating the coordinate information of the face area in the input image from which the face area image is cut out and the time when the tracking process is performed.

  If the face tracking unit 523 cannot associate the face area of the current frame of interest with the face area of the previous frame, the face area of the current frame includes a person who has newly entered the shooting area. As a matter of fact, series data for the person is newly added to the collation table 410, and initialization processing is performed. That is, the face tracking unit 523 assigns a new sequence ID 411 to the sequence data, records the current time as the update time 412, and sets the tracking flag 413 to ON. The face tracking unit 523 records the coordinate information and the current time of the face area of the current frame as the tracking position information 414. Further, the face tracking unit 523 sets the authentication flag 416 to OFF, and outputs to the face matching unit 524 a face area image created from the newly added series data and the face area of the current frame associated with the series data. .

  On the other hand, if the face tracking means 523 can associate the face area of the current frame of interest with the face area of the previous frame, the series data for that person has already been created in the matching table 410. Therefore, the face tracking unit 523 performs an update process of adding the coordinate information and the current time of the face area of the current frame to the tracking position information 414 of the corresponding series data in the matching table 410. Then, the face tracking unit 523 outputs the face area image created from the series data and the face area of the current frame associated with the series data to the face matching unit 524.

  Further, the face tracking unit 523 turns off the tracking flag 413 of the corresponding series data in the collation table 410 when there is a face area of the previous frame that is not associated with any face area of the current frame. The tracking process for the person is terminated.

  The face collating unit 524 includes a similarity calculating unit 525, an authentication candidate person specifying unit 526, a detection degree calculating unit 527, and an authentication determining unit 528. The face matching unit 524 includes a person's face shown in the input image and a registered face image. Authenticate a person by matching the face with the person. Here, the similarity calculation unit 525 and the authentication candidate person specifying unit 526 correspond to the similarity determination unit of the present invention, and the detection degree calculation unit 527 and the authentication determination unit 528 correspond to the identity determination unit of the present invention. The face matching unit 524 performs a matching process on the face area image associated with the series data for which the authentication flag 416 illustrated in FIG. 4 is OFF among the series data output from the face tracking unit 523, and performs matching. The history 415 and the authentication flag 416 are updated.

The matching history 415 includes a frame matching result 420 corresponding to the tracking position information 414 for each input image. FIG. 5 shows an example of the matching history 415. As shown in FIG. 5, in the matching history 415, the frame matching result 420 is managed for each face area image tracked by the face tracking unit 523.
Time 421 is the time when the face area image is acquired. Note that the frame number of the input image may be used instead of the time.
The first person ID 422 is the highest of the similarity (matching score) between the face information representing the feature of the face shown in the face area image and each of the plurality of registered face information stored in the storage unit 510 ( This is the registered person ID of the registered person for which the degree of similarity calculated as (first place) is calculated.
The similarity 423 is the similarity obtained for the first person ID 422 with respect to the face information of the face area image.

  The face matching unit 524 stores, in the matching table 410, the matching history 415 calculated from the most recent N images (N is an integer of 2 or more, for example, 10) retroactively from the latest input image. To do. Alternatively, the face matching unit 524 may continue to accumulate the matching history 415 while the person is being tracked, that is, while the series data is stored in the matching table 410.

  Returning to FIG. 4, the authentication flag 416 is a flag indicating whether or not the authentication for the person is successful. That is, when it is determined that the person is one of the registered persons, it is set to ON, and when the authentication result is not obtained as in the state where the person has just started tracking, or the person is registered. If it is determined that the person is not a person, it is set to OFF.

The similarity calculation unit 525 calculates the similarity between the face information about the face area image detected from the input image and each of the registered face information stored in the storage unit 510. The similarity calculating unit 525 extracts a face feature amount from the face area image to be processed, and a value obtained by normalizing a difference between the extracted face feature amount and the face feature amount of the registered face image stored in the storage unit 510. The reciprocal is obtained as the similarity. Note that the facial feature amount extraction method is the same as the facial feature amount calculation method calculated by the setting unit 521 for the registered face image, and thus description thereof is omitted.
Alternatively, the similarity calculation unit 525 may calculate the similarity using a known face matching technique. For example, the similarity calculation unit 525 calculates and calculates the sum of squares of the difference between the luminance values of each pixel included in the face area image and the corresponding pixel of the registered face image while shifting the position of the face area image and the registered face image. The reciprocal of the normalized value obtained by dividing the smallest sum of squares by the number of pixels included in the face area image is obtained as the similarity.

  The authentication candidate person specifying unit 526 specifies a registered person having registered face information similar to the face information about the face area image to be processed as an authentication candidate person. Here, if the highest similarity among the similarities with each of the registered face information stored in the storage unit 510 is equal to or greater than the first threshold, the registered person whose highest similarity is calculated is the authentication candidate person. And The first threshold is set to a value for determining that a certain degree of similarity is recognized. For example, when the similarity is calculated in the range of 0 to 1, it can be set to 0.7. The authentication candidate person specifying unit 526 stores the registered person ID of the registered person specified as the authentication candidate person in the first person ID 422 of the matching history 415 shown in FIG. 5 and stores the similarity in the similarity 423.

Based on the number of times (candidate count) that a registered person or a complementary person of the registered person is set as the first person ID 422 in the latest N comparison results, the detection degree calculating means 527 It is determined whether or not the person is a registered person. Here, the highest value among all the similarities calculated for the face area images extracted from the most recent N input images, that is, the highest value among the latest N similarities 423 in the matching history 415 is obtained. If it is equal to or greater than the second threshold, the registered person ID stored in the first person ID 422 corresponding to the highest value is specified. The second threshold is set to a lower limit value at which the face shown in the face area image can be regarded as the face of the registered person shown in the registered face image. For example, the second threshold is calculated in the range of 0 to 1 in similarity. Can be set to 0.85. The degree-of-detection calculation means 527 determines the number of times that a registered person corresponding to the specified registered person ID is specified as an authentication candidate person for the latest N face area images, that is, the registered person ID of the latest N 1 The number recorded in the ranked person ID 422 is calculated as the number of first hits for the registered person.
Furthermore, the detection degree calculation means 527 determines whether or not a complementary person ID is set for the registered person in the registration table. When the complementary person ID is set, the number of times that the complementary person corresponding to the complementary person ID is identified as the authentication candidate person for the latest N face area images, that is, the N complementary person IDs are the latest. The number recorded in the first person ID 422 is calculated as the number of first hits for the complementary person. Then, the value obtained by adding the number of first hits for the complementary person to the number of first hits for the registered person is again set as the number of first hits (candidate number) for the registered person.
Then, the detection degree calculation unit 527 sets the ratio of the number of first hits for the registered person to the number N of the latest face area images as the first hit rate.

  The authentication determination unit 528 determines whether or not a condition in which the face shown in the N face area images with the most recent first-rank hit rate is the face of the registered person is satisfied. The condition is, for example, that the first-rank hit rate is equal to or higher than the authentication threshold. The authentication threshold value is set to a value that can determine whether or not the face shown in the face area image is the face of the registered person, and can be set to 70%, for example. When the first-rank hit rate is equal to or higher than the authentication threshold, the authentication determination unit 528 determines that the face shown in the face area image is the face of the specified registered person, and the person who passes through the shooting area It is determined that the registered person is identified. In the collation table 410, the authentication flag 416 of the series data is set to ON, and a signal indicating authentication success is output to the output unit 200.

  On the other hand, when a registered person whose first hit rate is equal to or higher than the authentication threshold is not detected, the authentication determination unit 528 leaves the authentication flag 416 of the series data OFF as an authentication failure. In addition, immediately after the start of tracking and when the verification history 415 for N input images has not been acquired, the authentication flag 416 of the series data is kept OFF.

Here, FIG. 5 and FIG. 5 show the effects produced by adding a value obtained by adding the number of first hits for the complementary person to the number of first hits for the registered person to obtain the number of first hits for the registered person. This will be described with reference to 6 (a) and FIG. 6 (b).
FIG. 6A shows a registered face image 601 of a registered person whose registered person ID is ID01, and a registered face image 602 of a registered person whose registered person ID is ID04 whose face is very similar to the registered person. It shows. FIG. 6B shows face area images 611 to 620 extracted at times t1 to t10 of the matching history 415 shown in FIG.
In the face area image of the same person extracted over a plurality of input images, the similarity to the registered face information of the person is always the highest among the similarities calculated for the face information of each face area image. desirable. However, as shown in FIG. 6A, when the persons whose faces are similar to each other are registered, the degree of similarity to the registered face information of the person is not the highest, and the person is similar to the person. In some cases, the degree of similarity with respect to the registered face information becomes first.

As shown in FIG. 6B, for the face area images 611, 612, 615, 617, 619 and 620 at times t1, t2, t5, t7, t9 and t10, photographing such as illumination, facial expression, face orientation, etc. The conditions generally match the shooting conditions of the registered face image 601. As a result, for the face information of these face area images, the similarity to the registered face information of the registered face image 601 is the highest, and as shown in FIG. 5, the authentication candidate person (first person ID 422) is registered with ID01. It is determined that the person is a person.
On the other hand, as shown in FIG. 6B, the shooting conditions of the face area images 614 and 620 at times t4 and t8 are closer to the shooting conditions of the registered face image 602 than the shooting conditions of the registered face image 601. As a result, for the face information of these face area images, the similarity to the registered face information of the registered face image 602 is the highest, and the authentication candidate person (first person ID 422) is registered with ID04 as shown in FIG. It is determined that the person is a person.
In addition, as shown in FIG. 6B, the face orientations 613 and 616 at times t3 and t6 are greatly deviated from the front. As a result, the face information of these face area images has a low similarity between the registered face information of the registered face image 601 and the registered face information of the registered face image 602, and as shown in FIG. It is determined that the first person ID 422) is a registered person of ID07 and a registered person of ID05, which is neither a registered person of ID01 nor a registered person of ID04.

  As shown in FIG. 5, the similarity 0.93 at time t5 is the highest among the similarities calculated at times t1 to t10, which is equal to or greater than the second threshold (0.85). Therefore, the detection degree calculation means 527 obtains the first hit count and the first hit rate for the registered person with ID01 specified as the authentication candidate person at time t5. In this example, the number of first hits is distributed among the registered people of ID01 and the registered people of ID04. As a result, the number of first hits of the registered person of ID01 is six times, and the first hit rate is 60%. . Therefore, when the authentication threshold is set to 70%, the first-rank hit rate does not exceed the authentication threshold, and it is determined that the person who passes through the shooting area is not a registered person with ID01. In this case, if the authentication threshold is lowered, it is possible to determine that the person who passes through the imaging region is the registered person of ID01. However, in face image authentication, a so-called “similarity of others” may occur depending on the orientation of the face, lighting conditions, etc., and a phenomenon in which the similarity of others is higher than the similarity of the principal may occur accidentally. Therefore, there is a high possibility that a registered person other than the person will be mistakenly accepted, so it is not preferable to simply lower the authentication threshold.

  Therefore, the registered person of ID04 whose face is similar to the registered person of ID01 is associated with the registered person of ID01 as a complementary person, and the first hit for the registered person of ID04 is added to the number of first hits for the registered person of ID01. The value obtained by adding the number of times is again set as the number of first hits for the registered person of ID01. Thereby, it is suppressed that the number of first hits is distributed among registered persons whose faces are similar to each other. The number of first hits of a registered person with ID01 is 8, and the first hit rate is 80%. Since the first hit rate exceeds the authentication threshold, a person who passes through the shooting area is a registered person with ID01. Can be determined.

  The operation of the face image authentication apparatus 10 according to the present invention will be described below with reference to the flowchart shown in FIG. Note that the operation flow described below operates on the microprocessor constituting the image processing unit 500. Prior to the processing described below, the collation table 410 shown in FIG. 4 is initialized to be blank. The operation described below is started when the face image authentication apparatus 10 is turned on or when the authentication mode is set.

  First, the face image authentication device 10 acquires an input image obtained by capturing an imaged region by the image capturing unit 100, and sends the input image to the collating unit 520 of the image processing unit 500 (step S701). Next, the face detection unit 522 of the collation unit 520 extracts a face area in which a person's face is shown from the acquired input image, and cuts out the face area from the input image to create a face area image (step). S702). Next, the face detection unit 522 determines whether one or more face areas have been extracted (step S703). If no face area is extracted, the process proceeds to step S701, and the processes in steps S701 to S703 are repeated for the input images sequentially acquired until the face area is extracted. On the other hand, when one or more face areas are extracted, the face tracking unit 523 tracks all the extracted face areas and updates the collation table 410 (step S704).

  Next, the face matching unit 524 determines whether there is any series data in which the tracking flag 413 is ON and the authentication flag 416 is OFF in the matching table 410 (step S705). If tracking is in progress and there is no unauthenticated tracking target, the process proceeds to step S701. On the other hand, when there is a tracking target that is being tracked and that has not been authenticated, the face matching unit 524 executes an authentication process for each piece of series data of the tracking target (step S706). Details of the authentication process will be described later.

  Next, the face matching unit 524 determines whether there is any series data for which the authentication process has not yet been executed among the series data for which the tracking flag 413 is ON and the authentication flag 416 is OFF. (Step S707). If there is a tracking target that has not yet been subjected to the authentication process in the extracted face area, the face collating unit 524 proceeds to step S706 and executes the authentication process for the series data. On the other hand, when the authentication process is completed for all the tracking targets, the process proceeds to step S701, and the face image authentication apparatus 10 acquires a new input image.

  FIG. 8 is a flowchart showing the operation of the authentication process executed by the face matching unit 524. The authentication process shown in FIG. 8 is executed in step S710 of FIG.

  First, the similarity calculation unit 525 extracts face information from the face area image of the current frame corresponding to the series data, and calculates the similarity with all registered face information stored in the storage unit 510 ( Step S801). The authentication candidate person specifying unit 526 determines whether or not the highest similarity among the calculated similarities is greater than or equal to the first threshold (step S802), and the highest similarity is less than the first threshold. In this case, no particular process is performed, the authentication process is terminated, and the process proceeds to step S707. On the other hand, if the highest similarity is equal to or higher than the first threshold, the registered person whose highest similarity is calculated is identified as an authentication candidate person, and the registered person ID of the registered person is the first person in the matching history. It memorize | stores in ID422, and memorize | stores the similarity in the similarity 423 (step S803).

  Next, the detection degree calculation unit 527 determines whether or not the number of collations is a predetermined number N or more, that is, whether or not the number of frames of the collation history 415 accumulated in the series data is N or more (step S804). ). When the number of verifications is less than the predetermined number N, the detection degree calculation unit 527 ends the authentication process without performing any particular process. On the other hand, when the number of collations is equal to or more than the predetermined number N, the highest value among all the similarities calculated for the face area images extracted from the latest N input images, that is, the latest N pieces of collation history 415. The highest value among the similarities 423 is specified (step S805).

Next, the detection degree calculation unit 527 determines whether or not the highest value in the specified similarity 423 is greater than or equal to the second threshold (step S806). When the identified maximum value is less than the second threshold value, the detection degree calculation unit 527 ends the authentication process without performing any particular process. On the other hand, if the specified maximum value is equal to or greater than the second threshold, the number of registered person IDs stored in the first person ID 422 corresponding to the highest value is recorded in the latest N first person IDs 422. Is calculated as the number of first hits for the registered person corresponding to the registered person ID (step S807).
Next, the detection degree calculation unit 527 determines whether or not a complementary person ID is set for the registered person in the registration table (step S808). When the complementary person ID is not set, the detection degree calculation unit 527 does not perform any particular process and moves the process to step S810. On the other hand, when the complementary person ID is set, the number of the complementary person ID recorded in the latest N first person IDs 422 is calculated as the number of first hits for the complementary person corresponding to the complementary person ID. To do. Then, a value obtained by adding the number of first hits for the complementary person to the number of first hits for the registered person is again set as the number of complementary first place hits for the registered person (step S809).
Next, the detection degree calculation means 527 calculates the first-rank hit rate in the N frames of the determination period from the calculated first-hit number of times (step S810).

  Next, the authentication determination unit 528 determines whether or not the first-rank hit rate is greater than or equal to the authentication threshold (step S811). If the first hit rate is less than the authentication threshold, the authentication determination unit 528 ends the authentication process without performing any particular process. On the other hand, if the first hit rate is equal to or greater than the authentication threshold, the authentication determination unit 528 outputs a signal indicating authentication success to the output unit 200, sets the authentication flag 416 to ON (step S812), and ends the authentication process. To do.

  When a signal indicating successful authentication is output to the output unit 200 in step S812, the face collating unit 524 unlocks the electric lock via the output unit 200. Thus, a person who has succeeded in authentication can enter the room. It is assumed that the electric lock is automatically locked as soon as the person's entry is confirmed or after a predetermined time has elapsed.

  As described above, the face image authentication apparatus to which the present invention is applied stores registered face information of a registered person in association with a complementary person whose face is similar to that of the registered person. Then, based on the number obtained by adding the number of first hits of the complementary person associated with the registered person to the number of first hits of the registered person identified as an authentication candidate for the most recent face area images, It is determined whether or not the face shown in the face area image is the face of the registered person. As a result, the face image authentication apparatus suppresses the dispersal of registered persons whose similarity is first, even if persons with similar faces such as twins are registered. Can be prevented.

  The present invention is not limited to the above-described embodiment, and many changes and modifications can be made. For example, the face matching means determines whether or not the image quality of the face area image is suitable for the matching process, that is, whether the image quality is suitable for the calculation of the first-rank hit rate, and is suitable for the matching process. If it is determined that the image does not exist, the image may be excluded from the target of the collation process. For example, when an autofocus type lens is used as the lens of the imaging unit, the face matching unit extracts the frequency component of the face region image, and the face region image whose frequency component is a predetermined value or less is suitable for the matching process. It is determined that the face area image is not present, and the image is excluded from the target of the matching process. As a result, an out-of-focus image can be excluded in advance from the target of collation processing, and the authentication performance of the face image authentication device can be improved. It should be noted that the frames excluded from the verification processing are not added to the parameters in the calculation of the first hit rate.

  Further, when the setting unit receives an instruction for setting the association between the registered person and the complementary person from the operation unit, the setting unit calculates the similarity between the registered face information about the registered person and the registered face information about the complementary person, The registered face information of the registered person may be associated with the registered face information of the complementary person only when the similarity is equal to or more than a predetermined threshold value continuously one or more times. The predetermined threshold is set to a lower limit value at which it can be considered that the face of the registered person and the face of the complementary person cannot be distinguished. The predetermined threshold is preferably set to the same value as the second threshold. For example, when the similarity is calculated in the range of 0 to 1, it may be set to 0.80 to 0.85. it can. Thereby, it is possible to prevent the complementary person from being erroneously set by the administrator. Furthermore, it is possible to prevent a person who can be sufficiently distinguished from a registered person from being set as a complementary person.

  In addition, when the setting unit receives an instruction to register the registered face image of the registered person from the operation unit, the setting unit calculates the similarity between the registered face information regarding the registered face image and all the registered face information already registered. It may be calculated. Then, when there is registered registered face information whose similarity is equal to or higher than a predetermined threshold, the setting unit displays a screen for confirming whether to set as a complementary person on the display unit. Thereby, even when the administrator does not know that the newly registered person has twins or the like, it is possible to prevent the setting of the complementary person from being omitted. Alternatively, the setting means may automatically set a complementary person when there is registered registered face information whose similarity is equal to or greater than a predetermined threshold. This eliminates the need for the administrator to set a complementary person and improves convenience.

  In addition, in setting the complementary person, it is only necessary that the registered persons are associated with each other, and the complementary group ID may be used instead of the registered person ID of the complementary person. Moreover, you may associate a complementary person among three or more registered persons.

In addition, the authentication determination means does not perform authentication determination based on whether or not the first hit rate is equal to or higher than the authentication threshold, but does not perform authentication determination based on whether or not the number of first hits within a predetermined period is equal to or higher than the authentication threshold. A determination may be made.
Alternatively, the authentication determination means determines whether or not the number of consecutive successes having the highest degree of similarity is equal to or greater than an authentication threshold, that is, the similarity for a specific registered person or a complementary person of the registered person is continuously equal to or greater than the authentication threshold. The authentication determination may be made depending on whether or not it is ranked first. In that case, the matching history referred to by the face matching means may be the number of authentication thresholds, and the face matching means may store the matching history for the number of authentication thresholds in the matching table.

  Further, the detection degree calculation means does not calculate the first hit number and the first hit rate only for the registered person for which the highest value of the similarities for the most recent N face area images is calculated, but is similar. A configuration may be adopted in which the number of first hits and the first hit rate are obtained for a plurality of registered persons whose degrees are equal to or greater than the second threshold, and authentication is performed in descending order of similarity.

  Further, the authentication determination means does not determine that the passerby is the registered person when the first-rank hit rate of the registered person having the highest similarity is equal to or higher than the authentication threshold, but simply has the highest first-order hit ratio. If the first-person hit rate of the registered person is equal to or higher than the authentication threshold, the passerby may be determined as the registered person. Further, when the similarity calculated for the registered person with the highest first-rank hit rate is equal to or higher than the second threshold, the passerby may be determined as the registered person. In these cases, among the registered persons set as complementary persons, the registered person with the highest similarity is not determined as a passerby, but the first hit rate or first hit number is the highest. The registered person who becomes the value is finally determined to be a passerby.

  In addition, the authentication determination unit is configured to determine whether or not the authentication candidate person who has obtained the highest value of similarity is a registered person based on the first-rank hit rate, but performs authentication determination based on other factors. May be. For example, a difference obtained by subtracting an average value of similarities calculated for all other registered persons from an average value of similarities calculated for the registered person whose highest similarity value has been calculated is calculated as a relative score. The highest registered person is the authentication candidate person of each frame, or when the highest relative score of the authentication candidate person of each frame is equal to or greater than the threshold, the first hit rate for the registered person is determined. As the relative score, the deviation value of the highest value of the similarity with respect to the similarity calculated for all registered persons may be calculated.

DESCRIPTION OF SYMBOLS 10 Face image authentication apparatus 100 Imaging part 200 Output part 300 Display part 400 Operation part 500 Image processing part 510 Memory | storage part 520 Collation part 521 Setting means 522 Face detection means 523 Face tracking means 524 Face collation means 525 Similarity calculation means 526 Authentication candidate Person specifying means 527 Detection degree calculating means 528 Authentication determining means

Claims (4)

For each of a plurality of registered persons registered in advance, a storage unit that stores a correspondence relationship between registered face information representing the facial features of the registered person and a complementary person that is another registered person related to the registered person; ,
An image acquisition unit for sequentially acquiring input images obtained by imaging a predetermined imaging region;
Face detection means for extracting a face area including a human face from the input image;
The registered face similar to the input face information among the plurality of registered persons is collated with each of the registered face information of the plurality of registered persons for the input face information representing the facial features included in the extracted face area. Similarity determination means for identifying a registered person of information as an authentication candidate person;
In a plurality of input images having different shooting times, either the registered person or the complementary person associated with the registered person is reflected in the plurality of input images based on the number of candidates specified as the authentication candidate person. Identity determination means for determining that the face is the face of the registered person,
A face image authentication apparatus characterized by comprising:   The identity determination unit determines whether the number of candidates for the same registered person in the plurality of input images or the ratio of the number of candidates to the number of frames of the plurality of input images satisfies the determination criterion. The face image authentication device according to claim 1, wherein the face image authentication device is determined to be present.   The identity determination unit performs a determination based on the number of candidates for a registered person who has obtained the highest similarity as a result of matching the plurality of input images among the plurality of registered persons. The face image authentication device described. An operation unit;
Setting means for associating the complementary person with the registered person according to an operation from the operation unit;
The setting means associates one registered person with the other registered person as a complementary person when the similarity of the registered face information of the plurality of registered persons instructed by the operation unit is equal to or greater than a predetermined threshold. The face image authentication device according to claim 1.

Images