JP5766096B2 - Face image authentication device - Google Patents

Description

  The present invention relates to a face image authentication apparatus, and more particularly to a face image authentication apparatus that detects a user based on a face image registered in advance.

  Conventionally, in order to authenticate a person walking in a monitoring area, an image of an area including a person's face is extracted as a face area image from the monitoring image, and the extracted face area image and a plurality of registered face images registered in advance are extracted. A face image authentication device (a so-called walk-through type face image authentication device) is used that calculates a collation score, that is, a similarity in time series order.

  The walk-through type face image authentication device extracts face area images from images obtained by photographing a person walking at a plurality of points or time points, and calculates the similarity between these face area images and registered face images. And the person of the registration face image whose similarity is more than a certain level is specified for each point or time point. Further, the maximum value or the average value of the similarity at each point or each time point is calculated for each specified person. Finally, the person is authenticated based on the maximum value or the average value calculated in this way (Patent Document 1).

JP 2004-118359 A

  When considering the similarity to the registered person in time series for the face of the same person extracted across multiple monitoring images, that is, sequentially obtaining the similarity for the most recent face area images for the same person When doing so, it is desirable that both similarities be high. In this case, an accurate authentication result can be obtained even if a maximum value or an average value of a plurality of similarities is used.

  However, in a situation where the faces of passing people are sequentially photographed, variations in facial expressions, face orientations, etc. that occur naturally during walking can occur. When such a change occurs, even if the person himself / herself is similar, the degree of similarity is low, but a so-called “similarity of others” may occur in which the person in the face area image is determined to be a person other than the person himself / herself. Therefore, it is impossible to ignore the phenomenon that the similarity of others is higher than the similarity of the person. Since this leads to acceptance of others, it is inappropriate to simply make a determination based on a plurality of maximum similarity values. Further, when the determination is made based on the average value of the plurality of similarities, the average value of the plurality of similarities may not exceed the authentication threshold, and the person may not be determined correctly.

  In some cases, the highest similarity is lower than the authentication threshold due to the presence or absence of glasses, changes in facial expressions, and the like. For example, when a person registered with the naked eye walks with glasses, when calculating the similarity between the registered face image and the face area image, the difference between the images is small in the part other than the glasses. The degree becomes higher. However, since the texture of the spectacle portion differs between the registered face image and the face area image, the difference between the registered face image and the face area image in the spectacle portion increases. As a result, the similarity is low for the entire face, and the similarity is lower than the authentication threshold even if the person's similarity has the maximum value. In this case, since the degree of similarity does not exceed the authentication threshold, it is difficult to determine the person even if the maximum value or the average value of a plurality of degrees of similarity is used.

  Although it is conceivable to lower the authentication threshold so that the person can be determined, the registered person whose similarity is the maximum due to variations in facial expressions, face orientations, etc. that occur naturally during walking is other than that person In such a situation, it is possible that a registered person other than the principal is erroneously authenticated.

  Therefore, when authenticating a person walking in the monitoring area using the face image authentication device, it is necessary to ensure the acceptance of the person even in a situation where the person authentication is difficult due to a change in facial expression, face orientation, and the like.

  An object of the present invention is to provide a face image authentication apparatus that can ensure the identity acceptance even in a situation where the identity authentication is difficult due to a change in facial expression, face orientation, and the like.

  The present invention for solving such a problem includes an imaging unit that sequentially acquires a monitoring image obtained by photographing a person in a monitoring area, a storage unit that stores in advance face images of a plurality of registered persons as registered face images, and a monitoring image. A face image authentication apparatus comprising a collation unit that performs person authentication as to whether or not a person in a monitoring area is a registered person, wherein the collation unit extracts an image of an area including a person's face from the monitoring image. Face detection means for extracting as a face area image, face tracking means for tracking the face area image with a predetermined number of monitoring images acquired sequentially, and similarity calculation means for calculating the similarity between the face area image and the registered face image Authentication candidate person specifying means for specifying an authentication candidate person from a plurality of registered persons based on the similarity, and for the authentication candidate person, the face area image specified as the authentication candidate person for a predetermined number of monitoring images. Detect percentage of registered people And an authentication threshold value determining means for determining, for each authentication candidate person, an authentication threshold value for authenticating whether or not the face area image belongs to an authentication candidate person. And an authentication determination unit that determines that an authentication candidate person whose similarity exceeds an authentication threshold is a person in the monitoring area.

  In the face image authenticating device according to the present invention, it is preferable that the authentication threshold value determining means lowers the authentication threshold value as the registered person detection rate increases.

  In the face image authentication apparatus according to the present invention, it is preferable that the collation unit performs person authentication using a plurality of latest monitoring images determined in advance.

  In the face image authentication device according to the present invention, the collation unit further includes image quality determination means for determining whether or not the face area image has an image quality suitable for calculating the registered person detection rate, and the appropriate image quality It is preferable to perform person authentication using the face area image determined to be.

  The face image authentication apparatus according to the present invention has an effect that it is possible to ensure the user's acceptance by adaptively determining the authentication threshold value even in a situation where the user's authentication is difficult due to changes in facial expressions, face orientations, and the like.

It is a schematic block diagram of the face image authentication apparatus to which this invention is applied. It is a schematic diagram showing the installation example of an imaging part in case a face image authentication apparatus is installed in the entrance of an office building. It is a schematic diagram of the collation table memorize | stored in a memory | storage part. It is a schematic diagram of the frame collation result which comprises a part of collation table shown in FIG. It is a figure which shows the example of the face area image for demonstrating the case where the image quality criteria based on illumination conditions are used as an image quality criteria. It is a schematic diagram explaining the method of adjustment of an authentication threshold value. It is a schematic diagram of the similarity obtained with the face image authentication apparatus. It is a schematic diagram which shows that a user changes direction of a face and is image | photographed. It is a flowchart which shows operation | movement of the face image authentication apparatus by this invention.

Hereinafter, a face image authentication apparatus according to an embodiment of the present invention will be described with reference to the drawings.
In the authentication in the face image authentication device, there are situations where it is difficult to authenticate the person due to changes in facial expressions, face orientations, and the like. Therefore, the face image authentication apparatus to which the present invention is applied calculates a registered person detection rate, which is the ratio of the number of times that the degree of similarity to the most recently tracked plurality of face area images is the highest, and is included in the face area image. An authentication threshold, which is a reference for authenticating whether or not the face is the face of an authentication candidate person, is determined according to the registered person detection rate. Thus, by adaptively determining the authentication threshold, the face image authentication device attempts to ensure the user's acceptance even in situations where it is difficult to perform the user authentication due to changes in facial expressions, face orientations, and the like.

  FIG. 1 is a diagram showing a schematic configuration of a face image authentication apparatus 10 to which the present invention is applied. As illustrated in FIG. 1, the face image authentication device 10 includes an imaging unit 100, an output unit 200, and an image processing unit 300. Hereinafter, each part of the face image authentication device 10 will be described in detail.

The imaging unit 100 is a monitoring camera that captures a predetermined monitoring area, and is, for example, a photoelectric conversion element (for example, a CCD sensor, a C-MOS, or the like) that is arranged two-dimensionally and outputs an electrical signal corresponding to the amount of received light. And an imaging optical system for forming an image of the monitoring region on the photoelectric conversion element.
The imaging unit 100 is installed so that the face of a person who passes through the monitoring area (a person in the monitoring area to be collated by the face image authentication apparatus 10 is referred to as a target person) can be sequentially photographed. Then, the imaging unit 100 acquires a monitoring image obtained by capturing the monitoring area at predetermined time intervals (for example, 200 msec). The imaging unit 100 is connected to the image processing unit 300 and passes the acquired monitoring image to the image processing unit 300.

  FIG. 2 schematically shows an installation example of the imaging unit 100 when the face image authentication device 10 is installed at the entrance of an office building. As shown in FIG. 2, for example, when the face image authentication device 10 is installed at the entrance 253 of an office building, the imaging unit 100 includes a wall on which the entrance 253 is installed so as to include a passage leading to the entrance 253 in the monitoring area. It is attached to the top or ceiling of the camera with the shooting direction slightly downward and toward the passage. As a result, the imaging unit 100 can capture the target person heading toward the entrance 253 (heading in the traveling direction 254) at a predetermined time interval. FIG. 2 shows a state in which the imaging unit 100 sequentially captures the same target person 250, 251, 252 heading toward the entrance 253 at times t, t + 1, t + 2.

  The monitoring image can be a grayscale or color multi-tone image. In the present embodiment, the monitoring image is a color image having 1280 pixels by 960 pixels and having 8-bit luminance resolution for each of the RGB colors. However, a monitor image having a resolution and gradation other than this embodiment may be used.

  The output unit 200 includes a communication interface connected to, for example, an electric lock or an external device that controls the electric lock, and a control circuit thereof. When the output unit 200 receives a signal indicating successful authentication of the target person from the image processing unit 300, the output unit 200 outputs a signal requesting, for example, unlocking of the electric lock to the connected device.

  The image processing unit 300 is configured by, for example, a so-called computer. The image processing unit 300 authenticates the target person based on the monitoring image received from the imaging unit 100, and detects a target person that is difficult to be authenticated due to a change in facial expression, face orientation, or the like. For this purpose, the image processing unit 300 includes a storage unit 310 and a collation unit 320. Furthermore, the collation unit 320 includes a face detection unit 321, a face tracking unit 322, and a face collation unit 323.

  The storage unit 310 includes a semiconductor memory such as a ROM and a RAM, or a magnetic recording medium and its access device or an optical recording medium and its access device. The storage unit 310 stores in advance a computer program and various parameters for controlling the face image authentication apparatus 10. The storage unit 310 stores a registered face image used for verification in association with a registered person ID of a corresponding registered person (hereinafter referred to as a registered person). The storage unit 310 also stores a collation table for managing information related to the target person generated by the image processing. Details of this collation table will be described later.

  Each unit of the verification unit 320 is a functional module implemented by a microprocessor, a memory, a peripheral circuit thereof, and software operating on the microprocessor. Alternatively, these means may be integrated by firmware. Moreover, you may comprise some or all of these means with an independent electronic circuit, firmware, a microprocessor, etc. Hereinafter, each means of the collation part 320 is demonstrated in detail.

Each time the face detection unit 321 receives a monitoring image from the imaging unit 100, the face detection unit 321 detects a face area that is an area in which the face of the target person is reflected from the received monitoring image, and extracts a face area image.
In order to detect a face area, the face detection unit 321 uses, for example, an inter-frame difference process or a background difference process, and changes with time in luminance values in a plurality of monitoring images acquired by the imaging unit 100 are performed. Extract regions. Then, the face detection unit 321 extracts, from the extracted change areas, a change area that seems to be a person from a feature amount such as the size of the change area as a person area. Then, the face detection unit 321 performs edge pixel extraction on the extracted person region using a Sobel filter or the like so that the inclination direction of the luminance change can be known. Then, the face detection unit 321 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels, and uses the area surrounded by the edge distribution as a face area. Extract. In this case, the face detection unit 321 can detect an elliptical edge distribution using, for example, a generalized Hough transform.

Alternatively, the face detection unit 321 may detect a face area using an Adaboost classifier. This method is described in a paper by P. Viola and M. Jones “Rapid Object Detection Using a Boosted Cascade of Simple Features” (Proc. The IEEE International Conference on Computer Vision and Pattern Recognition, vol.1, pp.511-518, 2001). ) Can be referred to.
Then, the face detection unit 321 creates a face region image by cutting out the extracted face region from the monitoring image, and outputs the face region image and the coordinate information of the face region in the monitoring image to the face tracking unit 322.

The face tracking unit 322 performs a tracking process using a known tracking technique on a face area extracted from the face detection unit 321 over a plurality of monitoring images continuously acquired at a predetermined time interval. The face area image is tracked by associating the face areas in which the face is reflected.
For example, the face tracking unit 322 includes the position of the center of gravity of the face area extracted from the latest monitoring image (hereinafter referred to as the face area of the current frame) and the face area (hereinafter referred to as the previous area) extracted from the monitoring image of the previous frame. The distance of the center of gravity position (referred to as a face area of the frame) is obtained, and when the distance is equal to or less than a predetermined threshold, the face area is associated with the same person. The face area in the monitoring image before and after the movement when the target person moves a certain distance when moving away from the imaging unit 100 and when the target person moves the same distance when moving closer to the imaging unit 100. The position difference is different. Therefore, for example, by setting a predetermined threshold as the size of the face area, it is appropriately determined whether the face area of the current frame and the face area of the previous frame are from the same person regardless of the position of the target person in the monitoring area. Can be evaluated. When a plurality of face areas are extracted, it is checked whether or not face areas with the closest distance between the centroid positions correspond to each other.

  Alternatively, the face tracking unit 322 may perform face area tracking processing using a method such as an optical flow or a particle filter.

When the face tracking unit 322 associates the face area, the face tracking unit 322 updates the collation table stored in the storage unit 310. FIG. 3 shows an example of the collation table. As shown in FIG. 3, the collation table 410 manages the series data for each target person. The series data is a set of data represented in each row of the collation table 410, and the face tracking unit 322 updates the trial number 411, the series ID 412, the update time 413, the tracking flag 414, and the tracking position information 415 among the series data. To do.
In the verification table 410 shown in FIG. 3, the trial number 411 is an identification number for distinguishing the series data of the target person being tracked from other series data, and while the target person exists in the monitoring area. That is, the same identification number continues to be assigned while being tracked by the face tracking means 322. The series ID 412 is an identification number for identifying the target person being tracked from other target persons, and corresponds to the trial number 311 on a one-to-one basis. The series ID 412 is assigned so as to be uniquely determined. For example, a random number may be generated based on the time when the face tracking unit 322 starts tracking. Alternatively, the sequence ID 412 and the trial number 411 correspond one-to-one, and either may be omitted. The update time 413 represents the time when the face detection unit 321 finally extracted the face area image for the target person and the face tracking unit 322 finished the tracking process.
The tracking flag 414 is a flag indicating whether or not the tracking of the target person is continued. The tracking flag 414 is turned on when the face tracking unit 322 starts tracking the target person, and turned off when the tracking is finished. That is, when the tracking flag 414 is ON, the target person exists in the monitoring area, and when the tracking flag 414 is OFF, the target person has already been authenticated and has entered the room or left without being authenticated. It will be that.
The tracking position information 415 is time information indicating the coordinate information of the face area in the monitoring image from which the face area image is cut out and the time when the tracking process is performed.

  If the face area of the current frame of interest cannot be associated with the face area of the previous frame, the face tracking unit 322 shows the target person newly entering the monitoring area in the face area of the current frame. As a result, series data for the target person is newly added to the collation table 410, and initialization processing is performed. That is, the face tracking unit 322 assigns a new trial number 411 and a sequence ID 412 to the series data, records the current time as the update time 413, and sets the tracking flag 414 to ON. The face tracking unit 322 records the coordinate information and the current time of the face area of the current frame as the tracking position information 415. In addition, the face tracking unit 322 sets the authentication flag 417 to OFF, and outputs the newly added series data and the face area image of the current frame associated with the series data to the face matching unit 323.

  On the other hand, when the face tracking unit 322 can associate the face area of the current frame of interest with the face area of the previous frame, the series data for the target person has already been created in the matching table 410. Therefore, the face tracking unit 322 performs update processing for adding the coordinate information and the current time of the face area of the current frame to the tracking position information 415 of the corresponding series data in the matching table 410. Then, the face tracking unit 322 outputs the series data and the face area image of the current frame associated with the series data to the face matching unit 323.

  Further, when there is a face area of the previous frame that is not associated with all face areas of the current frame, the face tracking unit 322 turns off the tracking flag 414 of the corresponding series data in the matching table 410. Then, the tracking process for the target person is terminated.

  The face collating unit 323 includes an image quality determining unit 324, a similarity calculating unit 325, an authentication candidate person specifying unit 326, a detection rate calculating unit 327, an authentication threshold determining unit 328, and an authentication determining unit 329, and outputs from the face tracking unit 322. Among the obtained series data, the face area image output from the face tracking unit 322 is matched with the series data for which the authentication flag 417 of the matching table 410 shown in FIG. The verification history 416 and the authentication flag 417 in the table 410 are updated.

The matching history 416 holds a frame matching result 420 corresponding to the tracking position information 415 of each frame. FIG. 4 shows an example of the frame matching result 420.
The acquisition time 421 is the time when the face area image of the current frame is acquired. The image quality flag 422 is a flag that indicates whether or not the image quality of the face area image is suitable for collation, that is, whether or not the face area image has an image quality appropriate for calculating similarity described later. When the means 324 determines that the image quality of the face area image is suitable for collation, it is set to OK by the image quality determination means 324, and when the image quality determination means 324 determines that the image quality of the face area image is not suitable for collation. It is set to NG by the image quality judging means 324.

  The frame first person 423 has the highest similarity (first rank) among the similarities (collation scores) between the face area image tracked by the face tracking unit 322 and each of the plurality of registered face images stored in the storage unit 310. This is the identification ID of the registered person. The similarity 424 is a matching score that identifies the first person person 423 in the frame.

  Returning to FIG. 3, the authentication flag 417 is a flag indicating whether or not the authentication of the target person has succeeded. In other words, when the authentication determination unit 329 determines that the target person is one of the registered persons, the authentication determination unit 329 is set to ON, and the authentication result is obtained by the authentication determination unit 329 as if the target person has just started tracking. If it is determined that the authentication determination unit 329 determines that the target person is not a registered person, the setting is set to OFF.

As described above, the image quality determination unit 324 determines whether or not the image quality of the face area image is suitable for the collation processing, that is, the face area image is suitable for calculating the first hit rate as a registered person detection rate described later. It is determined whether or not the image quality is good, and the determination result is stored in the image quality flag 422.
In general, while the imaging unit 100 captures a walking target person, for example, a situation in which the subject is out of focus, a situation in which the face becomes dark due to lighting, shielding, or the like may occur. The authentication performance of the face image authenticating apparatus 10 can be improved by excluding, in advance, images that are not suitable for the matching process acquired in the event from the target of the matching process.

  There are a plurality of image quality determination criteria for determining whether or not the image quality of the face area of the verification data is suitable for the verification processing. For example, when an autofocus type lens is used as the lens of the imaging unit 100 and an image quality determination standard that excludes an image that cannot be adjusted with focus adjustment is used, the image quality determination unit 324 uses the frequency component of the face area image. , The face area image whose frequency component is equal to or less than a predetermined value is determined to be a face area image that is not suitable for the collation processing, and the image quality flag 422 is set to NG.

  When the image quality criterion based on the illumination condition is used as another image quality criterion for determining whether the image quality of the face area of the verification data is suitable for the verification process, the brightness and darkness generated in the face area image The state is shown in FIG.

  FIG. 5A shows a face area image with appropriate brightness. In the face area image shown in FIG. 5A, there is no shadow that hinders the collation process. In this case, the image quality determination unit 324 sets the image quality flag 422 to OK.

  FIG. 5B shows a dark face area image as a whole. In the face area image shown in FIG. 5B, face texture information that is important information in the matching process tends to be lost. In this case, the image quality determination unit 324 obtains the average luminance of the face area image and calculates the average. If the luminance is below a predetermined value, the image quality flag 422 is set to NG because the face area image is too dark. The image quality determination unit 324 also sets the image quality flag 422 to NG even when it is determined that the face area image is too bright and the facial texture information is missing. Therefore, the image quality determination unit 324 sets the image quality flag 422 to OK when the average luminance is within a predetermined range.

  In order to determine whether or not the average luminance is in a predetermined range, the image quality determination unit 324, for example, has a pixel whose luminance value is 0 (saturated black or blacked out) and a luminance value of 255 in the face area image. It is determined whether or not there is at least one of the pixels (white saturation or whiteout). If none of these pixels exists, it is determined that the average luminance is within a predetermined range. When the face area image is a color image, it is determined whether or not each of the RGB face area images has at least one of a pixel having a luminance value of 0 and a pixel having a luminance value of 255.

  When the face area image is too bright or too dark, the luminance distribution of the face portion area of the face area image tends not only to concentrate on the bright side or dark side but also to reduce the luminance gradation. Therefore, in order to determine whether or not the average luminance is within a predetermined range, the image quality determination means 324 determines whether or not the luminance gradation of the face portion area of the face area image has a resolution of 4 bits or more. If the resolution is 4 bits or more, it may be determined that the average luminance is within a predetermined range. When the face area image is a color image, it is determined whether the luminance gradation of the face portion area of the face area image has a resolution of 4 bits or more in each of the RGB face area images.

  FIG. 5C shows a face area image in which light hits from the left side of the target person, the left half face is bright, but the right half face is dark. In this case, features that should not be originally can be extracted at the boundary between light and dark. Therefore, the image quality determination unit 324 has a brightness that exceeds the first value in a region that is greater than or equal to a predetermined area, or is less than a second value that is lower than the predetermined first value in a region that is greater than or equal to the predetermined area. In this case, the image quality flag 422 is set to NG. A face area image (frame) in which the image quality flag 422 is set to NG is excluded from the processing target of the authentication threshold value determination unit 328 as described later.

The similarity calculation unit 325 calculates the similarity between the face area image tracked by the face tracking unit 322 and each of a plurality of registered face images stored in the storage unit 310 in advance. The similarity calculation unit 325 calculates the similarity using a known face matching technique. For example, the similarity calculation unit 325 calculates and calculates the sum of squares of the difference between the luminance values of the pixels included in the face area image and the corresponding pixels of the registered face image while shifting the positions of the face area image and the registered face image. The reciprocal of the normalized value obtained by dividing the smallest sum of squares by the number of pixels included in the face area image is obtained as the similarity.
The authentication candidate person specifying unit 326 specifies the registered person who has the highest similarity, that is, the authentication candidate person, stores the information (identification ID of the registered person) in the frame first person 423, and the similarity is similar. Store at 424 degrees.

The detection rate calculation unit 327 has the highest similarity for each of the face area images extracted from the monitoring images for the most recent N frames (for example, 30 frames) retroactive from the current frame for each authentication candidate person. The first hit rate as the registered person detection rate is obtained from the number of recorded images, that is, the number of identification IDs recorded in the frame first person 423.
Therefore, the detection rate calculation unit 327 checks the first frame person 423 and the similarity 424 included in the latest N frames for the sequence data to which the same sequence ID 412 is assigned. Then, the number of authentication candidate persons having the identification ID recorded in the first frame person 423 having the highest value among the similarities 424 of N frames is counted, and the N frames are counted. The ratio is defined as the first hit rate.
In an ideal state, the similarity score is the highest for the same authentication candidate person in all the frame matching results 420, and therefore the first-rank hit rate is 100% for the authentication candidate person.
Similarly, the number of authentication candidate persons having other identification IDs of the first frame person 423 that did not become the highest value among the similarities 424 of N frames is also included in the N frames. May be counted, and the first hit rate may be obtained for each.

For example, as represented by the formula (1), the first-rank hit rate is the number of frames Nhit in which the authentication candidate person is ranked first in the frame comparison result 420 for the most recent N frames, and for the most recent N frames. It can be calculated using the number of frames I in which the image quality flag 422 is set to NG in the frame matching result 420 and the number of frames Ihit in which the image quality flag 422 is set to NG among the number of frames Nhit.

  The authentication threshold value determining unit 328 determines an authentication threshold value that is a reference for authenticating whether or not the face included in the face area image is the face of a registered person specified as an authentication candidate person according to the first-rank hit rate. To do. FIG. 6 shows a two-dimensional graph of the authentication threshold value table.

  As will be described later in detail, the authentication threshold value table is designed so that the authentication threshold value decreases as the first-order hit rate increases. The relationship between the first-rank hit rate and the authentication threshold is a straight line relationship as shown in FIG. 6A, a downwardly convex curve relationship as shown in FIG. 6B, and as shown in FIG. 6C. Furthermore, it is possible to use a relationship of a convex curve, a step-like relationship divided for each predetermined section as shown in FIG.

  The relationship between the first-rank hit rate and the authentication threshold shown in FIG. 6 may be given based on existing knowledge as a design item, but can also be obtained empirically based on a preliminary experiment result. That is, when the face image authentication device 10 is installed, a sample subject is recruited before the actual operation, and the subject actually walks in the direction 254 in front of the door 253 as shown in FIG. At that time, the obtained input face image is actually authenticated, and the relationship between the first hit rate and the similarity is obtained and used as sample data. It is desirable to prepare a large number of sample data in consideration of the operating conditions of the face image authentication device 10 such as a change in brightness due to sunshine fluctuations.

  For example, when determining the authentication threshold value for the first hit rate of 0.8, sample data having the first hit rate of 0.75 to 0.85 among the sample data is collected and allowed FAR (False Accept Rate : Adopting a similarity degree that results in an acceptance rate of others (for example, 0.1%) as an authentication threshold. Similarly, the authentication threshold for each first hit rate is determined while changing the first hit rate, and the coordinates of the determined first hit rate and the authentication threshold are connected by a smooth curve, and the first hit rate and the authentication threshold What is necessary is just to determine a relationship as a linear relationship as shown, for example to Fig.6 (a). At this time, a non-linear function of monotonously decreasing right shoulder may be applied.

  If the number of frames with the highest similarity is extremely small (for example, 3 frames compared to the last 30 frames), it is inappropriate to output the result with the person even if there are frames with high similarity. is there. Also, even if the number of frames with the highest similarity is large or the similarity is too low, it is inappropriate to output the result with the person. Therefore, when the number of frames with the highest similarity is extremely small, or when the similarity is too low, it is preferable to prohibit determination by the authentication determination unit 329 described below.

  The authentication determination unit 329 determines whether there are a plurality of frames with the highest similarity and any one of the similarities in the N frames nearest to the authentication candidate person exceeds the authentication threshold. When the number of frames having the highest similarity is plural and the similarity of any one of the authentication candidate persons exceeds the authentication threshold, the authentication determination unit 329 determines that the face included in the face area image is the authentication It is determined that the face is a registered person identified as a candidate person, and the target person who passes through the monitoring area is determined to be the registered person. In the matching table 410, the tracking flag 414 is set to OFF and the authentication flag 417 is set to ON, and a signal indicating successful authentication is output to the output unit 200.

  On the other hand, if none of the similarities of the authentication candidate person exceeds the authentication threshold even though the tracking is continued for a predetermined time (for example, 2 seconds), the authentication determination unit 329 checks the corresponding verification as an authentication failure. The authentication flag 417 in the table 410 is set to OFF. Also, immediately after the start of tracking, when the number of times that the degree of similarity is the highest is 1 or when the verification history 416 has not been acquired for N frames, the authentication determination unit 329 includes the corresponding verification table 410. The authentication flag 417 is set to OFF.

Here, the effect obtained by determining the authentication threshold according to the first hit rate is shown in FIG. 7 which is a schematic diagram of the similarity obtained by the face image authentication device, and the user changes the face orientation. This will be described with reference to FIG.
When considering the similarity to the registered person in time series for the face of the same person extracted across multiple monitoring images, that is, sequentially obtaining the similarity for the most recent face area images for the same person When doing so, it is desirable that both similarities be high. For example, in FIG. 7A, all the similarities are high, and a result that “the face of the face area image is the face of the person A” is obtained. In this case, the result that “the face of the face area image is the face of the person A” can be obtained accurately using any of the highest similarity, the average similarity, and the first-rank hit rate.

  However, in a situation where the faces of passing people are sequentially photographed, variations in facial expressions, face orientations, etc. that occur naturally during walking can occur. In FIG. 8, it is assumed that a registered face image 700 represents a face image facing the front. Reference numerals 701 to 705 denote face area images in each frame. As shown in FIG. 8, for the face area images 703 and 704, the illumination conditions are good and the face direction is also the front, so the conditions are the same as the registered face image 700, and the similarity between them is also high. . As a result, the face person (first person person frame 423) of each face area image is determined to be “person A”. On the other hand, in the face area images 701, 702, and 705, since the face direction is not front, the similarity is low even for the person himself, and the person of the face in the face area image 701 is determined to be “person B”. The face person in the face area image 702 is determined to be “person D”, and the face person in the face area image 705 is determined to be “person C”.

  When the similarity is displayed in a graph in the situation as shown in FIG. 8, the similarity exceeds the authentication threshold for the face area images 703 and 704, and the face images 701, 702, and 705 are displayed as shown in FIG. The similarity is lower than the authentication threshold. In this case, if the determination is made based on the highest degree of similarity as in the prior art, the person can be distinguished from the other person. However, as described above, in face image authentication, a phenomenon in which a so-called “other's similarity” occurs due to the orientation of the face, lighting conditions, and the like, and a phenomenon in which the similarity of the other person is higher than the similarity of the person cannot be ignored. Since this leads to acceptance of others, it is inappropriate to simply make a determination based on the highest similarity.

  Further, when the determination is performed based on the average value of the similarity, the average of the similarity may not exceed the authentication threshold value, and the identity may not be determined correctly. Further, even when the determination is made based on the first hit rate, the first hit rate may not exceed the authentication threshold, and the person may not be determined correctly.

  In addition, as shown in FIG. 7C, there is a situation in which the highest similarity is lower than the authentication threshold value due to the presence or absence of glasses, changes in facial expressions, and the like. For example, when calculating the similarity between a registered face image and a face area image when a person registered with the naked eye is wearing glasses, the difference between the images is small in a portion other than the glasses. Although the degree of similarity increases, the difference between the registered image and the face area image in the eyeglass part increases because the texture of the eyeglass part differs between the registered image and the face area image. Even if the person has the highest similarity, the highest similarity is lower than the authentication threshold. In this case, since the highest degree of similarity does not exceed the authentication threshold, it is difficult to determine the person himself using any of the highest degree of similarity, the average of the degree of similarity, and the first-rank hit rate.

  Although it is conceivable to lower the authentication threshold so that the person can be determined, the registered person who has the highest degree of similarity due to variations in facial expressions, face orientations, etc. that occur naturally during walking is not the person In a situation that can be determined, there is a possibility that a registered person having a high first-rank hit rate other than the person himself / herself is specified as an authentication candidate person.

Therefore, if the highest similarity is predicted to be high even if the person's first-rank hit rate is low due to changes in facial expressions, face orientation, etc. that occur naturally during walking, the authentication threshold value remains at the default high value. By doing so, only the person himself can be accepted.
On the other hand, if the highest similarity is expected to be low although the person's first-ranked hit rate is high due to the presence or absence of glasses, changes in facial expression, etc., the authentication threshold is lowered (for example, 40% of the default value) ) So that only the person can be accepted.
As described above, by adaptively determining the authentication threshold according to the first-rank hit rate, it is possible to ensure identity acceptance even in situations where identity authentication is difficult due to changes in facial expression, face orientation, etc. It is possible to cope with various fluctuations that may occur on a daily basis when the apparatus 10 is operated.

  The operation of the face image authentication apparatus 10 according to the present invention will be described below with reference to the flowchart shown in FIG. The flow of operations described below is controlled by a control unit (not shown) that operates on the microprocessor constituting the image processing unit 300 and controls the face image authentication apparatus 10 as a whole. Note that the collation table 410 is initialized to be blank before performing the processing described below. The operation described below is performed every time one monitoring image is acquired.

  First, the face image authentication apparatus 10 acquires a monitoring image obtained by capturing a monitoring area by the imaging unit 100, and sends the monitoring image to the matching unit 320 of the image processing unit 300 (step S401). Next, the face detection unit 321 of the collation unit 320 extracts a face area in which the face of the target person is shown from the acquired monitoring image, and cuts out the face area from the monitoring image to create a face area image ( Step S402). Next, the face detection unit 321 determines whether one or more face regions have been extracted (step S403). If no face area is extracted, the process proceeds to step S401, and the processes of steps S401 to S403 are repeated until the face area is extracted.

  On the other hand, when one or more face areas are extracted and a face area image is created in step S403, the face tracking unit 322 sets the tracking flag 414 in the matching table 410 to ON for all the extracted face areas. The association with the sequence ID 412 is implemented (step S404). Then, the face tracking unit 322 determines whether or not there is a group ID 412 in which the tracking flag 414 in the collation table 410 is ON, which is not associated with any of the face areas of the current frame (step S405). . If there is a sequence ID 412 in which the tracking flag 414 in the matching table 410 that is not associated with any of the face regions of the current frame is ON, the target person of the sequence ID 412 has entered the room with successful authentication, or It is thought that he was away from the monitoring area without being authenticated. Therefore, in that case, the face tracking unit 322 turns off the tracking flag 414 of the series ID 412 that is not associated with any of the face areas of the current frame in the matching table 410, and does not perform the tracking process for the face area thereafter. (Step S406).

  Then, the face tracking unit 322 determines whether there is a face area of the current frame that cannot be associated with the series ID 412 in which the tracking flag 414 in the matching table 410 is ON (step S407). If there is something that could not be associated with the series ID 412 for which the tracking flag 414 in the collation table 410 is ON, the face tracking unit 322 collates the series data for the target person in the face area that could not be associated. A new one is added to the table 410 (step S408). That is, the face tracking unit 322 assigns a new trial number 411 and a series ID 412 to the series data for the target person in the face area that could not be associated, records the current time as the update time 413, and sets the tracking flag 414. Set to ON. The face tracking unit 322 records the coordinate information and acquisition time of the face area that could not be associated as the tracking position information 415, and sets the authentication flag 417 to OFF.

  When the face area of the current frame can be associated with the sequence ID 412 in which the tracking flag 414 in the collation table 410 is ON, the face tracking unit 322 acquires coordinate information and acquisition of the face area that can be associated. Update processing is performed to add the time to the tracking position information 415 of the corresponding series data in the matching table 410. Then, the face tracking unit 322 outputs the updated series data and face area image to the face matching unit 322.

  Next, the face collating unit 323 determines whether there is series data in which the tracking flag 414 is ON and the authentication flag 417 is OFF (step S409). If the tracking flag 414 is ON and there is no series data in which the authentication flag 417 is OFF, the process proceeds to step S401. On the other hand, when there is series data for which the authentication flag 417 is OFF, the following processes of steps S410 to S415 are executed. Here, among the series data for which the tracking flag 414 is ON, the series data for which the authentication flag 417 is ON is not processed because the authentication has already been completed for the series data.

  The face matching unit 323 stores the frame matching result 420 in the matching history 416 when the series data is output from the face tracking unit 322. Each frame matching result 420 has a one-to-one correspondence with the tracking position information 415 in each frame, and the time information of the tracking position information 415 is stored in the acquisition time 421. The acquisition time 421 may be omitted as being the same as the time information of the tracking position information 415.

  The image quality determination means 324 determines whether the image quality of the face area of the current frame of the sequence data is suitable for the collation processing, and the image quality of the face area of the current frame of the sequence data is suitable for the collation processing. If there is, the image quality flag 422 is set to OK, and if the image quality of the face area of the current frame of the sequence data is not suitable for collation processing, the image quality flag 422 is set to NG (step S410).

  Next, the similarity calculation unit 325 calculates the similarity between all the registered images stored in the storage unit 310 for the face area of the current frame of the sequence data (step S411). Then, the authentication candidate person specifying unit 326 stores the information of the registered person having the highest similarity (identification ID of the registered person) in the frame first person 423, and stores the similarity in the similarity 424 (step 424). S412). When the image quality determination unit 324 determines that the image quality of the face area of the current frame of the sequence data is not suitable for the collation process, the similarity of the frame is determined by the authentication threshold determination unit 328. Since it is not used, the calculation of the similarity may be omitted.

  Next, the detection rate calculation means 327 calculates the first-rank hit rate of the authentication candidate person who has the highest similarity in the frame comparison result 420 for the most recent N frames using Equation (1). Then, the authentication threshold value determination unit 328 sets an authentication threshold value that is a reference for authenticating whether or not the face included in the face area image is the face of the registered person specified as the authentication candidate person according to the first-rank hit rate. (Step S413).

  Note that when the tracking by the face tracking unit 322 has just started, the number of frames accumulated in the matching history 416 is small, and the first hit rate greatly depends on the result of one frame, so that authentication processing is not performed. It is necessary to. For this reason, for example, it is determined in step S409 whether or not the number of frames necessary for the authentication process is accumulated in the verification history 416, and the number of frames required for the authentication process is not accumulated in the verification history 416. It is preferable that the authentication process is prohibited and the process proceeds to step S401. The number of frames required for the authentication process is set as appropriate according to the installation environment of the face image authentication apparatus 10, but is preferably within a range of 30 to 100% of the number of frames N.

  Next, the authentication determination unit 329 has a plurality of (for example, three) frames with the highest similarity of the authentication candidate person, and the similarity 424 of any one of the authentication candidate persons exceeds the authentication threshold. It is determined whether or not (step S414). When the number of frames in which the similarity of the authentication candidate person is highest is one or the similarity 424 of the authentication candidate person does not exceed the authentication threshold, the process proceeds to step S401. On the other hand, if the number of frames with the highest similarity of the authentication candidate person is plural and any of the similarity degrees 424 of the authentication candidate person exceeds the authentication threshold, the authentication determination unit 329 Is output to the output unit 200, the tracking flag 414 is set to OFF, and the authentication flag 417 is set to ON (step S415).

  When a signal indicating successful authentication is output to the output unit 200 in step S 415, the face matching unit 323 unlocks the electric lock via the output unit 200. As a result, the target person who has succeeded in authentication can enter the room. It is assumed that the electric lock is locked immediately after the entry of the target person can be confirmed or automatically after a predetermined time has elapsed.

  The present invention is not limited to the above-described embodiment, and many changes and modifications can be made. For example, in the above embodiment, a case where a registered person having the highest similarity in the latest N frames is determined as an authentication candidate person when specifying an authentication candidate person from a plurality of registered persons based on the similarity degree will be described. However, a registered person having a similarity other than the highest similarity in the latest N frames can be determined as an authentication candidate person in descending order of similarity. In this case, when it is determined that the face included in the face area image is not the face of the registered person having the highest similarity in the latest N frames, the registration having a similarity other than the highest similarity in the latest N frames. For the person, the determination of the authentication threshold according to the first-rank hit rate and the authentication determination are performed in the order of registered persons having a high similarity until the authentication is successful. Therefore, a plurality of registered persons are sequentially set as authentication candidate persons in descending order of similarity in the latest N frames.

  Further, when an authentication candidate person is specified from a plurality of registered persons based on the similarity, the registered person having the highest similarity in the latest face area image (current frame) can be determined as the authentication candidate person. In this case, for a registered person having the highest similarity in the current frame, an authentication threshold is determined and authentication is determined according to the first hit rate. Further, when it is determined that the face included in the face area image is not the face of the registered person having the highest similarity in the current frame, the registered persons other than the registered person having the highest similarity in the recently extracted face area image In addition, the determination of the authentication threshold according to the first-rank hit rate and the authentication determination can be performed in the order of registered persons having high similarity until the authentication is successful. Accordingly, a plurality of registered persons are sequentially set as authentication candidate persons in descending order of similarity in the current frame.

  Further, when determining at least one of a plurality of registered persons as an authentication candidate person based on the similarity, the registered person having the highest first-rank hit rate in the latest N frames may be determined as the authentication candidate person. it can. In this case, for the registered person with the highest first hit rate in the latest N frames, the authentication threshold is determined and the authentication is determined according to the first hit rate. Further, when it is determined that the face included in the face area image is not the face of the registered person having the highest first hit rate in the latest N frames, a face other than the registered person having the highest first hit rate in the latest N frames is determined. For the registered person, the determination of the authentication threshold according to the first place hit rate and the authentication determination can be performed in the order of the registered person having the highest first place hit rate until the authentication is successful. Therefore, a plurality of registered persons are sequentially set as authentication candidate persons in descending order of the first-order bit rate in the latest N frames.

Further, in the above embodiment, the case has been described in which it is determined whether or not the maximum value of the similarity in the latest N frame of the authentication candidate person has exceeded the authentication threshold. You may determine whether the average value or the median value exceeded the authentication threshold value.
Further, in the above embodiment, the first-rank hit rate is obtained based on the number of similarities with respect to the face area image of the latest N frames of the authenticated person, but the similarity is second or lower. You may obtain | require a hit rate similarly including the case. In this case, it is preferable to obtain a hit rate from the same expression as Expression (1) by increasing the weight in the higher rank.

  Furthermore, in the above embodiment, the case where the frame in which the image quality flag 422 is set to NG is excluded when calculating the first-rank hit rate has been described. For example, the number of frames in which the image quality flag 422 is set to NG is described. When the number of the most recent frames used in calculating the first hit rate is 50 (for example, when the number of the latest frames used in calculating the first hit rate is 50), In the case where the number of frames set to NG is 3 or less), it is not necessary to exclude the frame in which the image quality flag 422 is set to NG when calculating the first hit rate.

DESCRIPTION OF SYMBOLS 10 Face image authentication apparatus 100 Imaging part 200 Output part 300 Image processing part 310 Storage part 320 Collation part 321 Face detection means 322 Face tracking means 323 Face collation means 324 Image quality determination means 325 Similarity calculation means 326 Authentication candidate person specification means 327 Detection Rate calculation means 328 Authentication threshold value determination means 329 Authentication determination means

Claims (4)

An imaging unit that sequentially acquires a monitoring image obtained by photographing a person in the monitoring area, a storage unit that stores in advance face images of a plurality of registered persons as registered face images, and a person in the monitoring area is registered using the monitoring image A face image authentication device comprising a verification unit for performing person authentication as to whether or not a person is a person,
The collation unit
Face detection means for extracting an image of an area including a human face from the monitoring image as a face area image;
Face tracking means for tracking the face area image with a predetermined number of monitoring images sequentially acquired;
Similarity calculation means for calculating the similarity between the face area image and the registered face image;
An authentication candidate person specifying means for specifying an authentication candidate person from the plurality of registered persons based on the similarity;
Detection rate calculation means for calculating the ratio of the face area image identified as the authentication candidate person with respect to the predetermined number of monitoring images as the registered person detection rate for the authentication candidate person;
Authentication threshold value determining means for determining an authentication threshold value for authenticating whether or not the face area image belongs to the authentication candidate person for each authentication candidate person according to the registered person detection rate;
An authentication determination unit that determines that an authentication candidate person whose similarity exceeds the authentication threshold is a person in the monitoring area.   The face image authentication apparatus according to claim 1, wherein the authentication threshold value determination unit lowers the authentication threshold value as the registered person detection rate increases.   The face image authentication apparatus according to claim 1, wherein the collation unit performs person authentication using a plurality of predetermined latest monitoring images.   The collation unit further includes image quality determination means for determining whether or not the face area image has an image quality appropriate for the calculation of the registered person detection rate, and uses the face area image determined to have the appropriate image quality. The face image authentication device according to claim 1, wherein person authentication is performed.

Images