JP7511158B2 - Facial recognition system and method for updating registered facial images - Google Patents

Description

The present invention relates to a face authentication system that compares a face image of a person to be authenticated obtained by photographing the person to be authenticated with a registered face image for each user, and a registered face image updating method that updates the registered face image in the face authentication system using an information processing device.

In recent years, facial recognition technology has become widespread for identifying people for purposes such as entrance and exit management. This facial recognition technology identifies the person to be authenticated by comparing the facial image of the person to be authenticated, which is acquired by photographing the person, with a registered facial image for each user. However, this type of facial recognition system has the problem that the matching accuracy decreases as time passes after the facial image is registered and the person's face changes during that time due to aging, etc.

To solve the problem of reduced matching accuracy due to the registered face images becoming outdated, a technique has been known in which face images of the person to be authenticated acquired during authentication are stored, and at an appropriate time, an appropriate face image is selected from the stored face images and registered in place of the old face image (see Patent Document 1). With this technique, the updating work of the administrator is automated, reducing the workload on the administrator.

JP 2002-259980 A

Now, in a facial recognition system, one-to-N facial matching is performed, matching one person to be authenticated with many registered people, but in a large-scale system, the processing load of facial matching becomes large due to the large number of registered people. For this reason, the system is operated with one registered facial image for each user to reduce the processing load of facial matching.

On the other hand, with conventional technology, there is a possibility that an inappropriate face image may be selected during the update process. In particular, in a system that operates with one registered face image per user, if an inappropriate face image is selected, the reliability of future face matching will drop significantly. For this reason, it is desirable for an administrator to visually check and select an appropriate face image. In this case, it is desirable to reduce the workload of the administrator as much as possible.

The main objective of the present invention is to provide a face authentication system and a registered face image update method that allows an administrator to efficiently select an appropriate face image from the face images of the person to be authenticated by visual confirmation, thereby reducing the burden of the update work on the administrator.

The face authentication system of the present invention is a face authentication system that compares a face image of a person to be authenticated acquired by photographing the person to be authenticated with a registered face image for each user, and includes a face authentication machine that photographs the person to be authenticated and acquires the face image of the person to be authenticated, and a server device that stores the registered face image for each user and executes a comparison process between the face image of the person to be authenticated and the registered face image. The server device stores the face image of the person to be authenticated acquired from the face authentication machine, and also stores authentication log management information that associates and manages the face image of the person to be authenticated, the authentication result, and the user regarding an authentication event that occurred in the face authentication machine , and displays a screen having a threshold designation unit that allows an administrator to designate a threshold value for extraction judgment when extracting a target person based on a matching score. and based on the authentication log management information, extracting as subjects, from among the users who have been successfully authenticated, those whose matching score tendency value is less than the extraction judgment threshold value specified by the administrator, extracting as subjects a plurality of candidate face images from the authenticated person face images of the subjects, presenting the extracted plurality of candidate face images to the administrator and displaying on the administrator device a screen for allowing the administrator to select a candidate face image, and performing an update process in which the selected candidate face image replaces the currently registered face image as a new registered face image, and when the administrator specifies the extraction judgment threshold value, a value that is larger than the matching judgment threshold value by a predetermined value is displayed on the screen of the administrator device as an initial value of the extraction judgment threshold value .

Further, a registered face image updating method of the present invention is a method for updating a registered face image by an information processing device in a face authentication system that compares a face image of a person to be authenticated acquired by photographing the person to be authenticated with a registered face image for each user, the method storing the face image of the person to be authenticated acquired from a face authentication machine, and storing authentication log management information that associates and manages the face image of the person to be authenticated, the authentication result, and the user regarding an authentication event that occurred in the face authentication machine, and displays on an administrator device a screen having a threshold designation unit that allows an administrator to designate a threshold value for extraction judgment when extracting a target person based on a matching score, and displays a screen of the person to be authenticated acquired from a face authentication system that matches and manages a registered face image of the person to be authenticated acquired from a face authentication machine, the authentication log management information storing the registered face image of the person to be authenticated acquired from a face authentication machine, and the authentication log management information storing the registered face image of the person to be authenticated acquired from a face authentication machine, the authentication log management information storing the registered face image of the person to be authenticated acquired from a face authentication machine, Among users, those whose matching score tendency value is less than the extraction judgment threshold value specified by the administrator are extracted as subjects, a plurality of candidate face images are extracted from the authenticated person face images of the subjects, the extracted plurality of candidate face images are presented to the administrator, a screen for allowing the administrator to select a candidate face image is displayed on the administrator device, and an update process is performed in which the selected candidate face image replaces the currently registered face image as a new registered face image, and when the administrator specifies the extraction judgment threshold value, a value that is a predetermined value larger than the matching judgment threshold value is displayed on the screen of the administrator device as the initial value of the extraction judgment threshold value.

According to the present invention, candidate face images that are candidates for new registered face images are presented to the administrator, so that the administrator can visually confirm the candidate face images. In addition, by using a threshold value for extraction judgment in a predetermined range that is greater than the threshold value for matching judgment, users who have been successfully authenticated and have a low matching score tendency value (low-score authenticated users) are extracted as targets, so that the update work can be performed on users who truly need updating. This allows the administrator to efficiently select an appropriate face image as a new registered face image from the face images of the person to be authenticated by visual confirmation, thereby reducing the burden of the update work on the administrator.

FIG. 1 is a diagram showing the overall configuration of a face authentication system according to an embodiment of the present invention. FIG. 13 is an explanatory diagram showing the contents of an authentication log management table processed by the face authentication server 2. A block diagram showing the schematic configuration of a checker 1 and a face authentication server 2. FIG. 2 is an explanatory diagram showing an overview of the processing performed by the face authentication server 2. FIG. 13 is an explanatory diagram showing a management screen displayed on a manager terminal 3. FIG. 13 is an explanatory diagram showing a filtering setting screen displayed on the administrator terminal 3. FIG. 13 is an explanatory diagram showing a candidate face image selection screen displayed on the administrator terminal 3. FIG. 2 is an explanatory diagram showing a registration face image enlargement screen and a candidate face image enlargement screen displayed on the administrator terminal 3.

A first invention made to solve the above-mentioned problems is a face authentication system that compares a face image of a person to be authenticated acquired by photographing the person to be authenticated with a registered face image for each user, the system comprising: a face authentication machine that photographs the person to be authenticated and acquires the face image of the person to be authenticated; and a server device that stores the registered face image for each user and executes a process of comparing the face image of the person to be authenticated with the registered face image, the server device stores the face image of the person to be authenticated acquired from the face authentication machine, and also stores authentication log management information that associates and manages the face image of the person to be authenticated, the authentication result, and the user regarding an authentication event that has occurred in the face authentication machine, and has a screen having a threshold designation unit that allows an administrator to designate a threshold value for extraction judgment when extracting a target person based on a matching score. on an administrator device, and among the users who have been successfully authenticated based on the authentication log management information, extract as subjects those users whose matching score tendency value is less than the extraction judgment threshold value specified by the administrator, extract a plurality of candidate face images from the authenticated person face images of the subjects, present the extracted plurality of candidate face images to the administrator, and display on the administrator device a screen for allowing the administrator to select a candidate face image, and perform an update process in which the selected candidate face image replaces the currently registered face image as a new registered face image, and when the administrator specifies the extraction judgment threshold value, a value that is larger than the matching judgment threshold value by a predetermined value is displayed on the screen of the administrator device as an initial value of the extraction judgment threshold value .

According to this, candidate face images that are candidates for new registered face images are presented to the administrator, so that the administrator can visually confirm the candidate face images. In addition, by using a threshold value for extraction judgment in a predetermined range that is larger than the threshold value for matching judgment, users who have been successfully authenticated and have a low matching score tendency value (low-score authenticated users) are extracted as targets, so that the update work can be performed on users who truly need updating. This allows the administrator to efficiently select an appropriate face image as a new registered face image from the face images of the person to be authenticated by visual confirmation, thereby reducing the burden of the update work on the administrator.

In addition, the second invention is configured such that the server device displays a period designation section on the screen of the administrator device, which allows the administrator to designate a target period, and tally up authentication events that fall within the target period designated by the administrator.

This allows you to narrow down the authentication events you are looking for by the date and time that the authentication event occurred.

In addition, in a third invention, the server device is configured to display a number designation section on a screen of the administrator device, which allows an administrator to designate the number of candidate face images, extract the number of candidate face images designated by the administrator, and display the extracted multiple candidate face images in a line on the screen of the administrator device.

This allows the administrator to compare multiple candidate face images and select the most suitable candidate face image as the new registration image.

In addition, in a fourth invention, the server device is configured such that, when extracting the candidate face images, the server device determines whether the target face image corresponds to an impersonation detected during face authentication, and excludes face images corresponding to an impersonation from the candidates for extraction of the candidate face images.

This will prevent spoofed facial images from being selected as registered facial images.

In addition, in a fifth invention, the server device is configured to determine whether or not the target face image satisfies image quality related to face matching when extracting the candidate face image, and to exclude face images that do not satisfy the image quality from the candidates for extracting the candidate face image.

This makes it possible to prevent face images that do not meet the image quality requirements, i.e., face images that are clearly inappropriate as registered face images, from being extracted as candidate face images.

A sixth aspect of the present invention is a method for updating a registered face image by an information processing device in a face authentication system that compares a face image of a person to be authenticated acquired by photographing the person to be authenticated with a registered face image for each user, the method comprising: storing the face image of the person to be authenticated acquired from a face authentication device; storing authentication log management information that associates and manages the face image of the person to be authenticated, the authentication result, and the user relating to an authentication event that occurred in the face authentication device; displaying on an administrator device a screen having a threshold designation unit that allows an administrator to designate a threshold value for extraction judgment when extracting a target person based on a matching score ; and displaying, based on the authentication log management information, a screen of users who have been successfully authenticated. Among these, users whose matching score tendency value is less than the extraction judgment threshold value specified by the administrator are extracted as subjects, multiple candidate face images are extracted from the authenticated person face images of the subjects, the multiple extracted candidate face images are presented to the administrator, a screen for allowing the administrator to select a candidate face image is displayed on the administrator device, and an update process is performed in which the selected candidate face image replaces the currently registered face image as a new registered face image, and when the administrator specifies the extraction judgment threshold value, a value that is a predetermined value larger than the matching judgment threshold value is displayed on the screen of the administrator device as the initial value of the extraction judgment threshold value .

As with the first invention, this allows the administrator to efficiently select an appropriate facial image from the facial images of the person to be authenticated by visual confirmation, thereby reducing the burden of updating work on the administrator.

The following describes an embodiment of the present invention with reference to the drawings.

Figure 1 is a diagram showing the overall configuration of a face authentication system according to this embodiment.

This face authentication system includes a checker 1 (face authentication machine), a face authentication server 2 (server device), and an administrator terminal 3. The checker 1, the face authentication server 2, and the administrator terminal 3 (administrator device) are connected to each other via a network.

Checker 1 photographs the face of the person to be authenticated, obtains a facial image of the person to be authenticated, and transmits the facial image of the person to be authenticated to face authentication server 2. Checker 1 also receives the authentication result from face authentication server 2 and executes an operation according to the authentication result. For example, if checker 1 is for restricting entry to pre-registered individuals, it will perform control such as opening a gate if authentication is successful.

The facial recognition server 2 performs facial recognition based on the face image of the person to be authenticated received from the checker 1, and transmits the authentication result to the checker 1. At this time, the facial recognition server 2 compares the face image of the person to be authenticated acquired from the checker 1 with the registered face image of each person registered in the registrant database, and if the matching score is equal to or greater than a predetermined threshold value, it identifies the person in the registered face image as the person to be authenticated. Furthermore, if multiple people with matching scores equal to or greater than the threshold value are found, it identifies the person with the highest matching score as the person to be authenticated.

The administrator terminal 3 is operated by an administrator. By accessing the face authentication server 2, the administrator terminal 3 displays a management screen related to the face authentication server 2. In particular, in this embodiment, a management screen (update operation screen) related to updating registered face images registered in the registrant database is displayed. On this management screen, the registered face image and multiple candidate face images extracted from the authenticated person's face image acquired during face authentication for the same person are displayed on the administrator terminal 3, and the administrator visually checks and selects the most suitable candidate face image. In the face authentication server 2, a process (update) is performed in which the candidate face image selected by the administrator is replaced with the currently registered face image as a new registered face image.

In this embodiment, the registrant database is configured to be built in the face authentication server 2, but the registrant database may be configured to be built in a database server separate from the face authentication server 2.

Next, we will explain the authentication log management table processed by the face authentication server 2. Figure 2 is an explanatory diagram showing the contents of the authentication log management table.

The facial recognition server 2 manages an authentication log management table (authentication log management information). This authentication log management table registers authentication log information for each authentication event in which the checker 1 photographs the person to be authenticated and the facial recognition server 2 executes facial recognition processing. Each time an authentication event occurs, authentication log information related to that authentication event is added to the authentication log management table.

During face authentication, one-to-N face matching, that is, one-to-one face matching between one face image of the person to be authenticated and one registered person, is repeated the same number of times as the number of registered people, and authentication log information is created for each one-to-N face matching. In other words, one authentication log is generated (output) for each 1:N face matching.

The authentication log information for each authentication event includes the authentication date and time, the checker 1 that performed the authentication, the face image ID of the person to be authenticated, the authentication result registrant ID, the authentication result, and the matching score obtained during authentication.

Here, the authentication log information and the face image of the person to be authenticated are linked by the face image ID of the person to be authenticated contained in the authentication log information. Note that in this embodiment, multiple face images of the person to be authenticated are acquired for authentication of one person and face matching is performed, so multiple authentication logs are generated for authentication of one person and a face image of the person to be authenticated is linked to each log.

The authentication result included in the authentication log information indicates whether the authentication was successful or not. Specifically, "OK" means that there was at least one person for whom the match was successful (the match score exceeded a certain threshold), i.e., the person to be authenticated was able to be identified, and "NG" means that there was no person for whom the match was successful, i.e., the person to be authenticated could not be identified.

In addition, in this embodiment, the checker 1 performs a process to detect spoofing, that is, the act of impersonating another person to receive authentication by displaying an image of another person's face on the screen of a smartphone or tablet terminal and holding it over the checker 1, or by holding a photo of another person's face over the checker 1. If spoofing is detected, the result is "NG" even if the matching score is equal to or exceeds a predetermined threshold value.

Next, the general configuration of the checker 1 and the face authentication server 2 will be described. FIG. 3 is a block diagram showing the general configuration of the checker 1 and the face authentication server 2. FIG. 4 is an explanatory diagram showing an overview of the processing performed by the face authentication server 2.

The checker 1 includes a camera 11, a communication unit 12, a memory unit 13, and a processor 14.

Camera 11 captures the face of the person undergoing face authentication.

The communication unit 12 communicates with the face recognition server 2 via the network.

The memory unit 13 stores programs executed by the processor 14.

The processor 14 performs various processes related to face authentication by executing the programs stored in the memory unit 13. In this embodiment, the processor 14 performs processes such as spoofing detection and obtaining a face image of the person to be authenticated.

In the spoofing detection process, the processor 14 detects spoofing, that is, the act of impersonating another person and receiving authentication by displaying an image of another person's face on the screen of a smartphone or tablet terminal and holding it over the checker 1, or by holding a photo of another person's face over the checker 1. This spoofing detection process, for example, detects the unevenness of the face based on changes in the image (frame) captured by the camera 11, and determines whether the subject is a real face or a face displayed on the screen.

In the process of acquiring the face image of the person to be authenticated, the processor 14 detects the face of the person (person to be authenticated) from the image captured by the camera 11, and acquires the face image of the person to be authenticated by cutting out the image area of the face from the captured image. The acquired face image of the person to be authenticated is then sent from the communication unit 12 to the face authentication server 2 together with the result of the spoofing detection. At this time, the above process is performed, for example, five times per second. Therefore, for example, if a person is in front of the checker for two seconds (within the angle of view of the camera of checker 1 for two seconds), the above process is performed ten times.

The face recognition server 2 includes a communication unit 21, a memory unit 22, and a processor 23.

The communication unit 21 communicates with the checker 1 and the administrator terminal 3 via the network.

The storage unit 22 stores programs executed by the processor 23, etc. The storage unit 22 also stores an authentication log management table (see FIG. 2), face images of the person to be authenticated, and registration information of a registrant database, etc. Attribute information and registered face images for each user are registered in the registrant database.

The authentication log information for each authentication event registered in the authentication log management table is deleted when a predetermined period of time (e.g., six months) has passed since the occurrence of the authentication event. This predetermined period of time can be set to any period of time from a setting screen (not shown). The face image of the person to be authenticated and the registered face image are stored in the memory unit 22 in an encrypted state. Although one piece of registered face information may be registered in the registered user database for each user, two or more pieces of registered face information may be registered.

The processor 23 performs various processes related to face recognition by executing the programs stored in the memory unit 22. In this embodiment, the processor 23 performs face recognition processing, authentication log management processing, matching score aggregation processing, subject extraction processing, face image filtering processing, candidate face image extraction processing, and registered face image update processing.

In the face authentication process, the processor 23 performs face authentication based on the face image of the person to be authenticated received from the checker 1 and the result of spoofing detection. At this time, the processor 23 compares the facial feature information extracted from the face image of the person to be authenticated with the facial feature information of each registrant registered in the registrant database to calculate a matching score, and compares this matching score with a predetermined threshold value (e.g., 0.75) to determine whether the person to be authenticated is the same person as any of the registrants (face matching process). Here, if the matching score is equal to or greater than the threshold value, it is determined that face matching has been successful, and the person to be authenticated is identified. On the other hand, if the matching score is less than the threshold value, it is determined that face matching has failed. Furthermore, if spoofing is detected, it is determined that face authentication has failed.

In this embodiment, the checker 1 transmits a facial image of the person to be authenticated to the facial authentication server 2, and the facial authentication server 2 extracts facial feature information from the facial image of the person to be authenticated. However, the checker 1 may extract facial feature information from the facial image of the person to be authenticated, and the facial feature information may be transmitted from the checker 1 to the facial authentication server 2.

In addition, in this embodiment, multiple facial images of the person to be authenticated (e.g., 5 per second) are acquired for authentication of one person, and in the face matching process, the facial feature information of each of the multiple facial images of the person to be authenticated is compared with the facial feature information of each registered person, a matching score is calculated for each of the multiple facial images of the person to be authenticated, and the person to be authenticated is identified.

In the authentication log management process, the processor 23 manages an authentication log management table (authentication log management information) (see FIG. 2). This authentication log management table registers authentication log information for each authentication event in which the checker 1 photographs the person to be authenticated and the face recognition server 2 executes face recognition processing. Each time an authentication event occurs, the processor 23 adds the authentication log information related to that authentication event to the authentication log management table.

In the matching score tallying process, the processor 23 tallies up the authentication scores (matching scores) for each user (person to be authenticated) and checker 1 based on the authentication log management table. In this embodiment, the average value (average score) of the matching scores is calculated. In addition, this matching score tallying process is performed for authentication events in which authentication was successful.

In addition, in the matching score aggregation process, authentication events that meet certain extraction conditions are extracted as targets for aggregation. In this embodiment, aggregation is performed for authentication events that fall within the target period that is the extraction condition. The target period is specified by the administrator.

In addition, in this embodiment, as an extraction condition, the count is performed on users whose number of authentication events (number of authentications) included in the target period is equal to or greater than a predetermined threshold (for example, 10 times per checker). The number of authentications is counted for each user regardless of the authentication result. The threshold for the number of authentications is specified by the administrator. As a result, users whose number of authentications is less than the threshold are excluded from the update target. This makes it possible to avoid updating users for whom there are few samples and therefore little point in updating them.

In the target extraction process, the processor 23 extracts users (low-score authenticated users) who are targets for updating face images based on the average score obtained in the matching score aggregation process. Specifically, users whose average score is less than a predetermined threshold value (e.g., 0.8) are extracted. The threshold value for extraction judgment, i.e., the threshold value for the average score, is specified by the administrator. The threshold value for extraction judgment is a value greater than the threshold value for matching judgment (e.g., 0.75). Note that in the matching score aggregation process, matching score trend values such as the median score or the most frequently occurring score value may be obtained in addition to the average score. In this way, by identifying users who have tended to have low scores because their faces have changed due to aging or other factors over the years since their images were registered, it becomes possible to perform update processing targeting users who need to be updated.

In addition, in the target extraction process, if the number of users whose average score is less than the threshold exceeds a predetermined number (e.g., 10 people), target users are extracted so that the number does not exceed the predetermined number. At this time, the number of users to be extracted is selected in order starting from the user with the lowest average score. The number of users to be extracted is specified by the administrator.

In the face image filtering process, the processor 23 filters face images (authenticated person face images for each authentication event) linked to all authentication logs of users (low-scoring authenticated persons) extracted in the target extraction process, and excludes inappropriate face images from the targets. In this embodiment, filtering is performed regarding image quality and impersonation.

In filtering related to image quality, the processor 23 determines whether the face image of the person to be authenticated satisfies a predetermined image quality (quality determination process), and face images of the person to be authenticated that do not satisfy the image quality are excluded from the target. Specifically, for example, a face image with an exaggerated facial expression such as a smile is determined not to satisfy the image quality. Note that if the face image of the person to be authenticated does not satisfy the image quality, the matching score will be low and matching will often fail, but there are also cases where matching is successful, in which case there is a risk that the face image of the person to be authenticated that does not satisfy the image quality will be extracted as a candidate face image.

In addition, in filtering related to spoofing, the processor 23 determines whether or not a face image is the result of spoofing based on the result of spoofing detection during face authentication (spoofing determination process), and spoofed face images are excluded from the target. Note that in the case of spoofing, face authentication will ultimately fail due to spoofing detection, but the matching score will be high in face matching, so the face matching itself is often successful. In this case, it is accumulated as an authentication event in which face matching was successful. For this reason, filtering is required to prevent spoofed face images from being extracted as candidate face images.

In the candidate face image extraction process, the processor 23 extracts a predetermined number (e.g., 10) of candidate face images to be used for face image updating from the face images narrowed down by the face image filtering process for each user (low score authenticated user) who is the target of face image updating. The number of images to be extracted is specified by the administrator. At this time, the face images to be extracted are those linked to authentication events in which authentication was successful, among the authentication events of low score authenticated users.

In addition, in this embodiment, the number of face images to be extracted is determined in descending order of suitability as a registered face image, and then the number of face images to be extracted is extracted as candidate face images. Specifically, the number of face images to be extracted is determined in descending order of matching score, and then the number of face images to be extracted is extracted as candidate face images, and then the number of face images to be extracted is determined in descending order of image quality.

In the registered face image update process, the processor 23 performs a process of replacing the currently registered face image with the candidate face image selected by the administrator from among the multiple candidate face images obtained in the candidate face image extraction process as a new registered face image. In this embodiment, the multiple candidate face images obtained in the candidate face image extraction process are displayed on the administrator terminal 3, and the administrator visually checks them and selects the most suitable candidate face image.

Next, we will explain the management screen displayed on the administrator terminal 3. Figure 5 is an explanatory diagram showing the management screen.

This management screen has an authentication log file selection section 31, a registered face image file selection section 32, and an import button 33.

In the authentication log file selection unit 31, when the user operates the selection button 34, a selection screen (not shown) for the authentication log file (including authentication log management information) is displayed, and when the user selects an authentication log file on this selection screen, the file name of the selected authentication log file is displayed in the import file display unit 35. Also, in the registration face image file selection unit 32, when the user operates the selection button 36, a selection screen (not shown) for the registration face image file is displayed, and when the user selects a registration face image file on this selection screen, the file name of the selected registration face image file is displayed in the import file display unit 37.

When the user selects an authentication log file in the authentication log file selection section 31 and also selects a registered face image file in the registered face image file selection section 32, and then operates the import button 33, the selected authentication log file, the face image of the person to be authenticated linked to that authentication log file, and the selected registered face image file are loaded. Note that the authentication log file and the face image of the person to be authenticated are stored in an encrypted state, so they are decrypted when they are loaded.

This management screen also has a first mode selection button 41, a second mode selection button 42, and a third mode selection button 43. When the administrator operates the first mode selection button 41, this management screen, i.e., a screen displaying statistical information on the authentication score for the user, is displayed. When the administrator operates the second mode selection button 42, a screen displaying statistical information on the authentication score for the checker 1 is displayed. When the administrator operates the third mode selection button 43, a screen displaying authentication log information is displayed.

In addition, this management screen has a first extraction condition setting section 45, a second extraction condition setting section 46, and a third extraction condition setting section 47.

The first extraction condition setting section 45 sets the extraction conditions (processing conditions) related to the matching score aggregation process. This first extraction condition setting section 45 is provided with an extraction button 51, a target period designation section 52, and an authentication count threshold designation section 53.

The target period designation unit 52 allows the administrator to designate the date and time that marks the start and end of the target period.

The authentication count threshold specification unit 53 allows the administrator to specify a threshold for the number of authentications (minimum number of authentications).

When the administrator specifies the target period in the target period specification section 52 and also specifies a threshold value for the number of authentications in the authentication count threshold specification section 53, and then operates the extract button 51, an extraction process related to the matching score aggregation process is executed, authentication events included in the target period are extracted, and further, users whose authentication count is equal to or exceeds the threshold value are extracted. Then, the list in the statistical information display section 61 is updated based on the extraction specification results.

The second extraction condition setting section 46 sets the extraction conditions (processing conditions) related to the subject extraction process. This second extraction condition setting section 46 is provided with an extraction button 54, a score threshold designation section 55, and a number of people to be extracted designation section 56.

The score threshold designation unit 55 allows the administrator to designate a threshold (extraction judgment threshold) related to the matching score trend value (e.g., the average, median, and recurrent score value of the matching score) when extracting users (low-score authenticated users) who are targets for updating their facial images. This extraction judgment threshold is a value greater than the threshold (matching judgment threshold) when determining whether or not face matching has been successful.

In this embodiment, the upper limit is set to a value (e.g., 0.80) that is a predetermined value (e.g., 0.05) greater than the threshold for matching (e.g., 0.75), and this upper limit is displayed as the initial value in the score threshold designation unit 55. In addition, the threshold for extraction can be selected within the range from the threshold for matching to the upper limit; specifically, a pull-down menu displays a selection of thresholds (e.g., 0.80, 0.79, 0.78, 0.77, 0.76).

The number of people to be extracted designation section 56 allows the administrator to designate the number of people to be extracted when extracting target users (low-score authenticated users).

When the administrator specifies the threshold value for extraction judgment in the score threshold designation section 55 and the number of people to be extracted in the number of people to be extracted designation section 56, and then operates the extraction button 54, a target extraction process is executed and users whose matching score tendency value is less than the threshold value are extracted, and further, users equal to the number of people to be extracted are extracted starting from the user with the lowest matching score tendency value (worst score N). Then, the list in the statistical information display section 61 is updated based on the extraction designation results.

The third extraction condition setting section 47 sets the extraction conditions (processing conditions) related to the candidate face image extraction process. This third extraction condition setting section 47 is provided with an extraction button 57 and a face image extraction number designation section 58. In the face image extraction number designation section 58, the administrator can designate the number of candidate face images to be extracted (e.g., 10).

When the administrator operates the extraction button 57 after specifying the number of images to be extracted in the facial image extraction number specification section 58, the candidate facial image extraction process is executed, and the number of facial images to be extracted is extracted as candidate facial images in descending order of suitability (matching score or image quality) as candidate facial images. Then, the list in the statistical information display section 61 is updated based on the extraction specification results.

The statistical information display unit 61 also displays in a table the results of the matching scores for all checkers 1 and the results of the matching scores for each checker 1. In this table, the number of face matches and the minimum, maximum, and average results of the matching scores are displayed for each user.

The statistical information display section 61 is also provided with vertical and horizontal scroll operation sections 62, 63 (scroll button, scroll bar). The administrator can operate these scroll operation sections 62, 63 to shift the display range of the list in the vertical and horizontal directions.

This management screen also has a settings button 65. When the administrator operates this settings button 65, the screen transitions to a filtering settings screen (see Figure 6).

The management screen also has an execute button 66. When the administrator operates this execute button 66, the screen transitions to a candidate face image selection screen (see FIG. 7).

Next, we will explain the filtering setting screen displayed on the administrator terminal 3. Figure 6 is an explanatory diagram showing the filtering setting screen.

This filtering setting screen allows the administrator to specify whether filtering (narrowing) is required for each item related to image quality, and whether filtering is required for spoofing. Specifically, the filtering setting screen has a check box 71 for each item related to image quality, and a check box 72 for spoofing.

Image quality filtering allows administrators to specify whether or not filtering is required for the following items: wearing sunglasses, wearing a mask, having multiple faces in the image, the image of a face that is too small, the image of a face that is out of focus or blurred, backlighting, blown-out highlights, wearing a hat, smiling or other exaggerated facial expressions, eyes obscured by hair, eyes closed, facial features obscured by hands or other objects, and a face not facing forward (sideways, upwards, or downwards).

This filtering setting screen has a setting button 73. When the administrator operates this setting button 73, setting information regarding the face image filtering process is registered based on the input contents of the check boxes 71 and 72.

Next, we will explain the candidate face image selection screen displayed on the administrator terminal 3. Figure 7 is an explanatory diagram showing the candidate face image selection screen.

This candidate face image selection screen is provided with a registered face image display section 81 and a candidate face image display section 82. The registered face image display section 81 displays the currently registered face image. The candidate face image display section 82 displays a predetermined number of extracted candidate face images (e.g., 10 images) in an order. This allows the administrator to visually confirm the suitability of the candidate face images while comparing them with the currently registered face image, and select the candidate face image that will become the new registered face image.

The registered face image display section 81 is also provided with a forward button 83a and a back button 83b. When the administrator operates the forward button 83a and the back button 83b, the administrator can change the users to be viewed from among the extracted users (low score authenticated users), and the registered face images of each user can be displayed one by one in the order of extraction.

This candidate face image selection screen also has a number display section 84 and a file name display section 85 relating to the registered face images. In the number display section 84, the number of the user being displayed and the total number of extracted users (worst score N) are displayed as fractions. Here, for example, 9/10 indicates that the user is the ninth out of 10 extracted users.

In addition, when the administrator performs an operation to enlarge the registered image (e.g., double-clicking on the image), the registered face image display unit 81 transitions to a registered image enlargement screen (see FIG. 8(A)).

In addition, the candidate face image display section 82 is provided with a radio button 86 for each candidate face image. The administrator can select a candidate face image by operating this radio button 86. When a candidate face image is selected, the selected candidate face image is highlighted. For example, the display frame of the candidate face image is highlighted.

In addition, when the administrator performs an operation to enlarge the candidate image (e.g., double-clicking on the image), the candidate face image display unit 82 transitions to a candidate image enlargement screen (see FIG. 8 (B)).

This candidate face image selection screen also has a number display section 87 and a management information display section 88 relating to the candidate face image. In the number display section 87, the number of the candidate face image selected in the candidate face image display section 82 and the total number of extracted candidate face images are displayed as fractions. Here, for example, 1/10 indicates that the candidate face image is the first out of 10 extracted. In the management information display section 88, management information relating to the selected candidate face image is displayed. Specifically, the authentication log ID, authentication event ID, person ID of the person to be authenticated, authentication date and time, authentication result, and matching score are displayed.

This candidate face image selection screen also has a file save button 89. When the administrator selects a candidate face image in the candidate face image display section 82 and operates the file save button 89, the selected candidate face image and management information (attribute information) related to the candidate face image are saved in the batch registration file. When the operation of selecting the candidate image related to the extracted user (the user with the worst score N) is completed, the administrator performs a batch registration operation. This causes an update process to be performed based on the candidate image and management information for each person saved in the batch registration file. In other words, a process is performed in which each of the candidate images saved in the batch registration file is replaced with a current registered image as a new registered image.

In this embodiment, one registered face image is registered for each user, and therefore one registered face image is displayed on the registered face image display unit 81; however, multiple registered face images may be registered for each user, in which case the multiple registered face images are displayed side-by-side on the registered face image display unit 81. Also, the administrator may be able to select multiple candidate face images from the multiple candidate face images displayed on the candidate face image display unit 82, up to the maximum number of registered images.

In addition, among the candidate face images displayed in the candidate face image display section 82, those that the administrator has not selected are assumed to be inappropriate as registered face images, and therefore may be deleted or may not be extracted as candidate face images the next time the system is updated.

Next, we will explain the enlarged registered face image screen and the enlarged candidate face image screen displayed on the administrator terminal 3. Figure 8 is an explanatory diagram showing the enlarged registered face image screen and the enlarged candidate face image screen.

As shown in FIG. 8(A), the enlarged registered face image screen is provided with an enlarged registered face image display section 91. In this enlarged registered face image display section 91, the registered face image is enlarged and displayed. In addition, this enlarged registered face image screen is provided with a number display section 92 and a file name display section 93, similar to the number display section 84 and file name display section 85 in the candidate face image selection screen (see FIG. 7).

As shown in FIG. 8(B), the candidate face image enlargement screen is provided with an enlarged candidate face image display section 95. In this enlarged candidate face image display section 95, the candidate face image is enlarged and displayed. Similar to the number display section 87 and management information display section 88 in the candidate face image selection screen (see FIG. 7), a number display section 96 and a management information display section 97 are also provided.

As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can also be applied to embodiments in which modifications, substitutions, additions, omissions, etc. have been made. It is also possible to combine the components described in the above embodiments to create new embodiments.

The face authentication system and registered face image updating method of the present invention have the effect of enabling an administrator to efficiently select an appropriate face image from the face images of the person to be authenticated by visual confirmation, thereby reducing the burden of the administrator's updating work, and are useful as a face authentication system that compares the face image of the person to be authenticated obtained by photographing the person to be authenticated with the registered face image for each user, and as a registered face image updating method that updates the registered face image in the face authentication system using an information processing device.

1 Checker (Facial Recognition Machine)
2. Face recognition server (server device)
3. Administrator terminal (administrator device)
11 Camera 12 Communication unit 13 Storage unit 14 Processor 21 Communication unit 22 Storage unit 23 Processor 45 First extraction condition setting unit 46 Second extraction condition setting unit 47 Third extraction condition setting unit 51 Extraction button 52 Target period designation unit (period designation unit)
53 Authentication count threshold designation section 54 Extract button 55 Score threshold designation section (threshold designation section)
56 Extraction number designation section 57 Extraction button 58 Face image extraction number designation section (number designation section)
61 Statistical information display section 65 Setting button 66 Execute button 81 Registered face image display section 82 Candidate face image display section

Claims (6)

A face authentication system that compares a face image of a person to be authenticated obtained by photographing the person to be authenticated with a registered face image for each user,
A face authentication device that photographs a person to be authenticated and acquires a face image of the person to be authenticated;
a server device that stores the registered face image for each user and executes a process of matching the face image of the person to be authenticated with the registered face image,
The server device includes:
storing the face image of the person to be authenticated acquired from the face authentication device, and accumulating authentication log management information that associates and manages the face image of the person to be authenticated, the authentication result, and the user regarding an authentication event that occurred in the face authentication device;
displaying on the administrator device a screen having a threshold designation section for allowing the administrator to designate a threshold value for extraction judgment when extracting the target person based on the matching score;
extracting, as subjects, users whose matching score tendency value is less than the extraction judgment threshold value designated by an administrator, from among the users who have been successfully authenticated based on the authentication log management information;
extracting a plurality of candidate face images from the face images of the person to be authenticated ;
displaying a screen on the administrator device that presents the extracted plurality of candidate face images to an administrator and allows the administrator to select a candidate face image;
An update process is performed to replace the currently registered face image with the selected candidate face image as a new registered face image;
A face authentication system characterized in that, when an administrator specifies the threshold value for the extraction judgment, a value that is a predetermined value larger than the threshold value for the matching judgment is displayed on a screen of the administrator device as an initial value for the threshold value for the extraction judgment . The server device includes:
displaying a period designation section on a screen of the administrator device, whereby an administrator can designate a target period;
The face authentication system according to claim 1 , wherein the facial recognition events included in the target period designated by an administrator are counted. The server device includes:
a number designation section for allowing an administrator to designate the number of candidate face images is displayed on a screen of the administrator device;
Extract the number of candidate face images specified by the administrator,
2. The face authentication system according to claim 1, wherein a plurality of extracted candidate face images are displayed side by side on a screen of the administrator device. The server device includes:
The face recognition system according to claim 1, characterized in that, when extracting the candidate face images, it is determined whether the target face image corresponds to an impersonation detected during face recognition, and face images corresponding to an impersonation are excluded from the extraction of the candidate face images. The server device includes:
The face recognition system according to claim 1, characterized in that, when extracting the candidate face images, it is determined whether the target face image satisfies the image quality related to face matching, and face images that do not satisfy the image quality are excluded from the extraction of the candidate face images. A method for updating a registered face image by an information processing device in a face authentication system that compares a face image of a person to be authenticated obtained by photographing the person to be authenticated with a registered face image for each user, comprising:
storing the face image of the person to be authenticated acquired from the face authentication device, and accumulating authentication log management information that associates and manages the face image of the person to be authenticated, the authentication result, and the user regarding an authentication event that occurred in the face authentication device;
displaying on the administrator device a screen having a threshold designation section for allowing the administrator to designate a threshold value for extraction judgment when extracting the target person based on the matching score;
extracting, as subjects, users whose matching score tendency value is less than the extraction judgment threshold value designated by an administrator, from among the users who have been successfully authenticated based on the authentication log management information;
extracting a plurality of candidate face images from the face images of the person to be authenticated ;
displaying a screen on the administrator device that presents the extracted plurality of candidate face images to an administrator and allows the administrator to select a candidate face image;
An update process is performed to replace the currently registered face image with the selected candidate face image as a new registered face image;
A registered face image updating method, characterized in that , when an administrator specifies the threshold value for extraction judgment, a value that is a predetermined value larger than the threshold value for matching judgment is displayed on a screen of the administrator device as an initial value of the threshold value for extraction judgment .

Images