KR20240052387A - Feature registration system for facial recognition - Google Patents

Abstract

The present invention registers multiple types of features (feature quantities) with different facial directions for the same person so that they are not affected by the direction of the user's face, etc., when facial recognition of the same person is performed multiple times a day. This is about a feature registration system for facial recognition.
The feature registration system for facial recognition of the present invention is based on a first standard that can determine whether the first feature amount of the registered face image is similar to the second feature amount of the newly captured face image, a first face authentication means for determining whether the person in the face image is the same person as the person in the registered face image; Photographing means for photographing a plurality of facial images of the same person taken sequentially within a predetermined time so that the faces are in different directions; Extraction means for extracting, from the plurality of photographed face images of the same person, facial images of the same person with different facial orientations that are not judged to be the same person as the person in the face image registered in the first face authentication means. ; According to the second standard that adds the third feature amount of the extracted face image to the first feature amount of the registered face image, the person in the face image to be newly captured is a face image with the face in a different direction. It is characterized by having a second face authentication means to determine whether the extracted person is the same person.

Description

Feature registration system for facial recognition}

The present invention registers multiple types of features (feature quantities) with different facial directions for the same person so that they are not affected by the direction of the user's face, etc., when facial recognition of the same person is performed multiple times a day. This is about a feature registration system for facial recognition.

Recently, due to disasters such as the coronavirus, many companies have users working from remote locations such as their homes, and there are cases where confidential information is leaked due to access to information processing terminals by third parties, including family members. Managing employees is difficult.

If the appearance of the user (employee) changes every day due to the presence or absence of glasses, etc., the judgment standard obtained from the reference facial image for face recognition of the same person may be shaken, and the recognition rate of the same person may fluctuate from day to day.

In order to solve this problem, the present invention looks at the similarity between the features (feature quantity) of the previously registered reference face image and the features of the face image newly captured with a camera, and when the similarity satisfies the tolerance, Events are detected using the features of the previously registered reference face image, and conversely, in cases where the similarity does not satisfy the tolerance, a face newly captured with a camera is used to increase the recognition rate of the same person without being affected by the user's appearance. We propose to detect events using temporary features obtained from images.

As a prior art, Domestic Patent No. 10-2312152 relates to a facial recognition server and facial recognition system for facial recognition according to changes in space and time environment, extracting normal authentication results that are normally approved as a registered user among the authentication results, Normal authentication results are classified by user, and the optimal reference image for each user is determined based on the normal authentication results classified for each user. However, even in Domestic Patent No. 10-2312152, it is difficult to determine that the same person is the same person if the direction of the face is different.

The technical problem that the present invention aims to solve is to recognize multiple types of features (features, etc. The goal is to provide a feature registration system for facial recognition that registers facial features.

Another technical problem to be solved by the present invention is to determine the similarity between the first feature (feature amount) of the registered face image and the second feature of the newly captured face image, It is equipped with a first face authentication means for determining whether the person in the face image is the same as the person in the registered face image, and continuously captures face images of the same person so that the faces are in different directions within a preset time, and among them, , extracting a face image of the same person with a different face orientation that is not judged to be the same person as the person in the face image registered in the first face authentication means, and based on the first feature of the registered face image, According to the second standard that adds the third feature of the extracted face image, facial recognition is performed to determine whether the person in the newly captured face image is the same person as the person extracted from the face image with the face in a different direction. The purpose is to provide a feature registration system for

In order to solve the above problem, the feature registration system for facial recognition is based on a first standard that can determine whether the first feature amount of the registered face image and the second feature amount of the newly captured face image are similar. , a first face authentication means for determining whether the person in the newly captured face image is the same person as the person in the registered face image; Photographing means for photographing a plurality of facial images of the same person taken sequentially within a predetermined time so that the faces are in different directions; Extraction means for extracting, from the plurality of photographed face images of the same person, facial images of the same person with different facial orientations that are not judged to be the same person as the person in the face image registered in the first face authentication means. ; According to the second standard that adds the third feature amount of the extracted face image to the first feature amount of the registered face image, the person in the face image to be newly captured is a face image with the face in a different direction. It is characterized by including a second face authentication means to determine whether the extracted person is the same person.

The feature registration system further includes guide means for guiding the user to move the face in a different direction while the image capture means is photographing.

The feature registration system further includes event detection means for repeatedly detecting event information corresponding to an operation performed by a person photographed using a photographing means in front of the photographing means for each motion, wherein the event detection means is configured to detect the event. In the case of detecting information, the second face authentication means is configured to detect event information of the person determined to be the same person.

The feature registration system includes event recording means for repeatedly recording event information corresponding to an operation performed in front of the photographing means by a person photographed using a photographing means, for each motion, and the event recording means recording genital event information. In this case, the event information of the person determined by the second face authentication means to be the same person is also recorded.

The feature registration system determines whether the person in the newly captured face image is the same person as the person in the registered face image according to the first standard for determining whether the registered face image and the newly captured face image are similar. A first face authentication means for determining; Photographing means for photographing a plurality of facial images of the same person taken sequentially within a predetermined time so that the faces are in different directions; From the plurality of photographed face images of the same person, extraction means for extracting face images of the same person with different facial orientations, without determining that the first face authentication means is the same person as the person in the face image registered. ; A second face that determines whether the person in the newly captured face image is the same person as the person from whom the face image with a different face direction was extracted, according to a second standard that adds the extracted face image to the registered face image. This can be accomplished by providing an authentication means.

The feature registration system determines whether the person in the newly captured face image is the same person as the person in the registered face image according to the first standard for determining whether the registered face image and the newly captured face image are similar. A first face authentication means for determining; Photographing means for photographing a plurality of facial images of the same person taken sequentially within a predetermined time so that the faces are in different directions; Extraction means for extracting, from the plurality of photographed face images of the same person, facial images of the same person with different facial orientations that are not judged to be the same person as the person in the face image registered in the first face authentication means. ; A second face that determines whether the person in the newly captured face image is the same person as the person from whom the face image with a different face direction was extracted, according to a second standard that adds the extracted face image to the registered face image. This may be accomplished by including an authentication method.

The feature registration system for facial recognition of the present invention provides multiple types of features with different facial orientations for the same person so as not to be affected by the direction of the user's face when facial recognition of the same person is performed multiple times a day. By registering (feature quantity) and adjusting the criteria for determining whether it is the same person, the recognition rate of facial recognition is increased.

In the present invention, according to a first standard for determining the similarity between the first feature (feature quantity) of the registered face image and the second feature of the newly captured face image, the person in the newly captured face image is, It is provided with a first face authentication means for determining whether the person in the registered face image is the same as the person, and continuously captures face images of the same person so that the faces are in different directions within a preset time, and among them, the first face authentication means Extracts a face image of the same person with a different face orientation that is not judged to be the same person as the person in the registered face image, and uses the first feature of the registered face image to match the third feature of the extracted face image. According to the second standard with additional features, it is determined whether the person in the newly captured face image is the same person as the person extracted from the face image with the face in a different direction.

1 is a configuration diagram schematically showing the configuration of a telework system.
FIG. 2 is a flowchart showing detection and recording processing of an operation event performed by the teleworker's personal terminal 100 while the user (teleworker) is working from home using a telecommuting system.
Figure 3 is a flowchart showing a procedure for registering a face image that serves as a standard for face authentication in the present invention.
Figure 4 is a flowchart showing a procedure for photographing one face image that serves as a standard for face authentication in the present invention.
Figure 5 is a flowchart showing a procedure for shooting multiple facial images, which are the standard for face authentication, in video format in the present invention.

Hereinafter, the feature registration system for facial recognition of the present invention will be described in detail with reference to the attached drawings.

The feature registration system for facial recognition uses multiple types of features (feature quantities) for the same person with different facial orientations so that they are not affected by the direction of the user's face when facial recognition of the same person is performed multiple times a day. It is a system for registering and is operated in conjunction with the user's personal terminal 100 and the management server 120. The personal terminal 100 is a personal terminal such as a microcomputer, laptop, or smartphone, and is equipped with an arithmetic processing unit (not shown).

The feature registration system for facial recognition of the present invention can be applied to a telework system.

In order to explain authentication using features registered in a feature registration system for facial recognition, the present invention will be described below using an example applied to a work-from-home system.

However, it should be noted that the present invention can be applied anywhere to the technology for frequently rinsing face authentication, and therefore the present invention is not limited to a work-from-home system.

The telecommuting system uses facial recognition technology to detect behavioral events of the telecommuter, such as “sitting down” or “leaving the desk,” or “substitution” or “peeping” by a third party other than the telecommuter. Safety accidents are detected and recorded so that managers can view them.

When a telecommuter starts work, a photo of the telecommuter's face is taken, personal authentication is performed using the features of the photographed face photo and the already registered facial image, and the degree of probability that the teleworker is the teleworker (authentication level) is calculated. Hereinafter, the face image and its feature quantities are collectively referred to as 'feature quantities', and the feature quantities used for personal authentication are referred to as authentication feature quantities.

1 is a configuration diagram schematically showing the configuration of a telework system.

This includes one or more telecommuter personal terminals (100), one or more supervisory terminals (110), one or more management servers (120), a local area network (LAN) (130), a router (140), and the Internet (150). It is connected through.

The teleworker's personal terminal 100 detects the motion events and security incidents of the teleworker used, transmits them to the management server 120, and collects the feature quantities (face image and face image) of the teleworker used to detect the motion event. The characteristic quantity acquired from is transmitted to the management server 120.

The supervisory terminal 110 performs an approval operation of the characteristic quantity recorded in the management server 120 and audits the traces recorded in the management server 120.

The management server 120 records the traces (i.e., the teleworker's operation events and security incidents) received from the telecommuter's personal terminal 100, or processes the audit operation of the supervisory terminal 110 for the traces. .In addition, the feature quantity received from the teleworker's personal terminal 100 is recorded, and the approval operation of the supervisory terminal 110 for the characteristic quantity is processed.

A face authentication system can be built using the teleworker's personal terminal (100) and management server (120).

FIG. 2 is a flowchart showing detection and recording processing of an operation event performed by the teleworker's personal terminal 100 while the user (teleworker) is working from home using a telecommuting system.

2 is performed in the calculation processing unit (not shown) of the telecommuter's personal terminal 100. Figure 2 shows that after registration of the facial image that serves as the standard for face authentication in Figure 3 is completed, the individual's face is authenticated, a WEB camera is used, and motion events that authenticate each individual are detected and recorded for a work-from-home task. This is a flow chart that is carried out during the process.

In the work start step (S401), the user logs in to the teleworker's personal terminal 100, and the teleworker's personal terminal 100 starts the resident application and sends the teleworker's account ID and information to the management server 120 in the background. Log in with your password. The management server 120 compares the information of the teleworker stored in the account storage unit (not shown) and, if it matches, allows login.

In the motion event generation step (S402), whenever a motion event of the teleworker occurs, the possibility that the user (teleworker) using the teleworker's personal terminal 100 is a teleworker registered in the management server 120 is calculated. As a result, in the motion event detection step (S403), the feature quantity used for motion event detection is acquired.

In the operation event generation step (S402), the telework system uses a recording means (WEB camera, etc.) to capture a video image of the face of the person (teleworker) while the teleworker is working. New capture A fourth feature quantity (a feature quantity obtained from face images facing in various directions) is obtained from a still image of each face image. When using the fourth feature quantity to determine whether a person is the same person as the person in the already registered face image, the third feature quantity of the already registered face image is added to the first criterion for discrimination. According to the criteria in 2, it is determined whether the person in the newly compared face image is the same person as the person in the already registered face image (second face authentication means).

The second standard is (the threshold for determining that the registered first feature is similar to the base of the fourth feature of the newly captured image in the motion event occurrence step (S402)) + (the registered third feature) The threshold is expanded to the threshold for judging that the fourth feature quantity of a newly captured image is similar to the feature quantity as the base.

By satisfying the second standard, the same person will be recognized, and even in cases where it is necessary to continuously perform face authentication using a photography means (WEB camera, etc.), even if the timing of the face changes accidentally, the same person will be recognized. It can be recognized as a human with good precision.

In the motion event detection step (S403), motion events (labor motion events such as “seating” and “leaving” or safety accidents such as “substitution” and “peeping”) are detected and the traces are sent to the management server. Send to (120).

In the motion event detection step (S403), the telework system detects event information in real time, which is a movement performed by a person (teleworker) taking pictures using a photographing means (WEB camera) in front of the photographing means (WEB camera). By analyzing the image of the motion content of the person photographed, each motion of the photographed person is repeatedly detected (event detection means). Here, the event detection means detects event information (seating, leaving, substitution, peeking). In this case, event information of the person determined to be the same person by the first face authentication means in S808 is also detected, and event information of the person determined to be the same person by the second face authentication means in the motion event generation step (S402) is also detected. is also detected. This broadens the range of recognition of the same person even if the direction of the face is different, improves the recognition rate of the same person, and properly records event information (seating, leaving) even if the direction of the face is different. , it becomes possible to detect (look into).

In the task completion step (S404), the user (teleworker) logs out of the teleworker's personal terminal 100, and the teleworker's personal terminal 100 logs out of the management server 120 in the background.

Figure 3 is a flowchart showing a procedure for registering a face image that serves as a standard for face authentication in the present invention.

As a preparation before starting work from home, this is a procedure to register a facial image that becomes the standard for face authentication.

After registering the facial image that serves as the basis for face authentication in Figure 3, detection and recording of the motion event in Figure 2 can be performed while working from home.

In the registration list creation step (S601), the feature registration system creates a registration list A that registers the face image that becomes the standard for face authentication.

In the first discriminability determination step (S602), the feature registration system determines whether or not face authentication is possible with the face image already in the registration list A. If face authentication is possible, proceed to the photography method reception step (S604). If face authentication is not possible, proceed to the face image capturing step (S603).

In the face image capturing step (S603), the feature registration system is a procedure for capturing one facial image that becomes the standard for face authentication. Details about this will be described later with reference to FIG. 4 .

In the capturing method reception step (S604), the feature registration system accepts the capturing method.

In the photographing method determination step (S605), the feature registration system determines whether the received photographing method is a method of photographing a single face image or a method of photographing multiple face images. If it is determined that the method is to shoot one image, the process proceeds to the face image shooting step (S603). If it is determined that multiple shots are being taken, the process proceeds to the determination step (S607) to determine whether to register multiple shots.

In the first empty state determination step (S606), the feature registration system determines whether the created registration list A is empty. If it is determined to be empty, the process proceeds to the second empty state determination step (S608). If it is determined that the state is not empty, the process proceeds to the determination step (S607) to determine whether to register multiple pages.

In the determination step (S607) whether to register multiple images, the feature registration system is a method of shooting multiple facial images as videos, which are the standard for face authentication. Details about this will be described later with reference to FIG. 5 .

In the second empty state determination step (S608), the feature registration system determines whether the created registration list A is empty. If it is determined that the state is empty, the process proceeds to the second discriminability determination step (S616). If it is determined that the state is not empty, the process proceeds to the face image registration request step (S609). Since steps S609 to S614 of FIG. 3 are not directly related to the present invention, their description is omitted here.

Figure 4 is a flowchart showing a procedure for photographing one face image that serves as a standard for face authentication in the present invention. This is a procedure for capturing one facial image, which is the standard for face authentication in the face image capturing step (S603).

In the cancellation determination step (S701), the feature registration system determines whether or not it is a cancellation. If it is a cancellation, it ends. If it is not a cancellation, it proceeds to the shooting time reception step (S702).

In the shooting time acceptance step (S702), the feature registration system accepts the shooting time (timing) when the user presses the shooting button (not shown) on the screen.

In the face image acquisition step (S703), the feature registration system captures a face image using a WEB camera 3 seconds after the user presses the button, and acquires one face image of the user (teleworker) sitting in front of the personal terminal.

In the acquisition success judgment step (S704), the feature registration system determines whether or not it has succeeded in acquiring one face image. If it is successful in acquiring one face image, the process proceeds to the image display processing method reception step (S706). If one face image cannot be acquired, the process proceeds to the failure notification step (S705).

In the failure notification step (S705), the feature registration system notifies the user (teleworker) sitting in front of the personal terminal that acquisition of one face image has failed.

In the image display processing method reception step (S706), the feature registration system displays the acquired face image of the user (teleworker) and receives a processing method for the face image (select one of registration, rephotography, and cancellation). do.

In the received processing method determination step (S707), the feature registration system determines whether the received processing method is registered or not, and if so, proceeds to the registration list addition step (S708). In case of re-shooting or cancellation, the process proceeds to the shooting time reception step (S702).

In the registration list addition step (S708), the feature registration system adds and preserves one acquired face image to registration list A. Thereafter, the first face authentication means registers the first feature quantity acquired from the face image additionally saved by the user for use in authentication.

Figure 5 is a flowchart showing a procedure for shooting multiple facial images that serve as a standard for face authentication in video format in the present invention. That is, Figure 5 shows a procedure for shooting multiple facial images as videos, which are the standard for face authentication in the determination step (S607) of whether to register multiple images.

In the cancellation determination step (S801), the feature registration system determines whether or not it is a cancellation, and if it is a cancellation, it ends. If it is not a cancellation, it proceeds to S802.

In the temporary discriminator creation step (S802), the feature registration system creates a registration list B of the temporary discriminator.

In the projection period acceptance step (S803), the feature registration system accepts the shooting time of the video (for example, 5 seconds).

In the continuous shooting step (S804), the feature registration system uses a shooting means (WEB camera, etc.) installed near the display unit of the feature registration system to change the direction of the user's (teleworker's) face within a predetermined time (e.g. Multiple face images of the same person (telecommuter) are captured in succession (for example, for about 5 seconds). In addition, it is necessary to obtain a second feature quantity from each of the multiple photographed face images of the same person (teleworker), so they need to be still images.

Also, here, using a shooting means, you may shoot a video within a certain period of time (for example, about 5 seconds) so that the direction of the user's (teleworker's) face is different in the video. If it is a video, the user (teleworker) You can easily obtain multiple still images of the same person (telecommuter) with different orientations of the user's face, without placing a load on the computer.

In the non-searched image determination step (S805), the feature registration system determines whether there is an unretrieved face image in each face image of the same person (teleworker) in multiple images, and if so, the feature registration system determines whether there is a face image in the face image search step (S806). ), and if there is none, proceed to the unidentified image judgment step (S807).

In the face image search step (S806), the feature registration system searches for one face image of the same person (teleworker) among multiple face images of the same person (teleworker) taken continuously. The S805~S806 loop is a loop that searches for and cuts out all face images (still images) from the captured video.

In the unidentified image judgment step (S807), the feature registration system determines whether there is a face image of the same person (telecommuter) among the multiple searched face images of the same person (teleworker). If there is one, the process proceeds to the face image determination step (S808). If not, the process proceeds to the candidate display processing method reception step (S811).

In the face image discrimination step (S808), the feature registration system uses the first feature amount (a feature obtained from a face image facing the front) of the face image already registered in the registration list addition step (S708)8 and continuous shooting. In step S804, the person in the newly photographed face image is identified according to the first standard that can determine whether the second feature quantity (feature quantity acquired from a face image that is not facing the front) of the newly photographed face image is similar. , It is determined whether the person is the same as the person in the already registered face image (first face authentication means).

The first standard is (a threshold for determining that the registered first feature value is similar to the base and the second feature value of the image newly captured in the continuous shooting step (S804)).

If the first face authentication means determines that the person is the same person as the person in the already registered face image in the determination step (S809) to identify the person as the person, the face image is added and stored in the registration list of the temporary discriminator. It doesn't work. Conversely, in the first face authentication means, if it is determined that the person is not the same person as the person in the already registered face image, the registration candidate list addition step (S810) is performed, and the face image is added to the registration list of the temporary discriminator. are additionally preserved.

In the registration candidate list addition step (S810), the feature registration system uses the first standard to select a face from among multiple (for example, about 20) face images of the same person taken in the continuous shooting step (S804). The first face authentication method in the image discrimination step (S808) does not determine that it is the same person as the person in the already registered face image (it is the same person, but it is not facing the front, so the first criterion is that it is the same person). is not recognized), extract several (e.g., about 3) face images of the same person with different facial orientations (extraction means).

That is, because the direction of the face has changed too much, it is determined that the feature quantity acquired from the selected face image is not similar to the first feature quantity.

In addition, here, the extraction means is a plurality of face images (one or more) of the same person with different facial directions (e.g., facing up, facing down, facing left, facing right). Extract. The extracted face images of the same person are added and stored in the registration list B of the temporary discriminator.

Face images of the same person that are additionally stored in the temporary discriminator's registration list are displayed in a list on the screen.

In the candidate display processing method reception step (S811), the feature registration system identifies and displays the face image selected by the user and receives a processing method for the face image (selection of one of registration, rephotography, and cancellation).

In the processing method determination step (S812), the feature registration system determines whether or not it is registered. In the case of registration, the feature registration system proceeds to the registration list addition step (S813). In the case of reshooting or cancellation, the feature registration system proceeds to the cancellation determination step (S801). do.

In the registration list addition step (S813), the feature registration system adds and preserves the selected face image to registration list A.

In order for the second face authentication means to be used for authentication, here, the feature quantity acquired from the face images manually selected by the user is the third feature quantity (the feature quantity acquired from the face image that is not facing the front), and the first feature quantity is also used for authentication. As a standard, it is additionally registered as a characteristic quantity that is not recognized as the same person.

In the present invention, face image information is information for identifying each face image stored for each face image, and includes user ID, shooting date and time, and face image data. Again. In the present invention, face image registration information includes user ID, shooting date and time, face image list, approval status, and approver.

As described above, although the present invention has been described with reference to limited embodiments and drawings, the present invention is not limited to the above embodiments, and those skilled in the art can make various modifications and modifications from these descriptions. Transformation is possible. Accordingly, the spirit of the present invention should be understood only by the scope of the claims stated below, and all equivalent or equivalent modifications thereof shall fall within the scope of the spirit of the present invention.

100: Personal terminal 110: Supervisor terminal 120: Management server
130: Local area network 140: Router 150: Internet network

Claims (6)

According to a first standard that can determine whether the first feature amount of the registered face image and the second feature amount of the newly captured face image are similar, the person in the newly captured face image is compared to the registered face image. A first face authentication means for determining whether the person is the same person,
Photographing means for photographing a plurality of facial images of the same person photographed sequentially within a predetermined time so that the orientation of the face is different;
Extraction means for extracting, from the plurality of photographed face images of the same person, facial images of the same person with different facial orientations that are not judged to be the same person as the person in the face image registered in the first face authentication means. class,
According to the second standard that adds the third feature amount of the extracted face image to the first feature amount of the registered face image, the person in the face image to be newly captured is a face image with the face in a different direction. A feature registration system for facial recognition, comprising a second face authentication means for determining whether the extracted person is the same person. According to paragraph 1,
A feature registration system for facial recognition, further comprising guide means for guiding the user to move the face in a different direction while the photographing means is photographing the face. According to paragraph 2,
further comprising event detection means for repeatedly detecting event information corresponding to an operation performed by a person photographed using the photographing means in front of the photographing means for each motion;
A feature registration system for facial recognition, wherein when the event detection means detects the event information, the second face authentication means detects the event information of the person determined to be the same person. According to any one of claims 1 to 3,
Event recording means for repeatedly recording event information corresponding to an operation performed in front of the photographing means by a person photographed using the photographing means for each motion;
A feature registration system for facial recognition, wherein when the event recording means records genital event information, the second face authentication means also records event information of the person determined to be the same person. According to the first standard for determining whether the registered face image and the newly captured face image are similar, the first criterion for determining whether the person in the newly captured face image is the same person as the person in the registered face image a face authentication means,
Photographing means for photographing a plurality of facial images of the same person photographed sequentially within a predetermined time so that the orientation of the face is different;
From the plurality of photographed face images of the same person, extraction means for extracting face images of the same person with different facial orientations, without determining that the first face authentication means is the same person as the person in the face image registered. class,
A second face that determines whether the person in the newly captured face image is the same person as the person from whom the face image with a different face direction was extracted, according to a second standard that adds the extracted face image to the registered face image. A feature registration system for facial recognition, characterized in that it has an authentication means. According to the first standard for determining whether the registered face image and the newly captured face image are similar, the first criterion for determining whether the person in the newly captured face image is the same person as the person in the registered face image a face authentication means,
Photographing means for photographing a plurality of facial images of the same person photographed sequentially within a predetermined time so that the orientation of the face is different;
Extraction means for extracting, from the plurality of photographed face images of the same person, facial images of the same person with different facial orientations that are not judged to be the same person as the person in the face image registered in the first face authentication means. class,
A second face that determines whether the person in the newly captured face image is the same person as the person from whom the face image with a different face direction was extracted, according to a second standard that adds the extracted face image to the registered face image. A feature registration system for facial recognition, characterized in that it has an authentication means.