JP7133949B2 - Face authentication method and device - Google Patents

Description

The present invention relates to face recognition technology.

Face authentication technology, which is one of biometric authentication technologies, is an authentication technology that determines whether a user is a valid user based on the face shown in a still image or moving image. Face recognition technology has the advantage of being able to confirm the person to be authenticated in a non-contact manner.

An object of the present invention is to provide face recognition technology.

A face authentication method according to one embodiment comprises steps of selecting a current authentication mode to be performed for face authentication from among a plurality of authentication modes, performed by a processor; determining one or more recognizers to be used for face authentication based on the mode; and extracting feature information from face information using at least one of the determined one or more recognizers. and displaying the presence or absence of authentication success based on the extracted feature information.

The face authentication method further includes obtaining a recognizer based on determinations of one or more recognizers, and extracting the feature information includes the Applying image information from the determined facial region of the image to the obtained recognizer.

The face recognition method includes the step of performing a first normalization of the determined face region of the image to produce the image information in a predetermined form suitable for application to the acquired recognizer. and the obtained recognizer may be a trained neural network or a machine learning model.

The face recognition method performs the following operations on the determined face region of the image to generate each image information in a predetermined form suitable for application to different acquired recognizers based on the current authentication mode. further comprising a step of normalizing each, wherein the step of extracting the feature information comprises extracting each feature information from each piece of video information using the different acquired recognizers, and determining whether the authentication is successful; The displaying may include displaying whether the authentication was successful based on at least one of the respective extracted feature information.

Each of the steps of normalizing may include synthesizing image information regarding occluded or expected occluded areas in the determined facial region of the image.

The face authentication method further includes the step of performing face authentication, wherein the step of performing face authentication is the step of performing first authentication based on first feature information extracted from each of the extracted feature information. and selectively performing a second authentication based on second feature information extracted from each of the extracted feature information; and depending on whether or not the second authentication is selectively performed. selectively performing third authentication based on the extracted first feature information and the extracted second feature information; and determining whether the face authentication is successful based on the result of the third authentication. and determining.

The step of extracting feature information includes extracting respective feature information from each normalization of a determined face region of a video using a different determined recognizer based on the current authentication mode; The step of extracting feature information may be selectively and partially performed by giving a weight to one of usability and security of the face authentication based on the current authentication mode.

determining the one or more recognizers comprises determining a plurality of different recognizers for the face recognition; performing the face authentication based on

The step of performing face authentication may include selectively performing respective authentication procedures corresponding to the different recognizers based on the current authentication mode.

In the face authentication method, each of the authentication procedures may include a step of comparing similarities between the extracted feature information and the corresponding registered information with corresponding thresholds.

The plurality of different recognizers includes a first neural network model recognizer and a second neural network model recognizer, wherein the first neural network model recognizer and the second neural network model recognizer use training data to: respectively, wherein the first neural network model recognizer performs low-light facial feature extraction for face recognition in a low-light environment, and the second neural network model recognizer trains a face region including one or more occluded regions; can be learned with training data for

The step of extracting the feature information includes obtaining the first feature information from image information of the determined face region of the image using a first recognizer, and the face recognition method further comprises the step of performing face recognition. and determining a first authentication result based on the first feature information; and performing a second recognition using a second recognizer based on the first authentication result and the current authentication mode. and determining whether to extract bi-characteristic information, wherein the plurality of authentication modes may include at least one of a first authentication mode and a second authentication mode.

The step of extracting the feature information includes determining a determined facial region of another image or the determination of the image using the second recognizer in response to a determination of whether to extract the second feature information. extracting the second feature information from the video information or other video information from the face region, wherein determining whether to extract the second feature information includes extracting the second feature information. The step of performing face authentication includes the step of determining a second authentication result based on the second feature information, and the step of determining whether or not the authentication is successful based on the second authentication result. can contain.

The step of performing face authentication includes determining a final authentication result as failure in response to the current authentication mode selected as a first authentication mode, in response to a case where the second authentication result is a failure. and determining the final authentication result as successful in response to the second authentication result being successful.

In the step of performing face authentication, the second feature information is extracted in response to the current authentication mode selected as the second authentication mode and in response to the first authentication result being successful. and determining the final authentication result as successful in response to the second authentication result being successful.

In the step of performing face authentication, if the current authentication mode is selected as a first authentication mode and if the first authentication result is authentication failure, the step of determining to extract the second feature information; determining a final authentication result as successful authentication and ending the face authentication procedure if the first authentication result is successful authentication.

determining a final authentication result as authentication failure in response to the current authentication mode selected as the second authentication mode, if the first authentication result is authentication failure, and ending the face authentication procedure; determining to extract the second characteristic information if the first authentication result is successful authentication.

The step of determining the first authentication result further includes determining the first authentication result based on a comparison of the first feature information and first registered feature information extracted in advance using the first recognizer. can contain.

The face authentication method may optionally further include updating the first registered feature information using the first feature information when the first authentication result is successful authentication.

The step of determining the second authentication result may determine the second authentication result by comparing the second feature information and the second registered feature information extracted using the second recognizer.

The face authentication method may optionally further include updating the second registered feature information using the second feature information when the second authentication result is successful authentication.

The step of determining the first authentication result includes calculating a degree of similarity between the first feature information and the first registered feature information extracted using the first recognizer; determining the first authentication result based on a comparison result between the first threshold value obtained.

The step of extracting the feature information comprises: detecting a face area from an input video; detecting landmarks of the face from the detected face area; and detecting the face area based on the detected landmarks. and normalizing to generate said information of said face.

A computer-readable recording medium can record a program for executing the method on a computer.

A face authentication device according to one embodiment includes at least one processor, wherein the at least one processor selects a current authentication mode to be performed for face authentication from among a plurality of authentication modes, and selects the selected current authentication mode. determining one or more recognizers among a plurality of recognizers based on an authentication mode; extracting feature information from face information using the determined one or more recognizers; and extracting the extracted feature information It is configured to display the presence or absence of authentication success based on.

The one or more processors obtain a recognizer based on determination of the one or more recognizers, and the one or more processors obtain the obtained feature information to generate the extracted feature information. A recognizer may be applied with image information from the determined facial region of the image.

The one or more processors perform a first normalization of the determined facial region of the video to produce the video information in a predetermined form suitable for application to the obtained recognizer; The obtained recognizers can be trained neural networks or machine learning models.

The one or more processors are configured to transform the determined face region of the image to generate each image information in a form predetermined to be suitable for application to different acquired recognizers based on the current authentication mode. and extracting respective feature information from each image information using the different acquired recognizers, and the authentication is successful based on at least one of the extracted respective feature information. It can indicate whether or not

The one or more processors may synthesize image information regarding occlusion areas or expected occlusion areas in the determined facial regions of the image for at least one of the respective normalizations.

The one or more processors perform first authentication based on first feature information extracted from each of the extracted feature information, and second features extracted from each of the extracted feature information. selectively performing a second authentication based on information, and depending on whether said second authentication was selectively performed, based on said extracted first characteristic information and said extracted second characteristic information; selectively perform a third authentication, and determine whether the face authentication is successful based on the result of the third authentication.

The one or more processors extract respective feature information from each normalization of the determined face region of the video using different determined recognizers based on the current authentication mode; may be selectively and partially performed by giving a weight to one of usability and security of the face authentication based on the current authentication mode.

The one or more processors determine a plurality of different recognizers for the face recognition based on the current authentication mode, and select combinations and/or arrangements of the different recognizers depending on the current authentication mode. may be configured to perform said authentication based on:

The one or more processors may be configured to selectively perform respective authentication procedures corresponding to the different recognizers based on the current authentication mode.

The face authentication device compares the similarities between the extracted feature information and the corresponding registered information by the one or more processors so that the authentication procedures are different from each other from corresponding thresholds. can contain.

The one or more processors extract first feature information from video information derived from a determined facial region of a video using a first recognizer to perform the feature information extraction and perform the authentication. determining a first authentication result based on the first feature information, and whether or not to extract second feature information using a second recognizer based on the first authentication result and the current authentication mode. and the plurality of authentication modes may include at least one of a first authentication mode and a different second authentication mode.

The at least one processor, in response to determining whether to extract the second feature information, uses the second recognizer to determine the determined facial region of another image or the determined facial region of the image. extracting the second feature information from other video information from the face region or from the video information, and determining whether or not to extract the second feature information is a decision to extract the second feature information; The authentication may be performed by determining a second authentication result based on the second characteristic information, and whether or not the authentication is successful may be determined based on the second authentication result.

The at least one processor determines a final authentication result as authentication failure in response to the second authentication result being authentication failure when the current authentication mode is selected as the first authentication mode, and It may be configured to determine the final authentication result as successful authentication in response to the second authentication result being successful authentication.

The at least one processor determines to extract second characteristic information in response to the first authentication result being authentication failure when the current authentication mode is selected as the second authentication mode. , determining a final authentication result as successful authentication in response to the second authentication result being successful authentication.

The one or more processors extract the second characteristic information in response to the current authentication mode being selected as a first authentication mode and in response to the first authentication result being authentication failure. and the first authentication result is authentication success, the final authentication result is determined as authentication success, and the face authentication procedure is terminated.

The one or more processors determine a final authentication result as authentication failure when the first authentication result is authentication failure in response to the current authentication mode being selected as a second authentication mode. It may be configured to determine to extract the second feature information when an authentication procedure is terminated and the first authentication result is successful authentication.

The one or more processors may be configured to compare the first feature information and first registered feature information previously extracted using the first recognizer to determine the first authentication result.

The one or more processors may be selectively configured to update the first registered characteristic information with the first characteristic information if the first authentication result is a successful authentication.

The one or more processors may be configured to compare the second feature information and second enrolled feature information previously extracted using the second recognizer to determine the second authentication result.

The one or more processors may be selectively configured to update the second registered characteristic information using the second characteristic information if the second authentication result is a successful authentication.

The one or more processors calculate a similarity between the first feature information and the first registered feature information extracted in advance using the first recognizer, and calculate the similarity and a preset first threshold value. may be configured to determine the first authentication result based on a comparison result between and.

The one or more processors detect facial regions from an input video, detect facial landmarks from the detected facial regions, and normalize the facial regions based on the detected landmarks. can be configured.

A computing device according to one embodiment includes a first recognizer including a neural network model for extracting first feature information from an input video in which a user's face region is indicated, and a first recognizer different from the first feature information from the input video. a second recognizer including a different neural network model for extracting second feature information; a first authentication result comparing the extracted first feature information with the registered first registered feature information; and one or more processors for determining successful face authentication if at least one of second authentication results comparing the second feature information and the registered second registered feature information is successful.

The neural network model of the first recognizer and the different neural network model of the second recognizer may be learned based on different learning data.

The different neural network model of the second recognizer may be learned based on training data that is expected to include or be expected to include an occlusion region in the learning face region.

An image that replaces an area in which an occluded area is expected to exist from the face area with image information of a corresponding area of an average image, an average value image, or a single color image is used for extracting the second feature information. Different neural network models of the second recognizer may be input.

The area where the occlusion area is expected to exist may be an area where occlusion is indicated by a combination of one or more of glasses, sunglasses, a hat, or a mask from the face area.

The one or more processors release a lock mode of the computing device if the face authentication is successful, and in response if the first authentication result and the second authentication result are authentication failures, The one or more processors do not retain lock mode or release lock mode.

A computing device according to one embodiment includes a first recognizer including a neural network model for extracting first feature information from an input video in which a user's face region is indicated, and a first recognizer different from the first feature information from the input video. a second recognizer including a different neural network model for extracting second feature information; a first authentication result comparing the extracted first feature information with the registered first registered feature information; and one or more processors determining that the face authentication fails if at least one of second authentication results comparing the second feature information and the registered second registered feature information fails.

The neural network model of the first recognizer and the different neural network model of the second recognizer may be learned based on different learning data.

The different neural network model of the second recognizer may be trained based on learning data in which an occluded region exists or is expected to exist in the learning face region.

An image that substitutes image information of a corresponding area of an average image, an average value image, or a single color image for an area in which an occluded area is expected to exist from the face area is used to extract the second feature information. Different neural network models of the second recognizer may be input.

The area where the occlusion area is expected to exist may be an area where occlusion is indicated by a combination of one or more of glasses, sunglasses, a hat, or a mask from the face area.

If the face authentication fails, the face authentication device determines an authentication result in a financial service or payment service using the computing device as an authentication failure, and the first authentication result and the second authentication result are authenticated. In response to being successful, the one or more processors determine the authentication result as successful authentication with the payment service or the financial service.

The computing device further includes a transceiver, and when the authentication result is determined to be successful in the payment service or the financial service, the one or more processors cause the transceiver to transmit payment information to an external terminal. perform financial transactions with the financial service; and/or provide financial information to a user interface with the financial service.

The computing device comprises a first recognizer including a neural network configured to extract first feature information from an input video including a facial region of a user; and second feature information different from the first feature information. a second recognizer comprising a different neural network configured to extract from an input video; and a current authentication mode executed to authenticate the user from one of a first authentication mode and a second authentication mode. and one or more processors configured to determine, when the current authentication mode is determined as the first authentication mode, the one or more processors determine between a first authentication result and a second authentication result determining successful face recognition in response to at least one successful authentication, wherein said first authentication result is obtained by comparing said first characteristic information with first registered characteristic information; An authentication result is obtained by comparing the second characteristic information with the second registered characteristic information, and if the current authentication mode is determined as the second authentication mode, the one or more processors perform the first authentication Determining face authentication as unsuccessful in response to at least one of the result and the second authentication result being unsuccessful.

The first authentication mode is a predetermined unlock mode, and the unlock mode is arranged to be automatically performed by one or more processors when the computing device is in a locked state, or the user may be determined to be executed by the one or more processors when attempting to release the user interface of the computing device.

The second authentication mode is a predetermined payment mode, and the payment mode is to be executed by the one or more processors when the user accesses or selects a payment service of the computing device. can be determined.

According to the present invention, face authentication technology can be provided.

FIG. 4 is a diagram for explaining face authentication according to one embodiment; 4 is a flowchart for explaining a face authentication method according to one embodiment; 4 is a flowchart for explaining face registration procedures and authentication procedures according to an embodiment; FIG. 10 is a diagram illustrating an example of a face registration procedure according to one embodiment; 6 is a flowchart for explaining an authentication method in a first authentication mode according to one embodiment; 4 is a flowchart for explaining an unlock mode among authentication modes according to an embodiment; 9 is a flowchart for explaining an authentication method in a second authentication mode according to another embodiment; 8 is a flowchart for explaining a payment mode among authentication modes according to another embodiment; It is a figure which shows the structure of the face authentication apparatus which concerns on one Embodiment. 1 illustrates an example computing device, according to one embodiment. FIG. It is a figure which shows an example of the learning apparatus which concerns on one Embodiment.

Any specific structural or functional descriptions of embodiments according to the inventive concepts disclosed herein are presented merely for purposes of describing the embodiments according to the inventive concepts. , embodiments in accordance with the inventive concepts may be embodied in various forms and are not limited to the embodiments set forth herein.

Terms such as first or second may be used to describe multiple components, but such terms should be construed only for the purpose of distinguishing one component from another. For example, a first component can be named a second component, and similarly a second component can be named a first component.

Singular expressions include plural expressions unless the context clearly dictates otherwise. As used herein, the terms "including" or "having" indicate the presence of any of the features, numbers, steps, acts, components, parts, or combinations thereof described in the specification. , one or more other features, figures, steps, acts, components, parts, or combinations thereof, are not to be precluded.

Unless defined otherwise, all terms, including technical or scientific terms, used herein are to be commonly understood by one of ordinary skill in the art to which the embodiments belong. have the same meaning. Commonly used pre-defined terms are to be construed to have a meaning consistent with the meaning they have in the context of the relevant art, and unless expressly defined herein, are ideally or excessively It is not to be interpreted in a formal sense.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Identical reference numerals appearing in the drawings indicate identical parts.

FIG. 1 is a diagram for explaining face authentication according to one embodiment.

Face recognition is a verification method that determines whether the user is a valid user based on the face of the user attempting verification. Referring to FIG. 1, the face recognition device is included and operates in a computing device 120 . Computing device 120 may be, for example, a smart phone, wearable device, tablet computer, netbook, laptop, desktop, personal digital assistant (PDA), set-top box, consumer electronics, biometric door lock, security device, or vehicle starter. obtain.

The facial recognition device authenticates the face of the user 110 to determine if the service performed via the computing device 120 can be used. For example, the face authentication device acquires an input image of the face of the user 110 using the camera 130 included in the computing device 120 . Camera 130 may be, for example, a video capture device such as a digital still camera or a video camera. The input video is a video that is input to the face authentication device and is a target video for face authentication.

The face recognition device detects a face region 140 from an input image and extracts features from the face region 140 using a plurality of recognizers. A recognizer is a model that outputs features corresponding to input information, and may be, for example, a neural network model trained using training data.

The face authentication device compares the features extracted by recognition with the features registered through the face registration procedure, and determines whether or not the authentication is successful based on the comparison result. A valid user 110 pre-registers his/her face with a computing device 120 via a face registration procedure, and the computing device 120 stores characteristics of the valid user 110 determined via the face registration procedure.

According to one embodiment, a face recognition apparatus, which performs face recognition, can improve the accuracy of face recognition for input images under various conditions by using a plurality of recognizers. The face authentication device can perform face authentication more efficiently by setting the relationship between the authentication results of the recognizer according to the purpose of the authentication mode.

There are an authentication mode in which usability is more important than security and an authentication mode in which security is more important than usability. For example, the former authentication mode is used in authentication situations such as user login and lock release, and the latter authentication mode is used in authentication situations such as payment and financial services. The required authentication strength may differ depending on the authentication mode. The face authentication method or face authentication device described below can flexibly determine the authentication strength according to the purpose of the authentication mode. If face recognition is performed with an authentication strength suitable for the purpose of authentication, convenience of use and security, which are conflicting with respect to the authentication strength, are all satisfied.

FIG. 2 is a flowchart for explaining a face authentication method according to one embodiment.

Referring to FIG. 2, at step 201, the face recognition device identifies the current verification mode. Authentication modes include modes that satisfy various authentication strengths between usability and security depending on the purpose of use of the face authentication device. For example, the authentication mode includes an unlock mode emphasizing usability or a payment mode emphasizing security.

At step 203, the face authentication device determines at least one recognizer to be used for face authentication based on the current authentication mode. The face authentication device determines a plurality of recognizers to be used for face authentication, and determines a combination of recognizers to be used for face authentication based on the current authentication mode.

Each of the recognizers has strong characteristics according to the conditions by learning from learning data shot under various conditions. A face authentication device can more accurately authenticate a user using input images captured under various conditions by using a plurality of recognizers. A recognizer-based authentication procedure may be determined based on the current authentication mode.

At step 205, the face recognition device acquires feature information from the face region shown in the input image using the determined at least one recognizer. At step 207, the face authentication device determines whether or not the authentication is successful based on the feature information.

A face authentication device detects a face area from an input video. For example, the face recognition device may detect a face region using a Haar-based cascade adaboost classifier, a Viola-Jones detector, or the like. However, the scope of the embodiments is not limited to this, and the face recognition device can detect face regions using various face region detection schemes.

Although not shown in the drawings, according to an embodiment, the face authentication device may first perform image pre-processing on the received input image. The image preprocessing process includes one or more processes for processing the input image in a suitable form for face recognition. For example, the preprocessing process includes adjusting the size of the input image, rotating the input image, removing noise contained in the input image, increasing the contrast of the input image, and increasing the contrast of the input image. a deblurring process for removing blurring, a process for removing a background area, a warping process for correcting distortion contained in the input image, a process for cropping a specific area in the input image, or a process for binarizing the input image. and so on. When image pre-processing is performed, 'input image' below may be replaced with 'input image on which image pre-processing has been performed'.

The face authentication device acquires first feature information from video information of the face area using the first recognizer. When the face region has undergone the normalization process, the face authentication apparatus acquires first feature information from the image information of the face region normalized using the first recognizer. The first recognizer may be a neural network model pre-trained with training data. The first feature information is in the form of a feature vector, and the image information indicates pixel values of pixels included in the image.

The face authentication device determines a first authentication result based on the first feature information. The face authentication device compares the first feature information with first registered feature information registered in advance, and determines the first authentication result based on the comparison result. The face authentication device determines the first authentication result as authentication success or authentication failure according to the comparison result. For example, if the degree of similarity between the first feature information and the first registered feature information is greater than a preset threshold, it is determined that authentication has succeeded, and if the degree of similarity is equal to or less than the threshold, it is determined that authentication has failed. be done.

The face authentication device decides whether to continue the additional authentication process. The face authentication device determines whether or not to acquire the second feature information using the second recognizer based on the first authentication result and the authentication mode. The authentication mode is predetermined according to the authentication status. Authentication modes include, for example, authentication modes where it is important to prevent False Rejection (FR), such as unlocking, and false acceptance (FA), such as payment and financial services. Includes important authentication modes. Here, error rejection means judging a valid user, and error recognition means authenticating an unauthorized user as a valid user.

For example, in the case of unlocking a smartphone, if most of the users who attempt facial recognition on the smartphone are likely to be legitimate users, it is more important than security to prevent erroneous rejection in terms of usability. is. In this case, face authentication is performed based on low authentication strength. For example, if even one of a plurality of authentication results obtained by different recognizers is determined as successful authentication, the final authentication result is determined as successful authentication. On the other hand, in the case of payment services, high security to prevent authentication of unauthorized users is more important than usability. In this case, face authentication is performed based on high authentication strength. For example, if any one of a plurality of authentication results obtained by different recognizers is determined to be an authentication failure, the final authentication result is determined to be an authentication failure. However, the authentication status and authentication mode described above are merely examples, and various forms of authentication modes combining the authentication results of a plurality of recognizers can exist.

The face recognition device obtains the second feature information from the image information of the face region using the second recognizer in response to the determination to obtain the second feature information. The second recognizer can also be a neural network model pre-trained with training data in the same way as the first recognizer. According to one embodiment, the second recognizer may be different than the first recognizer. The first recognizer and the second recognizer can exhibit different authentication performance depending on conditions. For example, the first recognizer may have a higher recognition rate than the second recognizer with respect to an input image captured in a low-illumination environment, and the second recognizer may detect an occlusion region by hiding the face in the face region. ) may be higher than that of the first recognizer.

The face authentication device determines a second authentication result based on the second feature information. The face authentication device compares the second feature information with second registered feature information registered in advance, and determines the second authentication result based on the comparison result. The face authentication device determines the second authentication result as authentication success or authentication failure according to the comparison result. For example, if the degree of similarity between the second feature information and the second registered feature information is greater than a preset threshold, it is determined that authentication has succeeded, and if the similarity is equal to or less than the threshold, it is determined that authentication has failed. be done.

The face authentication device determines a final authentication result based on the second authentication result. For example, if the second authentication result is authentication success, the final authentication result is also determined as authentication success, and if the second authentication result is authentication failure, the final authentication result is also determined as authentication failure.

According to an embodiment, if the face recognition device uses more than two recognizers, the final recognition result is determined as the recognition result performed using the last recognizer.

FIG. 3 is a flowchart for explaining face registration procedures and face authentication procedures according to one embodiment.

According to one embodiment, a face authentication procedure 320 is performed after the face registration procedure 310 . In the face registration procedure 310, after a face area is detected from the image 311 for face registration, facial landmarks are detected from the detected face area.

The detected face regions are normalized based on the landmarks to suit each recognizer input. For example, a normalized image 312 is generated by normalizing the face region to suit the input of the first recognizer. Also, a normalized image 313 is generated by normalizing the face region so that it is suitable for input to the second recognizer.

A first enrollment feature is extracted 314 from the normalized image 312 using the first recognizer. For example, if the first recognizer is a recognizer that is resistant to low illumination, the first recognizer compensates for the decrease in recognition rate due to low illumination. Also, a second enrolled feature is extracted 315 from the normalized image 313 using a second recognizer. For example, if the second recognizer is a recognizer that is resistant to areas occluded by eyeglasses, the second recognizer compensates for a decrease in recognition rate due to occluded areas. The registered features extracted through the above process are stored 316 in a database or the like.

In the face authentication procedure 320, the face authentication device receives an input image 321 to be authenticated from the image acquisition device. In face authentication procedure 320 , the process of extracting feature information may be similar to face registration procedure 310 . The face recognition device detects a face area from an input image 321, normalizes it to be suitable for input to the first recognizer, inputs the normalized image 322 to the first recognizer, and extracts 324 first features. The face recognition device detects a face region from an input image 321, normalizes it to be suitable for input to the second recognizer, inputs the normalized image 323 to the second recognizer, and extracts 325 second features.

The face authentication device determines 326 whether or not the authentication is successful using the extracted first and/or second features. The face authentication device determines a first authentication result based on a comparison result between the first feature and the first registered feature stored in the database, and compares the second feature with the second registered feature stored in the database. A second authentication result is determined based on the comparison result between. A final authentication result is determined based on one or both of the first authentication result and the second authentication result, depending on the authentication mode. For example, the final authentication result is determined according to the first authentication result, in which case the process for determining the second authentication result may not be performed.

FIG. 4 is a diagram illustrating an example of a face registration procedure according to one embodiment.

FIG. 4 is a more specific version of the face registration procedure shown in FIG. 3, and the specific technical elements are merely examples. Referring to FIG. 4, a valid user image 401 is input for face registration. A face area is detected 403 from the image 401, and landmarks are detected 405 from the detected face area. In one embodiment, the face region may be normalized 411, 421 to suit each recognizer input based on the detected landmarks.

The face recognition device performs a first normalization 411 on the face region to make it suitable for input to a first recognizer 417 . The first recognizer 417 receives the image information of the normalized face area 413 and outputs a first registered feature 419 based on the input image information. The face recognition device performs a second normalization 421 on the face region to make it suitable for input to a second recognizer 427 . A second recognizer 427 outputs a second registered feature 429 based on the input image information of the normalized face area 423 .

The first registered features 419 extracted by the first recognizer 417 and the second registered features 429 extracted by the second recognizer 427 are stored in the database 430 . In this manner, enrollment features may be obtained that are used to determine the presence or absence of face authentication via the face enrollment procedure.

FIG. 5 is a flowchart for explaining an authentication method in the first authentication mode according to one embodiment.

In one embodiment, the first authentication mode is a mode in which easy authentication of valid users is more important than security. For example, the first authentication mode may be an authentication mode applied to unlock a smart phone or the like. In this case, if any one of the plurality of authentication results in the first authentication mode is determined as successful authentication, the face authentication device determines the final authentication result as successful authentication, and terminates the face authentication procedure.

In step 510, the facial recognition device performs a first verification process. In the first authentication process, the face authentication device extracts first feature information from the input image using the first recognizer. As described with reference to FIG. 2, the face recognition apparatus detects a face area from an input image, detects landmarks from the detected face area, and performs a normalization process based on the detected landmarks. A face authentication device extracts first feature information from the normalized face region.

In step 515, the face recognition device determines whether the first recognition process is successful. The face authentication device determines whether or not the first authentication process is successful based on a comparison result between the first registered feature information stored in the database and the extracted first feature information. The face authentication device determines whether to acquire the second feature information using the second recognizer based on the result of the first authentication process and the authentication mode.

If the result of comparison between the first registered feature information and the first feature information satisfies a preset condition, the face authentication device determines 540 that the authentication result of the first authentication process is authentication success, and performs the face authentication procedure. finish. Here, the authentication result may be the final authentication result. If the comparison result between the first registered feature information and the first feature information does not satisfy the preset condition, the face authentication device determines the authentication result as authentication failure and performs the second authentication process. As described above, the face authentication device determines to acquire the second feature information when the authentication result of the first authentication process is authentication failure, and when the authentication result of the first authentication process is authentication success, the final authentication result is determined as authentication success, and the face authentication procedure ends.

At step 520, the face recognition device performs a second verification process. In the second authentication process, the face authentication device extracts second feature information from the input image using a second recognizer. The face recognition device processes the input image to be suitable for input of the second recognizer in a manner similar to that of the first recognizer, and then inputs the processed input image to the second recognizer.

At step 525, the face recognition device determines whether or not the second verification process is successful. The face authentication device compares the second registered feature information stored in the database with the extracted second feature information to determine whether or not the second authentication process has been successful. The face authentication device determines whether or not to acquire the third feature information using the third recognizer based on the authentication result of the second authentication process and the authentication mode.

If the comparison result between the second registered feature information and the second feature information satisfies the preset condition, the face authentication device determines 540 that the authentication result is authentication success, and ends the face authentication procedure. Here, the authentication result may be the final authentication result. If the result of comparison between the second registered feature information and the second feature information does not satisfy a preset condition, the face authentication device determines the authentication result as authentication failure. If the third authentication process is not performed, the face authentication device determines the final authentication result as authentication failure if the authentication result of the second authentication process is authentication failure, and if the authentication result of the second authentication process is authentication success , determine the final authentication result as authentication success.

In step 530, if the authentication result of the second authentication process is determined to be authentication failure, the face authentication device performs a third authentication process. However, the third authentication process is merely an example and may be omitted, and additional authentication processes using other types of recognizers may be included in the face authentication procedure. In the third authentication process, the face authentication device can acquire third feature information using a third recognizer, for example, from the result of synthesizing the parameters of the first authentication process and the parameters of the second authentication process. For example, the parameters may include feature information extracted from facial regions by each recognizer.

In step 535, the face recognition device determines whether or not the third verification process is successful. A face authentication apparatus according to an embodiment combines first feature information extracted from a first recognizer and second feature information extracted from a second recognizer to generate combined feature information. The face authentication device determines the difference between the registered feature information obtained by combining the first registered feature information corresponding to the first feature information and the second registered feature information corresponding to the second feature information and the combined feature information. If the comparison result satisfies the preset condition, the authentication result is determined as successful authentication (540), and the face authentication procedure ends. On the other hand, if the comparison result does not satisfy the preset conditions, the face authentication device determines the authentication result as authentication failure (550), and terminates the face authentication procedure. Here, the authentication result may be the final authentication result.

FIG. 6 is a flowchart for explaining the unlock mode among the authentication modes according to one embodiment.

As an example, the face authentication device performs unlock mode. The face authentication device detects a face area from the input image 601 (603). The face authentication device detects landmarks from the detected face area (605). The facial recognition device performs a first normalization 611 on the facial region to make it suitable for input to the first recognizer 617 .

The face authentication device normalizes the detected face area by, for example, matching the position of the detected landmark with a predefined reference position or adjusting the size of the detected face area. If the lower face region is not captured by moving away from the camera's field of view, the face recognition device may fill the lower face region of the input image with a region corresponding to the lower face region of the average image or the average value image.

The face recognition device inputs the normalized image 613 to the first recognizer 617 . The face recognition device extracts a first feature 619 using a first recognizer 617 . The face authentication device compares the extracted first feature 619 with the first registered features pre-stored in the database to determine whether or not the authentication is successful (651). Here, the first registered feature is a feature extracted from a valid user input video using the first recognizer 617 .

As an example, the face authentication device calculates the degree of similarity between the first feature 619 and the first registered feature, and compares the degree of similarity with a preset first threshold to determine whether or not authentication has succeeded. The facial recognition device determines 655 that the verification was successful, for example, if the determined similarity is greater than a first threshold. Conversely, the face recognition device determines that the recognition has failed if the similarity is less than or equal to the threshold.

When the authentication result is determined to be authentication success (655), the face authentication device determines whether to update the first registered feature based on the first feature information (631). The face authentication device updates the first registered feature by comparing, for example, a threshold set higher than the first threshold with the determined similarity (633). When the authentication result is determined to be authentication success (655), the face authentication device determines the final authentication result to be authentication success, and ends the face authentication procedure. When the face authentication device determines that the authentication result is authentication failure, the face authentication device performs second normalization (621).

The face recognition device inputs the normalized image 623 to the second recognizer 627 . The face authentication device according to one embodiment, regardless of whether or not the blocking region such as glasses exists in the face region, determines the region where the blocking region is expected to exist as the average image, the average value video, or the Fill with the image information of the corresponding portion of any single color image. An average image is generated by averaging the pixel values that have corresponding locations among the training images. The average value image is generated by averaging pixel values of all pixels included in the training image and assigning the average pixel value to all pixels. The average value image is shown because the pixel value may vary depending on each pixel position, but the average value image is not shown because all pixels have the same pixel value. However, this is only an example, and the processing method for the case where the face area is occluded is not limited to this.

The face recognition device extracts second features 629 using a second recognizer 627 . The face authentication device compares the extracted second feature 629 with the second registered features pre-stored in the database to determine whether or not the authentication is successful (653). Here, the second registered feature is a feature extracted from the valid user input video using the second recognizer 627 .

When the face authentication device determines that the authentication result is authentication success (655), it determines whether to update the second registered feature based on the second feature 629 (641). If the facial recognition device determines to update, it updates 643 the second enrollment feature based on the second feature 629 . The face authentication device determines the final authentication result as authentication success.

When the face authentication device determines the authentication result as authentication failure (657), the face authentication device determines the final authentication result as authentication failure. The face authentication device may end the face authentication procedure, or may receive another input image and perform the face authentication procedure again. For example, the face authentication device may repeat the face authentication procedure for input images received for 6 seconds by continuously photographing the camera until the final authentication result is determined as successful authentication.

FIG. 7 is a flow chart for explaining an authentication method in a second authentication mode according to another embodiment.

The second authentication mode according to one embodiment is an authentication mode in which security is more important than usability. For example, the second authentication mode may be a mode for confirming whether the user is valid for payment. In this case, if any one of the plurality of authentication results is determined to be authentication failure in the second authentication mode, the face authentication device determines that the final authentication result is authentication failure, and terminates the face authentication procedure.

In step 710, the facial recognition device performs a first verification process. In the first authentication process, the face authentication device extracts first feature information from the input image using the first recognizer. The method described with reference to FIG. 2 is used for the process of extracting feature information.

In step 715, the face recognition device determines whether or not the first verification process is successful. The face authentication device determines whether or not the first authentication process is successful based on a comparison result between the first registered feature information stored in the database and the extracted first feature information. The face authentication device determines whether to acquire the second feature information using the second recognizer based on the result of the first authentication process and the authentication mode.

If the result of comparison between the first registered feature information and the first feature information does not satisfy a preset condition, the face authentication device determines that the authentication result of the first authentication process is an authentication failure (740). End the certification process. Here, the authentication result may be the final authentication result. If the comparison result between the first registered feature information and the first feature information satisfies a preset condition, the face authentication device determines the authentication result as successful authentication and performs the second authentication process. As described above, the face authentication device determines to acquire the second feature information when the authentication result of the first authentication process is authentication success, and when the authentication result of the first authentication process is authentication failure, the final The authentication result is determined as authentication failure, and the face authentication procedure is terminated.

At step 720, the face recognition device performs a second verification process. In the second authentication process, the face authentication device extracts second feature information from the input image using a second recognizer. The process of extracting feature information uses the method described with reference to FIG.

In step 725, the face recognition device determines whether or not the second verification process is successful. The face authentication device compares the second registered feature information stored in the database with the extracted second feature information to determine whether or not the second authentication process has been successful. The face authentication device determines whether or not to acquire the third feature information using the third recognizer based on the second authentication result and the authentication mode.

If the comparison result between the second registered feature information and the second feature information does not satisfy the preset condition, the face authentication device determines the authentication result as authentication failure (740) and terminates the face authentication procedure. Here, the authentication result may be the final authentication result. When the comparison result between the second registered feature information and the second feature information satisfies a preset condition, the face authentication device determines the authentication result as authentication success. When the third authentication process is not performed, if the authentication result of the second authentication process is authentication failure, the face authentication device determines the final authentication result as authentication failure, and the authentication result of the second authentication process is authentication success. In this case, the final authentication result is determined as authentication success.

In step 730, if the authentication result of the second authentication process is determined to be authentication success, the face authentication device performs a third authentication process. Similar to that shown in FIG. 5, the third authentication step is exemplary only and may be omitted, and additional authentication steps using other types of recognizers may be included in the face authentication procedure.

In step 735, the face recognition device determines whether or not the third verification process is successful. Feature information obtained by combining first feature information extracted through a first recognizer and second feature information extracted from a second recognizer, and first registered feature information and a second feature corresponding to the first feature information If the comparison result between the second registered feature information combined with the second registered feature information corresponding to the information satisfies a preset condition, the face authentication device determines that the authentication result is authentication success (750). End the certification process. If the comparison result does not satisfy the preset condition, the face recognition device determines the recognition result as recognition failure (740), and terminates the face recognition procedure. Here, the authentication result may be the final authentication result.

FIG. 8 is a flowchart for explaining a payment mode among authentication modes according to another embodiment.

As an example, the face authentication device performs payment mode. Similar to the face recognition procedure shown in FIG. 6, the face recognition device detects a face area from the input image 601 (603). The face authentication device detects landmarks from the detected face area (605). The face recognition device performs a first normalization on the face region to make it suitable for input to the first recognizer 617 . The face recognition device inputs the normalized image 613 to the first recognizer 617 . The face recognition device extracts a first feature 619 using a first recognizer 617 . The face authentication device compares the extracted first feature 619 with the first registered features pre-stored in the database to determine whether or not the authentication is successful (651).

When the face authentication device determines that the authentication result is authentication success (655), it determines whether to update the first registered feature based on the first feature (631), and performs second normalization (621). . The face authentication device updates the first registered feature by comparing, for example, a threshold set higher than the first threshold with the determined similarity (633). When the face authentication device determines that the authentication result is authentication failure, the face authentication device determines that the final authentication result is authentication failure, and ends the face authentication procedure.

The face recognition device inputs the normalized image 623 to the second recognizer 627 . For example, if the lower face region has not been captured by leaving the field of view of the camera, the face recognition device may select the lower face region of the input image as the average image, the average value image, or the lower face area of any single color image. Fill with a portion corresponding to the face area, or areas where occlusion is indicated by hidden elements of the face such as glasses, sunglasses, masks or hats, etc. It can be filled with the corresponding part of the image.

The face recognition device extracts second features 629 using a second recognizer 627 . The face authentication device compares the extracted second feature 629 with the second registered features pre-stored in the database to determine whether or not the authentication is successful (653). Here, the second registered feature is a feature extracted from the valid user input video using the second recognizer 627 .

When the face authentication device determines that the authentication result is authentication success (655), it determines whether to update the second registered feature based on the second feature 629 (641). If the facial recognition device determines to update, it updates 643 the second enrollment feature based on the second feature 629 . The face authentication device determines the final authentication result as authentication success. When the face authentication device determines the authentication result as authentication failure (657), it determines the final authentication result as authentication failure. The face authentication device may end the face authentication procedure, or may receive another input image and perform the face authentication procedure again.

FIG. 9 is a diagram showing the configuration of a face authentication device according to one embodiment.

Referring to FIG. 9, the face recognition device 900 receives an input image for face recognition. The face authentication device 900 determines whether or not the authentication is successful based on the result of comparison between the features extracted from the input image and the registered features pre-stored in the database 930 . A valid user pre-registers features or images of his/her face through a face registration process, and the registered information is stored in the database 930 .

The face recognition device 900 can perform one or more operations described or illustrated herein with respect to face recognition methods and provide results of face recognition to the user. The face authentication device 900 outputs the results of face authentication in the form of voice, vibration, characters, pictures, moving images, or the like. An authentication result can be output.

Face authentication device 900 includes one or more processors 910 and memory 920 . Memory 920 is connected to processor 910 and stores instructions executable by processor 910 , data operated on by processor 910 or data processed by processor 910 . Memory 920 may be a non-transitory computer-readable storage medium, such as a high-speed random access memory and/or a non-volatile computer-readable storage medium (e.g., one or more disk storage devices, flash memory devices, or , and other non-volatile solid-state memory devices).

Processor 910 executes instructions to perform one or more of the operations described with reference to FIGS. 1-8. For example, the processor 910 detects facial regions and facial landmarks in the input video, and normalizes the input video to be suitable for each recognizer input based on the detected landmarks. Processor 910 uses the reference image to pad a portion of the face area that is not captured out of the field of view of the camera or is occluded by an external object such as eyeglasses.

Processor 910 identifies the current authentication mode and determines at least one recognizer to utilize for face authentication based on the current authentication mode. Processor 910 obtains feature information from the face region shown in the input video using the determined at least one recognizer.

The processor 910 extracts the features of the input face from the face area using the recognizer, compares the extracted features with the registered features, and determines whether or not the authentication is successful based on the comparison result. Here, the face area may be replaced with a normalized or padded face area. The processor 910 determines a final authentication result according to the authentication result of each recognition process based on the authentication mode, ends the face authentication procedure, or performs an authentication process with another recognizer. In this way, processor 910 determines whether or not authentication is successful based on the feature information.

According to one embodiment, the processor 910 performs the facial recognition procedure in unlock mode. When determining that the authentication result of the authentication process by the first recognizer is successful, the processor 910 determines that the final authentication result is successful, and ends the face authentication procedure. When the processor 910 determines that the authentication result of the authentication process by the first recognizer is authentication failure, the processor 910 performs the authentication process by the second recognizer. Here, the final authentication result is determined according to the authentication result by the second recognizer.

According to another embodiment, the processor 910 performs face recognition procedure in payment mode. When the processor 910 determines that the authentication result of the authentication process by the first recognizer is authentication failure, the processor 910 determines that the final authentication result is authentication failure, and ends the face authentication procedure. When the processor 910 determines that the authentication result of the authentication process by the first recognizer is successful, the processor 910 performs the authentication process by the second recognizer. Here, the final authentication result is determined according to the authentication result by the second recognizer.

FIG. 10 is a diagram illustrating an example of a computing device according to one embodiment.

Referring to FIG. 10, a computing device 1000 acquires an image including a user's face and performs a face recognition process by comparing features extracted from the acquired image with registered features. Computing device 1000 corresponds to, but is not limited to, computing device 120 shown in FIG. 1, face recognition device 900 shown in FIG. 9, or learning device 1100 shown in FIG. .

Computing device 1000 includes processor 1010 , memory 1020 , camera 1030 , storage device 1040 , input device 1050 , output device 1060 and network interface 1070 . Processor 1010 , memory 1020 , camera 1030 , storage device 1040 , input device 1050 , output device 1060 and network interface 1070 communicate via communication bus 1080 .

Camera 1030 captures still images, moving images, or both. When the user attempts face authentication, the processor 1010 captures an image of the user's face area, for example, adjusts the camera 1030 to acquire or capture an image of the user including the face, adjusts the camera 1030 so that it automatically captures images and automatically authenticates the user without initialization. Additionally, camera 1030 may be coordinated by processor 1010 while other functions of computing device 1000 are performed, such as when operating as a personal camera. Camera 1030 refers to multiple cameras such as, but not limited to, a color video/movie camera and a depth or infrared camera or TOF (time of flight) module.

Processor 1010 performs the functions and instructions that operate on computing device 1000 . For example, processor 1010 executes instructions stored in memory 1020 or storage device 1040 . Processor 1010 may be one or more processors, such as or including processor 910 shown in FIG. 9 or processor 1110 shown in FIG. Processor 1010 is configured to perform one or more of the operations or combinations thereof associated with the descriptions of FIGS. 1-11. For example, processor 1010 may be configured to perform face authentication and enrollment by comparing facial images captured by camera 1030 with enrolled facial images or corresponding enrolled features. Additionally, processor 1010 may be configured to coordinate other functions of computing device 1000 . For example, computing device 1000 may be a mobile device such as a mobile phone, tablet, or personal computer, and processor 1010 may be configured to perform other typical functions of computing device 1000. . For example, the processor 1010 may perform a lock-out operation upon successful authentication of the user's captured facial image to allow the user access, as described above, or otherwise. may be configured to perform a similar function.

Memory 1020 may be a non-volatile computer-readable recording medium or device that stores information used for face authentication. Memory 1020 may be, but is not limited to, memory such as memory 920 shown in FIG. 9 and/or memory 1120 shown in FIG. Memory 1020 may be a computer-readable storage medium or computer-readable storage device. Further, memory 1020 represents multiple types of memory.

As an example, while face registration or authentication, respectively, is being performed, the computing device 1000 may display the currently captured face image or preview image acquired by the camera 1030 on a display screen for viewing by the user. , visual feedback, example face images, preview images, or other guides. The exemplary facial image or preview image provides visual feedback or motivation to guide the user to provide the computing device 1000 with the overall facial image. The visual feedback may be a display of currently captured video frames from which the input video is selectively captured for facial recognition, or may provide the user with exemplary guidelines or overlays for preferred positions and sizes. may It may be a video taken for face authentication. With such facial recognition, no facial image or preview image may be provided, for example, as in the example performed automatically without user initiation.

FIG. 11 is a diagram illustrating an example of a learning device according to an embodiment;

Referring to FIG. 11, a training device 1100 trains a feature extractor 1130 used for face authentication, such as, by way of non-limiting example, one or more feature extractor combinations disclosed herein. configured as follows. For example, feature extractor 1130 may be a neural network model trained based on a plurality of training and reference images and configured to output result values corresponding to input values. Along with training the exemplary feature extractor 1130, the training device 1100 may train additional classification or comparison or objective portions of the neural network.

Learning device 1100 includes one or more processors 1110 and memory 1120 . Memory 1120 may be a non-volatile computer readable medium or device coupled to processor 1110 and may be used to process data processed by processor 1110 and/or data processed by processor 1110 during neural network training. , stores instructions that can be executed by the processor 1110 to train the neural network.

As an example, when one or more training videos are input to the training device 1100, the processor 1110 detects respective facial regions in the training videos, extracts features from the detected facial regions, For example, extractor 1130 represents a separate feature recognizer or extractor discussed herein. The feature recognition or extraction may not be related to facial images, and in another example, the feature recognizer or extractor performs recognition on pre-detected facial images and uses it for ultimate authentication purposes. May be combined. As a different example, the processor 1110 performs learning by calculating the loss caused by the difference between the feature extracted by the feature extractor 1130 and the intentional feature, and the feature extractor 1130 learns to reduce the calculated loss. The feature extractor 1130 can be trained by adjusting the parameters contained in 1130 .

The embodiments described above may be embodied in hardware components, software components, or a combination of hardware and software components. For example, the devices and components described in this embodiment include processors, controllers, ALUs (arithmetic logic units), digital signal processors, microcomputers, FPAs (field programmable arrays), PLUs (programmable logic units), microprocessors, or different devices that execute and respond to instructions, using one or more general-purpose or special-purpose computers. The processing device executes an operating system (OS) and one or more software applications that run on the operating system. The processing unit also accesses, stores, manipulates, processes, and generates data in response to executing the software. For convenience of understanding, the processing device may be described as one being used, but those of ordinary skill in the art will understand that the processing device may include multiple processing elements and/or It can be seen that there are multiple types of processing elements. For example, a processing device may include multiple processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software includes computer programs, code, instructions, or a combination of one or more of these, to configure and, independently or jointly, to direct the processor to operate as desired. Software and/or data are any type of machine, component, physical device, virtual device, computer storage medium or device, or signal transmitted to be interpreted by a processing device and to provide instructions or data to the processing device. Permanently or temporarily embodied through waves. The software may be stored and executed in a distributed fashion over network coupled computer systems. Software and data are stored on one or more computer-readable media.

The method according to the present embodiment is embodied in the form of program instructions executed via various computer means and recorded in a computer-readable recording medium. The recording media may include program instructions, data files, data structures, etc. singly or in combination. The recording medium and program instructions may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well known and available to those having skill in the computer software arts. . Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. It includes media and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code, such as that generated by a compiler, as well as high-level language code that is executed by a computer, such as using an interpreter. A hardware device may be configured to act as one or more software modules to perform the operations of the present invention, and vice versa.

Even though the embodiments are described by means of limited drawings as described above, those skilled in the art can apply various technical modifications and variations based on the above. . For example, the techniques described may be performed in a different order than in the manner described and/or components such as systems, structures, devices, circuits, etc. described may be combined or combined in a manner different than in the manner described. , may be substituted or substituted with other components or equivalents to achieve suitable results.

110 user 120 computing device 130 camera 140 face regions 310, 320 face recognition procedures 312, 313, 322, 323 normalized image 401 images 413, 423 normalized face region

Claims (62)

In a face authentication method executed by a processor,
selecting a current authentication mode to be performed for face authentication from among a plurality of authentication modes;
determining one or more recognizers to be used for face recognition based on the selected current authentication mode among a plurality of recognizers;
extracting feature information from face information using at least one of the determined one or more recognizers;
a step of displaying the presence or absence of authentication success based on the extracted feature information;
including
the one or more recognizers include a first recognizer and a second recognizer;
The first recognizer has a higher recognition rate than the second recognizer for an input image shot in a low-illuminance environment, and the second recognizer has an occluded area due to face hiding in the face area. The recognition rate for the input image is higher than that of the first recognizer,
face recognition method. further comprising obtaining a recognizer based on the determination of the one or more recognizers;
extracting the feature information includes applying video information from a determined facial region of a video to the acquired recognizer to generate the extracted feature information;
The face authentication method according to claim 1. further comprising performing a first normalization of the determined facial region of the image to produce the image information in a predetermined form suitable for application to the obtained recognizer;
wherein the obtained recognizer is a trained neural network or machine learning model;
The face authentication method according to claim 2. respectively performing a normalization on the determined facial regions of the image to generate each image information in a form predetermined to be suitable for application to different acquired recognizers based on the current authentication mode; further comprising
extracting the feature information includes extracting respective feature information from each piece of video information using the different acquired recognizers;
The step of displaying whether or not the authentication was successful includes displaying whether or not the authentication was successful based on at least one of the extracted characteristic information.
The face authentication method according to any one of claims 1 to 3. each of the steps of normalizing includes synthesizing image information relating to an occlusion region or an expected occlusion region in the determined facial region of the image;
The face authentication method according to claim 4. further comprising performing the face authentication;
The step of performing face authentication includes:
performing a first authentication based on first feature information extracted from each of the extracted feature information;
selectively performing a second authentication based on second feature information extracted from each of the extracted feature information;
selectively performing a third authentication based on the extracted first feature information and the extracted second feature information depending on whether the second authentication was selectively performed;
determining whether the face authentication is successful based on the result of the third authentication;
5. The face authentication method according to claim 4, comprising: the step of extracting feature information includes extracting respective feature information from each normalization of a determined face region of a video using a different determined recognizer based on the current authentication mode;
The step of extracting each feature information is selectively and partially performed by giving a weight to one of usability and security of the face authentication based on the current authentication mode.
The face authentication method according to claim 1. Determining the one or more recognizers comprises:
determining a plurality of different recognizers for said face recognition;
performing the face authentication based on selected combinations and/or arrangements of the different recognizers depending on the current authentication mode;
The face authentication method according to claim 1. performing face authentication includes selectively performing respective authentication procedures corresponding to the different recognizers based on the current authentication mode;
The face authentication method according to claim 8. Each of the authentication procedures includes a step of comparing respective similarities with corresponding thresholds between the feature information extracted differently from each other and the corresponding registration information;
The face authentication method according to claim 9. The first recognizer and the second recognizer are each trained using learning data,
The first recognizer performs low-light facial feature extraction for face recognition in a low-light environment, and the second recognizer is trained with training data regarding face regions including one or more occluded regions.
The face authentication method according to claim 8. The step of extracting the feature information includes:
obtaining first feature information from image information of the determined face region of the image using the first recognizer;
The face authentication method further comprises performing face authentication,
The step of performing face authentication includes:
determining a first authentication result based on the first characteristic information;
determining whether to extract second feature information using the second recognizer based on the first authentication result and the current authentication mode;
including
the plurality of authentication modes includes at least one of a first authentication mode and a second authentication mode;
The face authentication method according to claim 8. The step of extracting the feature information includes determining a determined facial region of another image or the determination of the image using the second recognizer in response to a determination of whether to extract the second feature information. extracting the second feature information from the video information or other video information from the face region, wherein determining whether to extract the second feature information includes extracting the second feature information. is the decision that
The step of performing face authentication includes:
determining a second authentication result based on the second characteristic information;
determining whether or not the authentication is successful based on the second authentication result;
13. The face authentication method of claim 12, comprising: The step of performing face authentication includes:
In response to said current authentication mode being selected as a first authentication mode,
determining a final authentication result as failure in response to the second authentication result being failure;
determining the final authentication result as successful in response to the second authentication result being successful;
14. The face authentication method of claim 13, comprising: The step of performing face authentication includes:
In response to the current authentication mode selected as a second authentication mode,
determining that the second characteristic information is to be extracted in response to the first authentication result being successful;
determining the final authentication result as successful in response to the second authentication result being successful;
15. The face authentication method of claim 14, comprising: The step of performing face authentication includes:
When the current authentication mode is selected as the first authentication mode,
determining to extract the second feature information if the first authentication result is authentication failure;
determining a final authentication result as authentication success and ending the face authentication procedure when the first authentication result is authentication success;
13. The face authentication method of claim 12, comprising: In response to the current authentication mode selected as the second authentication mode,
determining a final authentication result as authentication failure and ending the face authentication procedure when the first authentication result is authentication failure;
determining to extract the second feature information if the first authentication result is successful authentication;
16. The face authentication method of claim 15, comprising: The step of determining the first authentication result further includes determining the first authentication result based on a comparison of the first feature information and first registered feature information extracted in advance using the first recognizer. include,
The face authentication method according to claim 12. optionally further comprising updating the first registered feature information using the first feature information when the first authentication result is successful authentication;
The face authentication method according to claim 18. The step of determining the second authentication result compares the second feature information with the second registered feature information extracted using the second recognizer to determine the second authentication result.
The face authentication method according to claim 13. optionally further comprising updating the second registered feature information using the second feature information when the second authentication result is successful authentication;
The face authentication method according to claim 20. The step of determining the first authentication result includes:
calculating a similarity between the first feature information and the first registered feature information extracted using the first recognizer;
determining the first authentication result based on a comparison result between the similarity and a preset first threshold;
13. The face authentication method of claim 12, comprising: The step of extracting the feature information includes:
detecting a face region from an input video;
detecting facial landmarks from the detected facial region;
normalizing the face region based on the detected landmarks to generate the information of the face;
The face authentication method of claim 1, comprising: A computer-readable recording medium recording a program for executing the method according to any one of claims 1 to 23 on a computer. In a face authentication device, the face authentication device includes at least one processor, the at least one processor comprising:
select a current authentication mode to be performed for face authentication from multiple authentication modes,
determining one or more recognizers among a plurality of recognizers based on the selected current authentication mode;
extracting feature information from face information using the determined one or more recognizers;
configured to display whether or not authentication has been successful based on the extracted characteristic information;
the one or more recognizers include a first recognizer and a second recognizer;
The first recognizer has a higher recognition rate than the second recognizer for an input image shot in a low-illuminance environment, and the second recognizer has an occluded area due to face hiding in the face area. The recognition rate for the input image is higher than that of the first recognizer,
face recognition device. The one or more processors obtain a recognizer based on determinations of one or more recognizers;
The one or more processors apply video information from a determined facial region of a video to the acquired recognizer to generate the extracted feature information.
The face authentication device according to claim 25. The one or more processors
performing a first normalization of the determined facial region of the image to produce the image information in a predetermined form suitable for application to the obtained recognizer;
wherein the obtained recognizer is a trained neural network or machine learning model;
27. The face authentication device according to claim 26. The one or more processors
each normalizing the determined face region of the image to generate each image information in a predetermined form suitable for application to different acquired recognizers based on the current authentication mode;
extracting respective feature information from each video information using the different acquired recognizers;
displaying whether the authentication was successful based on at least one of the extracted feature information;
28. A face authentication device according to any one of claims 25-27. The one or more processors combine video information regarding occlusion regions or expected occlusion regions in determined facial regions of the video for at least one of the respective normalizations.
The face authentication device according to claim 28. The one or more processors
performing first authentication based on first feature information extracted from each of the extracted feature information;
selectively performing second authentication based on second feature information extracted from each of the extracted feature information;
selectively performing a third authentication based on the extracted first feature information and the extracted second feature information, depending on whether the second authentication was selectively performed;
Determining whether the face authentication was successful based on the result of the third authentication;
The face authentication device according to claim 28. The one or more processors
extracting respective feature information from each normalization of the determined face region of the video using different determined recognizers based on the current authentication mode;
The one or more processors selectively and partially weight one of usability and security of the face authentication based on the current authentication mode.
The face authentication device according to claim 25. The one or more processors
determine a plurality of different recognizers for the face authentication based on the current authentication mode;
configured to perform said authentication based on selected combinations and/or arrangements of said different recognizers, depending on said current authentication mode;
The face authentication device according to claim 25. the one or more processors are configured to selectively perform respective authentication procedures corresponding to the different recognizers based on the current authentication mode;
33. A face authentication device according to claim 32. By the one or more processors, each of the authentication procedures includes comparing respective similarities from corresponding thresholds between different extracted feature information and corresponding enrollment information;
34. A face authentication device according to claim 33. The one or more processors
extracting first feature information from video information derived from the determined face region of the video using the first recognizer to extract the feature information;
determining a first authentication result based on the first feature information to perform the authentication;
configured to determine whether to extract second feature information using the second recognizer based on the first authentication result and the current authentication mode;
The plurality of authentication modes includes at least one of a first authentication mode and a different second authentication mode,
33. A face authentication device according to claim 32. The at least one processor
a determined facial region of another image or another image from the determined facial region of the image using the second recognizer in response to determining whether to extract the second feature information; extracting the second feature information from the information or the video information, and determining whether to extract the second feature information is a decision to extract the second feature information;
determining a second authentication result based on the second characteristic information to perform the authentication;
configured to determine whether or not the authentication is successful based on the second authentication result;
36. A face authentication device according to claim 35. The at least one processor
when the current authentication mode is selected as the first authentication mode,
determining a final authentication result as an authentication failure in response to the second authentication result being an authentication failure;
configured to determine a final authentication result as successful authentication in response to the second authentication result being successful authentication;
37. A face authentication device according to claim 36. The at least one processor
When the current authentication mode is selected as a second authentication mode,
determining to extract second feature information in response to the first authentication result being authentication failure;
configured to determine a final authentication result as successful authentication in response to the second authentication result being successful authentication;
38. A face authentication device according to claim 37. The one or more processors
In response to the current authentication mode being selected as a first authentication mode,
determining to extract the second feature information in response to the first authentication result being authentication failure;
If the first authentication result is authentication success, the final authentication result is determined as authentication success and the face authentication procedure is terminated.
36. A face authentication device according to claim 35. The one or more processors
In response to the current authentication mode being selected as a second authentication mode,
if the first authentication result is authentication failure, determining the final authentication result as authentication failure and ending the face authentication procedure;
configured to determine to extract the second characteristic information if the first authentication result is successful authentication;
36. A face authentication device according to claim 35. The one or more processors are configured to compare the first feature information and first enrolled feature information pre-extracted using the first recognizer to determine the first authentication result.
36. A face authentication device according to claim 35. The one or more processors are selectively configured to update the first registered characteristic information with the first characteristic information if the first authentication result is a successful authentication.
42. A face authentication device according to claim 41. The one or more processors are configured to compare the second feature information and second enrolled feature information pre-extracted using the second recognizer to determine the second authentication result.
37. A face authentication device according to claim 36. The one or more processors are selectively configured to update the second registered characteristic information using the second characteristic information if the second authentication result is a successful authentication.
44. A face authentication device according to claim 43. The one or more processors
calculating the degree of similarity between the first feature information and the first registered feature information extracted in advance using the first recognizer;
configured to determine the first authentication result based on a comparison result between the similarity and a preset first threshold;
36. A face authentication device according to claim 35. The one or more processors
Detects the face area from the input video,
detecting facial landmarks from the detected facial region;
configured to normalize the face region based on the detected landmarks;
The face authentication device according to claim 25. a first recognizer including a neural network model for extracting first feature information from an input video showing a user's facial region;
a second recognizer including a different neural network model for extracting second feature information different from the first feature information from the input image;
A first authentication result obtained by comparing the extracted first feature information and the registered first registered feature information, and a second authentication result obtained by comparing the extracted second feature information and the registered second registered feature information. one or more processors that determine face authentication as successful if at least one of the authentication results is successful;
including
The first recognizer has a higher recognition rate than the second recognizer for an input image shot in a low-illuminance environment, and the second recognizer has an occluded area due to face hiding in the face area. The recognition rate for the input image is higher than that of the first recognizer,
computing device. The neural network model of the first recognizer and the different neural network model of the second recognizer are learned based on learning data different from each other,
48. A computing device as recited in claim 47. The different neural network model of the second recognizer is learned based on learning data in which an occluded region exists in the learning face region or is expected to exist,
48. A computing device as recited in claim 47. An image that replaces an area in which an occluded area is expected to exist from the face area with image information of a corresponding area of an average image, an average value image, or a single color image is used for extracting the second feature information. input to different neural network models of the second recognizer;
48. A computing device as recited in claim 47. The area where the occlusion area is expected to exist is an area where occlusion is indicated by a combination of one or more of glasses, sunglasses, a hat, or a mask from the face area.
51. A computing device as recited in claim 50. The one or more processors
if the face authentication is successful, unlocking the computing device;
responsive to the first authentication result and the second authentication result being authentication failure, the one or more processors do not retain or release the lock mode;
52. A computing device according to any one of claims 47-51. a first recognizer including a neural network model for extracting first feature information from an input video showing a user's facial region;
a second recognizer including a different neural network model for extracting second feature information different from the first feature information from the input image;
A first authentication result obtained by comparing the extracted first feature information and the registered first registered feature information, and a second authentication result obtained by comparing the extracted second feature information and the registered second registered feature information. one or more processors that determine face authentication as unsuccessful if at least one of the authentication results fails;
including
The first recognizer has a higher recognition rate than the second recognizer for an input image shot in a low-illuminance environment, and the second recognizer has an occluded area due to face hiding in the face area. The recognition rate for the input image is higher than that of the first recognizer,
computing device. The neural network model of the first recognizer and the different neural network model of the second recognizer are learned based on learning data different from each other,
54. A computing device as recited in claim 53. The different neural network model of the second recognizer is learned based on learning data in which an occluded region exists or is expected to exist in the learning face region,
54. A computing device as recited in claim 53. An image that substitutes image information of a corresponding area of an average image, an average value image, or a single color image for an area in which an occluded area is expected to exist from the face area is used to extract the second feature information. input to different neural network models of the second recognizer;
54. A computing device as recited in claim 53. The area where the occlusion area is expected to exist is an area where occlusion is indicated by a combination of one or more of glasses, sunglasses, a hat, or a mask from the face area.
57. A computing device as recited in claim 56. The computing device comprises:
If the face authentication fails, determining the authentication result in the financial service or payment service using the computing device as an authentication failure;
In response to the first authentication result and the second authentication result being successful authentication, the one or more processors determine the authentication result as successful authentication with the payment service or the financial service.
58. A computing device according to any one of claims 53-57. said computing device further comprising a transceiver;
When the authentication result is determined to be successful in the payment service or the financial service, the one or more processors cause the transceiver to transmit payment information to an external terminal or perform a financial transaction through the financial service. and/or configured to provide financial information to a user interface by the financial service;
59. A computing device as recited in claim 58. a first recognizer comprising a neural network configured to extract first feature information from an input video comprising a facial region of a user;
a second recognizer comprising a different neural network configured to extract second feature information from the input video that is different from the first feature information;
one or more processors configured to determine a current authentication mode to be implemented to authenticate the user from one of a first authentication mode and a second authentication mode;
including
When the current authentication mode is determined as the first authentication mode, the one or more processors perform facial recognition in response to at least one of a first authentication result and a second authentication result being successful authentication. determined to be successful, the first authentication result is obtained by comparing the first characteristic information with the first registered characteristic information, and the second authentication result is obtained by comparing the second characteristic information with the second registered characteristic information. obtained by comparing
When the current authentication mode is determined as the second authentication mode, the one or more processors perform face authentication in response to at least one of the first authentication result and the second authentication result being unsuccessful. as a failure , and
The first recognizer has a higher recognition rate than the second recognizer for an input image shot in a low-illuminance environment, and the second recognizer has an occluded area due to face hiding in the face area. The recognition rate for the input image is higher than that of the first recognizer,
computing device. the first authentication mode is a predetermined unlock mode;
The unlock mode is arranged to be automatically executed by one or more processors when the computing device is in a locked state, or when the user attempts to unlock the user interface of the computing device. determined to be executed by one or more processors;
61. A computing device as recited in claim 60. the second authentication mode is a predetermined payment mode;
the payment mode is determined to be performed by the one or more processors when the user accesses or selects payment services on the computing device;
62. A computing device according to claim 60 or 61.

Images