KR102215535B1 - Partial face image based identity authentication method using neural network and system for the method - Google Patents

Abstract

According to the present invention, an identity authentication method using a learned neural network includes the steps of: extracting a plurality of partial images according to a predetermined criterion from the facial image in the identification of a user; extracting partial feature information by inputting the partial image into a plurality of individually learned neural networks according to the criteria, and calculating a partial similarity by comparing the extracted partial feature information with pre-stored user partial feature information; recalculating an integrated similarity using the calculated partial similarity; and performing identity authentication of the user using the recalculated integrated similarity. According to the present invention, more accurate identification is possible even when a part of the face in the image is difficult to recognize.

Description

Partial face image based identity authentication method using neural network and system for the method}

The present invention relates to an identity authentication method, and to identity authentication using a neural network.

As the non-face-to-face identity authentication method is applied to various industrial fields, various methods for verifying the identity of users online are being devised.

In addition, with the development of artificial intelligence technology, artificial intelligence technologies are being applied in various fields, and methods of detecting and tracking objects in images through neural network models are being developed instead of the existing image processing methods. The neural network model used for artificial intelligence can detect objects faster and more accurately in the input image than in general image processing through learning.

Methods of authenticating the user's identity by applying artificial intelligence technology to the ID card image and comparing the user's facial image inputted from the camera in real time with the facial image in the ID card are being applied in actual financial services.

However, in the case of identification cards, various technologies such as coating and hologram are combined to prevent contamination and forgery, so there is a problem that the recognition performance differs according to various environmental factors such as light reflection, picture paper, and damage.

Therefore, nowadays, a technology for directly determining identity with the naked eye and a technology for determining identity based on image recognition are concurrently used to recognize identity in a non-face-to-face manner. Therefore, for more efficient recognition, identity authentication showing strong performance against environmental factors. Methods need to be devised.

An object of the present invention is to propose an identity authentication method having robust performance against environmental factors.

In addition, an object of the present invention is to propose a method capable of more accurate identification even when a part of the facial part in a photographed image is difficult to recognize.

In the identification authentication method using a learned neural network according to the present invention for solving the above technical problem, the method comprising: extracting a plurality of partial images according to a predetermined criterion from a facial image in the user's identification card; Extracting partial feature information by inputting the partial image into a plurality of neural networks individually learned according to the criterion and integrating the partial feature information; Calculating a similarity by comparing the integrated integrated feature information and pre-stored user feature information; And performing identification of the user according to the similarity.

It is preferable that the predetermined criterion is determined to include at least some of the feature elements constituting the face.

In the integrating step, it is preferable to generate the integrated feature information by combining the feature information according to the positional relationship of the feature element in the face.

It is preferable that the predetermined criterion is determined to simultaneously include at least some of the feature elements.

In the integrating step, it is preferable to integrate partial feature information generated for a partial image including one feature element and partial specific information generated for a partial image simultaneously including a plurality of feature elements using a common feature element. Do.

In the identification authentication method using a learned neural network according to the present invention for solving the above technical problem, the method comprising: extracting a plurality of partial images according to a predetermined criterion from a facial image in the user's identification card; Extracting partial feature information by inputting the partial image into a plurality of neural networks individually learned according to the criteria, and calculating partial similarity by comparing the extracted partial feature information with pre-stored user partial feature information; Recalculating the integrated similarity by using the calculated partial similarity; And performing identity authentication of the user by using the recalculated integrated similarity.

In the recalculating step, it is preferable to recalculate the integrated similarity by weighting the similarity according to the weight determined for each criterion.

In an identity authentication server performing identity authentication using a learned neural network according to the present invention for solving the above technical problem, a partial image extracting unit for extracting a plurality of partial images according to a predetermined criterion from a facial image in a user's identification card ; A feature information extracting unit that extracts partial feature information by inputting the partial image into a plurality of neural networks that are individually learned according to the criteria; A similarity calculation unit for integrating the partial feature information and calculating a similarity by comparing the combined integrated feature information and pre-stored user feature information; And an authentication unit that authenticates the user's identity according to the similarity.

It is preferable that the predetermined criterion is determined to include at least some of the feature elements constituting the face.

In the integrating step, it is preferable to generate the integrated feature information by combining the feature information according to the positional relationship of the feature element in the face.

In an identity authentication server performing identity authentication using a learned neural network according to the present invention for solving the above technical problem, a partial image extracting unit for extracting a plurality of partial images according to a predetermined criterion from a facial image in a user's identification card ; A partial feature information extracting unit for extracting feature information by inputting the partial image into a plurality of neural networks individually learned according to the criteria; A similarity calculation unit for calculating a partial similarity by comparing the extracted partial characteristic information and pre-stored user partial characteristic information, and recalculating an integrated similarity using the calculated partial similarity; And performing identity authentication of the user by using the recalculated integrated similarity.

It is preferable that the similarity calculation unit weights the similarity according to a weight determined for each criterion to recalculate the integrated similarity.

According to the present invention, the efficiency of identity authentication can be further improved in a non-face-to-face situation.

In addition, even if a part of the face cannot be recognized depending on the damage of the ID card or the surrounding environment, it is possible to increase the user's convenience by enabling identification to be recognized.

In addition, it is possible to more accurately determine a forgery situation by performing identity authentication according to various facial components, and to reduce the possibility of occurrence of a problem due to identity theft.

1 is a diagram showing an identity authentication system according to an embodiment of the present invention.
2 is a diagram showing an identity authentication method according to an embodiment of the present invention.
3 is a diagram showing a specific operation of an identity authentication method according to an embodiment of the present invention.
4 to 5 are diagrams showing characteristic information of an identity authentication method according to an embodiment of the present invention.
6 is a diagram showing an identity authentication method according to another embodiment of the present invention.
7 is a diagram illustrating a specific operation of an identity authentication method according to another embodiment of the present invention.
8 is a diagram showing the configuration of an authentication server that performs an identity authentication method according to an embodiment of the present invention.

The following content merely illustrates the principle of the invention. Therefore, although those skilled in the art can implement the principles of the invention and invent various devices included in the concept and scope of the invention, although not clearly described or illustrated herein. In addition, it should be understood that all conditional terms and examples listed in the present specification are, in principle, clearly intended only for the purpose of understanding the concept of the invention, and are not limited to the specifically listed embodiments and states. .

The above-described objects, features, and advantages will become more apparent through the following detailed description in connection with the accompanying drawings, and accordingly, a person of ordinary skill in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

In addition, when it is determined that a detailed description of a known technology related to the invention may unnecessarily obscure the subject matter of the invention, a detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an identity authentication system according to an embodiment of the present invention.

Referring to FIG. 1, the identity authentication system according to the present embodiment includes a user 10 performing authentication using a user terminal 1000, and authentication determining whether to authenticate using information input from the user 10. It may be configured as a server 100.

The user 10 may perform a video call for authentication, or may directly photograph his/her ID image 25 and transmit user information to the authentication server 100.

Upon receiving the transmitted user information, the authentication server 100 compares the received information with information in a pre-stored database to determine whether they are the same person and performs authentication.

Alternatively, it is also possible to receive identification information of the user 10 from a separate certification authority, and to determine whether the person is the same by comparing the received identification information with the user's face 30 on a video call or an identification card photographed by the user 10. It is possible.

However, in the case of the identification card used at this time, the quality of the face image 20 of the user 10 printed in the identification card itself may be deteriorated due to damage caused by the shooting environment or the identification card itself. ) Can cause inconvenience.

In addition, since the current authentication method is performed by comparing the feature values of the entire facial image 20 and the stored feature values, a problem may occur in the entire authentication due to an effect on some areas of the facial image 20.

Accordingly, in order to solve this problem, the identity authentication system according to the present invention proposes a method of dividing the face, not the entire face image 20, into a plurality of partial images and determining whether or not the person is the same by using the divided images.

Hereinafter, an authentication method of the identity authentication system according to the present embodiment will be described with reference to FIG. 2.

2 is a flowchart illustrating an identity authentication method according to an embodiment of the present invention.

First, a plurality of partial images are extracted from the facial image 20 in the identification card for identification (S100).

In this case, the partial image may be extracted according to a predetermined criterion.

For example, partial images including human eyes, nose, mouth, and eyebrows may be extracted as feature elements in the face.

In addition, it is also possible to extract a partial image including each feature element overlapping. For example, a partial image including both eyes or a partial image including both eyes and eyebrows may be extracted. Alternatively, it is possible to extract a partial image including the tip of the nose and the mouth as part of the nose.

When a partial image is extracted according to the above criteria, the partial feature information is extracted by inputting it into a plurality of neural networks 30 individually learned according to the criteria, and the partial feature information is integrated according to the criteria (S200).

That is, in the present embodiment, the neural network 30 for extracting feature information for an image used for authentication may be trained to extract feature information for each partial image, not feature information for the entire face image.

Accordingly, a plurality of neural networks 30 are used instead of a single neural network for facial authentication, and feature information for each partial image may be extracted through the neural network 30.

Hereinafter, it will be described in more detail with reference to FIG. 3.

Referring to FIG. 3, the facial image 20 may be divided into a plurality of partial images 20-n according to a predetermined criterion as described above.

In addition, the neural networks 30 learned for each partial image 20-n may be trained to reinforce values for the feature elements included in the partial image 20-n, and for this purpose, the neural network 30 It may be a convolution neural network (CNN) model composed of a layer that performs a convolution operation of.

Accordingly, the layers inside the neural network 30 into which the partial images 20-n are input perform a convolution operation to emphasize feature values according to the shape or color of the feature element included in the image.

Feature information of the number of dimensions determined as final output may be output for each partial image.

Feature information for the next output partial images may be combined into one integrated feature information 42 through an integration network 40.

That is, when an operation according to the neural network 30 is performed on each of a plurality of partial images to generate a plurality of feature information, the feature information is integrated for authentication. In this case, for the integration of the feature information, a positional relationship between facial feature elements, which is a reference for extracting a partial image, may be used.

Referring to FIG. 4, for example, if the integrated feature information 42 is determined in a two-dimensional matrix form, the feature is mapped to a position determined according to an anatomical standard of feature elements included in each two-dimensional partial feature information. Information can be integrated.

That is, each of the extracted feature information is generated from an independent partial image, but the feature elements included in the partial image have a determined relationship between their physical positions, such as, for example, eyes and eyebrows. Accordingly, the feature information generated from the partial image of the eye may be mapped to be positioned below the integrated feature information 42 that is relatively integrated compared to the feature information generated for the eyebrow.

That is, in the present embodiment, feature information on a partial image is integrated using a relative positional relationship between facial feature elements that are a reference for extracting a partial image.

Accordingly, the feature information for the left eye may be mapped to the upper left position in the integrated feature information 4a, and the feature information 4b and 4c for the nose and mouth may be integrated so that they are mapped at the upper and lower positions of each other.

In addition, in the present embodiment, since partial image information may include feature elements in duplicate, it is possible to use values in the generated feature information as a reference when integrating.

For example, it is possible to integrate the feature information so as to have the same or similar feature values for the common tip of the nose among the feature information for the entire nose by using feature information on a partial image including the tip of the nose and the mouth at the same time.

Alternatively, referring to FIG. 5, it is possible for the integrated network to map each partial feature information into a one-dimensional matrix like a fully connected network. Accordingly, values included in the partial feature information can be flattened (5a, 5b, 5c) to be integrated into a one-dimensional matrix.

The degree of similarity is calculated by comparing the integrated integrated feature information 42 and previously stored user feature information.

The user feature information to be compared is feature information extracted from the face image 20 of the user 10 used for the last authentication, and may be a value generated through the neural network 30 for the entire face image 20.

At this time, the integrated feature information 42 and the user feature information are configured in the same dimension, so that the degree of similarity can be determined.

For example, if each feature information appears as an n-dimensional vector, the similarity to the vector A and the vector B for the integrated feature information 42 may be determined as a cosine similarity (Equation 1).

[Equation 1]

Alternatively, the degree of similarity may be determined using a distance between each feature information or a difference between values in the feature information.

When the similarity is calculated through the above process, the identity of the user 10 is authenticated according to the similarity.

Identity authentication may be performed by comparing similarity and a predetermined threshold. The threshold value is a value determined to be the same person, for example, can be determined as a probability value between 0 and 1, for example, when 0.5 is the threshold value, if the calculated similarity is 0.5 or more, authentication as the same person can be allowed. .

The above threshold can be determined internally by the identity authentication system, and is variably determined by using information such as the location where the ID card is photographed, the model of the user terminal 1000 used for the ID card shooting, and the resolution of the image as an environmental factor used for authentication. It is also possible to become.

According to this embodiment, authentication is performed using partial feature information generated based on each partial image, so even if some of the feature elements are hidden or damaged depending on the state of the ID or the shooting environment, the entire The degree of similarity is adjusted so that the same person can be better judged.

Hereinafter, an identity authentication method according to another embodiment of the present invention will be described.

As the identity authentication method according to the second embodiment, the identity authentication system according to the present embodiment compares partial feature information with partial feature information of the user 10, instead of integrating feature information, and calculates a similarity and directly used for authentication. Do.

6, in the same manner as in the above-described embodiment, a plurality of partial images are extracted from the facial image 20 in the identification card for identification (S110). In this case, the partial image may be extracted according to a predetermined criterion.

Next, partial feature information is extracted using a plurality of neural networks 30 (S210).

Referring to FIG. 7, the neural networks 30 trained for each partial image may be configured to perform a convolution operation in the form of a Convolution Neural Network (CNN) model to reinforce feature values of feature elements included in the partial image.

Therefore, the layers inside the neural network 30 to which the partial images are input perform a convolution operation to emphasize features according to the shape or color of the feature element included in the image, and feature information determined as the final output is applied to each partial image. Can be output.

However, in the second embodiment, instead of mapping each feature information to one integrated feature information 42, it may be implemented to calculate the similarity by comparing the partial feature information with the user partial feature information stored in the database.

That is, partial feature information is extracted by inputting to a plurality of neural networks 30 individually learned according to the criteria for classifying partial images, and partial similarity is calculated by comparing the extracted partial feature information with pre-stored user partial feature information ( S310).

As described above, the partial similarity may be calculated using a cosine similarity or distance between vectors in the feature information.

When the partial similarity is calculated, the integrated similarity is recalculated using the calculated partial similarity (S410). The integrated similarity 72 may be calculated as a sum or average of general partial similarities, or may be calculated by weighting the similarity according to a weight determined for each extraction criterion for partial images.

For example, a similarity between feature information for a partial image including a plurality of feature elements simultaneously among the partial images may be determined to have a higher weight than a similarity for a single partial image with respect to an eyebrow.

When the integrated similarity 72 is recalculated according to the determined weight, identity authentication is performed using the integrated similarity 72 (S510).

That is, the integrated similarity 72 is compared with a predetermined threshold value, and if it is greater than or equal to the threshold value, it is determined as the same person and authentication is allowed.

Hereinafter, a configuration of the identity authentication server 100 that performs the identity authentication method according to the present embodiment will be described with reference to FIG. 8.

The identity authentication server 100 includes a partial image extracting unit 110, a feature information extracting unit 120, a similarity calculating unit 130, and an authentication unit 140.

The partial image extracting unit 110 extracts a plurality of partial images according to a predetermined criterion from the facial image 20 in the identification card of the user 10.

The predetermined criterion may be determined to extract partial images including an eye, a nose, a mouth, etc. as feature elements that influence the determination of the same person in the face.

The feature information extraction unit 120 extracts partial feature information by inputting the extracted partial image into a plurality of neural networks 30 that are individually learned according to the criteria. Also, although not shown, the feature information extraction unit 120 may include a neural network 30 that has been trained to output feature information for a plurality of partial images.

The neural network 30 is pre-trained for each feature element included in each of the partial images, and may be learned to emphasize the shape or color of the feature element.

Accordingly, partial feature information for each partial image extracted with respect to the facial image 20 is generated.

Next, the similarity calculation unit 130 integrates the partial feature information according to the criterion, and calculates the similarity by comparing the integrated feature information 42 with pre-stored user feature information. The similarity calculation unit may include a user characteristic information database 132 that stores and manages characteristic information of each user 10 as reference information for authentication in advance.

The degree of similarity may be determined by comparison between the integrated feature information 42 and the user feature information stored in the user feature information database 132.

The authentication unit 140 performs identity authentication of the user 10 according to the determined similarity, and the authentication result is provided to the user terminal 1000 again, so that the user 10 continues to provide services according to the result of non-face-to-face authentication. To be used

In addition, the identity authentication server may perform the authentication method according to the second embodiment.

Accordingly, the similarity calculation unit 130 may calculate the partial similarity by comparing the partial feature information extracted from the feature information extraction unit 120 with the user partial feature information stored in the user feature information database 132.

Also, the similarity calculation unit 130 may calculate the integrated similarity by recalculating the integrated similarity using the calculated partial similarity and weighting the partial similarity.

The authentication unit 140 finally determines whether the person is the same person using the integrated similarity recalculated through the above process, and transmits the determination result to the user terminal 1000.

According to the present invention, it is possible to further increase the efficiency of identity authentication in a non-face-to-face situation.

In addition, even if a part of the face cannot be recognized depending on the damage of the ID card or the surrounding environment, it is possible to increase the user's convenience by enabling the identification to be recognized.

In addition, it is possible to more accurately determine a forgery situation by performing identity authentication according to various facial components, and to reduce the possibility of occurrence of a problem due to identity theft.

Furthermore, various embodiments described herein may be implemented in a recording medium that can be read by a computer or a similar device using, for example, software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions, in some cases herein. The described embodiments may be implemented by the control module itself.

According to the software implementation, embodiments such as procedures and functions described in the present specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. The software code can be implemented as a software application written in an appropriate programming language. The software code may be stored in a memory module and executed by a control module.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the technical field to which the present invention belongs can make various modifications, changes, and substitutions within the scope not departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but are for description, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (14)

In the identity authentication method using a learned neural network,
Extracting a plurality of partial images according to a predetermined criterion from the facial image in the user's ID card;
Extracting partial feature information by inputting the partial image into a plurality of neural networks individually learned according to the criteria, and integrating the partial feature information;
Calculating a similarity by comparing the integrated integrated feature information and pre-stored user feature information; And
Including the step of performing the identity authentication of the user according to the similarity,
The neural network further includes an integrated network integrating the extracted partial feature information,
The integrating step is characterized in that the integration by mapping the partial feature information to a location determined by using a positional relationship of facial feature elements as a reference for extracting the partial image through the integrated network. The method of claim 1,
The predetermined criterion is an identity authentication method, characterized in that it is determined to include at least some of the feature elements constituting the face delete The method of claim 2,
And the predetermined criterion is determined to simultaneously include at least some of the characteristic elements. The method of claim 4,
In the integrating step, partial feature information generated for a partial image including one feature element and partial specific information generated for a partial image simultaneously including a plurality of feature elements are combined using a common feature element. Identity verification method. delete delete In an identity authentication server that performs identity authentication using a learned neural network,
A partial image extraction unit for extracting a plurality of partial images according to a predetermined criterion from the facial image in the user's ID card;
A feature information extracting unit for extracting partial feature information by inputting the partial image into a plurality of neural networks individually learned according to the criteria, and integrating the partial feature information;
A similarity calculation unit for calculating a similarity by comparing the integrated integrated characteristic information and pre-stored user characteristic information; And
Including an authentication unit that performs identification of the user according to the similarity,
The neural network further includes an integrated network integrating the extracted partial feature information,
The feature information extracting unit integrates the partial feature information by mapping the partial feature information to a location determined using the positional relationship of facial feature elements as a reference for extracting the partial image through the integrated network. server. The method of claim 8,
The identity authentication server, characterized in that the predetermined criterion is determined to include at least some of the feature elements constituting the face. delete delete delete A computer-readable recording medium storing a program for performing the identity authentication method according to any one of claims 1, 2, 4, and 5. A program stored in a computer-readable recording medium for performing the identification authentication method according to any one of claims 1, 2, 4, and 5.

Images