KR102213445B1 - Identity authentication method using neural network and system for the method - Google Patents

Abstract

According to one embodiment of the present invention, an identity authentication method using a learned neural network includes the steps of: calculating a first degree of degradation according to a first degradation factor of an ID image in an input image using a trained first neural network; calculating a similarity by comparing the ID image with the face image of the user; and comparing the calculated similarity with a predetermined threshold to determine whether they are the same. The similarity may be a value adjusted according to the first degree of degradation. According to the present invention, it is possible to further increase the accuracy of identity authentication in a non-face-to-face situation.

Description

Identity authentication method using neural network and system for the method}

The present invention relates to an identity authentication method, and to identity authentication using a neural network.

As the non-face-to-face identity authentication method is applied to various industrial fields, various methods for verifying the identity of users online are being devised.

In addition, with the development of artificial intelligence technology, artificial intelligence technologies are being applied in various fields, and methods of detecting and tracking objects in images through neural network models are being developed instead of the existing image processing methods. The neural network model used for artificial intelligence can detect objects faster and more accurately in the input image than in general image processing through learning.

Therefore, methods of authenticating the user's identity by comparing the user's face image input in real time from the camera with the face image in the ID card by applying artificial intelligence technology to the ID card image are actually applied in financial services.

However, in the case of identification cards, various technologies such as coating and hologram are combined to prevent contamination and forgery, so there is a problem that the recognition performance differs according to various environmental factors such as light reflection, shadow, and damage.

For example, the image printed on the ID card may not be clear due to deterioration due to physical factors such as scratches on the ID card itself, and depending on the shooting environment, some areas may be reflected and photographed or covered by light reflection or shadow. .

In addition, according to such deterioration factors, the object in the ID card and the actual object in use can be judged as the same person even though they are not the same person, which can damage the stability of the non-face-to-face service system.

Therefore, for more accurate recognition, it is necessary to devise an identity authentication method that shows robust performance against environmental factors.

An object of the present invention is to propose an identity authentication method having robust performance against environmental factors.

An object of the present invention is to propose an identity authentication method having an improved false acceptance rate (FAR) for environmental factors.

An object of the present invention is to propose an identity authentication method having an improved false rejection rate (FRR) for environmental factors.

An identity authentication method using a learned neural network according to an embodiment of the present invention for solving the above technical problem uses a learned first neural network to obtain a first degree of deterioration according to a first deterioration factor of an identity card image in an input image. Calculating; Calculating a similarity by comparing the identification card image with the user's face image; And determining whether they are the same by comparing the calculated similarity with a predetermined threshold value, and the similarity may be a value adjusted according to the first degree of deterioration.

Calculating a second degree of deterioration according to a second deterioration factor of the identification card image using a second neural network; And extracting an identification card image to be used for authentication from among the input images in consideration of the calculated second deterioration degree, wherein the calculating using the first neural network includes a first deterioration of the extracted identification card image. It is desirable to calculate the degree.

In the extracting step, a second degree of deterioration of the ID card image in the frame is continuously calculated based on the frame of the input image, and an ID card image in the frame to be used for authentication according to the amount of change in the calculated second degree of deterioration. It is desirable to extract.

In the extracting step, each of the plurality of frame ID images in which the amount of change of the second deterioration degree calculated for each frame is less than or equal to a predetermined standard is extracted, and the calculating step is based on the similarity between each of the extracted identification card images and the facial image. It is desirable to calculate the integrated similarity.

The similarity is preferably calculated by weighting the calculated similarity by the first degree of deterioration.

It is preferable that the first deterioration element is a static element that is physically formed on the identification card and affects the state of the identification card image.

It is preferable that the second deterioration factor is a dynamic factor that is formed according to the photographing environment of the identification card and affects the state of the identification card image.

An identity authentication server using a learned neural network according to an embodiment of the present invention for solving the above technical problem uses a learned first neural network to obtain a first degree of deterioration according to a first deterioration factor of an identity card image in an input image. A first deterioration degree calculating unit that calculates; A similarity calculation unit for calculating a similarity by comparing the identification card image with the user's facial image; And an authentication unit that compares the calculated similarity with a predetermined threshold to determine whether they are the same, wherein the similarity is a value adjusted according to the first degree of deterioration.

A second deterioration degree calculating unit for calculating a second deterioration degree according to a second deterioration factor of the identification card image using a second neural network; And an image extracting unit for extracting an identification card image to be used for authentication among the input images in consideration of the calculated second deterioration degree, and the first deterioration degree calculating unit calculates a first deterioration degree of the extracted identification card image. It is desirable to do.

The similarity is preferably calculated by weighting the calculated similarity by the first degree of deterioration.

It is preferable that the first deterioration element is a static element that is physically formed on the identification card and affects the state of the identification card image.

It is preferable that the second deterioration factor is a dynamic factor that is formed according to the photographing environment of the identification card and affects the state of the identification card image.

According to the present invention, it is possible to further improve the accuracy of identity authentication in a non-face-to-face situation.

In addition, when taking an ID card, an ID card image that has a relatively low impact on the surrounding environment can be used for authentication to increase the stability of the service and lower the false rejection rate for the same person.

In addition, by requesting a higher level of reliability for damage or contamination of ID cards, it is possible to lower the false acceptance rate, more accurately judge the forgery situation, and reduce the possibility of problems caused by identity theft.

1 is a diagram showing an identity authentication system according to an embodiment of the present invention.
2 to 4 are diagrams showing an identity authentication method according to an embodiment of the present invention.
5 is a diagram showing the configuration of a neural network for identity authentication according to an embodiment of the present invention.
6 is a diagram illustrating an example of determining the same person for identity authentication according to an embodiment of the present invention.
7 is a diagram showing the configuration of a neural network for identity authentication according to an embodiment of the present invention.
8 is a diagram illustrating an example of sampling an image of an identification card for identification according to an embodiment of the present invention.
9 is a diagram showing the configuration of an identity authentication server for identity authentication according to an embodiment of the present invention.

The following content merely illustrates the principle of the invention. Therefore, although those skilled in the art can implement the principles of the invention and invent various devices included in the concept and scope of the invention, although not clearly described or illustrated herein. In addition, it should be understood that all conditional terms and examples listed in the present specification are, in principle, clearly intended only for the purpose of understanding the concept of the invention, and are not limited to the specifically listed embodiments and states. .

The above-described objects, features, and advantages will become more apparent through the following detailed description in connection with the accompanying drawings, and accordingly, a person of ordinary skill in the technical field to which the invention pertains will be able to easily implement the technical idea of the invention. .

In addition, when it is determined that a detailed description of a known technology related to the invention may unnecessarily obscure the subject matter of the invention, a detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing an identity authentication system according to an embodiment of the present invention.

Referring to FIG. 1, the identity authentication system according to the present embodiment includes a user 10 performing authentication using a user terminal 1000, and authentication determining whether to authenticate using information input from the user 10. It may be configured as a server 100.

The user 10 may perform a video call for authentication, or may take a picture of his/her ID card image 20 and transmit it to the authentication server 100.

The authentication server 100 receiving the transmitted user 10 information determines whether the identification card image 20 and the user 10 in the video call are the same.

Specifically, the identity card image 20 used for authentication is the user's image printed on the identity card such as an ID photo among the entire identity card image 25 that the user 10 photographs and transmits using the camera device in the user terminal 1000. It may be an image, and the authentication server 100 may determine whether the owner of the identification card is the true owner by comparing the ID photo image with the facial image 30 of the user 10 directly input through a video call.

However, in the case of the ID card used at this time, the quality of the face image 30 itself printed in the ID card may be degraded due to damage due to the shooting environment or the ID card itself, which is misrecognized and the user 10 ) Can cause inconvenience.

For example, when the status of the identification card itself is defective, scratches may occur due to long-term use, or the printed image may be deteriorated, resulting in poor clarity.

In such a situation, when comparing specific information determined through the identification card image 20 deteriorated using a neural network with the characteristic information of the facial image 30 acquired through a video call, a problem of determining that the person is the same person even though the person is not the same person may arise. . The rate of false acceptance of such a rejection situation is referred to as a false acceptance rate (FAR), and is it improved in this embodiment? We propose an authentication method with FAR.

Alternatively, due to the influence of the photographing environment, a shadow may be generated by lighting when photographing an identification card, or light may be reflected on a hologram or a coating material to degrade the quality of the photographed identification card image 20. Therefore, the ratio of false judgments that can be recognized as different people even though they are the same person and rejected even though they are allowed is called the false rejection rate (FRR). Therefore, this embodiment proposes an authentication method that improves FRR together with FAR.

Hereinafter, an authentication method of the identity authentication system according to the present embodiment will be described with reference to FIG. 2.

2 is a flowchart illustrating an identity authentication method according to an embodiment of the present invention.

First, a first degree of deterioration according to the first deterioration factor of the ID image 20 in the input image is calculated using the learned first neural network (S100).

The first deterioration factor is a feature that is physically formed on the identification card itself, and may be a factor that deteriorates the quality of the identification card image 20 when photographing the identification card.

Specifically, the first deterioration factor may include scratches due to the use of an identification card, contamination, or other deterioration in which the sharpness of printed photos or characters is deteriorated.

The first neural network may be trained to identify such a first deterioration element, and may be trained to output a value of the existence probability of the first deterioration element for the input ID image 20.

Referring to FIG. 5, the first neural network 50 according to the present embodiment may be configured as a convolution neural network (CNN) model including a layer that performs a plurality of convolution operations.

Accordingly, when the identification card image 20 is input, the convolution operation is performed in a direction to emphasize feature values according to the shape or color of the feature element included in the image while passing through the layers inside the first neural network 50.

The feature values of the ID image 20 input to the first neural network 50 are generated in the form of a new feature map while passing through each convolution layer, and the feature map generated through repetitive operation is a fully connected layer. Identification card status information 52 is output as a result of classification according to the (Fully-connected layer).

The classification result is generated according to the classification according to the learning of the first neural network 50, for example, the first deterioration factor, which is a probability value of the degree according to the degree of scratch and other deterioration factors (fading, contamination, etc.). 1 Deterioration can be output as a value.

To this end, the first neural network 50 may be formed on the identification card itself to learn first deterioration elements determined to hinder recognition of the identification card image 20 through learning data labeled on each identification card image 20.

Since the above first deterioration element is physically formed on an identification card and has a fixed property, it may be defined as a static deterioration element in this embodiment.

Next, the similarity is calculated by comparing the ID image 20 in the image with the facial image 30 of the user 10 (S200).

In general, whether objects included in two images are identical may be performed by comparing feature information extracted for each image.

For feature information, a separate learned neural network can be used.

In this case, the neural network is a feature necessary for human identification, and can be learned to reinforce feature values such as patterns for anatomical features such as eyes, nose, and mouth included in the face, and can be implemented as a CNN model.

Specifically, the similarity between the photographed and transmitted identification card image 20 and the n-dimensional feature information determined by the output of the neural network for each of the facial images 30 of the user 10 input during a video call may be compared.

For example, if each feature information appears as an n-dimensional vector, the similarity to the vector A for the feature information of the identification card image 20 and the vector B for the feature information of the facial image 30 on a video call is cosine. It can be determined by the degree of similarity (Equation 1).

[Equation 1]

Alternatively, the degree of similarity may be determined using a distance between each feature information or a difference between values in the feature information.

When the degree of similarity is calculated through the above process, it is determined whether the user 10 is the same person according to the degree of similarity (S300).

In detail, whether they are the same may be performed by comparing the calculated similarity and a predetermined threshold. The threshold value is a value determined to be the same person, for example, can be determined as a probability value between 0 and 1, for example, when 0.5 is the threshold value, if the calculated similarity is 0.5 or more, authentication as the same person can be allowed. .

The above threshold can be determined internally by the identity authentication system, and is variably determined by using information such as the location where the ID card is photographed, the model of the user terminal 1000 used for the ID card shooting, and the resolution of the image as an environmental factor used for authentication. It is also possible to become.

However, in the present embodiment, the calculated similarity may be readjusted using the first degree of deterioration as described above in order to reduce the false judgment rate due to the deterioration factor of the identification card itself.

Referring to FIG. 3, the method for authenticating an identity according to the present embodiment may further include calculating a weighted similarity by weighting the calculated similarity by the first deterioration degree (S250).

Accordingly, in the step of calculating the similarity (S200), a final similarity may be calculated by multiplying the first deterioration degree by the weight with respect to the similarity calculated through the comparison between the two images.

Referring to FIG. 6, when the first degree of deterioration is output from the first neural network and is calculated as 0.3 (scratch 0.2 + other deterioration 0.1), 0.15 (15%) obtained by multiplying the similarity between two images by 0.5 and the first deterioration degree 0.3 is the final similarity. Can be calculated.

As described above, when the threshold value is 0.5 as a criterion for determining the same person, the final similarity according to the weighted product is less than the threshold value, and thus it is finally determined that the person is not the same person.

That is, if the final similarity calculated by reflecting the first degree of deterioration as the reliability of the similarity determination is used, the false judgment rate of the identification card due to static deterioration can be lowered.

Further, in the present embodiment, the identity authentication method may further consider dynamic deterioration factors in addition to static deterioration elements.

Referring to FIG. 4, in the identification authentication method according to the present embodiment, a second degree of deterioration according to a second deterioration factor of an identification card image 20 is calculated using a second neural network (S10), and the calculated second deterioration degree In consideration of the input image, the step (S20) of extracting an identification card image 20 to be used for authentication may be performed in advance.

That is, the identification card image 20 to be used for authentication of the same person in the photographed image of the identification card may be extracted in consideration of the second deterioration factor of variable influence factors such as shadows, reflections, and holograms as the photographing environment of the identification card.

In this case, the learned second neural network may be used to determine the degree of deterioration according to the second deterioration factor.

Referring to FIG. 7, the second neural network 70 may calculate a second degree of degradation for each frame with respect to an image frame input to the camera device of the user terminal 1000 for photographing an identification card.

Specifically, when the second neural network 70 is also composed of a CNN model, and when the ID image (20-n) included in the image frame is continuously input, the second neural network 70 passes through the inner layers, and according to the shape or color of the feature element included in the image. A convolution operation is performed to emphasize feature values.

In this embodiment, values in the identification card image 20 input to the second neural network 70 may be output in the form of a feature map while passing through each convolutional layer, and the output feature map is a classification result through a fully connected layer. It is output as ID card status information 72 for each frame.

The classification result is a classification according to learning of the second neural network 70, and a degree of deterioration according to the second deterioration factor may be output. In the present exemplary embodiment, the degree of shadow, the degree of hologram generation, and the degree of reflection may be output as a predicted probability value as the second deterioration degree of the identification card image 20 for each image frame.

Therefore, the second neural network 70 is variably generated according to the photographing environment of the identification card and can be learned through learning data labeled on each of the identification card images 20 with second deterioration elements that hinder the recognition of the identification card image 20. have.

At this time, since the second deterioration element is formed on the identification card as a factor according to the external environment and has a variable property, it may be defined as a dynamic deterioration element in this embodiment.

In addition, the second degree of deterioration output from the second neural network 70 is calculated as identification card status information 72 for each frame, and in this embodiment, an identification card image to be used for identification authentication using the identification card status information 72 for each frame ( 20) can be finally extracted.

Specifically, referring to FIG. 8, the identification card status information 72 for each frame calculated for each image frame may be generated as a continuous value, and through this, a change amount for each section of the second degree of degradation may be calculated. In this embodiment, a frame corresponding to a point or an inflection point where the amount of change is less than a predetermined standard is sampled using the amount of change for each section, and the identity image (20-1, 20-2, 20-3, 20-4) within the frame is authenticated. Can be used for

As described above, with respect to the identification card images 20-1 to 4 extracted through the above process, the first deterioration degree and similarity are determined, and the same person is finally determined.

In addition, when a plurality of image frames are sampled as shown in FIG. 8, in the step S200 of calculating the similarity, it is possible to calculate the similarity between each of the identification card images 20-1 to 4 and the facial image 30.

That is, when the similarity of each frame is calculated, the integrated similarity may be further calculated by integrating the similarity. For example, the average of the similarity for each frame may be calculated as the integrated similarity or the highest similarity may be determined as the integrated similarity.

The integrated similarity calculated through the above process is also adjusted according to the first degree of deterioration as described above, and whether or not they are the same is determined based on the adjusted final similarity.

Therefore, the identity authentication method according to the present invention reduces the false rejection rate (FRR) by extracting the identity card images (20-1 to 4) with relatively little influence from the dynamic deterioration factor and using it for identity authentication. By reflecting the reliability according to the static deterioration factor to the similarity, an improved false acceptance rate (FAR) can be achieved.

Hereinafter, an identity authentication server 100 that performs identity authentication using a learned neural network will be described with reference to FIG. 9.

The identity authentication server 100 includes a first deterioration degree calculating unit 110, a second deterioration degree calculating unit 120, a similarity degree calculating unit 130, an authentication unit 140, and an image extracting unit 150.

The first deterioration degree calculating unit 110 calculates a first deterioration degree according to the first deterioration factor of the identification card image 20 in the input image using the learned first neural network 50.

The first neural network 50 calculates the degree of deterioration according to the static deterioration factor in the identification card through learning, and calculates the first deterioration degree of the input identification card image 20.

The similarity calculation unit 130 calculates a similarity by comparing the identification card image 20 with the facial image 30 of the user 10. Specifically, for identity authentication, it may be calculated using a separate neural network learned to extract feature information on the face of the user 10.

Authentication is determined using the calculated similarity. In this embodiment, the similarity may be calculated by weighting the first degree of degradation and the similarity calculated as a value adjusted according to the first degree of degradation.

Accordingly, the authentication unit 140 may determine whether the first degree of deterioration is the same as the similarity by comparing the final similarity and a predetermined threshold value in which the reliability is reflected.

In addition, the second deterioration degree calculation unit 120 may calculate a criterion for extracting the identification card image 20 used for identification. In more detail, a second degree of deterioration according to the second deterioration factor of the identification card image 20 input as an image may be calculated using the second neural network 70 based on the image frame.

The image extraction unit 150 extracts an identification card image 20 to be used for identification authentication by sampling a specific frame using the calculated second deterioration degree.

The first deterioration degree calculating unit 110 reflects the result of the second deterioration degree calculating unit 120 and has relatively little influence of the dynamic deterioration factor, so that the identification card image 20 extracted from the image extracting unit 150 is The first degree of deterioration for can be calculated.

According to the present invention described above, the accuracy of identification authentication in a non-face-to-face situation can be further improved.

In addition, when taking an ID card, an ID card image that has a relatively low impact on the surrounding environment can be used for authentication to increase the stability of the service and lower the false rejection rate for the same person.

In addition, by requesting a higher level of reliability for damage or contamination of ID cards, it is possible to lower the false acceptance rate, more accurately judge the forgery situation, and reduce the possibility of problems caused by identity theft.

Furthermore, various embodiments described herein may be implemented in a recording medium that can be read by a computer or a similar device using, for example, software, hardware, or a combination thereof.

According to hardware implementation, the embodiments described herein include application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), It may be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing other functions, in some cases herein. The described embodiments may be implemented by the control module itself.

According to the software implementation, embodiments such as procedures and functions described in the present specification may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein. The software code can be implemented as a software application written in an appropriate programming language. The software code may be stored in a memory module and executed by a control module.

The above description is merely illustrative of the technical idea of the present invention, and those of ordinary skill in the technical field to which the present invention belongs can make various modifications, changes, and substitutions within the scope not departing from the essential characteristics of the present invention. will be.

Accordingly, the embodiments disclosed in the present invention and the accompanying drawings are not intended to limit the technical idea of the present invention, but are for description, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings. . The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present invention.

Claims (14)

In the identity authentication method using a learned neural network,
Calculating a first degree of deterioration according to a first deterioration factor of the ID image in the input image using the learned first neural network;
Calculating a similarity by comparing the identification card image with the user's facial image; And
Comprising the step of comparing the calculated similarity and a predetermined threshold value to determine whether the same person,
In the calculating of the similarity, the final similarity is recalculated by reflecting the first deterioration degree as a weight to the similarity,
The determining step comprises comparing the recalculated final similarity with the threshold value to determine whether they are the same. The method of claim 1,
Calculating a second degree of deterioration according to a second deterioration factor of the identification card image using a second neural network; And
Further comprising the step of extracting an identification card image to be used for authentication among the input images in consideration of the calculated second deterioration degree,
The calculating by using the first neural network comprises calculating a first degree of deterioration of the extracted ID card image. The method of claim 2,
In the extracting step, a second degree of deterioration of the ID card image in the frame is continuously calculated based on the frame of the input image, and an ID card image in the frame to be used for authentication according to the calculated amount of change of the second degree of deterioration. Identity authentication method, characterized in that to extract. The method of claim 3,
In the extracting step, each of the ID card images in a plurality of frames in which the amount of change of the second deterioration degree calculated for each frame is less than or equal to a predetermined standard is extracted,
The calculating step comprises calculating an integrated similarity degree from the similarity between the extracted respective identification card images and the facial image. delete The method of claim 1,
The first deterioration factor is a static factor that is physically formed on the identification card and affects the state of the identification card image. The method of claim 2,
The second deterioration factor is a dynamic factor that is formed according to a photographing environment of the identification card and affects a state of the identification card image. In an identity authentication server that performs identity authentication using a learned neural network,
A first deterioration degree calculation unit that calculates a first deterioration degree according to a first deterioration factor of the ID image in the input image using the learned first neural network;
A similarity calculation unit for calculating a similarity by comparing the identification card image with the user's facial image; And
Comprising an authentication unit to determine whether the same person by comparing the calculated similarity and a predetermined threshold,
The similarity calculating unit recalculates the final similarity by reflecting the first deterioration degree as a weight to the similarity,
And the authentication unit compares the recalculated final similarity with the threshold value to determine whether they are the same. The method of claim 8,
A second deterioration degree calculating unit for calculating a second deterioration degree according to a second deterioration factor of the identification card image using a second neural network; And
Further comprising an image extracting unit for extracting an identification card image to be used for authentication among the input images in consideration of the calculated second deterioration degree,
The identification authentication server, characterized in that the first deterioration degree calculator calculates a first deterioration degree of the extracted identification card image. delete The method of claim 8,
The first deterioration factor is a static factor that is physically formed on the identification card and affects the state of the identification card image. The method of claim 9,
The second deterioration factor is a dynamic factor that is formed according to a photographing environment of the ID card and affects a state of the ID card image. A computer-readable recording medium storing a program for performing the identification authentication method according to any one of claims 1 to 4 and 6 to 7. A program stored in a computer-readable recording medium for performing the identification authentication method according to any one of claims 1 to 4 and 6 to 7.

Images